Executive functioning deficits have been found to underlie primary symptoms of hoarding, such as difficulty discarding belongings and significant clutter. Cognitive flexibility—the ability to inhibit irrelevant material and attend flexibly between different mental sets—may be impaired as well, as individuals experience difficulty staying on task and are often distracted by specific possessions that tend to evoke an exaggerated emotional response. The present study investigated cognitive flexibility deficits via eye-tracking technology as a novel approach. Participants (N = 69) with high and low self-reported hoarding symptoms were asked to respond to a series of auditory cues requiring them to categorize a small target number superimposed on one of three distractor image types: hoarding, nature, or a blank control. Across a range of behavioral and eye-tracking outcomes (including reaction time, accuracy rate, initial orientation to distractors, and viewing time for distractors), high hoarding participants consistently demonstrated greater cognitive inflexibility compared to the low hoarding group. However, high hoarding participants did not evidence context-dependent deficits based on preceding distractor types, as performance did not significantly differ as a function of hoarding versus nature distractors. Current findings indicate a pervasive, more global deficit in cognitive flexibility. Those with hoarding may encounter greater difficulty disengaging from previous stimuli and attending to a given task at hand, regardless of whether the context of the distractor is specifically related to hoarding. Implications and future directions for clarifying the nature of cognitive inflexibility are discussed.