Background Cognitive impairments post-stroke are common. Assessment of cognition typically involves pen-and-paper tasks, which are often reliant on linguistic and motor function, creating barriers for many stroke survivors. The characteristics of stroke survivors excluded from cognitive assessments have never been investigated. Aims ( 1 ) To determine if the stroke samples included in studies evaluating clinimetric properties of cognitive assessments represent the stroke population, ( 2 ) to identify the different modes of cognitive assessments, and ( 3 ) to ascertain whether the different modes of cognitive assessments influence the stroke samples used in the studies. Summary of review We systematically reviewed studies that evaluated at least one clinimetric property of a cognitive assessment in adult stroke survivors from January 2000 to October 2013. Eligibility criteria, reasons for drop-outs and missing data were extracted. A theming process was employed to synthesize the data. From the initial yield of 3731 articles, 109 were included. Six broad categories describing reasons for exclusion were identified. Cognitive impairments were the most common (68%), then communication issues (62%), endurance problems (42%), sensory loss (39%), psychiatric illness (38%) and motor limitations (27%). The most prevalent assessment mode was pen-and-paper (73%), then virtual reality (11%), computer (6%), observational functional performance (5%), informant (3%) and telephone (3%). Regardless of mode, issues with cognition and communication were the most frequently used exclusion criteria. Conclusions Our findings indicate that cognitive assessments are not tested in representative stroke samples. Research is needed to identify valid and reliable cognitive assessments that are feasible in a wider range of stroke survivors.