Multiple sclerosis (MS) is a common and central nervous system illness associated with diffuse or widespread plaques affecting the myelin sheath that can be observed in both the brain and spinal cord. Demyelination has traditionally been viewed as a major disease mechanism in MS; however, recent research suggests that damage to axons can also occur [1]. Traditionally viewed as a disease affecting white matter, there is increasing evidence associating MS with reduced volume involving selected subcortical nuclei including the thalamus [2]. Given the widespread demyelinization and axonal involvement in MS it is not surprising that MS can present with numerous physical, neuropsychological, and neuropsychiatric symptoms [3,4].

Although the location of MRI plaques and suspected gray matter alterations can be found throughout the brain, at least a portion of the neuropsychological disabilities seen in MS suggest the presence of wide spread frontal-systems deficits involving problems with attention/concentration [5,6], executive control [7], episodic memory [8], and attenuated scores on tests that assess information processing speed [6,9]. For some MS patients information processing speed deficits; whether ascribed to motor skills, working memory capacity, or visuospatial operations [7,10] have been shown to predict long-term neuropsychological decline [11].

The clock drawing test [12-14] has been shown to detect neuropsychological impairment associated with a wide variety of neurobehavioral disorders [12]. The clock drawing test is typically comprised of two test conditions. In the command condition the patient is asked to draw the face of a clock, put in all of the numbers, and set the hands for ‘10 after 11’. This is followed by a copy test condition where patients are asked to copy a model of a clock. In patients with mild cognitive impairment (MCI) and dementia clock drawings to command and copy yield complementary but different information [15-17].

There has been only limited research regarding the utility of the clock drawing test to assess for cognitive disabilities in MS. Barak et al. [18] studied 107 patients with MS, most with relapsing/ remitting disease and a mean Expanded Disability Status Scale (EDSS) score of 4.3 suggesting at least mild to moderate disease involvement. Impaired clock drawing scores were correlated with reduced performance on tests that assess working memory and episodic memory. Rogers and Panegyres [19] have suggested that the clock drawing test can be an effective screening measure for cognitive impairment for MS patients. Benito-Leo [20] found greater impairment on the clock drawing test was associated with worse quality of life in MS patients.

While most clock drawing test procedures score patients drawings using a 5-10 point ordinal scale, the new digital Clock Drawing Test (dCDT; [21]) offers a unique opportunity to capture behavior that has previously been unobtainable. This is possible because the dCDT uses a digital pen embedded with a camera. As the patient draws, a movie or video uploads the patient’s clock drawings. In this sense the dCDT is able to capture behavior in real time.

In addition to total time to completion other behavior that can be captured include a number of discrete latency parameters that measure processing speed and decision-making including command and copy intra-component latencies or the time elapsed between clock drawing components; inter-digit latency or the average time between drawing numbers within the clock face; and quartile drawing time or total clock drawing time divided into four equal segments.

The purpose of the current research was to see how digital clock drawing technology can assess processing speed in MS and might compliment and provide additional information over and above traditional clock drawing methods in patients with MS. Thus, in the current research clock drawings were initially scored using a non-digital, 10-point ordinal scoring system [18]. Using normal control (NC) test performance MS patients were classified into impaired versus non-impaired MS clock drawing groups. The current research tested the hypotheses that [1] patients in the impaired MS clock drawing group will display greater impairment on dCDT measures assessing processing speed and decision-making compared to non-impaired MS and NC participants; and [2] that selected dCDT measures will be associated with impairment on traditional neuropsychological tests related to processing speed and executive control, neurocognitive deficits that are known to be present in MS.

Participants

The current research examined a group of 43 patients with relapsing-remitting MS. All patients were recruited from the Drexel Neurology Multiple Sclerosis Clinic. MS patients were diagnosed using McDonald Criteria [22,23] and all MS patients were on disease modifying medication (Avonex, Betaseron, Copaxone, Rebif, Tysabri) as indicated by their treating neurologist. Exclusion criteria for MS patients included non-native English speakers; the presence of other neurological conditions such as epilepsy; major psychiatric illness such as major depression; a bipolar disorder; drug or alcohol abuse; other major medical illness such as cancer; and any exacerbation of MS illness within six months. The MS Expanded Disability Status Scale (EDSS; [24]) scores ranged from 0 to 7.0. For MS patients depression was assessed with the Beck Depression Inventory-II (BDI-II; [25]; (Table 1) and information obtained from a clinical interview.

Table 1: Demographic Information.

A healthy, community-dwelling normal control group (NC; n=39) was recruited along with MS patients. Exclusion criteria for NCs were the same as the MS exclusion criteria described above. Participants were administered a protocol of neuropsychological tests that assessed executive functioning, information processing speed, and declarative memory.

MS patients were divided into two groups (impaired vs. non-impaired) based upon NC clock performance using a 10-point, ordinal rating scale [18]. For the current study impairment was defined as exceeding <1.5 standard deviations of total mean errors (i.e., errors on both conditions) made by the NC group. On average the NC group made 2.03 total clock errors (range: 0-6). Using NC performance a cut score of 4.1 total errors was calculated and MS participants were categorized accordingly. Thus, MS patients with 4 or fewer total clock errors were grouped as non-impaired (n=23; mean errors: 2.64+1.45) while MS participants with more than 4 errors were classified as impaired (n=20; mean errors: 6.44+1.35). The three groups did not differ for age or education. The two MS groups did not differ on the Beck Depression Inventory. Informed written consent was obtained according to Institutional Review Board guidelines and the Declaration of Helsinki.

The digital clock drawing test

Patients use a digital pen (Anoto Inc.) that works as an ordinary ballpoint pen while capturing pen position 80 times/ second at ± 0.002. Software was developed that classifies each pen stroke (e.g., as a clock face, clock hand, digit, etc.). Davis et al. [21] has shown that with healthy controls this software initially classifies pen strokes with up to 84% accuracy. All data is time-stamped enabling the program to record a movie or a video of the drawing. This assists in making judgments regarding clock elements (e.g., discriminating hour hands and spokes, etc.) and enhances classification accuracy. The spatial resolution of the pen also enables the drawing to be enlarged by up to 100x making apparent phenomena fractions of a millimeter in size and other clock elements that are not visible on the paper with the naked eye.

Among the parameters available using the dCDT is total time to completion for command and copy test conditions. Total time to completion may provide a measure of gross, overall information processing speed. In the current research command and copy total time to completion was subordinated into several constituent variables as described below.

Intra-component latencies

This behavior refers to the latency or the elapsed time as participants transition from drawing one component of the clock (say, the clock face) to another component of the clock (say, digits or the first clock hand). These variables may provide a means to assess complex cognitive constructs involving information processing speed, decision-making, and mental planning. The three intra-component latencies assessed in the current research are described below.

Post-Clock Face Latency (PCF-L): Latency between end of last clock face stroke and the beginning of the next stroke.

Pre-First Hand Latency (PFH-L): Latency from the end of the last stroke prior to drawing the first clock hand.

Inter-Digit Latency: This is the average time (latency) between drawing all digits contained within the clock.

Quartile Latencies

Total drawing time was divided into four equal segments or quartiles. Differentially slower dCDT latencies involving latter quartiles in MS compared to NC participants may provide a mean to assess the capacity to sustain a complex mental set and marshal the necessary neurocognitive resources to bring complex tests to fruition.

Drawing (i.e., ink) time versus non-drawing (i.e., think) time

Total time to completion can also be expressed as percent inking or drawing time versus percent thinking or non-drawing time.

Percent Drawing Time: This was calculated as the percent of total time participants actually spent drawing or putting ink on the page. Percent drawing time may provide a means to assess active test engagement.

Percent Non-Drawing Time: This measure was obtained by subtracting drawing time from total time to completion and may provide a means to assess occult or covert test involvement.

Neuropsychological protocol

The domains of neuropsychological functioning assessed included information processing speed, executive control, and declarative memory. To guard against possible statistical errors, analyses associating dCDT parameters with neuropsychological test were conducted only using MS patients. Not all NC participants and MS patients were assessed with all neuropsychological tests. For descriptive purposes neuropsychological test results for all three groups can be found in Table 2.

Table 2: Neuropsychological Test Performance.

Executive Control: was assessed with tests of letter (letters ‘FAS’) and category (‘animal’) fluency [26]. For both fluency tests participants were given 60s to generate exemplars. On the letter fluency test participants were asked to exclude numbers, names of people/ places, and multiple forms of the same word. The dependent variable for both tests was the total output that was generated.

Processing Speed: was assessed with the WAIS-III Digit Symbol subtest. This test was administered using standard procedures. The dependent variable was the total number of correct responses. Processing speed was also assessed with Symbol Copy subtest from the WAIS-R-Neuropsychological Instrument [27,28]. On this test participants were given two minutes to simply copy as many of the WAIS-III Digit Symbol stimuli as possible. As described by Kaplan et al. [28] the Symbol Copy subtest removes the incidental memory component and the necessity to scan back and forth from the stimulus key to the test form and provides a measure of graphomotor speed. The dependent variable obtained from this test was the number of items completed in two minutes.

Declarative Memory: was assessed with the California Verbal Learning Test II (CVLT-II; 29). This test was administered and scored according to standard instructions. However, the recognition discriminability index was calculated with the following algorithm: [1 - (omitted targets+false positives)/48)] x 100. Thus, a patient obtaining 100% correctly identified all 16 targets while rejecting all false positive responses. Standard administration and scoring were otherwise employed. The CVLT-II dependent variables of interest were total delayed free recall and the recognition discriminability index as described above. These variables were chosen because of their association with the presence of a clinical amnesia [30]. Only MS patients were assessed with this test.

Statistical analyses

Demographic variables such as race and gender were assessed with Chi Square analyses. Other demographic variables such as age and education as well dCDT performance were assessed with 1-way analysis of variance (ANOVA) with Tukey post-hoc tests. The relationship between dCDT performance and neuropsychological test performance was assessed with linear regression analyses.

Demographic information

69.51% of all participants were female; however, the NC group was comprised of more men than the impaired MS clock drawing (x²[1]=8.64, p<0.003), and the non-impaired MS clock drawing group (x²[1]=6.08, p<0.014). The three groups were equally split between African-American and Caucasian participants (African-American, n=39; 47.57%: Caucasian; n=43; 52.43%, respectively; x²[1]=2.73, ns). When statistical adjustments were made for age, education, race, gender, depression (Beck Depression Inventory-II) and type of disease modifying medication there was no effect on dCDT performance.

Command drawing test performance

Command Total Time to Completion: The 1-way ANOVA for total time to completion was significant (F[2,79]=9.84, p<0.001) with post-hoc (Tukey) analyses finding longer total time to completion for the impaired MS clock drawing group compared to other groups (p<0.001, both analyses). The non-impaired MS and NC groups did not differ on this measure (Table 3 and 4).

Table 3: Digital Clock Drawing: Command Test Condition (means and standard deviations).

Table 4: Digital Clock Drawing: Copy Test Condition (means and standard deviations).

Command Intra-Component Latencies: For command intra-component latencies there was no effect for group for post-clock face latency. Significant group effects were noted for pre-first hand latency (F[2,79]= 3.99, p<0.022) and the inter-digit latency (F[2,79]=5.72, p<0.005). Follow-up analyses found slower pre-first hand latency for the impaired MS clock drawing group compared to non-impaired MS (p<0.033) and NC participants (p<0.039); and, slower inter-digit latency also for the impaired MS clock drawing group compared to the non-impaired MS (p<0.007) and NC participants (p<0.014). For post-clock face latency 89.00% of all participants drew the number 12 after the clock face. For the pre-first hand latency 89.02% of participants drew either the number 11 (34.145%) or the clock center dot (58.53%) before the first clock hand.

Command Quartile Latencies: Separate 1-way ANOVAs were conducted for quartile latencies. For quartiles 1 and 2 there were no between-group differences; however, between-group differences were obtained for quartiles 3 and 4 (3rd quartile – F[2,79]= 4.89, p<0.010; 4th quartile – F[2,79]= 8.79, p<0.001). Follow-up tests found longer 3rd quartile latency for the impaired MS compared to the non-impaired MS (p<0.032) and NC groups (p<0.011). Similarly, in the 4th quartile impaired MS patients presented with a longer latency compared to the non-impaired (p<0.005) and NC groups (p<0.001).

Command Drawing (ink) versus Non-Drawing (think) Time: Percent drawing and non-drawing time was assessed between-group and no differences were found. Interestingly, across all three groups approximately 60 percent of total clock drawing time was spent thinking (i.e., not drawing) rather than inking or drawing. Percent inking versus thinking time was also assessed within-group. All three groups presented with greater non-drawing (thinking) compared to drawing (inking) time (impaired MS clock drawing group: t[19]=4.69, p<0.001; non-impaired MS clock drawing group: t[22]=5.16, p<0.001; NC group: t[38]=7.53, p<0.001).

Copy drawing test performance

Copy total time to completion: A similar series of 1-way ANOVAs was employed to assess copy test performance. Total time to completion continued to be significant ([F2, 79]=10.32, p<0.001) with follow-up analyses demonstrating longer time to completion for the impaired MS clock drawing group compared to non-impaired (p<0.002) and NC (p<0.001) participants. The non-impaired MS and NC groups did not differ on this measure.

Copy intra-component latencies: Significant group effects were obtained for the post-clock face latency (F[2,79]= 10.92, p<0.001) and inter-digit latency (F[2,79]= 7.94, p<0.001). Follow-up comparisons found that the impaired MS clock drawing group produced slower post-clock face latencies compared to non-impaired (p<0.003) and NC (p<0.001) groups. The impaired MS clock drawing group also produced slower inter-digit latencies compared to non-impaired (p<0.004) and NC (p<0.001) groups.

For the post-clock face latency 92.67% of participants drew the number 12 after the clock face. For the pre-first hand latency 92.68% of participants first drew either the number 11 (50.00 percent) or the clock center dot (42.68 percent) before the first clock hand.

Copy quartile latencies: For quartile latencies significant group effects were obtained for all four quartiles (1st quartile–F[2,79]=5.00, p<0.009; 2nd quartile–F[2,79]=5.84, p<0.004; 3rd quartile–F[2,79]=9.37, p<0.001; 4th quartile–F[2,79]=9.82, p<0.001). Follow-up comparisons found slower 1^st and 2^nd quartiles for the impaired MS clock drawing group compared to the NC group (p<0.007 and p<0.003, respectively). For the 3^rd quartile a longer latency was noted for the impaired MS group compared to the non-impaired MS (p<0.016) and NC groups (p<0.001). A similar result was obtained for 4th quartile with the impaired MS group producing a longer latency compared to the impaired MS (p<0.009) and NC (p<0.001) groups.

Copy Drawing (ink) versus Non-Drawing (think): When assessed between-group there was no difference between percent drawing versus percent non-drawing time. For all participants approximately 54 percent of total time to completion was spent not drawing compared to drawing. When assessed within-group only the NC participants generated more non-drawing time compared to non-drawing time (t[38]=4.10, p<0.001.

Intra-component regression analyses

A series of step-wise regression analyses were conducted to assess the relationship between dCDT intra-component latency and neuropsychological test performance. As stated above only MS patients were used in these analyses. To minimize the number of statistical comparisons all command and copy intra-component latencies were summed to create a single intra-component score. The rationale underlying this decision was based, in part, on the fact that the patterns of performance regarding intra-component performance were similar between the command and copy test conditions. In the regression analyses described below total intra-component latency was the dependent variable and neuropsychological test performance were independent variables.

The first regression analysis examined the relationship between intra-component latency and performance of executive control (letter/‘animal’ fluency). Only output from the ‘animal’ fluency entered the model such that increasing intra-component latency was related diminished ‘animal fluency’ output (r=0.317, R²=0.100, F[1,39]=4.34, p<0.013, beta=-0.317, p<0.040. For tests of information processing speed only the Symbol Copy subtest entered the model where diminished output was associated with longer intra-component latency (r=0.650, R²=0.423, F[1,21]=15.36, p<0.001, beta=-.650, p<0.001). When measures from the CVLT-II were examined only delayed recognition discriminability entered the model with slower intra-components associated with lower recognition scores (r=0.531, R²=0.282, F[1,22]=8.63, p<0.009, beta=-.531, p<0.008). Scatter plots displaying these analyses can be found in Figure 1a-c.

Figure 1a: Scatterplots for Total (command & copy) Digital Clock Drawing Intra-Component Latencies and Neuropsychological Test Performance. Figure 1a: WAIS-R-NI Symbol Copy & Total Digital Clock Intra-Components.

Figure 1b: Semantic (‘animal’) Output & Total Digital Clock Intra- Components.

Figure 1c: CVLT-II Recognition Discriminability Index & Total Digital Clock Intra-Components.

The association between slower intra-component latency and reduced CVLT-II recognition test performance prompted a follow-up stepwise regression where the four CVLT-II recognition foils type were the independent variables, i.e., list B shared list A category, list B non-shared list A category, semantic foils, unrelated foils. Only list B, shared list A category foils entered the model (R²=0.452, r=0.204, F[1,22]=5.65, p<0.027; beta=0.452, p<0.027) where slower latency was associated with increasing numbers of list B, shared list A category foils.

The purpose of the current research was 2-fold–to assess how dCDT parameters provides additional and complimentary information regarding processing speed and decision-making in relation to traditional clock scoring methods; and, to assess the association between dCDT parameters and conventional neuropsychological test performance. In the current research similar profile were observed in both the command a copy test conditions. Overall, patients in the impaired clock drawing group exhibited longer time to completion, longer latencies for selected intra-components, and longer latencies for the 3rd and 4th quartile of their drawing. When clock drawing command and copy intra-component latencies were summed regression analyses found longer intra-component latency to be associated with lower output on the ‘animal’ fluency test, less information completed on the WAIS-R-NI Symbol Copy subtest, and a lower score on the CVLT-II Recognition Discriminability subtest. Overall, these data suggest that digital clock drawing parameters compliment traditional clock scoring methods and provide greater detail regarding the brain-behavior relationships that underlie impaired clock drawing behavior in MS.

As noted above total time to completion in the command test condition was slower for the impaired MS compared to other groups; however, when simple total time to completion is parceled into meaningful constituent variables a more nuanced profile emerged suggesting subtle derailment of complex cognitive operations. The analysis of quartile latency data revealed no differences for the first and second quartiles. Between-group differences emerged only for the latter two quartiles where the impaired MS group was slower than other groups. Also, despite the fact that for all groups the proportion of non-drawing (thinking) versus drawing (inking) was the same, there were clear differences in time allocation.

Slower time to completion for the impaired MS group for the latter two test quartiles could suggest the need to recruit greater neurocognitive resources in order to sustain the necessary mental set to bring the test to fruition. Additional support for this notion might be obtained to the extent future research using functional imaging technology finds differential activation in specific brain regions on latter clock drawing quartiles or selected clock drawing components.

The impaired MS group also produced generally slower command intra-components latencies compared to other groups. A question to be asked is whether longer intra-component latency reflects slowness in motor versus cognitive operations. The data described above demonstrate that in either the command or copy test conditions almost 90 percent of participants drew the number 12 immediately after the clock face and the number 11 or the clock center dot immediately before the first clock hand. This suggests that as specific clock drawing transition junctions are confronted participants know what to do and where to go, but selected participants required more time to initiate a decision that is ultimately correct. We would maintain that this reflects underlying slowness in cognitive operations (i.e., bradyphrenia) rather than slowness in motor operations (i.e., bradykinesia). Such behavior could also be consistent with difficulty in sustaining mental set. However, we acknowledge further research is necessary to fully explore this issue.

Performance in the copy test condition was analogous but somewhat different compared to the command test condition. Total time to completion continued to be slower for the impaired MS compared to other groups. However, the relative percent of non-drawing time to drawing time across the three groups appears to be more complex. Within-group analyses showed that the NC group continued to require more non-drawing versus drawing time. By contrast, both MS groups devoted equal time for thinking versus drawing. The reason(s) for this command versus copy test difference are unclear at the present time and require further research.

The analysis of copy quartile latencies continues to suggest differences in time allocation. For the first and second quartiles the impaired MS group was slower than the NC group. For the third and fourth quartiles the impaired MS groups was slower compared to both non-impaired MS and NC groups. The differential slowness across all four quartiles seen in the impaired MS clock drawing group could suggest a greater need to scan back and forth from the copy model to patient’s drawings. This might suggest that copy test performance for the impaired MS clock drawing group is not as automatized as compared to other groups. Additional research is necessary to explore this possibility.

The impaired MS group displayed longer latencies for several copy intra-components. These data are striking given that in the command test condition participants in the impaired MS group generally demonstrated correct decision-making regarding their behavior after drawing the clock face and before drawing the first clock hand. These data, in conjunction with differential slowness involving all quartiles suggest that copy test performance in the impaired MS group appears not as automatized as for other groups. These data also suggest that difficulty in processing speed often seen in MS is complex, a function of test demands, and not necessarily monotonic in nature.

Our second prediction examined the relationships between dCDT parameters and conventional neuropsychological test performance. Among all of the neuropsychological measures assessed, slower intra-component latency was most strongly related to diminished output on the WAIS-R-NI Symbol Copy subtest suggesting that graphomotor processing speed is a major underlying factor contributing to impaired clock drawing in MS. However, slow intra-components was also observed in conjunction with low output on the ‘animal’ fluency test, and lower performance on the CVLT-II recognition discriminability index with selected responding to CVLT list A category items that are shared with CVLT list B. This profile suggests that dCDT slower intra-components may also be related to impair executive control. We acknowledge that more research is required to better understand the relationship between dCDT parameters and conventional neuropsychological tests.

Kaplan [13,14] maintained that the utility of the clock drawing test in clinical assessment and the degree clock drawing behavior conveys knowledge regarding underlying brain-behavior relationships is often revealed through the analysis of process and errors. The ability of the dCDT to capture behavior in real time is unique compared to other common neuropsychological tests and may provide an analysis akin to phasic psychophysiological behavior [31]. In preliminary research Shoyama et al. [32] obtained clock drawings from a healthy control group in conjunction with functional Near-Infrared Spectroscopy (fNIRS), a functional imaging technique. These researchers found that as clock drawing time to completion increased greater pre-frontal oxygen- hemoglobin was recruited. Libon and colleagues [33] used fNIR and compared MS and NC participates using a modified Stroop test. Preliminary data show no difference the number of correct responses or the number of errors produced. However, in order for MS patients to maintain this level of performance greater levels of pre-frontal oxygen- hemoglobin was necessary. The association between fNIR measurements and the dCDT parameters described above could provide greater insight into the brain-behavior relationships underlying clock drawing in MS.

The current research is not without limitations. First, only relapsing-remitting MS patients were studied. Different findings might have been obtained if other MS patient groups were studied. Second, the number of participants in both MS groups was modest and only a portion of NC and MS participants were assessed with all neuropsychological tests. Third, because of modest sample size separate analyses for the impaired versus non-impaired MS clock drawing groups were not conducted. All of these factors need to be addressed in future research. Thus, the data reported above must be viewed as preliminary. Nonetheless, given the ease of administration of the dCDT and its non-threatening nature, a practical benefit of the dCDT might its use to screen for subtle cognitive disabilities in MS. Also, parameters obtained from the dCDT might be suitable as outcome measures in research examining disease modifying therapies. In sum, despite the preliminary nature of the current research the precision in which behavior can be measured, digital pen technology appears to convey complimentary information in conjunction with traditional clock scoring methods and offers a unique perspective to assess underlying brain-behavior relationships in MS.

1. Helune GJ, Stott H, Pinkston J (2008). Multiple Sclerosis. In: Morgan JE, Ricker JH. Textbook of Clinical Neuropsychology. New York, USA: Taylor and Francis, pp: 599-615.
2. Benedict RH, Hulst HE, Bergsland N, Schoonheim MM, Dwyer MG, et al. (2013) Clinical significance of atrophy and white matter mean diffusivity within the thalamus of multiple sclerosis patients. Multiple Sclerosis 19:1478-1484.
3. Brassington JC, Marsh NV (1998) Neuropsychological aspects of multiple sclerosis. Neuropsychol Rev 8: 43-77.
4. Gordon PA, Lewis MD, Wong D (1994) Multiple sclerosis: strategies for rehabilitation counselors. Journal of Rehabilitation 60: 34-38.
5. Beatty WW, Wilbanks SL, Blanco CR, Hames KA, Tivis R, et al. (1996) Memory disturbance in multiple sclerosis: reconsideration of patterns of performance on the selective reminding test. J ClinExpNeuropsychol 18: 56-62.
6. Litvan I, Grafman J, Vendrell P, Martinez JM (1988) Slowed information processing in multiple sclerosis. Arch Neurol 45: 281-285.
7. DeLuca J, Chelune, GJ, Tulsky DS, Lengenfelder J, Chiaravalloti ND (2004) Is speed of processing or working memory the primary information processing deficit in multiple sclerosis? Journal of Clinical and Experimental Neuropsychology 26: 550-562
8. Denney DR, Sworowski LA, Lynch SG (2005) Cognitive impairment in three subtypes of multiple sclerosis. Arch ClinNeuropsychol 20: 967-981.
9. Grafman J, Rao S, Bernardin L, Leo GJ (1991) Automatic memory processes in patients with multiple sclerosis. Arch Neurol 48: 1072-1075.
10. Janculjak D, Mubrin Z, Brinar V, Spilich G (2002) Changes of attention and memory in a group of patients with multiple sclerosis. ClinNeurolNeurosurg 104: 221-227.
11. Bergendal G, Fredrikson S, Almkvist O (2007) Selective decline in information processing in subgroups of multiple sclerosis: an 8-year longitudinal study. EurNeurol 57: 193-202.
12. Freedman M, Leach L, Delis, D.C Kaplan E, (1994) Clock Drawing: A neuropsychological assessment, Oxford University Press, New York.
13. Kaplan E (1988) A process approach to neuropsychological assessment. In T Boll & Bryant BK (Eds), Clinical neuropsychology and brain function: Research, measurement, and practice, Washington, DC: American Psychological Association.
14. Kaplan E (1990) The process approach to neuropsychological assessment of psychiatric patients. J Neuropsychiatry ClinNeurosci 2: 72-87.
15. Cosentino S, Jefferson A, Chute DL, Kaplan E, Libon DJ (2004) Clock drawing errors in dementia: neuropsychological and neuroanatomical considerations. CognBehavNeurol 17: 74-84.
16. Grossman H. Libon DJ (2009). Differentiation of MCI Subtype Using a Conceptual Composite Score on the Clock Drawing Test. Abstract presented at the 37th Annual Meeting of the International Neuropsychological Society, Atlanta, GA.
17. Libon DJ, Malamut BL, Swenson R, Sands LP, Cloud BS (1996) Further analyses of clock drawings among demented and nondemented older subjects. Arch ClinNeuropsychol 11: 193-205.
18. Barak Y, Lavie M, Achiron A (2002) Screening for early cognitive impairment in multiple sclerosis patients using the clock drawing test. J ClinNeurosci 9: 629-632.
19. Rogers JM, Panegyres PK (2007) Cognitive impairment in multiple sclerosis: evidence-based analysis and recommendations. J ClinNeurosci 14: 919-927.
20. Benito-León J, Morales JM, Rivera-Navarro J (2002) Health-related quality of life and its relationship to cognitive and emotional functioning in multiple sclerosis patients. Eur J Neurol 9: 497-502.
21. Davis R, Penney DL, Pittman D, Libon DJ, Swenson R, et al. (2011) The Digital Clock Drawing Test (dCDT) - I: Development of a new computerized quantitative system. Abstract presented at the 39TH annual meeting of the International Neuropsychological Society, Boston, MA.
22. McDonald WI, Compston A, Edan G, Goodkin D, Hartung HP, et al. (2001) Recommended diagnostic criteria for multiple sclerosis: guidelines from the International Panel on the diagnosis of multiple sclerosis. Ann Neurol 50: 121-127.
23. Polman CH, Reingold SC, Edan G, Filippi M, Hartung HP, et al. (2005) Diagnostic criteria for multiple sclerosis: 2005 revisions to the "McDonald Criteria". Ann Neurol 58: 840-846.
24. Kurtzke JF (1983) Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology 33: 1444-1452.
25. Beck AT, Steer, R A, Brown GK (1996) Manual for the Beck Depression Inventory-II. San Antonio, TX: Psychological Corporation.
26. Strauss E, Sherman E, Spreen O (2006) A compendium of neuropsychological test: administration norms and commentary. Oxford University Press, New York, USA
27. Joy S, Kaplan E, Fein D (2004) Speed and memory in the WAIS-III Digit Symbol--Coding subtest across the adult lifespan. Arch ClinNeuropsychol 19: 759-767.
28. Kaplan E, Fein D, Morris R, Delis D (1991). The WAIS-R as a neuropsychological instrument. The Psychological Corporation, San Antonio, TX.
29. Delis DC, Kramer JH, Kaplan E, Ober BA (1987). The California Verbal Learning Test. New York: Psychology Corporation.
30. Price CC, Garrett KD, Jefferson AL, Cosentino S, Tanner JJ, et al. (2009) Leukoaraiosis severity and list-learning in dementia. ClinNeuropsychol 23: 944-961.
31. Lacey JI, BC Lacey (1962) The law of initial value in the longitudinal study of autonomic constitution: Reproducibility of autonomic responses and response patterns over a four-year interval.
32. Shoyama M, Nishioka T, Okumura M, Kose A, Tsuji T, et al. (2011) Brain activity during the Clock-Drawing Test: multichannel near-infrared spectroscopy study. ApplNeuropsychol 18: 243-251.
33. Libon DJ, Scull L, Choi L, Tabby D, Eppig J (2014) Assessing prefrontal neural integrity in Multiple Sclerosis with Functional Near-Infrared Spectroscopy (fNIR). Abstract presented to the 42nd meeting of the International Neuropsychological Society, Seattle, WA.