World Journal Of Pediatrics-articles

Paul SS Russell, Anna Daniel, Sushila Russell, Priya Mammen, Julie S Abel, Lydia E Raj, Satya Raj Shankar, Naveen Thomas

Author Affiliations: Autism Clinic, Child & Adolescent Psychiatry Unit, Department of Psychiatry, Christian Medical College, Vellore 632 002, Southern India (Russell PS, Daniel A, Russell S, Mammen P, Abel JS, Raj LE, Shankar SR, Thomas N)

Corresponding Author: Paul Russell, MD, Autism Clinic, Child & Adolescent Psychiatry Unit, Department of Psychiatry, Christian Medical College, Vellore 632 002, Southern India (Tel: +91(0416) 2284307; Fax: +91 (0416) 32788 or 32368; Email: russell@cmcvellore.ac.in)

Background: Since there is no established measure for autism in India, we evaluated the diagnostic accuracy, reliability and validity of Childhood Autism Rating Scale (CARS).

Methods: Children and adolescents suspected of having autism were identified from the unit's database. Scale and item level scores of CARS were collected and analyzed. Sensitivity, specificity, likelihood ratios and predictive values for various CARS cut-off scores were calculated. Test-retest reliability and inter-rater reliability of CARS were examined. The dichotomized CARS score was correlated with the ICD-10 clinical diagnosis of autism to establish the criterion validity of CARS as a measure of autism. Convergent and divergent validity was calculated. The factor structure of CARS was demonstrated by principal components analysis.

Results: A CARS score of ≥33 (sensitivity = 81.4%, specificity = 78.6%; area under the curve = 81%) was suggested for diagnostic use in Indian populations. The inter-rater reliability (ICC=0.74) and test-retest reliability (ICC=0.81) for CARS were good. Besides the adequate face and content validity, CARS demonstrated good internal consistency (Cronbach's α=0.79) and item-total correlation. There was moderate convergent validity with Binet-Kamat Test of Intelligence or Gessell's Developmental Schedule (r=0.42; P=0.01), divergent validity (r=-0.18; P=0.4) with ADD-H Comprehensive Teacher Rating Scale, and high concordance rate with the reference standard, ICD-10 diagnosis (82.52%; Cohen's κ=0.40, P=0.001) in classifying autism. A 5-factor structure explained 65.34% of variance.

Conclusion: The CARS has strong psychometric properties and is now available for clinical and research work in India.

Key words: autism; diagnostic accuracy; India; reliability; validation

There has been a documented increase in the prevalence of autism worldwide.[1,2] The Indian Academy of Pediatrics, as stated in the aims of its Vision 2007, plans to conduct epidemiological studies and enhance early identification of autism.^[3] To achieve these goals, establishing the psychometric properties of an appropriate autism measure that suits the local culture becomes imperative. A satisfactory diagnostic measure for autism is currently unavailable,^[4] partly because of inadequate validation procedures that do not satisfy the Cochrane and Holland criteria needed for the validation of measures.^[5]

Numerous measures related to autism have been partly or fully validated in other countries for school aged children.^[6] In India the Autism Behavior Checklist (ABC), Checklist for Autism in Toddlers (CHAT), Modified Checklist for Autism in Toddlers (M-CHAT), Autism Diagnostic Interview-Revised (ADI-R), Childhood Autism Rating Scale (CARS), Gilliam Autism Rating Scale (GARS) and Autism Diagnostic Observation Schedule (ADOS) are widely used for either screening or diagnosis of autism although none of these measures have been validated for this population. Among these autism assessment instruments reviewed, Childhood Autism Rating Scale (CARS)^[7] is promising as a diagnostic measure because of its simplicity, conceptual relevance, high concordance with DSM-III/III-R/IV diagnosis of autism, acceptability, cost effectiveness, utility among different populations^[7-12] and strong psychometric properties when validated in other non-Western cultures.^[13,14]

Therefore this study was conducted to establish the psychometric properties, namely the diagnostic accuracy, reliability and validity of CARS among children with autism in India using the criterion-referenced approach of validation, following the criteria of Cochrane and Holland for validating measures^[15] and STARD guidelines for diagnostic accuracy.^[16]

This study was conducted at the Autism Clinic, Child and Adolescent Psychiatry Unit of a tertiary care, teaching hospital in Southern India. This facility does not have a geographical catchment population. The charts of children and adolescents referred to the clinic with a suspected diagnosis of autism (Pervasive Developmental Disorder of ICD-10) were identified from the unit's database for a six year period of 2001 to 2007. We collected the data for each clinic visit made by the child and considered eligible for this study if suspected to have autism at any point during the clinical course. Children with a diagnosis of overactive disorder associated with mental retardation and stereotyped movements (F84.4) were excluded because of their uncertain nosological status.^[17] Case-notes for each eligible participant were reviewed and the following psychological and clinical data were collected to determine the various aspects of validation.

The ICD-10 based clinical diagnosis^[18] of autism (pervasive developmental disorders) [childhood autism (F84.0), atypical autism (F84.1), Rett's syndrome (F84.2), other childhood disintegrative disorder (F84.3), and Asperger's Syndrome (F84.5)], made by the consultant psychiatrists and later endorsed by the multidisciplinary team consisting of special educators, occupational therapists, speech therapists and psychia-tric nurses, was used as the reference standard in this study. The diagnoses were made by direct observations of children in semistructured play based activities and parent interviews.

The Childhood Autism Rating Scale (CARS)^[7] is a 15-item behavior-rating scale designed to detect and quantify symptoms of autism as well as to distinguish them from other developmental disabilities. Each item on the CARS is scored on a Likert scale, from 1 (no signs of autism) to 4 (severe symptoms). The maximum CARS score is 60, and the cut-off for a diagnosis of autism is 30. Children with scores of 30.5 to 37 are rated as mildly-moderately autistic, and 37.5 to 60 as severely autistic. The scale and item level scores of CARS were collected from the psychologist reports, occupational therapist's record as well as speech therapists notes, and they were indexes for validation in this study.

The Binet Kamat Scale of intelligence (BKT)^[19] is the Indian adaptation of the Stanford-Binet Scale of Intelligence. Some of the test items and materials were amended to suit Indian conditions, such as Indian coins, typically Indian pictorial scenes, vocabulary and Indian concepts. The intelligence scale assessed the child's skills in six areas: memory, language, conceptual thinking, reasoning, numerical reasoning, visuo-motor coordination and social intelligence. Gesell's Developmental Schedule (GDS)^[20] gives the developmental skills in four areas: motor behavior, adaptive behavior, language and personal as well as social behavior. These two scales were selected from psychological reports of these children to measure the convergent validity of the CARS. ADD-H Comprehensive Teacher Rating Scale or ACTeRS^[21] contains 24 questions and is used for children between the ages of 5 and 12 years, and measures 4 areas of behaviors of attention deficit, hyperactivity, oppositional behavior and social skills. Details of this scale from the psychological assessment notes were used to measure the divergent validity.

All the details about autism, intellectual disability and attention deficit hyperactivity disorder (ADHD) were made by the multidisciplinary treatment team ahead of the time when data were collected. The CARS was assessed after autism was clinically diagnosed by the psychiatrists in the team. However, the CARS was rated independently by clinical psychologists or rehabilitation psychologists and speech therapists with experience of working with children with developmental disabilities for a mean (SD) duration of 12.74 (8.21) years. The CARS ratings were based on the behavioral observation of the children by these raters further supported by information from the parents as well. They were not aware of the psychiatrists' clinical diagnosis minimizing the rater bias. A consultant psychiatrist independently collected the details of the ICD-10 clinical diagnosis. These data were available in the patients' clinical case-notes made by the psychiatrists, psychological assessment notes, special educators' reports, occupa-tional therapy details or speech therapist's notes. The data were extracted from these sources by two graduate psychologists, an occupational therapist, and a speech therapist independently, and they were protected by reversible anonymisation and restricted to others. The study was reviewed and approved by the local institutional review board.

Sensitivity, specificity, likelihood ratios and predictive values for various CARS cut-off scores were calculated to determine the optimal screening threshold with receiver operating characteristic (ROC) analyses and contingency tables. The nominal area under the curve (AUC) was computed to evaluate the overall diagnostic efficiency of the test. The test-retest reliability and the inter-rater reliability of the CARS were examined with the intra class correlation. For internal consistency, Cronbach's α coefficient was calculated. To identify the items that contribute to and discriminate between children who score high and low on the total set of items, we performed an item-total correlation. By determining the criterion validity of the CARS as a measure of autism, the dichotomized CARS score (score of 33 deciding the 'caseness') was found to be correlated with the ICD-10 clinical diagnosis of autism. The concordance (overlapping cases) of the ICD-10 diagnosis of autism and CARS diagnosis of autism was computed as the quotient of the cases classified as autism by both the measures and the number of cases classified as autism by neither of the measures.

Cohen's kappa was also calculated to assess the agreement between the ICD-10 diagnosis and dichotomized total CARS score. The convergent and divergent validity was calculated by correlating CARS score with the total GDS or BKT score and ACTeRS score respectively, as it was hypothesized that the CARS diagnosis of autism would be conceptually more closely related to another childhood disability with problems in communication and socialization than ADHD that measures decreased attention span and restlessness as its construct in children. The factor structure of the CARS was demonstrated by principal components analysis with varimax rotation. CARS items were removed if they failed to load on any factor (loading <0.40) or had unacceptably high secondary loadings (>0.30).^[22] The data were analyzed using SPSS version 16.

Of the 103 children, 86 had autism, 14 had no autism, and 3 had no data available. Among the children with autism, 28 had childhood autism, 54 had atypical autism, 3 had Asperger's syndrome, and 1 had Rett's syndrome. Among the 14 children without autism, 7 had average intelligence, 6 had compromised intelligence, and 1 had compromised intelligence with selective mutism. Among the entire study sample, there were 10 children with average intelligence, 72 children with mild, moderate, severe or profound intellectual disability, and 21 children with unspecified intellectual disability and autism. None of the 7 children without autism with average intelligence had any other developmental disorder. The mean (SD) CARS score was 35.40 (4.54) with a range of 22 to 44.50. The mean (SD) chronological age and mental age of the children were 5.10 (2.20) and 2.54 (1.41) years, respectively. There was a higher representation in boys than in girls, which probably reflected the higher prevalence of autism in boys as against girls globally. The geographic distribution of participants within

India

, the socio-demographic details of primary care-givers (mostly mothers) and disability details are shown in Table 1.

Various parameters of diagnostic accuracy for differing cut-off points of the CARS were tested against the reference standard of ICD-10 clinical diagnosis. Table 2 summarizes these results. A score of ≥33 in the CARS achieved a sensitivity of 81.4% (95% CI=71.6-89), a specificity of 78.6%, (95% CI=49.2-95.1), a positive likelihood ratio of 3.8 (95% CI=2.8-5.1), a negative likelihood ratio of 0.24 (95% CI=0.08-0.70), a positive predictive value of 95.9%, and a negative predictive value of 40.7%; therefore it was ideal as a screening cut-off score to identify possible cases of autism. The AUC in the ROC for the CARS was 0.81 (95% CI=0.72-0.88; z=6.14; P=0.0001) with its standard error at 0.06 (Fig.). Among the 14 children without autism the primary disorders noted were seizures (n=2), cerebral palsy (n=3), multiple co-morbidity (n=1). None of these primary disorders were detected up as autism, although some children (n=2) with intellectual disability were misclassified as autism.

The test-retest reliability at one year was studied to assess the reproducibility of the CARS and the intra class correlation coefficient was found to be ICC=0.81. The inter-rater reliability of the CARS as measured with intra class correlation coefficient was also found to be ICC= 0.74.

When we examined the validity of the CARS, Cronbach's α coefficient for the whole scale was high (α = 0.79), suggesting that the CARS in this population has satisfactory internal consistency and the item-total correlation ranged from 0.26 to 0.75 (Table 3). None of the 15 items was assigned a score of 0 by more than half of the children with autism in this study, suggesting that the content validity was appropriate to their neurodevelopmental disorder. As part of the criterion validity analysis, there was a high concordance rate of 82.52% (ICD-10 vs. CARS=89/103 vs. 81/103) [Cohen's κ=0.40 (95% CI=0.15-0.65); P=0.001] between the CARS and reference standard of ICD-10 diagnosis in identifying autism among the children. Interestingly only less than a quarter of the population were not found to be suffering from any type of autism by ICD-10 (14/103) or CARS (22/103). The convergent validity between the CARS and BKT or GDS, calculated with Pearson's product-moment correlation coefficient, was moderately acceptable (r=0.42, P=0.01). Divergent validity calculated by correlating CARS scores to ACTeRS showed non-significant associations (r=-0.18; P=0.4) demonstrating that the CARS discriminates autism from other childhood psychiatric disorders like ADHD.

To investigate the construct validity, we explored the factor structure of the items in the CARS. We extracted those factors with an eigen value of 1 and thus a 5-factor structure was derived. There was no CARS item that did not achieve the required factor loading to load to at least one factor, but items 1 (relationship to people), 3 (affect), 7 (visual responsiveness) and 12 (non-verbal communication) cross-loaded to many factors. None of the remaining 12 items cross-loaded but they all loaded distinctively to different factors (Table 4). CARS items 2 (imitation), 8 (auditory responsiveness), and 11 (verbal communication) loaded on to factor 1 (abnormal sensation-communication); items 4 (use of body), 5 (relation to nonhuman objects), and 15 (general impression) loaded to factor 2 (restricted interests-relations); items 6 (adaptation to environmental changes), 10 (anxiety reaction) loaded to factor 3 (negative emotionality-adaptability); items 9 (near receptor responsiveness) and 14 (intellectual functioning) loaded to factor 4 (odd sensory exploration-intellect); and finally the item 13 (activity level) loaded to factor 5 (difficulty in activity regulation). This five-factor structure explained 65.34% of the variance.

While the Childhood Autism Rating Scale (CARS) uses the well-tested autism symptom list from the DSM, it still has to be evaluated empirically before field use in another culture with different psychosocial constructs and therefore it was done in the present study. Thus our study is the first to document that the psychometric properties of diagnostic accuracy, reliability and validity of the CARS in the Indian population are as acceptable as in Western populations^[7,9,23] and other non-Western cultures.^[14]

A threshold score of ≥33 in the CARS was considered ideal as a diagnostic cut-off score to identify cases of autism in Indian populations. This higher cut-off score has better diagnostic accuracy properties compared to the validation in Japan with a score of 30/30.5 predicting a sensitivity of 71%, a specificity of 75%, a positive predictive value of 77%, and a negative predictive value of 69%.^[14] Our score is also higher than the cut-off score of 27 recorded in another study.^[24] This difference in scores could be seen because of the child population (of under 8 years) versus adolescents (of above 13 years) with autism in our previous study.^[24] Also, other studies have used DSM-III/III-R/IV as reference criteria and this is the first time the CARS being validated against ICD-10 criteria possibly resulting in a higher diagnostic threshold. Etiological reasons could contribute to this higher score as well. Many participants in our sample had co-morbid conditions like intellectual disability and attention deficit hyperactivity disorder, which will be scored on the CARS items of 'intellectual functioning' and 'activity level', increasing the threshold score for the 'caseness'. The AUC associated ROC curve for a screening is considered a measure of the overall diagnostic efficacy of the test. In our study the AUC 0.81 (P=0.0001) suggested that the overall diagnostic accuracy of the CARS was high. The CARS (score of ≥33) was also found to be as accurate as ICD-10 in the detection of autism, thus demonstrating a high concordance rate. However, there was some overlap between autism and intellectual disability although other primary disorders like seizures, cerebral palsy and multiple disabilities in our population were clearly discriminated.

The test-retest reliability of 0.81 in this study was similar to the reported 0.85,^[25] although a relatively lower test-retest reliability of 0.71^[26,27] and a higher reliability of 0.90^[9,28] were also documented. The inter-rater reliability varied in the literature from 0.53,^[9] 0.62^[13] to 0.71^[26,27] and we recorded a better inter-rater reliability of 0.74.

The face and content validity of the CARS as a measure for autism has long been established by consensus among clinicians.^[17] The CARS items are shown to be consistent with all three symptom clusters of DSM-III which are highly consistent with ICD symptom clusters, therefore the content validity of the CARS in this study was as good as reported elsewhere.^[29] Our review of internal consistency for the CARS revealed that it ranged from 0.62 to 0.87 in the previous studies.^[13] Therefore, Cronbach's α coefficient of internal consistency of 0.79 was comparable with other cultures and demonstrated that the items of the CARS were homogeneous while used in the Indian context. However, the item-total correlation in determing the role of each of the 15 CARS items in the entire test showed that the items of activity level and intellectual functioning were ineffective in contributing to the total score. This suggests that the two items are not measuring the same construct as the test and these items are not able to successfully discriminate between those who performed well and those who performed poorly in the CARS. These items could also measure the intellectual disability (95%) and attention deficit hyperactivity disorder (53%), two more highly prevalent co-morbid conditions among children with autism in

India

^[30] than the core construct of autism itself, thus excluding these items could improve the construct validity. This hypothesis needs further testing.

The convergence between the CARS and BKT or GDS score was moderately high, suggesting that the construct of childhood disability with impairment in socialization and communication measured by the CARS and BKT or GDS are theoretically related to each other, and thus were also observed to be related to each other in our study. Moderate correlation between CARS scores and developmental level has been demonstrated in the past years, indicating a significant variance.^[11] Our finding is also comparable with the convergent validity of 0.40 seen in a previous study comparing the CARS and Autism Diagnostic Interview-Revised^[28] and 0.39 for the CARS and Autism Behavior Checklist.^[31] The negative correlation between CARS and ACTeRS shows CARS has the ability to differentiate theoretical constructs that are different from its own.

Factor analysis of the CARS has yielded diverse structure models across different studies.^[8,12,32,33] For instance, DiLalla et al^[8] demonstrated a 3-factor structure explaining 64% of the variance and the other two studies along with ours documented a 5-factor structure explaining 40% and 57.2%^[12,32] respectively. Our item loading, the 5-factor structure and 65.3% of the variance being explained, makes it closer to the existing model offered by Stella et al.^[32] Main methodological differences, such as population characteristics, factor-extraction and factor-retention procedure, language version, and statistical approach, are aspects that might explain the variability of findings across these factor analyses of the CARS.^[12] Comparison of the factor structure of the CARS in other non-Western cultures could not be done as such data are not available in published literature.

The first limitation is the study was conducted in a tertiary-care hospital; the participants may not be representative of the children with autism in the general population. Therefore, using this study as the focus, further studies on community samples to establish the sensitivity and specificity of the CARS are required. Second, although the sample size of 103 participants is adequate for an exploratory factor analysis, a larger sample size can generate more stable factor structure models, thereby improving the confidence in the validity of identified constructs as well as providing more accurate values of sensitivity, specificity, and predictive values. Third, the high prevalence of autism in this sample should theoretically improve the statistical power and stability of the analyses regarding sensitivity and specificity. In community and clinical samples with different prevalence rates of autism, the changing prevalence might lead to different diagnostic scores. Fourth, using another scale that measures autism would have given the best convergent validity, and using specific domains of socialization and communication from the BKT and GDS would have given better convergent validity than what has been documented in this study as the symptom clusters of communication and socialization of autism are likely to converge with these two domains of the BKT or GDS.

The present study suggests that the CARS has strong psychometric properties in a high-risk sample of children for autism. Although CARS development predates the ICD-10 and many newer measures are available, its brevity, good psychometric properties, conceptual relevance, and flexible administration procedures lend support to the measure being used in India for screening procedures.

Ethical approval: This study was approved by the local Institutional Review Board.

Competing interest: There is no conflict of interest. No benefits in any form have been received from any extramural grants directly or funds indirectly.

Contributors: Russell PS was involved in the conception and designing of the study, data analysis, data interpretation, and drafting the article. Russell PS will also act as the guarantor. Daniel A, Russell S, Mammen P, Abel JS, Raj LE, Shankar SR, and Thomas N were involved in the conception and designing of the study as well as revising the article.

1 Wing L, Potter D. The epidemiology of autistic spectrum disorders: is the prevalence rising? Ment Retard Dev Disabil Res Rev 2002;8:151-161.

2 Prior M. Is there an increase in the prevalence of autism spectrum disorders? J Paediatr Child Health 2003;39:81-82.

3 Thacker N. Indian pediatrics and child survival. Indian Pediatr 2007;44:647-648.

4 Elliman DA, Dezateux C, Bedford HE. Newborn and childhood screening programmes: criteria, evidence, and current policy. Arch Dis Child 2002;87:6-9.

5 Baird G, Charman T, Cox A, Baron-Cohen S, Swettenham J, Wheelwright S, et al. Current topic: screening and surveillance for autism and pervasive developmental disorders. Arch Dis Child 2001;84:468-475.

6 Ozonoff S, Goodlin-Jones BL, Solomon M. Evidence-based assessment of autism spectrum disorders in children and adolescents. J Clin Child Adolesc Psychol 2005;34:523-540.

7 Schopler E, Reichler RJ, Renner BR. Childhood Autism Rating Scale. Los Angeles: Western Psychological Services, 1988.

8 DiLalla DL, Rogers SJ. Domains of the Childhood Autism Rating Scale: relevance for diagnosis and treatment. J Autism Dev Disord 1994;24:115-128.

9 Nordin V, Gillberg C, Nyden A. The Swedish version of the Childhood Autism Rating Scale in a clinical setting. J Autism Dev Disord 1998;28:69-75.

10 Rellini E, Tortolani D, Trillo S, Carbone S, Montecchi F. Childhood Autism Rating Scale (CARS) and Autism Behavior Checklist (ABC) correspondence and conflicts with DSM-IV criteria in diagnosis of autism. J Autism Dev Disord 2004;34: 703-708.

11 Perry A, Condillac RA, Freeman NL, Dunn-Geier J, Belair J. Multi-site study of the Childhood Autism Rating Scale (CARS) in five clinical groups of young children. J Autism Dev Disord 2005;35:625-634.

12 Magyar CI, Pandolfi V. Factor structure evaluation of the childhood autism rating scale. J Autism Dev Disord 2007;37: 1787-1794.

13 Kurita H, Miyake Y, Katsuno K. Reliability and validity of the Childhood Autism Rating Scale―Tokyo version (CARS-TV). J Autism Dev Disord 1989;19:389-396.

14 Tachimori H, Osada H, Kurita H. Childhood autism rating scale―Tokyo version for screening pervasive developmental disorders. Psychiatry Clin Neurosci 2003;57:113-118.

15 Cochrane AL, Holland WW. Validation of screening procedures. Br Med Bull 1971;27:3-8.

16 Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Standards for Reporting of Diagnostic Accuracy. Clin Chem 2003;49:1-6.

17 Sponheim E. Changing criteria of autistic disorders: a comparison of the ICD-10 research criteria and DSM-IV with DSM-III-R, CARS, and ABC. J Autism Dev Disord 1996;26: 513-525.

18 World Health Organization. The International Classification of Disease (ICD-10) Classification of Mental and Behavioral Disorders: Clinical Descriptions and Diagnostic Guidelines. Delhi: Oxford University Press, 1992.

19 Kamat VV. Measuring intelligence of Indian Children. Bombay: Oxford University Press, 1967.

20 Gesell A. The first five years of life: the preschool years. New York: Harper and Brothers Publishers, 1940.

21 Ullmann RK. ACTeRS useful in screening learning disabled from attention deficit disordered (ADD-H) children. Psychopharmacol Bull 1985;21:339-344.

22 Raubenheimer JE. An item selection procedure to maximize scale reliability and validity. SA Journal of Industrial Psychology 2004;30:59-64.

23 Garfin DG, McCallon D, Cox R. Validity and reliability of the Childhood Autism Rating Scale with autistic adolescents. J Autism Dev Disord 1988;18:367-378.

24 Mesibov GB, Schopler E, Schaffer B, Michal N. Use of the childhood autism rating scale with autistic adolescents and adults. J Am Acad Child Adolesc Psychiatry 1989;28:538-541.

25 Sturmey P, Matson JL, Sevin JA. Brief report: analysis of the internal consistency of three autism scales. J Autism Dev Disord 1992;22:321-328.

26 Schopler E, Reichler RJ, DeVellis RF, Daly K. Toward objective classification of childhood autism: Childhood Autism Rating Scale (CARS). J Autism Dev Disord 1980;10:91-103.

27 Sevin JA, Matson JL, Coe DA, Fee VE, Sevin BM. A comparison and evaluation of three commonly used autism scales. J Autism Dev Disord 1991;21:417-432.

28 Saemundsen E, Magnusson P, Smari J, Sigurdardottir S. Autism Diagnostic Interview-Revised and the Childhood Autism Rating Scale: convergence and discrepancy in diagnosing autism. J Autism Dev Disord 2003;33:319-328.

29 Volkmar FR, Klin A, Siegel B, Szatmari P, Lord C, Campbell M, et al. Field trial for autistic disorder in DSM-IV. Am J Psychiatry 1994;151:1361-1367.

30 Kalra V, Seth R, Sapra S. Autism―experiences in a tertiary care hospital. Indian J Pediatr 2005;72:227-230.

31 Eaves RC, Milner B. The criterion-related validity of the Childhood Autism Rating Scale and the Autism Behavior Checklist. J Abnorm Child Psychol 1993;21:481-491.

32 Stella J, Mundy P, Tuchman R. Social and nonsocial factors in the Childhood Autism Rating Scale. J Autism Dev Disord 1999;29:307-317.

33 Szatmari P, Merette C, Bryson SE, Thivierge J, Roy MA, Cayer M, et al. Quantifying dimensions in autism: a factor-analytic study. J Am Acad Child Adolesc Psychiatry 2002;41:467-474.