Psychometric properties of VEAN-Hi (Valoración del Espectro Autista para Hispanos), Autism Spectrum Assessment for Hispanic Children (ASA-HiCh) A free open access instrument

Corresponding Author:

Lilia Albores-Gallo MD. PhD
Children’s Psychiatric Hospital, Genetic Epidemiology, Clinical and Community, Av San Buenaventura no: 86. Col. Belisario Dominguez Tlalpan, CP. 14080 Mexico
Tel: (+521 )55-23162517

Abstract

Introduction: Early intervention in children with Autism Spectrum Disorder (ASD) improve prognosis. Although instruments can be translated, adapted and validated their performance is suboptimal compared to the original studies. The need to design culturally and context appropriate instruments for earlier assessment of Hispanic children with autism is urgent.

Methods: A validity study with a case control design with 70 ASD children and 16 children with Typical Development (TD) was performed. Autism diagnosis was confirmed with the Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime version (Kiddie SADS-PL/ASD- supplement) and the ADI-R.

Results: Internal consistency examined by Cronbach’s alpha for the 26 items of the total sample was α=0.81, p<0.01. The one-week test-retest reliability was 0.92, (95% CI 0.83 - 0.97), p<0.001. Using the criterion of ASD diagnosis vs. TD group a ROC curve (receiver operating characteristic) was constructed. With a cutoff of the Total ASA-HiCh score of 17, yielded an area under the curve of 0.84 (95% CI 0.74-0.94), sensitivity of 75.6 %, specificity of 87.5% and a positive predictive value of 93.9% and negative predictive value of 58.3%. For the K-SADS/ ASD classification diagnosis and the ASA-HiCh score 17 the kappa value was k= 0.60, p<0.01. For children <9 years-old Kappa coefficient was k=0.72, p<0.01, and for children <5 years-old k=0.87, p<0.01. In study 2 the kappa value for DSM-IV criteria was k=0.39, p<0.001, and for DSM-5 criteria was k=0.88, p<0.001. The medians for the total score of ASA-HiCh between ASD and TD groups, were analyzed by the Mann-Whitney U test, and were 34.51 and 14.88 respectively (U=102.0 Z -4.02, p=0.001).

Conclusion: The ASA-HiCh demonstrated good psychometric properties, future studies are necessary to determine the discriminant validity between ASD and other psychiatric disorders.

Keywords

Psychometric, Autism Spectrum, Hispanic children, Autism diagnosis

Introduction

In Mexico, the prevalence of autism spectrum disorders (ASD) is 0.87% [95% CI (0.62, 1.1%)] [1], a figure that coincides with the reported prevalence for the Hispanic population in the US [2,3] and higher than some Latin American countries like Brazil, Venezuela and Argentina [4-6]. Furthermore, this study conducted in our country showed that parents noticed the symptoms in the first 24 months of life but the diagnosis was not made until after 3 years of age, although most families had contact with health professionals before 24 months [1]. Although it is assumed that physicians and pediatricians can notice early symptoms of autism, the reality is that ASD are not included in the curriculum of Mexican general practitioners and pediatricians [7] and there are few standardized strategies to assess and treat these children [8]. On the other hand, Mexican versions of some tools for assessing autism, show suboptimal performance and few professionals know how to use them because of the lack of access and low usage. Very often, Spanish-speaking researchers validate instruments to address the ethical claim of assessing the transcultural validity of the instruments, since several centers routinely use them without knowing their psychometric properties [8]. In 2014, Soto analyzed 12 studies on the translation and validation of instruments for autism and found that, in most, at least minor modifications had to be made to increase the understanding of the questions while preserving their conceptual equivalence [9-11]. Sometimes these changes impact the retranslation process, but are necessary to achieve functional equivalence to the original version. The following describes some of these tools available in Mexico.

The instrument for Autism Detection in Early Childhood (ADEC) [12] is an interactive observational tool for detecting autism in children from 12 to 36 months old, designed in Australia that does not require training. In Mexico, the instrument was translated, adapted and validated on 115 children between 15 and 72 months of age using the ADI-R [13] and the Childhood Autism Rating Scale (CARS) [14] as gold standard to confirm the diagnosis. The sensitivity for the first and second phase of the study was 0.79-0.94 and specificity of 0.88-1.00. The instrument manual is available in English but the publication of the Spanish version is pending, and perhaps that is why the instrument is very little known in Mexico [15].

The Social Responsiveness Scale [16] evaluates the social responsiveness of a child aged 4-18 years from the parent and teacher report. It consists of 65 items on a Likert scale from 1 to 4, 17 questions have a reverse wording. Various studies show that a cutoff raw scores of 70 for males and 65 for females are appropriate for identifying children of the community while a score >85 correctly identifies children with developmental problems in the clinical setting. This instrument was validated in Mexico by Fombonne, Marcin, Bruno, Tinoco, & Marquez in 2012 [17]. The internal consistency for the parent and teacher version was 0.92. The correlation of scores for the parent and teacher version was moderate r = 0.49; P <0.001. The discriminant validity of the instrument was established by constructing ROC curves (Receiver Operating Characteristic) in which the area under the curve (AUC) is 0.96 for the parent and teacher version. This study identified that the average score for the Mexican sample was significantly higher compared to the US and Germany, suggesting that future studies need to determine the best cutoff point for the Mexican population.

In 2012, Albores-Gallo validated the Mexican version of the M-CHAT (Modified Checklist for Identification of Autism) [18]. This screening tool consists of 23 items of which six [2,7,9,13-15] are considered key for screening autism. Although the Mexican version of the M-CHAT showed an internal consistency of 0.76 for the total items and 0.70 for the six key items, these figures are suboptimal according to Glascoe [19], and only one of the key items was discriminant in the Mexican population [18]. This result is consistent with validation studies conducted in China [20], Japan [21], Sri Lanka [22], and Spain [23] and scrutiny studies with the M-CHAT in the US [24,25]. The inconsistency of the key items to identify autism in different countries is an obstacle to perform international collaboration studies. In addition, the Hispanic population in the United States (mostly Mexican) has an items failure rate higher compared to non- Hispanics (28.5% vs.17.4%), and similar figures when comparing Spanish and English versions (30.3% vs.20%) [26], suggesting significant effects of translation and adaptation to the Spanish language [27].

Although instruments can be translated, adapted and validated in our population, their performance is suboptimal with respect to the original studies. The need to design culturally appropriate instruments for earlier assessment and identification of children with autism is urgent.

The purpose of this study was to develop a screening instrument with excellent psychometric properties for the detection of ASD, culturally appropriate for the Spanish-speaking population, useful for clinical and epidemiological purposes, and freely accessible under the terms suggested by various researchers [28-30].

Material and Methods

Approval from the Hospital Research Committee and authorization from community health centers were obtained. All parents of children signed an informed consent prior to participation in the study.

▪ Study population and sample size

The study 1 sample was 80 children, 3 children from the clinical sample did not complete the assessment and 4 questionnaires from the community were eliminated because they were incomplete. In the study 1, 68 children were evaluated with a range of 3-17 years (M = 8, SD 3.88) with a presumptive diagnosis of any ASD (Autistic disorder, Asperger’s disorder or pervasive developmental disorder not otherwise specified) (Table 1). These were corroborated by the ASD supplement of the diagnostic interview Schedule for Affective Disorders and Schizophrenia for School-Age Children- Present and Lifetime version (K-SADS-PL/ ASD) [31]. Ten parents were chosen randomly for the test-retest one week later. The group identified as typically developing (TD) consisted of children (N=16) without developmental disorders recruited in a general medicine service of the community. Those children with sensory impairments, such as blindness, deafness, mental retardation or genetic syndromes, were excluded.

Table 1: Sociodemographics for ASD and TD groups.

The study 2 sample was composed by 18 children with an age range of 2.6 to 12 years and a mean age of 5.8, SD 3.10, 88.9% were masculine (N=16), participating in larger study of gene association. These children were evaluated with Autism Diagnostic Interview-Revised (ADI-R) and a clinical interview with DSM-IV [32] and DSM-5 [33] criteria.

▪ Instruments

The diagnostic interview K-SADS-PL contains a section for assessing autism spectrum disorders in children and adolescents that does not require training. It allows for discrimination between the diagnoses of Autistic disorder, Asperger’s disorder, pervasive developmental disorder not otherwise specified (PDDNOS) and childhood disintegrative disorder.

In 2014, Zavaleta and Albores [34] studied the inter-rater reliability of the Mexican K-SADSPL/ ASD version. After performing the Spanish translation, the reverse translation into English, the instrument was adapted and tested in a sample of 40 children and adolescents with an age range of 4-17 years and a presumptive diagnosis of autism spectrum disorder. The interrater reliability for the “present and past time” for the following diagnoses was: autism 0.79 and 0.74; Asperger disorder 0.85 and 1.0; pervasive developmental disorder not otherwise specified (PDD-NOS) 0.72 and 0.41 respectively. The kappa coefficients for the expert’s inter-rater reliability range from good to excellent for the following diagnoses in the present and in the past: Autism 0.89 and 0.87; Asperger 0.77 and 1.00; PDDNOS 0.69 and 0.64 respectively. Its administration lasts 30 minutes and recollects information from the parent and the child or adolescent. To provide the best clinical estimate, the clinician can incorporate data from observation of the child during the interview.

▪ ADI-R

Autism Diagnostic Interview (Autism Diagnostic Interview-Revised) (ADI-R) [13]. The ADIR is a semi structured interview that should be administrated by a clinician with experience evaluating children with autism. It is the gold standard for autism diagnosis of children and adults with mental ages older than 18 months.

The interview is organized according to the DSM-IV criteria. It contains questions to explore the child’s developmental history and symptoms of autism. The ADI-R algorithm generates scores for the three main domains of autistic symptomatology: (A) qualitative problems of reciprocal social behavior, (B) delayed language development, and (C) stereotyped behaviors and restrictive interests. It has an interrater reliability of 0.83 to 0.94 (Table 2).

Tabla 2: Factors items and variance.

The autism diagnosis in the second clinical group was confirmed through the ADI-R, in addition to a semi structured clinical interview with DSM-IV and DSM-5 criteria. Clinicians who conducted the interviews were blind to the questionnaire results. Inconsistencies between both criteria were solved by consensus.

Procedure

The international literature on the instruments was revised to assess autism in children and adolescents and items that represented the 3 dimensions of autism spectrum disorders according to DSM-IV were drafted.

Some of the CHAT and M-CHAT items were rephrased to make them more detailed. For example, the M-CHAT item which assess if child has interest in other children was rephrased to ask if the child has interest in playing with other children of the same age. Surprisingly in our previous -validity study of the M-CHAT- this item was interpreted as the child (usually a boy) being romantically interested in other children (boys), as many parents ask us for clarification.

The Japanese validity study [21] also showed problems with this item as cited by Soto et al [9]. Other items which explores pointing (a behavior which is considerate rude in some countries including Mexico) [35] were rephrased to provide a more detailed acceptable context. For example: Does your child use his finger to point out things he is interested such as toys, a fountain, balloons etc. An additional item to explore protoimperative pointing to differentiate from protodeclarative pointing was necessary because in our experience Mexican parents confuse both behaviors.

Subsequently a list of 30 items was drafted, the instrument whose answers have a Likert format to measure the frequency of symptoms (hardly ever, sometimes, often) because many Mexican informants find difficult to rate symptoms in a yes/no format.

The questionnaire is self-administered to be answered by parents or guardians (or the person who has more knowledge of the child’s behavior).

A pilot study was conducted so that informants could identify those items considered difficult to understand or confusing.

According to the results and comments obtained, the authors corrected the instrument’s drafting, whose final version consists of 26 items in a likert scale and 4 additional items to explore the age of first words, language regression yes/no, general development yes/no and atypical behavior yes/ no format.

For each item, a value of 0 is assigned to the answer “hardly ever,” 1 when answering “sometimes” or 2 if they answered “almost always.” Except for items 1, 2, 3, 4, 5, 7, 8, 9, 11, 12, 13, 14, 15, 17, 19, and 22 which are reverse scored (2 for the answer “hardly ever, “ 1 when responding “sometimes” and 0 if they answered “ almost always”), adding up the total of the items (recoded and not recoded) for the final grade.

Data Collection

All the parents answered the Autism Spectrum Assessment for Hispanic Children (ASA-HiCh) and signed the informed consent to participate in the study, 10 parents were randomly chosen from the clinical population (with ASD diagnosis) for the test-retest of the ASA-HiCh instrument. Subsequently, two child psychiatrists blind to the ASA-HiCh scores, administered the supplement for Autism Spectrum Disorders K-SADS-PL to the parents of clinical group, to confirm the diagnosis of autism. Parents of children from the general population completed DSM-IV checklists of ASD symptoms to confirm that children had typical development.

Statistical Analysis

Internal consistency was determined by Cronbach’s alpha for the total sample. To analyze the test-retest reliability, intraclass correlation coefficients of the items in the first and second application of the ASA-HiCh instrument were calculated.

To analyze the difference in the medians of the ASA-HiCh total score between groups the Mann-Whitney U test was used.

Through a ROC curve, the area under the curve and its confidence intervals were investigated, the best cutoff point for discriminating the ASD group with TD group was determined, and the sensitivity and specificity were calculated for it.

For criterion validity, kappa coefficients were calculated between the categorical diagnoses of: K-SADS-PL /ASD, the DSM-IV and DSM-5 clinical interviews and the categorical dichotomic results of the ASA-HiCh instrument (ASD vs. typical development) using the best cutoff point observed by ROC curve.

Construct validity was performed using factor analysis by principal components method with varimax rotation of the items of the total sample.

Results

Internal consistency and test-retest reliability

Internal consistency examined by Cronbach’s alpha for the 26 items of the total sample was α = .81, p =.001.

For the test-retest reliability with one week in between both applications, through the ICC intraclass correlation coefficients were 0.92, CI 95% .83 a 0.97, p < 0.001.

▪ Construct validity

A principal component analysis with varimax rotation was selected, to look for the presence of latent unobserved variables and determine the structure of the scale. The method of Kaiser [36] (eigenvalues greater than 1) was used to retain the factors. A 7 (seven) factors solution explained 64.06% of the total variance with factor loadings between 0.32 to 0.91 per item These 7 factors are described below:

• Abnormal relationship with people: includes items 15, 14, 26, 23 and 21 α = .74

• Communication/Language: items 8, 25, 24 and 18. α = .83

• Joint attention: items 4, 5 and 22. α = .77

• Sensory self-stimulation and hypersensitivity: items 16, 19, 10 and 6. α = .68

• Symbolic play: items 3, 12, 2 and 9. α = .54

• Self-absorption: items 17, 20 and 23. α = -.20

• Social Reciprocity: items 7, 1 and 11. α = .53

▪ Criterion validity

The criterion validity for ASA-HiCh was demonstrated by calculating the ROC curve (receiver operating characteristic), taking the different cutoffs from TOTAL score of ASAHiCh by using the criterion of ASD group versus the Typical Development control group through the DSM-IV criteria.

The cutoff point selected was the one that showed the best balance between sensitivity and specificity which corresponded to a score greater than or equal to 17, and an area under the curve of 0.84 (95% CI .74 - .94) with a sensitivity of 75.6 % and specificity of 87.5% and a positive predictive value of 93.9% and negative predictive value of 58.3% (Figure 1).

The Kappa value was 0.60, p<.01 between the ASA-HiCh score and the allocation to one of two groups (ASD or TD), confirming the diagnosis in the clinical group by the interview K-SADSPL supplement ASD. For children less than 9 years old Kappa coefficient was higher k=0.72, p<0.01, and even higher for children less than 5 years old k=0.87, p<0.01,

Kappa value and the clinical interview with DSM-5 criteria (ASD vs No-ASD) was k=0.88, p<0.001, and for DSM-IV (Autism, Asperger and PDD-NOS, No-ASD) k=0.39, p<0.0001, (Table 3).

Table 3: Convergent Validity. Kappa coefficients between K-SADS, DSMIV, DSM-5 and ASA-HiCh cutoff score 17.

▪ Discriminant Validity

The medians for the total score of ASA-HiCh between clinical and community groups, were analyzed by the Mann-Whitney U test, and were 34.51 and 14.88 respectively (U = 102.0 Z -4.02, p<.01).

Discussion

In this study we analyzed the psychometric data for the instrument Autism Spectrum Assessment for Hispanic Children (ASA-HiCh). Internal consistency, test-retest reliability, criterion discriminant and construct validity were investigated in a sample of children diagnosed with Autism Spectrum Disorders and a community control group with typical development. The ASA-HiCh scale obtained a good internal consistency α = 0.81, this performance is similar to other screening instruments for ASD like the Mexican version of CHAT-M, with an alpha of 0.85 and the ABC Mexican validity study. The test-retest reliability was excellent with r = 0.92 and using intraclass correlation coefficients with confidence interval from 0.83 to 0.97, these results confirm that the construct of ASD has stability over time as shown by a recent study that found that only 9% of children with autism dx lose that diagnosis.

The criterion validity between the ASA-HiCh instrument and the K-SADS-PL/TEA showed adequate properties with the cutoff = 17 (sensitivity 75.6%, specificity 87.5% and Kappa 0.60), these being appropriate figures to assess ASD [19] according to the fact that the construct is very complex and heterogeneous. The higher kappa coefficients values obtained in children with ages lower than 5 years, suggests the instrument performance is better in youngsters. The highest kappa value for the DSM-5 could be biased because the children in the second study had a lower mean age than children in the first study. Adding more cases and controls could determine if the instrument has in fact more convergent validity with the DSM-5 criteria than the DSM-IV. Another possibility is that including items for assessing sensorial hypo and hypersensitivity could explain the high concordance with the DSM-5.

Future studies should analyze if girls need a different cutoff to maximize sensitivity and specificity.

The 7 subscales inferred by factor analysis have face validity; however, in the future their usefulness should be tested.

The ASA-HiCh has the additional advantage of being validated in a Mexican sample of children and not being a translation of existing instruments that often do not take into account cultural differences. Mexicans are the largest group (63%) of Hispanics in the United States followed by Puerto Rican (9.2%), Cuban (3.5%), and Salvadoran (3.3%) Dominican (2.8%) Guatemalan (2.1%) and Colombian (1.8%) [37]. Given the uneven performance of many instruments to assess ASD in Hispanics [26,38], the ASA-HiCh is a promising instrument. Future studies should evaluate the Spanish equivalence with other Latin American countries. The ASA-HiCh is an instrument that was designed to be freely available to be used in Spanish-speaking countries that share social characteristics, and infrastructure for medical and educational services.

Limitations

This study has several limitations that deserve careful analysis. The design we used to validate this instrument was a cross-sectional study of cases and controls that often raises the positive predictive value and decreases the negative predictive value [39] by including a high proportion of children with autism as was our result (VPP 93.9% vs .VPN 58.3%). In addition, most (but not all) children who participated in this study were without medication and therefore, it cannot be ruled out that some scores on certain symptoms might be inaccurate. Therefore, it is necessary to carry out additional studies to analyze the effect of psychiatric comorbidities and intellectual disability in the discriminant validity and sensitivity to change of the scale in response to pharmacological or behavioral treatment, and the predictive validity (to be used as a screener in the community).

The reduced number of girls made it impossible to analyze the psychometric data of the instrument by sex. It is common that autism in girls is not as well diagnosed as boys according to some researchers [40,41]. Recent studies confirm that compared with boys, girls with Asperger’s syndrome are usually identified 20 months later [42], this despite the girls’ parents express concern at earlier stages [43]. For the same reason, some screening instruments use special cut points for females as they usually have lower scores [44] see Constantino & Charman for an excellent review [45]. The small sample size did not allow us to compare verbal vs. non-verbal children and there is some evidence that certain items are language influenced and therefore impact the total punctuation.

A strength of the study is that children were evaluated with two gold standards: the K-SADSPL/ ASD [31], valid and adapted for our country [34] and a semi-structured interview with the diagnostic criteria of DSM-IV, and the clinical group 2 was evaluated with ADI-R and a semi-structured interview which allows the diagnosis with DSM-IV and DSM-5 criteria.

Conclusions

The Autism Spectrum Assessment for Hispanic Children is an instrument with good psychometric data suitable for use with clinical or research purposes.

Harrison et al. [46] recommends more collaboration efforts to develop measures with cross-cultural validity. As recognized by Harrison [46] sometimes it is necessary to develop new measures to reach cross cultural equivalence for regions with common cultural and language background. We welcome Latin-American researchers’ initiatives to make contributions and possible modifications to the ASA-HiCh as a free open source instrument to reach transcultural validity as has been suggested by some researchers [28-30].

References

1. Fombonne E, Marcin C, Manero AC, et al.Prevalence of Autism Spectrum Disorders in Guanajuato, Mexico: The Leon survey. J. Autism. Dev. Disord46(5), 1669-1685 (2016).
2. Pedersen A, Pettygrove S, Meaney FJ, et al.Prevalence of autism spectrum disorders in Hispanic and non-Hispanic white children. Pediatrics129(3), e629-635 (2012).
3. Autism and Developmental Disabilities Monitoring Network Surveillance Year 2006 Principal Investigators, Centers for Disease Control and Prevention (CDC). Prevalence of autism spectrum disorders-Autism and Developmental Disabilities Monitoring Network, United States, 2006. Morb. Mortal. Wkly. Rep58(10), 01–20 (2009).
4. Paula CS, Ribeiro SH, Fombonne E, et al. Brief Report: Prevalence of Pervasive Developmental Disorder in Brazil: A Pilot Study. J. Autism. Dev. Disord 41(12), 1738-1742 (2011).
5. Montiel-Nava C, Peña JA. Epidemiological findings of pervasive developmental disorders in a Venezuelan study. Autism. Int. J. Res. Pract12(2), 191-202 (2008).
6. Lejarraga H, Menendez AM, Menzano E, et al.Screening for developmental problems at primary care level: a field programme in San Isidro, Argentina. Paediatr. Perinat. Epidemiol22(2), 180-187 (2008).
7. Facultad de Medicina UNAM.
8. Harris B, Barton EE. Autism services in Mexico: A qualitative survey of education professionals. Int. J. Sch. Educ. Psychol30(1),01-13 (2016).
9. Soto S, Linas K, Jacobstein D, et al.A review of cultural adaptations of screening tools for autism spectrum disorders. Autism19(6), 646-661 (2014).
10. Beaton DE, Bombardier C, Guillemin F, et al.Guidelines for the process of cross-cultural adaptation of self-report measures. Spine25(24), 3186–3191 (2000).
11. Banville D, Desrosiers P, Genet-Volet Y. Translating questionnaires and inventories using a cross-cultural translation technique. J. Teach. Phys. Educ.19(3), 374-387 (2000).
12. Hedley D, Young R, Angelica M, et al.Cross-cultural evaluation of the Autism Detection in Early Childhood (ADEC) in Mexico. Autism. Int. J. Res. Pract 14(2), 93-112 (2010).
13. Lord C, Rutter M, Le Couteur A. Autism Diagnostic Interview-Revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. J. Autism. Dev. Disord24(5), 659-685 (1994).
14. Schopler E, Reichler RJ, DeVellis RF, et al.Toward objective classification of childhood autism: Childhood Autism Rating Scale (CARS). J. Autism. Dev. Disord 10(1), 91-103 (1980).
15. Young RL. Autism Detection in Early Childhood:ADEC Manual. Australian Council of Educational Research (2007).
16. Constantino JN, Davis SA, Todd RD, et al.Validation of a brief quantitative measure of autistic traits: comparison of the social responsiveness scale with the autism diagnostic interview-revised. J. Autism. Dev. Disord33(4), 427-433 (2003).
17. Fombonne E, Marcin C, Bruno R, et al.Screening for Autism in Mexico. Autism. Res 5(3):180-189 (2012).
18. Albores-Gallo L, Roldán-Ceballos O, Villarreal-Valdes G, et al. M-CHAT Mexican Version Validity and Reliability and Some Cultural Considerations. ISRN. Neurol2012(1), 408694 (2012).
19. Glascoe FP. Screening for developmental and behavioral problems. Ment. Retard. Dev. Disabil. Res. Rev 11(3), 173-179 (2005).
20. Wong V, Hui LHS, Lee WC, et al.A Modified Screening Tool for Autism (Checklist for Autism in Toddlers [CHAT-23]) for Chinese Children. Pediatrics114(2), e166-e176 (2004).
21. Inada N, Koyama T, Inokuchi E, et al.Reliability and validity of the Japanese version of the Modified Checklist for autism in toddlers (M-CHAT). Res. Autism. Spectr. Disord 5(1), 330-336 (2011).
22. Perera H, Wijewardena K, Aluthwelage R. Screening of 18-24-month-old children for autism in a semi-urban community in Sri Lanka. J. Trop. Pediatr55(6), 402-425 (2009).
23. Canal-Bedia R, García-Primo P, Martín-Cilleros MV, et al.Modified Checklist for Autism in Toddlers: Cross-Cultural Adaptation and Validation in Spain. J. Autism. Dev. Disord 41(10), 1342-1351 (2010).
24. Yama B, Freeman T, Graves E, et al.Examination of the Properties of the Modified Checklist for Autism in Toddlers (M-CHAT) in a Population Sample. J. Autism. Dev. Disord 42(1), 23-34 (2012).
25. Miller JS, Gabrielsen T, Villalobos M, et al.The Each Child Study: Systematic Screening for Autism Spectrum Disorders in a Pediatric Setting. Pediatrics127(5), 866-871 (2011).
26. Windham GC, Smith KS, Rosen N, et al.Autism and Developmental Screening in a Public, Primary Care Setting Primarily Serving Hispanics: Challenges and Results. J. Autism. Dev. Disord44(7), 1621-1632 (2014).
27. Kimple KS, Bartelt EA, Wysocki KL, et al.Performance of the modified checklist for autism in toddlers in Spanish-speaking patients. Clin. Pediatr (Phila) 53(7), 632-638 (2014).
28. Ashwood KL, Buitelaar J, Murphy D, et al.European clinical network: autism spectrum disorder assessments and patient characterisation. Eur. Child. Adolesc.Psychiatry 24(8), 985-995(2015)
29. de Vries PJ. Thinking globally to meet local needs: autism spectrum disorders in Africa and other low-resource environments. Curr. Opin. Neurol 29(2), 130-136 (2016).
30. Durkin MS, Elsabbagh M, Barbaro J, et al.Autism screening and diagnosis in low resource settings: Challenges and opportunities to enhance research and services worldwide: Enhancing Autism Research and Services Worldwide. Autism. Res8(5), 473-476 (2015).
31. Birmaher B, Ehmann M, Axelson DA, et al.Schedule for affective disorders and schizophrenia for school-age children (K-SADS-PL) for the assessment of preschool children--a preliminary psychometric study. J. Psychiatr. Res 43(7), 680-686 (2009).
32. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. Washington, DC (2000).
33. American Psychiatric Association, American Psychiatric Association, editors. Diagnostic and statistical manual of mental disorders: DSM-5. 5th ed. Washington, D.C: American Psychiatric Association947 (2013).
34. Zavaleta-Ramírez P, Náfate-López O, Villarreal-Valdés G, et al.Confiabilidad interevaluador del K-SADS-PL-2009/trastornos del espectro autista (TEA). Salud. Ment 37(1), 461-466 (2014).
35. Author GC, Communication’ ‘Say Anything to Anyone Anywhere: 5 Keys to Successful Cross-Cultural. Gestures to Avoid in Cross-Cultural Business: In Other Words, “Keep Your Fingers to Yourself!”
36. Kaiser HF. The application of electronic computers to factor analysis. Measurement. 20(1), 141-151 (1960).
37. Bureau USC. U.S. Census Bureau, 2014 American Community Survey 1-Year Estimates. In American FactFinder.
38. Pedersen A, Pettygrove S, Meaney FJ, et al.Prevalence of Autism Spectrum Disorders in Hispanic and Non-Hispanic White Children. Pediatrics129(3), e629-e635 (2012).
39. Charman T, Gotham K. Measurement Issues: Screening and diagnostic instruments for autism spectrum disorders - lessons from research and practise. Child. Adolesc. Ment. Health 18(1), 52-63 (2013).
40. Lai M-C, Lombardo MV, Auyeung B,et al.Sex/Gender Differences and Autism: Setting the Scene for Future Research. J. Am. Acad. Child. Adolesc. Psychiatry54(1), 11-24 (2015).
41. Giarelli E, Wiggins LD, Rice CE, et al.Sex differences in the evaluation and diagnosis of autism spectrum disorders among children. Disabil. Health. J3(2), 107-116 (2010).
42. Begeer S, Mandell D, Wijnker-Holmes B, et al.Sex Differences in the Timing of Identification Among Children and Adults with Autism Spectrum Disorders. J. Autism. Dev. Disord43(5), 1151-1156 (2013).
43. Horovitz M, Matson JL, Sipes M. The relationship between parents’ first concerns and symptoms of autism spectrum disorders. Dev. Neurorehabilitation14(6), 372-377 (2011).
44. Kamio Y, Inada N, Moriwaki A, et al.Quantitative autistic traits ascertained in a national survey of 22 529 Japanese schoolchildren. Acta. Psychiatr. Scand128(1), 45-53 (2013).
45. Constantino JN, Charman T. Gender bias, female resilience, and the sex ratio in autism. J. Am. Acad. Child. Adolesc. Psychiatry51(8), 756-758 (2012).
46. Harrison AJ, Slane MM, Hoang L, et al.An international review of autism knowledge assessment measures. Autism (2016).