American Sign Language Expressive Skills Test

Overview

This test is the second in a series of language assessments in American Sign Language (ASL) produced by Northern Signs Research and the research team lead by Charlotte Enns at the University of Manitoba, Canada. The first assessment, the ASL Receptive Skills Test (ASL-RST) (Enns, Zimmer, Boudreault, Broszeit, & Rabu, 2013), measures children’s understanding of ASL morphology and syntax. The purpose of the ASL Expressive Skills Test (ASL-EST) is to complement the ASL-RST by assessing children’s expressive signed language, specifically their ability to produce a signed narrative with appropriate ASL grammatical features. The ASL Expressive Skills Test is based on the Assessing British Sign Language Production Test (Narrative Skills) (Herman, et. al., 2009). Our research team is very grateful for the work done by the BSL team to create the original format (specifically the Spider Story video) and scoring procedures for BSL grammar structures and narrative skills that we were able to build on and adapt to use in ASL.

The test adaptation process included the following steps (this process is further outlined and compared to the German Sign Language adaptation in Enns, Kolbe, & Becker, 2021):

Consultation with ASL linguists and Deaf teachers to determine the feasibility of test adaptation (would the test task elicit appropriate ASL structures/narrative?)
Pilot testing with Deaf adults (fluent in ASL) to determine the target structure expectations for the storytelling task, and then revising the scoring procedures accordingly
Creation of three videos (based on original BSL Spider Story) to provide parallel versions of the test for comparison over time
Conducting pilot testing with children (n=47) using all three stories/versions of the test to ensure that scores could be compared across the different stories
Revising scoring procedures based on pilot testing; changes, compared to BSL, included similar Grammar items, more emphasis on Role Shift, and combining Narrative Content and Structure
Collecting normative data on a representative sample (n=215) of deaf children across Canada and the United States

The reason for developing three different stories/versions of the test (parallel elicitation videos) was to prevent children from becoming overly familiar with one story through the process of longitudinal testing, or pre- and post-testing procedures for research purposes.

The ASL Expressive Skills Test is for any children using ASL. It is primarily for deaf children (aged 4 – 13 years) who are learning ASL as a first language, or any deaf children in this age group who use ASL, even with limited exposure to ASL or exposure to other signing systems. The test could also be used with older deaf children with learning disabilities or delayed/disordered language, although a comparison to the normative group would not be possible. With these children, the test could be used to determine their visual language developmental needs and strengths. The test might also be used with hearing children who use ASL as their primary way of communicating.

Testers must be fluent in ASL, meaning that they are native or near native signers (deaf people). Hearing people can take the training workshop and help with scoring, but the test must be administered, i.e., the language sample elicited, by a deaf person. All testers must complete the online training workshop and become certified as ASL Expressive Skills Test examiners (see Northern Signs Research website for workshop dates and details).

Test Administration

The ASL Expressive Skills Test uses a stimulus (video) to encourage children to tell a story, which is recorded for later analysis and scoring. The child watches the three-minute video, where two characters act out the story through gestures and facial expressions, no signing or talking. When the child finishes watching the video, they tell the story in ASL to a deaf person who is not familiar with the video. It is important for the child to believe that the listener has not seen the video so that they tell a more detailed story. Following the child’s telling of the story, which is videotaped for later analysis, the child must also answer three comprehension questions. The child’s responses to these questions are also video recorded. The three stimulus stories (videos) can be used interchangeably because they are parallel in terms of the story structure, number of events, two characters, and the types of ASL grammatical features that are elicited.

Scoring

After the session with the child, the analysis is completed by the trained tester repeatedly viewing the video recording and following the standardized scoring system (total raw score = 60). The Score Form is in an electronic Excel workbook format, and includes four tabs:

Cover page (background information, comments, section scores, percentile results, overview of strengths and needs)
Grammar – Total 30 points (spatial verbs- 13, directional verbs -3, agreement verbs -6, aspect - 3, manner -4, additional vocabulary -1)
Role Shift – Total 6 points (facial expression/mannerisms – 2, shoulder/head shifting – 2, eye gaze/spatial location – 2)
Narrative – Total 18 points (setting - 2, initiating events - 6, complicating actions – 2, climax – 4, resolution – 1, sequence – 1, evaluation – 1, real-life connection – 1) and Questions – Total 6 points.

Each individual child’s score can be compared to the normative group and be given a percentile ranking for their age in each of the categories (Grammar, Role Shift, Questions, Narrative) as well as for their total score, thereby indicating the child’s overall level of expressive ASL development and more diagnostic information related to the specific categories.

Due to the variation in deaf children’s language acquisition, there are a range of scores that are “normal” for each age. For this reason, the standard deviations of the mean are large, and the medians (50^th percentiles) are better estimates of age-appropriate levels than the means. The ASL-EST groups the percentiles into five age groups: a) 4 years, b) 5 years, c) 6 years, d) 7 & 8 years, and e) 9 years and above. The first three groups (4, 5, and 6 years) cover only one year because there is significant growth and development in language and narratives abilities at these ages. The remaining two groups cover several years as development is leveling off (less variable) at these older ages. Each age group has raw scores that fit into the five percentile categories: 10^th, 25^th, 50^th, 75^th, and 90^th percentiles.

Normative Sample

All the testing was completed in schools for the deaf in Canada and the United States, by members of the research team. All children were deaf and had been exposed to ASL before the age of 3 years. In order to ensure that children had average nonverbal intelligence, the Test of Nonverbal Intelligence (TONI-4) (Brown, Sherbenou, & Johnsen, 2010) was administered to most children, unless they were too young (less than 5 years of age) or the school had conducted similar testing within six months. Children with scores less than one standard deviation below the mean, including those with diagnosed disabilities (e.g., autism), were excluded.

We collected data from a total of 215 deaf children between the ages of 3.5 years to 13.9 years of age. The sample was not exactly gender balanced, with 125 girls and 90 boys, but we found no statistical evidence for differences in test performance based on gender (ANOVA Sig=0.933, greater than 0.05). Also, most of the children (n=144) had at least one deaf parent, with the remaining 71 children having non-deaf parents. Again, no statistical evidence was found for differences in test performance between children with deaf parents or non-deaf parents (Asymp. Sig=0.659, greater than 0.05). It is important to keep in mind that we had a selective sample, and only children with early exposure to ASL (before the age of 3 years) were included. We recognize that controlling for early exposure to ASL and nonverbal IQ resulted in a sample that may not be representative of the general population of deaf children, as considerable variability exists in terms of deaf children’s cognitive and language abilities. However, for this test we wanted to have a normative sample that represents what achievements are possible when children have early and rich full access to language.

Test Validity

To check if the test was measuring what it was designed to measure (children’s ASL abilities), scores (based on the categories of average, above average, below average) from the ASL Expressive Skills Test were compared with the same children’s scores on the ASL Receptive Skills Test using a Pearson’s correlation. A highly significant correlation (0.91, p<0.01) was found, suggesting good concurrent test validity.

Test Reliability

The reliability of the ASL Expressive Skills Test was investigated using inter-scorer comparisons and test-retest analyses with the three versions of the test.

Inter-scorer reliability was assessed by having 10% (30 videos) of the data independently scored by two different trained testers and comparing the results. Statistical analysis using Pearson’s correlation resulted in a highly significant correlation of 0.87 (p<0.01), indicating inter-scorer reliability was very good.

The test-retest reliability was based on the 47 children participating in the pilot testing who each completed two versions of the test (retold two different stories) within the same testing session. Parametric statistics (ANOVA) were used to compare the children’s scores between all combinations of the three stories, and no statistical evidence was found for any differences (Sig=0.288, greater than 0.05). A Test of Homogeneity of Variances was used to validate the assumption of the homogeneity for ANOVA and had similar results (Sig=0.234, greater than 0.05). These analyses indicate strong test-retest reliability within participants and across all three versions of the test.