Web-based ASL Vocabulary Test

The American Sign Language Vocabulary Test (ASL-VT; Mann, Roy & Morgan, 2015) is an adaptation of the  British Sign Language Vocabulary test (BSL-VT; Mann, 2009; Mann & Marshall, 2012) to assess deaf children’s vocabulary knowledge “by specifically measuring the degree of strength of the mappings between form and meaning for items in the core lexicon” (Mann & Marshall, 2012, p. 1031). It was adapted in close collaboration with a panel of deaf and hearing experts of ASL as part of a study that explored the use of dynamic assessment procedures (mediated learning) with signing deaf children aged 6-10 years (Mann, Peña & Morgan, 2014; 2015).

 

Development of the instrument

Like the web-based BSL Vocabulary Test, the ASL-VT consists of four tasks that measure vocabulary knowledge: (1) meaning recognition, (2) form recognition, (3) meaning recall, and (4) form recall. The same items are used across all tasks. Given the smaller age range of the US target group for the present study (6-10 years), only a subset of 80 items was considered for adaptation, based on the performance of UK children in this age range. Following the discussions with the deaf experts, 66 of these 80 items were accepted for adaptation without further changes and could be translated directly to ASL. The remaining 14 items required changes or were replaced.

 

Like the BSL-VT, the ASL-VT consists of two receptive and two production tasks. The two receptive tasks use a multiple-choice format and can be self-administered. The two production tasks require an administrator, who scores each response, based on four options and also documents the given response in a text box on the computer screen, using English glosses.

The task formats are as follows: Meaning recognition task: test takers see a pre-recorded ASL sign followed by four pictures. They have to select the picture that corresponds to the meaning of the signed prompt (Video 1). In the form recognition task, test takers see a picture, followed by four pre-recorded ASL signs and have to select the sign that matches the meaning of the picture prompt (Video 2). In the form recall task, test takers see a picture and have to produce the corresponding ASL sign and in the meaning recall task children see a pre-recorded ASL sign and have to generate three ASL signs with an associated meaning.

Because the ASL-VT was part of a study that required pre- and post test measures, item were presented in two sets, A and B, with 40 items each. 

 

A pre-pilot and pilot of the completed set of test terms were carried out before finalizing the prototype of the adapted ASL vocabulary test. In the pre-pilot, two deaf adults completed all vocabulary tasks and provided feedback on any items they considered problematic and/or suggestions of format-related changes. This was done as an additional measure to ensure test validity. Both participants scored close to 100% correct on all tasks. Based on the feedback, any final revisions were made and uploaded to the test website.

Participants in the pilot study were twenty deaf children between the ages of 6-10 years (8 boys and 12 girls) in the target age range (M=8;5, SD=1.3)

 

 

Psychometric information (from Mann, Roy, & Morgan, 2015)

 

Reliability for internal consistency (Cronbach’s alpha)

- meaning recall (.94)

- form recall (.86)

- form recognition (.71)

- meaning recognition (.55)

 

Item analysis

-12 items passed by all participants on three of the four levels of vocabulary knowledge (meaning recognition, form recognition, and form recall)

- none of these items were passed by all participants on the meaning recall task.

- one item in the form recognition task failed by all children

- no one item failed by all participants across all four levels

 

Inter-rater reliability (Cohen's kappa)

1. Agreements between raters’ judgments for scoring the form recall task (k=.91)

2. Agreements between raters’ judgments for scoring the meaning recall task (k=.85)

 

3. Items from the two receptive tasks were automatically scored by the computer upon selection of the response.

 

Content validity

1. Feedback related to the target and distractor items by a panel of deaf and hearing experts: revision of 14 items

2. Feedback by a group of undergraduate students on the quality and clarity of the test images: revision of 4 images

3. Mode of Acquisition (MoA, Wauters et al., 2008) ratings of target items reflect the range/spread of item types from concrete to abstract signs: spread of ratings over most of the range (1-5) except ‘5’ (signs that are acquired exclusively through language)

 

Construct validity

1. Correlation analysis between participants’ performance on the different tasks and age:

-  age &  meaning recognition = 0.49*

-  age & form recognition  = 0.68**

-  age & form recall = 0.63**

-  age & meaning recall = 0.43 ns

(Ns = not significant, *<.05 **p<.01)

 

2. Analysis of differences between participants’ performances across tasks:

Main effect of task, F (3, 57) = 213.253, p<.001, ηρ. = .918

Form recall > meaning recall (p<.001)

Meaning recognition > form recognition (p<.05)

Meaning recognition > meaning recall (p<.001)

Form recognition > form recall (p=.09)

3. Known-groups validity: comparison of deaf children’s performance on the two receptive tasks to age-matched hearing children with no previous knowledge of sign.

 

Strengths and weaknesses

Strengths: The ASL-VT test assesses vocabulary breath and depth and shows well documented psychometrics.

Weakness: Norms are needed for larger, more diverse sample

AUTHOR

Summarized by Wolfgang Mann (2016).

For more information regarding this test, please contact  Wolfgang Mann at the University of Roehampton.