Test Battery for Auslan Morphology and Syntax

The Test Battery for Australian Sign Language (Auslan) Morphology and Syntax (Schembri et al., 2002) is an adapted version from the  Test Battery for ASL Morphology and Syntax (Supalla et al., 1995, unpublished). One of the major issues in developing the test battery for Australian Sign Language (Auslan) was that, compared to other signed languages, there is a lack of descriptive research identifying morphosyntactic structures in Auslan. Therefore, the ASL test battery was chosen in order to gather more data about the nature of the linguistic structures of Auslan. A long term goal is to develop an assessment to be used for deaf children and adults learning Auslan as a second language.
 

The participants of the pilot were 25 deaf native signers. From these 25 participants, only twelve completed the entire Auslan test battery. The remaining thirteen completed only parts of the test battery. Their ages ranged from 16 to 58 years old.
 

Like the ASL test battery, the Auslan test battery tests both language comprehension and production of Auslan morphology and syntax.
  

Instrument
The discussed tests of the Auslan test battery in Schembri et al. (2002) are limited to the noun-verb production (NVP), the noun-verb comprehension (NVC), the sign order comprehension (SOC), and parts of the verbs of motion production (VMP), because they were completed by all 25 subjects (with the exception of the SOC test, which was completed only by 21 subjects). Therefore it is only possible to present these four tests.
  

The goal of the Noun-Verb Production (NVP) test is to determine if the signer produces a distinction in the number, duration, and/or manner of the movements between derivationally-related nouns and verbs in Auslan. In citation form, the nominal may be made with a shorter, restrained, and/or repeated movement, while the verbal may have a single, longer, continuous movement (Schembri et al., 2002, p. 8-9).
 

The task involved a test instruction in Auslan. Afterwards participants watched 35 short ‘skits’ “in which a person performs two separate actions, each action involving a different object” (Schembri et al., 2002, p. 9). Following each skit, the video was stopped and the participants were engaged in describing what they just saw. The participants’ responses were videotaped for later analysis.
  

In the original ASL test battery, 20 noun-verb pairs were elicited throughout this task. These 20 ASL noun-verb pairs have equivalent derivationally-related noun-verb pairs in Auslan. The results were analyzed independently by two coders. One of them was a deaf native signer.
  

In summary, the NVP tasks successfully elicited the target noun-verb pairs. Sometimes the participants did not produce the desired target. In such cases, the test administrator tried to clarify the situation with the participants and elicited some more information about the test materials. These tasks need to be modified for use in an Auslan assessment tool.
  

The Noun-Verb Comprehension (NVC) task aims to determine whether differences in the number, duration, and manner of movements were interpreted by native signers as signaling a morphological distinction between derivationally-related nouns and verbs in Auslan” (Schembri et al., 2002, p. 14).
  

The participants watched a short video clip where a native signer of Auslan produced either a nominal or a verbal form of a noun-verb pair. The signs were produced in isolation with no morphosyntactic cues. For each item, the participants saw one sign and then a pair of pictures. One of the pictures show an object, the other picture shows an action related to that object. The participants were asked to identify which of the pictures represented what they had seen. The picture material was used from the original Ted Supalla’s et al. (1995, unpublished) ASL test battery. All responses were videotaped for later analysis.

The results reveal that certain items elicited variable responses. The different responses reflect the patterns of lexical variation in Auslan noun-verb pairs. The materials of the NVC tasks need to be revised before they can be included in an Auslan assessment instrument.
 

The Verb of Motion Production (VMP) task aims to “determine how native signers of Auslan produce polymorphemic verbs of motion and location, and to gather normative data on what constitutes appropriate use of productive verbal morphology for manner, path, location, direction, and referent class object” (Schembri et al., 2002, p. 20). The participants watched a series of animated films showing simple motion events. These events involved movements of an object from one location to another. There was a total of 80 animated clips. The responses were videotaped and later analyzed. The first 40 animated clips involved a single moving object, referred to as the ‘central object’. The remaining 40 clips involved two objects: the central object and a stationary ‘secondary object’. In both cases the participant need to represent the referents or objects by the appropriate classifier handshape morpheme.
  

The VMP task was coded by two deaf native signer and a hearing non-native signer. The inter-rater agreement among the three rater was over 90% in all cases. In the original ASL test battery (Supalla et al., unpublished), the used handshape morphemes are divided into categories, size and shape specifiers and semantic classifiers. Schembri et al. (2002) state that, due to amount of data of the VMP task, the focus is only on the semantic classifiers.
  

The results of this task showed how little is known about how deaf native signers of Auslan produce differently various classes of referents in classifier verbs of motion. The VMP results did not provide the kind of normative data that would make the design of an assessment tool unproblematic, due to variations in the possible ways native signers of Auslan may represent various classes of referents in classifier verbs of motion. Additionally, the large amount of data and the complex coding process of the data would not make it easy to use the Auslan test battery as an assessment tool.

 
The Sign Order Comprehension (SOC) task aims to gather normative data about the sign order in Australian Sign Language.

 
The participants watched a series of 44 video clips, where each showed a single clause in Auslan. Of the 44 video clips, four were part of the explanation of the task and four were practice items. This left 36 items that were included in the analysis. Each of the clauses contained only three signs, produced in one of the three possible orders (SVO, OSV, VOS). In eighteen of these clauses, word order is the only grammatical signal to assign the roles of subject and object. In nine of these eighteen clauses, the subject was also non-manually marked as the topic. The remaining eighteen clauses used “a combination of word order other than SVO and non-manually marking of topicalized constituent to express subject and object roles” (Schembri et al., 2002, p. 18). That made four different types of clauses for this task. The verb signs were not spatially modified for persons, and no spatial cues were available to determine the roles of each of the nominals. After watching the video, the participants were shown a pair of pictures, which differed only in terms of which person was the agent or patient. The participants need to identify which of the two pictures represent the event described in the task. The participants had only to rely on the combination of word order and non-manual marking cues to determine the meaning of the clause.

 
The results show a very high number of correct responses, with an overall mean score of 32.6 correct out of 36 items, with a range from 19 to 36 correct. A problem with the current test designs is, as adapted from the ASL test battery, that in the test instructions one example of each clause type and the correct response is modeled for the participants. It is not clear if the participants are able to learn the rule from the instructions and then apply it to the test items.
  

Test adaptation
The ASL battery was chosen because it is the most extensive test for signed languages. The adaptation for linguistic research purposes of another signed language (i.e. Auslan) is relatively easy due to the design of the test materials.

 
It was, therefore, possible to use the ASL test battery as a starting point in the development of an assessment instrument for Auslan by using the materials to collect normative data on the key grammatical features of Auslan as used by native signers (Schembri et al., 2002, p. 5).

 
The original ASL test battery provided background information about the design, administration, and analysis of the test, which could be used for the adaptation of it for the Auslan test battery. Examples for coding sheets and picture materials that were used for several tasks were included. The videotape included film sequences that were used as test stimuli and test instructions in ASL. Even if the information is provided, some modification for using the test material for Auslan need to be made. Some tasks of the ASL version needed to be refilmed for Auslan since the test stimuli was in ASL, e.g. the sign order comprehension test and the ASL clause test. The same applied for the test instructions of each task. Table 1 provides an overview for each of the tests and which modifications needed to be made for the Auslan test battery.
 

Table 1: Modifications of the ASL test battery for the Auslan test battery (from Schembri et al., 2002)

Test name

Modifications for the Auslan Test Battery

Test 1: Lexical phonology production (LPP)

Instructions refilmed, printed materials adapted

Test 2: Narrative production (NP)

Instructions refilmed, original test stimuli used

Test 3: Verb agreement production (VAP)

Instructions and test stimuli refilmed

Test 4: Noun-verb production (NVP)

Instructions refilmed, original test stimuli used

Test 5: Noun-verb comprehension (NVC)

Instructions and test stimuli refilmed, original printed materials used

Test 6: Verb of motion production (VMP)

Instructions refilmed, original test stimuli used

Test 7: Demographic questionnaire (QQ)

Instructions refilmed, original printed materials adapted

Test 8: Aspect and number inflection production (ANIP)

Instructions and test stimuli refilmed

Test 9: Sign order comprehension (SOC)

Instructions and test stimuli refilmed, original printed materials used

Test 10: Verb reversal production (VRP)

Not adapted

Test 11: Verb agreement comprehension: Single verbs (VAC-1)

Instructions and test stimuli refilmed

Test 12: Verb agreement comprehension: Two verb sequences (VAC-2)

Instructions and test stimuli refilmed

The entire adaptation, including decisions about appropriate test lexical items used as stimuli and refilming of the test materials, was done by a hearing signed language researcher who is also an Auslan/English interpreter and a deaf native signer. The deaf native signer was filmed for the test instructions in Auslan. A group of three other deaf native signers assisted in the refilming of the test materials.
  

After a draft version of the Auslan test battery was completed, a pilot study was conducted in order to collect normative data to ensure the test materials were suitable as instruments for as a proficiency assessments of Auslan. Seven out of the initial twelve task of the ASL test battery needed to be refilmed. Those seven refilmed and translated tasks were administered to two Auslan/English interpreters as a pre-pilot. This was done in order to avoid using deaf native signers, which would have an impact on the number of deaf native signers as participants for the study itself. No modifications were made based on the results of the pre-pilot study, and the pilot was conducted.
  

The NVP, NVC, and the VMP needed substantial modifications before it could be included in the revised version of the Auslan test battery. The project of the Auslan test battery has both theoretical and practical significance. The project contributes to the understanding of the morphosyntactic structures of Auslan, since the research literature does not provid a detailed description of grammatical features of Auslan and their use by a range of deaf native signs from different social and regional backgrounds.
  

An improved understanding of the grammatical features of Auslan will help or will lead to the production of a valid and reliable assessment tool for the grammatical proficiency of Australian Sign Language. Modified versions could be used in bilingual programs serving deaf children or in teaching Auslan as a second language, but the development of such as assessment tool lies still ahead.
  

Schembri et al. (2002) point out the following issues that came up during this stage of the project. (1) Because the original items of the NVP and the NVC task show a large variation among the produced Auslan noun-verb pairs, more Auslan noun-verb pairs showing less variation need to be included in a revised version. Also, some materials of the ASL test battery need to undergo some modifications in order to ensure test validity. (2) The test design of the SOC task may need some modifications. (3) The results of the VMP task revealed that a significant amount of variation in the choice of classifier handshape morpheme is possible. The large amount of data produced and the amount of time needed to analyze all this data need modification before being included in an Auslan test battery.
  

This test is not available, since it reports from work in progress. A future goal of the Auslan test battery, besides gathering data on the morphosyntactic structures of Auslan, is to develop an assessment that can be used for deaf children and for hearing adults who learn Auslan as a second language.
  

Among the strengths of the Auslan test battery are that (1) it gathers information on specific morphosyntactic aspects of Auslan, (2) it might allow a cross-linguistic comparison on specific morphosyntactic aspects of signed language, e.g. ASL, and (3) it would be a future goal to develop an assessment instrument to be used with deaf children and second language learners of a signed language.
  

Among the weaknesses of the test battery are that (1) it is so far intended for research purpose only and (2) it can not yet be used as a baseline assessment in school.
 

AUTHOR

From: Tobias Haug: “Review of Sign Language Assessment Instruments”, an earlier version of that paper 2005.

For more information regarding this test, please contact  Adam Schembri at the University of Bermingham.