Profile of Multiple Language Proficiencies

The purpose of the Profile of Multiple Language Proficiencies (PMLP) is to assess the full range of expressive language abilities in deaf children using the same scale, namely spoken English, Manually Coded English (MCE) and American Sign Language (ASL). It also enables a comparison across languages and modalities. The PMLP aims to be sensitive to language development. Normally assessment instruments focus rather on a single language or modality (Goldstein & Bebko, 2003).

The PMLP is different from the  Language Proficiency Profile (LPP-2; Bebko, Calderon, & Treder, 2003) which assesses global communication proficiency. Instead, the PMLP focuses on different modes of communication.

 

Development of the scale

The development of the PMLP took place in two parts, a (1) pilot study, which resulted in (2) the development of the Profile of Multiple Language Proficiencies. The basis for both parts of the project were language samples obtained from the children in (naturalistic) semi-structured conversations and telling a short story based on a picture book (Goldstein & Bebko, 2003). This conversation offers children the opportunity to produce language spontaneously (as compared to an interview-form) and they were not restricted to using a certain form/type of communication (i.e. ASL, MCE, or English).

 

Pilot study

The goal of the pilot study was to evaluate whether it is possible to use a continuum to assess ASL and English skills relative to each other considering the diverse linguistic background of the target population. A naturalistic semi-structured conversation was used to collect data on this continuum. This “continuum” is a scale with English on the one end of the continuum (score: 0) to a score of 20, using ASL only. A score of 10 indicated a use of a balanced mix of ASL and English.

 

Sample: The language sample consisted of 31 deaf children (14 boys, 17 girls), aged 7-13 years (mean10;4). Two children had deaf parents. The children came from two schools in central and eastern Ontario area (Goldstein & Bebko, 2003). All had a hearing loss of at least 75 dB.

 

Rating of the language samples: The samples were rated by three coders, who were fluent in ASL and English. After the rating, the results of the three coders were correlated in order to establish inter-rater reliability. The inter-rater reliability was very poor, there was an error band of +/-8 on the 20-level scale (normally a 70-80% agreement between coders is needed to establish inter-rater reliability). The major difficulties of this scale were: (1) “the tendency to pit English and ASL skills in opposition to each other” (Goldstein & Bebko, 2003, p. 455), i.e. it did not recognize that a child can be strong in both languages. (2) Related to this issue, the scale could not discriminate between children who had higher or lower scores in both languages. This resulted in the development of a scale where English and ASL are evaluated separately (Goldstein & Bebko, 2003).

 

For the main study, the scale was revised in such a way that it covers the developmental aspect and evaluates separately the skills of deaf and hard-of-hearing children across different communication modalities, i.e. spoken English, signed English, and ASL (see Appendix in Goldstein & Bebko, 2003).

 

Main study

Participants were the same 31 deaf students that were involved in the pilot study.

The raters did not know the children before the study. All three had a background in language development in deaf children, both for English and ASL. One was a CODA who was a hearing university student (also participated in the pilot study), the second rater was a deaf high school teacher who was fluent in spoken English and ASL communication forms (Goldstein & Bebko, 2003). The third rater was a hearing person trained as educational interpreter. All three raters were asked to code/rate all three types of communication to match the design of the pilot study.

 

Procedure: The children were brought individually to the interview room. During the interview, the children and the interviewer were video recorded. The children were informed that they could use any form of communication they want, i.e. ASL, MCE and/or English. The interviewer would use sign language and voice, i.e. simultaneous communication. The interview consisted of three parts: (1) four charts of the Rapid Automatized Naming Task (RAN; colors, numbers, animals, and objects) (Denckla & Rudel, 1976). The goal using the RAN was to how fast a child could access sets of labels to measure automatized language skills. (2) This part had the form of an interview-like conversation, where the child could choose a topic of their own interest and converse with the interviewer more socially. In addition, the interviewer asked questions about school, language preference, family, and hobbies. (3) In the last part, the child looked a picture book (without written language) and was asked to tell what was taking place in the pictures. Each interview lasted about 30-45 minutes.

 

The raters started to look at the three modes of communication used by the children, using the PMLP scale, which describes the competencies of these three modes of communication (see Appendix Goldstein & Bebko, 2003). The scale of the PMLP is a synthesis of different languages measures, and is developmental and dynamic in nature. All the different measures that served as the basis of the PMLP target the different modes of communication, more specifically the Speech and Language Development Chart (Gard, Gilman, & Gorman, 1980), the Pragmatic Development Chart (Schulman, 1983), the Kendall Communicative Proficiency Profile (KCPP; Kendall Demonstration Elementary School, 1980), parts of the Conversational Proficiency Interview (Akamatsu & Muselamn, 1998), and work by Dworetzky (1996) and Phillips (1999) were used to design/create the developmental criteria within the PMLP. The basic structure of the PMLP is based on the levels of English language development (Phillips, 1999): (1) Pre-Word, (2) Pre-Sentence, (3) 3-Word-Combinations, (4) Simple Sentence, (5) Compound Sentences, (6) Complex Sentences, (7) Early Fluency (added by the authors), and (8) Native Fluency (renamed into Full Fluency by the authors). In order to also cover the developmental milestones of ASL, the authors reviewed existing ASL acquisition studies and linked them to the developmental milestones of English.

 

Table 1: Selected examples of the PMLP scale (the first description of each subcategory is given) (Goldstein & Bebko, 2003, pp. 460)

Level

ASL

Signed/Spoken English

Level 1 (pre-linguistic)

Expresses him or herself through facial expression and natural gestures

Has no command of English syntax

Level 2 (word combination)

Exhibits one- and two-sign structures that may be displayed in PSE or ASL

Uses one- and two-word English structures

Level 3 (beginning sentence)

Able to combine isolated signs to create simple questions, stereotypical phrases, and expressions

Able to use English combinations of nouns, pronouns, verbs, propositions, adjectives, conjunctions, “wh” questions, and negative marker

Level 4 (full simple sentence)

Describes, asks and answers questions, and uses social phrases

Able to use a variety of the five simple sentences, e.g. NP + VP

Level 5 (compound sentence)

Able to narrate, describe, compare, or contrast

Able to use conjunctions such as and, but, or to join two simple sentences

Level 6 (complex sentence)

Uses appropriate rhythm and stress

Able to use adverbial phrases or clauses when combining simple sentence patterns or compound sentences but with some errors or hesitations

Level 7 (early fluency)

Developing a larger sign vocabulary (signs not specific to English signs)

Able to use at least 1500-2000 words

Level 8 (full fluency)

Uses appropriate syntactical structures naturally and consistently without errors or hesitations

Uses syntactical structures naturally and consistently without errors or hesitations

 

The PMPL scale is designed in such a way that there are categories for the different levels, and within each level are different subcategories/defined criteria or skills that apply to the modes of communication, i.e. English, signed English and ASL. The raters were advised that in order to fulfill a criterion of the scale, a child should have at least half of the skills listed in one category, “most of the skills that characterize lower levels, and only one or two skills that characterize higher levels” (Goldstein & Bebko, 2003, p. 457). In the next step that raters were advised on how to use the scale. At the end each child should have a rating for each subcategory, i.e. mode of communication. In cases where a particular language/modality was not used, the child received the rating of Level 1 or that language/modality.

 

Results: The results across the raters were close to one another in terms of the range of scores they gave and they tended to cluster at the lower end of the scale in each subcategory. The agreement among the three raters (inter-rater reliability) was also investigated: allowing a +/- 1error of the rating in each scale produced an agreement between the three raters of 71-84% in the subcategories of spoken English and ASL (Goldstein & Bebko, 2003). Within the signed English subcategory, only an agreement of 68-90% between two raters and 55% between the remaining two raters emerged (allowing a +/- 2 error band produced a 87-100% agreement).

One of the initial results is that raters with different linguistic backgrounds (e.g., CODA, deaf) are able to use the PMLP scale, with a “moderate to high degree of concordance within a narrow to moderate range of allowable errors” (Goldstein & Bebko, 2003, p. 458). The different backgrounds of the raters also helped to clarify the inter-rater reliability. The highest agreement in the English category was between the raters for whom English was currently the dominant language (CODA and interpreter), the highest agreement in the ASL subcategory was found between the two raters for whom ASL is the native language, i.e. the deaf teacher and the CODA. There was no significant correlation between age and performance and therefore the PMLP is limited with respect to its sensitivity to the development of communication. The authors argue that the length of time using the current dominant language was a better predictor of “their use of language-based cognitive strategies than was age” (Goldstein & Bebko, 2003, p. 459). The scale can be used in the classroom by teachers as a tool for evaluating the strengths and weaknesses of deaf children across the different languages/modalities. However, more research in using the scale by practitioners is needed.

 

Changes to the original PMLP
There have been certain changes to the procedure described by the authors 2003 (Goldstein & Bebko, 2003). The changes include a formal protocoll and scoring sheet for the administration across the three modes of language use and the authors suggested to use certain pictures for the three tasks (G. Goldstein, personal communication, December 23, 2013).

 

Availability of the test

The PMLP has been used since 2003 by speech and language pathologists and teachers and given to researchers on the condition to share the results with the authors of the PMLP to provide better norms in the future (G. Goldstein, personal communication, December 23, 2013).

 

Strengths: (1) enables the evaluation of strengths and weaknesses of children across modalities, (2) could be used in schools by raters with different backgrounds, (3) validity of the scale (based on review of studies and existing scales)

 

Weaknesses: (1) availability of psychometric properties.

AUTHOR

Summarized by Tobias Haug (2013).

For more information regarding this test, please contact  Gayle Goldstein at York University, Canada.