Popup window

Print this page

Pilot Test Outcomes

Expand All | Collapse All

Chinese (Mandarin)

Pilot Test Outcomes

Chinese (Mandarin)

Multiple-Choice Items (MCQs) and Short-Answer (SAs) items
Number of MCQs and SAs Prepared170
Number of MCQs and SAs Piloted*74
Number of MCQs/SAs Identified for Further Review17
Number of MCQs/SAs Deleted Post Pilot Test Review1
Open-Response Items (ORIs)
Number of ORIs Developed23
Number of ORIs Piloted*4
Number of ORIs Identified for Further Reviewnone
Number of ORIs Deleted Post Pilot Test Reviewnone

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

Dance

Pilot Test Outcomes

Dance

Multiple-Choice Items (MCQs)

NOTE: As described in the "Pilot Testing" section of the Technical Manual, in addition to the review of test items by expert judges (members of the BRC and CACs), when permitted by sufficient candidate populations, empirical data were collected about statistical and qualitative characteristics of the new items through pilot testing. For this field there exists an insufficient candidate population for collecting statistical information through pilot testing. Therefore, pilot test information for multiple-choice questions is unavailable for the field.

Open-Response Items (ORIs)
Number of ORIs Developed9
Number of ORIs Piloted*9
Number of ORIs Identified for Further Reviewnone
Number of ORIs Deleted Post Pilot Test Reviewnone

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

English as a Second Language

Pilot Test Outcomes

English as a Second Language

Multiple-Choice Items (MCQs)
Number of MCQs Prepared336
Number of MCQs Piloted*209
Number of MCQs Identified for Further Review40
Number of MCQs Deleted Post Pilot Test Review11
Open-Response Items (ORIs)
Number of ORIs Developed23
Number of ORIs Piloted*18
Number of ORIs Identified for Further Review1
Number of ORIs Deleted Post Pilot Test Review1

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

German

Pilot Test Outcomes

German

Multiple-Choice Items (MCQs)

NOTE: As described in the "Pilot Testing" section of the Technical Manual, in addition to the review of test items by expert judges (members of the BRC and CACs), when permitted by sufficient candidate populations, empirical data were collected about statistical and qualitative characteristics of the new items through pilot testing. For this field there exists an insufficient candidate population for collecting statistical information through pilot testing. Therefore, pilot test information for multiple-choice questions is unavailable for the field.

Open-Response Items (ORIs)
Number of ORIs Developed20
Number of ORIs Piloted*4
Number of ORIs Identified for Further Reviewnone
Number of ORIs Deleted Post Pilot Test Reviewnone

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

Italian

Pilot Test Outcomes

Italian

Multiple-Choice Items (MCQs)

NOTE: As described in the "Pilot Testing" section of the Technical Manual, in addition to the review of test items by expert judges (members of the BRC and CACs), when permitted by sufficient candidate populations, empirical data were collected about statistical and qualitative characteristics of the new items through pilot testing. For this field there exists an insufficient candidate population for collecting statistical information through pilot testing. Therefore, pilot test information for multiple-choice questions is unavailable for the field.

Open-Response Items (ORIs)
Number of ORIs Developed20
Number of ORIs Piloted*8
Number of ORIs Identified for Further Reviewnone
Number of ORIs Deleted Post Pilot Test Reviewnone

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

Music

Pilot Test Outcomes

Music

Multiple-Choice Items (MCQs)
Number of MCQs Prepared323
Number of MCQs Piloted*176
Number of MCQs Identified for Further Review31
Number of MCQs Deleted Post Pilot Test Review3
Open-Response Items (ORIs)
Number of ORIs Developed19
Number of ORIs Piloted*7
Number of ORIs Identified for Further Review4
Number of ORIs Deleted Post Pilot Test Reviewnone

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

Portuguese

Pilot Test Outcomes

Portuguese

Multiple-Choice Items (MCQs)

NOTE: As described in the "Pilot Testing" section of the Technical Manual, in addition to the review of test items by expert judges (members of the BRC and CACs), when permitted by sufficient candidate populations, empirical data were collected about statistical and qualitative characteristics of the new items through pilot testing. For this field there exists an insufficient candidate population for collecting statistical information through pilot testing. Therefore, pilot test information for multiple-choice questions is unavailable for the field.

Open-Response Items (ORIs)
Number of ORIs Developed20
Number of ORIs Piloted*6
Number of ORIs Identified for Further Reviewnone
Number of ORIs Deleted Post Pilot Test Reviewnone

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

Russian

Pilot Test Outcomes

Russian

Multiple-Choice Items (MCQs)

NOTE: As described in the "Pilot Testing" section of the Technical Manual, in addition to the review of test items by expert judges (members of the BRC and CACs), when permitted by sufficient candidate populations, empirical data were collected about statistical and qualitative characteristics of the new items through pilot testing. For this field there exists an insufficient candidate population for collecting statistical information through pilot testing. Therefore, pilot test information for multiple-choice questions is unavailable for the field.

Open-Response Items (ORIs)
Number of ORIs Developed23
Number of ORIs Piloted*4
Number of ORIs Identified for Further Reviewnone
Number of ORIs Deleted Post Pilot Test Reviewnone

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

Sheltered English Immersion

Pilot Test Outcomes

Sheltered English Immersion

Multiple-Choice Items (MCQs)
Number of MCQs Prepared185
Number of MCQs Piloted*117
Number of MCQs Identified for Further Review22
Number of MCQs Deleted Post Pilot Test Review3

*This chart includes information regarding multiple-choice items piloted prior to the first operational administration and may be a subset of all items developed. Additional items are piloted as non-scorable on operational test forms, until all items are piloted.

Open-Response Items

In consultation with the Massachusetts Department of Education and the Department of Justice review, the open-response item for the Sheltered English Immersion test was designed as a single, five-part open-response item requiring candidates to show their knowledge of sheltering content. Candidates use one of 10 mentor texts for the basis of their response. Each mentor text is an informational text that is representative of the kind of content and academic language features a student might encounter in texts in a given content area. Candidates are expected to spend about 20–30 minutes to prepare an adequate response to each of the five parts of the assignment, for a total of about 2–2 ½ hours for the section. The multiple-choice section counts for 60 percent and the open-response section counts for 40 percent of the candidate's total test score.

Piloting of the new open-response item type focused on evaluating the item format. Pilot test responses to the open- response item format were reviewed for the following characteristics:

  • Candidate ability to complete the assignment
  • Testing time required to complete the assignment
  • Administration process and computer screen layout
  • Application of the proposed score scale to the responses

A review of the data and item comments collected indicated that the candidates were able to appropriately and adequately respond to the new open-response item type and were able to produce a response within the targeted response time of 2 ½ hours, leaving sufficient time to respond to the multiple-choice questions. In addition, candidates were able to move throughout the various open-response item components with relative ease.

Theater

Pilot Test Outcomes

Theater

Multiple-Choice Items (MCQs)
Number of MCQs Prepared272
Number of MCQs Piloted*269
Number of MCQs Identified for Further Review32
Number of MCQs Deleted Post Pilot Test Reviewnone
Open-Response Items (ORIs)
Number of ORIs Developed14
Number of ORIs Piloted*5
Number of ORIs Identified for Further Reviewnone
Number of ORIs Deleted Post Pilot Test Reviewnone

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.