Supplemental Test Development Information

Music

	Number
Committee Participants: Total	11

Public School Educators	9
Higher Education Faculty	2

Gender
Female	7
Male	4

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	2
Hispanic	0
White	4
Other	0
Unspecified	5

Music

	Number
Committee Participants: Total	5

Public School Educators	4
Higher Education Faculty	1

Gender
Female	2
Male	3

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	2
Asian or Pacific Islander	1
Hispanic	0
White	0
Other	1
Unspecified	1

Deaf/Hard of Hearing
Yes	0
No	5

Music

	Number
Committee Participants: Total	15

Public School Educators	12
Higher Education Faculty	3

Gender
Female	7
Male	8

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	2
Hispanic	0
White	7
Other	0
Unspecified	6

Music

INTACT CLASSROOM SESSIONS 2015
04/11/2015	Gordon College
04/27/2015	Salem State University
05/02/2015	University of Massachusetts - Amherst
05/05/2015	Westfield State University
05/09/2015	Gordon College

STAND ALONE EVENTS AT MTEL OPERATIONAL TEST ADMINISTRATIONS

03/07/2015
05/09/2015
07/11/2015

Music

Multiple-Choice Items (MCQs)

Number of MCQs Prepared	323
Number of MCQs Piloted*	176
Number of MCQs Identified for Further Review	31
Number of MCQs Deleted Post Pilot Test Review	3

Open-Response Items (ORIs)

Number of ORIs Developed	19
Number of ORIs Piloted*	7
Number of ORIs Identified for Further Review	4
Number of ORIs Deleted Post Pilot Test Review	none

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

Music

Number of Qualifying Score Participants: 15

Number of Participants Providing Ratings: 15 (100%)

Question [Rating Scale: 1 (not at all well) to 5 (very well)]	Mean Rating
How well do you feel you understood the training that you received for making your qualifying score judgments?	4.8
How confident are you of the qualifying score judgments you made?	4.7
How satisfied are you that you were able to complete the work you were asked to do in the time that was provided?	5.0
How successful were the coordination and logistics of the qualifying score conference?	5.0
Overall, how satisfied are you with the qualifying score process in which you participated?	4.7

German

	Number
Committee Participants: Total	8

Public School Educators	8
Higher Education Faculty	0

Gender
Female	6
Male	2

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	0
Hispanic	0
White	5
Other	1
Unspecified	2

German

	Number
Committee Participants: Total	13

Public School Educators	11
Higher Education Faculty	2

Gender
Female	9
Male	4

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	4
Asian or Pacific Islander	1
Hispanic	2
White	1
Other	5
Unspecified	0

Deaf/Hard of Hearing
Yes	1
No	12

German

	Number
Committee Participants: Total	7

Public School Educators	7
Higher Education Faculty	0

Gender
Female	5
Male	2

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	0
Hispanic	0
White	4
Other	1
Unspecified	2

German

Multiple-Choice Items (MCQs)

NOTE: As described in the "Pilot Testing" section of the Technical Manual, in addition to the review of test items by expert judges (members of the BRC and CACs), when permitted by sufficient candidate populations, empirical data were collected about statistical and qualitative characteristics of the new items through pilot testing. For this field there exists an insufficient candidate population for collecting statistical information through pilot testing. Therefore, pilot test information for multiple-choice questions is unavailable for the field.

Open-Response Items (ORIs)

Number of ORIs Developed	20
Number of ORIs Piloted*	4
Number of ORIs Identified for Further Review	none
Number of ORIs Deleted Post Pilot Test Review	none

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

German

Number of Qualifying Score Participants: 7

Number of Participants Providing Ratings: 7 (100%)

Question [Rating Scale: 1 (not at all well) to 5 (very well)]	Mean Rating
How well do you feel you understood the training that you received for making your qualifying score judgments?	4.7
How confident are you of the qualifying score judgments you made?	4.7
How satisfied are you that you were able to complete the work you were asked to do in the time that was provided?	5.0
How successful were the coordination and logistics of the qualifying score conference?	4.9
Overall, how satisfied are you with the qualifying score process in which you participated?	4.4

Chinese (Mandarin)

	Number
Committee Participants: Total	8

Public School Educators	6
Higher Education Faculty	2

Gender
Female	7
Male	1

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	6
Hispanic	0
White	2
Other	0
Unspecified	0

Chinese (Mandarin)

	Number
Committee Participants: Total	13

Public School Educators	11
Higher Education Faculty	2

Gender
Female	9
Male	4

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	4
Asian or Pacific Islander	1
Hispanic	2
White	1
Other	5
Unspecified	0

Deaf/Hard of Hearing
Yes	1
No	12

Chinese (Mandarin)

	Number
Committee Participants: Total	8

Public School Educators	5
Higher Education Faculty	3

Gender
Female	7
Male	1

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	6
Hispanic	0
White	1
Other	0
Unspecified	1

Chinese (Mandarin)

INTACT CLASSROOM SESSIONS 2013–2015
10/04/2013	Brandeis University
10/17/2013	University of Massachusetts - Amherst
04/29/2015	Tufts University
05/02/2015	Smith College
05/09/2015	Gordon College

STAND ALONE EVENTS AT MTEL OPERATIONAL TEST ADMINISTRATIONS

10/26/2013
03/01/2014
05/10/2014
07/12/2014
10/25/2014
03/07/2015
05/09/2015
07/11/2015

STAND ALONE EVENT(S) IN STATE(S) OTHER THAN MASSACHUSETTS

11/16/2013

Chinese (Mandarin)

Multiple-Choice Items (MCQs) and Short-Answer (SAs) items

Number of MCQs and SAs Prepared	170
Number of MCQs and SAs Piloted*	74
Number of MCQs/SAs Identified for Further Review	17
Number of MCQs/SAs Deleted Post Pilot Test Review	1

Open-Response Items (ORIs)

Number of ORIs Developed	23
Number of ORIs Piloted*	4
Number of ORIs Identified for Further Review	none
Number of ORIs Deleted Post Pilot Test Review	none

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

Chinese (Mandarin)

Number of Qualifying Score Participants: 8

Number of Participants Providing Ratings: 8 (100%)

Question [Rating Scale: 1 (not at all well) to 5 (very well)]	Mean Rating
How well do you feel you understood the training that you received for making your qualifying score judgments?	4.8
How confident are you of the qualifying score judgments you made?	4.5
How satisfied are you that you were able to complete the work you were asked to do in the time that was provided?	4.9
How successful were the coordination and logistics of the qualifying score conference?	5.0
Overall, how satisfied are you with the qualifying score process in which you participated?	5.0

Italian

	Number
Committee Participants: Total	5

Public School Educators	3
Higher Education Faculty	2

Gender
Female	4
Male	1

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	0
Hispanic	0
White	3
Other	0
Unspecified	2

Italian

	Number
Committee Participants: Total	16

Public School Educators	14
Higher Education Faculty	2

Gender
Female	11
Male	5

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	3
Asian or Pacific Islander	2
Hispanic	2
White	3
Other	6
Unspecified	0

Deaf/Hard of Hearing
Yes	3
No	13

Italian

	Number
Committee Participants: Total	7

Public School Educators	5
Higher Education Faculty	2

Gender
Female	6
Male	1

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	0
Hispanic	0
White	4
Other	0
Unspecified	3

Italian

INTACT CLASSROOM SESSIONS 2013–2015
10/23/2013	Brandeis University

STAND ALONE EVENTS AT MTEL OPERATIONAL TEST ADMINISTRATIONS

10/26/2013
03/01/2014
05/10/2014
07/12/2014
10/25/2014

STAND ALONE EVENT(S) IN STATE(S) OTHER THAN MASSACHUSETTS

11/16/2013

Italian

Multiple-Choice Items (MCQs)

NOTE: As described in the "Pilot Testing" section of the Technical Manual, in addition to the review of test items by expert judges (members of the BRC and CACs), when permitted by sufficient candidate populations, empirical data were collected about statistical and qualitative characteristics of the new items through pilot testing. For this field there exists an insufficient candidate population for collecting statistical information through pilot testing. Therefore, pilot test information for multiple-choice questions is unavailable for the field.

Open-Response Items (ORIs)

Number of ORIs Developed	20
Number of ORIs Piloted*	8
Number of ORIs Identified for Further Review	none
Number of ORIs Deleted Post Pilot Test Review	none

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

Italian

Number of Qualifying Score Participants: 7

Number of Participants Providing Ratings: 7 (100%)

Question [Rating Scale: 1 (not at all well) to 5 (very well)]	Mean Rating
How well do you feel you understood the training that you received for making your qualifying score judgments?	4.4
How confident are you of the qualifying score judgments you made?	4.3
How satisfied are you that you were able to complete the work you were asked to do in the time that was provided?	4.4
How successful were the coordination and logistics of the qualifying score conference?	4.6
Overall, how satisfied are you with the qualifying score process in which you participated?	4.7

Russian

	Number
Committee Participants: Total	7

Public School Educators	3
Higher Education Faculty	4

Gender
Female	6
Male	1

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	0
Hispanic	0
White	5
Other	0
Unspecified	2

Russian

	Number
Committee Participants: Total	15

Public School Educators	12
Higher Education Faculty	3

Gender
Female	10
Male	5

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	5
Asian or Pacific Islander	1
Hispanic	2
White	2
Other	5
Unspecified	0

Deaf/Hard of Hearing
Yes	2
No	13

Russian

	Number
Committee Participants: Total	3

Public School Educators	2
Higher Education Faculty	1

Gender
Female	3
Male	0

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	0
Hispanic	0
White	1
Other	0
Unspecified	2

Russian

Multiple-Choice Items (MCQs)

NOTE: As described in the "Pilot Testing" section of the Technical Manual, in addition to the review of test items by expert judges (members of the BRC and CACs), when permitted by sufficient candidate populations, empirical data were collected about statistical and qualitative characteristics of the new items through pilot testing. For this field there exists an insufficient candidate population for collecting statistical information through pilot testing. Therefore, pilot test information for multiple-choice questions is unavailable for the field.

Open-Response Items (ORIs)

Number of ORIs Developed	23
Number of ORIs Piloted*	4
Number of ORIs Identified for Further Review	none
Number of ORIs Deleted Post Pilot Test Review	none

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

Russian

Number of Qualifying Score Participants: 3

Number of Participants Providing Ratings: 3 (100%)

Question [Rating Scale: 1 (not at all well) to 5 (very well)]	Mean Rating
How well do you feel you understood the training that you received for making your qualifying score judgments?	4.7
How confident are you of the qualifying score judgments you made?	4.7
How satisfied are you that you were able to complete the work you were asked to do in the time that was provided?	5.0
How successful were the coordination and logistics of the qualifying score conference?	5.0
Overall, how satisfied are you with the qualifying score process in which you participated?	5.0

Portuguese

	Number
Committee Participants: Total	9

Public School Educators	8
Higher Education Faculty	1

Gender
Female	7
Male	2

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	0
Hispanic	1
White	2
Other	1
Unspecified	5

Portuguese

	Number
Committee Participants: Total	16

Public School Educators	14
Higher Education Faculty	2

Gender
Female	11
Male	5

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	3
Asian or Pacific Islander	2
Hispanic	2
White	3
Other	6
Unspecified	0

Deaf/Hard of Hearing
Yes	3
No	13

Portuguese

	Number
Committee Participants: Total	6

Public School Educators	5
Higher Education Faculty	1

Gender
Female	6
Male	0

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	0
Hispanic	1
White	2
Other	1
Unspecified	2

Portuguese

Multiple-Choice Items (MCQs)

NOTE: As described in the "Pilot Testing" section of the Technical Manual, in addition to the review of test items by expert judges (members of the BRC and CACs), when permitted by sufficient candidate populations, empirical data were collected about statistical and qualitative characteristics of the new items through pilot testing. For this field there exists an insufficient candidate population for collecting statistical information through pilot testing. Therefore, pilot test information for multiple-choice questions is unavailable for the field.

Open-Response Items (ORIs)

Number of ORIs Developed	20
Number of ORIs Piloted*	6
Number of ORIs Identified for Further Review	none
Number of ORIs Deleted Post Pilot Test Review	none

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

Portuguese

Number of Qualifying Score Participants: 6

Number of Participants Providing Ratings: 6 (100%)

Question [Rating Scale: 1 (not at all well) to 5 (very well)]	Mean Rating
How well do you feel you understood the training that you received for making your qualifying score judgments?	4.8
How confident are you of the qualifying score judgments you made?	5.0
How satisfied are you that you were able to complete the work you were asked to do in the time that was provided?	4.8
How successful were the coordination and logistics of the qualifying score conference?	4.7
Overall, how satisfied are you with the qualifying score process in which you participated?	4.5

Theater

	Number
Committee Participants: Total	10

Public School Educators	8
Higher Education Faculty	2

Gender
Female	6
Male	4

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	0
Hispanic	0
White	6
Other	1
Unspecified	3

Theater

	Number
Committee Participants: Total	5

Public School Educators	4
Higher Education Faculty	1

Gender
Female	2
Male	3

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	2
Asian or Pacific Islander	1
Hispanic	0
White	0
Other	1
Unspecified	1

Deaf/Hard of Hearing
Yes	0
No	5

Theater

	Number
Committee Participants: Total	8

Public School Educators	6
Higher Education Faculty	2

Gender
Female	7
Male	1

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	1
Asian or Pacific Islander	0
Hispanic	0
White	5
Other	0
Unspecified	2

Theater

INTACT CLASSROOM SESSIONS 2015
04/11/2015	Emerson College
04/17/2015	Emerson College
04/27/2015	Salem State University

STAND ALONE EVENTS AT MTEL OPERATIONAL TEST ADMINISTRATIONS

03/07/2015
05/09/2015
07/11/2015

COMPUTER-BASED PILOT TESTING

02/01/2016–04/01/2016

Theater

Multiple-Choice Items (MCQs)

Number of MCQs Prepared	272
Number of MCQs Piloted*	269
Number of MCQs Identified for Further Review	32
Number of MCQs Deleted Post Pilot Test Review	none

Open-Response Items (ORIs)

Number of ORIs Developed	14
Number of ORIs Piloted*	5
Number of ORIs Identified for Further Review	none
Number of ORIs Deleted Post Pilot Test Review	none

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

Theater

Number of Qualifying Score Participants: 8

Number of Participants Providing Ratings: 8 (100%)

Question [Rating Scale: 1 (not at all well) to 5 (very well)]	Mean Rating
How well do you feel you understood the training that you received for making your qualifying score judgments?	4.8
How confident are you of the qualifying score judgments you made?	4.6
How satisfied are you that you were able to complete the work you were asked to do in the time that was provided?	5.0
How successful were the coordination and logistics of the qualifying score conference?	4.9
Overall, how satisfied are you with the qualifying score process in which you participated?	4.8

Dance

	Number
Committee Participants: Total	7

Public School Educators	5
Higher Education Faculty	2

Gender
Female	6
Male	1

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	0
Hispanic	1
White	5
Other	0
Unspecified	1

Dance

	Number
Committee Participants: Total	13

Public School Educators	11
Higher Education Faculty	2

Gender
Female	9
Male	4

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	4
Asian or Pacific Islander	1
Hispanic	2
White	1
Other	5
Unspecified	0

Deaf/Hard of Hearing
Yes	1
No	12

Dance

	Number
Committee Participants: Total	6

Public School Educators	4
Higher Education Faculty	2

Gender
Female	5
Male	1

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	0
Hispanic	0
White	4
Other	0
Unspecified	2

Dance

INTACT CLASSROOM SESSIONS 2013–2015
10/02/2013	University of Massachusetts - Amherst
10/08/2013	Bridgewater State University
02/05/2014	Salem State University
02/20/2014	Bridgewater State University

STAND ALONE EVENTS AT MTEL OPERATIONAL TEST ADMINISTRATIONS

07/13/2013
10/26/2013

STAND ALONE EVENT(S) IN STATE(S) OTHER THAN MASSACHUSETTS

11/16/2013

Dance

Multiple-Choice Items (MCQs)

NOTE: As described in the "Pilot Testing" section of the Technical Manual, in addition to the review of test items by expert judges (members of the BRC and CACs), when permitted by sufficient candidate populations, empirical data were collected about statistical and qualitative characteristics of the new items through pilot testing. For this field there exists an insufficient candidate population for collecting statistical information through pilot testing. Therefore, pilot test information for multiple-choice questions is unavailable for the field.

Open-Response Items (ORIs)

Number of ORIs Developed	9
Number of ORIs Piloted*	9
Number of ORIs Identified for Further Review	none
Number of ORIs Deleted Post Pilot Test Review	none

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

Dance

Number of Qualifying Score Participants: 6

Number of Participants Providing Ratings: 6 (100%)

Question [Rating Scale: 1 (not at all well) to 5 (very well)]	Mean Rating
How well do you feel you understood the training that you received for making your qualifying score judgments?	4.8
How confident are you of the qualifying score judgments you made?	4.7
How satisfied are you that you were able to complete the work you were asked to do in the time that was provided?	4.8
How successful were the coordination and logistics of the qualifying score conference?	4.8
Overall, how satisfied are you with the qualifying score process in which you participated?	4.7

English as a Second Language (ESL)

	Number
Committee Participants: Total	24

Public School Educators	19
Higher Education Faculty	5

Gender
Female	24
Male	0

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	1
Hispanic	5
White	12
Other	1
Unspecified	5

English as a Second Language (ESL)

	Number
Committee Participants: Total	16

Public School Educators	12
Higher Education Faculty	4

Gender
Female	11
Male	5

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	4
Asian or Pacific Islander	1
Hispanic	3
White	3
Other	5
Unspecified	0

Deaf/Hard of Hearing
Yes	3
No	13

English as a Second Language (ESL)

	Number
Committee Participants: Total	12

Public School Educators	8
Higher Education Faculty	4

Gender
Female	11
Male	1

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	1
Asian or Pacific Islander	0
Hispanic	1
White	7
Other	0
Unspecified	3

English as a Second Language

INTACT CLASSROOM SESSIONS 2013–2015
10/01/2013	Lincoln School (Brookline Public Schools)
10/05/2013	Simmons College Gordon College Salem State University
10/08/2013	Wheaton College
10/18/2013	University of Massachusetts - Boston
10/31/2013	Wheelock College
01/30/2014	Wheaton College
02/01/2014	Gordon College
02/04/2014	Elms College
02/05/2014	Salem State University
02/06/2014	University of Massachusetts - Amherst
02/11/2014	Lincoln School (Brookline Public Schools)
02/14/2014	Wheelock College
02/19/2014	Worcester State University
04/25/2015	Cambridge College
04/27/2015	Salem State University
05/02/2015	Simmons College Framingham State University
05/05/2015	Westfield State University
05/09/2015	Gordon College

STAND ALONE EVENTS AT MTEL OPERATIONAL TEST ADMINISTRATIONS

10/26/2013
03/01/2014
05/10/2014
07/12/2014
10/25/2014

English as a Second Language

Multiple-Choice Items (MCQs)

Number of MCQs Prepared	336
Number of MCQs Piloted*	209
Number of MCQs Identified for Further Review	40
Number of MCQs Deleted Post Pilot Test Review	11

Open-Response Items (ORIs)

Number of ORIs Developed	23
Number of ORIs Piloted*	18
Number of ORIs Identified for Further Review	1
Number of ORIs Deleted Post Pilot Test Review	1

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

English as a Second Language

Number of Qualifying Score Participants: 12

Number of Participants Providing Ratings: 12 (100%)

Question [Rating Scale: 1 (not at all well) to 5 (very well)]	Mean Rating
How well do you feel you understood the training that you received for making your qualifying score judgments?	4.9
How confident are you of the qualifying score judgments you made?	4.7
How satisfied are you that you were able to complete the work you were asked to do in the time that was provided?	4.8
How successful were the coordination and logistics of the qualifying score conference?	4.6
Overall, how satisfied are you with the qualifying score process in which you participated?	4.8

Sheltered English Immersion

	Number
Committee Participants: Total	27

Public School Educators	21
Higher Education Faculty	6

Gender
Female	26
Male	1

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	1
Asian or Pacific Islander	2
Hispanic	3
White	9
Other	1
Unspecified	11

Sheltered English Immersion

	Number
Committee Participants: Total	18

Public School Educators	14
Higher Education Faculty	4

Gender
Female	12
Male	6

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	6
Asian or Pacific Islander	1
Hispanic	3
White	3
Other	5
Unspecified	0

Deaf/Hard of Hearing
Yes	3
No	15

Sheltered English Immersion

	Number
Committee Participants: Total	13

Public School Educators	11
Higher Education Faculty	2

Gender
Female	13
Male	0

Ethnicity
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	2
Hispanic	1
White	5
Other	0
Unspecified	5

Sheltered English Immersion

INTACT CLASSROOM SESSIONS 2013–2014
10/05/2013	Gordon College
10/18/2013	University of Massachusetts - Boston
10/19/2013	Brookline School District
10/31/2013	Wheelock College
02/17/2014	Clarion Hotel, Northampton
04/02–14/2014	Evaluation Systems, Hadley
07/12, 07/16–18/2014	Evaluation Systems, Hadley

STAND ALONE EVENTS AT MTEL OPERATIONAL TEST ADMINISTRATIONS

10/26/2013
03/01/2014

Sheltered English Immersion

Multiple-Choice Items (MCQs)

Number of MCQs Prepared	185
Number of MCQs Piloted*	117
Number of MCQs Identified for Further Review	22
Number of MCQs Deleted Post Pilot Test Review	3

*This chart includes information regarding multiple-choice items piloted prior to the first operational administration and may be a subset of all items developed. Additional items are piloted as non-scorable on operational test forms, until all items are piloted.

Open-Response Items

In consultation with the Massachusetts Department of Education and the Department of Justice review, the open-response item for the Sheltered English Immersion test was designed as a single, five-part open-response item requiring candidates to show their knowledge of sheltering content. Candidates use one of 10 mentor texts for the basis of their response. Each mentor text is an informational text that is representative of the kind of content and academic language features a student might encounter in texts in a given content area. Candidates are expected to spend about 20–30 minutes to prepare an adequate response to each of the five parts of the assignment, for a total of about 2–2 ½ hours for the section. The multiple-choice section counts for 60 percent and the open-response section counts for 40 percent of the candidate's total test score.

Piloting of the new open-response item type focused on evaluating the item format. Pilot test responses to the open-response item format were reviewed for the following characteristics:

Candidate ability to complete the assignment
Testing time required to complete the assignment
Administration process and computer screen layout
Application of the proposed score scale to the responses

A review of the data and item comments collected indicated that the candidates were able to appropriately and adequately respond to the new open-response item type and were able to produce a response within the targeted response time of 2 ½ hours, leaving sufficient time to respond to the multiple-choice questions. In addition, candidates were able to move throughout the various open-response item components with relative ease.

Foundations of Reading

Number of Qualifying Score Participants: 13

Number of Participants Providing Ratings: 13 (100%)

Question [Rating Scale: 1 (not at all well) to 5 (very well)]	Mean Rating
How well do you feel you understood the training that you received for making your qualifying score judgments?	5.0
How confident are you of the qualifying score judgments you made?	4.4
How satisfied are you that you were able to complete the work you were asked to do in the time that was provided?	4.7
How successful were the coordination and logistics of the qualifying score conference?	4.8
Overall, how satisfied are you with the qualifying score process in which you participated?	4.9

Foundations of Reading

Committee participants	Number
Total	32
Public School Educators	19
Higher Education Faculty	13

Gender	Number
Female	30
Male	0
No response	2

Ethnicity	Number
American Indian or Alaska Native	0
Black	1
Asian or Pacific Islander	0
Hispanic	0
White	28
Other	0
Unspecified	3

Foundations of Reading

Bias Review Committee Composition

Committee participants	Number
Total	9
Public School Educators	8
Higher Education Faculty	1

Gender	Number
Female	8
Male	1
No response	0

Ethnicity	Number
American Indian or Alaska Native	0
Black (not of Hispanic origin)	4
Asian or Pacific Islander	0
Hispanic	2
White	2
Other	1
Unspecified	0

Alignment between the MTEL® Test Objectives for Foundations of Reading (190) and the 2017 Massachusetts Curriculum Framework for English Language Arts and Literacy

MTEL 190 Test Objectives: I. Foundations of Reading Development	Curriculum Framework
0001 Demonstrate knowledge of principles and evidence-based instructional practices for developing language and emergent literacy skills, including phonological and phonemic awareness, concepts of print, and the alphabetic principle.	Reading Standards for Foundational Skills [RF]: Print Concepts: Pre-K–1.RF.1 Phonological Awareness: Pre-K–1.RF.2 Phonics and Word Recognition: Pre-K–1.RF.3
0002 Demonstrate knowledge of principles and evidence-based instructional practices for developing beginning reading skills, including phonics, high-frequency words, and spelling.	Reading Standards for Foundational Skills [RF]: Phonics and Word Recognition: Pre-K–5.RF.3
0003 Demonstrate knowledge of principles and evidence-based instructional practices for developing word analysis skills and strategies, including syllabication, structural or morphemic analysis, and orthographic skills.	Reading Standards for Foundational Skills [RF]: Phonics and Word Recognition: Pre-K–5.RF.3
0004 Demonstrate knowledge of principles and evidence-based instructional practices for developing reading fluency at all stages of reading development.	Reading Standards for Foundational Skills [RF]: Fluency: K–5.RF.4

MTEL 190 Test Objectives: II. Development of Reading Comprehension	Curriculum Framework
0005 Demonstrate knowledge of principles and evidence-based instructional practices for promoting academic language development, including vocabulary development.	Language Standards [L]: Conventions of Standard English: Pre-K–6.L.1, K–6.L.2 Knowledge of Language: 2–6.L.3 Vocabulary Acquisition and Use: Pre-K–6.L.4–5
0006 Demonstrate knowledge of principles and evidence-based instructional practices for promoting comprehension and analysis of literary texts.	Reading Standards for Literature [RL]: Key Ideas and Details: Pre-K–6.RL.1–3 Craft and Structure: Pre-K–6.RL.4–6 Integration of Knowledge and Ideas: Pre-K–6.RL.7–9
0007 Demonstrate knowledge of principles and evidence-based instructional practices for promoting comprehension and analysis of informational texts.	Reading Standards for Informational Text [RI]: Key Ideas and Details: Pre-K–6.RI.1–3 Craft and Structure: Pre-K–6.RI.4–6 Integration of Knowledge and Ideas: Pre-K–6.RI.7–9

MTEL 190 Test Objectives:
III. Reading Assessment and Instruction

Curriculum Framework

0008 Apply knowledge of principles and evidence-based best practices for assessing reading development.

N/A

0009 Apply knowledge of principles and evidence-based best practices of reading instruction.

Reading Standards for Literature [RL]:

Range of Reading and Text Complexity:Pre-K–6.RL.10

Reading Standards for Informational Text [RI]:

Range of Reading and Text Complexity: Pre-K–6.RI.10

MTEL 190 Test Objectives: IV. Integration of Knowledge and Understanding: Foundational Reading Skills	Curriculum Framework
0010 Prepare an organized, developed analysis on a topic related to the development of foundational reading skills.	Reading Standards for Foundational Skills [RF]: Phonological Awareness: Pre-K–1.RF.2 Phonics and Word Recognition: Pre-K–5.RF.3 Fluency: K–5.RF.4

MTEL 190 Test Objectives:
V. Integration of Knowledge and Understanding: Reading Comprehension

Curriculum Framework

0011 Prepare an organized, developed analysis on a topic related to the development of reading comprehension.

Language Standards [L]:

Conventions of Standard English: Pre-K–6.L.1, K–6.L.2
Knowledge of Language: 2–6.L.3
Vocabulary Acquisition and Use: Pre-K–6.L.4–5

Reading Standards for Literature [RL]:

Key Ideas and Details: Pre-K–6.RL.1–3
Craft and Structure: Pre-K–6.RL.4–6
Integration of Knowledge and Ideas: Pre-K–6.RL.7–9

Reading Standards for Informational Text [RI]:

Key Ideas and Details: Pre-K–6.RI.1–3
Craft and Structure: Pre-K–6.RI.4–6
Integration of Knowledge and Ideas: Pre-K–6.RI.7–9

Foundations of Reading

Committee participants	Number
Total	14
Public School Educators	10
Higher Education Faculty	4

Gender	Number
Female	13
Male	1
No response	0

Ethnicity	Number
American Indian or Alaska Native	0
Black (not of Hispanic origin)	2
Asian or Pacific Islander	1
Hispanic	1
White	10
Other	0
Unspecified	0

English as a Second Language

INTACT CLASSROOM SESSIONS 2013–2015
10/01/2013	Lincoln School (Brookline Public Schools)
10/05/2013	Simmons College Gordon College Salem State University
10/08/2013	Wheaton College
10/18/2013	University of Massachusetts - Boston
10/31/2013	Wheelock College
01/30/2014	Wheaton College
02/01/2014	Gordon College
02/04/2014	Elms College
02/05/2014	Salem State University
02/06/2014	University of Massachusetts - Amherst
02/11/2014	Lincoln School (Brookline Public Schools)
02/14/2014	Wheelock College
02/19/2014	Worcester State University
04/25/2015	Cambridge College
04/27/2015	Salem State University
05/02/2015	Simmons College Framingham State University
05/05/2015	Westfield State University
05/09/2015	Gordon College

STAND ALONE EVENTS AT MTEL OPERATIONAL TEST ADMINISTRATIONS

10/26/2013
03/01/2014
05/10/2014
07/12/2014
10/25/2014

Pilot Test Outcomes
Foundations of Reading

Pilot testing was conducted at CBT centers from February 18, 2020, through March 3, 2020. The following information is a summary of the data collected during this specific time frame.

Multiple-Choice Items (MCQs) and Short-Answer Items (SAs)

Number of MCQs Prepared	113
Number of MCQs Piloted*	51
Number of MCQs Identified for Further Review	4
Number of MCQs Deleted Post Pilot Test Review	1

Open-Response Items (ORIs)

Number of ORIs Developed	29
Number of ORIs Piloted*	24
Number of ORIs Identified for Further Review	1
Number of ORIs Deleted Post Pilot Test Review	1

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

Summary of Qualifying Score Conference Evaluation Forms
Foundations of Reading

Question Rating Scale: 1–5 1=not at all (well/confident/satisfied/successful), 5=very (well/confident/satisfied/successful)	Median Rating
How well do you feel you understood the training that you received for making your qualifying score judgments?	5
How confident are you of the qualifying score judgments you made?	5
How satisfied are you that you were able to complete the work you were asked to do in the time that was provided?	5
How successful were the coordination and logistics of the qualifying score conference?	4
Overall, how satisfied are you with the qualifying score process in which you participated?	4

13 total participants

Foundations of Reading Content Validation Survey Population/Sample/Respondents Demographics (Public School Sample)

Key

Population: The number and percent of licensed and assigned educators in the content area, extracted from the database of full-time equivalent (FTE) teachers assigned to each teaching area for each school district in Massachusetts, as provided by the Massachusetts Department of Elementary and Secondary Education for use in drawing a random sample of educators.
Sample (with oversampling): The number and percent of licensed and assigned educators sampled to receive the survey. In order to obtain appropriate representation of minority groups (e.g., American Indian, Asian, Black, and Hispanic) in the survey results, oversampling of minority groups was conducted for those fields for which the entire population was not sampled. In these instances, the survey results were weighted appropriately to take the oversampling into account.

Foundations of Reading	Population		Sample (with oversampling)
Foundations of Reading	N	%	N	%
Total	30740	100	425	100
Female	27628	89.87	380	89.41
Male	3112	10.12	45	10.58
American Indian or Alaska Native	34	0.11	2	0.47
Black (not of Hispanic origin)	732	2.38	19	4.47
Asian or Pacific Islander	353	1.14	9	2.11
Hispanic	638	2.07	16	3.76
White	28864	93.89	375	88.23
Other	119	0.38	4	0.94

Foundations of Reading Survey Return Rate by Field and Return Status: Public School Sample

Returned Surveys

Field	Number Sent	Number Not Distributed	Eligible		Ineligible		Total		Adjusted Return Rate
Field	Number Sent	Number Not Distributed	N	%	N	%	N	%	Adjusted Return Rate
061 English	541	30	191	35.30%	32	5.90%	223	43.60%	39.90%
063 Mathematics	609	28	205	33.70%	53	8.70%	258	44.40%	38.80%
064 General Science	530	15	154	29.10%	58	10.90%	212	41.20%	33.70%
065 Middle School Mathematics	581	14	160	27.50%	78	13.40%	238	42.00%	32.70%
190 Foundations of Reading	498	18	166	33.30%	41	8.20%	207	43.10%	37.80%
ALL	2759	105	876	31.80%	262	9.50%	1138	42.90%	36.60%

Foundations of Reading Survey Return Rate by Field and Return Status: Faculty Sample

Returned Surveys

Field	Number Sent	Number Not Distributed	Eligible		Ineligible		Total		Adjusted Return Rate
Field	Number Sent	Number Not Distributed	N	%	N	%	N	%	Adjusted Return Rate
061 English	127	9	18	14.20%	1	0.80%	19	16.10%	15.40%
063 Mathematics	129	8	24	18.60%	0	0.00%	24	19.80%	19.80%
064 General Science	106	9	9	8.50%	2	1.90%	11	11.30%	9.50%
065 Middle School Mathematics	121	9	21	17.40%	3	2.50%	24	21.40%	19.30%
190 Foundations of Reading	138	5	33	23.90%	0	0.00%	33	24.80%	24.80%
ALL	621	40	105	16.90%	6	1.00%	111	19.10%	18.30%

Demographic Summary Report: Public School Sample

Teacher Sample
Field: Foundations of Reading

Question	Answer	Absolute Frequency	Adjusted Percent
Currently Teaching	Yes	166	100
Currently Teaching	No	0	0
Hold a valid Massachusetts teaching license	Yes	166	100
Hold a valid Massachusetts teaching license	No	0	0
What is the highest level of education you have attained?	Bachelor's degree	16	9.7
	Master's degree	142	86.1
	Doctoral degree	2	1.2
	Other	5	3
What is your ethnicity?	American Indian or Alaska Native	1	0.6
	Black (not of Hispanic origin)	4	2.4
	Asian or Pacific Islander	5	3
	Hispanic	0	0
	White (not of Hispanic origin)	150	90.9
	Other	5	3
What is your first (native) language?	English	154	93.3
	French	4	2.4
	Portuguese	1	0.6
	An Asian language	3	1.8
	Other	3	1.8
What is your gender?	Female	136	82.4
What is your gender?	Male	29	17.6
What grade level(s) have you taught in Massachusetts during this or the previous school year? (Select all that apply.)	PreK–6th	129	73.3
	5th–8th	25	14.2
	5th–12th	5	2.8
	8th–12th	17	9.7
How many years of professional teaching experience do you have? (Count partial years as full years.)	1–3 years	4	2.4
	4–10 years	29	17.5
	11 years or more	133	80.1

Foundations of Reading Demographic Summary Report: Faculty Sample

Faculty Sample
Field: Foundations of Reading

Question	Answer	Absolute Frequency	Adjusted Percent
Teaching undergraduate or graduate arts and science or education courses in which prospective teachers may have been enrolled	Yes	33	100
	No	0	0
What is the highest level of education you have attained?	Bachelor's degree	0	0
	Master's degree	7	21.2
	Doctoral degree	24	72.7
	Other	2	6.1
What is your ethnicity?	American Indian or Alaska Native	0	0
	Black (not of Hispanic origin)	0	0
	Asian or Pacific Islander	0	0
	Hispanic	0	0
	White (not of Hispanic origin)	30	93.8
	Other	2	6.3
What is your first (native) language?	English	32	100
	Spanish	0	0
	Other	0	0
What is your gender?	Female	31	96.9
What is your gender?	Male	1	3.1
How many years have you taught as a faculty member at a higher education institution? (Count partial years as full years.)	1–3 years	2	6.3
	4–10 years	10	31.3
	11 years or more	20	62.5
In which higher education institution department do you currently hold your primary appointment?	Education department only	29	90.6
	Both the Education department and another department	2	6.3
	Arts and Sciences or Fine Arts department(s) only	1	3.1
Have you ever taught in the subject of the field and grade level indicated above in Massachusetts?	Yes	25	75.8
	No	8	24.2
How recently have you taught in the subject of the field and grade level indicated above in Massachusetts? (Count partial years as full years.)	1–3 years	4	16.7
	4–10 years	7	29.2
	11 years or more	13	54.2

Foundations of Reading Objective Rating Report: Public School Sample

Number of Objectives: 11

Importance Ratings

Objective Number	N	Mean	SD	SE
1	165	4.35	0.69	0.05
2	164	4.29	0.73	0.06
3	163	3.97	0.77	0.06
4	161	4.3	0.7	0.06
5	162	4.26	0.7	0.06
6	163	4.47	0.63	0.05
7	165	4.43	0.65	0.05
8	164	4.23	0.76	0.06
9	165	4.39	0.73	0.06
10	163	3.88	0.8	0.06
11	164	3.94	0.77	0.06

Number of Respondents: 165

Response Distribution (in %)

NR	2	3	4	5
1	1	10	43	46
1	2	10	44	43
2	1	27	45	26
3	0	13	41	43
2	0	14	43	40
2	0	7	38	53
1	0	8	40	51
1	1	16	40	41
1	1	13	34	52
2	2	33	39	25
1	2	27	45	25

Foundations of Reading Objective Rating Report: Faculty Sample

Objective Rating Report: Ordered by Objective Number
Unweighted

Faculty Sample
Field: Foundations of Reading

Number of Objectives: 11

Importance Ratings

Objective Number	N	Mean	SD	SE
1	33	4.61	0.61	0.11
2	32	4.53	0.62	0.11
3	33	4.36	0.65	0.11
4	32	4.41	0.56	0.1
5	33	4.61	0.5	0.09
6	33	4.76	0.44	0.08
7	31	4.77	0.43	0.08
8	32	4.47	0.51	0.09
9	32	4.53	0.72	0.13
10	33	4.18	0.68	0.12
11	32	4.38	0.66	0.12

Number of Respondents: 33

Response Distribution (in %)

NR	2	3	4	5
0	0	6	27	67
3	0	6	33	58
0	0	9	45	45
3	0	3	52	42
0	0	0	39	61
0	0	0	24	76
6	0	0	21	73
3	0	0	52	45
3	3	3	30	61
0	0	15	52	33
3	0	9	42	45

Foundations of Reading Descriptive Statement Rating Report: Public School Sample

Descriptive Statement Rating Report: Ordered by Objective Number
Unweighted

Number of Objectives: 11

Importance Ratings

Objective Number	N	Mean	SD	SE
1	164	4.32	0.74	0.06
2	166	4.28	0.7	0.05
3	165	4.12	0.76	0.06
4	164	4.3	0.73	0.06
5	163	4.2	0.75	0.06
6	165	4.39	0.64	0.05
7	164	4.38	0.65	0.05
8	164	4.23	0.77	0.06
9	164	4.29	0.7	0.05
10	161	4.01	0.79	0.06
11	161	4.09	0.74	0.06

Number of Respondents: 166

Response Distribution (in %)

NR	1	2	3	4	5
1	1	1	11	41	46
0	0	2	9	48	41
1	0	2	18	46	34
1	0	2	8	45	43
2	0	2	14	45	37
1	0	1	7	46	46
1	0	1	7	45	46
1	0	2	15	41	41
1	0	1	12	44	42
3	0	2	25	42	29
3	0	2	17	49	30

Foundations of Reading Descriptive Statement Rating Report: Faculty Sample

Descriptive Statement Rating Report: Ordered by Objective Number
Unweighted

Faculty Sample
Field: Foundations of Reading

Number of Objectives: 11

Importance Ratings

Objective Number	N	Mean	SD	SE
1	33	4.36	0.86	0.15
2	33	4.24	0.87	0.15
3	31	4.16	0.9	0.16
4	32	4.16	0.95	0.17
5	31	4.23	0.96	0.17
6	31	4.29	0.86	0.16
7	30	4.3	0.75	0.14
8	31	4.35	0.8	0.14
9	33	4.21	1.08	0.19

Number of Respondents: 33

Response Distribution (in %)

NR	1	2	3	4	5
0	0	6	6	33	55
0	0	6	9	39	45
6	0	6	12	36	39
3	0	9	9	36	42
6	3	3	6	39	42
6	3	0	6	42	42
9	0	3	6	42	39
6	0	3	9	33	48
0	3	6	12	24	55

Foundations of Reading Composite Rating Report: Public School Sample

Composite Rating Report
Unweighted

Teacher Sample
Field: Foundations of Reading

Importance Ratings

N	Mean	SD	SE
162	4.31	0.73	0.06

Number of Respondents: 162

Response Distribution (in %)

NR	1	2	3	4	5
0	1	0	12	41	46

Foundations of Reading Composite Rating Report: Faculty Sample

Composite Rating Report
Unweighted

Education Faculty Sample
Field: Foundations of Reading

Importance Ratings

N	Mean	SD	SE
33	4.3	0.92	0.16

Number of Respondents: 33

Response Distribution (in %)

NR	1	2	3	4	5
0	3	3	3	42	48

Foundations of Reading Objective Ratings Summary

Objective	Number		Mean Objective Rating		Description of Objective
Objective	Educators	Faculty	Educators	Faculty	Description of Objective
1	163	33	4.39	4.61	Demonstrate knowledge of principles and research-based instructional practices for developing language and emergent literacy skills, including phonological and phonemic awareness, concepts of print, and the alphabetic principle.
2	162	32	4.34	4.53	Demonstrate knowledge of principles and research-based instructional practices for developing beginning reading skills, including phonics, sight words, and spelling.
3	161	33	4.01	4.36	Demonstrate knowledge of principles and research-based instructional practices for developing word analysis skills and strategies, including syllabication, structural or morphemic analysis, and orthographic skills.
4	160	32	4.34	4.41	Demonstrate knowledge of principles and research-based instructional practices for developing reading fluency at all stages of reading development.
5	160	33	4.27	4.61	Demonstrate knowledge of principles and research-based instructional practices for promoting academic language development, including vocabulary development.
6	161	33	4.49	4.76	Demonstrate knowledge of principles and research-based instructional practices for promoting comprehension and analysis of literary texts.
7	163	31	4.45	4.77	Demonstrate knowledge of principles and research-based instructional practices for promoting comprehension and analysis of informational texts.
8	162	32	4.26	4.47	Apply knowledge of principles and research-based best practices for assessing reading development.
9	163	32	4.43	4.53	Apply knowledge of principles and research-based best practices of reading instruction.
10	161	33	3.9	4.18	Prepare an organized, developed analysis on a topic related to the development of foundational reading skills.
11	162	32	3.96	4.38	Prepare an organized, developed analysis on a topic related to the development of reading comprehension.
Overall	163	33	4.26	4.51

Biology

Committee participants	Number
Total	31
Public School Educators	27
Higher Education Faculty	4

Gender	Number
Female	24
Male	2
No response	5

Ethnicity	Number
American Indian or Alaska Native	0
Black	3
Asian or Pacific Islander	0
Hispanic	0
White	23
Other	0
Unspecified	5

Biology

Bias Review Committee Composition

Committee participants	Number
Total	10
Public School Educators	9
Higher Education Faculty	1

Gender	Number
Female	8
Male	2
No response	0

Ethnicity	Number
American Indian or Alaska Native	0
Black (not of Hispanic origin)	6
Asian or Pacific Islander	2
Hispanic	0
White	1
Other	1
Unspecified	0

Alignment between the MTEL® Test Objectives for Biology (066) and the 2017 Massachusetts Curriculum Framework for Biology

MTEL 066 Test Objectives: I. Molecules: Structures and Processes	Curriculum Framework
0001 Apply knowledge of the chemical components of living systems and basic principles of biochemistry.	HS–LS1–5. Use a model to illustrate how photosynthesis uses light energy to transform water and carbon dioxide into oxygen and chemical energy stored in the bonds of sugars and other carbohydrates.
	HS–LS1–6. Construct an explanation based on evidence that organic molecules are primarily composed of six elements, where carbon, hydrogen, and oxygen atoms may combine with nitrogen, sulfur, and phosphorus to form monomers that can further combine to form large carbon-based macromolecules.
	HS–LS1–7. Use a model to illustrate that aerobic cellular respiration is a chemical process whereby the bonds of food molecules and oxygen molecules are broken and new bonds form, resulting in new compounds and a net transfer of energy.
0002 Apply knowledge of the processes that generate cellular energy.	HS–LS1–3. Provide evidence that homeostasis maintains internal body conditions through both body-wide feedback mechanisms and small-scale cellular processes.
	HS–LS1–5. Use a model to illustrate how photosynthesis uses light energy to transform water and carbon dioxide into oxygen and chemical energy stored in the bonds of sugars and other carbohydrates.
	HS–LS1–7. Use a model to illustrate that aerobic cellular respiration is a chemical process whereby the bonds of food molecules and oxygen molecules are broken and new bonds form, resulting in new compounds and a net transfer of energy.

MTEL 066 Test Objectives: II. Organisms: Structure and Processes	Curriculum Framework
0003 Apply knowledge of the cell cycle and cell structure and function.	HS–LS1–3. Provide evidence that homeostasis maintains internal body conditions through both body-wide feedback mechanisms and small-scale cellular processes.
	HS–LS1–4. Construct an explanation using evidence for why the cell cycle is necessary for the growth, maintenance, and repair of multicellular organisms. Model the major events of the cell cycle, including (a) cell growth and DNA replication, (b) separation of chromosomes (mitosis), and (c) separation of cell contents.
	HS–LS4–4. Research and communicate information about key features of viruses and bacteria to explain their ability to adapt and reproduce in a wide variety of environments.
0004 Apply knowledge of the structures, structural organization, and life processes of unicellular and multicellular organisms (i.e., archaea, bacteria, protists, fungi, plants, and animals).	HS–PS1–7. Use mathematical representations and provide experimental evidence to support the claim that atoms, and therefore mass, are conserved during a chemical reaction. Use the mole concept and proportional relationships to evaluate the quantities (masses or moles) of specific reactants needed in order to obtain a specific amount of product.
	HS–LS1–2. Develop and use a model to illustrate the key functions of animal body systems, including (a) food digestion, nutrient uptake, and transport through the body; (b) exchange of oxygen and carbon dioxide; (c) removal of wastes; and (d) regulation of body processes.
	HS–LS1–3. Provide evidence that homeostasis maintains internal body conditions through both body-wide feedback mechanisms and small-scale cellular processes.
	HS–LS1–4. Construct an explanation using evidence for why the cell cycle is necessary for the growth, maintenance, and repair of multicellular organisms. Model the major events of the cell cycle, including (a) cell growth and DNA replication, (b) separation of chromosomes (mitosis), and (c) separation of cell contents.
	HS–LS4–4. Research and communicate information about key features of viruses and bacteria to explain their ability to adapt and reproduce in a wide variety of environments.
0005 Apply knowledge of human anatomy and physiology.	HS–LS1–2. Develop and use a model to illustrate the key functions of animal body systems, including (a) food digestion, nutrient uptake, and transport through the body; (b) exchange of oxygen and carbon dioxide; (c) removal of wastes; and (d) regulation of body processes.
0005 Apply knowledge of human anatomy and physiology.	HS–LS1–3. Provide evidence that homeostasis maintains internal body conditions through both body-wide feedback mechanisms and small-scale cellular processes.

MTEL 066 Test Objectives: III. Ecosystems: Interactions, Energy, and Dynamics	Curriculum Framework
0006 Analyze interactions and dynamics of populations, communities, ecosystems, and biomes.	HS–LS2–1. Analyze data sets to support explanations that biotic and abiotic factors affect ecosystem carrying capacity.
	HS–LS2–2. Use mathematical representations to support explanations that biotic and abiotic factors affect biodiversity, including genetic diversity within a population and species diversity within an ecosystem.
	HS–LS2–6. Analyze data to show ecosystems tend to maintain relatively consistent numbers and types of organisms even when small changes in conditions occur but that extreme fluctuations in conditions may result in a new ecosystem. Construct an argument supported by evidence that ecosystems with greater biodiversity tend to have greater resistance to change and resilience.
0007 Apply knowledge of the cycling of materials and the transfer of energy through an ecosystem.	HS–LS2–4. Use a mathematical model to describe the transfer of energy from one trophic level to another. Explain how the inefficiency of energy transfer between trophic levels affects the relative number of organisms that can be supported at each trophic level and necessitates a constant input of energy from sunlight or inorganic compounds from the environment.
	HS–LS2–5. Use a model that illustrates the roles of photosynthesis, cellular respiration, decomposition, and combustion to explain the cycling of carbon in its various forms among the biosphere, atmosphere, hydrosphere, and geosphere.
0008 Analyze the effects of human activities on ecosystems and the environment.	HS–LS2–1. Analyze data sets to support explanations that biotic and abiotic factors affect ecosystem carrying capacity.
	HS–LS2–2. Use mathematical representations to support explanations that biotic and abiotic factors affect biodiversity, including genetic diversity within a population and species diversity within an ecosystem.
	HS–LS2–6. Analyze data to show ecosystems tend to maintain relatively consistent numbers and types of organisms even when small changes in conditions occur but that extreme fluctuations in conditions may result in a new ecosystem. Construct an argument supported by evidence that ecosystems with greater biodiversity tend to have greater resistance to change and resilience.
	HS–LS2–7. Analyze direct and indirect effects of human activities on biodiversity and ecosystem health, specifically habitat fragmentation, introduction of non-native or invasive species, overharvesting, pollution, and climate change. Evaluate and refine a solution for reducing the impacts of human activities on biodiversity and ecosystem health.

MTEL 066 Test Objectives: IV. Heredity and Biological Evolution	Curriculum Framework
0009 Apply knowledge of the molecular basis of genetics.	HS–LS1–1. Construct a model of transcription and translation to explain the roles of DNA and RNA that code for proteins that regulate and carry out essential functions of life.
	HS–LS3–4– M A . Use scientific information to illustrate that many traits of individuals, and the presence of specific alleles in a population, are due to interactions of genetic factors and environmental factors.
0010 Apply knowledge of the principles of genetics to understand the inheritance and variation of traits.	HS–LS3–1. Develop and use a model to show how DNA in the form of chromosomes is passed from parents to offspring through the processes of meiosis and fertilization in sexual reproduction.
	HS–LS3–2. Make and defend a claim based on evidence that genetic variations (alleles) may result from (a) new genetic combinations via the processes of crossing over and random segregation of chromosomes during meiosis, (b) mutations that occur during replication, and/or (c) mutations caused by environmental factors. Recognize that mutations that occur in gametes can be passed to offspring.
	HS–LS3–3. Apply concepts of probability to represent possible genotype and phenotype combinations in offspring caused by different types of Mendelian inheritance patterns.
	HS–LS3–4– M A . Use scientific information to illustrate that many traits of individuals, and the presence of specific alleles in a population, are due to interactions of genetic factors and environmental factors.
0011 Apply knowledge of the theories and mechanisms of evolution to understand biological change and the diversity of life.	HS–LS2–2. Use mathematical representations to support explanations that biotic and abiotic factors affect biodiversity, including genetic diversity within a population and species diversity within an ecosystem.
	HS–LS3–4– M A . Use scientific information to illustrate that many traits of individuals, and the presence of specific alleles in a population, are due to interactions of genetic factors and environmental factors.
	HS–LS4–1. Communicate scientific information that common ancestry and biological evolution are supported by multiple lines of empirical evidence, including molecular, anatomical, and developmental similarities inherited from a common ancestor (homologies), seen through fossils and laboratory and field observations.
	HS–LS4–2. Construct an explanation based on evidence that Darwin's theory of evolution by natural selection occurs in a population when the following conditions are met: (a) more offspring are produced than can be supported by the environment, (b) there is heritable variation among individuals, and (c) some of these variations lead to differential fitness among individuals as some individuals are better able to compete for limited resources than others.
	HS–LS4–4. Research and communicate information about key features of viruses and bacteria to explain their ability to adapt and reproduce in a wide variety of environments.
	HS–LS4–5. Evaluate models that demonstrate how changes in an environment may result in the evolution of a population of a given species, the emergence of new species over generations, or the extinction of other species due to the processes of genetic drift, gene flow, mutation, and natural selection.

MTEL 066 Test Objectives: V. Integration of Knowledge and Understanding	Curriculum Framework
0012 Prepare an organized, developed analysis of a key topic in biology related to Molecules: Structures and Processes or Heredity and Biological Evolution.	HS–LS1–1. Construct a model of transcription and translation to explain the roles of DNA and RNA that code for proteins that regulate and carry out essential functions of life.
	HS–LS1–3. Provide evidence that homeostasis maintains internal body conditions through both body-wide feedback mechanisms and small-scale cellular processes.
	HS–LS1–4. Construct an explanation using evidence for why the cell cycle is necessary for the growth, maintenance, and repair of multicellular organisms. Model the major events of the cell cycle, including (a) cell growth and DNA replication, (b) separation of chromosomes (mitosis), and (c) separation of cell contents.
	HS–LS1–5. Use a model to illustrate how photosynthesis uses light energy to transform water and carbon dioxide into oxygen and chemical energy stored in the bonds of sugars and other carbohydrates.
	HS–LS1–6. Construct an explanation based on evidence that organic molecules are primarily composed of six elements, where carbon, hydrogen, and oxygen atoms may combine with nitrogen, sulfur, and phosphorus to form monomers that can further combine to form large carbon-based macromolecules.
	HS–LS1–7. Use a model to illustrate that aerobic cellular respiration is a chemical process whereby the bonds of food molecules and oxygen molecules are broken and new bonds form, resulting in new compounds and a net transfer of energy.
	HS–LS2–2. Use mathematical representations to support explanations that biotic and abiotic factors affect biodiversity, including genetic diversity within a population and species diversity within an ecosystem.
	HS–LS3–1. Develop and use a model to show how DNA in the form of chromosomes is passed from parents to offspring through the processes of meiosis and fertilization in sexual reproduction.
	HS–LS3–2. Make and defend a claim based on evidence that genetic variations (alleles) may result from (a) new genetic combinations via the processes of crossing over and random segregation of chromosomes during meiosis, (b) mutations that occur during replication, and/or (c) mutations caused by environmental factors. Recognize that mutations that occur in gametes can be passed to offspring.
	HS–LS3–3. Apply concepts of probability to represent possible genotype and phenotype combinations in offspring caused by different types of Mendelian inheritance patterns.
	HS–LS3–4– M A . Use scientific information to illustrate that many traits of individuals, and the presence of specific alleles in a population, are due to interactions of genetic factors and environmental factors.
	HS–LS4–1. Communicate scientific information that common ancestry and biological evolution are supported by multiple lines of empirical evidence, including molecular, anatomical, and developmental similarities inherited from a common ancestor (homologies), seen through fossils and laboratory and field observations.
	HS–LS4–2. Construct an explanation based on evidence that Darwin’s theory of evolution by natural selection occurs in a population when the following conditions are met: (a) more offspring are produced than can be supported by the environment, (b) there is heritable variation among individuals, and (c) some of these variations lead to differential fitness among individuals as some individuals are better able to compete for limited resources than others.
	HS–LS4–4. Research and communicate information about key features of viruses and bacteria to explain their ability to adapt and reproduce in a wide variety of environments.
	HS–LS4–5. Evaluate models that demonstrate how changes in an environment may result in the evolution of a population of a given species, the emergence of new species over generations, or the extinction of other species due to the processes of genetic drift, gene flow, mutation, and natural selection.
0013 Prepare an organized, developed analysis of a key topic in biology related to Organisms: Structures and Processes or Ecosystems: Interaction, Energy, and Dynamics that emphasizes the application of science and engineering practices in a classroom setting.	HS–LS1–2. Develop and use a model to illustrate the key functions of animal body systems, including (a) food digestion, nutrient uptake, and transport through the body; (b) exchange of oxygen and carbon dioxide; (c) removal of wastes; and (d) regulation of body processes.
	HS–LS1–3. Provide evidence that homeostasis maintains internal body conditions through both body-wide feedback mechanisms and small-scale cellular processes.
	HS–LS1–4. Construct an explanation using evidence for why the cell cycle is necessary for the growth, maintenance, and repair of multicellular organisms. Model the major events of the cell cycle, including (a) cell growth and DNA replication, (b) separation of chromosomes (mitosis), and (c) separation of cell contents.
	HS–LS2–1. Analyze data sets to support explanations that biotic and abiotic factors affect ecosystem carrying capacity.
	HS–LS2–2. Use mathematical representations to support explanations that biotic and abiotic factors affect biodiversity, including genetic diversity within a population and species diversity within an ecosystem.
	HS–LS2–4. Use a mathematical model to describe the transfer of energy from one trophic level to another. Explain how the inefficiency of energy transfer between trophic levels affects the relative number of organisms that can be supported at each trophic level and necessitates a constant input of energy from sunlight or inorganic compounds from the environment.
	HS–LS2–5. Use a model that illustrates the roles of photosynthesis, cellular respiration, decomposition, and combustion to explain the cycling of carbon in its various forms among the biosphere, atmosphere, hydrosphere, and geosphere.
	HS–LS2–6. Analyze data to show ecosystems tend to maintain relatively consistent numbers and types of organisms even when small changes in conditions occur but that extreme fluctuations in conditions may result in a new ecosystem. Construct an argument supported by evidence that ecosystems with greater biodiversity tend to have greater resistance to change and resilience.
	HS–LS2–7. Analyze direct and indirect effects of human activities on biodiversity and ecosystem health, specifically habitat fragmentation, introduction of non-native or invasive species, overharvesting, pollution, and climate change. Evaluate and refine a solution for reducing the impacts of human activities on biodiversity and ecosystem health.
	HS–LS4–4. Research and communicate information about key features of viruses and bacteria to explain their ability to adapt and reproduce in a wide variety of environments.

Biology

Committee participants	Number
Total	11
Public School Educators	9
Higher Education Faculty	2

Gender	Number
Female	6
Male	2
No response	3

Ethnicity	Number
American Indian or Alaska Native	0
Black (not of Hispanic origin)	1
Asian or Pacific Islander	1
Hispanic	0
White	7
Other	0
Unspecified	2

English as a Second Language

INTACT CLASSROOM SESSIONS 2013–2015
10/01/2013	Lincoln School (Brookline Public Schools)
10/05/2013	Simmons College Gordon College Salem State University
10/08/2013	Wheaton College
10/18/2013	University of Massachusetts - Boston
10/31/2013	Wheelock College
01/30/2014	Wheaton College
02/01/2014	Gordon College
02/04/2014	Elms College
02/05/2014	Salem State University
02/06/2014	University of Massachusetts - Amherst
02/11/2014	Lincoln School (Brookline Public Schools)
02/14/2014	Wheelock College
02/19/2014	Worcester State University
04/25/2015	Cambridge College
04/27/2015	Salem State University
05/02/2015	Simmons College Framingham State University
05/05/2015	Westfield State University
05/09/2015	Gordon College

STAND ALONE EVENTS AT MTEL OPERATIONAL TEST ADMINISTRATIONS

10/26/2013
03/01/2014
05/10/2014
07/12/2014
10/25/2014

Pilot Test Outcomes
Biology

Pilot testing was conducted at CBT centers from March 28, 2022, through April 22, 2022. The following information is a summary of the data collected during this specific time frame.

Multiple-Choice Items ( M C Q's ) and Short-Answer Items ( S A's )

Number of MCQs Prepared	62
Number of M C Q's Piloted*	62
Number of M C Q's Identified for Further Review	16
Number of M C Q's Deleted Post Pilot Test Review	2

Open-Response Items ( O R I's )

Number of O R I's Developed	15
Number of O R I's Piloted*	9
Number of O R I's Identified for Further Review	0
Number of O R I's Deleted Post Pilot Test Review	0

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

Biology

Number of Qualifying Score Participants: 11

Number of Participants Providing Ratings: 10 (91%)

Question Rating Scale: 1–5 1=not at all (well/confident/satisfied/successful), 5=very (well/confident/satisfied/successful)	Median Rating
How well do you feel you understood the training that you received for making your qualifying score judgments?	5
How confident are you of the qualifying score judgments you made?	4
How satisfied are you that you were able to complete the work you were asked to do in the time that was provided?	5
How successful were the coordination and logistics of the qualifying score conference?	5
Overall, how satisfied are you with the qualifying score process in which you participated?	5

Biology Content Validation Survey Population/Sample/Respondents Demographics (Public School Sample)

Key

Population: The number and percent of licensed and assigned educators in the content area, extracted from the database of full-time equivalent (FTE) teachers assigned to each teaching area for each school district in Massachusetts, as provided by the Massachusetts Department of Elementary and Secondary Education for use in drawing a random sample of educators.
Sample (with oversampling): The number and percent of licensed and assigned educators sampled to receive the survey. In order to obtain appropriate representation of minority groups (e.g., American Indian, Asian, Black, and Hispanic) in the survey results, oversampling of minority groups was conducted for those fields for which the entire population was not sampled. In these instances, the survey results were weighted appropriately to take the oversampling into account.

Biology	Population		Sample (with oversampling)
Biology	N	%	N	%
Total	2,094	100	300	100
Female	1,365	65.18	196	65.33
Male	729	34.81	104	34.66
American Indian or Alaska Native	0	0	0	0
Black (not of Hispanic origin)	27	1.28	4	1.33
Asian or Pacific Islander	50	2.38	7	2.33
Hispanic	50	2.38	7	2.33
White	1,955	93.36	280	93.33
Other	12	0.57	2	0.66

Biology Survey Return Rate by Field and Return Status: Public School Sample

Returned Surveys

Field	Number Sent	Number Not Distributed	Eligible		Ineligible		Total		Adjusted Return Rate
Field	Number Sent	Number Not Distributed	N	%	N	%	N	%	Adjusted Return Rate
066 Biology	296	17	62	20.90%	11	3.70%	73	24.70%	21.80%
067 Chemistry	293	7	58	19.80%	14	4.80%	72	24.60%	20.80%
068 Mathematics (Elementary)	298	8	72	24.20%	13	4.40%	85	28.50%	25.30%
069 Physics	298	27	59	19.80%	8	2.70%	67	22.50%	20.30%
070 Technology/Engineering	298	22	49	16.40%	17	5.70%	66	22.10%	17.40%
071 Digital Literacy and Computer Science	32	2	19	59.40%	0	0.00%	19	59.40%	59.40%
074 Earth and Space Science	149	3	21	14.10%	10	6.70%	31	20.80%	15.10%
ALL	1,664	86	340	20.40%	73	4.40%	413	24.80%	21.40%

Biology Survey Return Rate by Field and Return Status: Faculty Sample

Returned Surveys

Field	Number Sent	Total
Field	Number Sent	N	%
066 Biology	37	7	18.91%
067 Chemistry	36	5	13.88%
068 Mathematics (Elementary)	3	1	33.33%
069 Physics	28	2	7.14%
070 Technology/Engineering	1	0	0%
071 Digital Literacy and Computer Science	1	0	0%
074 Earth and Space Science	22	0	0%
ALL	128	15	11.71%

Demographic Summary Report: Public School Sample

Teacher Sample
Field: Biology

Question	Answer	Absolute Frequency	Adjusted Percent
Currently Teaching	Yes	62	100
Currently Teaching	No	0	0
Hold a valid Massachusetts teaching license	Yes	62	100
Hold a valid Massachusetts teaching license	No	0	0
What is the highest level of education you have attained?	Bachelor's degree	4	6
	Master's degree	55	89
	Doctoral degree	3	5
	Other	0	0
What is your ethnicity?	American Indian or Alaska Native	0	0
	Black (not of Hispanic origin)	0	0
	Asian or Pacific Islander	0	0
	Hispanic	2	3
	White (not of Hispanic origin)	59	95
	Other	0	0
What is your first (native) language?	English	58	94
	French	0	0
	Portuguese	1	2
	An Asian language	0	0
	Russian	0	0
	Spanish	1	2
	American Sign Language	0	0
	Italian	0	0
	Other	0	0
What is your gender?	Female	48	77
What is your gender?	Male	10	16
What grade level(s) have you taught in Massachusetts during this or the previous school year? (Select all that apply.)	Pre K to 6th	0	0
	5th to 8th	0	0
	5th to 12th	3	5
	8th to 12th	56	90
How many years of professional teaching experience do you have? (Count partial years as full years.)	1 to 3 years	3	5
	4 to 10 years	11	18
	11 years or more	47	76

Biology Demographic Summary Report: Faculty Sample

Faculty Sample
Field: Biology

Question	Answer	Absolute Frequency	Adjusted Percent
Teaching undergraduate or graduate arts and science or education courses in which prospective teachers may have been enrolled	Yes	7	100
	No	0	0
What is the highest level of education you have attained?	Bachelor's degree	0	0
	Master's degree	1	14
	Doctoral degree	1	86
	Other	0	0
What is your ethnicity?	American Indian or Alaska Native	0	0
	Black (not of Hispanic origin)	0	0
	Asian or Pacific Islander	0	0
	Hispanic	0	0
	White (not of Hispanic origin)	7	100
	Other	0	0
What is your first (native) language?	English	7	100
	Spanish	0	0
	Other	0	0
What is your gender?	Female	4	57
What is your gender?	Male	3	43
How many years have you taught as a faculty member at a higher education institution? (Count partial years as full years.)	1 to 3 years	0	0
	4 to 10 years	2	29
	11 years or more	5	71
In which higher education institution department do you currently hold your primary appointment?	Education department only	0	0
	Both the Education department and another department	1	14
	Arts and Sciences or Fine Arts department(s) only	6	86
Have you ever taught in the subject of the field and grade level indicated above in Massachusetts?	Yes	2	29
	No	5	71
How recently have you taught in the subject of the field and grade level indicated above in Massachusetts? (Count partial years as full years.)	1 to 3 years	0	0
	4 to 10 years	0	0
	11 years or more	2	29

Biology Objective Rating Report: Public School Sample

Number of Objectives: 13

Importance Ratings

Objective Number	N	Mean	SD	S E
1	60	4.12	0.74	0.1
2	60	3.95	0.81	0.1
3	60	4.17	0.81	0.1
4	60	3.43	0.91	0.12
5	60	3.85	0.8	0.1
6	60	4.18	0.75	0.1
7	60	4.05	0.7	0.09
8	61	4.03	0.87	0.11
9	61	4.54	0.56	0.07
10	61	4.46	0.59	0.08
11	61	4.52	0.65	0.08
12	61	3.74	0.89	0.11
13	61	3.84	0.78	0.1

Number of Respondents: 61

Response Distribution (in %)

2	3	4	5
0	13	27	20
2	15	27	16
2	9	26	23
9	24	19	8
2	18	27	13
1	9	28	22
1	10	34	15
4	10	27	20
0	2	24	35
0	3	27	31
0	5	19	37
4	22	21	14
3	15	32	11

Biology Objective Rating Report: Faculty Sample

Objective Rating Report: Ordered by Objective Number
Unweighted

Faculty Sample
Field: Biology

Number of Objectives: 13

Importance Ratings

Objective Number	N	Mean	SD	S E
1	7	4.29	0.76	0.29
2	7	4.43	0.53	0.2
3	6	4.33	0.82	0.33
4	7	4	0.58	0.22
5	6	4.17	0.75	0.31
6	7	4.14	0.69	0.26
7	6	4.17	0.41	0.17
8	7	4.43	0.53	0.2
9	7	4.14	0.69	0.26
10	7	4.29	0.49	0.18
11	7	4.43	0.53	0.2
12	7	4.43	0.53	0.2
13	7	4.43	0.53	0.2

Number of Respondents: 7

Response Distribution (in %)

3	4	5
1	3	3
0	4	3
1	2	3
1	5	1
1	3	2
1	4	2
0	5	1
0	4	3
1	4	2
0	5	2
0	4	3
0	4	3
0	4	3

Biology Descriptive Statement Rating Report: Public School Sample

Descriptive Statement Rating Report: Ordered by Objective Number
Unweighted

Number of Objectives: 13

Importance Ratings

Objective Number	N	Mean	SD	S E
1	62	4.18	0.78	0.01
2	62	4.03	0.87	0.11
3	61	4.1	0.91	0.12
4	62	3.34	1.04	0.13
5	61	3.72	1.02	0.13
6	62	4.1	0.78	0.1
7	61	4.13	0.83	0.11
8	62	4.13	0.82	0.1
9	62	4.02	0.78	0.1
10	62	4.29	0.71	0.09
11	60	4.28	0.78	0.1
12	61	3.44	0.94	0.12
13	60	3.72	0.9	0.12

Number of Respondents: 62

Response Distribution (in %)

1	2	3	4	5
0	1	11	26	24
0	4	10	28	20
0	4	10	23	24
2	11	22	18	9
1	8	12	26	14
0	2	10	30	20
0	3	8	28	22
0	1	14	23	24
0	4	6	37	15
0	1	6	29	26
1	1	3	30	25
0	10	23	19	9
0	5	20	22	13

Biology Descriptive Statement Rating Report: Faculty Sample

Descriptive Statement Rating Report: Ordered by Objective Number
Unweighted

Faculty Sample
Field: Biology

Number of Objectives: 13

Importance Ratings

Objective Number	N	Mean	SD	S E
1	7	4.29	0.76	0.29
2	7	4	0.58	0.22
3	7	4.14	0.69	0.26
4	7	4.29	0.49	0.18
5	7	3.71	0.95	0.36
6	7	4.57	0.53	0.2
7	7	4.14	0.38	0.14
8	7	4.14	0.38	0.14
9	7	4.29	0.49	0.18
10	7	4.29	0.49	0.18
11	7	4.29	0.49	0.18
12	6	3.5	1.05	0.43
13	7	4.14	0.69	0.26

Number of Respondents: 7

Response Distribution (in %)

2	3	4	5
0	1	3	3
0	1	5	1
0	1	4	2
0	0	5	2
1	1	4	1
0	0	3	4
0	0	6	1
0	0	6	1
0	0	5	2
0	0	5	2
0	0	5	2
1	2	2	1
0	1	4	2

Biology Composite Rating Report: Public School Sample

Composite Rating Report
Unweighted

Teacher Sample
Field: Biology

Importance Ratings

N	Mean	SD	S E
61	4.11	0.7	0.09

Number of Respondents: 61

Response Distribution (in %)

NR	1	2	3	4	5
0	0	1	9	34	18

Biology Composite Rating Report: Faculty Sample

Composite Rating Report
Unweighted

Education Faculty Sample
Field: Biology

Importance Ratings

N	Mean	SD	S E
7	4.43	0.53	0.2

Number of Respondents: 7

Response Distribution (in %)

NR	1	2	3	4	5
0	0	0	0	4	3

Biology Objective Ratings Summary

Objective	Number		Mean Objective Rating		Description of Objective
Objective	Educators	Faculty	Educators	Faculty	Description of Objective
1	60	7	4.12	4.29	Apply knowledge of the chemical components of living systems and basic principles of biochemistry.
2	60	7	3.95	4.43	Apply knowledge of the processes that generate cellular energy.
3	60	6	4.17	4.33	Apply knowledge of cell structure and function and of the cell cycle.
4	60	7	3.43	4	Apply knowledge of the structures, structural organization, and life processes of unicellular and multicellular organisms (i.e., archaea, bacteria, protists, fungi, plants, and animals).
5	60	6	3.85	4.17	Apply knowledge of human anatomy and physiology.
6	60	7	4.18	4.14	Analyze interactions and dynamics of populations, communities, ecosystems, and biomes.
7	60	6	4.05	4.17	Apply knowledge of the cycling of materials and the transfer of energy through an ecosystem.
8	61	7	4.03	4.43	Analyze the effects of human activities on ecosystems and the environment.
9	61	7	4.54	4.14	Apply knowledge of the molecular basis of genetics.
10	61	7	4.46	4.29	Apply knowledge of the principles of genetics to understand the inheritance and variation of traits.
11	61	7	4.52	4.43	Apply knowledge of the theories and mechanisms of evolution to understand biological change and the diversity of life.
12	61	7	3.74	4.43	Prepare an organized, developed analysis of a key topic in biology related to Molecules: Structures and Processes or Heredity and Biological Evolution.
13	61	7	3.84	4.43	Prepare an organized, developed analysis of a key topic in biology related to Organisms: Structures and Processes or Ecosystems: Interaction, Energy, and Dynamics that emphasizes the application of science and engineering practices in a classroom setting.
Overall	61	7	4.11	4.43

Chemistry

Committee participants	Number
Total	22
Public School Educators	20
Higher Education Faculty	2

Gender	Number
Female	11
Male	9
No response	2

Ethnicity	Number
American Indian or Alaska Native	0
Black	0
Asian or Pacific Islander	3
Hispanic	0
White	16
Other	1
Unspecified	2

Chemistry

Bias Review Committee Composition

Committee participants	Number
Total	10
Public School Educators	9
Higher Education Faculty	1

Gender	Number
Female	8
Male	2
No response	0

Ethnicity	Number
American Indian or Alaska Native	0
Black (not of Hispanic origin)	6
Asian or Pacific Islander	2
Hispanic	0
White	1
Other	1
Unspecified	0

Alignment between the MTEL® Test Objectives for Chemistry (067) and the 2017 Massachusetts Curriculum Framework for Chemistry

MTEL 067 Test Objectives: I. MATTER AND ITS INTERACTIONS: PERIODIC PROPERTIES	Curriculum Framework
0001 Apply knowledge of atomic and subatomic structure and the principles of quantum theory.	8.MS–PS1–1. Develop a model to describe that (a) atoms combine in a multitude of ways to produce pure substances which make up all of the living and nonliving things that we encounter, (b) atoms form molecules and compounds that range in size from two to thousands of atoms, and (c) mixtures are composed of different proportions of pure substances.
	HS–PS1–1. Use the periodic table as a model to predict the relative properties of main group elements, including ionization energy and relative sizes of atoms and ions, based on the patterns of electrons in the outermost energy level of each element. Use the patterns of valence electron configurations, core charge, and Coulomb's law to explain and predict general trends in ionization energies, relative sizes of atoms and ions, and reactivity of pure elements.
0002 Apply knowledge of periodic properties and the organization of the periodic table.	HS–PS1–1. Use the periodic table as a model to predict the relative properties of main group elements, including ionization energy and relative sizes of atoms and ions, based on the patterns of electrons in the outermost energy level of each element. Use the patterns of valence electron configurations, core charge, and Coulomb's law to explain and predict general trends in ionization energies, relative sizes of atoms and ions, and reactivity of pure elements.
	HS–PS1–2. Use the periodic table model to predict and design simple reactions that result in two main classes of binary compounds, ionic and molecular. Develop an explanation based on given observational data and the electronegativity model about the relative strengths of ionic or covalent bonds.

MTEL 067 Test Objectives: II. MATTER AND ITS INTERACTIONS: CHEMICAL STRUCTURE AND REACTIONS	Curriculum Framework
0003 Apply knowledge of the nomenclature and structure of inorganic and organic compounds.	HS–PS2–6. Communicate scientific and technical information about the molecular-level structures of polymers, ionic compounds, acids and bases, and metals to justify why these are useful in the functioning of designed materials.
0004 Apply knowledge of chemical bonding and the relationship between bond type and the properties of substances.	HS–PS1–3. Cite evidence to relate physical properties of substances at the bulk scale to spatial arrangements, movement, and strength of electrostatic forces among ions, small molecules, or regions of large molecules in the substances. Make arguments to account for how compositional and structural differences in molecules result in different types of intermolecular or intramolecular interactions.
	HS–PS2–7( M A ). Construct a model to explain how ions dissolve in polar solvents (particularly water). Analyze and compare solubility and conductivity data to determine the extent to which different ionic species dissolve.
0005 Apply knowledge of physical and chemical properties and physical and chemical changes.	HS–PS1–3. Cite evidence to relate physical properties of substances at the bulk scale to spatial arrangements, movement, and strength of electrostatic forces among ions, small molecules, or regions of large molecules in the substances. Make arguments to account for how compositional and structural differences in molecules result in different types of intermolecular or intramolecular interactions.
0006 Apply knowledge of the types and characteristics of chemical reactions.	HS–PS1–10( M A ). Use an oxidation-reduction reaction model to predict products of reactions given the reactants, and to communicate the reaction models using a representation that shows electron transfer (redox). Use oxidation numbers to account for how electrons are redistributed in redox processes used in devices that generate electricity or systems that prevent corrosion.
	HS–PS1–2. Use the periodic table model to predict and design simple reactions that result in two main classes of binary compounds, ionic and molecular. Develop an explanation based on given observational data and the electronegativity model about the relative strengths of ionic or covalent bonds.
0007 Apply knowledge of the quantitative relationships expressed in chemical equations.	HS–PS1–7. Use mathematical representations and provide experimental evidence to support the claim that atoms, and therefore mass, are conserved during a chemical reaction. Use the mole concept and proportional relationships to evaluate the quantities (masses or moles) of specific reactants needed in order to obtain a specific amount of product.

MTEL 067 Test Objectives: III. MATTER AND ITS INTERACTIONS: SUBSTANCES, MIXTURES, AND SOLUTIONS	Curriculum Framework
0008 Apply knowledge of the mass relationships in chemical substances.	HS–PS1–7. Use mathematical representations and provide experimental evidence to support the claim that atoms, and therefore mass, are conserved during a chemical reaction. Use the mole concept and proportional relationships to evaluate the quantities (masses or moles) of specific reactants needed in order to obtain a specific amount of product.
0009 Analyze the properties of solutions.	HS–PS1–5. Construct an explanation based on kinetic molecular theory for why varying conditions influence the rate of a chemical reaction or a dissolving process. Design and test ways to slow down or accelerate rates of processes (chemical reactions or dissolving) by altering various conditions.
	HS–PS2–7( M A ). Construct a model to explain how ions dissolve in polar solvents (particularly water). Analyze and compare solubility and conductivity data to determine the extent to which different ionic species dissolve.
	HS–PS2–8( M A ). Use kinetic molecular theory to compare the strengths of electrostatic forces and the prevalence of interactions that occur between molecules in solids, liquids, and gases. Use the combined gas law to determine changes in pressure, volume, and temperature in gases.

MTEL 067 Test Objectives: IV. MOTION AND STABILITY: FORCES AND INTERACTIONS	Curriculum Framework
0010 Apply knowledge of the kinetic molecular theory, the nature of phase changes, and the gas laws.	HS–PS2–8( M A ). Use kinetic molecular theory to compare the strengths of electrostatic forces and the prevalence of interactions that occur between molecules in solids, liquids, and gases. Use the combined gas law to determine changes in pressure, volume, and temperature in gases.
0011 Apply knowledge of the connection between molecular structure and forces between particles.	HS–PS2–8( M A ). Use kinetic molecular theory to compare the strengths of electrostatic forces and the prevalence of interactions that occur between molecules in solids, liquids, and gases. Use the combined gas law to determine changes in pressure, volume, and temperature in gases.
	HS–PS2–7( M A ). Construct a model to explain how ions dissolve in polar solvents (particularly water). Analyze and compare solubility and conductivity data to determine the extent to which different ionic species dissolve.
0012 Apply knowledge of the principles of chemical equilibrium.	HS–PS1–6. Design ways to control the extent of a reaction at equilibrium (relative amount of products to reactants) by altering various conditions using Le Chatelier's principle. Make arguments based on kinetic molecular theory to account for how altering conditions would affect the forward and reverse rates of the reaction until a new equilibrium is established.

MTEL 067 Test Objectives: V. ENERGY IN CHEMICAL SYSTEMS	Curriculum Framework
0013 Apply knowledge of the factors that affect reaction rates and methods of measuring reaction rates.	HS–PS1–5. Construct an explanation based on kinetic molecular theory for why varying conditions influence the rate of a chemical reaction or a dissolving process. Design and test ways to slow down or accelerate rates of processes (chemical reactions or dissolving) by altering various conditions.
	HS–PS1–6. Design ways to control the extent of a reaction at equilibrium (relative amount of products to reactants) by altering various conditions using Le Chatelier's principle. Make arguments based on kinetic molecular theory to account for how altering conditions would affect the forward and reverse rates of the reaction until a new equilibrium is established.
0014 Apply knowledge of the principles of thermodynamics and calorimetry.	HS–PS1–4. Develop a model to illustrate the energy transferred during an exothermic or endothermic chemical reaction based on the bond energy difference between bonds broken (absorption of energy) and bonds formed (release of energy).
	HS–PS3–4b. Provide evidence from informational text or available data to illustrate that the transfer of energy during a chemical reaction in a closed system involves changes in energy dispersal (enthalpy change) and heat content (entropy change) while assuming the overall energy in the system is conserved.
0015 Apply knowledge of the energy relationships in chemical bonding and chemical reactions.	HS–PS1–4. Develop a model to illustrate the energy transferred during an exothermic or endothermic chemical reaction based on the bond energy difference between bonds broken (absorption of energy) and bonds formed (release of energy).
	HS–PS3–4b. Provide evidence from informational text or available data to illustrate that the transfer of energy during a chemical reaction in a closed system involves changes in energy dispersal (enthalpy change) and heat content (entropy change) while assuming the overall energy in the system is conserved.
0016 Apply knowledge of oxidation-reduction reactions to electrochemistry.	HS–PS1–10( M A ). Use an oxidation-reduction reaction model to predict products of reactions given the reactants, and to communicate the reaction models using a representation that shows electron transfer (redox). Use oxidation numbers to account for how electrons are redistributed in redox processes used in devices that generate electricity or systems that prevent corrosion.

MTEL 067 Test Objectives: VI. INTEGRATION OF KNOWLEDGE AND UNDERSTANDING	Curriculum Framework
0017 Prepare an organized, developed analysis of a key topic in chemistry related to periodic properties, properties of matter, and/or chemical bonding.	8.MS–PS1–1. Develop a model to describe that (a) atoms combine in a multitude of ways to produce pure substances which make up all of the living and nonliving things that we encounter, (b) atoms form molecules and compounds that range in size from two to thousands of atoms, and (c) mixtures are composed of different proportions of pure substances.
	HS–PS1–1. Use the periodic table as a model to predict the relative properties of main group elements, including ionization energy and relative sizes of atoms and ions, based on the patterns of electrons in the outermost energy level of each element. Use the patterns of valence electron configurations, core charge, and Coulomb's law to explain and predict general trends in ionization energies, relative sizes of atoms and ions, and reactivity of pure elements.
	HS–PS1–3. Cite evidence to relate physical properties of substances at the bulk scale to spatial arrangements, movement, and strength of electrostatic forces among ions, small molecules, or regions of large molecules in the substances. Make arguments to account for how compositional and structural differences in molecules result in different types of intermolecular or intramolecular interactions.
	HS–PS1–7. Use mathematical representations and provide experimental evidence to support the claim that atoms, and therefore mass, are conserved during a chemical reaction. Use the mole concept and proportional relationships to evaluate the quantities (masses or moles) of specific reactants needed in order to obtain a specific amount of product.
	HS–PS2–6. Communicate scientific and technical information about the molecular-level structures of polymers, ionic compounds, acids and bases, and metals to justify why these are useful in the functioning of designed materials.
	HS–PS2–7( M A ). Construct a model to explain how ions dissolve in polar solvents (particularly water). Analyze and compare solubility and conductivity data to determine the extent to which different ionic species dissolve.
	HS–PS2–8( M A ). Use kinetic molecular theory to compare the strengths of electrostatic forces and the prevalence of interactions that occur between molecules in solids, liquids, and gases. Use the combined gas law to determine changes in pressure, volume, and temperature in gases.
0018 Prepare an organized, developed analysis of a key topic in chemistry related to chemical reactions, mixtures and solutions, and/or energy that emphasizes the application of science and engineering practices in a classroom setting.	HS–PS1–2. Use the periodic table model to predict and design simple reactions that result in two main classes of binary compounds, ionic and molecular. Develop an explanation based on given observational data and the electronegativity model about the relative strengths of ionic or covalent bonds.
	HS–PS1–4. Develop a model to illustrate the energy transferred during an exothermic or endothermic chemical reaction based on the bond energy difference between bonds broken (absorption of energy) and bonds formed (release of energy).
	HS–PS1–5. Construct an explanation based on kinetic molecular theory for why varying conditions influence the rate of a chemical reaction or a dissolving process. Design and test ways to slow down or accelerate rates of processes (chemical reactions or dissolving) by altering various conditions.
	HS–PS1–6. Design ways to control the extent of a reaction at equilibrium (relative amount of products to reactants) by altering various conditions using Le Chatelier's principle. Make arguments based on kinetic molecular theory to account for how altering conditions would affect the forward and reverse rates of the reaction until a new equilibrium is established.
	HS–PS1–7. Use mathematical representations and provide experimental evidence to support the claim that atoms, and therefore mass, are conserved during a chemical reaction. Use the mole concept and proportional relationships to evaluate the quantities (masses or moles) of specific reactants needed in order to obtain a specific amount of product.
	HS–PS1–10( M A ). Use an oxidation-reduction reaction model to predict products of reactions given the reactants, and to communicate the reaction models using a representation that shows electron transfer (redox). Use oxidation numbers to account for how electrons are redistributed in redox processes used in devices that generate electricity or systems that prevent corrosion.
	HS–PS2–7( M A ). Construct a model to explain how ions dissolve in polar solvents (particularly water). Analyze and compare solubility and conductivity data to determine the extent to which different ionic species dissolve.
	HS–PS2–8( M A ). Use kinetic molecular theory to compare the strengths of electrostatic forces and the prevalence of interactions that occur between molecules in solids, liquids, and gases. Use the combined gas law to determine changes in pressure, volume, and temperature in gases.
	HS–PS3–4b. Provide evidence from informational text or available data to illustrate that the transfer of energy during a chemical reaction in a closed system involves changes in energy dispersal (enthalpy change) and heat content (entropy change) while assuming the overall energy in the system is conserved.

Chemistry

Committee participants	Number
Total	14
Public School Educators	13
Higher Education Faculty	1

Gender	Number
Female	5
Male	3
No response	6

Ethnicity	Number
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	1
Hispanic	1
White	11
Other	0
Unspecified	1

English as a Second Language

INTACT CLASSROOM SESSIONS 2013–2015
10/01/2013	Lincoln School (Brookline Public Schools)
10/05/2013	Simmons College Gordon College Salem State University
10/08/2013	Wheaton College
10/18/2013	University of Massachusetts - Boston
10/31/2013	Wheelock College
01/30/2014	Wheaton College
02/01/2014	Gordon College
02/04/2014	Elms College
02/05/2014	Salem State University
02/06/2014	University of Massachusetts - Amherst
02/11/2014	Lincoln School (Brookline Public Schools)
02/14/2014	Wheelock College
02/19/2014	Worcester State University
04/25/2015	Cambridge College
04/27/2015	Salem State University
05/02/2015	Simmons College Framingham State University
05/05/2015	Westfield State University
05/09/2015	Gordon College

STAND ALONE EVENTS AT MTEL OPERATIONAL TEST ADMINISTRATIONS

10/26/2013
03/01/2014
05/10/2014
07/12/2014
10/25/2014

Pilot Test Outcomes
Chemistry

Pilot testing was conducted at CBT centers from March 28, 2022, through April 22, 2022. The following information is a summary of the data collected during this specific time frame.

Multiple-Choice Items ( M C Q's ) and Short-Answer Items ( S A's )

Number of M C Q's Prepared	60
Number of M C Q's Piloted*	30
Number of M C Q's Identified for Further Review	10
Number of M C Q's Deleted Post Pilot Test Review	0

Open-Response Items ( O R I's )

Number of O R I's Developed	14
Number of O R I's Piloted*	3
Number of O R I's Identified for Further Review	0
Number of O R I's Deleted Post Pilot Test Review	0

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms until all items are piloted.

Chemistry

Number of Qualifying Score Participants: 14

Number of Participants Providing Ratings: 12 (86%)

Question Rating Scale: 1–5 1=not at all (well/confident/satisfied/successful), 5=very (well/confident/satisfied/successful)	Median Rating
How well do you feel you understood the training that you received for making your qualifying score judgments?	5
How confident are you of the qualifying score judgments you made?	5
How satisfied are you that you were able to complete the work you were asked to do in the time that was provided?	5
How successful were the coordination and logistics of the qualifying score conference?	5
Overall, how satisfied are you with the qualifying score process in which you participated?	5

Chemistry Content Validation Survey Population/Sample/Respondents Demographics (Public School Sample)

Key

Population: The number and percent of licensed and assigned educators in the content area, extracted from the database of full-time equivalent (FTE) teachers assigned to each teaching area for each school district in Massachusetts, as provided by the Massachusetts Department of Elementary and Secondary Education for use in drawing a random sample of educators.
Sample (with oversampling): The number and percent of licensed and assigned educators sampled to receive the survey. In order to obtain appropriate representation of minority groups (e.g., American Indian, Asian, Black, and Hispanic) in the survey results, oversampling of minority groups was conducted for those fields for which the entire population was not sampled. In these instances, the survey results were weighted appropriately to take the oversampling into account.

Chemistry	Population		Sample (with oversampling)
Chemistry	N	%	N	%
Total	914	100	300	100
Female	524	13.38	172	57.33
Male	390	9.96	128	42.66
American Indian or Alaska Native	0	0	0	0
Black (not of Hispanic origin)	10	0.25	3	1
Asian or Pacific Islander	41	1.04	13	4.33
Hispanic	17	0.43	6	2
White	841	21.48	276	92
Other	5	0.12	2	0.66

Chemistry Survey Return Rate by Field and Return Status: Public School Sample

Returned Surveys

Field	Number Sent	Number Not Distributed	Eligible		Ineligible		Total		Adjusted Return Rate
Field	Number Sent	Number Not Distributed	N	%	N	%	N	%	Adjusted Return Rate
066 Biology	296	17	62	20.90%	11	3.70%	73	24.70%	21.80%
067 Chemistry	293	7	58	19.80%	14	4.80%	72	24.60%	20.80%
068 Mathematics (Elementary)	298	8	72	24.20%	13	4.40%	85	28.50%	25.30%
069 Physics	298	27	59	19.80%	8	2.70%	67	22.50%	20.30%
070 Technology/Engineering	298	22	49	16.40%	17	5.70%	66	22.10%	17.40%
071 Digital Literacy and Computer Science	32	2	19	59.40%	0	0.00%	19	59.40%	59.40%
074 Earth and Space Science	149	3	21	14.10%	10	6.70%	31	20.80%	15.10%
ALL	1,664	86	340	20.40%	73	4.40%	413	24.80%	21.40%

Chemistry Survey Return Rate by Field and Return Status: Faculty Sample

Returned Surveys

Field	Number Sent	Total
Field	Number Sent	N	%
066 Biology	37	7	18.91%
067 Chemistry	36	5	13.88%
068 Mathematics (Elementary)	3	1	33.33%
069 Physics	28	2	7.14%
070 Technology/Engineering	1	0	0%
071 Digital Literacy and Computer Science	1	0	0%
074 Earth and Space Science	22	0	0%
ALL	128	15	11.71%

Demographic Summary Report: Public School Sample

Teacher Sample
Field: Chemistry

Question	Answer	Absolute Frequency	Adjusted Percent
Currently Teaching	Yes	58	100
Currently Teaching	No	0	0
Hold a valid Massachusetts teaching license	Yes	58	100
Hold a valid Massachusetts teaching license	No	0	0
What is the highest level of education you have attained?	Bachelor's degree	7	12
	Master's degree	43	74
	Doctoral degree	6	10
	Other	2	3
What is your ethnicity?	American Indian or Alaska Native	0	0
	Black (not of Hispanic origin)	0	0
	Asian or Pacific Islander	2	3
	Hispanic	2	3
	White (not of Hispanic origin)	52	90
	Other	2	3
What is your first (native) language?	English	53	91
	French	1	2
	Portuguese	1	2
	An Asian language	1	2
	Russian	0	0
	Spanish	1	2
	American Sign Language	0	0
	Italian	0	0
	Other	1	2
What is your gender?	Female	39	67
What is your gender?	Male	18	31
What grade level(s) have you taught in Massachusetts during this or the previous school year? (Select all that apply.)	Pre K to 6th	0	0
	5th to 8th	0	0
	5th to 12th	0	0
	8th to 12th	56	97
How many years of professional teaching experience do you have? (Count partial years as full years.)	1 to 3 years	2	3
	4 to 10 years	17	29
	11 years or more	39	67

Chemistry Demographic Summary Report: Faculty Sample

Faculty Sample
Field: Chemistry

Question	Answer	Absolute Frequency	Adjusted Percent
Teaching undergraduate or graduate arts and science or education courses in which prospective teachers may have been enrolled	Yes	5	100
	No	0	0
What is the highest level of education you have attained?	Bachelor's degree	0	0
	Master's degree	0	0
	Doctoral degree	5	100
	Other	0	0
What is your ethnicity?	American Indian or Alaska Native	0	0
	Black (not of Hispanic origin)	0	0
	Asian or Pacific Islander	1	20
	Hispanic	1	20
	White (not of Hispanic origin)	3	60
	Other	0	0
What is your first (native) language?	English	4	80
	Spanish	0	0
	Other	1	20
What is your gender?	Female	1	20
What is your gender?	Male	3	60
How many years have you taught as a faculty member at a higher education institution? (Count partial years as full years.)	1 to 3 years	0	0
	4 to 10 years	1	20
	11 years or more	4	80
In which higher education institution department do you currently hold your primary appointment?	Education department only	0	0
	Both the Education department and another department	0	0
	Arts and Sciences or Fine Arts department(s) only	5	100
Have you ever taught in the subject of the field and grade level indicated above in Massachusetts?	Yes	1	20
	No	4	80
How recently have you taught in the subject of the field and grade level indicated above in Massachusetts? (Count partial years as full years.)	1 to 3 years	0	0
	4 to 10 years	0	0
	11 years or more	1	20

Chemistry Objective Rating Report: Public School Sample

Number of Objectives: 18

Importance Ratings

Objective Number	N	Mean	SD	S E
1	57	4.32	0.83	0.11
2	54	4.52	0.64	0.09
3	56	3.82	0.94	0.13
4	55	4.44	0.63	0.09
5	57	4.39	0.7	0.09
6	57	4.39	0.67	0.09
7	56	4.63	0.49	0.07
8	56	4.34	0.69	0.09
9	55	4.02	0.83	0.11
10	56	4.32	0.66	0.09
11	56	4	0.71	0.1
12	57	3.74	0.84	0.11
13	57	3.63	0.77	0.1
14	57	3.58	0.84	0.11
15	57	3.79	0.77	0.1
16	57	3.33	0.91	0.12
17	56	4.04	0.97	0.13
18	56	4.16	0.87	0.12

Number of Respondents: 57

Response Distribution (in %)

1	2	3	4	5
0	1	10	16	30
0	0	4	18	32
0	4	18	18	16
0	0	4	23	28
0	0	7	21	29
0	0	6	23	28
0	0	0	21	35
0	0	7	23	26
0	1	15	21	18
0	0	6	26	24
0	0	14	28	14
1	2	17	28	9
0	2	25	22	8
1	3	22	24	7
0	1	21	24	11
2	6	25	19	5
0	4	13	16	23
0	2	11	19	24

Chemistry Objective Rating Report: Faculty Sample

Objective Rating Report: Ordered by Objective Number
Unweighted

Faculty Sample
Field: Chemistry

Number of Objectives: 18

Importance Ratings

Objective Number	N	Mean	SD	S E
1	5	4.2	0.84	0.37
2	5	4.4	0.89	0.4
3	5	3.4	0.89	0.4
4	5	4.6	0.55	0.24
5	5	4.4	0.55	0.24
6	5	4.4	0.89	0.4
7	5	4.6	0.55	0.24
8	5	4.6	0.89	0.4
9	5	3.6	0.89	0.4
10	5	3.6	0.89	0.4
11	5	4.2	1.1	0.49
12	5	3.6	0.89	0.4
13	5	3.4	1.14	0.51
14	5	3.6	0.89	0.4
15	5	3.6	0.55	0.24
16	5	3	0.71	0.32
17	5	3.4	0.55	0.24
18	5	3.6	0.55	0.24

Number of Respondents: 5

Response Distribution (in %)

2	3	4	5
0	1	2	2
0	1	1	3
1	1	3	0
0	0	2	3
0	0	3	2
0	1	1	3
0	0	2	3
0	1	0	4
0	3	1	1
0	3	1	1
0	2	0	3
0	3	1	1
1	2	1	1
0	3	1	1
0	2	3	0
1	3	1	0
0	3	2	0
0	2	3	0

Chemistry Descriptive Statement Rating Report: Public School Sample

Descriptive Statement Rating Report: Ordered by Objective Number
Unweighted

Number of Objectives: 18

Importance Ratings

Objective Number	N	Mean	SD	S E
1	58	4.03	0.75	0.1
2	58	4.29	0.75	0.1
3	58	3.6	1.04	0.14
4	57	4.26	0.86	0.11
5	57	4.23	0.68	0.09
6	57	4.12	0.76	0.1
7	58	4.38	0.72	0.09
8	57	4.19	0.77	0.1
9	58	4.05	0.94	0.12
10	57	4.02	0.81	0.11
11	57	3.98	0.88	0.12
12	56	4.16	0.71	0.09
13	58	3.84	0.87	0.11
14	57	4	0.94	0.13
15	57	3.98	0.81	0.11
16	58	3.83	0.09	0.12
17	57	3.58	1.05	0.14
18	56	3.98	0.96	0.13

Number of Respondents: 58

Response Distribution (in %)

1	2	3	4	5
0	1	12	29	16
0	1	7	24	26
2	7	14	24	11
0	3	6	21	27
0	0	8	28	21
0	1	10	27	19
0	1	5	23	29
0	1	9	25	22
1	3	9	24	21
0	2	12	26	17
0	4	10	26	17
0	1	7	30	18
0	2	21	19	16
1	3	10	24	19
0	3	10	29	15
0	2	23	16	17
2	6	18	19	12
1	2	14	19	20

Chemistry Descriptive Statement Rating Report: Faculty Sample

Descriptive Statement Rating Report: Ordered by Objective Number
Unweighted

Faculty Sample
Field: Chemistry

Number of Objectives: 18

Importance Ratings

Objective Number	N	Mean	SD	S E
1	5	4.2	0.84	0.37
2	5	4.4	0.89	0.4
3	5	3.6	1.14	0.51
4	5	4.4	0.89	0.4
5	5	4	1.22	0.55
6	5	4.4	0.89	0.4
7	5	3.8	1.1	0.49
8	5	4.4	0.89	0.4
9	5	4.2	0.84	0.37
10	5	3.4	0.89	0.4
11	5	4.2	0.84	0.37
12	5	3.6	0.55	0.24
13	5	3.8	0.84	0.37
14	5	3.8	0.84	0.37
15	5	3.8	0.84	0.37
16	5	4	1	0.45
17	5	3.4	0.89	0.4
18	5	4	0.71	0.32

Number of Respondents: 5

Response Distribution (in %)

2	3	4	5
0	1	2	2
0	1	1	3
1	1	2	1
0	1	1	3
1	0	2	2
0	1	1	3
1	0	3	1
0	1	1	3
0	1	2	2
1	1	3	0
0	1	2	2
0	2	3	0
0	2	2	1
0	2	2	1
0	2	2	1
0	2	1	2
1	1	3	0
0	1	3	1

Chemistry Composite Rating Report: Public School Sample

Composite Rating Report
Unweighted

Teacher Sample
Field: Chemistry

Importance Ratings

N	Mean	SD	S E
58	4.07	0.75	0.01

Number of Respondents: 58

Response Distribution (in %)

NR	1	2	3	4	5
0	0	1	11	29	17

Chemistry Composite Rating Report: Faculty Sample

Composite Rating Report
Unweighted

Education Faculty Sample
Field: Chemistry

Importance Ratings

N	Mean	SD	S E
5	3.8	0.45	0.2

Number of Respondents: 5

Response Distribution (in %)

NR	1	2	3	4	5
0	0	0	1	4	0

Digital Literacy and Computer Science Objective Ratings Summary

Objective	Number		Mean Objective Rating		Description of Objective
Objective	Educators	Faculty	Educators	Faculty	Description of Objective
1	57	5	4.32	4.2	Apply knowledge of atomic and subatomic structure and the principles of quantum theory.
2	54	5	4.52	4.4	Apply knowledge of periodic properties and the organization of the periodic table.
3	56	5	3.82	3.4	Apply knowledge of the nomenclature and structure of inorganic and organic compounds.
4	55	5	4.44	4.6	Apply knowledge of chemical bonding and the relationship between bond type and the properties of substances.
5	57	5	4.39	4.4	Apply knowledge of physical and chemical properties and physical and chemical changes.
6	57	5	4.39	4.4	Apply knowledge of the types and characteristics of chemical reactions.
7	56	5	4.63	4.6	Apply knowledge of the quantitative relationships expressed in chemical equations.
8	56	5	4.34	4.6	Apply knowledge of the mass relationships in chemical substances.
9	55	5	4.02	3.6	Analyze the properties of solutions.
10	56	5	4.32	3.6	Apply knowledge of the kinetic molecular theory, the nature of phase changes, and the gas laws.
11	56	5	4	4.2	Apply knowledge of the connection between molecular structure and forces between particles.
12	57	5	3.74	3.6	Apply knowledge of the principles of chemical equilibrium.
13	57	5	3.63	3.4	Apply knowledge of the factors that affect reaction rates and methods of measuring reaction rates.
14	57	5	3.58	3.6	Apply knowledge of the principles of thermodynamics and calorimetry.
15	57	5	3.79	3.6	Apply knowledge of the energy relationships in chemical bonding and chemical reactions.
16	57	5	3.33	3	Apply knowledge of oxidation-reduction reactions to electrochemistry.
17	56	5	4.04	3.4	Prepare an organized, developed analysis of a key topic in chemistry related to Matter and Its Interactions: Periodic Properties or Motion and Stability: Forces and Interactions.
18	56	5	4.16	3.6	Prepare an organized, developed analysis of a key topic in chemistry related to Matter and Its Interactions: Chemical Structure and Reactions; Matter and Its Interactions: Substances, Mixtures, and Solutions; or Energy in Chemical Systems that emphasizes the application of science and engineering practices in a classroom setting.
Overall	57	5	4.07	3.8

Mathematics (Elementary)

Committee participants	Number
Total	35
Public School Educators	31
Higher Education Faculty	4

Gender	Number
Female	30
Male	3
No response	2

Ethnicity	Number
American Indian or Alaska Native	0
Black	2
Asian or Pacific Islander	1
Hispanic	3
White	22
Other	2
Unspecified	5

Mathematics (Elementary)

Bias Review Committee Composition

Committee participants	Number
Total	10
Public School Educators	10
Higher Education Faculty	0

Gender	Number
Female	8
Male	2
No response	0

Ethnicity	Number
American Indian or Alaska Native	0
Black (not of Hispanic origin)	6
Asian or Pacific Islander	2
Hispanic	0
White	1
Other	1
Unspecified	0

Alignment between the MTEL® Test Objectives for Mathematics (Elementary) (068) and the 2017 Massachusetts Curriculum Framework for Mathematics (Elementary)

MTEL 068 Test Objectives:
I. Number Systems and Operations

Curriculum Framework

0001 Apply number theory, structures of numeration systems, and arithmetic properties to the real number system.

1. O A .A Represent and solve problems involving addition and subtraction.

1. O A .B Understand and apply properties of operations and the relationship between addition and subtraction.

1. O A .C Add and subtract within 20.

1. O A .D Work with addition and subtraction equations.

1.NBT.A Extend the counting sequence.

1.NBT.B Understand place value.

1.NBT.C Use place value understanding and properties of operations to add and subtract.

2. O A .A Represent and solve problems involving addition and subtraction.

2. O A .B Add and subtract within 20.

2. O A .C Work with equal groups of objects to gain foundations for multiplication.

2.NBT.A Understand place value.

2.NBT.B Use place value understanding and properties of operations to add and subtract.

3. O A .A Represent and solve problems involving multiplication and division.

3. O A .B Understand properties of multiplication and the relationship between multiplication and division.

3. O A .C Multiply and divide within 100.

3. O A .D Solve problems involving the four operations, and identify and explain patterns in arithmetic.

3.NBT.A Use place value understanding and properties of operations to perform multi-digit arithmetic.

4. O A .B Gain familiarity with factors and multiples.

4.NBT.A Generalize place value understanding for multi-digit whole numbers less than or equal to 1,000,000.

4.NBT.B Use place value understanding and properties of operations to perform multi-digit arithmetic on whole numbers less than or equal to 1,000,000.

5.NBT.A Understand the place value system.

6.NS.B Compute fluently with multi-digit numbers and find common factors and multiples.

6.NS.C Apply and extend previous understandings of numbers to the system of rational numbers.

8.NS.A Know that there are numbers that are not rational, and approximate them by rational numbers.

0002 Demonstrate knowledge of the principles and operations related to whole numbers, decimals, and percents.

4. O A .A Use the four operations with whole numbers to solve problems.

4. O A .C Generate and analyze patterns.

5.NBT.B Perform operations with multi-digit whole numbers and with decimals to hundredths.

6.NS.A Apply and extend previous understandings of multiplication and division to divide fractions by fractions.

6.NS.C Apply and extend previous understandings of numbers to the system of rational numbers.

0003 Understand principles and operations related to integers and fractions.

3.NF.A Develop understanding of fractions as numbers for fractions with denominators 2, 3, 4, 6, and 8.

4.NF.A Extend understanding of fraction equivalence and ordering for fractions with denominators 2, 3, 4, 5, 6, 8, 10, 12, and 100.

4.NF.B Build fractions from unit fractions by applying and extending previous understandings of operations on whole numbers for fractions with denominators 2, 3, 4, 5, 6, 8, 10, 12, and 100.

4.NF.C Understand decimal notation for fractions, and compare decimal fractions.

5.NF.A Extend understanding of fraction equivalence and ordering for fractions with denominators 2, 3, 4, 5, 6, 8, 10, 12, and 100.

5.NF.B Apply and extend previous understandings of multiplication and division to multiply and divide fractions.

6.NS.A Apply and extend previous understandings of multiplication and division to divide fractions by fractions.

7.NS.A Apply and extend previous understandings of operations with fractions to add, subtract, multiply, and divide rational numbers.

MTEL 068 Test Objectives: II. Algebraic Thinking and Relations	Curriculum Framework
0004 Analyze patterns and the properties of functions and relations.	8.F.A Define, evaluate, and compare functions. 8.F.B Use functions to model relationships between quantities.
0005 Understand how to manipulate and simplify algebraic expressions and translate problems into algebraic expressions, equations, and inequalities.	5. O A .A Write and interpret numerical expressions. 5. O A .B Analyze patterns and relationships. 6. E E .A Apply and extend previous understandings of arithmetic to algebraic expressions. 6. E E .B Reason about and solve one-variable equations and inequalities. 6. E E .C Represent and analyze quantitative relationships between dependent and independent variables. 7. E E .A Use properties of operations to generate equivalent expressions. 7. E E .B Solve real-life and mathematical problems using numerical and algebraic expressions and equations. 8. E E .B Understand the connections between proportional relationships, lines, and linear equations. 8. E E .C Analyze and solve linear equations and pairs of simultaneous linear equations.
0006 Understand properties and applications of ratios and proportions.	6.RP.A Understand ratio and rate concepts and use ratio reasoning to solve problems. 7.RP.A Analyze proportional relationships and use them to solve real-world and mathematical problems. 8. E E .B Understand the connections between proportional relationships, lines, and linear equations.
0007 Analyze properties and applications of linear relations and functions.	8.F.A Define, evaluate, and compare functions. 8.F.B Use functions to model relationships between quantities.

MTEL 068 Test Objectives:
III. Geometry

Curriculum Framework

0008 Analyze the characteristics and properties of two- and three-dimensional figures.

1.G.A Reason with shapes and their attributes.

2.G.A Reason with shapes and their attributes.

3.G.A Reason with shapes and their attributes.

3.MD.C Geometric measurement: understand concepts of area and relate area to multiplication and to addition.

3.MD.D Geometric measurement: recognize perimeter as an attribute of plane figures and distinguish between linear and area measures.

4.MD.A Solve problems involving measurement and conversion of measurements from a larger unit to a smaller unit.

4.MD.C Geometric measurement: Understand concepts of angle and measure angles.

4.G.A Draw and identify lines and angles, and classify shapes by properties of their lines and angles.

5.MD.C Geometric measurement: Understand concepts of volume and relate volume to multiplication and to addition.

5.G.B Classify two-dimensional figures into categories based on their properties.

6.G.A Solve real-world and mathematical problems involving area, surface area, and volume.

7.G.A Draw, construct and describe geometrical figures and describe the relationships between them.

7.G.B Solve real-life and mathematical problems involving angle measure, area, surface area, and volume.

8.G.B Understand and apply the Pythagorean Theorem.

8.G.C Solve real-world and mathematical problems involving volume of cylinders, cones, and spheres.

0009 Understand the principles and properties of coordinate and transformational geometry.

5.G.A Graph points on the coordinate plane to solve real-world and mathematical problems.

6.G.A Solve real-world and mathematical problems involving area, surface area, and volume.

8.G.A Understand congruence and similarity using physical models, transparencies, or geometry software.

MTEL 068 Test Objectives:
IV. Measurement, Data, and Probability

Curriculum Framework

0010 Understand principles, concepts, and procedures related to measurement.

1.MD.A Measure lengths indirectly and by iterating length units.

1.MD.C Represent and interpret data.

2.MD.A Measure and estimate lengths in standard units.

2.MD.B Relate addition and subtraction to length.

2.MD.D Represent and interpret data.

3.MD.A Solve problems involving measurement and estimation of intervals of time, liquid volumes, and masses of objects.

3.MD.B Represent and interpret data.

3.MD.C Geometric measurement: understand concepts of area and relate area to multiplication and to addition.

4.MD.A Solve problems involving measurement and conversion of measurements from a larger unit to a smaller unit.

4.MD.B Represent and interpret data.

4.MD.C Geometric measurement: Understand concepts of angle and measure angles.

5.MD.A Convert like measurement units within a given measurement system.

5.MD.B Represent and interpret data.

0011 Understand the fundamental principles of probability.

7.SP.C Investigate chance processes and develop, use, and evaluate probability models.

0012 Understand descriptive statistics and the methods used in collecting, organizing, reporting, and analyzing data.

3.MD.B Represent and interpret data.

4.MD.B Represent and interpret data.

6.SP.A Develop understanding of statistical variability.

6.SP.B Summarize and describe distributions.

7.SP.A Use random sampling to draw inferences about a population.

7.SP.B Draw informal comparative inferences about two populations.

8.SP.A Investigate patterns of association in bivariate data.

MTEL 068 Test Objectives: V. Integration of Knowledge and Understanding	Curriculum Framework
0013 Prepare an organized, developed analysis on a topic related to one or more of the following: number systems; operations; relations; algebraic thinking.	The Number System Numbers and Operations Ratios and Proportional Relationships Expressions and Equations Operations and Algebraic Thinking
0014 Prepare an organized, developed analysis on a topic related to one or more of the following: measurement; data; geometry.	Geometry Measurement and Data Statistics and Probability

Mathematics (Elementary)

Committee participants	Number
Total	9
Public School Educators	9
Higher Education Faculty	0

Gender	Number
Female	7
Male	1
No response	1

Ethnicity	Number
American Indian or Alaska Native	0
Black (not of Hispanic origin)	1
Asian or Pacific Islander	0
Hispanic	0
White	6
Other	1
Unspecified	1

English as a Second Language

INTACT CLASSROOM SESSIONS 2013–2015
10/01/2013	Lincoln School (Brookline Public Schools)
10/05/2013	Simmons College Gordon College Salem State University
10/08/2013	Wheaton College
10/18/2013	University of Massachusetts - Boston
10/31/2013	Wheelock College
01/30/2014	Wheaton College
02/01/2014	Gordon College
02/04/2014	Elms College
02/05/2014	Salem State University
02/06/2014	University of Massachusetts - Amherst
02/11/2014	Lincoln School (Brookline Public Schools)
02/14/2014	Wheelock College
02/19/2014	Worcester State University
04/25/2015	Cambridge College
04/27/2015	Salem State University
05/02/2015	Simmons College Framingham State University
05/05/2015	Westfield State University
05/09/2015	Gordon College

STAND ALONE EVENTS AT MTEL OPERATIONAL TEST ADMINISTRATIONS

10/26/2013
03/01/2014
05/10/2014
07/12/2014
10/25/2014

Pilot Test Outcomes
Mathematics (Elementary)

Pilot testing was conducted at CBT centers from March 28, 2022, through April 22, 2022. The following information is a summary of the data collected during this specific time frame.

Multiple-Choice Items ( M C Q's ) and Short-Answer Items ( S A's )

Number of M C Q's Prepared	119
Number of M C Q's Piloted*	54
Number of M C Q's Identified for Further Review	21
Number of M C Q's Deleted Post Pilot Test Review	0

Open-Response Items ( O R I's )

Number of O R I's Developed	19
Number of O R I's Piloted*	16
Number of O R I's Identified for Further Review	0
Number of O R I's Deleted Post Pilot Test Review	0

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms, until all items are piloted.

Mathematics (Elementary)

Number of Qualifying Score Participants: 9

Number of Participants Providing Ratings: 9 (100%)

Question Rating Scale: 1–5 1=not at all (well/confident/satisfied/successful), 5=very (well/confident/satisfied/successful)	Median Rating
How well do you feel you understood the training that you received for making your qualifying score judgments?	5
How confident are you of the qualifying score judgments you made?	5
How satisfied are you that you were able to complete the work you were asked to do in the time that was provided?	4
How successful were the coordination and logistics of the qualifying score conference?	4
Overall, how satisfied are you with the qualifying score process in which you participated?	5

Mathematics (Elementary) Content Validation Survey Population/Sample/Respondents Demographics (Public School Sample)

Key

Population: The number and percent of licensed and assigned educators in the content area, extracted from the database of full-time equivalent (FTE) teachers assigned to each teaching area for each school district in Massachusetts, as provided by the Massachusetts Department of Elementary and Secondary Education for use in drawing a random sample of educators.
Sample (with oversampling): The number and percent of licensed and assigned educators sampled to receive the survey. In order to obtain appropriate representation of minority groups (e.g., American Indian, Asian, Black, and Hispanic) in the survey results, oversampling of minority groups was conducted for those fields for which the entire population was not sampled. In these instances, the survey results were weighted appropriately to take the oversampling into account.

Mathematics (Elementary)	Population		Sample (with oversampling)
Mathematics (Elementary)	N	%	N	%
Total	492	100	300	100
Female	406	82.52	247	82.33
Male	86	17.47	53	17.66
American Indian or Alaska Native	0	0	0	0
Black (not of Hispanic origin)	12	2.43	7	2.33
Asian or Pacific Islander	10	2.03	6	2
Hispanic	7	1.42	4	1.33
White	462	93.9	282	94
Other	1	0.2	1	0.33

Mathematics (Elementary) Survey Return Rate by Field and Return Status: Public School Sample

Returned Surveys

Field	Number Sent	Number Not Distributed	Eligible		Ineligible		Total		Adjusted Return Rate
Field	Number Sent	Number Not Distributed	N	%	N	%	N	%	Adjusted Return Rate
066 Biology	296	17	62	20.90%	11	3.70%	73	24.70%	21.80%
067 Chemistry	293	7	58	19.80%	14	4.80%	72	24.60%	20.80%
068 Mathematics (Elementary)	298	8	72	24.20%	13	4.40%	85	28.50%	25.30%
069 Physics	298	27	59	19.80%	8	2.70%	67	22.50%	20.30%
070 Technology/Engineering	298	22	49	16.40%	17	5.70%	66	22.10%	17.40%
071 Digital Literacy and Computer Science	32	2	19	59.40%	0	0.00%	19	59.40%	59.40%
074 Earth and Space Science	149	3	21	14.10%	10	6.70%	31	20.80%	15.10%
ALL	1,664	86	340	20.40%	73	4.40%	413	24.80%	21.40%

Mathematics (Elementary) Survey Return Rate by Field and Return Status: Faculty Sample

Returned Surveys

Field	Number Sent	Total
Field	Number Sent	N	%
066 Biology	37	7	18.91%
067 Chemistry	36	5	13.88%
068 Mathematics (Elementary)	3	1	33.33%
069 Physics	28	2	7.14%
070 Technology/Engineering	1	0	0%
071 Digital Literacy and Computer Science	1	0	0%
074 Earth and Space Science	22	0	0%
ALL	128	15	11.71%

Demographic Summary Report: Public School Sample

Teacher Sample
Field: Mathematics (Elementary)

Question	Answer	Absolute Frequency	Adjusted Percent
Currently Teaching	Yes	72	100
Currently Teaching	No	0	0
Hold a valid Massachusetts teaching license	Yes	72	100
Hold a valid Massachusetts teaching license	No	0	0
What is the highest level of education you have attained?	Bachelor's degree	6	8
	Master's degree	62	86
	Doctoral degree	1	1
	Other	3	4
What is your ethnicity?	American Indian or Alaska Native	0	0
	Black (not of Hispanic origin)	1	1
	Asian or Pacific Islander	0	0
	Hispanic	0	0
	White (not of Hispanic origin)	71	99
	Other	0	0
What is your first (native) language?	English	70	97
	French	0	0
	Portuguese	2	3
	An Asian language	0	0
	Russian	0	0
	Spanish	0	0
	American Sign Language	0	0
	Italian	0	0
	Other	0	0
What is your gender?	Female	68	94
What is your gender?	Male	4	6
What grade level(s) have you taught in Massachusetts during this or the previous school year? (Select all that apply.)	Pre K to 6th	0	0
	5th to 8th	0	0
	5th to 12th	3	5
	8th to 12th	56	90
How many years of professional teaching experience do you have? (Count partial years as full years.)	1 to 3 years	3	5
	4 to 10 years	11	18
	11 years or more	47	76

Mathematics (Elementary) Demographic Summary Report: Faculty Sample

Faculty Sample
Field: Mathematics (Elementary)

Question	Answer	Absolute Frequency	Adjusted Percent
Teaching undergraduate or graduate arts and science or education courses in which prospective teachers may have been enrolled	Yes	1	100
	No	0	0
What is the highest level of education you have attained?	Bachelor's degree	0	0
	Master's degree	0	0
	Doctoral degree	1	100
	Other	0	0
What is your ethnicity?	American Indian or Alaska Native	0	0
	Black (not of Hispanic origin)	0	0
	Asian or Pacific Islander	0	0
	Hispanic	0	0
	White (not of Hispanic origin)	1	100
	Other	0	0
What is your first (native) language?	English	1	100
	Spanish	0	0
	Other	0	0
What is your gender?	Female	1	100
What is your gender?	Male	0	0
How many years have you taught as a faculty member at a higher education institution? (Count partial years as full years.)	1 to 3 years	0	0
	4 to 10 years	1	100
	11 years or more	0	0
In which higher education institution department do you currently hold your primary appointment?	Education department only	1	100
	Both the Education department and another department	0	0
	Arts and Sciences or Fine Arts department(s) only	0	0
Have you ever taught in the subject of the field and grade level indicated above in Massachusetts?	Yes	0	0
	No	1	100
How recently have you taught in the subject of the field and grade level indicated above in Massachusetts? (Count partial years as full years.)	1 to 3 years	0	0
	4 to 10 years	0	0
	11 years or more	0	0

Mathematics (Elementary) Objective Rating Report: Public School Sample

Number of Objectives: 14

Importance Ratings

Objective Number	N	Mean	SD	S E
1	72	4.72	0.51	0.06
2	72	4.69	0.55	0.06
3	71	4.61	0.62	0.07
4	72	3.89	0.91	0.11
5	72	4.24	0.72	0.09
6	72	4.06	0.93	0.11
7	72	3.42	0.93	0.11
8	72	4.21	0.79	0.09
9	71	3.48	0.88	0.1
10	72	4.38	0.68	0.08
11	72	3.26	0.79	0.09
12	72	4	0.84	1
13	71	4.2	0.87	0.1
14	71	4.2	0.84	0.1

Number of Respondents: 72

Response Distribution (in %)

1	2	3	4	5
0	0	2	16	54
0	0	3	16	53
0	0	5	18	48
0	4	22	24	22
0	1	9	34	28
0	4	17	22	29
1	9	31	21	10
0	2	10	31	29
1	6	31	24	9
0	0	8	29	35
0	11	35	22	4
0	1	22	25	24
0	1	18	18	34
0	1	16	22	32

Mathematics (Elementary) Objective Rating Report: Faculty Sample

Objective Rating Report: Ordered by Objective Number
Unweighted

Faculty Sample
Field: Mathematics (Elementary)

Number of Objectives: 14

Importance Ratings

Objective Number	N	Mean	SD	S E
1	1	5	N/A	N/A
2	1	5	N/A	N/A
3	1	5	N/A	N/A
4	1	4	N/A	N/A
5	1	5	N/A	N/A
6	1	5	N/A	N/A
7	1	4	N/A	N/A
8	1	5	N/A	N/A
9	1	5	N/A	N/A
10	1	5	N/A	N/A
11	1	4	N/A	N/A
12	1	4	N/A	N/A
13	1	5	N/A	N/A
14	1	5	N/A	N/A

Number of Respondents: 5

Response Distribution (in %)

NR	1	2	3	4	5
0	0	0	0	0	1
0	0	0	0	0	1
0	0	0	0	0	1
0	0	0	0	1	0
0	0	0	0	0	1
0	0	0	0	0	1
0	0	0	0	1	0
0	0	0	0	0	1
0	0	0	0	0	1
0	0	0	0	0	1
0	0	0	0	1	0
0	0	0	0	1	0
0	0	0	0	0	1
0	0	0	0	0	1

Mathematics (Elementary) Descriptive Statement Rating Report: Public School Sample

Descriptive Statement Rating Report: Ordered by Objective Number
Unweighted

Number of Objectives: 14

Importance Ratings

Objective Number	N	Mean	SD	S E
1	72	4.54	0.69	0.08
2	70	4.69	0.47	0.06
3	72	4.6	0.55	0.06
4	72	4.18	0.84	0.1
5	71	4.41	0.79	0.09
6	72	4.33	0.73	0.09
7	72	4.1	0.79	0.09
8	72	4.49	0.63	0.07
9	71	4.14	0.91	0.11
10	72	4.51	0.58	0.07
11	72	4.1	0.82	0.1
12	71	4.31	0.77	0.09
13	72	4.1	0.86	0.1
14	72	4.35	0.73	0.09

Number of Respondents: 72

Response Distribution (in %)

1	2	3	4	5
0	2	2	23	45
0	0	0	22	48
0	0	2	25	45
0	2	14	25	31
0	3	4	25	39
0	1	8	29	34
0	3	10	36	23
0	0	5	27	40
1	4	7	31	28
0	0	3	29	40
0	2	15	29	26
0	1	10	26	34
0	4	11	31	26
0	2	5	31	34

Mathematics (Elementary) Descriptive Statement Rating Report: Faculty Sample

Descriptive Statement Rating Report: Ordered by Objective Number
Unweighted

Faculty Sample
Field: Mathematics (Elementary)

Number of Objectives: 14

Importance Ratings

Objective Number	N	Mean	SD	S E
1	1	5	N/A	N/A
2	1	5	N/A	N/A
3	1	5	N/A	N/A
4	1	5	N/A	N/A
5	1	5	N/A	N/A
6	1	5	N/A	N/A
7	1	5	N/A	N/A
8	1	5	N/A	N/A
9	1	5	N/A	N/A
10	1	5	N/A	N/A
11	1	5	N/A	N/A
12	1	5	N/A	N/A
13	1	5	N/A	N/A
14	1	5	N/A	N/A

Number of Respondents: 1

Response Distribution (in %)

NR	1	2	3	4	5
0	0	0	0	0	1
0	0	0	0	0	1
0	0	0	0	0	1
0	0	0	0	0	1
0	0	0	0	0	1
0	0	0	0	0	1
0	0	0	0	0	1
0	0	0	0	0	1
0	0	0	0	0	1
0	0	0	0	0	1
0	0	0	0	0	1
0	0	0	0	0	1
0	0	0	0	0	1
0	0	0	0	0	1

Mathematics (Elementary) Composite Rating Report: Public School Sample

Composite Rating Report
Unweighted

Teacher Sample
Field: Mathematics (Elementary)

Importance Ratings

N	Mean	SD	S E
72	4.46	0.63	0.7

Number of Respondents: 72

Response Distribution (in %)

NR	1	2	3	4	5
0	0	1	2	32	37

Mathematics (Elementary) Composite Rating Report: Faculty Sample

Composite Rating Report
Unweighted

Education Faculty Sample
Field: Mathematics (Elementary)

Importance Ratings

N	Mean	SD	S E
1	5	N/A	N/A

Number of Respondents: 1

Response Distribution (in %)

NR	1	2	3	4	5
0	0	0	0	0	1

Mathematics (Elementary) Objective Ratings Summary

Objective	Number		Mean Objective Rating		Description of Objective
Objective	Educators	Faculty	Educators	Faculty	Description of Objective
1	72	1	4.72	5	Apply knowledge of connections between engineering, technology, mathematics, and natural sciences.
2	72	1	4.69	5	Apply knowledge of engineering design in developing technological solutions to problems within given specifications.
3	71	1	4.61	5	Apply knowledge of technology and engineering in social contexts.
4	72	1	3.89	4	Apply knowledge of the selection and safe use of appropriate materials, tools, equipment, and machines in technology and engineering.
5	72	1	4.24	5	Apply knowledge of manufacturing processes used to create products that meet stated requirements.
6	72	1	4.06	5	Apply knowledge of processes used to communicate data and information.
7	72	1	3.42	4	Apply knowledge of design factors, material selection, and constraints in building structures.
8	72	1	4.21	5	Analyze the effects of forces in construction technology.
9	71	1	3.48	5	Apply knowledge of the principles and characteristics of transportation technology.
10	72	1	4.38	5	Apply knowledge of fluid systems and their role in technological systems.
11	72	1	3.26	4	Apply knowledge of thermal systems and their role in technological systems.
12	72	1	4	4	Apply knowledge of electrical principles and components and their roles in technological systems.
13	71	1	4.2	5	Apply knowledge of basic principles of energy, work, and power and their relationship to mechanical systems.
14	71	1	4.2	5	Prepare an organized, developed analysis on a topic related to instruction in engineering design and manufacturing using materials and tools.
Overall	72	1	4.1	4.71

Digital Literacy and Computer Science

Committee participants	Number
Total	27
Public School Educators	25
Higher Education Faculty	2

Gender	Number
Female	18
Male	7
No response	2

Ethnicity	Number
American Indian or Alaska Native	0
Black	0
Asian or Pacific Islander	0
Hispanic	0
White	24
Other	1
Unspecified	2

Digital Literacy and Computer Science

Bias Review Committee Composition

Committee participants	Number
Total	6
Public School Educators	4
Higher Education Faculty	2

Gender	Number
Female	5
Male	1
No response	0

Ethnicity	Number
American Indian or Alaska Native	0
Black (not of Hispanic origin)	3
Asian or Pacific Islander	0
Hispanic	1
White	2
Other	0
Unspecified	0

Alignment between the MTEL® Test Objectives for Digital Literacy and Computer Science (071) and the 2017 Massachusetts Curriculum Framework for Digital Literacy and Computer Science

MTEL 071 Test Objectives:
I. COMPUTING AND SOCIETY

Curriculum Framework

0001 Apply knowledge of safety and security concepts and appropriate digital interactions.

K–2. C A S .a Safety and Security

3–5. C A S .a Safety and Security

6–8. C A S .a Safety and Security

9–12. C A S .a Safety and Security

0002 Apply knowledge of the responsible, ethical, and legal use of technology resources and digital content.

K–2. C A S .b Ethics and Laws

3–5. C A S .b Ethics and Laws

6–8. C A S .b Ethics and Laws

9–12. C A S .b Ethics and Laws

0003 Apply knowledge of the impacts of technology on society.

6–8. C A S .c Interpersonal and Societal Impact

MTEL 071 Test Objectives:
II. DIGITAL TOOLS AND COLLABORATION

Curriculum Framework

0004 Apply knowledge of the use of digital tools and resources to create content, communicate, and collaborate.

K–2.DTC.a Digital Tools

3–5.DTC.a Digital Tools

6–8.DTC.a Digital Tools

9–12.DTC.a Digital Tools

K–2.DTC.b Collaboration and Communication

3–5.DTC.b Collaboration and Communication

6–8.DTC.b Collaboration and Communication

9–12.DTC.b Collaboration and Communication

0005 Apply knowledge of digital tools and techniques for conducting research.

K–2.DTC.c Research

3–5.DTC.c Research

6–8.DTC.c Research

9–12.DTC.c Research

MTEL 071 Test Objectives:
III. COMPUTING SYSTEMS

Curriculum Framework

0006 Apply knowledge of characteristics, functions, and uses of computing devices and components.

K–2. C S .a Computing devices

3–5. C S .a Computing devices

6–8. C S .a Computing devices

9–12. C S .a Computing devices

K–2. C S .b Human and Computer Partnerships

3–5. C S .b Human and Computer Partnerships

6–8. C S .b Human and Computer Partnerships

9–12. C S .b Human and Computer Partnerships

0007 Apply knowledge of characteristics, functions, and uses of networks and services.

K–2. C S .c Networks

3–5. C S .c Networks

6–8. C S .c Networks

9–12. C S .c Networks

MTEL 071 Test Objectives: IV. COMPUTATIONAL THINKING	Curriculum Framework
0008 Apply concepts related to abstraction and algorithms.	K–2.CT.a Abstraction 3–5.CT.a Abstraction 6–8.CT.a Abstraction 9–12.CT.a Abstraction
	K–2.CT.b Algorithms 3–5.CT.b Algorithms 6–8.CT.b Algorithms 9–12.CT.b Algorithms
0009 Apply concepts related to data representation, modeling, and simulation.	K–2.CT.c Data 3–5.CT.c Data 6–8.CT.c Data 9–12.CT.c Data
	K–2.CT.e Modeling and Simulation 3–5.CT.e Modeling and Simulation 6–8.CT.e Modeling and Simulation 9–12.CT.e Modeling and Simulation
0010 Apply concepts related to computer programming.	K–2.CT.d Programming and Development 3–5.CT.d Programming and Development 6–8.CT.d Programming and Development 9–12.CT.d Programming and Development
0011 Apply concepts related to program development and testing.	K–2.CT.d Programming and Development 3–5.CT.d Programming and Development 6–8.CT.d Programming and Development 9–12.CT.d Programming and Development

MTEL 071 Test Objectives:
V. INTEGRATION OF KNOWLEDGE AND UNDERSTANDING

Curriculum Framework

0012 Prepare an organized, developed analysis on a topic related to the use of digital tools to create an artifact.

K–2. C A S .a Safety and Security

3–5. C A S .a Safety and Security

6–8. C A S .a Safety and Security

9–12. C A S .a Safety and Security

K–2. C A S .b Ethics and Laws

3–5. C A S .b Ethics and Laws

6–8. C A S .b Ethics and Laws

9–12. C A S .b Ethics and Laws

K–2.DTC.a Digital Tools

3–5.DTC.a Digital Tools

6–8.DTC.a Digital Tools

9–12.DTC.a Digital Tools

K–2.DTC.b Collaboration and Communication

3–5.DTC.b Collaboration and Communication

6–8.DTC.b Collaboration and Communication

9–12.DTC.b Collaboration and Communication

0013 Prepare an organized, developed analysis on a topic related to computer science.

K–2.CT.a Abstraction

3–5.CT.a Abstraction

6–8.CT.a Abstraction

9–12.CT.a Abstraction

K–2.CT.b Algorithms

3–5.CT.b Algorithms

6–8.CT.b Algorithms

9–12.CT.b Algorithms

K–2.CT.c Data

3–5.CT.c Data

6–8.CT.c Data

9–12.CT.c Data

K–2.CT.d Programming and Development

3–5.CT.d Programming and Development

6–8.CT.d Programming and Development

9–12.CT.d Programming and Development

K–2.CT.e Modeling and Simulation

3–5.CT.e Modeling and Simulation

6–8.CT.e Modeling and Simulation

9–12.CT.e Modeling and Simulation

Digital Literacy and Computer Science

Committee participants	Number
Total	9
Public School Educators	9
Higher Education Faculty	0

Gender	Number
Female	6
Male	2
No response	1

Ethnicity	Number
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	1
Hispanic	0
White	6
Other	1
Unspecified	1

English as a Second Language

INTACT CLASSROOM SESSIONS 2013–2015
10/01/2013	Lincoln School (Brookline Public Schools)
10/05/2013	Simmons College Gordon College Salem State University
10/08/2013	Wheaton College
10/18/2013	University of Massachusetts - Boston
10/31/2013	Wheelock College
01/30/2014	Wheaton College
02/01/2014	Gordon College
02/04/2014	Elms College
02/05/2014	Salem State University
02/06/2014	University of Massachusetts - Amherst
02/11/2014	Lincoln School (Brookline Public Schools)
02/14/2014	Wheelock College
02/19/2014	Worcester State University
04/25/2015	Cambridge College
04/27/2015	Salem State University
05/02/2015	Simmons College Framingham State University
05/05/2015	Westfield State University
05/09/2015	Gordon College

STAND ALONE EVENTS AT MTEL OPERATIONAL TEST ADMINISTRATIONS

10/26/2013
03/01/2014
05/10/2014
07/12/2014
10/25/2014

Digital Literacy and Computer Science

Number of Qualifying Score Participants: 9

Number of Participants Providing Ratings: 9 (100%)

Question Rating Scale: 1–5 1=not at all (well/confident/satisfied/successful), 5=very (well/confident/satisfied/successful)	Median Rating
How well do you feel you understood the training that you received for making your qualifying score judgments?	5
How confident are you of the qualifying score judgments you made?	4
How satisfied are you that you were able to complete the work you were asked to do in the time that was provided?	5
How successful were the coordination and logistics of the qualifying score conference?	4
Overall, how satisfied are you with the qualifying score process in which you participated?	4

Digital Literacy and Computer Science Content Validation Survey Population/Sample/Respondents Demographics (Public School Sample)

Key

Population: The number and percent of licensed and assigned educators in the content area, extracted from the database of full-time equivalent (FTE) teachers assigned to each teaching area for each school district in Massachusetts, as provided by the Massachusetts Department of Elementary and Secondary Education for use in drawing a random sample of educators.
Sample (with oversampling): The number and percent of licensed and assigned educators sampled to receive the survey. In order to obtain appropriate representation of minority groups (e.g., American Indian, Asian, Black, and Hispanic) in the survey results, oversampling of minority groups was conducted for those fields for which the entire population was not sampled. In these instances, the survey results were weighted appropriately to take the oversampling into account.

Digital Literacy and Computer Science	Population		Sample (with oversampling)
Digital Literacy and Computer Science	N	%	N	%
Total	33	100	33	100
Female	16	48.48	16	48.48
Male	17	51.51	17	51.51
American Indian or Alaska Native	0	0	0	0
Black (not of Hispanic origin)	1	3	1	3
Asian or Pacific Islander	2	6	2	6
Hispanic	0	0	0	0
White	30	90.9	30	90.9
Other	0	0	0	0

Digital Literacy and Computer Science Survey Return Rate by Field and Return Status: Public School Sample

Returned Surveys

Field	Number Sent	Number Not Distributed	Eligible		Ineligible		Total		Adjusted Return Rate
Field	Number Sent	Number Not Distributed	N	%	N	%	N	%	Adjusted Return Rate
066 Biology	296	17	62	20.90%	11	3.70%	73	24.70%	21.80%
067 Chemistry	293	7	58	19.80%	14	4.80%	72	24.60%	20.80%
068 Mathematics (Elementary)	298	8	72	24.20%	13	4.40%	85	28.50%	25.30%
069 Physics	298	27	59	19.80%	8	2.70%	67	22.50%	20.30%
070 Technology/Engineering	298	22	49	16.40%	17	5.70%	66	22.10%	17.40%
071 Digital Literacy and Computer Science	32	2	19	59.40%	0	0.00%	19	59.40%	59.40%
074 Earth and Space Science	149	3	21	14.10%	10	6.70%	31	20.80%	15.10%
ALL	1,664	86	340	20.40%	73	4.40%	413	24.80%	21.40%

Demographic Summary Report: Public School Sample

Teacher Sample
Field: Digital Literacy and Computer Science

Question	Answer	Absolute Frequency	Adjusted Percent
Currently Teaching	Yes	19	100
Currently Teaching	No	0	0
Hold a valid Massachusetts teaching license	Yes	19	100
Hold a valid Massachusetts teaching license	No	0	0
What is the highest level of education you have attained?	Bachelor's degree	2	11
	Master's degree	15	79
	Doctoral degree	2	11
	Other	0	0
What is your ethnicity?	American Indian or Alaska Native	0	0
	Black (not of Hispanic origin)	0	0
	Asian or Pacific Islander	1	5
	Hispanic	0	0
	White (not of Hispanic origin)	18	95
	Other	0	0
What is your first (native) language?	English	19	100
	French	0	0
	Portuguese	0	0
	An Asian language	0	0
	Russian	0	0
	Spanish	0	0
	American Sign Language	0	0
	Italian	0	0
	Other	0	0
What is your gender?	Female	8	42
What is your gender?	Male	10	53
What grade level(s) have you taught in Massachusetts during this or the previous school year? (Select all that apply.)	Pre K to 6th	0	0
	5th to 8th	3	16
	5th to 12th	3	16
	8th to 12th	11	58
How many years of professional teaching experience do you have? (Count partial years as full years.)	1 to 3 years	0	0
	4 to 10 years	4	21
	11 years or more	15	79

Digital Literacy and Computer Science Objective Rating Report: Public School Sample

Number of Objectives: 13

Importance Ratings

Objective Number	N	Mean	SD	S E
1	19	4.05	0.62	0.14
2	18	4.17	0.79	0.19
3	18	4.06	0.64	0.15
4	19	4.26	0.81	0.18
5	18	3.78	0.88	0.21
6	19	3.74	0.81	0.18
7	19	3.79	0.71	0.16
8	19	4.53	0.61	0.14
9	19	3.89	0.74	0.17
10	19	4.68	0.48	0.11
11	19	4.53	0.61	0.14
12	19	3.53	0.9	0.21
13	19	4.05	0.62	0.14

Number of Respondents: 19

Response Distribution (in %)

2	3	4	5
0	3	12	4
0	4	7	7
0	3	11	4
1	1	9	8
1	6	7	4
0	9	6	4
0	7	9	3
0	1	7	11
0	6	9	4
0	0	6	13
0	1	7	11
2	8	6	3
0	3	12	4

Digital Literacy and Computer Science Descriptive Statement Rating Report: Public School Sample

Descriptive Statement Rating Report: Ordered by Objective Number
Unweighted

Number of Objectives: 13

Importance Ratings

Objective Number	N	Mean	SD	S E
1	19	3.89	0.88	0.2
2	18	3.83	0.86	0.2
3	18	3.89	0.68	0.16
4	19	3.95	0.62	0.14
5	19	4	0.67	0.15
6	19	3.89	0.81	0.19
7	19	3.95	0.71	0.16
8	19	3.89	0.66	0.15
9	19	3.74	0.56	0.13
10	19	4.16	0.5	0.12
11	19	3.95	0.78	0.18
12	19	3.58	0.69	0.16
13	18	3.56	0.92	0.22

Number of Respondents: 19

Response Distribution (in %)

2	3	4	5
1	5	8	5
1	5	8	4
0	5	10	3
0	4	12	3
0	4	11	4
1	4	10	4
1	2	13	3
0	5	11	3
0	6	12	1
0	1	14	4
1	3	11	4
1	7	10	1
2	7	6	3

Digital Literacy and Computer Science Composite Rating Report: Public School Sample

Composite Rating Report
Unweighted

Teacher Sample
Field: Digital Literacy and Computer Science

Importance Ratings

N	Mean	SD	S E
19	4	0.88	0.2

Number of Respondents: 19

Response Distribution (in %)

NR	1	2	3	4	5
0	0	1	4	8	6

Digital Literacy and Computer Science Objective Ratings Summary

Objective	Number		Mean Objective Rating		Description of Objective
Objective	Educators	Faculty	Educators	Faculty	Description of Objective
1	19	0	3.93	0	Apply knowledge of safety and security concepts and appropriate digital interactions
2	18	0	4.03	0	Apply knowledge of the responsible, ethical, and legal use of technology resources and digital content.
3	18	0	3.93	0	Apply knowledge of the impacts of technology on society.
4	19	0	4.12	0	Apply knowledge of the use of digital tools and resources to create content, communicate, and collaborate.
5	18	0	3.66	0	Apply knowledge of digital tools and techniques for conducting research.
6	19	0	3.62	0	Apply knowledge of characteristics, functions, and uses of computing devices and components.
7	19	0	3.67	0	Apply knowledge of characteristics, functions, and uses of networks and services.
8	19	0	4.39	0	Apply concepts related to abstraction and algorithms.
9	19	0	3.78	0	Apply concepts related to data representation, modeling, and simulation.
10	19	0	4.55	0	Apply concepts related to computer programming.
11	19	0	4.39	0	Apply concepts related to program development and testing.
12	19	0	3.41	0	Prepare an organized, developed analysis on a topic related to the use of digital tools to create an artifact.
13	18	0	3.67	0	Prepare an organized, developed analysis on a topic related to computer science.
Overall	19	0	3.94	0

English

Committee participants	Number
Total	31
Public School Educators	22
Higher Education Faculty	9

Gender	Number
Female	22
Male	5
No response	4

Ethnicity	Number
American Indian or Alaska Native	0
Black	3
Asian or Pacific Islander	2
Hispanic	3
White	18
Other	1
Unspecified	4

English

Bias Review Committee Composition

Committee participants	Number
Total	7
Public School Educators	7
Higher Education Faculty	0

Gender	Number
Female	6
Male	0
No response	1

Ethnicity	Number
American Indian or Alaska Native	1
Black (not of Hispanic origin)	3
Asian or Pacific Islander	0
Hispanic	1
White	2
Other	0
Unspecified	0

Alignment Between the Massachusetts Tests for Educator Licensure® (MTEL®) Test Objectives for English (061) and Massachusetts Curriculum Framework and Subject Matter Knowledge (SMK) Guidelines for English, 5 to 12

MTEL 061 Test Objectives: I. Reading and Language	Curriculum Framework
0001 Apply knowledge of the characteristics of major genres of literature.	R.CCR.2 Determine central ideas or themes of a text and analyze their development; summarize the key supporting details and ideas. R.CCR.4 Interpret words and phrases as they are used in a text, including determining technical, connotative, and figurative meanings, and analyze how specific word choices shape meaning or tone. R.CCR.5 Analyze the structure of texts, including how specific sentences, paragraphs, and larger portions of a text relate to each other and the whole. RL.5.1, 5.2, 5.4, 5.9; RL.6.1, 6.2, 6.4–6.6, 6.9; RL.7.1–7.6; RL.8.1–8.6; RL.9–10.1–RL.9–10.6, RL.9–10.9; RL.11–12.1–RL.11–12.7	Teacher candidates must demonstrate the necessary depth and breadth of content knowledge needed to support all students in mastering expectations outlined in the following Massachusetts Curriculum Framework: a. 2017 English Language Arts ( E L A )/Literacy Framework: i. Grades 5 to 12
0002 Apply knowledge of American literature from the seventeenth through the twenty-first century that represents a range of American perspectives reflecting diversity of gender; race; ethnicity; sexual orientation; nation of origin; religion; age; disability; and cultural, economic, and geographic backgrounds.	RL.11–12.9 Demonstrate knowledge of eighteenth, nineteenth and early–twentieth century foundational works of American literature, including how two or more texts from the same period treat similar themes or topics. Appendix B: A Literary Heritage	Teacher candidates must demonstrate the necessary depth and breadth of content knowledge needed to support all students in mastering expectations outlined in the following Massachusetts Curriculum Framework: a. 2017 English Language Arts( E L A )/Literacy Framework: i. Grades 5 to 12
0003 Apply knowledge of British literature from the Anglo-Saxon through the contemporary period.	Appendix B: A Literary Heritage	Teacher candidates must demonstrate the necessary depth and breadth of content knowledge needed to support all students in mastering expectations outlined in the following Massachusetts Curriculum Framework: a. 2017 English Language Arts( E L A )/Literacy Framework: i. Grades 5 to 12
0004 Apply knowledge of contemporary and historical literature from Africa, Asia, Latin America, the Caribbean, and Europe from ancient times through the twenty-first century.	Appendix B: A Literary Heritage	Teacher candidates must demonstrate the necessary depth and breadth of content knowledge needed to support all students in mastering expectations outlined in the following Massachusetts Curriculum Framework: a. 2017 English Language Arts( E L A )/Literacy Framework: i. Grades 5 to 12
0005 Apply knowledge of informational texts.	R.CCR.1 Read closely to determine what a text states explicitly and to make logical inferences from it; cite specific textual evidence when writing or speaking to support conclusions drawn from a text. R.CCR.2 Determine central ideas or themes of a text and analyze their development; summarize the key supporting details and ideas. R.CCR.4 Interpret words and phrases as they are used in a text, including determining technical, connotative, and figurative meanings, and analyze how specific word choices shape meaning or tone. R.CCR.6 Assess how point of view or purpose shapes the content and style of a text. R.CCR.8 Delineate and evaluate the argument and specific claims in a text, including the validity of the reasoning as well as the relevance and sufficiency of the evidence. RI.5.2, RI.5.4–RI.5.9; RI.6.2–RI.6.9; RI.7.2–RI.7.9; RI.8.2–RI.8.9; RI.9–10.1–RI.9–10.9; RI.11–12.1–RI.11–12.9	Teacher candidates must demonstrate the necessary depth and breadth of content knowledge needed to support all students in mastering expectations outlined in the following Massachusetts Curriculum Framework: a. 2017 English Language Arts( E L A )/Literacy Framework: i. Grades 5 to 12
0006 Apply knowledge of the structure and development of Standard American English.	L.CCR.4 Determine or clarify the meaning of unknown and multiple-meaning words and phrases by using context clues, analyzing meaningful word parts, and consulting general and specialized reference materials, as appropriate. L.CCR.5 Demonstrate understanding of figurative language, word relationships, and nuances in word meanings. L.CCR.6 Acquire and use accurately a range of general academic and domain-specific words and phrases sufficient for reading, writing, speaking, and listening at the college and career readiness level; demonstrate independence in gathering vocabulary knowledge. L.5.4–L.5.6; L.6.4–L.6.6; L.7.4–L.7.6; L.8.4–L.8.6; L.9–10.4–L.9–10.6; L.11–12.4–L.11–12.6	Teacher candidates must demonstrate the necessary depth and breadth of content knowledge needed to support all students in mastering expectations outlined in the following Massachusetts Curriculum Framework: a. 2017 English Language Arts( E L A )/Literacy Framework: i. Grades 5 to 12
0007 Apply knowledge of theory, research, and instructional practice related to language acquisition and reading.	N/A	N/A

MTEL 061 Test Objectives:
II. Rhetoric and Composition

Curriculum Framework

0008 Apply knowledge of principles of rhetoric and characteristics of effective writing and writing instruction.

W.CCR.4 Produce clear and coherent writing in which the development, organization, and style are appropriate to task, purpose, and audience.

W.CCR.5 Develop and strengthen writing as needed by planning, revising, editing, rewriting, or trying a new approach.

W.CCR.6 Use technology to produce and publish writing and to interact and collaborate with others.

W.5.4–W.5.6; W.6.4–W.6.6; W.7.4–W.7.6; W.8.4–W.8.6; W.9–10.4–W.9–10.6; W.11–12.4–W.11–12.6

L.CCR.1 Demonstrate command of the conventions of standard English grammar and usage when writing or speaking.

L.CCR.2 Demonstrate command of the conventions of standard English capitalization, punctuation, and spelling when writing.

L.5.1–L.5.3; L.6.1–L.6.3; L.7.1–L.7.3; L.8.1–L.8.3; L.9–10.1–L.9–10.3; L.11–12.1–L.11–12.3

Teacher candidates must demonstrate the necessary depth and breadth of content knowledge needed to support all students in mastering expectations outlined in the following Massachusetts Curriculum Framework:

a. 2017 English Language Arts( E L A )/Literacy Framework:

i. Grades 5 to 12

0009 Apply knowledge of techniques for writing arguments.

W.CCR.1 Write arguments to support claims in an analysis of substantive topics or texts, using valid reasoning and relevant and sufficient evidence.

W.5.1.a–d; W.6.1.a–e; W.7.1.a–e; W.8.1.a–e; W.9–10.1.a–e; W.11–12.1.a–e

Teacher candidates must demonstrate the necessary depth and breadth of content knowledge needed to support all students in mastering expectations outlined in the following Massachusetts Curriculum Framework:

a. 2017 English Language Arts( E L A )/Literacy Framework:

i. Grades 5 to 12

0010 Apply knowledge of techniques for writing informative/explanatory texts.

W.CCR.2 Write informative/explanatory texts to examine and convey complex ideas and information clearly and accurately through the effective selection, organization, and analysis of content.

W.5.2.a–e; W.6.2.a–f; W.7.2.a–f; W.8.2.a–f; W.9–10.2.a–f; W.10–11.2.a–f

Teacher candidates must demonstrate the necessary depth and breadth of content knowledge needed to support all students in mastering expectations outlined in the following Massachusetts Curriculum Framework:

a. 2017 English Language Arts( E L A )/Literacy Framework:

i. Grades 5 to 12

0011 Apply knowledge of techniques for conducting academic research to build and present knowledge.

W.CCR.8 When conducting research, gather relevant information from multiple print and digital sources, assess the credibility and accuracy of each source, and integrate the information while avoiding plagiarism.

W.5.7–W.5.9; W.6.7–W.6.9; W.7.7–W.7.9; W.8.7–W.8.9; W.9–10.7–W.9–10.9; W.11–12.7–W.11–12.9

Teacher candidates must demonstrate the necessary depth and breadth of content knowledge needed to support all students in mastering expectations outlined in the following Massachusetts Curriculum Framework:

a. 2017 English Language Arts( E L A )/Literacy Framework:

i. Grades 5 to 12

0012 Apply knowledge of techniques for writing narratives.

W.CCR.3 Write narratives to develop experiences or events using effective literary techniques, well-chosen details, and well-structured sequences.

W.5.3.a–g; W.6.3.a–e; W.7.3.a–e; W.8.3.a–e; W.9–10.3.a–e; W.11–12.3.a–e

Teacher candidates must demonstrate the necessary depth and breadth of content knowledge needed to support all students in mastering expectations outlined in the following Massachusetts Curriculum Framework:

a. 2017 English Language Arts( E L A )/Literacy Framework:

i. Grades 5 to 12

0013 Apply knowledge of techniques for speaking and/or expressive communication and listening and/or receptive communication to use in a variety of contexts.

SL.CCR.1 Prepare for and participate effectively in a range of conversations and collaborations with diverse partners, building on others' ideas and expressing their own clearly and persuasively.

SL.CCR.2 Integrate and evaluate information presented in diverse media and formats, including visually, quantitatively, and orally.

SL.CCR.3 Evaluate a speaker's point of view, reasoning, and use of evidence and rhetoric.

SL.CCR.4 Present information, findings, and supporting evidence such that:

Listeners can follow the line of reasoning.
The organization, development, vocabulary, and style are appropriate to task, purpose, and audience.

SL.CCR.5 Make strategic use of digital media and visual displays of data to express information and enhance understanding of presentations.

SL.CCR.6 Adapt speech to a variety of contexts and communicative tasks, demonstrating command of formal English when indicated or appropriate.

SL.5.1–SL.5.6; SL.6.1–SL.6.6; SL.7.1–SL.7.6; SL.8.1–SL.8.6; SL.9–10.1–SL.9–10.6; SL.11–12.1–SL.11–12.6

Teacher candidates must demonstrate the necessary depth and breadth of content knowledge needed to support all students in mastering expectations outlined in the following Massachusetts Curriculum Framework:

a. 2017 English Language Arts( E L A )/Literacy Framework:

i. Grades 5 to 12

MTEL 061 Test Objectives:
III. Integration of Knowledge and Understanding

Curriculum Framework

0014 Prepare an organized, developed written analysis of a literary or informational text.

R.CCR.2 Determine central ideas or themes of a text and analyze their development; summarize the key supporting details and ideas.

R.CCR.4 Interpret words and phrases as they are used in a text, including determining technical, connotative, and figurative meanings, and analyze how specific word choices shape meaning or tone.

R.CCR.5 Analyze the structure of texts, including how specific sentences, paragraphs, and larger portions of a text relate to each other and the whole.

R.CCR.1 Read closely to determine what a text states explicitly and to make logical inferences from it; cite specific textual evidence when writing or speaking to support conclusions drawn from a text.

R.CCR.2 Determine central ideas or themes of a text and analyze their development; summarize the key supporting details and ideas.

R.CCR.4 Interpret words and phrases as they are used in a text, including determining technical, connotative, and figurative meanings, and analyze how specific word choices shape meaning or tone.

R.CCR.6 Assess how point of view or purpose shapes the content and style of a text.

W.CCR.1 Write arguments to support claims in an analysis of substantive topics or texts, using valid reasoning and relevant and sufficient evidence.

W.CCR.4 Produce clear and coherent writing in which the development, organization, and style are appropriate to task, purpose, and audience.

W.CCR.5 Develop and strengthen writing as needed by planning, revising, editing, rewriting, or trying a new approach.

W.CCR.9 Draw evidence from literary or informational texts to support analysis, interpretation, reflection, and research.

Teacher candidates must demonstrate the necessary depth and breadth of content knowledge needed to support all students in mastering expectations outlined in the following Massachusetts Curriculum Framework:

a. 2017 English Language Arts( E L A )/Literacy Framework:

i. Grades 5 to 12

0015 Prepare an organized, developed argument in response to a written text.

R.CCR.1 Read closely to determine what a text states explicitly and to make logical inferences from it; cite specific textual evidence when writing or speaking to support conclusions drawn from a text.

R.CCR.2 Determine central ideas or themes of a text and analyze their development; summarize the key supporting details and ideas.

R.CCR.4 Interpret words and phrases as they are used in a text, including determining technical, connotative, and figurative meanings, and analyze how specific word choices shape meaning or tone.

R.CCR.6 Assess how point of view or purpose shapes the content and style of a text.

R.CCR.8 Delineate and evaluate the argument and specific claims in a text, including the validity of the reasoning as well as the relevance and sufficiency of the evidence.

W.CCR.1 Write arguments to support claims in an analysis of substantive topics or texts, using valid reasoning and relevant and sufficient evidence.

W.CCR.4 Produce clear and coherent writing in which the development, organization, and style are appropriate to task, purpose, and audience.

W.CCR.5 Develop and strengthen writing as needed by planning, revising, editing, rewriting, or trying a new approach.

Teacher candidates must demonstrate the necessary depth and breadth of content knowledge needed to support all students in mastering expectations outlined in the following Massachusetts Curriculum Framework:

a. 2017 English Language Arts( E L A )/Literacy Framework:

i. Grades 5 to 12

Subject Matter Knowledge Guidelines for English Language Arts and Literacy Updated September 2019 (English Language Arts and Literacy Curriculum Framework [2017] College and Career Readiness Standards)	MTEL 061 Test Objectives
R.CCR.1–R.CCR.10	0001, 0002, 0003, 0004, 0005
W.CCR.1	0009, 0015
W.CCR.2	0010
W.CCR.3	0012
W.CCR.4	0008, 0009, 0010, 0012, 0014, 0015
W.CCR.5	0008, 0014, 0015
W.CCR.6	0008
W.CCR.7	0011
W.CCR.8	0011
W.CCR.9	0011, 0014, 0015
SL.1	0013
SL.2	0013
SL.3	0013
SL.4	0013
SL.5	0013
SL.6	0013
L.1	0008
L.2	0008
L.3	0008
L.4	0001, 0005, 0006
L.5	0001, 0005, 0006
L.6	0006

English

Committee participants	Number
Total	19
Public School Educators	16
Higher Education Faculty	3

Gender	Number
Female	12
Male	7
No response	0

Ethnicity	Number
American Indian or Alaska Native	0
Black (not of Hispanic origin)	2
Asian or Pacific Islander	1
Hispanic	2
White	13
Other	0
Unspecified	1

English as a Second Language

INTACT CLASSROOM SESSIONS 2013–2015
10/01/2013	Lincoln School (Brookline Public Schools)
10/05/2013	Simmons College Gordon College Salem State University
10/08/2013	Wheaton College
10/18/2013	University of Massachusetts - Boston
10/31/2013	Wheelock College
01/30/2014	Wheaton College
02/01/2014	Gordon College
02/04/2014	Elms College
02/05/2014	Salem State University
02/06/2014	University of Massachusetts - Amherst
02/11/2014	Lincoln School (Brookline Public Schools)
02/14/2014	Wheelock College
02/19/2014	Worcester State University
04/25/2015	Cambridge College
04/27/2015	Salem State University
05/02/2015	Simmons College Framingham State University
05/05/2015	Westfield State University
05/09/2015	Gordon College

STAND ALONE EVENTS AT MTEL OPERATIONAL TEST ADMINISTRATIONS

10/26/2013
03/01/2014
05/10/2014
07/12/2014
10/25/2014

Pilot Test Outcomes
English

Pilot testing was conducted at CBT centers from February 18, 2020, through March 16, 2020. The following information is a summary of the data collected during this specific time frame.

Multiple-Choice Items ( M C Q's ) and Short-Answer Items ( S A's )

Number of M C Q's Prepared	113
Number of M C Q's Piloted*	50
Number of M C Q's Identified for Further Review	4
Number of M C Q's Deleted Post Pilot Test Review	0

Open-Response Items ( O R I's )

Number of O R I's Developed	21
Number of O R I's Piloted*	18
Number of O R I's Identified for Further Review	3
Number of O R I's Deleted Post Pilot Test Review	3

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms until all items are piloted.

English

Number of Qualifying Score Participants: 19

Number of Participants Providing Ratings: 19 (100%)

Question Rating Scale: 1–5 1=not at all (well/confident/satisfied/successful), 5=very (well/confident/satisfied/successful)	Median Rating
How well do you feel you understood the training that you received for making your qualifying score judgments?	5
How confident are you of the qualifying score judgments you made?	5
How satisfied are you that you were able to complete the work you were asked to do in the time that was provided?	5
How successful were the coordination and logistics of the qualifying score conference?	4
Overall, how satisfied are you with the qualifying score process in which you participated?	4

English Content Validation Survey Population/Sample/Respondents Demographics (Public School Sample)

Key

Population: The number and percent of licensed and assigned educators in the content area, extracted from the database of full-time equivalent (FTE) teachers assigned to each teaching area for each school district in Massachusetts, as provided by the Massachusetts Department of Elementary and Secondary Education for use in drawing a random sample of educators.
Sample (with oversampling): The number and percent of licensed and assigned educators sampled to receive the survey. In order to obtain appropriate representation of minority groups (e.g., American Indian, Asian, Black, and Hispanic) in the survey results, oversampling of minority groups was conducted for those fields for which the entire population was not sampled. In these instances, the survey results were weighted appropriately to take the oversampling into account.

English	Population		Sample (with oversampling)
English	N	%	N	%
Total	7,008	100	427	100
Female	5,110	72.91	312	73.06
Male	1,898	27.08	115	26.93
American Indian or Alaska Native	10	0.14	2	0.46
Black (not of Hispanic origin)	214	3.05	25	5.85
Asian or Pacific Islander	73	1.04	8	1.87
Hispanic	120	1.71	14	3.27
White	6,558	93.57	374	87.58
Other	33	0.47	4	0.93

English Survey Return Rate by Field and Return Status: Public School Sample

Returned Surveys

Field	Number Sent	Number Not Distributed	Eligible		Ineligible		Total		Adjusted Return Rate
Field	Number Sent	Number Not Distributed	N	%	N	%	N	%	Adjusted Return Rate
061 English	541	30	191	35.30%	32	5.90%	223	43.60%	39.90%
063 Mathematics	609	28	205	33.70%	53	8.70%	258	44.40%	38.80%
064 General Science	530	15	154	29.10%	58	10.90%	212	41.20%	33.70%
065 Middle School Mathematics	581	14	160	27.50%	78	13.40%	238	42.00%	32.70%
190 Foundations of Reading	498	18	166	33.30%	41	8.20%	207	43.10%	37.80%
ALL	2,759	105	876	31.80%	262	9.50%	1,138	42.90%	36.60%

English Survey Return Rate by Field and Return Status: Faculty Sample

Returned Surveys

Field	Number Sent	Number Not Distributed	Eligible		Ineligible		Total		Adjusted Return Rate
Field	Number Sent	Number Not Distributed	N	%	N	%	N	%	Adjusted Return Rate
061 English	127	9	18	14.20%	1	0.80%	19	16.10%	15.40%
063 Mathematics	129	8	24	18.60%	0	0.00%	24	19.80%	19.80%
064 General Science	106	9	9	8.50%	2	1.90%	11	11.30%	9.50%
065 Middle School Mathematics	121	9	21	17.40%	3	2.50%	24	21.40%	19.30%
190 Foundations of Reading	138	5	33	23.90%	0	0.00%	33	24.80%	24.80%
ALL	621	40	105	16.90%	6	1.00%	111	19.10%	18.30%

Demographic Summary Report: Public School Sample

Teacher Sample
Field: English

Question	Answer	Absolute Frequency	Adjusted Percent
Currently Teaching	Yes	191	100
Currently Teaching	No	0	0
Hold a valid Massachusetts teaching license	Yes	191	100
Hold a valid Massachusetts teaching license	No	0	0
What is the highest level of education you have attained?	Bachelor's degree	17	8.9
	Master's degree	164	85.9
	Doctoral degree	6	3.1
	Other	4	2.1
What is your ethnicity?	American Indian or Alaska Native	2	1.1
	Black (not of Hispanic origin)	4	2.1
	Asian or Pacific Islander	6	3.2
	Hispanic	2	1.1
	White (not of Hispanic origin)	168	88.9
	Other	7	3.7
What is your first (native) language?	English	184	96.8
	French	1	0.5
	Portuguese	1	0.5
	An Asian language	3	1.6
	Other	1	0.5
What is your gender?	Female	115	60.8
What is your gender?	Male	74	39.2
What grade level(s) have you taught in Massachusetts during this or the previous school year? (Select all that apply.)	Pre K to 6th	4	2
	5th to 8th	63	31.2
	5th to 12th	22	10.9
	8th to 12th	113	55.9
How many years of professional teaching experience do you have? (Count partial years as full years.)	1 to 3 years	7	3.7
	4 to 10 years	52	27.4
	11 years or more	131	68.9

English Demographic Summary Report: Faculty Sample

Faculty Sample
Field: English

Question	Answer	Absolute Frequency	Adjusted Percent
Teaching undergraduate or graduate arts and science or education courses in which prospective teachers may have been enrolled	Yes	18	100
	No	0	0
What is the highest level of education you have attained?	Bachelor's degree	0	0
	Master's degree	0	0
	Doctoral degree	17	94.4
	Other	1	5.6
What is your ethnicity?	American Indian or Alaska Native	0	0
	Black (not of Hispanic origin)	0	0
	Asian or Pacific Islander	0	0
	Hispanic	2	11.8
	White (not of Hispanic origin)	15	88.2
	Other	0	0
What is your first (native) language?	English	16	88.9
	Spanish	2	11.1
	Other	0	0
What is your gender?	Female	12	66.7
What is your gender?	Male	6	33.3
How many years have you taught as a faculty member at a higher education institution? (Count partial years as full years.)	4 to 10 years	4	22.2
	11 years or more	14	77.8
In which higher education institution department do you currently hold your primary appointment?	Education department only	4	22.2
	Arts and Sciences or Fine Arts department(s) only	14	77.8
Have you ever taught in the subject of the field and grade level indicated above in Massachusetts?	Yes	9	52.9
	No	8	47.1
How recently have you taught in the subject of the field and grade level indicated above in Massachusetts? (Count partial years as full years.)	1 to 3 years	2	22.2
	4 to 10 years	3	33.3
	11 years or more	4	44.4

English Objective Rating Report: Public School Sample

Number of Objectives: 15

Importance Ratings

Objective Number	N	Mean	SD	S E
1	189	4.2	0.76	0.06
2	189	3.95	0.89	0.07
3	189	3.23	0.79	0.06
4	188	3.34	0.9	0.07
5	190	4.23	0.78	0.06
6	189	4.17	0.83	0.06
7	188	3.8	0.82	0.06
8	189	4.32	0.7	0.05
9	188	4.35	0.73	0.05
10	188	4.23	0.72	0.05
11	190	4.11	0.77	0.06
12	190	3.87	0.86	0.06
13	190	4.11	0.78	0.06
14	190	4.42	0.69	0.05
15	189	4.46	0.64	0.05

Number of Respondents: 190

Response Distribution (in %)

NR	1	2	3	4	5
1	1	0	17	43	38
1	2	3	25	40	30
1	1	14	54	24	7
2	3	10	47	28	10
1	0	2	16	39	42
1	0	2	20	35	41
2	0	3	37	37	23
1	0	1	12	42	45
2	0	1	12	37	49
2	0	0	16	43	39
1	0	1	21	43	34
1	0	3	35	34	28
1	0	1	23	41	35
1	0	1	8	38	52
1	0	0	8	38	53

English Objective Rating Report: Faculty Sample

Objective Rating Report: Ordered by Objective Number
Unweighted

Faculty Sample
Field: English

Number of Objectives: 15

Importance Ratings

Objective Number	N	Mean	SD	S E
1	17	4.65	0.61	0.15
2	17	4.35	0.79	0.19
3	18	3.83	0.79	0.19
4	18	4	0.84	0.2
5	17	3.71	0.99	0.24
6	18	4.11	0.96	0.23
7	18	4.11	0.76	0.18
8	18	4.5	0.62	0.15
9	18	4.22	0.88	0.21
10	18	4.06	0.87	0.21
11	18	4.17	0.86	0.2
12	18	3.83	0.71	0.17
13	18	4.06	0.8	0.19
14	18	4.33	0.84	0.2
15	18	4.28	0.75	0.18

Number of Respondents: 18

Response Distribution (in %)

NR	2	3	4	5
6	0	6	22	67
6	0	17	28	50
0	6	22	56	17
0	6	17	50	28
6	11	28	33	22
0	6	22	28	44
0	0	22	44	33
0	0	6	39	56
0	6	11	39	44
0	0	33	28	39
0	6	11	44	39
0	0	33	50	17
0	0	28	39	33
0	6	6	39	50
0	0	17	39	44

English Descriptive Statement Rating Report: Public School Sample

Descriptive Statement Rating Report: Ordered by Objective Number
Unweighted

Number of Objectives: 15

Importance Ratings

Objective Number	N	Mean	SD	S E
1	190	4.31	0.79	0.06
2	187	3.97	0.88	0.06
3	189	3.68	0.91	0.07
4	188	3.67	0.89	0.07
5	189	4.29	0.79	0.06
6	190	4.06	0.87	0.06
7	188	3.93	0.89	0.06
8	190	4.21	0.78	0.06
9	191	4.34	0.75	0.05
10	187	4.25	0.71	0.05
11	189	4.18	0.82	0.06
12	191	4.18	0.73	0.05
13	188	4.11	0.81	0.06
14	188	4.22	0.75	0.05
15	189	4.23	0.74	0.05

Number of Respondents: 191

Response Distribution (in %)

NR	1	2	3	4	5
1	1	2	11	39	47
2	2	4	18	47	28
1	1	9	29	41	18
2	2	8	28	46	16
1	0	3	12	38	46
1	1	5	18	42	35
2	1	3	28	37	29
1	0	3	14	43	40
0	1	2	9	41	48
2	1	2	7	52	36
1	1	2	14	43	39
0	1	1	14	50	35
2	1	1	18	44	34
2	1	1	13	46	38
1	0	2	12	46	39

English Descriptive Statement Rating Report: Faculty Sample

Descriptive Statement Rating Report: Ordered by Objective Number
Unweighted

Faculty Sample
Field: English

Number of Objectives: 15

Importance Ratings

Objective Number	N	Mean	SD	S E
1	18	4.28	0.89	0.21
2	18	4	1.14	0.27
3	18	3.89	0.96	0.23
4	18	3.78	0.81	0.19
5	17	3.71	0.99	0.24
6	18	3.67	1.19	0.28
7	18	4	0.84	0.2
8	18	3.94	0.73	0.17
9	18	4	0.77	0.18
10	17	4	0.71	0.17
11	18	4.22	0.65	0.15
12	18	4	0.84	0.2
13	18	3.78	0.88	0.21
14	18	4.22	0.81	0.19
15	18	4	0.84	0.2

Number of Respondents: 18

Response Distribution (in %)

NR	1	2	3	4	5
0	0	6	11	33	50
0	6	6	11	39	39
0	0	11	17	44	28
0	0	6	28	50	17
6	0	11	28	33	22
0	0	28	6	39	28
0	0	6	17	50	28
0	0	6	11	67	17
0	0	6	11	61	22
6	0	0	22	50	22
0	0	0	11	56	33
0	0	6	17	50	28
0	0	11	17	56	17
0	0	6	6	50	39
0	0	6	17	50	28

English Composite Rating Report: Public School Sample

Composite Rating Report
Unweighted

Teacher Sample
Field: English

Importance Ratings

N	Mean	SD	S E
190	4.11	0.73	0.05

Number of Respondents: 190

Response Distribution (in %)

NR	1	2	3	4	5
0	0	3	14	53	30

English Composite Rating Report: Faculty Sample

Composite Rating Report
Unweighted

Education Faculty Sample
Field: English

Importance Ratings

N	Mean	SD	S E
18	4.06	0.8	0.19

Number of Respondents: 18

Response Distribution (in %)

NR	1	2	3	4	5
0	0	6	11	56	28

English Objective Ratings Summary

Objective	Number		Mean Objective Rating		Description of Objective
Objective	Educators	Faculty	Educators	Faculty	Description of Objective
1	186	17	4.22	4.65	Apply knowledge of the characteristics of major genres of literature.
2	186	17	3.98	4.35	Apply knowledge of American literature from the seventeenth through the twenty-first century that represents a range of American perspectives reflecting diversity of gender; race; ethnicity; sexual orientation; nation of origin; religion; age; disability; and cultural, economic, and geographic backgrounds.
3	186	18	3.21	3.83	Apply knowledge of British literature from the Anglo-Saxon through the contemporary period.
4	185	18	3.3	4	Apply knowledge of contemporary and historical literature from Africa, Asia, Latin America, the Caribbean, and Europe from ancient times through the twenty-first century.
5	187	17	4.26	3.71	Apply knowledge of informational texts.
6	186	18	4.21	4.11	Apply knowledge of the structure and development of Standard American English.
7	185	18	3.8	4.11	Apply knowledge of theory, research, and instructional practice related to language acquisition and reading.
8	186	18	4.33	4.5	Apply knowledge of principles of rhetoric and characteristics of effective writing and writing instruction.
9	185	18	4.37	4.22	Apply knowledge of techniques for writing arguments.
10	185	18	4.24	4.06	Apply knowledge of techniques for writing informative/explanatory texts.
11	187	18	4.08	4.17	Apply knowledge of techniques for conducting academic research to build and present knowledge.
12	187	18	3.89	3.83	Apply knowledge of techniques for writing narratives.
13	187	18	4.16	4.06	Apply knowledge of techniques for speaking and/or expressive communication and listening and/or receptive communication to use in a variety of contexts.
14	187	18	4.44	4.33	Prepare an organized, developed written analysis of a literary or informational text.
15	186	18	4.52	4.28	Prepare an organized, developed written analysis of an argument.
Overall	187	18	4.07	4.15

General Science

Committee participants	Number
Total	23
Public School Educators	19
Higher Education Faculty	4

Gender	Number
Female	5
Male	18
No response	0

Ethnicity	Number
American Indian or Alaska Native	0
Black	1
Asian or Pacific Islander	0
Hispanic	2
White	12
Other	2
Unspecified	6

General Science

Bias Review Committee Composition

Committee participants	Number
Total	10
Public School Educators	10
Higher Education Faculty	0

Gender	Number
Female	8
Male	2
No response	0

Ethnicity	Number
American Indian or Alaska Native	0
Black (not of Hispanic origin)	6
Asian or Pacific Islander	2
Hispanic	0
White	1
Other	1
Unspecified	0

Alignment Between the Massachusetts Tests for Educator Licensure® (MTEL®) Test Objectives for M A 064 General Science (5 to 8) and Massachusetts Curriculum Framework for Science and Technology / Engineering.

MTEL 064 Test Objectives:
I. Technology / Engineering

Curriculum Framework

0001 Apply knowledge of engineering design.

5- E S S 3 dash 2 (M A) Test a simple system designed to filter particulates out of water and propose one change to the design to improve it.

5-LS2-2( M A ) Compare at least two designs for a composter to determine which is most likely to encourage decomposition of materials.

5.3-5- E T S 3 dash 2 (M A) Use sketches or drawings to show how each part of a product or device relates to other parts in the product or device.

6.MS- E T S 1 dash 1 Define the criteria and constraints of a design problem with sufficient precision to ensure a successful solution. Include potential impacts on people and the natural environment that may limit possible solutions.

6.MS- E T S 1 dash 5 (M A) Create visual displays of solutions to a design problem. Accurately interpret and apply scale and proportion to visual representations.

6.MS- E T S 1 dash 6 (M A) Communicate a design solution to an intended user, including design features and limitations of the solution.

7.MS- E T S 1 dash 2 Evaluate competing solutions to a given design problem using a decision matrix to determine how well each meets the criteria and constraints of the problem. Use a model of each solution to evaluate how variations in one or more design features, including size, shape, weight, or cost, may affect the function or effectiveness of the solution.

7.MS- E T S 1 dash 4 Generate and analyze data from iterative testing and modification of a proposed object, tool, or process to optimize the object, tool, or process for its intended purpose.

7.MS- E T S 1 dash 7 (M A) Construct a prototype of a solution to a given design problem.

7.MS-LS2-5 Evaluate competing design solutions for protecting an ecosystem. Discuss benefits and limitations of each design.

7.MS-PS3-3 Apply scientific principles to design, construct, and test a device to minimize or maximize thermal energy transfer.

7.MS- E T S 3 dash 4 (M A) Show how the components of a structural system work together to serve a structural function. Provide examples of physical structures and relate their design to their intended use.

0002 Demonstrate knowledge of tools, materials, and manufacturing.

6.MS- E T S 2 dash 1 (M A) Analyze and compare properties of metals, plastics, wood, and ceramics, including flexibility, ductility, hardness, thermal conductivity, electrical conductivity, and melting point.

6.MS- E T S 2 dash 2 (M A) Given a design task, select appropriate materials based on specific properties needed in the construction of a solution.

6.MS- E T S 2 dash 3 (M A) Choose and safely use appropriate measuring tools, hand tools, fasteners, and common hand–held power tools used to construct a prototype.

8.MS- E T S 2 dash 4 (M A) Use informational text to illustrate that materials maintain their composition under various kinds of physical processing; however, some material properties may change if a process changes the particulate structure of a material.

8.MS- E T S 2 dash 5 (M A) Present information that illustrates how a product can be created using basic processes in manufacturing systems, including forming, separating, conditioning, assembling, finishing, quality control, and safety. Compare the advantages and disadvantages of human vs. computer control of these processes.

0003 Demonstrate knowledge of technological systems.

5.3-5- E T S 3 dash 1 (M A) Use informational text to provide examples of improvements to existing technologies (innovations) and the development of new technologies (inventions). Recognize that technology is any modification of the natural or designed world done to fulfill human needs or wants.

5.3-5- E T S 3 dash 2 (M A) Use sketches or drawings to show how each part of a product or device relates to other parts in the product or device.

5- E S S 3 dash 1 (M A) Obtain and combine information about ways communities reduce human impact on the Earth’s resources and environment by changing an agricultural, industrial, or community practice or process.

6.MS-PS4-3 Present qualitative scientific and technical information to support the claim that digitized signals (sent as wave pulses representing 0s and 1s) can be used to encode and transmit information.

6.MS- E T S 2 dash 1 (M A) Analyze and compare properties of metals, plastics, wood, and ceramics, including flexibility, ductility, hardness, thermal conductivity, electrical conductivity, and melting point.

6.MS- E T S 2 dash 2 (M A) Given a design task, select appropriate materials based on specific properties needed in the construction of a solution.

7.MS- E T S 3 dash 1 (M A) Explain the function of a communication system and the role of its components, including a source, encoder, transmitter, receiver, decoder, and storage.

7.MS- E T S 3 dash 2 (M A) Compare benefits and drawbacks of different communication systems.

7.MS- E T S 3 dash 3 (M A) Research and communicate information about how transportation systems are designed to move people and goods using a variety of vehicles and devices. Identify and describe subsystems of a transportation vehicle, including structural, propulsion, guidance, suspension, and control subsystems.

7.MS- E T S 3 dash 4 (M A) Show how the components of a structural system work together to serve a structural function. Provide examples of physical structures and relate their design to their intended use.

7.MS- E T S 3 dash 5 (M A) Use the concept of systems engineering to model inputs, processes, outputs, and feedback among components of a transportation, structural, or communication system.

8.MS- E T S 2 dash 4 (M A) Use informational text to illustrate that materials maintain their composition under various kinds of physical processing; however, some material properties may change if a process changes the particulate structure of a material.

8.MS- E T S 2 dash 5 (M A) Present information that illustrates how a product can be created using basic processes in manufacturing systems, including forming, separating, conditioning, assembling, finishing, quality control, and safety. Compare the advantages and disadvantages of human vs. computer control of these processes.

MTEL 064 Test Objectives: II. Earth and Space Science	Curriculum Framework
0004 Demonstrate knowledge of the components of the solar system and universe and their interactions.	5- E S S 1 dash 1 Use observations, first–hand and from various media, to argue that the Sun is a star that appears larger and brighter than other stars because it is closer to Earth. 5- E S S 1 dash 2 Use a model to communicate Earth's relationship to the Sun, Moon, and other stars that explain (a) why people on Earth experience day and night, (b) patterns in daily changes in length and direction of shadows over a day, and (c) changes in the apparent position of the Sun, Moon, and stars at different times during a day, over a month, and over a year. 6.MS- E S S 1 dash 1 A Develop and use a model of the Earth–Sun–Moon system to explain the causes of lunar phases and eclipses of the Sun and Moon. 6.MS- E S S 1 dash 5 (M A) Use graphical displays to illustrate that Earth and its solar system are one of many in the Milky Way galaxy, which is one of billions of galaxies in the universe. 8.MS- E S S 1 dash 1 b Develop and use a model of the Earth-Sun system to explain the cyclical pattern of seasons, which includes Earth's tilt and differential intensity of sunlight on different areas of Earth across the year. 8.MS- E S S 1 dash 2 Explain the role of gravity in ocean tides, the orbital motions of planets, their moons, and asteroids in the solar system.
0005 Apply knowledge of Earth's geosphere, geologic history, and processes.	6.MS- E S S 1 dash 4 Analyze and interpret rock layers and index fossils to determine the relative ages of rock formations that result from processes occurring over long periods of time. 6.MS- E S S 2 dash 3 Analyze and interpret maps showing the distribution of fossils and rocks, continental shapes, and seafloor structures to provide evidence that Earth's plates have moved great distances, collided, and spread apart. 7.MS- E S S 2 dash 2 Construct an explanation based on evidence for how Earth's surface has changed over scales that range from local to global in size. 8.MS- E S S 2 dash 1 Use a model to illustrate that energy from Earth's interior drives convection that cycles Earth's crust, leading to melting, crystallization, weathering, and deformation of large rock formations, including generation of ocean sea floor at ridges, submergence of ocean sea floor at trenches, mountain building, and active volcanic chains.
0006 Demonstrate knowledge of Earth's hydrosphere, atmosphere, weather, and climate.	5- E S S 2 dash 1 Use a model to describe the cycling of water through a watershed through evaporation, precipitation, absorption, surface runoff, and condensation. 5- E S S 2 dash 2 Describe and graph the relative amounts of salt water in the ocean; fresh water in lakes, rivers, and groundwater; and fresh water frozen in glaciers and polar ice caps to provide evidence about the availability of fresh water in Earth's biosphere. 7.MS- E S S 2 dash 4 Develop a model to explain how the energy of the Sun and Earth's gravity drive the cycling of water, including changes of state, as it moves through multiple pathways in Earth's hydrosphere. 8.MS- E S S 2 dash 5 Interpret basic weather data to identify patterns in air mass interactions and the relationship of those patterns to local weather. 8.MS- E S S 2 dash 6 Describe how interactions involving the ocean affect weather and climate on a regional scale, including the influence of the ocean temperature as mediated by energy input from the Sun and energy loss due to evaporation or redistribution via ocean currents.
0007 Demonstrate knowledge of natural resources, natural hazards, and human impacts on the environment.	5- E S S 3 dash 1 Obtain and combine information about ways communities reduce human impact on the Earth's resources and environment by changing an agricultural, industrial, or community practice or process. 5- E S S 3 dash 2 (M A) Test a simple system designed to filter particulates out of water and propose one change to the design to improve it. 5-LS2-2( M A ) Compare at least two designs for a composter to determine which is most likely to encourage decomposition of materials. 5.3-5- E T S 3 dash 1 (M A) Use informational text to provide examples of improvements to existing technologies (innovations) and the development of new technologies (inventions). Recognize that technology is any modification of the natural or designed world done to fulfill human needs or wants. 6.MS- E T S 1 dash 1 Define the criteria and constraints of a design problem with sufficient precision to ensure a successful solution. Include potential impacts on people and the natural environment that may limit possible solutions. 6.MS- E T S 2 dash 1 (M A) Analyze and compare properties of metals, plastics, wood, and ceramics, including flexibility, ductility, hardness, thermal conductivity, electrical conductivity, and melting point. 6.MS- E T S 2 dash 2 (M A) Given a design task, select appropriate materials based on specific properties needed in the construction of a solution. 7.MS- E S S 3 dash 4 Construct an argument supported by evidence that human activities and technologies can mitigate the impact of increases in human population and per capita consumption of natural resources on the environment. 7.MS-LS2-4 Analyze data to provide evidence that disruptions (natural or human-made) to any physical or biological component of an ecosystem can lead to shifts in all its populations. 7.MS-LS2-5 Evaluate competing design solutions for protecting an ecosystem. Discuss benefits and limitations of each design. 7.MS-LS2-6( M A ) Explain how changes to the biodiversity of an ecosystem—the variety of species found in the ecosystem—may limit the availability of resources humans use. 8.MS- E S S 3 dash 1 Analyze and interpret data to explain that the Earth's mineral and fossil fuel resources are unevenly distributed as a result of geologic processes. 8.MS- E S S 3 dash 5 Examine and interpret data to describe the role that human activities have played in causing the rise in global temperatures over the past century.

MTEL 064 Test Objectives: III. Life Science	Curriculum Framework
0008 Demonstrate knowledge of the characteristics and processes of living organisms.	5.LS1-1 Ask testable questions about the process by which plants use air, water, and energy from sunlight to produce sugars and plant materials needed for growth and reproduction. 5.LS2-1 Develop a model to describe the movement of matter among producers, consumers, decomposers, and the air, water, and soil in the environment to (a) show that plants produce sugars and plant materials, (b) show that animals can eat plants and/or other animals for food, and (c) show that some organisms, including fungi and bacteria, break down dead organisms and recycle some materials back to the air and soil. 5.PS3-1 Use a model to describe that the food animals digest (a) contains energy that was once energy from the Sun, and (b) provides energy and nutrients for life processes, including body repair, growth, motion, body warmth, and reproduction. 6.MS-LS1-1 Provide evidence that all organisms (unicellular and multicellular) are made of cells. 6.MS-LS1-2 Develop and use a model to describe how parts of cells contribute to the cellular functions of obtaining food, water, and other nutrients from its environment, disposing of wastes, and providing energy for cellular processes. 6.MS-LS1-3 Construct an argument supported by evidence that the body systems interact to carry out essential functions of life. 6.MS- E T S 1 dash 5 (M A) Create visual representations of solutions to a design problem. Accurately interpret and apply scale and proportion to visual representations. 7.MS-LS1-4 Construct an explanation based on evidence for how characteristic animal behaviors and specialized plant structures increase the probability of successful reproduction of animals and plants. 7.MS-LS2-3 Develop a model to describe that matter and energy are transferred among living and nonliving parts of an ecosystem and that both matter and energy are conserved through these processes. 8.MS-LS1-5 Construct an argument based on evidence for how environmental and genetic factors influence the growth of organisms. 8.MS-LS1-7 Use informational text to describe that food molecules, including carbohydrates, proteins, and fats, are broken down and rearranged through chemical reactions forming new molecules that support cell growth and/or release of energy. 8.MS-LS3-4( M A ) Develop and use a model to show that sexually reproducing organisms have two of each chromosome in their cell nuclei, and hence two variants (alleles) of each gene that can be the same or different from each other, with one random assortment of each chromosome passed down to offspring from both parents. 8.MS- E T S 2 dash 4 (M A) Use informational text to illustrate that materials maintain their composition under various kinds of physical processing; however, some material properties may change if a process changes the particulate structure of a material.
0009 Apply knowledge of the characteristics of populations, communities, ecosystems, and biomes.	5.LS2-1 Develop a model to describe the movement of matter among producers, consumers, decomposers, and the air, water, and soil in the environment to (a) show that plants produce sugars and plant materials, (b) show that animals can eat plants and/or other animals for food, and (c) show that some organisms, including fungi and bacteria, break down dead organisms and recycle some materials back to the air and soil. 6.MS-LS4-1 Analyze and interpret evidence from the fossil record to describe organisms and their environment, extinctions, and changes to life forms throughout the history of Earth. 6.MS- E T S 1 dash 1 Define the criteria and constraints of a design problem with sufficient precision to ensure a successful solution. Include potential impacts on people and the natural environment that may limit possible solutions. 6.MS- E T S 1 dash 5 (M A) Create visual representations of solutions to a design problem. Accurately interpret and apply scale and proportion to visual representations. 7.MS-LS2-1 Analyze and interpret data to provide evidence for the effects of periods of abundant and scarce resources on the growth of organisms and the size of populations in an ecosystem. 7.MS-LS2-2 Describe how relationships among and between organisms in an ecosystem can be competitive, predatory, parasitic, and mutually beneficial and that these interactions are found across multiple ecosystems. 7.MS-LS2-3 Develop a model to describe that matter and energy are transferred among living and nonliving parts of an ecosystem and that both matter and energy are conserved through these processes. 7.MS-LS2-4 Analyze data to provide evidence that disruptions (natural or human-made) to any physical or biological component of an ecosystem can lead to shifts in all its populations. 7.MS-LS2-5 Evaluate competing design solutions for protecting an ecosystem. Discuss benefits and limitations of each design. 7.MS-LS2-6( M A ) Explain how changes to the biodiversity of an ecosystem—the variety of species found in the ecosystem—may limit the availability of resources humans use.
0010 Apply principles related to the inheritance of characteristics.	7.MS-LS1-4 Construct an explanation based on evidence for how characteristic animal behaviors and specialized plant structures increase the probability of successful reproduction of animals and plants. 8.MS-LS1-5 Construct an argument based on evidence for how environmental and genetic factors influence the growth of organisms. 8.MS-LS3-1 Develop and use a model to describe that structural changes to genes (mutations) may or may not result in changes to proteins, and if there are changes to proteins there may be harmful, beneficial, or neutral changes to traits. 8.MS-LS3-2 Construct an argument based on evidence for how asexual reproduction results in offspring with identical genetic information and sexual reproduction results in offspring with genetic variation. Compare and contrast advantages and disadvantages of asexual and sexual reproduction. 8.MS-LS3-3( M A ) Communicate through writing and in diagrams that chromosomes contain many distinct genes and that each gene holds the instructions for the production of specific proteins, which in turn affects the traits of an individual. 8.MS-LS3-4( M A ) Develop and use a model to show that sexually reproducing organisms have two of each chromosome in their cell nuclei, and hence two variants (alleles) of each gene that can be the same or different from each other, with one random assortment of each chromosome passed down to offspring from both parents. 8.MS-LS4-4 Use a model to describe the process of natural selection, in which genetic variations of some traits in a population increase some individuals' likelihood of surviving and reproducing in a changing environment. Provide evidence that natural selection occurs over many generations.
0011 Demonstrate knowledge of principles related to the theory of biological evolution.	5.3-5- E T S 3 dash 1 (M A) Use informational text to provide examples of improvements to existing technologies (innovations) and the development of new technologies (inventions). Recognize that technology is any modification of the natural or designed world done to fulfill human needs or wants. 6.MS-LS4-1 Analyze and interpret evidence from the fossil record to describe organisms and their environment, extinctions, and changes to life forms throughout the history of Earth. 6.MS-LS4-2 Construct an argument using anatomical structures to support evolutionary relationships among and between fossil organisms and modern organisms. 8.MS-LS1-5 Construct an argument based on evidence for how environmental and genetic factors influence the growth of organisms. 8.MS-LS3-1 Develop and use a model to describe that structural changes to genes (mutations) may or may not result in changes to proteins, and if there are changes to proteins there may be harmful, beneficial, or neutral changes to traits. 8.MS-LS4-4 Use a model to describe the process of natural selection, in which genetic variations of some traits in a population increase some individuals' likelihood of surviving and reproducing in a changing environment. Provide evidence that natural selection occurs over many generations. 8.MS-LS4-5 Synthesize and communicate information about artificial selection, or the ways in which humans have changed the inheritance of desired traits in organisms.

MTEL 064 Test Objectives: IV. Physical Science	Curriculum Framework
0012 Apply knowledge of the structure and properties of matter.	5-PS1-2 Measure and graph the weights (masses) of substances before and after a reaction or phase change to provide evidence that regardless of the type of change that occurs when heating, cooling, or combining substances, the total weight (mass) of matter is conserved. 5-PS1-3 Make observations and measurements of substances to describe characteristic properties of each, including color, hardness, reflectivity, electrical conductivity, thermal conductivity, response to magnetic forces, and solubility. 5-PS1-4 Conduct an experiment to determine whether the mixing of two or more substances results in new substances with new properties (a chemical reaction) or not (a mixture). 5- E S S 3 dash 2 (M A) Test a simple system designed to filter particulates out of water and propose one change to the design to improve it. 6.MS-PS1-8( M A ) Conduct an experiment to show that many materials are mixtures of pure substances that can be separated by physical means into their component pure substances. 6.MS- E T S 2 dash 1 (M A) Analyze and compare properties of metals, plastics, wood, and ceramics, including flexibility, ductility, hardness, thermal conductivity, electrical conductivity, and melting point. 7.MS-LS2-3 Develop a model to describe that matter and energy are transferred among living and nonliving parts of an ecosystem and that both matter and energy are conserved through these processes. 8.MS-LS1-7 Use informational text to describe that food molecules, including carbohydrates, proteins, and fats, are broken down and rearranged through chemical reactions forming new molecules that support cell growth and/or release of energy. 8.MS-PS1-1 Develop a model to describe that (a) atoms combine in a multitude of ways to produce pure substances which make up all of the living and nonliving things that we encounter, (b) atoms form molecules and compounds that range in size from two to thousands of atoms, and (c) mixtures are composed of different proportions of pure substances. 8.MS-PS1-2 Analyze and interpret data on the properties of substances before and after the substances interact to determine if a chemical reaction has occurred. 8.MS-PS1-5 Use a model to explain that atoms are rearranged during a chemical reaction to form new substances with new properties. Explain that the atoms present in the reactants are all present in the products and thus the total number of atoms is conserved. 8.MS- E T S 2 dash 4 (M A) Use informational text to illustrate that materials maintain their composition under various kinds of physical processing; however, some material properties may change if a process changes the particulate structure of a material.
0013 Apply knowledge of the states of matter, particle motion, and heat.	5.PS1-1 Use a particle model of matter to explain common phenomena involving gases, and phase changes between gas and liquid and between liquid and solid. 6.MS-PS1-6 Plan and conduct an experiment involving exothermic and endothermic chemical reactions to measure and describe the release or absorption of thermal energy. 6.MS-PS1-7( M A ) Use a particulate model of matter to explain that density is the amount of matter (mass) in a given volume. Apply proportional reasoning to describe, calculate, and compare relative densities of different materials. 6.MS- E T S 2 dash 1 (M A) Analyze and compare properties of metals, plastics, wood, and ceramics, including flexibility, ductility, hardness, thermal conductivity, electrical conductivity, and melting point. 6.MS- E T S 2 dash 2 (M A) Given a design task, select appropriate materials based on specific properties needed in the construction of a solution. 7.MS- E S S 2 dash 4 Develop a model to explain how the energy of the Sun and Earth's gravity drive the cycling of water, including changes of state, as it moves through multiple pathways in Earth's hydrosphere. 7.MS-PS3-3 Apply scientific principles of energy and heat transfer to design, construct, and test a device to minimize or maximize thermal energy transfer. 7.MS-PS3-4 Conduct an investigation to determine the relationships among the energy transferred, how well the type of matter retains or radiates heat, the mass, and the change in the average kinetic energy of the particles as measured by the temperature of the sample. 7.MS-PS3-6( M A ) Use a model to explain how thermal energy is transferred out of hotter regions or objects and into colder ones by convection, conduction, and radiation. 8.MS- E S S 2 dash 1 Use a model to illustrate that energy from Earth's interior drives convection that cycles Earth's crust, leading to melting, crystallization, weathering, and deformation of large rock formations, including generation of ocean sea floor at ridges, submergence of ocean sea floor at trenches, mountain building, and active volcanic chains. 8.MS- E S S 2 dash 6 Describe how interactions involving the ocean affect weather and climate on a regional scale, including the influence of the ocean temperature as mediated by energy input from the Sun and energy loss due to evaporation or redistribution via ocean currents. 8.MS-PS1-2 Analyze and interpret data on the properties of substances before and after the substances interact to determine if a chemical reaction has occurred. 8.MS-PS1-4 Develop a model that describes and predicts changes in particle motion, relative spatial arrangement, temperature, and state of a pure substance when thermal energy is added or removed.
0014 Apply knowledge of different forms of energy and the conservation of energy.	6.MS-PS1-6 Plan and conduct an experiment involving exothermic and endothermic chemical reactions to measure and describe the release or absorption of thermal energy. 7.MS- E S S 2 dash 4 Develop a model to explain how the energy of the Sun and Earth's gravity drive the cycling of water, including changes of state, as it moves through multiple pathways in Earth's hydrosphere. 7.MS-LS2-3 Develop a model to describe that matter and energy are transferred among living and nonliving parts of an ecosystem and that both matter and energy are conserved through these processes. 7.MS-PS3-1 Construct and interpret data and graphs to describe the relationships among kinetic energy, mass, and speed of an object. 7.MS-PS3-2 Develop a model to describe the relationship between the relative positions of objects interacting at a distance and their relative potential energy in the system. 7.MS-PS3-3 Apply scientific principles of energy and heat transfer to design, construct, and test a device to minimize or maximize thermal energy transfer. 7.MS-PS3-4 Conduct an investigation to determine the relationships among the energy transferred, how well the type of matter retains or radiates heat, the mass, and the change in the average kinetic energy of the particles as measured by the temperature of the sample. 7.MS-PS3-5 Present evidence to support the claim that when the kinetic energy of an object changes, energy is transferred to or from the object. 7.MS-PS3-6( M A ) Use a model to explain how thermal energy is transferred out of hotter regions or objects and into colder ones by convection, conduction, and radiation. 7.MS-PS3-7( M A ) Use informational text to describe the relationship between kinetic and potential energy and illustrate conversions from one form to another. 8.MS- E S S 2 dash 1 Use a model to illustrate that energy from Earth's interior drives convection that cycles Earth's crust, leading to melting, crystallization, weathering, and deformation of large rock formations, including generation of ocean sea floor at ridges, submergence of ocean sea floor at trenches, mountain building, and active volcanic chains.
0015 Apply knowledge of the concepts of force, motion, work, and power.	5.PS2-1 Support an argument with evidence that the gravitational force exerted by Earth on objects is directed toward Earth's center. 5.3-5- E T S 3 dash 2 (M A) Use sketches or drawings to show how each part of a product or device relates to other parts in the product or device. 6.MS-PS2-4 Use evidence to support the claim that gravitational forces between objects are attractive and are only noticeable when one or both of the objects have a very large mass. 7.MS-PS2-3 Analyze data to describe the effect of distance and magnitude of electric charge on the strength of electric forces. 7.MS-PS2-5 Use scientific evidence to argue that fields exist between objects with mass, between magnetic objects, and between electrically charged objects that exert force on each other even though the objects are not in contact. 7.MS-PS3-1 Construct and interpret data and graphs to describe the relationships among kinetic energy, mass, and speed of an object. 8.MS-PS2-1 Develop a model that demonstrates Newton's third law involving the motion of two colliding objects. 8.MS-PS2-2 Provide evidence that the change in an object's speed depends on the sum of the forces on the object (the net force) and the mass of the object.
0016 Apply knowledge of the characteristics and properties of waves, electricity, magnetism, and electromagnetism.	5-PS1-3 Make observations and measurements of substances to describe characteristic properties of each, including color, hardness, reflectivity, electrical conductivity, thermal conductivity, response to magnetic forces, and solubility. 5.3-5- E T S 3 dash 1 (M A) Use informational text to provide examples of improvements to existing technologies (innovations) and the development of new technologies (inventions). Recognize that technology is any modification of the natural or designed world done to fulfill human needs or wants. 5.3-5- E T S 3 dash 2 (M A) Use sketches or drawings to show how each part of a product or device relates to other parts in the product or device. 6.MS-PS4-1 Use diagrams of a simple wave to explain that (a) a wave has a repeating pattern with a specific amplitude, frequency, and wavelength, and (b) the amplitude of a wave is related to the energy of the wave. 6.MS-PS4-2 Use diagrams and other models to show that both light rays and mechanical waves are reflected, absorbed, or transmitted through various materials. 6.MS-PS4-3 Present qualitative scientific and technical information to support the claim that digitized signals (sent as wave pulses representing 0s and 1s) can be used to encode and transmit information. 7.MS-PS2-3 Analyze data to describe the effect of distance and magnitude of electric charge on the strength of electric forces. 7.MS-PS2-5 Use scientific evidence to argue that fields exist between objects with mass, between magnetic objects, and between electrically charged objects that exert force on each other even though the objects are not in contact.

MTEL 064 Test Objectives:
V. Integration of Knowledge and Understanding

Curriculum Framework

0017 Prepare an organized, developed analysis of a topic related to key concepts in Earth and Space Science, Life Science, or Physical Science.

0018 Prepare an organized, developed analysis of a topic related to key concepts in Earth and Space Science, Life Science, or Physical Science that emphasizes the application of the principles of science practices.

5- E S S 1 dash 1 Use observations, first–hand and from various media, to argue that the Sun is a star that appears larger and brighter than other stars because it is closer to Earth.

5- E S S 1 dash 2 Use a model to communicate Earth's relationship to the Sun, Moon, and other stars that explain (a) why people on Earth experience day and night, (b) patterns in daily changes in length and direction of shadows over a day, and (c) changes in the apparent position of the Sun, Moon, and stars at different times during a day, over a month, and over a year.

5- E S S 2 dash 1 Use a model to describe the cycling of water through a watershed through evaporation, precipitation, absorption, surface runoff, and condensation.

5- E S S 2 dash 2 Describe and graph the relative amounts of salt water in the ocean; fresh water in lakes, rivers, and groundwater; and fresh water frozen in glaciers and polar ice caps to provide evidence about the availability of fresh water in Earth's biosphere.

5- E S S 3 dash 1 Obtain and combine information about ways communities reduce human impact on the Earth's resources and environment by changing an agricultural, industrial, or community practice or process.

5- E S S 3 dash 2 (M A) Test a simple system designed to filter particulates out of water and propose one change to the design to improve it.

5.LS1-1 Ask testable questions about the process by which plants use air, water, and energy from sunlight to produce sugars and plant materials needed for growth and reproduction.

5.LS2-1 Develop a model to describe the movement of matter among producers, consumers, decomposers, and the air, water, and soil in the environment to (a) show that plants produce sugars and plant materials, (b) show that animals can eat plants and/or other animals for food, and (c) show that some organisms, including fungi and bacteria, break down dead organisms and recycle some materials back to the air and soil.

5-LS2-2( M A ) Compare at least two designs for a composter to determine which is most likely to encourage decomposition of materials.

5.3-5- E T S 3 dash 1 (M A) Use informational text to provide examples of improvements to existing technologies (innovations) and the development of new technologies (inventions). Recognize that technology is any modification of the natural or designed world done to fulfill human needs or wants.

5.PS3-1 Use a model to describe that the food animals digest (a) contains energy that was once energy from the Sun, and (b) provides energy and nutrients for life processes, including body repair, growth, motion, body warmth, and reproduction.

6.MS- E S S 1 dash 1 A Develop and use a model of the Earth–Sun–Moon system to explain the causes of lunar phases and eclipses of the Sun and Moon.

6.MS- E S S 1 dash 4 Analyze and interpret rock layers and index fossils to determine the relative ages of rock formations that result from processes occurring over long periods of time.

6.MS- E S S 1 dash 5 (M A) Use graphical displays to illustrate that Earth and its solar system are one of many in the Milky Way galaxy, which is one of billions of galaxies in the universe.

6.MS- E S S 2 dash 3 Analyze and interpret maps showing the distribution of fossils and rocks, continental shapes, and seafloor structures to provide evidence that Earth's plates have moved great distances, collided, and spread apart.

6.MS-LS1-1 Provide evidence that all organisms (unicellular and multicellular) are made of cells.

6.MS-LS1-3 Construct an argument supported by evidence that the body systems interact to carry out essential functions of life.

6.MS-LS4-1 Analyze and interpret evidence from the fossil record to describe organisms and their environment, extinctions, and changes to life forms throughout the history of Earth.

6.MS-LS4-2 Construct an argument using anatomical structures to support evolutionary relationships among and between fossil organisms and modern organisms.

6.MS- E T S 1 dash 1 Define the criteria and constraints of a design problem with sufficient precision to ensure a successful solution. Include potential impacts on people and the natural environment that may limit possible solutions.

6.MS- E T S 1 dash 5 (M A) Create visual representations of solutions to a design problem. Accurately interpret and apply scale and proportion to visual representations.

6.MS- E T S 1 dash 6 (M A) Communicate a design solution to an intended user, including design features and limitations of the solution.

6.MS- E T S 2 dash 1 (M A) Analyze and compare properties of metals, plastics, wood, and ceramics, including flexibility, ductility, hardness, thermal conductivity, electrical conductivity, and melting point.

6.MS- E T S 2 dash 2 (M A) Given a design task, select appropriate materials based on specific properties needed in the construction of a solution.

7.MS- E S S 2 dash 2 Construct an explanation based on evidence for how Earth's surface has changed over scales that range from local to global in size.

7.MS- E S S 2 dash 4 Develop a model to explain how the energy of the Sun and Earth's gravity drive the cycling of water, including changes of state, as it moves through multiple pathways in Earth's hydrosphere.

7.MS- E S S 3 dash 2 Obtain and communicate information on how data from past geologic events are analyzed for patterns and used to forecast the location and likelihood of future catastrophic events.

7.MS- E S S 3 dash 4 Construct an argument supported by evidence that human activities and technologies can mitigate the impact of increases in human population and per capita consumption of natural resources on the environment.

7.MS-LS1-4 Construct an explanation based on evidence for how characteristic animal behaviors and specialized plant structures increase the probability of successful reproduction of animals and plants.

7.MS-LS2-1 Analyze and interpret data to provide evidence for the effects of periods of abundant and scarce resources on the growth of organisms and the size of populations in an ecosystem.

7.MS-LS2-2 Describe how relationships among and between organisms in an ecosystem can be competitive, predatory, parasitic, and mutually beneficial and that these interactions are found across multiple ecosystems.

7.MS-LS2-3 Develop a model to describe that matter and energy are transferred among living and nonliving parts of an ecosystem and that both matter and energy are conserved through these processes.

7.MS-LS2-4 Analyze data to provide evidence that disruptions (natural or human-made) to any physical or biological component of an ecosystem can lead to shifts in all its populations.

7.MS-LS2-5 Evaluate competing design solutions for protecting an ecosystem. Discuss benefits and limitations of each design.

7.MS-LS2-6( M A ) Explain how changes to the biodiversity of an ecosystem—the variety of species found in the ecosystem—may limit the availability of resources humans use.

8.MS- E S S 1 dash 1 b Develop and use a model of the Earth-Sun system to explain the cyclical pattern of seasons, which includes Earth's tilt and differential intensity of sunlight on different areas of Earth across the year.

8.MS- E S S 1 dash 2 Explain the role of gravity in ocean tides, the orbital motions of planets, their moons, and asteroids in the solar system.

8.MS- E S S 2 dash 1 Use a model to illustrate that energy from Earth's interior drives convection that cycles Earth's crust, leading to melting, crystallization, weathering, and deformation of large rock formations, including generation of ocean sea floor at ridges, submergence of ocean sea floor at trenches, mountain building, and active volcanic chains.

8.MS- E S S 2 dash 5 Interpret basic weather data to identify patterns in air mass interactions and the relationship of those patterns to local weather.

8.MS- E S S 2 dash 6 Describe how interactions involving the ocean affect weather and climate on a regional scale, including the influence of the ocean temperature as mediated by energy input from the Sun and energy loss due to evaporation or redistribution via ocean currents.

8.MS- E S S 3 dash 1 Analyze and interpret data to explain that the Earth's mineral and fossil fuel resources are unevenly distributed as a result of geologic processes.

8.MS- E S S 3 dash 5 Examine and interpret data to describe the role that human activities have played in causing the rise in global temperatures over the past century.

8.MS-LS1-5 Construct an argument based on evidence for how environmental and genetic factors influence the growth of organisms.

8.MS-LS1-7 Use informational text to describe that food molecules, including carbohydrates, proteins, and fats, are broken down and rearranged through chemical reactions forming new molecules that support cell growth and/or release of energy.

8.MS-LS3-1 Develop and use a model to describe that structural changes to genes (mutations) may or may not result in changes to proteins, and if there are changes to proteins there may be harmful, beneficial, or neutral changes to traits.

8.MS-LS3-2 Construct an argument based on evidence for how asexual reproduction results in offspring with identical genetic information and sexual reproduction results in offspring with genetic variation. Compare and contrast advantages and disadvantages of asexual and sexual reproduction.

8.MS-LS3-3( M A ) Communicate through writing and in diagrams that chromosomes contain many distinct genes and that each gene holds the instructions for the production of specific proteins, which in turn affects the traits of an individual.

8.MS-LS3-4( M A ) Develop and use a model to show that sexually reproducing organisms have two of each chromosome in their cell nuclei, and hence two variants (alleles) of each gene that can be the same or different from each other, with one random assortment of each chromosome passed down to offspring from both parents.

8.MS-LS4-4 Use a model to describe the process of natural selection, in which genetic variations of some traits in a population increase some individuals' likelihood of surviving and reproducing in a changing environment. Provide evidence that natural selection occurs over many generations.

8.MS-LS4-5 Synthesize and communicate information about artificial selection, or the ways in which humans have changed the inheritance of desired traits in organisms.

5.PS1-1 Use a particle model of matter to explain common phenomena involving gases, and phase changes between gas and liquid and between liquid and solid.

5-PS1-2 Measure and graph the weights (masses) of substances before and after a reaction or phase change to provide evidence that regardless of the type of change that occurs when heating, cooling, or combining substances, the total weight (mass) of matter is conserved.

5-PS1-4 Conduct an experiment to determine whether the mixing of two or more substances results in new substances with new properties (a chemical reaction) or not (a mixture).

5.PS2-1 Support an argument with evidence that the gravitational force exerted by Earth on objects is directed toward Earth's center.

5.3-5- E T S 3 dash 2 (M A) Use sketches or drawings to show how each part of a product or device relates to other parts in the product or device.

6.MS-LS4-1 Analyze and interpret evidence from the fossil record to describe organisms and their environment, extinctions, and changes to life forms throughout the history of Earth.

6.MS-LS4-2 Construct an argument using anatomical structures to support evolutionary relationships among and between fossil organisms and modern organisms.

6.MS-PS1-6 Plan and conduct an experiment involving exothermic and endothermic chemical reactions to measure and describe the release or absorption of thermal energy.

6.MS-PS1-7( M A ) Use a particulate model of matter to explain that density is the amount of matter (mass) in a given volume. Apply proportional reasoning to describe, calculate, and compare relative densities of different materials.

6.MS-PS1-8( M A ) Conduct an experiment to show that many materials are mixtures of pure substances that can be separated by physical means into their component pure substances.

6.MS-PS2-4 Use evidence to support the claim that gravitational forces between objects are attractive and are only noticeable when one or both of the objects have a very large mass.

6.MS-PS4-1 Use diagrams of a simple wave to explain that (a) a wave has a repeating pattern with a specific amplitude, frequency, and wavelength, and (b) the amplitude of a wave is related to the energy of the wave.

6.MS-PS4-3 Present qualitative scientific and technical information to support the claim that digitized signals (sent as wave pulses representing 0s and 1s) can be used to encode and transmit information.

6.MS- E T S 1 dash 5 (M A) Create visual representations of solutions to a design problem. Accurately interpret and apply scale and proportion to visual representations.

6.MS- E T S 1 dash 6 (M A) Communicate a design solution to an intended user, including design features and limitations of the solution.

6.MS- E T S 2 dash 1 (M A) Analyze and compare properties of metals, plastics, wood, and ceramics, including flexibility, ductility, hardness, thermal conductivity, electrical conductivity, and melting point.

6.MS- E T S 2 dash 2 (M A) Given a design task, select appropriate materials based on specific properties needed in the construction of a solution.

6.MS- E T S 2 dash 3 (M A) Choose and safely use appropriate measuring tools, hand tools, fasteners, and common hand-held power tools used to construct a prototype.

7.MS-PS2-3 Analyze data to describe the effect of distance and magnitude of electric charge on the strength of electric forces.

7.MS-PS2-5 Use scientific evidence to argue that fields exist between objects with mass, between magnetic objects, and between electrically charged objects that exert force on each other even though the objects are not in contact.

7.MS-PS3-1 Construct and interpret data and graphs to describe the relationships among kinetic energy, mass, and speed of an object.

7.MS-PS3-2 Develop a model to describe the relationship between the relative positions of objects interacting at a distance and their relative potential energy in the system.

7.MS-PS3-3 Apply scientific principles of energy and heat transfer to design, construct, and test a device to minimize or maximize thermal energy transfer.

7.MS-PS3-4 Conduct an investigation to determine the relationships among the energy transferred, how well the type of matter retains or radiates heat, the mass, and the change in the average kinetic energy of the particles as measured by the temperature of the sample.

7.MS-PS3-5 Present evidence to support the claim that when the kinetic energy of an object changes, energy is transferred to or from the object.

7.MS-PS3-6( M A ) Use a model to explain how thermal energy is transferred out of hotter regions or objects and into colder ones by convection, conduction, and radiation.

7.MS-PS3-7( M A ) Use informational text to describe the relationship between kinetic and potential energy and illustrate conversions from one form to another.

8.MS-PS1-1 Develop a model to describe that (a) atoms combine in a multitude of ways to produce pure substances which make up all of the living and nonliving things that we encounter, (b) atoms form molecules and compounds that range in size from two to thousands of atoms, and (c) mixtures are composed of different proportions of pure substances.

8.MS-PS1-2 Analyze and interpret data on the properties of substances before and after the substances interact to determine if a chemical reaction has occurred.

8.MS-PS1-4 Develop a model that describes and predicts changes in particle motion, relative spatial arrangement, temperature, and state of a pure substance when thermal energy is added or removed.

8.PS1-5 Use a model to explain that atoms are rearranged during a chemical reaction to form new substances with new properties. Explain that the atoms present in the reactants are all present in the products and thus the total number of atoms is conserved.

8.MS-PS2-1 Develop a model that demonstrates Newton's third law involving the motion of two colliding objects.

8.MS-PS2-2 Provide evidence that the change in an object's speed depends on the sum of the forces on the object (the net force) and the mass of the object.

General Science

Committee participants	Number
Total	4

Gender	Number
Female	3
Male	1
No response	0

Ethnicity	Number
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	0
Hispanic	0
White	4
Other	0
Unspecified	0

English as a Second Language

INTACT CLASSROOM SESSIONS 2013–2015
10/01/2013	Lincoln School (Brookline Public Schools)
10/05/2013	Simmons College Gordon College Salem State University
10/08/2013	Wheaton College
10/18/2013	University of Massachusetts - Boston
10/31/2013	Wheelock College
01/30/2014	Wheaton College
02/01/2014	Gordon College
02/04/2014	Elms College
02/05/2014	Salem State University
02/06/2014	University of Massachusetts - Amherst
02/11/2014	Lincoln School (Brookline Public Schools)
02/14/2014	Wheelock College
02/19/2014	Worcester State University
04/25/2015	Cambridge College
04/27/2015	Salem State University
05/02/2015	Simmons College Framingham State University
05/05/2015	Westfield State University
05/09/2015	Gordon College

STAND ALONE EVENTS AT MTEL OPERATIONAL TEST ADMINISTRATIONS

10/26/2013
03/01/2014
05/10/2014
07/12/2014
10/25/2014

Pilot Test Outcomes
General Science

Pilot testing was conducted at CBT centers from November 15, 2021, through January 5, 2022. The following information is a summary of the data collected during this specific time frame.

Multiple-Choice Items ( M C Q's ) and Short-Answer Items ( S A's )

Number of M C Q's Prepared	163
Number of M C Q's Piloted*	66
Number of M C Q's Identified for Further Review	15
Number of M C Q's Deleted Post Pilot Test Review	0

Open-Response Items ( O R I's )

Number of O R I's Developed	22
Number of O R I's Piloted*	8
Number of O R I's Identified for Further Review	0
Number of O R I's Deleted Post Pilot Test Review	0

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms until all items are piloted.

General Science

Number of Qualifying Score Participants: 4

Number of Participants Providing Ratings: 4 (100%)

Question Rating Scale: 1–5 1=not at all (well/confident/satisfied/successful), 5=very (well/confident/satisfied/successful)	Median Rating
How well do you feel you understood the training that you received for making your qualifying score judgments?	5
How confident are you of the qualifying score judgments you made?	5
How satisfied are you that you were able to complete the work you were asked to do in the time that was provided?	5
How successful were the coordination and logistics of the qualifying score conference?	5
Overall, how satisfied are you with the qualifying score process in which you participated?	5

General Science Content Validation Survey Population/Sample/Respondents Demographics (Public School Sample)

Key

Population: The number and percent of licensed and assigned educators in the content area, extracted from the database of full-time equivalent (FTE) teachers assigned to each teaching area for each school district in Massachusetts, as provided by the Massachusetts Department of Elementary and Secondary Education for use in drawing a random sample of educators.
Sample (with oversampling): The number and percent of licensed and assigned educators sampled to receive the survey. In order to obtain appropriate representation of minority groups (e.g., American Indian, Asian, Black, and Hispanic) in the survey results, oversampling of minority groups was conducted for those fields for which the entire population was not sampled. In these instances, the survey results were weighted appropriately to take the oversampling into account.

General Science	Population		Sample (with oversampling)
General Science	N	%	N	%
Total	2,269	100	214	100
Female	1,460	64.34	138	64.48
Male	809	3.92	76	35.51
American Indian or Alaska Native	1	0.04	0	0
Black (not of Hispanic origin)	52	2.29	10	4.67
Asian or Pacific Islander	51	2.24	9	4.2
Hispanic	48	2.11	8	3.73
White	2,106	92.81	185	86.44
Other	12	0.52	2	0.93

General Science Survey Return Rate by Field and Return Status: Public School Sample

Returned Surveys

Field	Number Sent	Number Not Distributed	Eligible		Ineligible		Total		Adjusted Return Rate
Field	Number Sent	Number Not Distributed	N	%	N	%	N	%	Adjusted Return Rate
061 English	541	30	191	35.30%	32	5.90%	223	43.60%	39.90%
063 Mathematics	609	28	205	33.70%	53	8.70%	258	44.40%	38.80%
064 General Science	530	15	154	29.10%	58	10.90%	212	41.20%	33.70%
065 Middle School Mathematics	581	14	160	27.50%	78	13.40%	238	42.00%	32.70%
190 Foundations of Reading	498	18	166	33.30%	41	8.20%	207	43.10%	37.80%
ALL	2,759	105	876	31.80%	262	9.50%	1,138	42.90%	36.60%

General Science Survey Return Rate by Field and Return Status: Faculty Sample

Returned Surveys

Field	Number Sent	Number Not Distributed	Eligible		Ineligible		Total		Adjusted Return Rate
Field	Number Sent	Number Not Distributed	N	%	N	%	N	%	Adjusted Return Rate
061 English	127	9	18	14.20%	1	0.80%	19	16.10%	15.40%
063 Mathematics	129	8	24	18.60%	0	0.00%	24	19.80%	19.80%
064 General Science	106	9	9	8.50%	2	1.90%	11	11.30%	9.50%
065 Middle School Mathematics	121	9	21	17.40%	3	2.50%	24	21.40%	19.30%
190 Foundations of Reading	138	5	33	23.90%	0	0.00%	33	24.80%	24.80%
ALL	621	40	105	16.90%	6	1.00%	111	19.10%	18.30%

Demographic Summary Report: Public School Sample

Teacher Sample
Field: General Science

Question	Answer	Absolute Frequency	Adjusted Percent
Currently Teaching	Yes	154	100
Currently Teaching	No	0	0
Hold a valid Massachusetts teaching license	Yes	154	100
Hold a valid Massachusetts teaching license	No	0	0
What is the highest level of education you have attained?	Bachelor's degree	16	10.4
	Master's degree	127	82.5
	Doctoral degree	6	3.9
	Other	5	3.2
What is your ethnicity?	American Indian or Alaska Native	0	0
	Black (not of Hispanic origin)	6	3.9
	Asian or Pacific Islander	1	0.7
	Hispanic	4	2.6
	White (not of Hispanic origin)	139	91.4
	Other	2	1.3
What is your first (native) language?	English	148	96.7
	French	0	0
	Portuguese	1	0.7
	An Asian language	0	0
	Russian	1	0.7
	Spanish	2	1.3
	Other	1	0.7
What is your gender?	Female	86	57
What is your gender?	Male	65	43
What grade level(s) have you taught in Massachusetts during this or the previous school year? (Select all that apply.)	Pre K to 6th	15	8.4
	5th to 8th	129	72.1
	5th to 12th	17	9.5
	8th to 12th	18	10.1
How many years of professional teaching experience do you have? (Count partial years as full years.)	1 to 3 years	4	2.6
	4 to 10 years	36	23.4
	11 years or more	114	74

General Science Demographic Summary Report: Faculty Sample

Faculty Sample
Field: General Science

Question	Answer	Absolute Frequency	Adjusted Percent
Teaching undergraduate or graduate arts and science or education courses in which prospective teachers may have been enrolled	Yes	9	100
	No	0	0
What is the highest level of education you have attained?	Bachelor's degree	0	0
	Master's degree	1	11.1
	Doctoral degree	8	88.9
	Other	0	0
What is your ethnicity?	American Indian or Alaska Native	0	0
	Black (not of Hispanic origin)	0	0
	Asian or Pacific Islander	1	11.1
	Hispanic	0	0
	White (not of Hispanic origin)	8	88.9
	Other	0	0
What is your first (native) language?	English	8	88.9
	Spanish	0	0
	Other	1	11.1
What is your gender?	Female	4	44.4
What is your gender?	Male	5	55.6
How many years have you taught as a faculty member at a higher education institution? (Count partial years as full years.)	1 to 3 years	0	0
	4 to 10 years	4	44.4
	11 years or more	5	55.6
In which higher education institution department do you currently hold your primary appointment?	Education department only	3	33.3
	Both the Education department and another department	1	11.1
	Arts and Sciences or Fine Arts department(s) only	5	55.6
Have you ever taught in the subject of the field and grade level indicated above in Massachusetts?	Yes	4	44.4
	No	5	55.6
How recently have you taught in the subject of the field and grade level indicated above in Massachusetts? (Count partial years as full years.)	1 to 3 years	1	25
	4 to 10 years	1	25
	11 years or more	2	50

General Science Objective Rating Report: Public School Sample

Number of Objectives: 18

Importance Ratings

Objective Number	N	Mean	SD	S E
1	151	3.64	0.87	0.07
2	152	3.19	0.87	0.07
3	152	3.41	0.87	0.07
4	151	4.11	0.67	0.05
5	150	4.06	0.71	0.06
6	152	4.2	0.71	0.06
7	152	4.2	0.76	0.06
8	154	4.33	0.64	0.05
9	153	4.03	0.76	0.06
10	152	4.13	0.76	0.06
11	152	4.03	0.78	0.06
12	153	4.32	0.68	0.06
13	151	4.23	0.69	0.06
14	154	4.08	0.78	0.06
15	154	4.01	0.76	0.06
16	153	3.73	0.76	0.06
17	153	3.94	0.85	0.07
18	154	3.95	0.81	0.07

Number of Respondents: 154

Response Distribution (in %)

NR	1	2	3	4	5
2	0	8	36	36	18
1	2	17	47	25	7
1	1	9	49	27	13
2	0	1	16	55	27
3	0	2	16	55	25
1	0	1	13	49	35
1	0	1	19	40	40
0	0	0	9	49	42
1	0	2	21	49	28
1	0	1	21	42	35
1	0	2	22	45	29
1	0	1	8	47	43
2	0	1	13	48	36
0	0	3	19	46	32
0	0	1	24	47	28
1	0	1	42	39	18
1	1	3	25	42	28
0	0	3	27	43	27

General Science Objective Rating Report: Faculty Sample

Objective Rating Report: Ordered by Objective Number
Unweighted

Faculty Sample
Field: General Science

Number of Objectives: 18

Importance Ratings

Objective Number	N	Mean	SD	S E
1	9	4	0.71	0.24
2	9	3	0.71	0.24
3	9	4	0.87	0.29
4	9	4.11	0.6	0.2
5	9	4.11	0.78	0.26
6	9	4.56	0.73	0.24
7	9	4.67	0.5	0.17
8	9	4.78	0.44	0.15
9	9	4.22	0.67	0.22
10	9	4.33	0.71	0.24
11	9	4.33	0.1	0.33
12	9	4.33	0.71	0.24
13	9	4	0.87	0.29
14	9	4.33	0.87	0.29
15	9	4.11	0.93	0.31
16	9	3.89	0.93	0.11
17	9	4.56	0.53	0.18
18	9	4.44	0.73	0.24

Number of Respondents: 9

Response Distribution (in %)

2	3	4	5
0	22	56	22
22	56	22	0
0	33	33	33
0	11	67	22
0	22	44	33
0	11	22	67
0	0	33	67
0	0	22	78
0	11	56	33
0	11	44	44
0	33	0	67
0	11	44	44
0	33	33	33
0	22	22	56
0	33	22	44
0	44	22	33
0	0	44	56
0	11	33	56

General Science Descriptive Statement Rating Report: Public School Sample

Descriptive Statement Rating Report: Ordered by Objective Number
Unweighted

Number of Objectives: 18

Importance Ratings

Objective Number	N	Mean	SD	S E
1	154	3.82	0.85	0.07
2	146	3.57	0.91	0.08
3	150	3.59	0.86	0.07
4	152	4.12	0.75	0.06
5	153	4.15	0.72	0.06
6	151	4.13	0.71	0.06
7	152	4.1	0.72	0.06
8	150	4.2	0.74	0.06
9	152	4.18	0.69	0.06
10	152	4.06	0.7	0.06
11	153	3.95	0.79	0.06
12	153	4.1	0.71	0.06
13	150	4.09	0.68	0.06
14	152	3.99	0.75	0.06
15	150	3.87	0.76	0.06
16	153	3.82	0.81	0.07
17	151	3.5	0.94	0.08
18	152	3.78	0.84	0.07

Number of Respondents: 154

Response Distribution (in %)

NR	1	2	3	4	5
0	1	5	27	45	21
5	2	6	39	32	16
3	2	4	40	37	14
1	1	1	14	52	31
1	0	2	14	51	32
2	0	1	16	51	31
1	0	1	17	51	29
3	0	1	17	42	38
1	0	0	16	48	34
1	0	1	19	52	27
1	0	3	25	45	26
1	0	1	17	52	29
3	0	0	18	52	27
1	1	1	22	51	24
3	0	3	27	47	20
1	0	5	28	46	20
2	2	11	35	36	14
1	0	5	34	39	21

General Science Descriptive Statement Rating Report: Faculty Sample

Descriptive Statement Rating Report: Ordered by Objective Number
Unweighted

Faculty Sample
Field: General Science

Number of Objectives: 18

Importance Ratings

Objective Number	N	Mean	SD	S E
1	9	4.56	0.53	0.18
2	9	3.89	1.05	0.35
3	9	4.11	0.6	0.2
4	9	4.22	0.44	0.15
5	9	4	0.87	0.29
6	9	4.44	0.53	0.18
7	9	4.33	0.71	0.24
8	9	4.11	0.93	0.31
9	9	4.22	0.44	0.15
10	9	4.33	0.5	0.17
11	9	4.33	0.71	0.24
12	9	4.11	0.6	0.2
13	9	4.22	0.67	0.22
14	9	3.44	0.88	0.29
15	9	4.11	0.33	0.11
16	9	4.44	0.53	0.18
17	9	3.22	1.09	0.36
18	9	3.44	1.13	0.38

Number of Respondents: 9

Response Distribution (in %)

1	2	3	4	5
0	0	0	44	56
0	11	22	33	33
0	0	11	67	22
0	0	0	78	22
0	11	0	67	22
0	0	0	56	44
0	0	11	44	44
0	11	0	56	33
0	0	0	78	22
0	0	0	67	33
0	0	11	44	44
0	0	11	67	22
0	0	11	56	33
0	22	11	67	0
0	0	0	89	11
0	0	0	56	44
11	0	56	22	11
11	0	33	44	11

General Science Composite Rating Report: Public School Sample

Composite Rating Report
Unweighted

Teacher Sample
Field: General Science

Importance Ratings

N	Mean	SD	S E
152	4.05	0.71	0.06

Number of Respondents: 152

Response Distribution (in %)

NR	1	2	3	4	5
0	0	1	21	51	27

General Science Composite Rating Report: Faculty Sample

Composite Rating Report
Unweighted

Education Faculty Sample
Field: General Science

Importance Ratings

N	Mean	SD	S E
9	4.11	0.6	0.2

Number of Respondents: 9

Response Distribution (in %)

NR	1	2	3	4	5
0	0	0	11	67	22

General Science Objective Ratings Summary

Objective	Number		Mean Objective Rating		Description of Objective
Objective	Educators	Faculty	Educators	Faculty	Description of Objective
1	147	9	3.67	4	Apply knowledge of engineering design.
2	148	9	3.21	3	Demonstrate knowledge of tools, materials, and manufacturing.
3	148	9	3.45	4	Demonstrate knowledge of technological systems.
4	147	9	4.11	4.11	Demonstrate knowledge of the components of the solar system and universe and their interactions.
5	146	9	4.07	4.11	Apply knowledge of Earth's geosphere, geologic history, and processes.
6	148	9	4.19	4.56	Demonstrate knowledge of Earth's hydrosphere, atmosphere, weather, and climate.
7	148	9	4.2	4.67	Demonstrate knowledge of natural resources, natural hazards, and human impacts on the environment.
8	150	9	4.33	4.78	Demonstrate knowledge of the characteristics and processes of living organisms.
9	149	9	4.05	4.22	Apply knowledge of the characteristics of populations, communities, ecosystems, and biomes.
10	148	9	4.14	4.33	Apply principles related to the inheritance of characteristics.
11	148	9	4.03	4.33	Demonstrate knowledge of principles related to the theory of biological evolution.
12	149	9	4.3	4.33	Apply knowledge of the structure and properties of matter.
13	147	9	4.24	4	Apply knowledge of the states of matter, particle motion, and heat.
14	150	9	4.1	4.33	Apply knowledge of different forms of energy and the conservation of energy.
15	150	9	4	4.11	Apply knowledge of the concepts of force, motion, work, and power.
16	149	9	3.75	3.89	Apply knowledge of the characteristics and properties of waves, electricity, magnetism, and electromagnetism.
17	149	9	3.95	4.56	Prepare an organized, developed analysis of a topic related to key concepts in earth and space science and/or life science.
18	150	9	3.95	4.44	Prepare an organized, developed analysis of a topic related to key concepts in physical science that emphasizes the application of the principles of science practices.
Overall	150	9	3.99	4.21

Mathematics (Secondary)

Committee participants	Number
Total	24
Public School Educators	22
Higher Education Faculty	2

Gender	Number
Female	15
Male	6
No response	3

Ethnicity	Number
American Indian or Alaska Native	0
Black	2
Asian or Pacific Islander	1
Hispanic	1
White	13
Other	1
Unspecified	6

Mathematics (Secondary)

Bias Review Committee Composition

Committee participants	Number
Total	4
Public School Educators	4
Higher Education Faculty	0

Gender	Number
Female	4
Male	0
No response	0

Ethnicity	Number
American Indian or Alaska Native	0
Black (not of Hispanic origin)	1
Asian or Pacific Islander	0
Hispanic	1
White	2
Other	0
Unspecified	0

Alignment Between the Massachusetts Tests for Educator Licensure® (MTEL®) Test Objectives for M A 063 Mathematics and Massachusetts Curriculum Framework for Mathematics 8 to 12.

MTEL 063 Test Objectives:
I. Number Systems and Operations

Curriculum Framework

0001 Apply knowledge of the properties and structure of the real number system.

Know that there are numbers that are not rational, and approximate them by rational numbers (8.NS.1–2)

Work with radicals and integer exponents (8. E E .1.3–4)

Extend the properties of exponents to rational exponents (N-RN.1–2)

Use properties of rational and irrational numbers (N-RN.3)

0002 Apply knowledge of the properties and structure of the complex number system and linear algebra.

Perform arithmetic operations with complex numbers (N-CN.1–3)

Represent complex numbers and their operations on the complex plane (N-CN.4–6)

Use complex numbers in polynomial identities and equations (N-CN.7–9)

Represent and model with vector quantities (N-VM.1–3)

Perform operations on vectors (N-VM.4–5)

Perform operations on matrices and use matrices in applications (N-VM.6–12)

Solve systems of equations (A- R E I .8–9)

MTEL 063 Test Objectives: II. Relations, Functions, and Algebra	Curriculum Framework
0003 Analyze and apply algebraic techniques to expressions, equations, and inequalities.	Work with radicals and integer exponents (8. E E .2) Analyze and solve linear equations and pairs of simultaneous linear equations (8. E E .7) Interpret the structure of expressions (A-SSE.1–2) Write expressions in equivalent forms to solve problems (A-SSE.3) Perform arithmetic operations on polynomials (A-APR.1) Understand the relationship between zeros and factors of polynomials (A-APR.2) Rewrite rational expressions (A-APR.6–7) Create equations that describe numbers or relationships (A- C E D .1) Understand solving equations as a process of reasoning and explain the reasoning (A- R E I .1–2) Solve equations and inequalities in one variable (A- R E I .3–4)
0004 Apply the principles and properties of relations and functions.	Define, evaluate, and compare functions (8.F.1–2) Understand the concept of a function and use function notation (F- I F .1–3) Analyze functions using different representations (F- I F .7–10) Build a function that models a relationship between two quantities (F-BF.2) Build new functions from existing functions (F-BF.3–4)
0005 Apply the principles and properties of linear, absolute value, and quadratic relations and functions.	Understand the connections between proportional relationships, lines, and linear equations (8. E E .5–6) Analyze and solve linear equations and pairs of simultaneous linear equations (8. E E .8) Define, evaluate, and compare functions (8.F.3) Use functions to model relationships between quantities (8.F.4–5) Create equations that describe numbers or relationships (A- C E D .2–3) Solve systems of equations (A- R E I .6–7) Represent and solve equations and inequalities graphically (A- R E I .10–12) Interpret functions that arise in applications in terms of the context (F- I F .4–6) Analyze functions using different representations (F- I F .7–10) Build a function that models a relationship between two quantities (F-BF.1) Construct and compare linear, quadratic, and exponential models and solve problems (F- L E 1–2) Interpret expressions for functions in terms of the situation they model (F- L E .5)
0006 Apply the principles and properties of exponential and logarithmic relations and functions.	Represent and solve equations and inequalities graphically (A- R E I .10–11) Interpret functions that arise in applications in terms of the context (F- I F .4–6) Analyze functions using different representations (F- I F .7–10) Build a function that models a relationship between two quantities (F-BF.1) Build new functions from existing functions (F-BF.5) Construct and compare linear, quadratic, and exponential models and solve problems (F- L E 1–4) Interpret expressions for functions in terms of the situation they model (F- L E .5)
0007 Apply the principles and properties of polynomial, radical, and rational relations and functions.	Understand the relationship between zeros and factors of polynomials (A-APR.3) Represent and solve equations and inequalities graphically (A- R E I .10–11) Interpret functions that arise in applications in terms of the context (F- I F .4–6) Analyze functions using different representations (F- I F .7–10) Build a function that models a relationship between two quantities (F-BF.1)
0008 Apply the principles and properties of trigonometric functions and identities.	Represent and solve equations and inequalities graphically (A- R E I .10–11) Interpret functions that arise in applications in terms of the context (F- I F .4–6) Analyze functions using different representations (F- I F .7–10) Build a function that models a relationship between two quantities (F-BF.1) Extend the domain of trigonometric functions using the unit circle (F-TF.1–4) Model periodic phenomena with trigonometric functions (F-TF.5–7) Prove and apply trigonometric identities (F-TF.8–9)

MTEL 063 Test Objectives: III. Geometry and Measurement	Curriculum Framework
0009 Apply the principles, concepts, and procedures related to units and measurement.	Solve real-world and mathematical problems involving volume of cylinders, cones, and spheres (8.G.9) Reason quantitatively and use units to solve problems (N-Q.1,3) Prove theorems involving similarity (G-SRT.5) Define trigonometric ratios and solve problems involving right triangles (G-SRT.6–8) Explain volume formulas and use them to solve problems (G-GMD.1–3) Apply geometric concepts in modeling situations (G-MG.2,4)
0010 Apply the axiomatic structure of Euclidean geometry.	Understand congruence and similarity using physical models, transparencies, or geometry software (8.G.5) Understand and apply the Pythagorean Theorem (8.G.6–7) Understand congruence in terms of rigid motions (G- C O .7–8) Prove geometric theorems and, when appropriate, the converse of theorems (G- C O .9–11) Make geometric constructions (G- C O .12–13) Prove theorems involving similarity (G-SRT.4) Define trigonometric ratios and solve problems involving right triangles (G-SRT.6–8) Apply trigonometry to general triangles (G-SRT.9–11) Understand and apply theorems about circles (G-C.1–4) Find arc lengths and areas of sectors of circles (G-C.5)
0011 Apply the principles and properties of Euclidean geometry to solve problems involving two- and three-dimensional objects.	Solve real-world and mathematical problems involving volume of cylinders, cones, and spheres (8.G.9) Define trigonometric ratios and solve problems involving right triangles (G-SRT.8) Explain volume formulas and use them to solve problems (G-GMD.1–3) Visualize relationships between two-dimensional and three-dimensional objects (G-GMD.4) Apply geometric concepts in modeling situations (G-MG.1,3)
0012 Apply the principles and properties of coordinate and transformational geometry and the characteristics of non-Euclidean geometries.	Understand congruence and similarity using physical models, transparencies, or geometry software (8.G.1–4) Understand and apply the Pythagorean Theorem (8.G.8) Experiment with transformations in the plane (G- C O .1–5) Understand congruence in terms of rigid motions (G- C O .6–8) Understand similarity in terms of similarity transformations (G-SRT.1–3) Translate between the geometric description and the equation for a conic section (G-GPE.1–3) Use coordinates to prove simple geometric theorems algebraically (G-GPE.4–7)

MTEL 063 Test Objectives: IV. Probability, Statistics, Calculus, and Discrete Mathematics	Curriculum Framework
0013 Apply the principles, properties, and techniques of probability.	Understand independence and conditional probability and use them to interpret data from simulations or experiments (S-CP.1–3) Use the rules of probability to compute probabilities of compound events in a uniform probability model (S-CP.6–9)
0014 Apply the principles and concepts of descriptive statistics to the problem–solving process.	Investigate patterns of association in bivariate data (8.SP.1–4) Summarize, represent, and interpret data on a single count or measurement variable (S- I D .1–4) Summarize, represent, and interpret data on two categorical and quantitative variables (S- I D .5–6) Interpret linear models (S- I D .7–9) Understand and evaluate random processes underlying statistical experiments (S- I C .1–2) Make inferences and justify conclusions from sample surveys, experiments, and observational studies (S- I C .3–6) Understand independence and conditional probability and use them to interpret data from simulations or experiments (S-CP.4–5) Calculate expected values and use them to solve problems (S-MD.1–4) Use probability to evaluate outcomes of decisions (S-MD.5–7)
0015 Apply principles and techniques of limits, continuity, and differential calculus.	blank
0016 Apply the principles and techniques of integral calculus.	blank
0017 Apply the properties and techniques of discrete mathematics.	blank

MTEL 063 Test Objectives:
V. Integration of Knowledge and Understanding

Curriculum Framework

0018 Prepare an organized, developed analysis on a topic related to one or more of the following: functions, algebra, geometry, and measurement.

Solve systems of equations (A- R E I .6–9)

Analyze and solve linear equations and pairs of simultaneous linear equations (8. E E .7–8)

Interpret the structure of expressions (A-SSE.1–2)

Write expressions in equivalent forms to solve problems (A-SSE.3)

Perform arithmetic operations on polynomials (A-APR.1)

Understand the relationship between zeros and factors of polynomials (A-APR.2–3)

Rewrite rational expressions (A-APR.6–7)

Create equations that describe numbers or relationships (A- C E D .1–3)

Understand solving equations as a process of reasoning and explain the reasoning (A- R E I .1–2)

Solve equations and inequalities in one variable (A- R E I .3–4)

Define, evaluate, and compare functions (8.F.1–3)

Understand the concept of a function and use function notation (F- I F .1–3)

Analyze functions using different representations (F- I F .7–10)

Build a function that models a relationship between two quantities (F-BF.1–2)

Build new functions from existing functions (F-BF.3–5)

Understand the connections between proportional relationships, lines, and linear equations (8. E E .5–6)

Use functions to model relationships between quantities (8.F.4–5)

Represent and solve equations and inequalities graphically (A- R E I .10–12)

Interpret functions that arise in applications in terms of the context (F- I F .4–6)

Construct and compare linear, quadratic, and exponential models and solve problems (F- L E 1–4)

Interpret expressions for functions in terms of the situation they model (F- L E .5)

Extend the domain of trigonometric functions using the unit circle (F-TF.1–4)

Model periodic phenomena with trigonometric functions (F-TF.5–7)

Solve real-world and mathematical problems involving volume of cylinders, cones, and spheres (8.G.9)

Reason quantitatively and use units to solve problems (N-Q.1,3)

Define trigonometric ratios and solve problems involving right triangles (G-SRT.6–8)

Explain volume formulas and use them to solve problems (G-GMD.1–3)

Apply geometric concepts in modeling situations (G-MG.1–4)

Understand and apply the Pythagorean Theorem (8.G.6–8)

Understand congruence in terms of rigid motions (G- C O .6–8)

Prove geometric theorems and, when appropriate, the converse of theorems (G- C O .9–11)

Apply trigonometry to general triangles (G-SRT.9–11)

Understand and apply theorems about circles (G-C.1–4)

Find arc lengths and areas of sectors of circles (G-C.5)

Visualize relationships between two–dimensional and three–dimensional objects (G-GMD.4)

Experiment with transformations in the plane (G- C O .1–5)

Understand similarity in terms of similarity transformations (G-SRT.1–3)

Translate between the geometric description and the equation for a conic section (G-GPE.1–3)

Use coordinates to prove simple geometric theorems algebraically (G-GPE.4–7)

0019 Prepare an organized, developed analysis on a topic related to one or more of the following: number systems and operations; probability, statistics, calculus, and discrete mathematics.

Know that there are numbers that are not rational, and approximate them by rational numbers (8.NS.1–2)

Work with radicals and integer exponents (8. E E .1.3–4)

Extend the properties of exponents to rational exponents (N-RN.1–2)

Use properties of rational and irrational numbers (N-RN.3)

Represent and model with vector quantities (N-VM.1–3)

Perform operations on vectors (N-VM.4–5)

Perform operations on matrices and use matrices in applications (N-VM.6–12)

Understand independence and conditional probability and use them to interpret data from simulations or experiments (S-CP.1–5)

Use the rules of probability to compute probabilities of compound events in a uniform probability model (S-CP.6–9)

Investigate patterns of association in bivariate data (8.SP.1–4)

Summarize, represent, and interpret data on a single count or measurement variable (S- I D .1–4)

Summarize, represent, and interpret data on two categorical and quantitative variables (S- I D .5–6)

Interpret linear models (S- I D .7–9)

Understand and evaluate random processes underlying statistical experiments (S- I C .1–2)

Make inferences and justify conclusions from sample surveys, experiments, and observational studies (S- I C .3–6)

Calculate expected values and use them to solve problems (S-MD.1–4)

Use probability to evaluate outcomes of decisions (S-MD.5–7)

Mathematics (Secondary)

Committee participants	Number
Total	9
Public School Educators	8
Higher Education Faculty	1

Gender	Number
Female	7
Male	1
No response	1

Ethnicity	Number
American Indian or Alaska Native	0
Black (not of Hispanic origin)	0
Asian or Pacific Islander	1
Hispanic	2
White	5
Other	0
Unspecified	1

English as a Second Language

INTACT CLASSROOM SESSIONS 2013–2015
10/01/2013	Lincoln School (Brookline Public Schools)
10/05/2013	Simmons College Gordon College Salem State University
10/08/2013	Wheaton College
10/18/2013	University of Massachusetts - Boston
10/31/2013	Wheelock College
01/30/2014	Wheaton College
02/01/2014	Gordon College
02/04/2014	Elms College
02/05/2014	Salem State University
02/06/2014	University of Massachusetts - Amherst
02/11/2014	Lincoln School (Brookline Public Schools)
02/14/2014	Wheelock College
02/19/2014	Worcester State University
04/25/2015	Cambridge College
04/27/2015	Salem State University
05/02/2015	Simmons College Framingham State University
05/05/2015	Westfield State University
05/09/2015	Gordon College

STAND ALONE EVENTS AT MTEL OPERATIONAL TEST ADMINISTRATIONS

10/26/2013
03/01/2014
05/10/2014
07/12/2014
10/25/2014

Pilot Test Outcomes
Mathematics (Secondary)

Pilot testing was conducted at CBT centers from February 18, 2020, through March 16, 2020. The following information is a summary of the data collected during this specific time frame.

Multiple-Choice Items ( M C Q's ) and Short-Answer Items ( S A's )

Number of M C Q's Prepared	123
Number of M C Q's Piloted*	50
Number of M C Q's Identified for Further Review	6
Number of M C Q's Deleted Post Pilot Test Review	0

Open-Response Items ( O R I's )

Number of O R I's Developed	21
Number of O R I's Piloted*	18
Number of O R I's Identified for Further Review	3
Number of O R I's Deleted Post Pilot Test Review	3

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms until all items are piloted.

Mathematics (Secondary)

Number of Qualifying Score Participants: 9

Number of Participants Providing Ratings: 7 (78%)

Question Rating Scale: 1–5 1=not at all (well/confident/satisfied/successful), 5=very (well/confident/satisfied/successful)	Median Rating
How well do you feel you understood the training that you received for making your qualifying score judgments?	5
How confident are you of the qualifying score judgments you made?	5
How satisfied are you that you were able to complete the work you were asked to do in the time that was provided?	5
How successful were the coordination and logistics of the qualifying score conference?	4
Overall, how satisfied are you with the qualifying score process in which you participated?	5

Mathematics (Secondary) Content Validation Survey Population/Sample/Respondents Demographics (Public School Sample)

Key

Population: The number and percent of licensed and assigned educators in the content area, extracted from the database of full-time equivalent (FTE) teachers assigned to each teaching area for each school district in Massachusetts, as provided by the Massachusetts Department of Elementary and Secondary Education for use in drawing a random sample of educators.
Sample (with oversampling): The number and percent of licensed and assigned educators sampled to receive the survey. In order to obtain appropriate representation of minority groups (e.g., American Indian, Asian, Black, and Hispanic) in the survey results, oversampling of minority groups was conducted for those fields for which the entire population was not sampled. In these instances, the survey results were weighted appropriately to take the oversampling into account.

Mathematics (Secondary)	Population		Sample (with oversampling)
Mathematics (Secondary)	N	%	N	%
Total	4,227	100	443	100
Female	2,394	56.63	248	55.98
Male	1,833	43.36	195	44.01
American Indian or Alaska Native	5	0.11	1	0.22
Black (not of Hispanic origin)	144	3.4	28	6.32
Asian or Pacific Islander	141	3.33	27	6.09
Hispanic	122	2.88	23	5.19
White	3,792	89.7	359	81.03
Other	23	0.54	5	1.12

Mathematics (Secondary) Survey Return Rate by Field and Return Status: Public School Sample

Returned Surveys

Field	Number Sent	Number Not Distributed	Eligible		Ineligible		Total		Adjusted Return Rate
Field	Number Sent	Number Not Distributed	N	%	N	%	N	%	Adjusted Return Rate
061 English	541	30	191	35.30%	32	5.90%	223	43.60%	39.90%
063 Mathematics	609	28	205	33.70%	53	8.70%	258	44.40%	38.80%
064 General Science	530	15	154	29.10%	58	10.90%	212	41.20%	33.70%
065 Middle School Mathematics	581	14	160	27.50%	78	13.40%	238	42.00%	32.70%
190 Foundations of Reading	498	18	166	33.30%	41	8.20%	207	43.10%	37.80%
ALL	2,759	105	876	31.80%	262	9.50%	1,138	42.90%	36.60%

Mathematics (Secondary) Survey Return Rate by Field and Return Status: Faculty Sample

Returned Surveys

Field	Number Sent	Number Not Distributed	Eligible		Ineligible		Total		Adjusted Return Rate
Field	Number Sent	Number Not Distributed	N	%	N	%	N	%	Adjusted Return Rate
061 English	127	9	18	14.20%	1	0.80%	19	16.10%	15.40%
063 Mathematics	129	8	24	18.60%	0	0.00%	24	19.80%	19.80%
064 General Science	106	9	9	8.50%	2	1.90%	11	11.30%	9.50%
065 Middle School Mathematics	121	9	21	17.40%	3	2.50%	24	21.40%	19.30%
190 Foundations of Reading	138	5	33	23.90%	0	0.00%	33	24.80%	24.80%
ALL	621	40	105	16.90%	6	1.00%	111	19.10%	18.30%

Demographic Summary Report: Public School Sample

Teacher Sample
Field: Mathematics (Secondary)

Question	Answer	Absolute Frequency	Adjusted Percent
Currently Teaching	Yes	204	99.5
Currently Teaching	No	1	0.5
Hold a valid Massachusetts teaching license	Yes	205	100
Hold a valid Massachusetts teaching license	No	0	0
What is the highest level of education you have attained?	Bachelor's degree	23	11.2
	Master's degree	167	81.5
	Doctoral degree	8	3.9
	Other	7	3.4
What is your ethnicity?	American Indian or Alaska Native	0	0
	Black (not of Hispanic origin)	8	3.9
	Asian or Pacific Islander	12	5.9
	Hispanic	7	3.4
	White (not of Hispanic origin)	175	85.8
	Other	2	1
What is your first (native) language?	English	179	88.2
	French	1	0.5
	Portuguese	1	0.5
	An Asian language	7	3.4
	Russian	1	0.5
	Spanish	5	2.5
	American Sign Language	1	0.5
	Italian	2	1
	Other	6	3
What is your gender?	Female	113	57.1
What is your gender?	Male	85	42.9
What grade level(s) have you taught in Massachusetts during this or the previous school year? (Select all that apply.)	Pre K to 6th	2	0.9
	5th to 8th	18	8.3
	5th to 12th	11	5.1
	8th to 12th	186	85.7
How many years of professional teaching experience do you have? (Count partial years as full years.)	1 to 3 years	9	4.4
	4 to 10 years	52	25.4
	11 years or more	144	70.2

Mathematics (Secondary) Demographic Summary Report: Faculty Sample

Faculty Sample
Field: Mathematics (Secondary)

Question	Answer	Absolute Frequency	Adjusted Percent
Teaching undergraduate or graduate arts and science or education courses in which prospective teachers may have been enrolled	Yes	24	100
	No	0	0
What is the highest level of education you have attained?	Bachelor's degree	0	0
	Master's degree	1	4.2
	Doctoral degree	23	95.8
	Other	0	0
What is your ethnicity?	American Indian or Alaska Native	0	0
	Black (not of Hispanic origin)	0	0
	Asian or Pacific Islander	0	0
	Hispanic	1	4.3
	White (not of Hispanic origin)	22	95.7
	Other	0	0
What is your first (native) language?	English	21	87.5
	Spanish	0	0
	Other	3	12.5
What is your gender?	Female	15	65.2
What is your gender?	Male	8	34.8
How many years have you taught as a faculty member at a higher education institution? (Count partial years as full years.)	1 to 3 years	3	12.5
	4 to 10 years	5	20.8
	11 years or more	16	66.7
In which higher education institution department do you currently hold your primary appointment?	Education department only	4	16.7
	Both the Education department and another department	1	4.2
	Arts and Sciences or Fine Arts department(s) only	19	79.2
Have you ever taught in the subject of the field and grade level indicated above in Massachusetts?	Yes	5	20.8
	No	19	79.2
How recently have you taught in the subject of the field and grade level indicated above in Massachusetts? (Count partial years as full years.)	1 to 3 years	2	40
	4 to 10 years	2	40
	11 years or more	1	20

Mathematics (Secondary) Objective Rating Report: Public School Sample

Number of Objectives: 19

Importance Ratings

Objective Number	N	Mean	SD	S E
1	203	4.49	0.65	0.05
2	204	3.61	0.88	0.06
3	205	4.72	0.49	0.03
4	204	4.4	0.67	0.05
5	204	4.45	0.6	0.04
6	205	3.89	0.78	0.05
7	205	4.11	0.76	0.05
8	204	3.8	0.81	0.06
9	204	4.04	0.83	0.06
10	205	3.91	0.89	0.06
11	204	3.96	0.83	0.06
12	204	3.71	0.89	0.06
13	205	3.75	0.82	0.06
14	203	3.7	0.89	0.06
15	204	3.37	0.92	0.06
16	204	3.2	0.94	0.07
17	204	3.2	0.92	0.06
18	204	4.28	0.78	0.05
19	202	4.13	0.82	0.06

Number of Respondents: 205

Response Distribution (in %)

NR	1	2	3	4	5
1	0	1	5	37	56
0	1	6	38	39	16
0	0	0	2	24	74
0	0	0	10	39	50
0	0	0	5	44	50
0	1	1	28	48	22
0	0	1	19	46	33
0	0	3	32	44	20
0	0	1	28	35	35
0	0	5	27	38	29
0	0	1	29	40	29
0	0	8	31	40	20
0	0	3	40	36	21
1	0	6	37	35	21
0	0	15	44	26	14
0	2	20	44	23	10
0	3	18	43	28	8
0	0	1	13	40	45
1	0	3	16	43	36

Mathematics (Secondary) Objective Rating Report: Faculty Sample

Objective Rating Report: Ordered by Objective Number
Unweighted

Faculty Sample
Field: Mathematics (Secondary)

Number of Objectives: 19

Importance Ratings

Objective Number	N	Mean	SD	S E
1	24	4.63	0.49	0.1
2	23	3.61	0.84	0.17
3	24	4.71	0.62	0.13
4	24	4.71	0.55	0.11
5	24	4.46	0.51	0.1
6	24	4.13	0.8	0.16
7	23	4.17	0.78	0.16
8	24	4.21	0.66	0.13
9	24	4.25	0.61	0.12
10	24	4.13	0.85	0.17
11	24	3.96	0.81	0.16
12	24	3.67	0.96	0.2
13	24	4.25	0.68	0.14
14	24	4.42	0.72	0.15
15	24	4.13	0.8	0.16
16	24	4.08	0.83	0.17
17	22	3.86	0.83	0.18
18	24	4.42	0.65	0.13
19	22	4.27	0.94	0.2

Number of Respondents: 24

Response Distribution (in %)

NR	1	2	3	4	5
0	0	0	0	38	63
4	0	8	33	42	13
0	0	0	8	13	79
0	0	0	4	21	75
0	0	0	0	54	46
0	0	4	13	50	33
4	0	0	21	38	38
0	0	0	13	54	33
0	0	0	8	58	33
0	0	4	17	42	38
0	0	0	33	38	29
0	0	8	42	25	25
0	0	0	13	50	38
0	0	0	13	33	54
0	0	0	25	38	38
0	0	0	29	33	38
8	0	4	25	42	21
0	0	0	8	42	50
8	4	0	4	42	42

Mathematics (Secondary) Descriptive Statement Rating Report: Public School Sample

Descriptive Statement Rating Report: Ordered by Objective Number
Unweighted

Number of Objectives: 19

Importance Ratings

Objective Number	N	Mean	SD	S E
1	201	4.2	0.71	0.05
2	202	3.76	0.82	0.06
3	204	4.34	0.74	0.05
4	204	4.1	0.78	0.05
5	203	4.27	0.71	0.05
6	203	4.11	0.82	0.06
7	203	4.11	0.77	0.05
8	204	4.08	0.8	0.06
9	204	4.07	0.72	0.05
10	204	3.99	0.87	0.06
11	203	4.02	0.8	0.06
12	201	3.94	0.82	0.06
13	204	3.99	0.77	0.05
14	201	3.96	0.8	0.06
15	203	3.97	0.78	0.05
16	203	3.88	0.82	0.06
17	201	3.73	0.83	0.06
18	203	4.05	0.83	0.06
19	203	4.05	0.77	0.05

Number of Respondents: 204

Response Distribution (in %)

NR	1	2	3	4	5
2	0	1	13	49	35
1	1	5	25	51	16
0	0	2	9	41	47
0	0	3	17	47	33
1	0	1	8	50	39
1	1	2	14	49	33
1	0	2	16	49	32
0	0	2	18	46	32
0	0	1	18	52	28
0	0	5	20	44	30
1	0	2	20	47	29
2	0	4	23	45	25
0	0	1	24	48	26
2	0	3	24	45	26
1	0	2	22	49	25
1	0	4	25	46	23
2	1	3	32	45	16
1	1	1	21	43	32
1	0	1	20	49	28

Mathematics (Secondary) Descriptive Statement Rating Report: Faculty Sample

Descriptive Statement Rating Report: Ordered by Objective Number
Unweighted

Faculty Sample
Field: Mathematics (Secondary)

Number of Objectives: 19

Importance Ratings

Objective Number	N	Mean	SD	S E
1	24	4.08	0.72	0.15
2	24	4	0.72	0.15
3	23	4.04	0.64	0.13
4	24	3.71	1.04	0.21
5	24	4.04	0.75	0.15
6	24	4.04	0.75	0.15
7	24	4.04	0.62	0.13
8	24	3.92	0.83	0.17
9	24	4.13	0.74	0.15
10	24	3.92	0.83	0.17
11	23	3.61	0.84	0.17
12	24	3.88	0.85	0.17
13	23	3.87	0.87	0.18
14	23	4	0.74	0.15
15	24	3.96	0.91	0.19
16	24	3.88	0.99	0.2
17	24	3.63	0.88	0.18
18	24	3.67	1.13	0.23
19	22	3.73	0.98	0.21

Number of Respondents: 24

Response Distribution (in %)

NR	1	2	3	4	5
0	0	0	21	50	29
0	0	0	25	50	25
4	0	0	17	58	21
0	0	17	21	38	25
0	0	0	25	46	29
0	0	4	13	58	25
0	0	0	17	63	21
0	0	8	13	58	21
0	0	0	21	46	33
0	0	4	25	46	25
4	0	8	33	42	13
0	0	4	29	42	25
4	0	8	17	50	21
4	0	0	25	46	25
0	0	8	17	46	29
0	0	13	17	42	29
0	0	13	25	50	13
0	8	4	21	46	21
8	4	0	33	33	21

Mathematics (Secondary) Composite Rating Report: Public School Sample

Composite Rating Report
Unweighted

Teacher Sample
Field: Mathematics (Secondary)

Importance Ratings

N	Mean	SD	S E
202	4.07	0.73	0.05

Number of Respondents: 152

Response Distribution (in %)

NR	1	2	3	4	5
0	0	3	14	55	27

Mathematics (Secondary) Composite Rating Report: Faculty Sample

Composite Rating Report
Unweighted

Education Faculty Sample
Field: Mathematics (Secondary)

Importance Ratings

N	Mean	SD	S E
24	3.88	0.8	0.16

Number of Respondents: 24

Response Distribution (in %)

NR	1	2	3	4	5
0	0	4	25	50	21

Mathematics (Secondary) Objective Ratings Summary

Objective	Number		Mean Objective Rating		Description of Objective
Objective	Educators	Faculty	Educators	Faculty	Description of Objective
1	196	24	4.49	4.63	Apply knowledge of the properties and structure of the real number system.
2	197	23	3.6	3.61	Apply knowledge of the properties and structure of the complex number system and linear algebra.
3	198	24	4.72	4.71	Analyze and apply algebraic techniques to expressions, equations, and inequalities.
4	197	24	4.4	4.71	Apply the principles and properties of relations and functions.
5	197	24	4.44	4.46	Apply the principles and properties of linear, absolute value, and quadratic relations and functions.
6	198	24	3.88	4.13	Apply the principles and properties of exponential and logarithmic relations and functions.
7	198	23	4.1	4.17	Apply the principles and properties of polynomial, radical, and rational relations and functions.
8	197	24	3.79	4.21	Apply the principles and properties of trigonometric functions and identities.
9	197	24	4.03	4.25	Apply the principles, concepts, and procedures related to units and measurement.
10	198	24	3.89	4.13	Apply the axiomatic structure of Euclidean geometry.
11	197	24	3.95	3.96	Apply the principles and properties of Euclidean geometry to solve problems involving two- and three-dimensional objects.
12	197	24	3.69	3.67	Apply the principles and properties of coordinate and transformational geometry and the characteristics of non-Euclidean geometries.
13	198	24	3.71	4.25	Apply the principles, properties, and techniques of probability.
14	196	24	3.68	4.42	Apply the principles and concepts of descriptive statistics to the problem-solving process.
15	197	24	3.36	4.13	Apply principles and techniques of limits, continuity, and differential calculus.
16	197	24	3.18	4.08	Apply principles and techniques of integral calculus.
17	197	22	3.16	3.86	Apply the properties and techniques of discrete mathematics.
18	197	24	4.28	4.42	Prepare an organized, developed analysis on a topic integrating knowledge from at least two of the following: number sense and operations; relations, functions, and algebra; geometry and measurement; probability, statistics, calculus, and discrete mathematics.
19	195	22	4.13	4.27	Prepare an organized, developed analysis on a topic integrating knowledge from at least two of the following: number sense and operations; relations, functions, and algebra; geometry and measurement; probability, statistics, calculus, and discrete mathematics.
Overall	198	24	3.92	4.22

Middle School Mathematics

Committee participants	Number
Total	33
Public School Educators	26
Higher Education Faculty	7

Gender	Number
Female	24
Male	0
No response	9

Ethnicity	Number
American Indian or Alaska Native	0
Black	2
Asian or Pacific Islander	4
Hispanic	0
White	14
Other	2
Unspecified	9

Middle School Mathematics

Bias Review Committee Composition

Committee participants	Number
Total	10
Public School Educators	10
Higher Education Faculty	0

Gender	Number
Female	8
Male	2
No response	0

Ethnicity	Number
American Indian or Alaska Native	0
Black (not of Hispanic origin)	6
Asian or Pacific Islander	2
Hispanic	0
White	1
Other	1
Unspecified	0

Alignment Between the Massachusetts Tests for Educator Licensure® (MTEL®) Test Objectives for M A 063 Middle School Mathematics and Massachusetts Curriculum Framework for Mathematics 5 to 8.

MTEL 065 Test Objectives:
I. Number System and Quantity

Curriculum Framework

0001 Apply the structure and properties of the real number system.

Write and interpret numerical expressions (5. O A .1)

Understand the place value system (5.NBT.1–4)

Perform operations with multi-digit whole numbers and with decimals to hundredths (5.NBT.5–7)

Compute fluently with multi-digit numbers and find common factors and multiples (6.NS.2–4)

Know that there are numbers that are not rational, and approximate them by rational numbers (8.NS.1–2)

Work with radicals and integer exponents (8. E E .1,3–4)

0002 Use rational numbers, ratios, and proportional relationships.

Use equivalent fractions as a strategy to add and subtract fractions (5.NF.1–2)

Apply and extend previous understandings of multiplication and division to multiply and divide fractions (5.NF.3–7)

Apply and extend previous understandings of multiplication and division to divide fractions by fractions (6.NS.1)

Apply and extend previous understandings of numbers to the system of rational numbers (6.NS.5–8)

Apply and extend previous understandings of operations with fractions to add, subtract, multiply, and divide rational numbers (7.NS.1–3)

Understand ratio and rate concepts and use ratio and rate reasoning to solve problems (6.RP.1–3)

Analyze proportional relationships and use them to solve real-world and mathematical problems (7.RP.1,3)

MTEL 065 Test Objectives: II. Algebra, Functions, and Modeling	Curriculum Framework
0003 Use patterns to model and solve problems.	Analyze patterns and relationships (5. O A .3) Analyze proportional relationships and use them to solve real-world and mathematical problems (7.RP.2) Understand the connections between proportional relationships, lines, and linear equations (8. E E .5)
0004 Apply algebraic techniques to expressions and equations.	Apply and extend previous understandings of arithmetic to algebraic expressions (6. E E .1–4) Reason about and solve one-variable equations and inequalities (6. E E .5–8) Use properties of operations to generate equivalent expressions (7. E E .1–2) Solve real-life and mathematical problems using numerical and algebraic expressions and equations (7. E E .3–4) Work with radicals and integer exponents (8. E E .2) Analyze and solve linear equations and pairs of simultaneous linear equations (8. E E .7)
0005 Demonstrate knowledge of relations and functions.	Represent and analyze quantitative relationships between dependent and independent variables (6. E E .9) Define, evaluate, and compare functions (8.F.1–2) Use functions to model relationships between quantities (8.F.4–5)
0006 Apply the properties of linear relations and functions.	Analyze proportional relationships and use them to solve real-world and mathematical problems (7.RP.2) Understand the connections between proportional relationships, lines, and linear equations (8. E E .5–6) Analyze and solve linear equations and pairs of simultaneous linear equations (8. E E .8) Define, evaluate, and compare functions (8.F.3) Use functions to model relationships between quantities (8.F.4)
0007 Apply the principles and properties of nonlinear relations and functions.	Use functions to model relationships between quantities (8.F.5)

MTEL 065 Test Objectives: III. Geometry and Measurement	Curriculum Framework
0008 Apply principles, concepts, and procedures related to measurement.	Convert like measurement units within a given measurement system (5.MD.1) Understand concepts of volume and relate volume to multiplication and to addition (5.MD.3–5) Solve real-world and mathematical problems involving area, surface area, and volume (6.G.1–2) Solve real-life and mathematical problems involving angle measure, area, surface area, and volume (7.G.5–6) Solve real-world and mathematical problems involving volume of cylinders, cones, and spheres (8.G.9)
0009 Apply the principles of Euclidean geometry and proof.	Draw, construct, and describe geometrical figures and describe relationships between them (7.G.1–2) Solve real-life and mathematical problems involving angle measure, area, surface area, and volume (7.G.5) Understand congruence and similarity using physical models, transparencies, or geometry software (8.G.5)
0010 Apply properties of two- and three-dimensional figures.	Classify two-dimensional figures into categories based on their properties (5.G.3–4) Solve real-world and mathematical problems involving area, surface area, and volume (6.G.4) Draw, construct, and describe geometrical figures and describe relationships between them (7.G.2–3) Solve real-life and mathematical problems involving angle measure, area, surface area, and volume (7.G.4) Understand and apply the Pythagorean Theorem (8.G.6–7) Solve real-world and mathematical problems involving volume of cylinders, cones, and spheres (8.G.9)
0011 Apply the principles and properties of coordinate and transformational geometries.	Graph points on the coordinate plane to solve real-world and mathematical problems (5.G.1–2) Solve real-world and mathematical problems involving area, surface area, and volume (6.G.3) Understand congruence and similarity using physical models, transparencies, or geometry software (8.G.1–4) Understand and apply the Pythagorean Theorem (8.G.8)

MTEL 065 Test Objectives:
IV. Statistics and Probability

Curriculum Framework

0012 Understand the principles, techniques, and applications of statistics.

Represent and interpret data (5.MD.2)

Develop understanding of statistical variability (6.SP.1–3)

Summarize and describe distributions (6.SP.4–5)

Use random sampling to draw inferences about a population (7.SP.1–2)

Draw informal comparative inferences about two populations (7.SP.3–4)

Investigate patterns of association in bivariate data (8.SP.1–4)

0013 Understand the principles of probability.

Investigate chance processes and develop, use, and evaluate probability models (7.SP.5–8)

MTEL 065 Test Objectives:
V. Integration of Knowledge and Understanding

Curriculum Framework

0014 Prepare an organized, developed analysis on a topic cited from the Massachusetts Mathematics Curriculum Framework grades 5–8.

Write and interpret numerical expressions (5. O A .1)

Understand ratio and rate concepts and use ratio and rate reasoning to solve problems (6.RP.1–3)

Analyze proportional relationships and use them to solve real-world and mathematical problems (7.RP.1–3)

Analyze patterns and relationships (5. O A .3)

Understand the connections between proportional relationships, lines, and linear equations (8. E E .5)

Apply and extend previous understandings of arithmetic to algebraic expressions (6. E E .1–4)

Reason about and solve one-variable equations and inequalities (6. E E .5–8)

Use properties of operations to generate equivalent expressions (7. E E .1–2)

Solve real-life and mathematical problems using numerical and algebraic expressions and equations (7. E E .3–4)

Work with radicals and integer exponents (8. E E .2)

Analyze and solve linear equations and pairs of simultaneous linear equations (8. E E .7)

Represent and analyze quantitative relationships between dependent and independent variables (6. E E .9)

Define, evaluate, and compare functions (8.F.1–2)

Use functions to model relationships between quantities (8.F.4–5)

Analyze proportional relationships and use them to solve real-world and mathematical problems (7.RP.2)

Understand the connections between proportional relationships, lines, and linear equations (8. E E .5–6)

Analyze and solve linear equations and pairs of simultaneous linear equations (8. E E .8)

Define, evaluate, and compare functions (8.F.3)

Solve real-world and mathematical problems involving area, surface area, and volume (6.G.1–4)

Solve real-life and mathematical problems involving angle measure, area, surface area, and volume (7.G.4–6)

Solve real-world and mathematical problems involving volume of cylinders, cones, and spheres (8.G.9)

Draw, construct, and describe geometrical figures and describe relationships between them (7.G.1–3)

Classify two-dimensional figures into categories based on their properties (5.G.3–4)

Understand and apply the Pythagorean Theorem (8.G.6–8)

Graph points on the coordinate plane to solve real-world and mathematical problems (5.G.1–2)

0015 Prepare an organized, developed analysis on a topic related to one or more of the following: statistics, probability, and algebra.

Understand the place value system (5.NBT.1–4)

Perform operations with multi-digit whole numbers and with decimals to hundredths (5.NBT.5–7)

Know that there are numbers that are not rational, and approximate them by rational numbers (8.NS.1–2)

Use equivalent fractions as a strategy to add and subtract fractions (5.NF.1–2)

Apply and extend previous understandings of multiplication and division to multiply and divide fractions (5.NF.3–7; 6.NS.1)

Apply and extend previous understandings of numbers to the system of rational numbers (6.NS.5–8)

Apply and extend previous understandings of operations with fractions to add, subtract, multiply, and divide rational numbers (7.NS.1–3)

Understand ratio and rate concepts and use ratio and rate reasoning to solve problems (6.RP.1–3)

Represent and analyze quantitative relationships between dependent and independent variables (6. E E .9)

Graph points on the coordinate plane to solve real-world and mathematical problems (5.G.1–2)

Represent and interpret data (5.MD.2)

Develop understanding of statistical variability (6.SP.1–3)

Summarize and describe distributions (6.SP.4–5)

Use random sampling to draw inferences about a population (7.SP.1–2)

Draw informal comparative inferences about two populations (7.SP.3–4)

Investigate patterns of association in bivariate data (8.SP.1–4)

Investigate chance processes and develop, use, and evaluate probability models (7.SP.5–8)

Middle School Mathematics

Committee participants	Number
Total	8
Public School Educators	5
Higher Education Faculty	3

Gender	Number
Female	7
Male	1
No response	0

Ethnicity	Number
American Indian or Alaska Native	0
Black (not of Hispanic origin)	1
Asian or Pacific Islander	0
Hispanic	0
White	5
Other	0
Unspecified	2

English as a Second Language

INTACT CLASSROOM SESSIONS 2013–2015
10/01/2013	Lincoln School (Brookline Public Schools)
10/05/2013	Simmons College Gordon College Salem State University
10/08/2013	Wheaton College
10/18/2013	University of Massachusetts - Boston
10/31/2013	Wheelock College
01/30/2014	Wheaton College
02/01/2014	Gordon College
02/04/2014	Elms College
02/05/2014	Salem State University
02/06/2014	University of Massachusetts - Amherst
02/11/2014	Lincoln School (Brookline Public Schools)
02/14/2014	Wheelock College
02/19/2014	Worcester State University
04/25/2015	Cambridge College
04/27/2015	Salem State University
05/02/2015	Simmons College Framingham State University
05/05/2015	Westfield State University
05/09/2015	Gordon College

STAND ALONE EVENTS AT MTEL OPERATIONAL TEST ADMINISTRATIONS

10/26/2013
03/01/2014
05/10/2014
07/12/2014
10/25/2014

Pilot Test Outcomes
Middle School Mathematics

Pilot testing was conducted at CBT centers from February 18, 2020, through March 16, 2020. The following information is a summary of the data collected during this specific time frame.

Multiple-Choice Items ( M C Q's ) and Short-Answer Items ( S A's )

Number of M C Q's Prepared	154
Number of M C Q's Piloted*	50
Number of M C Q's Identified for Further Review	14
Number of M C Q's Deleted Post Pilot Test Review	0

Open-Response Items ( O R I's )

Number of O R I's Developed	19
Number of O R I's Piloted*	17
Number of O R I's Identified for Further Review	5
Number of O R I's Deleted Post Pilot Test Review	5

*This chart includes information regarding items piloted prior to the first operational administration and may be a subset of all items developed. Additional multiple-choice items are piloted as non-scorable on operational test forms until all items are piloted.

Middle School Mathematics

Number of Qualifying Score Participants: 8

Number of Participants Providing Ratings: 8 (100%)

Question Rating Scale: 1–5 1=not at all (well/confident/satisfied/successful), 5=very (well/confident/satisfied/successful)	Median Rating
How well do you feel you understood the training that you received for making your qualifying score judgments?	5
How confident are you of the qualifying score judgments you made?	4
How satisfied are you that you were able to complete the work you were asked to do in the time that was provided?	5
How successful were the coordination and logistics of the qualifying score conference?	5
Overall, how satisfied are you with the qualifying score process in which you participated?	5

Middle School Mathematics Content Validation Survey Population/Sample/Respondents Demographics (Public School Sample)

Key

Population: The number and percent of licensed and assigned educators in the content area, extracted from the database of full-time equivalent (FTE) teachers assigned to each teaching area for each school district in Massachusetts, as provided by the Massachusetts Department of Elementary and Secondary Education for use in drawing a random sample of educators.
Sample (with oversampling): The number and percent of licensed and assigned educators sampled to receive the survey. In order to obtain appropriate representation of minority groups (e.g., American Indian, Asian, Black, and Hispanic) in the survey results, oversampling of minority groups was conducted for those fields for which the entire population was not sampled. In these instances, the survey results were weighted appropriately to take the oversampling into account.

Mathematics (Secondary)	Population		Sample (with oversampling)
Mathematics (Secondary)	N	%	N	%
Total	4,333	100	221	100
Female	2,793	64.45	141	63.8
Male	1,540	35.54	80	36.19
American Indian or Alaska Native	2	0.04	2	0.09
Black (not of Hispanic origin)	152	3.5	14	6.33
Asian or Pacific Islander	143	3.3	13	5.88
Hispanic	121	2.79	11	4.97
White	3,895	89.89	179	80.99
Other	20	0.46	2	0.9

Middle School Mathematics Survey Return Rate by Field and Return Status: Public School Sample

Returned Surveys

Field	Number Sent	Number Not Distributed	Eligible		Ineligible		Total		Adjusted Return Rate
Field	Number Sent	Number Not Distributed	N	%	N	%	N	%	Adjusted Return Rate
061 English	541	30	191	35.30%	32	5.90%	223	43.60%	39.90%
063 Mathematics	609	28	205	33.70%	53	8.70%	258	44.40%	38.80%
064 General Science	530	15	154	29.10%	58	10.90%	212	41.20%	33.70%
065 Middle School Mathematics	581	14	160	27.50%	78	13.40%	238	42.00%	32.70%
190 Foundations of Reading	498	18	166	33.30%	41	8.20%	207	43.10%	37.80%
ALL	2,759	105	876	31.80%	262	9.50%	1,138	42.90%	36.60%

NR	2	3	4	5
1	1	10	43	46
1	2	10	44	43
2	1	27	45	26
3	0	13	41	43
2	0	14	43	40
2	0	7	38	53
1	0	8	40	51
1	1	16	40	41
1	1	13	34	52
2	2	33	39	25
1	2	27	45	25

NR	2	3	4	5
0	0	6	27	67
3	0	6	33	58
0	0	9	45	45
3	0	3	52	42
0	0	0	39	61
0	0	0	24	76
6	0	0	21	73
3	0	0	52	45
3	3	3	30	61
0	0	15	52	33
3	0	9	42	45

NR	1	2	3	4	5
1	1	1	11	41	46
0	0	2	9	48	41
1	0	2	18	46	34
1	0	2	8	45	43
2	0	2	14	45	37
1	0	1	7	46	46
1	0	1	7	45	46
1	0	2	15	41	41
1	0	1	12	44	42
3	0	2	25	42	29
3	0	2	17	49	30

NR	1	2	3	4	5
0	0	6	6	33	55
0	0	6	9	39	45
6	0	6	12	36	39
3	0	9	9	36	42
6	3	3	6	39	42
6	3	0	6	42	42
9	0	3	6	42	39
6	0	3	9	33	48
0	3	6	12	24	55

2	3	4	5
0	13	27	20
2	15	27	16
2	9	26	23
9	24	19	8
2	18	27	13
1	9	28	22
1	10	34	15
4	10	27	20
0	2	24	35
0	3	27	31
0	5	19	37
4	22	21	14
3	15	32	11

3	4	5
1	3	3
0	4	3
1	2	3
1	5	1
1	3	2
1	4	2
0	5	1
0	4	3
1	4	2
0	5	2
0	4	3
0	4	3
0	4	3

1	2	3	4	5
0	1	11	26	24
0	4	10	28	20
0	4	10	23	24
2	11	22	18	9
1	8	12	26	14
0	2	10	30	20
0	3	8	28	22
0	1	14	23	24
0	4	6	37	15
0	1	6	29	26
1	1	3	30	25
0	10	23	19	9
0	5	20	22	13

2	3	4	5
0	1	3	3
0	1	5	1
0	1	4	2
0	0	5	2
1	1	4	1
0	0	3	4
0	0	6	1
0	0	6	1
0	0	5	2
0	0	5	2
0	0	5	2
1	2	2	1
0	1	4	2

1	2	3	4	5
0	1	10	16	30
0	0	4	18	32
0	4	18	18	16
0	0	4	23	28
0	0	7	21	29
0	0	6	23	28
0	0	0	21	35
0	0	7	23	26
0	1	15	21	18
0	0	6	26	24
0	0	14	28	14
1	2	17	28	9
0	2	25	22	8
1	3	22	24	7
0	1	21	24	11
2	6	25	19	5
0	4	13	16	23
0	2	11	19	24

2	3	4	5
0	1	2	2
0	1	1	3
1	1	3	0
0	0	2	3
0	0	3	2
0	1	1	3
0	0	2	3
0	1	0	4
0	3	1	1
0	3	1	1
0	2	0	3
0	3	1	1
1	2	1	1
0	3	1	1
0	2	3	0
1	3	1	0
0	3	2	0
0	2	3	0

NR	2	3	4	5
1	1	10	43	46
1	2	10	44	43
2	1	27	45	26
3	0	13	41	43
2	0	14	43	40
2	0	7	38	53
1	0	8	40	51
1	1	16	40	41
1	1	13	34	52
2	2	33	39	25
1	2	27	45	25

NR	2	3	4	5
0	0	6	27	67
3	0	6	33	58
0	0	9	45	45
3	0	3	52	42
0	0	0	39	61
0	0	0	24	76
6	0	0	21	73
3	0	0	52	45
3	3	3	30	61
0	0	15	52	33
3	0	9	42	45

NR	1	2	3	4	5
1	1	1	11	41	46
0	0	2	9	48	41
1	0	2	18	46	34
1	0	2	8	45	43
2	0	2	14	45	37
1	0	1	7	46	46
1	0	1	7	45	46
1	0	2	15	41	41
1	0	1	12	44	42
3	0	2	25	42	29
3	0	2	17	49	30

NR	1	2	3	4	5
0	0	6	6	33	55
0	0	6	9	39	45
6	0	6	12	36	39
3	0	9	9	36	42
6	3	3	6	39	42
6	3	0	6	42	42
9	0	3	6	42	39
6	0	3	9	33	48
0	3	6	12	24	55

2	3	4	5
0	13	27	20
2	15	27	16
2	9	26	23
9	24	19	8
2	18	27	13
1	9	28	22
1	10	34	15
4	10	27	20
0	2	24	35
0	3	27	31
0	5	19	37
4	22	21	14
3	15	32	11

3	4	5
1	3	3
0	4	3
1	2	3
1	5	1
1	3	2
1	4	2
0	5	1
0	4	3
1	4	2
0	5	2
0	4	3
0	4	3
0	4	3

1	2	3	4	5
0	1	11	26	24
0	4	10	28	20
0	4	10	23	24
2	11	22	18	9
1	8	12	26	14
0	2	10	30	20
0	3	8	28	22
0	1	14	23	24
0	4	6	37	15
0	1	6	29	26
1	1	3	30	25
0	10	23	19	9
0	5	20	22	13

2	3	4	5
0	1	3	3
0	1	5	1
0	1	4	2
0	0	5	2
1	1	4	1
0	0	3	4
0	0	6	1
0	0	6	1
0	0	5	2
0	0	5	2
0	0	5	2
1	2	2	1
0	1	4	2

1	2	3	4	5
0	1	10	16	30
0	0	4	18	32
0	4	18	18	16
0	0	4	23	28
0	0	7	21	29
0	0	6	23	28
0	0	0	21	35
0	0	7	23	26
0	1	15	21	18
0	0	6	26	24
0	0	14	28	14
1	2	17	28	9
0	2	25	22	8
1	3	22	24	7
0	1	21	24	11
2	6	25	19	5
0	4	13	16	23
0	2	11	19	24

2	3	4	5
0	1	2	2
0	1	1	3
1	1	3	0
0	0	2	3
0	0	3	2
0	1	1	3
0	0	2	3
0	1	0	4
0	3	1	1
0	3	1	1
0	2	0	3
0	3	1	1
1	2	1	1
0	3	1	1
0	2	3	0
1	3	1	0
0	3	2	0
0	2	3	0

Supplemental Test Development Information

Technology/Engineering

NR	2	3	4	5
1	1	10	43	46
1	2	10	44	43
2	1	27	45	26
3	0	13	41	43
2	0	14	43	40
2	0	7	38	53
1	0	8	40	51
1	1	16	40	41
1	1	13	34	52
2	2	33	39	25
1	2	27	45	25

NR	2	3	4	5
0	0	6	27	67
3	0	6	33	58
0	0	9	45	45
3	0	3	52	42
0	0	0	39	61
0	0	0	24	76
6	0	0	21	73
3	0	0	52	45
3	3	3	30	61
0	0	15	52	33
3	0	9	42	45

NR	1	2	3	4	5
1	1	1	11	41	46
0	0	2	9	48	41
1	0	2	18	46	34
1	0	2	8	45	43
2	0	2	14	45	37
1	0	1	7	46	46
1	0	1	7	45	46
1	0	2	15	41	41
1	0	1	12	44	42
3	0	2	25	42	29
3	0	2	17	49	30

NR	1	2	3	4	5
0	0	6	6	33	55
0	0	6	9	39	45
6	0	6	12	36	39
3	0	9	9	36	42
6	3	3	6	39	42
6	3	0	6	42	42
9	0	3	6	42	39
6	0	3	9	33	48
0	3	6	12	24	55

2	3	4	5
0	13	27	20
2	15	27	16
2	9	26	23
9	24	19	8
2	18	27	13
1	9	28	22
1	10	34	15
4	10	27	20
0	2	24	35
0	3	27	31
0	5	19	37
4	22	21	14
3	15	32	11

3	4	5
1	3	3
0	4	3
1	2	3
1	5	1
1	3	2
1	4	2
0	5	1
0	4	3
1	4	2
0	5	2
0	4	3
0	4	3
0	4	3

1	2	3	4	5
0	1	11	26	24
0	4	10	28	20
0	4	10	23	24
2	11	22	18	9
1	8	12	26	14
0	2	10	30	20
0	3	8	28	22
0	1	14	23	24
0	4	6	37	15
0	1	6	29	26
1	1	3	30	25
0	10	23	19	9
0	5	20	22	13

2	3	4	5
0	1	3	3
0	1	5	1
0	1	4	2
0	0	5	2
1	1	4	1
0	0	3	4
0	0	6	1
0	0	6	1
0	0	5	2
0	0	5	2
0	0	5	2
1	2	2	1
0	1	4	2

1	2	3	4	5
0	1	10	16	30
0	0	4	18	32
0	4	18	18	16
0	0	4	23	28
0	0	7	21	29
0	0	6	23	28
0	0	0	21	35
0	0	7	23	26
0	1	15	21	18
0	0	6	26	24
0	0	14	28	14
1	2	17	28	9
0	2	25	22	8
1	3	22	24	7
0	1	21	24	11
2	6	25	19	5
0	4	13	16	23
0	2	11	19	24

2	3	4	5
0	1	2	2
0	1	1	3
1	1	3	0
0	0	2	3
0	0	3	2
0	1	1	3
0	0	2	3
0	1	0	4
0	3	1	1
0	3	1	1
0	2	0	3
0	3	1	1
1	2	1	1
0	3	1	1
0	2	3	0
1	3	1	0
0	3	2	0
0	2	3	0