Dimensions of Learning II

EDU 503

Belhaven University

Unit 5, Part 2

Assessment

1

Class Introduction

? Test-item development is a critical

component of classroom instruction. Test-

item development for day-to-day instructional

practice is often without blueprint specifics

and meeting the following targets:

? Measuring desired skill(s) and objective

? Minimum scoring time

? Objective rather than subjective

? Informing the daily instructional practice

2

Topics we?ll cover

? Developing Good Test Items?learn and explore

rules for creating a better test item for a valid

assessment.

3

Class Objectives

? Learn and explore rules for creating a better test

item for a valid assessment.

4

Biblical Foundation

? And whatever you do, in word or in deed, do

everything in the name of the Lord Jesus,

giving thanks to God the Father through him.

Colossians 3:17

5

Deciding on a testing format

6

? The choice of item format is sometimes

determined by your instructional objectives.

? At other times, the advantages and

disadvantages of the different formats should

influence your choice.

? What is the goal of your test?

? Design items where guessing is minimized

? Test items should be objective in scoring

Print the handout ?Test Item
Activity?; then record your answers
on your printout

Pretest your knowledge: Determine
whether each of the test items are
good (G) or poor (P)

Interactive activity

True-False items

8

? True-false items

? require less time to construct

? are most prone to guessing
? have a variety of other faults

? Faults include (features that make poor items):

? Absolutes in wording (i.e., always, all, never, only)

? Double negatives

? Opinionated

? Double-barreled statements (i.e., use of and, or)

? Excessive wordiness

? A tendency to reflect statements taken verbatim from
readings (w/o context, or out of context)

Section I ? True/False

T/F items are popular because they are easy and quick to write ? or seem

to be; however, good T/F items are not that easy to write.

Instructions: Consider the following and use common sense to determine

which are good (G) and which are poor (P).

__1. High IQ children always get high grades in school

__2. Will Rogers stated, ?I never met a man I didn?t like.?

__3. If a plane crashed on the Mexican-US border, half the survivors

would be buried in Mexico and half in the United States.

__4. The use of double negatives is not an altogether undesirable

characteristic of diplomats and academicians.

__5. Prayer should not be outlawed in schools.

__6. Of the objective items, true/false items are the least time consuming

to construct.

__7. The trend toward competency testing of high school graduates began

in the late 1970s and represents a big step forward for slow learners.

? 1- always is an absolute and absolutes (all, always, never) cue student that the question is

false

? 2-G ? in order to answer, students would have to know that Will Rogers made the statement

? 3-survivors are not buried, and students would suspect this is a trick question. This statement

should have perhaps used fatalities instead of survivors

? 4-not and undesirable are both negatives ? a single negative in a test question is confusing

enough for students, make positive statements instead ? The use of double negatives is an

altogether desirable trait of diplomats and academicians. However, this item is still flawed in

that it states an opinion, not a fact. It is better written ?According to the National Institute of

Diplomacy, the use of double negatives is a desirable trait of diplomats and academicians.?

? 5-opinion ? therefore not obviously true or false. Rewritten, ?The ACLU has taken the position

that prayer should not be outlawed in schools? and Notice the negative not is italicized, when

using a negative, italicize!

? 6-G-this is factual information from the text

? 7-this is a double-barreled item?there are two parts, the learner has to determine if both are

true or both are false or is one false and one true ? simply construct two items here

The trend toward competency testing of high school graduates began in the late 1970s

The trend toward competency testing represents a big step forward for slow learners

True/False key

1. Instructions for marking true or false should be explained

2. Construct statements that are definitely true or definitely false ?

opinions must be attributed to a source

3. Use relatively short statements and eliminate extraneous material

4. Keep true and false statements at approximately the same length

and approximately of equal number

5. Avoid double negatives, absolutes and terms denoting indefinite

degrees (long time, regularly, never, only),

6. Avoid patterns TTFFTTFF OR TFTFTFTF and so on,

7. Avoid taking statements from the text and presenting them out of

context

8. And, always allowing the student the opportunity for a restricted

response to explain why the statement is false is good practice

How to construct good true/false items

Completion Items

Completion items (fill in the blank)
? rival true-false items in ease of construction.
? Since answers must be supplied, they are

least subject to guessing.
? require more scoring time

Faults include:
? Too many blanks
? Lack of specificity (too many potential

responses)
? Failure to state a problem

Section II – Completion Items

Instructions: Consider the following and use common sense to determine which

are good (G) and which are poor (P)

__1. The evolutionary theory of ______ is based on the principle of ________.

__2. Columbus discovered America in ______.

__3. The capital of Mexico is ______.

__4. In what year did William J. Clinton become president of the United States?

______

__5. ______ blanks cause much frustration in ______.

__6. ______ was the first American to ______.

Completion – Key

? 1-avoid more than one blank per question. The evolutionary

theory of Darwin is based on the principle of survival of the

fittest

? 2-most people would choose Good; however, a plausible

answer could be fifteenth century

? 3-poor as ?is larger than the capital of Alaska,? or ?is a

beautiful city? ? rewritten, ?The name of the capital city of

Mexico is _____.?

? 4-G because no other answer seems plausible here

? 5 & 6 too many blanks

Suggestions for Writing Completion (or

Supply) Items

If possible, items should require a single-word answer/brief and
definite statement.

The answer required is factually correct.

Omit clue/key words
Don?t eliminate so many elements that the sense of the content is impaired.

Write the blank near the end of the sentence rather than near the
beginning.

This will prevent awkward sentences.

If the problem requires a numerical answer, indicate the units in which
it is to be expressed (pounds, inches, etc.)

Multiple-Choice Items

Multiple-choice items
? Measure behavior at higher levels of taxonomy (ex:

comprehension +)
? most difficult of the objective items to construct
? Should be used with caution on younger children

Faults included:
? Grammatical cues or specific determiners
? Multiple defensible answers
? Unordered option lists
? Stem clues
? Opinionated statements
? Lack of a problem statement in the stem
? Redundant wording
? Wordiness in the correct option
? Use of ?all of the above?
? Indiscriminate use of ?none of the above?

Section III – Multiple Choice

They are unique in that they enable the teacher to measure at the higher levels of taxonomy as well as at the basic

knowledge level.

Instructions: Consider the following and use common sense to determine which are good (G) and which are poor (P).

? 1-grammatical clue ?an? eliminates options a, b, and d – better written

Grant was a/an and there are multiple defensible answers! Better

written ? Who was elected president after the Civil War? A) U.S.

Grant B) Andrew Johnson C) Abraham Lincoln D) Andrew Jackson

? 2-G ? always arrange dates in chronological order

? 3-stem clue ? eliminate the word free

? 4-opinion- rewritten better is ?The USDA states that the principal ?.?

? 5-more than one defensible answer

? 6-gramatical clue ? are ? make substitute is/are AND always use this

format sparingly AND, italicize NOT

? 7-eliminate redundancy in options, also length of options is a giveaway

and test-wise students will be able to guess correctly ? avoid where

correct answers are 1.5 times the length of incorrect options

? 8-?none of the above? should be used sparingly, test-wise students

tend to choose ?none of the above? and the correct option is e

? 9-G

Multiple Choice – Key

Suggestions for Writing Multiple-Choice Items for

Higher-Order thinking
1-use pictures and graphs to measure higher-order knowledge

2-use analogies that demonstrate relationships among terms

(Physician is to humans as veterinarian is to ? a) fruits b) animals c) minerals d)

vegetables)

3-require usage of previously learned principles/recall analogies (figuring miles per

gallon, perimeter, etc.)

4-the stem should formulate ?a problem?; the response needs to be short but include

only the material needed to make the problem clear ? don?t add extraneous information

5-be sure there is only one best choice

6-wrong answer choices should be plausible

7-eliminate grammatical clues, keep length equal, rotate the position of the correct

answer

8-include four or five options to minimize guessing

9-avoid ?all of the above? and use ?none of the above? sparingly

Matching Items

Matching Items
? fairly easy to construct
? uses reversal of options and descriptions
? BUT?

? May lack of clarity/specificity in directions
? Dissimilar and non-ordered lists

Faults include
? Non-homogeneous lists (ex: list with states, capitals,

cities)
? Improper ordering of lists (ex: mixing up descriptors with

indicators)
? Easy guessing ? should be more answers than questions
? Poor directions ? should ID bases of matching
? Too many responses
? Ambiguous lists ? should use first & last names

Matching ? Analyze the following:

__1. Lincoln a) President during the twentieth century

__2. Nixon b) Invented the telephone

__3. Whitney c) Delivered the Emancipation Proclamation

__4. Ford d) Recently resigned from office

__5. Bell e) Civil rights leader

__6. King f) Invented the cotton gin

__7. Washington g) Our first president

__8. Roosevelt h) Only president elected for more than two terms

__1. A president not elected to office a) Gerald Ford

__2. Delivered the Emancipation Proclamation b) Thomas Jefferson

__3. Only president to resign from office c) Abraham Lincoln

__4. Only president elected for more than two terms d) Richard Nixon

__5. Our first president e) Franklin Roosevelt

f) Theodore Roosevelt

g) George Washington

h) Woodrow Wilson

__1. Invented the cotton gin a) Alexander Graham Bell

__2. One of his inventions was the telephone b) Henry Bessemer

__3. One of his inventions was the telegraph c) Thomas Edison

d) Guglielmo Marconi

e) Eli Whitney

f) Orville Wright

Martin Luther King would be presented in T/F or M/C format since there are no other

civil rights workers listed.

A better constructed layout

Faults of matching:

1-lists that are not homogeneous ? the above contains presidents,

inventors, and a civil rights leaders. Three separate matching exercises

should be developed. This better helps student eliminate implausible

options, and three separate matching exercises would enable the teacher

to provide better distractors, eliminate guessing, etc.

2-wrong order ? columns should be reversed (description first/options

second). The student should read the longer description then glance

down the list of names. As it is written the student reads the name

Lincoln and then has to read through a lengthy list for the right answer

3-easy guessing ? same number names as descriptions is a no

no?.there should be at least three more names ? this cuts down on

guessing

4-too many correct responses for ?president during the twentieth century?

(Ford, Nixon, Roosevelt) ? or does Ford mean Henry Ford, the inventor?

5-In the directions, specify if each option is used once or more than once

Constructing essay items

? Where possible, use restricted range rather than extended range ? and
use a ?box? for the answer (1/3 of a page, 1/2 of a page, etc.). Students
must restrict their answer to the text box provided-this will eliminate
?bluffing?

? Use a pre-determined scoring rubric

? Implement the scoring rubric consistently with all students

? Remove or cover names to control for bias

? Score all responses to one item before scoring the next item

? Keep scores from previous items hidden when scoring subsequent
items

? Use essay items when test security is in question

? Use essay items when few test items are necessary

? Use essay items when high-level cognitive processes cannot be fully
measured with objective items ? using words as predict, give reasons
for, compare and contrast, NOT who, when, what

Gender and Racial Bias in Test Items

To avoid gender and/or racial biases in test items ?

? avoid using stereotypes

? be sure to make equal reference to both

males and females and to various ethnic

groups

? balance role and power references

Recap of the class

26

1. Begin writing items far enough in advance that you will have time
to revise them.

2. Match items to intended outcomes at the proper difficulty level to
provide a valid measure of instructional objectives. Limit the
question to the skill being assessed.

3. Be sure each item deals with an important aspect of the content
area and not with trivia.

4. Be sure that the problem posed is clear and unambiguous.

5. Be sure that each item is independent of all other items. The
answer to one item should not be required as a condition for
answering the next item. A hint to one answer should not be
imbedded in another item.

6. Be sure the item has one correct or best answer on which experts
would agree.

7. Prevent unintended clues to the answer in the statements or
question. Grammatical inconsistencies such as a or an give
clues to the correct answer and will help those students who are
not well prepared for the test.

8. Avoid replication of the textbook in writing test items: don?t quote
directly from textual materials.

What?s next?

? Complete the assigned reading

? Answer the discussion question

? Complete the writing assignments

27

References

Kubiszyn, T., & Borich, G. D. (2016). Educational testing &

measurement. Hoboken, MJ: Wiley & Sons.

Payne, D. A. (2003). Applied educational assessment. Belmont, CA:

Wadsworth/Thomas Learning.

Popham, J. W. (2014). Classroom assessment: What teachers need

to know. Upper Saddle River, NJ: Pearson Education Inc.

28

Dimensions of Learning II

EDU 503

Belhaven University

Unit 5, Part 1

1

Assessment

Class Introduction

2

? Online assessments? overarching goal is to

improve student learning via a systematic and

documented process.

? Focused on individual learner and school achievement

? Reliable measurement of an instructional design

model

? Outcome-based

? Compares with an adopted national standard and

provides a norm comparison of like

curriculum/objectives/targets

? Aids schools in determining learning priorities

(lack/knowledge of skills)

Topics we?ll cover

? Online Assessment Models?gain awareness of

current trends in online assessment models and

necessary test-taking skills.

? The Mississippi Department of Education?s

resources?gain awareness for a better-informed

instructional practice

3

Class Objectives

? Gain awareness of current trends in online

assessment models and necessary test-

taking skills.

? Explore online resources for a better-

informed assessment practice and

instructional practice.

4

Biblical Foundation

? The precious possession of a man is

diligence; the diligent seizes opportunity.

Proverbs 12:27

5

21st Century Assessment

6

The Assessment Model is a top/down structure

? Federal ? ESSA — recap

? President Obama’s ESSA speech – signing day

December 2015

? ESSA/NCLB – Some Comparisons

? MDE ESSA information

? State ?

? Accountability Standards

? (Business rules, page 25)

? Local ?

? College and Career Readiness Standards

https://www.mdek12.org/Accred/AAS

Page 25, performance

standards followed by

?Business Rules?

Beginning on page 43,

Graduation options, State

Board Policy 3804 & 3803

& concordance tables

Accountability Standards and Business Rules

Office of Student Assessment-MAAP

Choose a hyperlink to see data

in spreadsheet format

Office of Student Assessment (OSA)

https://www.mdek12.org/OSA

Office of Student Assessment (OSA)

See parent

resources

A look at the Normal

Distribution Student data and how you should

perceive all results

0 ? 20 percentile low

21 ? 40 percentile low average

41 ? 60 percentile average

61 ? 80 percentile high average

81 ? 99 percentile high

A student scores in the 50th percentile on a standardized test. How

will the student likely score on the same test on a different day?

A student scores in the 80th percentile on a standardized test. How

will the student likely score on the same test on a different day?

Why do you need to know who your ?bubble? students are?

The normal in a normal distribution ?.

Blueprints

Blueprints

BLUEPRINTS ? are they printed on your desk ? AND have you printed the

next year?s grade level blueprints and standards ? you must know where your

students ?are going?; what has been added to your students? skill requirement

for next year?

Consider ? If you teach 8th/9th grade Social Studies, should you also have copies of the US

History blueprint on your desk? If there are ten Civil Rights questions on the US History test

and your middle school framework has a Civil Rights strand, then you should PORTFOLIO

assess your students so they will have a portfolio (folder) to take with them to high school.

You must prepare the foundation for those strands and objectives of which your students

will be tested for their high school exit exams.

https://www.mdek12.org/sites/default/files/Offices/MD

E/OAE/OEER/Literacy/Writing/mde_questar_writing_ti

ps_handout_2.12.19.pdf

http://www.act.org/cont

ent/act/en/products-

and-services/state-

and-district-

solutions/mississippi.ht

ml

ACT Documents ?

Document libraries, etc.

ACT

https://www.mdek12.org/OA/ODSP

https://www.mdek12.org/OCGR/mact

See how districts and schools earn points!

Public Access Documents

Testing Calendar

Before the year begins, locate the testing calendar ?

Accountability Results
https://www.mdek12.org/OPR/Reporting/Accountability/2019

https://www.mdek12.org/OPR/Reporting/AccountabilityPrior years? accountability:

Standards and Frameworks
https://www.mdek12.org/OAE/college-and-career-readiness-standards

Before teaching, before testing,

before lesson planning, teachers

must ?.

-Read the standards

-Read and study the scaffolding

document

must know the grade before,

the grade you teach and the

next grade

(a 5th-grade teacher should have

4th, 5th and 6th in mind at all times)

These should be printed and on

your desk at all times. These

along with your lesson plans are

your guide to leading your students

to success.

Practice tests ? are you using these and have you informed

your students and parents of where to find online practice

tests/resources? (the beginning of the year and at the beginning of

the 2
nd

semester is a good time to provide informational hyperlinks

again)

Math and ELA testlets

https://www.mdek12.org/OSA/MAAP

Item Samplers and Practice Tests may answer parental

questions and serve as a student support for all pupils.

https://ms.nextera.questarai.com/tds/#practice

Note: This is one part of your writing assignment.

Join mailing lists

Subscribe to MDE?s listservs

Listservs

Recap of the class

25

We educators are responsible for graduating children. Through

provided online resources, we can better prepare our students and

substantially support our daily instructional practice.

? We use grade-level (and next grade-level) practice tests and

blueprints

? Align curriculum content with practice tests and blueprints

? Prepare students for grade level success using practice tests and blueprints

? We adhere to standards, benchmarks, etc., because our goal is that

our students will be content ready for their high school exit exams

(state tests).

? We keep copies of practice test questions and blueprints for our

grade and the next grade level on our desks because our goal is

preparing our students for academic success leading to graduation.

? We share online resources with students and their parents.

? We join listservs in our quest for a bett