Assignment 6
PSYED 2001/EDUC 2201: Introduction to Research Methodology
This homework assignment is based on an article about the development of an instrument known
as the Student Course Engagement Questionnaire (SCEQ) to measure undergraduate students’
degree of engagement in a course. Each question is accompanied by a summary of the relevant
section of the article. You should be able to complete the assignment without reading the full
text of the article, but it is available in a separate document for students who may be interested in
reading it. Type your answers below each question.
1.
In the introduction section of the article the authors express the viewpoint that observing
students’ overt behaviors (such as asking questions during class) would NOT yield a
valid measure of their engagement because such behaviors are influenced to some extent
by students’ personalities and cultural background. To what extent does your experience
as a student and/or as a teacher support the authors’ viewpoint?
2.
The authors began the process of instrument development by “asking undergraduates and
faculty to describe what engaged students do, feel, and think”. Identify two observable
behaviors that you believe to be typical of what engaged students (in a face-to-face
undergraduate course) do.
3.
The method of selecting a sample for the initial “tryout” of the instrument is described in
the excerpt below.
We recruited participants from among students at the University of Colorado at
Denver, an urban commuter campus that comprises not only a typical college
population but also a substantial number of nontraditional students–older,
working adults who attend school part time. The sampling process was not totally
random; not all students had an equal chance to participate. Rather, we recruited
participants by visiting a variety of classes that represented two levels (upper
division and lower division) in each of three disciplines–psychology, political
science, and mathematics. We collected data from 266 undergraduates (90 men,
176 women). Their ages ranged from 18 to 56 years (M = 23.02, SD = 6.29),
which reflected the composition of the general student body.
To what extent do you think this sample of 266 undergraduates is representative of the
population of undergraduate students in American colleges and universities?
4.
In evaluating the criterion-related validity of the instrument, the authors used the
following criteria. Keeping in mind the principle that the ideal criterion is in itself a valid
measure of the trait or characteristic measured by the instrument in question, what do you
see as the strengths/weaknesses of each of these criteria?
1
PSYED 2001/EDUC 2201: Introduction to Research Methodology
Assignment 6
(1) Students self report of their engagement in the course assessed by the two
items designed by the authors as specified below.
“How engaged are you in this class? (six-point Likert scale with 1 = not at all
engaged, all the way up to 6 = extremely engaged)”
“How engaged are you in this class, compared to the other courses you’re taking
this semester? (six-point Likert scale with 1 = less engaged than in any of my
other courses all the way up tp 6 = more engaged than in any of my other
courses).”
Strengths:
Weaknesses:
(2) Students’ goal orientation for the class (performance goals vs. learning goals)
assessed by an item designed by the authors as specified below.
“If I had to choose between getting a good grade and being challenged in class, I
would choose:–‘good grade’—‘being challenged.’
(Students who chose ‘good grade’ were classified as having performance goals;
students who chose ‘being challenged’ were classified as having learning goals.)
Strengths: (hint: if you can’t find strength for this item, you can leave it blank)
Weaknesses:
(3) Three measures of students’ achievement in the course: average grade on
weekly homework assignments, grade on midterm exam, and grade on final exam.
Strengths:
Weaknesses:
2
PSYED 2001/EDUC 2201: Introduction to Research Methodology
Assignment 6
5.
The authors found that the instrument had high reliability based on the internal
consistency approach, but they did not assess test-retest reliability for the instrument.
Why the lack of evidence of test-retest reliability to be a concern? (Consider that the
authors suggest that a possible use of the instrument would be to compare students’ level
of engagement before and after an intervention such as a change in instructional method.)
6.
Do you think it would be safe to assume that the SCEQ would be valid for graduate
students based solely on evidence of its validity for undergraduate students? Why or
why not?
3

Purchase answer to see full
attachment