Developing
an Assessment Plan
for Majors,
Interdisciplinary Programs, and Other Academic Programs
The
typical assessment cycle.
In
general:
·
Assessment
is cyclic: results from one assessment
cycle inform the next assessment cycle.
·
Reflecting
upon the results is critical: build
time for the faculty to do this.
·
Use
the results to adjust the curriculum or pedagogy for the main reason for
assessment: to improve student
learning.
Basic Assessment Planning Questions:
v
How will the learning outcomes be established?
v
What are the outcomes: What would an ideal student know,
be able to do, or value?
v
What measures will be used with each outcome?
v
How will the results be reviewed and decisions made?
v
What is the timeline for implementing the plan?
v
What technical assistance may be needed?
What is an effective assessment plan?
An effective plan will:
1. start with the Department’s mission statement;
2. be built upon the program’s goals or learning outcomes;
3. involve more than one type of measure;
4. be developed and carried out by the program faculty;
5. address the real questions and concerns faculty may have about student learning in their program;
6. be used to improve student learning, not merely satisfy a requirement that the program faculty must assess student learning.
Example of an institutional assessment plan from IUPUI, 1994.
Goals, objectives, learning outcomes—What’s the
difference?
Think of what your students should be able to do or know or believe that would demonstrate what they have learned: this is what the literature on assessment refers to when speaking of learning outcomes. Goals and objectives both refer to what the Department intends students to learn. Goals often include objectives. For example: “Students will know the content of their field” is a broad goal. “Students will be able to define key terms in the field” is a more narrow learning objective. Don’t worry about the difference: concentrate on determining what the outcomes should be and what evidence will demonstrate those. As Dary Erwin[1] phrased it, “One must know what is to be assessed before one knows how to assess it.”
How many students do we collect data from, how many
sections: all or a sample?
That depends upon factors such as the number of majors that you have. If you have a large program, a sample taken from a cross-section of the students will do. If you have a small program, e.g., 15 or fewer seniors in the major, then assessing all makes more sense. If there are many sections of a course, then you can sample by student or by section.
If we haven’t specified our objectives or our learning outcomes, how should we do it?
You might work through
the following questions:
1. What would the ideal graduate of our program look
like (knowledge, skills, beliefs and values)?
2. What experiences (assignments, papers, productions, internships,
etc.) do students carry out through our program that would provide
evidence of their achievements?
3. What standards would we expect our students to
achieve for those experiences?
4. Can we express those experiences and standards in
ways that would both guide our students in determining whether they have
achieved what we want and provide us clear criteria for our assessments?
Are there other ways of determining our learning outcomes?
Inventories:
1. Review the syllabi for all of your courses to list
what is taught in each course. Based
upon that, what appear to be the broad goals or the learning outcomes for the
program? Create a spreadsheet that
lists the broad goals or the learning outcomes in the left hand column, then
list all the courses across the top row, and then note which courses address
which goals. Sometimes, doing this
exercise reveals gaps in the program or unnecessary repetition of the same
skills in many courses. This inventory,
in itself, is a form of assessment, but not an assessment of what students
learn.
2. List all the major assignments and tests in all your
courses. Given the breadth and depth of
all the courses, is the distribution of these assignments appropriate for
addressing the learning outcomes you want from your program.
Research:
1. Contact colleagues from across the nation to learn
what they are doing?
2. Go online to find out what other departments are
doing in your field? Perhaps start with
the assessment website at Skidmore.
3. Note assessment sessions at your national conferences.
4. If your discipline has teaching journals, review
articles on assessment.
Review:
1. Catalog copy to determine whether you tell
prospective majors what they should expect to learn by the time they graduate
from your program.
2. Other materials you have already produced: annual reports, program reviews,
accreditation reports, recruiting materials.
We have the learning objectives or outcomes? How do we decide the best way to assess them?
1. Review the section of this handbook on different
assessment methods. Decide which are
most appropriate for the type of work your students are expected to do, what
you want to learn about their achievements, and when the best time to assess
that work would be.
2. Use techniques that you are already familiar with in
your discipline. For example, if it is
not common to do statistical analyses in your field, don’t try to force it for
assessment purposes. If qualitative
studies are inappropriate in your field, don’t force yourselves to develop
descriptive studies.
3. Determine whether you are already assessing student
work in ways that could be employed for assessment purposes. For example, do all your students have to
take a capstone course or perform field research or undergo an internship experience? Can you systematically evaluate those for
assessment purposes? Are student performances evaluated by groups of faculty or
professionals in the field? Can those
be adapted for assessment purposes? Do
the faculty as a group or a sub-group of the faculty evaluate student progress
in the earlier stages of your program? Can those evaluations be employed as one
of your assessment methods?
4. The assessment process should involve the entire
faculty. All faculty should review the
results and discuss whether any improvements need to be made. Don’t ask one faculty member to carry out
any one assessment. Individual faculty
give grades in their courses and can gather the examples of student work that
you need, but groups of faculty review the evidence for assessment purposes.
How to Write Learning
Objectives
Learning objectives specify both an observable behavior and the object of that
behavior.
For example:
Students will be able to
write a research paper.
In addition, the criterion could also be specified:
Students will be able to write a research paper in the appropriate scientific style.
Optionally, the condition under which the behavior occurs can be specified:
At the end of their field research, students will be able to write a research paper in the appropriate scientific style.
Note that the verb you choose will help you focus on what you assess. For example:
Students will be able to do research.
The verb do is vague. Do you mean identify an appropriate research question, review the literature, establish hypotheses, use research technology, collect data, analyze data, interpret results, draw conclusions, recommend further research, or all of those? Each of the verbs in those statements is appropriately specific.
More examples. The more specific example is easier to assess than the broad example:
A.
Broad:
Students will demonstrate knowledge of the history, literature and function of the theatre, including works from various periods and cultures.
More specific:
Students will be able to explain the theoretical
bases of various dramatic genres and illustrate them with examples from plays
of different eras.
Even more specific, specifying
the conditions:
During the senior dramatic literature course, the students will be able to explain the theoretical bases of various dramatic genres and illustrate them with examples from plays of different eras.
B.
Broad:
The student will be able to discuss philosophical questions.
More
specific:
The student is able to develop relevant examples and to express the significance of philosophical questions.
C.
Broad:
Students will be able to think in an interdisciplinary manner.
More specific:
Asked to solve a problem in the student’s field, the student will be able to draw from theories, principles, and/or knowledge from other disciplines to help solve the problem.
D.
Broad:
Each student will be able to function as a team member.
More specific:
Each student will reflect upon his or her contributions to a team effort, ability to accept other team members as resources, and willingness to accept compromises if required to achieve a team goal.
E.
Broad:
Students will understand how to use
technology effectively.
More specific:
Each student
will be able to use word processing, spreadsheets, databases, and presentation
graphics in preparing their final research project and report.
Summary of assessable
learning outcomes:
1. They use verbs that indicate how the student work can be observed.
2. They focus on what the student should do, not what the instructor teaches.
3. They reflect what students should be able to do after a course ends, not simply what they do during the course.
4. They usually can be assessed in more than one way.
5. They can be understood by someone outside the discipline.
Bloom’s Taxonomy of
Educational Objectives[2]
has served to exemplify the verbs that faculty might use in writing their
program’s learning objectives:
Cognition
Knowledge Comprehension Application
define annotate apply
describe explain demonstrate
recall give examples illustrate
state predict solve
list infer manipulate
summarize interpret interview
identify generalize construct
point to calculate draw
match convert perform
Analysis Synthesis Evaluation
subdivide write evaluate
compare create assess
contrast compose critique
identify formulate prioritize
infer outline defend
distinguish plan judge
diagram conceive recommend
illustrate hypothesize defend
categorize predict select
Sample
affective learning verbs[3]:
Volunteer, support, question, praise, join, defend, challenge, attempt
Sample psychomotor learning verbs[4]:
Dance, sing, draw, bend, operate, reach, relax, shorten, lift, throw, hit