I.
Methodology:
Phyllis Roth and Kate Berheide conducted an e-mail survey of all our faculty and asked them to write back, identifying what they considered to be characteristics of students whom they would describe as “engaged.” 40 faculty responded.
Since there was no formal structure to the responses, those faculty who responded did so in varied ways, ranging from discussing one or two characteristics quite extensively and sometimes eloquently to listing a number of characteristics with little or no discussion. Therefore I decided to identify each characteristic that each faculty member mentioned and tally each as a separate response, even though one faculty member may have listed 5 characteristics and another faculty member may have listed 1. If a faculty member talked about the same characteristic in several ways, I counted it once.
Then, chosing to lump rather than split, I tried to identify the general characteristics that were mentioned, even though several faculty would describe the same characteristic in very different ways. This initial clustering of characteristics resulted in 14 characteristics, which I have ranked by the percentage of faculty who mentioned each characteristic (see section II below). Then, for purposes of our definitions, I attempted to collapse the 14 characteristics into fewer categories so that we could more easily have a working definition that would not overwhelm a reader or reviewer and that could serve assessment purposes. I have summarized those below (see section III).
I have ranked fourteen characteristics below in order of decreasing frequency of response by different faculty (the percentage indicates the percentage of faculty who mentioned anything related to this category):
Rank (topic
identifier letter) Characteristic (percentage)
1. (A) Do a deeper analysis, question one’s own assumptions, achieve more in-depth understanding, make conceptual leaps, develop internal standards, develop creative solutions: 35%
2. (B) Continue working after a class, lab, or field trip: 25%
2. © Voluntarily extend the topic of a class into additional research, further readings, or outside activities such as service learning or student activities: 25%
3. (D)Apply what has been learned to a different context, problem, or task: 22.5%
4. (E) Discover a passion for something, become obsessed with the topic, become physically excited about the topic, demonstrate enthusiasm: 20%
4. (F) Suggest new readings, new approaches, new questions: 20%
5. (G) Be willing to take on more work or work on weaknesses: 15%
6. (H) Arrive on site or in class prepared to begin: 12.5%
6. (I) Talk to others about their learning, involve other audiences, mentor or tutor students in the subject: 12.5%
7. (J) Risk failure to learn more: 10%
8. (K) Develop career or academic goals as a result of the study, course or major: 7.5%
8. (L) Develop a real voice in writing, argue persuasively, exhibit craftsmanship or identifiable style: 7.5%
9. (M) Connect learning to personal experiences, prior knowledge, interests, other courses they are or have taken: 5%
10. (N) Collaborate with others: 2.5%
III.
Clustering the Characteristics:
Attempting to determine which of the initial characteristics related to one another, I have clustered the following, as identified by the letters above:
1. Develop a deeper analysis, achieve greater understanding, make conceptual leaps, develop internal standards, develop creative solutions, and question one’s own assumptions. (A)
2. Exhibit a real voice, craftsmanship, or identifiable style. (L)
3. Prepare thoroughly for class, be willing to undertake more work, and continue work after class or in new contexts. (B, C, D, G, N)
4. Risk failure to learn more. (J)
5. Demonstrate a passionate commitment to the study now and in the future. (H, I, J, K)
6. Apply learning to a new context, work, or activity and relate learning to personal experiences or prior knowledge. (F, M, E)
I note that characteristics 1 and 2 refer to the products of one’s work; items 3, 4, and 5 refer to the attitudes or predispositions of the students toward their study or the topic; and item 6 refers to the students extending what they have learned to something different from the course itself, ranging from current student activities to careers.
A disclaimer: I recognize that my interpretation of the faculty comments is just that, an interpretation perhaps biased by my own subjective understandings of student learning and personal responses to what the faculty have told us. I also recognize that, sadly, my clustering or lumping of their responses necessarily eliminates many nuances and eloquent phrases. Finally, a subsequent study might begin with these initial responses of the faculty and employ a Delphi technique[1] approach to enable the faculty to see these possibilities and rank them. That might lead to greater consensus among the total faculty.
Based upon the responses of 40 faculty, the identification of 14 broad characteristics derived from the faculty responses, and the subsequent clustering of those into 6 categories, I conclude that the faculty identified “student engagement” to be revealed by the attitudes that students bring to their academic work, the products of their work, and the extension of their learning beyond the class itself.
At this point, we can say that the faculty have provided us with 6 characteristics of student engagement or student outcomes that broadly define student engagement. We can use these as we assess student engagement in the future or we can attempt to redefine them so that they more accurately reflect what we hope for in an engaged Skidmore student.
--Ray Rodrigues
[1] The Delphi technique reflects the Hegelian dialectic: thesis and antithesis leading to synthesis, with synthesis becoming the new thesis. Essentially, using this technique, a first survey identifies possibilities, and subsequent surveys each use the responses from the previous survey to construct new surveys, moving toward and attempting to achieve a final consensus.