Use of Student Satisfaction Data

There has been an interesting change in medical education over the time that I have been a faculty member. I am sure that many people involved in education have seen the same change. It appears that the phenomenon occurs across multiple, different disciplines. The change is related is how we use student satisfaction data. And even more specifically, what importance is put on student satisfaction data, how it’s used for curricular change, and how it is used to drive the curriculum.

I had a lot of great experiences when I was a medical student. But, I had one really interesting (not-so-great) experience. We had a professor who was very well-liked. He was a great teacher and a student advocate. He was one of those teachers that was always available to meet with a student who didn’t understand concepts or to do individual tutoring sessions. He was very personable. He was also a great lecturer, he always delivered content that was relevant and useful. His test questions were consistent with the lectures that he gave and the information that was in the lectures that he gave.

As a student, he was what most of us thought of as the gold standard for a faculty educator. But he was not a researcher. And in many academic departments and medical schools, you have to be a researcher first, and a teacher second. If you don’t get enough research grants, then your job is at risk. And that’s what happened in this situation: his contract was not renewed, because he did not receive tenure.

My medical school class was the group of students that worked with him the most because of the course that he taught. We were very upset by the school's decision. We wanted to do something to express our dissatisfaction with this result. He was one of our best teachers. We did not think it was right to get rid of one of our best teachers. So, we wrote a very professional letter to the Dean of the medical school, expressing our feelings about this decision. The entire class signed it. There were over a hundred people in the class, everyone signed this letter. Then our class president took this letter, you can call it a petition, I guess, but it was really just a letter, to the Dean of the medical school.

What happened next is shocking to me now, but at the time it was not surprising. The Dean basically threw the class president out of his office and told him never to come back. He said something to the effect of “This is my medical school, and I’m charge and the students don’t have any say in what goes on.” Actually, at the time, there was a story going around that he said, “If you complain about anything else, I’ll break your kneecaps,” or something like that. But that may have just been a story that was made up by med students afterwards.

Fast forward to 2014...  For every course that we deliver in our medical school (and in every medical school that I know of) we ask the students to comment on the process.  We ask them to comment on the policies that are in place. We ask them to comment on the learning objectives and whether the content matched the learning objectives. We ask them about the content of the course. We ask them a lot for their opinion about their satisfaction with the curriculum. There’re some really good things about this. Clearly, the dean’s response to my class’s dissatisfaction was not a great response. But I think it is possible for the pendulum to swing too far the other direction.

In a lot of medical schools, students have this idea that they are helping to determine the curriculum. I think that is a dangerous and, in many ways, nonsensical idea. In addition, the LCME looks at student satisfaction on the Graduate Questionnaire, a survey that is sent out to all medical students after graduation. In the accreditation visits for each medical school, the LCME uses that data to determine accreditation for the medical schools. Again, this is a very dangerous proposition in many ways.

Now, don’t get me wrong. I am very interested in student satisfaction data. But the data that I think that they should be giving us is how the content was delivered, how accessible were the faculty, did they follow the policies that we have set forth for grading? Did the questions that were on their test match up with the content that was delivered during their lectures? Was their lecture style appropriate? Did they understand the lecture? Were their slides helpful and additive to the lectures? Those are the kind of questions that student satisfaction data would be pretty useful in answering. And we do some of that.

But often, we ask them other questions. For example: do you think that the information in this course will help you to be a better doctor? That question bothers me a lot. I really have trouble seeing how a medical student would know the answer to that or how their answer to that would change what I want to do as far as a course that I am teaching. I see medical students in their comments, write things like, “the information in this lecture is not important,” or “the information in this course is fluff.” When I think about that, I wonder. Does a student have the framework to make that call?

One of the things that is important for faculty and course directors and medical schools to do is to think about what the curriculum should look like. What information do you want the students to learn? To some extent, that is driven by the national examinations, like USMLE. There is content that every student will be tested on in the national standardized tests. The curriculum needs to do a good job of preparing the students to take those tests. But how the other pieces of the curriculum are emphasized should be the judgment of the faculty.

It often seems like, students are giving their opinion. That is opinion is based on what they think is important, but with little background or context. So for instance, we had a student comment that said, “I didn’t like going to this clinical site.” I thought it was an interesting comment. We specifically chose that as a clinical site for all the students, because we wanted them to have the experience. It was an underserved practice. We wanted them to be exposed to a clinic in an underserved area so that they might see how it could be useful to them in their future practice. Most of the students really got that. But occasionally, there’d be a student who just didn’t really think that that was important. So I think, okay, so you didn’t think it was important, but I do.

Another thing that students comment on in satisfaction surveys are things that are outside of their expertise. It is often comments like “I don’t understand why this test counts for so much of my grade.” I think it should count for that much of your grade. It doesn’t make any difference that the student doesn’t agree with me.

I say all of this to come back to the curricular review and development process. Every course in medical school should be evaluated on a regular basis. The satisfaction surveys that students do are given a lot of weight. If students make a lot of negative comments, or there are several things that they do not like, there is a push from the Dean’s office to look at those comments. The administration may encourage the director to change their course in some way. That is problematic, because it takes the power of determining the direction of the medical curriculum away from the faculty and gives it to students. The faculty are the most experienced and most able to determine what the curriculum should look like but this gives curricular control to the least experienced and least able to make decisions, the medical students.

I’m not advocating that we should go back to the place where the dean can kick you out of his office and tell you he doesn’t want you to ever come back, but I do think that it’s important for the medical school faculty to drive the curriculum. Student comments should be limited to the delivery and the process and the policies, and stay out of what the content is and how that content is weighted.