Feedback or Compliments? Which is better?

I was getting ready to do journal club a couple of weeks ago and I came across an interesting study from a couple of years ago. Boehler and colleagues at the Southern Illinois University School of Medicine, decided to do a study of feedback (1), which happens to be one of my favorite topics.

Feedback is an old term that actually comes from rocketry and missiles. Radio signals came from the rocket back to the control tower on Earth. The direction of the rocket is then controlled by sending signals back to the rocket. Those signals are the feedback.

But it is hard! Faculty do not give students feedback often enough. At the University of Michigan, medical students were asked about the third year Surgery clerkship.(2) When asked (did you always get feedback?), students were more likely to Disagree or Strongly Disagree (p<0.001). Medical students thought that feedback was poor when compared with the opinions of faculty and residents (p<0.002). And 50% of the medical students believed that they were an inconvenience to the clinical service that they were a part of.

And faculty do not give good enough feedback. At the University of Missouri-Columbia, I did a study of PGY-1 residents who had been evaluated by faculty and senior residents.(3) We looked at the end of block written feedback comments. What we found was that 82% of all the comments were positive. 38% of all comments fell into two categories. The first category was generic comments such as “did a great job” or “is a pleasure to work with.” The second category was personal attributes, such as “has a great sense of humor.” The level or gender of the evaluator did not affect the comments (p = 0.17)

Feedback is really important in clinical skills acquisition. It helps students to learn about their own strengths and deficiencies. It offers insight into what he or she actually did and the consequences of actions. There is a dissonance between the intended result and the actual result, which hopefully becomes an impetus for change in the learner.

So Boehler’s project had two hypotheses: 1) medical students receiving compliments would be more satisfied than those receiving feedback, and 2) medical students receiving feedback would demonstrate improved performance, whereas those receiving compliments would not. They took a group of second and third year medical students at the medical school and studied their ability to learn to tie a knot. More specifically, a two-handed square surgical knot. This is a basic skill that all medical students need to learn.

The authors video-taped all of the students tying a knot. Then they gave all the students knot-tying instruction by an expert academic surgeon who was well-respected, supportive, credible, and trustworthy. They video-taped the students tying a knot again. Then the students were randomly assigned to two groups. One group received specific feedback about their knot tying skills (ie: this hand needs to be in this position), the other group received generic compliments about their technique (ie: you are doing a great job). After this, the students were video-taped a third time as they were tying a knot.

Each of the knot-tying videos were evaluated by three experienced surgeons who were blinded to the students. They used a validated knot-tying scale to score the skill of the students. They also asked the students how satisfied they were with the instruction given by the surgeon (feedback versus compliments).

What do you think they found?

As you might have guessed, the students liked the compliments better. They were more satisfied (p=0.005). But more importantly, the group that received specific feedback did a better job with their knot tying than the group that just got compliments (21.9 vs 17.0, p=0.008).

It is interesting (to me at least) that both groups got better with practice. This is known as the time on task effect. The longer you work at something the better you get. The group that got feedback just got better at a faster rate, than the group that only got compliments.

There is often a tension between what a learner perceives as their deficit and what an objective observer sees as the deficit. As a teacher, when we do a good job of pointing out the deficits, it can help the learner to improve at a faster rate. But importantly, the learner may not be as satisfied with this method. It is a strong statement for limiting the use of student satisfaction data in evaluating teacher performance. Students can be less satisfied but have improvement in their performance on a task.

References

(1)       Boehler ML,et al. An investigation of medical student reactions to feedback: a randomizedcontrolled trial. Med Education  2006;40: 746-749.

(2)       Sumit K. De,et al. Attending, house officer, and medical student perceptions about teachingin the third-year medical school general surgery clerkship. J AM Coll Surg.2004; 199(6): 932–942.

(3)       Ringdahl EN,Delzell JE, Kruse RL. Evaluation of interns by senior residents and faculty: isthere any difference? Med Educ 2004; 38(6): 646-51.