Feedback or Compliments? Which is better?

I was getting ready to do journal club a couple of weeks ago and I came across an interesting study from a couple of years ago. Boehler and colleagues at the Southern Illinois University School of Medicine, decided to do a study of feedback (1), which happens to be one of my favorite topics.

Feedback is an old term that actually comes from rocketry and missiles. Radio signals came from the rocket back to the control tower on Earth. The direction of the rocket is then controlled by sending signals back to the rocket. Those signals are the feedback.

But it is hard! Faculty do not give students feedback often enough. At the University of Michigan, medical students were asked about the third year Surgery clerkship.(2) When asked (did you always get feedback?), students were more likely to Disagree or Strongly Disagree (p<0.001). Medical students thought that feedback was poor when compared with the opinions of faculty and residents (p<0.002). And 50% of the medical students believed that they were an inconvenience to the clinical service that they were a part of.

And faculty do not give good enough feedback. At the University of Missouri-Columbia, I did a study of PGY-1 residents who had been evaluated by faculty and senior residents.(3) We looked at the end of block written feedback comments. What we found was that 82% of all the comments were positive. 38% of all comments fell into two categories. The first category was generic comments such as “did a great job” or “is a pleasure to work with.” The second category was personal attributes, such as “has a great sense of humor.” The level or gender of the evaluator did not affect the comments (p = 0.17)

Feedback is really important in clinical skills acquisition. It helps students to learn about their own strengths and deficiencies. It offers insight into what he or she actually did and the consequences of actions. There is a dissonance between the intended result and the actual result, which hopefully becomes an impetus for change in the learner.

So Boehler’s project had two hypotheses: 1) medical students receiving compliments would be more satisfied than those receiving feedback, and 2) medical students receiving feedback would demonstrate improved performance, whereas those receiving compliments would not. They took a group of second and third year medical students at the medical school and studied their ability to learn to tie a knot. More specifically, a two-handed square surgical knot. This is a basic skill that all medical students need to learn.

The authors video-taped all of the students tying a knot. Then they gave all the students knot-tying instruction by an expert academic surgeon who was well-respected, supportive, credible, and trustworthy. They video-taped the students tying a knot again. Then the students were randomly assigned to two groups. One group received specific feedback about their knot tying skills (ie: this hand needs to be in this position), the other group received generic compliments about their technique (ie: you are doing a great job). After this, the students were video-taped a third time as they were tying a knot.

Each of the knot-tying videos were evaluated by three experienced surgeons who were blinded to the students. They used a validated knot-tying scale to score the skill of the students. They also asked the students how satisfied they were with the instruction given by the surgeon (feedback versus compliments).

What do you think they found?

As you might have guessed, the students liked the compliments better. They were more satisfied (p=0.005). But more importantly, the group that received specific feedback did a better job with their knot tying than the group that just got compliments (21.9 vs 17.0, p=0.008).

It is interesting (to me at least) that both groups got better with practice. This is known as the time on task effect. The longer you work at something the better you get. The group that got feedback just got better at a faster rate, than the group that only got compliments.

There is often a tension between what a learner perceives as their deficit and what an objective observer sees as the deficit. As a teacher, when we do a good job of pointing out the deficits, it can help the learner to improve at a faster rate. But importantly, the learner may not be as satisfied with this method. It is a strong statement for limiting the use of student satisfaction data in evaluating teacher performance. Students can be less satisfied but have improvement in their performance on a task.

References

(1)       Boehler ML,et al. An investigation of medical student reactions to feedback: a randomizedcontrolled trial. Med Education  2006;40: 746-749.

(2)       Sumit K. De,et al. Attending, house officer, and medical student perceptions about teachingin the third-year medical school general surgery clerkship. J AM Coll Surg.2004; 199(6): 932–942.

(3)       Ringdahl EN,Delzell JE, Kruse RL. Evaluation of interns by senior residents and faculty: isthere any difference? Med Educ 2004; 38(6): 646-51.

General Surgery and Family Medicine are in the same boat

In its 21^st Report, Improving Value in GraduateMedical Education (1), the Council on Graduate Medical Education made an important recommendation. COGME looked at all of the data regarding the physician workforce and predictions of student specialty choice. Their Recommendation 2 states that “GME funding should be prioritized to accelerate physician workforce alignment with population and health delivery needs.” And even more specifically, that “increases in GME funding should be directed toward the following high priority specialties: Family medicine, Geriatrics, General internal medicine, General surgery, High priority pediatric subspecialties, Psychiatry. I have written a lot about family medicine and primary care, but why is general surgery on this list?

General surgery has been facing many of the same pressures that have afflicted family medicine over the past twenty years.(2) Between 1987 and 2002, the percentage of US allopathic seniors choosing general surgery declined from 7.8% to 5.8%. This occurred while the percentage of students choosing surgical fields remained fairly constant around 12%. This coupled with a 20% attrition from surgical residency programs is leading to a shortage of general surgeons.(3) And compounding the problem is that up to 80% of general surgery residents plan on completing a specialty fellowship after residency.(4) 

Why is this happening? When I was in medical school, the General Surgeons were to gods of the wards. They were terrible to behold—striking fear in medical students, residents, nurses, and staff alike. Their position was lofty. When we needed something important done, we could always count on the surgeons.  So, what happened?

In some ways, it is similar to what has happened to primary care. You could call them the primary care surgeons. (I would not want to offend them with this, but I mean it as a compliment). Those surgeons were true generalists. A good general surgeon could operate in the abdomen, the thorax, the breast, the extremities, even in the gynecologic organs. But then general surgery was picked apart. Graduating residents wanted to confine their practice to a particular area. Maybe they did not like doing breast biopsies and mastectomies, so they limited themselves to the abdomen. Maybe, their friend was the gynecologist in town and did not like them doing hysterectomies because it took away some of their business. Maybe, they did not do enough vascular surgeries to remain competent. Over time, all of these factors, and others (reimbursement, hospital privileging, and local politics) led to general surgeons who were more limited in their scope of practice.

The interesting thing about academic medicine is that residents tend to practice like their teachers. So, when academic general surgery started to fragment, residents and students started to see that as the norm. If you look at a Department of Surgery in a major academic medical center, there are surgeons who operate only on the colon, who only do transplants, who only operate on the liver, and who only do trauma. If you step back a bit, there are now Departments of Urology, Orthopedics, Neurosurgery, etc.  In the past, those were all part of the Department of Surgery. Dean’s offices at medical schools around the nation are happy with things this way. Each time they created a new department, the dean had another department and chair to tax for more money for the Dean’s office. Recruitment is easier too. It is getting pretty hard to find the surgeons that want to do everything.

I used to joke that someday we would have Left Pinky Finger Doctors, but now I am worried that may really happen. It has already happened in other specialties. More than 80% of graduates of Internal Medicine residencies choose to complete further specialty fellowship training, in endocrinology, nephrology, infectious diseases, and others. Most of those doctors only practice in their sub-specialty area without any general medicine. Pediatrics, has seen this also in a smaller way (only 40-50% of graduates sub-specialize). And now, General Surgery is going the same way.

I think it is a great loss to our profession! When all of those “old” guys have died out, we will have lost something important.

References

(1)  The Council on Graduate Medical Education. 2008. 21^st Report, Improving Value in Graduate Medical Education. http://www.hrsa.gov/advisorycommittees/bhpradvisory/cogme/Reports/twentyfirstreport.pdf

(2)  http://usatoday30.usatoday.com/news/health/2008-02-26-doctor-shortage_N.htm

(3)  Newton D, Grayson M. Trends in Career Choice by US Medical School Graduates. JAMA. 2003; 290(9): 1179-1182.

(4)  Fischer J. How to rescue general surgery. Am J Surgery. 2012; 204(4): 541–542.]