'When Trust In An Expert Is Unwise'

What an interesting column by David Leonhardt.

It appeared in the November 7, 2007 New York Times, and follows.

When Trust in an Expert Is Unwise

A few years ago, an economics graduate student named Henry Schneider drove his dad’s old Subaru station wagon up to Montreal. He had heard about a Canadian consumer interest group that had done undercover investigations of auto-repair shops, and he wanted to try a more academic version of its experiment.

So when he arrived in Montreal, he handed the Subaru over to the mechanics working for the group, the Automobile Protection Association, for a complete inspection. They found that it had a small hole in its exhaust pipe, a blown taillight and several other relatively minor problems. Mr. Schneider took careful notes. But he also did something that no ordinary car owner would do. He asked the mechanics to show him how to mess up the car in a couple of serious but obvious ways.

They taught him how to loosen the battery cable (which can prevent a car from starting) and how to suck out coolant (which can leave an engine vulnerable to overheating). Armed with this knowledge, Mr. Schneider drove home to Connecticut and undertook a devilish little test.

Over the next few months, he took the Subaru to 40 garages, loosening the battery cable and draining some coolant before each visit. He even wrote himself a script and memorized it, to make sure he was telling every garage the same thing. “We bought the car recently, and we should have had it looked at before we bought it, but we didn’t,” he would say. “It hasn’t started a few times. Can you check that out?” He also asked for a thorough inspection.

Mr. Schneider was trying to answer a question that has occurred to pretty much all drivers who have ever been given the unsettling news that a car needs more repairs than they had expected: Does it really? Or is the garage just looking to make some extra money off me?

Economists sometimes refer to this situation as an “expert service problem,” because the same expert who is diagnosing the flaw is the one who will be paid to fix it. In most of these cases, consumers aren’t sophisticated enough to make an independent judgment. That’s why they went to the expert.

The problem, of course, extends well beyond the car business. Anytime you call a plumber or roofer to your home or anytime you visit a doctor or dentist, you’re at risk of having an expert service problem.

There is one school of thought — and it dominated the field of economics for decades — that says we shouldn’t worry so much about these problems. Even if most consumers can’t distinguish between good and bad expert service, some can. These informed consumers will tell their friends and relatives which experts to use, and an expert’s reputation will eventually become well known. The market, according to this view, solves the problem.

But there haven’t been too many real-world tests of the theory, which is precisely what makes Mr. Schneider’s research so interesting — and so depressing.

At only 27 of the 40 garages did mechanics tell Mr. Schneider that he had a disconnected battery cable, the very problem to which he had pointed them by saying his car didn’t always start. Only 11 mentioned the low coolant, a problem that can ruin a car’s engine. Ten of the garages, meanwhile, recommended costly repairs that were plainly unnecessary, like replacing the starter motor or the battery. (Tellingly, his results were in line with what the Automobile Protection Association found when it performed its experiments in Canada.)

In all, only about 20 percent of the garages deserved a passing grade. “And that’s with a pretty low bar,” Mr. Schneider told me. “I’m even allowing them to have missed a blown taillight that should have been caught.”

The study — which helped Mr. Schneider get a job as an assistant professor at Cornell’s business school and which he has submitted to an academic journal — is part of a recent wave of economic research dealing with everyday matters. The most famous example is the best-selling book “Freakonomics.” (Its authors write a blog on The New York Times's Web site and a column in its magazine.)

But there has been a bit of a Freakonomics backlash lately, with some economists arguing that their peers are paying too much attention to cute cocktail party questions that don’t really matter. The world would be better off, the critics say, if economists returned to the sort of grand issues that had occupied John Maynard Keynes and Milton Friedman.

There is some truth to this critique, but I think it is more wrong than right. And the garage study shows why.

See, Mr. Schneider didn’t set out to study cars. His original goal was to examine the health care system. But he couldn’t very well give himself a heart murmur and then visit 40 cardiologists.

“It turns out it’s hard to get objective measures of people’s bodies,” as Thomas Hubbard, a Northwestern University professor who has also studied the economics of reputation, put it. “It’s a lot easier to get objective measures of people’s cars.”

Yet by studying a fairly narrow question, Mr. Schneider discovered a larger truth: the expert service problem really does appear to be a serious one. It isn’t that reputation is irrelevant. “Very few businesses will stay in business ripping off their customers,” Tony Molla, a spokesman for the National Institute for Automotive Service Excellence, argued after I sent him the study.

But when a situation is too complex for an amateur to grasp — and when it involves shades of gray — you probably shouldn’t expect to get a purely objective diagnosis from someone who has a financial incentive to give you something else.

That, in fact, may be the biggest flaw with today’s health care system. (Yes, even bigger than lack of health insurance.) If anything, Professor Hubbard argues that the expert service problem is more serious in medicine than in auto repair, because most people are less willing to question a doctor than to question a mechanic. Any effort to reform American medicine has to grapple with these conflicts of interest.

And what about your car? How can you be sure you’re not getting swindled?

“That’s an excellent question,” Mr. Schneider says. “When people ask me that, I say, ‘If you’re an uninformed consumer, it’s going to be very hard to know if you’re getting good service.’”

For an expensive repair, a second opinion makes sense, but it will be hard to know which garage to believe. Mr. Schneider noticed no performance difference between garages that talked him through what they found and less forthcoming garages. He did find that mechanics certified by Mr. Molla’s group performed somewhat better than others, but not enormously so.

Until some savvy entrepreneur starts a garage-rating business, the best solution may be the oldest one: asking for a recommendation from someone who is knowledgeable enough to distinguish between good service and bad. Just remember that a lot of people don’t know quite as much about cars — or their mechanic — as they think they do.