I am the Director of Auto Test from Consumer Reports to talk about top picks and reliability AMA

In various interviews, you've justified your decision to override the views of your Model-3 owning readers - the same ones who ranked it the best car out there - and de-recommend it, based on the logic that, "Well, these buyers are unusual, they're unusually passionate, but some 'average buyer' won't like it as much."

On what grounds do you base the notion that there is some completely different group of people reading your recommendation advice than the ones who are already subscribers to consumer reports and have already purchased the vehicle (and love it)?   Why do you think that future buyers will be any different at all than past ones?  Has this ever happened with any other Tesla?  Because Teslas keep getting ranked by owners at the top of your lists.  What is your rationalle for assuming some sudden, radical shift in buyers, and using this to make such a radical, complete reversal of actual owner opinions about the vehicle?

We can't overlook reliability just because owners are happy with the purchase. This isn't anything new to us - Jeep Wranglers and Chevy Corvettes also have great owner satisfaction from their owners but aren't very reliable. But just because Jeep Wrangler's love their Jeeps, doesn't mean that a Tesla Model 3 buyer will love it too.

Um holy carp, as a Tesla owner I almost feel like I need to apologize. I guess I'll just ask my question though. Where would a 1999, 2009, etc. car with average reliability rate compared to today's cars?

LOL!!! Don't apologize! I get it. Tesla's have a lot going for them. We are just laying out the facts. Today's cars are more reliable for sure, but over the last several years we are seeing a change in where the problems are. More electronic problems and new tech transmissions.

You're conducting surveys, so you're supposed to be identifying and controlling for biases. Who exactly advises you on this? Because there's some serious concerns in this regard (I'm hardly the first to raise concerns about your lack of controls for data biases).

In particular, the main things you hold against the Model 3 were paint and trim.  Now, you're surely aware of the fact that last year, Tesla was the most heavily shorted stock in the world, and that there are entire online communities of short sellers dedicated to finding and sharing any negative reports about Teslas they could find. As a consequence, Model 3 buyers were hypervigilant - often showing up to their car pickups with exhaustive "delivery checklists" of things to inspect, to make sure that the car is perfect.

Anyone who works in detailing - for example, applying paint protective film - will tell you that many if not most new cars have at least some sort of minor issues with the paint and trim. This is why the first step of PPF application is paint correction. It's just that the owner never noticed until the detailer pointed the issue out.  Most owners just aren't that hypervigilent during their vehicle inspection.

It's just a simple fact that the more closely you look for problems, the more you're going to find, and most Model 3 buyers were intensely scrutinizing their cars. Do you accept this? If you accept this, what did you do to control for this fact?  Because it appears that the answer is "absolutely nothing".  And yet "paint and trim" were among the main things you marked Model 3 down for.

Karen - I am going to go on http://techcastdaily.com/ and discuss this stuff in detail there. I hope that I can hit many of your questions there. I clearly can't type as fast as you!

Karen - I think I answered your question elsewhere - there are several issues that Model 3 owners are reporting to us. I'm not ignoring you, but I'd like to answer some others too. I'll try to get back to you - but I can spend all whole hour reading your questions!

Do you honestly think that "paint and trim" are "reliability" issues?

Are paint and trim quality not basic owner satisfaction issues, no different from, say, how smooth the display interface is and how comfortable the seats are?

Are you not effectively double-counting them, since such issues are already accounted for in your customer satisfaction survey?

Bubbles in paint, broken trim, and electronics problems, are issues for a new car buyer. They require getting your car fixed - something a new car buyer doesn't want to do.

Do you think that's an answer to the questions:

• "Do you think that 'paint and trim' are 'reliability' issues?"
• "Are paint and trim quality not basic owner satisfaction issues, no different from, say, how smooth the display interface is and how comfortable the seats are?"
• Are you not effectively double-counting them, since such issues are already accounted for in your customer satisfaction survey?

1 - if it is something that a owner has to get fixed - it is counted

2 - reliability can effect owner sat - but in the case of the Tesla Model 3, it doesn't seem to

3 - see #2

As a thoughtful consumer, I would put my family in a car with the worst paint in the world as long as the paint had nothing to do with the safety provided to my family by the vehicle.

I can easily get a car repainted but real safety has to be designed into the car from the ground up it seems to me.

I would have thought that a company that responded so quickly to the breaking problem CR identified would be window into how seriously safety is to the people of that company.

And I will note that there is no risk of Carbon Monoxide fatalities in the the entire Tesla line.

How can any combustion engine vehicle which can be fatal when used as intended get a recommendation? How many combustion engine vehicles have a real CO sensor in the car and a fuel cutoff valve interlinked with the alarm? CR could literally save lives each year if a strong stand of zero tolerance for CO related deaths. This has been going on for over a hundred years with combustion engines and it could have been stopped decades ago. Just look up the efforts of Herb Denenberg.

https://www.kidsandcars.org/2017/09/14/co-and-cars-unfinished-business/

I can't understand how a car with such a great safety record and the high reliability of electric motors not being recommended? I hope CR quickly takes another look.

We don't believe that consumers should have to have brand new vehicles fixed or repainted. If you like the features that we report on, by all means purchase the car.

In Larry Olsen's interview with CR's Patrick Olsen, it was stated CR bases its recommendations on a "triangle": 'they have to do well in our testing, they have to be reliable, and they have to be safe. And if you can't connect all of the parts of that, we're not going to recommend it.'

Model 3 did do well in your testing .
Paint and trim are not issues about reliability. You yourselves said that the vehicle (powertrain, etc) was reliable.
You yourselves said it's safe.

How does this not fit your "recommendation triangle"?

Are you claiming that paint and trim are "reliability" issues? If so, I can sympathize, as my paint rebooted while I was driving the other day...  ;)

Would not reliability be best assessed by recalls and NHTSA manufacturer communications?

http://www.rodneytanner.com/hotrod/wp-content/uploads/2019/02/2019-02-NHTSA-Recalls-and-Comms.png

​

Most of these are recommendations of yours.

We have about 500 Tesla Model 3's in our survey and the owners reported several issues including broken trim, broken glass, and other issues including the electronics. While we don't weigh these as heavily as powertrain issues, the amount of problems is it an issue for buyers. If these problems are fixed like Tesla is claiming, we will see it in our data later this year.

I used to participate in a reliability study called “true delta” and their main focus was on unscheduled trips to the repair shop.

I found this metric to be a great barometer for how reliable a vehicle is (in the traditional sense of the word) and weeded out things like misaligned trim which (for me) can wait until the next service visit.

If something can wait until the next service visit to be dealt with, I still consider the vehicle reliable. If I must take unscheduled time to deal with it, I consider the vehicle to be unreliable.

Does Consumer Reports plan to start separating things like powertrain failures or brake/suspension malfunctions (what I consider needing immediate attention) from things like loose carpet, misaligned window trim, or glitchy Bluetooth connection interfaces?

I feel like casting “unreliable” across headlines for these five models is perhaps an unrealistic way to present it. That word, to me, equates with being left stranded on the side of the road, not with asking the dealer to adjust chrome trim pieces while the car is in for a scheduled tire rotation.

hing can wait until the next service visit to be dealt with, I still consider the vehicle reliable. If I must take unscheduled time to deal with it, I consider the vehicle to be unreliable.

Does Consumer Reports plan to start separating things like powertrain failures or brake/suspension malfunctions (what I consider needing immediate attention) from things like loose carpet, m

We do a lot of this in our analysis. That's why we never talk about PPH (problems per hundred) - because we weigh the severity. We show the issues that are more prevalent.

Thanks for the clarification. As a subscriber the way you describe these cars is not helpful, honestly. “Unreliable” is a very severe word to me in regards to an automobile.

I had an unreliable VW and it left me on the side of the road and needed a stack of unscheduled repairs and time off of work to deal with it. The same car had absolutely impeccable fit and finish, perfect paint, and flawless trim fitment.

I would really like to see your ratings broken out into genuine reliability problems (mechanical things that are broken) and irritations (trim, paint, infotainment). Being unable to determine, as a subscriber, how these very different concerns are lumped together negates the ratings for me. I just largely ignore your auto ratings when shopping for a car because of this even though I’m a big fan of your testing procedures and the publication in general.

Appreciate the feedback. Let me see what we can do. Breaking up the stuff that makes you break down on the side of the road is certainly more severe. BTW - are you a vegan too?

Was vegan but I’m too weak. Just vegetarian now. One of the two since 1998 :-)

But you got to keep the username huh? I find getting enough salt helps.

Oh I meant weak-willed. Like for cheese. I felt fine haha.

LOL - I get it. There's some really good vegan cheese out there now. I drink a lot of Cashew Milk too.

I'm signing off. Thank you for all the questions. I'm really sorry if I didn't get the chance to answer all of them. I'll try to do another soon.

We have discussed many of these topics on our Talking Cars video podcast. Past episodes can be found at: http://cr.org/talkingcars

Here is our latest show, where we discussed Tesla and our Annual Auto issue:

https://www.consumerreports.org/cars-driving/talking-cars-191-behind-the-scenes-of-the-2019-autos-spotlight/

​

​

​

You simultaneously de-recommended six separate vehicles. One would hardly know it. You wrote an entire separate article for the Model 3, and dedicated a separate tweet to it specifically. Even in your multi-car article, you focus the headline and lead text on only two cars, one of those being the Model 3. How is this not clickbait? Do you think that this sort of excess negative focus on a specific, "click-grabbing" model during a downgrade of six vehicles is what a fair, impartial organization would do?

dedicated a separate tweet to it specifically. Even in your multi-car article, you focus the headline and lead text on only two cars, one of those being the Model 3. How is this not clickbait? Do you think that this sort of excess negative focus on a specific, "click-grabbing" model du

Our coverage focused on Top Picks and the Best Cars. We did write an article about the Model 3 to help answer some anticipated questions. I'm also doing this AMA to help answer questions. (It's hard to keep up with you K.) Do you have any questions on anything besides Tesla? I'd be happy to answer.

Funny that you seem to welcome clicks and media attention about your Model 3 downrating, yet not questions as to statistical biases and logical gaps in your methodology.

I don't want you to misunderstand me. I think that Consumer Reports does an important service. I don't think you have any sort of "Anti-Tesla" bias, and that you do very much mean well. Your job is an important one. But because it is, and because you wield so much power as a result, you bear an obligation not to resort to clickbait and indefensible methodologies and results.

I'm hardly the first to raise questions about the statistical defensibility of your methodologies or claims of clickbait. But I am hoping that some of this might come through in regards to the need to do better. It's an important job. Please treat it with the respect that it deserves.

Karen - I absolutely welcome these questions and am happy to answer - it's just hard to keep up with your pre-written questions.

Hey Jake, longtime CR follower here! To distract you from the bazillion Tesla Model 3 questions, what road tests should I look forward to over the coming months? Thanks for the AMA by the way!!!

We're working on the Audi A8, Cadillac XT4 and the Toyota RAV4 right now.

How many times a day do you say “I mean look”? :)

Hey, you try to do video! :) I'm an engineer by training.

Which car maker has the best automated safety features? Specifically I'm speaking of auto braking, auto lane keep assist, and adaptive cruise control.

The truth is you don't have to spend a lot of money anymore for this equipment. Subaru's systems (Eyesight) perform quite well - but I wish they would make it standard in the Impreza.

Was there any car that was really poor in your tests or any car that made you shake your head?

Here are the cars that did worst in our reliability ratings

https://www.consumerreports.org/car-reliability-owner-satisfaction/10-least-reliable-cars/

and if you are a CR member, you'll have access to the cars that perform poorly in our testing

https://www.consumerreports.org/buying-a-car/worst-overall-cars-minivan-suvs-trucks/

​

Giulia?! Didn't know about that. Whew, I actually sold that to pick up my Model S...

Was there any model that tested very poorly, so bad that it's memorable?

The Fisker Karma was particularly bad. Billed as a Tesla competitor - but not so much. Our test cars barely ran enough to test and when it did run - it wasn't very good. Also not a big fan of the new Mitsubishi Mirage.

Hey Jake - I know the team is testing the Jag I-Pace, and said something like how it was one of the first true competitors to Tesla currently available. I'm curious how you think the MPG-e of the I-Pace compares to a Model S or Model X so far? And more importantly, do you think buyers of the car will care if there's a significant % difference in economy as long as they can get adequate range?

I think it's all about range. The Jag doesn't seem as efficient as the Teslas - but if they had a bigger battery that might fix it.

I know Toyota SUVs and Trucks are extremely reliable (look at the 4Runner, wow!) because they’re so long in the tooth but with that reliability with their old models comes lack of newer safety gear and older and less efficient engines and transmissions. Do you think it’s time Toyota updates the 4Runner, Tundra, and Sequoia? The only thing stopping me from getting a new 4Runner is thinking they’ll release an updated platform and engine.

Yes - some of them are getting ancient for sure. Honestly, I think they are way overdue. At least the Tundra and Sequoia have FCW and AEB now. https://www.consumerreports.org/car-safety/cars-with-advanced-safety-systems/

What car do you drive?

I drive the test cars. In my garage I have an 87 Toyota MR2 race car - SCCA Improved Touring. But not exactly a daily driver anymore.

Can you tell Keith Barry I said hey?

I'm on it.