StaffAllpar HomeMore NewsCarsTrucksUpcomingRepairsTest drives

How reliable is CR’s reliability study?

by Bill Cawthon on

Consumer Reports’ annual reliability survey is influential and has come to be the “gold standard” in the eyes of many consumers. But is the survey itself reliable?

Consumers Union does not claim that its study is scientifically rigorous. Other sources, including Allpar, have covered some of the flaws in the CR survey, and we find that TrueDelta adopts a more rigorous and transparent approach to vehicle problem reporting.

Consumer-Reports-Graphic-Web

The larger issue is whether the reliability study is even consistent with itself, at least in terms of the information the magazine releases.

In automotive terms, reliability is the ability of a vehicle to do what is is supposed to do on a consistent basis. Turn the key; it starts. Press on the brakes; it stops.

Considering that the average age of the vehicles on American roads and highways is more than eleven years and that vehicles with more than 100,000 miles on them are no longer a newsworthy rarity, one has to conclude most cars and trucks are pretty reliable over time, if properly maintained.

After the first three years, most light vehicles sold in the U.S. have reached the end of their “bumper-to-bumper” warranty and at the end of five years, most have reached the limit of their powertrain warranty. This is also the period where wear and tear have taken their toll and visits to repair facilities become both more common and more expensive.

What reliability isn’t, is a beauty contest or a measure of how well a consumer likes or is able to master a given component.

CR keeps data on vehicles going back ten years. It notes major generational changes, which is good, but fails to account for other changes.

In the case of Chrysler, ten years ago it was part of DaimlerChrysler, one of the less successful mergers, followed by the Cerberus interlude; in both times, concerns about quality were subordinate to squeezing every last penny out of costs.  Chrysler today is largely the product of the Fiat takeover, and that goes back only to the middle of 2009.

Allpar looked at the Consumer Reports reliability ratings for a number of Chrysler products including sedans, sport-utility vehicles and light trucks.

The ratings data published by CR covers 17 parameters, each of which gets a grade on a five-point scale.  Allpar assigned a numeric value to the circles, with “5” meaning “Much Better Than Average,” “4” meaning “Better Than Average,” “3” meaning “Average,” “2” meaning “Worse Than Average” and “1” meaning “Much Worse Than Average.”

We looked at CR data going back only to the 2011 model year, to separate issues with design and manufacture from problems caused by wear-and-tear or poor maintenance. This is also the data CR uses for its new vehicle reliability predictions.

The Chrysler 200, Dodge Avenger, and Dodge Dart all have notes that there is insufficient data to rate them (which means less than 100 surveys were received). This didn’t prevent CR from labeling the Dart as one of the ten least reliable cars.

There’s no data for the Jeep Compass prior to the current year; and the Jeep Cherokee is apparently being judged by the Jeep Liberty, because the scores shown are for the Liberty, not the new Cherokee. The two share few, if any, parts and designs (one is rear wheel drive, one is front drive; they have different engines, transmissions, all wheel drive systems, bodies, etc.)

Allpar compared the ratings for the “Better Than Average” 2014 Buick Regal to the Chrysler 300 and the Dodge Charger V6, which were rated “Much Worse Than Average.”

Based on CR‘s own ratings, the Buick was superior in only four categories (Transmission Minor, Suspension, Body Integrity and Audio System). “Transmission Minor” covers things like rough shifting, adjustments, leaks and such; from what we could tell, the Chrysler products got dinged for rough shifting.

By CR’s three-year rule, the “Better Than Average” rating for the 2014 Regal doesn’t make any sense; the Regal was rated “Much Worse Than Average” in 2011 and 2013 and “Average” in 2012.

The Chrysler was superior in seven metrics, including Engine Minor, Drive System, Fuel System, Electrical System, Brakes, Paint/Trim and Body Hardware. Dodge was the winner in Engine Major and there were five ties, only two of which, Engine Cooling and Exhaust (a three-way tie), included the Buick.

The average of the scores a 3.9 for the Chrysler 300, a 3.7 for the Charger and a 3.5 for the Buick.

CR notes that certain critical parameters, such as engine and transmission, receive more weight in their calculations, but in almost all of these areas, the Chrysler Group cars outscored the Buick.

In fact, the 2011-2013 “Much Worse Than Average” Jeep Wrangler Unlimited scored slightly better than the Buick.

The Buick Verano, which scored a 4.3, had better scores, though data was only available from 2012 and 2013. Yet both it and the Regal were rated “Better Than Average.”

This is not to say CR is presenting false data in any way: the numbers seem to be valid, within the limitations of the study. Allpar took their information, assigned a different (but equivalent) scale, and ran those numbers as a method of comparing CR‘s ratings. The issues suggest using other sources, perhaps in addition to those ratings, to make an informed decision.

Consumer Reports is a registered trademark of Consumers Union of the United States, Inc. It is used under the doctrine of fair use, in an editorial context, for critical evaluation. No approval of this use by Consumers Union is implied or should be inferred. See other issues with Consumers Reports auto ratings and their response.

Bill Cawthon grew up in the auto industry in the 1950s. His Dad worked for Chrysler and Bill spent a number of Saturdays down on the plant floor at Dodge Main in Hamtramck. Bill is also the U.S. market correspondent for just-auto.com, a British auto industry publication, and a member of the Texas Auto Writers Association, which has named the Jeep Grand Cherokee the “SUV of Texas” several times and named the Ram 1500 as the “Truck of Texas” two years running.

Bill has owned five Plymouths (including the only 1962 “Texan”), one Dodge and one Chrysler and is still trying to figure out how to justify a Wrangler. He also has owned at least one of every 1:87 scale model of a Chrysler product. You can reach him directly at (206) 888-7324 or by using the form.


Back to the Chrysler 100

Kidde fire-extinguisher recall and Ram RVs
mack engine
Silver Mack

More Mopar Car
and Truck News

Some popular Allpar pages





Dodge Demon

2018 Wrangler JL



Staff details/contactsTerms of ServiceInformation is presented to the best of our knowledge. Plans change and sometimes mistakes are made. Decisions or purchases made based on this site's verbiage or images are done at the reader's own risk. Also see the Allpar News archives, 1997-2008 • Copyright © 2008-2017, Allpar LLC. All rights reserved. • Mopar, Dodge, Jeep, Chrysler, HEMI, and certain other names are trademarks of Fiat Chrysler Automobiles.