Problems with CR!

Filed under: Cars and stuff, Off-Topic on March 23, 2007 by CanadianJeepYJ at 5:53 pm

I found out about a blog written by Michael Karesh of TrueDelta from a news article here at allpar. The original blog by Michael Karesh can be found here.

In Michael’s blog, he discusses the “anomalies” that exist in CR’s reporting about the “reliability” of particular vehicles. The majority of the vehicles discussed in Michael’s blog consists of vehicles that have different bodystyles of the same basic car structure, e.g., Dodge Magnum and the Chrysler 300. As Michael states in his blog these vehicles “share the great majority of their parts, including powertrains. This is especially the case if both are made in the same plant.”

Michael then reports on such “anomalies” such as the V8 Magnum scoring 55 points higher then the V8 300 yet the V6 Magnum scores 25 points worse then the V6 300. Michael goes on to discuss a wide variety of vehicles where this same “anomaly” or situation occurs.

Michael runs a site, named TrueDelta, where his analysis not only includes the mean or average number of successful repair trips per year but he also includes the standard deviation (error bars, uncertainties, variance, confidence intervals) of his data. This type of data reporting is crucial to any analytic undertaking. An example of his data reporting can be found here at his website.

From Michael’s analysis it is easy to see, with no questions asked, that the “reliability” or the number of trips per year that the 06 Chevy HHR is larger then the average number of trips per year of a 06 VW Jetta. Error bars are included!!! Similarly, his analysis shows that a 06 Chevy HHR is not more “reliable” then an 06 Charger/Magnum/300 or visa versa.

From surveys, people report to CR about a number of items about their particular vehicle. They report on the vehicles cooling, suspension, transmission, engine, etc. For each vehicle, each one of these sub-classification, e.g., transmission, will have a distribution of numbers related to the “reliability” or happiness of the consumer. (CR’s data is obtained by polling information of their subscribers). To calculate the average CR would add all of the data and divide by the number of data points. To calculate the error of their average CR would (or does but who knows since they do not publish these results) calculate a root mean squared uncertainty. This uncertainty is found when each data point is subtracted from their calculated mean, then squared, then summed, then divided by the number of data points (or N-1), then finally the square root of this number obtains their uncertainty or standard deviation. Another possible method is to fit the data to a Gaussian then the standard deviation would just be the second moment of the distributions or sigma = sqrt[{x^2} - {x}^2]). (These should be averages in x, but I cannot place the “average” symbols here). In summary each item that CR reports, e.g., transmission or engine, should have a mean and a standard deviation associated with the mean.

For example if CR asks 4 owners of a Chrysler 300 to rate their suspension of their vehicle; rating the suspension from 0 to 10 they may be a variety of answers. Lets say for illustrative purposes the responses that they get are as follows
Person A may say 7
Person B may say 3
Person C may say 8
Person D may say 10

CR would report the suspension of the 300 to be (7+3+8+10)/4 = 7. To calculate the error of their reported average they would calculate {[(7-7)^2+(3-7)^2+(8-7)^2+(10-7)^2]/4}^(1/2) = 2.5 (If we used N-1 or 4-1=3 for the normalization constant, the error would be 2.9).

It would be wrong for CR to just report that the average Chrysler 300 customer rates their suspension of their vehicle as 7 out of 10. They must also report the variance of their data and state that the average 300 customer rates their suspension to be 7 +/- 2.5 out of ten.

Then CR must then give an average of all of the sub-classifications of each vehicle. (I take it this is where they rate or calculate the reliability of the vehicle). To accomplish this they will find an average of each of the sub-classification. To find their uncertainty on their “reliability” of the vehicle they must add the square of each of their uncertainties of the sub-classifications to find the “average uncertainty” for each vehicle.

For example, Vehicle A was surveyed and the average and uncertainty of each of the 5 classification or sub-classes was found to be the following (for illustrative purposes)
Engine: 5 +/- 3
Transmission: 7 +/- 3
Suspension: 6 +/- 4
Cooling Ability: 7 +/- 3
Noise Levels: 5 +/- 4

The average score for the overall vehicle would be (5+7+4+7+5)/5 = 5.6 (With equal weighting for each of the sub-classifications).

We wouldn’t add their uncertainties like this (2+3+2+3+4)/5 = 2.8 (This is wrong).
You add the squares of their uncertainties, then take the square root such that [(2^2+3^2+1^2+3^2+4^2)/5]^(1/2) = 2.898.

So the “reliability” of Vehicle A is found to be 5.6 +/- 2.9.

The same survey for Vehicle B might show:
Engine: 6 +/- 4
Transmission: 8 +/- 2
Suspension: 5 +/- 5
Cooling Ability: 7 +/- 3
Noise Levels: 4 +/- 3

Which would have a “reliability” if 6 +/- 3.5.

CR, in my opinion, would rate Vehicle B higher or more “reliable” then Vehicle A. Sure the average is higher along with the upper bounds of the error bars (6+3.5=9.5 versus 5.6+2.9=8.5) but the lower end of the error bars puts vehicle B at 2.5 (6-3.5=2.5) versus 2.7 for vehicle A (5.6-2.9=2.7).

However, it is my opinion that it would be silly or not prudent to say that Vehicle B is more “reliable” then Vehicle A due to the larger area of overlap between the error bars associated with each vehicle. A reduction in the error bars would be needed before a concrete discussion was made about the relative reliability of either vehicle. One may reduce the size of the error bars by increasing the number of data points.

In order to calculate the reliability of brands the average reliability of each of the brands vehicles should then be calculated. In order to accomplish this CR would have to make a decision on how to weight each vehicle in their calculations.

For illustrative purposes, if brand X produces 3 Vehicles A, B, and C.
Vehicle A has a calculated “reliability” rating of 5 +/- 2.
Similarly Vehicle B has 7 +/- 1 and Vehicle C has 8 +/- 1.

Assuming equal weighting for each vehicle, brand X would have an average reliability rating of 6.7 +/- 1.4.

However, if brand X sold 10,000 units a year comprised of 9,000 units of Vehicle A, 750 units of Vehicle B, and 250 units of Vehicle C. You would think that you should included the weights of these units into your calculation for the “reliability” of brand X. Assuming this, brand X would have a “reliability” rating of 5.23 +/- 0.69 (instead of the equal weighting of the vehicles found to be 6.7 +/- 1.4).

CR doesn’t state how they weight each vehicle when they calculate an average for each brand. Obviously this is important information since it dramatically changes the results.

In summary, I want to congratulate Michael Karesh of TrueDelta for finally creating a second option, relative to CR, for consumers to obtain information regarding manufactures of automobiles. His analysis has been open and honest. It contains the most BASIC ingredients, ones which are lacking in CR reporting, regarding statistical analysis of data collection.

Problems with CR! Part I

Filed under: Chrysler Corp on by CanadianJeepYJ at 5:50 pm

I found out about a blog written by Michael Karesh of TrueDelta via allpar here. The original blog by Michael Karesh can be found here.

Michael in his blog discusses the “anomalies” that exist in CR’s reporting of the “reliability” of each vehicle. The majority of the vehicles discussed in Michael’s blog consists of vehicles that have different bodystyles of the same basic car structure, e.g., Dodge Magnum and the Chrysler 300. As Michael states in his blog these vehicles “share the great majority of their parts, including powertrains. This is especially the case if both are made in the same plant.”

Michael then reports on such “anomalies” such as the V8 Magnum scoring 55 points higher then the V8 300 yet the V6 Magnum scores 25 points worse then the V6 300. Michael goes on to discuss a wide variety of vehicles where this same “anomaly” or situation occurs.

Michael runs a sit, named TrueDelta where his analysis not only includes the mean or average number of successful repair trips per year but he also includes the standard deviation (error bars or confidence intervals) of his data. This type of data reporting is crucial to any analysis undertaking. An example of his data reporting can be found here at his website.

From Michael’s analysis it is easy to see, with no questions asked that the “reliability” or the number of trips per year that the 06 Chevy HHR will have is larger then the average number of trips per year a 06 VW Jetta will have. Error Bars Included!!! Similarly, his analysis shows that one cannot say that the 06 Chevy HHR is more “reliable” then an 06 Charger/Magnum/300 or visa versa.

One possible reason why CR does not discuss their error bars or standard deviations from their means is due the possible large values of their error bars. People report to CR about a number of items about their vehicle. They report on the vehicles cooling, suspension, transmission, engine, etc. For each vehicle, each one of these sub-classification, e.g., transmission, of the vehicle will have a distribution of numbers related to the “reliability” or happiness of the consumer. (CR’s data is obtained by polling information). To calculate the average CR would add all of the data and divide by the number of data points. To calculate the error of their average or mean CR would (or do, who knows) would calculate a root mean squared uncertainty. Each data point would be subtracted from their calculated mean then squared, summed, divided by the number of data points (or N-1) then the square root of this number obtains the uncertainty or the standard deviation. (Another possible method is to fit the data to a Gaussian then the standard deviation would just be sigma = sqrt[ - ^2]). In summary each item that CR reports, e.g., transmission or engine, should have a mean and a standard deviation.

For example if CR asks four owners of a Chrysler 300 to rate their suspension of their vehicle rating the suspension from 0 to 10.
Person A may say 7
Person B may say 3
Person C may say 8
Person D may say 10

CR would report the suspension of the 300 to be (7+3+8+10)/4 = 7. To calculate the error of their reported average we would calculate {[(7-7)^2+(3-7)^2+(8-7)^2+(10-7)^2]/4}^(1/2) = 2.5 (If we used N-1 for the normalization constant the error would be 2.9).

It would be wrong for me to just report that the average 300 customer reports that the suspension of their vehicle rates a 7 out of 10. We must include the variance of the data and state that the average 300 customer rates their suspension to be 7 +/- 2.5 out of ten.

Then CR must then give an average of all of the sub-classifications of their car….I take it this is where they rate their reliability of the vehicle. There they will again find the average of each of the sub-classification, but to find their uncertain here they must add the square of their uncertainties to find the “average uncertainty.” Here is were the their uncertainties will grow.

What to do about CR?

Filed under: Cars and stuff on March 15, 2007 by Dave at 7:27 am

Consumer Reports has, using questionable statistics, managed to exclude just about every American vehicle from its top car lists, and even its “top American car” list fails to have a single domestic minivan - or any Chrysler vehicle at all, including those that have gained high reliability ratings. One problem is the lack of alternatives; there really are no better American reports. That’s not to say one person isn’t trying; Michael Karesh is trying to build a system that addresses some of Consumer Reports’ reliability survey problems. You can be part of the solution by signing up to be a member of his panel, and giving him enough data about your car(s) to have a credible counter to Consumer Reports. The path for Michael is difficult, not just because CR is an established institution that journalists and judges alike seem to feel is beyond reproach, no matter what their actions. Try out his site.

Allpar serious downtime

Filed under: Allpar on March 14, 2007 by Dave at 2:02 pm

As you might have noticed, the site has gone down for LONG periods of time. This is just about the most aggravating thing that can happen to a webmaster, and it’s been happening despite assurances like:

“We apologize for this, there was emergency hardware replacement which took place on our edge routers, this issue is now resolved. Typically we like to send out a notice prior to maintenance, however the nature of the issue which was discovered require immediate attention to prevent a larger issue as the day went on. … We certainly have the reliability of the service in mind when such things are performed and I can assure you that at this point there should not be any reason for further service disruption stemming from this.”

And then the next day, we get a 60-minute outage during the busiest part of the day.

And of course the nice people at Hostrocket now have voice-over-IP so the phones go down along with their service. And their site goes down, too. No, they don’t have a $60/year account somewhere else “just in case.”

I am considering failover possibilities, including going to managed DNS via a service like EasyDNS, where I can quickly repoint the IP address to a backup server at 1and1. At the moment the DNS system is based on the server itself which seemed to make sense (single point of failure instead of two) but that doesn’t make sense now. The problem with this is caching; it can take up to 20 minutes for the IP address to change. Still, that’s better than an hour or, if Hostrocket folds up their tents without warning (some of us had this experience in the past), a few days.

The backup site can’t be mirrored effectively wtih the forums and all, but it can provide a “90%” solution. If I rented a second dedicated server just for these times, it would be easier than using a 1and1 VPS - there’s be more bandwidth and such for daily mirroring.

Or I could just get a better hosting service. That’s really the best solution. So I’m taking recommendations now. We are currently paying about $245 for a setup with 700 GB transfer limit, 80 GB hard drive, 4 GB RAM, 2.8 GHz dual-core Xeon processor, daily backup to hard drive array, and WHM/Cpanel. That includes tech support, supposedly. If you know of something really reliable and reputable, let me know.

Chryslertakeover

Filed under: Chrysler Corp, DCX on March 12, 2007 by Rich at 5:37 pm

First off, there’s a great editorial here

http://www.allpar.com/ed/chryslertakeover.html that has inspired me to write. I remember chryslertakeover.com, and thank Mr. Vitale for his work. I remember feeling this was a ‘bad idea’ in 1998. There was a time, not long ago, when Chrysler was busy saving Mercedes’ bacon and Zetsche was the toast of Detroit, that I thought this whole DCX thing might work out long term.

I believe I was mistaken.

It’s clear now that if Chrysler stays within the clutches of Daimler, it will slowly but surely be completely absorbed with only Jeep and Dodge Trucks left. Every “turnaround” seeks, supposedly, to grow by contraction - a short term answer intended to impress those looking to make a quick buck on the stock rather than actually grow the company long term.

There’s another angle, the one that suspects this is all a ruse to soften up the UAW. It’s a mighty sophisticated ruse then, as this has all the earmarks of really being an effort to sell. I used to place more credence in this idea than I currently do, but given what we’ve seen with DCX so far I hesitate to completely rule it out.

It is also possible that they simply will not get their asking price, and will eventually take Chrysler back off the market. In my opinion that is the least likely scenario, as they have created many, many wounds that they would then have to heal, a difficult task when they already have a plate full of those.

Key elements in Daimler want Chrysler gone. That also has been made clear. This not only makes a retention of Chrysler by Daimler seemingly unlikely, it also makes it undesirable in the extreme. For, if you cannot sell that which you want to be rid of, you can always dismantle it, jettisoning that which you don’t want and only keeping the bits that you still value.

Chrysler - as a company - ceased to exist in 1998. If Daimler does not find a buyer (and of course, maybe even if they do), Chrysler could very well cease to exist as a brand as well.

Work hard and do your best though we dislike you

Filed under: Chrysler Corp, DCX on March 6, 2007 by Dave at 9:35 am

Work hard. Contribute your all. Sorry we don’t care about you.

That’s the message coming from DaimlerChrysler for its American employees.

Tom LaSorda urges employees to remember that Chrysler Group is investing in the future, to work hard and try to pull the fat from the fire after all those people from Stuttgart put it there.

Unfortunately, the DaimlerChrysler Boards keep undermining that message by making it clear that anyone who gives them enough cash will get the Chrysler Group. There is no apparent concern for the thousands and thousands of Chrysler Group employees in their actions and statements; indeed, they’ve (probably not Dieter) made enough sleazy off-the-record statements to the press about the Americans to show that at least some of the Board apparently would like everyone at Chrysler Group to lose their jobs as part punishment for trying to taint Mercedes with what we can only describe as higher quality.

Some of the buyers for Chrysler are clearly not good in the long term for Chrysler employees’ jobs. General Motors poses a serious risk, depending on what they would do with Chrysler after acquiring it. Would they simply take over the flexible manufacturing, minivan platform, Jeep name (and perhaps engineers), the upcoming automated-manual transmission, Phoenix V6, and a few other technologies, and can the rest of the company? How about the various private equity firms? Some of them seem fairly sleazy while others seem very above-board, but that doesn’t seem to be a factor; everyone gets the same financial packet, as far as we can tell.

Those waiting for a sign from above only hear “all options are open.” There’s no BMW-style “we’ll give away Rover complete with a billion or so of our cash to help you keep it running for a while.” No symbolic gestures are being made for the Chrysler employees. Instead, it’s a constant stream of insult and innuendo from the cowardly masked board members countered only by Dieter Zetsche’s references to the skills and expertise of Chrysler’s engineers.

Whatever happens, it should happen quickly. This chapter should close as soon as it can. Among other things, I can’t believe this attitude towards employees motivates people to root out and fix quality problems, to stand up to managers and demand that problems be fixed, to put in that extra time to get it right - or even to give Chrylser another chance instead of sending off that newly polished resume to Toyota. I hear they’re hiring.

Next Page »

Powered by WordPress

Cars - Engines - Viper - History - Forums - Repairs - Reviews - Other car reviews - About us - Terms of Service - News

Allpar Internal Search Engine:

dodge cars

Please read the terms of use! Mopar, Dodge, Jeep, and Chrysler are trademarks of Chrysler LLC. We are not Chrysler. We are not responsible for the consequences of actions taken based on this site and make no guarantees regarding validity or applicability of information or advice. The Webmaster is not an expert. We cannot take responsibility for posts on this unmoderated weblog. Weblog materials copyright 2006-2007, Allpar LLC. All rights reserved. Recommend this page!

Bad Behavior has blocked 1103 access attempts in the last 7 days.