07.10.09
Selection Effects: Statistics are not in good hands with Allstate (part 1 in an ongoing series)
Most people are ignorant of basic statistical facts. While this is fine for much of ordinary life, a lack of understanding of statistics will quickly lead investors and stock traders to lose money. So while an inability to understand statistics on the part of both Republicans and Democrats means we get plenty of stupid editorials instead of the acknowledgment that we can never know whether Al Franken or Norm Coleman won the election in Minnesota (the error in counting votes was greater than the margin of victory and different and equally plausible / legal methods for validating ballots led to different outcomes), in investing and trading, misunderstanding statistics leads to quick and painful losses.
This ongoing series will focus on selection bias (also called selection effect) and how it leads to inappropriate conclusions. I will not concentrate solely on trading and investing but rather will highlight selection effects in many different fields. Selection effects arise when conclusions are made from sampled data that are not representative of the population of data as a whole. An example from my field (psychology) is the use of college students in most memory research. A researcher who ran experiments only on college students and tried to say the results exemplified people in general would be making false statements because of a selection effect. To make generalizations about the population as a whole, the sample needs to be randomly drawn from that population.
Today’s selection effect comes from Allstate. On their website they proudly announce, “People who switched from Geico to Allstate saved $473 a year on average.” This statement can be interpreted in two ways. The first interpretation is that Allstate insurance is on average cheaper than GEICO. That is an incorrect interpretation; it not supported by the data and it is probably not true for the majority of car insurance customers. The second and correct interpretation is that Allstate and GEICO have different underwriting standards, and for some subset of GEICO customers, Allstate will be cheaper. Those people will likely switch to Allstate if they check out Allstate’s prices. For the most part, people who switch insurance companies do so because they can get insurance cheaper elsewhere. Those who cannot get cheaper insurance do not switch. Therefore, the people who switch from GEICO to Allstate will by definition be those who can get insurance cheaper with Allstate. Furthermore, people generally do not switch insurance unless they can save a significant amount of money; this makes the the average savings from switching seem higher. This type of change in observed average as a result of a selection effect is one of the more insidious types of selection effects; I will give further examples of this in future posts.
Allstate’s statistic would look even dumber if we had the same statistic from every insurance company. We would likely find that 90% of people who switch insurance companies save money when they switch. On average, the switchers would save hundreds of dollars no matter what company they switched to. This tells us nothing about which insurance is the cheapest. So if you are searching for insurance, check out many different insurance companies; depending upon your special circumstances it is hard to predict which one will be cheapest. I recently switched to GEICO from Farmer’s (and saved hundreds of dollars on auto insurance); but back when I had a house and rental real estate, Farmer’s was the cheapest.
Disclosure: I am just about to switch from Farmer’s to GEICO for all my property/casualty insurance needs. I am currently long 30 shares of Berkshire Hathaway B stock and short 1 share of Berkshire Hathaway A stock (in an arbitrage trade leaving me with no significant net position). GEICO is a wholly-owned subsidiary of Berkshire Hathaway. My disclosure policy suffers occasional bouts of selection effects itself.
RJ said,
July 10, 2009 at 6:43 pm
Michael,
I have been watching those Allstate ads for quite a few years now, I never thought about looking at those savings from this perspective. In general, I have learned that you have to shop for auto insurance every year if you want to save money. And yes, I am waiting for Milemeter to start providing insurance in NJ/NY!
RJ
Daniel M. Ryan said,
July 11, 2009 at 12:42 am
It sounds like there’s survivorship bias buried in there as well.