Poll of polls including the 'quick count'

on Jul 2, 2012
As expected Peña Nieto won and AMLO came in second place, but the polls way overestimated the voting intention of Peña Nieto, underestimated AMLO's expected vote, and to a lesser extent also underestimated Josefina Vázquez Mota's vote.

The Federal Elections Board in Mexico runs a "quick count" (conteo rápido), a random sample of returns from voting booths across the country, which serves as a highly accurate exit poll. Measuring the euclidian distance from the normalized "quick count" to the voting preferences, the most accurate pollsters were:  SDP Noticias-Covarrubias, Grupo Reforma, Ipsos-Bimsa, and UNO TV-María de las Hera. The worst performing pollsters were Milenio-GEA ISA and Indemerc.

Rerunning my polls of polls using the quick count results as if they were the result of a poll taken on election day with a massive sample size and free of pollster bias (I plan on rerunning this analysis when the final election results are known) I get the chart at the top the post. The results are quite striking, particularly with regard to Pena Nieto, and indicate a relatively big bias among most pollsters. I'm very curious about what it is the polling firms did wrong to oversample PRI voters or otherwise bias their results.

There's always the possibility that AMLO had a late surge, but it doesn't look like there was one since some of the private exit polls also tended to put Peña Nieto ahead of what the "quick count" predicted. And exit polls are usually much more accurate than polling. It's very likely there was bias.

Reference

Simon Jackman. Bayesian Analysis for the Social Sciences. John Wiley & Sons, 2009

Don't miss a thing: Follow me @diegovalle on Twitter, grab the RSS feed or subscribe by email

6 comments:

Eduardo Flores said...

El voto "neto", es que no incluye los "no sabe/no contesto", o los incluye?

Diego Valle-Jones said...

El voto "neto" o "efectivo" es el que no incluye a los "no sabe/no contesto". En el conteo rápido también excluí los votos nulos o no registrados y lo normalice para sumar 100%

Eduardo Flores said...

Gracias!

rolandmex said...

In the 14 years I'm living in Mexico I've learned to mistrust any kind of polls in this country, having seen that even the pollsters in the street alter results.

Examples:
1) I buy a box of beer in the supermarket. Right next to the stand is a girl from the brewery offering me a cap for buying their product. She also has a poll sheet with a couple of questions, but she only asks for my name, which she fills in. The questions themselves are already answered by her. (I've been seeing this at least a dozen times over the years)

2) for a couple of years I worked near la Alameda Central in Mexico City, where a pollster base seemed to be. Every sooften when I passed the area I was asked to participate in a poll. Yes, there were a few people answering questions, but not enough to have a base for statistics. I suspect they were filled like in point 1).

3) I have asked around and nobody within my circle of acquaintances, friends and family has ever answered polls on the phone, and very few do answer questions in the street. Pollster companies (almost) never reveal who they ask (sex, age, education, social class), how large their base is and where the people are from.

4) Corruption.
a) Almost everyone will answer what you want to hear for a few pesos or a box of rice, beans and tortillas (at the moment there is a scandal going on because of Soriana shopping cards that were given out by the PRI in exchange for voting for Peña Nieto).
b) It is well known that the PRI and Televisa have an agreement to promote Peña Nieto, so it is very likely that pollster companies have been bribed to deliver results as desired. The PRI used the poll results in their TV spots in the last couple of weeks of the campaign, arguing that the candidate who's on top of the polls must be the best.

I'd suggest you follow the debate in Mexico for the next weeks to see what will be revealed about the (IMHO) manipulated polls. At the moment there are even talks to sue pollster companies for falsifying results (but you and I know that there's always a lot of talk in Mexico and very few results).

Thanks for being on top of the events in Mexico. As shown with these poll results the Mexican press is either not able or not willing to report in a objective way.

Saludos.

Arturo Sánchez Correa said...

Why are the entire trendlines shifted (up for PRD and down for PRI)? Is it that you are plotting the smoothed estimates? My guess is that the filtered estimates will have a big jump to get to the quick count, am I right?

Diego Valle-Jones said...

Arturo,

The trendlines are shifted because Reforma, María de las Heras and other pollsters who came close to the quick count results consistently placed the voting intention of Peña Nieto below (and AMLO above) other pollsters who did not come close to the quick count and since they got the election right they are given more weight when calculating the trendline

Post a Comment

Thank you for your interesting and insightful comments. I truly look forward to reading them.