Visualizing ENLACE 2009

on Sep 8, 2009
ENLACE is a test of student progress given to all students in Mexico. Yesterday, the test results for high school were released. The website the SEP setup has an area dedicated to statistics about the test, I would make the following suggestions:

1) Make public the total score for the test. When you only have access to the population percentiles who reach a certain level of proficiency it gives the impression that they don't want reporters to summarize the test results in an easy way. Plus with all the multiple comparisons you'll always be able to say that students improved their scores at some level.

2) Graphs are better than tables.  Look at the difference between the table [PDF] and the charts below (the table is in Spanish but easy to understand):
image


Just by looking at the graphs you can tell immediately that scores mostly stayed the same, the students marginally improved their scores in math while their scores in reading (lectura) were worse. The math scores don't look normal at all and they are much lower than the reading scores which do look somewhat normal. Chiapas, Guerrero and Sinaloa experienced the biggest decreases in reading scores.

3) The SEP should really consider making probability density plots, they would be even easier to understand than looking at percentages. Ha ha!

Mexican PISA Scores

on Sep 3, 2009
PISA 2006 was a world-wide test of 15 year olds, regardless of the grade they attended, in Math, Science and Reading. We can use the test results[PDF] to compare the quality of students and school systems within nations and between nations. Here is an overview of the results for Mexico:
The highest scoring region in Mexico was the Federal District (Mexico City), which also happens to be the richest part of the country. Additionally it looks like the scores follow the "closer to Canada" rule.


Anecdotally I've heard many Mexicans (who have gone to American schools) claim that Mexican schools are better than US schools, and apparently so have other people. PISA lets us test this assertion empirically. Furthermore, the PISA scores for the science section are available broken down by ethnicity in the US, which allows us to make a more in depth comparison.
On the combined science literacy scale in the United States, Black (non-Hispanic) students (409) and Hispanic students (439) scored lower, on average, than White (non-Hispanic) students (523), Asian (non-Hispanic) students (499), and students of more than one race (non-Hispanic) (501).
The correlation between the science scores and the math and reading scores is higher than .9 so the comparison should apply to all areas of the test.

The DF (Mexico City) with a score of 445.11 and Queretaro with a score of 440.43 are the only states that do better (barely) than US Hispanics (439). Since the difference between Hispanic students in the US and Mexican students is so great, we can probably lay the blame on the school system and teachers in Mexico.

The average score for all groups in the US was 489 and the average score for Mexico was 409, the test has a standard deviation of a 100 points so that would put Mexico .8 deviations below the US, which means that the average Mexican student scores at the 21st percentile in the US.

The results are averages and of course do not mean that there aren't individuals of extraordinary abilities, just that the percentage who meet a high threshold is much lower in Mexico. And in fact the percentage of students that reach the highest two levels in the exam is 3% in Mexico, compared to an average of 9% across all OECD countries
By world standards Mexico scores abysmally low, no wonder the government ordered the results of TIMSS-1995 destroyed before they could be published. I'm surprised that Argentina with the highest income per capita in Latin-America scores so low!

Not everything is bad news, now that teachers can no longer inherit or sell their jobs, and are chosen by a nationwide exam, test scores will probably go up.

Update: Only girls were tested in Chihuahua, and only high school students took the test in Michoacan