Unnaturally Long Attention Span

Once every three years, the Department of Education participates in the Program for International Student Assessment (PISA), which is a study comparing 15-year-olds' performance in reading literacy, mathematics literacy, and science literacy in 57 participating countries. In particular, this latest assessment focused on science literacy, which the test's methodology defines as

an individual’s scientific knowledge and use of that
knowledge to identify questions, to acquire new
knowledge, to explain scientific phenomena, and
to draw evidence-based conclusions about sciencerelated
issues, understanding of the characteristic
features of science as a form of human knowledge
and enquiry, awareness of how science and
technology shape our material, intellectual, and
cultural environments, and willingness to engage in
science-related issues, and with the ideas of science,
as a reflective citizen (OECD 2006, p.12).

Now, whenever I hear discussion concerning the academic performance of American students in the context of international comparison, it is usually from two types of sources: politicians and moms.

By politicians, I mean mainstream news sources and personalities that draw upon the popular urban legend meme of the US being the worst in education around the world in order to work a crowd.

[sample politician at a rally]

"...the US has the lowest reading and math scores worldwide. Our children are our future and we need more funding to pay our teachers today!"

Crowd: "Yeah!!"

Being an immigrant, by moms I mean this of course:

"You know when I was your age in Soviet Russia, we were doing vector calculus and number theory in middle school. In the snow. Uphill. This American education is rotting your brain."

So, what does the report show? US students scored 489, on a test where the mean and standard deviation have been normalized respectively to 500 and 100. So yes, the report does seem to support the idea that US students underperform their international peers. However, if you break down the data more closely you'll find two interesting features.

Firstly, they do a categorically analysis of the data by racial group, which shows very statistically significant disparities in the scores between different racial groups. They range the gamut from an average score for black students of 409 to an average score of white non-Hispanic students of 523. The state of the American education performance in a nation that is itself multi-cultural and multi-societal, is a complex and non-homogeneous issue. There are definitely serious problems and widening gaps in education performance that are developing in America, but the data suggests to me that uniform across-the-board type policies, such as No Child Left behind, or universal minimum wages for teachers may not be all that beneficial, and potentially harmful, to the overall big picture. Instead target areas which are weighting down our national average should be pinpointed as areas to take a closer look at, and the best policies in those specific areas considered. But of course, this type of fine-grained debate never happens in the mainstream discourse.

In comparison, another feature of this study, and similar types of studies, that further aggravates this effect (of American students' underperformance) is that the distribution of the participating countries in the study is heavily skewed. In the list of member countries shown in the report, I noticed that the vast majority of the regions tested are in Europe and Australia. Not only are these countries much more racial homogeneous, making the comparison to a heavily multi-racial US an ill one, most of these countries are composed primarily of the "non-Hispanic" white population that scored the highest in the US breakdown. Let's not even address the issue of bias in question construction. Sadly, the entire continent of Africa is not included in the study.

What can moms learn from this study? The report also presents a limited what's called decile analysis of the scores. That means they break down the average scores in the 0%-10% range, 10%-20%, ..., 90%-100% range, etc. What the study found is that in the 90th percentile, a.k.a. 90%-100% range the US students scored 628, compared to a lower 622 for international students in the 90th percentile. That means, if you are an immigrant mom, and in the position to wonder which country would provide the best education for your child, you should have no qualms about having your children educated in the US. There are many good schools and almost any immigrant hub (read: major metropolitan area) in the US has some of the best schools in the world.

Read the report for yourself. It has some fun sample questions from the test that was given to the 15 year olds.

Highlights from PISA 2006: Performance of US 15-Year-Old Students