Yesterday I noted that Finland is seeing declines in reading, math, and science achievement. For anyone unfamiliar with education policy debates in the U.S., you might wonder why I was paying special attention to a Nordic nation with a population about the size of South Carolina.

But for those who do pay attention to American education politics, you probably weren’t fazed by the Finland coverage. After all, Finland’s education policies have been given an inordinate amount of attention since they scored near the top of international achievement tests about a decade ago. “What would Finland do?” prompted a cottage industry of commentators about how we could copy whatever it was that Finland was doing and then, as a result, improve our own results.

But this was sloppy thinking. As Pat Wolf pointed out in the tweet below, researchers call this “selection on the dependent variable.” That is, you can’t just look at what the high performers are doing and try to copy them. Making policy prescriptions that way can easily confuse correlation with causation, and you can’t tell what really caused an outcome just by looking at what activities were completed.

Yeah, an entire gaggle of ed policy scholars made their careers out of encouraging America to imitate Finland when that country topped the PISA rankings. That's the problem with selection on the dependent variable research: you don't know if the outcome was caused or just a fluke https://t.co/RnxkF18WEd

— Patrick Wolf (@P_Diddy_Wolf) December 4, 2019

Put another way, were Finland’s results caused by their ethnically homogenous student body, their low teacher turnover and high bar to entry to the teaching profession, their school choice policies, their high-stakes standardized test given to high school seniors, their national curriculum, or something else? We don’t know! But that didn’t stop advocates from championing these ideas, or arguing that Finland was successful without some common reform ideas espoused here in the States.

I’m afraid we’re already starting to see this same “edu-tourism” in the wake of the recent NAEP results. Mississippi and DC stood out as two places that bucked the national trends, but it’s hard to say what caused those positive results. Instead of visiting those places and looking backward at what practices make them special, we should be consulting research on the specific policies those places have attempted. For example, we should pay much more attention to the empirical evidence on DCPS’ teacher evaluation program than on any policy prescriptions coming out of the NAEP results.

I don’t want to bash Finland, but I do hope Finland’s recent decline will serve as a cautionary tale. And no, I don’t mean trying to diagnose why Finland’s scores are now declining. That would be the exact same mistake but in the opposite direction! No, I mean that we should stop trying to identify policy prescriptions by blindly copying high performers.

–Guest post by Chad Aldeman