Robert Slavin, a huge figure in evidence-based education, has written a blog post claiming that John Hattie is wrong. Hattie pursues the approach of meta-meta-analysis. In other words, he brings different meta-analyses together to compute an overall effect size.
Slavin points out that many of the studies that sit underneath these meta-analyses are weak, poorly designed and often don’t seem to relate very well to the concept that is supposedly being investigated. It’s worth mentioning that Hattie accepts at least some of this criticism. That’s why Hattie insists on an effect size above d=0.40 rather than zero. However, Slavin notes that really well-designed studies rarely generate an effect size this large. Hattie is effectively filtering out the good stuff in order to make conclusions based on what is left.
One specific point Slavin makes is about experimenter-designed tests versus standardised tests. In my view, the former are valid and it…
View original post 392 more words