A recent article in Scientific American described how scholars are bringing new rigor to educational research by introducing methods from science and economics. I’m strongly in favor of more rigor in educational research, as well as in the broader social sciences. However, the article itself downplays the challenges involved in this and thus the limitations of experimental research.
This is clearly seen in the lead example discussed in the article, a study by Carnegie Mellon psychologist Anna Fisher and her grad student Karrie Godwin. Fisher noticed that while the walls in her university classroom were barren, a typical elementary classroom was adorned with all kinds of potentially distracting material. So Godwin carried out a small experiment in which two groups of 12 kindergartners listened to three stories about science and then were asked questions on the material. One of the groups heard the stories in a classroom with bare walls, and the other in a classroom decorated with material purchased by Godwin. Analysis of videotapes and the post-test found that the students in the bare classroom were more likely to pay attention and scored higher on the comprehension test.
There are lots of questions about this study, including whether a study of two small groups would yield enough statistical power to draw any conclusions. Let’s put those aside for a second, and assume for the sake of argument that statistically significant results were found. We still have a situation with presumed internal validity, but almost no external validity.
Think, for a minute, of a typical kindergarten classroom. What kinds of things would be found on the walls and in the room, and for what purposes? Well, perhaps the major purpose of kindergarten is to help develop children’s language and literacy ability. And literacy in kindergarten is not taught in isolated lessons, but is rather integrated in throughout the school day. So a teacher would like to have lots of visual materials to support literacy development, such as pictures of the alphabet, phonetic combinations, and sight words, that students could refer to at any time. Beyond that, a teacher would want to create a print-rich environment, with lots of things to read, information about children’s authors, and samples of the children’s own writing–which will also help the students develop their identity as readers and writers. Similar material would typically be displayed in support of numeracy, from numbers, to number facts, to number lines, to geometric shapes.
Of course a teacher also wants to inspire curiosity about how the world works, so there may be science posters, globes, pictures of scientists, and scientific displays. The teacher also wants students to develop positive character, so there may be some school slogans encouraging students to behave respectfully and kindly. The teacher also wants children to school as an inviting and enjoyable environment, so there is reason to make the class colorful and pleasing. Finally, the teacher will likely organize small group work, so there may be various stations organized around the room to facilitate that.
Finally, all these things are chosen by the teacher herself rather than chosen haphazardly by a graduate student for the purpose of a brief study.
Now, it may be the case that, absent all this material, students may be less distracted in hearing a single short story. But the purpose of kindergarten is to develop students’ knowledge, skills, and attitude over the course of a year to prepare them for a lifetime of learning, not merely to help them remember the details of one story.
So, as it turns out, the experiment carried out by Fisher and Godwin is not really that relevant to the true purposes of kindergarten. To shed light on the relationship of display material to achieving those purposes, you would need a very different kind of experiment.
Theoretically, you could randomly assign a large number of kindergarten teachers to either (a) teach in their normally decorated room or (b) teach in a barren room, for an entire year — and then measure the effects on children in a variety of ways. (Even that is limiting, because it doesn’t measure long-term effects). Very few kindergarten teachers would probably be willing to participate in such an experiment, and, even if willing, their schools or districts might balk. (You can imagine how parents would react to seeing little Johnny in a barren classroom for a year when the class next door is filled with rich learning materials–and then finding out that it is all part of a university experiment). Even if you could find teachers, schools, and districts willing to carry out a study, you would need lots of funding to conduct it. I doubt if there is any funding agency that would be interested in supporting such a study, or any educational researcher who would be interested in investing the time and effort on such a quixotic quest.
In other words, the only reason that this research question could be investigated experimentally by Fisher and Godwin was because their experiment did not match the real purposes of display material in kindergarten, or the ways that that material is made use of in actual kindergarten classrooms. To carry out a much more meaningful experiment would be almost impossible. And I doubt anyone will ever bother with such a study, because once we think about the actual purposes of display material in kindergarten classrooms, there are more reasons to think those materials are valuable than harmful. Indeed, once we look at the real world, a recent study of a very high performing school district with 85% Hispanic students, in Union City, New Jersey, found that “word-soaked” classrooms were an essential element of success.
So, by all means, let’s carry out rigorous experiments in schools. But let us also be cognizant of the limitations and challenges of experimental research in educational settings.
Papyrus News 