I must have been too busy shopping just before Christmas to follow the back-and-forth over the accuracy and fairness of a report in the Dallas Morning News on the early evaluation of a Texas program to raise the quality of Head Start and public and private preschool programs. The newspaper said the evaluation of the first 18 months of the now five-year-old program found that it "has yet to deliver on the investment." That language seems harsh, given that the evaluation report said its design and timing limited its power to make such statements that early in the program. Most of the evaluation had to do with implementation, rather than quality, questions. The evaluators wrote a letter to the Texas Education Agency disavowing the newspaper's conclusion. In fact, the report found that with one year of training provided through the state-funded program teacher performance improved significantly and those who received two years of training improved even more. Although the report acknowledged it was not possible to tell whether overall student performance improved the evaluators did detect gains in three important literacy related skills: rapid letter naming, rapid vocabulary naming, and phonological awareness (an understanding that words can be dissected into discrete sounds).

As journalists, we love rendering judgments, which is an important oress function. But good evaluations are tools for improving programs, as much as they are tools for rendering judgments. It's important that we not overinterpret such work--mistaking recommendations for improvement for fatal flaws. In addition, it's important that we step back and look at what is actually being evaluated. If a report looks at the first 18 months of a program that's operated for almost five years, and if the program today is dramatically different (it served 1,600 students in 03-04 and 27,000 today), caution is warranted.