Making educational data meaningful and usable

ShareThis

By ELEANOR CHUTE
Across America, educators, parents and students are swimming in an overload of data, much of which seems unrelated and difficult to use.

Over the past decade, education based on academic standards has become the norm. This has fueled the push for data to show how students are doing and to guide efforts to improve.

The trend has only grown since the 2002 signing of the federal No Child Left Behind Act, mandating that all students in public schools be proficient in math and reading by 2014.

Now a big buzzword is "data-driven instruction," which means using the data to try to figure out what a child needs and then trying to provide it.

Kati Haycock, director of the Education Trust, a Washington nonprofit group that advocates for achievement for all children, said data "can basically take us out of the dark ages of just kinda teaching and hoping, which is what a lot of folks have done for a very long time."

"Where this is all headed is personal learning plans on a kid-by-kid basis," said James Turner, director of the partnership for school-district improvement at the University of Pittsburgh School of Education.

The challenge, however, is making the data meaningful and usable.

This fall, Pennsylvania will give every district a new tool called Pennsylvania's Value-Added Assessment System, or PVAAS. The state will pay for the analysis at a cost of about $2 a student for more than 900,000 students.

In 2002, the state Board of Education passed a resolution calling for a value-added approach, which looks at how much of a student's progress can be attributed to the school.

Since then, it has been piloted in about 100 districts. Now all districts will get at least limited data, which will be expanded in future years.

PVAAS uses statewide achievement data in math and reading from the Pennsylvania System of School Assessment (PSSA).

Taking into consideration that students start a school year at different levels, it statistically compares a student to other students statewide who have similar testing histories. It then predicts how the student would do in an average school during the coming school year.

If they make more than the expected year's growth, then that extra improvement could be credited to the school. If they make less, questions are raised about the school's effectiveness.

"Every sound business has systems in place to keep track of what's working and what's not working and focus management efforts on fixing problems," said Alan Lesgold, dean of Pitt's School of Education. "Value-added testing is the first effort in the educational world to do the same thing."

PVAAS uses a complex statistical calculation based on the work of William Sanders, who more than a decade ago began a value-added system in Tennessee.

The Sanders model is "probably one of the more widely respected models for the value-added system," said Suzanne Lane, professor of research methodology in the School of Education at Pitt and former president of the National Council on Measurement in Education.

Lane noted that for the data to be good, the underlying test needs to be one of good quality.

Pilot schools, such as South Fayette, used data from other standardized tests when PSSA data weren't available.

"It changed our district because it allows us to look not only at student achievement but also at student growth," said South Fayette Superintendent Linda Hippert.

On the 2005 PSSA, South Fayette's fifth-graders ranked No. 1 in math and No. 3 in reading statewide. Its eighth-graders came in No. 4 in math and No. 7 in reading.

That favorable report, however, missed part of the picture that PVAAS catches.

For South Fayette, PVAAS revealed that one eighth-grader scored 130 points lower in math than would have been expected and another scored 400 points higher than would have been expected.

It showed that while some fifth-graders who scored low in math didn't meet the state standard, they did grow at a much faster rate than those scoring higher did, thus the low achievers may be headed toward proficiency.

PVAAS doesn't pinpoint skills, but it opens the door to new questions and conversations.

Hippert said, "I can go into the system and say I want to see every (eighth-grader) who is predicted with less than a 60 percent chance of being proficient. Then you can pull those kids and take a look at them and begin remediating in the ninth grade."

Value-added also can be used to spot proficient and advanced students who are declining but still are at least proficient.

"The beauty of the value-added philosophy is that every child deserves at least a year's worth of growth in a year. The highest-achieving kids are just as worthy of growth as the lowest-achieving kids," said Theodore Hershberg, professor of public policy and history at the University of Pennsylvania and a longtime supporter of value-added.

Throughout the country, value-added approaches are hot.

As a pilot, the U.S. Department of Education is permitting two states _Tennessee and North Carolina _ to be judged under the No Child Left Behind Act using value-added approaches, not just achievement levels. Other states, including Pennsylvania, are eager to be considered for such an approach.

(Education writer Eleanor Chute can be reached at echute(at)post-gazette.com.)