Improving teaching through examining student work

drawing by Stephen Augustus, 6th grade,
Hoover Middle School, Long Beach, CA

By John Norton

"Is it real? Is it real?"

Ms. Estrada's enigmatic smile gives nothing away. She's dressed for an archaeological dig -- khaki shorts, hiking boots, and a long paintbrush in her back pocket to carefully sweep dirt away from the fragile artifacts scattered around the human bones. A sign reads: "Use Caution! Paleolithic Grave Site! Archaeologist at Work!"

Estrada has shoved desks back against the walls to make room for her excavation site, arranged like a boxing ring with the tape strung from dayglo orange posts. A grid of twine criss-crosses the protected area -- the north-south lines are numbered; the east-west lines have letters.

The kids have been asked to become part of a field expedition that has just uncovered "an amazing find." It's their job to recreate the grid on paper and carefully plot the location of each artifact, using the same methods used by archaeologists and paleoanthropologists.

Estrada's students have been tracing the early history of the human race, often reading the actual work of scientists. They can talk about Neandertal Man, Homo erectus, Homo habilis, and other ancient hominids.
They're also learning about "Lucy," the 3.5 million-year-old Australopithecus skeleton discovered by Donald Johanson in 1974 along the banks of the Awash River in the Hadar Wilderness of Ethiopia. They will watch a video recreation of the event, read primary sources (including Johanson's own account of his team's discovery), and sense, perhaps, some of the excitement the scientist-historians felt on the day of their discovery:

"In that 110 degree heat we began jumping up and down. With nobody to share our feelings, we hugged each other sweaty and smelly, howling and hugging in the heat-shimmering gravel, the small brown remains of what now seemed almost certain to be parts of a single hominid skeleton lying all around us."

As today's "dig" progresses, some of the kids calm down enough to begin working on their grid maps. But a few continue to circle the pile of bones incongruously flopped on their familiar classroom floor.
"That's not real, is it, Ms. Estrada," says one girl who's determined to get to the bottom of this mystery.

"Hey," calls out a blonde-headed boy with a mischievous grin. "It says 'Made in China' on this skeleton's leg!"

"The methods of historians"

A few buildings away, Estrada's sixth-grade teaching colleague Janet Leis is leading a discussion about the cave paintings of Cro-Magnon Man. The kids are surrounded by thick picture books about the early humans, many of them authored by famous scientific explorers. These words are written in large letters on an erasable board at the front of the room: "Standard 3: Historical Perspective -- Students will analyze the development of early humankind."

Several students speculate about the reason for the cave art. One suggests that it was connected with a rite of passage, celebrating a boy's first kill and his passage into manhood. Another wonders if the drawings of elk and bear connect to tribal folktales. A third suggests the animal drawings (no people ever appear in these pictures) are religious in nature.

"We know they needed the animals to live," Leis nods. "Maybe that's why they worshipped them. But because we understand the methods of scientists and historians, we realize this is all scientific speculation. There's no way we can know for sure."

The students watch a brief videotape reporting on a French scientist's theory-developed from observing aborigines-that the early humans mixed saliva, charcoal, and pigment in the mouth and then spit out the mixture. "Oh, yuck," several middle schoolers respond in unison. But another mutters "Cool" as the scientist applies a similar mixture to a cave wall.

"We're going to try cave painting today," says Ms. Leis, "but I don't think I'm brave enough to have us mix this in our mouths and use our own spit." Several sounds of disappointment. They use bowls instead, stirring terra cotta clay and water with charcoal sticks and applying the mixture to brown paper.

"We're going back to 20,000 years B.C. There's no evidence people lived in the caves, so we're going to be quiet and respectful," Leis continues. "And it was dark so we're going to cut down the lights. And Cro-Magnon Man was among the first to have music, so we're going to put on some flutes, which have been found with their remains."

Each step of the way, Leis pushes the children toward authenticity. No cars or golden arches, no dinosaurs ("remember, they disappeared 65 million years before."). To the greatest extent possible, in 1997 A.D. Southern California, it's real history.

Teachers in the hot seat

Six hours later, Janet Leis is probably wishing she'd brought some of that calming flute music along with her. Today, she and 8th grade teacher Mary Massich are taking turns in the "hot seat" as their fellow history teachers -- all friends and colleagues -- methodically dissect samples of their work.

Seven teachers sit around a table piled high with food. "The last meal," one jokes. Massich goes first. She hands out a proposed performance assessment in U.S. history, in which students will be asked to demonstrate their understanding of a district standard: "Identify the Bill of Rights and its relevance to everyday life."

"I'm looking for ways to make this more meaningful," Massich says. "I want everybody to read this, do their scoring, and tell me what you think."

The teachers read silently through the assessment, which includes a chart students must fill in and a writing assignment, complete with a scoring rubric -- a guide that tells students what Massich expects to see in their writing. Each teacher has an "assessment task rating form" which the group is continuing to refine. It links back to a more detailed document -- "Hoover History Department Performance Tasks" -- which provides a rich description of what the department expects of its students, organized under three questions: How well did students understand the content? How well did they elaborate in writing? To what degree did they analyze information?

As each teacher calls out his or her scores for Massich's assignment, university professor and team coach Linda Whitney records them. Scores of "3" and "4" are considered good marks; "1's" and "2's" send up warning flags. It's a sensitive time. Massich's jaw muscles tighten as she hears a string of "1's" and "2's" in the category "Disciplinary process: Task asks students to inquire, research, or communicate."

The focus is on the work

Whitney suggests they start the discussion by having Massich rate her own work. And despite a little defensiveness in her tone, she's tough on herself -- it's not her first time. As the discussion progresses around the table, the teachers are both frank and supportive. The focus is on the piece of work, not on Massich herself.

"I have a real concern about something in your rubric," says 6th grade teacher Alicia Estrada. "To get four points, students have to demonstrate a 'thorough understanding' of the Amendments. To get three points, they just have to demonstrate 'an understanding.' What's the difference? How do they know what 'thorough' means?"

And so the conversation continues, as the teachers probe the draft assignment, helping Massich sharpen her own thinking about its quality and purpose. Massich will take their feedback and refine her work substantially, she says. "I really need to think this whole piece through again."

After an hour, the group turns its attention to some work by Janet Leis's 6th graders, growing out of their study of ancient hominids and the discovery of the "Lucy" skeleton. Where Massich was asking her colleagues to analyze a proposed task, Leis has brought in a finished assignment, complete with student work products that can be used to help judge the effectiveness of her unit. The group spends the next 90 minutes exploring the Lucy project in great depth, relying once again on their performance framework to guide the discussion. (Read some of their conversation.)

By 6:30 p.m. everyone is exhausted but all agree that -- once again -- the courage to come to the table and bare your professional soul has paid off.

How do learn to do this?

Educators would be quick to tell you that what the Hoover teachers are doing every Thursday afternoon from 4 to 6 (and sometimes later) is highly unusual.

And by most educator (and public) standards, the teaching going on in these teachers' classrooms (as evidenced by Alicia Estrada and Janet Leis' lessons today) is so exemplary -- so rich in hands-on activities, higher thinking, and historical detail -- that it doesn't need much improvement. So why are these teachers subjecting themselves to this professional gauntlet?

The answer can be traced back to a Saturday lunch at Mary Massich's home, nearly five years ago. "That's when we really began developing camaraderie," Leis says. When a former Hoover principal suggested the group apply for a California Demonstration Grant, "some of us were a little skeptical, because we knew it meant a lot of time. But we knew it was a great opportunity so we said yes," Leis recalls.

The grant was funded, and the Hoover history teachers received $25,000 a year for the next three years to strengthen their department. They spent money on primary source material and other teaching tools, but most importantly, they began purchasing their own professional development.

"We hired consultants and professors, based on what we decided we needed to know more about," says Massich, who has been the de facto leader of the group. "If we wanted to know more about a teaching strategy or a period in history, we either went to the source or we brought the source to us." The group drew on experts at Berkeley, Cal State Long Beach, and UCLA, where Dr. Amanda Podany, an expert on Mesopotamia, taught the Hoover team how to use primary sources from the time period with 6th graders.

For three years the team absorbed large helpings of history and teaching strategy. When the State of California offered them a fourth-year grant to become a model school (they were the only history program to be asked to continue), the teachers balked.

"We felt there was a critical piece missing," says Massich. "We felt the need to sit back and reflect on our own teaching and look at how the staff development was actually showing up in our lessons. Was it trickling down to the kids?"

"We felt there was a critical piece missing," says Massich. "We felt the need to sit back and reflect on our own teaching and look at how the staff development was actually showing up in our lessons. Was it trickling down to the kids?"

Luckily, Massich and her colleagues were able to convince grant officials to let them focus on reflective practice instead. "So we began the process of looking at student work to see if the students were actually achieving what we had set out for them to do."

The decision to reflect on their own work came just in time for Massich, she says. "I had been teaching American History in 8th grade for eight years and I have a real passion for it, but I was beginning to feel 'content-weary.' I was thinking about doing something different. Then we started this process of analyzing student work, and I realized that I needed to make some big changes in my teaching. My weariness vanished."

Janet Leis says the group's decision coincided with a district workshop in August 1996 presented by Ruth Mitchell, a national consultant on standards and teacher collaboration. While many LBUSD teachers reacted negatively when Mitchell challenged them to question their own teaching, Leis says, "they probably weren't ready to hear it. But we were. She gave examples of standards-based lessons and we realized what we were asking kids to do didn't always show higher level learning. She said we have to really know what we want kids to learn and do, and that came as kind of a revelation."

Collaboration does not require consensus

Teachers who meet together to examine student work are still the rare exception -- not the rule. But interest in the process is growing nationally. It's one of the fundamental tools used by The Coalition of Essential Schools, a well-regarded reform group based at Brown University.

"Unlike a standardized test, [student work] speaks directly and revealingly of what teachers and students actually do and learn," writes Coalition author Kathleen Cushman. "Like a compass reading, it can then translate into informed action; changed perceptions of students; revised curricula and teaching strategies; new goals, and a sense of direction for a faculty."

So long as collaborative groups respect each member and follow a non-judgmental process, Cushman says, their discussions can deepen understanding even among people "who come at school change with very different beliefs and assumptions."

That's clearly the case for the Hoover team, whose members often disagree about issues like academic standards, teaching strategies, and grading. The team includes not only the regular 6th, 7th and 8th grade history teachers, but Yvette Gow, who teaches special education, and Maria Estrada, an English Language Development teacher.

Two of the seven team members are fairly new to Hoover. Brad Rudy has been teaching full-time for less than two years, and Tim Evans transferred to the school from Avalon a year ago. Evans, a mid-career teacher, admits that he lacks the deep knowledge of history that the team's veteran members (Massich, Alicia Estrada, and Leis) have developed as a result of the grant and their own passion for the subject. (Estrada went to Cambodia this summer to increase her first-hand historical knowledge of the native country of many Long Beach families.) Evans says the collaborative experiences make him realize how important content knowledge is to high-quality teaching. "I want more of that for myself," he says.

"It was nice and comfortable conversation," says university coach Linda Whitney. "Then we started asking more pointed questions, like whether there was rigor in the lesson -- whether kids were likely to meet standards. The questions became more profound as we went along."

Rudy did his student teaching at Hoover with Leis as his mentor "and I can look back now and see they were already preparing me to be a part of this team." When the time came to collaborate, Rudy says, "I was reluctant at first to put my work on the table.

"I was new and I felt I was under the microscope. I was worried they would rip my lesson apart. But there was more dwelling on the positive -- how students could get more out of it. You come out of a session wrung out mentally and emotionally. But you say, 'O.k., the kids are going to benefit and that's what it's all about.'"

Every Hoover team member has a similar story to tell about the painful process of getting better.

"A lot of lessons are teacher-centered," says Yvette Gow. "What the teacher wants to teach or is most comfortable teaching. By working with the history department group, I've really come to ask, 'What is it my students need most? And how am I going to deliver that?' If I wasn't with this group, I would not be at the level of lesson preparation and execution I am now."

Alicia Estrada says the group sessions are "such a stimulant. They always make you think a lot harder about what you're doing." That's tough on perfectionists, she admits. "It's making me miserable! I beat myself up all the time. I'm always asking, 'what kind of teacher am I?' I'm having to learn that I can't teach a perfect lesson every time. You just have to accept that you're building a foundation."

University coach Linda Whitney says the team's reflections grew deeper over time. "At first, and this is predictable I think, the conversations were fairly safe. 'What was your objective? What did you want your students to know?' It was nice and comfortable conversation. Then we started asking more pointed questions, like whether there was rigor in the lesson -- whether kids were likely to meet standards. The questions became more profound as we went along."

The idea is slow to spread

Mary Massich speaks for all of the Hoover teachers when she says the collaborative process has been "the most powerful thing in my professional life." So the idea must be spreading like a California wildfire, right?

Not really.

Teachers from within and without the school come and observe the Hoover history team, Leis says, but the idea is slow to catch on. "I think you have to have a level of readiness for it, and not every teacher is at that point."

Many teachers, Leis believes, "feel like we're supposed to know it all and be perfect because we're teachers. And we don't."

"We just kind of have to get humble with each other and realize that two, and three, and four heads are better than one," she says. "Together we can come up with something really awesome, but by ourselves we just don't get the same product. When we're through, I feel like it's our piece of work, not just mine, and I feel stronger about what I'm doing."

Hoover principal Gary Graves sees the history team's work as one of several key strategies to improve the school. "I've learned a lot from watching these teachers," he says. He's particularly struck by the powerful impact of the state demonstration grant "which allowed them to make a lot of their own decisions about how to improve their teaching." And he's convinced that CSULB coach Linda Whitney has been an important catalyst in the team's growth over the last year.

Acting on what he's learned, Graves has helped convince Whitney and the Hoover English department to pursue the study of student work as well. And the school has set aside $5,000 grants for each subject-area team that they can use to design professional development for themselves.

"In our school, just like any school, I see a good core of teachers who are willing to try new ideas," Brad Rudy says. "You always have some who just don't want to change, no matter what. And some are just skeptical because they've been asked to change so often without much result. But we can show that what we're doing works. It's not just change for change's sake. Our students' work is the best evidence."

Spreading the word across the school system

As a college professor and former consultant to the California History Project, Linda Whitney has seen the collaboration process trigger some powerful changes in several Southern California schools. She's also seen the process falter when teachers succumb to their fears about possible imperfections in their teaching.

"Teachers have to come to see that this process is not about teachers, but about students," Whitney says. "I think there's this feeling that you came out of college, and you had all the skills and knowledge that you needed, and that whatever you did was supposed to be right.

"But this work shows that good teaching is a developmental process, and in fact, it's never going to end. It's all right to say you don't know. These aren't mistakes we're looking at. They're examples of a teacher's work that's going through the process of getting better and better."

Is the kind of work the Hoover history teachers are doing likely to spread throughout the school system? It's already happening at a few schools, although probably not at the level the Hoover teachers have taken it. A year ago, interest in their work was modest at best - only a handful of teachers showed up at a workshop they presented during an LBUSD innovations conference, and several left quickly when they realized what the collaboration process entailed.

But interest and awareness about reflective practice has grown among middle school teachers in the last year, and Mary Massich hopes for a much bigger crowd at this year's "Carpe Diem" staff development meeting. "We've got a better spot on the agenda," she says. When district leaders went to New York in September to meet with foundation officials, Massich was one of two teachers they took along.

Still, the district's commitment to as much "bottom up" reform as possible makes it unlikely that the Hoover model will spread quickly.

While the LBUSD central leadership is enthusiastic about the benefits of teacher collaboration around student work, there is not yet "a cohesive movement" across the school system, says Kristi Kahl, who directs middle school reform for the district. Schools themselves will make the decision about whether to include the study of student work in their reform process.

"We know it requires a great deal of time from teachers, yet we also know how important it is to take this time," Kahl adds.

Whitney says she's convinced that structured reflection on teacher and student work is one of the most powerful tools schools can use to raise student achievement -- "and that's what it's all about."

"There needs to be a lot more of this work in our schools," Whitney said at the end of the Hoover group's two-and-a-half-hour dissection of Massich and Leis' lessons. "You have to risk your ego but afterwards you think, "Wow!", I really gained something by enduring this process, and now I'm willing to throw myself out there even more.

"These teachers at Hoover have become anxious to learn," she says. "They did not have to risk this, but they did. Now they love it when somebody puts something on the table and they can talk about it. They love the learning process. This group is learning what teaching is -- what it really means to be a teacher."

Want to begin collaborative work in your school? Hoover teachers have some tips.

Look at Janet Leis' student work and examine the actual collaborative process yourself.