Formative Assessments: Our Compass for Understanding Affective, Cognitive, and Physical Aspects of Information Search Processes

Original photo by Buffy Hamilton

I’m currently working with a section of American Literature/Composition students who have been asked to look at a historical or current event that embodies a degree of hysteria or abuse of power and compare that to the hysteria or abuse of power in the play The Crucible.   This research task is a fairly typical high school assignment for this text and course.  Initially, the teacher wanted me to cover five major areas of research in one class period in lecture format with no hands-on learning activities because she initially thought they had a greater degree of prior knowledge.   However, after a series of email conversations, we worked out a series of learning experiences to implement this week to address student learning needs in a richer and more meaningful way for students.   As many of you know, these collaborative conversations are sometimes really difficult and uncomfortable to broach, especially when you are new to a school or don’t know a teacher very well as you want to be respectful yet honest when you realize a particular request for instruction might not be realistic or effective for students.    I also try to remember that many teachers have never had the experience of working with a school librarian who is genuinely interested in being a co-teacher and instructional designer, so they might not want to ask much of you simply because they’ve never had that kind of expectation or instructional services.

We began our efforts earlier this week by introducing the students to specific databases and search tools in GALILEO through our project LibGuide.  We also talked briefly about some basic search features and tips for each of the databases.  I also showed students how to sign up for our EasyBib account and how to export potential sources to their EasyBib project bibliography.  In an ideal world, I would not attempt to cover so much territory in one session, especially with students who have little experience using databases, but when forging new collaborative partnerships with faculty, it often takes time to establish the trust and rapport needed to jointly fine-tune project pacing and the instructional design process.  It is also a regular challenge many school librarians face as we try to balance our desire to go deep with hands-on learning activities and time to engage in inquiry while respecting the pressures and time constraints classroom teachers face with pacing calendars, common assessments, and other aspects of modern classroom life.

Yesterday we formed “Birds of Feather” groups by research interest.   Topic areas/groups included McCarthyism, post 9/11 racial profiling, Japanese Internment Camps, and then “undecided” for students who were still exploring.   I created topic placeholders for each table to make it easier for students to self-form groups because I wanted them to work together to talk about search terms and databases they were trying so that they could hopefully tap into the power of collaborative thinking.   My plan was for them to collaborate as searchers, to work on their search maps, and then complete a group debrief/”ticket out the door” assessment we’d do with big sticky pads (see below).

Teens being teens, though, some decided to sit with friends rather than by topic; as a result, we lost a little instructional time because the teacher wanted to get them in the “like” groups at the beginning of the period before we proceeded any further.   Once everyone was settled in, though, we jumped into the day’s agenda:  I reminded them how to get back to the LibGuide (and they also had a set of notes/graphic organizer I had provided the day earlier to help them as well) and did a brief introduction to search mapping.    I did not take as much time as I normally would because:

1.  We had already lost about 10 minutes of class.
2.  I originally thought this might be an optional activity,  but after observing their initial seach efforts on Tuesday, I realized they needed this form of scaffolding to help them navigate the databases with some deliberation and intention.

I told the students I would collect their maps and whatever progress they had made as their ticket out the door and provide feedback the next day.  They then set about searching the databases and beginning their search maps; the teacher and I walked about providing feedback and answering questions with individual students and small groups. This was also a great opportunity to just observe and see what students were doing and how they might be thinking. Toward the end of the class period, I asked students to flip their maps over and complete the simple and fast self-assessment below:

I collected their maps at the end of class; this morning, I read through their work as well as their self-assessment reflections for about 90 minutes taking time to provide brief written comments and making notes about points of confusion and questions.  By looking at this simple self-assessment, I was able to see patterns of misunderstanding or difficulty while noting specific questions to address/answer.   In particular, I could see that students were struggling with:

1.  Slowing down the process and doing pre-search with more intention as they worked with the search mapping process (new for all students).  This “slowing down” process  also included reading beyond skimming and scanning.   I call this “doing the work”; I recommend this brief but terrific post by Pegasus Librarian on this challenge of reading and search; also see her brilliant post Information Literacy in a Utopian High School.
2.  Working through informational text—-some were having difficulty pulling out the big ideas, so I pulled together some resources from Cris Tovani and Stephanie Harvey on text annotation (this is not new or unfamiliar territory!).  I also decided to create a template for a Venn diagram to help students visualize their big ideas since the teacher had suggested that to some students yesterday, and it seemed to help them make sense of the research task (another area that I soon realized later in the day was a clear area of misunderstanding for some)
3. How to deal with dead-ends with search terms and understanding that one database would not be enough for this research task as well as additional strategies for search terms specific to databases.

By looking at the students’ work, I could provide some specific written feedback and help resources (either in print format that I attached to their maps and/or as resources I added to the LibGuide this morning).

When students arrived today, we returned their maps.  I then shared how I had looked at their work and the three big areas of concern/patterns of response; I also showed them resources added to the LibGuide (handouts, graphic organizers, how to videos, website links) to help them negotiate those challenges.

I also tried to allay their uncertainty and discomfort by talking about the fact that search is often just difficult and takes a great deal of reading, browsing, and persistence in trying different strategies.   We also talked about “digging in”, making sure our conversation with peers was constructive, and advocating for ourselves by asking questions and choosing a different seat if the current workspace was too distracting.  Last but not least, we celebrated the exemplary search maps from Day 1, and I showed them where they could download or print a copy of these model search maps (which of course are still in progress) on the LibGuide.

The class was then given the rest of the period to continue their presearch/search mapping , and they were encouraged to seek help from both me and the teacher.  She circulated about and answered content related questions for the research task while I set up an area where students could come and do 1:1 or small group conferencing with me, an activity that was very insightful for me as well as the students.    These mini-conference/conversations were very revealing and helped me see that some students just needed verbal reinforcement of what to do while others clearly didn’t understand the research task at all.  Other students needed a visual graphic organizer to unpack the research task (see this simple Venn diagram I was inspired to create for the students after hearing Ms. Sidell, their teacher, suggest it to a student yesterday).

It was also a great opportunity to ask students about what they had learned in their initial reading yesterday, and that opened up honest yet encouraging conversations with individual students about taking time to actually read the texts and the importance of reading in developing search vocabulary and ideas for interpretation and discussion in their papers.

These two methods of formative assessment in 24 hours reminded me to look to the work of Carol Kuhlthau to help me contextualize much of what I was seeing and  hearing from the students.    Her Information Search Process model underscores the affective aspect of information seeking behaviors and reminds us that we need to help students acknowledge, honor, and own the feelings of confusion, doubt, and frustration that can come with the messiness of Initiation and Exploration.  Teachers, students, and librarians can embrace the discomfort and leverage these teachable moments as opportunities to help students hone and grow their strategy toolkit that will ultimately help them develop persistence and resilience in the face of challenging information seeking tasks.  I highly recommend you read these three posts from my colleagues and fellow practitioners Heather Hersey, Marci Zane, Meg Donhauser, and Cathy Stutzman:

• Confusion and Frustration Were Rampant
• Why Can’t You Just Tell Us What to Do
• Teacher Strategies :  When Learning Gets Emotional (this is not just for kids–it’s for us as adults as well! Points 2 and 6 especially resonate with me)

As many of you know, working through these rough patches is often a healthy mix of tough love and patient TLC with a generous helping of practical strategies for learners.  By engaging in formative assessment in various formats early in the inquiry process, I stay grounded and can better understand the stumbling blocks for the students.   It also helps me to unpack what might initially be seen as negative behavior such as resistance or even hostility when in reality, that behavior is camouflaging sincere student distress and fear, especially if they have little experience in negotiating challenging or difficult academic endeavors.  Consequently, I’m then better positioned to offer practical coaching and help rather than getting stuck in the weeds of frustration that can spring up as quickly for us as it does the students.    These formative assessments also give us evidence and guideposts for adjusting instruction, something that is particularly important when we are helping the classroom teacher think through the kinds of learning activities students need and how to appropriately pace them by being responsive to student needs.

How are you engaging in formative assessment?  Where does this intersect with Kuhlthau’s ISP for you, and how do you help students and teachers work through the messy and uncomfortable chaos that is in inherent in search and information seeking tasks?  How do you help frame the feelings of uncertainty and confusion in a constructive and positive light?

Holistic and Individualized Formative Assessment of Research and Inquiry Processes

For the last two weeks, our students have been immersed in investigating information and constructing new understandings as they have been composing their research design proposals, revising sections of their proposals, and doing additional research after focusing and narrowing their topics and research questions.  As they have gone back and forth in refining their topics and questions and doing the subsequent additional research, we’ve seen our students move back and forth between confusion/doubt/uncertainty/discomfort and clarity.  Most students are not used to doing this sort of deep dive with a topic, making their own choices about the topic and research questions, articulating how they demonstrating growth in their learning, or selecting their learning products; consequently, the messiness of choice and ownership of their projects has been a new experience (and uncomfortable to varying extents) for them.  Sarah Rust and I have also experienced a spectrum of emotions in this inquiry process as well;  we know our students will grow from these experiences yet we too feel some of that same uncertainty and frustration as our students when they wobble or stall in spite of our efforts to scaffold and support with individualized feedback, resources, and reflective questioning.   Like our colleagues Heather, Meg, Marci, and Cathy, we provide  them strategies and feedback that will propel them forward and give them the tools to self-help, but as we have told them, we cannot make the decisions for them or give them the answers.   We stay calm and reiterate that we are focused on how and what they are learning, not grades—of course, this discourse is a departure from the narrative they have heard their entire school lives in our test-driven culture.

The individualized and fluid nature of working with 50+ students who are all doing different topics is also a newer experience for us and definitely for the students.   Over the last two weeks, any given day has been a potpourri of joy, exasperation, delight, and doubt as students have drafted their research design proposals for their multigenre projects.  This kind of work is where the collaborative partnership Sarah and I have is critical because you have an instructional partner to be responsive to these kinds of learning experiences and individual student needs.  Because we both bring different strengths to the table and can process what we are observing with student work together, we are much better positioned to truly help our students than if we were doing this in a solitary or prescriptive, rigid way.

After receiving the drafts of their proposals for their projects, Sarah and I have employed a variety of strategies to personalize the feedback for each student at their points of need.  Here are some of the action steps we’ve tried:

Individualizing and Capturing Feedback Through Mail Merge and Databases

I created a database in Word of all of our students in 2nd and 3rd periods.  Data fields I created included:

• First name
• Last name
• Class Period
• Comments About the Narrowed/Focused Topic
• Multigenre Products Students Selected
• Publishing Platform of Virtual or Paper (Word/PDF)
• General Notes (comments about student self-selected learning targets, what they know about their topic at this point, what they want to learn, research questions, their working bibliographies, and search terms/strategies.
• Next Steps–specific tasks and suggestions to help the students move forward.  These action steps could also include requests for students to schedule 1:1 help or to participate in some of the small-group help sessions we set up in response to the patterns of thinking and gaps we saw in the proposals.

I went through each proposal and typed in my feedback for each student in the appropriate fields in the database document.   I then used the Mail Merge wizard in Word to create a “form letter” that imported this feedback and printed out the feedback documents for each student on colored paper or in color.  Once I printed completed feedback forms, I stapled them to the research design proposal draft and returned to the student as soon as possible for them so that they could move forward or make revisions.  I also provided a copy to Sarah so that she could begin developing a list of needs to address and to prioritize which students needed her help and areas of expertise.  The master database provides us an archived record of the formative assessment to use as we look at student growth; it is also easily accessible to reprint should a student lose his/her copy of the feedback form.

It did take quite a bit of time to methodically go through each proposal and to generate the personalized feedback.  However, I so appreciate the opportunity to engage in this sort of assessment because it helps me get to know the students as learners.  This work also improves my instruction because I can easily see patterns of understanding and confusion and helps me to be a more reflective and effective practitioner as well as instructional designer.

Conferencing/Coaching/Triage 1:1 and Small Group

 

Using this information as our starting point, Sarah and I have  been meeting with students the last few days (late last week and all of this week)to discuss the feedback we’ve provided them ; we use the feedback forms as a strategic entry point for face to face conference/coaching conversation with students.   We have been organizing our 1:1 meetings and small group sessions through a variety of mediums each day:

• Students can sign up for specific individual help each day—we have used large post-it paper and our Verb dry erase boards as our parking lots for students to indicate they need assistance or have questions.
• Students can sign up for small group help or indicate they want to join a future small group work session through our Verb dry erase boards.  For example, after reviewing all the research design proposals, I realized I needed to do some small group instruction on additional search techniques with Boolean operators and additional instruction on mining Academic Search Complete.
• For those who might be shy or reluctant to place themselves in one of these help request parking lots, we’ve also been sure to work through our class rosters and are checking in with each student so that we are sure to meet with EVERY student and “check up” on their progress, successes, questions, and worries.

Yesterday, Sarah called students up by the class roster whereas I started with my list of student requested help.  Today we approached the scheduling of the 1:1 conferences by working through the class rosters and having students first check in with Sarah about some of their recent process work; students then moved to my table for to discuss the feedback they received from us on their research design proposals.   We each set up a help area with our mobile tables and our green Hon rolling chairs so that we had comfortable spaces to talk to students and where they could spread out their work and/or where we could show them specific resources or skills on our laptops if they needed some concrete visualization or examples.  Some conferences are brief while others are more extended, but typically, each meeting can last 3-10 minutes—it all depends on student need and how the conversation evolves in the conference.  We also keep notepads, large lined sticky notes, and/or Google Docs available at the conference table to jot down notes from each meeting while students bring along their folders of their process work, drafts of their design proposals, and the individualized design proposal feedback form.

In just these first few days we’ve been meeting with students, it’s very apparent when students feel confident (and skills/processes/ideas they’re self-assured about as well) and where students feel fuzzy, unsure, and/or anxious.   We’ve also observed that most of our students are not used to this level of accountability, and some seem a bit uncomfortable with it when you are asking them questions to nudge them to dig deeper or be more specific with details; we sense many are also not used to these types of conferences that puts the responsibility and decision making on them as students.   We are framing this conference/coaching sessions from a stance of discussions to help them think through their choices, to clarify their own thinking/choices/next steps, and to move forward with their projects since we don’t want them to see the messiness and muckiness of inquiry as punitive.   These sessions have also helped us identify those who might benefit from some of our upcoming small group mini-lessons but who may not have initially signed up for assistance.  Last but not least, I believe these conferences convey to our students that each person matters and that we care about them and their topics.

Reflections

While we cannot do their work for them, we can give students every opportunity to get personal assistance in a low-key setting —we want them to know they cannot fall through the cracks or simply fly under our radars.  While I’ve done this sort of work before, this is probably the biggest chunk of time I’ve had in a collaborative partnership for this level of assessment and 1:1 student conferencing.  This approach requires us to be agile and responsive as each day is different and every student need varies.  This kind of conferencing/coaching is time consuming and messy; while the prep for the small group work is pretty straightforward, the 1:1 help is definitely open-ended.  I have been inspired and am improving my own conferencing skills with students by watching Sarah (who is a master at this process) and by my friend and fellow school librarian Heather Hersey.  Her post about the importance of conferencing helped me to think about focusing on all aspects of their inquiry work and design proposals rather than just sources or their bibliographies; it also inspired my idea for using the mail merge form and database to capture feedback and use that as a starting point for the student conferences/coaching sessions.

Sarah and I have also been discussing how intense this kind of work is and how you have to be comfortable with making adjustments as needed to timelines and your plans in order to be responsive to the students.    The processes are messy, yet this “mucking around in ideas” is the grist for the growth and critical thinking that happens as both we and our students problem solve, question, and revise our ideas and stances.  Neither of us has any idea how someone would do this kind of process-driven, organic, fluid, and reflective work alone!  We love that our combined talents help the students as well as each other; we also are appreciative of having someone else each day who can help you see things you might have missed or to think about a particular situation or challenge with fresh eyes.   We are also excited we can model collaborative learning for our students—how often do they get to be in a learning environment where there are at least 2-3 adults who can help them and provide them the kind of specific and personalized attention they are receiving?  Most importantly, this type of collaboration is a catalyst for inquiry work and for integrating more formative kinds of assessments that benefit students and impact learning.

We expect the 1:1 and small group conferences, coaching, and small group instruction to continue the next 7 days of school leading up to our Thanksgiving break.  I hope to share more images, video, written/video reflections and feedback from both of us as well as our students in an upcoming post later this month.   I’m also thinking about how to better integrate the conferencing/coaching/conversation aspect into the inquiry approach (and at an earlier point in time) with research using Cris Tovani’s conceptualization of these conversations as data and formative assessment (see her text, So What Do They Really Know?  Assessment That Informs Teaching and Learning).

How are you approaching assessment with inquiry work?  How do you negotiate and embrace the challenges of time and fluidity with this approach to learning and research?  How do you scale this kind of learning experience when there are always challenges of time, space, and staffing?

Simple Yet Powerful Formative Assessment of IR with Sarah Rust

Every Wednesday is Independent Reading (IR) day here in our Language Arts classes here at NHS.  Today, Language Arts teacher Sarah Rust, one of our awesome collaborative partners, did this very simple yet interesting formative assessment with her students.   The instructions:

Students selected a sticky note of a color of their choosing and then composed their responses.  As an extra touch to celebrate the concept of IR, Ms. Rust then took their responses and fashioned them into the letters “IR.”   While this idea seems simple on the surface, the student responses were revealing and showed a wide range of book selections as well as reactions to the IR experience.  These can be a springboard to future IR learning activities and learning experiences for book selection and peer sharing.

 

 

It’s another reason why sticky notes are my favorite “technology” as of late!  This approach is a great way to do a quick individual assessment of student learning or where they are with their current IR as well as make an artistic class statement that represents every student voice.

Sticky Notes as Formative Assessment for Information Literacy Instruction: Coding Student Responses

Yesterday I blogged about our pre-searching activities and the use of sticky notes for some gentle formative assessment.  Today I want to share how I went about coding the student responses not only to get a sense of students’ thinking during the two days of pre-searching, but to also use the data as a baseline of sorts in hopefully looking a broad collection of their work as we try to track their trajectory of growth and progress through this extended research unit.

Coding Information Sources

I began by removing the sticky notes for each period from the whiteboards and affixing them to large post-it notes and labeling each grouping by period and response type.  The next challenge was to think of categories for coding the student responses.  The “information sources used” was the easiest starting point, so I began there.

I listed all the information sources from the LibGuide for the project and then tallied responses.  I wound up adding Google as another category since some students indicated they had used this search engine.  Here are the results by period:

 

In both classes, it appears Gale Opposing Viewpoints was a starting point for the majority of students; Gale Science in Context was next in popularity.  2nd period seemed to like SweetSearch and self-selected information sources while 3rd period leaned more heavily toward Academic Search Complete.

When we look at the updated topics roster (while taking into account the intiial list of topics they had generated), the numbers are not too surprising.  I know that many of them will benefit from some guidance into specific databases and search tools that will align with their topic choices as we move deeper into the project, but I’m not terribly surprised by what I see from the first two days of the risk free pre-search time to just hone down an interest area for one broad topic.  This data, though, does suggest to me that there may be sources unfamiliar to students or they have used minimally in the past (as do the results from the information literacy skills needs survey we did via index cards with Ms. Rust a few weeks ago).

Questions

My categories for coding the questions students generated included:

• Who
• What
• Where
• When
• How or Why?
• Topic Clarification
• Question about the research or the assignment
• Other (other types of questions i.e. Is Finland’s educational system superior to the United States?)
• None

2nd period posed 15 “how/why” questions and 11 questions that fell under “other”; there were four “who” questions and 6 “what” questions; three students did not note any questions.  3rd period generated questions that primarily fell under “what” (4), “how/why” (4), research/assignment questions (6), or “other” (6); five students did not generate any questions.  Clearly, there is a stark contrast between the two classes in the types of questions they generated.  This data may indicate that 3rd period may need more guided help in engaging more deeply with their articles OR strategies for generating questions.

Discoveries and Insights

For this group of sticky note responses, I created these coding categories:

• Fact or concrete detail
• Concept/Conceptual
• Question
• Reflection
• Commentary/Opinion/Reaction

Once I began taking a pass through the student responses, I realized I need four additional categories:

• Topic Ideas
• Sources
• None
• Other

Second period students primarily recorded facts or concrete details for their notes; however, several used this space to think through additional topic ideas; the pattern was nearly identical in 3rd period.  I was not surprised by these findings since students spent only two days doing light pre-search and I knew in advance that getting enough information to eliminate some topic areas of interest would be where many would expend their time and energy.

Final Thoughts

The pre-search activity and days were designed to help students rule out some topics and have time to explore those of interest and our sticky note method of formative assessment was one we felt would give us feedback without imposing a structure that would be time-consuming for students since we really wanted them to channel their energies into reading and learning more about their topic lists.  While some of the data I coded was not surprising, I was really struck by the differences in the types of questions they generated.  Right now I don’t know if this means one class might need more help in generating questions from informational texts or if perhaps they were approaching the reading and activity in a different way that didn’t lend itself to composing lots of questions at that early juncture in time.

If you are incorporating pre-search as part of the connecting cycle of inquiry, what kinds of formative assessments do you use?  If you code student responses, how do you approach that process, and how do you use that data to inform your instructional design?   I find this kind of work interesting—I am looking forward to seeing if any of these gray areas or perceived gaps come to light as we move further into our research unit this month.