information literacy

Holistic and Individualized Formative Assessment of Research and Inquiry Processes

For the last two weeks, our students have been immersed in investigating information and constructing new understandings as they have been composing their research design proposals, revising sections of their proposals, and doing additional research after focusing and narrowing their topics and research questions.  As they have gone back and forth in refining their topics and questions and doing the subsequent additional research, we’ve seen our students move back and forth between confusion/doubt/uncertainty/discomfort and clarity.  Most students are not used to doing this sort of deep dive with a topic, making their own choices about the topic and research questions, articulating how they demonstrating growth in their learning, or selecting their learning products; consequently, the messiness of choice and ownership of their projects has been a new experience (and uncomfortable to varying extents) for them.  Sarah Rust and I have also experienced a spectrum of emotions in this inquiry process as well;  we know our students will grow from these experiences yet we too feel some of that same uncertainty and frustration as our students when they wobble or stall in spite of our efforts to scaffold and support with individualized feedback, resources, and reflective questioning.   Like our colleagues Heather, Meg, Marci, and Cathy, we provide  them strategies and feedback that will propel them forward and give them the tools to self-help, but as we have told them, we cannot make the decisions for them or give them the answers.   We stay calm and reiterate that we are focused on how and what they are learning, not grades—of course, this discourse is a departure from the narrative they have heard their entire school lives in our test-driven culture.

The individualized and fluid nature of working with 50+ students who are all doing different topics is also a newer experience for us and definitely for the students.   Over the last two weeks, any given day has been a potpourri of joy, exasperation, delight, and doubt as students have drafted their research design proposals for their multigenre projects.  This kind of work is where the collaborative partnership Sarah and I have is critical because you have an instructional partner to be responsive to these kinds of learning experiences and individual student needs.  Because we both bring different strengths to the table and can process what we are observing with student work together, we are much better positioned to truly help our students than if we were doing this in a solitary or prescriptive, rigid way.

After receiving the drafts of their proposals for their projects, Sarah and I have employed a variety of strategies to personalize the feedback for each student at their points of need.  Here are some of the action steps we’ve tried:

Individualizing and Capturing Feedback Through Mail Merge and Databases

I created a database in Word of all of our students in 2nd and 3rd periods.  Data fields I created included:

  • First name
  • Last name
  • Class Period
  • Comments About the Narrowed/Focused Topic
  • Multigenre Products Students Selected
  • Publishing Platform of Virtual or Paper (Word/PDF)
  • General Notes (comments about student self-selected learning targets, what they know about their topic at this point, what they want to learn, research questions, their working bibliographies, and search terms/strategies.
  • Next Steps–specific tasks and suggestions to help the students move forward.  These action steps could also include requests for students to schedule 1:1 help or to participate in some of the small-group help sessions we set up in response to the patterns of thinking and gaps we saw in the proposals.

rust-feedback1

rust-feedback2

I went through each proposal and typed in my feedback for each student in the appropriate fields in the database document.   I then used the Mail Merge wizard in Word to create a “form letter” that imported this feedback and printed out the feedback documents for each student on colored paper or in color.  Once I printed completed feedback forms, I stapled them to the research design proposal draft and returned to the student as soon as possible for them so that they could move forward or make revisions.  I also provided a copy to Sarah so that she could begin developing a list of needs to address and to prioritize which students needed her help and areas of expertise.  The master database provides us an archived record of the formative assessment to use as we look at student growth; it is also easily accessible to reprint should a student lose his/her copy of the feedback form.

It did take quite a bit of time to methodically go through each proposal and to generate the personalized feedback.  However, I so appreciate the opportunity to engage in this sort of assessment because it helps me get to know the students as learners.  This work also improves my instruction because I can easily see patterns of understanding and confusion and helps me to be a more reflective and effective practitioner as well as instructional designer.

Conferencing/Coaching/Triage 1:1 and Small Group

help help help help 2

 

help help 3

Using this information as our starting point, Sarah and I have  been meeting with students the last few days (late last week and all of this week)to discuss the feedback we’ve provided them ; we use the feedback forms as a strategic entry point for face to face conference/coaching conversation with students.   We have been organizing our 1:1 meetings and small group sessions through a variety of mediums each day:

  • Students can sign up for specific individual help each day—we have used large post-it paper and our Verb dry erase boards as our parking lots for students to indicate they need assistance or have questions.
  • Students can sign up for small group help or indicate they want to join a future small group work session through our Verb dry erase boards.  For example, after reviewing all the research design proposals, I realized I needed to do some small group instruction on additional search techniques with Boolean operators and additional instruction on mining Academic Search Complete.
  • For those who might be shy or reluctant to place themselves in one of these help request parking lots, we’ve also been sure to work through our class rosters and are checking in with each student so that we are sure to meet with EVERY student and “check up” on their progress, successes, questions, and worries.

Yesterday, Sarah called students up by the class roster whereas I started with my list of student requested help.  Today we approached the scheduling of the 1:1 conferences by working through the class rosters and having students first check in with Sarah about some of their recent process work; students then moved to my table for to discuss the feedback they received from us on their research design proposals.   We each set up a help area with our mobile tables and our green Hon rolling chairs so that we had comfortable spaces to talk to students and where they could spread out their work and/or where we could show them specific resources or skills on our laptops if they needed some concrete visualization or examples.  Some conferences are brief while others are more extended, but typically, each meeting can last 3-10 minutes—it all depends on student need and how the conversation evolves in the conference.  We also keep notepads, large lined sticky notes, and/or Google Docs available at the conference table to jot down notes from each meeting while students bring along their folders of their process work, drafts of their design proposals, and the individualized design proposal feedback form.

student conf notes

In just these first few days we’ve been meeting with students, it’s very apparent when students feel confident (and skills/processes/ideas they’re self-assured about as well) and where students feel fuzzy, unsure, and/or anxious.   We’ve also observed that most of our students are not used to this level of accountability, and some seem a bit uncomfortable with it when you are asking them questions to nudge them to dig deeper or be more specific with details; we sense many are also not used to these types of conferences that puts the responsibility and decision making on them as students.   We are framing this conference/coaching sessions from a stance of discussions to help them think through their choices, to clarify their own thinking/choices/next steps, and to move forward with their projects since we don’t want them to see the messiness and muckiness of inquiry as punitive.   These sessions have also helped us identify those who might benefit from some of our upcoming small group mini-lessons but who may not have initially signed up for assistance.  Last but not least, I believe these conferences convey to our students that each person matters and that we care about them and their topics.

Reflections

While we cannot do their work for them, we can give students every opportunity to get personal assistance in a low-key setting —we want them to know they cannot fall through the cracks or simply fly under our radars.  While I’ve done this sort of work before, this is probably the biggest chunk of time I’ve had in a collaborative partnership for this level of assessment and 1:1 student conferencing.  This approach requires us to be agile and responsive as each day is different and every student need varies.  This kind of conferencing/coaching is time consuming and messy; while the prep for the small group work is pretty straightforward, the 1:1 help is definitely open-ended.  I have been inspired and am improving my own conferencing skills with students by watching Sarah (who is a master at this process) and by my friend and fellow school librarian Heather Hersey.  Her post about the importance of conferencing helped me to think about focusing on all aspects of their inquiry work and design proposals rather than just sources or their bibliographies; it also inspired my idea for using the mail merge form and database to capture feedback and use that as a starting point for the student conferences/coaching sessions.

Sarah and I have also been discussing how intense this kind of work is and how you have to be comfortable with making adjustments as needed to timelines and your plans in order to be responsive to the students.    The processes are messy, yet this “mucking around in ideas” is the grist for the growth and critical thinking that happens as both we and our students problem solve, question, and revise our ideas and stances.  Neither of us has any idea how someone would do this kind of process-driven, organic, fluid, and reflective work alone!  We love that our combined talents help the students as well as each other; we also are appreciative of having someone else each day who can help you see things you might have missed or to think about a particular situation or challenge with fresh eyes.   We are also excited we can model collaborative learning for our students—how often do they get to be in a learning environment where there are at least 2-3 adults who can help them and provide them the kind of specific and personalized attention they are receiving?  Most importantly, this type of collaboration is a catalyst for inquiry work and for integrating more formative kinds of assessments that benefit students and impact learning.

We expect the 1:1 and small group conferences, coaching, and small group instruction to continue the next 7 days of school leading up to our Thanksgiving break.  I hope to share more images, video, written/video reflections and feedback from both of us as well as our students in an upcoming post later this month.   I’m also thinking about how to better integrate the conferencing/coaching/conversation aspect into the inquiry approach (and at an earlier point in time) with research using Cris Tovani’s conceptualization of these conversations as data and formative assessment (see her text, So What Do They Really Know?  Assessment That Informs Teaching and Learning).

Cris Tovani's Conversation Calendars

How are you approaching assessment with inquiry work?  How do you negotiate and embrace the challenges of time and fluidity with this approach to learning and research?  How do you scale this kind of learning experience when there are always challenges of time, space, and staffing?

Moving from Our Mindmaps to More Focused Topics with Question Lenses and Musical Peer Review

music activity-collage

In my last post that is part of this series, I shared how we used mindmapping after our second round of pre-searching to begin honing in our a more specific topic for our research.

After students shared out their mindmaps and big ideas, we asked them to look at their topic through different question “lenses” using an activity shared with me by my colleague Heather Hersey, a school librarian in Seattle.  Heather, who adapted her version of the handout from An Educator’s Guide to Information Literacy: What Every High School Senior Needs to Know (Ann Marlow Riedling, 2009), graciously shared her template with us, and we used it to help students look at their topic from multiple perspectives by “trying on” a question lens (see below).

Large Group Share of Mindmaps and Topic Triangles

This exercise was a stretch for the students as it forced them to look at their topic from an angle they might not have considered; we asked them to see if they could generate a question for at least 2-3 of the question lenses.  For some students, this was a helpful endeavor as it helped them in their thinking process about narrowing a focus even more for their topic; other students, though, were a little overwhelmed by the task and/or the cognitive dissonance that came along with the activity.   I love the activity because it is yet another way to encourage wonder, but in retrospect, I think it might have been more beneficial had we incorporated it sooner.   After conferring with Heather via email, I realized I missed that she utilizes it earlier in the pre-search process; I will consider following her lead on the timing in the future although I want to get student feedback on this aspect before making a decision for future efforts.

Once students had some time to muck around with this exercise, we asked them to then see if they could arrive at a more focused topic and compose a research topic statement or focused essential research question—we wanted to give them the option to write about their more focused in either format.  This statement/question writing was the springboard to an activity, “Musical Questions:  Broadening and Narrowing our EQs,” that we did on Friday (October 24) that we adapted from Marci Zane, Heather Hersey, Meg Donhauser, and Cathy Stutzman.   Sarah Rust came up with her own brilliant take on this activity:

1.  We arranged our tables in a conference style arrangement so that students could move about the “square” easily.

Musical Research Topic

2.  At the beginning of both classes, Sarah instructed the students to take out their document with the focused research topic statement or question they had composed earlier in the week.

Musical Research Topic

Musical Research Topic

3.  We explained that there would be four rounds of peer review; during each review students could provide four types of feedback:

  • a suggestion to broaden the topic statement/research question if needed
  • a suggestion to narrow the topic statement/research question
  • pose a question to nudge the peer’s thinking
  • share a constructive piece of feedback to help the peer with fine tuning that statement or question.

We asked that they put their names next to their feedback in case the owner of the statement/question needed to confer with them later about their statements or questions they had shared.

4.  We then explained that when there was no music, they would get up and begin walking around the table until music began playing (we let the students choose the music, and interestingly enough, they chose 80s and 90s tunes!  Their input made the activity more fun and gave them a sense of ownership).  Once the music started, they were to sit down in the closest chair and document and begin writing their feedback to the peer.   We gave them roughly 3-4 minutes to provide the feedback before stopping the music and starting our rotation again.

Musical Research Topic

Musical Research Topic

We honestly were not sure what to expect with the activity, so we were delighted by the positive response from our students.  We were also joined during our first class by assistant principal Christine Dailey who was there not only as an observer, but also as a participant—her jumping in and working side by side with the students was fantastic and of course, great modeling for our kids.  After the four rounds, we asked students to return back to the original seat and work and read over the feedback provided.  They then had an opportunity to pair-share with the person sitting next to them and to reflect on next steps or ideas that came from the four rounds of feedback from their peers.

Musical Research Topic

From what we observed, many students found this activity helpful in getting concrete suggestions for being more specific with the wording and in some cases, the scope, of their topic statements/research questions. We definitely would do this activity again as it gave students the chance to collaborate and work together  in a meaningful way as we continued to work through our research/inquiry processes! We would like to thank our students for their efforts, Christine Dailey for her time and feedback, and Heather, Meg, Marci, and Cathy for so generously sharing their ideas and experiences with us to create learning experiences that have helped us all as learners.

In addition, a big thanks to our New Jersey/Washington state friends for reminding us that the affective aspects of inquiry that our students are cycling through—confusion/frustration/doubt and clarity—are normal and that activities like these help mitigate some of the emotional lows or challenges our students are feeling, especially as the inquiry approach pushes them out of their comfort zones.  Even when you’ve experienced this process with students in the past, it is always reassuring to hear from others their approach to honoring that uncertainty and helping nudge students forward when they get stuck.  This same post has also given us some helpful guidance as we’ve wrestled with issues of grading and assessment of process-oriented, formative work.

In my next post, I’ll share how we are moving through the research design proposal process for the multigenre projects that students will be crafting.  The next couple of weeks will be interesting as students complete their proposals and move recursively between investigating, constructing, and expressing.   Until then, we’d love to hear your ideas and strategies for helping students narrow their topic statements and questions after pre-search–what have you tried that has been successful?  We’d love to hear your suggestions!

 

Pre-Searching Round Two: Moving Toward a More Specific Topic for Research and Inquiry

presearch round 2students

After introducing students to some basics of information evaluation, we began our second phase of pre-searching on Monday, October 6.   Our learning targets included (based on our district content area standards and the AASL Standards for the 21st Century Learner):

  • I can use prior background knowledge as context for new learning.
  • I can find, evaluate, and select appropriate sources to answer questions.
  • I can read widely and fluently to make connections with self, the world, and previous reading.
  • I can use my library time wisely to think deeply on my work and stay on task.

We began the conversation by discussing how this next-round of pre-search was going to be a little more strategic and structured since our first phase had given us a topic commitment and now it was time to start “cropping” the big picture to narrow our topic (hat tip to Pegasus Librarian for this wonderful metaphor and to my friend Kristin Fontichiaro for pointing me to it).

We then introduced our structures , steps, and resources for helping us go more deeply into our pre-search to help us read and reflect more intentionally while evaluating our information sources.

pre-search round 2 directions

 

We required students to print or create a hard copy of any information sources they were using so that they could highlight and annotate the text.  We then took time to discuss strategies for annotating informational text and how annotations help us think more deeply and purposefully about a text.  We drew heavily from reading and literacy expert Cris Tovani to create this handy “help” sheet on annotating texts for our students:

Annotating Text Strategies

We then shared with the students how the text annotations would be the bridge to our modified KWL for pre-search and how this reflective thinking, while time intensive for the present, would be essential and instrumental to building our existing knowledge of the topic so that we could hone in on a more specific focus.

On the backside of the hard copy of this chart was the information source evaluation checklist we had worked with the previous week in our research/inquiry circles.   We explained how we would use the CRAAP test and our assessment tool to evaluate the information source.  Once students had read and annotated an article, completed a KWL for that article, and completed the information evaluation assessment tool for that article, we asked them to staple that together as a “packet” and then add the information source to their EasyBib working bibliography.  We ended with a short EasyBib refresher and pointed students to specific tutorial videos we’ve created for a variety of resources.  

tempcheckWe then turned the students loose, and they began immersing themselves in the work.  Over the next few days, the primary role for Sarah, Jennifer, and me as instructors was to facilitate; most of our efforts were spent answering 1:1 questions and individual conferencing to help students keep moving forward or adjust their searching.   After doing a “temperature check” on Friday, October 10, we realized students needed one more additional day for searching, reading, annotating, and doing their metacognitive work with the KWL and information evaluation tool. This was an opportunity for students to wrap up their work while others took advantage of the extra day to get some additional  intensive and extended 1:1 help—most requests were related to search terms and techniques.  For these students, the personalized help was beneficial in moving them from a place where they felt stuck to discovering new sources.

The content in these pre-search “packets” will be the fodder for helping us move forward with the next step in narrowing topics:  mindmapping. We formally started this process of mindmapping today, and I’ll be writing more about that soon as well as the assessments and self-assessments we’re designing to think about where we are in our learning before moving forward into our next phase of inquiry!

Follow our journey:

Hashtag:  #rustyq

Our LibGuide

Blog post 1:  Inquiring with Students: What Do or Can “Good” Research Projects Look Like?

Blog post 2:  Beginning Our Research and Inquiry Experiences with Pre-Searching

Blog post 3:  Sticky Notes as Formative Assessment for Information Literacy Instruction: Coding Student Responses

Blog post 4:  Collaborative Information Source Evaluation: Research/Inquiry Circles and the CRAAP Test

Collaborative Information Source Evaluation: Research/Inquiry Circles and the CRAAP Test

info evaluation activity collage

Last week, students completed the gentle entry-level phase of pre-search (see the end of this post for more detailed reflections); teacher Sarah Rust and I felt it would be helpful to introduce information and source evaluation skills to our students before moving forward into the next round of pre-searching.    We grouped students into collaborative “research” or “inquiry” circles based on their initial topic interests.  We plan to use these research circles as a medium for workshopping with small groups as we move deeper into research and inquiry; these groups will also help us move into collaborative learning experiences.

On Thursday, October 2, we grouped students and then introduced them to the CRAAP test with this terrific video from the Academy of Art University; while this structure for evaluating information originally was designed for online resources, we discussed how it was important to evaluate ALL forms of information, including ones traditionally considered authoritative.    We talked about the messiness of information evaluation and context  of authority using the framework of the CRRAP method. Using the recent Secret Service security breaches as our research topic, students then were asked collaboratively look at seven different information sources we posted on our project LibGuide and to work together to evaluate each information source using the CRAAP test as their guide.   We asked them to use this checklist to guide their assessment and to tally their scores for each source.  Students worked together all period and for about the first quarter of class on Friday, October 3.

After students finished up their assessments on Friday, we instructed each group then posted their score on a dry-erase board on our Verb easel; we labeled each whiteboard with a sticker for the source so that the “parking lots” for their scores would be easy to post.

Each group then came up to the easel and shared/defended their assessments of each source.

As they did this, I took rough notes about how each group scored sources and notes of any comments or reasoning they shared.  You can see my notes below:

Sarah, Jennifer, and I were fascinated by the students’ responses.  Just a few things some students/groups noticed:

  • Databases may be great, but if they are only providing background information and not answering one’s research question, the content there may not always be the best fit.  We were impressed they made this distinction.
  • One group commented that they would like to know if the journalists for the Washington Post article had previously written about problems with the Secret Service security issues or if this was their first effort on writing about that topic.  Again, we were happily surprised they were this discerning in their evaluation.
  • Several groups noted that just because a source was government publication, it was not necessarily credible since they might be interested in putting a certain spin on the value and integrity of the Secret Service; this level of questioning could be a reflection of previous instruction elsewhere that values interrogating all sources, but we also wondered if that stance might also be a reflection (at least, in part) of the politically conservative nature of the community.
  • Discussions emerged about different news publications and outlets and how their reputation to lean left or right might impact the objectivity of the articles or news videos.
  • Several students indicated they would like GALE to include more information about the authors of reference articles in databases like Opposing Viewpoints in Context.
  • Scores were pretty consistent from group to group within specific class periods and across both class sections.

We were incredibly happy with the way students engaged with each other and the assessment task as groups.  Our goal was for them to have an opportunity to debate and wrestle with their evaluation of each source within their groups and to share that thinking out loud with the larger class; this approach accomplished that outcome.  I definitely would introduce information evaluation in this way again, and this springboard activity seemed to fit a wide range of prior experiences with these concepts.  As we’ve engaged in pre-search “phase 2″ this week, we’ve incorporated this CRAAP framework into their metagcognitive learning activites.  I’ll share more about those processes in a new blog post next week.

Follow our journey:

Hashtag:  #rustyq

Our LibGuide

Blog post 1:  Inquiring with Students: What Do or Can “Good” Research Projects Look Like?

Blog post 2:  Beginning Our Research and Inquiry Experiences with Pre-Searching

Blog post 3:  Sticky Notes as Formative Assessment for Information Literacy Instruction: Coding Student Responses

Conversation 3: Student Reflections on Inquiry, Choice, Participatory Learning, Information, and Digital Literacy

Last week, we held a large group share/think/brain dump/reflect session with our Media 21 students over a series of four days after students completed initial written self-assessment and summative reflections.  This video is the first of a series of conversations in which students share their summative reflections about their experiences in a collaboratively taught English course by Susan Lester, English teacher, and Buffy Hamilton, school librarian in 2011-2012.    I’d like to thank our students for their willingness and permission to share with a global audience as well as their participation in these conversations.   While these are lengthy conversations, I hope the thoughts and insights they share will be helpful to other teachers, librarians, students, administrators, and community members in thinking about the possibilities of learning and libraries and the potential of the collaborative partnerships we can forge.  I’ll be following up this series of video conversations with a written post highlighting the insights, reflections, and self-assessments shared by our students.

In this discussion, Ella and Cynda discuss information literacy standards they’ve mastered, how participatory learning has built their confidence as students, and the decisions behind their multigenre, transmedia learning products.   You can see Ella and Cynda’s work by clicking  here.

Resources: