Southeastern Library Assessment Conference 2015 Session Roundup: Correlation Between Library Instruction and Student Retention

CC image from
CC image from

Note:  This is the second in a series of posts on sessions I attended at the Southeastern Library Assessment Conference on November 17.  Please see the first post with more background information here.

Session Description

Grand Valley State University Libraries has found a statistically significant positive correlation between librarian-led instruction and student retention. The libraries’ Head of Instructional Services and the university’s Institutional Analysis department worked together to connect library instruction with student-level data and then analyze that data over several academic years. This presentation will outline the data collection process, analysis methods, limitations, and outcomes of a longitudinal study.

Session Notes

The most interesting session we attended  was “Correlation Between Library Instruction and Student Retention” from Mary O’Kelly, Librarian and Head of Instructional Services at Grand Valley State University.  Here are my notes and reflections from one of the best, most interesting, and most thoughtful conference sessions I have ever attended during my career as a librarian.

O’Kelly became the Head of Instructional Services in 2012 and very quickly began thinking about assessment planning.  Major questions she wanted to pursue included:

  • What questions do we have about our instructional program?
  • What data do we need to those questions?
  • How will we get it?
  • Who will analyze and interpret the results?

She also determined that she would need help gathering reliable data and expert help with the data analysis.    She conducted literature reviews in these broad areas:

  1.  Relationships between library services and student retention
  2.   High-impact educational practices
  3.  Practices to impact student retention in positive way

O’Kelly was particularly interested in retention as part of her research because:

  • Retention is a significant issue in higher education.
  • Retention is a priority at her institution and in higher education at large.
  • She found nothing in the literature about information literacy instruction and student retention (though there were other kinds of studies on correlation between non-instructional aspects of library service and correlation).
  • She felt it was a top priority for the library to demonstrate its values and impact on larger institutional goals.
  • She wanted to see if the data would support library instruction and collaboration between faculty and librarians as a high impact practice.

Research Methods

O’Kelly and her staff used LibAnalytics from Springshare to collect library instruction data.   For each guide used with library instruction, staff entered:
A.  the course code, course number, and section in separate fields
B.  The professor name and librarian name
C.  Other:  date, location, duration, content
D.  Attendance was not taken; absences were within a margin of error

This research endeavor presented an opportunity to establish a new relationship with the university’s Office of Institutional Analysis.  O’Kelly worked closely with Rachael Passarelli, the office’s research analyst.  Together they began with 23 questions about the library’s instructional program.  Initially, they used some of these questions to adjust instruction and to think about important data points for instruction:

  • Which programs were not being reached?
  • How many students were reached at each grade level?
  • What percentage of instruction sessions were in the library?
  • What is the distribution of the number of sessions over the course of an academic year?

After developing the of working list of 23 questions and conducting her literature review,  the one that bubbled to the top was focused on the relationship between information literacy instruction and student retention.This initial pass at the data led them to the big research question:

Of the students who saw a librarian in class, what percentage of them re-enrolled for the following fall compared to students who did not see a librarian? 

The null hypothesis:  there is no relationship between library instruction and student retention.  Retention was defined as re-enrollment the following fall semester.

After the data was collected over the course of an academic year, the LibAnalytics dataset and questions were sent to Passarelli and the Office of Institutional Analysis.  Passarelli pulled student records from course enrollment in the LibAnalytics data as part of her analysis.  Only courses with at least one library session were analyzed; she also used the chi-square test of independence using SAS.  A fixed p-value of .05 was used to test significance; a general linear model was used to control for ACT scores, high school GPA, socioecnomic status, and first generation status.  The research project was IRB exempt since privileged student data was stripped before the data analysis was sent back to the library.


The results over three years of data collection showed the following:

data results

As you can see, the findings were replicable over three years and statistically significant.  In addition, the magnitude increased each year.   Consequently, the null hypothesis has been rejected.   Data also shows the highest retention correlation with freshmen and graduate students.  In order to triangulate data and rule out faculty effect, a recent follow-up analysis that compared the retention of students whose faculty had a librarian come into any of their classes compared to faculty who did not (analysis was by faculty, not by student).  This follow-up analysis also showed a significant correlation, p-value=.0001.


  • LibAnalytics record input is subject to human error
  • Attendance is estimated
  • Online instruction is excluded
  • Results cannot be generalized to other institutions
  • Retention is not persistence to graduation
  • Reasons why students withdraw are often complicated
  • Correlation is not causation (my note:  I am deeply appreciative of this distinction since so many library studies of recent years crow as though their results are causation when in fact, they are not.)

Discussion/Next Steps

One of the limitations of library use studies is the student motivation factor.  For O’Kelly’s study intrinsic motivation for library engagement is removed because whole-class data was used.  In addition, the large sample size is a strength of this study.  O’Kelly wants to further explore why students are using the library and to consider the role of course assignments (that are given by faculty) in library use.  At this time, the library instruction is very targeted because it is based on the school core curriculum, not the ACRL standards/framework.

Because faculty are including a librarian in assignment design and delivery, they are introducing the library as an academic support service to students.  In light of her research showing faculty engagement with instruction librarians is correlated with student retention and student participation in library instruction is correlated with student retention, O’Kelly now wonders, “What’s the cause?”  She now wants to test this working hypothesis:

Faculty engagement with library instruction is a high impact practice that positively affects student retention.

O’Kelly will be publishing her research in early 2016; I will be share to alert you when her formal study is published.  For now, you can see her slides here.  I was most impressed by the depth of thought and how she tried to cover every angle possible with her research methodology.  As I stated earlier, I also appreciate that she is stresses her research shows correlation, not cause, a distinction I think is often lost in library studies in people’s efforts to advance advocacy agendas.  The other attendees were also clearly impressed with her research methodology, attention to detail, and the clear and deliberate way she communicated her work.    The session left me thinking about how her efforts might inspire my own research as a high school librarian and what data points matter to my local school and learning community.  I hope to write more about where this may lead or go in the spring of 2016.

Southeastern Library Assessment Conference 2015: Introduction and Space Assessment Session 1

library asssessment conf

My friend and former Norcross High colleague Jennifer Lund and I attended the Southeastern Library Assessment Conference on November 16 that was held at the historic Georgian Terrace Hotel in Atlanta, Georgia.  Though we were probably the only school librarians there, we felt welcome and gleaned many pearls of wisdom from the sessions we attended.  I was sadly only able to attend Day 1 (Monday, 11/16) due to district meetings I needed to attend on the second day (11/17), but I got MORE than my money’s worth from the sessions I attended.  I highly recommend this conference if you are looking for smart, thoughtful perspectives that are grounded in evidence based practice and data collection with integrity.  The conference was limited to 125 people and had a pleasant, intimate feel; in addition, we were served a gourmet lunch buffet (it was fabulous) and many delicious amenities throughout the day (Starbucks coffee, tea, water, sodas, cookies).  Many thanks to the conference organizers who did a fantastic job with every aspect of the conference—it is by far one of the best and most meaningful conference experiences I’ve had in my career—every session had substance.

This is the first in a series of posts on the sessions Jennifer and I attended on Monday, November 16, 2015.

Space Assessment: How They Use It, What They Want, Sara DeWaay, University of North Carolina, Charlotte

Session Description:  Getting student input on the library space can be a multi-layered effort. Come hear about the methods used to get an understanding of use patterns, as well as the students’ desires for a small branch library, as we work to transition towards a flexible space.

My Notes:
The emphasis was on users and feedback from students; Sara thought about the feedback in terms of “low cost easy” vs. “high cost hard” solutions and ideas from the students.  When she began the group study, she thought of the library space in zones:  group study, circulation area, lounge, quiet study, flexible, and creativity.  She began by doing a literature review on space assessment, and she focused on both qualitative and quantitative assessment methods.  She also looked at space assessment from a “before” and “afterwards” perspective since assessment should continue after the space remodel or redesign is initially completed.  She also did research on user centered design.  She formed a Student Advisory group; positive aspects of this group included input, support, connection, and ownership for the students, but challenges were maintaining momentum and a sustained sense of meaningfulness for the students after their participation ended.  In the future, Sara would try to make sure students received some sort of course credit for participation, perhaps as part of a project based learning assignment related to space design.

She organized a student event where students could come and vote on designs; approximately 40-50 students participated.  She basically used big notepads where students could vote with sticky notes on larger sheets of bulletin board or flip chart paper housed on easels.  For example:


She also used flip charts to get feedback from students using open-ended questions; she interspersed the flip charts with the buffet of food to “guide” them to this part of the feedback session.    Students also had a chance to mark up floor plans; she provided them a variety of tools for doing this activity including crayons, sharpies, ballpoint pens, colored pencils, and regular pencils.  Students then could tape their proposed floor plan on the wall.  Afterwards, she coded the feedback from the student floor plans using categories like “atmosphere” (and specific elements assigned something like letters A-J) and “physical space” (specific aspects were numbered 1-14).  This method of floor plan coding then allowed her to look at the data in a “layered” way (example:  2B).

Another strategy was student surveys.  Unfortunately, her sample size of 40 was not ideal, but nonetheless, she was able to ask more detailed questions about services as well as questions about the library in comparison to other spaces in the building.  She also had library student assistants help track space use; using iPads and Suma, they were able to gather data and plug it into LibAnalytics to get a better idea of space usage.

Once she looked at all the data, she was able to better understand student needs and could classify possible changes and redesign elements into these categories:

  • Low cost/easy to do
  • Low-cost/difficult to do
  • High cost/easy to do
  • High cost/ difficult to do

Unfortunately, the budget for the renovation was put on hold, but if it moves forward, Sara would get faculty input in the future and do similar activities with staff.  The major takeaway for me from this session was the idea of space assessment as cyclical—it should be ongoing and is important to do even after you complete a renovation or redesign project to make sure the new space is continuing to work for students or to see what areas of new need/adjustment may be needed.  This idea was especially helpful for Jennifer and me since she has opened a new library space, and I’m in the middle of working on a redesign project for the library here at Chattahoochee High.

My next post will be about the second session we attended on battling survey fatigue.

Researchers as Artists, Artists as Researchers: Tinkering, Messiness, and Meaning Making in Libraries as Learning Studios


Last week, I sent out a needs assessment to our faculty.  Initially, I was concerned it was too lengthy, but as a new media specialist here at Chattahoochee High, I feel a sense of urgency to get some idea of what teachers have done in the past, what they are interested in now, and their points of need.  In spite of some lingering reservations, I shared the assessment with our faulty via email.  The next morning, AP Studio Art teacher Dorsey Sammataro came by to see me because she was intrigued by information literacy concepts embedded in the survey.  Long story short, the survey opened a really exciting conversation between us about certain concepts and skills she saw on the survey and how it dovetailed with the needs for a new unit she is piloting related to 2D Design Service Learning and Natural and Human-Made Environments.  Students have started thinking about topics of importance to them but need help growing strategies for search, developing search vocabulary, and becoming more comfortable with web-based resources as well as databases that students can mine to find inspiration for ideas and issues that can then inspire their art.  Ms. Sammataro identified this working list of issues and topics of importance to her students in the course:

  • Issues of socioeconomic equity (rich, poor, middle class)
  • GMOs/Food
  • Education and Equity
  • Human trafficking
  • Environment
  • Cultural appropriation: identifying its effects in everyday life and raising awareness of it
  • Assimilating into a culture:  how, why, impact on those assimilate—what is gained, what is lost
  • Adolescent mental health issues
  • Body image
  • Emotional health
  • A sense of unity and connection to peoples and cultures in other parts of the world
  • Stereotypes and assumptions people make about specific ethnicities
  • Bullying
  • Abandonment of self because of depression/mental illness as well as abandoned communities and/or groups of people

When she said they would be having a group debrief about the first work of art they had created that had come out of their initial pass at these topics, I asked if I could come listen in, and she enthusiastically said yes!  I was able to join them and listen to most of the 50 minute small group discussion as they talked about:

  • expanded insights about their topic ideas—this aspect of the discussion was quite meaty/weighty as students drew from personal experiences.
  • what they had learned about their idea through their initial research and first efforts at crafting 2D art around it.
  • what community resources (people, groups) might be resources for our work and ideas.
  • how and why one might abandon a topic and how the process of making art around a topic may help you realize that topic is not your true passion.
  • one student shared she had discovered she needs a strong intention for figurative pieces, so the idea/topic of interest is particularly crucial for art making.
  • an extended conversation about the importance of time, space, and ownership of experimentation for both literal and experimental/abstract pieces (echoes of Nancie Atwell’s concept of what writers need); the importance of trying new things, art forms.  In the words of one student, “Don’t be afraid to stray from the path of success.”
  • some students discovered they liked new art forms they didn’t think they would like.
  • one student shared how she was excited about the idea that inspired her art but when it came to do the printing process it was very humbling because it was more difficult than she imagined and the piece didn’t turn out quite as she envisioned, yet this trial and error process was important and valuable to her.
  • some discovered it was more difficult than they anticipated to turn an idea/topic/issue into an art piece.
  • one student shared how important it is to find out what you really are passionate about and then wondered how to better go about mining it to yield more strategic ideas/subtopics or focal points for expression of that through art.

I was struck by how deeply invested the students were in these topics and the group discussion; I was also appreciative of their honesty and openness, something that is not easy to do among peers or with a new adult on the staff who is listening to what they have to say.    Their perspectives on these topics as well as their insights on art making processes had a depth I had not anticipated; it also got me thinking about the parallels between making meaning from art and making meaning from working with information (and some form of research whether formal, informal, or some hybrid).   A few wonderings I’m now contemplating:

  • How do the two (research and art) inform each other, and how might looking at art-making processes foreground our conceptualization of “research”?  I can’t help but wonder if some of the precepts of Dennis Sumara’s work with “literary anthropology” in studying reading literacies might be applied when we think of art, the learning environment of a studio, and research intersect as a site of “information literacy interpretation.”
  • How might a library function as a studio where meaning making is elevated across multiple forms of literacy, particularly information literacy processes?  How is research art?  How might research and the cultivation of information literacy skills in art students impact their art-making processes?  What insights from an art studio might we draw upon in designing a library as a learning studio, and what does research look like in this environment?  How will it translate to learning spaces then outside the library and impact a larger learning community and culture where research seems increasingly marginalized in K-12 public schools by the impact of standardized testing?  What tools, resources, experiences, and learning design drivers do artists and learners need in a research/library learning studio as well as an art studio?
  • How is the act of crafting art like acts of crafting research processes and products?
  • Research and art can both be organic, recursive, and frequently non-linear (even though there are those who would like to prescribe models that are contradictory in nature).    Many K-12 teachers, professors, and yes, even some librarians tend to emphasize the consumption aspect of research rather than frontloading the grittier messy work of mucking about in information; students often miss the experience of wrestling with the friction of ideas that comes when one goes beyond regurgitating facts and engages in higher level thinking; it is often the final product, a paper, that gets the most emphasis.  Yet this creative process is viewed positively when it comes to crafting art—-how might it be viewed if we embrace meaning making as the core of research as it is in art?

Building on the extensive work and efforts Jennifer Lund and I invested in developing the concept of library as learning studio at Norcross High (see any of my posts from the last two years), this budding collaborative partnership with the AP Studio Art students and Ms. Sammataro (and my larger/big picture efforts to now develop the library as learning studio concept at Chattahoochee High) may offer opportunities for us to explore these wonderings together by working from an inquiry stance.   I hope to dwell in these ideas and look forward to see how my thinking is shaped by my experiences with Ms. Sammataro and her students.

Mindmapping Our Presearch Notes: Seeing Patterns and Gaps


For the last month or so, I’ve been working with a section of Honors 9th Language Arts (hopefully, another more comprehensive post coming on this endeavor later in the spring).   After completing a class study of To Kill a Mockingbird, the students selected a motif of choice and began presearching a topic of choice related to the motif.   After completing a presearch search term map and arriving at a narrowed topic (which I’ve blogged about earlier this semester), we moved forward with another and more focused round of presearch while using EasyBib to capture information sources and take notes.   After approximately two and a half weeks, most students had a body of notes on their focused topic.  However, after many 1:1 student conferences and a formative assessment of collecting and reading their notes, the teacher and I realized many students were struggling with:

1.  Recording relevant information from their information sources.

2.  Taking notes in “bite-sized” portions.

3.  Being discerning about information that would help them go beyond merely reporting and instead, help them dig into the higher level thinking and questions that we wanted to anchor their inquiry.

After addressing some of these challenges with a mini-lesson and small group or individual conferences, we felt the students needed a more concrete way of discovering the patterns of information as well as the gaps in their notes. We asked students to print out their e-notes from EasyBib and gave them supplies (Sharpies, markers, large/oversized  sticky notes) to help them map out the information they had collected up to that point in their notes.  We discussed some strategies for identifying major topics and subtopics as well a sample mindmap of notes. Students were assigned a working area with a partner so that they had a “research buddy” to help them think through their process as needed and worked on their maps about a day and a half.

We wanted the class to have an opportunity to look at their peers’ maps and provide feedback; we knew our students would need a little scaffolding to provide meaningful peer feedback, so we took a few minutes to review their peer review activity guidelines with them.  We spread out the maps and placed pads of lined sticky notes at each table of notes mindmaps.  We asked students to write their feedback on these sticky notes and to include their name, their feedback, and the feedback category number (see the handout embedded below).  We discussed ideas for meaningful feedback (including a list of idea/conversation starters) that were numbered so that they could include the “feedback category” ID number on the sticky note as an extra layer of clarification.

As students walked around giving feedback, we also instructed them to keep a running record of the maps they were reviewing and quick notes about what they were seeing (see page 2 of the document embedded above).







We initially thought the students could complete the activity in 12-15 minutes, but in spite of our best efforts to be proactive, we had about a quarter of the class that had difficulty staying on task, participating, or making a legitimate effort to provide meaningful feedback.  Because our studio space had been reserved for another activity the following day, we had to adjust our plans to complete the activity in the teacher’s classroom, a space that really was not conducive to the activity or an ideal learning environment for this kind of activity.  However, we had no choice, so we had to adjust as best we could.   I hung some of the maps on whatever wall space I could find; for the rest of the maps, I had students place desks together in pairs and utilized that surface space for the remaining maps.  We thought they could complete feedback within another ten minutes, but some of the very same behavior issues that plagued us the day before were again problematic even after we enlisted the assistance of an assistant principal to conference with some of the students outside of class as part of our efforts to address the previous day’s issues.  However, we stayed the course and tried to redirect students as needed so that we could get as much helpful peer review for everyone as possible.



Once we brought the peer review to a close, students paired up once again with their research buddy and used the Making Thinking Visible learning structure of Compass Points to help students reflect on the mindmapping process and peer review activity.  Each pair received a graphic organizer to complete their ideas they were to share and discuss.


After having 5-7 minutes to discuss and record their observations and ideas, we asked each pair to do a quick share of their notes with the large group.   Students noted effective map organization strategies as well as what constituted “good” or quality information from the notes in the maps.  However, many students noticed that quite a few maps lacked depth of information; others noted that better organization was needed in structuring topics and subtopics. In spite of some of the challenges we encountered, we felt most students truly benefitted from the mapping activity itself as well as the peer review.

When we returned from spring break, we returned maps to the students along with the sticky note feedback others gave from the peer review activity.  We then asked students to think about what they had in their notes and maps that was helpful and what was missing.  Students then had the class period to complete two thinking/reflection exercises:

1.  The question lenses activity that I borrowed last semester from my friend Heather Hersey (and blogged about; also see Sarah Ludwig’s awesome adaptation of this thinking exercise–I would have totally done her version if we had more time in our schedule for this project).  We framed this thinking exercise as a way of addressing gaps or “struggle areas” of their mindmaps and as a means of thinking about next steps for our new round of additional research for the week.


2.  They then had time to complete  mindmapping reflection questions via a Google Survey embedded in the project LibGuide.

We collected the hard copies of the question lens activity; I downloaded the responses from the Google Sheet as an Excel spreadsheet and then ran a mail merge so that the teacher and I both had easy to read hard copies of the student responses.    Common feedback ran along the lines of these statements:

  • The mindmapping process helped me better organize my ideas.
  • The mindmapping process helped me realize I didn’t have enough information about my topic.
  • The mindmapping process gave me new perspectives I had never considered.
  • The peer feedback helped me rethink one of my subections and a new direction for research.
  • Organizing my topics and subtopics was harder than I realized.
  • The mindmap challenged me to make sense of my notes (more intentional thought as opposed to just randomly taking notes).
  • I realized some of the information I had taken notes on really did not fit with my narrowed topic focus.
  • Mindmapping helped me visualize how my pieces of information fit and relate to each other.
  • Writing in short phrases or brief key ideas was challenging for me; I wanted to copy my notes as they were in complete sentences.
  • Mindmapping was challenging because categorizing my ideas and information into subtopics was difficult.
  • The mindmapping process helped me to see I need to slightly change my topic focus from A to B.
  • I realized the notes I have are lacking in meaningful detail.
  • The mindmapping process has allowed me to see/find a deeper personal connection to my overall topic.
  • The mindmapping process has helped me to see I need to go deeper with my information and further develop the topic.
  • The mindmapping processes helped me better see the strengths and weaknesses of my research and better refine my subtopics.
  • The peer feedback helped me to see I need to regroup my ideas on my map.
  • The question lenses activity helped me to look at my project from a different point of view and to rethink what information I should now focus on gathering.

I’m happy the mindmapping activity and peer review provided students the opportunity to wrestle with their notes and the information they had gathered during our first round of presearch.  My hope was that the process would nudge their thinking because it was clear within the first few days of working with the students that they were used to reporting information as opposed to researching, a distinction my colleagues who blog at Letting Go have made in previous posts.  I know that for some students, the uncertainty and our pushing them to think more deeply beyond shallow, surface level work has been uncomfortable, but we have tried to give them as much support as we can to help them develop new strategies and resilience in this inquiry focused project.  I think it is especially important with freshmen to provide and scaffold these kinds of learning experiences, particularly if they have had few or no opportunities to develop these kinds of information literacy skills and processes.

Their teacher and I are proud nearly every student has either had the confidence and persistence to move forward this past week wherever they have been on the spectrum of the quality and depth of their work.  Several have regrouped and have been digging in to act on the next steps they identified from their insights and reflections on their work.   It takes grit on the part of students, teachers, and librarians to grapple with these kinds of challenges, but it is so gratifying to see the individual growth and forward momentum for each student.  They are now starting to sketch out their multigenre products, and we’ll be moving forward with creating those artifacts and the supporting notes narratives/compositions.