ARLD Instructioanl Dialogue Notes

Assessment: Where We Are and Where We Need to Go
Notes for a Dialogue sponsored by the Academic and Research Libraries Division (ARLD) of the Minnesota Library Association
Minnesota State University, Mankato, December 9th, 2002

Background

Assessment is a contested issue in higher education. The North Central Association (recently renamed the Higher Learning Commission) like other accreditation bodies, has emphasized developing a "culture of evidence" on campuses that emphasizes measuring student learning outcomes and using the results of that assessment in planning and in resource allocation. To some critics, this emphasis on developing such a culture is a distraction from teaching and a bureaucratization of academic work. The word "evidence" implies an adversarial climate that places the burden of proof on the professoriate - and the old ways of assessing learning don't count. Indeed, the language of some NCA documents suggests showing evidence that an institution gathers evidence is more important than what the evidence shows.

Libraries, at the same time, are growing increasingly intersted in asking not just "how many?" or "how much?" but "so what?" The motivation for this change in focus is not so much driven by accreditation pressures as by a genuine interest in how well the library supports teaching and learning. The new Standards for College Libraries adopted by the ACRL in January 2000 illustrate this shift by reframing standards that graded libraries on the basis of quantifiable (though somewhat arbitrary) inputs and outputs into standards that include selected inputs, outputs, and outcomes. Similarly, the Information Literacy Competency Standards for Higher Education also adopted by the ACRL in January 2000, emphasizes not what libraries provide but rather on student learning outcomes. The SAILS project at Kent State University is one atttempt to create a standardized and shareable test that will yield national benchmark data on information literacy, thanks to an IMLS grant. Other measures might be less concrete and more quantitative in nature, such as using rubrics for scoring portfolios, focus groups, observation, or interviews. There certainly is room for involving faculty outside the library in assessing how well students have grasped inquiry skills. By focusing on learning outcomes, libraries can reposition their mission as not just a collection of materials supported by services but as the common ground for student learning.

What can libraries gain by adopting a focus on student learning outcomes?

we can use what we learn to improve what we do
we can situate instruction as our primary mission in the entire operation of the library
with student learning as a focus, we can align our purposes more visibly with those of the faculty in the disciplines - an enhance collaboration
by learning more about how students learn and about where they have difficulty, we can deepen our profession's knowledge base

Assessment at Gustavus

With other academic departments on our campus, the library was asked to develop a departmental assessment plan in 1998. The process was guided by NCA criteria that spell out that assessment plans

should be tied to a mission statement
should identify specific outcomes of student learning
should identify evidence that will be used to assess student learning
should identify persons and procedures for interpreting the evidence
and should lay out a process for disseminating and using results

We opted not to use the Information Literacy Competency Standards, but rather met and reviewed our mission, then extracted from it four things we hoped Gustavus graduates would master:

Students will develop an understanding of how knowledge is produced and disseminated and will recognize that they play a role in knowledge production.
Students will understand how knowledge is organized and will be able to use that understanding to pursue information independently.
Students will be competent and confident in the use of print and electronic research tools in their major discipline(s).
Students will develop a sensitivity to and an appreciation of the diversity and wealth of knowledge created by different communities throughout time.

Finding out whether, in fact, students can do these things is not easy. We decided to use a combination of measures, including getting feedback from students and faculty involved in formal library sessions, holding annual focus groups with the same cohort of students across their four years, interviewing seniors about how they feel their research skills have grown over their college years, examining student work for signs of information literacy, and keeping track of formal student research-based presentations and publications. We've added to this other measures from time to time: studying the use of space in the public areas of the library, surveying students, faculty, and alumni, and a Web useability study - all of which focus on how the library supports student learning.

lessons learned

Start with what you really want to know; though there may be no easily "measurable" ways to assess student learning of difficult material, qualitative methods of inquiry can provide authentic insights.
Make a short list of ways to collect and analyze a manageable amount of data; then cut it in half. Creating too ambitious an assessment plan is setting yourself up for failure.
Don't just pay attention to what has been learned, but also to the learning process. Capturing a sense of the experience leading up to learning can be more informative than looking at final results.
Map out a process and timelines for gathering, interpreting, and using data. Be sure to set aside time to process what you've learned into action.
Get used to the fact that assessment of student learning will not tell you specifically how you're doing. There are many people involved in creating an information literate student. Though we can't control all of the variables, insights gained by looking at the big picture will give us lots to talk about with colleagues in other disciplines.

Readings on Assessment

American Association for Higher Education. Assessment Forum: 9 Principles of Good Practice for Assessing Student Learning. A short list of practical ideas about what makes assessment work.

Fraser, Bruce T., Charles R. McClure, and Emily H. Leahy. "Toward a Framework for Assessing Library and Institutional Outcomes" portal: Libraries and the Academy 2.4 (2002): 505-528. (online access for subscribing libraries) Addresses the use of the E-Metrics project in ARL libraries and considers assessment challenges and issues generally.

Hardesty, Larry. "Academic Libraries and Regional Accreditation." Library Issues: Briefings for Faculty and Administrators. 21.4 (March 2001). Covers various accreditation associations and their approaches to libraries--as well as how libraries fit into the accreditation process.

Journal of Academic Librarianship 28.1 (Jan-March 2002) Available through Science Direct. (online access for subscribing libraries) This issue includes a number of articles on assessment.

Lindauer, Bonnie Gratch. "Comparing the Regional Accreditation Standards: Outcomes Assessment and Other Trends." Journal of Academic Librarianship 28.1 (Jan-March 2002): 14-25. Available through Science Direct. (online access for subscribing libraries) Analyzes current and draft standards from regional accreditation commissions, finding more emphasis on outcomes, distance education, and information literacy.

Lindauer, Bonnie Gratch. "Defining and Measuring the Library's Impact on Campuswide Outcomes. College and Research Libraries 59.6 (November 1998): 546-570. Links library goals for learning with accreditation expectations and identiies ways libraries contribute to student learning--and methods of assessing the teaching-learning library.

Lopez, Cecelia. "Assessment of student learning: challenges and strategies." Journal of Academic Librarianship 28.6 (Nov.-Dec. 2002): 356-367. Available through Science Direct. (online access for subscribing libraries) An argument in favor of assessment and a description of the NCA's assessment matrix. Opens with an exhortation for librarians to participate. Ironically, though it argues librarians should be included in assessment, it avoids mention that the NCA has the lowest bar when it comes to explicitly including libraries in assessing or even influencing student learning.

Pausch, Lois M. and Mary Pagliero Popp. "Assessment of Information Literacy: Lessons from the Higher Education Assessment Movement." Choosing Our Futures: ACRL 1997 National Conference Papers. An overview of how the assessment movement intersects with information literacy's emphasis on student learning.

Pritchard, Sarah M. "Determining Quality in Academic Libraries." Library Trends 44.3 (Winter 1996): 572-594. (online access for subscribing libraries) An overview of previous attempts and an outline of the issues facing libraries.

Ralph A. Wolff. "Using the Accreditation Process to Transform the Mission of the Library." New Directions for Higher Education 90 (Summer 1995): 77-91. Discusses how self-study for accreditation can lead to situating the library more firmly in the center of teaching and learning. An interesting description of how conceptual changes in how we perceive the information landscape and the role of libraries in higher education drive a new emphasis on teaching and learning collaborations.

Ray, Kathlin L. "The Postmodern Library in an Age of Assessment." ACRL X: Crossing the Divide March 15-18, 2001. An interesting analysis of how changing libraries demand changing models for management, planning, and assessment.

Barbara Fister 12/2002