E-mailThis reading list is meant to help with the first stage of an OCUL user study: What do we know already from previous studies?
First and foremost, we need a vision
"We suggest clients look five or ten years into the future and ask the question, "What will using our site be like on that date? What experience will the user have?" Team members from the best organizations can answer this question. They have a consistent, clear idea what the user's experience will be like in the future. Having a clear vision lets the team chart a direction for their design, helping identify if any design idea is moving them closer to the vision or farther away... It's critical the vision not focus on future technology but instead on future experience. What are the steps in today's process that makes things cumbersome or frustrating? How could the experience become more delightful?" Thinking in the Right Terms: 7 Components for a Successful Web Site Redesign
We need to understand our users
"The University of Minnesota Libraries received support from the Andrew W. Mellon Foundation to develop a multi-dimensional model for assessing support for scholarship in the context of a large research campus. The project team explored discipline-specific needs for facilities, information content, services,tools, and expertise in the humanities and social sciences. The goal was to develop a model for bringing greater coherence to these distributed resources through physical and virtual means, and also a research support environment that could be modeled, prototyped, and evaluated." A Multi-Dimensional Framework for Academic Support: Final Report
"Our first task was to identify one trenchant research question to guide the project. The question we developed was, What do students really do when they write their research papers? Between the assignment of a research paper and the finished, submitted product was a black box that largely concealed the processes undertaken by the student. We wanted to take a peek into that box to see what we could find." Studying Students: The Undergraduate Research Project at the University of Rochester (pdf)
We need to build something better
"The author seeks to answer the following questions: What characterizes the queries that end users submit to online IR systems? What search features do people use? What features would enable them to improve on the retrievals they have in hand? What features are hardly ever used? What do end users do in response to the system's retrievals? Are end users satisfied with their online searches?" [Markey, K. (2007). Twenty-five years of end-user searching, part 1: Research findings. Journal of the American Society for Information Science and Technology, 58(8), 1071-1081.]
"Reading and synthesizing Marcum, Calhoun, Bates, Mann, Hildreth, Borgman, Anderson, etc., should be mandatory for everyone who cares about the future of the online library catalog. The next steps must be to engage all interested parties in serious dialogue, system prototyping, decision making, and action so the online library catalog of the future hits the ground running just as mass digitization projects end." [The Online Library Catalog: Paradise Lost and Paradise Regained?]
"Richard Wallis of Talissuggested during his Access 2007 presentation that if the catalogue knew a bit about you, it could present better search results to you. Using the example of the keyword lotus, an engineering student would get items about the car of the same name while the botanist would get items about the genus Nelumbo. Mark Leggot, using his metaphor of the search box as Lego brick, suggested in his Access 2007 presentationthat it shouldn't be difficult to embed a library search box into a course-specific page that was already constrained to the sources that were relevant to both the subject of the course and the course level." [Access 2007 Wrap Up Part Three - The Importance of Being Relevant]
We need to get into the places where our users are
For me there are five parts of a strategy for maintaining the library as a vibrant enterprise worthy of support from our campuses...
4. Reposition library and information tools, resources, and expertise so that they are embedded into the teaching, learning, and research enterprises. This includes both human and, increasingly, computer-mediated systems. Emphasis should be placed on external, not library-centered, structures and systems.
5. Migrate the focus of collections from purchasing materials to curating content. [A Strategy for Academic Libraries in the First Quarter of the 21st Century]
"We suggest three main areas, with specific recommendations for each, where our institutions can help to alleviate these pressures.
- Enhance and improve the user interface
- Connect the citation network to user workflow
- Embrace standards and technologies that will allow present and future network discovery systems to make use of what we offer" [Scholr 2.0]
Other recent studies...
Information Behaviour of the Researcher of the Future
"A new report, commissioned by JISC and the British Library, counters the common assumption that the 'Google Generation' - young people born or brought up in the Internet age - is the most adept at using the web. The report by the CIBER research team at University College London claims that, although young people demonstrate an ease and familiarity with computers, they rely on the most basic search tools and do not possess the critical and analytical skills to asses the information that they find on the web."http://www.jisc.ac.uk/media/documents/programmes/reppres/gg_final_keynote_11012008.pdf
Observing Students in their Native Habitat
John Law, ProQuest CSA (USA)
This presentation was delivered at Scholars Portal Day in December 2007, and similar versions have been presented at various other recent conferences. The study shows that yes, students use Google as a starting point and reference tool, but they also use other research tools quite competently... if they can find them. Library marketing/instruction helps with this. Barriers include library websites that do a poor job of exposing the available resources, and authentication. Student Research Behaviors - OCUL.PPT
Scholars Portal Search User Study (2005)
In the summer of 2005, OCUL's Scholars Portal Search Working Group conducted a user study to better understand how to position Scholars Portal Search in OCUL institutions, in light of staff feedback solicited earlier in the year. The study involved a sampling of users at several OCUL institutions, and investigated their satisfaction with aggregated search results in Scholars Portal Search. There was a balance of users who were fairly new to academic research (undergraduates) and users who routinely conduct in-depth research using information retrieval systems (graduate students and faculty). In this study, it was observed that satisfaction with the results of a search of multiple unrelated databases did not depend on the level of experience of the user. Users were satisfied with the results of the all databases search when their topic was fairly narrow; this was true of both novice and experienced users. They also found them useful for other reasons, such as interdisciplinarity and serendipity. Results from a search of a single database or a group of databases related to a discipline were more satisfactory for some users, particularly less experienced searchers. Most users wanted to be able to search without needing to select the appropriate databases. SPS User Results Aug31final.pdf
Portals for Undergraduate Subject Searching: Are They Worth It?
This very good article in the October 2007 issue of Portal: Libraries and the Academy draws this conclusion:
"After analyzing our campus and library computing environment, the state of portal technology, and student feedback from the needs assessment, our thinking about using a portal approach to facilitate subject searching changed. Because of the factors discussed previously, the task force concluded that a virtual college library offering subject access by college or discipline is the wrong level of access for undergraduates. We recommended, instead, the construction of library Web pages for particular courses or course assignments. Creating course-specific pages retains a subject approach but gives students access where they prefer it---at the course level.
There is a need for more research into the complexities of how undergraduates seek information and how libraries can expose more quality content to students. Libraries would benefit from studies about the interplay of students' library experience, class rank, and progress in their major. Librarians assume that students will transfer technologies they use in their personal life to their academic life. We must have a more nuanced understanding of this behavior to create library tools for the new generation."
http://muse.jhu.edu./journals/portal_libraries_and_the_academy/v007/7.4nichols.pdf
User Consultation Report, Queen's University Library (Usability Matters)
This was a broad study to explore both the current state of the library website(s) as well as opinions and ideas for future enhancements and new services/features. Findings include: researchers want personalized search tools; "subject" emerged as the preferred categorization approach; "home" library model does not carry over to the online environment; search refining options are as important as "advanced search"; "single search" is not a significant priority; instructional support efforts should be focused on directing researchers to the appropriate resources (ideally specific to courses or better yet assignments) and contextual help. http://library.queensu.ca/libdocs/QUL_UserConsultation_Report.pdf
University of Toronto Library website redesign projects
Helping students find articles: two paths to resource discovery (OLA SuperConference 2008 Poster Session)
Heather Cunningham, Marc Lalonde, Gerstein Science Information Centre, University of Toronto Libraries.
"Finding journal articles is one of the most daunting challenges a student faces when using a library Web site. The University of Toronto Libraries (UTL) Web team employed three techniques to incorporate student input into the library Web page 2005-2006 re-design process: card sorting exercise, several rounds of usability testing and a persona. The results of card sorting and usability testing will illustrate why patron-preferred terminology and a prominent Google-like search box was deployed on the UTL home-page. The 2006 Gerstein Science Information Centre (GSIC) Web page redesign incorporated many UTL evidence-based elements. The GSIC home-page differs by not having a search box. Usability results and Dr. French, the persona of a veteran medical researcher, informed this design. These two approaches for finding articles address the needs of the university's diverse user community."
University of Western Ontario website redesign project
Western has conducted user studies related to their current redesign project... (need to follow up with contacts)
White, H., Wright, T., and Chawner, B. 2006. Usability evaluation of library online catalogues. In Proceedings of the 7th Australasian User interface Conference - Volume 50 (Hobart, Australia, January 16 - 19, 2006). W. Piekarski, Ed. ACM International Conference Proceeding Series, vol. 169. Australian Computer Society, Darlinghurst, Australia, 69-72.
From the abstract: "The evaluation found severe usability problems with online catalogues--we found so many problems we were forced to use a card sorting technique to understand and classify the problems."
Prabha, Chandra, Lynn Silipigni Connaway, Lawrence Olszewski, and Lillie R. Jenkins. 2007. "What is enough? Satisficing information needs." Journal of Documentation, 63,1: 74-89. Pre-print available online at: http://www.oclc.org/research/publications/_archive/2007/prabha-satisficing.pdf_ _
Nancy Fried Foster and Ryan Randall. 2007. Designing the Academic Library Catalog: A Review of Relevant Literature and Projects
Relates to University of Rochester's planning of their project ot develop the eXtensible Catalog. http://www.extensiblecatalog.info/wp-content/uploads/2007/09/Designing%20the%20Academic%20Library%20Catalog.pdf
Karen Calhoun, Prepared for the Library of Congress, March 2006, The Changing Nature of the Catalog and its Integration with Other Discovery Tools
Let's not forget about this report that helped get this whole conversation started... http://www.loc.gov/catdir/calhoun-report-final.pdf
LASSIE - Libraries and Social Software in Education
LASSIE is a 9 month project funded by the Centre for Distance Education at the University of London. Its recently published five case studies and produced a literature review on social software, libraries, and education.
MINES for Libraries: Measuring the Impact of Networked Electronic Services and the Ontario Council of University Libraries' Scholars Portal, "MINES for Libraries™ adapts a long-established methodology to account for the use of information resources in the digital environment. The survey is based on methods developed to determine the indirect costs (Franklin, 2001) of conducting grant-funded R&D activities, and was adopted as part of ARL's New Measures program in May 2003." Data was collected from sixteen OCUL libraries, comprising more than 20,000 uses, between May, 2004 and April, 2005. http://www.libqual.org/documents/admin/FINAL%20REPORT_Jan26mk.pdf
What do we think we know?
Following on our March 7 meeting with Usability Matters, please add here your understanding of what we already know about user research behaviour:
- students understand the value of going beyond Google to specific sources in a field, but they can't find them
- students value the recommendations/endorsements of their instructors (ie more so than a librarian alone)
- in implementations of Endeca, only 20-30% of users make use of facets (but we don't know why)
- federated searching:
- everyone appreciates the single starting point
- novice searchers are often confused by 'one big search' results and don't have the skills to hone their search
- expert searchers like the serendity of the 'one big search' and also recognize the value of tools specific to their discipline
Add Comment