User Research

Each year, NYU’s group for IT professionals, TorchTech, hosts an annual Share Fair, which showcases technology projects across the university.  This year, we at Digital Library Technology Services used the opportunity to promote Guerrilla Usability Testing, using the ENM search prototype as a test subject.  We set up two laptops at our assigned table area, attached a fabulous sign (see above), and, for 2 hours, marshalled passers-by into 4- 10 minute testing sessions. We asked users to complete a few simple tasks (such as  “Search for ‘poet'”) while speaking aloud their impressions, assumptions, and expectations.

Some ideas we hoped to seed:

• one can do this kind of testing anywhere, with minimal requirements or preparation
• one can do this with a product that is still in progress
• one can use these sessions to share design and development ideas among colleagues, and to break down silos

We also, happily, achieved some secondary goals – namely, showing off the unique tech here, including our blazing fast search results, and getting more data points on how users comprehend the interface. This approach of ebook discovery using back-of-book indexes is unique, and our goal is a user experience that intuitively makes sense, operates smoothly and as expected, and provides exciting, useful results. We now have a few more ideas about what works (intrigues, pleases, or helps) and what doesn’t (or simply isn’t noticed.)

Many thanks to our colleagues from the NYU technology community for generously sharing their time and thoughts with us!

PDF handout here: TorchTechShareFair2018_Guerrilla_UX_testing.pdf

When I came on board as the Digital Production Editor for ENM, one of the first things I did was try to figure out what people want from ebooks. For me, as the curator and shaper of our topic map data, it was important to understand what user needs this data could address.

What do users expect from ebooks? With the goals of investigating that question, informing our design process, and suggesting areas for further research, as Digital Production Editor I compiled a literature review (organized by themes) and drafted accompanying user stories, which the rest of the ENM team then contributed to and revised. The research and user stories morphed into two separate documents: one with condensed theme summaries and user stories, and the other with all the nitty gritty details and quotes from sources. Following are some of the highlights; the (recently revised and updated) combined report can be found in the NYU Faculty Digital Archive.¹

Personas and motivations

A key overarching framework that emerged for me is the distinction between reader and user: Roger Schonfeld suggests that a reader’s objective is to read the book, whereas a user may have other objectives “such as skimming the illustrations, consulting an index, or conducting text mining.”² As noted recently by Joseph Esposito, “books perform all kinds of tasks.”

In addition to considering broad objectives, it was helpful to think about different motivations for ebook use. Following are some categorizations from a few different sources:

Modes³

• receptive reading (reading sequentially)
• responsive reading (active engagement with pauses and rereading)
• skimming (getting an overview)
• searching (looking for answers to a query)
• scanning (looking for a specific word or phrase)

Motivations⁴

• finding relevant content
• selective reading
• fact finding
• extended reading

Activities⁵

• reading cover to cover in depth
• reading a section in depth
• comparing treatment of ideas between monographs
• skimming in whole or in part
• exploring references
• searching for a particular topic

Many studies and articles indicated that ebooks are often used as a reference source, tool for research, or source for fact-checking. For these purposes, users skim, search, and jump around the text. These sorts of motivations can help us rethink the features and formats of our content. In his recent definition of “exploratory resources,” Roger Schonfeld advocates for thinking of reference content as “workflow and infrastructure” and “exploratory resources” as those “that allow a researcher to learn about a subject area or field of study and then deepen their learning through a curated set of potential further readings and other content types.”

Research also suggests that readers tend to prefer print books for their deep, extended, “receptive” reading, partly because they do not like looking at screens for long periods of time. Some readers also prefer print for selective reading of smaller chunks, perhaps because they grow frustrated with ebook platforms. Another factor to take into account is an individual’s familiarity with a given book. Familiarity could be important factor that influences the purposes and motivations with which readers/users approach an e-book.

These findings suggested to us that we focus our interface design work on features that will best help users and readers working in “researcher mode.”

Themes

Grouping insights from the literature review into themes galvanized our team to organize our user stories and start prioritizing what we wanted to focus on (we’ll cover more on how our design process evolved after this research in another post). The themes identified were:

• Navigating via menus and arrows
• Navigating via internal links
• User orientation within a book
• Full-text search within a book
• Annotating
• Supplementary information
• Navigating a collection
• Tablet experience
• Citation generation
• Copying/pasting
• Downloading/printing
• Text-to-speech

To sum up several issues that cut across these themes, research confirms the desirability of full-text search, annotation, and navigation using internal links within an ebook. Happily, that aligns with our ENM goals of providing annotation, full-text search, and navigation via the topic map. Downloading, printing, and copying/pasting are also important to users.

In terms of future directions, though navigation via internal links has been studied, expectations about the directionality of links could be further investigated. Relatedly, though users may expect ebooks to have Internet-style hyperlinks in them, conclusions are mixed regarding the appropriateness or value of links to external content, and it is unclear what sort of content users desire from external links. Finally, user expectations for navigating not just within a book but also between ebooks is also an area to explore.

References

1. Alexandra Provo, “Ebook User Expectations,” August 2017, http://archive.nyu.edu/handle/2451/39638.
2. Roger C. Schonfeld, “Forward,” in E-Books in Academic Libraries, ed. Suzanne M. Ward, Robert S. Freeman, and Judith M. Nixon (Purdue University Press, 2015), https://muse.jhu.edu/book/43208/, p. iii.
3. From A.K. Pugh (1978), used in Thayer (2011) and cited in Robert S. Freeman and E. Stewart Saunders, “E-Book Reading Practices in Different Subject Areas: An Exploratory Log Analysis,” in E-Books in Academic Libraries, ed. Suzanne M. Ward, Robert S. Freeman, and Judith M. Nixon (Purdue University Press, 2015), https://muse.jhu.edu/book/43208/, p. 226.
4. From Abdullah and Gibb (“Students’ attitudes towards e-books in a Scottish higher education institute: Part 2: Analysis of e-book usage”, 2008), cited in Tao Zhang and Xi Niu, “The User Experience of E-Books in Academic Libraries: Perception, Discovery, and Use,” in E-Books in Academic Libraries, ed. Suzanne M. Ward, Robert S. Freeman, and Judith M. Nixon (Purdue University Press, 2015), https://muse.jhu.edu/book/43208/,  p. 211.
5. From the 2015 Ithaka S+R Faculty survey.