A research team at the Center for Studies in Higher Education (CSHE) at UC Berkeley has undertaken a two-year project to understand more clearly how - and if - these online materials are used in undergraduate teaching contexts. Our research focuses on the humanities and social sciences out of concern that their unique modes of scholarship and pedagogy may require different educational technology solutions than those employed in science, technical, and vocational fields.

A report of preliminary findings from the first year is now online ( http://digitalresourcestudy.berkeley.edu/ ). The early analyses of faculty focus groups and more than 500 UC faculty responses to surveys suggest that although many faculty in the humanities and social sciences do use digital resources in their teaching, they may struggle with finding and reusing materials. Many in our sample also encountered significant barriers to their digital resource use, including lack of time, inadequate support, and limited access to classroom technology.

Why bother understanding users?

Our research is specifically targeting a suite of questions that will move us closer to a better understanding of current and future user demand for available digital resources. Through a variety of methods, most importantly talking to a sample of faculty across various disciplines and institutions, we are seeking a more nuanced understanding of faculty opinion about the role of digital materials in teaching. We have been particularly careful in our work to ask faculty what they value (as opposed to telling them what is valuable). In addition to surveys of faculty, and discussions with faculty, graduate students, librarians, site owners, and educational technology professionals, we are also conducting an extensive literature review of current user studies and exploring what methods are best suited for understanding users in different contexts.

What faculty say they use

Digital materials take many forms. They are found in media sites such as the New York Times, and in collections assembled by independent organizations and by individual scholars. There are ambitious attempts to put up course web pages or whole courses (e.g., MIT OpenCourseWare , World Lecture Hall , Monterey Institute for Technology and Education ), clearinghouses of individual learning objects (e.g., MERLOT ), and digital library/museum collections (e.g., MOAC , Harvard Open Collections Program ). Media types encompass images, videos, audio recordings, news resources, digitized books, journal articles, data archives, maps, historical documents, curricular materials, as well as others.

Our preliminary efforts have reinforced our initial suspicions that there is a wide variety of digital resources being used in higher education. The digitized resources faculty report using are not just text and do not just emanate from libraries, or any other single source. According to our surveys, the most commonly used resources (in approximate order of frequency of use) are images and visual materials, news and other media sources, online reference resources, digital film or videos, and portals. Curricular materials, blogs, digital coursepacks, and simulations were low on the list of things faculty in our sample say they use frequently.

Perhaps not unlike students, faculty in our sample say they use Google-type search engines as the most common way to find resources, including images. Other prominent sources of materials are those garnered directly from faculty's own personal collections. Online journals and public/free image databases are also high on the list of preferred ways of locating desired resources.

Discipline, institution type, and individual teaching philosophy matter

We see a spectrum of user types emerging, ranging from the non-user, to the inexperienced, novice user, to the highly proficient and advanced user of digital resources. Nonusers are themselves diverse. Often pejoratively referred to as "Luddites" by those more enamored of technology, nonusers in fact include those who are passionately opposed to the use of technologies in their classroom for a variety of valid pedagogical reasons (e.g., they cannot substitute for preferred teaching approaches). Nonusers also include self-described enthusiasts frustrated by technical and non-technical barriers, and those simply without enough time to think about, let alone use, technology in teaching.

Variation among disciplines appears to be of some importance; the humanities and social sciences are not a monolith. Faculty who use texts extensively depend on different kinds of sources, for different pedagogical goals, than those who rely more heavily on images. Maps are heavily used by historians and anthropologists/archaeologists; political scientists are the heaviest users of news archives. Those in art and architecture appear to have different profiles than their colleagues in other disciplines-relying somewhat more frequently on images and video. Those who teach subjects that require three-dimensional visualization and/or historical reconstruction may need particularly sophisticated resources and tools.

We suspect in many cases that scholarly practice is often linked tightly to pedagogical approach in various sectors of humanities and social science teaching, leading many faculty to place a high value on integrating their own resources and research into their teaching practice. More than 75% of our respondents who are faculty members in anthropology/archaeology, art/architecture, and history say they use their own personal collections as sources for digital materials; more than 60% of respondents in languages/literature and political science say they rely on their own collections.

Barriers

Based on our work to date, we can say that it is not at all easy for most of our respondents to use the plethora of digital resources available to them. Many want a one-stop shop in which they can find and re-aggregate snippets from available resources into a customized resource for their own use. Consequently, they are unsatisfied both with their ability to find the resources they need and with the tools available to manage those digital resources in different contexts.

The most-cited obstacles to the effective use of digital resources were the availability, reliability, and expense of the necessary equipment (both in the classroom and for personal use). One faculty member explains his/her reluctance to use technology in the classroom:

". . .I find that the computer in class anchors me to a certain spot and at t imes to a certain order of presentation. I need freedom to improvise, change direction, and physically move around…Finally, I hate the tension that equipment introduces into the classroom, the fear of breakdown, the suspense, the frequent waste of time…"

Other obstacles mentioned included difficulty locating high-quality, pedagogically relevant materials from credible sources, and the sheer volume of available materials. Lack of time is a common theme. One faculty member explained,

"I came across an adage that 'e-mail allows me to do in one hour what I never had to do before.' So it goes with course WWW sites and digital instructional media too."

Another explains the time and resource challenges of integrating personal resources into new media,

"I own a personal collection of 40,000 35-mm slides, so to put it mildly, I am very invested in 20th century technology. I would need real help-both in machines and time-to convert teaching to power point, although I see some of its genuine advantages."

As our research continues, we are exploring how or if different obstacles may be more characteristic of different disciplines, different institutions, or different types of users.

What methods, and for what questions?

Much of our work is targeted toward understanding the overall value of different types of "user" studies, which can be expensive to conduct well. Specifically, how can we begin to assess overall user demand for digital resources, and what analytic methods are useful for the various phases of decision-making faced by creators and stewards of digital resources? Our review of the current literature on use studies and our own work point to the importance of distinguishing between "user demand" vs. "usability of brand" questions when studying and making inferences about user behavior and motivations.

For example, "usability" studies can be helpful in a site's content design and for improving usefulness of a website or tool in a particular context, but they are limited to the evaluation of existing resources, and their methods often tell us only about current users of a particular site. If not combined with methods to rigorously assess non-users and/or non-enthusiasts, such studies tell us nothing about that site's value or usability for a wider potential audience in diverse educational environments. More importantly, in focusing on usability, many studies ignore the broader questions of usefulness for faculty members' important needs and goals. We suspect that the application of inappropriate methods to questions of funding start-ups, targeting new audiences, or finding the resources to maintain or improve an existing project could potentially result in costly choices.

At the midpoint of the research project, we are refining an understanding of the complex digital resources landscape faced by today's college instructors in the social sciences and humanities. We expect that a better understanding of the concerns and experiences of those instructors will provide the groundwork for integrating more effective, useful, and usable digital resources into varied teaching and learning contexts.

Finally, we would like to take this opportunity to thank those UC faculty who took their valuable time to participate in our discussions and to fill out our survey. We will continue to update our findings on our web site as they become available, and we encourage feedback from interested members of the UC community.

This project is funded by the Andrew W. Mellon Foundation and the William and Flora Hewlett Foundation, in partnership with Center for Information Technology Research in the Interest of Society (CITRIS), the Hewlett-Packard Company, the California Digital Library (CDL), and the Vice Chancellor of Research, UC Berkeley