The third phase of investigation was conducted during the formative research phase of the project in 2012. Following the literature review, and the environmental scan, in order to begin to develop specifications for what would become the Smithsonian Learning Lab, two phases of teacher research and prototyping were conducted. The first engaged a group of 20 educators in California in the spring of 2012. The second involved rapid prototyping with 69 teachers in Washington DC later that summer. The patterns of behavior observed and recorded during these two phases enabled SCLDA to design initial prototypes grounded in research and then test and quickly revise those prototypes. For a detailed look at this work (as well as all the research instruments, etc.) see the Report of Findings: Teacher Research and Prototype Testing and its appendix (available for download and reuse on the research wiki).

It's notable that the relatively small sample sizes raise some questions about the generalizations of what was uncovered during this phase. However, when taken together with the literature review and the environmental scan and the available user analytics (based on thousands of teachers), the outcomes may offer insights and frameworks for other museums interested in exploring similar questions.

Phase 1: Project Design and Methodology

The aim of this phase was to take what we had learned already and apply it to focused teacher inquiry, exploration, and evaluation. To do this, our teacher group was involved in an iterative series of investigations, which resulted in classroom-tested lessons designed by the teachers using Smithsonian assets. The research team employed a combination of heuristic, quantitative, and qualitative measures, utilizing surveys, observation, focus groups, and monitoring of online conversations in order to capture a range of interactions.

A central tenant of the research design and methodology of the Learning Lab project is to listen carefully and watch closely the behaviors of teachers. Rather than using a large sample of teachers and conducting more traditional survey research, the first phase of the project involved identifying a group of teachers to serve as an intimate research group, or focus group. We hoped to surface more actionable recommendations that could be further studied with a larger sample of teachers in Phase 2.

The teacher participants engaged in a series of common use-regimens to address our research question:

How did educators find, analyze, modify, or organize the resource(s) for classroom use or electronic distribution to students? Given options include:

• How do educators prefer to search for resources?
• How do educators prefer to save a quality resource they have identified?
• Did educators elect to add annotative data to a resource when utilities were provided?
• Did educators find and share resource(s) with others through on-site (push to FB, Tweet, etc.) or off-site means (copy url into email and send to others, etc.)?

At the workshop, teachers searched for Smithsonian resources using SmithsonianEducation.org, Brokers of Expertise, Thinkfinity, Gooru, or OER Commons.

Part two of the teacher participants work, done independently, was to integrate two digital items into a proposed teaching/learning activity guide, lesson plan, project description, etc., for their classroom. After teaching that lesson, they reported their successes, failures, and reflections.

Our deeper look at teacher behaviors in this phase confirmed previous findings and expanded our understanding. Where the user analytics and Foresee data showed us what teachers were looking for, we now know more about why teachers might choose a particular resource over another and how they might use it to build a lesson.

Phase 2: Project Design and Methodology

Throughout the summer of 2012, Council for Chief State School Officers (CCSSO) “State Teachers of the Year" were in Washington DC for weeklong digital learning workshops. Participants were largely K–12 classroom teachers as well as five teachers from Phase 1 invited to provide some continuity across the two phases of teacher research.

Our research team had one hour a day with these groups (one hour per day for four days with three distinct groups over three weeks) so the challenge was to design activities that could yield answers to important questions about teachers' needs and their use of digital resources. Each week there were three full iterations of a development-and-deployment methodology using the accelerated delivery of Agile software development.

Three distinct groups of educators engaged in a series of activities, each of which was designed to address a series of research questions:

• How do various related searches ranging from global to granular, render results within the prototype structure, and how are those results best consumed by the educators for classroom use?
• Once teachers find viable resources for use in the classroom, how do they choose to display and use those resources in the classroom?
• How readily can participants assemble their own unique collections and integrate instructional interactive modules?
• If given tools to include Common Core standards and external web resources, will teachers feel confident in analyzing and identifying appropriate additional resources and standards as correlated to Smithsonian resources?

With these questions in mind, engineers and researchers elected to begin with paper-based interface activities. This quick and dirty prototyping confirmed some basic structures we had hoped to build into a digital prototype. The developers worked quickly to create a web environment with a functional user interface and a set of search/save/organize/adapt tools tied directly to the existing Smithsonian database of digitized assets. Full access to this prototype (shown below) is available at http://scems.navnorth.com.

When taken together, the three weeks of teacher workshops enabled us to confirm a set of behaviors that have implications for our development, including interface design and metadata management. When taken further and compared with the literature, environmental scan, and our earlier research, as well as what we know about best practices and usage patterns, the workshop experience helped us develop the following guiding principles (as well as a full set of specifications for building the prototype into the Learning Lab).

We can summarize the prototype testing findings as follows:

When searching for or identifying content:

• Teachers prefer to search by entering a general search term, then filtering further if needed. They also prefer the gallery view to review their search results. The workshop participants want more intelligence in their searches and results to guide them toward the most valuable resources. This “intelligence" includes auto-complete typing, auto-correct spelling, and similar items.
• When seeking content online, teachers use a diversity of locations to find what they need and have little loyalty to one site in particular, although they go to educational sites more frequently than non-educational sites. In seeking collaboration, the researchers have concluded, the teachers use both education and non-education sites equally.
• Participants used the Facebook Share option that was provided, but the most popular method of sharing was emailing the link to themselves or a colleague.

When analyzing content, teachers want:

• to save resources that they find useful, and will employ any available mains to do this, even if the site does not provide this function.
• the flexibility to organize and annotate resources according to their own schemas.
• flexibility in the types of viewing methods available: one for whole-class interaction and one for individual interaction.
• the ability to have students use the site and its tools as much as the teacher.
• content that is aligned, or close-to-aligned, with Common Core state standards.

When extracting content to create an interactive lesson, teachers want:

• the use of interactive modules with the resources found in the Smithsonian collection.
• a variety of tools.
• better visibility of the tools, including prompts and explanations for their use.
• more intuitive design and flow between tools and facets of the prototype.
• the ability to upload resources from other sources to augment their collections
• more intuitive design and flow between tools and facets of the prototype.

The full Report of Findings: Teacher Research and Prototype Testing and its appendix is available for download and reuse on the research wiki.

This post summarizes, and at times excerpts from an original research report commissioned by the Smithsonian Center for Learning and Digital Access from our formative research collaborators Cross & Joftus and Navigation North in 2012. The following individuals, in addition to the author, made significant contributions to the research design, data collection, analysis, and writing of the original reports and should be acknowledged and credited: for SCLDA: Michelle Smith, Pino Monaco, Melissa Wadman; Navigation North Learning Solutions: Joe Hobson, Brian Ausland, Daniel Krieger, MaryRose Lovgren; and Cross & Joftus: Christopher Cross, Virginia Adams Simon, John Ittelson, Clark Quinn, Dilan Maherdran, Virginia McMunn, Griffith Montgomery, Jillian Ryan; and Steve Midgley.

The teacher research and prototyping was funded by a Smithsonian Youth Access Grant administered by the Office of the Assistant Secretary for Education and Access, with contributions by the Pearson Foundation, Brokers of Expertise of the California Department of Education, and the Council of Chief State School Officers.

Images: teacher photos by Joe Hobson, Navigation North