NOVA Labs

Description

For nearly 40 years, NOVA’s been reporting from the frontlines of research, translating the ideas and sharing the passion of scientists who've changed the world. Now, through these Lab experiences, we’re adapting the storytelling skills we use in our TV programs to help teens and lifelong learners take part in research themselves. Our guiding principle? The best way to appreciate science is to be involved in it.

NOVA Labs is a new digital platform where "citizen scientists" can actively participate in the scientific process. From predicting solar storms and designing renewable energy systems to tracking cloud movements and learning cybersecurity strategies, NOVA Labs participants can take part in real-world investigations by visualizing, analyzing, and sharing the same data that scientists use.

Each Lab is unique, and focuses on a different area of active research. But all of them illustrate key concepts with engaging and informative videos and guide participants as they answer scientific questions or design solutions to current problems. Experts in the field are available as well to answer users’ questions and to propose new routes of investigation.

We will use both qualitative and quantitative information to gauge the progress and success of the NOVA Labs Initiative. The effectiveness of the initiative will be measured through external evaluation undertaken by The Lifelong Learning Group (LLG), an independent research and evaluation consulting group. LLG will direct evaluation the Labs to assess the achievement of targeted outcomes among student and teacher audiences, individually and as compared to other Labs. We will use web analytics and social media tools to gather quantitative information about traffic to the Labs (usage and reach) and about the time users spend with them (engagement).

Partnerships

Effectiveness and Impact

Evaluation findings and impact statements:

Internally, we work with a group of approximately twenty-five teen advisors, all between the ages 13 to 17, who provide formative feedback about the Labs during development and production. Our teen advisors provide quick turnaround feedback on all aspects of the Lab, including: design, content, and usability. Our work with the teen advisors helps to ensure that Labs are age appropriate and engaging. Externally, NOVA contracts with The Lifelong Learning Group (LLG), part of COSI, to direct formative and summative evaluation of NOVA Labs. The description of services for the formative evaluation of the pilot Sun Lab is attached.

The summative evaluation for both NASA-funded Labs is still in process and planned for completion by winter 2013-14, however formative evaluation of the pilot Sun Lab has been completed, and an excerpt of the findings follow:

Original Report Prepared by: Jessica Sickler, M.S.Ed. September 2012
Overall Appeal: Factors Driving Teen Use of Site When asked to think hypothetically about what would likely drive them to use a site like NOVA Labs, the consensus was overwhelmingly that the site would be used and valuable for school-related contexts The benefits for school use, however, were many and attractive to teens in the focus group discussions. Several teens mentioned specific courses or teachers from their own experience who would use the site and that they felt it would have improved their experience in the class: Teens who explored the site on their own in the think-alouds (without direction about where to go or what to look at) did not tend to understand that the purpose of the site was to conduct one’s own investigations with authentic data. Two features that seemed to be more helpful in explaining the purpose of the site were the intro video and the infographic, both in the slideshow on the home page.

For participants in the focus groups, the overall concept of NOVA Labs was very positively received, with teens generally thinking it had cool and interesting aspects that would add to their school experiences. The video features and lab features were generally the most instantly compelling ideas for most of the teens.

The other compelling attribute of the overall NOVA Labs concept was its authenticity. This was partially impacted by the relationship with NASA, which seemed to create a positive impression on teens – as identified via the logo on the home page and through the scientists’ affiliations. But it was most impacted when teens were told or read that the site was sharing real data from NASA satellites and instruments. In both focus group discussions, teens sought verification on this point, that what they were seeing was real data, and not just content for an educational purpose.

“Probably using the live data from NASA [is most interesting]. … this was just more like, okay, this is actual data from – that's like currently being collected. I thought that was cool.”

“Like those are like the sunspots on the sun [in the image] like at the moment, or are they just like, hey, we're just going to throw these on for the Sun Lab?”

In response to the Facebook Meet-the-Scientist Event example (Phillip Chamberlain), results were somewhat positive in both the focus group and think-aloud samples. The idea of a Q&A opportunity with a real scientist was seen as a positive and unique opportunity and outlet for participants.

Conclusions and Recommendations:

Overall, the concept and core features of the NOVA Labs framework were well received by teen test audiences. The authenticity of interacting with real data from real NASA research and instruments was very appealing to those who realized those opportunities. Similarly, the use of high quality video segments that were brief, explanatory, and used strong graphics and animations to explain concepts were very appealing to the teen test audiences. The biggest challenge for the site overall seems to be how to reach the target audience. The pathway of formal education seems to be very promising, as nearly all of the students felt their use of it would be for a school-related purpose. Teens felt it would be a very positive addition to their schoolwork, seeing great benefit in high quality, interesting, and interactive content like this Lab.

Although these tests were artificial in some regards – the users were selected to test the website in a brief period of time out of context from real use – themes emerged in usability and content that may help inform the team in the development of future NOVA Labs.

Reaching Teen Audiences:

The plan to use dissemination via schools, teachers, and formal education contexts seems a fruitful approach with great potential impact. From the descriptions of teens, the site aligns with content they are currently learning and the types of resources their teachers like to employ – although NOVA Labs was felt to be of a higher quality than much of what they’ve encountered previously.

Non-formal uses of the site may require identifying smaller segments of the teen audience with particular interest in the content area of a given Lab.

Navigation and Usability

Improvements to navigation and site architecture are already underway and established by the NOVA Labs team, so basic navigation was not a main focus of this evaluation. The focus was on specific user challenges in interacting with the features, which may impact understanding. Thus, these comments and suggestions are made knowing that some may already be addressed in current plans.

Although challenging for any website, opportunities to decrease reliance on reading large quantities of text in order to be successful are advised. The use of videos was a huge strength of NOVA Labs in this regard. However, in Lab activities, the reliance on reading written instructions often hindered understanding the purpose of an activity (for teens exploring on their own). Even with the strong video content, the quantity of reading on the site was a frequent critique.

The use and style of the current videos is a strong and successful strategy to maintain. The style, tone, narration, and length of the videos were all tremendous attributes for the audience. They were relatable, understandable, and the look even pulled in a number of viewers.

Continue and consider opportunities to further use animations and diagrams to help explain important concepts. These were very well-received by teens and the combination of those diagrams with narration helped them feel like they were understanding the content.

Maintain the engaging style of narration in the videos, as well as the inclusion of scientists who are strong presenters / communicators. This added to the engaging qualities of the videos, as well as their feeling of reliability of information. The Sun Lab: The Lab activities are generally difficult for teens; they were not insurmountable, but they were challenging. There did not appear to be a concern that any of the challenges were too easy or simplistic.

Maintain the strong use of guided exploration activities in the Lab environment. Even teens intrigued by the option to do free exploration felt that guidance was needed to feel successful and know what you are doing. In fact, there were requests across teens to make the current guided explorations MORE step-by-step to reduce confusion. Consider enhancements to an Open Investigation area that might give users more guidance in how to get started on their own – orienting them to the features on the page that they might use and how. Label features with both technical names and more common explanations of what they do. If there is a specific video that provides the tutorial, perhaps link to it directly on this page (for users who have not completed the full Boot Camp sequence).

The Lifelong Learning Group (LLG), part of COSI, will direct summative evaluation, to be undertaken in fall and winter 2013-14. As currently envisioned, this summative evaluation will explore NOVA Labs success in achieving the following outcomes:
Youth participants in the Lab will:

• Increase knowledge of STEM career paths.
• Increase interest in pursuing careers or higher education in STEM fields.
• Increase knowledge of Lockheed’s leading role in science research.
• Demonstrate targeted STEM skills through work in the Lab