• Have any questions?
  • info.zbook.org@gmail.com

Sustainability: Professional Development Summative .

3m ago
6 Views
0 Downloads
1.74 MB
41 Pages
Last View : 1d ago
Last Download : n/a
Upload by : Camden Erdman
Share:
Transcription

Sustainability: Professional DevelopmentSummative Evaluation ReportCommunity Environmental Services in partnership with OMSIPrepared byRenée Bogin Curtis, EvaluatorCommunity Environmental Services, Portland State UniversityDecember 31, 2014Funded by the National Science Foundation, Grant No. DRL-0917595With the generous support of:

Table of ContentsExecutive Summary . 2Introduction to the Sustainability Project . 4Overview of ExhibitSEED . 5The Development of ExhibitSEED . 8Measures of Success . 10Evaluation Methods . 11Findings . 13Conclusions and Recommendations for the Field . 24Appendix I. Post-Workshop Participant Survey . 26Appendix II. Delayed Post-Workshop Survey. 28Appendix III. Logic Model . 29Appendix IV. Green Exhibit Checklist. 30Appendix V. Decision-Making Tool . 37This material is based upon work supported by the National Science Foundation under Grant No. DRL-0917595.Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s)and do not necessarily reflect the views of the National Science Foundation. November 2014 Portland State UniversitySustainability: Professional Development Summative Evaluation Report1

Executive SummaryOverview of the Sustainability ProjectAs part of the National Science Foundation funded Sustainability: Promoting Sustainable DecisionMaking in Informal Education project, the Oregon Museum of Science and Industry (OMSI) and itspartners developed a bilingual (Spanish/English) exhibition, event series, outreach campaign including apublic website, and a professional development website and workshop. The goal of these deliverableswas to promote sustainable decision making by building skills that allow participants to weigh thetradeoffs of their choices and choose more sustainable practices. The project duration was September2009–April 2015. Portland State University, in collaboration with OMSI, supported many phases ofevaluation during the project and led all summative evaluation activities.Overview of ExhibitSEEDExhibitSEED (Exhibit Social Environmental and Economic Development) refers to the professionaldevelopment website and the Museum Advisory Committee (MAC) workshops created by OMSI duringthe Sustainability project. ExhibitSEED resources are based on a three pillars approach to sustainabilitythat includes environmental, economic, and social considerations. The website (www.ExhibitSEED.org)launched in 2012 as a place for exhibit and museum professionals to find resources for developing,designing, and building more sustainable exhibits. The website tools were developed from theperspective of an interactive science museum, but the information is valuable to exhibit professionals inall types of museums. The MAC workshops were conducted in 2013 at five museums in different regionsof the country. They provided opportunities for attendees– 100 design and fabrication professionalsfrom 81 organizations– to practice sustainable decision-making skills and use the ExhibitSEED tools.The summative study evaluated whether the tools achieved the following impacts:1. Workshop participants will understand considerations for the development, design, andfabrication of more environmentally, economically, and socially sustainable exhibits.2. Website users will become aware of skills associated with environmentally, economically,and socially sustainable decision making in exhibit development, design, and fabrication.3. The Green Exhibit Checklist (GEC) will foster skills associated with environmentallyresponsible decision making in exhibit design and fabrication.Key ResultsThe findings are based on data from 85 post-workshop surveys conducted at the end of each event, 45delayed post-workshop surveys conducted within 6–8 months of the workshop, and a team debriefconducted after the final workshop with workshop facilitators and planners. The data do not alwaysdirectly measure the percentages outlined in the intended outcomes, yet relevant data were evaluatedand overall indicated the deliverables successfully achieved their intended impacts and outcomes.Museum Advisory Committee (MAC) WorkshopsBased on a scale of 1–5 (5 being the highest and 1 the lowest), participants in the post-workshop surveygave the MAC Workshop very high mean ratings in terms of its usefulness (4.52), relevancy to work(4.59), and applicability to practices (4.44). Those participants also rated the workshop fairly high interms of its effectiveness at facilitating sustainable practice (4.0) and very high in terms of theirlikelihood to apply the tools (4.47). These ratings indicate success in encouraging industry professionalsto understand considerations for the development, design, and fabrication of more environmentally,economically, and socially sustainable exhibits. In the post-delayed surveys, many participants alsoindicated ways that the three pillar model impacted their thinking or approach after the workshop.Sustainability: Professional Development Summative Evaluation Report2

ExhibitSEED WebsiteBased on a scale of 1–5 (5 being the highest and 1 the lowest), participants in the post-workshop surveygave the ExhibitSEED website high mean ratings in terms of its usefulness (4.19), relevancy to work(4.24), and applicability to practices (4.02). These ratings indicated successful awareness of skillsassociated with environmentally responsible decision making. In the post-delayed surveys, 78% ofparticipants reported use of the website often because of the website’s good tools and information.Green Exhibit ChecklistBased on a scale of 1–5 (5 being the highest and 1 the lowest), participants in the post-workshop surveygave the Green Exhibit Checklist (GEC) high mean ratings in terms of its usefulness (4.14), relevancy towork (4.25), and applicability to practices (4.08). These ratings indicated successful acquisition of skillsassociated with environmentally responsible decision making. In the post-delayed surveys about half ofparticipants (49%) had used the GEC, while many others stated they have not yet had the opportunity.Considering the Three Pillars of SustainabilityBased on a scale of 1–5 (5 being the highest and 1 the lowest), participants in the post-workshop surveyrated moderate-to-high confidence in their ability to consider each of the three pillars with the highestrating in environmental (4.35), fairly high confidence in economic (4.00), and moderately highconfidence in the social (3.85), with fairly high confidence in the three pillars combined (4.05).Project Team and Host Museum Staff DebriefThe debrief session revealed that impressions among participants about the workshops and tools werelargely positive. Debrief comments identified the following highlights for attendees: the chance tonetwork with others in the industry around the sustainability topic, new channels to continue theconversation, and the possibility of additional or longer workshops in the future.Conclusions & Recommendations for the FieldGenerally, the deliverables achieved the desired impacts. Overall, participants reported positive andvaluable experiences with the workshops and tools. The largest value of exposure to the workshop andtools over time appears to be the integration of sustainability considerations into general thinking andthe approach to design and development. This impact on general thinking suggests the projectdeliverables helped to foster sustainable decision-making skills. Some participants reported barriers touse including uncertainty about how best to use the tools or limitations in the workplace due to lack ofsupport or absence of new development opportunities. The tools may be most useful for new projectsor opportunities. Participants’ ability to apply the three pillars approach was strong but varied by pillar.Confidence was strongest in the use the environmental pillar and weakest in use of the social pillar.While recollection and/or use of some tools diminished over time, the majority of participants reportedcontinued use of the ExhibitSEED website tools and the continued impact of the workshop on theirthoughts and decisions around sustainability.Specific recommended actions include the following: Continue the conversation. Promote more participation and encourage norm development. Advance and promote the workshop themes. Recognize and respond to regional, industry, and workplace barriers. Further evaluate tools and resources.Sustainability: Professional Development Summative Evaluation Report3

Introduction to the Sustainability ProjectAs part of the National Science Foundation funded Sustainability: Promoting Sustainable DecisionMaking in Informal Education project, the Oregon Museum of Science and Industry (OMSI) and itspartners developed a bilingual (Spanish/English) exhibition, event series, outreach campaign, andwebsite for the public. The goal of these deliverables was to promote sustainable decision making bybuilding skills that allow participants to weigh the tradeoffs of their choices and choose moresustainable practices. The project duration was September 2009 – April 2015.The project team used a positive, story-based approach to engage the target public audience ofEnglish- and Spanish-speaking families in the Portland metropolitan area. The deliverables focused onpeople and places in the area and highlighted simple choices available to everyone.The project also features professional audience deliverables that build upon the groundbreaking workalready done at OMSI in the area of sustainable exhibit development, design, and fabrication.Project Goals1. Promote sustainable decision making.2. Promote sustainable practices for developing, designing, and fabricating exhibits.Target AudiencesPublic Audience Families, with a focus on families with middle and high school aged youth Residents of the Portland metro area (Clackamas, Washington, and Multnomah counties) Underserved audience: parents who prefer to speak Spanish and their familiesProfessional Audience Exhibit developers, designers, and fabricators in non-profit and for-profit sectors Secondary audiences include museum administrators and others working in Informal ScienceEducation (ISE) organizationsProject Deliverables A 1,500 square foot bilingual (Spanish/English) exhibition, Clever Together/ Juntos somosingeniosos, designed to engage the public in developing an understanding of the skills needed forsustainable decision making in their everyday lives Bilingual (Spanish/English) outreach campaign that uses access points outside of the museum toencourage people to engage with sustainability-related stories and activities via phone orcomputer (Local Voices, Clever Choices/ Nuestras voces, nuestras decisiones campaign) Eight bilingual (Spanish/English) museum events about sustainable living ExhibitSEED workshops for museum professionals on sustainable exhibit development A set of documented practices and guidelines, tools, and resources for sustainable exhibitdevelopment, design, and fabrication that may become an industry standard (ExhibitSEED.org)Project Partners Coalition for a Livable Future Metro Regional Government Portland Bureau of Planning and Sustainability Portland Community College VerdeSustainability: Professional Development Summative Evaluation Report 4

Overview of ExhibitSEEDExhibitSEED (Exhibit Social Environmental and Economic Development) is a professional developmentwebsite and workshop created by OMSI during the course of the Sustainability project. ExhibitSEEDresources are based on a three pillars approach to sustainability that includes environmental, economic,and social considerations. The website (www.ExhibitSEED.org), launched in the fall of 2012, is a placefor exhibit and museum professionals to find resources for developing, designing, and building moresustainable exhibits. The resources on this website were developed from an interactive sciencemuseum perspective, but the information is valuable to exhibit professionals in all types of museums.The Oregon Museum of Science and Industry (OMSI) developed ExhibitSEED.org with input from localand national museum industry and design advisors. This team included the Museum AdvisoryCommittee (MAC) partners that also hosted ExhibitSEED workshops for museum professionals in theirregions.Museum Advisory Committee:Children’s Museum of HoustonThe Franklin InstituteMiami Science MuseumScience Museum of MinnesotaDesign Advisors:Madison Children’s MuseumGyroscope, Inc.Paul Orselli’s WorkshopNational Building MuseumThe intended impacts for the ExhibitSEED deliverables were:1. Through the ExhibitSEED workshops, participants will understand considerations for thedevelopment, design, and fabrication of more environmentally, economically, and sociallysustainable1 exhibits.2. Through the website, users will become aware of skills associated with environmentally,economically, and socially sustainable decision making in exhibit development, design, andfabrication.3. Through the Green Exhibit Checklist, the target audience will foster skills associated withenvironmentally responsible decision making in exhibit design and fabrication.1“Sustainability is commonly defined as the concept of providing society with current needs in a way that doesn’tharm the ability of future generations to meet their needs. Definitions often include the three componentsenvironment, economy, and society (sometimes referred to as the three “Ps”: planet, prosperity, and people).” –Trautmann, Charles H. (2007). A Change in the Weather: European Museum Coverage of Global Climate Change.Sciencecenter Report. p.3. Retrieved on October 3, 2014 fromhttp://informalscience.org/images/research/A Change in the Weather.pdf.Sustainability: Professional Development Summative Evaluation Report5

ExhibitSEED WebsiteThe resources included on the website and introduced in the workshops included the following: Sustainable Practices: Practical tips for incorporating social, economic, and environmentalconsiderations into each phase of exhibit development including proposal writing, projectmanagement, content research and development, design, prototyping and visitor testing,production, evaluation, and end-life Decision Making Tool: An activity intended to inspire a well-rounded conversation that leadsteams to decisions that consider all three (social, environmental, and economic) aspects ofsustainability The Green Exhibit Checklist: A tool to evaluate the environmental sustainability of exhibitproduction Materials Guide: An online guide to help exhibit designers and builders choose materials thatare better for the environment, safe for visitors and workers, and make economic sense Case Studies: Individual case studies about how museums have integrated the three pillars2 ofsustainability into their operations Envisioning Sustainability Activity: An activity designed to help teams explore the concept ofsustainabilityWorkshopsThe team hosted five one-day, hands-on ExhibitSEED workshops. At these workshops, exhibitdevelopers, designers, and fabricators discussed practical skills for creating more sustainableinteractive exhibits using case studies and practical tools. To encourage participation from diversedisciplines, institutions, and geographic areas, there was no charge for the workshop, and participantscould apply for travel assistance.The first workshop was conducted at OMSI in February 2013 and acted as a pilot workshop for remedialevaluation. The team used reflections from the OMSI workshop and the remedial evaluation report toupdate the workshop before conducting subsequent Museum Advisory Committee (MAC) workshops inApril and May 2013. MAC workshops were held at the Children’s Museum of Houston, Miami ScienceMuseum, Science Museum of Minnesota, and The Franklin Institute in Philadelphia. Each workshop hadapproximately 25 participants from the region surrounding the host museum. A total of 131 peopleregistered for the workshops from 110 institutions.Green Exhibit ChecklistA key element of ExhibitSEED was the Green Exhibit Checklist (GEC). The GEC was a tool to evaluatethe environmental sustainability of exhibits. The goal of the GEC was to inspire exhibit teams to planexhibits with five key environmental considerations in mind and extra points awarded for innovation: Reducing new materials Using local resources Reducing waste Reducing energy consumption Reducing toxic emissions2The United Nations General Assembly defines the three pillars as, “efforts [to] promote the integration of thethree components of sustainable development – economic development, social development and environmentalprotection – as interdependent and mutually reinforcing pillars.” – 2005 World Summit Outcome, ResolutionA/60/1. (15 September 2005). p.11-12. Retrieved on October 3, 2014 worldsummitoutcome resolution 24oct2005 en.pdfSustainability: Professional Development Summative Evaluation Report6

The initial outline of the GEC was developed in 2007 when designers at OMSI looked to the LEEDstandards for buildings to try to create a similar scoring system for exhibits. LEED assesses and assignsa numerical score for the environmental sustainability of building construction. However, the LEEDsystem has many categories that do not apply to exhibits and involves complex calculations thatseemed unnecessarily complicated for the purposes of museum professionals. The OMSI exhibitdesigners and production staff worked together to find a simpler approach based on basic material anddesign information. Since its creation, OMSI has used the GEC to assess past exhibit builds and to setsustainability goals for future projects. See Figure 1 for an example of the GEC cover sheet.Figure 1. Cover sheet for the Green Exhibit Checklist (GEC). A complete version of the GEC is includedin Appendix IV and online at Sustainability: Professional Development Summative Evaluation Report7

The Development of ExhibitSEEDExhibitSEED deliverables were developed based on the team’s experiences creating the Clever Togetherexhibition, literature research, partner and advisor input, and early project evaluation.Front-End and Remedial Project EvaluationThe purpose of the evaluation was to understand the professional audiences’ needs and concerns withregard to designing and fabricating more sustainable exhibits. The OMSI evaluation team conductedfront-end research to identify existing needs and interests of the field through targeted surveys ofindustry professionals in the winter of 2010, at the Spring 2010 Sustainability Symposium (a conferencehosted by OMSI to inform the Sustainability project), and through additional targeted survey researchin spring 2012 to obtain industry feedback on the ExhibitSEED website and Green Exhibit Checklist. Theevaluation team conducted remedial evaluation of the workshops and website content by conductingobservations and surveys with participants at the first ExhibitSEED workshop held at OMSI in February2013.Developing the DeliverablesIn the original National Science Foundation (NSF) grant, OMSI propose

Sustainability: Professional Development Summative Evaluation Report 5 Overview of ExhibitSEED ExhibitSEED (Exhibit Social Environmental and Economic Development) is a professional development website and workshop created by OMSI during the course of the Sustainability project. ExhibitSEED resources are based on a three pillars approach to sustainability that includes environmental, economic,