Transcription

Paper ID #15701A Rubric to Assess Civil Engineering Students’ Grand Challenge SustainableEntrepreneurship ProjectsDr. Claire L. A. Dancz, Clemson UniversityClaire L. A. Dancz is a Postdoctoral Research Fellow in Civil Engineering and online active experientiallearning and assessment with Clemson Online at Clemson University. Dr. Dancz received her B.S. inEnvironmental Microbiology and Biology from Michigan State University, her M.S. in Civil Engineeringfrom University of Pittsburgh, and Ph.D. in Sustainable Engineering from Arizona State University. Herareas of research include modular, course, and blended models for integrating sustainability into civilengineering programs, entrepreneurship for engineering grand challenges and service-learning, and assessment in engineering education. Dr. Dancz has developed and evaluated open-access online active andexperiential learning activities that immerse engineering students in sustainability and enable students toexercise their voice in solving grand challenges. As a Kolbe R certified consultant, Dr. Dancz utilizesconation and team science to recruit and retain students with diverse problem-solving instincts to improvecommunication, leadership, and impact the diversity of engineers as global change-makers.Dr. Jeffery M Plumblee II, Clemson UniversityJeff Plumblee, PhD, MBA is a Postdoctoral Research Fellow in online service-learning at Clemson University. Plumblee founded the award winning Clemson Engineers for Developing Countries (CEDC) in2009 while pursuing a doctorate in civil engineering. He has helped to grow the organization to 100 students per semester, including 2-5 interns living in Haiti year-round. The program has overseen in excessof 2 million in sustainable infrastructure and economic development projects in Haiti. He is currently exploring ways to offer similar opportunities to a wider audience, including bringing the CEDC model intoa domestic context, leveraging technology to virtually link students with service-learning opportunitiesand resources throughout the world, and starting a design challenge for high school students to addressthe needs of the less fortunate.Dylan Bargar, Clemson UniversityDr. Penelope Walters Brunner, Clemson UniversityDR. PENELOPE BRUNNER is the Director of Assessment and Planning for Clemson’s College of Engineering. In this role, she works with academic departments and administrative offices on assessmentreporting and strategic planning alignments.Prior to joining Clemson, Dr. Brunner was an Associate Vice President at the College of Charleston. Asan associate professor within the University of North Carolina system, she taught courses in Managementand Management Information Systems. Her national and international consultancies involve working witha variety of accreditation agencies including Middle States, Western Association, AACSB, and NCATE.A native Oklahoman, Dr. Brunner holds MA, MBA, and EdD degrees from the University of Tulsa.Dr. Karen A High, Clemson UniversityDr. Karen High is the Associate Dean for undergraduate studies in the College of Engineering and Scienceat Clemson University. She also holds an academic appointment in the Engineering Science and Educationdepartment and joint appointments in the Chemical and Biomolecular Engineering department as well asthe Environmental Engineering and Earth Sciences department. Prior to this Dr. Karen was at OklahomaState University where she was a professor for 24 years and served as the Director of Student Servicesas well as the Women in Engineering Coordinator. She received her B.S. in chemical engineering fromUniversity of Michigan in 1985 and she received her M.S. in 1988 and her Ph.D. in 1991 in chemicalengineering both from Pennsylvania State University. Dr. Karen’s educational emphasis includes: criticalthinking, enhancing mathematics, engineering entrepreneurship in education, communication skills, K-12engineering education, and promoting women in engineering. Her technical work and research focuseson sustainable chemical process design, computer aided design, mixed integer nonlinear programing, andmulticriteria decision making.c American Society for Engineering Education, 2016

Paper ID #15701Dr. Leidy Klotz, Clemson UniversityLeidy Klotz is an engineering faculty member at Clemson University, where he developed and teachescourses like the one described in this paper. He does research on decision making and education forsustainability.Prof. Amy E. Landis, Clemson UniversityDr. Landis joined Clemson in June 2015 as the Thomas F. Hash ’69 Endowed Chair in SustainableDevelopment. Previously she was an Associate Professor at Arizona State University in the School ofSustainable Engineering and the Built Environment. She began her career as an Assistant Professor at theUniversity of Pittsburgh, after having obtained her PhD in 2007 from the University of Illinois at Chicagounder the supervision of Dr. Thomas L. Theis. She has developed a research program in sustainableengineering of bioproducts. Her research ranges from design of systems based on industrial ecology andbyproduct synergies, life cycle and sustainability assessments of biopolymers and biofuels, and designand analysis of sustainable solutions for healthcare. Since 2007, she has lead seven federal researchprojects and collaborated on many more, totaling over 7M in research, with over 12M in collaborativeresearch. At ASU, Dr. Landis continues to grow her research activities and collaborations to includemultidisciplinary approaches to sustainable systems with over 60 peer-reviewed publications. Dr. Landisis dedicated to sustainability engineering education and outreach; she works with local high schools, afterschool programs, local nonprofit organizations, and museums to integrate sustainability and engineeringinto K-12 and undergraduate curricula.c American Society for Engineering Education, 2016

A Piloted Rubric to Assess Civil Engineering Students’Grand Challenge Sustainable Entrepreneurship ProjectsAbstractTo prepare the next generation of civil engineers to tackle 21st century challenges, engineeringeducation must commit to deepening engineer’s social consciousness through exposure tosocietal problems in addition to teaching technical competencies. The National Academy ofEngineering (NAE) Grand Challenges for Engineering offers a framework for exposing studentsto the role of a modern engineer and the complex global challenges that require engineeringintervention. In response to these challenges, many U.S. engineering schools have adopted theGrand Challenge Scholars (GCS) program to educate a new generation of engineeringprofessionals equipped to sustainably address society’s most imminent problems. This paperpresents the development of a holistic rubric to assess student scholarship and informcompetencies related to Grand Challenges. The rubric builds on best practices in assessment andevaluation of the five key NAE GCS program components, including 1) hands-onproject/research experience, 2) interdisciplinary curriculum, 3) entrepreneurship, 4) globaldimension, and 5) service-learning. The authors discuss potential applications of the rubric toevaluate course-level outcomes, including student projects from an interdisciplinary courseentitled “Creatively Applying Science for Sustainability.” In the course, students work to addressa societal Grand Challenge in a semester-long project and in interdisciplinary student projectsthat tackle Grand Challenges on an international scale. This rubric fills a literature gap inassessing 21st century global engineering skills by measuring capabilities based on five key NAEGCS program components and provides a mechanism to understand and influence the quality ofstudent education and experiences within Grand Challenge-focused courses and programs.IntroductionThe next generation of engineering professionals must be prepared to solve complex andmultidisciplinary problems in a sustainable and global context. Engineering education canprovide students with the tools to approach these grand challenges of the 21st century whileconsidering aspects that are key for designing sustainable systems.1 Despite this, engineeringeducation faces several challenges, including, but not limited to, addressing low diversitypercentages, high attrition rates, and the need to better engage and prepare students for the role ofa 21st century engineer.2Since the 1970s the representation of women in Science, Technology, Engineering andMathematics (STEM) occupations has grown unevenly from 3% to 26%.3 While the percent ofwomen in math and science has continued to grow, growth in engineering has stagnated around13% since 1990.3 Also, the number of bachelor’s degrees awarded in science and engineeringhas increased, while the percentage of women earning bachelor’s degrees in computer scienceand engineering has decreased in the last 10 years.4 In addition, while underrepresented

minorities account for more than 30% of the total United States’ workforce, only 12% areenrolled in science and engineering undergraduate degree programs and 16% are employed insome STEM occupations.3,4 The President’s Council of Advisors on Science and Technology(PCAST) and relevant research recommend that creating an educational experience wherestudents have a connection to their degree and a connection to their technical community cancontribute to increasing diversity in STEM. Sustainability is one theme that can create thisconnection for many students. Research indicates that students who hope to addresssustainability issues related to energy, water, and the environment demonstrate increased interestin pursuing engineering degrees; increasing the connection between sustainability andengineering could broaden participation of underrepresented populations, including women.5Furthermore, fewer than 40% of students enrolled in STEM majors complete their degree.2,6There are many reasons for a student to move from STEM to another discipline, includingintellectual compatibility and institutional support.7 However, according to a recent NationalAcademy of Science report, Changing the Conversation, one of the most significant contributingfactors to high attrition rates is that courses no longer appeal to our youth.8,9Youth are seeking careers that can make a difference, thus strategies for engineering educationneed to bring exciting topics and engaging methods into the classroom to motivate studentstoward goals that matter to them. Sustainable engineering offers a solution to these pressingchallenges by providing context for the role of a modern engineer in solving 21st centuryproblems. Sustainability topics in engineering curricula can address many of the underlyingfactors facing diversity and retention of students who otherwise leave STEM majors due to lackof engagement and/or motivation.5The National Academy of Engineering (NAE) developed and issued the Grand Challenges forEngineering, with five of the fourteen directly related to sustainability (solar energy, carbonsequestration, nitrogen cycle, clean water, and infrastructure).10 The Grand Challenges offer aframework for exposing engineering students to the role of an engineer in modern society. TheWhite House Strategy for American Innovation and the United Nations MillenniumDevelopment Goals have identified many of the Grand Challenges as global challenges that willrequire diverse, innovative solutions.11,12 Adoption of these challenges within engineeringcurricula has been cited to engage a diverse array of interested students by establishingcontextualized linkages between course content and the contributions an engineer makes to solveglobal issues through systems-thinking innovation.13Having acknowledged the need for graduates trained in solving 'Grand Challenge'-scaleproblems, a natural outcome was to develop a university program to facilitate the training. TheGrand Challenge Scholars (GCS) program was created and adopted by Duke’s Pratt School ofEngineering, The Franklin W. Olin College of Engineering, and the University of SouthernCalifornia’s Viterbi School of Engineering. The program has since expanded; over the next

decade 122 schools across the country have pledged to graduate at least 20 students specificallytrained in solving large-scale problems like the Grand Challenges.14The GCS program was developed such that each school could develop its own methods forstudent fulfillment of five program competencies. These five GCS program competencies areshown in Figure 1. The program competencies within the GCS program are intended to providethe foundation for graduates to tackle large-scale challenges, such as the 14 outlined in the NAEGrand Challenges for Engineering.14Hands-on Project or Research Experience Related to a Grand ChallengeInterdisciplinary Curriculum A curriculum that complements engineering fundamentals with courses in otherfields, preparing engineering students to work at the overlap with public policy,business, law, ethics, human behavior, risk, and the arts, as well as medicine and thesciencesEntrepreneurship Preparing students to translate invention to innovation; to develop market venturesthat scale to global solutions in the public interestGlobal Dimension Developing the students’ global perspective necessary to address challenges that areinherently global as well as to lead innovation in a global economyService Learning Developing and deepening students’ social consciousness and their motivation tobring their technical expertise to bear on societal problems through mentoredexperiential learning with real clientsFigure 1. Grand Challenge Scholars Program Competencies.Since the Grand Challenge Scholars (GCS) program takes different forms at different institutions(and even different forms for students within an institution), it is critical to create a mechanismfor standardizing the five program competencies within the GCS program. Rubrics can be usedto promote student learning, improve instruction, and support effective programs because theymake expectations and criteria explicit.15 The proposed rubric in this paper serves as a qualitycontrol mechanism, ensuring that each student participant in the GCS program fulfills aminimum level of rigor and/or experience in each of the five program competencies.

Rubric DevelopmentThe GC Rubric was developed by mining best practices in the literature on assessment andevaluation of the five GCS program competencies, including 1) hands-on project/researchexperience, 2) interdisciplinary curriculum, 3) entrepreneurship, 4) global dimension, and 5)service-learning. Criteria for all five GCS program competencies were generated such that GCcompetencies are measured based on student project assessments (shown Table 1). The first GCcompetency, hands-on project/research experience, rubric components include documentation ofresearch methods, such as problem identification, data collection, and analysis of findings.16,17Interdisciplinary curriculum, the second GC competency, rubric components containdemonstrating the relationship between multiple disciplines such as showing the potentialconflict of the same problem viewed from two different perspectives.18,19 Rubric criteria for theentrepreneurship GC competency considers critical thinking, customer-appropriate valuepropositions, effectively delivering final product and the relation of personal liberties toentrepreneurship.20-22 The global dimension GC competency covers the temporal scale ofcontemporary Grand Challenges and was assessed through understanding global systems,cultures, and a student’s personal role of social responsibility.23-25 The fifth GC competency,service learning, assesses students’ civic action, service to others, and understanding of differingperspectives of communities intimately affected by Grand Challenges.26-28The rubric is applied using the following four metrics: “does not meet expectations”characterizes a student performance that does not display any of the desired activity,“developing” characterizes student performance that displays some of the desired target activity,“meets expectations” characterizes student performance that displays the minimal level of abilityexpected, and “proficient” designates student performance that exceeds “meets expectations” andevidences mastery of the target activity.20 Students are scored by two external evaluators withexpertise in Engineering Grand Challenges. The external evaluators viewed students’ finalpresentations in which students presented a comprehensive overview of the problems, thecommunity stakeholders they engaged, their process for addressing the problem, and their finalsolution. The two evaluators agreed on final scoring while applying the rubric and viewing thepresentation together.

Table 1. Grand Challenge Scholars (GCS) Rubric for Evaluating Student WorkGCS ProgramCompetency1. Hands-onProject/ ResearchExperience2. InterdisciplinaryCurriculum3. Entrepreneurship4. Global Dimension5. Service LearningRubric b.c.d.Identify the problemCollect data with supporting methodologyAnalyze data and generate resultsPresent conclusions and applications of project/research findingsDiscuss problem from multiple perspectivesShow connections between two or more disciplinesIntegrate conflicting insights from two or more disciplinesDemonstrate interdisciplinary understanding of the problemCollaborate as a teamApply critical and creative thinking to ambiguous problemConstruct customer-appropriate value propositionPersist and learn through failureEffectively manage projects through final delivery processDemonstrate social responsibilityRelate personal liberties to entrepreneurshipDemonstrate global and cultural self-awareness and curiosityEngage and learn from global culturesDevelop intercultural sensitivity and empathyRecognize personal and social responsibilityUnderstand global systemsDefine civic action and reflect on personal roleConnect and extend knowledge to civic engagement and serve othersCommunicate differing perspectives of communities and culturesCollaboratively work across and within a community to provide a serviceRef16,1718,1920-2223-2526-28The rubric was created by the authors for this study and for use at Clemson University toevaluate GCS projects based on the 5 GCS program competencies. The rubric criteria weremined and adapted from best practices in the literature.Rubric ApplicationTo demonstrate the use of the proposed rubric, the following section describes two real-world,Grand Challenge-themed student projects, which have been evaluated by the rubric to assesswhether they fulfill any, or multiple, GCS program components based on the GCS rubric.Authors applied the rubric to select students’ semester projects from the Fall 2015 ClemsonUniversity course entitled "Creatively Applying Science for Sustainability." The coursecombines sustainability concepts with applications in students' design ventures in aninterdisciplinary, flipped-course setting. Students chose a wide variety of challenges to addresswithin the design venture, including local and international issues, with topics ranging fromaddressing infrastructure to human health to sustainability education. Students work through thesix milestones project assignments in tandem with the six course themes: Our Grand Challenges;

Systems and Sustainability; Evaluating Sustainability; Creating- Sustainable Design Process;Creating- Sustainable Design Principles; and Creating- Finding Deep Simplicity. For thesemester project, students first identify their Grand Challenge and, optionally, form teams. Eachunit is required to perform background research to understand the Grand Challenge, its impact onsociety and stakeholders, inherent cultural or ethical considerations, and relevant cause-andeffect relationships. Students then define minimum requirements for success and constraints,create a best-case scenario, and develop criteria for which they can evaluate solutions. Afterwhich, students brainstorm and define a possible solution and they begin to design and solicitfeedback from stakeholders, peers, and experts. Students then refine their prototype, consideroperations and maintenance, generate a basic business model, and continue improving theirsolution. At the end of the semester students present their solution, reflect on their experience,and develop a path forward. The authors piloted the rubric by assessing two student groupprojects. The first one, entitled “Economic Use of CMU Blocks,” explored the recycle of wastematerials into concrete masonry units (CMU), the impact on material quality, and the potentialuse of these units to support Haiti’s infrastructure. The second, entitled “Banana Bags”, exploredthe use of banana fibers to create bags in Cameroon and address the plastic bag black market.“Economic Use of CMU Blocks” student group attempted to find a solution for Haiti's poorinfrastructure and waste management issues. They explored the possibility of using wasteproducts in building CMUs, a common construction material in Haiti. By working with ClemsonEngineers for Developing Countries (a student organization actively working in Haiti), theydefined a problem and connected students with Haitian stakeholders. Faculty members from theGlenn Department of Civil Engineering gave advice concerning materials andfeasibility. Students determined waste availability, obtained an understanding of potentialadditives to strengthen or serve as substitutes in concrete, and developed plans to test certainprevalent waste products in CMUs. At the end of their project, students anticipated obstacles thatcould prevent the project from continuing, including the possibility that Haitians may use thehigher strength block due to higher costs and lack of central waste management system or evenbasic incentives for collecting waste.The “Banana Bags” project attempted to find a solution for Cameroon’s plastic bag blackmarket. The student leading the project, a native of Cameroon, explored the unintendedconsequences of a national banning of plastic bags in 2014. Plastic bags are primarily used in thetransport of groceries and goods. When the Cameroon government banned plastics bags, withoutany alternative object to substitute for the bags, a black market was created. This studentidentified a potential industrial symbiotic relationship between the plastic bag black market andCameroon’s chief export of bananas. Banana fibers remaining from the production of bananasexhibit properties excellent for weaving into baskets and bags. This student proposed to use thewaste output of one export as the input to the black market plastic bag import as a solution thatnot only addressed the issue but also provided jobs and monetary flows for locals.

The GCS rubric application of the Creatively Applying Science for Sustainability Fall 2015project, “Economic Use of CMU Blocks,” revealed that students perform ‘proficiently’ by goingabove minimal expectations in seven assessment criteria; students ‘meet expectations’ for twelveand reach ‘developing’ for four assessment criteria and “does not meet expectations” for one outof twenty-four assessment criteria (Table 2). It is anticipated that students perform at the ‘meetsexpectation’ level for each GC program competency at minimum. The GC rubric application ofthe “Banana Bags” project revealed that the student perform ‘proficiently’ by going aboveminimal expectations in nine assessment criteria; students ‘meet expectations’ for nine and reach‘developing’ for four assessment criteria, and “does not meet expectations” for two out oftwenty-four assessment criteria (Table 3). The rubric assessment of both student project revealsareas in which students excel, by a function of their own effort, by the nature of the problem theyare attempting to solve, or both. It also reveals the areas where weakness may lead to theinability of the project to address a Grand Challenge holistically.

Table 2. Rubric Assessment of Creatively Applying for Sustainability Fall 2015 student project“Economic Use of CMU Blocks” in Haiti.1. Hands-onProject/ ResearchExperience2. InterdisciplinaryCurriculum3. Entrepreneurship4. Global Dimension5. Service LearningStudent Project Assessment b.c.Identify the problemCollect data with supporting methodologyAnalyze data and generate resultsPresent conclusions and applications of project/research findingsDiscuss problem from multiple perspectivesShow connections between two or more disciplinesIntegrate conflicting insights from two or more disciplinesDemonstrate interdisciplinary understanding of the problemCollaborate as a teamApply critical and creative thinking to ambiguous problemConstruct customer-appropriate value propositionPersist and learn through failureEffectively manage projects through final delivery processDemonstrate social responsibilityRelate personal liberties to entrepreneurshipDemonstrate global and cultural self-awareness and curiosityEngage and learn from global culturesDevelop intercultural sensitivity and empathyRecognize personal and social responsibilityUnderstand global systemsDefine civic action and reflect on personal roleConnect and extend knowledge to civic engagement and serve othersCommunicate differing perspectives of communities and culturesd.Collaboratively work across and within a community to provide a serviceMeetsExpectationsProficientGC RubricDoes Not MeetExpectationsDevelopingStudent Performance

Table 3. Rubric Assessment of Creatively Applying for Sustainability Fall 2015 student project“Banana Bags” in Cameroon.1. Hands-onProject/ ResearchExperience2. InterdisciplinaryCurriculum3. Entrepreneurship4. Global Dimension5. Service b.c.Identify the problemCollect data with supporting methodologyAnalyze data and generate resultsPresent conclusions and applications of project/research findingsDiscuss problem from multiple perspectivesShow connections between two or more disciplinesIntegrate conflicting insights from two or more disciplinesDemonstrate interdisciplinary understanding of the problemCollaborate as a teamApply critical and creative thinking to ambiguous problemConstruct customer-appropriate value propositionPersist and learn through failureEffectively manage projects through final delivery processDemonstrate social responsibilityRelate personal liberties to entrepreneurshipDemonstrate global and cultural self-awareness and curiosityEngage and learn from global culturesDevelop intercultural sensitivity and empathyRecognize personal and social responsibilityUnderstand global systemsDefine civic action and reflect on personal roleConnect and extend knowledge to civic engagement and serve othersCommunicate differing perspectives of communities and culturesd.Collaboratively work across and within a community to provide a serviceMeetsExpectationsProficientStudent Project Assessment CriteriaDevelopingGC RubricDoes Not MeetExpectationsStudent PerformanceFuture DirectionThough this rubric is still in the development phase, authors intend to include the rubric as anassessment piece of a new Grand Challenge minor at Clemson University, as well as to integrateit into our existing GCS program. This will help to ensure that students in the GCS programand/or Grand Challenge minor fulfill each of the program competencies, serving as a qualitycontrol mechanism to provide consistency among student experiences.In addition to analyzing the fulfillment of program objectives on an individual student basis,authors plan to develop a similarly structured rubric to assess entire existing programs, courses,and organizations on campus that are interested in whether their student experiences fulfill aprogram competency (or multiple program competencies) of the GCS program. These rubricswill have the ability to work in tandem; to qualify an entire program, course, or organization, it isnecessary to review the requirements of each student, as well as to assess a representative sampleof final deliverables within the rubric to determine if all deliverables are expected to meet

program competencies. Authors expect to garner interest from Clemson's Engineers WithoutBorders chapter, Clemson Engineers for Developing Countries, the proposed Grand Challengeminor offering, and Grand Challenge Scholars program at Clemson University, among others.In the future the evaluators will utilize Inter-Rater Reliability (IRR) best practices to understandthe impact of the evaluators on rubric results as additional evaluators are engage with the rubric.IRR is defined as the process through which two or more raters classify subjects or objectsindependent of one another.29 High IRR verifies that the raters can be used interchangeably,thereby establishing the rater as an abstract entity to the main focus of study, the subjects.29,30Furthermore, the evaluators plan to examine the impact of the rubric on student learning byestablishing a control course without introduction to the rubric and experimental course thatintroduces and integrates the rubric throughout the semester.Beyond Clemson, the rubric can be used by any institution interested in assessing any or all ofthe five program competencies of the GCS program, whether or not they currently have a GCSprogram. This application could be useful for institutions that already have a GCS program tocreate a cross-institutional minimum standard to ensure that NAE program competencies arefulfilled. For institutions that are considering starting a GCS program, this rubric could assesstheir potentially appropriate programs to determine the institutional readiness for a GCS programand highlight any deficiencies or gaps that may need to be filled, as well as potential institutionalstrengths. Finally, the rubric coul

Dr. Jeffery M Plumblee II, Clemson University Jeff Plumblee, PhD, MBA is a Postdoctoral Research Fellow in online service-learning at Clemson Uni-versity. Plumblee founded the award winning Clemson Engineers for Developing Countries (CEDC) in 2009 while pursuing a doctorate in civil enginee