mopdf.com

IntroductionInvesting today in the employment ofyoung people means investing in thepresent and future of our societies.Guy Ryder, ILO Director-GeneralYoung people are estimated to account forover 35 per cent of the unemployed population worldwide (ILO, 2017a). While the globalyouth unemployment rate stabilized at 13 percent in 2016, it rose slightly to 13.1 per centin 2017. The estimated figure of 70.9 millionunemployed youth in 2017 is an important improvement from the crisis peak of 76.7 million in 2009, but this number is expected torise by a further 200,000 in 2018, reaching atotal of 71.1 million. More importantly, 39 percent of young workers in the emerging anddeveloping world – 160.8 million youth – areliving close to or in extreme poverty, i.e. onless than US 3.10 a day (at 2011 purchasingpower parity (PPP)). More than two in everyfive young people in today’s workforce areunemployed or are working but remain poor,a striking reality that is impacting societiesacross the world.Despite this challenging situation, the adoption of the Sustainable Development Goals(SDGs) in 2015, and their explicit focus on decent work, offers hope for young people making the transition to the world of work. The ILOand its constituents – governments, workers’and employers’ organizations – are workingtowards global targets on decent work and inclusive growth. The interests of young peopleare strongly represented in the SDGs, specifically in Goal 1 on fighting poverty, Goal 4on providing quality education and Goal 8 ondecent work and economic growth. The wellbeing of the most vulnerable young peopleis explicitly addressed in the Goal 8.6 target,“By 2020, substantially reduce the proportion of youth not in employment, educationor training”. For youth employment practitioners, this group of excluded and marginalizedyoung people is a focus of primary concern.The trajectory of this growing young generation will depend on their social and economicintegration, their ability to live fulfilling livesand their participation in and contribution tosociety.Taking effective action towards realizingthese SDGs requires a systematic approachto measurement and evaluation. The UnitedNations (UN) community has laid out a results-based reporting plan, designed to assess and monitor the realization of the SDGsand the contributions of specific UN agenciesand member States. In monitoring and evaluating SDG implementation, the Agenda 2030document provides guidance for systematicand rigorous follow-up processes (UN, 2015).The Agenda 2030 document recognizes thatthe review process for the SDGs requires“enhanced capacity-building support for developing countries, including the strengthening of national data systems and evaluationprogrammes”.This guide contributes to our collective efforts in moving towards fulfilling the SDGsand securing better youth employment prospects by promoting the implementation ofeffective results measurement systems inOVERVIEW3

GUIDE ON RESULTS MEASUREMENT OF DECENT JOBS FOR YOUNG PEOPLEyouth employment programmes and interventions. With an extensive section on attributingchanges to youth employment interventions,this guide fosters evidence-based policy andprogramming choices by equipping ILO partners and stakeholders with the tools to attribute their actions to positive youth relatedoutcomes. Guidance on results measurementof decent jobs for youth is provided with thepremise that, for jobs to be created, employment conditions to be improved and SDGs tobe realized, we need more evidence on “whatworks”. And, more importantly, can we say“how” and “why” interventions work in orderto inform future programme design?Although knowledge gaps remain, we cansafely say that the evidence base is strongerthan it was 10 years ago and it continues togrow at an impressive rate. The recently released systematic review on “Interventions toimprove the labour market outcomes of youth”(Kluve et al., 2017) identified 113 rigorousevaluation reports to include in its meta-analysis, 74 of which have been released since2010. The main findings of the review werealso encouraging: Overall, youth employmentinterventions can produce positive job andincome effects for young people. As globalevidence increases, so will the effectivenessof youth employment interventions, resultingin the creation of more decent jobs for youngpeople.This guide proposes a framework that linkspractical, implementation-focused measurement and monitoring to research-oriented impact evaluation. This is achieved in a seriesof seven “notes”, which lead readers throughthe key steps in diagnosing and formulatingyouth employment interventions, setting up aresults measurement system and defining corresponding key youth employment indicators.It then moves on to providing an overview ofthe types of evaluation approaches availableto measure youth employment outcomes, including non-experimental, quasi-experimental and experimental methods that encouragereflection on the challenge of attribution. Themenu of evaluation methods available is thendiscussed and, finally, guidance is providedon how to ensure that the findings from evidence-based evaluations are taken up indialogue and policy formulation processes related to employment and young people. Thetechnical notes are supplemented by interactive case studies supplied at the end of eachnote, to be used as a complement to classroom based learning on this topic.ABOUT THIS GUIDEObjectiveWith this guide, we aim to equip readers withthe basic set of concepts and tools neededto make informed decisions about how bestto measure and evaluate the results of youthemployment interventions. We review the entire life cycle of a youth employment intervention, starting with diagnostics and programmedesign, then moving on to setting up a monitoring system and measuring results. We seekto provide a clear understanding of the variety4OVERVIEWof evaluation options available and the issuesto consider in choosing the most appropriatemethod, given the learning objectives and operational context. Additionally, the guide describes how to manage an impact evaluation,if that is the assessment method of choice.Our overarching goal is to strengthen thefoundation of sound programming and policy-making by increasing the number of quality evaluations in the youth employment field,thereby facilitating the process of scaling up

and replicating successful interventions. Wewant to promote a perception of evaluationas a tool for both internal and external learning rather than as simply an accountabilityexercise. Using evaluation to learn about theperformance of an intervention can facilitateadaptive management where problems canbe recognized early and projects adjusted accordingly. Learning that is shared publicly canhelp others to select the most effective intervention models, ensuring that resources spenton youth employment are optimally used.Learning toolThe guide can also be used to complementlearners taking training courses on resultsmeasurement and what works in youth employment. From 2010 to 2017, the ILO hastrained over 2,000 constituents on resultsmeasurement in youth employment using anexperiential learning approach called “Evaluation Clinics”. This guide closely follows thelearning modules and curriculum of the ILOEvaluation Clinics but provides learners withmore in-depth information and tools to buildon the knowledge gained in the Clinics.AudienceWhile this guide is primarily a reference toolfor ILO constituents, it can also be used by allthose involved in the implementation of youthemployment programmes.The guide has therefore been drafted with thefollowing organizations in mind.Primary audiencesGovernments: Particularly ministries andagencies which focus on employment, youth,labour market services and training. The information may be more relevant to agenciesand institutions which are directly responsible for delivering and monitoring differentprogrammes, such as youth employmentagencies, public employment service agencies and technical and vocational training institutes. It can be used to assist in designingnew public employment programmes.Employers’ organizations: Organizationsrepresenting the interests of private employers who offer youth vocational education,employment and entrepreneurship opportunities. They will be able to use the guide bothto inform their advocacy on important legislative and regulatory issues and to offer valueadded services and advice to their members,based on the latest research and evidence.Companies have an increasingly large stakein issues involving youth and making surethey are prepared with the requisite skills andattitudes to become the employers and employees of the future.Workers’ organizations: Workers’ councils and trade unionists who are interested inthe potential of youth employment interventions for realizing decent work for youth. Theguide will be particularly useful in formulatingprogrammes and interventions for evidencebased advocacy around key policy issues.Workers’ organizations have an interest inlearning about the conditions under whichemployment, and in particular decent employment, can be increased.Youth-led organizations: not-for-profit organizations working in the youth employmentfield whose staff and members are predominantly made up of young people. Most organizations use the UN’s definition of youth whichis a person between the ages of 15 to 24.Secondary audiencesCivil society: NGOs are one of the main implementers of youth employment programmesOVERVIEW5

GUIDE ON RESULTS MEASUREMENT OF DECENT JOBS FOR YOUNG PEOPLEin many countries. As the guide covers the fulllife cycle of a youth employment intervention,it can be used at every stage of a project. Thefocus on evaluation and evaluation methodsallows intervention results to be translated intolessons to inform both follow-up projects andother practitioners.Donors: Multilateral and bilateral development agencies provide a considerable shareof the funding for youth employment programmes. Donors, who have a fiduciary interest in maximizing the impact of their grantfunding and ensuring accountability in theuse of resources, will be able to use the guideto set appropriate evaluation schedules, aswell as to design and commission evidencebased youth programmes.Researchers: Researchers conducting fieldresearch and liaising with programme managers and implementing organizations willfind the information presented in this guiderelevant for adapting standard research methods to local programme conditions. A context-sensitive approach to impact evaluationis presented that facilitates research design.Key referencesThis guide complements existing materialson results measurement, applying them to thespecific area of youth employment. It buildson the following works:XX Measuring success of youth livelihoodinterventions: A practical guide to monitoring and evaluation (Hempel and Fiala, 2011).6OVERVIEWAuthored as a contribution to the GlobalPartnership for Youth Employment, this workwas used as the key reference document.XX ILO Policy Guidelines for Evaluation (ILO,2017b) and ILO Development CooperationManual (ILO, 2015), which set out the overall framework for project monitoring, reporting and evaluation in an ILO context.XX Monitoring and evaluation of youth employment programmes: A learning package(ILO, 2013). This guidebook provides adviceon monitoring the performance of youthemployment programmes and measuringboth short- and long-term outcomes.XX Practical toolkits that emphasize generalmonitoring and evaluation (e.g. Gospariniet al., 2003; Kellogg Foundation, 2004;Kellogg Foundation, 2017, 2004) and otherpublications that focus specifically on impactevaluation (e.g. Baker, 2000; Duflo et al.,2006; Gertler et al., 2016; Khandker et al.,2010; Ravallion, 2008).Considering similar resources on the topicof monitoring and evaluation, our Guideis unique in that it is the only guidance toolthat orients all elements of the results measurement life cycle to the topic of youth employment. These unique elements include achapter advising practitioners on the appropriate youth employment indicators to select(Note 2) ; a list of challenges, and their solutions, that are specific to youth employmentfocused impact evaluations (Note 5); and areflection on the existing body of youth employment evidence and advice on how thisevidence can lead to policy change (Note 7).

OVERVIEW OF THE GUIDE AND HOW TO USE ITWhile the guide leads the reader through allstages involved in formulating youth employment programmes, starting with the diagnostic phase and closing with evidence uptake inyouth employment policy formulation, the mainthrust of the guide is on monitoring, resultsmeasurement and evaluation. Figure 0.1 setsout a simple results measurement cycle andthe relating key elements of the seven notes.measurement. The second note covers labourmarket indicators with particular relevance foryouth employment interventions, while Note 3concentrates on setting up the monitoring system. Note 4 introduces readers to the importance of evaluation before presenting differentapproaches, including performance and impact evaluations. In order to address the challenge of attribution, Note 5 presents a numberof different methods which can help decisionmakers weigh the desired level of rigour withthe feasibility of conducting the research. Note6 guides readers through the step-by-stepThe first note focuses on designing a youth employment intervention and establishing a solidtheory of change as a basis for quality resultsFIGURE 0.1 RESULTS MEASUREMENT CYCLENote 1:Diagnose thechallengeNote 7:Uptake ofevidenceNote 2:SelectindicatorsNote 3:Set-up themonitoringsystemPlanEvaluateand learnNote 4 & 5:Plan theevaluationNote ntNote: 6AnalyzeresultsNote: 6CollectdataOVERVIEW7

GUIDE ON RESULTS MEASUREMENT OF DECENT JOBS FOR YOUNG PEOPLEprocess of implementing a youth employmentfocused impact evaluation. The guide ends byoffering practical advice on ensuring that evaluation evidence is taken up in policy formulation processes.Although it is important to be familiar with allparts of the measurement process, it is notnecessary to read the guide from beginning toend. Instead, each note is conceived as a standalone entity that can be read independently ofthe others, according to each reader’s needs.Table 0.1 indicates which notes are most relevant to different types of readers.Case studiesAt the end of each of the seven notes, we present case studies detailing how different youthemployment interventions apply results measurement strategies. The case studies illustratethe main points in each note and ask readersto apply the concepts they have learned to“real world” situations. The case studies aredesigned to complement classroom-basedlearning, as pedagogic exercises to be discussed in small groups with the assistance ofan expert or facilitator in results measurementin youth employment.While all of the case studies relate to youth labour market interventions, some of them arederived from experiences of the ILO in supporting organizations which are seeking toimprove their results measurement systemsand implement impact evaluation projects.Several of these case studies are drawn fromorganizations that were offered support under ILO’s Fund for Evaluation in Employment,a technical and financial support programmefor youth employment researchers and organizations in the Arab States and Africa regions.The case studies are accompanied by an appendix “Answer key for case studies”, whichis available separately from the seven notevolume of Guide. This answer key appendix8OVERVIEWis intended to be used by facilitators to assist in small group discussions about the casestudies.Overview of key termsThe guide addresses the effective monitoringand measurement of outcomes of youth employment interventions with a specific focuson impact evaluation. It is important here tomake reference to four key terms which areused extensively throughout the guide: resultsmeasurement system, monitoring, evaluationand impact evaluation.Results measurement system refers to theoverall processes, plans, tools and resourcesthat are used to determine whether a programme has been implemented accordingto the plan (monitoring) and is having the desired result (evaluation).1 A results measurement system specifies:XX indicators to be trackedXX milestones (mid-stream) and end targetsXX data collection toolsXX the personnel who will gather, record andanalyse the data andXX the types of reports that will be prepared,including for whom, why and how often.The key activities of a results measurementsystem are monitoring and evaluation:Monitoring tracks the implementation andprogress of an intervention in order to supportprogramme management. Monitoring:XX involves the collection of data on specificimplementation and results indicatorsXX assesses compliance with work plans andbudgetsXX uses information for project managementand decision-makingXX is ongoing1Also known as a monitoring and evaluation (M&E)system.

Table 0.1 Reader’s guideNoteTitle1Diagnosing,planning anddesigning youthemploymentinterventions2Concepts anddefinitions ofemploymentindicatorsrelevant for youngpeople3Establishing amonitoringSystem4Enhancing youthemploymentlearning throughevaluation5Impact evaluationmethods foryouthemploymentinterventions6A step-by-stepguide to impactevaluation7Evidence uptakein policyformulationDescriptionGuides readers throughan employment diagnosticanalysis as the basis fordeveloping the theory ofchange then proceeds toprogramme designReviews concepts anddefinitions of employmentindicators relevant foryoung people in the areasof employment opportunities, employment quality,employment access andemployment skillsPresents the main steps indeveloping a resultsmeasurement system,including how to collectand analyse data.Asks which type of evaluation best suits an individual programme. The answerdepends on evaluationquestions, the context andcharacteristics of the project, and availableresourcesPresents the main featuresof an impact evaluationand focuses on finding agood comparison group toreliably demonstrateimpact. Presents different(quasi-)experimentalmethods for conductingan impact evaluationMoves from the conceptual to the practical level,describing the major stepsinvolved in carrying out animpact evaluation and providing practical resources.These steps cover theentire process, from initialpreparations to the dissemination of resultsHelps readers to plan howevaluation results can beused to influence policyand improve programming. Communication ofresults and stakeholderengagement strategiesare discussedCase studyGovern- Employers/ Civil ResearDonorsments workers society chersLabour market diagnostics for the promotion of rural youthlivelihoods in Zambia Selecting indicatorsfor the NorthernUganda YouthEntrepreneurshipProgramme Establishing a monitoring system for theJordan EconomicGrowth andEmployment Project Developing Terms ofReference for a midterm evaluation of ayouth employmentproject in Egypt Assessing ruralmicroenterprisegrowth through different evaluationmethods Survey design andimplementation forNeqdar Nesharek inEgypt Uptake of evidenceon the effects of skillstraining on youngpeople’s financialbehaviou

The Guide on Measuring Decent Jobs for Youth represents a collaborative effort involv-ing expertise from the International Labour Organization (ILO) and its main constituents as well as from external partners. The Guide is a product of the ILO's Youth Employment Programme Unit (YEP) and is developed in close collaboration with the other branches