WHO/TCM/MRS/2007.2Regulatory Support Series, No.12Practical Guidance forConducting a Review(based on the WHO Data Collection Toolfor the Review of Drug Regulatory Systems)Geneva2007

WHO Guidance for the assessment of drug regulatory systemsThis document has been produced with the financial assistance of the European Community. Theviews expressed herein are those of the authors and can therefore in no way be taken to reflect theofficial opinion of the European Community. World Health Organization 2007All rights reserved. Publications of the World Health Organization can be obtained from WHO Press,World Health Organization, 20 Avenue Appia, 1211 Geneva 27, Switzerland (tel.: 41 22 791 3264;fax: 41 22 791 4857; e-mail: [email protected]). Requests for permission to reproduce or translateWHO publications – whether for sale or for noncommercial distribution – should be addressed to WHOPress, at the above address (fax: 41 22 791 4806; e-mail: [email protected]).The designations employed and the presentation of the material in this publication do not imply theexpression of any opinion whatsoever on the part of the World Health Organization concerning thelegal status of any country, territory, city or area or of its authorities, or concerning the delimitation ofits frontiers or boundaries. Dotted lines on maps represent approximate border lines for which theremay not yet be full agreement.The mention of specific companies or of certain manufacturers’ products does not imply that they areendorsed or recommended by the World Health Organization in preference to others of a similarnature that are not mentioned. Errors and omissions excepted, the names of proprietary products aredistinguished by initial capital letters.All reasonable precautions have been taken by the World Health Organization to verify the informationcontained in this publication. However, the published material is being distributed without warranty ofany kind, either expressed or implied. The responsibility for the interpretation and use of the materiallies with the reader. In no event shall the World Health Organization be liable for damages arisingfrom its use.Printed in (for external printing)Printed by the WHO Document Production Services, Geneva, Switzerland

WHO Guidance for the assessment of drug regulatory systemsAuthorsDr Alain Prat, Department of Technical Cooperation for Essential Drugs and TraditionalMedicines, World Health Organization, Geneva, Switzerland.

WHO Guidance for the assessment of drug regulatory systemsAcknowledgementsWe are grateful to the European Commission for its generous contribution to this project.This document and the methodology it contains builds on the pioneering work of EshetuWondemagegnehu and Valerio Reggi have who conducted similar studies around the worldsince 2001. We are very grateful to them for taking a lead and showing the way forward forothers working on this subject.Special thanks are due to the following people who contributed much time and expertise inreviewing the previous draft of this guidance: Deus Mubangizi and Ben Botwe, incommenting on the revised draft: Celeste Aurora Sanchez-Gonzalez, Lucky S. Slamet,Margareth Sigonda, Hiiti Sillo, and Milan Smid.Finally, we would like to thank Rutendo Kuwana for his invaluable contribution in editing thisdocument.

WHO Guidance for the assessment of drug regulatory systemsContentsAuthors . 3Acknowledgements . 4Introduction. 61.General information - Module 1 . 132.National Regulatory System - Module 2 . 143.National Regulatory Authority - Module 3. 164.Marketing Authorization (MA) - Module 4 . 255.Licensing of Manufacturers - Module 5 . 306.Licensing of importers, exporters, wholesalers and distributors - Module 6 . 347.Licensing of pharmacies and retail outlets - Module 7 . 378.Registration of pharmacy personnel - Module 8 . 399.Post Marketing surveillance and controls - Module 9 . 4110.Control of drug promotion and advertising - Module 10 . 4611.Pharmacovigilance - Module 11 . 4912.Clinical Trials - Module 12 . 5213.Regulatory Inspections and enforcement activities - Module 13 . 5614.Quality Control Laboratory - Module 14. 6115.Control of Narcotics, Psychotropic Substances and Precursors - Module 15 . 6516.International Cooperation and Harmonisation Module 16 . 67Glossary . 68Abbreviations. 75Annexes . 76Annex 1: Terms of reference for the assessment Team . 77Annex 2: Recommended Regulatory Assessor’s Competencies. . 78Annex 3: Example of a Confidentiality Agreement . 79Annex 4: Example of a Declaration of no conflicting Interest by Assessors. 80Annex 5: Draft program of a Regulatory Assessment . 81Annex 6: List of institutions to be visited and Personnel to be met . 83Annex 7: Example of the contents of an assessment Report Format . 84Annex 8: Example of a format for a Plan of corrective actions. 86Annex 9. Example of a format of a implementation report . 87Annex 10. Quantitative indicators for regulatory purposes. 88Annex 11. Main source of documented evidence . 92Annex 12 - Overview of the assessment tool . 93

IntroductionWHO's role in medicines regulation is based on its constitutional mandate and various WorldHealth Assembly resolutions. These require WHO to support member states in their efforts toimplement national medicines policies and programs, to ensure equity of access to essentialmedicines, medicines safety and quality, and the appropriate use of medicines. Morespecifically, WHO is requested to provide guidance and support on the setting up of efficientnational medicines regulatory mechanisms, to ensure that available medicines are of goodquality and to fight against counterfeited medicines.TCM continues to assist countries on regulatory issues such as assessment of regulatorycapacities and skills building. Greater focus is however being given to stimulating thecountry's commitment to effective regulation and to enable countries to determinethemselves how their resources can be developed and applied most effectively.TCM will work with countries in assessing national regulatory systems to monitor progress,identify gaps and develop strategies, in cooperation with national authorities, to improvemedicines regulation. TCM will also help the country to introduce self-assessmentmechanisms into their regulatory systems, thereby stimulating their own commitment tosustained improvement of their regulatory performance and to capacity building.TCM will propose in both cases to the country to enter and follow its own Global RegulatorySupport Process consisting of planning, assessing, reporting on identified gaps, providingrecommendations, collaborating in the establishment of an institutional plan of action,providing support activities and follow-up.Experience in developed countries has shown that development of regulatory capacitiesoccurs in phases, over a long period of time. Factors such as the level of development of thepharmaceutical sector, the availability of trained human resources, infrastructure, the sizeand sophistication of the regulatory authority and financial resources, influence the regulatoryfunctions that can be carried out.Developing countries that are totally dependant on imported products and which have limitedqualified human and other resources will thus need to start with limited priority activities,expanding them gradually as their pharmaceutical sector develops and resources becomeavailable.Poor or inadequate regulation can lead to the prevalence of poor standard, counterfeit,harmful and ineffective drugs on national markets and in the international commerce. Thiscan result in serious harm to the health of individual consumers and even to the health of awider population. Therefore, countries must continuously strengthen key drug regulatoryresponsibilities so as to ensure the safety, quality and efficacy of drugs and the accuracy ofproduct information.Countries should assess their drug regulatory performance using indicators that focus onstructures and inputs, processes and outcomes. They should identify any strengths andweaknesses, the reasons for them and consider alternative regulatory options, using themost appropriate and practical choices.The WHO Data Collection Tool for the review of Drug regulatory Systems has beendesigned1 to help regulatory authorities perform such an assessment. More information onthe design and development of this tool is provided in annex 12. This guidance providestechnical advise on how to conduct this review.1WHO Data Collection Tool for the Review of Drug Regulatory Systems (WHO/TCM/MRS/2007.2)Regulatory Support Series No12

WHO Guidance for the assessment of drug regulatory systemsPrinciples of Drug RegulationThe drug regulation structures that exist today (drug laws, drug regulatory agencies, drugevaluation boards, quality control (QC) laboratories, drug information centres, etc.) haveevolved over time. Drug regulation is a public policy response to the perceived problems orneeds of the society. Each country establishes its own regulatory framework and regulatorypriorities according to existing and expected health risks and must adapt continuously tochanging needs. Consequently, drug laws need to be updated to keep pace with thechanges in the environment for which they apply.One of the key concepts growing in extent is to combine post-authorization and preauthorization activities in management of risks of pharmaceutical products and risk-minimization activities. This should also be considered by countries in the development andimplementation of a comprehensive regulatory framework.Drug laws, norms and standardsLegal structures form the foundation of drug regulation. Some drug laws traditionally omit orexempt certain areas of the pharmaceutical activity from their scope of control, resulting in aregulatory gap. To protect the public from harmful and dubious drugs and practices, druglaws should be comprehensive enough to cover all areas of pharmaceutical activity in thecountry.While drug laws provide the basis for drug regulation, regulatory tools such as standards andguidelines equip drug regulatory authorities with the practical means of implementing therelevant laws. Standards and guidelines should be established in a written form for all drugregulatory functions. These tools should then be used to guide the regulatory practice, aswell as being made publicly available to all the parties involved in order to bring transparencyto the drug regulatory process.Structure of Drug Regulatory AuthoritiesDrug regulation encompasses a variety of functions, such as licensing, inspection ofmanufacturing facilities and distribution channels, import and export controls, productassessment and registration, pharmacovigilance, Quality Control, control of drug promotionand advertising and control of drug clinical trials. Each of these functions targets a differentaspect of the pharmaceutical activity and all of them should act in concert for an effectiveconsumer protection.In some countries, all functions related to drug regulation come under the jurisdiction of asingle agency, which has the full authority in the command and control of these functions, aswell as bearing the responsibility for their effectiveness. In other cases, drug regulatoryfunctions are assigned to two or more agencies, at either the same or different levels ofgovernment. Two phenomena are found in the structural design of drug regulatory authoritieswhich can present problems in regulatory effectiveness — fragmentation and uncoordinateddelegation. Drug regulatory structures should thus be designed in such a way to provide for acentral coordinating body with overall responsibility and accountability for all aspects of drugregulation for the entire country.The drug regulatory authorities (DRA) in some countries are given non-regulatory functionssuch as drug manufacturing, procurement and/or delivery of services. Conflicts of interest inmandates and resource allocation can occur among these multiple functions and should beproperly managed.The DRA’s financial sustainability is a critical factor in the continued implementation of thevarious drug regulatory functions. The fees charged for regulatory services and governmentsubvention are in general two different ways of financing the drug regulatory authorities.

WHO Guidance for the assessment of drug regulatory systemsThe government should thus be fully committed to ensuring the financial sustainability of thedrug regulation.One of the major problems faced by the DRAs is the shortage of qualified personnel. Anumber of strategies can be considered in order to alleviate the shortage of humanresources: better human resource planning; sharing and pooling of international resourceson education and training, information and market surveillance; instituting incentives,prioritizing and streamlining work processes, job enlargement and job enrichment.Implementing drug regulationSeveral drug regulated areas, such as the informal sector, post-marketing surveillance andcontrol of drug information receive relatively little attention in the implementation process.Counterfeit products, products of dubious quality and flawed information, especiallyexaggerated claims of efficacy are often found to be widespread in the informal sector.Unlicensed manufacturers, importers, wholesalers, retailers and even persons engaged inthe pharmaceutical business pose difficult challenges to drug regulation. No matter howthoroughly the pre-marketing assessment is conducted, it is only one of the functions neededif the efficacy and, more importantly the safety of drugs are to be assured. Post-marketingsurveillance functions, such as pharmacovigilance, QC testing and re-evaluation ofregistered products, should represent priority areas in drug regulation as well.Drug information is distributed as widely as drug products themselves. Systems of regulatingdrug information include pre-approvals and self-regulation. However, monitoring of theaccuracy and appropriateness of information is generally inadequate, and the effectivenessof existing systems of regulation is unknown.Monitoring and evaluationThe regulatory process should be routinely and systematically monitored in order to identifythe problems in the process and determine whether the activities actually carried out areconsistent with the intended course of action. Several approaches such as self-reviews,supervisory body reviews and peer reviews may be used for assessing the DRA’sperformance: These approaches can complement one another in appraising the DRA’sperformance as well as assisting it to identify areas for possible improvements.Drug Regulatory Assessment ProcessThe regulatory assessment should be understood as part of a global process of improvement.It is the preliminary step for an authority, a country or an organization to identify its gaps andweaknesses. The results should be the basis for a global perspective and plan for theimprovement of its structure, capacities and efficiency.WHO can take part in this process in providing expertise to perform this assessment as wellas to provide the support for implementing the necessary corrective actions and to participatein the global increase of the level of quality of the organizations that deal with medicalproducts.The scope of the regulatory assessment could vary with respect to its objectives. A globalassessment of the regulatory system will review all the organizations having responsibilitiesover all kinds of pharmaceutical products. Some assessments can focus on horizontalfunctions such as regulatory inspections for all categories of products or some verticalfunctions such as the review of the authorization, licensing and vigilance functions for aspecial category of products. Some assessments might also be Organization/structureoriented in the review of all functions performed by a specific authority.Each Regulatory assessment should be organized based on the following steps: planning,preparation, site visit, report, action plan and follow up. The different steps could varydepending on the kind of assessment performed (self-assessment or external assessment).

WHO Guidance for the assessment of drug regulatory systemsThe process described below was developed by WHO/TCM for its own regulatory systemassessments. It could be adapted to satisfy particular needs or organizations. Expression of a needAn assessment should always be based on an expressed need. This need can be formal forexample an official request from a country to WHO, or informal such as in the case of a selfassessment. It could be internal for example following a program of quality management orexternal if a Ministry of Health asks for an assessment of one of its regulatory authorities.This is a major step because it will have an impact on the rest of the process: the definition ofthe scope, objectives, etc. Following this request, the first terms of reference of theassessment should be drafted (see annex 1). Assessment teamAn assessment team comprising at least two assessors should be designated to undertakethe assessment . The criteria for this selection should be based on the scope and theobjectives of the assessment. For example the review of the whole regulatory system can beconducted by an assessor without specific qualification but if the review has to focus on aquality control laboratory, the assessment team can be composed of a pharmacist and ananalyst. The recommended regulatory assessment competencies are as provided underannex 2. The assessors should belong to an independent group from outside the assessedorganization and should not be involved in the processes reviewed. It is recommended thatin case of recruitment of external experts, they be required to sign a confidentialityagreement (see annex 3) and a declaration of no conflict of interest (see annex 4).The assessors should be selected with due regard to their education and past experiences inthe assessed domain. If the assessment team is composed of several assessors, a teamleader should be nominated as well as a "rapporteur" in charge of taking notes and keepingthe documentation collected during the assessment. Usually a designated representative ofthe country or the assessed organization, accompanys the assessment team during itsassessment.If the organization requesting the assessment is not the assessed organization, the finalteam's composition should be agreed on between the two organizations. PreparationThe terms of reference of the assessment should be discussed, consolidated and agreed tobetween the assessment team and the requestor.The duration of the review visit depends on the kind of assessment and should usually lastfrom four to five working days. It might be longer depending on the scope of the assessment.A precise date of the assessment should be planned in collaboration with all the involvedparties.The assessment team should begin by collecting the information on the regulatory authority.A request for documentation should be sent to the country/organization in advance toprepare the assessment (see table 1 in chapter 2. and annex 11). Useful information can becollected from previous/past assessments performed. The assessment team should begin byreviewing the collected information and completing the WHO Data Collection Tool. It isadvisable to use electronic tools for collecting the information during the preparation, theassessment and reporting.Based on this information, the team leader should prepare a draft agenda mentioning thedifferent organizations, activities to be assessed (see annex 5) and the responsible person tobe visited for each activity (see annex 6).

WHO Guidance for the assessment of drug regulatory systems VisitBefore embarking on the assessment and setting up any meeting, the assessment teamshould make a briefing to the senior level officials of the organization in most of the cases theMinistry of Health or the General Directorate in the case of a national regulatory authority. Opening sessionBefore the visit, an opening meeting should be organized to define the practical aspects ofthe assessment, to review the plan, to check the availability of personnel and to agree on thetimetable of the assessment. Performing the assessmentDuring the visit, the assessment team should collect the needed information on the assessedorganization following the order of the WHO Data Collection Tool. The assessors shouldmainly ask for documentation, review the records, interview the staff and make observations.Data should be collected and information should be registered directly in the tool. Gaps orweakness should be identified. Preliminary recommendations could be discussed with themembers of the organization.On certain matters or issues the assessor could also undertake interviews outside of theNRA's organization or the institution assessed’ for example representatives of stakeholders,to collate opinions or information on how the NRA is perceived or operates.The assessors should review samples of records to check the implementation of theregulation, the guidance or the internal procedures of the organization.The assessment team should be encouraged to organize at the end of each day a debriefingsession with the personnel on the outcomes of the assessment in order to expose the mainfindings identified. Closing sessionAt the end of the visit, the assessment team should prepare a preliminary report containingthe scope covered by the assessment, the strengths and gaps identified and therecommendations on corrective actions that should be undertaken.The assessment team should then organise a closing session with the regulatory authority'sstaff that have been assessed. The estimated time frame for this closing session is half a day.During this closing session, the mains gaps should be presented and the recommendationsproposed by the assessment team should be discussed. The assessment team and theorganization's staff should be able to reach a common agreement on the recommendations.The preliminary assessment report should be amended taking into account the discussionswith the regulatory authority's staff.Following this discussion, a preliminary action plan should be developed by the assessmentteam and senior officials of the organization to specify how the recommendations could beimplemented.Situations can arise when the assessors do not share the same sentiments about a matter(gaps identified, recommendations of correctives actions, etc.). In such a case, the assessorshould first go back to the documented evidence that cannot be challenged and this shouldthen be recognized as pivotal. The assessor should always keep in mind that the content ofthe report is developed independently of the action plan which is a compromise between theassessment team and the assessed organization.

WHO Guidance for the assessment of drug regulatory systemsFollow-upA draft report (see annex 7) containing the findings, gaps and recommendations should bemade by the assessment team and should be submitted to the assessed organization timelyfor comments. The preliminary action plan of corrective actions (see annex 8) should beannexed to the draft report.This draft report should be reviewed by the assessed organization which can send technicalcomments or propose some amendments to the report of the assessment team.The report should be amended if necessary and finalized by the assessment team. The finalreport should then be submitted officially to the assessed organization and/or requestor, ifdifferent. Following the regulatory assessmentFollowing the final report, the assessed organization should prepare a final action plan ofcorrective actions. This action plan should take an operational format with the list ofcorrective actions that will be undertaken, the prospective deadlines or time limits for theircompletion with a clear mention of the responsibilities of the completion of these actions. Thefinancial aspects or resources required can be added to this plan or mentioned separately.Necessary actions should then be undertaken by the organization with or without the supportof external interventions. The implementation of the action plan should be reviewed anddocumented under the format of an implementation plan (see annex 9).Different categories of assessmentsFollowing the different steps in the assessment process, different regulatory assessmentscan be performed for a certain period of time.Preliminary assessmentA preliminary assessment can be organized to have a global overview of the system and toidentify the main gaps or deficiencies.Follow-up assessmentA follow-up assessment can be organized in a second intend after a preliminary assessmentand after corrective actions have been implemented to check or to verify the effectiveness ofthe actions undertaken or to review the level of implementation.Different regulatory assessments can be performed based on the scope for examplespecialized assessment can focused on;- specific regulatory functions (Licensing, Inspection, Laboratory, Etc ) or- specific products (marketing authorization decision making process for thepharmaceutical product X).Assessment durationThe duration of an assessment of a regulatory system will vary form one type of assessmentto another, the level of maturity of the Drug regulatory authority and also from the variousfunctions that have to be assessed. The time allocated should be based on factors such asthe size of the organization and the number of locations. Assessment duration will dependalso on the audit frequency . Annex 2 from Guidance on the Application of ISO/IEC Guide 62provides useful information on determination of the duration for assessment.

WHO Guidance for the assessment of drug regulatory systemsAssessment MethodologyThe assessment methodology is based on several concepts which should be taken intoaccount and followed.An assessment should not be based on impressions, feelings or any subjectiveconsiderations. Furthermore it is important for the assessor to collect objective evidence ofhis observation. The laws or regulations published should be collected and any reference tointernal procedures or Standard operating procedures/instructions should be quoted. Thesedocuments will be used afterwards as the basis for the proposed recommendations and willcontribute to demonstrate the adequacy of the propositions to address any identified gaps.Evidence may be collect by different means such as: Interviewing personnel. Reading documents. Reviewing manuals. Studying records. Reading reports. Scanning files. Analyzing data. Observing activities. Examining conditions.The evidence collected through interviews should, whenever possible, be confirmed by moreobjective means. Investigational clues that point to possible deficiencies or gaps should bethoroughly investigated. The assessor should examine the collected evidence and documentthe gaps identified.The assessor should not limit his activities to checking the presence or the absence of adocument or a law. He should pursue to find the evidence on the implementation of thedocuments, procedures, guidance or laws.For example, a law might have been published but no regulation has been adoptedafterwards or the regulation has not been explained to other stakeholders through aguidance. As for the administrative procedures, the questioning methodology is the same;e.g. a procedure might have been established but not implemented. In such cases, theassessors should take samples of the records to identify the level of implementation of theprocedure.The regulatory assessment is a method to improve a regulatory system and it could beperformed within the organization by internal personnel or by an external expert or a

well as bearing the responsibility for their effectiveness. In other cases, drug regulatory functions are assigned to two or more agencies, at either the same or different levels of government. Two phenomena are foun