Report on the Efficiency and Effectiveness ofthe Office of the Controller and AuditorGeneral of New Zealand by an InternationalPeer Review Team.March 2008

Table of ContentsPage1.2.3.4.Introduction . 4Conduct of the Review. 4Overview . 5Governance and Management . 9Organisation Structure. 10Governance Framework . 11Management . 13Performance Management . 19Culture and Values. 20External Stakeholder Relationships . 21External Performance Assessment. 245. Financial Statements and Assurance Auditing . 24Financial Statements Audit. 25Business Model. 26Audit Appointment/Allocation . 30Audit Fee Setting and Monitoring . 34Annual Audit Performance. 36Schools Audits . 36Audit New Zealand . 38Long Term Council Community Plans (LTCCPs) . 45The Controller Function . 476. Performance Auditing, Good Practice Guides and Inquiries . 506. 50Performance Auditing . 50Good Practice Guides. 65Performance Reporting. 66Inquiries. 667. Suggestions for Improvement . 707.17.27.3Governance and Management . 70Financial Statements and Assurance Auditing . 71Performance Auditing, Good Practice Guides and Inquiries . 738. Note of Appreciation . 76Attachments: of Reference . 77Review Team Members. 78Interviews conducted. 79Background Review Documents. 83Background on the New Zealand Public Sector. 84Documents Examined . 891

Abbreviations – A-ZANZASDASPAssistant Auditor-GeneralAustralasian Council of Auditors-GeneralAuditor-General (although the full title is Controller and Auditor-General, theapproach in this Report is to use the accepted abbreviation. This also applies tothe use of the term Deputy Auditor-General).Audit New ZealandAudit Status DatabaseAudit Service ProviderC.CAGController and Auditor-GeneralD.DCAGDeputy Controller and Auditor-GeneralE.EAPEEOEmployee Assistance ProgrammeEqual Employment OpportunityF.FRSBFTEFinancial Reporting Standards BoardFull Time EquivalentH.HRHuman ResourcesI.ICTIFRSINTOSAIISITInformation and Communications TechnologyInternational Financial Reporting StandardsInternational Organization of Supreme Audit InstitutionsInformation ServicesInformation TechnologyL.LGNZLTCCPLocal Government New ZealandLong Term Council Community PlansM.MOUMPMemorandum of UnderstandingMember of ParliamentN.NAMSNational Asset Management Steering Group, set up by Association of LocalGovernment EngineersNew Zealand Qualifications AuthorityA.AA-GACAGA-GNZQAO.OAGOFFICEOPCOffice of the Auditor-GeneralThe Auditor-General’s Office, including the Office of the Auditor-General andAudit New Zealand and, where necessary, Appointed AuditorsOfficers of Parliament CommitteeP.PAGPPGPWCPerformance Audit GroupProfessional Practices GroupPricewaterhouseCoopersQ.R.QAR&DQuality AssuranceResearch and DevelopmentS.SOLGMSPASAISociety of Local Government ManagersSouth Pacific Association of Supreme Audit Institutions2

Glossary of TermsAllocation ModelA process whereby Audit Service Providers are selected by the Auditor-General against an establishedset of criteriaAMP PlanA private sector superannuation plan managed by AMP Financial ServicesAppointed AuditorAn Appointed Auditor is the person appointed by the Auditor-General to sign the audit opinion of apublic entity on his behalf. Appointed Auditors are normally either partners of chartered accountingfirms or directors of Audit NZ, although there are exceptions for the audits of some smaller entities.Climate SurveysOrganisational surveys conducted by independent consultants to assess staff attitudes and attributesContestable ModelA process whereby Audit Service Providers are generally selected on the basis of an open tender processController FunctionThe Controller functions are described in Section 5.10 of this reportEnhanced Financial AuditsEnhanced financial audits are described in sub-section 5.3.5 of this reportLeadership Development CentreNew Zealand’s training centre for senior State sector managers and leadersLeadership Teams The OAG Leadership Team comprises the Auditor-General, Deputy Auditor-General and allAssistant Auditors-GeneralThe Audit New Zealand Executive Leadership Team comprises the Executive Director, his GeneralManagers and the Assistant Auditor-General responsible for Corporate ServicesThe combined Leadership Team comprises all members of the OAG and Audit New ZealandLeadership TeamsProfessional Practices GroupThe team responsible for developing and enhancing Audit New Zealand’s professional competence. Itreports to the General Manager, Professional PracticesQuality Control ManualAudit New Zealand’s document containing all policies and procedures of the organisation. It formallydocuments the organisational system of quality control. All employees carrying out audit or assurancemust comply with the Manual.Select CommitteesMulti-party Parliamentary Committees which conduct annual examinations of the budgets andperformance of public entities within their subject areaSubstantiatorMember of the Performance Audit Group, independent of a particular performance audit, who carries outa review to ensure that there is evidence in the working papers to support the statements made in thedraft report for that auditTeamMateAudit New Zealand’s computer software programme which provides electronic management of auditsand permits audit working papers to be stored electronically rather than in a ‘paper’ format“Team Space”Locations or folders provided on the Office’s intranet for specific staff teams and groupings3

1.Introduction1.1.This Report was prepared by a Review Team in response to Terms of Referenceprovided by the CAG, Mr Kevin Brady (see Attachment 1). Drafts of the Terms ofReference were provided to Review Team Members before being finalised. Briefbackground information on each of the five Team Members is included in Attachment 2.The Coordinator of the Review Team, Pat Barrett, was fully consulted on membershipof the Review Team. His suggestion to conduct the Review in accordance with theAustralasian Council of Auditors-General Peer Reviews and Voluntary DevelopmentQuality Assurance Reviews – Core Reference Modules was accepted by Mr Brady.However, a number of areas specifically relevant to the New Zealand Office wereadded. It was also agreed that a Draft Report would be provided to the Controller andAuditor-General by 18 December 2007, with a response from Mr Brady by 28 February2008 and a Final Report to be delivered by 31 March 2008. All commitments were met.2.Conduct of the Review2.1.Initial discussions concerning the conduct of the Review took place in Wellington on 23and 24 November 2006 between the Review Coordinator and the Controller andAuditor-General and Deputy Controller and Auditor-General. These discussions alsoprovided an opportunity for familiarisation with the OAG and Audit New Zealand, asdid discussions with senior staff at the time (see Attachment 3). Further Reviewpreparation included examination of Background Documents (see Attachment 4) and arange of other Documents provided by the Office which were added to progressivelyover the Review period (see Attachment 6). These Documents included the previousPeer Review of the New Zealand Audit Office in 2001 under the auspices of theAustralasian Council of Auditors-General, and the most recent Peer Review by theAustralian National Audit Office of two performance audits prepared by thePerformance Audit Group within the OAG. As noted in Attachment 3, the ReviewCoordinator had discussions with the latter Peer Review Team in Wellington on 23November 2006 and in Canberra on 13 December 2006.2.2.The extensive nature and content of the foregoing discussions and documentationprovided had two main benefits. They allowed for a considerable range of issues andviews to be considered by the Review Team well before the on-site review conducted inSeptember. As a result, the Team had the opportunity to be better prepared and focussedto make best use of their time in Wellington, including further interviews withnominated stakeholders and those requesting interviews with the Team. The pre-reviewactivities were a factor in determining the dates for the on-site review. It was decidedthat a two week period from 17 to 28 September was appropriate. One Member of theTeam located in Wellington was able to commence interviews in the week beginning 10September, which proved to be of considerable assistance in completing an extensiverange of interviews.2.3.Following consideration of the Terms of Reference, it was decided that the Reviewwould cover three segments of the operations of the Office as follows: Governance and Management (including all Corporate Services) Financial Statement and Assurance Auditing (including the Controller Function andLTCCP audits); and Performance Auditing, Good Practice Guides and Inquiries.4

2.4.3.Review Team Members were selected on the basis of experience and professionalbackground and qualifications in each of these three areas. Because of the extent ofactivity and resources involved in the second segment above, the Controller andAuditor-General agreed to have two Team Members cover that segment. Obviously,there were both overarching and common issues impacting on all three segments thatneeded to be coordinated to avoid unnecessary duplication of activity. While all TeamMembers were sufficiently experienced to manage this situation effectively, it was alsoapparent that the Review would benefit from their different perceptions of the issues.This was considered particularly relevant to ensuring a more cohesive organisationinvolving the respective roles of, and the relationships between, the OAG (as describedat sub-section 4.2.1) and Audit New Zealand (as described at sub-section 5.8.1).Overview“ audit does not exist merely to check on what public organizations are doing, itmust also act as a catalyst for change through constructive but critical challenge.Increasing insight and facilitating foresight is set to feature ever more prominently inthis task.”Joseph McHugh, Deputy Editor, Public Finance, CIPFA, London, 200713.1.The Auditor-General’s Office (the Office) is a relatively small but highly regardedorganisation both in New Zealand and internationally. The Office operates in thecontext of a developed, highly performing and sophisticated national economy and aninnovative public service that has been used as a model by many countries. On the onehand, limited resource availability and product scope might inhibit performancecompared with that of other similar Offices but, on the other, the foregoing factors tendto raise expectations of the Office. The Review Team was conscious of such a tensionbut concluded, in relation to a number of performance criteria, that the Office would ratehighly both absolutely and relatively in any international comparison.3.2.A central requirement of any professional audit practice is to know and understand theenvironment in which audit activity is conducted. The Team not only had theopportunity to speak with an extensive range of stakeholders of the Office to test its ownknowledge and understanding of the New Zealand Public Sector, it also had theadvantage of tapping into the wide-ranging experience and knowledge of Neil Walter,throughout the Review, as a Team Member. Neil provided considerable backgroundinformation for the Team, including a Paper on the New Zealand Public Sector asAttachment 5 to this Report. The Paper provided a useful context for the Review,particularly with its focus on performance and results.1 Comment included in a Public Management and Policy Association Report entitled “Watchdogs Straining at theLeash”. Edited by Michaela Lavender, November 2007 (page 5)5

3.3.The Review Team was impressed by the robust legislative framework applying to thepublic sector and to the Office. The Public Audit Act 2001 was intended to strike anacceptable balance between: the independence of the Auditor-General, in particular the ability to act withoutdirection or improper influence by the Executive or the Legislature; the need for a sound working relationship between the Auditor-General, Parliamentand the Executive; and the need for the Auditor-General to be properly accountable to Parliament.3.4.Importantly, establishing the Auditor-General as an Officer of Parliament was to ensurethe Auditor-General’s independence. The Office accepted that the Controller andAuditor-General, as an Officer of Parliament and a Corporation Sole has the same levelof accountability as comparable entities in the public sector. It is recognised that theAuditor-General’s role as an Officer of Parliament does create a tension between beingaccountable to Parliament while recognising that Parliament is best served by anAuditor-General free from any political interference. The same comment applies to thestatutory position of Deputy Auditor-General. The effectiveness of the Office isenhanced by the seamless integration of governance at the top of the organisation,reflecting the complementary and cooperative relationship between the two currentappointees.3.5.While some concerns were expressed to the Review Team about the clarity of roles andrelationships between the Auditor-General (and the Office) and Parliament, thelegislative intent is quite clear. Differing perceptions about action taken, or not taken,are best addressed by direct and open communication. Independence is called intoquestion when one is involved in processes or related decision-making and is alsoresponsible for the review or audit of the activity. It was clear that the Auditor-Generaland the Office are very sensitive to issues of independence and accountability and theappropriate action, including independent review, to provide assurance to the variousstakeholders, not least to Parliament itself.3.6.Particular issues examined by the Review Team included the organisation and structureof the Office (covering the role and effectiveness of sector managers and any apparentduplication of activities); relationships with its various stakeholders; audit planning,selection, methodology and conduct (including contestable and “allocation” approachesto auditor selection and the coverage and conduct of “enhanced” financial statementaudits); knowledge management, performance management and assessment; qualityassurance; implementation of audit findings and recommendations; and the complex andrapidly-changing nature of the work of the Office.3.7.The Office has undergone significant and wide-ranging change since the last PeerReview in 2001. Its workload has increased in both volume and complexity. Itslegislative and operating environment has altered in a number of ways. Its budget andstaffing establishment have expanded. And significant improvements have been made toits structures, systems and operations.3.8.The relationships of the office with its key stakeholders are generally very positive. Itswork is respected across government. Central agencies in particular see it as supportiveof their efforts to improve public sector performance. It is highly regarded by localgovernment. Parliamentary Select Committees depend heavily on advice from the OAG6

and appointed auditors in their examination of departmental estimates and annualreports. Ministers find it helpful to receive annual financial audit reports on theirportfolio agencies and spoke well of the work of the Office. Parliament also values theOffice’s work, although the recent inquiry into electoral advertising has obviouslystrained the Auditor-General’s relations with political parties. Recent independentstakeholder surveys bear out the high levels of satisfaction encountered by our team.3.9.Except as discussed in sub-section 5.3, the financial and assurance work of the Office islargely non-discretionary. The main areas of discretionary work are performance auditsand inquiries, which make up around 10 per cent of the Office’s activities. While thefinancial audit and assurance work of the Office is well established and accepted,performance auditing is still developing and growing in importance, reflecting asignificant increase in resources in 2004. Overall, we found a high level of satisfactionwith the OAG’s performance audits. Interviews with Ministers, Select Committees andpublic entities yielded a range of generally positive comments. The results of the OAGcommissioned 2007 Stakeholder Feedback Interviews showed all respondents satisfiedwith the quality of OAG performance audits and 86 per cent satisfied with theirusefulness. There was only the occasional questioning as to whether all the subjectscovered really warranted scrutiny by the Auditor-General.3.10. We found considerable process rigour around quality assurance, with no fewer than fivereviews conducted in the year preceding our own review. These reviews have allcommented positively on the OAG’s performance audits, while also usefully signpostingways in which further improvement can be made. We note that, at present, much of thequality assurance effort is focussed towards the end of an audit or indeed after an auditreport has been published. We consider there would be merit in applying more of thisresource to earlier points in the audit lifecycle.3.11. With 14 performance audits to deliver in 2007-08 and a core audit staff of 15performance auditors, there is a question as to whether present core staffing levels forperformance audits provide sufficient continuity and capacity to do justice to the rangeand number of audits tackled each year. The selection processes for performance auditscould also be refined, primarily around a more proactive engagement with Members ofParliament and public entities. While improving quality of audits and recommendationsis of prime importance to the reputation and credibility of the Office, there is also theimperative to ensure that recommendations are actually being implemented effectively.A closer working relationship with Select Committees could assist in this respect.3.12. May 2006 saw the introduction of an Inquiries Manual designed to rationalise processesand procedures across the Office, centred upon good practice established in earlierinquiries. The Manual represents an important decision-making tool and is an importantmeans for the Auditor-General to demonstrate, with clear and comprehensivedocumentation, that processes have been diligently and consistently applied.3.13. In 2006-07, there were two specific quality assurance exercises applied to the inquiriesfunction which identified a number of areas of the Inquiries Manual that might needstrengthening, as well as a need in some inquires for more documentation to properlyrecord key decisions and positively demonstrate compliance with the Manual.3.14. As with the OAG’s performance audits, the 2007 Stakeholder Feedback Report found100 per cent of respondents satisfied with the OAG’s handling of inquiries, comparedwith a figure of 75 per cent in 2006.7

3.15. Our overall assessment is that the Office has coped well with the challenges of recentyears and is performing its tasks professionally and well. It has the feel of an outwardlooking and forward-looking organisation. The feedback from stakeholders on itsperformance was generally positive, as noted earlier, with some suggesting that theOffice could do even more to contribute to improving public sector management in NewZealand. We found management and staff alike to be well attuned to the changingrequirements and expectations placed on the Office. Both the OAG and Audit NewZealand now seem to us to be well positioned to respond to the further challenges thatlie ahead.3.16. As with other Audit Offices and private sector accounting firms, the OAG and AuditNew Zealand have an ongoing problem of attracting and retaining suitable professionalsnot only to undertake audit programmes, but also to maintain – and hopefully improve –research and development capacity, add value to audits and improve relationships withall stakeholders. At least three related factors need to continue to be addressed to meetstaffing concerns – providing suitable personal development and professional training aswell as state-of-the-art audit tools; providing a comprehensive and varied auditprogramme that is relevant to stakeholders as well as being demanding and interesting tostaff; and promoting stakeholder relationships that enhance understanding andacceptance of the work of the auditor.3.17. The Review Team saw its main tasks as information gathering, assessment, assuranceand adding value. Full access was provided to all documentation and relevant personnelboth within and outside the Office. The Team had sufficient expertise to undertake allassessments without any other assistance. This reflected both the professionalqualifications and the wide experience of individual Team Members. Thus it had fullconfidence in its ability to provide assurance across the full range of the Office’sactivities – of both an audit and a non-audit nature.3.18. The Team considered it would best add value largely by supporting and reinforcingobserved better practice. This included offering a number of suggestions forimprovement based on Team Members’ knowledge and experience. The opening quoteto this overview was a recent reflection of the views of Auditors-General in the UnitedStates and United Kingdom, suggesting the need for more audit focus on strategicplanning, governance, risk management and performance measurement and reporting. Inaddition, the Team took into account the increased involvement of the private sector inpublic service delivery and issues associated with cross-agency delivery of governmentprogrammes.3.19. A Draft Report was provided to the Auditor-General for any comments and/orsuggestions – basically on facts and accuracy. The final Report includes his commentswhere relevant.3.20. The following sections of the Report, led by individual Team Members, outline what wefound and what improvements we suggest.8

4.Governance and Management4.1Introduction4.1.1 Governance has been defined as “ the processes by which organisations are directed,controlled and held to account. It encompasses authority, accountability, stewardship,leadership, direction and control exercised in the organisation. It includes how anorganisation is managed, its corporate and other structures, its culture, its policies andstrategies and the way it deals with its various stakeholders.”24.1.2 The Controller and Auditor-General is an officer of Parliament. His powers andresponsibilities are set out in the Public Audit Act 2001. The Auditor-General isindependent of executive government and Parliament in discharging his statutoryfunctions but is answerable to Parliament for the stewardship of the public resourcesentrusted to him. His primary task is to provide independent assurance to Parliamentand the public that public entities and local authorities are operating in accordance withParliament’s intentions and accounting for their performance. The governance issuesfacing the Auditor-General can be quite complicated – not least because he deals with awide range of stakeholders in a fast-changing and often complex environment.4.1.3 The Public Audit Act 2001 also sets out statutory status for the position of DeputyController and Auditor-General, who is also appointed by the Governor-General on therecommendation of Parliament. It confers on the Deputy position the same powers andmany of the responsibilities of the Auditor-General. The two positions were obviouslyintended to be complementary and mutually supportive, and the Team found that this ishow the relationship is working in practice. It was impressed with the sense ofpartnership that characterises the working relationship between the Auditor-General andDeputy Auditor-General.4.1.4 Recent years have seen significant changes in the accounting and auditing professions aswell as in New Zealand’s legislative and operating environment. A new Public AuditAct was passed in 2001. New accounting standards based on the International FinancialReporting Standards which came into effect on 1 January 2007 (earlier in the case oflocal government) affect virtually all public entities. The Local Government Act 2002generated major new work for the Office, including the requirement to audit localauthorities’ Long Term Council Community Plans (LTCCPs). Stakeholder demands onthe Office are rising. More complex and more stringent managerial standards are nowimposed on all public entities.4.1.5 The Office has undertaken a major programme of organisational adaptation andimprovement in response to these changes. Its annual budget increased from 36.8million in 2001-02 to 69 million in the 2006-07 year. Over the same period the staffingof the OAG and Audit New Zealand has gone from 248 to 288. Strategic planning hasbeen put on a more systematic footing. The contestable model of audit allocation hasbeen replaced by an allocation model. It has developed a new project management unitand a research and development capability. It has merged the strategic planning andcorporate service functions of the OAG and Audit New Zealand. Effort has gone into thedevelopment of knowledge management, risk management and business continuityframeworks. There is a stronger focus on professional development. An independentAudit and Risk Committee has been established, along with an internal audit function.2Australian National Audit Office, August 2003. Public Sector Governance. Better Practice Guide. Canberra.9

Interaction with key external stakeholders has been stepped up. The Office’s planningand reporting processes have become more robust and transparent. Numerous externalreviews and surveys have been commissioned by both the OAG and Audit NewZealand.4.1.6 In terms of its output, the Office has placed a progressively stronger emphasis onperformance and programme audits. Its regular financial audits now involve closerscrutiny of such areas as probity, waste, sustainable development, stewardship of assetsand compliance and assessing the quality of service performance information in annualreports. It consults more widely on its discretionary work programme. More attention isbeing paid to the quality of its reports. A number of good practice guides have beenproduced aimed at lifting performance in the public sector. Links have been maintainedand strengthened with regional and international professional groupings.4.1.7 To give a brief picture of its current situation, the Office is now responsible for over4000 financial audits and delivers 20

TeamMate Audit New Zealand's computer software programme which provides electronic management of audits and permits audit working papers to be stored electronically rather than in a 'paper' format "Team Space" Locations or folders provided on the Office's intranet for specific staff teams and groupings