Performance Audit Manual
COPYRIGHT NOTICE — This document is intended for internal use. It cannot be distributed to or reproduced by third parties without prior written permission from the Copyright Coordinator for the Office of the Auditor General of Canada. This includes email, fax, mail and hand delivery, or use of any other method of distribution or reproduction. CPA Canada Handbook sections and excerpts are reproduced herein for your non-commercial use with the permission of The Chartered Professional Accountants of Canada (“CPA Canada”). These may not be modified, copied or distributed in any form as this would infringe CPA Canada’s copyright. Reproduced, with permission, from the CPA Canada Handbook, The Chartered Professional Accountants of Canada, Toronto, Canada.
4042 Audit Scope and Approach
Dec-2022
Overview
Considering what elements to be included in the audit (scope) and determining the nature of the work to be undertaken and how it will be undertaken (approach) are two interrelated concepts. Scope and approach involves understanding the subject matter of the audit and considering matters of significance and their auditability; determining the nature and extent of procedures to be performed (including establishing lines of enquiry); and considering including follow-up work and using the work of others.
Financial Administration Act Requirements for Special Examinations
Section 131(1) Each parent Crown corporation shall cause
[...]
(b) financial and management control and information systems and management practices to be maintained, in respect of itself and each of its wholly-owned subsidiaries, if any.
Section 131(2) – The books, records, systems and practices referred to in subsection (1) shall be kept and maintained in such manner as will provide reasonable assurance that
(a) The assets of the corporation and each subsidiary are safeguarded and controlled;
[...]
(c) The financial, human and physical resources of the corporation and each subsidiary are managed economically and efficiently and the operations of the corporation and each subsidiary are carried out effectively.
Section 138(1) Each parent Crown corporation shall cause a special examination to be carried out in respect of itself and its wholly-owned subsidiaries, if any, to determine if the systems and practices referred to in paragraph 131(1)(b) were, in the period under examination, maintained in a manner that provided reasonable assurance that they met the requirements of paragraphs 131(2)(a) and (c):
Section 138(3) Before an examiner commences a special examination, he shall survey the systems and practices of the corporation to be examined and submit a plan for the examination, including a statement of the criteria to be applied in the examination, to the audit committee of the corporation, or if there is no audit committee, to the board of directors of the corporation.
Section 138(5) An examiner shall, to the extent he considers practicable, rely on any internal audit of the corporation being examined conducted pursuant to subsection 131(3).
Section 139(2) The report of an examiner under subsection (1) shall include
(a) a statement whether in the examiner’s opinion, with respect to the criteria established pursuant to subsection 138(3), there is reasonable assurance that there are no significant deficiencies in the systems and practices examined.
OAG Policy
The audit team shall perform a risk-based planning exercise to determine the scope and approach of the audit. [Nov‑2015]
Audits shall have a clear scope that focuses on the extent, timing, and nature of the audit. The audit team shall select issues on the basis of their significance, their planned value added, and their auditability. [Nov‑2015]
The engagement leader shall approve the scope and approach of the audit as documented in the audit logic matrix. [Nov‑2015]
In performance audits, the audit team shall follow up on issues that led to recommendations in a previous audit report if those issues continue to be of interest to Parliament and/or pose a significant risk. [Nov‑2015]
In special examinations, the audit team shall consider significant deficiencies identified and recommendations made in the previous special examination report when determining the scope of the audit. [Nov‑2015]
In special examinations, the audit team shall identify all affiliated entities of the Crown corporation and decide whether these affiliated entities are to be scoped into the audit. [Nov‑2011]
The scope of all special examinations of Crown corporations shall, at a minimum, cover ‘core’ systems and practices which are assessed using the Office’s standard criteria. Based on a risk and control assessment performed in the planning phase the engagement leader can justify expanding the scope of the special examination beyond the mandatory ‘core’ systems and practices. [Nov-2017]
OAG Guidance
What CSAE 3001 means for determining the audit scope and approach
CSAE 3001 requires that the audit team consider the concept of significance when planning and performing the assurance engagement, including when determining the audit approach (i.e. determining the nature, timing, and extent of procedures). An audit of a significant matter is one that may influence the decisions of the “intended users” (members of Parliament for performance audits and board of directors of the Crown corporation for special examinations).
CSAE 3001 also requires that the audit team obtains an understanding of the subject matter in order to be able to identify and assess the risks of significant deviations and provide the basis for designing and performing audit procedures to respond to the assessed risks and obtain sufficient appropriate evidence to support the audit conclusion.
The audit team bases its audit scope and approach on its understanding of the subject matter, its significance, and an assessment of the risks of deviations. The concepts of audit scope and approach are inter-related. Together with the audit objective (OAG Audit 4041 Audit objectives) and criteria (OAG Audit 4043 Audit criteria), they form the overall audit design. For more information, see OAG Audit 4010 Understanding the subject matter in planning an audit, OAG Audit 4020 Risk assessment, OAG Audit 4025 Internal controls, and OAG Audit 4044 Developing the audit strategy: audit logic matrix.
Identifying Audit Scope and Approach
Audit scope
The audit scope includes a description of what elements are included in the audit. Typically, scope includes the following three elements:
- what entities (or parts thereof) are included in the audit,
- what programs or activities or functions are included in the audit, and
- what is the period covered by the audit.
Scoping decisions include a rationale for including key elements as well as for elements explicitly not included.
Performance audits
Audit teams have considerable latitude in determining the scope of a performance audit due to the Auditor General Act. Section 7(2) of the Act states that the Auditor General “shall call attention to anything that he considers to be of significance and of a nature that should be brought to the attention of the House of Commons . . .”
Audit teams use professional judgment to determine the audit objective, and select the relevant scope and audit approach to conclude on that objective.
Special examinations
To meet the objective of a special examination as set out in the Financial Administration Act, the audit team selects systems and practices considered essential to providing the corporation with reasonable assurance that it is meeting its statutory control objectives (i.e. assets are safeguarded and controlled, resources are managed economically and efficiently, and operations are carried out effectively).
The audit team can consult with the appropriate internal specialists (if applicable) before scoping the special examination. Major scoping decisions for special examinations include determining the effect of affiliate entities on the scope and deciding which systems and practices are considered essential for responding to risks assessed during planning (see Matters of significance below).
Audit approach
The audit approach corresponds to the nature, extent, and timing of the work being undertaken in order for the audit team to provide reasonable assurance to the intended user of the report on the audit conclusion, against the audit objective.
Key considerations for audit scope and approach
The planning phase involves the important tasks of elaborating a detailed approach; documenting the nature, timing, and extend of procedures to be performed; and the reasons for selecting them. This work is iterative and is not necessarily undertaken in a linear fashion. It ensures that the audit team has a good understanding of what work to undertake and how to undertake it. It also helps ensure that sufficient appropriate evidence will be obtained to provide a reasonable basis to support the conclusion expressed in the report. The team should document the professional judgments used to make the selection of procedures to be performed and other key decisions during the planning phase. The audit logic matrix (OAG Audit 4044 Developing the audit strategy: audit logic matrix) and other risk tools and templates (OAG Audit 4020 Risk assessment) will help auditors document the decisions made and logic used.
The audit team needs to consider the following elements when making decisions about scope and approach:
- significance,
- auditability,
- period under examination,
- lines of enquiry,
- suitability of criteria,
- inclusion of follow-up work,
- using the work of others,
- audit effort required during the examination phase,
- evidence-gathering techniques and testing strategies required.
With the exception of suitability of criteria that is discussed under OAG Audit 4043 Audit criteria, each of these considerations are covered in the following sections.
Matters of significance
Carefully scoping the audit early in the process helps increase its efficiency and effectiveness. The statement of scope should be clear about any related areas that are not included. This information is set out in the audit logic matrix (OAG Audit 4044 Developing the audit strategy: audit logic matrix) along with planning documents provided to entities, such as the audit plan summary (OAG Audit 4090 Audit plan summary for performance audits), or the Special Examination Plan (OAG Audit 4100 Special examination plan). OAG Audit 7030 Drafting the audit report sets out how scoping considerations are described in the audit report.
The audit team identifies matters of significance to include in the scope of an audit, based on an understanding of the subject matter, risk assessment, and an evaluation of internal controls (if relevant).
Performance audits
When the decision is made to carry out an audit (OAG Audit 1510 Selection of performance audit topics), the activity to be audited is defined in broad terms. Once planning work begins, clearly defining the audit scope is important to determine the budget, human resources, and time required to conduct the examination work, and to determine what will be reported. Scoping the audit involves narrowing the audit to relatively few matters of significance that pertain to the audit objective and that can be audited within allocated resources. To identify matters of significance to parliamentarians, the audit team conducts research to understand the subject matter (OAG Audit 4010 Understanding the subject matter in planning an audit), keeping in mind the following questions:
-
Are there areas that have an important impact on the entity’s results? A project or program may have a relatively small budget, but affect a large segment of the population or have a significant impact on the environment.
-
Are there areas involving large dollar amounts? Generally, areas with larger dollar amounts warrant more attention.
-
Will the audit of the issue make a difference; that is, will it result in improved performance, accountability, or value for money?
-
Are there issues with high visibility or of current concern?
-
Are there areas that have undergone a significant degree of change? Examples of changes within an entity are new programs, increased staff turnover, and reorganization; examples of changes to an entity’s environment are amendments to enabling legislation, federal hiring freezes, and budget cuts.
-
Is the timing appropriate for auditing the issue?
Planned value added in Performance audits
To inform a scoping decision, the audit team determines the value that the audit will add in terms of the assurance, information, and advice to be provided. This information is used to determine whether each line of enquiry is needed to support the overall planned value added of the audit. For further information, see Guidance to Integrate Value added into the Performance Audit Process and Guidance to Integrate Results Measures into Performance Audits, under the Audit Guidance section below.
Special examinations
The conclusion of a special examination includes an opinion statement on whether there were any significant deficiencies in the Crown corporation’s systems and practices selected for examination.
Determining essential systems and practices
Generally, “systems” are formally approved procedures, while “practices” are what is actually being done. Although the two terms are mostly used together, they are not synonymous.
The OAG has developed a set of “core” systems and practices and related standard criteria that should be examined in every special examination. Taken together, these ‘core’ systems and practices and standard criteria are deemed to be applicable in all cases, and in most cases sufficiently comprehensive to provide the basis for an audit opinion on whether the Corporation has reasonable assurance of achieving its statutory control objectives. Although the control objectives are identified separately in the FAA, each is affected by a variety of systems and practices acting in concert. Moreover, any given system and practice influences more than one control objective. Core recognizes the complex relationships both among control objectives and the systems and practices of the corporation.
Through consultations, the OAG concluded that the Act does not require the inclusion of all systems and practices within the scope of a special examination. Rather, the scoping of a special examination allows for the selection of specific systems, practices and criteria that reflect each Crown corporation’s circumstances, provided the selection is sufficiently comprehensive to determine if the corporation has reasonable assurance of achieving the statutory control objectives. Core systems and practices were developed in order to manage the risk of the OAG not examining a selection of systems and practices that is sufficiently comprehensive to fulfill its mandate for special examinations under the FAA. In order to identify this minimum set of systems and practices to be examined, the following assessments were performed:
-
In order to answer the question, “What is required for the Crown corporation to cause financial and management control and information systems and practices to be maintained (FAA S. 131 1 (b))?” an analysis and cross-walk was performed to compare the FAA requirements with both ‘best management practices’ from recognized sources as well as with expectations for management systems and practices found in documents linked to the changes to the FAA in 1984.
-
In order to answer the question, “What is expected for a Crown corporation to have reasonable assurance that its assets are safeguarded and controlled, its resources are managed economically and efficiently, and its operations are carried out effectively (FAA S. 131 2 (a) and (c))?” an analysis and cross-walk was performed to identify the expectations for systems and practices to be in place to achieve these statutory control objectives from documents written for guidance on conducting special examinations around the time of the change in the FAA.
These assessments identified a ‘core’ set of systems and practices that were deemed to be applicable in all cases, and in most cases sufficiently comprehensive to determine if a Crown corporation is achieving its statutory control objectives. Core systems and practices mitigate coverage risk.
Based on the risk and internal control assessment for the Crown corporation under audit, the special examination team determines whether to expand the scope of the examination beyond the mandatory core systems and practices and related standard criteria. Where there are significant risks facing the corporation, and where a significant deficiency in the mitigating systems and practices could prevent the Crown corporation from having reasonable assurance that its assets are safeguarded and controlled, its resources are managed economically and efficiently, and its operations are carried out effectively, teams must consider two options:
-
If the risk is covered by the core systems and practices, the team considers increasing audit effort in the area of higher risk.
-
If the risk is not covered by the core systems and practices, the team can then justify adding to the scope based on the rationale that a significant deficiency in the non-core system(s) and practice(s) could prevent the Crown corporation from having reasonable assurance that its assets are safeguarded and controlled, its resources are managed economically and efficiently, and its operations are carried out effectively.
The engagement team members should ask themselves the following questions when applying professional judgment in deciding which systems and practices are essential to include within the scope of the audit:
-
Is this system and practice of particular interest to the board of directors? To Parliament? To the public?
-
Was this system and practice the subject of a recommendation or significant deficiency in the previous special examination report?
-
Are controls poorly designed or not adequately implemented?
When selecting additional, non-core systems and practices for examination, the audit team should keep in mind its responsibility to conclude on the audit objective, which is to determine whether the systems and practices we selected for examination were providing the corporation with reasonable assurance that its assets were safeguarded and controlled, its resources were managed economically and efficiently, and its operations were carried out effectively. The team should ensure that it clearly understands the corporation’s mandate—why it was created and what its objectives are—to assess whether its operations are being carried out effectively.
Core systems and practices and standard criteria, as well as additional guidance for expanding the scope of a special examination based on risk and internal controls assessment, can be found in the document “Special Examination Audit Approach.”
Determining the effect of affiliated entities on the scope
The scoping decision for a special examination should be based on how significant the less than wholly-owned (LTWO) affiliated entity’s operations are to the parent Crown corporation. The scope of a special examination could include the systems and practices in place in the parent Crown corporation to manage the LTWO-affiliated entity. For example, in lines of enquiry related to governance, risk management, strategic planning, and so on, the audit team could include the systems and practices in place to manage the LTWO-affiliated entity. Alternatively, the audit team could develop a separate line of enquiry to examine the parent’s management systems in relation to the LTWO-affiliated entity.
-
Either of these audit approaches would allow the audit team to examine the systems and practices designed to manage a significant LTWO-affiliated entity in order to gain reasonable assurance that the parent Crown corporation can fulfill its mandate and meet its statutory control objectives.
-
If the audit team is unable to obtain sufficient appropriate evidence to conclude against the selected criteria for all of the operations of the parent Crown corporation, including any significant LTWO-affiliated entity, the OAG may need to express a qualification in the special examination conclusion.
-
To avoid a qualification of the conclusion, the audit team may need to collect evidence that exists only in the LTWO-affiliated entity. For example, the team may need documents related to the LTWO-affiliated entity’s strategic planning and risk management.
In this case, Section 144 of the Financial Administration Act (FAA) applies. Note that this section of the FAA does not use the wording “wholly-owned” as found in the special examination mandate in subsection 138(1). Therefore, based on Section 144 of the FAA, the OAG can obtain access to records or documents.
For further information on affiliated entities, see OAG Audit 4010 Understanding the subject matter in planning an audit.
Auditability
Auditability relates to the ability to carry out the audit according to professional standards and audit policies. Although some areas may be significant, they may not be auditable for one or more of the following reasons:
- the area is outside the mandate of the OAG;
- the audit team does not have or cannot acquire the required expertise;
- the area is undergoing significant and fundamental change;
- suitable criteria or approaches are not available to assess performance; or
- the information or evidence required is not available or cannot be obtained efficiently.
The audit team should refer to its assessment of engagement risk when considering auditability (OAG Audit 4020 Risk assessment).
Period covered by the audit
Another consideration relevant to the scope is the period covered by the audit. This is the time frame covered by the audit work for which a conclusion will be formed; for example, the three fiscal years before the audit.
The audit team may decide to have a shorter audit period, such as one-year. However, in consultation with the OAG Internal Specialist Research and Quantitative Analysis, the team may analyze multiple years of data to ensure sufficient appropriate evidence that includes a longer term trend analysis.
An important consideration when selecting the period covered by the audit is that the audit team must obtain sufficient appropriate evidence to conclude on the audit objective. It is important that the collective weight of the evidence provides persuasive support for the findings in the report (OAG Audit 1051 Sufficient appropriate audit evidence).
The audit team should select a time frame to enable it to assess the entity’s performance against the criteria for a period that is long enough to cover any events that the team wants to take into account, such as a change in operations. The audit team should examine all parts of the audit period. If different lines of enquiry or different audit tests cover different time periods, this should be disclosed in the About the Audit section of the report. The team should also include a rationale for the period(s) chosen. For example:
“The audit will assess practices in place during the period of the audit, 1 April 2019 to 31 March 2020. To assess whether commitments and standards reflect client needs and the organizations’ responsiveness to service quality issues, we also examined the files for activities that occurred during the period of 1 July 2017 to 31 March 2020. This three-year span was selected as there were significant changes to service delivery approaches in June 2019 and we expected to see evidence of the results of these changes.”
Special examinations
The special examination provides a conclusion on the Crown corporation’s systems and practices that are current and in place at the time of the audit. The audit conclusion relates to these systems and practices as being able to provide assurance to the corporation that it met its statutory control objectives during this period. In consequence, the period covered by the audit for a special examination is often from the beginning of the planning phase until the end of the examination phase (end of field work)—however, the team may select another period to properly evaluate systems and practices in scope of the engagement.
Lines of enquiry
Lines of enquiry are a way to organize the audit work and correspond to areas to be audited within the scope. Lines of enquiry address the issues that the audit team wants to examine. Each line of enquiry should clearly address the risks identified in the team’s risk assessment (OAG Audit 4020 Risk assessment).
The number of lines of enquiry may depend on the nature, scope, and complexity of the audit issues and the value that each line of enquiry will be adding. Lines of enquiry are organized around audit issues, subject areas, systems and practices, or stages in a process. For example, for an audit of contracting, there could be lines of enquiry for the contract awarding process, the administration of contracts, and payment.
Lines of enquiry are determined to help the audit team finalize its audit approach as documented in the audit logic matrix. See OAG Audit 4044 Developing the audit strategy: audit logic matrix.
Inclusion of follow-up work
The team determines if any recommendations or issues from previous audits will be included in the scope. In determining the audit approach to address past issues, teams should consider the action taken by management to address the issues and whether it has a reasonable level of assurance on actions taken in response. For follow-up work on a matter that has previously been audited, the audit team may change or broaden the scope to consider changing circumstances or the emergence of new risks since the time of the original audit.
Audit issues from the original audit may also be dropped for the follow-up work, if they are no longer of interest, no longer pose a significant risk, or become irrelevant or obsolete. The rationale for dropping them is documented in the audit file.
Performance audits
If the audit team draws on recommendations from the Public Accounts Committee or other parliamentary committees, it should treat these as new or redefined issues.
Special examinations
The team reviews findings from the previous special examination to identify the systems and practices that represented a significant deficiency or warranted a recommendation for improvement. Summarizing the significant deficiencies or recommendations from the previous special examination report provides the audit team with considerable context and a strong starting point from which to gain an understanding of the entity and determine scope.
Using the work of others
While scoping the audit and determining the audit approach to be followed, the team needs to determine the possibility of using the work of others and relying on it. The team can consider relying on the work of internal audit; using the work of audited entity’s experts; and/or involving internal or external experts to assist with aspects of the audit work for which the team does not have sufficient experience or expertise.
As explained under OAG Audit 2070 Use of experts, OAG Audit 4030 Reliance on internal audit and OAG Audit 4045 Evidence-gathering methods, the team needs to assess whether the work performed by others is relevant before using it for the audit. It also needs to assess whether reasonable assurance can be obtained concerning the expertise, competence, and integrity of external experts and/or internal auditors.
Relying on the work of others and on existing information could reduce the number of hours and the extent of audit effort required.
Special examinations
The Financial Administration Act requires the audit team to consider relying on work performed by the Crown corporation’s internal audit function, as far as it is practicable to do so. Auditors should assess the relevant systems and practices identified in their lines of enquiry, as well as areas where they may rely on the internal audit function.
Determining nature and extent of procedures including limitations
During planning, in order to allocate sufficient resources to the audit project, the audit team assesses and documents the expected extent of audit effort required to obtain reasonable assurance. This assessment requires professional judgment and should consider the concepts of significance (OAG Audit 2020 Significance) and risk, including the impact on engagement risk (OAG Audit 4020 Risk assessment).
Determining the nature and extent of audit procedures is also driven by sources of evidence as set out in the audit logic matrix (see OAG Audit 4044 Developing the audit strategy: audit logic matrix), by evidence-gathering techniques to be used during the audit (OAG Audit 4045 Evidence-gathering methods) and possible scope limitations (i.e. inability to obtain sufficient appropriate evidence), and its impact on the conclusion (OAG Audit 7040 Audit conclusion).
Next steps
The audit scope and approach, including lines of enquiry, should be included in the audit logic matrix (OAG Audit 4044 Developing the audit strategy: audit logic matrix), which also includes the objectives, criteria and their sources, audit questions, data sources, and other information. The team must document its scope and approach decisions and the professional judgment exercised. The use of templates and tools discussed in OAG Audit 4020 Risk assessment and OAG Audit 4025 Internal controls facilitate the teams’ documentation.
Audit Guidance
Guidance to Integrate Value Added into the Performance Audit Process
Guidance on Risk-based Planning and Scoping for Direct Engagement
Guidance to Integrate Results Measures into Performance Audits
Special Examination Audit Approach