Guidance to Integrate Value Added into the Performance Audit Process

Objectives

The objectives of this guide are

• to help audit teams identify and document the value added (VALAD) of proposed audits when selecting audit topics;
• to help audit teams document with greater precision the VALAD of the audit during the planning phase; and
• to remind audit teams to consider the planned VALAD of the audit when assessing audit findings, developing the report strategy, and drafting the audit report.

Key points of intersection in the performance audit process

There are several points of intersection in the performance audit process for determining, assessing, and communicating VALAD, as shown in this chart.

Integrating VALAD into the audit process

To integrate VALAD into the performance audit process, practitioners should state the expected VALAD throughout the audit process in the form of VALAD statements. An audit can add value by providing information, advice, and assurance.

Some of the key characteristics of good VALAD statements include the following elements:

• They are based, to the extent possible, on SMART principles (specific, measurable, achievable, relevant and realistic, time-based).
• They are expressed as an end result.
• They reflect, to the extent possible, quantitative impacts, such as cost savings, cost avoidance, streamlining (efficiency), and reduction of time delays. However, not every audit lends itself to quantitative impacts. In such cases, VALAD statements are solely qualitative.
• They indicate who the audience of the information, advice, and assurance is.
• They indicate why the planned VALAD is important (so what).
• They indicate whether the root causes (why so) of potential audit findings will be identified.

Establishing VALAD statements during the selection of audit topics

Strategic Audit Plan. One of the key objectives of a strategic audit plan (SAP) is to ensure that the Office’s resources are focused on the areas of highest significance and relevance to Parliament and that the audits will add value to Parliament and Canadians. Therefore, audit teams are asked to propose audits that, among other things, will make a difference and add value. This is done through the Audit Report Submission.

Audit Report Submission. Practitioners are asked to include the value proposition in the report submission. VALAD statements in this phase allow the Office to decide whether the audit should proceed or not, based on the value proposition. The value proposition is to be seen as part of a business case for doing the audit. (See Appendix 1 for an example.)

The following questions will help practitioners consider the VALAD of the proposed audits.

Will the audit add value by providing assurance on the subject matter? For example:

• Why does Parliament need assurance on the proposed subject matter (for example, is it important subject matter or financially material)?
• Is the Office the sole provider of that assurance?
• Has internal audit or program evaluation assessed the subject matter area recently? If so, will that assessment affect the VALAD of the proposed audit?
• If the audit focuses on known deficiencies, how will it add value?

Will the audit add value by providing advice on the subject matter? For example:

• Given the identified or suspected issues, what type of advice can the Office provide that would lead to significant improvements in accountability or government operations?
• Without carrying out the proposed audit, would the entity likely fix the problems?

Will the audit add value by providing information on the subject matter? For example:

• Would the audit provide new insights by, for example, disclosing information not currently public?
• Would the information provided in the report build or increase the awareness, or grasp the importance or scale of a subject, issue, or identified deficiency?
• Would the information provided be timely in relation to a public policy matter or debate or matter of significant risk to Parliament, the public, or both?
• Would the audit quantify the impact of shortcomings (for example, case illustrations, quantification of cost savings, and quantification of results gap)?
• Would the audit demystify myths?
• Is the audit likely to reveal information that management or government would not wish to become public?
• Would the audit provide information on the consequences of the observations (so what) as well as the root causes (why so)?

Will the audit add value by providing other benefits concerning the subject matter? For example:

• What additional knowledge will the audit acquire (such as criteria or approaches that may lead to other audits) or be useful to other product lines?
• Other than Parliament and management, will other parties benefit from the audit (for example, interest groups and academics)?
• Is it likely that management would take action in areas not scoped for the audit?

Establishing VALAD statements during the planning phase of the audit

Audit Logic Matrix (ALM). Towards the end of the planning phase, the audit team has a deeper knowledge and understanding of the underlying subject matter and the entity(ies). Practitioners are asked to expand on the VALAD statements developed at the Audit Report Submission stage and use their newly acquired knowledge to write more precise VALAD statements. Also, practitioners are asked to indicate how each line of enquiry (LOE) will contribute to the overall VALAD statement(s). At the ALM stage, VALAD statements allow the Office to decide whether the audit, including each proposed LOE, should proceed or not. (See Appendix 2 for an example.)

Considering VALAD statements during the reporting phase of the audit

Draft Audit Report. Practitioners are asked to consider VALAD statements included in the ALM when drafting the report.

Considering VALAD after the audit

Post-audit Survey. Within four weeks of an audit report being tabled in Parliament, a post-audit survey is sent to entities. Entities are asked to respond to questions related to the VALAD of the audit report, recognizing that Parliament is our client. Responses to the questionnaire are to be shared with principals.

Example of VALAD statements in an Audit Report Submission and an ALM

Below are examples of VALAD statements using an audit report that was tabled in November 2011: Regulating Pharmaceutical Drugs—Health Canada.

Appendix 1—Planned VALAD statements in the Audit Report Submission

State the expected outcomes of the performance audit identified to date that justify doing the audit.
	VALAD Statement
Assurance to be provided	Parliament will have a reliable assessment on how Health Canada is meeting its regulatory responsibilities in managing the human prescription drugs program. This is important because Canadians who purchase and consume pharmaceuticals authorized for sale in Canada rely on the government and industry to monitor the safety of these drugs.
Advice (recommendations) to address gaps–problems–risks	Because of significant financial pressures, the audit will provide advice on using a risk-based approach to its activities. We will also provide advice on the extent Health Canada needs to improve its conflict of interest process. This is important because officials responsible for reviewing drug submissions routinely handle commercially sensitive information that could be used for personal gain.
Informational—improving transparency/enhancing understanding	The audit will provide information on the timeliness of communications on the safety risks to Canadians concerning prescription drugs. This is important because if there are safety risks in using a particular drug, health professionals and Canadians need to know this quickly to reduce the health impacts. The audit will fill some of the information gaps left by Health Canada to deliver on increasing public expectation for transparency and accountability for results.
Other benefits	The audit will contribute to developing criteria that will be used in Health Canada audits on regulatory programs (for example, biologics and natural health products).

Appendix 2—Planned VALAD statements in the ALM

Review the VALAD statements provided in the Audit Report Submission. Update the statements, if necessary, and add specificity.
Assurance
Parliament will have a reliable assessment on whether Health Canada is fulfilling its key responsibilities for clinical trials and submission reviews, including the transparency of its actions.	LOE 1 LOE 2
Parliament will have a reliable assessment on whether Health Canada manages the risk to the review process arising from conflict of interest.	LOE 2
Parliament will have a reliable assessment on whether Health Canada monitors the safety of drugs on the market and communicates safety concerns in a timely manner.	LOE 3
Advice
The audit will provide advice to increase the transparency of key decisions related to clinical trials and the review process.	LOE 1 and 2
The audit will provide advice to better manage potential conflicts of interest related to the review process.	LOE 2
The audit will provide advice to increase the timeliness of safety assessments and the communication of safety risks to those who need to know.	LOE 3
Information
Interested parliamentarians and stakeholders will understand the extent to which recommendations from the 2004 and 2008 House Committee reports have been implemented.	LOE 1 to 3
Interested parliamentarians and stakeholders will understand for the first time, the time between identified post-market safety issues and Health Canada’s action. (so what)	LOE 3
Interested parliamentarians and stakeholders will have indications of key root causes for deficiencies. (why so)	LOE 1 to 3
Other benefits
The audit will contribute to developing criteria that will be used in Health Canada audits on regulatory programs (for example, blood, biologics, and natural health products).