Annual Audit Manual
COPYRIGHT NOTICE — This document is intended for internal use. It cannot be distributed to or reproduced by third parties without prior written permission from the Copyright Coordinator for the Office of the Auditor General of Canada. This includes email, fax, mail and hand delivery, or use of any other method of distribution or reproduction. CPA Canada Handbook sections and excerpts are reproduced herein for your non-commercial use with the permission of The Chartered Professional Accountants of Canada (“CPA Canada”). These may not be modified, copied or distributed in any form as this would infringe CPA Canada’s copyright. Reproduced, with permission, from the CPA Canada Handbook, The Chartered Professional Accountants of Canada, Toronto, Canada.
7592 Considerations when performing CAATs
Jul-2017
In This Section
Assess data quality and availability
Determine data obtained is complete and accurate
Structure, sort, analyze and disaggregate data
Overview
This topic explains:
- Key considerations when obtaining detailed electronic data from the entity
OAG Guidance
Understanding the type of data that will be available to perform CAATs can be performed during planning in conjunction with obtaining an understanding of the information system, including the related business processes, relevant to financial reporting.
We need to assess whether quality data will be available in a usable form that is easily accessible for efficient use.
A Data Analytics specialist can assist engagement teams in determining whether appropriate data is available and coordinate with the entity to obtain the data from the entity’s system.
OAG Guidance
First Time Retrieval
In case of the first time retrieval of the data, we perform an initial assessment of the data needed and, if needed, an IT Audit specialist would be engaged. The data requirements will differ depending on the objectives of the audit procedure and the period of coverage. Once agreed, a formal data request could be communicated to the entity’s IT department with set expectations as to the format of the data, mode of transfer and timing. We need to retain the extraction scripts of the data for our records and document how we achieved the completeness of the population and the accuracy of the fields that we selected.
Repeat Retrieval
After the first year, the prior year’s data request can be used as a starting point. We can modify the request based on our understanding of any changes in the scope of our procedures we plan to perform, after consultation within our team and evaluating any system / process changes. When sending in the data request include sample files, as well as extraction scripts from the prior year to assist the entity’s IT department process the request.
OAG Guidance
Based on the timing of the audit and the lead time required to perform the analysis, data needs to be requested far enough in advance to determine that the analysis is available on time, taking into account the time which may be needed to resolve any data retrieval issues, such as quality, completeness and accuracy of the data (right fields, all the right records, etc.)
OAG Guidance
Data is normally obtained in one of the following formats. However, depending on the capabilities of the entity’s application and the tool expected to be used to analyze the data, other formats may be acceptable. Local specialists from Data Analytics and Research Methods can help us assess which tool is most appropriate for the situation, help obtain the data in a suitable format and convert the format, if needed. Typical formats are as follows:
- MS Excel.
- XML.
- DBF (dBase),
- Delimited (use ~ or ^ or | as delimiter),
- Flat file,
- ASCII print file, or
- ODBC compatible (i.e. MS Access),
- any other format with prior consultation with the entity.
A record layout or data definitions describing each column, type (numeric, text, etc.), field length (if applicable) needs to be obtained from the entity.
OAG Guidance
Data can be obtained via CD, DVD, secured FTP or using other mobile storage devices (USB storage). Check that data transferred is encrypted and password protected. Large files can be compressed using an application such as WinZip or WinRAR. (For further security over the file, a password can be applied to a zipped file by the entity and obtained from the entity upon receipt of the file.)
OAG Guidance
Data needs to be stored on OAG servers at a central location with access restricted to the engagement team working on the engagement with predefined naming conventions. Regulations and guidance need to be reviewed for any additional security and privacy requirements. Retain the data after the audit period in accordance with OAG Audit and any regulations.
OAG Guidance
Our intentions and reasons for requesting the data need to be communicated to the entity. A secured FTP site needs to be used while transferring the data over FTP. If the data is transferred using other means such as CD, DVD or via email, in case the size of data is small, determine that data is appropriately encrypted and password protected.
OAG Guidance
Perform procedures to determine the completeness and accuracy of the data obtained for use in the CAAT. The objective in determining the completeness and accuracy data to be used in a CAAT differs from the objective of addressing the financial statement assertions for a particular FSLI. The objective in determining that data to be used in a CAAT is complete and accurate is to verify that the data received from the client reflect what has actually been recorded.
Assurance over the completeness and accuracy of data can be obtained through either test of controls or substantive testing.
OAG Guidance
Once the appropriate data is obtained and completeness and accuracy has been assessed, we can use computerized software tools to perform the CAAT. CAATs can be utilized to structure, sort, analyze and disaggregate the client data. Potential CAATs include: running reports or queries based on expected use of the data, disaggregating transactions by source and linking to the entity’s processes and systems, or running relevant reports to search for conditions indicative of possible control deficiencies or anomalies to target test.
OAG Guidance
To verify that the CAAT was executed appropriately, agree the information in the output report to the underlying client data originally obtained and previously tested for completeness and accuracy. For example, agree total customer cash receipts per the output of the CAATs analysis to the customer cash receipt data assessed for completeness and accuracy. Transactions identified by the CAAT as unusual based on the predefined criteria will be the key focus of testing as they represent unusual items that may present higher risk (although substantive procedures over ‘usual’ transactions would normally need to be performed as well).
OAG Guidance
Findings need to be communicated to the entity contact as agreed during the planning phase after the findings are discussed with the engagement team. Findings can be communicated to the entity in a standard format and in a timeframe to allow reasonable time for the entity to respond.