2042 Considerations for using detailed electronic entity data in audit procedures

Sep-2022

Assess data quality and availability

Assess whether quality data will be available in a useable form and is easily accessible for efficient use.

Obtain detailed electronic entity data and determine its completeness and accuracy

Obtain the data from the entity’s systems and import it into whatever OAG or vendor tool will be used for the analysis. Perform procedures to determine the completeness and accuracy of the data, and, where possible, tie the data out to source documents subject to other audit procedures, such as a trial balance.

Structure, sort, analyze, disaggregate, and sample depending on use of the data

Run reports / queries based on the expected use of the data.

Disaggregate the transactions by their source and link to the entity’s processes and systems. Run relevant reports to identify conditions indicative of possible control deficiencies.

Risk assessment and developing audit strategy

Use source reports and additional risk assessment analytics to help determine areas of higher risk. Develop audit strategy based on our understanding of the entity, its internal control and our risk assessment.

Respond to risk and gather evidence

Use detailed electronic entity data to support:

• Controls testing
• Substantive analytics
• Substantive tests of detail
• Journal entry testing
• Fraud detection

Communicate with management and those charged with governance

Use detailed electronic entity data to enhance our required communications with management and those charged with governance and provide value added comments.

Document the use of electronic entity data in the audit file

We need to determine that the documentation in the audit file meets the requirements as set out in OAG Audit 1100.

The audit documentation of the use of electronic entity data in our audit needs to reflect the procedures performed to obtain the data, the validation of the data obtained, the testing performed and evidence obtained to verify the electronic data (e.g. agreeing revenue transaction data to customer orders or contracts for a manufacturing company) and the conclusions we reached.

Identifying and Assessing the Risks of Material Misstatement

General

Assess data quality and availability—The first step in using data is to assess whether quality data will be available in a usable form and is easily accessible for efficient use. Related considerations include:

• The nature of the entity’s business activities and processes. Data intensive businesses, e.g., those with numerous transactions that are highly systems dependent, will generally have more readily accessible data.

• The nature of the systems being used by the entity and their degree of integration. Normally it will be easier to extract or have the entity export data from standard packaged accounting or ERP software with integrated modules. Also, it is more likely that either an OAG or vendor tool will have standard importing and reporting capabilities for such software. When standard entity packages or extraction and reporting tools are not available, more customized solutions can be developed, but likely only efficient when used for several years.

• Depending on our purpose in using the data, we may want to consider the design and operating effectiveness of the entity’s internal controls over such data and the level of sophistication of the entity’s IT department when determining our other procedures. Accessing the entity’s data will normally require help from the IT personnel with an understanding of their systems and data structures. The more knowledgeable the entity is in this area, the more efficiently and expediently we will be able to obtain the data.

• The knowledge of the engagement team and availability of data analytics specialists. Engagements teams, including Data Analytics specialists, may have sufficient knowledge to effectively and efficiently use detailed electronic entity data at smaller entities. The larger and more complex the entity and its processes and systems, the more likely that IT Audit specialists will be needed.

See OAG Audit 7592 for detailed data retrieval considerations. Also, see OAG Audit 7033.1 for the need to evaluate the reliability of the data.

General

Obtain Data and Verify Completeness and Accuracy—A number of techniques are available to obtain detailed electronic entity data from the entity, depending on the systems they are using. A Data Analytics specialist can assist in this determination.

When to obtain the data will depend (among other factors) on whether this is the first year we are using the detailed electronic entity data or a subsequent year. In the first year, we may need to go through some or all the steps below to understand what data is available and needed to support our audit procedures. Obtaining the data may be done in phases as we determine what data is needed, i.e., a data request related to accounts and transactions to perform analytics and support our understanding of the business, in some circumstances, could be used later to support specific requests related to substantive tests.

In subsequent years we may know in advance what data will be needed and can make the data request to the entity early in the audit process (after confirming that there have been no significant changes in the process or systems).

It is recommended that we start with small and manageable data sets and expand where plausible. General Ledger, Accounts Receivable, Property Plant & Equipment and Accounts Payable sub-ledger and transaction level data may be desirable to obtain. Depending on the nature of the entity’s business and known risk areas, other ledger data could be helpful, e.g., Inventory. Two years of historical data and/or current year-to-date data on a quarterly basis may be desirable in certain circumstances.

Once the data is obtained from the entity, the data extracted needs to be verified for its completeness and accuracy. With the use of some CAATs, some data integrity checks will be performed by the CAATs but additional completeness and accuracy checks will likely be necessary. At a minimum, the data would be agreed to the entity’s trial balance. If you are extracting non‑financial data (for example, a file with all changes to standing data over a certain period) procedures over the accuracy and completeness of the data also need to be performed and documented.

See OAG Audit 7592 for detailed data retrieval considerations. Also, see OAG Audit 4028.4 for some examples on report testing with and without ITGC reliance. We would assess the completeness and accuracy of the source data (to verify that data was completely and accurately entered into the system) and also assess the completeness and accuracy of the report (to verify that the report logic completely and accurately captures the data entered into the system).

Note: Due to technical knowledge needed to access and analyze data, these procedures are normally performed by Data Analytics specialists. Consult Data Analytics specialists when planning to use detailed data or perform CAATs as part of audit procedures. See also OAG Audit 3102 on involvement of IT Audit. It is possible that IT Audit or Data Analytics specialists may be needed to develop the initial data retrieval in the first year, but the core assurance team, with some help from IT Audit or Data Analytics & Research Methods, can perform the procedures in subsequent years, assuming no significant changes in the entity’s processes or systems.

Understand the entity and its environment, including its system of internal control

Disaggregated Data—To support our understanding of the entity and its environment, the transactions would be disaggregated by the source of the entries recorded (e.g., sales, purchases, wages, adjusting entries) with their sources linked to applications used for processing. The appropriateness of the sources used would be assessed and linked to our understanding of the entity’s processes and systems.

As detailed in OAG Audit 5034, understanding the information system and communication involves determining how financial information is generated, including mapping the linkage between significant processes and the financial statements. Related considerations include:

• How is the data prepared (e.g., regular data feeds from the general ledger system to a data warehouse or via the use of a reporting tool, or download to spreadsheet)?

• What is the source of the information (e.g., database/extract from data warehouse, general and sub- ledgers, budget, spreadsheets, external third party vendor or source)?

• Which key financial statement areas are predominantly system derived (e.g., are the transactions that make up a financial statement area summarized electronically)?

• How do the key financial statement line items link to business processes?

• What transactions occur within the significant business processes?

• How are these transactions initiated (are they initiated electronically)?

Reports generated by the entity can support our understanding of the entity. A standard report that is often used shows a breakdown of ledger accounts by its sources. These sources give evidence of where the recorded entries originate and can be used as part of the basis for our understanding of the entity’s process flow and systems. Of special interest would be entries originating from unknown sources and a breakdown of manual vs. electronic feeds. Manual feeds may be indicative of non-routine processes.

Reports can also be run on accounts that can give a preliminary indication of whether adequate controls are in place for audit reliance. For example, a report that shows payments for goods or services without a required invoice could be an indication that payment controls are not operating effectively and therefore warrants further consideration. If the entity has a control procedure where the initiation and approval of manual journal entries is supposed to be segregated but a detailed summary of recorded manual journal entries shows instances where the initiation and approval is performed by the same person, this is likely an indication the control is not designed or operating effectively and warrants further investigation. We can develop a more efficient and effective audit plan by detecting such risks early in planning. In this example, we would avoid performing testing of controls already determined not to be designed or operating effectively and instead plan substantive procedures to appropriately respond to the assessed risk.

See OAG Audit 5034 and OAG Audit 5035.4 for further guidance on determining how financial information is generated, including mapping the linkage between significant processes and the financial statements and assertions.

Risk Assessment Analytics

Perform risk assessment analytics and evaluate reports for anomalies

Using the data available, additional risk assessment analytical procedures, covering one or more of the five types (trend, ratio analysis, reasonableness, regression analysis, scanning), can be performed at this stage to further our understanding of audit risks. In some circumstances, the data may also be used in substantive analytical procedures.

Identify Risks Relevant to the Audit Whether due to Fraud and/or Error

Data analysis may assist us in focusing on unusual transactions.

Using prior audit experience and our understanding of the entity and its business processes, routine transactions could be defined as the transactions that normally follow a specific path. We refer to transactions that follow the typical path as “expected”. We need to apply our understanding of the entity and business processes to determine the criteria for which transactions are expected.  When a transaction does not meet our criteria, it is considered an unexpected transaction and we apply judgment in determining an appropriate response, including any substantive procedures necessary to evaluate whether a material misstatement may exist. For example, revenue may normally be recognized only when there is a customer purchase order and a shipping document posted in an entity’s system. Data analysis may help us identify the transactions recorded outside the norm, based on pre-defined criteria, such as the transaction’s timing, source or type, whether the transaction is recorded manually, involves unusual circumstances, etc. In the preceding example, if revenue was recognized although there was no customer purchase order posted in the system, this would be an unexpected transaction that may warrant further investigation.

Related Guidance

OAG Audit 7032 – Suitability of Analytical Procedures

OAG Audit 5012.2 – Use Of Risk Assessment Analytical Procedures

OAG Audit 7591 – Use of Computer Assisted Audit Techniques (CAATs)

Develop Audit Strategy

Use detailed electronic entity data to help with risk assessment and developing the audit strategy

The use of detailed electronic entity data can help us develop our audit strategy in a number of ways. The use of more effective risk assessment analytical procedures that indicate unusual or non-routine transactions can identify accounts of significant risks early in the audit process, enabling us to plan for these risks up front. For example, analytics on data disaggregated by journal entry source may reveal an unexpected high number of non-routine transactions.

Also, disaggregated analytics can provide information on account sources below materiality levels requiring little or no audit procedures.

Evidence of possible deficiencies in the design or operating effectiveness of controls can impact the planned level of controls reliance. For example, if data indicates that there may be deficiencies in controls over authorization of manual journal entries, then our audit plan may need to include more substantive procedures in order to obtain sufficient audit evidence. By determining this in the planning phase of the audit, efficiencies may be realized and possibly a more effective audit plan may be developed.

Related Guidance

See OAG Audit 2043 on how to use data reporting to help understand the entity’s process flows and risks.

Respond to Risk and Gather Evidence

Tests of Controls

Controls testing—The use of detailed electronic entity data can support our controls testing in several areas, including:

• Using this data can assist in our sample selection, e.g., generating a list of items from the system and selecting a random or haphazard sample, or testing a complete population. Automating this process may save considerable time.

• Testing for conditions that would not be present if controls are operating effectively, e.g., credit memos without required approvals, or users with unexpected system access rights. Note that this is information that may be indicative of control deficiencies but can not substitute for an actual test of the underlying control.

• Developing an efficient audit response to an identified control deficiency. For example, our access controls testing has determined that the controller has inappropriate access to process transactional data. Performing an electronic analysis of the transactional data or related security data to identify and target actual instances of the controller using this access may provide sufficient evidence to evaluate the magnitude of the risk of misstatement due to error of fraud as a result of these access rights.

• In some circumstances we may use this data to perform dual purpose testing procedures for example to perform controls testing and substantive analytical procedures OAG Audit 6053.

Related Guidance

See OAG Audit 7591 on CAAT testing techniques for further details.

Substantive Analytical Procedures

The use of CAATs with detailed electronic entity data can support our substantive analytical procedures or tests of details by automating aspects of the test or procedure to make the audit more effective and efficient.

Example

Subsequent events testing may be performed as a kind of scanning analytic on the accounts payable file by filtering, sorting and stratifying the transactions to identify any items for follow up.

Related Guidance

OAG Audit 7591 provides further details on CAAT techniques and examples of the most common uses of CAATs in substantive tests.

Tests of Details

CAATs can be very useful when performing tests of details. CAATs using detailed electronic entity data can automate the process which can provide efficiencies. The effectiveness of the testing may also be improved by allowing for a greater variety of tests not practical in a manual environment and allowing the tests to take into account a larger set of transactions than we may be able to consider in a manual test. Using CAATs alone will generally not provide sufficient audit evidence. Testing of the underlying data by reference to third party evidence (e.g. customer contract) and seeking other corroborative evidence is still needed to support our audit conclusions.

Journal Entry testing—For example, CAATs can be used to generate a list of journal entries meeting our pre-defined fraud risk criteria that need to be tested (i.e. corroborating evidence for the journal entries needs to be obtained), e.g., journal entries with unexpected account combinations, journal entries posted by unexpected users.

Search for Unrecorded Liabilities—For example, CAATs can be used to generate a list of invoices entered into the entity’s system after the period end date, for which the invoice document date and date in the general ledger are in different periods. Using this analysis, we may concentrate our completeness testing on these items as part of our search for unrecorded liabilities. In addition to performing this substantive test, engagement teams may also utilize this output when testing the entity’s controls over the completeness of accounts payable.

Related Guidance

For further guidance on journal entry testing see OAG Audit 5509.

To assist in fraud detection—As part of tests of details, CAATs can also assist in fraud detection by enabling us to sort through large volumes of data, use advanced statistical techniques, and search or compare data from different sources for unusual combinations.

Example

For example, CAATs can be used on unusually large volumes of transactions taking place just prior to a period end, which may be indicative of revenue recognition manipulation, (e.g., sales being recorded in the wrong accounting period to meet expectations). Statistical analysis (e.g., Benfords Law analysis) can help detect fraudulent transactions. Transfers from Profit & Loss to Balance Sheet accounts vulnerable to manipulation such as inventory and accounts receivable may also be indicative of fraudulent activity.

Related Guidance

See OAG Audit 5514 for further examples of how CAATs can be used to detect fraud.

Communication of Results

Communication with Management and Those Charged with Governance

We need to communicate our audit findings to management and those charged with governance, as appropriate. The use of detailed electronic entity data to support our findings can be helpful by providing clear quantification of the nature and extent of identified issues.