7044 Audit sampling

Sep-2022

OAG Guidance

CAS 530 recognizes both non-statistical and statistical approaches for audit sampling therefore professional judgement is required when determining which approach is appropriate for an engagement.

The primary advantages of statistical audit sampling are a statistically derived sample size and a statistically determined evaluation of sampling risk. However, the advantages of statistical sampling may not outweigh the primary disadvantages, which include the use of formal techniques to determine sample size, to select the sample and to evaluate the results. Therefore, the preferable approach will depend on the circumstances of each audit and require the use of professional judgment.

The choice of non-statistical or statistical sampling does not directly affect decisions about the auditing procedures to be applied, the appropriateness of the audit evidence obtained with respect to individual items in the sample, or the actions that might be taken in light of the nature and cause of particular misstatements.

The underlying principles, procedures and matters relevant to the planning and performance of audit sampling are similar for both non‑statistical and statistical methods. Thus, while the procedures and methods used in non‑statistical sampling are less formal, they are nonetheless rigorous. Ordinarily, the application of non‑statistical sampling will result in a sample size comparable to the sample size resulting from an efficient and effectively designed statistical sample, considering the same sampling parameters. The audit sampling guidance in OAG Audit 7044.1, while directed to non‑statistical sampling applications, is grounded in statistical theory and is designed to provide the necessary information to determine appropriate sample sizes and selection techniques, and properly project and evaluate results from non‑statistical audit sampling.

On engagements where we believe statistical sampling is the preferred approach consider consulting the OAG Internal Specialist of Research and Quantitative Analysis. For further guidance on statistical sampling, see OAG Audit 7044.2.

OAG Guidance

Because of sampling risk and the fact that items tested in audit sampling are selected essentially using a random selection method, haphazard selection method or a systematic method, we first consider the use of targeted testing. Audit sampling will typically be used on populations that are made up of many homogeneous items when it is more efficient than targeted testing based on coverage (e.g., coverage would require a large number of items to be tested) or when, because of the nature of the population, there are insufficient good “targets” to allow use of risk‑based targeted testing (e.g., many homogeneous items with few or no particularly higher‑risk or unusual items).

When we use audit sampling, our primary concern is overstatement. However, it can be an effective technique to test for unrecorded items, if the plan for detecting such misstatements involves selecting from a source that will provide appropriate evidence of whether there is understatement. For example, when testing for understatement of accounts payable the population is not the accounts payable listing but rather subsequent disbursements, unpaid invoices, suppliers’ statements, unmatched receiving reports or other appropriate populations. Also, audit sampling is not appropriate when a population comprises non‑homogeneous items. For example, a software development entity may enter into long‑term contracts to develop customised solutions. If the terms and conditions of the development contracts were non‑standardised, the use of audit sampling would not be appropriate for testing costs and revenues associated with the development contracts.

Audit sampling is a tool primarily designed to obtain sufficient appropriate audit evidence to support the assertion that a sampling population is fairly stated. It is not well suited for populations with high error rates or to estimate misstatements for purposes of correcting a materially misstated account. As such audit sampling would ordinarily not be used when estimated misstatement is more than 5 percent of the sample population or more than 70 percent of tolerable misstatement. Consider the appropriate testing strategy before defaulting to audit sampling. When audit sampling is not considered practical because the resulting sample size is very large, we look to obtain evidence from other procedures if appropriate (e.g., controls testing, substantive analytics or targeted testing). In some cases, by adding or increasing evidence from other procedures, sampling may become practical because the desired level of evidence from the sample can be reduced and/or after more targeted testing, the sampling population (numerator in the sample size formula—see further in this section) decreases which results in a smaller sample size. In some cases where a high proportion of misstatement is expected or observed, it may be necessary for the entity to correct errors to reduce the error rate or to perform testing that can be audited to provide the necessary evidence. If this is the case, consider consultation with the OAG Internal Specialist of Research and Quantitative Analysis.

In addition to the general limitations referred to above, there are specific considerations related to performing non‑statistical sampling at the Supplemental level of assurance. As explained further in OAG Audit 7044.1, the Supplemental level of assurance provides limited confidence (and therefore limited evidence) and will not by itself provide sufficient substantive evidence at the FSLI assertion level. Therefore we would only use this level when we performed sufficient other substantive procedures related to the relevant FSLI assertion.

In other words, the limited substantive evidence provided by a non‑statistical sample at the Supplemental level of assurance needs to be combined with other substantive procedures to provide the desired substantive evidence at the FSLI assertion level. For further guidance on the Supplemental level of assurance, refer to OAG Audit 7044.1.

OAG Guidance

As explained in OAG Audit 7011.1, in certain circumstances we may apply a “two step” approach to detailed testing of revenue transactions. In particular, if specific criteria are satisfied we can initially limit the sample size to 120. If no misstatements related to our test objectives are identified for the relevant assertions under testing, further revenue transactions testing would not be required.

The two-step approach to revenue transaction testing recognizes that the representative selection of revenue transactions is not a stand‑alone test and once it is performed to a certain level of precision, we can obtain the desired level of assurance from a combination of the representative selection of transactions and other audit procedures.  Specifically, the two step approach recognizes that due to the fact that performance materiality as percentage of revenue can be extremely small (often less than 0.5 percent) non‑statistical samples of revenue can become quite large. When precision gets extremely small (due to the size of the account balance and/or materiality level) expanding the number of selections beyond 120 provides little incremental audit evidence and such evidence needed to achieve the desired level of assurance is provided by other procedures (e.g., confirmation of accounts receivable at a moderate level of evidence) supporting an expectation of zero misstatement due to error or fraud, combined with the other conditions described above.

When we obtain evidence at the Supplemental level of assurance assuming no misstatements result from the testing performed, 120 transactions represents tolerable misstatement in relation to the balance tested of 0.33 percent. The logic being that once we have obtained evidence at a precision of say 0.33 percent of the revenue balance tested, we have useful audit evidence, combined and corroborated with other evidence, to reduce the remaining risk of material misstatement to a sufficiently low level.  As noted in OAG Audit 7011.1, in order to use the 120 testing levels, an engagement team would need to meet certain criteria, including achieving at least a moderate level of evidence over accounts receivable and performing some targeted testing on revenue transactions.  These additional criteria combined with a test of details based on a representative selection of revenue transactions achieve our related audit objectives.

Consider whether the “two step” approach may be applied to testing revenue transactions. In such cases follow the guidance in OAG Audit 7011.1.

OAG Guidance

To assist with consistency of application we have developed an eight step approach to non‑statistical and statistical sampling. Each of the eight steps is outlined further in this topic and forms the basis of application when performing sampling as part of OAG Audit.

OAG Audit 7044.1 Eight Step Approach to Performing Non‑Statistical Audit Sampling

OAG Audit 7044.2 Eight Step Approach to Performing Statistical Audit Sampling