6053 Extent of controls tests

Sep-2022

OAG Guidance

Identifying the testing population

For purposes of control testing, determining a testing population involves defining the appropriate and complete set of data (e.g., the instances of the control for which we are seeking to test operating effectiveness) from which we will select the sample and about which we wish to draw conclusions.

There may be a tendency to designate all instances of certain controls operating or transactions processed centrally as a single population for testing purposes. However, when assessing the homogeneity of a control testing population, a homogeneous testing population is limited to those control instances designed and operating in a manner such that it would be reasonable to draw conclusions across the entire population from the results of one sample.

Careful consideration is applied when determining the testing populations and whether they are homogeneous for sampling. Our experience and understanding of the business process and control activities are important in determining the homogeneity of a testing population. When a population is determined to be homogeneous for testing a control, we have determined the characteristics of the population allow for conclusions to be drawn across the entire population based upon the sample to be tested.

Our conclusion related to the homogeneity of a population for controls testing is based on carefully considering each of the factors described in the table below, as applicable.

In making our assessment of homogeneity, we document our overall rationale and judgment as to the homogeneity of the testing population. The relative importance of these factors will vary depending on the engagement-specific facts and circumstances.

When assessing homogeneity for control testing populations the evaluation may need to be performed on a control-by-control basis and not across a process in total.

We consider the following factors when determining populations of control instances for testing (after obtaining a thorough understanding of the business processes and controls relevant to the audit):

Factor	Homogeneity Not indicative<------------------------------------------------------------------> Indicative
Commonality of entity-level controls (ELCs), including the control environment	Different ELCs	Some ELCs are the same but not all	Same ELCs
Commonality of supervisory oversight and/or direct monitoring of the operation of the controls	Multiple supervisors	Limited number of supervisors	One supervisor
Uniformity of the entity’s policies and procedures, including execution of the controls	Diverse	Similar	Identical
Number of individuals executing the controls and their reporting lines	Larger number		Smaller number
Competency of the individuals executing the controls	Varied levels of competency		Consistent levels of competency
Commonality of training provided to individuals performing the controls	Diverse	Similar	Identical
Historical results and prior knowledge of the entity’s process and control results	Findings and understanding of the process indicate lack of consistency across the population		Findings and understanding of the process indicate consistency across the population

For example, consider the situation where there are multiple preparers and reviewers at the entity’s 50 components, all based in a single country, for reconciling a particular significant account (e.g., accounts receivable). Each component prepares monthly reconciliations for its individual accounts receivable balance and reviewers are independent of preparers. The method for reconciliation preparation, review, resolution of differences and reporting are in accordance with the entity’s standardized account reconciliation policy and procedures. The entity has a strong control environment. The control operators receive common training as to how to perform and/or review the reconciliations, and there are consistent levels of competency in the individuals involved in executing the control. Expectations, policies, and procedures are communicated to the preparers of the reconciliations, and component management monitors compliance with corporate-wide policies and procedures. The engagement team’s prior experience validates the consistency in the preparation and review of the reconciliations and the competency of the preparers and reviewers. In this situation, the engagement team would test the operating effectiveness of the oversight by component management concurrent with the testing of the reconciliation control to further validate the consistency in the preparation and review of the reconciliations. Given this fact pattern, it would be appropriate for the engagement team to conclude that the reconciliation processes across these 50 components are homogeneous and accordingly plan to test them as one population.

In contrast, in the situation where more than one individual performs the same control and the engagement team is not able to conclude that there is one homogeneous population due to the absence of factors described above (for example, the monitoring by component management), the engagement team would instead determine sample sizes and make sample selections separately for each of the unique testing populations. For example, if two individuals perform the reconciliations and each is considered its own testing population, then separate testing would need to be performed for each population.

Extent of controls testing

Consider the following guidance, which supplements on the points raised in CAS 330.A28, when considering the extent of testing of controls:

• The frequency of the control:
  • The more frequently a manual control procedure is performed(e.g., daily as opposed to monthly), generally the more items we test. The number of different individuals / locations performing the control will also impact the sample size.
• The expected deviation from the control:
  • Consider the risk that observation of the control, answers to inquiries and even signature evidence of operation may not be sufficient to determine whether the control operated effectively for the entire period during which we seek to rely on it. If evidence from other audit procedures indicates the control is not effective, we do not plan to rely on the control.
• The relevance and reliability of the audit evidence needed to be satisfied that the control operates effectively:
  • The more significant the risk addressed by the control and the more important the control is to addressing the risk, the more persuasive our evidence needs to be.
• The extent to which audit evidence is obtained from tests of other controls related to the assertion:
  • The more assurance we expect to receive from other audit procedures, including tests of other controls related to the risk that the control addresses, the fewer items we may need to test.
• The type and nature of the control we plan to test:
  • Generally, more items will be tested for manual controls than automated controls because manually applied controls are more prone to mistakes and random failures, whereas previously tested automated controls that have not changed will continue to be reliable, as long as the Information Technology General Controls (ITGCs) around the relevant computer systems and applications are working effectively. See OAG Audit 6054 for guidance on Automated Controls.
  • When other important controls depend on the operation of a specific control, we generally test more items of that control, e.g., an Information Technology General Control that supports the operation of important application controls.
  • When manual oversight or judgment is a necessary part of a control we plan to test (e.g. exception reports, analysis, evaluation, data input, information matching), we generally test more items.
  • Generally when control procedures are more complex we test more items.
  • The more susceptiblethe control is to management override, the more itemswe would expect to test.
• Whether there has been change in the design or operation of the controls:
  • We consider testing more itemswhere there has been change in the design or operations of the controls including volume or nature of transactions that might adversely affect operating effectiveness or changes in key personnel who perform the control or monitor its performance.

OAG Guidance

We may want to assess control risk as low, or in other situations we may assess control risk as high due to design and/or operating deficiencies. However, in cases where we do not assess control risk as low, we may still want to seek some assurance over controls and may do so by testing fewer items than is required to achieve a high level of assurance. Thus we may want to place some reliance on controls in establishing the nature, timing and extent of substantive procedures, but not at a high level. Audit theory and standards support a moderate level of controls testing to support an assessment that a control reduces control risk to a moderate level with the necessary addition of appropriately designed substantive tests (these substantive tests would be less than in circumstances where the audit strategy is no controls reliance and may be more than in circumstances where the audit strategy is high controls reliance as described below). (For a definition of control risk, refer to OAG Audit 5042).

Based on our planned controls reliance, we need to test an appropriate number of instances of controls to achieve the desired level of assurance. For example, to achieve a high level of assurance, we would generally need to test more instances of the control, than for situations when we only plan to achieve a moderate level of assurance. The following subsections provide guidance on appropriate sample sizes to be used when different levels of assurance are sought.

A Test of Controls template is available on the Intranet under Template and Checklists to assist with the appropriate application of the sample size guidance for all types of controls.

See OAG Audit 4028.4 for further guidance on the reliability of information generated by an IT application used in our audit.

High level of assurance from controls testing

In determining the number of items to test for manual controls, the numbers of items to test to achieve a high level of assurance from controls testing are provided below. Based on the frequency or on the assumed population of the control, we test an appropriate number of items.

Frequency of control	Assumed population of controls occurrences	Number of items to test
Annual	1	1
Quarterly	4	2
Monthly	12	2
Weekly	52	5
Daily	250	20
Multiple times per day	Over 250	25

The sample sizes above would ordinarily be sufficient to achieve a high level of assurance from controls. In rare circumstances where achieving a high level of assurance is necessary and we conclude that the sample sizes above will not provide sufficient evidence over the operating effectiveness of individual controls, we apply the sample sizes in the table below. Such circumstances would be rare, but may, for example, include situations when a combination of the following factors is present:

• We plan to rely on a complex control
• The control is related to a significant risk
• The control involves significant judgment
• We expect a high control deviation rate
• There is a significant element of manual oversight in control application.

Frequency of control	Assumed population of controls occurrences	Number of items to test
Annual	1	1
Quarterly	4	2
Monthly	12	2 to 5
Weekly	52	5, 10, 15
Daily	250	20, 30, 40
Multiple times per day	Over 250	25, 45, 60

Moderate level of assurance from controls testing

Where a moderate level of assurance is determined to be appropriate the numbers of items to test to achieve a moderate level of assurance from controls testing are provided below based on the frequency or the assumed population of the control.

Frequency of control	Assumed population of controls occurrences	Number of items to test
Annual	1	1
Quarterly	4	1
Monthly	12	2
Weekly	52	4
Daily	250	10
Multiple times per day	Over 250	15

The number of items to test reflected above provides a moderate level of assurance from control testing and only partially mitigates the risk to which the control relates. Our level of controls reliance is reflected in the Audit planning template in the planned level of controls reliance of High, Partial or None. Partial reliance is selected when we consider that the control (or controls) on which we intend to place reliance will only partially mitigate the risk to which they/it relates. When we plan to use a moderate level of assurance for controls testing, we will not be able to obtain High controls reliance because use of the sample sizes in the moderate assurance table above will only achieve Partial controls reliance for the individual control being tested. If we change our audit strategy from High controls reliance to Partial controls reliance, we may need to adjust the planned substantive evidence and perform different or additional substantive tests. OAG Audit 4024 provides guidance regarding the impact of controls reliance on substantive testing strategies and includes a definition and examples of when Partial controls reliance may be appropriate.

Also note that if we plan to test multiple controls, addressing the same assertions for the FSLI, at a Moderate level of assurance, in aggregate, we may be able to achieve high controls reliance for those assertions. Determining the level of assurance in such circumstances is a matter of professional judgment.

Example– Moderate level of assurance

We plan our testing for a FSLI where the inherent risk is assessed at the high end of the range of normal risk. We plan to rely on one control that operates on a weekly basis and addresses all relevant assertions for the FSLI. We plan to achieve Partial controls reliance and Low substantive evidence. We plan to test the control for a moderate level of assurance. The following table illustrates our planned approach:

Inherent risk	Expected controls reliance	Planned substantive evidence	Level of control assurance	Control sample size
Normal	Partial	Low	Moderate	4

Number of Occurrences of the Control

Based on the frequency the control operates (e.g., annually, quarterly, etc.) or number of occurrences of the control we will determine the number of items to test. For example, a sample size for a monthly control is based on a population of 12. Similarly, the sample size for testing of a daily control assumes a population of roughly 250, and the sample size for testing a control operating multiple times a day assumes a large population, effectively a population over 250.

Where the population of occurrences falls between the levels identified in the table above, we are able to interpolate the number of items to test between the levels indicated, exercising professional judgment to determine the appropriate sample size. Use the assumed population column as a reference when we know the number of occurrences during the period under review but the frequency does not align with the first column.

The reference to “items” in the third column refers to the number of occurrences of the control and will ordinarily be applied to tests involving reperformance and/or inspection of evidence, although there may be circumstances when inspection at these levels is not necessary. The table can also be applied to observation but in this case we determine how much evidence is enough to conclude on the effectiveness of the operation of the controls.

Test Design Considerations

Consider the following points:

• If the nature of an examination test will be such that reperforming the test will not take significantly longer, we would ordinarily reduce audit risk for similar effort by reperforming the control.
• The number selected from the table will ordinarily apply to the number of reperformances or the number of inspections or the number of these types of test combined. It would not normally be necessary to use sample sizes when the nature of the testing is inquiry or observation. In practice a combination of different types of tests may be efficient. For example, for a control performed multiple times a day (e.g., signature evidence), if we test 25 items we may decide that it is appropriate to obtain evidence from 20 tests of inspection and 5 tests of reperformance. However, note that in this case we do not achieve a ’reperformance’ level of evidence when combining reperformance with inspection, therefore whether a combination approach is appropriate depends on the risk and what level of evidence is sought.

Nature of evidence

The audit evidence obtained from testing controls will be influenced by the persuasiveness of evidence obtained for each control we test. When determining the persuasiveness of the evidence needed to support the conclusion that a control operates effectively, we may consider the following factors (note that the following list is not intended to be all‑inclusive):

• The inherent risk associated with the related account(s) and assertion(s) (The higher the inherent risk, the more persuasive the audit evidence that will be desired);
• The level of reliance placed on the control (The higher the level of reliance, the more persuasive the audit evidence that will be desired);
• The nature and materiality of misstatements that the control is intended to prevent or detect (The more material a potential misstatement associated with the operation of the control, the more persuasive the audit evidence that will be desired);
• The results of the previous years’ testing of the control, combined with an assessment of any changes in the control in the current year (Assuming no significant changes in the control, the more effective the control was determined to be in the prior year, the lower the level of audit evidence that may be required in the current year);
• Whether the account to which the control relates has a history of errors (The more significant the history of errors, the more persuasive the audit evidence that will be desired);
• Whether there have been changes in the volume or nature of transactions that might adversely affect control design or operating effectiveness (The more significant the change in the volume or nature of transactions, the more persuasive the audit evidence that will be desired);
• The effectiveness of entity-level controls that we have tested, especially controls that monitor other controls (The more effective the ELCs, the less persuasive the audit evidence may need to be);
• The nature of the control (The more routine a control, the less persuasive the evidence can be);
• The degree to which the control relies on the effectiveness of other controls (e.g., the control environment or information technology general controls) (The more effective the other independent controls, the less persuasive the audit evidence may need to be);
• The competence of the personnel who perform the control or monitor its performance and whether there have been changes in key personnel who perform the control or monitor its performance (The more competent the personnel, the less persuasive the evidence can be. The auditor may obtain more persuasive evidence to the extent the personnel performing the control is new to the organization or to the process);
• The complexity of the control and the significance of the judgments that need to be made in connection with its operation (The more complex the control and/or the more significant the judgments made in connection with its operation, the more persuasive the evidence that will be desired).

Persuasiveness

Obtaining ’more persuasive’ evidence means we may:

• Increase the quantity of the evidence (i.e., the extent of testing),
• Obtain evidence that is more relevant or reliable (i.e., vary the nature and timing of testing), and/or
• Test the control for a higher level of assurance (i.e. high assurance instead of moderate).

We would generally seek to place greater reliance on the effectiveness of a control and therefore aim to obtain more persuasive evidence if the control we intend to test:

• relates to a significant risk; or
• is particularly important to our testing plan, e.g., we judge the control to be particularly important to the sub‑process or FSLI, or the individual control test is the primary or only test to assess the effectiveness of the control for a particular assertion.

Source of audit evidence

We need to obtain evidence that supports not only that the control is operating but also that the control functions as designed to achieve the objective(s) of the control. Depending on the nature of the control, evidence can take many forms including:

• Detailed memorandum prepared by the control operator that outlines the considerations, rationale, and conclusions reached
• Email correspondence from the control operator demonstrating follow‑up and resolution of items that meet the criteria/thresholds
• Corroborating inquiry via discussions with others that interact with the control operator in his/her performance of the review (It is important to remain professionally skeptical, and remember that open ended questions generally provide more evidence than closed or leading questions, as does corroborating with multiple individuals)
• Detailed meeting minutes that include the considerations, rationale, conclusions, as well as how those were arrived at– more than just an executive summary
• Notations on forms, documents, or analyses that demonstrate the substance of the execution of the review
• Demonstrated recalculations, footing, cross footing, and agreement to underlying data and support when the control is designed to achieve those objectives

Control operators may conduct meetings to review certain information and take necessary action as part of relevant information processing and business performance review controls. These meetings are often conducted to review complex or subjective areas, such as significant judgments, estimates, assumptions underlying estimates, and other financial information. Examples include, but are not limited to, meetings to review estimates of future cash flows, income tax provisions and related accounts, assumptions used in accounting for business combinations, business performance reviews, and other judgmental areas. These types of controls may not lend themselves to reperformance testing. Rather, we may determine that observing such meetings is the most effective method of obtaining sufficient appropriate audit evidence to conclude the control is operating effectively.

Our planned procedures when obtaining evidence of the operating effectiveness of a control through observation at meetings include considering and documenting specific details regarding what we observed. Some examples of such details are:

• The specific actions requested and questions raised by the control operator during the meeting
• Understanding and evaluating thresholds used by the control operator to identify variances and investigate differences
• Completeness of the discussions, including whether specific matters addressed supported the designed precision level of the control
• Follow‑up items or changes resulting from the discussion which also provide evidence of the precision and effectiveness of the control design
• Whether the control operator demonstrated the requisite knowledge and experience
• The source of the information used in executing the control– for guidance on testing information generated by an IT application refer to OAG Audit 4028.4.
• Whether documents and schedules used in the operation of the control are consistent with the control operator’s ability to perform the control effectively as designed
• The linkage of what we observed in the meeting and how it supports the operating effectiveness of the control
• How agenda items or meeting summaries provide evidence of the precision and effectiveness of the review
• Other meeting information, including the date, who attended, etc.

Mere references to the occurrence of or attendance at meetings, or the inclusion of meeting agendas in the audit file without documenting details of the evidence obtained such as those described above, is unlikely to provide sufficient evidence of our control testing.

For example, to be sufficiently persuaded that the division controller’s variance analysis is effective, our tests might include the following for a sample of months: (1) tying out the source data and recalculating the variances, (2) obtaining corroborating evidence of the matters investigated by the controller, and (3) checking all matters requiring action were investigated. If the controller’s documented resolution of a significant variance in sales to Customer A is that the entity entered into a new arrangement with Customer A -‑ then we may need to obtain corroborating evidence of that new customer arrangement and its impact on sales as a means of testing the effectiveness and degree of precision of the controller’s analysis. (This example is intended to be illustrative, i.e., neither prescriptive nor complete.)

We need to determine that the control operates effectively on its own without considering the results of our substantive testing and we need to determine that the control operates at the level of precision contemplated in the design of the control. The audit evidence we seek needs to be sufficiently persuasive to make these determinations.

The nature and timing of audit procedures may be impacted by the fact that some of the accounting data and other information may be available only in electronic form or only at certain points or periods in time. See OAG Audit 6052 for further guidance on the use of electronic documentation.

Example Determining number of instances to test– High level of assurance

A company has a control that operates on a monthly basis in a large number of instances (one individual performs monthly bank reconciliations for all 50 of the company’s bank accounts). In the example, the first step is to convert the information given into the population of instances of the control’s operation on an annual basis, in this case 12 months X 50 bank reconciliations or 600. The next step is to select the period in the table whose population corresponds to the population we are testing. In this case, 600 would represent a large population and therefore we would select a sample size for controls operating multiple times a day (i.e., 25).

If for the example above, we had 15 reconciliations instead of 50, the total annual population would be 180. In this case, considering that the frequency is greater than 52 and lower than 250, we may determine an appropriate sample size based on the frequency of the control in relation to our controls test guidance sample sizes (i.e., 15).

Moreover, in the case of having more than one individual performing the same control, and in the absence of a central monitoring or review function, sample sizes are determined per individual basis, for example: if there are two individuals who perform the 600 reconciliations, then we will be testing 25 items for each individual, with a total of 50 items tested.

In other words, deciding on the number of items to test requires a determination as to whether the population we are testing is sufficiently homogeneous to permit it to be treated as one population and, therefore, permit the results of our testing to be appropriately projected to that population. If, for example, there are uniform procedures and review by the same supervisor, it would be reasonable to presume that the controls performed by the different individuals represented a single population. In such cases, we generally select roughly a similar quantity of our test items from each different individual, presuming one population but allowing the results of our testing to support or refute that presumption.

If a central monitoring or review function was in place it may be possible to treat the controls performed by different individuals as a single population. In this case in addition to testing the reconciliation process itself, we would perform a separate test of the supervisory review and approval of the reconciliations to support our conclusion that the reconciliations could be treated as a single homogeneous population. The extent of this test (i.e. number of items selected for testing) would depend mainly on the frequency of the supervisory control procedure.

Confidence Levels

While our controls testing is typically not a statistical approach, our guidance is consistent with the fundamentals of sampling theory. Less frequent controls (quarterly to daily) represent relatively small populations where statistical theories are of little help. For less frequent controls, the sample sizes providence in this guidance represent good judgment in determining how much evidence is necessary to conclude that a control is operating effectively, providing no exceptions are found.

When a control operates multiple times a day, the population is sufficiently large to apply sampling theory. The type of conclusions that can be reached considering our sample sizes and if applying statistical theories are illustrated below:

• Sample size of 25 with no exceptions - Conclusion: 90% confident that the exception rate is no higher than 8.8%.
• Sample size of 25 with no exceptions - Conclusion: 95% confident that the exception rate is no higher than 11.3%.
• Sample size of 15 with no exceptions - Conclusion: 73% confident that the exception rate is no higher than 8.5%.
• Sample size of 15 with no exceptions - Conclusion: 80% confident that the exception rate is no higher than 10.2%.

FAQs related to the application of section Control sample size guidance—Manual controls:

1. How do you determine the sample size for a control test in situations where a monthly control occurs over several accounts during the year (e.g. the control operator performs the control for 50 accounts on a monthly basis, i.e. 12 x 50 accounts = 600 frequency)?

Example:

The engagement team is planning to test a control relating to reconciling detailed accounts receivable records to the general ledger, which occurs 50 times each month (i.e., there are 50 different accounts receivable reconciliations being prepared monthly), and it is seeking a moderate level of assurance from testing this control.

Alternative 1: Assume the engagement team has determined the accounts receivable reconciliations represent a homogenous population for the purpose of testing the control, and it determined a sample size of 2 months is appropriate for testing the operating effectiveness of the control. Accordingly, in determining how many instances of the control to test in order to be persuaded that the control is operating consistently as designed, the engagement team considered the following data points:

• The aggregate number of instances the control operates throughout the year is 600 (i.e., 50 occurrences each month times 12 months) - This approximates multiple times per day. By analogy, this equates to 15 instances to test in order to achieve a moderate level of assurance using control sample size guidance. Using this approach, the engagement team would consider spreading the 15 instances to test over the 2 months selected for testing (i.e., 7 instances for one month and 8 instances for the other month).
• Given the control operates 50 times each month, this equates to 5 instances per month selected for testing by analogy to accept-reject testing - with no exceptions (i.e., 10 in the aggregate).

For this example, after considering these two data points and the risk associated with the control, available evidence, and experience from prior audits, the engagement team used professional judgment to determine the appropriate sample size (e.g., test 7 instances in one month and 8 instances in the other month for a total of 15 instances).

Alternative 2: An alternative to determine the number of instances of the control to test is to define the testing population as all instances of the control’s operation throughout the audit period (i.e. 600 in this example, rather than the 12 monthly occurrences used in alternative 1 above) and to select the sample for testing from across that entire population. Using the control sample size guidance for a moderate level of assurance, the engagement team determined the appropriate sample size to be 15 instances. By reference to the control sample size guidance for a moderate level of assurance the engagement team would select a minimum of 15 instances of the control to test– but the selections in this case would span the entire year and not just a few selected months, since that is how the engagement team is now defining the population and sampling unit.

2. When engagement teams plan to rely on the testing performed by Internal Audit do we need to independently test some of the controls or can we just reperform some of the testing performed by Internal Audit?

After assessing competence and objectivity of Internal Audit, the auditing standards allow the auditor to rely on the testing performed by Internal Audit without independently testing any controls. However, engagement teams should be selective in applying this guidance. Generally we should only be relying on Internal Audit for lower risk controls that are simple to execute by the control operator. Testing of controls which involve a high level of judgment or controls in higher risk areas of the file should be completed by the engagement team. In addition, we should be doing at least some independent testing of even the routine controls ourselves. This is an area of professional judgment, which should take into consideration an overall evaluation regarding the extent of work we are planning to use to determine that we are sufficiently involved in the audit to provide a basis for expressing our audit opinion.

3. When reperforming Internal Audit’s work, is there specific guidance on how much we have to reperform or is this based on professional judgment?

The guidance states that the auditor needs to select at least "some" of the controls for reperformance, i.e. selecting a "few" of the samples tested by Internal Audit and then reperform their work to see whether we get the same test results. How many controls we select for reperformance testing is a matter of professional judgment and accordingly, there is no set minimum level for this. However, we need to reperform enough controls to be comfortable with their work. In practice we often see engagement teams selecting somewhere between 5% and 10% of the controls for reperformance testing.

Determining the number of items to test when we are reperforming a control is also a matter of professional judgement. In practice we often see engagement teams selecting somewhere between 3‑5 items of the original sample of 20‑25. Engagement teams can use this as a benchmark, although again there is no formal policy for how many items we should select for reperformance. Reperformance would also include evaluating where the sample was selected from and how it was selected to determine if these steps were completed appropriately.

In addition to reperforming some of the testing done by Internal Audit, we also need to review all the work performed by Internal Audit that we are relying on. This should be done by an individual with the appropriate amount of experience and as part of the review, this individual should be assessing whether a reasonable sample size was used by Internal Audit and whether Internal Audit used the appropriate testing methodology.

OAG Guidance

We may design a test of controls to be performed concurrently with a substantive test of details using the same transactions. We refer to this as a dual purpose test. Although the objectives of the control and substantive tests are different, both may be accomplished concurrently. For example, we may examine a vendor invoice to determine whether it has been approved as part of the entity’s controls and to provide substantive audit evidence for the cut‑off assertion for the purchase of inventories. We must carefully consider the design and evaluation of such tests to check that both objectives are achieved.

Determining the sample size and sample selection method

When designing dual purpose tests, we first design the test of the control and then determine whether this test could efficiently provide substantive evidence as well. We need to test the objectives associated with the controls test and the substantive test using the appropriate sample size for each. In the example above, assume our control occurred multiple times per day and required a high level of assurance from testing and we selected accept-reject testing with a low desired level of evidence as our test of details. This scenario would yield a control sample of 25 invoices and 16 accept-reject tests. We would select the larger of the two sample sizes, 25, solely for determining the number of invoices to request from the client; however, we would perform testing over the number of items determined by the specific test in order to achieve each of the control and substantive testing objectives. This means we would test all 25 items for the controls test and would test the first 16 items selected for our accept-reject testing. It’s important to note that to maintain the integrity of our testing, it is the first 16 items out of total 25 items selected that we must use for our accept-reject testing. We are not able to pick and choose which 16 items we use for our accept-reject testing.

With both a control test and an accept-reject test, as in our example above, samples are selected on a random basis. Where the sample selection does not occur in the same way for the control and substantive tests that we are performing, the auditor must carefully consider the way in which the sample is selected so that each item has an equal opportunity of being selected. On the contrary, if we selected dollar-unit sampling as our substantive test to accompany our control test, there is a risk that bias may be introduced in the selection probabilities of items given that our dollar-unit sampling will favour high-dollar items. Should you wish to pursue a dual-purpose testing strategy using statistical sampling, consider consulting with the Internal Specialist—Research and Quantitative Analysis.

See the table below for a summary of how our substantive test impacts our ability to use dual-purpose testing:

Type of Test	Control Test	Accept-reject Test	Target Test	Non-Statistical Sampling	Statistical Sampling
Sampling Method	Random selection	Random or Haphazard selection	Targeted items (based on risk, materiality, etc.)	Random, Haphazard or Systematic selection	Dollar unit sampling
Allowable to be combined with Controls Testing for Dual Purpose Testing?	N/A	Yes	No	Possibly, so long as the sampling method allows that each sampling unit has a known probability of being selected and the auditor avoids bias by selecting items which have typical characteristic of the population	Possibly, consider contacting the Internal Specialist— Research and Quantitative Analysis to ensure that preference is not given to higher dollar sampling units

Dealing with errors in testing

Extrapolating errors would occur in the same way that they normally would for the type of test used. Auditors should also keep in mind that if dual purpose tests were performed incorporating a test of 25 control instances and an accept-reject test of 16 and then found an error in the 22nd sample item, we would not include that error in the accept-reject testing. That 22nd error would already be included in the allowable error of the accept-reject sampling.