7033.2 Defining a significant difference or threshold

Jul-2017

OAG Guidance

Step 2: Define a Significant Difference or Threshold

The maximum acceptable difference that can be accepted without further investigation is commonly called the “threshold.” Thresholds may be defined either as numerical values or as percentages of the items being tested.

The concepts and principles related to the determination of the appropriate threshold to use for substantive analytics are influenced by materiality and the desired level of assurance from the analytic. The more precise the expectation, the lower is the threshold. Similarly, the more assurance desired, the lower the threshold. Determination of the threshold involves considering the possibility that a combination of misstatements in the specific account balance, class of transactions, or disclosure could aggregate to an unacceptable amount. Increasing the desired level of assurance as the risk of material misstatement increases is achieved by reducing the amount of difference from the expectation that can be accepted without further investigation.

The threshold would generally not exceed performance materiality when we perform substantive analytics on most accounts. However, when we use substantive analytics on revenue and cost of sales, thresholds greater than performance materiality can be used to achieve a Low or Moderate level of assurance. Refer to the guidance further in this section.

To prevent bias in our judgment, determine the threshold while planning the substantive analytical procedures, i.e., before Step 3, in which the difference between the expectation and the recorded amount are computed. If materiality for particular classes of transactions, account balances or disclosures is established and we apply substantive analytics to those accounts, the threshold related to such analytics does not exceed those specific materiality levels and would ordinarily be lower (except for revenue and cost of sales when Low or Moderate levels of assurance are desired).

The appropriate threshold can be determined using statistical techniques or professional judgment. The factors that influence the threshold are discussed further in this topic.

Threshold Levels—Revenue and Cost of Sales

When we plan to obtain a High level of evidence from performance of substantive analytical procedure over revenue and cost of sales, the threshold may not exceed performance materiality. However, when we have performed other substantive procedures on the relevant assertions for revenue and/or cost of sales and plan to obtain a Moderate or Low level of evidence from substantive analytics, we can use a threshold that is in excess of performance materiality. This would only be applicable for revenue and cost of sales accounts, as they tend to be very large as compared to materiality and using thresholds below performance materiality is oftentimes not practicable.

The following table summarizes the maximum thresholds we can apply when performing substantive analytics over revenue and cost of sales, prior to considerations for disaggregated analytics:

Level of assurance*	Maximum threshold**
High	Performance materiality
Moderate	Lesser of 3% of the population or class of transactions or 2 x performance materiality
Low	Lesser of 3% of the population or class of transactions or 3 x performance materiality

*As further explained below, substantive analytics performed at Moderate or Low level cannot be used as the only substantive procedures at the assertion level and would always need to be combined with other substantive procedures.

**Thresholds cannot exceed 3 percent of the population or class of transactions subject to the analytical procedures balance. We may use lower thresholds, when we consider it appropriate in the circumstances of the engagement.

We can only use thresholds higher than performance materiality on revenue and cost of sales, when the following pre-conditions have been met:

• Substantive analytics do not represent the only substantive procedure performed over the related assertion. In other words, substantive analytics performed with a threshold exceeding performance materiality would not provide sufficient audit evidence on their own and would need to be combined with other substantive procedures. These other procedures, as a minimum, would include testing of the related balance sheet accounts (accounts receivable for revenue, inventory and accounts payable for cost of sales) as of the end of the period that is subject to substantive analytics. Other substantive procedures (targeted testing, audit sampling) over revenue or cost of sales need to be performed depending on the assessed risk and other engagement circumstances.
• We are not expecting errors in revenue or cost of sales that are significant individually or in the aggregate based on prior audit experience and errors identified in other audit procedures such as testing of related balance sheet accounts.
• Our independent expectation used for substantive analytical procedure needs to be sufficiently precise for the desired level of assurance.
• The desired level of assurance needs to be determined and justified in advance of performing the analytic. This determination would be based on our risk assessment and the level of assurance provided by other audit procedures. If a difference in excess of the threshold is observed, we need to investigate the difference and not settle for a lower level of assurance. For example, it would not be appropriate to plan to achieve a Moderate level of assurance, observe a difference of 2.5 x Performance Materiality, and simply accept that the test delivered Low assurance. Unexpected differences increase the risk that there is a material misstatement and need to be investigated.
• Thresholds used may not exceed 3 percent of the FSLI balance (i.e., the threshold would never exceed 3 percent of revenue or cost of sales).

As noted in OAG Audit 5505 and OAG Audit 5506, CAS 240 indicates that there is a presumed significant risk of fraud in revenue recognition. We perform a granular risk assessment to identify the specific risk of fraud at the entity and design procedures that are directly responsive to the assessed risks of material misstatement due to fraud at the assertion level. We may use substantive analytical procedures designed for moderate or low assurance in situations where a significant risk of fraud exists on the engagement and has been appropriately addressed (i.e. substantive analytics may be used even when the risk of fraud has not been rebutted, provided that the substantive analytics are combined with other substantive procedures sufficient to address the risk). See OAG Audit 7011.1 for considerations related to auditing revenue and for additional considerations related to auditing income statement accounts other than revenue.

OAG Guidance

Defining a Significant Difference or Threshold

The primary factors considered when defining a threshold are:

• desired level of evidence from the analytical procedure;
• the precision of the expectation;
• overall materiality and performance materiality, including impact of disaggregation; and
• type of analytical procedure and the rigor with which it is applied.

Each of these items is discussed further below.

Apply care when defining thresholds, as analytical procedures—even those with appropriate expectations—may fail if thresholds are inappropriate. For example, when assurance and precision are both high and an inappropriately large threshold is defined, the analytical procedure will not be effective as it may result in a failure to address differences which may indicate or represent material misstatements. Conversely, when assurance and precision are low and the threshold is small, the procedure will not be efficient as it will require investigation of differences of such a small size that they will have only a negligible likelihood of being material misstatements.

Where multiple measures are used for a threshold, such as dollar and percentage, the threshold is defined as $X “OR” Y% so that both measures do not need to be exceeded to trigger investigation. Using “AND” could result in material differences not being investigated.

Impact of Assurance on Thresholds

The threshold will generally become smaller as the precision of the expectation increases through the use of better and/or more information. Further, better disaggregation, improved data reliability and/or predictability, or more precise types of analytical procedures have a similar impact of decreasing the threshold.

Large Balances

As noted above, thresholds generally do not exceed performance materiality (except for revenue and cost of sales, when Low or Moderate assurance is desired). Thresholds exceeding performance materiality would be limited to revenue and cost of sales which tend to be very large and where other substantive procedures are performed over the relevant assertions (and other certain pre-conditions are met).

Small Balances

If we perform testing on accounts that are either less than performance materiality or where accounts have been disaggregated into components that are less than performance materiality, thresholds are set at an amount sufficiently low in order that we meet our objectives for the analytical procedure. In these instances, professional judgment is applied to determine an appropriate threshold. Generally, the thresholds would be more closely associated with the size of the balance, after taking into consideration the factors discussed in the “Impact of Assurance on Thresholds” block above.

Disaggregated Balances

In cases where accounts are disaggregated for testing, the thresholds that would be appropriate for the disaggregated components of an account would be lower than the threshold that would be appropriate for the aggregated account. This takes into account “aggregation risk,” or the risk that differences, which in the disaggregated account components are not individually material misstatements, aggregate to a material misstatement in the account.

A key consideration in determining the thresholds when disaggregating an account is the relationship between the amount and/or other qualitative factors of the disaggregated components of the account and performance materiality.

For example, disaggregating a $100 million account into four approximately equal $25 million components where performance materiality is $5 million and the aggregate account threshold is $4 million (80 percent of performance materiality). The range to determine the appropriate threshold to apply to each of the $25 million disaggregated account components:

• $4 million = 100 percent of aggregate account threshold.
• $1 million = Aggregate threshold prorated based on amount (i.e. $4 million aggregate account threshold divided by four).

Within this range, the thresholds are determined individually for each component by applying professional judgment. Those thresholds could be different for each of the components despite the components being similar in $ amount and in certain circumstances they could be lower than provided by this range.

Also consider the way in which the account is disaggregated (e.g., by date, business unit, product line, etc.) and the size of the remaining components as these factors could impact the thresholds.  For example, disaggregating a $50 million P&L account balance based on time:

• Activity in the account occurs consistently over time and the account is disaggregated into monthly components that are approximately similar in size; we may determine that a threshold of 20 percent of the aggregate account threshold for each component would be reasonable.
• Activity in the account occurs consistently over time and the account is disaggregated into quarterly components that are approximately similar in size; we may determine that a threshold of 65 percent of the aggregate account threshold for each of the first through third quarters and then 45 percent of the aggregate account threshold for the fourth quarter would be reasonable. (In this case the lower threshold applied to the 4th quarter could have been to address specific year-end risks.)
• Activity in the account is seasonal and the account is disaggregated into two components; one component is the combination of “off-season” first through third quarters which in total represents 40 percent of the account activity and the fourth quarter, which represents the remaining 60 percent of the account activity. We may determine that a reasonable threshold for the combined three off-season quarters is 70 percent of the aggregate account threshold and for the fourth quarter it is 50 percent of the aggregate account threshold.

Another point of reference that may be helpful, by analogy, is the result of allocating materiality to each unit of disaggregation using the framework for allocating materiality to components in OAG Audit 2334 . For example, assume we are testing the reasonableness of the revenue FSLI, using an analytic that is disaggregated into 6 product lines and 4 quarters. We plan to obtain a Moderate level of assurance from this substantive analytic and plan to use the largest threshold for the FSLI, i.e., 2 x performance materiality:

Recorded revenue	$1.5 billion
Performance materiality	$15 million
Threshold (2 x Performance Materiality)	$30 million
Number of disaggregated units	24 units

When we disaggregate the revenue population by product line and by quarter, we are essentially creating 24 sub-populations or units of disaggregation and need to set the threshold for each unit of disaggregation.

As we plan to achieve a Moderate level of assurance, our maximum threshold would be $30 million, however we would not use this level for each unit of disaggregation in this example.

While this is a matter of judgment, we might take the approach that the 24 ”units” are akin to components, and referencing to the component materiality framework by analogy, we can allocate 500 percent of the maximum threshold (i.e., $30 million x 500% =$150 million) to the 24 units (assuming the group performance materiality haircut is 50 percent). Absent any further adjustments, we would be assigning a threshold of $6.25 million ($150 million divided by 24) to each of the units of disaggregation (assuming the units are of equal sizes). When the units are of different size or there are other relevant factors that may affect the disaggregation, we would need to consider those, as appropriate.

Other Methods for Determining Thresholds

Thresholds may be determined statistically. For example, regression analysis expresses significance mathematically in terms of confidence levels and precision ranges that can be logically related to performance materiality for the account balance. Regardless of how we determine the amount that will be considered significant, the threshold amount is carefully established to verify that significant differences are identified in a rational and consistent manner.

Types of Analytical Procedure and Thresholds

For details regarding the correlation between the type of analytical procedure and the precision that it can provide, see OAG Audit 7032. The rigor with which the procedure is performed also affects its precision and assurance. It is important to recognize that this relationship between precision and assurance also impacts the size of the threshold. As discussed above, there is generally an inverse relationship between the amount of assurance and precision and the size of the threshold.

Scanning analytics are different from the other types of analytical procedures in that scanning analytics search within accounts or other entity data to identify items that meet predetermined criteria. In this instance, our expectations are very precise and it is probable that all items which fall outside the expectation are examined (i.e., the threshold is zero).

Finalizing the Threshold

When finalizing the threshold, it is important to reconsider the objectives set for the analytical procedure and whether we have been able to develop a threshold consistent with those objectives. Assess carefully the consideration of the planned assurance on the procedure and the designed assurance for the analytical procedure to be effective and efficient. If assurance is greater or lesser than originally intended, assess the impact on the audit plan and adjust the analytical procedure or the audit plan as necessary.

OAG Guidance

For substantive analytical procedures, document our threshold and the rationale used to develop it, as they are designed to provide substantive audit assurance. For example, based on our performance materiality, the disaggregation of data, the predictability of the account, and the absence of non-recurring events in the account, we have established our threshold for this analytic at $45,000.

The threshold is implied in the development of our expectations; explanation of the unusual or unexpected item is sufficient to enable an experienced auditor independent of the audit to understand the basis for our conclusion. For instance, in our expectation we indicate that we expect revenues to increase 10 percent from the prior period. Consequently, compare the prior year revenue plus 10 percent to the actual results of the current period. If the actual results differ from our expectation, the explanation of the difference will reinforce what our expectation was and why we concluded there was a difference.

For documenting this step of the four-step process, use the substantive analytics procedures for the specific financial statement line item (FSLI) from the relevant cabinet: IFRS or PSAS-Substantive Tests.