7073.4 Step 4: Test Method(s)

Sep-2020

OAG Guidance

The understanding of how management selects or designs, and applies, the methods used, including the use of models, which we obtain in accordance with the section Entity’s Information System Relating to Accounting Estimates in OAG Audit 7073.1, serves to provide the basis for our assessment of the appropriateness of management’s methods.

When the applicable financial reporting framework does not prescribe the method of measurement for a particular accounting estimate and there is no established accounting convention for the area in question, consider the need to engage an auditor’s expert or a specialist in accounting or auditing in the audit to assist with evaluating the appropriateness of management’s method(s). Refer to OAG Audit 3100 for guidance on engaging Specialists in Accounting or Auditing.

Methods for developing accounting estimates may change depending on changes in the entity’s circumstances. While the entity might aim to provide consistency of accounting policies and disclosures (to ensure comparability of financial statements), changes in methods may be necessary to appropriately respond to market and other changes. Consider if the method used in the current period is appropriate and, where there has been a change as compared to the previous year, evaluate whether it was justified.

Where a change does not appear to be based on changes in the entity’s circumstances, consider if it may represent a risk of management bias or fraud. For further guidance on the nature of, and our response to, indicators of management bias, refer to the section Management Bias in OAG Audit 7071 and OAG Audit 7073.9 respectively. Refer to OAG Audit 5504 for guidance on assessing and documenting fraud risks.

OAG Guidance

We test complex models to verify that the internal logic and formulae in the model are accurate and appropriate, in that they reflect the expected relationships between data and assumptions and support the proper application of relevant requirements of the applicable financial reporting framework. Whether the complex model used in developing an estimate is appropriate in the circumstances may depend on a number of factors, such as the nature of the entity and its environment, including the industry in which it operates and the specific asset or liability being measured. These factors also help us determine the nature, timing, and extent of the audit procedures to be performed over the complex model.

Matters to consider in testing the model

Depending on the circumstances (including whether the complex model is one that is commercially available for use in a particular sector or industry, or a proprietary model), matters that we may consider in testing the model (in addition to those listed in CAS 540.A99), include whether:

• Controls are implemented over:

  • the design and development, or selection, of a particular model for a particular purpose; and

  • determining that model data and assumptions are subject to all changes intended by management, and not subject to any unintended changes, during activities such as input, storage, retrieval, transmission or processing;

• The model is adequately documented, including the model’s intended applications and limitations and its key parameters, required inputs, and results of any validation analysis performed;

• The theoretical model being used is appropriate. For example, there are a number of option pricing models and it is important that the limitations inherent in the assumptions underlying each one are understood and taken into account in the valuations;

• The model is commonly used by other market participants and has been previously demonstrated to provide a reliable estimate;

• The model operates as intended and there are no flaws in its design, particularly under extreme conditions;

• The inputs to the models are complete and appropriate for the model;

• Controls have been designed and implemented to prevent those responsible for developing the model from bring unduly influenced by traders or others who may not be objective;

• The model is periodically calibrated to the market to verify that its output is a genuine reflection of market prices, including how sensitive the model is to changes in variables and whether this reflects market behavior;

• Market variables and assumptions are used consistently and whether new conditions justify a change in the market variables or assumptions used;

• For a fair value model, the model maximizes the use of relevant observable inputs and minimizes the use of unobservable inputs;

• Adjustments are made to the output of the model including, in the case of fair value measurements, whether such adjustments reflect the assumptions marketplace participants would use in similar circumstances; or

• The model was developed and applied by individuals with appropriate levels of competence, knowledge and experience.

In planning our test of a model we need to appropriately identify and test any IT dependencies in accordance with the guidance in OAG Audit 4028. This will include considering control over access to the model to protect against unapproved changes such that it does not operate consistent with management’s communicated intention. Also consider the guidance in OAG Audit 2051 where spreadsheets or other end‑user computing tools are used in the modelling process.

When testing complex models, consider if the use of specialists or experts may be necessary. See OAG Audit 3090 and OAG Audit 3100 for guidance on the use of specialists and experts.

Adjustments to model outputs (“Overlays”)

Complex estimates can often be built upon rigorous examination and analysis of historical data and its relationship to changes in current factors in the entity and its environment, as well as the application of management judgment. Such judgment is based upon the specific knowledge of each entity and the current business environment and how the judgments may be affected by unique events and conditions that cannot be easily captured by quantitative models. In these circumstances, the entity may therefore adjust its modelled output, sometimes referred to as an “overlay adjustment” or “overlay”. Use of overlays may represent an opportunity for management to override controls and therefore we need to consider the use of overlays when evaluating the appropriateness of management’s methods.

Examples where overlays may be applied:

• Circumstances where economic, political, natural or other global events occur prior to period end but before re‑measurement of the estimate can occur;

• To remedy known imperfections in models for which adjustments are expected;

• To integrate non‑modelled risk factors;

• To account for considerations relating to business cyclicality; and

• To remedy known cases where there is a lack of data availability.

Where overlays are applied we consider testing the operating effectiveness of management’s approval process around the use of the overlay and perform appropriate substantive tests of details such as:

• Evaluating the reasonableness of the entity’s assessment of the overlays, including:

  • Understanding the rationale for the overlay and agreeing the overlay to appropriate supporting documentation

  • Testing any significant assumptions inherent in the overlay calculation

  • Testing mathematical accuracy of the overlay calculation

  • Assessing for any ‘double‑counting’ of effects (i.e., whether allowance for such events is already reflected in the model or elsewhere)

  • Assessing the appropriateness of the overlay allocation (i.e., whether the overlay is appropriately distributed among different entities or segments covered by the underlying model)

• Understanding any additional data used in the calculation to develop the overlays (in addition to the data used in the estimate model) and assessing the impact on IT testing strategy and overall audit plan (i.e., approach to determining completeness and accuracy of additional data); or

• Evaluating the suitability of external data used in the overlay calculation and determining whether the data used in the overlay calculations is relevant to the accounting estimate.