7073.6 Step 6: Test Data

Sep-2022

OAG Guidance

Some financial reporting frameworks establish a fair value hierarchy to develop increased consistency and comparability in fair value measurements and related disclosures. In such cases, inputs are classified into different levels such as:

• Level 1 inputs—Quoted prices (unadjusted) in active markets for identical assets or liabilities that the entity can access at the measurement date.

• Level 2 inputs—Inputs other than quoted prices included within level 1 that are observable for the asset or liability, either directly or indirectly. If the asset or liability has a specified (contractual) term, a level 2 input must be observable for substantially the full term of the asset or liability. Level 2 inputs include the following:

  • Quoted prices for similar assets or liabilities in active markets.
  • Quoted prices for identical or similar assets or liabilities in markets that are not active.
  • Inputs other than quoted prices that are observable for the asset or liability (for example, interest rates and yield curves observable at commonly quoted intervals, implied volatilities and credit spreads).
  • Inputs that are derived principally from, or corroborated by, observable market data by correlation or other means (market‑corroborated inputs).

• Level 3 inputs—Unobservable inputs for the asset or liability. Unobservable inputs are used to measure fair value to the extent that relevant observable inputs are not available, thereby allowing for situations in which there is little, if any, market activity for the asset or liability at the measurement date.

The hierarchy of inputs is a way of defining a continuum of the nature and reliability of evidence used to support valuation. Estimation uncertainty increases as the observability of inputs declines, i.e., estimation uncertainty is likely to be higher for Level 3 inputs than for Level 1 inputs. Therefore we obtain an understanding of where the inputs used to value an asset or liability are obtained from and design tests that take into account the level of observability of those inputs. We evaluate the available evidence and understand both the fair value hierarchy and the risk of management bias in management’s categorization of financial instruments in the fair value hierarchy.

Related Guidance

Please refer to the section Reliable Audit Evidence in OAG Audit 1051 for general guidance on the reliability of audit evidence. Please refer to the section Test Relevance and Reliability of Data below for guidance on testing of relevance and reliability of data used in developing accounting estimates.

OAG Guidance

Obtaining audit evidence about the appropriateness of the data used in making an estimate will always need to be part of the audit procedures we perform. Our objective is to evaluate whether the data, including data used by management experts, is relevant to management’s estimate determination and whether the data is reliable (i.e. whether the data is complete and accurate).

The concept of data reliability includes the following:

• Reliability of data supporting management’s assumptions (e.g., historical employee turnover rates and service periods used in developing current period assumptions when estimating post‑employment benefits obligations); and

• Reliability of underlying data to which management or management experts apply those assumptions in its estimation method (e.g., workforce characteristics at period end).

Some accounting estimates may be subject to multiple data inputs that might have differing impacts on the measurement of the estimate. The planned nature, timing, and extent of our audit procedures to test the relevance and reliability of the data may vary depending on how important the particular data is to the measurement of the estimate. For example, how sensitive the estimate is to changes in the data.

Considering the source, relevance, and reliability of external data

Accounting estimates, and particularly fair value accounting estimates, may use data inputs that are obtained from sources external to the entity we are auditing. We need to perform procedures to assess whether external data used in developing estimates is from a reputable and credible source, relevant to the estimate and reliable.

Determine the nature, timing, and extent of procedures

Determining the nature, timing, and extent of controls testing, if applicable, and substantive audit procedures to assess the reliability of data used as inputs will depend on the entity’s facts and circumstances and may be affected by factors such as:

• The nature of the data used (e.g., routinely generated information or one‑off/ad hoc information);

• The source(s) of data used (e.g., external or entity prepared, manually prepared / maintained in spreadsheet format or generated by a system under the internal controls of the entity we are auditing);

• The significance of the underlying data (i.e., the sensitivity of the estimate to changes in the data); and

• Results of other audit procedures that may be relevant to the completeness and accuracy of data used as model inputs (i.e. audit procedures performed for other purposes that can be leveraged).

The determination of the desired level of evidence from substantive tests will depend on the assessed risk of material misstatement, the expected level of reliance on controls, and evidence obtained from other substantive audit procedures that provides evidence about the completeness and accuracy of the data (i.e. other audit procedures that can be leveraged).

We consider the guidance in the section Information Generated by an IT Application in OAG Audit 4028.4 with respect to understanding the nature and source of the underlying data, including information generated by an IT application. When testing whether data used in developing the estimate has been correctly transferred from its source into the entity’s calculation (i.e., testing an attribute), for example by agreeing it to other source documents, we may use accept‑reject testing provided that the accuracy of the data in those source documents was otherwise tested using an appropriate form of controls and/or substantive testing. We consider the guidance in the section Methods and Modelsin OAG Audit 7073.1 in evaluating the appropriateness of applying accept‑reject testing to data used in making accounting estimates.

In defining the population(s) of data to test, teams may define the population(s) differently depending on the characteristics and sources of the data and the nature of measurements (i.e., estimated amounts) selected for testing.

In some cases we define our testing population based on the sources of the data. For example, when testing impairment of property, plant and equipment the data relating to the fair values of the assets within a cash generating unit (CGU) may have been provided by an external valuation expert engaged by management, and the data related to the calculation of the estimated future cash flows included in the discounted cash flow model may come from internal sources such as historical results and budgets.

In this situation, we could define our testing population based on each CGU (i.e., selecting those CGUs that would give us appropriate coverage) and test all the data inputs within the selected CGUs.

Alternatively if we are able to establish the data population from a particular source is homogeneous (e.g., one external valuation report obtained by management covering assets across multiple CGUs), we might determine that it is more effective and efficient to define our data testing population based on this source. In this case, we could define the population as assets included in the valuation report and test the data within that population applying appropriate testing techniques (e.g., targeted testing, non‑statistical sampling). We would then need to perform procedures to verify that data used in the fair values of the assets within each CGU were correctly transferred from that external valuation report into the estimate calculation, for instance using accept‑reject testing or by testing relevant data input controls.

Consider utilizing the work of internal audit and/or relying on service organization auditor’s reports when testing the reliability of the data used in developing accounting estimates.

For more complex accounting estimates consider the need to involve auditor’s experts or specialists when testing the appropriateness and reliability of the data.

Consider factors noted in CAS 540.A127‑129 when performing the evaluation of the reliability of data used in making accounting estimates originating from an external information sources.

Related Guidance

Please refer to OAG Audit 1051 for further guidance regarding data reliability.

Please refer to OAG Audit 6043 for further guidance regarding using service organization auditor’s report as audit evidence.

Please refer to OAG Audit 6030 for further guidance regarding the use of the work of internal audit.

OAG Guidance

Related Guidance

Please refer to OAG Audit 1051 for further guidance regarding data reliability.

Please refer to OAG Audit 3095 for further guidance regarding involvement of an auditor’s expert.

OAG Guidance

Auditing management’s valuation of financial instruments often involves obtaining pricing information from a number of sources. Consideration needs to be given to existing requirements under the applicable financial reporting framework when planning and executing audit procedures using pricing information obtained from third parties. Depending on the facts and circumstances, we may consider:

• our understanding and evaluation of management’s process and controls for determining the value of financial instruments. Management is responsible for the fair value estimates included in its financial statements and our audit procedures would reflect the assessments and documentation prepared by management (i.e., we need to understand and evaluate management’s process and design and execute procedures accordingly);

• whether the data provided by information sources reflects fair value as defined by the applicable financial reporting framework;

• the sufficiency of audit evidence supporting management’s securities valuation assertions; and

• the nature and extent of additional audit evidence and procedures that may be appropriate when securities valuation information from third parties is limited or unavailable given the level of audit risk and materiality.

Considerations when using third party information sources

This guidance is intended to identify potential considerations for engagement teams when using third-party information sources as audit evidence related to the securities valuation assertion. These considerations are intended to provide a framework for evaluating whether our planned audit procedures are commensurate with the assessed level of risk related to the valuation assertion, including consideration of the materiality to the financial statements taken as a whole. These considerations are also focused on those valuation matters that require the most professional judgment.

Depending on the facts and circumstances, we may consider the need to:

• assess management’s procedures and controls relevant to the valuation assertion and the impact, if any, on our risk assessment;

• read the applicable agreements or underlying documents for more complex securities to understand relevant characteristics and terms likely to affect fair value in the current circumstances;

• obtain an understanding of the relevant market characteristics of the securities (including the markets in which they are transacted, liquidity of the market, market participants, valuation risk) in determining what constitutes relevant observable market data;

• evaluate fair value measurement. Practically, most of the market characteristics can be determined by grouping the securities by similar characteristics and then identifying whether use of a specialist or internal expert may be appropriate;

• obtain an understanding of the methodology and source and nature of the information used by the pricing service or broker for the price or quote or third party. Obtaining the price or quote alone, even though from a third party, may not be sufficient audit evidence by itself. If we perform procedures to gain an understanding of the methodology and source of information used by a pricing service at an interim date, we evaluate whether the pricing service has changed its methodology or source of information relative to the types of financial instruments being valued, and, if so, the effect of such changes on the pricing information provided at period end;

• consider the effect of subsequent events that come to our attention that could affect the measurement of fair value of financial instruments as of the reporting date. The determination of whether such information represents an adjusting or non‑adjusting subsequent event is judgmental and will be based on the specific facts and circumstances. Consideration would be given to consulting with the Financial Instruments Specialist when determining the impact of such events on valuations and related disclosures; and

• document important judgments and conclusions completely and on a timely basis. Our documentation would generally reflect:

  • evaluation of management’s process and the reasonableness of significant assumptions used in the determination of fair value by either management or a third party,
  • consideration of the factors discussed in this section,
  • underlying rationale used in reaching well-reasoned judgments,
  • consideration of alternative conclusions (by management and us), and
  • conclusion regarding the reasonableness of management’s estimates.

Additional considerations when using information provided by third parties are described below.

Pricing services

Pricing services aggregate, evaluate, and report pricing data, which reflects their view of the current value of a range of securities. Examples of pricing services include Interactive Data Corp (IDC), Reuters, and Loan Pricing Corporation. Depending upon the security, the value reported may be based upon a combination of various factors, such as:

• observed trades of the identical security;

• ‘benchmark/matrix pricing’—observed trades of similar securities, potentially accompanied by adjustments to reflect the circumstances of the client’s specific instrument;

• broker quotes (single or multiple);

• model-based valuations (e.g., discounted cash flows); and

• internal fundamental analysis and research such as credit analysis and risk ratings.

• consensus pricing ‑ an algorithmic approach using multiple broker quotes

Pricing information received from pricing services generally does not represent offers to buy or sell a particular security. As such, the information provided needs to be considered carefully. Neither management nor we would assume that the information provided by these third parties reflects fair value as defined by the applicable financial reporting framework without evaluating the nature and source of the information provided or corroborating with other relevant market information.

Quoted market prices for derivatives and securities listed on national exchanges or over-the-counter markets may be available from sources such as financial publications, the exchanges, or pricing services. Quoted market prices from active markets are generally considered to provide the primary and the most market observable evidence of the fair value of the securities.

As noted above, pricing information provided by pricing services is generally only indicative (i.e., it does not reflect an offer to transact at the reported price) and is usually accompanied by a disclaimer as to this fact. Therefore, management performs an evaluation to assess the appropriateness of the pricing information provided for use as the fair value measurement. As auditors, we evaluate whether management has sufficient documentation supporting their assertions. The nature and extent of this evaluation will depend upon the nature of the security and client-specific facts and circumstances, including the significance to the financial statements taken as a whole and, in certain cases, may require significant judgment.

In order to evaluate whether the information provided by the pricing service is representative of fair value, we consider:

• the client controls for assessing data from pricing services such as review of prices, comparisons of multiple sources of data, price tolerance check, day to day movements in pricing, current market activity (before and after measurement date), evaluation of stale pricing data, comparison to actual trades executed by the client and due diligence procedures;

• the nature of the security and general availability of observable market data for the security category;

• the pricing service’s valuation methodologies for the security or security category including how outliers may be identified in a security group, which may require discussions with the pricing service;

• obtaining an understanding, on a security by security basis, if necessary based on risk and materiality of the individual securities, of the pricing service’s underlying source of information and adjustments to the underlying information, if any, made by the pricing service;

• the quality of the pricing service, such as its historical accuracy and level of experience;

• whether there is any inconsistent information in the market place regarding the pricing assertion; and

• whether the pricing service or broker is independent of the entity (i.e., whether the pricing service has a relationship with the entity by which entity management has the ability to directly or indirectly control or significantly influence the pricing service).

We may, after appropriate consideration, conclude that the pricing data provided by the pricing service is representative of fair value for certain securities and document the basis for our conclusions. However, for certain securities for which valuation is more subjective, we may conclude that additional audit evidence needs to be obtained to further corroborate the valuation assertion. In such circumstances, we may consider additional procedures such as:

• comparing the reported price with evidence of a recent transaction for the security from another information source; and

• comparing the reported price to other relevant observable market information (e.g., prices of similar securities).

• where the client’s reported price reflects a fair value estimate based on a model (including a modeled valuation developed by a pricing service), considering the complexity of the security and subjectivity of the valuation and determining the need to test management’s process, including the method, data and significant assumptions, or develop an independent expectation of the estimated fair value of the security (see OAG Audit 7073.3)

If a pricing service provides a service organization report (e.g., CSAE 3416) describing controls that address the valuation assertion, exercise due care in determining the level of audit evidence obtained from the service organization as this may not sufficiently address the valuation assertion for certain securities or for the entire period appropriate for the entity. These considerations may impact the nature, timing and extent of relevant audit procedures.

In certain instances, management may engage an expert to assist in estimating fair value depending on the significance of the security valuation, and/or we may determine that it is necessary to develop an independent estimate to evaluate the value of the security.

Broker quotes

Quotes from broker-dealers who may make a market for certain types of securities can also be used by management to provide evidence of fair value. Quotes may be obtained from broker-dealers, including inter-dealer brokers.

For certain securities, significant judgment may be required to determine whether broker-dealer quotes provide sufficient audit evidence to support a fair value measurement. Although broker quotes are generally representative of traded market levels, this may not be the case in illiquid or thinly traded markets. For example, some broker quotes may be developed based on proprietary models rather than market transactions and therefore result in a higher degree of estimation uncertainty.

Using a broker quote to test valuation assertions may require special knowledge to understand the circumstances in which the quote was developed. For example, the quote may not be based on recent trades and may only be an indication of interest and not an actual price for which a counterparty will purchase or sell the underlying security. Therefore, consider the need to perform additional procedures to evaluate whether broker quotes reflect fair value. The more significant the position in the security and the more subjectivity in the valuation, the more likely it is that additional audit evidence and evaluation may be necessary.

Factors that may be considered in evaluating whether broker quotes are representative of fair value and/or whether additional audit evidence may be necessary include:

• market observability or other relevant market information that is available to support or contradict the pricing assertion, for example, whether the broker quote reflects recent market transacted prices (e.g., close to measurement date), trades of similar securities with similar credit ratings, or current bid/ask pricing;

• the number of broker quotes obtained and the closeness of the range of quotes being provided;

• trading volumes in the quoted security;

• whether the quoting broker was involved in initiating the trade, i.e., the broker’s independence;

• the process for obtaining/developing the quote (i.e., understanding the methodology used by the broker and evaluating the reasonableness of any significant assumptions used);

• management’s process for evaluating the reasonableness of the broker quote (especially if there is limited or no market activity); and

• the history of the quality of the previous quotes by the broker.

Consider these factors even if the entity has obtained multiple broker quotes.

Note that the “standard” disclaimer on broker quotes does not mean that the pricing information provided is not a relevant data point to consider.

We may, after appropriate consideration, conclude that the broker quote is representative of fair value for certain securities and document the basis for their conclusions. However, for certain securities for which valuation is more subjective, we may conclude that additional audit evidence needs to be obtained to further corroborate the valuation assertion. In this circumstance, consider the need to perform additional procedures such as:

• comparing the reported price to other relevant observable market information and reviewing recent purchase and sales activity of the security, and

• comparing the price to the price of similar securities.

For certain securities, the broker quote may reflect an estimate of fair value developed using proprietary valuation methods or models. Examples of valuation models include the present value of expected future cash flows, option-pricing models, matrix pricing, option-adjusted spread models, and fundamental analysis. When the broker estimates fair value based on a proprietary model, additional audit procedures may be appropriate, e.g., corroborating against other pricing sources and pricing models, including when the valuation is based on significant assumptions that are highly subjective or particularly sensitive to changes in the underlying circumstances.

Brokers may restrict accessibility to information about their proprietary valuation models and significant assumptions used to provide estimated fair values, which may preclude us from assessing the reasonableness of their methods and significant assumptions. In these circumstances, we will need to perform additional procedures to corroborate the estimated fair value. The nature and extent of the additional procedures will depend on the nature of the security and availability of relevant observable market data. Depending on the facts and circumstances, we may consider the need to perform additional procedures such as:

• evaluating the nature of the security and expected cash flows based on an understanding of the security features and the underlying collateral and priority of distribution;

• understanding and evaluating the entity’s investment manager/advisor’s approach to valuing the security;

• obtaining an understanding of the data/significant assumptions to the valuation model and evaluating the reasonableness against available market information; and

• corroborating the value of the security with additional, independent sources.

In certain instances, management may need to engage an expert to assist in estimating fair value depending on the significance of the security valuation and/or we may determine that it is necessary to develop an independent estimate to corroborate the value of the security.

Consensus pricing sources

Consensus pricing sources reflect calculated prices based on underlying information provided by other parties. For example, Bloomberg Generic (BGN) is the default pricing source on Bloomberg, which is based on a calculated average of available pricing information provided by banks. Other consensus pricing services include Mark-it’s Totem Service, and FT Interactive Data. The quality of the available pricing contributions may vary considerably. Note that Bloomberg provides pricing for a number of security types including equities and liquid fixed income securities. The considerations below regarding consensus pricing sources may be less applicable for readily marketable securities that are actively traded.

To evaluate whether consensus pricing sources are representative of fair value and/or whether additional audit evidence and evaluation may be necessary, engagement teams consider:

• understanding the methodology used by the consensus pricing source;

• understanding the contributors of information to the consensus pricing source and the quality of the pricing contributions, e.g., the number and names of contributors, as well as the time the prices were provided (for BGN, the data can be assessed through the ALLQ (All Quotes) screen on Bloomberg);

• Understanding whether the contributor source is the same source that is used by the entity.

The considerations above regarding consensus pricing sources may be less relevant for readily marketable securities that are actively traded. Also consider the guidance listed in the “Pricing services” and “Broker quotes” sections above when evaluating information obtained from consensus pricing sources.

Consider consulting with the Financial Instruments Specialist before undertaking any related work on financial instruments.

Investment managers/advisers

Third-party investment managers advise clients as to investment strategy and investment decision-making as well as providing various performance analyses. The use of a third-party investment manager does not obviate our clients from their responsibility to ensure that the financial statements reflect the fair value of their financial instruments. As noted with regard to other sources of pricing information described in this section, valuation information received from a client’s investment manager must be corroborated by additional supporting information. Generally, reliance by the client and the auditor on information obtained from the investment manager, without corroborative pricing data or an understanding of underlying pricing sources and/or methodologies and significant assumptions, is not sufficient. Third-party investment managers generally are willing to share their understanding of the pricing methodologies and sources with their clients and their auditors. Engagement teams should use the knowledge obtained from the investment manager (which may include CSAE 3416—Type 2 reports from service organizations), along with the considerations outlined herein to determine what additional procedures may need to be performed.

Limited access to underlying valuation information from pricing sources

Transparency into how prices were developed (including assumptions, methods, etc.) by third party information sources may be limited or unavailable for certain securities, particularly for certain illiquid structured securities. If other relevant observable market data is not available (e.g., benchmark securities, valuation information related to underlying collateral, etc.), the entity may need to model the estimated fair value based on hypothetical market participant assumptions. In these circumstances, consider the involvement of a specialist or internal expert.

We need to obtain an understanding of the methods and significant assumptions used by the specialist. In some situations where the specialist or expert will not provide us with the relevant assumptions due to the proprietary or limited nature of the information, we may be able to perform alternative procedures, such as those described in this section, to understand and evaluate the reasonableness of the valuation of such securities. Such procedures would be commensurate with our assessment of risk and materiality of the security. Consider consulting with the Financial Instruments Specialist in these situations.

Management’s expert

In the event management uses an expert to provide modeled values of complex investment securities (for example, cash flow projections for a private equity investment), we obtain an understanding of the methods and assumptions used by management’s expert, in the same way as we would if management developed their own assumptions. Management’s assumptions include both assumptions developed by management under the guidance of the board of directors and assumptions developed by an expert employed by management. In most situations, management’s expert uses assumptions consistent with market participants and applies these assumptions to entity-specific data provided by our client. In some situations where management’s expert will not provide us with the relevant assumptions due to the proprietary or limited nature of the information, we may be able to perform alternative procedures to understand and evaluate the reasonableness of the valuation of such securities, which may include developing an independent point estimate or range as described in OAG Audit 7073.3. Such procedures would be commensurate with our assessment of risk and materiality of the security. Consider consulting with an expert in these situations.