7033.1 Evaluate reliability of data and develop an independent expectation

Sep-2022

OAG Guidance

Step 1.1 Data Reliability

One factor of particular importance when determining whether data is reliable for purposes of designing analytical procedures is the source of the information available. Obtaining assurance that the source data are reliable involves auditing internally prepared data or determining that the data are independent (i.e., prepared externally). Internal data produced from systems and records that are separate and distinct from the accounting records or that are not subject to manipulation by persons in a position to influence accounting activities are generally considered more reliable than internal accounting data.

Source data are not independent if they are generated from the information they are intended to predict. For example, it would be inappropriate to predict sales from commissions when commissions are themselves calculated as a percentage of sales. On the other hand, using sales to predict commissions would be appropriate.

The more reliable the data is, the more precise the expectation. In addition to the factors listed in CAS 520.A12, the reliability of the data used to form expectations is influenced by

• source of the information available

  • whether the data was obtained from independent sources outside the entity or from sources within the entity,
  • whether sources within the entity were independent of those who are responsible for the amount audited,
  • whether the expectations were developed using data from a variety of sources.

• comparability of the information available,

• nature and relevance of the information available, and

• whether the data was subjected to audit testing in the current or prior year.

In summary, there is a correlation between the reliability of the data and the quality of the expectation derived from the data, and thus the amount of assurance to be gained from the analytical procedure. Generally, the more precise an expectation for an analytical procedure, the greater will be the potential reliability of that procedure. As such, while data reliability is considered for risk assessment and overall conclusion analytics, it is more critical when executing substantive analytics.

Entity-Prepared Analytical Data

Throughout the course of the audit, the entity may provide us with audit evidence that they have prepared either in the ordinary course of business or specifically at our request. This audit evidence could include analytical data that may be useful in our efforts to perform risk assessment, substantive, and/or overall conclusion analytical procedures. Regardless of the rigor with which the entity prepares their analytical data we cannot substitute the entity’s work for our own. If we determine that it would be efficient to use the analytical data prepared by the entity, verify that the data is both properly prepared and that we continue to apply the four-step process for performing substantive analytical procedures. Generally, analytical data that the entity prepares is for their internal purposes which are different from our objectives. Even if the entity specifically prepares the analytical data at our request to address our objectives, still approach analytical procedures independently. Accordingly, establish an independent expectation and threshold. After calculating the difference, the entity-prepared analytics may then also be used to investigate the results.

To determine whether the entity’s analytical data is properly prepared, consider the following:

• given our objectives, the suitability of using the analytical data;

• degree of disaggregation;

• reliability of the data used in the analytical data (including the strength of the entity’s internal control over internally generated data);

• predictability of the amount or ratio;

• extent of reliance on the analytical data; and

• final evaluation.

These considerations are similar to those undertaken when developing our own analytical procedures.

Consequently, develop independent expectations and thresholds and utilize professional skepticism to determine the plausibility and sufficiency of the investigations (corroborated when appropriate) and reach our own conclusions.

Upon completing the use of the entity analytical data, consider whether our specific audit objectives have been achieved.

Internal and External Factors

Use professional judgment to understand and account for internal and external factors that affect financial relationships to verify that we are entitled to the audit evidence derived from the analysis. There is a wide variety of factors, and ways that these factors could influence the results of the analytical procedures. These factors are critical to consider when performing analytical procedures utilizing company data, and more importantly, when analytical procedures are used to benchmark the company against its competitors or the industry.

Examples of these factors include

• internal factors

  • management policies,
  • accounting methods,
  • internal controls,
  • quality of financial and operational information.

• external factors

  • economic, social, legal, and governmental;
  • industry and competitor.

As a result of these factors, interpret analytical procedures with care as the results can be intentionally or unintentionally misleading. Results may be intentionally misleading due to issues such as fraud or manipulation of results by management. Results may be unintentionally misleading due to mistake or error. See OAG Audit 5500 for guidance on addressing fraud in the audit.

Performing Testing for Determining Reliability of Data

When using information provided by the entity in analytical procedures, we need to evaluate the reliability of the information. We determine that information is sufficient and appropriate by performing procedures to: a) test the accuracy and completeness of the information or test the controls over the accuracy and completeness of that information; and b) evaluate whether the information is sufficiently precise and detailed for purposes of the substantive analytic. The nature, timing and extent of procedures performed over that data will vary depending on the nature of the data and its importance to the analytic, and the results of other procedures performed. When data is tested substantively, procedures for this purpose may include agreeing data to appropriate source documents, performing full and false inclusion testing reperforming mathematical calculations, etc.

When we assess the reliability of the data by testing the internal controls over the accuracy and completeness of the information, appropriate consideration needs to be given to any IT dependencies of those controls and whether or not testing has been performed over the related ITGC’s and information processing controls.

OAG Guidance

Step 1.2 Develop an Independent Expectation

The development of an appropriately precise, objective expectation is an important step in effectively using substantive analytical procedures.

An expectation is a prediction of a recorded amount or ratio. The prediction can be a specific number, a percentage, a direction or an approximation, depending on the desired precision, as indicated below. Develop expectations by identifying plausible relationships (e.g., between store square footage and retail sales, market trends and entity revenues) that are reasonably expected to exist based on our knowledge of the business, industry, trends, or other accounts.

Objectives

Refer to the objectives of the analytical procedure as these may have some influence over the expectation:

• Inherent and audit risks—The expectation is more precise as these risk factors increase.

• Understanding the entity and prior audit knowledge and experience—As our experience of the entity increases it will be possible to develop more precise expectations.

• Assertions—Identify the assertion(s) being addressed.

• Reliance on the procedure—The greater the reliance being sought from the procedure, the more precise the expectation has to be.

• Prior results—If past testing has been successful then the expectations we develop would be more precise.

Gather Information to Develop an Independent Expectation

To be able to produce good quality expectations we first need to have detailed knowledge of the entity, business and industry. Consequently, appropriate research and knowledge sharing within the team is important. The expectation begins to be developed at the team planning meetings with the engagement manager and partner.

Depending on the particular requirements of our procedures, we may select from a variety of data sources to form expectations. Examples of information generally available to further assist in developing expectations for analytical procedures include the following:

• information about the industry and the entity’s competitors and/or peers (consider benchmarking where appropriate);

• analyst reports (for publicly traded entities);

• budgets and forecasts;

• publicly reported entity information (e.g., management’s discussion and analysis of results of operations in annual and interim filings, webcasts, and conference calls);

• entity financial and operational information for current and comparable prior period; and

• information from discussions with management about their objectives, risks, and the control mechanisms used to monitor those risks.

Developing an independent expectation does not mean that the expectation is based entirely on information external to the entity. Such external information may often be difficult to obtain for smaller entities and research of external information sites may be unproductive.

Therefore focus on the key pieces of external information needed to develop appropriate analytical procedure in the key areas of the financial statements, for example, information on industry average sales prices for the entity’s products.

Information sources for developing an expectation:

• Last year’s figures updated with known changes from last years

It may be appropriate to use the prior year amount as a starting point for developing an expectation. If prior year figures are used as a starting point, document the rationale for the appropriateness of the prior year amount as a starting point and then the factors considered when developing an expectation from that starting point. Where there have been major changes at an entity (e.g., significant changes in product lines, loss of major customers, acquisitions, disposals etc.), prior year will clearly not be an appropriate starting point without factoring these events into the expectation.

• Relationships between elements of financial data

Relationships between elements of financial data may also be used as a starting point for developing an expectation. For example, if sales increase by 5 percent it is reasonable to assume that, all things being equal, accounts receivable will increase by a similar amount.

• Relationships between elements of financial and non-financial data

Examples of such ratios are sales revenue/production volume, rental income/number of apartments rented or salary costs/number of employees.

Evaluate the Information Gathered

Consider unusual items that might influence existing relationships. Weak, uninformed expectations limit the potential effectiveness of analytical procedures. This does not preclude us from using simple expectations, such as “the current year will be consistent with the prior year balance,” but apply professional judgment in developing expectations that are appropriate to achieving our objectives. Simply using prior year numbers with no independent consideration is inadequate. Further, rigor is increased if expectations are established prior to viewing the current year results.

When developing or applying an expectation based on analytical procedures designed in prior periods, consider factors that may have affected the relationship, such as seasonality, lags (the item of audit interest may be related to data from a prior period), and the possibility that relationships that made sense in the past no longer apply due to changing market conditions or entity actions.

Considerations for Developing an Expectation

The potential effectiveness of an analytical procedure and the degree of reliance that can be placed on the procedure is affected by the quality of the expectation that is developed. The closeness of our expectation to the “correct” amount is called the degree of precision. The degree of desired precision will differ with the specific purpose of the analytical procedure.

The higher the desired level of assurance, the more precise the expectation needs to be. The expectation needs to be sufficiently precise such that a significant difference between it and the recorded amount could be an indicator of a misstatement.

This is summarized in the table below:

Desired level of evidence	Definition of a significant difference	Necessary precision of expectation
High	Smaller	Very precise
Moderate	Moderate	Moderately precise
Low	Larger	Less precise

Consideration of each analytical procedure’s objectives is important to properly establish the required degree of precision.

Key Factors Affecting the Precision of the Expectation

There are four key factors that affect the precision of analytical procedures: disaggregation, data reliability, predictability, and type of analytical procedure (see further in this topic of OAG Audit).

OAG Guidance

Step 1.3 Disaggregated Analytical Procedures

The more detailed the level at which analytical procedures are performed, the greater the potential precision of the procedure. Further, analytical procedures performed at a high level may mask significant, but offsetting, differences that are more likely to come to our attention when procedures are performed on disaggregated data. The objective of the audit procedure will determine whether data for an analytical procedure is disaggregated and to what degree it is disaggregated.

Disaggregated analytics can be best thought of as looking at the composition of a balance(s) based on time (e.g., by month or by week) and the source(s) (e.g., by geographic region or by product) of the underlying data elements.

Time

Balances can be disaggregated by time, where the discrete periods reviewed sum to the whole of the period being reported upon. Generally, the smaller the unit of time used to disaggregate financial data, the more precise the resulting expectation. The objective of the analytical procedure helps define the units of time to be used.

For example, expectations formed using quarterly data are likely to be more precise than ones formed using annual data. Similarly, expectations formed using monthly data are likely to be more precise than ones formed using quarterly data.

Source

The ‘source’ is represented by the types of information that form the logical subsets of the underlying data and are likely to be those reviewed by management. Examples of types of information include geography or size. Generally, disaggregation of financial information by source results in a more precise expectation and more effective analytical procedures. The objective of the analytical procedure, and the level of assurance desired, helps to determine the degree to which financial transactions is disaggregated by source.

The source of the financial transactions may, for example, refer to certain organizational, geographic, or product-related attributes or types, sizes, or demographics of customer(s). Examples of disaggregation with these sources include:

• Organizational—Entity, business unit, management unit, operating facilities.
• Geographic—Continent, country, region, town, etc.
• Account balance or class of transaction attribute—Value or risk.
• Product—product, product line, product group.
• Customer type—Product- or service-lines sold.
• Customer size—Large, medium or small purchase volumes.
• Customer demographics—Location and size of business.

Size of Balance and Homogeneity

The size of the recorded account balance is also a factor to consider when developing an expectation. If the recorded balance is too large or comprised of non-homogeneous components that make it difficult for us to develop an expectation that will be sufficiently precise for its intended purpose, disaggregate the information to obtain a more precise expectation. The objective of the analytical procedure helps to define the degree to which account balances is disaggregated.

For example, for most entities, annual revenues constitute a very large balance in relation to materiality, and it may not be possible to develop an expectation of annual revenues, in the aggregate, that is sufficiently precise. Therefore, total revenue is generally disaggregated into smaller amounts for which more precise expectations can be developed (e.g., by time, source, or another attribute).

Other Attributes

Exercise professional judgment to identify other key attributes that are considered as drivers for disaggregation. Our understanding of the business is a primary resource for identifying such attributes.

Disaggregated Analytics and Fraud Risk Related to Revenue

Since there is a rebuttable presumption that there is a fraud risk relating to improper revenue recognition, we ordinarily use disaggregated analytical procedures with regards to revenue. As discussed above, time, source and size of balance are all particularly important to consider when determining how to effectively disaggregate revenue for testing. Other factors to consider may include such specific areas as the composition of revenues, specific attributes of revenue transactions, and unique industry considerations.

Substantive analytical procedures performed on disaggregated data are designed and executed to verify audit objectives are attained effectively and efficiently. Examples of analytical procedures performed on disaggregated revenue data could include

• analyzing comparative customer sales by the smallest time period available for the purpose of identifying purchasing patterns and that may indicate outliers or exceptions,

• comparing revenues for major customers by week for three years to identify unusual trends in revenue recognition,

• comparing gross margins by sales region to identify unusual balances or margins, and

• analyzing the average revenue and margin on contracts by salesperson to identify outliers.

Disaggregation and Data Availability

Both the availability of data and the type and degree of disaggregation commonly used by management could influence the decision as to which disaggregated data to use for an analytical procedure. If the available information is more disaggregated than required, assess whether we can place greater assurance on our work by utilizing this additional disaggregation to develop more precise expectations and thresholds. Likewise, if information is less granular than required, verify that audit procedures performed elsewhere are adjusted to compensate for the lower assurance derived from the analytical procedure.

Disaggregation and CAATs

The size of the entity and/or the volume of information will impact the decision to incorporate the use of computer assisted audit techniques (CAATs) in disaggregated analytics. Consider the use of CAATs during the early planning stages of the audit. CAATs provide a means to view large amounts of data in a format that can provide transparency otherwise not attainable through other auditing procedures.

CAATs can be used to support all types of analytical procedures (e.g., trend, reasonableness, scanning analytics) through data manipulation (including sorting) and stratification. Of particular note is the considerable assistance they can provide when performing both scanning analytics or accept / reject testing on significant quantities of data (e.g., journal entry testing, disaggregated revenue analytics, etc.)

Our specialists/experts, such as the Data Analytics Specialist, could assist us to effectively and efficiently design and execute CAATs.

In summary, there is a direct correlation between the degree of data disaggregation and ability to derive precise expectations from the data. Generally, the more precise an expectation is for an analytical procedure, the greater the potential assurance can be obtained from that procedure.

OAG Guidance

Step 1.4 Predictability

Expectations for some amounts and ratios are likely to be more precise than for others. Examples of factors affecting the predictability of expectations include:

• In a stable entity or economic environment, relationships are usually more predictable than in a dynamic or unstable environment.

• Relationships involving income statement accounts tend to be more predictable than those involving balance sheet accounts since the former represent transactions over a period of time, while balance sheet accounts represent amounts at a point in time.

• Relationships involving transactions subject to management discretion may be less predictable (e.g., the timing of cash payments).

• The nature of certain transactions and correlation between them (e.g., the trend of sales may correlate to the trend in accounts receivable).

Other factors that influence the predictability of a ratio or account include

• significant market events,
• changes in entity strategy,
• accounting changes,
• industry factors,
• economic factors, and
• management incentives.

The use of non-financial data (e.g., number of employees, occupancy rates, units produced) in developing an expectation may increase our ability to predict account relationships. However, the information is subject to data reliability considerations mentioned above.

In summary, there is a direct relationship between the predictability of the data and the quality of the expectation derived from the data. Generally, the more precise an expectation is for an analytical procedure, the greater will be the potential reliability of that procedure.

OAG Guidance

Step 1.5 Type of Analytical Procedure

There are five types of analytical procedure commonly used for performing analytical procedures on an audit engagement. All five types of analytical procedures may be used as substantive procedures and will influence the precision of the expectation. Choose among these procedures based on our objectives for the procedure (i.e., purpose of the test, desired level of assurance):

• Trend analysis
• Ratio analysis
• Reasonableness testing
• Regression analysis
• Scanning analytics

Each of the five types uses a different method to form an expectation (see OAG Audit 7032). The first four are ranked from lowest to highest in order of their inherent precision. Scanning analytics are different from the other types of analytical procedures in that scanning analytics search within accounts or other entity data to identify anomalous individual items, while the other types use aggregated financial information.

Expectations can be developed for analytical procedures as basic as a simple trend analysis (using the prior-year balance, adjusted for expected changes, as the expectation for the current year), or as complex as multiple regression analysis incorporating both financial and non-financial data. Select the most appropriate type of analytical procedure by considering the nature of the account, purpose of the test, and desired level of assurance.

If we need a high level of assurance from a substantive analytical procedure, develop a relatively precise expectation by selecting an appropriate analytical procedure (e.g., a reasonableness test instead of a simple trend or “flux” analysis). Determining which type of substantive analytical procedure to use is a matter of professional judgment.

In summary, there is a direct correlation between the type of analytical procedures selected and the precision it can provide. Generally, the more precision inherent in an analytical procedure used, the greater the potential reliability of that procedure.

OAG Guidance

Step 1.6 Finalizing the expectation

When finalizing the expectation, it is important to reconsider the objectives set for the analytical procedure and whether we have been able to develop an expectation consistent with those objectives. Consideration of the planned reliance on the procedure and the designed reliance is carefully assessed for the analytical procedure to be effective and efficient. If reliance is greater or less than originally intended, assess the impact on the audit plan and adjust the analytical procedure or the audit plan as necessary.

OAG Guidance

Document adequately the expectation and how it was derived. The following are key documentation points to consider:

• How the expectation was developed, including key data sources and information, and the reliability of that information.
• A description of the expectation.