Testing misstatement

Testing in an audit sampling context

In an audit sampling test the auditor generally assigns performance materiality, $$\theta_{max}$$, to the population which expresses the maximum tolerable misstatement (as a fraction or a monetary amount). The auditor then inspects a sample of the population to compare the following two hypotheses:

$H_-:\theta<\theta_{max}\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, H_+:\theta\geq\theta_{max}$.

The evaluation() function allows you to make a statement about the credibility of these two hypotheses after inspecting a sample. The output for testing as discussed in this vignette is only displayed when you enter a value for materiality argument.

Frequentist hypothesis testing using the p-value

This will be added in a future version of jfa.

Bayesian hypothesis testing using the Bayes factor

Bayesian hypothesis testing uses the Bayes factor, $$BF_{-+}$$ or $$BF_{+-}$$, to make a statement about the evidence provided by the sample in support for one of the two hypotheses $$H_-$$ or $$H_+$$. The subscript The Bayes factor denotes which hypothesis it favors. By default, the evaluation() function returns the value for $$BF_{-+}$$.

As an example of how to interpret the Bayes factor, the value of $$BF_{-+} = 10$$ (provided by the evaluation() function) can be interpreted as: the data are 10 times more likely to have occurred under the hypothesis $$H_-:\theta<\theta_{max}$$ than under the hypothesis $$H_+:\theta\geq\theta_{max}$$. $$BF_{-+} > 1$$ indicates evidence for $$H_-$$, while $$BF_{-+} < 1$$ indicates evidence for $$H_+$$.

$$BF_{-+}$$ Strength of evidence
$$< 0.01$$ Extreme evidence for $$H_+$$
$$0.01 - 0.033$$ Very strong evidence for $$H_+$$
$$0.033 - 0.10$$ Strong evidence for $$H_+$$
$$0.10 - 0.33$$ Moderate evidence for $$H_+$$
$$0.33 - 1$$ Anecdotal evidence for $$H_+$$
$$1$$ No evidence for $$H_-$$ or $$H_+$$
$$1 - 3$$ Anecdotal evidence for $$H_-$$
$$3 - 10$$ Moderate evidence for $$H_-$$
$$10 - 30$$ Strong evidence for $$H_-$$
$$30 - 100$$ Very strong evidence for $$H_-$$
$$> 100$$ Extreme evidence for $$H_-$$

Example

As an example, consider that an auditor wants to verify whether the population contains less than 5 percent misstatement, implying the hypotheses $$H_-:\theta<0.05$$ and $$H_+:\theta\geq0.05$$. They have taken a sample of 40 items, of which 1 contained an error. The prior distribution is assumed to be a non-informative $$beta(1,1)$$ prior.

The output below shows that $$BF_{-+}=30.28$$, implying that there is very strong evidence for $$H_-$$, the hypothesis that the population contains misstatements lower than 5 percent of the population.

prior <- auditPrior(materiality = 0.05, method = "none", likelihood = "binomial")
stage4 <- evaluation(materiality = 0.05, nSumstats = 40, kSumstats = 1, prior = prior)
summary(stage4)
## # ------------------------------------------------------------
## #             jfa Evaluation Summary (Bayesian)
## # ------------------------------------------------------------
## # Input:
## #
## # Confidence:                    95%
## # Materiality:                   5%
## # Minimum precision:             Not specified
## # Sample size:                   40
## # Sample errors:                 1
## # Sum of taints:                 1
## # Method:                        binomial
## # Prior distribution:            beta(a = 1, ß = 1)
## # ------------------------------------------------------------
## # Output:
## #
## # Posterior distribution:        beta(a = 2, ß = 40)
## # Most likely error:             2.5%
## # Upper bound:                   11.055%
## # Precision:                     8.555%
## # Bayes factor-+:                30.282
## # Conclusion:                    Do not approve population
## # ------------------------------------------------------------

Sensitivity to the prior distribution

In audit sampling, the Bayes factor is dependent on the prior distribution for $$\theta$$. As a rule of thumb, when the prior distribution is very uninformative with respect to the misstatement parameter $$\theta$$, the Bayes factor overestimates the evidence in favor of $$H_-$$. You can mitigate this dependency using method = "median" in the auditPrior() function, which constructs a prior distribution that is impartial with respect to the hypotheses $$H_-$$ and $$H_+$$.

The output below shows that $$BF_{-+}=3.08$$, implying that there is anecdotal evidence for $$H_-$$, the hypothesis that the population contains misstatements lower than 5 percent of the population.

prior <- auditPrior(materiality = 0.05, method = "median", likelihood = "binomial")
stage4 <- evaluation(materiality = 0.05, nSumstats = 40, kSumstats = 1, prior = prior)
summary(stage4)
## # ------------------------------------------------------------
## #             jfa Evaluation Summary (Bayesian)
## # ------------------------------------------------------------
## # Input:
## #
## # Confidence:                    95%
## # Materiality:                   5%
## # Minimum precision:             Not specified
## # Sample size:                   40
## # Sample errors:                 1
## # Sum of taints:                 1
## # Method:                        binomial
## # Prior distribution:            beta(a = 1, ß = 13.513)
## # ------------------------------------------------------------
## # Output:
## #
## # Posterior distribution:        beta(a = 2, ß = 52.513)
## # Most likely error:             1.904%
## # Upper bound:                   8.561%
## # Precision:                     6.656%
## # Bayes factor-+:                3.078
## # Conclusion:                    Do not approve population
## # ------------------------------------------------------------