1. finance
2. accounting and auditing

Review of Choice Based and Matched Sample Studies
In Auditing Research*
Donald P. Cram
Unaffiliated
Iris Stuart
California State University Fullerton
Vijay Karan
California State University Fullerton
March 7, 2007
PRELIMINARY AND INCOMPLETE
*
Please address all correspondence to Donald P. Cram ([email protected]).
1
Review of Choice Based and Matched Sample Studies
In Auditing Research
Summary
Three methodological errors in the analysis of choice based and matched samples in
accounting research have been identified by Cram, Karan, Stuart (CKS, 2007). CKS
demonstrate that non-random samples, when collected, must be analyzed taking their
non-randomness properly into account. We discuss the misanalysis in nine streams of
audit research: auditor litigation, audit fees, impact of audit committees, quality of audits
and auditors, prediction of financial distress involving audit variables such as a going
concern qualification, other audit qualifications, and experiments on auditor judgment.
We tabulate and discuss 76 papers published in these research streams between 1980 and
2003 that used choice-based or matched samples, and we identify errors in 71 of these.
We provide evidence that the broad course of research in some of these streams may have
been misdirected due to reliance upon unjustified results of these works.
Key words: Matched sample, choice-based designs, auditing literature
Data availability: Data are available from public sources
2
INTRODUCTION
Audit researchers have used choice based and matched sample research designs
frequently in 76 papers from 1980-2003. They do so primarily for their power to reveal
statistically significant findings following collection of relatively small data sets. Choice
based and matched samples are frequently used to economize when data collection is
costly, especially when outcomes of one sort are rare and few would be obtained under
random selection. The research design of these non-random samples provides for
efficient collection of fewer data points. For example, all firms experiencing auditor
litigation during a period may be identified, and for comparison, rather than gathering
data for all non-litigation firms, a control sample containing matched firms, matching to
each litigation firm by industry and firm size, may be collected. This is appropriate if a
factor such as industry or firm size is likely to have a large effect on the likelihood of
auditor litigation but not itself be of primary research interest. In such a case, the use of a
matched sample design allows the researcher to focus power on estimating parameters for
variables of interest while applying control for those “nuisance” variables. Or, if
nuisance variables are likely to have a nonlinear effect, it suffices to match on those
variables without modeling and estimating their effects explicitly. Cram, Karan, Stuart
(CKS, 2007) identify, however, that choice based and matched samples have often been
misanalysed in accounting research, from Beaver (1966, 1968) and Altman (1968) until
now, and identify three errors. These errors, to some extent, have been noted before, but
CKS suggest sufficient guidance has not been provided within accounting research. CKS
are the first to identify six categories of studies using choice-based and matching
techniques which differ in the potential problems and most efficient solutions that apply.
The presence of these errors undermines both the internal and external validity of the
research.
Recent studies seek to identify determinants of the outcomes observed, e.g. of
change of auditors or no auditor change, and look at the sign, magnitude, and significance
3
of the coefficients in a model in order to accept or reject hypotheses on their
relationships. However, as Cram, Karan, Stuart (2007) demonstrate when improper
methods of analysis are used, the estimated coefficients’ signs, magnitudes, and
significance are unreliable.
When miss-specified models are being applied, research streams are more at risk
from the effect of “fishing” for results. Lys and Watts (1994) measure 14 client-specific
variable and 9 auditor-specific variables, but present a model involving just 5 and 3
variables, of which 4 variables are significant. Bell and Carcello (2000), in exploratory
work, find just 7 of 46 hypothesized characteristics to be significant in explaining auditor
dismissals.
Other studies depend on this literature by use of coefficients estimated in choicebased and matched samples, extending errors and introducing a new kind of error. For
example, a researcher may take and apply the estimated coefficients from a study of
determinants of audit litigation, in order to calculate a Z-score for use in other work. This
extends any errors from within the choice-based and matched analyses that derived the
coefficients. And, further it introduces a new error: it constructs an unconditional
estimate, which is not justified, from conditional results. For example, Krishnan and
Krishnan (1997) used the coefficients from Stice (1991) matched sample to assess the
likelihood of lawsuits against auditors. We assess that Stice’s paper’s coefficients are not
reliable due to Errors 1, 2, and 3 all occuring in the analysis. Second, the Stice paper
provides a conditional analysis, yielding coefficient estimates for some variables, but not
providing coefficients on matching variables of industry and size that are needed in an
unconditional estimate. The Z-score is incomplete. We do not search out and review
these studies relying upon choice-based and matched samples, unless they happened to be
in the auditing area and also included a new matched or choice-based sample.
The present paper explores the extent of misanalysis in audit research. We seek
to alert audit researchers to the need for reanalysis of previously published results, as they
4
suffer from problems of both internal and external validity. We also suggest that many
unpublished studies may benefit from correct reanalysis.
Auditing researchers often reason persuasively that industry or size or other
factors have large effects that must be controlled for, use those factors in selecting their
sample, but continue to perform analysis that, erroneously, does not account for the
matching. Matching on an effect does not accomplish the desired control if an unmatched
method is then used to analyze the sample. Therefore the analysis performed is
misspecified, and the researchers have created a strong possibility that their discussion of
the relative importance of other factors of research interest is not justified. Briefly, their
analysis suffers from the omission of multiple correlated variables which they themselves
have identified as being important, and hence all estimated coefficients and standard
errors in their analysis are biased unpredictably. Cram, Karan, Stuart (2007) term this
omission to be Error 1.
A related, further complication in the analysis arises if the matching has not been
perfect, e.g., if matching is done on a continuous variable such as assets or sales, and
“closest” rather than exact matches are accepted. Then, the remaining gap in size between
treatment and control observations could have a substantial effect, and a variable
measuring that difference in size needs to be included in the analysis of differences to
“soak up” its effect. Or, equivalently, that effect could be controlled for by including the
size variable itself in an analysis where matching is otherwise controlled for by including
pairwise dummy variables. Audit researchers often do not employ this refinement of
simpler matched analysis when it is needed. Cram, Karan, Stuart (2007) term this Error 2
in the discussion that follows.
Choice based and matched samples are not randomly selected. So, in addition to
controlling properly for matching variables, it is also necessary to adjust analyses to take
into account the differing sampling rates in strata of the collected data. This can be done
by reweighting each observation according to its stratum’s representativeness of the
5
general population. Analyses employing logit regression may be exempt from the need
for reweighting. Also exempt are some within-subject designs, such as studies of audit
fees paid by individual firms before and after a given event. Cram, Karan, Stuart (2007)
term the omission of necessary reweighting as Error 3.
The goals of this article, then, are to review the use of choice based and matched
sample designs in published auditing research. We identify model specification issues,
and clarify the correct use of matched sample research design in order to promote its use
as an effective tool. In this review of 73 papers, we establish that the extent of incorrect
analysis in auditing research—and by extension in other areas of accounting and financial
research--is large.
In some research streams it is possible to do cross-sectional studies on random
samples, so it is possible to perform meta-analysis across those and the ones employing
choice-based and matched samples. Hay Knechel Wong perform such a meta-analysis
across audit fee literature. In other streams, it is less feasible to collect random samples,
matched and choice-based samples comprise the majority of studies. When these are the
dominant approach, misanalysis is likely lead the research stream awry. In bankruptcy
prediction, the early studies such as Beaver (1966, 1968) and Altman (1968) and many
hundreds thereafter used matched samples and were misanalysed. From Ohlson (1980)
on, however, the use of random samples has largely supplanted the use of matched
samples.
This paper is organized as follows. In the next section, we discuss the research
designs used in choice based and matched sample studies in auditing research and
categorize them into six distinct groups. Then we discuss three types of errors that we
find to be common in many of these studies. Groups vary in terms of being subject to the
various errors and their remedies vary correspondingly. Then we establish the scope of
the problem by discussing the 76 studies in auditing research where we note choice
based, matched and stratified sample approaches have been applied. To be most useful to
6
audit researchers, we discuss these studies categorized within nine streams of audit
research. Also, we report on the frequency of these errors by category of the research
design used, by journals where these papers have been published, and by year of
publication. Finally, we summarize and conclude.
RESEARCH DESIGNS IN AUDIT RESEARCH
In audit research, Cram, Karan, Stuart (CKS 2007) identified six distinct research
designs involving samples that are nonrandom in that they are choice-based and/or
matched. We discuss, in addition, one more research design.
A choice-based research design is one where a subsample consisting of cases
having one outcome (e.g., firm-year observations where a bankruptcy or a going concern
qualified audit opinion occurs) is collected, and then a comparison sample of control
observations is selected from available data having different outcomes, and then the
analysis to follow uses the outcome as the dependent variable to be explained by other
variables. “Outcome-based” or “state-based” might be more descriptive, but choice-based
is the term generally used, as if the research is explaining discretionary management
decisions to declare bankruptcy or not, for example. Choice-based sampling is useful
when data collection is costly and one category of the outcome to be explained is rare, so
random sampling from the population would not yield very many observations of the rare
type unless very costly, large samples were collected. Medical research often takes
advantage of this method to examine determinants of patient outcomes such as cancers
that rarely occur.
A matched sample research design is one which incorporates non-proportional
sampling by its selection of pairs, triples, or other clusters of observations that are similar
in certain respects. Matched clusters may consist of “within-subject” data, e.g., the pair
of before-treatment and after-treatment observations of the same firm or other subject.1
1
Within-subject, before-and-after studies are not usually self-described as being matched samples in
accounting research; we found relatively few in our review of audit research. Unlike between-subject
studies, accounting researchers have usually analysed these taking into account the matching of subject
with itself at a later time. Many within-subject studies appear in experimental work and are appropriately
analysed: in analysis the term “blocking” refers to the control for matching that is accounted for.
7
Or, in a cross-sectional study, clusters may consist of pairs of firm-year observations that
a researcher assesses are similar on observable and available characteristics such as year,
industry, and size. These are “between-subject” studies and are usually misanalysed. In
medical research, there are many studies of twins, as many genetic and/or environmental
factors can be well controlled for, leaving power to discern the effects, across twins, of
differences of dietary or smoking or other habits.
Within the choice based research design, the control sample is often chosen based
not only upon the outcome but also by matching on other variables. Thus a choice-based
sample may or may not also be a matched sample; to be both, the control sample must be
selected on the outcome variable to be explained in analysis as well as matched to the
case sample on other variables. A sample of litigated firms paired to non-litigated firms,
with the pair-matching by industry and closest size, is both. If the comparison firms
were chosen randomly from the stratum of non-litigated firms, this would be just choice
based. We distinguish between “fully-matched” samples in which each case observation
is uniquely associated with one or more controls, and “semi-matched” samples. The
latter are samples that have pairings of case and controls that are nominally but not
meaningfully unique. For example, if several case observations in one industry are each
matched to a different randomly selected control from that industry, what is achieved is
what we term “semi-matching”; the pairs can just as well be combined into matched
many-to-many clusters. Each industry-case cluster and each industry-control cluster are
strata.
A different kind of nonrandom design is one that is merely stratified, meaning that
samples from different subpopulations are selected at varying sampling rates. To
generalize to the larger population, the data must be reweighted in any analysis so that the
true subpopulation sizes are proportionally represented. Kreutzfeldt and Wallace (1986),
Kreutzfeld and Wallace (1990), and Wallace and Kreutzfeldt (1995) use the same sample,
a sample from Arthur Anderson of 260 audit clients by stratified sampling in groupings
by size, industry, and publicly vs. privately held. The authors stated that hey wanted a
more representative sample than one chosen by pure random selection. Effectively,
8
however, they over-sampled in some strata. Hence this is a stratified sample, although it
is not choice-based and it does not involve matching.
These approaches to research design lead to seven distinct categories (see Figure
1), here described as (1) Choice Based Non Matched (CB-NM), (2) Choice Based Semi
Matched (CB-SM), (3) Choice Based Fully Matched (CB-FM), (4) Non Choice Based
Semi Matched (NCB-SM), and (5) Non Choice Based Fully Matched within-subjects
(NCB-FM-W), (6) Non Choice Based Fully Matched between-subjects (NCB-FM-B),
and (7) Stratified but neither choice-based nor matched. All seven categories are nonrandom samples, suffer from different potential errors and require differing forms of
analysis. Previous researchers have not recognized the need for differing forms of
analysis and have often analyzed their data incorrectly.
THREE COMMON ERRORS IN DATA ANALYSIS
Cram, Karan, Stuart (2007) identified three errors that can apply to one or more of
the seven research design categories. We summarize the research designs and the
potential for each error in Table 1.
Error 1: the Use of Unconditional Analysis, when Analysis Conditional upon
Effects of Matching Variables is Needed, can apply to all the matched sample categories,
but not to choice-based samples which do not involve matching.
Error 2: Failure to Control for Effect of Imperfectly Matched Variables, can
apply to CB-FM and NCB-FM-B categories. Matching is exact for NCB-FM-W
category. We chose not to review this error in the category of CB-SM, because we
estimated the effect of the error would likely be minor.
Error 3: Failure to Reweight Observations According to Differing Sampling
Rates, can apply to all research categories but NCB-FM-W. However, a “logit
exemption” to the need for reweighting applies to logit regression models in CB-NM,
CB-SM, and CB-FM categories, as long as fully saturated models are used.
In auditing research, we note there are just a few studies employing weighting to
correct for such problems. A limitation of the weighted exogenous sampling maximum
likelihood (WESML) method that these studies use, and of other weighting approaches, is
9
that exogenous population information is needed. In many research settings the required
information-- the proportions of all individuals from the population that fall in each of the
matching set categories-- might be readily available, such as when sampling from the
finite population of publicly traded firm-year observations appearing in CRSP and
Compustat databases, but often it is not. The vast majority of auditing studies using
choice-based and matched samples do not, however, employ exogenous weighting.
Many, instead, rely upon a special result which we term the “logit exemption”.
The logit exemption, at least as it applies to simple choice-based sampling, has
been known in accounting research since Palepu (1986). Notably, Maddala (1991) who
described the simple version of the exemption in an invited paper in The Accounting
Review, has been oft-cited and is at times paraphrased, imprecisely, to represent that as
long as logit regression is used, coefficients other than the intercept will not be biased.2
Zmijewski (1984) also has been cited on this point. It has not been understood that if
matching is used within choice-based samples, however, it is then necessary for an
intercept to be estimated for each matched set. Then each of those intercepts are likewise
biased, although coefficient estimates on included variables will be consistent.
Accounting researchers have also applied unweighted estimation to probit, OLS,
discriminant, and univariate analyses where the logit exemption obviously does not
apply. These analyses do not yield valid estimates. Five of the six research design
categories can include papers suffering from Error 3, because they involve nonrandom
sampling.
2
An extended version of the logit exemption is proven in CKS (2007). The logit exemption allows the use
of unweighted logit regressions to analyse samples that are choice-based, whether matched or not, data,
delivering asymptotically unbiased coefficient estimates and standard errors on non-intercept variables,
providing that the model is “fully saturated”, i.e. that an intercept is included for every level of each
matching variable. The typical choice-based application in accounting research , however, involves
matching, and erroneously estimates an unweighted and unsaturated model, which does not control for the
matching variables’ effects and does not enjoy the logit exemption from need to weight data to reflect
population proportions.
10
DISCUSSION OF AUDIT RESEARCH STUDIES
The use and frequent misanalysis of choice based and matched sample studies in
accounting research is not limited to the auditing area. However, these nonrandom
samples are widely used in auditing research, appearing in 24 articles during 2000-2003
alone, of which seven are in AJPT, the primary journal for American Accounting
Association audit researchers. We have collected, and summarize in Table 3, a list of 76
papers that is intended to be a fairly complete list of publications in auditing research,
from 1980 on, that feature choice based, matched, and stratified samples.3
3
We do not seek to include experimental studies that employed within-subjects designs although those can
be interpreted as matched sample studies. We would have included such a study if it was self-described as a
matched sample study, but found none. Our search to identify published papers in auditing research
included the following steps. We searched in ABI-Inform, JSTOR, and other literature databases on
“matched sample”, “choice-based”, “matching”, “stratified sample” and similar key words and phrases.
Also, we extensively browsed through back issues of The Accounting Review, the Journal of Accounting
Research, the Journal of Accounting and Economics, Auditing: A Journal of Practice & Theory,
Accounting and Business Research, the Journal of Accounting Auditing and Finance, and the International
Journal of Auditing to identify additional articles using matched sample designs, presented less
prominently. We observe that it is relatively rare for researchers to highlight their use of a choice-based or
matched sample design in title or abstract, perhaps because the method is so well accepted. We collected
complete copies of all articles identified, and from those identified other articles that these cited, and
collected those, obtaining those from outside our library where necessary. For authors who were found to
employ matched or choice-based sampling, as defined in this paper, we frequently made further efforts to
review their other papers. Although we do not include unpublished papers in our target set, we also
searched for working papers in accounting and finance by use of the Google search engine and the SSRN
11
The 76 papers listed in Table 3 comprise an interesting set of papers on its own.
Current work in the audit literature relies upon results of these papers. We believe that,
although it is beyond the scope of what we can prove in this paper, patterns prevailing in
this area extend to other areas in accounting and finance research.
We feel that the main results are not adequately supported in 71 of these 76
papers, and explain our general concerns in this section. More details of each individual
study are included in Table 3. We attempt to be nearly exhaustive in sampling from one
area of accounting research in order to avoid an appearance of criticizing authors
selectively. This review enables us to establish patterns that in fact apply more broadly.
We find one or more technical deficiencies along the lines identified above with almost
all of the choice based and matched sample analyses performed. Correct re-analysis in
most cases would be straightforward, were the original data available. Hence we
recommend reanalysis of many of these works before they are relied upon further. We do
not present replications, but at least in all of the studies using logit regression, correct
reanalysis would be easy, if the original data were available. Reanalysis of many others
website. Working papers yielded more citations of published papers using matching. From this set of more
than 400 papers in accounting and finance, we then restricted our attention to the 76 papers that we classify
as being in audit research. We deem a paper to be in audit research, for the purpose of this review, if it
focuses on audit litigation, uses auditors as subjects in a survey, or prominently employs audit-related
variables. As an example of a decision on the margin, we chose to include Beasley (1996), whose model
includes a variable denoting the existence or not of an audit committee, in this review, while we exclude
numerous other papers in corporate governance not having audit committee variables.
12
would also be feasible if exogenous sampling rate information also remains available.
Our speculation is that many results will be significantly refined upon replication.
We find that the vast majority of these papers suffer one or more of the three
technical deficiencies. Of the papers, 56 of the matched sample papers do not explicitly
control for matching in their analysis, thus suffering from Error 1. In the pair-matched
papers among these 56, the researchers should have evaluated pair-wise differences rather
than pooling all the data. The correct analysis also could have been implemented,
loosely speaking, by included dummy variables for each matched set. Of the remaining
20 papers that do not suffer from that error, seven are choice-based but not matched
samples, and five are merely stratified. Eight are matched samples which do control for
the matching, for example, by their having taken pairwise differences of the data, and
regressing pair-wise differences in outcomes upon those. Although these eight suffer
other errors, they do utilize the matching and effectively avoid Error 1. Papers suffering
from Error 1 lack internal validity. To be clear, there is no set of statistical assumptions
which would justify the standard errors and t-statistics reported. CKS show by
simulation that the effect of Error 1 can be to yield entirely erroneous conclusions.
On the second technical criticism, Error 2, we note 29 of the papers suffer from
lack of explicit control for “closest” imperfect matching. A closest-matched variable
such as size can still have influence. It might be controlled for by including a linear term.
But, as size or another variables contribution might be non-linear, in general, there is no
fully satisfactory resolution. The researcher, we argue, must show some effort to
eliminate the possibility that their results are merely driven by the omitted residual effect.
Sensitivity analyses including linear and quadratic terms, for example, might be
13
performed and given some discussion. Otherwise, the researcher has not established that
their effects of research interest are not merely the result of an omitted variable problem.
Papers suffering from Error 2 may lack internal validity. The main statistical assumption
which would justify excluding the residual effect is that size has no influence upon the
outcome. In a OLS regression, the omission of residual size would also be justifiable if
size was not correlated with any included variables (it is only omitted correlated variables
that cause bias in all other coefficients). However in logit, probit and other analyses, the
omission of even an uncorrelated variable that should be in the model will bias estimates
of all other coefficients.
On the third technical criticism, that of the need to reweight for sampling rates in
each stratum, numerous papers would be exempt for their use of logit regression, at least
insofar as for inferences based on non-intercept coefficients. (All of the matched sample
papers using logit, however, suffer from Error 1). Additionally, five are exempt from the
need to reweight for the reason that their matching is a within-subjects design, such as
before-and-after study of audit fee levels, where the subject is a firm which is randomly
selected from all firms having data available. And one study, G. Krishnan (2003), is
exempt from need to reweight because the researcher’s sampling rate is effectively 100%
in each of the strata sampled. The results, when presented as conditional to the
stratification variables, are then already weighted to reflect the available population. The
remainder do not enjoy the logit exemption from the need to reweight sampled data to
reflect population proportions, but only two of those, both being probit analyses, perform
the necessary reweighting. So 44 are in error for employing univariate, OLS regression,
14
discriminant analysis, and other analyses without necessary reweighting. These studies
suffer from Error 3 and lack external validity.
Overall, only five papers are free of all three errors that we classify. Two of
these, Colbert and Murray (1998) and Foster et al (1998), are logit regressions of nonmatched choice-based samples. The three others, Iyer and Iyer (1996), Maher et al.
(1992), and Sanders et al. (1996), are “before-and-after” studies of audit fee levels,
employing OLS regression of within-subject differences in audit fees upon differences in
other variables. A total of 30 suffer just one error, 24 suffer from just two, and 17 suffer
from all three errors. On average 1.7 of these three types of errors appear.
DISCUSSION OF AUDIT RESEARCH STUDIES BY RESEARCH STREAM
We identified nine research areas in auditing and discuss matched sample and
choice-based research in each of these areas.
Research Stream 1: Determinants of Auditor Litigation
Five papers investigate determinants of auditor litigation using choice-based
samples. Lys and Watts (1994)’s CB-FM sample consists of 163 firms whose auditors
were sued and 163 firm-year observations pair-matched by year, industry, Compustat
delisting code (if any), and then by closest logarithm of size. They report OLS
regressions explaining auditor litigation or not. Stice (1991), Shu (2000), and Heninger
(2001) are CB-SM studies employing logit regressions, and Krishnan and Krishnan
(1997) is a paper that extends the work in this area.
Stice creates two semi-matched samples for comparison to 49 firms having
auditor litigation: one is matched on year only, the other is matched by year and by
15
industry, and then one is selected randomly, achieving less than 49 year-industry sets.
Shu (2000) uses a list of audit litigation firms from Watts, augmented by additional
searching, yielding 282 firm-year observations, and constructs a semi-matched sample by
selecting 10 controls matched by year, for each litigation outcome. Heninger (2001)
identifies 67 auditee firms with lawsuits against auditors, and semi-matches to 67 firms
by year and industry, achieving somewhat fewer than 67 year-industry sets. These four
papers suffer from Error 1: they essentially suffer from the omission of dummy variables
that are required to be included. The authors argue convincingly that industry and/or year
are important but the analyses does not control for them (Error 1).
Lys and Watts include control for imperfect matching on logsize by including size
as a variable in their analysis, mitigating concern for Error 2 in their work, although
including logsize would be more internally consistent.
All of these papers suffer Error 3, the need for reweighting to population
proportions. Since saturated models are not employed in the logit analyses, they do not
enjoy the logit exemption for reweighting. Univariate analyses in these papers and the
OLS analysis in Lys and Watts also suffer from Error 3.
The impact of erroneous analysis in these papers carries through many other
papers that rely upon them for their measurement of audit litigation risk.
As one example, Krishnan and Krishnan (1997), itself a CB-SM paper, relies
upon Stice’s reported coefficients for a Z-score-type index of auditor litigation likelihood.
This application is inappropriate. If Stice’s sample had been choice-based but not
matched (CB-NM), then the coefficients other than the intercept would have been
accurate and higher index values would have represented higher auditor litigation
16
likelihood. However, use of an unconditional index based on the conditional estimates
from a matched sample is invalid. The index omits year-industry-specific intercepts that
cannot be estimated in a matched sample and that would likely be important (as Lys and
Watts, Stice, Shu, and Heninger do or would argue). And even if Stice had accounted for
the matching by including year-industry dummies in the logit regression, the intercepts
estimated would not be valid, for the same reason that the overall intercept in logit
regression of CB samples is not valid. Estimation of those intercepts for an index
requires a random sample, or reweighting of each observation according to the sampling
rate in each outcome-year-industry vis-a-vis the general population. Would such an
index nonetheless be informative about litigation likelihood, in the same way that
Altman’s Z-score turns out to be informative about bankruptcy, although it is
equivalently miss-specified? The answer to that is unclear: Altman’s Z-score has been
proven to be informative by classification performance tests by Altman and many others;
an auditor litigation score has not been proven informative in such ways. Further, it is
arguable that industry-year effects could be relatively stronger for auditor litigation and
omitting them would undermine the scores validity in two ways: first the direct omission
of the important effect of industry and year, second the coefficients included in the model
would be more severely miss-estimated.
As another example, more recent than the 76 that we tabulate, Krishnan and
Zhang (2005) similarly use a Z-score-like measure of audit litigation risk, but one based
on Shu (2000)’s coefficients instead of Stice’s. It is again incorrect to use the conditional
estimates from Shu to construct an unconditional index of audit litigation risk, one which
17
necessarily omits the year effect that Shu’s work deemed important, besides the fact of
error 1 in Shu’s work.
It seems that the impact of misanalysis of choice based and matched samples
significantly affects the estimation of audit litigation risk, which then continues to
influence research in corporate governance and other areas.
Research Stream 2: Impact of Audit Committees
The research in this area considers the impact of audit committees and the board
of directors on financial statement fraud. Research studies include three choice-based
samples and two matched but non-choice-based samples.
McMullen (1996) examined the reliability of financial reporting to determine if
the presence of an audit committee is associated with one or more of five outcomes
related to financial reporting problems (shareholder litigation alleging management fraud,
quarterly earnings restatement, SEC violation, illegal act, auditor change due to an
accounting disagreement between auditor and management). She collected five samples
of firm-year observations involving such outcomes over seven years, and constructed five
comparison samples from a set of 100 firms that was itself randomly selected. For each
outcome, she discarded comparison candidates that themselves experienced the outcome,
and for the remainder selected firm-year observations that semi-matched by year to the
treatment sample: each comparison sample had approximately the same distribution by
year as its treatment sample. She runs separate logit regressions to explain each of the
five 0-1 outcomes. The analysis failed to account for the semi-matching. To correct the
analysis, the researcher should have included stratification by year in the analysis.
Effectively, she omitted up to six dummy variables for year. Her result that audit
18
committee presence is negatively associated with each of the five financial reporting
problems is somewhat under question for the misanalysis.
Beasley (1996) examines the relation between the composition of the board of
directors and financial statement fraud. He selects a sample of 75 companies reporting
fraud between 1980 and 1991. Non-fraud firms are matched with fraud firms by closest
size, industry, and time period. The analysis fails to incorporate matching, so the results
may not be reliable.
To emphasize the importance of these works, take note of a recent paper relying
partially upon their contibutions, Srinivasan (2005), which itself uses a matched sample
analysed imperfectly. For evidence “that the likelihood of financial reporting problems
can be diminished with greater independence and expertise on boards and audit
committees” (p. 292), the previous publications he cites are all matched sample studies
(Beasley (1996), Dechow, Sloan, Sweeney (1996), Farber (200_), and Agrawal and
Chadha (200_).4 Srinivasan examines consequences for outside directors, on the audit
committee vs. not, when financial reporting failures occur. This bears on impact of the
audit committee, because incentives for board members tie to the effectiveness of the
audit committee work. In analysis across restating firms and non-restating firms matched
by industry, year, and performance, which a discussant deems to be the “central result” of
the paper, Srinivasan suffers Error 3 and, for some models, also Error 1, even though he
attempted a differences-in-differences approach that should have avoided Error 1, but did
not implement that thoroughly for all models.
4
The latter two were working papers at the time of his citation. Dechow, Sloan, Sweeney (1996) suffers
Error 1. We did not include it in our 75 papers reviewed herein as it appeared insufficiently focused upon
audit research. Farber (200 )
19
These are but a few of many corporate governance papers that used matched
samples. We included these because of the prominence of the audit committee variable.
Research Stream 3: Determinants of Audit Fees
A literature on pricing audit services is potentially important in providing
guidance in fee negotiation for audit firms and their clients. Six papers from our sample,
three papers using between-subjects matching and three using within-subjects matching,
contribute to this literature. This is the only research stream which has been studied by a
meta-analysis: Hay, Knechel, and Wong (HKW 2006) provide a review of 100 or so
papers including these six. HKW’s collected data could possibly be used to judge the
accuracy of the 6, by testing whether their results differ from the others’.
Simunic (1980) develops a model for determinants of audit fees and also
empirically tests it by OLS regressions explaining fees deflated by assets. He collected a
stratified sample by surveying financial officers at some of the 8,077 public U.S.
companies within four categories (OtherBig8 vs. nonBig8 auditor crossed with auditee
size greater or lesser than $125 million), selecting at random within each category. We
term this a NCB-SM sample. The OLS regression included intercepts for Big8 vs.
NonBig8 (literally a dummy for Big8, and an overall intercept.) However to fully
account for the stratification, the OLS regression should have included dummy variables
for each category, so three dummies plus the overall intercept were required. (Error 1).
Including one additional dummy for large size within Big8, and another for large size
within nonBig8 would have been consistent with the sampling done, and would have
achieved more efficient, consistent estimation of the coefficient on Big8 that was the
main result of interest.
20
We identify a sampling-related issue5: he had sampled at a different rate in each
category, sampling at a relatively high rate among the Big8 large auditee subuniverse
while collecting less than a proportional share of the nonBig8 small firms. For his results
to be generalizable to the universe of public companies, the regression would have to be
reweighted: each observation should be given weight equal to the inverse of the
sampling rate for its category. That requires information exogenous to the sample
collected, namely the numbers of available firms in each of four categories, out of the
universe of 8,077 from which he drew. For example, consider the 70 observations
collected in the NonBig8 small auditee category, and the 172 observations of Big8 large
auditees. If the universe of 8,077 had contained, say 6,000 and 500 of each of those
categories respectively, then weighting the NonBig8 small at 6000/70 and the Big8 Large
at 500/172 would be appropriate. Thus we find two “errors” in the model specification,
Error 1 and Error 3.
The reported results are technically incorrect in the sense that they are not the
maximum likelihood estimates that they are interpreted to be, and the corresponding
standard errors and p-values reported are not what would be yielded by the appropriately
weighted OLS regression that included the two omitted intercept variables. Reestimating the model in this way would yield different coefficient estimates and standard
errors for all variables, given the non-zero correlations amongst them. Could the results
be significantly different? It is certainly possible that some coefficients would drop
under or swing over the hurdle of achieving a .05 statistical significance level, and that
5
Simunic expresses concern about response bias, a distinct sampling problem that we do not address.
21
then the interpretation of results would be changed, and hypotheses advanced by
following researchers would differ, and so on. Menon and Williams (2001) note:
“Simunic (1980), in a seminal study, finds evidence of a Big 6 discount,
attributable to scale economies. However, subsequent studies, such as Francis
(1984), Craswell et al. (1995), and Palmrose (1986), detect a Big 6 price
premium, plausibly attributable to a higher quality of assurance offered by these
firms.” (p.117).
Perhaps a re-estimation of Simunic’s data could reconcile the reported results to those
reported by other researchers who used entirely random samples.
Two following studies that use low-balling to explain audit fees also suffer from
Error 1: Turpen (1990) employed a stratified sample similar to Simunic’s sample.
Turpen’s OLS analysis omitted the dummy variable for the OTC versus exchange traded
securities. Walker and Castarella (2000) collected a sample of 80 pairs matching by size
and by industry code. Their analysis omitted 80 indicator variables. In addition, for their
results to be generalizable, the OLS regressions in each needed to be reweighted using
inverse sampling rates as described above.
Three studies exploring changes in audit fees do not suffer from any of the three
errors. Maher et al. (1992), Sanders et al. (1995), and Iyer and Iyer (1996) each employ
matching only in the sense that they are within-subject studies. They collect data for 78
firms, 159 cities, and 270 U.K. firms at two dates: each firm-year observation is
“matched” to a later year’s observation for the same firm. They take pairwise
differences, and regress changes in fees on variables that are each changes in some other
measure. This accounts properly for the matching. In the OLS regression setting, this is
22
equivalent to including 78, 159, and 270 pair-indicator variables, respectively, in a
regression on the data for both years pooled together. These regressions are not missweighted either: the first firm-year observation for each pair can be viewed as randomly
selected from a universe of firms that exist at both the earlier and later dates. The
matching of corresponding later firm-years is collected at a 100% rate. If the pairs are
themselves viewed as strata, the sampling rates within each one is the same, so no
reweighting is needed. Survivorship bias is a form of selection bias that would apply in
considering generalizing their results to all firms. However, the results can be
generalized directly to the population of firms that existed at both dates.
The most recent of these studies, Walker and Castrella (2000), is the most
deficient with respect to the errors wet review, as it omits the greatest number of required
variables. We assess that its two main results, that audit fee discounting of new
engagements continues into more recent years, and that auditors charge more for firms
experiencing losses (perhaps as compensation for higher audit litigation risk) have not
been established by this work. The research stream should revisit these questions, rather
than rely on this study.
Hay, Knechel, and Wong’s (2006) meta-analysis of the audit fee research
literature includes all six of these papers. Its powerful conclusions on which variables
affect audit fee levels, based on meta-analysis of about a hundred papers, then, are partly
misinformed by the effect of misanalysis in these that we review here.
23
Research Stream 4: Prediction of Financial Distress and Fraud
Twenty-three papers in our census predict levels of financial distress, measured
by Going Concern Qualification (GCQ) or Bankruptcy, in choice-based studies. Some of
these papers use GCQ as a predictor of bankruptcy.
While studies that do this are numbered in the hundreds, we identify only 23
having an audit research focus or incorporating audit measures. Of these, six employ a
CB-NM design, i.e. they have one subsample collected by random selection from
financially distressed firms, another by random selection from non-distressed firm, with
matching not employed. Obtaining the same number of observations in each subsample
is not required. Mutchler (1985) and Lenard et al (1995, 1998, 2000) collect equally
sized samples; Foster et al (1998) and Cormier et al (1995) do not. Only one, Foster et
al 1998, is exempt from the need to employ reweighting because it uses a logit
regression. The others fail to apply necessary reweighting in their statistical analysis.
The purposes of the Lenard et al and Cormier et al papers include comparing the
performance among logit regression, discriminant analysis, recursive partitioning, and/or
neural network methods, and their authors advocate use of the non-logit methods to
achieve higher classification accuracy, based on the apparent performance of the models
applied without reweighting. We suggest the comparisons should have been performed
with each approach incorporating reweighting to address the different sampling rates in
the two subsamples. We must readily concede that the non-logit methods, even without
reweighting, may well yield models that achieve higher prediction performance than logit
models, but we note that, at least without reweighting, they are in effect “black box”
methods. In choice-based samples only the logit regressions yield coefficient and p-value
24
estimates that are valid for discussing the contribution of specific variables. The earliest
of these papers, Mutchler (1985), first discusses apparent univariate differences between
her GCQ vs. non-GCQ subsamples; this is based either on reported means of continuous
variables or percentages of firms exhibiting each of several 0-1 characteristics. With
hindsight, we note that extended discussion along these lines seems unjustified as
statistical significance of differences is not measured (as might be done by unmatched ttests). She goes on to apply discriminant analysis to various combinations of the
variables without reweighting.6 An unexpected result is that adding a good vs. bad news
identifier variable, constructed by evaluation of management discussion and analysis,
causes performance (measured by classification in hold-out samples) to deteriorate, while
it was strongly supposed to provide improvement. We conjecture that miss-weighting of
the observations could possible contribute to such an anomaly occurring, even when all
other statistical assumptions of the discriminant model (normal distribution of each
independent variable) are satisfied. We would expect that a Z-score computed from
Mutchler’s coefficients, such as Fleak and Wilson (1994) employ, would be informative
about GCQ likelihood, just not as informative as a comparable Z-score computed from a
reweighted analysis of Mutchler’s data.
Fleak and Wilson (1994) consider the information content of the going-concern
opinion. Their premise is that some going-concern qualifications provide information
affecting security returns and some do not. They created a stratified sample of distressed
firms. One stratum is all firms receiving going-concern qualified opinions from 19791986, the second is approximately 25 percent of all other available firms meeting a
6
Tatsuoka(19__) provides discussion of modification of discriminant analyses to avoid Error 1.
25
“troubled firm” criteria. They found that going-concern opinions that are unexpected are
associated with negative abnormal security returns. Their analysis suffers from error 3
only. To gain generalizability of their results, reweighting is needed.
Twelve papers explaining financial distress employ a CB-FM design: Kida
(1980), Levitan and Knoblett (1985), Koh (1991), Ponemon and Schick (1991), Citron
and Taffler (1992), Carlson et al (1998), Koh and Tan (1999), Citron and Taffler (2001),
Vanstraelen (2002 and 2003), Gaeremynck and Willekens (2003), and Geiger and Rama
(2003). They each identified a set of financially distressed firms, and then selected an
equally-sized, pair-matched comparison sample of non-distressed firm-year observations,
with pairing typically by industry, size, and year. The matching was not accounted for, in
any of these papers, in the various statistical models they estimated (all suffer from Error
1;) the authors erroneously believed that they had controlled for industry, size, year
and/or other matching variables merely by using matching in the sample selection stage,
while in fact it also needs to be controlled for in the analysis. As such, we argue all these
papers results should be disregarded: as Maddala (1991) put it, “nothing can be learned
from” such miss-specified models. Further, many of these suffer from Error 2 and/or
Error 3, although Koh (1991) stands out for applying appropriate reweighting in his
probit analysis.
Four of the financial distress papers employ a CB-SM design: Chen and Church
(1992), Kleinman and Anandarajan (1999), Morris and Strawser (1999), and Behn et al
(2001). Chen and Church use matching only by year to select an equal sized comparison
sample for their 127 GCQ firm-year observations, thereby creating a matched set for each
of five years. Morris and Strawser similarly use matching only by year to select an equal
26
sized comparison sample for 116 Texas banks bankrupted in 1990 and 1991. Behn et al
(2001) similarly create year-matched sets for 148 GCQ firms during four years. These
three studies apply logit regression to explain GCQ and fail to account for the semimatching (Error 1): essentially they needed to include a dummy variable for each year’s
set. Their results are only skewed by the omission of 4, 1, and 3 year-dummy variables,
respectively; this may seem minor but is required for the statistical reasoning to be valid,
and the authors had themselves identified that year was an important variable to be
controlled for.
Kleinman and Andarajan report that they match by size to their 61 GCQ firms,
but do not explain why they then obtain 173 control firm-year observations; we infer
their sample selection was equivalent to semi-matching on year and on size groupings.
Kleinman and Andarajan apply stepwise discriminant analysis which suffers from both
Error 1 and Error 3.
Research Stream 5: Prediction of Audit Qualification
A somewhat overlapping set of papers are those that explain audit qualification.
There are nine papers that address audit qualifications more generally than those, above,
addressing going concern qualifications alone.
An early CB-SM study in this area is Dopuch et al (1987) They find 275 firms
that experienced audit qualifications from 1969-1980, and semimatched on year alone.
The only concern we have is that they did not control for the semimatching in the
analysis by including year dummies.
Wilkerson (1987) identified 16 firms receiving uncertainty qualifications related
to allegations of federal antitrust law violations. He matched the firms receiving
27
uncertainty qualifications to firms identified with allegations of federal antitrust law
violations and not receiving uncertainty qualifications. A holdout sample from each
group was used to determine the predictive ability of a model based on financial and
market variables. Wilkerson’s analysis fails to incorporate matching.
Elliott (1992) identified 145 firms receiving “subject to” audit qualifications
between 1973 and 1978 and selected control firms with the closest earnings per share
measure (calculated by taking a year-to-year change in the EPS in the event year minus
the average change in EPS during prior years divided by the estimated standard deviation
of the change in EPS) to the sample firms. The analysis failed to control for the “closest”
match in the EPS measure.
DeFond and Jiambalvo (1993) examined factors related to auditor-client
disagreements. They identified 58 companies that reported auditor-client disagreements
between 1982 and 1986 and identified control firms that changed auditors but did not
report a disagreement. The control firms were matched by industry code (2-digit, 3-digit
or 4-digit.) The analysis failed to incorporate matching in the analysis and control for the
“closest” match was needed and was not done.
Kinney and McDaniel (1993) identified 85 firms from 1976-1988 making yearend announcements of corrections in previously reported interim earnings. The sample
was matched to control firms with no corrections in previously reported interim earnings
by industry and closest size. While the authors correctly used matching in the analysis of
the data, they failed to add additional controls for “closest” match. Given the differences
in occurrence rates in the two groups sampled, re-weighting is needed to generalize the
results beyond the current sample.
28
Buchman and Collins (1998) identified 60 firms receiving qualified opinions for
litigation from 1969-1977. Firms receiving unqualified opinions disclosing litigation
uncertainty in a footnote was matched to the firms receiving qualified opinion for
litigation by 3-digit SIC codes. The analysis failed to incorporate matching in the
analysis. Re-weighting of the sample in both populations was needed for generalization
of the results and was not reported.
Seipel and Tunnell (2000) reported results from a study examining the use of a
“subject to” audit opinion from 1983 to 1987. They identified 135 firms with “subject
to” opinions during this time period. The controls firms were matched by 2-digit SIC
code, and financial condition measured by Altman’s Z-score. The analysis fails to
incorporate matching. Additional control variables are need for “closest” matching and
are not reported. To generalize the results beyond the firms in the sample, re-weighting
is needed in the analysis.
Bartov et al (2001) identified 173 firms with qualified audit reports from 1980 to
1997. Each firm year in the test sample is matched with a control firm with an
unqualified opinion in the event year. Matching was done by 2-digit SIC code, auditor
type, and nearest asset amount. The analysis fails to incorporate matching. Additional
control is needed for “closest” matching and is not reported.
Peasnell et al (2001) identified firms with defective financial statements as judged
by the Financial Reporting Review Panel. The authors found that the firms judged to
have defective financial statements reported weaker performance than firms in the control
sample. The authors failed to incorporate matching in the analysis or to add additional
controls for “closest” matching.
29
Research Stream 6: Audit Quality
We group together eight papers in which audit quality is saliently discussed.
Audit firm size has been considered a reasonable proxy for perceived audit quality since
DeAngelo (1981). One of these papers directly tests that by comparing peer assessed
audit quality to audit firm size measured by number of CPA’s. Eight papers assume that
Big 8 / Big 6 / Big 4 audit firms are high quality and that non-Big 8 / Big 6 / Big 4 audits
are low quality. Of these one examines determinants of this proxy for audit quality
choice, while seven test whether audit quality, so measured, is associated with some
outcome. These latter papers implicitly test whether this proxy for audit quality has merit
as a proxy, by jointly testing whether the proxy enters with the expected sign into their
models. The research stream perhaps is important for auditors and firms, to verify that
higher quality audits are worthwhile and justify higher fees. For investors, the research
could verify that auditor quality proxy is a valid signal bearing on interpretation of
financial statements.
What is the purpose of using matched samples in these studies? In contrast to
other research streams, the purpose is not to economize on data collection costs. In fact
here data is discarded that has already been collected. The authors state reasons which do
not hold up as valid reasons, and the matching in these studies together with failure to
account for the matching in the analyses render their stated results to be invalid. For
example, Becker Defond Jiambalvo Subramanyam (___) state that they are attempting
“to increase comparability” by selecting a semi-matched sample, dropping Big 8 audited
firms in industry / year / cash flow size regions of the population where there are not also
non Big 6 audited firms . And Feltham, Hughes, Simunic (____) similarly state “we
30
control for the size effects ...by selecting two subsamples of firms, dichotomized by their
choice of small versus large auditor, such that the firm size distribution is the same in
both subsamples. This is accomplished by including a firm in a particular subsample
only if we can ‘match’ it with a firm of corresponding size in the other subsample.”
(p.381) This reasoning is invalid. It serves no purpose to drop firms selectively and then
apply an unmatched analysis. If data is selectively dropped to derive a fully- or semimatched sample, then only conditional results can be obtained, rather than the
unconditional results that are asserted.
The empirical relationship between size and audit quality is assessed by only one
paper, Colbert and Murray (1998), in this set. Using a CB-NM design Colbert and
Murray study small audit firms (half having 3 or fewer CPA’s) that are evaluated for
quality by state societies of CPAs or by the AICPA through a peer review process. They
measure size by the number of CPA’s working for the audit firm. Using ordered logit to
explain assessed quality in on-site reviews, they find the size variable to be significant.
As the study does not use a matched sample and uses logit regression to analyse the data,
the study is not subject to any of the three errors.
Clarkson and Simunic (1994) is the one CB-FM study. Clarkson and Simunic
(1994) use logit regression to explain firms’ choices of auditor type over 44 matched
pairs of data. It did not incorporate matching. Given the authors’ discussion, the correct
analysis incorporating the matching would be best conceptualized, for them, as running
the pairwise difference in audit quality (always 1) as a no-intercept logit regression over
44 pairwise differences of independent variables. This would have been appropriately
avoided Error 1 and is equivalent to running their logit regression modified to include 87
31
pair-indicator variables, over 88 observations. Other principal analyses include
univariate tests of differences in means; it is not clear in our readings whether they
applied matched t-tests which would be correct; if they applied unmatched t-tests their
reasoning was not valid. OLS regression results over a non-matched sample would be
valid; the same OLS regression over the pooled matched pairs is not. All of the analyses
performed suffered from one or both of Error 2 and Error 3, as well, because need to
account for imperfection in closest matching and need to provide reweighting to reflect
population proportions were not addressed.
In two papers, the authors create two samples, run models separately in each
sample, and claim results from comparisons made.
Feltham, Hughes, Simunic (1991) select three samples of 50, 40, and 37 fully
matched pairs from 251 high- and 141 low-audit-quality firms identified in prior
research, matching on just one variable for each: closest assets, closest IPO proceeds,
and closest MVE, respectively. They run an OLS regression explaining market value,
which do not incorporate matching, simultaneously within the high quality and low
quality subsamples, and report Chi-squared statistics from Chow tests whether
coefficients estimated in the different subsamples are different. For the tests to be valid,
the regression would need to include fixed effects for the pairings.
Allen matched cities with Moody’s bond ratings from 1978-1986 by size and Big
8 vs. non-Big 8 auditor. He ran separate logistic regressions within the 125 Big 8 Cities
subsample and the 125 non-Big 8 cities to predict bond ratings for the cities, as a function
of accounting ratios. He found that the model for Big 8 cities had higher explanatory
power. His primary hypothesis was that “accounting information audited by Big 8
32
auditors is better able to predict municipal bond ratings than is accounting information
audited by non-Big 8 auditors.” (p 119) He found rank correlation of .448 between
predicted and actual ratings for the Big 8 cities versus .185 correlation for the non Big 8,
and reported two tests. However, these analyses failed to account for the matching in
sample selection, and because the cities were matched by population size, additional
control for “closest” matching was also needed. Therefore Allen’s study suffers from
Error 1 and Error 2. His result, while reasonable sounding, is not established by his
analysis. We cannot ourselves imagine how to adjust his tests to be valid, given his
sample selection, but the onus is on the researcher, not us, to conduct the research in
manner so that appropriate tests can be performed. Taking a random sample and
controlling for city size explicitly in the model, then applying a bootstrapping approach to
statistically measure the difference in R-squareds (as Joos and Lang 1994 do) would be
appropriate.
Three papers are NCB-FM which use auditor quality, proxied by Big 8 / Big 6 /
Big 4 or not, as an explanatory variable in explaining various outcomes.
Teoh and Wong (1993) use matching by year, industry, and closest in size to
select 1282 pairs, and then run an OLS regression explaining CAR over a window from
earnings forecast to earnings announcement; their analysis does not incorporate
matching. Again, those not incorporating the matching should be disregarded. Teoh and
Wong essentially omitted 1,262 dummy variables that would have implemented control
for year, industry and size. Although Teoh and Wong use assets as their measure of size
in their sample selection, they include log of market value as a firm size measure in their
analysis; matching on log of market value originally would have been more appropriate.
33
Both studies suffer from Error 2 and Error 3: control for closest matching (in Teoh and
Wong’s case, by including at least a linear term for assets) was needed and also
reweighting was needed, for the results to be generalizable.
Bauwhede, Willekens, and Gaeremynck (2003) matched companies listed on the
Brussels Stock Exchange by industry and size with nonlisted firms in the same industry
and size to determine if Belgian companies engage in earnings management. The
analysis fails to incorporate matching. Reweighting is needed to generalize results to the
population and is not done.
Finally, two papers, Becker, DeFond, Jiambalvo, and Subramanyam (1998) and
Krishnan (2003) use a NCB-SM design. Becker, Defond, Jiambalvo, and Subramanyam
specifically that they do not use a matched sample design, but they nonetheless proceed
to select what we term a semi-matched sample. They match companies with Big 6
auditors and non-Big 6 auditors by year, industry, and cash flows to determine if
discretionary accruals are greater for companies with Big 6 or non-Big 6 auditors. Their
exposition of the selection process is a negative one: they discuss how they exclude
observations for firms not having a match, which yields the same result that including
only firms having matches would accomplish. It is not pair-matching, but this
accomplishes a many-to-many matching.7 Their analysis, not surprisingly, fails to
incorporate matching for year and industry. To obtain valid results they needed to have
included indicator variables for each year-industry-cash flow decile combination to
7
They state “We emphasize that our sample selection procedure is not a matched sample design. Rather,
the constraints on year, industry, and cash flow are employed in an attempt to induce a reasonable amount
of comparability across the Big Six and non-Big Six samples. Because the Big Six clients vastly
outnumber non-Big Six clients in the COMPUSTAT database, a one-to-one match does not yield a sample
that is reflective of the population. Our main result (that Big Six auditors have lower discretionary
34
incorporate matching in the analysis. Also, reweighting is needed to generalize their
results to the population and is not done.
Krishnan similiarly applies an exclusion process which amounts to matching
companies having Big 6 and non-Big 6 auditors by year, industry, and cash flows. He
seeks to determine if discretionary accruals are associated with stock returns or future
profitability. His analysis likewise fails to incorporate matching and omits reweighting
needed to generalize the results to the population.
In this grouping of research papers, all but one (Colbert and Murray) suffers Error
1, so the miscellaneous results stated (that audit quality is related to market value of
IPO’s, to ERC’s, to municipal bond ratings, etc.) have not in fact been supported.
Becker DeFond Jiambalvo and Subramanyam state in a footnote that their main result
(that Big Six audited firms have lower discretionary accruals) holds in non-reported
analysis over all Compustat firms, but there is no way for us to evaluate whether other
results are robust to the misspecifications that have been imposed.
Research Stream 7: Auditor Judgments
Research on auditor judgments examines the way auditors make decisions. The
use of matched samples may be a strategic decision because data collection from audit
personnel is very costly. The field may include such studies where there is matching in
the sense of within-subject, before and after data collection, but none are highlighted as
using matching. Use of blocking in ANOVA to correctly account for within-subject
matching is well understood and is equivalent to the use of dummy variables for each
matched set.
accruals) is not dependent on the sample selection procedure. Our results hold even when we use the
35
We identified only two papers in this field where the matching was across
subjects. Jamal and Tan (2001) use a sample consisting of 14 threesomes of auditors
(manager, “top” auditor, and “mediocre” auditor) who work together on client
engagements. They test for differences in performance on a judgment task across roles,
eg. managers’ vs. “top” seniors’ performance, by t-tests that appear to be unmatched.
Accounting for the matching within their sample selection would require use of matched
t-tests, instead, that would report results with seven fewer degrees of freedom. It is not
possible for us to assess whether doing so would yield higher or lower statistical
significance than what was reported, but the matched test is what is required by the
situation (the degrees of freedom are not “used up” by a choice after the fact, they were
not available in the sample collected). Unmatched tests do not have validity for
generalizing about the performance differences across roles.
Bedard and Graham (2002) sample contains 23 pairs of auditors who work
together on the same engagement. They treat one auditor with an aid intended to give a
negative orientation in a risk assessment judgment task, the other with one to give a
positive orientation. OLS regressions are applied to explain number of risk factors
identified as a function of negative vs. positive orientation as well as auditor experience.
Accounting for the pairing by including fixed effects is mandated by the data selection
process, but appears not to have been done. Coefficients and p-values would differ; it is
not discernable whether the “loss” of 23 degrees of freedom would decrease apparent
precision or whether precision would be enhanced by implementing the control for
within-client-engagement experience that may be obscuring precision.
population of COMPUSTAT firms.
36
Chung, Firth, and Kim, (2003), a NCB-FM-B study considers whether Big 6
auditing firms use more conservative accounting than non-Big 6 firms, when the
company reports bad news. They develop a sample of firms with a Big 6 auditor during
the 10-year period 1988-1997 and for each year selected and matched them to a sample
of firms with non-Big 6 auditors by 2-digit SIC code and closest in size. This paper
suffers from error 1, 2 and 3. The analysis fails to incorporate matching. Additional
control is needed for “closest size.” To generalize the results to the population, the
researcher needs to reweight the sample.
Research Stream 8: Auditor Changes
Auditor changes are events of concern to firms and investors because auditor
change may contain information content. Researchers are interested in auditor changes
because identifying factors associated with changes may be useful for predicting auditor
change. Knowing when clients change auditors or when auditors are likely to resign
from an engagement may allow an audit firm to control the overall audit risk.
The fourteen papers in this research stream can be grouped into three categories:
(1) market reaction to auditor change; (2) causes of auditor change; and (3) consequences
of auditor change. Seven papers are CB-FM and six papers are CB-SM. Eleven of the
thirteen papers suffer from error 1, a failure to take matching into account in the analysis
of the data. The results of all these papers are suspect, both in the results found and the
results not found. A number of researchers discuss areas where results were expected,
but not found (e.g. Dhaliwal, Schatzberg, and Trombley 1991; Schwartz and Soo 1995.)
In these situations, the failure to find significant results may be a result of either miss-
37
analyzing data (this biases the researcher against finding results) or in not controlling for
other factors in the analysis. Several studies used univariate analysis, rather than
multivariate analysis. Univariate analysis does not allow a researcher to control for other
variables that may affect the analysis and provides very inconclusive results in a situation
where many variables impact the result. Matched sample studies are used in this area of
research because data collection is costly. Data is hand-collected from annual reports, 8K’s, 10-K’s, proxy statements, Moody’s Manuals, and Standard & Poor’s Register of
Corporations. Using a matched sample when data collection is costly can be an efficient
way to analyze data.
Market Reaction to Auditor Changes
Three papers consider market reactions to auditor changes: Fried and Schiff
(1981), Schwartz and Soo (1995), and Dunn, Hillier and Marshall (1999.) Schartz and
Soo and Dunn, Hillier, and Marshall suffer from error 1 making the results reported in
their papers highly questionable.
Fried and Schiff (1981) use univariate pairwise t-tests of difference in market
returns of firms that switch auditors vs. beta- and industry-matched firms that do not
switch auditors. Fried and Schiff report a negative market reaction to a change in
auditors. Schwartz and Soo (1995) consider whether 8-K disclosures of auditor changes
by firms approaching bankruptcy are systematically different from 8-K disclosures of
non-bankrupt firms changing auditors. The results reported in the paper support the
researchers'contention that disclosures of bankrupt firms differ from the disclosures of
non-bankrupt firms. The researchers fail to find evidence that the market penalizes
bankrupt companies that delay the filing of the 8-K report, contrary to expectations.
38
Dunn, Hillier and Marshall (1999) consider the reaction of the stock market in the United
Kingdom to auditor resignation letters. Their results indicate that the market reacts
negatively to the auditor resignation letter, even though few auditors indicated client
problems in the letter. Because the last two studies suffer from error 1, the conclusions
reported by the researchers may not be supported by re-analysis of the evidence.
Causes of Auditor Changes
Seven papers examine causes for auditor changes. All seven papers in this area
suffer from error 1, so the results reported are highly questionable. While the conclusions
reached by the researchers may appear to be reasonable or consistent with a “commonsense” understanding of the phenomena of auditor change, the data analysis reported in
the papers does not provide evidence to support the research theories.
Schwartz and Menon (1985), consider the reasons bankrupt companies change
auditors. Possible factors associated with a change might include: audit qualifications,
reporting disputes, management changes, audit fees, increased need for larger auditors as
firm grows, but their analysis fails to support any of these explanations. Results indicate
only that failing firms have a greater tendency to switch auditors. Seabright, Levinthal,
and Fichman (1992) consider the role that personal attachments between management
and the auditor play in auditor changes. They found that changes in the resource
requirements of a company increased the liklihood of switching auditors. Attachments of
the CFO and Audit Committee to the auditor decreased the liklihood of ending the
relationship. Some of the reported results were contrary to predictions. Re-analysis
taking the matching into account may provide support for their predictions.
39
Krishnan and Krishnan (1997) examine factors influencing auditor changes by
auditor resignation or by dismissal. The study uses two semi-matched samples, by
industry and year or by year alone. The logit analysis fails to account for the matching
variables and hence suffers Error 1 and 3. An unpublished partial replication of this work
suggests that Krishnan and Krishnan’s main result (that an audit litigation risk variable is
significant in determining auditor resignation) is negated when the matching is accounted
for as it should be.8 Menon and Williams (1999), consider audit changes initiated by the
auditor rather than the client. They hypothesize that auditor decisions to resign or decline
to continue an engagement can be explained by audit error cost. Although their results
support their belief, the analysis suffers from error 1. The reported results may change on
re-analysis of the data.
Shu (2000) hypothesizes that auditor resignation is positively related to increased
client legal exposure and to a mismatch between the auditor and the client. Her results
provide support for her hypotheses. She finds significant and negative stock reactions
associated with auditor resignations. The greater the increase in litigation risk, the larger
the drop in stock price, if the resignation occurs during the audit. Because Shu'
s paper
suffers from error 1, her results may not be supported by re-analysis. Woo and Koh
(2001) examines the reasons for auditor changes in Singapore. Their sample includes
companies changing auditors and companies not changing auditors from 1986-1995.
They find that auditor changes are more likely in cases where firms engage smaller audit
firms, change their management composition, experience a lower diffusion of ownership,
experience an increase in income manipulation opportunities, have higher leverage, have
8
We do not have the partial replication results to share, as they have not been made available to us.
40
many subsidiaries or have more rapid growth. The researchers tested sixteen
characteristics to explain the direction of auditor change. Of the sixteen variables tested,
ten were found to be significant. Because their results failed to incorporate matching, the
variables identified may be inaccurate. The insignificant variables may also prove to play
a role in the audit change process if the data was subject to re-analysis.
Carcello and Neal (2003) examine auditor dismissals following a going concern
opinion report issued by Big 6 auditors between 1988 and 1999. They find that audit
committees with greater independence, greater governance expertise, and lower
stockholdings are more effective in shielding auditors from dismissal after issuance of a
going concern opinion. The researchers report a counter-intuitive positive relationship
between audit committee members’ financial expertise and dismissals following clean
opinions. Because this study suffers from error 1, the results reported in the paper may
change when re-analysed.
Consequences of Change in Auditors
Four papers consider the consequences of changing auditors in terms of auditorclient fit, the level of non-audit services consumed, the level of performance of the
company changing auditors, and the quality of financial statements of the company
changing auditors. DeBerg, Kaplan, and Pany (1993) does not suffer from error 1.
Eichenseher and Shields (1989), Dhaliwal, Schatzberg, and Trombley (1993), and
Johnson, Khurana, and Reynolds (2002) suffer from error 1, rendering the results from
their studies questionable.
Eichenseher and Shields (1989) posit that the demand for audit services is a
function of the firm'
s capital structure. The researchers consider the relationship between
41
the auditor (Big 8 or not) and four capital structure variables. None of the samples
generated models that were predictive. The paper suffered from error 1 so reanalysis may
be useful. DeBerg, Kaplan and Pany investigated whether there was an association
between the decision to change auditors and the level of non-audit services provided.
They failed to find an association in their analysis. The researchers did report that
companies reduced their consumption of non-audit services after a change in auditors.
Dhaliwal, Schatzberg, and Trombley (1993) investigate the information content of
disagreement disclosures surrounding auditor changes. They find that clients changing
auditors after a disagreement have poorer earnings performance, more debt, lower levels
of current assets, and poorer stock price performance than firms changing auditors
without a disagreement and other firms in the same industry. Because the research
suffers from error 1, the variables identified and the relationships between the variables
and auditor change are unreliable.
Johnson, Khurana, and Reynolds consider whether audit firm tenure is associated
with financial reporting quality. Johnson, Khurana, and Reynolds selected a
COMPUSTAT sample of companies audited by short-tenure firms(2-3 years), medium
tenure firms (4-8 years)and long tenure firms (9 years or longer) with companies matched
on industry and size. The analysis fails to incorporate matching so the results do not
reflect the condition nature of the research design. Reweighting is needed to generalize
to the population and is not performed. Bears on whether mandatory rotation of audit
firms is good policy.
42
Research Stream 9: Detecting Management Fraud
Research in the area of fraud detection is designed to identify factors present
when fraud occurs. This information would be useful for managers interested in
preventing fraud, for auditors responsible for designing the audit so material fraud is
found, and for the regulators making institutional changes likely to lessen the occurrence
of fraud within a company. Results of research done in this area indicate that analytical
procedures might be useful for identifying situations where fraud might be present.
Research conclusions related to the impact of the independence of the board of directors
or the presence of an audit committee are subject to error 1. Therefore the conclusions of
the research may not be supported during re-analysis. One research paper identified
fraud risk factors that might be associated with fraud, but this paper was also subject to
error 1, so the fraud risk factors identified may not stand up to reanalysis. Matched
sample and choice-based research designs have been used to investigate fraud due to the
costs associated with manual data collection.
Four research studies consider the potential for detecting management fraud in the
planning stage of an audit. Two of the studies are choice-based fully matched, one is
choice-based semi-matched, and one is a between subjects non-choice-based study.
Three of the four studies suffer from Error 1. Two of the studies use closest matching (on
size) in pair selection, hence would be required to account for that in the analysis, but do
not (suffering error 2.) Two of the studies use logit and hence are exempt from the need
to reweight data; the other two suffer error 3.
Green and Calderon (1995) identify 86 instances of 10-K statements subsequently
found to be fraudulent, and match to 86 non-fraud firm year observations, with matching
43
by industry, year, and “comparable” size. The work tests whether simple analytical
procedures in the early stage of an audit are effective in signaling management fraud.
Their analysis is by univariate ranked sums of differences, takes matching into account,
but suffers Error 2 for not addressing residual difference in size, and suffers Error 3
because reweighting is not done that would allow the results to generalize to any larger
population. The results of the study indicate that analytical procedures are fairly effective
at detecting management fraud.
Beasley (1996), also discussed in Research Stream 3 on Audit Committee Impact,
employs 75 matched pairs of fraud and non-fraud firms matched by year, stock exchange,
size (market value within 30%), and industry to consider whether including a larger
proportion of outside members on the board of directors reduces the likelihood of fraud.
The results indicate that firms without fraud have a significantly higher percentage of
outside members than firms with fraud. Audit committee presence was found not to be
significant, seemingly contrary to Beasley’s priors. This work was viewed as important,
innovative work on an important topic and received the 1995 AAA Competitive
Manuscript Award. The error suffered Error 1, which Cram, Karan, and Stuart (2007)
show is severe. Sarbanes-Oxley requires an independent audit committee, rather than an
independent board of directors implementing the reverse of what was implied by this
paper. It seems possible that, in reanalysis, Beasley’s main result, that percentage of
outside directors affects the likelihood of financial fraud, could be reversed, and that the
presence of audit committee instead is a significant driver.
Green and Choi (1997) use the sample of firms from Green and Calderon (1995)
to apply a neural network approach which appears to pool the data and not account for
44
the pairings; hence suffers Error 1 in addition to Errors 2 and 3. They assert that their
neural network model achieves higher reliability levels than previous models developed
to detect the presence of fraud in financial statements.
Bell and Carcello (2000) is a CB-SM study that develops a model to estimate the
likelihood of fraudulent financial reporting for an audit client based on the presence or
absence of fraud-risk factors. The researchers identified 7 factors from the list of 46
fraud risk factors that were associated with the likelihood of fraudulent financial
statements. Because this paper suffers from Error 1, the variables identified may not
prove to be significant after re-analysis. Also, variable believed to be insignificant might
be found to be significant, in reanalysis.
DISCUSSION OF AUDIT RESEARCH STUDIES BY JOURNAL AND YEAR
Frequency of Errors by Journal:
In Table 4, we tabulate the occurrence of errors across journals in which the
articles appear. The top three journals in terms of numbers of papers in our sample are
Auditing: a Journal of Practice & Theory, with 22 papers, The Accounting Review, with
10 papers, and Accounting and Business Research, with 9 papers. This is reflective of
where auditing papers appear. There is no obvious evidence of editorial differences
across journals relating to the errors addressed in this paper.
Frequency of Errors by Year:
Examination of trends over time, tabulated in Table 5, is cause for concern.
Overall, 75% of papers have suffered Error 1, the most essential error, and all but one of
24 recent papers (published during 2000-2003) suffer from it. The exception appeared in
a journal outside the mainstream of accounting research, in Decision Sciences, it
45
employed a Choice-Based Non-Matched sample, so did not require matching to be
accounted for in the analysis. The average number of conceptual errors has hovered
around two per paper.
Cross Tabulation by Journal Quality against Time Range:
When we group journals by quality category (Higher tier accounting journals,
AJPT, lower tier accounting journals, vs. non-accounting journals) and by year ranges, as
tabulated in Table 6, we begin to reveal some trends in the dispersion of the matched and
choice-based research designs. Audit researchers used matched and choice-based
samples first in the Upper tier journals (defined as CAR, JAE, JAR, TAR). Usage in
AJPT and in lower tier journals followed, and the cumulative usage is now about equal.
Audit research has appeared in journals that are not primarily accounting journals only
more recently. However, the average number of errors does not yet appear to differ
across journal quality categories. With the distribution of CKS (2007) and the present
research, we are interested to see which journal editors enforce changes going forward.
SUMMARY AND CONCLUDING REMARKS
This article examined the use of choice based and matched sample designs in
published auditing research, identified model specification issues, and clarified the
matched sample research design approach in order to promote its use as an effective tool.
Researchers have often selected matched samples based on industry and size, and
assumed this effectively controls for industry and size effects in their studies. But, as has
been demonstrated with simulations and formal proofs, this approach can and does lead
to incorrect conclusions. This leaves us with uncertainty in the accuracy of results of a
46
number of published papers within auditing and beyond (in other accounting and finance
research). Perhaps more importantly, the power to detect statistically significant
relationships may have been lost in numerous research studies that have gone
unpublished, despite researchers having collected data that may have held important
results.
We hope that our identification of the need to reanalyze various auditing research
studies will spur expanded use of the choice based and matched sample design in future
accounting research. Data analysis using properly specified models might allow
researchers to find new results in previously published choice based and matched sample
studies, or to resolve discrepancies of findings within and across papers. Additionally, a
better understanding of matched sample design may facilitate reanalysis of datasets that
have yet to yield publishable results, perhaps due to incorrect analysis. Further the
improved understanding should facilitate more rapid penetration of entirely new areas of
research interest where data collection is initially costly.
47
References
Abbott, L. J., Y. Park, and S. Parker. 2000. The Effects of Audit Committee Activity and
Independence on Corporate Fraud. Managerial Finance 26 (11): 55-67.
Agrawal, A. and S. Chadha. 2005. Corporate Governance and Accounting Scandals.
Journal of Law, Economics & Organization. 48: 371-406.
Allen, A. C. 1994. The Effect of Large-Firm Audits on Municipal Bond Rating
Decisions. Auditing: A Journal of Practice & Theory 13 (1): 115-125.
Altman, E.I. 1968. Financial Ratios as Predictors of Failure. Journal of Finance 23(4):
589-609.
Bartov, E., F. A. Gul, and J. S. L. Tsui. 2001. Discretionary-Accruals Models and Audit
Qualifications. Journal of Accounting and Economics 30: 421-452.
Bauwhede, H. V., M. Willekens, and A. Gaeremynck. 2003. Audit Firm Size, Public
Ownership, and Firms’ Discretionary Accruals Management”. International
Journal of Accounting 38:1-22.
Beasley, M. S. 1996. An Empirical Analysis of the Relation Between the Board of
Director Composition and Financial Statement Fraud. The Accounting Review 71
(4): 443-465.
Beaver, W. 1966. Financial Ratios as Predictors of Failure. Empirical Research in
Accounting: Selected Studies, 1966, supplement to Journal of Accounting
Research: 71-111.
Beaver, W. 1968. Market Prices, Financial Ratios, and the Prediction of Failure. Journal
of Accounting Research 6(2): 179-192.
Becker, C. L., M. L. DeFond, J. Jiambalvo, and K. R. Subramanyam. 1998. The Effect of
Audit Quality on Earnings Management. Contemporary Accounting Research 15
(1): 1-24.
Bedard, J. C., and L. E. Graham. 2002. The Effects of Decision Aid Orientation on Risk
Factor Identification and Audit Test Planning. Auditing: A Journal of Practice &
Theory 21 (2): 39-56.
Behn, B. K., S.E. Kaplan, and K. R. Krumwiede. 2001. Further Evidence on the
Auditor’s Going-Concern Report: The Influence of Management Plans. Auditing:
A Journal of Practice & Theory 20 (1): 13-28.
48
Bell, T. B., and J. V. Carcello. 2000. A Decision Aid for Assessing the Likelihood of
Fraudulent Financial Reporting. Auditing: A Journal of Practice & Theory 19 (1):
169-184.
Buchman, T., and D. Collins. 1998. Uncertainty about Litigation Losses and Auditors.
Journal of Business Research 43: 57-63.
Carcello, J. V., and T. L. Neal. 2003. Audit Committee Characteristics and Auditor
Dismissals following ‘New’ Going-Concern Reports. The Accounting Review 78
(1): 95-117.
Carcello, J. V., and A. L. Nagy. 2004. Audit Firm Tenure and Fraudulent Financial
Reporting. Auditing: A Journal of Practice & Theory 23(2): 55-69.
Carlson, S. J., G. W. Glezen, and M. E. Benefield. 1998. An Investigation of Investor
Reaction to the Information Content of a Going Concern Audit Report while
Controlling for Concurrent Financial Statement Disclosures. Quarterly Journal of
Business and Economics 37 (3): 25-39.
Chen, K. C. W., and K. B. Church. 1992. Default on Debt Oblication and the Issuance of
Going-Concern Opinions. Auditing: A Journal of Practice & Theory 11 (2): 3049.
Chung, Firth, Kim . 2003. Auditor Conservatism and Reported Earnings. Accounting and
Business Research 33 (1): 19-.
Citron, D. and R. Taffler. 1992. The Auditor Report under Going Concern Uncertainties:
an Empirical Analysis. Accounting and Business Research 22: 337-345.
Citron, D. and R. Taffler. 2001. Ethical Behavior in the U.K. Audit Profession: The Case
of the Self-Fulfilling Prophecy Under Going-Concern Uncertainties. Journal of
Business Ethics 29: 353-363.
Clarkson and Simunic. 1994. The Association Between Audit Quality, Retained
Ownership, and Firm-Specific Risk in U.S. vs. Canadian IPO Markets. Journal of
Accounting and Economics 17: 207-228.
Colbert, G., and D. Murray. 1998. The Association Between Auditor Quality and Auditor
Size: An Analysis of Small CPA Firms. Journal of Accounting, Auditing, and
Finance 13 (2): 135-150.
Cormier, D., M. Magnan, and B. Morard. 1995. The auditor’s consideration of the going
concern assumption. Journal of Accounting, Auditing, and Finance 10 (2): 201222.
49
Cram, D., V. Karan, and I. Stuart. 2005. Three Threats to Validity of Choice-Based and
Matched Sample Studies in Accounting Research. Working paper. Available at
the Social Science Research Network at http://ssrn.com/abstract=955031.
DeAngelo, L. 1981. Auditor Size and Audit Quality. Journal of Accounting and
Economics 3(3): 183-199.
DeBerg, C., S. Kaplan, and K. Pany. 1991. An Examination of Some Relationships
Between Non-Audit Services and Auditor Change. Accounting Horizons (March):
17-28.
Dechow, P.M., R.G. Sloan and A.P. Sweeney. 1996. Causes and Consequences of
Earnings Manipulation: An Analysis of Firms Subject to Enforcement Actions by
the SEC. Contemporary Accounting Research 13 (1):1-36.
DeFond, M. L., and J. Jiambalvo. 1993. Factors related to auditor-client disagreements
over income-increasing accounting methods. Contemporary Accounting Research
9 (2): 415-431.
Dhaliwal, D. S., J. W. Schatzberg, and M. A. Trombley. 1993. An Analysis of the
Economic Factors Related to Auditor-Client Disagreements Preceding Auditor
Change. Auditing: A Journal of Practice & Theory 12 (2): 22-38.
Dopuch, N., R. W. Holthausen, and W. Leftwich. 1987. Predicting Audit Qualifications
with Financial and Market Variables. The Accounting Review 62 (3): 431-454.
Dunn, J., D. Hillier, and A. P. Marshall. 1999. The Market Reaction to Auditor
Resignations. Accounting and Business Research 29 (2): 95-108.
Eichenseher, J. W., and D. Shields. 1989. Corporate capital structure and auditor ‘fit’.
Advances in Accounting, Supplement 1: 39-56.
Elliott, J. A. 1992. “Subject to” Audit Opinions and Abnormal Security Returns—
Outcomes and Ambiguities. Journal of Accounting Research 20 (2): 617-638.
Farber, D.B. 2005. Restoring Trust After Fraud: Does Corporate Governance Matter? The
Accounting Review 80(2): 539-561.
Feltham, G. A, J. S. Hughes, and D. A. Simunic. 1991. Empirical Assessment of the
Impact of Auditor Quality on the Valuation of New Issues. Journal of Accounting
and Economics 14: 375-399.
Fleak, S. K., and E. R. Wilson. 1994. The Incremental Information Content of the GoingConcern Audit Opinion. Journal of Accounting, Auditing, and Finance 9 (1): 149166.
50
Foster, B. P., T. J. Ward, and J. Woodroof. 1998. An Analysis of the Usefulness of Debt
Defaults and Going Concern Opinions in Bankruptcy Risk Assessment. Journal
of Accounting, Auditing and Finance 13 (3): 351-376.
Fried, D., and A. Schiff. 1981. CPA Switches and Associated Market Reactions. The
Accounting Review 56 (2): 326-341.
Gaeremynck, A., and M. Willekens. 2003. The Endogenous Relationship Between AuditReport Type and Business Termination: Evidence on Private Firms in a NonLitigious Environment. Accounting and Business Research 33 (1): 65-79.
Geiger, M.A. and D.V. Rama. 2003. Audit Fees, Nonaudit Fees, and Auditor Reporting
on Stressed Companies. Auditing: A Journal of Practice & Theory 22(2): 53-69.
Green, B. P., and T. G. Calderon. 1995. Analytical Procedures and Auditors’ Capacity to
Detect Management Fraud. Accounting Enquiries 5 (1): 1-48.
Green, B. P., and J. H. Choi. 1997. Assessing the Risk of Management Fraud Through
Neural Network Technology. Auditing: A Journal of Practice & Theory 16
(1):14-28.
Hay, D. C., W. R. Knechel, and N. Wong. 2006. Audit Fees: A Meta-analysis of the
Effect of Supply and Demand Attributes. Contemporary Accounting Research
23(1):141-91.
Heninger, W. G. 2001. The Association between Auditor Litigation and Abnormal
Accruals. The Accounting Review 76 (1): 111-126.
Iyer, V. M., and G. S. Iyer. 1996. Effect of Big 8 Mergers on Audit Fees: Evidence from
the United Kingdom. Auditing: A Journal of Practice & Theory 15 (2): 123-132.
Jamal, K., and H. Tan. 2001. Can Auditors Predict the Choices Made by Other Auditors?
Journal of Accounting Research 39 (3): 583-597.
Johnson, V. E., I. K. Khurana, and J. K. Reynolds. 2002. Audit-firm Tenure and the
Quality of Financial Reports. Contemporary Accounting Research 19 (4):637660.
Kida, T. 1980. An Investigation into Auditors’ Continuity and Related Qualification
Judgments. Journal of Accounting Research 18 (2): 506-523.
Kinney, W. R., and L. S. McDaniel. 1993. Audit Delay for Firms Correcting Quarterly
Earnings. Auditing: A Journal of Practice & Theory 12 (2): 135-142.
51
Kleinman, G., and A. Anandarajan. 1999. The Usefulness of Off-Balance Sheet Variables
as Predictors of Auditor’s Going Concern Opinions. Managerial Auditing Journal
14 (6): 273-285.
Koh, H. C. 1991. Model Predictions and Auditor Assessments of Going Concern Status.
Accounting and Business Research 21 (84): 331-338.
Koh, H. C., and S. S. Tan. 1999. A Neural Network Approach to the Prediction of Going
Concern Status. Accounting and Business Research 29 (3): 211-216.
Krishnan, G. 2003. Audit Quality and the Pricing of Discretionary Accruals. Auditing: A
Journal of Practice & Theory 22 (1):109-126.
Krishnan, J., and Krishnan, J. 1997. Litigation Risk and Auditor Resignations. The
Accounting Review 72 (4): 539-560.
Kreutzfeldt, R.W., and Wallace, W. A. 1986. Error Characteristics in Audit Populations:
Their Profile and Relationship to Environmental Factors. Auditing: A Journal of
Practice & Theory 6 (1):20-43.
Kreutzfeldt, R.W., and Wallace, W. A. 1990. Control Risk Assessments: Do They Relate
to Errors? Auditing: A Journal of Practice & Theory.
Krogstad, L., and G. Smith. 2003. Assessing the Influence of Auditing: A Journal of
practice & Theory: 1985-2000. Auditing: A Journal of Practice & Theory 22
(1): 195-204.
Lenard, M. J., P. Alam, and G. R. Madey. 1995. The Application of Neural Networks and
a Qualitative Response Model to the Auditor’s Going Concern Uncertainty
Decision. Decision Sciences 26 (2): 209-227.
Lenard, M. J., P. Alam, and G. R. Madey. 1998. The Design and Validation of a Hybrid
Information System for the Auditor’s Going Concern Decision. The Journal of
Management Information Systems 14 (4): 219-237.
Lenard, M. J., P. Alam, and D. Booth. 2000. An Analysis of Fuzzy Clustering and a
Hybrid Model for the Auditor’s Going Concern Assessment. Decision Sciences
31 (4): 861-884.
Levitan, A. S., and J. A. Knoblett. 1985. Indicators of Exceptions to the Going Concern
Assumption. Auditing: A Journal of Practice & Theory 5 (1): 26-39.
Lys, T. and R. Watts. 1994. Lawsuits Against Auditors. Journal of Accounting Research
32 (Supplement): 65-93.
52
Maddala, G. S. 1991. A Perspective on the Use of Limited-Dependent and Qualitative
Variables Models in Accounting Research. The Accounting Review 66: 788-807.
Maher, M., P. Tiessen, R. Colson, and A. Broman. 1992. Competition and Audit Fees.
The Accounting Review 67: 199-211.
McMullen, D. A. 1996. Audit Committee Performance: An Investigation of the
Consequences Associated with Audit Committees. Auditing: A Journal of
Practice & Theory 15 (1): 87-103.
Menon, K., and D. D. Williams. 1999. Error Cost and Auditors’ Termination Decisions.
Journal of Accounting, Auditing and Finance: 95-123.
Morris, R. E.,and J. R. Strawser. 1999. An Examination of the Effect of CPA Firm Type
on Bank Regulators’ Disclosure Decisions. Auditing: A Journal of Practice &
Theory 18 (2): 143-158.
Mutchler, J. 1985. A Multivariate Analysis of the Auditor’s Going-Concern Opinion.
Journal of Accounting Research 23(2): 668-682.
Palepu, K.G. 1986. Predicting Takeover Targets: a Methodological and Empirical
Analysis. Journal of Accounting and Economics 8: 3-35.
Peasnell, K. V., P. F. Pope, and S. Young. 2001. The Characteristics of Firms Subject to
Adverse Ratings by the Financial Reporting Review Board. Accounting and
Business Research 31(4): 291-311.
Ponemon, L. and A. Schick. 1991. Financially Distressed Companies and Auditor
Perceptions of the Twelve Characteristics of Decline. Auditing: A Journal of
Practice & Theory 10: 70-83.
Richardson, S.A. 2005. Discussion of Consequences of Financial Reporting Failure for
Outside Directors: Evidence from Accounting Restatements and Audit
Committee Members. Journal of Accounting Research 43(2): 334-342..
Sanders, G., A. Allen, and L. Korte. 1995. Municipal Audit Fees: Has Increased
Competition Made a Difference? Auditing: A Journal of Practice & Theory 14
(1): 105-114.
Schwartz, K. and K. Menon. 1985. Auditor Switches by Failing Firms. The Accounting
Review 60 (April): 248-261.
Schwartz, K. B., and B. S. Soo. 1995. An Analysis of Form 8-K Disclosures of Auditor
Changes by Firms Approaching Bankruptcy. Auditing: A Journal of Practice &
Theory 14 (1): 125-136.
53
Seabright, M. A., D. A. Levinthal, and M. Fichman. 1992. Role of Individual
Attachments in the Dissolution of Interorganizational Relationships. Academy of
Management Journal 35 (1): 122-160.
Seipel, C., and L. Tunnell. 2000. A Stochastic dominance analysis of the issuance of
qualified opinions. American Business Review: 32-39.
Shu, S. Z. 2000. Auditor Resignations: Clientele Effects and Legal Liability. Journal of
Accounting and Economics 29: 173-205.
Simunic, D. A. 1980. The Pricing of Audit Services: Theory and Evidence. Journal of
Accounting Research 18(1): 161-190.
Srinivasan, S. 2005. Consequences of Financial Reporting Failure for Outside Directors:
Evidence from Accounting Restatements and Audit Committee Members.
Journal of Accounting Research 43(2): 291-334. (Discussed by Richardson)
Stice, J. 1991. Using Financial and Market Information to Identify Pre-Engagement
Factors Associated with Lawsuits Against Auditors. The Accounting Review 65:
516-533.
Teoh, S. and T. J. Wong. 1993. Perceived Auditor Quality and the Earnings Response
Coefficient. The Accounting Review 68: 346-366.
Turpen, R. A. 1990. Differential Pricing on Auditors Initial Engagements: Further
Evidence. Auditing: A Journal of Practice & Theory 9(2): 60-76.
Vafeas, N. 2001. On Audit Committee Appointments. Auditing: A Journal of Practice &
Theory 20: 197-207.
Vanstraelen, A. 2002. Auditor Economic Incentives and Going-Concern in a Limited
Litigious Continental European Business Environment: Empirical Evidence from
Belgium. Accounting and Business Research 21(3): 171-186.
Vanstraelen, A. 2003. Going Concern Opinions, Auditor Switching, and the SelfFulfilling Prophecy Effects Examined in the Regulatory Context of Belgium.
Journal of Accounting Auditing and Finance: 231-252.
Walker, P. L., and J. R. Casterella. 2000. The Role of Auditee Profitability in Pricing
New Audit Engagements. Auditing: A Journal of Practice & Theory 19(1): 157167.
Wallace, W. A. and R. W. Kreutzfeldt. 1995. The Relation of Inherent and Control Risks
to Audit Adjustments. Journal of Accounting, Auditing and Finance: 459-481.
54
Wilkerson, J. E., Jr. 1987. Selecting Experimental and Comparison Samples for Use in
Studies of Auditor Reporting Decisions. Journal of Accounting Research 25: 161167.
Woo, E., and H. C. Koh. 2001. Factors Associated with Auditor Changes: A Singapore
Study. Accounting and Business Research 31(2): 133-144.
Zmijewski, M. 1984. Methodological Issues Related to the Estimation of Financial
Distress Prediction Models. Journal of Accounting Research 22: 59-82.
55
Figure 1: Research Design Categories for Choice Based and Matched Samples9
Choice Based
Matched
NCB-FM-W
CB-NM
CB-SM
CB-FM
NCB-SM
NCB-FM-B
Fully Matched
Category
Example
Count in
Audit Research
CB-NM
Choice Based Non-Matched
Palepu (1986)
CB-SM
Choice Based Semi-Matched
Heninger (2001)
16
CB-FM
Choice Based Fully Matched
Lys and Watts (1994)
21
NCB-FM-W Non Choice Based Fully Matched
7
Teoh and Wong (1993)
5
Iyer and Iyer (1996)
15
7
Within-Subject
NCB-FM-B
Non Choice Based Fully Matched
Between Subject
NCB-SM
Non Choice Based Semi-Matched
Krishnan (2003)
STRATIF
Stratified only (NCB, NM)
Kreutzfeldt and Wallace
_5
Total
76
9
Adapted from CKS (2007).
56
Table 1: By Research Stream
Research Stream
Paper
Count
Has
Error
2
0
Has
Error
3
2
Suffer
0
errors
0
Suffer
1
error
3
Suffer
2
errors
2
Suffer
3
errors
0
Avg #
Errors
5
Has
Error
1
5
5
5
1
1
0
4
0
1
1.4
7
3
0
3
3
2
2
0
.86
23
16
10
13
1
11
5
6
1.69
1-Determinants of Audit
Litigation
2-Impact of Audit
Committees
3-Determinants of Audit
Fees
4-Prediction of Financial
Distress Including GCQ
5-Prediction of Audit
Qualifications Other than
GCG
6-Audit Quality
7-Auditor Judgments
8-Auditor Changes
9-Detecting Mgmt Fraud
Subtotal
(Duplications)
1.4
9
7
6
4
0
2
6
1
1.89
8
6
14
4
81
(5)
7
3
12
3
61
(5)
4
1
5
2
29
(0)
7
4
8
2
44
(0)
1
0
0
0
5
(0)
0
5
6
2
35
(5)
3
0
5
1
24
(0)
4
1
3
1
17
(0)
2.25
1.33
1.79
1.75
1.66
--
Total
76
56
29
44
5
30
24
17
1.70
57
Table 2
Errors by Research Design10
CB-FM
21 papers
in audit
research
sample
CB-SM
16 papers
CB-NM
7 papers
NCB-FM-W
5 papers
NCB-FM-B
15 papers
NCB-SM
7 papers
STRATIF
5 papers
Total: 76
10
Treatment
Group
Selected on
basis of
outcome
Selected on
basis of
outcome
Selected on
basis of
outcome
Randomly
selected
sample of
firms
Randomly
selected
sample of
firms
Randomly
selected
sample of
firms
Stratified only
Control
Group
One firm selected as match for each
firm in treatment group from set of
firms having similar characteristics
by matching on “closest” values
Randomly selected from firms not
having the same outcome, but
matching by industry, year, size or
group level
Randomly selected from firms not
having the same outcome
Same subject, usually before and
after
One firm selected as match for each
firm in treatment group from set of
firms having similar characteristics
by matching on “closest” values
Randomly selected from firms not
having the same outcome, but
matching by industry, year, size or
group level
Select at different rates, but within
every group
Error 1
Error 2
Error 3
√
√
√
21
17
11
√
√
√
16
1
N/A
N/A
3
√
5
√
N/A
N/A
√
√
√
10
11
15
√
7
√
0
√
5
0
0
5
56
29
44
2
Adapted, with modifications, from CKS (2007). See that for additional guidance for researchers, by category.
58
Insert Table 3 here. (Note to reader: This Table 3 may appear out of order, at end, or be attached in a
separate PDF document file.)
59
Table 4: Categorization of Studies by Journal of Publication
Journal
ABR
ADVA
AE
AH
AJPT
AMBR
AMJ
CAR
DS
IJOA
JAAF
JAE
JAR
JBR
JMIS
JOBE
MAJ
MF
TAR
QJBE
Total
Count
of
papers
9
1
1
1
22
1
1
3
2
1
7
4
6
2
1
1
1
1
10
1
Suffer
Error 1
Suffer
Error 2
Suffer
0 errors
Suffer
1 error
Suffer
2 errors
8
0
1
1
6
1
0
1
0
0
1
3
2
0
0
0
1
1
3
0
Suffer
Error
3
4
1
1
1
12
1
0
2
2
1
3
2
5
1
1
0
1
1
4
1
9
1
0
0
17
1
1
3
0
1
2
4
3
2
0
1
1
1
8
1
76
55
Avg #
Errors
4
1
1
1
5
0
0
3
0
1
1
1
2
1
0
0
0
0
2
1
Suffer
3
errors
4
0
0
0
5
1
0
0
0
0
0
2
1
0
0
0
1
1
2
0
0
0
0
0
2
0
0
0
0
0
2
0
0
0
0
0
0
0
1
0
1
0
0
0
10
0
1
0
2
0
4
1
3
1
1
1
0
0
5
0
29
44
5
30
24
17
1.70
2.33
2.00
2.00
2.00
1.59
3.00
1.00
2.00
1.00
2.00
.86
2.25
1.67
1.50
1.00
1.00
3.00
3.00
1.50
2.00
Journals, in order of abbreviated titles above, are Accounting and Business Research, Advances in
Accounting, Accounting Enquiries, Accounting Horizons, Auditing: A Journal of Practice & Theory,
American Business Review, Academy of Management Journal, Contemporary Accounting Research,
Decision Sciences, International Journal of Accounting, Journal of Accounting Auditing and Finance,
Journal of Accounting and Economics, Journal of Accounting Research, Journal of Business Research,
Journal of Management Information Systems, Journal of Business Ethics, Managerial Auditing Journal,
Managerial Finance, The Accounting Review, and Quarterly Journal of Business and Economics.
60
Table 5: Categorization of Studies by Year of Publication
Suffer
Error 1
Suffer
Error 2
Suffer
Error 3
Suffer
0 errors
Suffer
1 error
<1980
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
Count
of
papers
0
2
1
0
0
0
3
1
2
0
1
2
5
5
4
4
6
3
2
6
5
7
7
3
7
1
0
2
0
2
1
1
4
3
3
3
1
2
2
3
5
6
7
3
7
1
1
2
0
0
0
0
4
2
3
2
2
0
1
0
3
3
1
1
3
2
1
3
1
0
1
2
4
2
3
4
5
0
1
4
3
4
0
1
3
0
0
0
0
0
0
0
0
1
0
0
1
1
0
2
0
0
0
0
0
Total
76
56
29
44
’80-‘84
3
1
2
‘85-‘89
7
5
‘90-‘94
20
’95-‘99
’00-‘03
Year
Suffer
3 errors
Avg #
Errors
1
0
1
1
2
0
1
0
2
0
1
3
2
1
1
2
3
6
1
2
Suffer
2
errors
0
1
0
0
0
1
1
3
1
3
1
1
0
0
3
0
2
1
2
4
1
0
2
0
0
0
0
2
1
1
2
1
0
1
0
3
2
0
0
1
2.00
2.00
2.33
1.00
1.00
2.00
1.50
2.40
1.40
2.25
2.25
1.33
0.67
2.00
1.17
2.20
1.86
1.14
1.67
1.85
5
30
24
17
1.70
3
0
1
1
1
2.00
2
5
0
4
1
2
1.71
14
11
15
1
4
9
6
2.00
22
13
6
13
4
9
4
5
1.45
24
23
8
8
0
12
9
3
1.63
61
Table 6: Categorization of Studies by Journal Quality Level and Year Range
Journal
Quality
Level/
Yr
Range
Count
of
papers
Suffer
Error 1
Suffer
Error 2
Suffer
Error
3
’80-‘84
’85-‘89
’90-‘94
’95-‘99
’00-‘03
Subtotal
3
4
8
3
5
23
1
3
6
3
5
18
2
1
5
0
1
9
3
2
6
1
1
13
’80-‘84
’85-‘89
’90-‘94
’95-‘99
’00-‘03
Subtotal
0
2
7
6
7
22
0
1
5
4
7
17
0
1
3
2
0
6
0
2
6
2
2
12
’80-‘84
’85-‘89
’90-‘94
’95-‘99
’00-‘03
Subtotal
0
1
4
10
7
22
0
1
2
4
7
14
0
0
3
4
5
12
0
1
3
7
2
13
’80-‘84
’85-‘89
’90-‘94
’95-‘99
’00-‘03
Subtotal
0
0
1
3
5
9
0
0
1
2
4
7
0
0
0
0
2
2
0
0
0
3
3
6
Total
76
55
29
44
Suffer
0 errors
Upper
AJPT
Lower
Other
5
Suffer
1 error
Suffer
2 errors
Suffer
3
errors
Avg #
Errors
0
0
1
0
0
1
1
3
0
2
3
9
1
0
4
1
2
8
1
1
3
0
0
5
2.00
1.50
2.13
1.33
1.40
1.74
0
0
0
2
0
2
0
1
2
2
5
10
0
0
3
0
2
5
0
1
2
2
0
5
-2.00
2.00
1.33
1.29
1.59
0
0
0
2
0
2
0
0
1
4
1
6
0
1
2
1
5
9
0
0
1
3
1
5
-2.00
2.00
1.50
2.00
1.77
0
0
0
0
0
0
0
0
1
1
3
5
0
0
0
2
0
2
0
0
0
0
2
2
--1.00
1.67
1.80
1.67
30
24
17
Upper Tier Accounting Journals defined as: CAR, JAE, JAR, TAR
Auditing defined as AJPT alone
Lower Tier Accounting Journals defined as: ABR, ADVA, AE, AH, IJOA, JAAF, JMIS, MAJ
Other Journals defined as: ACADMJ, AMBR, DS, JBR, JOBE, MF, QJBE.
1.70
Research
Stream
Author(s), Journal, Paper Title
Stice (1991). The Accounting Review,
“Using Financial and Market Information
to Identify Pre-Engagement Factors
Associated with Lawsuits Against
Auditors”.
Lys and Watts (1994).Journal of
Accounting Research, “Lawsuits Against
Auditors”. This paper was discussed by
Jennifer Francis’ “Discussion of Lawsuits
against Auditors”.
Krishnan, J. and Krishnan, J. (1997). The
Accounting Review , “Litigation Risk and
Auditor Resignations”
Shu (2000). Journal of Accounting and
Economics . “Auditor Resignations:
Clientele Effects and Legal Liability”
SSCI
citation
count
1/26/07
60
Research
Design
Category
auditor
litigation
auditor
litigation
CB-FM
dup - 21
cites
auditor
litigation
CB-SM
15
auditor
litigation
CB-SM
Heninger (2001). The Accounting Review ,
“The Association Between Auditor
Litigation and Abnormal Accruals”.
11
Beasley (1996). The Accounting Review ,
“An Empirical Analysis of the Relation
Between the Board of Director
Composition and Financial Statement
Fraud”
McMullen (1996). Auditing: A Journal of
Practice & Theory ,“Audit Committee
Performance: An Investigation of the
Consequences Associated with Audit
Committees”
60
Abbott, Park, Parker (2000). Managerial
Finance . “The Effects of Audit Committee
Activity and Independence on Corporate
Fraud”
Vafeas (2001). Auditing: A Journal of
Practice and Theory , “On Audit
Committee Appointments”.
5
14
auditor
litigation
audit
committee
impact
CB-SM
CB-SM
audit
committee
impact
NCB-SM
0
Principal Analyses Explaining
Choice and/or Using Matching
Identification of Cases, or Stratified Sample
CB-SM
32
Choice-Based and/or Matched Sample Selection
audit
committee
impact
audit
committee
impact
CB-FM
NCB-SM
Create two semi-matched samples from
Compustat firms: one is matched on year only
Identify 49 cases of auditor litigation, excluding
and then random selection, the other is matched
financial and service firms.
on year and industry (SIC3).
Probit regression of auditor
litigation or not
Fully-match 163 firm-year observations: match
by year, industry (3 digit SIC), and Compustat
Identify 163 auditee firms whose auditors were
delisting code, if any, and then select firm of
sued during 1955-1994, and for which Compustat
closest size (total assets).
data was available
OLS (and logit not reported)
regressions of litigation or not
Identify 67 auditee firms with lawsuits against
auditors having 8 years of Compustat data.
Logit regression of auditor
resignations with respect to
litigation risk and clientele
effects.
Identify 262 cases of new appointments of
directors to the audit committee during 19941998.
Fully-match 238 nonexecutive director
appointments: match on firm and year and on
professional affiliation of director (inside vs.
outside director), then select closest in age.
Yes
No
No
No
Yes
No
Yes
No
Yes
Yes
Yes
No
Yes
No
No
No
No
No
Yes
No
No
No
No
No
Yes
No
No
No
No
No
Yes
No
No
No
No
No
Yes
No
No
No
No
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
Yes
No
No
Logit regression explaining
fraud
Logit regression explaining
Semi-match firms: create 5 corresponding
samples of approximately the same size, matched litigation or not, with bootstrap
Identify 5 sets of firms having 5 financial
on year. Further, discussion of the determination
standard errors
reporting issues during 1982-1988 such as
of one variable (audit committee presence or not)
quarterly earnings corrections, ranging in number
suggests a pair-wise correspondence, not fully
from 62 to 96 firms.
described, underla
Fully-match to 78 firm-year observations in nonIdentify 78 firms subject to SEC Accounting and sanctioned firms matched by size, industry,
trading exchange, and time period.
Auditing Enforcement Releases.
Reweight-ing for
generalizing
Logistic regressions of
resignation versus dismissal
Semi-match 67 firm-year observations: match on Logit regression of litigation or
year, industry ( SIC4: 62; SIC3: 5), and then
not
randomly select one.
Semi-match 75 firm-year observations having
Identify 75 public firms having an occurrence of complete data: Match on year, stock exchange,
financial statement fraud publicly reported during firm size within +/-30% in market value, industry
(SIC4 or 3 or 2)
1980-1991.
Additional
control for
“closest”
matching
Needed Applied Needed Applied Needed Applied
?
?
?
?
?
?
Selection of Controls or Comparison Sample,
If Any
Semi-match firm-year observations of firms who
dismissed auditors (auditors did not resign),
Identify 141 firms whose auditors resigned during following Stice (1991): one sample of 141
matched on year only and then random selection,
1989-1995 and all required data available.
a second set of 141 is matched on year and
industry.
Two control groups, 1) for each auditor
resignation firm ten firmsare randomly drawn
from Compustat for the same year resulting in
269 auditor resignations over the years 1988 to 2,690 firms ,2) from 1263 randomly selected
1996.
client initiated auditor changes between 1987 and
1955, 433 firms selected based
Analyses Incorp.
Matching/Stratif
ication?
Linear probability model (OLS
regression) explaining 1-0
sanction or not
Logit regression of director
serving on audit committee or
not
Research
Stream
Author(s), Journal, Paper Title
SSCI
citation
count
1/26/07
Carcello and Neal (2003). The Accounting dup - 10
Review , “Audit Committee Characteristics
cites
and Auditor Dismissals following ‘New’
Going-Concern Reports”
Simunic (1980), Journal of Accounting
Research, " The Pricing of Audit Services:
Theory and Evidence"
182
Turpen (1990). Auditing: A Journal of
Practice and Theory , “Differential Pricing
on Auditors Initial Engagements: Further
Evidence”
13
Maher, Tiessen, Colson, Broman (1992).
The Accounting Review , “Competition and
Audit Fees”.
14
Sanders, Allen & Korte (1995). Auditing:
A Journal of Practice and Theory ,
“Municipal Audit Fees: Has Increased
Competition Made a Difference?”
3
Iyer and Iyer (1996). Auditing: A Journal
of Practice & Theory , “Effect of Big 8
Mergers on Audit Fees: Evidence from the
United Kingdom”
Walker and Casterella (2000). Auditing:
A Journal of Practice & Theory , “The
Role of Auditee Profitability in Pricing
New Audit Engagements”
2
Geiger and Rama (2003). Auditing: A
Journal of Practice & Theory , "Audit
Fees, Nonaudit Fees, and Auditor
Reporting on Stressed Companies"
Kida (1980). Journal of Accounting
Research . “An Investigation into
Auditors’ Continuity and Related
Qualification Judgments”
3
Research
Design
Category
55
Principal Analyses Explaining
Choice and/or Using Matching
Identification of Cases, or Stratified Sample
audit
committee
impact
CB-FM
Identify 66 nonfinancial public firms that received
a Going concern report from a big 6 auditor and
then dismissed the auditor during the period 19881999, but did not go bankrupt, and which had an
audit committee and information available. Also,
randomly
STRATIF
Survey firms in 4 strata: Big 8 vs Non Big 8
Auditor, small vs. large (large is > $125 million
sales). Obtain useful responses in each strata of:
Big 8 Large (172), Big 8 Small (117), Non Big 8
Large (38), and Non Big 8 Small (70)
Determinants
of audit fees
Determinants
of audit fees
NCB-SM
3
Choice-Based and/or Matched Sample Selection
Identify 327 public companies that changed
auditors during 1982-1984. Collect 57 usable
surveys from these.
Determinants
of audit fees
On the pooled sample of 187
First, fully-match each GC firm with a firm
receiving a going concern report but that did not
pairs, run logit regression of
dismiss its auditor: Match by year, industry and
whether dismissed auditor or
closest size. Drop four as matches not available,
not, on variables including
yielding 62 pairs. Second, fully-match clean
interactions with whether
opinion firms
received a going concern or not
NCB-FM-W
NCB-SM
Identify 80 firms with auditor tenure of 1 to 3
years
Determinants
of audit fees
Determinants
of audit fees
Prediction of
Fin Distress
including
GCQ
CB-FM
Semi-match 327 firm-year observations from
Compustat firms that did not change auditors
during 1980-1984: match proportionally by
exchange (Exchange-traded vs. OTC). Collect
89 usable surveys from these, then match
proportionally by year.
CB-FM
Yes
No
Yes
Yes
No
No
No
No
No
No
Yes
No
Yes
No
No
No
Yes
No
Yes
Yes
No
No
No
No
Yes
Yes
No
No
No
No
Yes
Yes
No
No
No
No
Yes
No
Yes
Yes
Yes
No
Yes
No
Yes
Yes
No
No
Yes
No
Yes
No
Yes
No
OLS regression of Log(Audit
Fees) over 57+89=146 firms.
OLS regression of change in
audit fees on other changes
OLS of change in audit fees
regressed on differences, over
159 observations.
Fully-match, prospectively: Each auditee is
OLS of change in audit fees on
regarded as its own control before and after big 8
differences
auditors become the big 6 by mergers during
1989.
Match to 80 firms that have auditor tenure of
Regression explaining audit fees
other than 1 to 3 years, based on total assets and
SIC
Fully-match 66 non-GCQ but financially stressed
66 firms receiving a first-time GCQ opinion and firms by a financial stress measure, industry and
size measured by sales.
meeting data requirements
Identify manufacturing firms having 10 goingconcern potential indicators, from those filing
during May 1974-April 1975, having complete
Moody’s data. Get sample size down to 20 by
eliminations “due to size, date of problem, or
finally on a random basis”.
Reweight-ing for
generalizing
OLS model explaining audit fees
NCB-FM-W
270 UK firms audited by Big 8 auditors and
available data in 1987 and 1991.
Additional
control for
“closest”
matching
Needed Applied Needed Applied Needed Applied
?
?
?
?
?
?
Selection of Controls or Comparison Sample,
If Any
Fully-match, prospectively: Each auditee is
78 firms in a convenience sample (a database for regarded its own control before and after
legislative, enforcement, and rule change events;
which 1977 and 1981 audit fees are known)
no further data collection required.
In 1989, 289 cities responded to another survey.
Determinants
of audit fees
Of these 159 appeared in 1985 survey.
NCB-FM-W 642 cities responding to survey request in 1985.
Determinants
of audit fees
Analyses Incorp.
Matching/Stratif
ication?
Identify 20 non-problem firms matched by year,
industry and asset size.
Logit regression explaining
going concern report
Discriminant analysis of
problem vs. non-problem firms,
using 5 financial ratios. Also,
compare this model’s
performance compared to
experimental subject auditor
judgments on same firms’ data.
Research
Stream
Author(s), Journal, Paper Title
SSCI
citation
count
1/26/07
Levitan and Knoblett (1985). Auditing: A
14
Prediction of
Journal of Practice & Theory , “Indicators
Fin Distress
of Exceptions to the Going Concern
including
Assumption”
GCQ
Mutchler (1985). Journal of Accounting
Research. “A Multivariate Analysis of the
Auditor’s Going-Concern Opinion”
45
Koh (1991) Accounting and Business
Research , “Model Predictions and Auditor
Assessments of Going Concern Status”
Not in
SSCI
Ponemon and Schick [1991]: Auditing: A
Journal of Practice and Theory,
“Financially Distressed Companies and
Auditor Perceptions of the Twelve
Characteristics of Decline”.
Chen and Church (1992). Auditing: A
Journal of Practice & Theory , “Default on
Debt Obligations and the Issuance of
Going-Concern Opinions”
2
25
Citron and Taffler (1992). Accounting and
Business Research, “The Auditor Report
Under Going Concern Uncertainties: An
Empirical Analysis”.
Not in
SSCI
Fleak and Wilson (1994). Journal of
Accounting, Auditing, and Finance . “The
Incremental Information Content of the
Going-Concern Audit Opinion”
Not in
SSCI
Cormier, Magnan, Morard (1995) Journal
of Accounting, Auditing, and
Finance," The auditor’s consideration of
the going concern assumption"
Not in
SSCI
Lenard, Alam, Madey (1995) Decision
Sciences . “The Application of Neural
Networks and a Qualitative Response
Model to the Auditor’s Going Concern
Uncertainty Decision”
36
Prediction of
Fin Distress
including
GCQ
Prediction of
Fin Distress
including
GCQ
Prediction of
Fin Distress
including
GCQ
Prediction of
Fin Distress
including
GCQ
Prediction of
Fin Distress
including
GCQ
Prediction of
Fin Distress
including
GCQ
Prediction of
Fin Distress
including
GCQ
Prediction of
Fin Distress
including
GCQ
Research
Design
Category
Choice-Based and/or Matched Sample Selection
Principal Analyses Explaining
Choice and/or Using Matching
Identification of Cases, or Stratified Sample
CB-FM
CB-NM
35 Compustat firms filing Chapter 11 bankruptcy
in 1980-1981
Identify 165 firms receiving going concern audit
qualifications during 1978-95.
CB-FM
Of one accounting firm’s clients, identify 43
firms in financial distress, have each audit mgr
rate 12 characteristics.
CB-SM
Identify 127 public industrial firms receiving a
going-concern opinion for the first time during
1982-1986. These get weight 1 in analysis.
CB-FM
STRATIF
CB-NM
CB-NM
Additional
control for
“closest”
matching
Reweight-ing for
generalizing
Needed Applied Needed Applied Needed Applied
?
?
?
?
?
?
Selection of Controls or Comparison Sample,
If Any
Fully-match 35 non-bankrupt Compustat firms by Discriminant analysis explaining
year, SIC code, and closest size (assets)
bankruptcy or not
Identify 119 firms having distress but not going
concern qualification, by random selection of
firms for review from 2,855 available
Identify 119 manufacturing firms having a Going
manufacturing firms. 684 were reviewed to find
Concern qualification during an 11 month period.
119 meeting distress criteria.
CB-FM
Analyses Incorp.
Matching/Stratif
ication?
Fully-match to 165 non-GC firms:matched on
industry, size (assets) and year.
Match according to size (revenues within 5%),
public vs. closely held control, and 2 digit SIC
code.
Yes
No
Yes
No
No
No
No
No
Yes
No
Yes
No
Yes
No
Yes
Yes
Yes
No
Yes
No (size
only
coarsely
)
Yes
No
Yes
No
No
No
No
No
Yes
No
Yes
No
Yes
No
No
No
No
No
Yes
No
No
No
No
No
Yes
No
No
No
No
No
Yes
No
Probit regression explaining
bankruptcy, using WESML
ANOVA and MANOVA on 12
constructs of organization
decline surveyed
2x2 tabulations, e.g. ,of auditor
switch or not vs. going concern
or not, (chi-square tests possible
but not performed)
OLS explaining Cumulative
Selected controls from all Compustat firms
distressed but not GC, selecting 25% of those,
Abnormal Returns around audit
opinion release dates
572, and then dropping for data reasons, yielding
325. Choice-based in that controls selected at
25% sampling rate.
Identify 112 firms apparently not facing
Logit, discriminant analysis,
financial difficulties (stock return > 0%)
recursive partitioning
Identify 138 Canadian firms that potentially face
explaining financial difficulty
financial difficulties (stock return < -50%)
status.
Identify 153 firms having Going-Concern
qualifications during 1979-1986 and all data
acceptable.
Selected 40 firms with unqualified audit opinions
from Compustat PST firms (presumably in the
From 1988 Disclosure II Database, identified 40 same year?)
firms having going conceren audit qualification
No
Discriminant analysis of goingconcern qualification or not
Semi-match 127 firm-year observations matched
Logit regression of Going
on year, from 1,015 Compustat firms having
Concern Opinion received or not
negative net worth or other “problem” status.
(Weighting of 1015/127 used in logit regression)
Fully-match two subsamples of 61 and 25
Identify 61 firm-year observations (for 38 UK
unqualified firm-year observations: Match by
firms) receiving going concern audit qualifications
year, industry, size, and financial distress status
but not going bankrupt; identify 25 failed firms
(percentile ranking of an Altman Z-score).
that received GC qualification.
Yes
Logit and Neural network
models explaining GC audit
qualifications
Research
Stream
Author(s), Journal, Paper Title
Carlson, Glezen, Benefield (1998)
Quarterly Journal of Business and
Economics. “An Investigation of Investor
Reaction to the Information Content of a
Going Concern Audit Report while
Controlling for Concurrent Financial
Statement Disclosures”
SSCI
citation
count
1/26/07
Not in Prediction of
SSCI
Fin Distress
including
GCQ
Foster, Ward, Woodroof (1998). Journal
of Accounting, Auditing and Finance .
“An Analysis of the Usefulness of Debt
Defaults and Going Concern Opinions in
Bankruptcy Risk Assessment”
Not in
SSCI
Lenard, Madey, Alam (1998) The Journal
of Management Information Systems ,
“The Design and Validation of a Hybrid
Information System for the Auditor’s
Going Concern Decision”
Kleinman and Anandarajan (1999).
Managerial Auditing Journal. “The
Usefulness of Off-Balance Sheet Variables
as Predictors of Auditor’s Going Concern
Opinions”
Koh and Tan (1999). Accounting and
Business Research , “A Neural Network
Approach to the Prediction of Going
Concern Status”
Not in
SSCI
Morris and Strawser (1999). Auditing: A
Journal of Practice and Theory, “An
Examination of the Effect of CPA Firm
Type on Bank Regulators’ Disclosure
Decisions”
Lenard, Alam, Booth (2000) Decision
Sciences . “An Analysis of Fuzzy
Clustering and a Hybrid Model for the
Auditor’s Going Concern Assessment”
Behn, Kaplan, Krumwiede (2001).
Auditing: A Journal of Practice & Theory ,
“Further Evidence on the Auditor’s GoingConcern Report: The Influence of
Management Plans”
Citron and Taffler (2001). Journal of
Business Ethics . “Ethical Behaviour in
the U.K. Audit Profession: The Case of the
Self-Fulfilling Prophecy Under GoingConcern Uncertainties”
Not in
SSCI
Not in
SSCI
0
3
0
1
Prediction of
Fin Distress
including
GCQ
Prediction of
Fin Distress
including
GCQ
Prediction of
Fin Distress
including
GCQ
Prediction of
Fin Distress
including
GCQ
Prediction of
Fin Distress
including
GCQ
Prediction of
Fin Distress
including
GCQ
Prediction of
Fin Distress
including
GCQ
Prediction of
Fin Distress
including
GCQ
Research
Design
Category
Choice-Based and/or Matched Sample Selection
Principal Analyses Explaining
Choice and/or Using Matching
Identification of Cases, or Stratified Sample
NCB-FM-B
Identify 88 firms that received Going Concern
Audit Reports (GCAR) during 1981-1988, using
NAARS.
CB-NM
Identify 82 firms going bankrupt during 19881991 for an estimation sample, and 44
bankrupcies during 1992-1993 for a holdout
sample. Select the firm-year the year before
bankruptcy.
CB-NM
32 bankrupt firms from 1989 and 26 bankrupt
firms from 1990
CB-SM
CB-FM
CB-SM
Identify 55 distressed but not bankrupt firms
during first period, and 40 during second period.
Selection of which firm-year observation was
mostly 1990. Choice-based, random selection of
firms, and analysis is one obs per firm during
time period apparen
Reweight-ing for
generalizing
ANCOVA explaining market
returns around report dates
Logit regression of bankruptcy
as a function of audit opinion
and other variables
32 firms randomly selected from 1562
nonbankrupt firms from 1989and 26 firms
randomly selected from 1602 nonbankrupt firms
from 1990
Stepwise and non-stepwise
From nonGCAR Big 6 audited firms, identify
From Big 6 audited clients in 1992 Compact
173 firms matched to GCAR firms on size (both discriminant analysis to explain
Disclosure database, Identify 61 firms receiving a
GCAR status
assets and revenues). Authors do not explain this
specific Going Concern audit report (GCAR)
matching procedure in detail. It is not clear how
during 1990-1992 and having complete data.
this matching was
Use sample of Koh (1991) 165 matched.
Neural network explaining
bankruptcy
Use sample of Koh (1991)
Identify 116 Texas banks closed during 19901991.
Additional
control for
“closest”
matching
Needed Applied Needed Applied Needed Applied
?
?
?
?
?
?
Selection of Controls or Comparison Sample,
If Any
Fully-match to 88 non-GCAR firm-year
observations by fiscal year, industry, and a Zscore measure of financial distress.
Analyses Incorp.
Matching/Stratif
ication?
Semi-match to 116 firm-year observations of non- Logit regression of bankruptcy
closed Texas banks: match by year, then
(closure) or not
randomly select one.
Yes
No
Yes
Yes
Yes
No
No
No
No
No
No
No
No
No
No
No
Yes
No
Yes
No
No
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
No
No
No
No
No
No
No
No
Yes
No
Yes
No
No
No
No
No
Yes
No
Yes
Yes
No
No
Apparently same control sample as Lenard et al
(1998)
CB-NM
CB-SM
CB-FM
Apparently same sample as Lenard et al (1998)
Semi-match 148 firm-year observations of
manufacturing firms: match only on year, then
148 publicly traded manufacturing firms receiving
randomly select one.
Going concern report within an unqualified audit
Identify 99 firms receiving GC qualifications
during 1987-1994 (of which 17 went bankrupt
before publishing next accounts (i.e. financial
statements, said British-wise)
Fully-match to 99(?) firms based on year,
exchange listing status, industry, size
(“turnover”, i.e. sales) and Z-score (of which 8
went bankrupt)
Logit regression explaining
going concern report
Logit regression explaining
bankruptcy by sales, Z-score,
opinion (GC or not), listing
status, loss
Research
Stream
Author(s), Journal, Paper Title
Vanstraelen (2002). Accounting and
Business Research , “Auditor Economic
Incentives and Going-Concern in a
Limited Litigious Continental European
Business Environment: Empirical
Evidence from Belgium”
Gaeremynck and Willekens (2003).
Accounting and Business Research , “The
Endogenous Relationship Between AuditReport Type and Business Termination:
Evidence on Private Firms in a NonLitigious Environment”
SSCI
citation
count
1/26/07
Not in Prediction of
SSCI
Fin Distress
including
GCQ
Not in
SSCI
Prediction of
Fin Distress
including
GCQ
Geiger and Rama (2003). Auditing: A
Journal of Practice & Theory , "Audit
Fees, Nonaudit Fees, and Auditor
Reporting on Stressed Companies"
dup - 3 Prediction of
cites
Fin Distress
including
GCQ
Vanstraelen (2003). Journal of
Accounting Auditing and Finance , “Going
Concern Opinions, Auditor Switching, and
the Self-Fulfilling Prophecy Effects
Dopuch, Holthausen, Leftwich (1987).
The Accounting Review , “Predicting Audit
Qualifications with Financial and Market
Variables”
Not in
SSCI
Wilkerson (1987). Journal of Accounting
Research , “Selecting Experimental and
Comparison Samples for Use in Studies of
Auditor Reporting Decisions”
6
Elliott (1992).Journal of Accounting
Research , “’Subject to’ Audit Opinions
and Abnormal Security
Returns—Outcomes and Ambiguities”.
DeFond and Jiambalvo (1993).
Contemporary Accounting Research ,
“Factors related to auditor-client
disagreements over income-increasing
accounting methods”
Kinney and McDaniel (1993). Auditing: A
Journal of Practice & Theory. “Audit
Delay for Firms Correcting Quarterly
Earnings”
Buchman and Collins (1998). Journal of
Business Research , “Uncertainty About
Litigation Losses and Auditors”.
45
Not in
SSCI
!!?!
Not in
SSCI
1
0
Prediction of
Fin Distress
including
GCQ
Prediction of
Audit
Qualification
other than
GCQ
Prediction of
Audit
Qualification
other than
GCQ
Research
Design
Category
Choice-Based and/or Matched Sample Selection
Principal Analyses Explaining
Choice and/or Using Matching
Identification of Cases, or Stratified Sample
CB-FM
CB-FM
CB-FM
CB-FM
CB-SM
CB-SM
Identify 392 bankrupt firms
Identify 114 Belgian firm bankruptcies in 1995
and 1996.
Identify 392 bankrupt firms
Fully-match by year, industry (4 digit NACE
code) and size (assets).
Prediction of
From NAARS, fully match 85 firms by 4 digit
Audit
SIC, sign of earnings change, having no
Identify 85 firms announcing, at year-end,
NCB-FM-B
Qualification
corrections of previously reported interim earnings extraordinary items, and closest in size
other than
(revenues).
Semi-match firm-year observations from firms
Prediction of
Audit
60 firms having audit opinions qualified due to disclosing litigation uncertainty but unqualified:
CB-SM
Match by year???, by industry (SIC3), and then
Qualification
litigation uncertainty and having full data.
other than
randomly select one.
No
Yes
No
No
No
Yes
No
Yes
No
No
No
Yes
No
Yes
Yes
No
No
Yes
No
Yes
No
No
No
Yes
No
No
No
Yes
Yes
Yes
No
No
No
No
No
Yes
Yes
Yes
No
Yes
No
Yes
No
Yes
No
No
No
Yes
Yes
Yes
No
Yes
No
Yes
No
No
No
Yes
No
Logit regression
Semi-match to 33 firms also investigated by the Logit regression explaining audit
qualification or not, on
Identify 16 firms investigated by the SEC for price same 9 SEC investigations, but not receiving
fixing during 1972-1981, and receiving audit audit qualifications. Each of 9 investigations had Cumulative Abnormal Returns
and other variables
qualifications due to the uncertainty, within 9 both outcomes represented.
separate investigations.
Fully-match to 145 firm-year observations not
having audit qualifications: match by year (as
close to same year-end as possible), industry
(usually 4 digit SIC), and closest by a measure of
Fully-match 40 firm-year observations of firms
changing auditors for other reasons: match by
closest year of change, and closest industry.
Yes
Logit regression explaining
going concern report
Probit regression with WESML
Semi-match to N firm-year observations of
NYSE and AMEX firms not having audit
Identify some number N of firm-years having an qualifications (allowing consistency exceptions):
auditor qualification for the first time during 1969- Match on year, then select randomly. After
dropping firms having incomplete CRSP and
1980, of NYSE and AMEX firms.
Compustat, and setting aside holdout sa
Prediction of
Audit
Randomly select 145 firms having a “Subject To”
NCB-FM-B
Qualification
audit qualification and CRSP data availability.
other than
Prediction of
Audit
Of firms changing auditors, identify 40 firms 8-K
Qualification
CB-FM
reporting disputes where auditor objected to
other than
management attempts to report higher earnings
GCQ
Reweight-ing for
generalizing
Logit regression of Going
Concern
Fully match 114 continuing firm-year
Logit regression of audit report
observations: Match by size, industry, and year. (clean or not) NOT SO SIMPLE
has 2 equation modeling
endogeneity
Fully-match 66 non-GCQ but financially stressed
firms by a financial stress measure, industry and
66 firms receiving a first-time GCQ opinion and
size measured by sales.
meeting data requirements
Additional
control for
“closest”
matching
Needed Applied Needed Applied Needed Applied
?
?
?
?
?
?
Selection of Controls or Comparison Sample,
If Any
Fully-match to create two control samples, one of
financially stressed and one of non-stressed
firms: Match by year, industry (4 digit NACE
code) and size (assets).
Analyses Incorp.
Matching/Stratif
ication?
Univariate matched t-tests of
cumulative pair-wise returns
differences
Logit regression explaining
disagreements
OLS of paired differences in
audit delay upon pairwise
difference in explanaotry
variables.
Univariate tests, and also a
pooled logit regression of
material loss or not
Research
Stream
Author(s), Journal, Paper Title
Bartov, Gul, Tsui [2000] Journal of
Accounting and Economics,
“ Discretionary-Accruals Models and
Audit Qualifications”.
Seipel and Tunnell (2000) American
Business Review . “A Stochastic
dominance analysis of the issuance of
qualified opinions”
SSCI
citation
count
1/26/07
20
Prediction of
Audit
Qualification
other than
Not in Prediction of
SSCI
Audit
Qualification
other than
GCQ
Peasnell, Pope, Young (2001). Accounting
and Business Research , “The
Characteristics of Firms Subject to
Adverse Ratings by the Financial
Reporting Review Board”
Not in
SSCI
Feltham, Hughes, Simunic (1991).Journal
of Accounting and Economics , “Empirical
Assessment of the Impact of Auditor
Quality on the Valuation of New Issues”.
24
Teoh and Wong [1993]:The Accounting
Review, “Perceived Auditor Quality and
the Earnings Response Coefficient”.
54
Allen (1994). Auditing: A Journal of
Practice & Theory , “The Effect of LargeFirm Audits on Municipal Bond Rating
Decisions”
0
Clarkson and Simunic (1994). Journal of
Accounting and Economics . “The
Association Between Audit Quality,
Retained Ownership, and Firm-Specific
Risk in U.S. vs. Canadian IPO Markets”
21
Becker, DeFond, Jiambalvo, and
Subramanyam (1998). Contemporary
Accounting Research , “The Effect of
Audit Quality on Earnings Management”
Not in
SSCI
Colbert and Murray (1998). Journal of
Accounting, Auditing, and Finance. “The
Association Between Auditor Quality and
Auditor Size: An Analysis of Small CPA
Firms”
Not in
SSCI
Prediction of
Audit
Qualification
other than
GCQ
Research
Design
Category
Choice-Based and/or Matched Sample Selection
Principal Analyses Explaining
Choice and/or Using Matching
Identification of Cases, or Stratified Sample
CB-FM
CB-FM
CB-FM
Fully match 173 firm-year observations Match on Logit regression explaining
year, 2 digit SIC, and Big 6 vs. non-Big-6
whether audit opinions are
auditor, then select nearest in assets (or, for one
qualified or not.
analysis, nearest in total accruals).
Identify control firm year observation for each,
Chi squared test of pair-wise
Stochastic Dominance status of
Identify 3 case samples of firms having audit matched by size, industry, year-end and/or
CARs
qualifications : 75 going concern, 37 litigation, 27 financial condition (Z-score)
asset valuations
Identify 173 Compustat firms having qualified
audits.
Identify 47 adverse FRRP rulings.
audit quality
Fully-matched pairs are chosen from within the
sample, prospectively: 110, 102, and 85 pairs are
251 cases of firms employing high quality (large)
found with adequate matching by assets, IPO
NCB-FM-B
auditors and 141 firms employing low quality
proceeds, or MVE. Within those, 50, 40, and 37
(small) auditors are identified.
pairs have market returns data available for some
analyses.
audit quality
Fully-match 1,263 big 8 audited firm-yearobservations to nonBig8 ones matching by year,
Identify 1,297 Compustat and CRSP firms having industry (4 digit SIC, or 3 or 2 digit if necessary),
NCB-FM-B non-big 8 auditors during 1980-1989, and 16,000 and then select closest in size (total assets).
having big 8 auditors.
audit quality
CB-FM
audit quality
NCB-SM
audit quality
CB-NM
Identify 174 IPOs in Canada during 1984 and
1987
Yes
No
Yes
No
No
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
No
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
No
No
Yes
No
Yes
No
Yes
No
Yes
No
No
No
Yes
No
No
No
No
No
No
No
OLS explaining market value
OLS regression of cumulative
abnormal returns between
forecast and earnings dates
Logit regression explaining
Compare 44 IPOs having high quality audit to 44
firms having low quality audit matched on size
high/low quality as function of
variables including firm-specific
(assets), underwriter prestige (high/low), and
risk measures.
terms of offering ("firm commitment" or "best
efforts")
OLS regression of est.
Semi-match 2,179 firm year observations of
firms having non-big 6 auditors and match on
discretionary accruals on big 6
10,397 firm-year observations of non-financial
dummy and other variables
year, industry, and decile of operating cash flows
firms having big 6 auditors over 1989-1992 and
(each selected control has at least one big 6
sufficient data.
counterpart).
Compare to 325 audit firms not receiving
qualified or adverse reports, selected by interval
Identify 97 audit firms that received a qualified or
sampling (essentially randomly) from
adverse report from AICPA’s Private Companies
alphabetical files of the PCPS as of 1996.
Practice Section peer review program.
Reweight-ing for
generalizing
Logit regression of FRRP
censure
Fully-match to non-big 8 audited cities matching Ordered logit models explaining
within 3% on population, then selecting closest bond ratings are run within each
sample separately
Identify 212 cities having a Moody’s bond rating match on population. Yield is 125 city pairs.
NCB-FM-B
during 1978-1986 and audited by a big 8 firm
audit quality
Additional
control for
“closest”
matching
Needed Applied Needed Applied Needed Applied
?
?
?
?
?
?
Selection of Controls or Comparison Sample,
If Any
Fully-match to firm-year observations not
receiving adverse FRRP rulings: Match by same
level-6 industry group, year, and closest assets
(require assets within 25%, else go to level-4
industry match), and data available.
Analyses Incorp.
Matching/Stratif
ication?
Ordered logit explaining peer
review outcomes: unqualified /
unqualified with comment /
qualified / adverse.
Research
Stream
Author(s), Journal, Paper Title
Bauwhede, Willekens, Gaeremynck
(2003). International Journal of
Accounting , “Audit Firm Size, Public
Ownership, and Firms’ Discretionary
Accruals Management”.
SSCI
citation
count
1/26/07
Not in audit quality
SSCI
Krishnan, G. (2003). Auditing: A Journal
of Practice & Theory. “Audit Quality and
the Pricing of Discretionary Accruals”
2
Kreutzfeldt and Wallace (1986) Auditing:
A Journal of Practice & Theory , "Error
Characteristics in Audit Populations:
Their Profile and Relationship to
Environmental Factors"
62
Kreutzfeldt and Wallace (1990) Auditing:
A Journal of Practice & Theory , "Control
Risk Assessments: Do They Relate to
Errors?"
Wallace and Kreutzfeldt (1995). Journal
of Accounting, Auditing and Finance ,
“The Relation of Inherent and Control
Risks to Audit Adjustments”
6
Jamal and Tan (2001). Journal of
Accounting Research . “Can Auditors
Predict the Choices Made by Other
Auditors?”
Bedard and Graham (2002). Auditing: A
Journal of Practice & Theory , “The
Effects of Decision Aid Orientation on
Risk Factor Identification and Audit Test
Chung, Firth, Kim (2003). Accounting and
Business Research , “Auditor
Conservatism and Reported Earnings”
Identify 31 industrial firms listed on the Brussels
Stock Exchange having consolidated financial
NCB-FM-B
statements and a comparable matching firm with
sufficient data.
audit quality
Not in
SSCI
3
0
0
auditor
judgment
auditor
judgment
auditor
judment
auditor
judgment
auditor
changes
40
Fully-match 136 firm-year observations: Match
on industry (at least 2 digit NACE code), size
(assets), and year, yielding 31 control firms, and
then collect other years’ data when available for
both case and control firm in a given year,
yielding 136 firm-year observations.
Mean and standard deviation of
number of audit adjustments as
percentage of account balances;
ANCOVA of same comparing
strata
STRATIF
Use sample of Kreutzfeldt and Wallace (1986) Note same sample also used in Wallace and
Identify 260 Arthur Anderson clients, stratified in Kreutzfeldt (1995)
sampling by client size, industry, and public vs.
privately held
STRATIF
Use sample of Kreutzfeldt and Wallace (1990):
select 260 Arthur Andersen audit clients by
stratified sampling in groups by size, industry, and
public versus private
NCB-FM-W
Fully matched in construction in terms of who
Within an auditing firm 14 sets of 3 persons who
works together (and may share common
work together (a manager, a top senior, and a
knowledge base).
mediocre senior) are identified.
OLS regression models
explaining number of audit
adjustments, or in other words,
error rate upon control risk
factors
ANOVA exploring differences
in accuracy of predictions about
audit task choices in other sets.
Matched pairs were defined prospectively in the
OLS regression of auditor
sample creation. In each pair, one auditor is
judgments of number of client
randomly assigned to the positive and one to the engagement risk factors present
negative treatment.
Fully-match to non-Big 6 firms: Match by year,
OLS regression of
industry (2 digit SIC code), and closest in size.
Earnings/price ratio
From all Compustat Plus firm-years in 1988-1997,
This reduces sample greatly. Then, winsorize
NCB-FM-B
select all audited by Big 6 firms.
and delete some observations. Yield is 3,860
observations in total.
Fully-match to 48 firms not switching auditors:
Univariate t-tests of pair-wise
Identify 48 firms switching auditors during 1972- match on firm’s Beta (from a market model), and differences in cumulative market
NCB-FM-B
by industry where possibly (by 4-, 3- 2-digit SIC,
returns
1975 and CRSP data available.
or not at all )
NCB-FM-W
23 pairs of senior auditors who share a client
engagement are identified.
auditor
changes
NCB-FM-B
Identify 132 public firms filing for bankruptcy
during 1974-1982, and other requirements.
Fully-match 132 non-bankrupt firm-year
observations, retrospectively: Match by year,
industry (SIC4), and size (revenues in 4th year
prior to bankruptcy of case firm).
Additional
control for
“closest”
matching
Reweight-ing for
generalizing
OLS regression explaining
discretionary accruals
Identify 3,316 firm-year observations in Non Big Regression of Stock returns on
operating cash flow,
Identify 15,342 firm-year observations in Big 6 6 audited firms that correspond in industry and
Nondiscretionary Accruals, and
audited firms for which corresponding Non Big 6 cash flow decile.
discretionay accruals
control observations in same 2 digit SIC and cash
conditioned on audit quality
flow decile are available.
STRATIF
Analyses Incorp.
Matching/Stratif
ication?
Needed Applied Needed Applied Needed Applied
?
?
?
?
?
?
Selection of Controls or Comparison Sample,
If Any
Identify 260 Arthur Anderson clients, stratified in
sampling by client size, industry, and public vs.
privately held
auditor
judgment
auditor
judment
Principal Analyses Explaining
Choice and/or Using Matching
Identification of Cases, or Stratified Sample
NCB-SM
Fried and Schiff (1981). The Accounting
Review , “CPA Switches and Associated
Market Reactions”
Schwartz and Menon (1985). The
Accounting Review . “Auditor Switches by
Failing Firms”.
Research
Design
Category
Choice-Based and/or Matched Sample Selection
2x2 tables, e.g. bankrupt or not
versus auditor switch or not,
with Chi-squared tests. Auditor
switch is factor in each pairwise,
2x2 test; bankruptcy is not.
Yes
No
Yes
Yes
Yes
No
Yes
No
Yes
Yes
Yes
No
No
No
No
No
Yes
No
No
No
No
No
Yes
No
No
No
No
No
Yes
No
Yes
No
No
No
No
No
Yes
No
No
No
No
No
Yes
No
Yes
No
Yes
No
Yes
Yes
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
Research
Stream
Author(s), Journal, Paper Title
SSCI
citation
count
1/26/07
Eichenseher and Shields (1989). Advances Not in
in Accounting, Supplement , “Corporate
SSCI
capital structure and auditor ‘fit’”.
DeBerg, Kaplan, Pany (1991). Accounting
Horizons, “An Examination of Some
Relationships Between Non-Audit
Services and Auditor Change”.
Not in
SSCI
Seabright, Levinthal, Fichman (1992).
Academy of Management Journal . “Role
of Individual Attachments in the
Dissolution of Interorganizational
Relationships”
80
Dhaliwal, Schatzberg, Trombley (1993).
Auditing: A Journal of Practice & Theory ,
“An Analysis of the Economic Factors
Related to Auditor-Client Disagreements
Preceding Auditor Change”
6
Schwartz and Soo (1995). Auditing: A
Journal of Practice and Theory , “An
Analysis of Form 8-K Disclosures of
Auditor Changes by Firms Approaching
Bankruptcy”
5
Krishnan, J. and Krishnan, J. (1997). The
Accounting Review , “Litigation Risk and
Auditor Resignations”
21
Research
Design
Category
auditor
changes
NCB-FM-B
auditor
changes
Identify 27 AMEX firms that changed auditors
during 1981-1982, and did not go bankrupt or
change auditors again before 198_
Fully-match 27 firm-year observations of firms
not changing auditors during 1974-1984: match
on size (assets), industry (3-digit SIC), and big 8
vs. non-big 8 auditor (matching to the former
auditor of the case firms)
OLS regression with jackknife
standard errors, explaining
auditor change or not (linear
probability model)
Univariate Wilcoxon matched
pair tests of Non-Audit Service
usage
Match to 170 non auditor switch companies by
year and size grouping (assets-based)
Logit models explaining aditor
switch or not
auditor
changes
auditor
changes
NCB-SM
auditor
changes
CB-FM
auditor
changes
Identify 170 cases of auditor changes within 32
industries (2 digit SIC) having 20 or more
companies
Semi-match to 71 firm-years within Smith’s
sample having an auditor change but without a
Within a sample collected by Smith (1988) of reported disagreement: match on industry (64 of
CRSP firms that changed auditors during 1973- 71 matches by 2 digit SIC). Also, create from
1982, identify 71 that issued a timely 8-K report Compustat a second comparison sample of
of a disagreement with the auditor.
industry averages, matched as c
Univariate comparisons of
Cumulative Abnormal Returns
Fully-match 59 firm-year observations from
Compustat firms changing auditors during 1985Identify 59 firms voluntarily making a auditor 1992 that did not go bankrupt: match by industry
change within 3 years of bankruptcy and having (SIC2 in most cases), size (total assets), and
filed 8-Ks, out of 307 firms filing for bankruptcy closest auditor change year???
during 1987-1992
a) Univariate Chi-squared on
various b)
Logit regression
of Bankruptcy c)
Univariate
t-test on CARs around events.
B. is choice-based.
Semi-match firm-year observations of firms who
dismissed auditors (auditors did not resign),
following Stice (1991): one sample of 141
Identify 141 firms whose auditors resigned during matched on year only and then random selection,
1989-1995 and all required data available.
a second set of 141 is matched on year and
industry.
Logistic regressions of
resignation versus dismissal
auditor
changes
NCB-FM-B
Identify 88 auditor resignations
Analyses Incorp.
Matching/Stratif
ication?
Additional
control for
“closest”
matching
Reweight-ing for
generalizing
Needed Applied Needed Applied Needed Applied
?
?
?
?
?
?
Selection of Controls or Comparison Sample,
If Any
Fully-match to AMEX or NYSE firms not
changing auditors, retrospectively: Match on
Identify 83 AMEX or NYSE listed non-bankrupt “comparable” industry, assets, net income, and
NCB-FM-B firms that changed auditors from one Big 8 auditor leverage. 20 firms not matched; yield is 63
to another Big 8 auditor during 1978-1982.
pairs.
CB-SM
Not in
SSCI
Principal Analyses Explaining
Choice and/or Using Matching
Identification of Cases, or Stratified Sample
CB-SM
Dunn, Hillier, Marshall (1999).
Accounting and Business Research , “The
Market Reaction to Auditor Resignations”
Choice-Based and/or Matched Sample Selection
From firms not having auditor resignation, match
Event study of cumulative
on industry, size of company, current status
abnormal returns (CARs) for test
(merged/bankrupt/continuing in operation)
sample and for control sample
Yes
No
Yes
Yes
Yes
No
Yes
Yes
Yes
No
Yes
No
Yes
No
No
No
No
No
Yes
No
No
No
Yes
No
Yes
No
Yes
No
a)Y b)N
c)Y
No
Yes
No
No
No
No
No
Yes
No
Yes
No
Yes
No
Research
Stream
Author(s), Journal, Paper Title
SSCI
citation
count
1/26/07
Menon and Williams (1999). Journal of
Not in
Accounting, Auditing and Finance , “Error SSCI
Cost and Auditors’ Termination
Decisions”
Shu (2000). Journal of Accounting and
Economics . “Auditor Resignations:
Clientele Effects and Legal Liability”
Woo and Koh (2001). Accounting and
Business Research , “Factors Associated
with Auditor Changes: A Singapore
Study”
Johnson, Khurana, and Reynolds (2002).
Contemporary Accounting Rsearch,
"Audit-Firm Tenure and the Quality of
Financial Reports"
Carcello and Neal (2003). The Accounting
Review , “Audit Committee Characteristics
and Auditor Dismissals following ‘New’
Going-Concern Reports”
dup - 15
auditor
changes
CB-SM
8
auditor
changes
Not in
SSCI
Beasley (1996). The Accounting Review ,
“An Empirical Analysis of the Relation
Between the Board of Director
Composition and Financial Statement
Fraud”
Green and Choi (1997). Auditing: A
Journal of Practice & Theory , “Assessing
the Risk of Management Fraud Through
Neural Network Technology”
dup 60
cites
CB-SM
269 auditor resignations over the years 1988 to
1996.
CB-SM
Identify 54 Stock Exchange of Singapore listed
firms that changed auditors during 1986-1995.
From Big 6 audited clients in 1986-1995 Compact
Disclosure database, Identify 821 firms having a
NCB-FM-B
"short-tenure" auditor relationship (2-3 years) and
having complete data.
auditor
changes
CB-FM
Identify 66 nonfinancial public firms that received
a Going concern report from a big 6 auditor and
then dismissed the auditor during the period 19881999, but did not go bankrupt, and which had an
audit committee and information available. Also,
randomly select 125 clean opinion firms that
dismissed auditors (excluding those without audit
committees or without financial data available).
NCB-FM-B
86 firms having 10-K statements subsequently
found to be fraudulent, and data available.
fraud
CB-SM
9
CB-FM
Logit regression of auditor
resignations with respect to
litigation risk and clientele
effects.
Semi-match to firms not changing auditors:
Match by year, country of incorporation, then
random selection of one.
Logit regression explaining
auditor change
Create two samples with full matching: identify
OLS regression explaining
821 firms having a medium-tenure audit client
unexpected accruals
relationship (4-8 years) and 821 firms having a
long-term audit relationship (9+ years), match by
year, industry (2-4 digit SIC), and closest in size
(total assets).
First, fully-match each GC firm with a firm
On the pooled sample of 187
receiving a going concern report but that did not
pairs of two types, run logit
dismiss its auditor: Match by year, industry and regression of whether dismissed
auditor or not, on variables
closest size. Drop four as matches not available,
including interactions with
yielding 62 pairs. Second, fully-match clean
whether received a going
opinion firms to 125 that did not dismiss
concern or not
auditors.
Semi-match 75 firm-year observations having
Identify 75 public firms having an occurrence of complete data: Match on year, stock exchange,
financial statement fraud publicly reported during firm size within +/-30% in market value, industry
(SIC4 or 3 or 2)
1980-1991.
fraud
86 firms having 10-K statements subsequently
found to be fraudulent, and data available.
Logit regression explaining
resigned versus continuing
auditor or declined versus
continuing auditor decisions.
Two control groups, 1) for each auditor
resignation firm ten firmsare randomly drawn
from Compustat for the same year resulting in
2,690 firms ,2) from 1263 randomly selected
client initiated auditor changes between 1987 and
1955, exclude nonCompustat and financial and
utility firms, then randomly select 433 firms.
Fully-match 86 nonfraud firm-year observations
from Compustat retrospectively: match by year,
size (“comparable” assets), and industry.
Fully-match 86 nonfraud firm-year observations
from Compustat: match by year, size, and
industry (4 digit SIC).
Analyses Incorp.
Matching/Stratif
ication?
Additional
control for
“closest”
matching
Reweight-ing for
generalizing
Needed Applied Needed Applied Needed Applied
?
?
?
?
?
?
Selection of Controls or Comparison Sample,
If Any
Create a control sample of 2168 firm year
observations (in 2168 distinct firms ) matched by
Identified 2 samples in the 1990-1996 period; 217 2 digit SIC, Big 6 auditor and year, matched to
firms whose Big 6 auditor resigned, 67 firms the observations in either case samples, and
retained the same auditor for the seven year
whose Big 6 auditor declined reappointment
period.
auditor
changes
auditor
changes
Green and Calderon (1995). Accounting
Enquiries , “Analytical Procedures and
Auditors’ Capacity to Detect Management
Fraud”
Principal Analyses Explaining
Choice and/or Using Matching
Identification of Cases, or Stratified Sample
Not in
SSCI
10
Research
Design
Category
Choice-Based and/or Matched Sample Selection
Yes
No
No
No
No
No
Yes
No
No
No
No
No
Yes
No
No
No
No
No
Yes
No
Yes
Yes
Yes
No
Yes
No
Yes
Yes
No
No
Yes
Yes
Yes
No
Yes
No
Yes
No
No
No
No
No
Yes
No
Yes
No
Yes
No
Univariate Wilcoxon signed
rank tests of differences
Logit regression explaining
fraud
Neural network explaining
Fraud or non-fraud
Research
Stream
Author(s), Journal, Paper Title
Bell and Carcello (2000). Auditing: A
Journal of Practice & Theory , “A
Decision Aid for Assessing the Likelihood
of Fraudulent Financial Reporting”
Total
SSCI
citation
count
1/26/07
6
Research
Design
Category
Principal Analyses Explaining
Choice and/or Using Matching
Identification of Cases, or Stratified Sample
Selection of Controls or Comparison Sample,
If Any
Employ sample of 77 audit client engagements of
an audit firm where material financial statement
fraud was identified during 1960-1989, as
identified in a prior study.
For comparison, solicit survey responses from a
stratified sample of non-fraud audit engagements
of the same firm during 1990, with stratification
ensuring industry representation was proportional
to the audit firm’s client base (but we presume in
differe
fraud
CB-SM
1027
Choice-Based and/or Matched Sample Selection
Analyses Incorp.
Matching/Stratif
ication?
Additional
control for
“closest”
matching
Reweight-ing for
generalizing
Needed Applied Needed Applied Needed Applied
?
?
?
?
?
?
Logit regressions explaining
fraud or not.
Yes
No
No
No
No
No