POWER ANALYSIS AND SAMPLE SIZE DETERMINATION IN THE CONTEXT OF CORRESPONDENCE ANALYSIS George Menexes1, Iannis Papadimitriou2 University of Macedonia, Thessaloniki, Greece The Correspondence Analysis is considered mainly as a descriptive technique designed to analyze simple two-way and multi-way tables containing some kind of correspondence or association between the rows and the columns. In the case of two variables where the data have been collected using the simple random sampling scheme, the statistical significance of the total inertia of the contingency table can be tested by means of the 2 distribution. If the total inertia is considered as an index of a global effect size then the observed power of the 2 test of independence can be estimated. In the context of the Statistical Power Analysis proposed by Cohen, the minimum required sample size can be estimated, a priori, in such way that a predetermined effect size can be detected as statistically significant by the 2 test, at significance level with power . In this study we introduce the concept of dynamic inertia and we propose a methodology for an a priori and post hoc Power Analysis of the 2 test using the square root of the total inertia and the dynamic inertia as measures of effect size. Using the proposed methodology the minimum required sample size can be estimated in case of a survey or of an experimental study in which the Correspondence Analysis method will be applied. Finally, a generalization of the method is attempted in the multivariate case. Keywords: Chi-square test, effect size, total inertia, dynamic inertia 0 INTRODUCTION A conceptual presentation of the methodology used in Correspondence Analysis (CA) can be provided in a great variety of ways (see Benzécri 1992, Greenacre 1993, Gifi 1996, Nishisato 1980, Israëls 1987). That is probably the main reason why this method “surfaced” on several occasions during the 20th century (Michailidis & De Leeuw, 1998; Clausen, 1998). The CA is mainly considered to be a descriptive method used to examine the relationship between two or more categorical variables. The corresponding results provide information, similar in nature to that produced by the techniques applied in factor analysis, and allow us to study the structure of the variables included in the analysis. One characteristic of this method is that the data from one sample is treated as if it represents the whole population under study. Nevertheless, in the case of two variables, where the collection of the relevant data has been carried out using simple random sampling, the statistical significance of the contingency table’s total inertia can be tested via the 2 distribution (Lebart, et al., 1977; Lebart, et al., 2000). Which is why, many authors (Weller & Romney, 1990; Greenacre, 1993; Van de Geer, 1993; Blasius, 1994; Micheloud, 1997; Clausen, 1998) combine the application of the CA with the 2 test of independence. 1 2 E-mail: [email protected], [email protected] E-mail: [email protected] 1 By tradition, statistical hypothesis testing in scientific research has shown an obvious preference towards the use of statistical significance as a criterion for the rejection or nonrejection of the null hypothesis H0 (Huck, 2000); with the result that more emphasis was placed on the testing and management of Error Type . In recent years, however, and particularly following Cohen’s studies (1962, 1965, 1988) related to Statistical Power Analysis in Behavioral Sciences, the attention of researchers is beginning to turn also towards the testing of Error Type II and the necessity of analyzing the power of statistical tests (Cohen, 1988; Murphy & Myors, 1998). This necessity has exceeded the boundaries of Behavioral Sciences and has made its presence in other scientific fields also (see Muller, et al., 1992; Hubbard & Armstrong, 1992; Verma & Goodale, 1995; Buhl-Mortensen, 1996; Thomas & Juanes, 1996; Heidelbaugh & Nelson, 1996; Meyer & Mark, 1996; Miller, et al., 1997; Sheppard, 1999; Evans & Viengkham, 2001; Foster, 2001; Di Stefano, 2001; Nutahara, et al., 2001). Hallahan & Rosenthal (1996) and Sheppard (1999) present an informative introduction to the theory of Power Analysis of statistical tests, while Thomas & Krebs (1997) attempt to make a comparative presentation of the software used for Power Analysis. In fact, Gatti & Harwell (1998) recommend the use of software for Power Analysis as opposed to traditional Power Charts, which do not facilitate the reading of the numerical values and can lead to false estimations, due to the frequently required linear interpolation. Within the framework of Statistical Power Analysis recommended by Cohen (1988), it is also possible to make an a priori estimation of the minimum required sample size, so that the 2 statistical test of independence for two categorical variables, at a significance level , with power , can detect a predefined Effect Size (ES) as statistically significant. The issue of Power Analysis in relation to the statistical test 2 has also concerned other researchers (Meng & Chapman, 1966; Nathan, 1972; Guenther, 1977; Lachin, 1977). The methodologies they propose seem to resolve the problem on a local basis (e.g. for specific sampling or experimental designs and specific alternative hypotheses) and not through a general methodological framework, like Cohen’s proposal, which, as we shall see later on, also allows us to connect Power Analysis to the total inertia of the contingency table of two categorical variables. In the present paper, we consider total inertia as an index of effect size, based on which we can estimate the observed power of the statistical test of independence 2. We introduce the concept of dynamic inertia of a contingency table with two categorical variables and propose a methodology for an a priori and post-hoc Power Analysis of the statistical test 2 , using the square root of the total inertia and the dynamic inertia as indexes of effect size. Through the methodology we recommend, it is possible to estimate the minimum required sample size for a sampling or experimental research, on whose data Correspondence Analysis will be applied. In Chapters 1-4, we present the basic notation and concepts related to the process of statistical hypothesis testing (Error Type , Error Type II, Error Type ½, Power, Observed Significance Level). Chapter 5 is a summary of the main arguments on which the criticism against statistical hypothesis testing is based. In Chapter 6, there is a brief presentation of a priori and post hoc Power Analysis within the framework recommended by Cohen. Chapter 7 includes an overview of the Power Analysis methodology for the 2 test of independence in the case of two categorical variables and of the relation between the total inertia of the relevant contingency table and the effect size, as determined by Cohen for the test of independence 2. In Chapter 8, we suggest ways of predetermining the effect size and highlight the relationships between the total inertia and other contingency coefficients. In Chapter 9 we introduce the concept of dynamic inertia of a contingency table with two categorical variables. Chapter 10 involves an attempt to generalize on the method for 2 calculating sample size in the case of multiple variables. In Chapter 11, we present the Power Analysis methodology in the case of the 2 goodness of fit test, in order to determine the number of significant factorial axes that will remain after CA is applied to a contingency table with two variables. Finally, Chapter 12 includes numerical examples (the calculations were made using MS-Excel software with the support of the add-in -face). 1 THE STATISTICAL SIGNIFICANCE OF TOTAL INERTIA Let us suppose that X and Y are two categorical variables with k and l categories respectively. We symbolize with: F: the k l contingency table of absolute frequencies with elements fij (i=1,…,k and j=1,…,l) which expresses the joint distribution of the variables and Y fi+ : the marginal absolute frequency of row i, i=1,…,k in table F f+j : the marginal absolute frequency of column j, j=1,…, l in table F : the grand total of table F with fi f i j N j P: the k l correspondence table whose elements are the elements of table F divided by ; i.e. the elements of table P are provided by the formula: f ij pij N , i=1,…,k and j = 1,…, l ri: the mass (or weight) of row i in table P, where: ri f ij p ij j N j , fi i=1,…,k and j=1,…, l N cj: the mass (or weight) of column j in table P, where: cj pij i i f ij f N N j , i=1,…,k and j=1,…, l It is a known fact (see Greenacre, 1993) that the total inertia IF of table F expresses a generalized variance and, more specifically, the weighted average of the squared 2 distances of the row profiles (or equivalents of the column profiles) from their center of gravity. To calculate IF, we can also use the following formulae (Blasius & Greenacre, 1994): ( pij IF i IF j ri c j ) 2 ri c j Q N [1.1] [1.2] From [1.2] Q=NIF [1.3] 3 2 In [1.2], the Q quantity is the statistic as follows: f ij Q i fi f N fi f j j that corresponds to table F and is calculated 2 j ( p ij N i j ri c j ) 2 [1.4] ri c j N When the data has been collected through simple random sampling, based on [1.3] and the acceptance that the preconditions are valid for the application of the 2 statistical test of independence (see Lancaster, 1969), the statistical significance of the quantity NIF can be tested through the 2 distribution with (k-1)(l-1) degrees of freedom (Lebart, et al., 1977; Lebart, et al., 2000). 2 ERROR TYPE AND ERROR TYPE II In any statistical test, the decision related to the rejection of H0 can be correct or false. An erroneous decision is reached when: a) H0 is rejected, when in reality it is true. We then say that an Error Type or error of the first kind is committed. The probability of committing an Error Type is designated by and is the conditional probability: = (rejection of H0 / H0 true) [2.1] b) H0 is not rejected, when in reality it is false. We then say that an Error Type or error of the second kind is committed. The probability of committing an Error Type II is designated by and is the conditional probability: = (non-rejection of H0 / H0 false) [2.2] When an H0 is tested, what is chosen as is a value that expresses the maximum probability of accepting the commitment of an Error Type . This probability is known as significance level and must be determined by the researcher prior to the sampling or execution of an experiment, so that the results of the statistical analyses do not affect its value (Hinkle, et al., 1988; Kachigan, 1991; Cohen, 1988). Thus, the value of should not be determined following certain preliminary data analyses, neither should it be modified in order to cater for the rejection or non-rejection of specific null hypotheses. Furthermore, the significance level expresses the probability of committing an Error Type only when: a) the measurements are valid and reliable and b) the preconditions are valid for the application of the corresponding statistical test. In practice, the conventional (arbitrary) values =0.10 or =0.05 or =0.01 are traditionally used (Hinkle, et al., 1988; Kirk, 1995; Hopkins, 1997; Hair, et al., 1995; Huck, 2000). For example, if in one test the significance level =0.05 or 5% is given and the H0 is rejected, then theoretically in 100 similar cases or 100 repetitions of the experiment, only 5 erroneous decisions are expected to be taken, i.e. rejections of H0, when it is actually correct. It therefore seems that the significance level expresses an error rate that is mainly related to 4 the statistical process and not to the value of the statistic (e.g. t, F and 2) of the test (Lohninger, 1999). The probability of a true H0 not being rejected is determined by the significance level : 1 = (non-rejection of H0 / H0 true) [2.3] The probability of a false H0 being rejected is determined by the probability called the power of the statistical test: =1 = (rejection of H0 / H0 false) and is [2.4] Relations [2.3] and [2.4] express the probability of a correct decision having been taken in a statistical test. Therefore, in order for someone to reach relatively safe and reliable conclusions, based on the available data, the statistical test should minimize and . However, any attempt to minimize one risk causes the other to increase. On a practical level, we attempt to reduce whichever risk is considered most important. One way of simultaneously minimizing both risks is by increasing the sample size (Zar, 1996); however, this is not always feasible due to natural, technical, financial and ethical restrictions. Still, which of the two errors is the most important? The answer is relative, and depends on many factors, such as the general purpose and specific goals of the research, its theoretical framework, the researcher’s knowledge or other justifications. In any case, nevertheless, any decision to reject or not reject a hypothesis must calculate and take into account both and . Statistical tests include several conventions as regards the predefinition of and . For example, many scientists who use statistical testing in their research set 0.05 and 0.20. This means that they consider the risk of committing an Error Type more serious than the risk of an Error Type . If we calculate the ratio: (committing an Error Type II) / (committing an Error Type I), where =0.05 and =0.20, we have, 0.20 / 0.05=4. In this case, the Error Type is considered 4 times more serious, or critical, than the Error Type II. If =0.20 then the power is =0.80. On the other hand, other scientists set the value 0.80 as the minimum accepted power of a statistical test and, if the latter has a smaller power, then they either do not carry out or redesign their research (Kirk, 1995). 3 ERROR TYPE II½ In order to avoid any erroneous conclusions, particular attention should be given to the fact that the test’s inability to reveal a statistically significant result (e.g. difference, effect, correlation) does not signify that the said difference or effect or correlation does not exist in the corresponding populations. This erroneous conclusion is often referred to as Error Type II½ (Kritzer, 1996). It is a logical error or fallacy and pertains to cases when, within a hypothetical syllogism, it is assumed that the conclusion’s argument is valid (i.e. when affirming the consequent) (Dometrius, 1992; Kargopoulos & Raftopoulos, 1998). 5 4 OBSERVED SIGNIFICANCE LEVEL (p-VALUE) The observed significance level of a statistical test is the probability of observing a value of the statistic that is greater or equal to the value given by the sample, provided that the H0 is true, i.e.: p=P(Z |z | / H0 is true) [4.1] where is the random variable that corresponds to the test’s statistic and z the statistic’s value for that specific sample (e.g. t, F and 2). The value of the observed significance level, which is based on the data, will support the decision about whether to reject the H0 or not. If a test’s observed significance level is smaller or equal to the predefined significance level , then the H0 is rejected at significance level (Dometrius, 1992; Kirk, 1995; Kinnear & Gray, 1999). If the observed significance level is greater than the predefined significance level , then the H0 is not rejected. The observed significance level expresses the probability of a statistical result, greater or equal to the observed result, happening “by chance” if the H0 is true (Bryman & Cramer, 1999). The value of the observed significance level expresses the lowest significance level at which the H0 can be rejected. It should however be noted, that in all cases what is actually valid in relation to H0 is unknown. 5 CRITICISM CONCERNING THE H0 SIGNIFICANCE TESTS The Null-Hypothesis Significance-Test Procedure (NHSTP) became the subject of criticism as early as the 1960s, and since then various writers have addressed the issue periodically (Yates, 1951; Kish, 1959; Rozeboom, 1960; Bakan, 1966; Morrison & Henkel, 1970; Pratt, 1976; Cox, 1977; Carver, 1978; Parkhurst, 1985; Guttman, 1985; Oakes, 1986; Chatfield, 1991; Loftus, 1991; Yoccuz, 1991; Schmidt, 1996). The main arguments against the NHSTP can be summarized as follows: The statistical significance of a result can be due to the appropriate choice of the sample size and the significance level . The H0 can never be true. Based on statistical significance we cannot reach conclusions concerning the reverse probability of the hypothesis, i.e. the probability that the H0 is true given the available data. Statistical significance provides no information about the values of the parameters of the populations. Testing for Error Type II is unjustifiably overlooked. Statistical significance cannot be used to reach conclusions related to the practical or clinical significance of a result. The binary logic of the NHSTP (H0 is either rejected or not) does not conform to the fact that knowledge is acquired one step at a time. The procedure in question carries the risk of stochastical and logical errors, as well as misconceptions (Menexes & Oikonomoy, 2002). However, in most cases, the criticism against it is not supported by statements related to the statistical procedure itself, but rather to the fact that the erroneous perceptions of researchers and the stochastically illiteracy are the factors that lead to a wrongful use and interpretation of the results from the H0 statistical significance tests. 6 6 POWER ANALYSIS Power Analysis is commonly carried out during the planning stage of a research or of an experiment, i.e. prior to data collection (a priori) and is used to estimate the probability of a false H0 being rejected. In other words, through Power Analysis we attempt to assess the degree of confidence that will be attributed to the test’s “ability” to actually provide statistically significant results. The power of a statistical test depends mainly on three factors (Cohen, 1988; Murphy & Myors, 1998): a) The significance level b) The sample size n, and c) The Effect size (ES). Effect Size can be generally defined as the extent or the magnitude of the phenomenon under study (Cohen & Cohen, 1983). It is a measurement of the degree to which a phenomenon is realized (Cohen, 1965). From a different viewpoint, ES can also be considered as the observed result’s degree of deviation from the H0 (Kramer & Rosental, 1999). Every statistical test has a different ES that can be measured in two ways (Cohen, 1988; Kramer & Rosental, 1999; Murphy & Myors, 1998): a) as a difference, standardized or not (e.g. Cohen's d, Hedges' g, Glass' delta), or b) as a correlation or contingency (e.g. r, r2, 2, 2, ). In conjunction with power, the three above-mentioned factors constitute a closed system, in the sense that if three of the system’s elements are known and fixed, then the fourth can also be fully defined (Cohen & Cohen, 1983). More specifically, for a given (fixed) n and ES the power of the test increases in line with , for a given and ES the power increases in line with n and for a given and n the power increases in line with ES. The aim of the Power Analysis is to appropriately balance the system’s four parameters, taking into account both the theoretical and practical objectives of the research, in combination with the resources (e.g. financial, technological) available to the researcher. This balance should not contradict the moral-ethical restrictions of the said research. n a practical level, Power Analysis can also give answers to the following two basic questions: a) At a significance level and for a power level , what is the minimum sample size n required for the implemented statistical test to diagnose an ES d as statistically significant? In such a case, d (e.g. 0.20) is an estimation of the minimum ES that can be of practical or clinical significance to the researcher and which is worth detecting as statistically significant. b) Given the sample size n, the significance level and the observed ES, what is the power of the statistical test? The answer to question a) is the a priori approach to power analysis, while the answer to question b) is the post-hoc approach. 7 POWER ANALYSIS ACCORDING TO COHEN FOR A CONTINGENCY TABLE WITH TWO VARIABLES Let us suppose that F is a contingency table of absolute frequencies of two categorical variables X and Y with k and l categories respectively. The general element of table F is fij with i=1,…,k and j=1,…,l (for the notation of the present Chapter, see also Chapter 1). Let us also suppose that the preconditions are valid for the application of the 2 statistical test of independence. The statistical test then realized is the following: H0: X and Y are independent vs Ha: not H0, 7 (at significance level ) 2 The null hypothesis H0 is rejected if Q 2 corresponds to table F and ( k 1)( l 1); , where Q is the statistic 2 is the critical value of the ( k 1)( l 1); 2 that distribution at a significance level with (k-1)(l-1) degrees of freedom (d.f.). Remark 1. If H0 is true, then Q asymptotically follows the 2 distribution with (k-1)(l-1) d.f. If, however, Ha is true, then Q has the limiting non-central 2 distribution, with a non-centrality parameter and (k-1)(l-1) d.f. (Cochran, 1952; Chapman & Nam, 1968; Lachin, 1977; Guenther, 1977). References providing more information concerning the noncentral 2 distribution can be found in Patnaik (1949), Sankaran (1963), Guenther (1964) and Han (1975). In general, the following is valid for the non-centrality parameter (Lachin, 1977): =nf( 0, a ), [7.1] where n is the sample size and f the function of the vectors of parameters 0 and a, which are involved in the statistical test 2 under the H0 and Ha respectively. From a different perspective, f can be considered as the observed result’s degree of deviation from the condition stated through H0 and therefore is a function of the statistical test’s corresponding ES. From [7.1], we find that: n= / f( 0, a ) [7.2] Therefore, if the parameter and its corresponding ES are estimated, then [7.2] can be used to calculate the minimum sample size required, at a significance level and power level , for the statistical test 2 to diagnose the corresponding ES as statistically significant. 7.1 Post-hoc Power Analysis Taking into account [2.2] and Remark 1, the observed Error Type follows: P Q obs where 2 ( k 1)( l 1);a 2 / Ha true P 2 2 nc ( k 1)( l 1) ( k 1)( l 1); a is the value of the non-central nc ( k 1)( l 1) (k-1)(l-1) d.f. Due to [2.4], the observed power expression: obs 1 In order to calculate According to Cohen (1988), obs obs, P obs 2 obs , is estimated as [7.3] distribution with a parameter of the 2 and test is given by the following 2 2 nc ( k 1)( l 1) ( k 1)( l 1); a [7.4] it is necessary to have an estimation of the parameter . 8 =nw2, [7.5] where n is the sample size and w an estimation of the ES, as determined by Cohen for the test of independence 2 with two variables. The ES w is generally estimated using the following formula: kl w p1i p 0i 2 , p0i i [7.6] where p1i and p0i are the relative frequencies of cell i on the contingency table, under the H0 and Ha respectively. In the case of table F in particular, formula [7.6] taking into account formula [1.1] can also be written thus: ( pij w i ri c j ) 2 IF ri c j j [7.7] The w index ranges between 0, which signifies the independence of the two variables, and the maximum value s , where s=min(k-1, l-1) (Cohen, 1988). The limiting maximum value of w signifies a perfect correlation between the two variables. From [7.7] w2 IF [7.8] In addition, from [1.2] it is obvious that: w 2 Q n [7.9] Due to [1.3] and [7.8], formula [7.5] is written as follows: =nw2 =nIF =Q [7.10] Therefore, the observed power of the test of independence following relation: obs P 2 nc ( k 1)( l 1) nI F 2 P ( k 1)( l 1); a 2 nc ( k 1)( l 1) Q 2 is provided by the 2 ( k 1)( l 1); a [7.11] In practice, the numerical calculation of [7.11] is carried out with the aid of noncentral 2 distribution tables (Haynam, et al., 1970) or with the use of software such as SAS and SPSS that include specific functions for all the relevant calculations. 7.2 A Priori Power Analysis Using [7.2] and [7.10], it is possible to make an a priori calculation of the minimum sample size required, when an estimation of the parameter and ES w is given. In this case, we can deduct the sample size from the following formula: 9 n w2 [7.12] IF For the 2 distribution, the values of parameter ( , , u) that correspond to power =1- and a significance level , with u degrees of freedom, can be found in tables (Haynam, et al., 1970; Pearson & Hartley, 1972) or can be calculated using relevant software. The problem lies in providing a predefined estimation of w or IF that has a clinical or practical significance within the framework of the research to be implemented. 8. PREDEFINING THE EFFECT SIZE A predefinition of w or IF can be achieved either through pilot research projects or following meta-analyses of previous comparable studies on the same research subject. In addition, Cohen’s conventions can also be used in relation to what can be considered as a “small”, “medium” or “large” effect size, within the framework of the statistical test of independence 2 (see Table 1). Table 1: Cohen’s Conventions And Correspondence Between w And IF Small ES Medium ES Large ES w=0.10 w=0.30 w=0.50 IF =0.01 IF =0.09 IF =0.25 What is of even greater interest, is the relation between w or IF and contingency coefficients based on the statistic Q. There are two basic reasons that lead us to the abovementioned proposal: a) These indexes can be calculated in a relatively easy manner using published results from other corresponding research studies b) they express the magnitude or the degree of association between the variables on a scale of 0 to a maximum value 1. Two of the most commonly used association indexes for contingency tables with two variables are the Contingency Coefficient C and Cramer’s V index. 8.1 The Relation Of w And IF To The Contingency Coefficient C It is known (Reynolds, 1984) that the C coefficient is given by the formula: C Q [8.1] Q n Due to [7.9] and [7.8], formula [8.1] can also be written thus: C Q n Q n n n w2 w2 1 10 IF IF 1 [8.2] From [8.2] C2 1 C2 w IF [8.3] Therefore, IF C2 1 C2 [8.4] Remark 2. The maximum value of the C coefficient depends on the dimensions of the contingency table, which means that a direct comparison between C indexes from tables with varying dimensions is not feasible (Cohen, 1988). 8.2 The Relation Of w And IF To Cramer’s V Coefficient The Cramer’s V index is given by the formula (Reynolds, 1984): Q , ns V [8.5] where s=min(k-1, l-1). Due to [7.9] and [7.8], formula [8.5] can also be written as follows: w V s IF s [8.6] From [8.6] w V s [8.7] Hence, IF V s IF V 2s [8.8] 9 DYNAMIC INERTIA OF A CONTINGENCY TABLE WITH TWO VARIABLES The concept of inertia plays a core role within the framework of Data Analysis. More specifically, the total inertia of a contingency table with two variables expresses a generalized variance, but can simultaneously be viewed as an index of the information contained within the table. In addition, it has been shown above that total inertia, due to its association with w, can also be regarded as an index of effect size, which expresses the degree or magnitude of 11 the association between two categorical variables. The maximum value of total inertia in the case of two variables is equal to s, where s=min(k-1, l-1). Definition: If IF is the observed inertia of the contingency table F and Imax=s the maximum possible inertia of F with only one constrain concerning the number of rows and columns in the table for a given sample size n, then we define as the dynamic inertia ID of table F the ratio: IF I max ID [9.1] Dynamic inertia expresses the observed inertia as a percentage of the maximum possible inertia. Through formulae [1.2] and [8.5], formula [9.1] can also be written thus: Q ns ID V2 [9.2] Therefore, the dynamic inertia of the contingency table is equal to the square of the contingency coefficient Cramer’s V. The quantity ID 100 expresses IF as a percentage (%) of Imax. For those scientists who use Data Analysis methods in their research and are familiar with the concept of inertia, dynamic inertia can constitute an alternative approach for predefining effect size. The formulae relating ID to w and IF are provided below. w From formulae [8.7] and [9.2] IF From formula [9.2] sI D sI D [9.3] [9.4] Remark 3. In practical applications, for an a priori determination of sample size, it is useful to carry out a relevant sensitivity test, setting limits for ES, and a cost-benefit analysis in order to balance the available resources with the research objectives. 10 CALCULATING SAMPLE SIZE IN THE CASE OF MULTIPLE VARIABLES In the case of q categorical variables, CA is usually applied to the generalized contingency table B (Burt’s table). According to Menexes & Papadimitriou (2004), that part of the total inertia in table B, which expresses the pair-wise correlations (contingencies) between variables and is worth investigating, is the interesting inertia I , defined by the following formula: g Ic I c 1 q (q 1) 2 , g 1, , qq 1 , 2 g I c is the sum of the inertias of the where c 1 [10.1] qq 1 different simple contingency tables of 2 the q variables in pairs. 12 In view of such an observation, we could claim that CA in fact analyzes a “package” of bivariate associations or, in other words, the pair-wise interactions of all variables. And if we consider that total inertia expresses a measure of the information contained within the relevant table or an overall effect size, then we could claim that in each case we are analyzing an average effect size index, which is the result of bivariate and not multivariate relations. In conclusion, it becomes obvious that pair-wise pre-testing of the associations between the variables to be analyzed using CA is a prerequisite. When the sample is derived from simple random sampling, the statistical test of independence 2 can be applied for the qq 1 pair-wise correlations of the variables, with an adjustment of the significance level 2 , for example according to Bonferroni (Girden, 1992; Brown & Melamed, 1990); otherwise, the goodness of fit of a suitable loglinear model can be tested. Therefore, in the case of multiple variables, the minimum required sample size can be calculated for each pair of variables in isolation and then the highest value can be selected qq 1 values that will emerge, so that the requirements of all tests are fulfilled, as from the 2 regards sample size. Naturally, such a process calls for a predefined ES for each bivariate association. 11 POWER ANALYSIS OF THE GOODNESS OF FIT TEST During the application of CA on the contingency table Fk l with two variables, the reconstruction of the table’s absolute frequencies is achieved through the transition formula (Benzécri, 1992): f ij fi f j n u ip v jp 1 p , i=1,…,k and j=1,…,l [11.1] p where p is the eigenvalue (inertia) of the factorial axis p and {uip , vjp} are the standardized coordinates of the rows and column in table F (Greenacre, 1993) respectively, on the factorial axis p where p=0,…r and r=mim(k-1, l-1). The statistical goodness of fit 2 test (Cohen, 1988) can be used to compare the observed frequencies fij in table F, from a random sample size n, with the expected frequencies ij specified by the null hypothesis Hp, which states that only the first p eigenvalues are statistically different to zero (Saporta & Tambrea, 1993). The weighted least squares estimates of the expected frequencies ij are the values that result from the transition formula, where only the first p terms are used in the sum (Andersen, 1991). The goodness of fit test can be realized with the statistic: f ij ij Qp i j 2 [11.2] ij In cases where p=0, i.e. when the two variables are independent, and provided the preconditions for the test’s application are satisfied, the statistic Q0 can be compared with the critical value of the 2 distribution with (k-1)(l-1) degrees of freedom, at a significance level 13 . If p=1, then the statistic Q1 asymptotically follows the 2 distribution with (k-2)(l-2) d.f. In general, under Hp, the statistic Qp can be compared with the critical value of the 2 distribution with (k-p-1)(l-p-1) degrees of freedom (Andersen, 1991; Saporta & Tambrea, 1993; Rao, 1995). The statistical test 2 can be used to test whether the p first factorial axes are sufficient in order to reconstruct table F. In practice, we carry out a series of tests, starting with p=0 until the null hypothesis Hp becomes “accepted” at a significance level . In this way, our interest focuses on the calculation of obs, so as to avoid committing Errors Type and Type ½. The method for calculating the observed power obs of the statistical goodness of fit 2 test can be applied in exactly the same manner as with the test of independence (Cohen, 1988). Remark 4. The statistic Qp has the disadvantage that when the observed frequencies fij are small, then the estimated frequencies ij can even acquire negative values, in which case the statistical test cannot be implemented. What Malinvaud (1987) has recommended on such fi f j an occasion, is to use the quantity as the denominator of the fraction in formula n [11.2]. This then leads us to a modified statistic Q p , given by the formula: f ij ij Qp i j fi f 2 n p 1 p 2 r [11.2] j n The statistic Q p is also a function of the remaining r-p eigenvalues and also asymptotically follows the 12 2 distribution with (k-p-1)(l-p-1) degrees of freedom. NUMERICAL EXAMPLES In the following examples, the numerical calculations required were carried out with the support of the add-in EXCEL -face, which is available at: ftp://ftp.stat.uiowa.edu/pub/rlenth/PiFace/. EXCEL software was chosen as the calculation platform, for two main reasons: a) it is widely-used, and b) it provides the option of creating scenarios and What if Analysis. The face also includes the following functions, amongst others: 1) Chi2PowerG 2) Chi2PowerNC The Chi2Power function accepts as arguments the desired significance level of the 2 test, the non-centrality parameter and the corresponding degrees of freedom. The function returns the observed power of the test and can therefore be used for a post-hoc Power Analysis approach. The Chi2PowerNC function accepts as arguments the desired significance level of the 2 test, the desired power of the test and the corresponding degrees of freedom. It returns the non-centrality parameter and can therefore be used for an a priori Power Analysis approach. 14 Example 1: Post hoc Power Analysis Let us suppose we have two categorical variables X and Y with three categories each. The sample size is n=80. The statistical test 2 has shown that Q=14.32. The value of Q for 4 d.f. is statistically significant at =0.05 (p<0.05). The problem that arises, is how to calculate the observed power obs of the 2 test that corresponds to =0.05. The observed inertia I from the corresponding contingency table with two variables can be calculated using the formula [1.2] and is I=0.179. Formula [7.7] shows us that the inertia I corresponds to an effect size w=0.423 (with a tendency towards a large ES according to Cohen’s conventions). Formula [7.10] can be used to estimate the non-centrality parameter , which is =14.32. From [7.11] and with the help of the Chi2Power function, the observed power of the test is estimated at obs=0.875. Consequently, the probability of the 2 test identifying an ES equal to the one observed as statistically significant, at =0.05, is approximately 87.5%. Example 2: A priori Power Analysis Let us suppose that during the planning stage of a research, attention is focused on testing the association between two categorical variables X and Y with three and four categories respectively. Past experience has shown that an ES that corresponds to a minimum inertia of 0.04 is considered clinically or practically significant, according to the objectives and theoretical framework of the said research. The problem in this case, is how to estimate the minimum required sample size n, so that the statistical test 2 (for 6 d.f.), at =0.10 with power =0.99, can identify the predefined ES as statistically significant. From formulae [7.7], [8.6] for s=2 and [9.2] it is seen that inertia I=0.04 corresponds to w=0.20, to Cramer’s index V=0.14 and to a dynamic inertia ID= 0.019. With the help of the Chi2PowerNC function, the non-centrality parameter =24.65 is calculated. Subsequently, using [7.12] the required sample size is estimated at n=617 sampling units. If the desired power level of the test decreases to 0.95, then the estimated sample size is n=447. And if we set =0.90, then the corresponding sample size is estimated at n=367. In this example, the predefinition of the clinically significant ES could be based either on the V index or on dynamic inertia. The size of the sample that we will eventually attempt to collect will also depend on the resources available. Therefore, the likely effect of other restrictions, which are not related to the subject of the research, should be taken into account during the research planning stage. Example 3: Calculation of sample size in the case of three variables Let us suppose that during the planning stage of a research, attention is focused on testing the pair-wise association between three categorical variables X, Y and Z, with three, four and five categories respectively. Past experience has shown that for the pair of variables (X, Y), an ES that corresponds to a minimum dynamic inertia of 0.005 is considered clinically or practically significant, while the corresponding ES for the pairs (X, Z) and (Y, Z) is 0.045 and 0.085 respectively. The problem in this case, is how to estimate the minimum required sample size for each of the three 2 tests, at =0.05 with power =0.80. The relevant data and results from this problem are presented in Table 2. The methodology of the a priori Power Analysis approach can be used to calculate the sample size for each pair of variables. The last column in Table 2 shows that with a sample size n=1503, the requirements of all three 2 tests are fulfilled. Due to the fact that three statistical 2 tests have to be carried out, the significance level can be adjusted according to 15 Bonferroni, so that the Cumulative Error Rate Type (see Huck, 2000) is not greater than 0.05 0.0167 and 0.05. In this case, the significance level for each test could be predefined at 3 then the relevant calculations can follow. Table 2: Sample Size For Each Pair Of Association 13 Correlation Pairs (X, Y) s 2 Clinically Significant ES: Dynamic Inertia ID 0.005 Sample Size n for =0.05 and =0.80 1503 (X, Z) 2 0.045 152 (Y, Z) 3 0.085 68 CONCLUSIONS In the case of two categorical variables, where the data has been collected using the simple random sampling method, it is possible to proceed with a combination of Correspondence Analysis and the statistical test of independence 2. In the present paper, we have introduced the concept of dynamic inertia and recommended a methodology that can be used to estimate both the observed power of the statistical test 2 (post hoc) as well as the minimum required sample size (a priori) in a sampling or experimental research, on whose data CA will be applied. In order to develop the proposed methodology, we have considered the total and dynamic inertia of a contingency table with two variables as alternative effect size indexes and used as a basis the Power Analysis framework of 2 statistical tests as proposed by Cohen. Finally, the following risk must also be pointed out: an experienced statistical analyst, who intentionally makes wrongful use of Statistics, can design a research in such a way so as to appropriately balance the Errors Type and Type , and after an appropriate selection of , , effect size and sample size, can direct the related conclusions towards certain “desirable” results. 14 REFERENCES Andersen, E. (1991). The Statistical Analysis of Categorical Data. BerlinHeidelberg: Springer-Verlag. Bakan, D. (1966). The test of significance in psychological research. Psychological Bulletin, 66, 423-437. Benzécri, J.-P. (1992). Correspondence Analysis Handbook. New York: Marcel Dekker, Inc. Blasius, J. & Greenacre, M. (1994). Computation of Correspondence Analysis. In: M. Greenacre and J. Blasius (eds), Correspondence Analysis in the Social Sciences. Recent Developments and Applications. London: Academic Press. Blasius, J. (1994). Correspondence Analysis in Social Science Research. In: M. Greenacre and J. Blasius (eds), Correspondence Analysis in the Social Sciences. Recent Developments and Applications. London: Academic Press. Brown, S. & Melamed, L. (1990). Experimental Design and Analysis. Newbury Park, CA: Sage. 16 Bryman, A. & Cramer, D. (1999). Quantitative Data Analysis with SPSS Release 8 for Windows: A Guide for Social Scientists. London and New York: Routledge. Buhl-Mortensen, L. (1996). Type-II Statistical Errors in Environmental Science and the Precautionary Principle. Marine Pollution Bulletin, Vol. 32, No. 7, 528-531. Carver, P. (1978). The case against statistical testing. Harvard Educational Review, 48, 378-399. Chapman, D. & Nam, J. (1968). Asymptotic Power of Chi-Square Tests for Linear Trends in Proportions. Biometrics, Vol. 24, No. 2, 315-327. Chatfield, C. (1991). Avoiding statistical pitfalls. Statistical Science, 6, 240-268. Clausen, S.-E. (1998). Applied Correspondence Analysis: An Introduction. Thousand Oakes, CA: Sage. Cochran, W. (1952). The chi2 Test of Goodness of Fit. The Annals of Mathematical Statistics, Vol. 23, No. 3, 315-345. Cohen, J. & Cohen, P. (1983). Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. New Jersey: Lawrence Erlbaum Associates, Inc. Cohen, J. (1962). The Statistical Power of Abnormal-Social Psychological Research: A Review. Journal of Abnormal and Social Psychology, 65, 145153. Cohen, J. (1965). Some Statistical Issues in Psychological Research. In: B. Wolman (ed.), Handbook of Clinical Psychology. New York: McGraw-Hill. Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. New Jersey: Lawrence Erlbaum Associates, Inc. Cox, R. (1977). The role of significance tests. Scandanavian Journal of Statistics, 4, 49-70. Di Stefano, J. (2001). Power analysis and sustainable forest management. Forest Ecology and Management, 154, 141-153. Dometrius, N. (1992). Social Statistics Using SPSS. New York: HarperCollins Publishers, Inc. Evans, T. & Viengkham, O. (2001). Inventory time-cost and statistical power: a case study of a Lao rattan. Forest Ecology and Management, 150, 313-322. Foster, J. (2001). Statistical power in forest monitoring. Forest Ecology and Management, 151, 211-222. Gatti, G. & Harwell, M. (1998). Advantages of Computer Programs Over Power Charts for the Estimation of Power. Journal of Statistics Education, Vol. 6, No. 3. Available at: http://www.amstat.org/publications/jse/v6n3/gatti.html Gifi, A. (1996). NonLinearMultivariate Analysis. Chichester: John Willey & Sons Ltd. Girden, E. (1992). ANOVA Repeated Measures. Newbury Park, CA: Sage. Greenacre, M. (1993). Correspondence Analysis in Practice. London: Academic Press. Guenther, W. (1964). Another Derivation of the Non-Central Chi-Square Distribution. Journal of the American Statistical Association, Vol. 59, No. 307, 957-960. Guenther, W. (1977). Power and Sample Size for Approximate Chi-Square Tests. The American Statistician, Vol. 31, No. 2, 83-85. Guttman, L. (1985). The illogic of statistical inference for cumulative science. Applied Stochastic Models and Data Analysis, 1, 3-10. 17 Hair, J., Anderson, R., Tatham, R. & Black, W. (1995). Multivariate Data Analysis With Readings. New Jersey: Prentice-Hall International, Inc. Hallahan, M. & Rosenthal, R. (1996). Statistical Power: Concepts, Procedures, and Applications. Behav. Res. Ther., Vol. 34, No. 5/6, 489-499. Han, C-P. (1975). Some relationships Betwwen Noncentral Chi-Squared and Normal Distributions. Biometrika, Vol. 62, No. 1, 213-214. Haynam, G.E., Govindarajulu, Z. & Leone, F.C. (1970). Tables of the Cumulative Non-central Chi-Square Distribution. In: H. L. Harter and D. B. Owen (eds), Selected Tables in Mathematical Statistics, Vol. I. Chicago: Markham Publishing Co. Heidelbaugh, S. & Nelson, W. (1996). A power analysis of methods for assessment of change in seagrass cover. Aquatic Botany, 53, 227-233. Hinkle, D., Wiersma, W. & Jurs, S. (1988). Applied Statistics for the Behavioral Sciences. Boston: Houghton Mifflin Company. Hopkins, W. (1997). A New View of Statistics. Available at: http://www.sportsci.org/resource/stats/index.html. Hubbard, R. & Armstrong, S. (1992). Are Null Results Becoming an Endangered Species in Marketing? Marketing Letters (3:2), 127-136. Huck, S. (2000). Reading Statistics and Research. New York: Addison Wesley Longman, Inc. Israëls, A. (1987). Eigenvalue techniques for Qualitative Data. Leiden: DSWO Press. Kachigan, S. (1991). Multivariate Statistical Analysis: A Conceptual Introduction. NY: Radius Press. Kargopoulos P. & Raftopoulos T. (1998). The Science of Logic & The Art of Thinking. Thessaloniki: Vanias Publishing House. Kinnear, P. & Gray, C. (1999). SPSS for Windows Made Simple. East Sussex: Psychology Press Ltd. Kirk, R. (1995). Experimental Design: Procedures for the Behavioral Sciences. Pasific Grove, CA: Brooks/Cole Publishing Company, ITP. Kish, L. (1959). Some statistical problems in research design. American Sociological Review, 24, 328-338. Kramer, S., and Rosental, R., (1999): Effect Sizes and Significance Levels in Small-Sample Research. In: R. Hoyle (ed.), Statistical Strategies for Small Sample Research. Thousand Oakes: Sage Publications, Inc. Kritzer, B. (1996). Surviving Statistical Spitting Matches. A Professional Development Seminar presentation for Senior Staff of the National Conference of State Legislatures, Madison, Wisconsin, October 10, 1996. Available at: http://www.polisci.wisc.edu/~kritzer/misc/legstaff/legstaff.htm Lachin, J. (1977). Sample Size Determinations for rxc Comparative Trials. Biometrics, Vol. 33, No. 2, 315-324. Lancaster, H. O. (1969). The Chi-Squared Distribution. John Willey & Sons, Inc. Lebart, L., Morineau, A. & Piron, M. (2000). Statistique Exploratoire Multidimensionnelle. Paris: Dunod. Lebart, L., Morineau, A. & Tabard, N. (1977). Techniques de la Description Statistique: méthodes et logiciels pour l’analyse des grands tableaux. Paris: Dunod. Loftus, R. (1991). On the tyranny of hypothesis testing in the social sciences. Contemporary Psychology, 36, 102-105. 18 Lohninger, H. (1999). Teach Me Data Analysis: Single User Edition, [Computer program manual]. New York: Springer. Malinvaud, E. (1987). Data Analysis in applied socio-economic statistics with special consideration of correspondence analysis. Marketing Science Conference Proceedings. HEC-ISA, Joy en Josas. Menexes, G. & Oikonomoy, A. (2002). Errors and Misconception in Statistical Hypothesis Testing. Workbooks of Data Analysis, 2, 52-64. (in Greek) Menexes, G. & Papadimitriou, I. (2004). Relations of inertia in simple contingency, generalized contingency (Burt) and indicator matrices for two or more variables. Workbooks of Data Analysis, 4, 42-69. (in Greek) Meng, R. & Chapman, D. (1966). The Power of Chi Square Tests for Contingency Tables. Journal of the American Statistical Association, Vol. 61, No. 316, 965-975. Meyer, T. & Mark, M. (1996). Statistical Power and Implications of Meta-Analysis for Clinical Research in Psychosocial Oncology. Journal of Psychosomatic Research, Vol. 41, No. 5, 409-413. Michailidis, G. & De Leeuw, J. (1998). The Gifi System of Descriptive Multivariate Analysis. Statistical Science, Vol. 13, No. 4, 307-336. Micheloud, F.-X. (1997). Jean Paul Benzécri’s Correspondence Analysis. Available at: http://www.micheloud.com/FXM/COR/E/index.htm Miller, J., Daly, J., Wood, M., Roper, M. & Brooks, A. (1997). Statistical power and its subcomponents-missing and misunderstood concepts in empirical software engineering research. Information and Software Technology, 39, 285-295. Morrison, E. & Henkel, E. (1970). Significance tests in behavioral research: Skeptical conclusions and beyond. In: D. E. Morrison and R. E. Henkel (ed.), The Significance Test Controversy---A Reader. Chicago: Aldine. Muller, K., LaVange, L., Landersman-Ramey, S. & Ramey, C. (1992). Power Calculations for General Linear Multivariate Models Including Repeated Measures Applications. Journal of the American Statistical Association, Vol. 87, No. 420, 1209-1226. Murphy, K. & Myors, B. (1998). Statistical Power Analysis: A Simple and General Model for Traditional and Modern Hypothesis Tests. New Jersey: Lawrence Erlbaum Associates, Inc. Nathan, G. (1972). On the Asymptotic Power of Tests for Independence in Contingency Tables from Stratified Samples. Journal of the American Statistical Association, Vol. 67, No. 340, 917-920. Nishisato, S. (1980). Analysis of Categorical Data: Dual Scaling and its Applications. Toronto: University of Toronto Press. Nutahara, H. et al. (2001). A simple computerized program for the calculation of the required sample size necessary to ensure statistical accuracy in medical experiments. Computer Methods and Programs in Biomedicine, 65, 133139. Oakes, M. (1986). Statistical Inference: A Commentary for the Social and Behavioral Sciences. Chichester: John Wiley & Sons, Inc. . : Pagano, M. & Gauvreau, K. (2000). . Parkhurst, F. (1985). Interpreting failure to reject a null hypothesis. Bulletin of the Ecological Society of America, 66, 301-302. 19 Patnaik, P. (1949). The Non-Central chi2 and F Distribution and their Applications. Biometrika, Vol. 36, No. 1/2, 202-232. Pearson, E.S. & Hartley, H.O. (eds) (1972). Biometrika Tables for Statisticians 2. London: Cambridge University Press. Pratt, W. (1976). A discussion of the question: for what use are tests of hypotheses and tests of significance. Communications in Statistics, Series A5, 779-787. Rao, R. (1995). The Use of Hellinger Distance in graphical Displays. In: E.-M. Tiit, T. Kollo and H. Niemi (eds), New Trends in Probability and Statistics Vol. 3 Multivariate Statistics and Matrices in Statistics, Zeist: VSP BV and Vilnius: TEV Ltd., (VSP/TEV). Reynolds, H.T. (1984). Analysis of nominal data. Thousand Oaks, CA: Sage Publications, Inc. Rozeboom, W. (1960). The fallacy of the null-hypothesis significance test. Psychological Bulletin, 57, 416-428. Sankaran, M. (1963). Approximations to the Non-Central Chi-Square Distribution. Biometrika, Vol. 50, No. 1/2, 199-204. Saporta, G. & Tambrea, N. (1993). About the Selection of the Number of Components in Correspondence Analysis. In: J. Janssen and C. Skiadas (eds), Applied Stochastic Models and Data Analysis. Singapore: World Scientific. Schmidt, L. (1996). Statistical significance testing and cumulative knowledge in psychology: implications for training of researchers. Psychological Methods, 1(2), 115-129. Sheppard, C. (1999). How Large should my Sample be? Some Quick Guides to Sample Size and the Power of Tests. Marine Pollution Bulletin, Vol. 38, No. 6, 439-447. Thomas, L. & Juanes, F. (1996). The importance of statistical power analysis: an example from Animal Behaviour. Anim. Behav., 52, 856-859. Thomas, L. & Krebs, C. (1997). A Review of Statistical Power Analysis Software. Bulletin of the Ecological Society of America, Vol. 78(2). Available at: http://sustain.forestry.ubc.ca/cacb/power/review/powrev.html Van de Geer, J. (1993). Multivariate Analysis of Categorical Data: Applications. Thousand Oaks, CA: Sage Publications, Inc. Verma, R. & Goodale, J. (1995). Statistical power in operations management research. Journal of Operations Management, 13, 139-152. Weller, S. & Romney A.K. (1990). Metric Scaling: Correspondence Analysis. Newbury Park, CA: Sage. Yates, F. (1951). The influence of Statistical Methods for Research Workers on the development of the science of statistics. Journal of the American Statistical Association, 46, 19-34. Yoccuz, G. (1991). Use, overuse, and misuse of significance tests in evolutionary biology and ecology. Bulletin of the Ecological Society of America, 72, 106111. Zar, J. (1996): Biostatistical Analysis. New jersey: Prentice-Hall International, Inc. 20
© Copyright 2024