Statistics at the Masters' Level in Mumbai University : A Sample Solution to a Typical Problem of Parametric Significance Testing

With this post, I put forth a solution to a prototypical question in the Statistics paper for Psychology Masters' students of Mumbai University. My aim is to not merely provide a solution to such problems but to motivate students towards solving the problem meaningfully. 

*Please note that this post is not exhaustive in terms of an explanation of concepts relevant to the problem solved in it. This is because statistical theory continuously builds from simple to complex concepts and making an all-inclusive post is virtually impossible.

---

Here's the question -

"A researcher was interested in studying difference between two groups of 20 individuals each, one to which she provided training in anxiety reduction and one to which she did not. These groups had been matched on their anxiety before training began. K-S tests and Levene's tests showed insignificant results. The mean for the trained group was found to be 25.6 with a variance of 25 and the mean for the untrained group was found to be 24.3 with a standard deviation of 4.7. The data from the groups shared 19.5% variance. Perform appropriate statistics and interpret the data."

---

Let's begin with the solution, taking one meaningful step at a time-

1. As the first step, let's read the question carefully and understand the problem that the researcher is seeking to address through the use of statistics - 

In the first sentence, we read that the researcher is seeking to compare two groups on a certain type of training. From our knowledge of basic statistics, we should know at this point that a comparison of two groups statistically involves the comparison of the mean scores of the two groups, since the mean is taken as a fair representation of a dataset under most conditions. As students of psychological research, we must immediately rephrase this in our minds as, "this problem involves a comparison of the means of two groups."

Again, in the same sentence, we read that the researcher is seeking to compare these distinct groups based on some training they did or did not receive. Again, as students of research, this must translate in our minds as, "this problem involves a single independent variable - 'provision of training', manipulated at two levels - 'provision of training' and 'no provision of training.'"

By the time of attempting such a question, from the syllabus we have already covered to this point, we should know that there are two statistical tests most suited to analysing data involving one independent variable with two levels - the suitable parametric alternative is the t-test whereas the suitable non-parametric alternative is the Mann-Whitney 'U' test. Keeping this consideration in mind, we can proceed with reading the problem.

The next sentence informs us about a baseline comparison between the two groups on which they were found to be matched with respect to their levels of anxiety. When two groups are matched with respect to a variable critical to a research investigation - the variable being anxiety in this case, from a statistical point of view, they can be treated as if they are one and the same group undergoing different conditions of the investigation. This is because though they are physically distinct, their standing with respect to the psychological attribute in question is identical. The important takeaway from this sentence of the problem is that we need to use that statistical method for analysis which involves a comparison of dependent samples.

Coming to the next sentence now, we are informed that one, a K-S test and two, a Levene's test performed on the data, both turned out to be insignificant. The K-S test is a test that detects normality in a dataset. It's insignificance indicates that the dataset is normally distributed. As far as the information regarding the Levene's test performed is concerned, it does not bear upon our selection of a statistical test involving dependent samples. Homogeneity of variances is a not an assumption of the dependent samples t-test since this test compares a single sample or two identical samples drawn from the same population.

At this point, we have all the information we need to decide which statistical test to use. The data involves one independent variable with two levels, is normally distributed and has been drawn from dependent samples. The test of choice here is certainly, the t-test for dependent samples.

Now that we know exactly how we should be analyzing the data, we ought to take note of the information provided to us that will further our purpose. We know that for being able to perform the t-test for dependent samples, we need -

• The means of the two groups for comparison;

• The standard error of the two groups for location of the difference of their means on the t-distribution;

• The correlation co-efficient of the association between the groups in order to account for the dependence between the groups and deriving an uninflated estimate of common standard error for transformation of the difference between means to a t-score.

Reading the next two sentences of the problem statement, we find all this information in different forms. How the information needed is to be extracted from the given statements is what we will deal with in a following section discussing computations for this problem.

Though the above analysis of the question might seem time-consuming, it is a really quick mental process once understood and practiced, something that appears lengthy only in writing - certainly nothing to worry about.

With this in mind, let's move on to the next step-

2. Let's review theory relevant to our problem at this point to build motivation to solve the problem - 

An important point that I would like to draw attention to before beginning with this section is that students of statistics ought to solve problems meaningfully. At this point, you are aspiring to make use of statistics in your career in psychology, so do take note that statisticians are not the equivalents of data-analysing software. Students working on statistical problems with this misconception, with the sole aim of applying formulas on numbers presented and coming up with answers are bound to wear out themselves. Enthusiasm in solving a problem can be developed only by making the problem relevant to one's field of study. 

Let's start this section accordingly, making a step-wise review of relevant theory of the dependent samples t-test - 

• Psychologists often seek to make comparisons between two conditions. Frequently, they seek to compare the effectiveness of a certain treatment or therapy that they have administered to individuals - by comparing those who have received the treatment with those who have not.

• In the present problem, the researcher seeks to compare the anxiety levels of two groups - one that has undergone certain treatment and one that has not;

• Accordingly, they conduct research on two sample groups who are or are not administered the treatment of interest. They then obtain datasets of scores from each of the groups they compare;

• The present problem describes research conducted on two sample groups comprising of 20 participants each -  one that received training to reduce anxiety and one that did not. A summary of descriptive statistics from the data obtained through this research is presented in the problem statement;

• They most often choose to compare the means of the data obtained from the two sample groups with respect to the effectiveness of their treatment since the mean is that representative score of the performance of the entire group, that gives weightage to every score in the dataset while computing its most central representative score;

• In the present problem, the means of both groups on their anxiety levels have been measured and presented;

• The representativeness of the mean, however, is limited to the center of a dataset. It does not represent the distribution of scores in the set. This implies that if two datasets do not follow the same distribution or spread of scores, then comparing their means directly will lead to erroneous conclusions. This, therefore, necessitates consideration of the spread of scores of datasets being compared. The most reliable measure of this spread is the standard deviation  of a dataset;

• In the present problem, the standard deviations of the groups being compared have been presented. These standard deviations differ from each other, suggesting that the datasets are not identically distributed. Though the samples originally came from the same population, sharing the same variances; the treatment administered to them has caused a change in their distribution. Directly comparing the mean anxiety levels of the two groups, therefore, would neither give an accurate nor a complete understanding of the difference in anxiety levels of the two groups. *Think about it - If you know that two groups of individuals have the same mean anxiety levels but you also know that in one group, most individuals have anxiety levels similar to the mean while in the other group, most individuals have anxiety levels very different from the mean, would you say that the anxiety levels of the two groups are similar?

• The method which enables comparison of means of two datasets while giving due weightage to the spread of their distributions is the transformation of raw mean scores to standard mean scores;

• In the present problem, if the data of anxiety levels of both groups can be scaled onto a common distribution, that is, a distribution with an identical spread of scores, then their means can be compared directly;

• The standard scores to which raw scores can be transformed is contingent on considerations of the spread of the scores, the scale of measurement used to obtain data and the size of datasets. When two datasets are normally distributed, when they have been measured using an interval scale and when they are small in size, their scores are best transformed to t scores which follow the Student's t distribution;

• In the present problem, the data presented are normally distributed as evidenced by the insignificantly result of the K-S test performed on the data. The data have been measured on an interval scale which is evidenced by the computation of means and standard deviations for the datasets. Lastly, the datasets are small - small being defined here by the rule of thumb that any sample size less than 30 is a small sample size - the present datasets have been based on 20 participants each. When datasets of this nature are standardized, that is, when the scores of such datasets are expressed in terms of their standard deviations or distances from the means, they are found to follow a t-distribution. A t-distribution is known to have a normal symmetric distribution of scores with heavy tails since it is based on small samples, which tend to have more extreme scores than large samples.

• The Student's t distribution serves as a standardized sampling distribution of sample means for it not only enables direct comparison of the means of two normally distributed, small datasets but also provides theoretical probabilities for these differences. These probabilities help researchers take a decision about the significance of difference between the means they are comparing. The probabilities associated with raw scores are unknown and hence, comparing raw scores directly does not lend to statistical significance testing;

• In the present problem, it is possible for the researcher to calculate the difference between the given means and then transform this difference to a t-score. Since the individual distributions follow a t-distribution, differences in their scores also follow the same. This t-score can then be used to obtain the probability associated with it by reference of a relevant t-table or a statistical calculator/software. Students ought to have a conceptual knowledge of the Student's t standard distribution and its probability function at this point. I am unable to revise it here due to the extensiveness of discussion required for it.

• When the Student's t-test is conducted on dependent samples - those that are same or identical with respect to the variable being measured at baseline, the conversion of the raw difference between means to a standardized t difference requires that due consideration be given to the association between data obtained from the conditions run. This association is accounted for in the estimation of the standard error used for transformation of the raw differences into a standard t-score;

• In the present problem, anxiety levels are compared between pairs of individuals who have had matching scores on baseline anxiety. By this virtue, every pair of individuals, that is, statistically a single individual - has a peculiar response to the anxiety training given to him because he is that individual, because he carries a peculiar baseline level of anxiety with him to begin with. This peculiarity of his affects his scores of both - the without training and after training anxiety obtained from him. This influence of his, which affects both conditions run on him is statistically accounted for by computing a correlation co-efficient that can be deducted from the estimated standard error during transformation of the raw difference between means to a standard difference. *Link this point to your knowledge of the independent samples t-test. The assumption of independence of data in that test controls for such undue individual differences.

Let's also make a really quick review of statistical hypothesis testing in statistical inference before we delve into the calculations required for the present problem-

• Researchers first generate research questions regarding phenomena of their interest, the inspiration for which comes from their experiences, observations, knowledge and/or other sources;

• They then review existing literature to investigate what is already known about their phenomena of interest;

• In accordance with their review, they set up hypotheses which are testable, tentative explanations for their phenomena of interest;

• They then undertake research investigation - in the form of observation or experimentation - to gather evidence that either supports or refutes their hypotheses;

• From this investigation, they gather data that serves as an empirical basis for them to take a decision on the validity of their hypotheses;

• They analyse this data using statistical procedures that enable them to determine whether or not their hypotheses hold true for the population and if so, to what extent they do;

• The critical part of this analysis is what is called significance testing- testing for occurrence of data trends as real effects or merely outcomes of chance;

• Such significance testing involves transformation of raw data into standardized data for which theoretical probabilities have been calculated and made available to researchers. These probabilities help researchers decide whether the trends seen in their data are due to sampling fluctuations or due to impact of their manipulations - that is, whether the trends are insignificant or significant; and whether they hold true for the population or not;

• Significance testing concludes the hypothesis testing procedure for a given instance of research in its investigation and the results obtained from it enable researchers in making informed decisions regarding their future course of investigation, namely, to proceed with investing their time and effort in pursuing their research question or to abandon it.

I reiterate here that since conduct of the Student's t-test is a part of a much larger process of what is called hypothesis testing or significance testing, students at the Masters' level must deal with problems presented to them keeping this holistic approach in mind and not break down problem-solving to a mindless formula-matching-and-application process.

Having made a review of relevant concepts at this point, let us proceed towards solving the problem given systematically 

(*Please note, I am not sharing the mathematical formulae and computations required for these problems for the sake of maintaining precision in the post)- 

3. Let's start answering this question step-wise, in line with a hypothesis-testing procedure:

For the given problem, we set the following hypotheses from a point of view making an inference from our data for the population -

Null hypothesis : "The mean anxiety level after receiving training is the same as the mean anxiety level upon receiving no training."

That is, μtraining = μno_training

Alternative hypothesis : "The mean anxiety level after receiving training is not the same as the mean anxiety level upon receiving no training."

That is, μtraining ≠ μno_training

We then transform the given descriptive data to forms that are needed to solve the problem.

The given data are:

Mean of Group 1, X̄training = 25.6;
Mean of Group 2, X̄no_training = 24.3;
Variance of Group 1, S2training = 25;
Standard Deviation of Group 2, Sno_training = 4.7;
Shared Percentage Variance between the two groups = 19.5;
Sample Size, n = 20


Using this given data, we then calculate statistics we need to be able to compute our t-score:

Standard Deviation of Group 1, Straining = 5;
Correlation between the two groups, r = 0.442
Standard Error of Group 1, SEtraining = 1.119;

Standard Error of Group 2, SEtraining = 1.051;

Now that we have all the data we need, we compute the t-score needed, check for its significance and note the results obtained as follows:

Degrees of freedom, df = 19;
t(19) = 1.327;
tabled values (two-tailed)
for α 0.05 = 2.093; α 0.01 = 2.861
d = 0.253

Having all the inferential data of our interest now, we draw the following conclusion about the veracity of the hypotheses stated:

Since the computed t-value is lesser than the critical values of t, we accept the null hypothesis which states, "The mean anxiety level after receiving training is the same as the mean anxiety level upon receiving no training," t(19) = 1.327, n.s.; d = 0.253.

At this point, significance testing ends with the conclusion that the increase in mean anxiety level post training of the participants is very likely to have been a result of sampling characteristics and not actual effectiveness of the training. The small effect size obtained from the data adds support to this conclusion. After drawing this conclusion, a researcher decides to either discard his line of investigation or to make some modifications to his intervention and re-administer is. What a researcher does further is based upon his theoretical beliefs and practical resources. For us, our work for the given statistical problem ends here.

----X----

I hope this post has successfully guided you through one instance of hypothesis testing from your syllabus. If you are looking for help for such problems, you can contact me on my number for home tuition for Psychology in Mumbai - 9892507784 or drop me a mail at my address - jyotikapsychology@gmail.com.

(Keywords - Statistics for Mumbai University MA Psychology, MA Psychology Statistics, Masters' Psychology Statistics, t-test, Mumbai University t-test for Psychology, Psychological Statistics at Mumbai University)