Introduction

The Student’s t-test is a vital statistical method used in inferential statistics. It assesses whether there is a significant difference between the means of two groups based on samples drawn from those groups. Essentially, the t-test determines if the populations from which these samples originate differ, considering that the samples measure a continuous quantitative variable.

The underlying principle of this statistical test is that if the two samples exhibit significant differences, it is reasonable to conclude that their respective populations are also distinct. Conversely, if the samples appear similar, we cannot dismiss the hypothesis that the populations are alike, indicating insufficient evidence to claim a difference.

Typically, a measure of central tendency, such as the mean, is used to compare two samples in a t-test. However, in some cases, the median is more appropriate, which is addressed through the Wilcoxon test. As this article is comprehensive, the Wilcoxon test is discussed separately, with examples illustrating when to use each test.

Both the Student’s t-test and the Wilcoxon test aim to compare samples to determine population differences. While the Student’s t-test is generally more powerful—capable of detecting smaller differences—it is sensitive to outliers and non-normal data distributions. Each test has various versions, necessitating an understanding of their distinctions and applications based on specific research questions and data characteristics.

This article will provide a detailed, step-by-step guide on performing the Student’s t-test for both independent and paired samples, using a simplified dataset for clarity. Following the manual calculations, we will replicate these tests using R to confirm our findings. Additionally, we will review the rationale behind hypothesis testing, interpret the p-value, and address the assumptions linked to this test.

Though the primary goal is to demonstrate how to compute the Student’s t-test by hand and in R, we will mention the assumptions and suggest alternatives if any are unmet. Interested readers can refer to the end of the article for further details on these aspects.

Null and Alternative Hypothesis

Before delving into the calculations, let’s clarify the null and alternative hypotheses:

• H0: ?1 = ?2 (the means of the two populations are equal)
• H1: ?1 ? ?2 (the means of the two populations are not equal)

Here, ?1 and ?2 represent the means of the two populations associated with the samples. While the t-test fundamentally compares the means of the samples, its primary aim is to evaluate the following hypotheses:

• H0: the populations are similar
• H1: the populations are different

This scenario applies when the goal is merely to ascertain whether the two populations differ concerning the dependent variable, without prior assumptions about the populations' sizes.

If there is a belief that one population is larger or smaller than the other, the t-test can also address:

• H0: ?1 = ?2
• H1: ?1 > ?2 (testing if the first population is larger)

or

• H0: ?1 = ?2
• H1: ?1 < ?2 (testing if the first population is smaller)

One-sided tests, while possible, are often discouraged in practice. If a researcher is firmly convinced of one population's superiority, they may question the need for significance testing. This philosophical debate is beyond this article's scope, but interested readers can refer to Rowntree (2000) for insights.

Hypothesis Testing

Hypothesis tests are prevalent in statistics, serving to determine if a belief is plausible given the available data. Most hypothesis tests can be summarized in four steps:

1. State the null and alternative hypotheses.
2. Calculate the test statistic, denoted as t-stat. Formulas vary based on the specific version of the Student’s t-test.
3. Identify the critical value based on the theoretical statistical distribution, its parameters, and the significance level (?). For the t-test, this may involve the normal or Student's t distribution.
4. Conclude by comparing the t-stat with the critical value. If the t-stat falls within the rejection region, the null hypothesis is rejected; otherwise, it is not rejected. Note that we never "accept" a hypothesis; we simply fail to reject it.

Different Versions of the Student’s t-test

The Student’s t-test comes in various forms, depending on whether the samples are independent or paired and whether the population variances are known or unknown:

Independent Samples: These are samples collected from distinct experimental units or individuals. For example, analyzing data from men and women separately or from patients in different treatment groups.

Paired Samples: These involve measurements from the same experimental units or individuals. For instance, in medical studies assessing treatment efficacy over time, where the same patients are measured before and after treatment. The dependency between the two samples must be factored into the test statistic calculations by focusing on the differences in measurements for each subject.

Choosing the right version of the t-test also depends on whether the population variances are known, unknown, equal, or unequal. In practice, variances are typically unknown, and the decision hinges on checking for equality.

How to Compute Student’s t-test by Hand?

The data used in this article are hypothetical and not representative of real-world variables. We assume all assumptions are met for illustrative purposes, and the significance level is set at ? = 5%.

For those preferring to apply these tests without manual calculations, a Shiny app is available that simplifies the process. Users can input their data and select the appropriate test version from the sidebar menu, complete with visual aids.

Scenario 1: Independent Samples with 2 Known Variances

For this scenario, consider the data below. Assume the two samples are independent, with both populations having a variance of 1, and we want to test whether the populations differ.

• 5 observations in each sample: n1 = n2 = 5
• Mean of sample 1 = 0.02
• Mean of sample 2 = 0.06
• Variances of both populations = 1

Following the four steps of hypothesis testing:

1. H0: ?1 = ?2 and H1: ?1 ? ?2
2. Test statistic:
3. Critical value: ±z(?/2) = ±z(0.025) = ±1.96
4. Conclusion: The rejection regions are from -? to -1.96 and from 1.96 to +?. The test statistic falls outside these regions, leading us to not reject the null hypothesis. At the 5% significance level, we conclude that there is no sufficient evidence to assert that the two populations are different.

Scenario 2: Independent Samples with 2 Equal but Unknown Variances

For the second scenario, consider the following data. The samples are independent, the variances are unknown but equal, and we wish to test if population 1 is larger than population 2.

• Sample 1: n1 = 6
• Sample 2: n2 = 5
• Mean of sample 1 = 1.247
• Mean of sample 2 = 0.1
• Variance of sample 1 = 0.303
• Variance of sample 2 = 0.315

Following the four steps:

1. H0: ?1 = ?2 and H1: ?1 > ?2
2. Test statistic:
3. Critical value: t(?, n1+n2-2) = t(0.05, 9) = 1.833
4. Conclusion: The rejection region is from 1.833 to +?. The test statistic lies within this region, leading us to reject the null hypothesis. At the 5% significance level, we conclude that population 1 is larger than population 2.

Scenario 3: Independent Samples with 2 Unequal and Unknown Variances

For the third scenario, consider the following data. The samples are independent, the population variances are unknown and unequal, and we want to test whether population 1 is smaller than population 2.

• Sample 1: n1 = 5
• Sample 2: n2 = 6
• Mean of sample 1 = 0.42
• Mean of sample 2 = 1.247
• Variance of sample 1 = 0.107
• Variance of sample 2 = 0.303

Following the four steps:

1. H0: ?1 = ?2 and H1: ?1 < ?2
2. Test statistic:
3. Critical value: t(?, df) where

The degrees of freedom (8.28) does not exist in the standard Student distribution table; we round down to 8.

4. Conclusion: The rejection region is from -? to -1.851. The test statistic lies in this region, leading us to reject the null hypothesis. At the 5% significance level, we conclude that population 1 is smaller than population 2.

Scenario 4: Paired Samples Where the Variance of the Differences is Known

In this scenario, consider the following data. The samples are dependent (matched), the variance of the differences in the population is known and equal to 1, and we want to test if the difference in the population is different from 0.

We begin by calculating the differences for all pairs of observations:

Following the four steps:

1. H0: ?D = 0 and H1: ?D ? 0
2. Test statistic:
3. Critical value: ±z(?/2) = ±z(0.025) = ±1.96
4. Conclusion: The rejection regions are from -? to -1.96 and from 1.96 to +?. The test statistic lies outside these regions, leading us to not reject the null hypothesis. At the 5% significance level, we conclude that the difference between the two populations is equal to 0.

Scenario 5: Paired Samples Where the Variance of the Differences is Unknown

In the final scenario, consider the data below. The samples are dependent (matched), the variance of the differences is unknown, and we want to test whether the treatment is effective in improving running capabilities (higher values indicate better performance).

We calculate the differences for all pairs of observations:

Following the four steps:

1. H0: ?D = 0 and H1: ?D > 0
2. Test statistic:
3. Critical value: t(?, n-1) = t(0.05, 4) = 2.132
4. Conclusion: The rejection region is from 2.132 to +?. The test statistic lies within this region, leading us to reject the null hypothesis. At the 5% significance level, we conclude that the treatment significantly enhances running capabilities.

This concludes the manual computation of different versions of the Student’s t-test. In the following sections, we will detail how to perform the same tests in R.