T-Test Calculator

What exactly does this calculate?

A one-sample, two-tailed t-test from summary figures. It computes the standard error s ÷ √n, the statistic t = (x̄ − μ) ÷ (s ÷ √n), degrees of freedom n − 1, and the p-value from the Student t distribution with those degrees of freedom. The p-value is exact (computed through the regularised incomplete beta function), not a normal-curve shortcut, so it stays correct for very small samples.

Why does the p-value for a small sample differ from what a z-test would give?

The t distribution has heavier tails than the normal one, especially with few degrees of freedom. For a sample mean of 103 against μ = 100 with s = 6 and n = 16, t = 2.0000 with 15 df and the p-value is 0.0639, so you fail to reject at 0.05. A normal curve would give about 0.0455 for the same statistic and wrongly reject. The gap shrinks as n grows.

What do I enter for the standard deviation, and what are the limits?

The sample standard deviation s, computed with an n − 1 divisor, which is the "sample" option in the variance calculator. It must be above 0. The sample size must be at least 2, since a single observation has no spread, and α must be strictly between 0 and 1. Use whole numbers for n because degrees of freedom are n − 1.

Is this a one-tailed or two-tailed test?

Two-tailed only: the alternative is that the true mean differs from μ in either direction. If you predicted a direction in advance and the sign of t agrees with it, the one-tailed p-value is half the displayed one. Choosing the direction after seeing the data is not valid.

What does the p-value mean, and what does it not mean?

It is the probability of getting a t at least this far from zero if the true mean really were μ and the test assumptions held. It is not the probability that the null hypothesis is true, and it does not measure the size of the difference. "Fail to reject" also does not prove the means are equal; with n = 10, a 2-point gap and s = 8 gives p = 0.4495, which mainly reflects limited data.

Why can a tiny difference still be "significant"?

Because the standard error shrinks with √n. A sample mean of 105 against μ = 100 with s = 10 and n = 5000 gives t = 35.36 and a p-value shown as "< 0.0001"; the same gap with n = 5 would not be significant. Statistical significance is not practical importance, so compare the difference itself with what matters in your field.

What assumptions does the t-test need?

Independent observations from a random sample, and data that are roughly normal when n is small. With larger samples the test tolerates moderate non-normality because the sample mean is close to normal, but strong skew or outliers still distort s and t. The calculator has no way to check any of this from summary numbers.

Which tests are not included?

Two-sample, Welch, and paired tests, proportion tests, effect sizes, power, and confidence intervals. A paired comparison can be run here by entering the mean and sample SD of the pairwise differences with μ = 0. The tool accepts summary statistics only, not raw data.