1. Define what you want to estimate or compare
Describe the intended result in plain language: a distribution, a comparison of means, an association, or an outcome examined alongside other variables. Identify the observation unit and how measurements were obtained. One participant measured twice is not two independent participants.
Distinguish the primary question from exploratory analyses. If the study involves multiple comparisons, plan how they will be addressed and interpreted before searching through outputs for desirable results.
2. Review measurement and data before naming a test
Create a data dictionary recording each variable, coding, units, possible values, and missing-data codes. Review impossible values, duplication, and all changes made to the dataset. Preserve an untouched source file and document the analytical version.
A numeric column may represent a category, rank, or quantity. A single Likert item is not automatically equivalent to a composite score. Justify score construction and use in relation to the instrument, measurement evidence, and study design.
3. Initial options to discuss
The table offers starting points, followed by review of assumptions and design details. UCLA resources connect the analysis family to the question and variables. NIST explains paired and independent samples, and equal- and unequal-variance approaches.
| Analytical aim | Possible starting point | Review before use |
|---|---|---|
| Compare a quantitative mean in two independent groups | Independent t-test, including Welch’s approach | Independence, distribution, outliers, and variances |
| Compare two quantitative measurements on the same units | Paired t-test | Correct pairing and the distribution of differences |
| Examine an association between variables | Pearson or Spearman, as appropriate | Relationship shape, measurement, ranks, and outliers |
| Examine two categorical variables | Chi-square or an appropriate exact test | Cell frequencies and independent observations |
4. An illustrative before-and-after example
Suppose the same teachers complete a measure before and after a training programme. Pair each teacher’s responses using an analytical identifier and examine within-person differences. If the two groups contain different participants, the comparison has a different structure. Column names such as “before” and “after” do not resolve that distinction.
If some participants lack a second measurement, document the number and available reasons. Decide how missingness will be addressed. Software defaults should not turn deletion and recoding into undocumented methodological decisions.
5. Interpret findings in relation to the question
Bring together descriptive information, the estimated size of a difference or association, uncertainty, and relevant assumptions. A discussion should communicate more than whether a result is labelled significant. Do not change the question after viewing the output and present an exploratory finding as a planned test.
Clustering within schools or teams, survey weights, and repeated observations can require an analysis that accounts for those structures. An easily accessible software command is not a replacement for understanding how the data were generated.
6. Prepare for an analytical consultation
These materials support a discussion of the analysis plan and limits of inference. Any additional processing can be scoped after the project is reviewed.
- The current research questions or hypotheses.
- The instrument and its scoring procedure.
- An organised file without unnecessary personal identifiers.
- A variable dictionary and missing-data codes.
- A description of sampling, measurements, and dependencies.
- Any existing output and the question it has not answered clearly.
Sources & further reading
Would you like to discuss your study?
Explore the support related to this guide, or share a short description of your needs to discuss the next step.