A questionnaire needs evidence for how its scores should be interpreted and how consistently it measures the intended concept. Those are the two questions behind validity and reliability. To answer them in Chapter 3, describe the checks you actually performed: reviewers who read the items, a pilot test that changed some wording, and, where it fits, an alpha value calculated from real responses. A stock sentence saying "the instrument was validated and found reliable" is not evidence of either.
A named reviewer or alpha value needs a real record behind it. UniDraft helps organise the instrument checks, analysis output and instructions you confirm into an editable project document. Start your questionnaire project report, then verify every method and value against your actual work. It does not validate questionnaires or calculate reliability for you.
The short answer
- Validity asks whether the questionnaire measures the right concept. Reliability asks whether it measures consistently. You need both, and one does not prove the other.
- An objective-to-item map, knowledgeable review and wording checks can contribute evidence; they do not automatically establish validity for every intended interpretation.
- Cronbach’s alpha is one possible internal-consistency estimate for an appropriate item set. It is not a test of consistency across time, and other reliability questions need other methods. A common rule of thumb is 0.70 or above, with values above about 0.90 treated as a warning of repeated items.
- Report only what happened. A review, a pilot or a calculation you did not carry out must not appear in Chapter 3 as if you did.
- Your department's or supervisor's instruction beats any general rule of thumb, including the ones in this guide.
Validity and reliability in plain terms
Think of a bathroom scale. If it always shows 3 kg too heavy, it is reliable (consistent) but not valid (accurate). If it shows a different weight every time you step on it, it is neither. Questionnaires behave the same way: a set of questions can produce tidy, consistent answers to the wrong concept.
Three kinds of validity come up most in student projects. The definitions below follow the scale-development review by Boateng and colleagues (see Sources):
| Type | What it asks | How a student usually shows it |
|---|---|---|
| Face validity | Do respondents judge the items appropriate and clear for the concept? | Pilot with people like your respondents and record their reactions |
| Content validity | Do the items adequately cover the whole domain you want to measure? | Objective-to-item map, plus review by people who know the topic |
| Construct validity | Do the scores relate to other measures the way the concept predicts? | Usually factor analysis or correlation checks; often beyond a first-degree project |
Tsang and colleagues describe face validity as the weakest of the approaches, useful mainly to see whether items look relevant and to encourage honest answers. Do not present it as your only evidence if your department expects more.
Construct validity needs care. Many undergraduate projects do not have the sample size or the time for a full factor analysis, and that is fine, provided you do not claim you established it. If you adopt an established instrument, say so and cite the source (the citation finder helps you check real references). Tell the reader what was already tested by its authors and what you checked yourself in your population.
Step 1: map every item to an objective
Before anyone reviews the form, make a two-column table. In one column put each research objective or research question or construct; in the other, the questionnaire items meant to measure it. Two things usually show up straight away:
- Orphan items that measure nothing in your objectives, such as "How satisfied are you with the campus environment?" in a study of online learning access. Remove them.
- Missing coverage, where an objective has one weak item or none. If Objective 2 is about how students access online classes, you probably need items on devices, data cost, electricity and connectivity, not just "I can access online classes".
This map is a starting record for discussing content coverage, not sufficient proof of validity by itself. Keep it: it goes in your working file, and many departments allow a version in the appendix. See the questionnaire design guide for how to write the items themselves.
Step 2: get expert review, honestly
Ask people who know the topic to read the draft against your objectives. Ask specific questions: does each item belong, is any important area missing, is any wording biased or double-barrelled ("Do lecturers teach well and mark fairly?").
In practice this often means your supervisor plus one or two others in the department, but your department decides who must review and whether a signed validation sheet is needed. Follow its rule.
Keep a short record:
| Record | What to keep |
|---|---|
| Reviewer feedback | Who reviewed it, their role, the date, and the specific comments |
| Revision log | Items added, split, reworded or removed, and why |
| Sign-off | A signed validation sheet only if your department requires one and the person actually signed |
Never write that a named lecturer "validated" the instrument unless they reviewed it. If your supervisor has not looked at it yet, say the review is planned.
Step 3: pilot the questionnaire
A pilot means giving the draft to a small group of people similar to your real respondents before the main survey, so problems show up while you can still fix them. Ask, for each confusing item, what the person thought it meant and why they answered as they did. Look for missing answer options, questions that are hard to answer on a phone, and how long it takes.
How big should the pilot be? There is no single official figure. Boateng and colleagues describe cognitive interviews with roughly 5 to 15 people across two or three rounds until the same problems stop appearing. Tsang and colleagues describe a preliminary test with roughly 30 to 50 respondents to look at response patterns. Both are guidance from the questionnaire-development literature, not a Nigerian regulator's rule, so ask your supervisor what your department accepts, then report what you actually did.
Practical points:
- Use people who are like your target group but will not be in your final sample if your method requires separation. In a study of final-year students in one department, pilot with students from a similar department or level.
- Do not let a pilot of eight friends become the "reliability test". It shows whether the wording works; it does not prove a scale is statistically reliable.
- Write down what changed. "Item 7 was split into two after three pilot respondents said it asked about two things" is far more convincing than "the pilot was successful".
Your sample-size calculation covers the main study; the pilot has a different job.
Step 4: reliability and Cronbach's alpha
Reliability concerns consistency. Cronbach’s alpha is an internal-consistency estimate for an appropriate item set. A high value does not itself prove one dimension or validity. A computed negative value can occur and needs investigation of the covariance structure, coding and item set rather than interpretation as negative reliability.
What the literature says about interpreting it:
- Tavakol and Dennick report that published recommendations for acceptable alpha range from 0.70 to 0.95, and that values above about 0.90 may signal redundant items, so they suggest 0.90 as a maximum.
- Tsang and colleagues give 0.70 as the usual minimum for adequate reliability and also warn about values of 0.90 and above.
- Alpha depends on the number of items, how related they are, and whether the test is unidimensional. Short scales tend to give lower alpha even if the items are good, and a scale mixing several constructs can still yield a high alpha.
- Alpha is a property of the scores from a specific sample, so a value published for another study does not carry over to yours. Calculate it on your own data.
Treat the published markers as context-sensitive guidance, not an automatic pass/fail scale. A higher estimate alone does not establish validity or a single dimension.
Worked example: interpreting alpha (illustrative numbers only)
These numbers are invented to show the arithmetic. They are not real data or a justified reliability-study sample size. Plan the sample for the actual design and precision needed.
Suppose four Likert items (1 to 5) are meant to measure "access to online learning", and eight pilot respondents answered like this:
| Respondent | Q1 | Q2 | Q3 | Q4 | Total |
|---|---|---|---|---|---|
| A | 4 | 3 | 5 | 2 | 14 |
| B | 3 | 4 | 3 | 4 | 14 |
| C | 5 | 4 | 4 | 3 | 16 |
| D | 2 | 3 | 3 | 4 | 12 |
| E | 4 | 5 | 3 | 2 | 14 |
| F | 3 | 2 | 4 | 3 | 12 |
| G | 5 | 4 | 4 | 5 | 18 |
| H | 2 | 3 | 2 | 3 | 10 |
Alpha = (k / (k - 1)) x (1 - sum of item variances / variance of total scores), where k is the number of items.
- Sum of the four item variances = 4.21; variance of the total scores = 6.21.
- Alpha = (4/3) x (1 - 4.21/6.21) = about 0.43.
Reading it: 0.43 is well below the usual 0.70 marker, but that value alone does not diagnose dimensionality or establish why the estimate is low. Do not stop at the number. Look at what happens when each item is left out in turn (an "if item deleted" table, which SPSS and similar programs can produce):
| Item removed | Alpha for the other three |
|---|---|
| Q1 | -0.39 (negative) |
| Q2 | 0.38 |
| Q3 | 0.38 |
| Q4 | 0.63 |
Removing Q4 increases alpha in this fictional sample; removing Q1 produces a negative estimate. Those arithmetic changes do not establish that Q4 is invalid or Q1 is scientifically essential. The next step is a judgement, not an automatic deletion:
- Read Q4. Is it worded in the opposite direction ("I often struggle to log in"), and did you forget to reverse-code it? That is a coding error, not a bad item.
- Does Q4 really measure a different idea, such as cost rather than access? Then it belongs to another construct, or needs its own items.
- If Q4 is simply unclear, revise it and re-pilot; do not just delete it because it improves the number.
Whatever you decide, write down the original value, the change and the reason. A revised alpha of 0.63 is still below 0.70, and you should say so and explain what you did about it. What you must not do is quietly report a better number than the data gave.
A last check: if a four-item scale had produced 0.96, the same literature would ask whether some questions repeat each other. High alpha is not automatically good.
When alpha does not apply
Do not calculate one alpha across the whole questionnaire "to get a number". Alpha needs items that measure the same construct on the same kind of scale.
- Demographic questions (age, level, department) have no internal consistency to measure.
- Single-item questions cannot have an alpha.
- Different constructs should each get their own alpha, for example one for "access" items and another for "attitude" items.
- Free-text or interview questions need checks suited to the chosen qualitative approach and a documented process. Do not automatically impose coder agreement on an approach that uses different assumptions.
- An established instrument may already have published validation. Cite it, then check that it works in your setting rather than assuming it does.
Write Chapter 3 in the right tense
In a proposal, say what you will do: who will review the instrument, whether and how it will be piloted, and which reliability check suits the design. In a completed project, say what happened. Your methodology chapter should include a short subsection on the instrument.
A structure that works (fill each bracket only with something real):
"The draft items were reviewed by [actual reviewer, role]. After feedback on [actual issue], items [numbers] were revised. The instrument was piloted with [number and description of participants] on [dates]. Internal consistency for the [named construct] items was assessed using Cronbach's alpha, giving [actual value] for [number] items."
If a step did not happen, delete that sentence. The alpha values themselves normally go in the methodology or as a small table, and your results chapter then reports the main-study findings.
Common mistakes and how supervisors flag them
| Mistake | How it gets flagged |
|---|---|
| "The instrument was validated by my supervisor" with no detail | "When, and what did they change?" |
| One alpha for the entire questionnaire | "Which construct does this measure?" |
| Alpha reported with no item count or sample | "Based on how many items and which respondents?" |
| Pilot respondents included in the final sample without saying so | "Were pilot answers analysed with the main study?" |
| Alpha copied from a published paper | "Did you calculate it on your own data?" |
| Deleting items until alpha looks good | "What did those items measure?" |
| Confusing "reliable" with "valid" | "Consistent is not the same as correct." |
If corrections come back on this section, keep a copy of the previous draft and record what changed; the guide to managing supervisor corrections covers that habit.
Pre-submission checklist
- Every item has a justified purpose and each measured construct has an appropriate item set.
- Reviewer names, roles and dates are true and you have their feedback.
- The pilot is described as it happened, with changes listed.
- Alpha (if used) is calculated per construct, on real responses, with item counts.
- Values below your target are reported honestly with what you did about them.
- Tense matches the document: plan in the proposal, past tense in the report.
- The thresholds you quote come from a real source you can name; the literature review guide helps if you need to build that support.
Keep a pilot decision log, not just an alpha number
Use this original worksheet to make the pilot useful to the final instrument. The example issue is hypothetical; record only what actually happened in your pilot.
| Record | What to preserve |
|---|---|
| Instrument version | Dated copy before testing and after revision |
| Recruitment | Who was eligible, how they were reached and who participated |
| Item problem | Exact item and what the respondent misunderstood |
| Decision | Revise, split, remove or retain, with a reason |
| Recheck | Whether the revised wording was tried again and what happened |
| Analysis record | Item set, scoring decisions, cases included and actual output |
For example, if a participant says “I cannot answer because I have never used that service”, investigate whether eligibility screening or a genuine not-applicable option is needed. Do not change a response into a neutral answer simply to fill the cell. Keep pilot material separate from the main export where the approved design requires it; the data-cleaning guide explains preserving raw records and decisions.
Before reporting alpha, check that the analysed items belong to the intended construct and that scoring matches the instrument. Preserve the original output and any justified revision history. A higher alpha alone does not establish validity or prove a single dimension, as Tavakol and Dennick explain. Do not delete meaningful content merely to chase a threshold. Report what you changed and why, including problems that remain.
Where UniDraft fits in
UniDraft does not run statistics, and it does not create reviewers, pilot data or alpha values. Those come from your real study. What it does is build your written final-year project as an editable Word (.docx) document around the topic, methodology and real results you confirm, applying your supervisor's instructions first, then your department's and university's format. You can upload your results tables and labelled images, and if your supervisor asks for changes you supply their corrections, confirm the extracted changes and get a new version while earlier versions are kept. Check the current project price and included correction rounds before checkout. The formatting checker is free and looks at fonts, spacing and margins in your .docx.
Sources
- Making sense of Cronbach's alpha (Tavakol and Dennick, 2011, International Journal of Medical Education): alpha ranges of 0.70 to 0.95, the 0.90 redundancy warning, dependence on item number and dimensionality, sample-specific scores.
- Best Practices for Developing and Validating Scales (Boateng et al., 2018, Frontiers in Public Health): definitions of face, content and construct validity; cognitive interviews of about 5 to 15 people.
- Guidelines for developing, translating, and validating a questionnaire (Tsang, Royse and Terkawi, 2017, Saudi Journal of Anaesthesia): pilot testing, expert review, face validity, alpha of 0.70 and above 0.90.