Reliability and Validity: Cronbach's Alpha Explained

Before you test any hypothesis with questionnaire data, your examiners will ask two questions: does your instrument measure consistently, and does it measure what it claims to measure? These are reliability and validity. Cronbach's alpha is the statistic most theses use for reliability, and this article shows you how to calculate it by hand and interpret it properly.

Reliability vs Validity

A simple analogy is a weighing scale. If it shows 61.2 kg, 61.3 kg and 61.2 kg when you step on it three times, it is reliable. If your true weight is 58 kg, it is reliable but not valid. A measure can be reliable without being valid, but an unreliable measure cannot be valid, because random noise swamps whatever it is trying to capture.

Types of Reliability

• Internal consistency: do the items in a scale hang together? Measured by Cronbach's alpha or McDonald's omega.
• Test–retest reliability: do people get similar scores when tested again after a few weeks? Measured by a correlation or intraclass correlation.
• Inter-rater reliability: do two judges agree? Measured by Cohen's kappa or the intraclass correlation.
• Split-half reliability: do two halves of the test agree? Adjusted with the Spearman–Brown formula.

Types of Validity

• Content validity: do the items cover the whole construct? Often judged by an expert panel, sometimes quantified with a content validity index.
• Construct validity: does the scale behave as theory predicts? This includes convergent validity (correlating with related measures) and discriminant validity (not correlating too strongly with different constructs), usually tested with factor analysis.
• Criterion validity: does the score predict or match an external standard, either at the same time (concurrent) or later (predictive)?

What Cronbach's Alpha Measures

Lee Cronbach published his coefficient alpha in 1951. It estimates how much of the variance in a total scale score is "true" variance shared by the items, rather than item-specific noise. The formula is

$$ \alpha = \frac{k}{k-1}\left(1 - \frac{\sum s_i^2}{s_T^2}\right) $$

where \( k \) is the number of items, \( s_i^2 \) is the variance of item \( i \), and \( s_T^2 \) is the variance of respondents' total scores. When items are strongly related, the total-score variance becomes much larger than the sum of item variances, so the bracket and alpha approach 1.

Worked Example: A Four-Item Study Motivation Scale

Six respondents answered four Likert items (1 = strongly disagree, 5 = strongly agree) on study motivation. Real reliability analysis needs far more respondents; six keeps every step visible.

Step 1: Item Variances

Use the sample variance, dividing by \( n - 1 = 5 \).

• Item 1: mean 3.5; squared deviations 0.25, 0.25, 2.25, 2.25, 0.25, 0.25 sum to 5.5; variance \( 5.5/5 = 1.100 \).
• Item 2: mean 3.667; squared deviations sum to 7.333; variance \( 1.467 \).
• Item 3: mean 3.5; squared deviations 0.25, 2.25, 2.25, 2.25, 2.25, 0.25 sum to 9.5; variance \( 1.900 \).
• Item 4: mean 3.5; squared deviations sum to 5.5; variance \( 1.100 \).

Sum of item variances:

$$ \sum s_i^2 = 1.100 + 1.467 + 1.900 + 1.100 = 5.567 $$

Step 2: Total Scores and Their Variance

Each respondent's total: R1 = 17, R2 = 11, R3 = 19, R4 = 9, R5 = 18, R6 = 11. The mean total is \( 85/6 = 14.167 \).

Deviations: 2.833, −3.167, 4.833, −5.167, 3.833, −3.167. Their squares sum to 92.833, so

$$ s_T^2 = \frac{92.833}{5} = 18.567 $$

Step 3: Apply the Formula

$$ \alpha = \frac{4}{3}\left(1 - \frac{5.567}{18.567}\right) = 1.333 \times (1 - 0.2998) = 1.333 \times 0.7002 = 0.934 $$

The scale has excellent internal consistency in this sample.

Worked Example 2: Alpha if Item Deleted

SPSS also prints "Cronbach's Alpha if Item Deleted". Recomputing alpha with each item removed in turn gives:

Let us verify the Item 2 row by hand.

Step 1: Remaining item variances. Items 1, 3 and 4 have variances that sum to \( 1.1 + 1.9 + 1.1 = 4.1 \).

Step 2: New totals. Without Item 2, the totals are 12, 8, 14, 6, 14 and 9, with mean \( 63/6 = 10.5 \).

Step 3: Total variance. The squared deviations from 10.5 are 2.25, 6.25, 12.25, 20.25, 12.25 and 2.25, which sum to 55.5, so \( s_T^2 = 55.5/5 = 11.1 \).

Step 4: Alpha with \( k = 3 \).

$$ \alpha = \frac{3}{2}\left(1 - \frac{4.1}{11.1}\right) = 1.5 \times 0.6306 = 0.946 $$

Removing Item 2 would raise alpha slightly, from .934 to .946, and it has the weakest item–total correlation. That alone is not a reason to delete it: a rise of .01 is trivial, and dropping an item can weaken content validity. Only remove items when the gain is substantial and theoretically justified.

How to Interpret Alpha

Commonly cited guidelines (often attributed to George and Mallery) are:

• \( \alpha \ge .90 \): excellent
• \( .80 \le \alpha < .90 \): good
• \( .70 \le \alpha < .80 \): acceptable
• \( .60 \le \alpha < .70 \): questionable
• \( \alpha < .60 \): poor

Cautions:

• Alpha rises with the number of items. A 20-item scale can reach .80 even if items correlate only weakly.
• Very high alpha (above .95) may signal redundancy, with items that are near-copies of each other.
• Alpha assumes one dimension. Compute it separately for each subscale, not for the whole multi-construct questionnaire.
• Reverse-worded items must be recoded first, otherwise alpha can even turn negative.
• Alpha is not validity. A highly consistent scale can still measure the wrong thing.

Many methodologists now also report McDonald's omega, which relaxes alpha's assumption that all items load equally on the construct.

Reporting in APA Style

Internal consistency of the four-item study motivation scale was excellent (Cronbach's α = .93). Corrected item–total correlations ranged from .74 to .92, and no item deletion would have improved alpha meaningfully.

Software

• SPSS: Analyze → Scale → Reliability Analysis; tick "Scale if item deleted" under Statistics.
• Stata: alpha item1-item4, item.
• Excel: compute with VAR.S for each item and the total, then apply the formula.
• R:

r library(psych) alpha(df[, c("item1", "item2", "item3", "item4")])

Did You Know?

• Cronbach's 1951 paper became one of the most cited papers in the history of psychology.
• Late in life, Cronbach wrote that he felt alpha was being over-used and that other approaches often suited researchers better.
• For items scored only right or wrong, alpha reduces to the older Kuder–Richardson formula 20 (KR-20).

Key Takeaways

• Reliability is consistency; validity is accuracy. Both are needed.
• \( \alpha = \frac{k}{k-1}\left(1 - \sum s_i^2 / s_T^2\right) \); values of .70 and above are usually acceptable.
• Recode reverse-worded items and compute alpha separately for each subscale.
• Do not delete items just to nudge alpha upward; protect content validity.
• Support validity with expert review, factor analysis and correlations with related measures.

Need reliability and validity analysis for your questionnaire or thesis? Amathify's experts and PhD scholars provide complete step-by-step solutions; reach out on WhatsApp or via the Ask a Question form.