Article — Residual Calculator (Observed − Predicted)
Residual calculator
A residual is the difference between an observed value and the corresponding value predicted by a model, written e = y − ŷ. If the observed value is 125 and the model predicted 120, the residual is +5. Positive residuals mean the model under-predicted; negative residuals mean it over-predicted. The smaller the residuals, the closer the fit.
Residuals are the workhorse of regression diagnostics. They tell you whether your model captures the structure of the data or systematically misses certain regions. They feed into goodness-of-fit metrics — sum of squared residuals (SSR), mean absolute error (MAE), root mean square error (RMSE), and R². Every line of statistical inference about a linear model runs through the residual matrix.
What is a residual?
A residual is a single number per data point: how far the actual observation deviates from the regression prediction at that point. If you have 100 data points and fit a line, you get 100 residuals — one per observation. They live in the same units as the dependent variable. A residual of 5 in a house-price regression means the model missed by $5 (or $5,000, depending on scale).
By construction, ordinary least squares regression chooses the line that minimizes the sum of squared residuals. Other algorithms minimize different residual functions: least absolute deviation minimizes Σ|e_i|, quantile regression minimizes asymmetric weighted residuals, and so on. Whatever you minimize defines what residual you care about.
The term "residual" in regression comes from Carl Friedrich Gauss's 1809 derivation of the least-squares method for astronomical orbit fitting. He was tracking the dwarf planet Ceres and needed to estimate its orbital parameters from imperfect telescope observations. The residuals — observation minus orbital prediction — were what he minimized to recover the orbit, a procedure that recovered Ceres in 1801 after it had been lost behind the sun.
The residual formula
The basic formula is simple subtraction. The aggregate metrics derive from it.
eᵢ = yᵢ − ŷᵢ individual residualSSR = Σ eᵢ² sum of squared residualsMAE = (1/n) × Σ|eᵢ| mean absolute errorRMSE = √(SSR / n) root mean square errorR² = 1 − SSR/SST coefficient of determinationeᵢ_std = eᵢ / s standardized residualThe squared formulation in SSR and RMSE was chosen for mathematical convenience — squares are differentiable, absolute values are not — but it has a side effect: large residuals dominate the metric. A single outlier can pull RMSE up substantially while leaving MAE roughly unchanged.
Residual vs error
The two words are often conflated but the distinction matters in formal statistics. Error refers to the deviation between an observation and the true unobservable population mean — a theoretical quantity. Residual refers to the deviation between an observation and the model's fitted prediction — an observed quantity. Residuals are sample estimates of errors.
Because you never see the true population, you never see true errors. You only see residuals, which approximate the errors with some sampling noise. This distinction underlies inference: standard errors of coefficients, confidence intervals and hypothesis tests all rely on assumptions about the true errors being modelled, and residuals are the only window into them.
Using residuals to judge model fit
Three numbers summarize how well a model fits across an entire dataset: SSR, RMSE and R². All three are derived from residuals.
- SSR (sum of squared residuals): always non-negative. Smaller is better. Scale-dependent — comparing across datasets requires normalization.
- RMSE: RMSE is in the units of y. RMSE of 5 on a house price model means typical prediction error is $5,000.
- MAE: also in units of y. Less sensitive to outliers than RMSE.
- R²: proportion of variance in y explained by the model. R² = 0.8 means 80% of variation in y is captured.
- Adjusted R²: R² penalized for the number of predictors. Standard for comparing nested models.
- F-statistic: ratio of explained to unexplained variance. Tests whether the model is better than the mean.
Residual plots and diagnostics
A scatter plot of residuals against fitted values is the single most useful diagnostic tool in linear regression. A good model produces a featureless cloud centered at zero. Patterns reveal trouble.
If residual variance grows with the fitted value (creating a fan or funnel shape on the residual plot), the OLS assumption of constant variance is violated. Standard errors of coefficients become biased — usually too small — leading to overconfident confidence intervals and inflated t-statistics. Common remedies include weighted least squares, log-transformation of y, or robust standard errors (Huber-White sandwich estimator).
Residual aggregates — SSR, MAE, RMSE
The three classical aggregates serve different purposes. SSR is the metric you minimized to fit the model; it is the obvious goodness-of-fit number, but its scale depends on n and on the units of y. RMSE corrects for n and keeps the units of y, making it the most interpretable single number. MAE is more robust to outliers but does not have the nice statistical properties of RMSE (it is not differentiable).
The relationship RMSE ≥ MAE always holds, with equality only when all residuals have the same magnitude. The ratio RMSE/MAE between roughly 1.0 and 1.5 typically indicates well-behaved residuals; ratios above 2 suggest heavy tails or outliers driving the squared metric.
Common residual mistakes
Always inspect a residual plot before trusting any regression output. R² of 0.95 can hide a clear curved pattern that signals a missing quadratic or interaction term. Numbers like RMSE summarize, but plots reveal.
The most common error is confusing residual with error in writing or speech. The two terms are interchangeable in casual use, but in formal statistical inference the distinction matters. Standard error of the regression, for example, refers to the estimated standard deviation of the true errors, computed from the residuals.
A second trap is computing residuals on training data and treating those as a measure of predictive accuracy. In-sample residuals are biased downward because the model has already been fit to those points. Out-of-sample residuals from held-out data, cross-validation or a test set are the honest measure of prediction error.
A third trap is reporting R² without considering whether the model has an intercept. Without an intercept, the formula R² = 1 − SSR/SST does not behave as expected and can be negative even when the model is reasonable. Always include an intercept unless you have a strong theoretical reason to force the line through the origin.
A fourth trap is using RMSE alone to compare models with different output scales. RMSE inherits the units of y. A model predicting log(price) has RMSE in log-dollars; a model predicting price has RMSE in dollars. Comparing the two requires careful unit conversion or a normalized metric like MAPE (mean absolute percentage error).