Error Propagation: Understanding How Uncertainty Spreads Through Calculations

When we perform calculations with measured values, we’re working with uncertainties. Every measurement carries some uncertainty, often expressed as absolute or relative error. These uncertainties don’t simply disappear when we use measurements in calculations; instead, they combine and can amplify depending on the mathematical operations involved. 

Error propagation, also known as the propagation of uncertainty, is the process of determining how uncertainties in measured variables affect the uncertainty in a computed result.

In this tutorial, we will learn about the mathematical rules that govern how uncertainties propagate through different operations and explore methods for quantifying uncertainty. We’ll work through practical examples that demonstrate how to calculate uncertainties in real-world scenarios, from simple arithmetic to complex functions involving correlation and advanced simulation techniques.

Key Concepts in Error Propagation

Before diving into the mathematical rules, let’s understand a few concepts behind error propagation:

• Absolute error represents the actual deviation from the true value, expressed in the same units as the measurement. If we measure a length as 10.0 ± 0.2 cm, the absolute error is 0.2 cm.
• Relative error expresses uncertainty as a fraction or percentage of the measured value. For our length measurement, the relative error would be 0.2/10.0 = 0.02 or 2%.
• Standard deviation is our primary measure of uncertainty in most scientific contexts, representing the spread of values around the mean. When we express a measurement as x ± σ, the σ represents one standard deviation.
• Variance is the square of the standard deviation. The variances of independent random variables add directly, a property that simplifies many calculations.
• Covariance measures how two variables change in relation to each other. When uncertainties are correlated, we must include covariance terms in our propagation formulas to obtain accurate results.

Understanding the difference between independent and correlated uncertainties is also important. Independent uncertainties arise from separate, unrelated sources, like reading errors from different instruments. Correlated uncertainties share a common source, such as using the same calibration standard for multiple measurements.

While our focus for this tutorial is on random errors that follow statistical patterns, it’s worth noting that systematic errors (consistent biases) require different treatment.

Mathematical Rules for Error Propagation

The propagation of uncertainty follows specific rules depending on the mathematical operation. Let’s explore each of these operations:

Addition and subtraction

When adding or subtracting measurements with uncertainties, we add the absolute uncertainties in quadrature (the square root of the sum of squares) for standard deviations.

For z = x ± y, where x has uncertainty σₓ and y has uncertainty σᵧ:

σz = √(σₓ² + σᵧ²)

Consider an example where we measure the total length of two rods. Rod A measures 15.3 ± 0.2 cm, and Rod B measures 8.7 ± 0.1 cm. The total length is:

Length = 15.3 + 8.7 = 24.0 cm
Uncertainty = √(0.2² + 0.1²) = √(0.04 + 0.01) = √0.05 = 0.22 cm

Therefore, the total length is 24.0 ± 0.22 cm.

Multiplication and division

For multiplication and division, we work with relative uncertainties. The relative uncertainty of the result equals the quadrature sum of the relative uncertainties of the inputs.

For z = xy or z = x/y:

(σz/z)² = (σₓ/x)² + (σᵧ/y)²

Suppose we want to calculate the area of a rectangle with a length of 5.0 ± 0.1 m and a width of 3.0 ± 0.05 m.

Area = 5.0 × 3.0 = 15.0 m²
Relative uncertainty in length = 0.1/5.0 = 0.02
Relative uncertainty in width = 0.05/3.0 = 0.0167
Relative uncertainty in area = √(0.02² + 0.0167²) = √(0.0004 + 0.000278) = 0.026
Absolute uncertainty in area = 15.0 × 0.026 = 0.39 m²

Therefore, the area is 15.0 ± 0.39 m².

Powers and roots

When raising a measurement to a power n, the relative uncertainty is multiplied by the absolute value of the power.

For z = xⁿ:

σz/z = |n| × (σₓ/x)

Consider a scenario where we measure a sphere’s radius as 2.5 ± 0.05 cm and want to calculate its volume using V = (4/3)πr³.

Volume = (4/3)π(2.5)³ = 65.45 cm³
Relative uncertainty in radius = 0.05/2.5 = 0.02
Relative uncertainty in volume = 3 × 0.02 = 0.06
Absolute uncertainty in volume = 65.45 × 0.06 = 3.93 cm³

Therefore, the volume is 65.5 ± 3.9 cm³.

Logarithmic and exponential functions

For natural logarithm: 

If z = ln(x), then σz = σₓ/x

For exponential: 

If z = eˣ, then σz/z = σₓ

Suppose we measure a value x = 10.0 ± 0.3 and calculate y = ln(x).

y = ln(10.0) = 2.303
Uncertainty in y = 0.3/10.0 = 0.03

Therefore, ln(10.0) = 2.303 ± 0.030.

Now that we’ve explored specific rules for different operations, let’s understand the unified mathematical foundation that underlies all of them.

Derivation Using Calculus

To build intuition, think about partial derivatives as answering the question: “If I change this input by a tiny amount, how much does my output change?” 

This sensitivity to change is exactly what we need to understand how uncertainties propagate. A large partial derivative means that the input strongly influences the output, so its uncertainty contributes more to the final uncertainty.

Let’s start with the mathematical foundation. For a function z = f(x, y), the total differential tells us:

dz = (∂f/∂x)dx + (∂f/∂y)dy

When we interpret dx and dy as small uncertainties and want to find the variance (σz²), we square both sides and take the expected value. 

This leads us to the general error propagation formula:

σz² = (∂f/∂x)² σₓ² + (∂f/∂y)² σᵧ² + 2(∂f/∂x)(∂f/∂y)σₓᵧ

Here, σₓᵧ represents the covariance between x and y. For independent variables, this term vanishes, giving us the simpler form we typically use.

Let’s verify this formula by deriving our multiplication rule. 

For z = xy:

• Partial derivative with respect to x: ∂z/∂x = y
• Partial derivative with respect to y: ∂z/∂y = x

Substituting into our general formula (assuming x and y are independent):

σz² = y²σₓ² + x²σᵧ²

Dividing both sides by z^² = (xy)^²:

σz²/z² = σₓ²/x² + σᵧ²/y²

Taking the square root gives us:

(σz/z)² = (σₓ/x)² + (σᵧ/y)²

which is exactly our multiplication rule for adding relative uncertainties in quadrature.

This general approach becomes useful when dealing with complex functions where memorized rules don’t apply. 

For instance, if we need to calculate the uncertainty in z = x^²sin(y) + e^xy, we can systematically apply partial derivatives rather than trying to combine multiple rules. The formula handles any differentiable function, making it our most useful tool for uncertainty analysis in complex calculations.

Correlation and Covariance in Error Propagation

Our error propagation formulas so far assume that uncertainties are independent, but what happens when they’re not? 

Correlation between uncertainties is more common than we might expect and can significantly affect our results. Correlation arises when measurements share a common source of uncertainty. 

Consider measuring the length and width of a metal block with the same ruler. If that ruler is actually 1% too short, both measurements will be systematically affected in the same way, and they’re positively correlated. Similarly, if temperature affects multiple sensors in an experiment, all readings taken at the same moment share this environmental influence.

When one measurement tends to be high while another tends to be high (positive correlation), or when one tends to be high while another tends to be low (negative correlation), we must account for this relationship in our uncertainty calculations.

When variables are correlated, we introduce the covariance term σₓᵧ in our propagation formula. For two correlated variables with a correlation coefficient ρ:

σₓᵧ = ρσₓσᵧ

The complete formula for addition becomes:

σz² = σₓ² + σᵧ² + 2ρσₓσᵧ

We’ll understand this concept better with an example. 

Let’s measure the perimeter of a rectangular metal plate at an elevated temperature. We measure length L = 10.0 ± 0.1 cm and width W = 5.0 ± 0.05 cm. The perimeter is P = 2L + 2W.

Without considering correlation:

P = 2(10.0) + 2(5.0) = 30.0 cm
σₚ = √[(2 × 0.1)² + (2 × 0.05)²] = √[0.04 + 0.01] = 0.22 cm

However, thermal expansion affects both dimensions similarly. If we determine that the correlation coefficient is ρ = 0.7 (strong positive correlation):

Covariance term: 2 × 2 × 2 × 0.7 × 0.1 × 0.05 = 0.028
σₚ = √[0.04 + 0.01 + 0.028] = √0.078 = 0.28 cm

Ignoring correlation underestimated our uncertainty by about 25%, a significant difference that could affect our conclusions.

We should include covariance when:

• Multiple measurements use the same instrument or calibration standard.
• Environmental conditions (temperature, pressure, humidity) affect all measurements simultaneously.
• Variables have known physical relationships (like dimensions expanding together).

We can safely omit covariance when measurements are truly independent, when taken with different instruments, at different times, or when correlation is negligible compared to other uncertainty sources. When in doubt, if changing conditions would affect both measurements in the same direction, they’re likely correlated, and this should be considered in our analysis.

Advanced Methods for Handling Uncertainty

When functions involve conditional logic, iterative calculations, or complex nonlinear relationships, Monte Carlo simulation is a good alternative.

Monte Carlo simulations

Monte Carlo simulation takes a direct approach: instead of propagating uncertainties through formulas, we simulate thousands of possible scenarios and observe the distribution of results. 

The process is:

1. Generate random samples from each input variable’s distribution.
2. Calculate the result for each set of samples.
3. Analyze the output distribution to determine uncertainty.

You can learn how to perform a monte-carlo simulation in Excel through our in-depth guide. However, for the completeness of this tutorial, let’s revisit a scenario where analytical methods become complex. 

Consider z = x^²/y where x = 10.0 ± 0.5 and y = 2.0 ± 0.2. While we could use partial derivatives, let’s see a conceptual approach to how Monte Carlo handles this:

for each of 10,000 iterations:
   x_sample = random value from normal(mean=10.0, std=0.5)
   y_sample = random value from normal(mean=2.0, std=0.2)
   z_sample = x_sample² / y_sample
  
result = mean of all z_samples = 50.2
uncertainty = standard deviation of z_samples = 5.3

The beauty of this approach is that it:

• Handles any distribution type: Our inputs don’t need to be normally distributed.
• Natural correlation handling: Remember our correlated perimeter measurement? Monte Carlo can generate correlated random samples directly.
• Complete output picture: We get the full distribution of results, revealing skewness or multiple peaks that standard deviation alone might not account for.
• Works with complex logic: Functions with if/then conditions or min/max operations can be handled.

So, when should we choose Monte Carlo over analytical methods? We can consider it when:

• Our function involves conditional statements or iterative calculations.
• Variables follow non-normal distributions (uniform, exponential, etc.)
• We need the full output distribution, not just its standard deviation.
• The analytical approach requires complex derivatives that we’re not confident in calculating.

For practical implementation, 10,000 samples typically provide good estimates for most applications, while 100,000 samples offer high precision. Modern computers can run these simulations in seconds for simple functions.

Computational considerations

For computationally intensive models, such as finite element simulations or climate models, where a single calculation might take hours, surrogate models are well-suited. 

Surrogate models are simplified mathematical functions that approximate the behavior of complex models, a faster substitute that captures the input-output relationships of the original model.

However, for most data analysis tasks, direct Monte Carlo simulation provides the ideal balance of accuracy and simplicity.

Conclusion

This article introduced error propagation, the mathematical framework for understanding how uncertainties in measurements affect calculated results. We learned the fundamental rules for propagating uncertainties through different mathematical operations, understood how correlation between variables impacts uncertainty calculations, and explored Monte Carlo simulation as a computational alternative for complex scenarios.

To deepen your skills in handling uncertainty and statistical analysis in real-world applications, consider exploring our Machine Learning Scientist with Python track. This track covers supervised and unsupervised learning, feature engineering, time series analysis, and deep learning, where understanding uncertainty and error analysis becomes important for evaluating model performance, confidence intervals, and making reliable predictions in data science projects.