Raw data is useless without context. I've spent years working in statistics to know that the spread of data tells a deeper story than the average ever could.
1. The Logic of Dispersion
A Low Standard Deviation indicates that the data points tend to be very close to the mean (average). A High Standard Deviation indicates that the data points are spread out over a wider range of values. In finance, this is often used as a direct measure of risk or volatility.
2. Population vs. Sample (Bessel's Correction)
When calculating for a Sample, we use \(n-1\) in the denominator instead of \(n\). This is known as Bessel's Correction. It corrects the bias in the estimation of the population variance, making the result more statistically accurate when the entire population isn't known.
3. The 68-95-99.7 Rule (Empirical Rule)
In a normal distribution:
- 68% of data falls within 1 standard deviation of the mean.
- 95% of data falls within 2 standard deviations.
- 99.7% of data falls within 3 standard deviations.
4. Step-by-Step Calculation
- Calculate the Mean (Average) of all data points.
- Subtract the Mean from each data point (Deviation).
- Square each deviation (to remove negative values).
- Calculate the average of those squared deviations (this is the Variance).
- Take the square root of the variance to find the Standard Deviation.
Frequently Asked Questions (FAQ)
What does a high standard deviation mean? ▶
A high standard deviation means that the data points are spread out over a wider range of values, indicating higher volatility or inconsistency in the data set.
What is the difference between Population and Sample SD? ▶
Population SD is used when you have data for every member of a group. Sample SD is used when you are estimating the SD of a large population based on a smaller subset. Sample SD uses "n-1" in its formula to correct for bias.
Why do we square the deviations? ▶
We square the deviations to ensure all values are positive (otherwise, they would sum to zero) and to give more "weight" to data points that are further away from the mean.
What is a "Normal Distribution"? ▶
A normal distribution is a bell-shaped curve where most data points cluster around the mean. In this distribution, 68% of data falls within one standard deviation of the average.
Can standard deviation be negative? ▶
No. Since standard deviation is the square root of variance (which is an average of squared numbers), it is always zero or a positive number.
