🧮 Essential Formulas

For the regression line \hat{y} = a + bx:

Slope: b = \frac{\sum(x_i - \bar{x})(y_i - \bar{y})}{\sum(x_i - \bar{x})^2} = \frac{s_{xy}}{s_x^2}

Alternative slope formula: b = r \cdot \frac{s_y}{s_x}

Y-intercept: a = \bar{y} - b\bar{x}

Where:

• r = correlation coefficient

• s_x, s_y = standard deviations

• s_{xy} = covariance

• \bar{x}, \bar{y} = sample means

Important: The regression line always passes through (\bar{x}, \bar{y})!

🔢 Key Statistics

Correlation coefficient: r = \frac{\sum(x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum(x_i - \bar{x})^2 \sum(y_i - \bar{y})^2}}

Coefficient of determination: R^2 = r^2

Residual: e_i = y_i - \hat{y}_i

Residual standard error: s_e = \sqrt{\frac{\sum e_i^2}{n-2}}

What R² tells us:

• R^2 = 0.25 → 25% of variation in y is explained by x

• R^2 = 0.81 → 81% of variation in y is explained by x

• Higher R^2 = better fit (closer to 1.0)

🎯 What is R²?

Coefficient of Determination (R²) measures the proportion of variation in the response variable explained by the explanatory variable.

Formula:

R^2 = r^2 (square of correlation)

Interpretation:

• R^2 = 0: No linear relationship

• R^2 = 0.25: Weak relationship (25% explained)

• R^2 = 0.64: Moderate relationship (64% explained)

• R^2 = 0.90: Strong relationship (90% explained)

• R^2 = 1: Perfect linear relationship

Key Points:

• Always between 0 and 1

• Higher = better fit

• Context matters for “good” values

• Can’t determine causation

🔍 Key Assumptions

1. Linearity - Relationship is actually linear - Check with scatterplot

2. Independence - Observations are independent - No time trends or clustering

3. Normality of Residuals - Residuals follow normal distribution - Check with residual plots

4. Constant Variance - Spread of residuals is consistent - No “fan” patterns

5. No Extreme Outliers - Outliers can heavily influence line - Check for unusual points

⚠️ Common Pitfalls

1. Correlation ≠ Causation - Strong correlation doesn’t prove cause - Could be confounding variables

2. Extrapolation Dangers - Don’t predict far outside data range - Relationships may change

3. Outlier Sensitivity - One extreme point can change everything - Always examine unusual observations

4. Non-linear Relationships - Linear regression assumes straight line - Curved relationships need different methods

5. Ecological Fallacy - Group-level patterns ≠ individual patterns - Be careful with aggregated data

🎯 Best Practices

• Always plot your data first
• Check residuals
• Consider context and causation
• Report limitations honestly

🧠 Key Formulas

Regression Line: \hat{y} = a + bx

Slope: b = r \cdot \frac{s_y}{s_x}

Intercept: a = \bar{y} - b\bar{x}

R-squared: R^2 = r^2

Residual: e_i = y_i - \hat{y}_i

🔍 Interpretation Skills

Slope: Change in y per unit change in x

Intercept: Value of y when x = 0

R²: % of y-variation explained by x

Always include units and context!

🛠️ Problem-Solving Steps

1. Plot the data (always start here!)
2. Calculate means \bar{x}, \bar{y}
3. Find correlation r
4. Calculate slope b = r \cdot s_y/s_x
5. Calculate intercept a = \bar{y} - b\bar{x}
6. Write equation \hat{y} = a + bx
7. Find R² = r²
8. Make predictions (within reason)
9. Interpret in context
10. Check assumptions

🚨 Red Flags to Watch

• Perfect correlation (r = ±1.0)
• Extrapolating too far
• Ignoring outliers
• Assuming causation
• Non-linear patterns