Principal Components Regression

Talking about what is Principal Components, and how to use it to squeeze some more value out of our regression models. The tldr is that this can help reduce the number of columns we look at.

In this post, we'll go over a technique called principal components regression. This isn't an actual regression model, but rather the principal components technique being applied to a simple least squares model (linear regression).

Table of Contents

1. Frequently Asked Questions (FAQ)

2. Implementing Principal components regression pcr model in R and Python

3. Principal Components Analysis

4. Linear Regression Model

5. Principal component regression

Frequently Asked Questions (FAQ)

I.) What does principal component regression do?
Principal component regression is a dimensionality reduction technique that can be used in place of multiple linear regression. The first step in principal component regression is to calculate the eigenvectors and eigenvalues of the correlation matrix.

The eigenvectors define the directions of the new axes, and the eigenvalues determine how much variance is associated with each new axis. The second step is to choose which components to keep by selecting those with the largest eigenvalues. The remaining eigenvectors are then used to transform the original data into a new set of variables, which we call principal components.

Finally, we use multiple linear regression on these principal components to predict our response variable.