Answer by Abhilash Kantamneni:
Consider a problem where you have a bunch of factors that can predict some outcomes (or responses). For example, let us say you were trying to predict whether a particular set of wines are more likely be paired with meat, or dessert. You are given just two factors: Price and Sugar.
If these variables are independent, in other words, if the Price of a wine is not collinear with the amount of sugar in the wine, then you can use a simple multiple regression model (Figure from). You will probably end up with some model that effectively boils down to "higher the sugar, more likely to be used for dessert, price not likely to be a factor".
Using this exampleas an illustration, what if the factors that determined wine pairings were Price, Acidity, Sugar and Alcohol? As the number of factors increase when compared to the number of observations, the assumption that the factors are independent and non-collinear might not always hold.
In other words, you the wine shopper might be thinking "I'd probably use a sugary wine for dessert, as long as it is not too acidic, but maybe just a little alcoholic, but if it is too expensive, I'll just skip it etc….." etc. As you build this predictor for wine pairings, you are not just concerned with how much each of these factors independently influence the outcomes, but what combination of these factors influence the outcomes and how.
This is where Principal Components Regression helps: an indirect modeling technique that maps the factors directly to outcomes by first performing a Principle Component Analysis directly to the factors, and then applying the Multilinear Regression. (source :)
PLS takes the idea further and maximises the factors that correlate to responses. PLS and PCR are almost similar, but you can think of PLS as a pruned PCR. (Read).
Again, from, the advantages of PLS/PCR when compared to Multi-Regression are:
- Multiple outcomes
- Dimension reduction
- 'Closer to reality' modeling by getting rid of non-colinearity assumptions.