Answer by Sumedha Sengupta:
In its simplest form:
In a sample of size, n, of paired observation, (x,y) the Method of Least Squares gives the estimates of the coefficients for a Best Fit straight line, namely, Y= mX+C that can represent the relationship between the correlated variables, x & y. The coefficients completely defines the straight line giving
'm' it's slope, and it's intersection, C on the y-axis. The method minimizes the total error involved in the sample, namely the total sum of squares of the differences between the observed values from the expected values.
Sometimes, the data in the sample shows higher order polynomials are better fit than the straight line . The methods applies for most of such cases, to estimate the coefficients of the polynomials for the best fit curve.