Appendix A-5.1

Derivation of regression coefficient

Let us specify the following linear equation

yt = b0 + b1x1t + b2x2t + + bkxkt

which tests the presence of k+1 coefficients b being different from zero establishing a correlation between the endogenous variable y and k variables x with n observations from t=1 to t=n. The equation can be stated more economically by substituting a matrix X for the k variables x1 xk and B for the k coefficients b1 bk

Y = XB

The estimated coefficients b* are calculated for the sum of squared error terms et to be minimum, where the error terms et are given by

et = yt - xt B

If E is the vector of the error terms et

e = Y - XB

The sum of squared error terms is

sum(et) = E'E

or = (Y-XB)'(Y-XB)

= Y'Y - 2Y'YB + B'X'XB

For the sum of square error terms to be minimum, the first derivative with respect to B must be null

dE'E / dB = - 2Y'Y + 2X'XB = 0

The matrix of estimated coefficients B* is given by

B* = (X'X)-1 (X'Y)

In the case of two variables

{ Sum(yt) }
X'Y = { Sum(xtyt) }

{ n Sum(xt) }
X'X = { Sum(xt) Sum(xt2) }

{Sum(xt2) Sum(xt) }
and (X'X) -1 = 1/(n Sum(xt2) - (Sum(xt)) 2 ){- Sum(xt) n }

Solving for b1 and b0 gives

b1 = (n Sum(xtyt) - Sum(xt) Sum(yt)) / (n Sum(xt2) - (Sum(xt)) 2 )

or b1 = Sum((dxt)(dyt)) / Sum((dxt)2)

and b0 = Sum(yt)/n - b1 Sum(xt)/n

or b0 = E(Y)/n - b1 E(X)

The matrix of variances Sbb of the estimated coefficients B is given by

Sbb = Se(X'X)-1

where Se = variance of the error term

The test of the hypothesis that each coefficient b is different from zero is conducted with the t statistic

t = b / sb

where sb= the standard deviation of b calculated
given as a square root of element in Sbb above

The t statistic has a Z normal distribution N(0,1) which value are given in tables. For instance, if we obtain and estimate for t of 1.35. The table shows for that value and 10 degrees of freedom (i.e. the number of observations, minus the number of variables, plus one), there is 90% confidence interval that the hypothesis that b is different from zero can be supported. Statisticians also say the level of significance in this case if 5% (i.e. half of 1-.90) that accepting the hypothesis that b=0 would an error.

To test the validity of the entire equation, the statistic used is R2

R2 = 1 - SSR/TD

or R2 = (TD - SSR)/TD

where TD = total deviation
SSR = sum of squared error terms or residuals

One may verify the maximum value of R2 is one. A R2 value in excess of .50 is usually taken to prove that a correlation is present.

When there are more than one exogenous variable it is necessary to calculate an adjusted R2 to account for the loss of degrees of freedom from the presence of the additional variables

Adjusted R2 = 1 - (SSR/ (N-K)) / (TD/(N-1))

where TD = total deviation
SSR = sum of squared error terms or residuals
N = the number of observations
K = the number of exogenous variables

A more rigorous test of the "goodness of fit" than R2 is the F statistic which has a Chi Square distribution which values can be found in tables for different number of variables v and observations o. The F(v,o) statistic is calculated by

F(v,o) = ((TD-SSR)/(K-1)) / (SSR/(N-K))

where TD = total deviation
SSR = sum of squared error terms or residuals
N = the number of observations
K = the number of exogenous variables

Finally, the Durbin-Watson statistic D-W used to determine the presence of autocorrelation is calculated by

D-W = Sum((et - et-1 )2 ) / Sum (et2)

where e = error term or residual

Tables of values of Durbin-Watson statistic appear in statistical manuals and allow to reject the hypothesis of autocorrelation if D-W is higher than the value for the number of observations and variables used in the regression.