Journal ofthe Nigérian Association o f Mathematical Physics 
. Volume 42, (July, 2017), pp229 -  238
© J. o f  N  AMP
Frequentist and Bayesian Estimation of Parameters of Linear Régression Model with
Correlated Explanatory Variables
A. A. Adepoju*, E. O. Adebajo and P. T. Ogundunmade 
Department of Statistics, University of Ibadan, Ibadan, Nigeria
Abstract
This paper addressed the popular issue o f  collinearity among explanatory 
variables in the context o f a multiple linear régression analysis, and the 
parameter estimations ofboth the classical and the Bayesian methods. Five 
sample sizes: 10, 25, 50, 100 and 500 each replicated 10,000 tintes were 
simulated using Monte Carlo method. Four levelss o f corrélation 
p  = 0.0,0.1,0.5, and 0.9 representing no corrélation, weak corrélation, 
moderate corrélation and strong corrélation were considered. The 
estimation techniques considered were; Ordinary Least Squares (OLS),
Feasible Generalized Least Squares (FGLS) and Bayesian Methods. The 
performances o f the estimators were evaluated using Absolute Bias 
(ABIAS) and Mean Square Error (MSE) o f  the estimâtes. In ail cases 
considered, the Bayesian estimators had the best performance. It was 
consistently most efficient than the other estimators, namely OLS and 
FGLS.
Keywords: Multicollinearity, Bayesian Estimation. Level of corrélation, Feasible Generalized Least 
Squares, Mean Square Error
1.0 Introduction
Régression analysis is a central tool in applied statistics that aims to answer the general question of how two 
or more explanatory variables influence the outcome of the response variable. However, this influence can be 
seen to dépend greatly on the degree of corrélation existing among the explanatory variables. Such 
relationship among the explanatory variables is referred to as multicollinearity. Multicollinearity is one of 
several problems confronting researchers using régression analysis. To most of these researchers especially 
economists, the single équation least-squares régression model is a very popular and useful model which is 
tried and true. Its properties and limitations hâve been extensively studied and documented and are, for most 
part, well-known. Discussion of problems that arise as particular assumptions are violated are been 
extensively discussed in literature [1].
The history of multicollinearity dates back to 1934 when the multicollinearity concept was formulated to refer 
to the condition when the variables handled are under influence of two or more relationships. The term 
multicollinearity is used to dénoté the presence of linear relationships (or near linear relationships) among 
explanatory variables which results in a breakdown of the least squares procedures. If the explanatory 
variables are perfectly linearly correlated, the corrélation coefficient for these variables is equal to unity [2], 
When this happens, it is therefore normally impossible to interpret estimâtes of individual coefficients when 
the explanatory variables are mainly highly inter-correlated, no matter what the goal of multiple régression 
analysis is. In addition, review of the literature indicates that the multicollinearity problem has been handled 
in a variety of different ways [3]. Unfortunately, this problem arises often in practice, since many économie 
variables such as income, wealth, etc. are likely to be interrelated. Time sériés data are also likely to exhibit 
multicollinearity. Many économie sériés tend to move in the same direction (e.g., production, income and 
employment). When two or more independent variables are correlated, the statistical estimation techniques are 
incapable of sorting out-the independent effects of each on the dépendent variable.
While régression coefficients estimated using correlated independent variables are unbiased, they tend to hâve 
larger standard errors than they would hâve in the absence of multicollinearity.
Corresponding Author: Adepoju, A.A., Email: pojuday@yahoo.com, +2348066430258
Journal o f  the Nigérian Association o f Mathematical Physics Volume 42, (July, 2017), 229 - 238
229
UNIVERSITY OF IBADAN LIBRARY
Frequentist and Bayesian... Adepoju, Adehajo and Ogundu \ „ -  \  AMP
This in tura means that the t ratios will be smaller. Thus it is likely that the r:-.-. • vvill not be
significant than in the case where no multicollinearity plagues the data. Ir. . . .  précision
associated with the estimated coefficients [4].
Multicollinearity has been indicated to weaken accurate inference through its e - ri errors of 
the individual parameter estimâtes, increase variance of estimators, yield hic:'. . . . .  ■. rmination
(R2), leads to wider confidence intervals as well as lower test statistics values (in a- - . . . ificant
tests. Similarly, this results in parameter estimâtes with incorrect signs and unlikeb. — _ L-: :ng it
more difficult to specify the correct model [5-7],
Several estimators hâve been used in estimating the parameters of a linear model in the . . . .  . . ,.:ed
explanatory variables with the OLS being the oldest and the most popular. The FGL'- . _ LS
estimator of a transformed isomorphic model that works on the shortcomings of OLS ha- .. - dT*. OV
by some authors. Bayesian on the other hand, offers ways to attain a reasonable e>:::r._:. 
parameters due to the possibility of including some sort of prior knowledge about the'.- r_ . . 
However, the majority of Bayesian inference problems according to [9] can be seen as e. _ .. . 
expectation of a function U(0) of interest under the parameter.
Multicollinearity is probably présent in ail régression analysis, since the independent variables _ • . ::
be totally uncorrelated. Thus whether or not multicollinearity is a problem dépends on __ -..
collinearity. The difficulty is that there is no statistical test that can détermine whether or not :: :_
problem. One method to search for the problem is to look for “high” corrélation coefficients be: . .  
variables included in a régression équation. Even then, however, this approach is not foolpr: : 
multicollinearity also exists if linear combinations of variables are used in a régression équation. There . - 
single préférable technique for overcoming muticollinearity, since the problem is due to the form of the date - 
Collinearity among explanatory variables is one feature of the data that is directly related to the amour.: 
information provided by the sample. When the sample is not informative enough to lead to significar.: 
conclusions, the only potential solution is to introduce more information. Although classical method rejects 
outright subjective information, it is not surprising that the search for operational solutions within ::s 
framework has failed to produce accepted techniques to combat multicollinearity. Hence, this studs 
demonstrated that there is in fact an appropriate place for subjective information especially when the 
regressors are correlated. It seems more useful to speak in terms of the multicollinearity problem’s severity 
rather than its existence. A case of perfect multicollinearity is rare, as is a zéro corrélation among explanatory 
variables (X's). Accordingly, multicollinearity will be defined here in terms of departures from independence. 
or from non-correlation, of the X's with one another. The major objective of this study is to compare the 
asymptotic behaviours of classical and Bayesian estimators at different levels of corrélation among the 
explanatory variables.
2.1 Material and Methods
In observational studies, the data generated by uncontrolled mechanisms may be subject to biases not présent 
in controlled experiments. The most common problem is interrelationships among the independent variables 
that hinder précisé identification of their separate effects. In such circumstances, régression parameters will 
tend to exhibit large sampling variances, perhaps leading to incorrect inferences regarding their significance, 
and there will be high corrélations between parameters. Possible solutions to multicollinearity include the 
introduction of extra information, for example via prior restrictions on the parameters based on subject matter 
knowledge; the multivariate réduction of the set of covariates (e.g. by principal components analysis) to a 
smaller set of uncorrelated predictors; ridge régression [10], in which the parameters are a function of a 
shrinkage parameter k>0, with least squares estimate 
Given the following model for multiple linear régression:
Vi  = P o - + P i X n  +  T.lj = 2 P j x ij + £ i £é~1V(0,<72) ...(2.1)
where i — 1,2, ...,n  *
2.2 Effect of Multicollinearity on Sample Size
The effect of collinearity on parameter estimâtes is relative to the sample size. Intuitively, both collineaeritv 
and sample size may be viewed as two very similar factors that détermine the variability in the sample, which 
is the primary source of information offered by the data for statistical inference. Hence, the extent of either 
effect (high collinearity or small sample size) on individual parameter inference must be interpreted 
accordingly. From a purely classical objectivist perspective that obstinately refuses ail prior information in 
statistical inference, this claim is undisputable. That is, one certainly cannot commit inferential exclusivitv to 
a set of data, and upon receiving vague inference from the data, dismiss this vagueness on the grounds that the
Journal o f the Nigérian Association o fM athematical Physics Volume 42, (July, 2017), 229 - 238
230
UNIVERSITY OF IBADAN LIBRARY
Frequentist and Bayesian... Adepoju, Adebujo and Ogundunmade J. o f  N A M P
data is poorly conditioned. Hence, this study also investigates the effect of varying corrélation among the 
regressors when sample sizes are 10, 25, 50, 100 and 500 respectively.
2.3 Detecting the Problem
A natural starting point to detecting multicollinearity problem is to look at the corrélations between predictors. 
Perhaps there are only two predictors, this is sufficient enough to detect any problem with collinearity: if the 
corrélation between the two predictors is zéro then there is no problem. If the corrélation is low then 
collinearity is probably just a rninor nuisance - but will still reduce statistical power (meaning that there is less 
possibility to detect an effect and the effect will be measured less accurately). However with a larger 
corrélation, there is more serious problem [11,12], Furthermore with more than two predictors, the 
corrélations between predictors can be misleading. Even if they are ail very low (and unless they are exactly 
zéro) they could conceal important multicollinearity problems. This will happen if the predictor's corrélations 
do not overlap - and thus they hâve a cumulative effect. Working out the severity of the multicollinearity 
problems is not that easy and it is generally a good idea to use collinearity diagnostic. Fortunately there are a 
number of multicollinearity diagnotics that can help detect problems. Two of these diagnostic methods include 
tolérance and inflation factor (VIF).
2.4 Problems of the commonly used remédiai Measures
In a situation where multicollinearity is detected, there are a number of ways in literature for dealing with it. 
However, the best remedy for multicollinearity is either to design a study to avoid it for instance, using an 
appropriate experimental design or increasing the sample size (which may not be feasible in some cases) to 
make your estimâtes sufficiently accurate. Increase in sample size decreases the effect of multicollinearity on 
the standard error (the larger the sample size, the smaller the standard error). If these methods are not feasible 
there are other options that may be helpful. One of which is to drop a predictor which may sometimes lead to 
a bigger problem such as misspecification error.
2.5 Model Estimation 
The Classical Approach
The problem with the classical régression models in dealing with multicollinearity is that the standard errors 
associated with parameters estimâtes only reflect error due to sampling and there is no way to incorporate 
uncertainty that is associated with the model spécification. Various classical estimation techniques hâve been 
used in literature on linear régression models. Of ail these estimation techniques, only the methods used here 
shall be discussed, the Ordinary Least Squares (OLS) and the Feasible General Least Squares (FGLS).
A. Ordinary Least Squares (OLS)
Among ail the various econometric methods that can be used to dérivé parameter estimâtes of econometric 
relationships, Ordinary Least Squares (OLS) stands on top priority list. It seeks the minimization of the sum of 
squares déviation of the actual observations on a variable front the values that would be obtained based on the 
régression équation. The ordinary least squares équation is given as: 
y  =  Xp  + e e~ N (0 ,a 2) ...(2.2)
The OLS is known to be biased and inconsistent when endogenous variables appear as regressors in an 
équation and may be inefficient. Thus the OLS estimator for estimating p is given as:
Pois = (x1x y 1x 1y  ...(2.3)
By decomposing fs:!ls estimator
Pois = (x1x y 1x 1y  
The variance is given as
VarOL\ f ) \ x \ = a 2(X 'X ) - ' (2 4)
B. Feasible Generalized Least Squares (FGLS)
The feasible generalised least squares is an ordinary least squares of the transformed variable that satisfies the 
standard least squares assumption. Feasible generalized Least Squares (FGLS) estimator is used when S is 
unknown. That is, the efficient estimation of P in feasible generalised least squares régression model does not 
require the knowledge of û  - a positive definite symmetric matrix. In the estimation of FGLS, fl is used 
instead of fl.
Thus the FGLS estimator for estimating fi is given as:
pfgis = ( x ' a - ' x r ' x ' s r ' y  ...(2.5)
Journal o f  the Nigérian Association o f  Mathematical Physics Volume 42, (Jtily, 2017), 229 - 238
231
UNIVERSITY 
F IBADAN LIBRARY
Frequentist and Bayesian... Adepoju, Adebajo and Ogundunmade J. o f  NAM P
with
varfgls[p\x\ = a 2( x ' . . . ( 2 .6)
The Bayesian Approach
Several authors hâve worked on model estimation using Bayesian approach, [13-17], Bayesian linear 
régression is an approach to linear régression in which the statistical analysis is undertaken within the context 
of Bayesian inference. It is commonly recommended as a means of dealing with multicollinearity. The 
thinking is similar to the recommendation for dealing with multicollinearity by increasing the sample size. If 
the analysis is based on more information then there should be no problem estimating the parameters more 
precisely [1,18-21], In the Bayesian approach, we increase the information in the analysis by incorporating 
information about the prior beliefs about the parameter estimâtes as opposed to adding new data point [22-25], 
When the régression model has errors that are normally distributed, and if a particular form of prior 
distribution is assumed. explicit results are available for the posterior probability distributions of the model’s 
parameters. The Bayesian approach works on the principle of Bayes theorem that is: 
p(0 |y) =  p (y |0 )p (0 )/p (y ) ... (2.7)
However as p(y) is independent of the data, this gives the posterior distribution as:
Posterior oc Likelihood x Prior
P(0 |y) =  p (y |0 )p(0) ...(2.8)
By convention;
p(0) - the prior distribution of 0 (i.e. the distribution prior to observing the data y);
p ( y |0 ) -  Ae likelihood function (i.e. the likelihood of observing the data given a particular parameter value 0) 
p(0 |y) - the posterior probability distribution (i.e. the distribution of 0 obtained after observing y and
combining the information in the data with the information in the prior distribution);
P (y) _ the marginal density, is the sum(or intégral) of p(y[0)p(ô) over ail possible values of 6.
Prior probability density function p(0)
Non-informative prior is assumed for the parameters of the model. The idea behind the use of this prior (also 
known as fiat, diffuse or locally-uniform prior) is to make inferences that are not greatly affected by central 
information or when external information is not available [26]. Two rules were suggested in [27] to serve as 
guide in choosing a prior distribution. The first one States that “if the parameter has any fixed value n, a finite 
range, or from -oo to +oo, its prior probability should be taken as uniformly distributed”. The second is that “if 
the parameter, by nature, can take any value from 0 to oo, the prior probability of its logarithm should be taken 
as uniformly distributed.
The prior pdf is given as
p(/? ,a2) ce -L ...(2.9)
p{6) -  constant (prior p d f  )
Likelihood Function p (y |0 )
Based on the assumption from our model that e~NIID (0, £), the likelihood function is given as; 
p(y\P ,o2) = — l—  e x p [ - - ^ £ f =1Q/; -flxi)2] ... (2.10)
{ 2 n ) 2 a N
Writing the likelihood in terms of précision gives
p(y\P.h) = - ■ yv n fcexpt" (/? -  /?)z ZtiXi2]} {h?exp[-^]} ...(2.11)
vvhere h =  -a%2 
Posterior PDF p (0 |y )
Combining the prior pdf p(0) with the likelihood function, gives a posterior distribution of the form;
p(P \y,h) = -  - \  ^ {/üexp[-  ̂  (/g -  P)2 i f U  * ,2]} {/i^expf- ^ ] }  xp{p, g2) oc ^  ...(2.12)
The Bayesian approach in this research work is estimated using Monte Carlo simulation. Thus the posterior 
distribution of p \o 2 is given as
Journal o f the Nigérian Association o f Mathematical Physics Volume 42, (July, 2017), 229 - 238
232
UNIVERSITY OF IBADAN LIBRARY
Frequentist and Bayesian... Adepoju, Adebajo and Ogundunmade J. o f  N  AM P
v { P \y .^ ï ) ~ N ^ E.Vpo2) ...(2.13)
where 8 represents the parameters i.e. 0, a 2 .
3.0 Methodology Review
Given that the number of observations is represented by n, the problem of estimating or predicting the value of 
a dépendent variable y on the basis of a set of measurements taken on several independent variables 
x 1,x 2, ... xn is considered.
Generally, the model that relates observations and parameters may be written as;
(y\x , fi, o 2)~ N orm a l(xp ,o i2) ... (3.1)
However, it is convenient to express the estimators by employing the following matrix notation: 
y  = x(] + e e~ N (0 ,a 2) ...(3.2)
where;
y  =  {y i.y 2. — yn)7 is nxl matrix of response variables
e = (elt e2, ... en) is nxl matrix of residual variables
x  =  (xlf x 2, ... xn)' is the nxk design matrix of explanatory variables;
0  = kx  1 is a vector of régression coefficients. 
n =  number of observations
Design of Monte Carlo Experiment
The experiment involved generating five explanatory variables from a uniform distribution for sample sizes 
10, 25, 50, 100 and 500 out of which three (3) were made to correlate with specified corrélation values of 0.0, 
0.1, 0.5 and 0.9 for no, weak, moderate and strong corrélations respectively. The data was generated by 
arbitrarily ftxing the following values for the true parameters.
0 !  =  0.1, p 2 = 0.99, 0 3 = 0 .13, 0 4 = 1.00, 0 5 = 0.11
Likewise, the spécifie distributions for the predetermined variables were stated. Furthermore, the estimation of 
the already stated parameterg of the model was carried out using both the classical and the Bayesian 
approaches.
4.0 Analysis and Interprétation of Results
The summary of results obtained for each sample size 10, 25, 50, 100 and 500 replicated 10,000 times under 
the degrees of corrélation 0.0, 0.1, 0.5, 0.9 are presented in the subséquent tables.
The Table below shows the sum absolute bias and the MSE of the three estimators for each of the sample size 
replicated 10,000 times under the varying degrees of corrélation.
Journal of the Nigérian Association o f  Mathematical Physics Volume 42, (July, 2017), 229 - 238
233
UNIVERSITY OF IBADAN LIBRARY
Frequentist and Bayesian... Adepoju, Adebajo and Ogundunmade J. o f  NAMP
Tablel: Absolute Biases across Corrélation
riio 0.0 0.1 0.5 0.9
N OLS FGLS BAYESIAN OLS FGLS BAYESIAN OLS FGLS BAYESIAN OLS FGLS BAYESIAN
10 0.0132 0.0130 0.0130 0.0134 0.0133 0.0131 0.0148 0.0137 0.0134 0.0157 0.0145 0.0136
25 0.0129 0.0128 0.0126 0.0135 0.0132 0.0127 0.0149 0.0135 0.0130 0.0168 0.0147 0.0134
50 0.0126 0.0125 0.0125 0.0128 0.0127 0.0125 0.0136 0.0130 0.0128 0.0152 0.0138 0.0132
100 0.0114 0.0114 0.0114 0.0116 0.0115 0.0115 0.0125 0.0118 0.0116 0.0138 0.0136 0.0120
500 0.0100 0.0098 0.0098 0.0101 0.0099 0.0098 0.0112 0.0103 0.0100 0.0125 0.0110 0.0103
Table2: Absolute Biases across Sample Sizes
N 10 25 50 100 500
RHO OLS FGLS BAYESIAN OLS FGLS BAYESIAN OLS FGLS BAYESIAN OLS FGLS BAYESIAN OLS FGLS BAYESIAN
0.0 0.0132 0.0130 0.0130 0.0129 0.0128 0.0126 0.0126 0.0125 0.0125 0.0114 0.0114 0.0114 0.0100 0.0098 0.0098
0.1 0.0134 0.0133 0.0131 0.0135 0.0132 0.0127 0.0128 0.0127 0.0125 0.0116 0.0115 0.01 15 0.0101 0.0099 0.0098
0.5 0.0148 0.0137 0.0134 0.0149 0.0135 0.0130 0.0136 0.0130 0.0128 0.0125 0.0118 0.0116 0.0112 0.0103 0.0100
0.9 0.0157 0.0145 0.0136 0.0168 0.0147 0.0134 0.0152 0.0138 0.0132 0.0138 0.0136 0.0120 0.0125 0.0110 0.0103
Table 3: MSE of the Estimators across Sample Sizes
N 10 25 50
RHO OLS FGLS BAYESIAN OLS FGLS BAYESIAN OLS FGLS BAYESIAN
0.0 0.000007156 0.000007032 0.000007028 0.000008382 0.000008280 0.000008280 0.000007116 0.000007070 0.000006966
0.1 0.000010656 0.000010374 0.000010178 0.000008462 0.000008180 0.000007662 0.000007188 0.000007094 0.000006842
0.5 0.000011884 0.000010542 0.000010468 0.000009602 0.000008038 0.000007508 0.000008700 0.000008020 0.000007820
0.9 0.000010418 0.000009006 0.000008048 0.000009972 0.000009374 0.000007926 0.000009912 0.000008208 0.000007508
N 100 500
RIIO OLS FGLS BAYESIAN OLS FGLS BAYESIAN
0.0 0.000006732 0.000006732 0.000006716 0.000004412 0.000004240 0.000004212
0.1 0.000005660 0.000005538 0.000005444 0.000005078 0.000004938 0.000004856
0.5 0.000006510 0.000005808 0.000005664 0.000005148 0.000004938 0.000004180
0.9 0.000007692 0.000007036 0.000005896 0.000006486 0.000005076 0.000004510
Journal o f tlie Nigérian Association of Mathematical Physics Volante 42, (July, 2017), 229 - 238
234
UNIVERSITY OF IBADAN LIBRARY
Frequentist and Bayesian... Adepoju, Adebajo and Ogundunmade J. o f  NAM P
Table 4: Performance of The OLS Estimator across Sample Sizes
N = 10 N = 25 N = 50 N = 100 N = 500
RHO ABIAS MSE ABIAS MSE ABIAS MSE ABIAS MSE ABIAS MSE
0.0 0.0132 0.000007156 0.0129 0.000008382 0.0126 0.000007116 0.0114 0.000006732 0.0100 0.000004412
0.1 0.0134 0.000010656 0.0135 0.000008462 0.0128 0.000007188 0.0116 0.000005660 0.0101 0.000005078
0.5 0.0148 0.000011884 0.0149 0.000009602 0.0136 0.000008700 0.0125 0.000006510 0.0112 0.000005148
0.9 0.0157 0.000010418 0.0168 0.000019972 0.0152 0.000009912 0.0138 0.000007692 0.0125 0.000006486
Table 5: Performance of the rGLS Estimator across Sample Sizes
N = 10 N = 25 N = 50 N = 100 N = 500
RHO ABIAS MSE ABIAS MSE ABIAS MSE ABIAS MSE ABIAS MSE
0.0 0.0130 0.000007032 0.0128 0.000008280 0.0125 0.000007070 0.0114 0.000006732 0.0098 0.000004240
0.1 0.0133 0.000010374 0.0132 0.000008180 0.0127 0.000007094 0.0115 0.000005538 0.0099 0.000004938
0.5 0.0137 0.000010542 0.0135 0.000008038 0.0130 0.000008020 0.0118 0.000005808 0.0103 0.000004938
0.9 0.0145 0.000009006 0.0147 0.000009374 0.0138 0.000008208 0.0136 0.000007036 0.0110 0.000005076
Table 6: Performance of the Bayesian Estimator across Sample Si/.cs
N = 10 N = 25 N = 50 N = 100 N = 500
RHO ABIAS MSE ABIAS MSE ABIAS MSE ABIAS MSE ABIAS MSE
0.0 0.0130 0.000007028 0.0126 0.000008280 0.0125 0.000006966 0.0114 0.000006716 0.0098 0.000004212
0.1 0.0131 0.000010178 0.0127 0.000007662 0.0125 0.000006842 0.0115 0.000005444 0.0098 0.000004856
0.5 0.0134 0.000010468 0.0130 0.000007508 0.0128 0.000007820 0.0116 0.000005664 0.0100 0.000004180
0.9 0.0136 0.000008048 0.0134 0.000007926 0.0132 0.000007508 0.0120 0.000005896 0.0103 0.000004510
Journal o f the Nigérian Association o f Mathematicai Physics Volume 42, (July, 2017), 229 - 238
235
UNIVERSITY OF IBADAN LIBRARY
Frequentist and Bayesian... Adepoju, Adebajo and Ogundunmade J. o f  NAM P
5.0 Discussion of Results
In Tables 1 and 2, the absolute bias (ABIAS) is used to judge the perfonnances o f the three 
techniques. The estimâtes o f ail the methods increase gradually as the level o f corrélation among the 
explanatory variables increases. However, the Bayesian method produced the smallest absolute 
biases for ail the corrélation levels considered. Ail the estimators showed an asymptotic behaviour, 
as their ABIAS estimâtes decrease as the sample size increases.
From Table 3, the MSE estimâtes of the methods increase consistently as the level o f  corrélation 
among the variables increases. As the sample size increases, the MSE o f ail the methods also 
decrease except at N = 25. Again the Bayesian method outperfonned the other estimators.
Tables 4, 5 and 6 give the summary o f the perfonnances o f the estimators using both ABIAS and 
MSE across sample sizes.
6.0 Conclusion
The criteria used for comparison were majorly their absolute biases and mean square errors (MSE). 
From the analysis, the general perfonnance o f the Bayesian method was better compared to OLS and 
FGLS estimators. As the sample sizes increases, the bias reduced and the estimated parameters are 
approaching the specified/true parameters.
The performances o f the OLS when there was no or a very weak corrélation (i.e. p — 0.0 and 0.1) 
among the explanatory variables were not too far from each another with the exception o f sample 
size 25 and mostly for the large samples. Consequently, OLS may best be used when there is no or 
very weak corrélations for large samples however, defïning how large is, is another bone o f 
contention.
Furthermore, from the performances o f FGLS, it can be best used when there is no corrélation, weak 
corrélation or moderate corrélation among explanatory variables.
The performance o f the Bayesian method is remarkable even for small samples cases n<50. To a 
very large extent, the Bayesian method is less sensitive to multicollinearity in estimating the 
parameters o f linear régression models in the presence o f correlated explanatory variables compared 
to the considered classical approaches. The Bayesian approach is asymptotically consistent in 
estimating the parameters o f linear régression models in the presence o f correlated explanatory 
variables. This is a strong point for the Bayesian method. Consequently, to detennine the 
contribution o f each explanatory variable for any sample size at any level o f corrélation, the 
Bayesian method should be preferred over the classical methods.
Thus, we would conclude on the basis o f this analysis and statistical investigation that the Bayesian 
approach is better and should be preferred in estimating the parameters o f linear model in the 
presence o f correlated explanatory variables.
References
[1] Farrar, Donald E.; Glauber, Robert R. (1967): Multicollinearity in Régression Analysis
[2] Koutsoyannis, A. (2001): Theory o f Econometrics. Palgrave, New York
[3] Kmenta Jan (1971): Eléments o f econometrics. Macmillan publishing Co., Inc. New York.
[4] Scheroeder, Larry D., Sjoquist, David E. and Stephan, Paula E. (1986): Understanding 
Régression Analysis: An Introduction Guide, Sage Publications: The International 
Professional Publishers, No. 07-057. Newbury Park, CA: Sage.
[5] Belsley, D. A., Kuh, E. and Welsch, R. E.(1980): Régression Diagnostics: Identifying 
influential Data and Sources o f Collinearity. John Wiley & Sons., New York
[6] Kmenta, Jan. (1986): Eléments o f Econometrics. Second Edition. Ann Arbor: The University 
o f Michigan Press.
Journal o f the Nigérian Association o f  Mathematical Physics Volume 42, (July, 2017), 229 - 238
236
UNIVERSITY OF IBADAN LIBRARY
Frequentist and Bayesian... Adepoju, Adebajo and Ogundunimulc J. oJ'NAMP
[7] Greene, W. H. (2002), "Econometric Analysis" .Fourth édition, New Jersey: Prentice- Hall.
[8] Higgins, J. and Whitehead, A. (1996): Borrowing Strength from Extemal Trials in a meta- 
analysis. Stat. Med. 15, 2733 - 2749.
[9] Geweke, J. (1989): Bayesian Inference in Econometric Models using Monte Carlo 
Intégration, Econometrica, 57, 1317-1340
[10] Marquardt, D. W. and Snee, R. D. (1975): Ridge Régression in Practice. Annals o f 
Statistics, 29: 3 -  20.
[11] Poirier, D. J. (1995): Immédiate Statistics and Econometrics: A comparative Approach. 
Cambridge, MA: MIT Press
[12] Barnett, V. (1999): Comparative Statistical Inference, Wiley, Chichester, UK
[13] Zellner, A. (1971). An introduction to Bayesian Inference in Econometrics John Wiley& 
Sons, Inc.
[14] Zellner, A. (1965): Bayesian Inference and Simultaneous Equation Econometric Models. 
Abstract published in Econometrica 34(5): 47. JSTOR
[15] Zellner, A. (1979): Statistical Analysis o f Econometrics and Statistics: The Zellner view and 
papers. Edward Elgar Publishing. 250-305
[16] Berger, J. (2006): The case for objective Bayesian analysis (with Discussion), Bayesian 
Analysis, vol. 1; 385-402.
[17] Berg, Bemd A. (2004): Markov Chain Carlo Simulations and their Statistical Analysis (With 
Web-Based Fortran Code). Hackensack, NJ; World Scientific.
[18] Kumar, T. Krishna (1975): Multicollinearity in Régression Analysis’. Review o f Economies 
and Statistics 57 (3): 365-366
[19] Samaniego, F. and Renau, D. (1994): Toward a réconciliation o f the Bayesian and 
Frequentist approaches to point estimation’, Journal o f the American Statistical Association, 
vol. 89; 947-957.
[20] Samaniego, F. J. (2010): A comparison o f the Bayesian and frequentist approaches io 
estimation. Springer, New York, NY, USA.
[21] Koop Gary (2003): Bayesian Econometrics. l st Ed., John W iley and Sons Ltd., UK
[22] Leamer, Edward E. (1994): Sturdy Econometrics. Hants, England: Edward Elgar
[23] Birkes, David and Dodge, Yadolah. (1993): Alternative Methods o f Régression. New York: 
John
[24] Judge, George, W. E. Griffiths, R. Carter Hill, Helmut Lutkepohl (1985): The Theory and 
Practice o f Econometrics. 2nd Ed., New York: John W iley & Sons.
Journal o f the Nigérian Association o f Mathematical Physics Volume 42, (July, 2017), 229 - 238
237
UNIVERSITY OF IBADAN LIBRARY
Frequentist and Bavesian... Adepoju, Adebajo and Ogundunmade J. o f  N  AM P
[25] Vestrup, E. and Samaniego, F. (2004): Bayes versus frequentist shrinkage in multivariate 
normal problems, Sankhya, vol.66; 109-139.
[26] Yang. R., and Berger, J. O. (1998): Catalog o f non-informative priors: Berger, James O. 
2000’Bayesian Analysis: A look at today and Thoughts o f Tom orrow’ Journal o f the 
American Statistical Association, 95 (452): 1269-1276
[27] Jeffreys, H. (1961 ) Theory o f Probability (3rd edn). Oxford University Press: Oxford
Journal o f the Nigérian Association o f Mathematical Physics Volume 42, (July, 2017), 229 - 238
238
UNIVERSITY OF IBADAN LIBRARY