Chter4 Predictions In Liner Regression Model Prediction o vlues o stud vrible An iortnt use o liner regression odeling is to redict the verge nd ctul vlues o stud vrible The ter rediction o vlue o stud vrible corresonds to knowing the vlue o E( (in cse o verge vlue nd vlue o (in cse o ctul vlue or given vlue o exlntor vrible We consider both the cses The rediction o vlues consists o two stes In the irst ste, the regression coeicients re estited on the bsis o given observtions In the second ste, these estitors re then used to construct the redictor which rovides the rediction o ctul or verge vlues o stud vribles Bsed on this roch o construction o redictors, there re two situtions in which the ctul nd verge vlues o stud vrible cn be redicted- within sle rediction nd outside sle rediction We describe the rediction in both the situtions Within sle rediction in sile liner regression odel Consider the liner regression odel x Bsed on sle o n sets o ired observtions ( xi, i ( i,,, n ollowing i xi i, where i s re identicll nd indeendentl distributed ollowing N(, The reters nd re estited using the ordinr lest squres estition s b o nd b o s where b bx s b s x xx s x x s x x x x n n n n x ( i ( i, xx ( i, i, i i i n i n i The itted odel is b bx Cse : Prediction o verge vlue o Suose we wnt to redict the vlue o E( or given vlue o x x Then the redictor is given b b bx Here stnds or en vlue Econoetrics Chter 4 Predictions In Liner Regression Model Shlbh, IIT Knur
Predictive bis The rediction error is given s E( b bx E( x b bx ( x ( b ( b x Then the rediction bis is given s ( ( ( E E Eb Eb x Thus the redictor is n unbised redictor o E( Predictive vrince: The redictive vrince o is PV ( Vr( b bx Vr b ( x x ( x x Vr( ( x x Vr( b ( x x Cov(, b n sxx ( x x n sxx Estite o redictive vrince The redictive vrince cn be estited b substituting b ˆ MSE s ( x x PV ( ˆ n sxx ( x x MSE n s xx Prediction intervl : The (- % rediction intervl or E( is obtined s ollows: The redictor distributed s is liner cobintion o norll distributed rndo vribles, so it is lso norll ~ N x, PV Econoetrics Chter 4 Predictions In Liner Regression Model Shlbh, IIT Knur
So i is known, then the distribution o E( PV ( is N (, So the (- % rediction intervl is obtined s E( Pz z / / PV ( which gives the rediction intervl or E( s ( x x ( x x z/, z/ n sxx n sxx is unknown, it is relced b ˆ MSE nd in this cse, the sling distribution o E( ( x x MSE n s xx is t -distribution with ( n degrees o reedo, ie, tn The (- % rediction intervl in this cse is E( Pt t MSE n s xx /, n /, n ( x x which gives the rediction intervl s ( x x ( x x t/, n MSE, t/, n MSE n sxx n sxx Note tht the width o rediction intervl E( is unction o x The intervl width is iniu or x x nd widens s x x increses This is exected lso s the best estites o to be de t x -vlues lie ner the center o the dt nd the recision o estition to deteriorte s we ove to the boundr o the x -sce Econoetrics Chter 4 Predictions In Liner Regression Model Shlbh, IIT Knur 3
Cse : Prediction o ctul vlue I x is the vlue o the exlntor vrible, then the ctul vlue redictor or is b bx Here ens ctul The true vlue o in the rediction eriod is given b x where indictes the vlue tht would be drwn ro the distribution o rndo error in the rediction eriod Note tht the or o redictor is the se s o verge vlue redictor but its redictive error nd other roerties re dierent This is the dul nture o redictor Predictive bis: The redictive error o is given b b bx ( x ( b ( b x Thus, we ind tht E( E( b E( b x E( which ilies tht is n unbised redictor o Predictive vrince Becuse the uture observtion is indeendent o, the redictive vrince o is PV ( E( E[( b ( x x( b ( b x ] Vr( b ( x x Vr( b x Vr( b Vr( ( x x Cov( b, b xcov( b, b ( x x Vr( b [rest o the ters re ssuing the indeendence o with,,, ] Vr( b [( x x x ( x x] Vr( b Vr( [( x x x] Cov( b, b Vr( b x Vr( b Vr( x Cov( b, b x x x n sxx sxx sxx n ( x x s xx x n Econoetrics Chter 4 Predictions In Liner Regression Model Shlbh, IIT Knur 4
Estite o redictive vrince The estite o redictive vrince cn be obtined b relcing b its estite ˆ MSE s ( x x PV ( ˆ n sxx ( x x MSE n sxx Prediction intervl: I is known, then the distribution o PV ( is N (, So the (- % rediction intervl or is obtined s Pz z / / PV ( which gives the rediction intervl or s ( x x ( x x z/, z/ n sxx n sxx is unknown, then PV ( ollows t -distribution with ( n degrees o reedo The (- % rediction intervl or in this cse is obtined s P t /, n t /, n PV ( which gives the rediction intervl or s ( x x ( x x t/, n MSE, t/, n MSE n sxx n sxx The rediction intervl is o iniu width t x The rediction intervl or or x nd widens s x is wider thn the rediction intervl or x increses becuse the rediction intervl deends on both the error ro the itted odel s well s the error ssocited with the uture observtions Econoetrics Chter 4 Predictions In Liner Regression Model Shlbh, IIT Knur 5
Within sle rediction in ultile liner regression odel Consider the ultile regression odel with k exlntor vribles s, where (,,, n ' is n vector o n observtion on stud vrible, x x x k x x xk xn xn xnk is n k trix o n observtions on ech o the k exlntor vribles, (,,, k ' is k vector o regression coeicients nd (,,, n ' is n vector o rndo error coonents or disturbnce ter ollowing N(, I n I intercet ter is resent, tke irst colun o to be (,,,' Let the reter be estited b its ordinr lest squres estitor b ( ' ' Then the redictor is b which cn be used or redicting the ctul nd verge vlues o stud vrible This is the dul nture o redictor Cse : Prediction o verge vlue o the objective is to redict the verge vlue o, ie, E(, then the estition error is given b E( b b ( ( ' ' H where H ( ' ' Then E E( which roves tht the redictor The redictive vrince o is ' H H ' H PV ( E E( ' E( E E tr H k b rovides unbised rediction or verge vlue The redictive vrince cn be estited b PV ( ˆ k where is obtined ro nlsis o vrince bsed on OLSE Econoetrics Chter 4 Predictions In Liner Regression Model Shlbh, IIT Knur ˆ MSE 6
is known, then the distribution o E( PV ( is N (, So the (- % rediction intervl or E( is obtined s E( Pz ( z PV / / which gives the rediction intervl or E( s z/ PV(, z/ PV( is unknown, it is relced b ˆ MSE nd in this cse, the sling distribution o E( PV ( is t -distribution with ( n k degrees o reedo, ie, t n k The (- % rediction intervl or E( in this cse is E( Pt/, nk t /, nk PV ( which gives the rediction intervl or E( s t /, nk PV (, t/, nk PV ( Cse : Prediction o ctul vlue o the redictor error is given b b b ( ( ' ' I ( ' ' H b is used or redicting the ctul vlue o stud vrible, then its rediction Thus E( Econoetrics Chter 4 Predictions In Liner Regression Model Shlbh, IIT Knur 7
which shows tht rovides unbised redictions or the ctul vlues o stud vrible The redictive vrince in this cse is PV ( E '( E ' H H E ' H trh ( nk The redictive vrince cn be estited b PV ( ˆ ( n k where ˆ MSE is obtined ro nlsis o vrince bsed on OLSE Coring the erornces o to redict ctul nd verge vlues, we ind tht in better redictor or redicting the verge vlue in corison to ctul vlue when PV ( PV ( or k( nk or k n ie when the totl nuber o observtions re ore thn twice the nuber o exlntor vribles Now we obtin the conidence intervl or is known, then the distribution o PV ( is N (, So the (- % rediction intervl or is obtined s Pz ( z PV / / which gives the rediction intervl or s z/ PV(, z/ PV( is unknown, it is relced b ˆ MSE nd in this cse, the sling distribution o PV ( is t -distribution with ( n k degrees o reedo, ie, t n k Econoetrics Chter 4 Predictions In Liner Regression Model Shlbh, IIT Knur 8
The (- % rediction intervl o in this cse is obtined s Pt/, nk t /, nk PV ( which gives the rediction intervl or s t /, nk PV (, t/, nk PV ( Outside sle rediction in ultile liner regression odel Consider the odel ( where is n vector o n observtions on stud vrible, is n k trix o exlntor vribles nd is n vector o disturbnces ollowing N I n (, Further, suose set o n observtions on the se set o k exlntor vribles re lso vilble but the corresonding n observtions on the stud vrible re not vilble Assuing tht this set o observtion lso ollows the se odel, we cn write ( where is n vector o uture vlues, is n k trix o known vlues o exlntor vribles nd is n vector o disturbnces ollowing N I n (, It is lso ssued tht the eleents o nd re indeendentl distributed We now consider the rediction o vlues or given ro odel ( This cn be done b estiting the regression coeicients ro odel ( bsed on n observtions nd use it is orulting the redictor in odel ( I ordinr lest squres estition is used to estite in odel ( s b ( ' ' then the corresonding redictor is b ( ' ' Econoetrics Chter 4 Predictions In Liner Regression Model Shlbh, IIT Knur 9
Cse : Prediction o verge vlue o stud vrible the i is to redict the verge vlue E(, then the rediction error is E( b ( b ( ' ' Then Thus E E( ' ' E rovides unbised rediction or verge vlue The redictive covrince trix o is Cov( E E( E( ' E ( ' ' ' ( ' ' ' ' ' E ' ( ' ' ( ' ' ' ( ' ' The redictive vrince o is PV( E E( ' E( tr Cov( tr( ' ' I is unknown, then relce b ˆ redictive vrince nd there estites re Cov ( ˆ ( ' ' ˆ ' PV ( tr ' ( Now we obtin the conidence intervl or E( MSE in the exressions o redictive covrince trix nd is known, then the distribution o E( PV ( is N (, So the (- % rediction intervl o E( is obtined s Econoetrics Chter 4 Predictions In Liner Regression Model Shlbh, IIT Knur
E( Pz z / / PV( which gives the rediction intervl or E( s z/ PV(, z/ PV( is unknown, it is relced b ˆ MSE nd in this cse, the sling distribution o E( PV ( is t -distribution with ( n k degrees o reedo, ie, t n k The (- % rediction intervl or E( in this cse is E( P t/, nk t /, nk PV ( which gives the rediction intervl or E( s t/, nk PV (, t/, nk PV ( Cse : Prediction o ctul vlue o stud vrible is used to redict the ctul vlue, then the rediction error is b ( b Then ( E E b E Thus rovides unbised rediction or ctul vlues The redictive covrince trix o in this cse is ' ' ' E ( b ( b' Cov E Vb E b ' ' ( ( (Using ( ( ' ' ' ( ' I n Econoetrics Chter 4 Predictions In Liner Regression Model Shlbh, IIT Knur
The redictive vrince o is PV( E ' tr Cov( ' tr( ' n The estites o covrince trix nd redictive vrince cn be obtined b relcing b ˆ MSE s ' Cov ( ˆ ( ' I n ' PV ( ˆ tr( ' n Now we obtin the conidence intervl or is known, then the distribution o PV ( is N (, So the (- % rediction intervl is obtined s Pz z / / PV( which gives the rediction intervl or s z/ PV(, z/ PV( is unknown, it is relced b ˆ MSE nd in this cse, the sling distribution o PV ( is t -distribution with ( n k degrees o reedo, ie, t n k The (- % rediction intervl or in this cse is P t/, nk t /, nk PV ( which gives the rediction intervl or s t/, nk PV (, t /, n k PV ( Econoetrics Chter 4 Predictions In Liner Regression Model Shlbh, IIT Knur
Siultneous rediction o verge nd ctul vlues o stud vrible The redictions re generll obtined either or the verge vlues o stud vrible or ctul vlues o stud vrible In n lictions, it not be rorite to conine our ttention to onl to either o the two It be ore rorite in soe situtions to redict both the vlues siultneousl, ie, consider the rediction o ctul nd verge vlues o stud vrible siultneousl For exle, suose ir dels with the sle o ertilizer to the user The interest o con would be in redicting the verge vlue o ield which the con would like to use in showing tht the verge ield o the cro increses b using their ertilizer On the other side, the user would not be interested in the verge vlue The user would like to know the ctul increse in the ield b using the ertilizer Suose both seller nd user, both go or rediction through regression odeling Now using the clssicl tools, the sttisticin cn redict either the ctul vlue or the verge vlue This cn segurd the interest o either the user or the seller Insted o this, it is required to segurd the interest o both b striking blnce between the objectives o the seller nd the user This cn be chieved b cobining both the redictions o ctul nd verge vlues This cn be done b orulting n objective unction or trget unction Such trget unction hs to be lexible nd should llow to ssign dierent weights to the choice o two kinds o redictions deending uon their iortnce in n given liction nd lso reducible to individul redictions leding to ctul nd verge vlue rediction Now we consider the siultneous rediction in within nd outside sle cses Siultneous rediction in within sle rediction Deine trget unction ( E( ; which is convex cobintion o ctul vlue nd verge vlue E( The weight is constnt ling between zero nd one whose vlue relects the iortnce being ssigned to ctul vlue rediction Moreover gives the verge vlue rediction nd gives the ctul vlue rediction For exle, the vlue o in the ertilizer exle deends on the rules nd regultion o rket, nors o societ nd other considertions etc The vlue o is the choice o rctitioner Consider the ultile regression odel E E I, (, ( ' n Estite b ordinr lest squres estition nd construct the redictor b Now elo this redictor or redicting the ctul nd verge vlues siultneousl through the trget unction Econoetrics Chter 4 Predictions In Liner Regression Model Shlbh, IIT Knur 3
The rediction error is b( E( b( ( b ( Thus E( E( b E( So rovides unbised rediction or The vrince is Vr( E( '( E( b' ' ' ( b E ' ( ' ' ' ' ' ( b' ' ' ( b' E ( ' ( ' ' ' ( tr ( ' ' trin ( k n The estites o redictive vrince cn be obtined b relcing b ˆ MSE s Vr( ˆ ( k n Siultneous rediction is outside sle rediction: Consider the odel described erlier under outside sle rediction s, E(, V( In n nk k n ; E(, V( In n k n k n The trget unction in this cse is deined s ( E( ; The redictor bsed on OLSE o is b; b ' ' The redictive error o is b ( E( b( ( ( b Econoetrics Chter 4 Predictions In Liner Regression Model Shlbh, IIT Knur 4
So Thus E( E( b E( rovides unbised rediction or The vrince o is Vr( E( '( ' ' ( ' ( E b b ' ' ' E ' ( ' ' ' ( ' ' ' tr ( ' ( ' ' n ssuing tht the eleents is nd re utull indeendent The estites o redictive vrince cn be obtined b relcing b ˆ MSE s ( ˆ ( ' ( ' ' Vr ' tr n Econoetrics Chter 4 Predictions In Liner Regression Model Shlbh, IIT Knur 5