Least Square Support Vector Machines as. an Alternative Method in Seasonal. Time Series Forecasting

Transcription

1 Appled Mahemacal Sceces, Vol. 9, 05, o. 4, HIKARI Ld, hp://d.do.org/0.988/ams Leas Square Suppor Vecor Maches as a Alerave Mehod Seasoal me Seres Forecasg A Shabr Deparme of Mahemacal Sceces, Faculy of Scece Uvers ekolog Malaysa, 830 Skuda, Johor, Malaysa Copyrgh 05 A Shabr. hs arcle s dsrbued uder he Creave Commos Arbuo Lcese, whch perms uresrced use, dsrbuo, ad reproduco ay medum, provded he orgal work s properly ced. Absrac he leas square suppor vecor maches (LSSSVM) model s a ovel forecasg approach ad has bee successfully used o solve me seres problems. However, he applcaos of LSSVM model a seasoal me seres forecasg has o bee wdely vesgaed. hs sudy ams a developg a LSSVM model o forecas seasoal me seres daa. o assess he effecveess of hs model, he arle passeger seres ehbs olear behavour ad shows mulplcave seasoal behavour was appled. I order o oba he opmal model parameers of he LSSVM, a grd search algorhm ad cross-valdao mehod were employed. I hs sudy, seasoal auoregressve egraed movg average (SARIMA) ad arfcal eural ework (ANN) models are employed for forecasg he same daa ses. Emprcal resuls dcae ha he LSSVM yelds well forecasg performaces. hus, he LSSVM model provdes a promsg alerave for seasoal me seres forecasg. Keywords: ARIMA, LSSVM, forecasg, me seres, seasoal Iroduco Seasoaly s especally observable he ecoomc ad busess me seres daa. Mos me seres coss of reds ad seasoal effecs. However, he seasoal me seres s a comple ad olear problem. Improvg accuracy me seres forecasg s a mpora ye ofe dffcul ask. Much effor has

2 608 A Shabr bee devoed over he pas several decades o he developme ad mproveme of me seres forecasg models. Oe of he bes-kow approaches he developme of me seres model s he seasoal auo-regressve egraed movg average (SARIMA) model. he SARIMA model s oe of he mos popular approaches seasoal me seres forecasg owg o s sascal properes, as well as he well-kow Bo Jeks mehodology used for cosrucg he model. he SARIMA model has bee successfully ulzed may felds of forecasg such as ecoomc, egeerg, foreg echage, sock marke ad socal [8]. Alhough he SARIMA model has bee hghly successful boh academc research ad dusral applcao durg he pas hree decades, suffers from a major lmao owg o s pre-assumed lear form of he model. Recely, arfcal eural ework (ANN) model has bee eesvely suded ad also used as a alerave forecasg seasoal daa paer [7]. Some leraures dcaed ha ANN ca oba desrable resuls seasoal ad red forecasg [4,, 5]. Whle some researchers clam ha ANN s usuccessful fdg ou seasoal effec he daa srucure [6, 9, 7]. Suykes e al. [8] proposed a modfed verso of suppor vecor maches (SVM) called leas squares suppor vecor maches (LSSVM). I rece years, LSSVM eeded o cope wh forecasg problems, ad has bee used successfully varous areas of paer recogo ad me seres forecasg problems [5, 7, 3, 6]. However, applcaos of he LSSVM models seasoal me seres daa have o bee wdely suded. herefore, hs sudy aemps o develop a LSSVM model he seasoal me seres forecasg problems. Mehodology. he Auoregressve Iegraed Movg Average he ARIMA models roduced by Bo ad Jeks [], has bee oe of he mos popular approaches o he aalyss of he me seres forecasg. he geeral compoes of a ARIMA models coss of seasoal ad o-seasoal pars, whch are commoly kow as SARIMA models. he form of SARIMA(p,d,q)(P,D,Q)s model ca be wre as ( a () p s d s D s B ) P( B )( B) ( B ) q( B) Q( B ) s where (B) ad (B) are polyomals of order p ad q, respecvely; ( B ) s s ad ( B ) are polyomals B of degrees P ad Q, respecvely; p order of o-seasoal auo regresso; d umber of regular dfferecg; q order of he o-seasoal movg average; P order of seasoal auo regresso; D umber of seasoal dfferecg; Q order of seasoal movg average; ad s legh of seaso. Radom errors, a are assumed o be depedely ad decally dsrbued wh a mea of zero ad a cosa varace. A more dealed descrpo of

3 Leas square suppor vecor maches 609 he ARIMA models s publshed by Bo ad Jeks [] ad hus, descrpos of hese mehods are o gve here.. he Leas Square Vecor Maches Model he LSSVM s a ew verso of SVM modfed by Suykes e al. [4]. LSSVM volves he soluo of a quadrac opmzao problem wh a leas squares loss fuco ad equaly cosras sead of equaly cosras. Cosder a rag sample se (, y ) wh pu R ad oupu y R. I feaure space SVM models ake he form y( ) w ( ) b () where he olear mappg () maps he pu daa o a hgher dmesoal feaure space. LSSVM roduces a leas square verso o SVM regresso by formulag he regresso problem as m R( w, e) w w e (3) subjec o he equaly cosras y( ) b e,,,..., (4) w ( ) o solve hs opmzao problem, Lagrage fuco s cosruced as L w w e { w ( ) b e y } (5) where s Lagrage mulplers. he soluo of (5) ca be obaed by parally dffereag wh respec o w, b, e ad L 0 w ( ) (6) w L 0 0 (7) b L e 0 e (8)

4 60 A Shabr L 0 w ( ) b e y 0,,,..., (9) he he wegh w ca be wre as combao of he Lagrage mulplers wh he correspodg daa rag. w ( ) e ( ) (0) Pug he resul of (0) o (), he he followg resul s obaed: y( ) ( ) ( ) b K(, ) b () where a posve defe kerel s defed as follows: K (, ) ( ) ( ) he vecor ad b ca be foud by solvg a se of lear equaos: 0 ( b 0 ) ( I j ) y where y y...;, ;...;,...; ; y followg LSSVM model for fuco esmao: where ;. hs fally leads o he y( ) K(, ) b (), b are he soluo o he lear sysem. Kerel fuco, K(, ) represes he hgh dmesoal feaure space ha s olearly mapped from pu space. May works have demosraed favorable performaces of he Radal bass fuco (RBF) [0]. herefore, RBF s used as he kerel fuco hs sudy. he RBF s gve by K(,) ep( ), where s he kerel parameers. 3 Idces of Performace Evaluao I hs sudy, oe-sep ahead forecasg s cosdered. I pracce, shor-erm forecasg resuls are more useful as hey provde mely formao for he correco of forecasg value. I hs sudy, hree performace crera such as

5 No. of Passegers ('000) Leas square suppor vecor maches 6 meas square error (MSE), mea absolue error (MAE) ad correlao (R) were used o compare LSSVM performace wh ARIMA model or radoal mehods. hese crera are gve below: MSE R ( ˆ ) ( ( ), MAE ˆ )( ˆ ˆ) ( ˆ ˆ) where s he acual ad ˆ s he forecased value of perod, ad s he umber of oal observao. 4 Case Sudy I order o eame wheher LSSVM gves beer resuls or o, he daa se of arle passegers from he leraure was used. he arle passeger daa se was frs used by Brow [] ad he by Bo ad Jeks []. he arle passeger daa se cosss of he oal umber ( housads) of passegers o eraoal arles from Jauary 949 o December 960. Fg. shows ha he daa have a upward red ogeher wh seasoal varao whose sze s roughly proporoal o he local mea level called mulplcave seasoaly. he arle passeger seres s orgal form ehbs olear behavour ad shows mulplcave seasoal behavour. As may oher sudes volvg hs me seres, he daa from he frs years (3 observaos) are used for modellg ad he las observaos are used for esg Mohs (Ja Dec. 960) Fg.. Arle passeger daa from 949 o 960 he seres volves akg aural logarhms of he daa followed by seasoal (D=) ad o-seasoal (d=) dfferecg o make he seres saoary. he sample auocorrelao fuco (ACF) ad sample paral auocorrelao fuco (PACF) of saoary hs seres s ploed Fg..

6 6 A Shabr Auocorrelao Fuco for C3 (wh 5% sgfcace lms for he auocorrelaos) Paral Auocorrelao Fuco for C3 (wh 5% sgfcace lms for he paral auocorrelaos) Auocorrelao Paral Auocorrelao Lag Lag Fg.. ACF ad PACF plos used for he seleco of ARIMA model for ar passeger seres he ACF s dampg ou epoeal waves wh sgfca spkes ear lags ad. he PACF also has sgfcace values a lag ad. hs dcaes a possble SARIMA(p,,q)(P,,Q)s model wh p, q, P ad Q equal o wh s =. All combaos are esed o deerme he bes model ou of hese caddae models. he defcao of bes model for hs seres based o mmum AIC s show able. able. Comparso of AIC for seleced caddae models Model SARIMA(,,)(,,) SARIMA(,,0)(,,0) SARIMA(,,0)(0,,) SARIMA(0,,)(,,0) SARIMA(0,,)(0,,) Bold values dcae he bes AIC AIC he model fally seleced was SARIMA(0,,)(0,,). he resdual ACF (RACF) ad he PACF (RACF) of he bes model are demosraed Fg. 3. he RACF ad RPACF le wh he cofdece lms, whch clearly suppors he fac ha he resduals from he bes model are whe ose. ACF of Resduals for C (wh 5% sgfcace lms for he auocorrelaos) PACF of Resduals for C (wh 5% sgfcace lms for he paral auocorrelaos) Auocorrelao Paral Auocorrelao Lag Lag Fg. 3. he resdual ACF (RACF) ad he PACF (RACF) of he bes model

7 Leas square suppor vecor maches 63 Oe of he mos mpora seps developg a sasfacory LSSVM model s he seleco of he pu varables. A classcal mehod he seleco of he pu varables from a me seres by usg a se of me delayed samples of seres []. Usg hs mehod, he umber of odes he pu layer s equal o he umber of delays or lagged varables,,,..., ), where s he me ( 3 k delay, ad k s he umber of chose delays. he oupu, value of a me seres defed as P, s he predced P f (,,,..., k ) where P s a predco me o he fuure. I hs case, we gradually creased he umber of lagged varables from o hrough ehausve umercal smulao rus. A ew mehod o deerme he umber of odes he pu layer s based o he Bo-Jeks aalyss. Sce oe of he ma goals of hs work s o cosruc a LSSVM model for he arle passegers, herefore, he Bo-Jeks approach s used o dscover whch lagged or delayed varables are ecessary o prese as pus o he LSSVM. Usg he Bo-Jeks echque o he arle passegers me seres preseed Fg., he bes model foud s SARIMA model of order (0,,)(0,,) gve by ( B )( B ) ( B)( B ) a he model ca be wre as a a 0. 47a 3 3 a For he arles daa, he oupu ca be defed as f (,, 3) All combaos of umber of odes are esed o deerme he bes pu model for he LSSVM model. I he rag ad esg of LSSVM model, he same pu srucures of he daa se from Bo-Jeks model were used. I order o oba he opmal model parameers of he LSSVM, a grd search algorhm ad cross-valdao mehod were employed. I hs sudy, a grd search of (, ) wh he rage 0 o 000 ad he rage 0.0 o.0 was coduced o fd he opmal parameers. For each hyper parameer par (, ) he search space, 0-fold cross valdao o he rag se was performed o predc he predco error. able shows he performace resuls obaed he esg perod of he LSSVM approach. he bes values of he MSE, MAE ad R were obaed usg

8 64 A Shabr lag ad. For furher aalyss, he bes performace of he SARIMA ad LSSVM were compared wh he bes resuls suded by Faraway ad Chafeld [3]. I able 3, shows ha LSSVM has good performace ad hs model ouperform SARIMA ad ANN erms of all he sadard sascal measures for arles seres. he performace of LSSVM predcg he ar passeger s superor o classcal SARIMA model. he resuls show ha LSSVM model ca be successfully appled o esablsh accurae ad relable forecasg of seasoal me seres. able. he MSE, MAE ad R sascs of LSSVM model es perod Lags MSE MAE R,,3,,3, Bold values dcae he bes MSE, MAE ad R able 3. Comparso of performace of he LSSVM model wh ARIMA, ANNs ad ARIMA models suded by Faraway ad Chafeld [3] Idces of Performace Evaluao Model SSE MSE MAE Faraway SARIMA [3] Faraway ANN[3] SARIMA(0,,)(0,,) LSSVM 5 Cocluso here are pley of models used o predc me seres daa. I hs paper, he LSSVM model was proposed for mprovg forecasg performace of seasoal me seres. o llusrae he capably of he LSSVM model, arles daa was chose as a case sudy. Oe of he mos mpora facors developg a sasfacory LSSVM model s he seleco of he pu varables. Emprcal resuls o ar passeger daa ses usg wo dffere models clearly reveal he effcecy of he LSSVM model erms of MSE ad MAE values. hese resuls show ha he LSSVM model provdes a robus modellg capable of capurg he seasoal aure of he me seres daa ad hus producg more accurae forecass. Ackowledgemes. he auhors hakfully ackowledged he facal suppor ha afforded by MOE, UM ad GUP Gra (VO 4F68).

9 Leas square suppor vecor maches 65 Refereces [] G.E.P. Bo, G.M. Jeks, me Seres Aalyss: Forecasg ad Corol, Holde-Day, Sa Fracsco, 976. [] R.G. Brow, Smoohg Forecasg ad Predco of Dscree me Seres, Prece Hall, Eglewood Clffs, 96. [3] J. Faraway ad C. Chafeld, me seres forecasg wh eural eworks: a comparave sudy usg he ar le daa, Appl. Sas., 47 (998), o., hp://d.do.org/0./ [4] P.H. Frases ad G. Drasma, Recogzg chagg seasoal paers usg arfcal eural eworks, Joural of Ecoomercs, 8 (997), hp://d.do.org/0.06/s (97) [5] D. Habay, A eper sysem based o leas square suppor vecor maches for dagoss of he valvular hear dsease, Eper Sysems wh Applcaos, 36 (009), o. 3, hp://d.do.org/0.06/j.eswa [6]. Hll, M. O Coor ad W. Remus, Neural ework models for me seres forecass, Maageme Sceces, 4 (996), o. 7, hp://d.do.org/0.87/msc [7] S. Ismal, A. Shabr ad R. Samsud, A hybrd model of self-orgazg maps (SOM) ad leas square suppor vecor mache (LSSVM) for me-seres forecasg, Eper Sysems wh Applcaos, 38 (0), o. 8, hp://d.do.org/0.06/j.eswa [8] M. Khashe, M.Bjar ad A.R. Ardal, Improveme of Auo-Regressve Iegraed Movg Average models usg Fuzzy Logc ad Arfcal Neural Neworks, Neurocompug, 7 (009), hp://d.do.org/0.06/j.eucom [9]. Kolark ad G. Rudorfer, me seres forecasg usg eural eworks, ACM SIGAPL APL Quoe Quad, 5 (994), o., hp://d.do.org/0.45/ [0] L. Lu ad W. Wag, Echage raes forecasg wh leas squares suppor vecor maches, Ieraoal Coferece o Compuer Scece ad Sofware Egeerg, (008), hp://d.do.org/0.09/csse [] K. Nam ad. Schaefer, Forecasg eraoal arle passeger raffc

10 66 A Shabr usg eural ework, Logsc ad rasporao, 3 (995), o. 3, [] N.H. Packard, J.P. Cruchfeld, J.D. Farmer ad R.S. Shaw, Geomery from a me seres, Phys. Rew. Le., 45 (980), o. 9, hp://d.do.org/0.03/physrevle.45.7 [3] R. Samsud, P. Saad ad A. Shabr, Rver flow me seres usg leas squares suppor vecor maches, Hydrol. Earh Sys. Sc., 5 (0), hp://d.do.org/0.594/hess [4] J.A.K. Suykes,. Va Gesel, J. De Brabaer, B. De Moor ad J. Vadewalle, Leas Squares Suppor Vecor Maches, World Scefc, Sgapore, 00. hp://d.do.org/0.4/5089 [5] Z. ag ad P.A. Fshwck, Feedforward eural es as models for me seres forecasg, ORSA Joural o Compug, 5 (993), o. 4, hp://d.do.org/0.87/joc [6] X. Yurog ad J. Lagzhog, Waer qualy predco usg LS-SVM wh parcle swarm opmzao, Secod Ieraoal Workshop o Kowledge Dscovery ad Daa Mg, (009), hp://d.do.org/0.09/wkdd [7] G. Zhag ad M. Q, Neural ework forecasg for seasoal ad red me seres, Europea Joural of Operaoal Research, 60 (005), hp://d.do.org/0.06/j.ejor Receved: Sepember, 05; Publshed: Ocober, 05