Week 2: Time Series Forecasting

sktime: Unified Python library for time series machine learning

Installing sktime from PyPI

To install sktime with core dependencies, excluding soft dependencies, via pip type:

pip install sktime

To install sktime with maximum dependencies, including soft dependencies, install with the all_extras modifier:

pip install sktime[all_extras]

Extracted from:

https://www.sktime.org/en/stable/installation.html

Probabilistic time series model

Let \({X_1, X_2, ...}\) be a sequence of random variables. Then the joint distribution of the random vector \(([X_1, X_2, ..., X_n])\) is

\[P[X_1 \le x_1, X_2 \le x_2, ..., X_n \le x_n]\]

where \(-\infty < x_1, ..., x_n < \infty\) and \((n = 1, 2, ...)\)

Mean function

The mean function of \({X_t}\) is

\[\mu_X(t)=E(X_t).\]

Covariance function

The covariance function of \({X_t}\) is

\[\gamma_X(r, s)=Cov(X_r, X_s)=E[(X_r-\mu_X(r))(X_s-\mu_X(s))]\]

for all integers \((r)\) and \((s)\).

The covariance function of \({X_t}\) at lag \((h)\) is defined by \[\gamma_X(h):=\gamma_X(h, 0)=\gamma(t+h, t)=Cov(X_{t+h}, X_t).\]

Autocovariance function

The auto covariance function of \({X_t}\) at lag \((h)\) is

\[\gamma_X(h)=Cov(X_{t+h}, X_t).\] Autocorrelation function

The autocorrelation function of \({X_t}\) at lag \((h)\) is

\[\rho_X(h)=\frac{\gamma_X(h)}{\gamma_X(0)}=Cor(X_{t+h}, X_t).\]

Weekly stationary

A time series \({X_t}\) is called weekly stationary if

• \(\mu_X(t)\) is independent of \(t\).

• \(\gamma_X(t+h, t)\)` is independent of \((t)\) for each \((h)\).

In other words the statistical properties of the time series (mean, variance, autocorrelation, etc.) do not depend on the time at which the series is observed, that is no trend or seasonality. However, a time series with cyclic behaviour (but with no trend or seasonality) is stationary.

Strict stationarity of a time series

A time series \(\{X_t\}\) is called weekly stationary if the random vector \([X_1, X_2..., X_n]\) and \([X_{1+h}, X_{2+h}..., X_{n+h}]\) have the same joint distribution for all integers \((h)\) and \((n > 0)\).

Simple time series models

1. iid noise

1. no trend or seasonal component

2. observations are independent and identically distributed (iid) random variables with zero mean.

3. Notation: \({X_t} \sim IID(0, \sigma^2)\)

4. plays an important role as a building block for more complicated time series.

import numpy
import matplotlib.pyplot as plt

mean = 0
std = 1 
num_samples = 1000
samples = numpy.random.normal(mean, std, size=num_samples)
samples

array([ 2.14145624e-01,  1.08921814e-01, -5.11987567e-01, -1.32186374e+00,
        1.09789903e+00,  2.93297923e-01, -6.34363715e-01, -6.28762591e-01,
       -5.64648110e-01, -1.68209070e+00,  1.41439281e+00, -1.11721795e+00,
       -9.22642781e-02, -6.30185036e-02, -2.18193430e+00, -7.70824476e-01,
        8.82166875e-02, -3.27811259e-01,  8.31132141e-01,  1.08099115e+00,
        4.60191765e-01, -1.28953308e-01,  7.04403399e-01,  3.61355409e-01,
       -3.89070439e-01, -8.56271407e-01, -8.87844968e-01,  2.20909912e-01,
       -2.33305110e-01,  1.00441521e+00,  1.79524597e+00,  7.49544794e-01,
        1.40201904e+00,  2.10113766e-01,  1.51214183e-01,  1.91841212e+00,
       -1.30684658e+00, -4.47171105e-01, -4.01838766e-01,  7.62403960e-01,
       -6.97385364e-01,  4.73001040e-01, -1.98080096e+00, -1.89375384e-01,
        2.99582476e-01,  6.15237566e-01,  3.87802417e-01,  1.14541042e+00,
        3.67713406e-01,  1.06750905e+00,  1.17741884e+00, -2.57156703e-01,
       -6.69413738e-01, -1.16652875e+00, -1.13729193e+00, -1.99347313e+00,
       -6.55448002e-01,  7.23559807e-01, -9.47080669e-01, -4.14025583e-01,
       -2.25526111e-01, -5.26183890e-01,  1.17769754e+00, -7.57991355e-01,
        1.07746573e+00, -1.60706269e+00, -1.65753029e+00, -1.30611705e+00,
       -6.62137173e-01,  7.21671210e-01, -2.56074108e-01,  3.91699280e-03,
        3.55385620e-01, -9.11462239e-01, -6.78663832e-01,  1.04026877e+00,
       -7.98926445e-02, -1.44173175e+00, -2.07586921e+00,  1.40533466e+00,
        1.38456053e-01, -3.83668269e-01,  1.65457521e-01,  1.09938583e+00,
       -3.87825866e-01,  1.10740363e+00, -1.07398779e+00,  3.82215603e-01,
        6.70685336e-04, -1.21018276e+00, -4.40313833e-01,  6.21440922e-01,
       -1.33086133e+00,  4.78559134e-01, -5.78023220e-03,  4.56562068e-01,
       -1.52449178e+00,  2.87515621e-02,  1.14212588e+00, -4.07441371e-01,
       -1.65367052e+00,  8.48812543e-02, -8.17389117e-01,  1.66251862e+00,
        4.24593134e-01, -5.02015419e-02, -2.04106409e-01,  5.01700271e-01,
       -5.61835871e-02, -6.60649233e-01, -1.12194305e+00,  1.44851734e+00,
        1.33086583e-01,  1.00772800e+00, -1.51104696e+00,  9.34566430e-01,
       -1.93479864e+00,  1.02666543e+00,  8.79010229e-01,  1.85358951e-01,
       -4.61297000e-01, -5.52581836e-01,  8.63892306e-01, -1.21424542e+00,
       -5.09829519e-01,  8.81153738e-01, -9.57550040e-01, -4.77108038e-01,
       -1.33679262e+00,  5.29430827e-01, -1.04993058e+00,  1.40356176e+00,
        2.13241066e+00,  4.52256220e-01,  1.32986932e+00, -6.26108311e-01,
        1.71025106e+00, -4.09161850e-01,  4.50057282e-01, -4.57590330e-01,
       -5.98032900e-01, -2.54161274e-01, -4.59919994e-01, -1.29074388e+00,
        4.32305388e-01, -3.66276253e-02, -5.91664269e-01, -7.02631090e-01,
       -1.34736800e+00, -1.06046142e+00,  7.66828216e-01,  4.83794470e-03,
       -2.35186897e-01,  1.40078681e+00,  3.73325101e-01,  2.49623812e-01,
        2.15320258e+00,  1.06798441e+00, -1.10852383e+00, -9.30411967e-01,
        9.03708629e-01, -3.39028265e-01, -5.70280940e-01, -7.63270306e-01,
        1.26838573e+00, -1.07192294e+00, -8.25899580e-01, -4.17665588e-01,
        4.09097508e-01, -1.62890688e+00, -2.38903557e-02,  8.17125714e-01,
       -3.53250654e-01,  8.74173184e-02, -1.96027360e+00, -3.63783657e-01,
       -1.77808913e+00, -5.39454390e-01, -1.60110554e+00, -3.72157076e-01,
        4.38914315e-01, -1.29904413e+00, -5.81971753e-01,  2.18264902e-01,
       -1.19900886e-01, -5.71571215e-01,  1.02972079e+00, -2.68560258e-01,
       -2.19223619e-01, -1.16976575e+00, -2.29519398e-01,  3.52760174e-01,
       -1.21245416e+00,  1.03410259e-01,  5.41393988e-02, -3.32707934e-01,
        9.02516937e-01,  4.72269008e-02,  7.69413254e-01,  4.90800018e-01,
       -7.13459071e-01, -1.51977738e-01,  3.04049492e-01, -2.73267720e-01,
       -5.77151701e-02, -5.96169308e-01, -1.19297597e+00, -4.20747209e-01,
       -1.92672752e-01,  2.20431462e-02,  1.40145901e+00, -1.60920604e-01,
        1.38273214e+00, -5.92526653e-01,  2.14122564e-01, -1.23302258e+00,
       -2.03624131e+00,  3.22286746e-01, -7.57108758e-01, -1.14915781e-01,
        9.45944409e-01,  9.37264674e-01,  3.70469974e-01,  4.08118869e-03,
       -1.24450117e+00, -1.31906211e+00, -4.56922516e-01,  1.13242281e+00,
       -1.37416472e+00, -1.05099141e+00, -5.73708659e-01,  3.96414230e-02,
       -1.51796221e+00, -1.59095251e+00, -1.81960535e+00, -6.65500228e-01,
        9.94888262e-02, -2.10574183e-01,  5.55617277e-01,  2.15586690e+00,
       -2.10506962e+00,  2.00110072e+00,  9.42773885e-01, -8.08627833e-01,
        1.55133565e+00,  2.44616308e-01,  6.01864642e-01,  9.10453727e-01,
        2.90188748e-01, -2.08787793e+00, -1.08091884e+00,  1.79563636e+00,
        1.43572422e+00,  9.64302422e-02,  1.66519679e+00, -5.80873609e-01,
       -1.75594544e+00,  1.06891993e+00,  3.44186405e-01,  9.38568401e-01,
        2.68194801e+00, -7.22255234e-01,  1.12844964e+00, -1.47252802e+00,
        1.35129058e-02, -8.10032907e-01, -9.04204948e-01, -1.35643199e+00,
       -2.00799783e-02, -3.11000492e-01,  1.31205007e+00, -1.95734062e-01,
       -1.91774712e-01,  1.55276757e+00, -1.84010689e-01,  1.36817631e-01,
        7.69003423e-01, -1.18069001e+00, -8.66809723e-01, -1.50164871e+00,
        1.05377609e+00,  2.66856883e-01,  3.77589123e-02,  2.60507723e-01,
       -2.47293329e-01, -3.49013899e-01,  7.03065270e-01, -9.01681386e-01,
       -3.45762694e-01, -2.23219755e-01,  9.11776595e-01,  5.59904020e-01,
        2.64120074e-01, -2.68006104e-01, -2.53480938e-01, -7.31962949e-01,
       -8.95921306e-01,  4.52615936e-01, -1.43271545e+00,  1.84809940e+00,
        2.39114998e-01,  1.60510742e-01, -8.91969283e-01, -1.08758341e+00,
        1.20563631e+00, -5.87115082e-01, -8.20190274e-01, -5.11063450e-01,
        2.18095671e-01, -2.34334969e-01, -5.40524303e-01,  1.07919892e+00,
       -1.89868830e-01, -6.78778693e-01,  4.80019020e-01, -1.87096461e+00,
       -1.50088438e+00, -5.36756251e-01,  1.00207566e+00, -2.84002024e+00,
       -1.92651316e+00, -7.63462133e-01, -1.46865078e+00,  9.68167224e-01,
        7.75448634e-01, -1.29403220e+00, -2.58678386e+00, -5.30352540e-01,
        5.91010898e-01,  8.44290192e-01, -3.27243070e-01,  1.13906041e+00,
        9.98349981e-01, -2.26853414e-02,  7.14408069e-01, -1.07143169e+00,
       -7.51824697e-01,  4.98055404e-02,  8.22179917e-03, -2.71967549e-01,
       -1.30392750e+00, -6.96976305e-02,  2.51959267e-01,  6.70740621e-01,
        1.46866086e+00,  6.02708080e-01, -1.19023754e+00,  1.89194722e-01,
       -6.19358519e-01,  4.39810561e-01,  1.91071875e+00,  2.80034976e-01,
        1.99370982e-02,  6.85563534e-01,  1.24371184e+00, -1.14202372e+00,
       -1.28618121e+00,  1.10951777e+00, -1.10199568e-01,  3.84412452e-01,
       -6.77353933e-01,  7.03340634e-01,  4.36671902e-01,  2.73264863e-01,
        2.63521113e-01,  2.50188363e-01, -2.97902680e-01,  1.61509739e-01,
       -1.13622727e+00, -1.16685182e+00, -1.10997964e+00,  6.49837619e-01,
        4.52671839e-01,  1.30486836e+00, -1.25098609e+00, -5.79584716e-01,
       -1.97991077e-01, -1.61087900e+00,  4.65085523e-01,  4.42396743e-01,
        1.03627600e+00, -1.58138050e+00,  1.54173142e-01,  9.27878363e-02,
        4.74035659e-01, -3.81493100e-01, -6.62789911e-02,  8.70793906e-01,
        3.44620545e-01,  1.25119067e+00,  6.53150990e-01,  5.82816202e-01,
        2.58107688e-01,  2.10709973e+00,  2.16594982e+00,  8.21949484e-03,
        3.29069522e-01, -5.08818577e-02,  1.51071793e+00, -8.33699352e-01,
       -2.63363887e-02, -2.29420367e+00,  1.07354582e+00, -1.54072722e+00,
        1.29769504e-02, -4.75216610e-01, -4.07533223e-01, -1.33336699e+00,
       -3.69594769e-02,  3.34242196e-01, -5.63508384e-01, -3.27034433e-01,
       -8.01226710e-01, -2.66648968e-01, -3.63916290e-01,  1.23960180e+00,
       -3.52672587e-01,  1.87830112e-01, -9.83218282e-02, -2.76943384e-01,
       -5.10308195e-02, -1.00705717e+00,  2.94158830e-01, -1.31579310e+00,
        1.26751542e+00, -2.43084705e-01, -3.42056914e-01, -5.99990071e-01,
       -6.28413001e-01, -1.58479167e-01, -1.28198651e+00, -2.40344621e-01,
        4.08137972e-01,  1.10898240e+00, -8.10195080e-01, -7.22431107e-01,
        2.02743615e+00,  1.60752351e+00, -1.46185441e-01, -1.29754596e+00,
        1.05810501e+00, -1.13384571e+00,  2.50523515e-01, -1.56148889e-01,
       -2.98176569e-01, -2.25779636e-01,  3.21927801e-01,  1.38364461e+00,
       -4.17915765e-04,  2.97547369e-01, -6.24622538e-01,  6.19865878e-01,
       -1.44911289e+00,  4.16640480e-01, -4.47099660e-02,  1.42966255e+00,
       -5.91808496e-01,  4.65178494e-01, -1.68192926e+00, -3.02829919e-02,
       -4.98443926e-01, -1.21420817e+00, -8.68684599e-03, -8.80267989e-01,
        1.33330238e+00, -1.93061610e+00,  1.59402555e-01,  6.27355273e-01,
        1.42023591e+00, -7.45813335e-01, -4.46806135e-01,  8.57264131e-01,
       -9.00623471e-01, -1.06124246e-01, -3.09440122e-01,  7.02753356e-01,
       -1.52539398e+00,  5.96717408e-01, -1.08263690e+00,  1.00064008e+00,
        3.17506045e-01,  5.20622371e-01, -1.55161825e+00, -2.46259214e-01,
        1.43615885e+00,  1.35343547e+00, -1.20921549e+00, -4.38549790e-01,
        3.48067042e-01,  3.66908218e-01,  8.52868197e-01, -6.81667511e-01,
       -4.74152568e-01,  8.57776584e-01,  1.74490711e+00,  2.75361917e-01,
       -6.06357676e-01,  1.55277051e+00,  2.24751643e-01, -5.10344365e-01,
        7.15929132e-02,  1.46133596e+00,  3.96468829e-01,  4.59840685e-01,
       -7.24378830e-02,  8.90649886e-01, -3.09058672e-01, -1.53690506e+00,
        8.42183198e-02, -4.93467712e-02,  1.52082567e+00, -4.45203268e-01,
        1.19237783e+00, -7.55652988e-01, -3.13654919e-01, -9.92084016e-01,
        4.60293400e-01, -6.01391855e-01, -1.13531971e+00, -5.53440486e-01,
        9.92809119e-01, -8.65292460e-02, -1.17163194e+00, -2.89359888e+00,
        9.93695407e-01, -5.16788466e-01, -6.27660718e-02, -1.78521997e+00,
       -6.87589688e-01,  1.99328524e+00, -3.74922963e-01,  1.37005100e+00,
        5.03010478e-01, -4.15159248e-01, -1.94114164e+00, -2.19040997e-01,
       -7.57500404e-01, -4.94731332e-01,  2.71196536e-02, -4.24699056e-01,
        4.10072207e-01,  6.96861284e-01, -1.61651206e-01,  1.71676988e-01,
        4.05112542e-01,  1.24637175e+00,  6.68964638e-01,  2.45635538e-01,
       -7.58980617e-01,  3.56229509e-03,  1.31595589e+00,  1.89437990e-01,
        2.47709010e-01, -5.32416684e-01,  1.31497905e+00,  8.33487109e-01,
       -6.20731285e-01,  1.28355051e-01, -1.72391432e+00, -1.78712130e+00,
       -1.63005421e+00, -1.02198887e+00, -5.32942431e-01,  1.44171284e+00,
       -4.82106499e-01, -9.01642794e-01, -1.01489436e+00, -3.43602990e-01,
        1.24446125e-01, -6.61831954e-01, -5.16450034e-02, -5.11185296e-01,
       -1.95776409e-01,  6.83291704e-01,  3.96852740e-01, -2.16193360e-01,
        1.47995485e+00, -5.26936352e-01, -2.00040486e-01, -1.34708128e+00,
        1.62940641e-01,  3.28064211e-01, -2.48068602e+00,  2.64188636e-01,
       -5.86392180e-01, -5.51301695e-01,  1.56078086e-01, -4.03049343e-01,
       -1.88641899e-01,  3.50758773e-01,  8.07052952e-01,  1.37399364e-01,
        1.07588769e+00, -8.35328442e-01,  3.69638153e-01, -9.70562260e-01,
       -1.06687010e+00, -9.88392664e-01, -6.32515897e-01,  1.72127292e+00,
        2.14858845e-01,  3.57550783e-01, -4.05970949e-01,  9.39954279e-01,
       -3.29744512e-01,  4.47827318e-01, -1.25227783e-01, -1.03803594e+00,
        2.85412891e-01,  1.67353180e-01,  4.99621679e-02, -1.69594716e-01,
       -6.15795829e-01,  1.17132031e+00,  1.68817729e+00, -1.14969479e-01,
        3.30014458e-01, -6.18759957e-01, -1.93347418e+00, -1.36062818e+00,
       -9.85459945e-01, -2.13212208e+00, -8.36902303e-01, -7.74322765e-01,
        6.96882367e-01, -9.17188342e-01, -2.80708446e+00,  4.43475313e-01,
        7.67671907e-01,  2.94820795e+00, -1.96608453e+00,  1.70904733e-01,
       -1.09351279e+00, -8.28134275e-01, -4.83674571e-01, -1.21016899e+00,
        2.80064374e-01, -4.47234613e-01, -1.16814527e+00,  3.59404597e-01,
       -6.82811560e-01, -1.50726764e+00,  3.56850915e-01, -4.11264767e-01,
       -1.52700398e+00,  1.79495223e-01,  8.96378733e-01, -5.47325116e-01,
        1.50869582e-01,  1.10027700e+00, -5.93008908e-02, -1.07236087e+00,
       -1.01751681e+00,  2.16355771e-01,  1.48166165e+00, -1.62002259e+00,
        1.55703098e+00,  6.51991949e-01,  2.60384671e-01, -2.01923890e+00,
       -4.46918337e-01, -5.87247865e-01, -2.57776288e-01, -2.15517373e+00,
       -9.92965092e-01,  1.10642181e+00, -1.02426678e+00, -3.40955298e-01,
       -6.57366288e-03,  1.02869176e+00,  1.18393949e+00,  3.98307644e-01,
       -4.14604733e-01, -9.14314946e-01, -1.87119093e+00, -1.81012731e+00,
       -6.82030442e-01, -6.02324915e-01, -1.41871700e+00, -7.30294132e-01,
       -9.09395770e-01, -1.87842393e+00,  1.28280157e+00, -1.09548808e+00,
        5.06335012e-01, -1.17709031e+00, -5.51488576e-01,  3.57939675e-01,
       -2.73318947e-01, -8.85718306e-01,  6.99264801e-01, -6.95838223e-02,
        1.05381517e+00, -1.86333957e+00,  1.22955148e+00, -2.76772619e+00,
        3.00113822e-01, -2.35719357e-01, -1.16255653e+00, -7.90067199e-02,
        6.94288661e-01,  1.76494776e+00,  7.90035509e-02,  1.16708112e+00,
        4.20752622e-01,  1.18679528e+00,  3.78908772e-01, -9.31616427e-01,
       -7.48236388e-01,  3.96374667e-01, -9.37187901e-01,  4.15277078e-01,
       -3.08107595e-01, -6.20771363e-01,  1.07298088e+00, -3.14959597e-01,
       -8.62924083e-01,  2.37308290e+00,  2.52199019e+00,  1.98896483e-01,
       -1.83186566e+00, -4.50825630e-01,  3.12575182e-01,  7.18939929e-01,
        1.95713912e-01,  5.90008339e-01,  8.34384609e-01,  1.15434508e+00,
       -8.61232041e-01,  1.39396643e-01,  4.54116679e-01, -2.37078020e-01,
       -4.25195131e-01,  5.14473133e-01, -8.66106796e-01,  7.70334642e-01,
        7.78192340e-01, -1.59952117e+00,  1.55514420e+00, -4.48232857e-01,
       -3.82311411e-01, -6.49425047e-01, -1.29301527e+00,  2.63216023e-01,
       -1.73307911e-01, -2.29347965e+00,  8.68007631e-01, -3.58490539e-01,
       -1.71280970e-02,  1.23707122e+00, -7.19158228e-01, -2.76601620e-01,
       -2.71152178e-01, -8.46982708e-01, -4.71652155e-01, -2.66382632e-01,
       -1.65065662e+00, -5.64137706e-01, -5.55674976e-01, -5.76521850e-01,
        4.18976534e-01, -4.29328942e-01,  3.24367814e-01,  7.96844216e-01,
        4.24202924e-01, -4.57870301e-01, -9.68070962e-01, -1.92319221e-01,
       -1.33264410e-01,  6.45665026e-01, -8.44762390e-02,  4.28293745e-01,
       -4.93675687e-01,  5.73891943e-01,  1.27893054e+00,  2.61916650e-01,
       -1.00117712e+00,  1.14963225e+00, -1.07356603e-01, -9.44272528e-02,
        1.44009953e+00, -8.53448241e-01,  7.41644271e-01, -9.55636743e-01,
        1.24033257e+00,  4.72853193e-01,  4.98377536e-01,  6.48176835e-01,
       -1.13664092e+00, -1.11173740e+00,  1.69433582e+00,  2.04927352e-01,
        1.72095944e+00,  1.13945412e-01, -2.78121635e-01,  1.22706658e+00,
       -1.36817292e+00,  9.33952760e-01,  1.03611573e+00,  4.52544530e-01,
       -8.46152509e-01,  1.89238280e-01,  1.32263775e-01, -1.16838274e-01,
        4.23124704e-01,  1.35231474e+00,  9.40102233e-02,  9.08133098e-01,
        2.90065192e-01,  1.53488635e+00, -2.20453781e+00,  1.20616522e-01,
       -9.05656892e-02, -2.20621691e-01, -1.35394151e+00, -1.27601202e+00,
       -1.60734550e-01, -1.00313335e+00,  2.18691344e+00,  1.66095188e+00,
        3.77950530e-01, -4.56302845e-01,  1.19378929e+00, -1.25954250e-01,
        3.99999645e-01,  7.91662917e-01,  2.21377152e+00, -2.04238729e+00,
       -1.68488302e+00, -1.04185646e+00,  1.51658457e+00, -1.32667548e+00,
        6.97498279e-01, -1.26664192e+00,  1.93908469e-01,  6.59680112e-01,
       -4.14294087e-01,  2.25837075e-01,  1.98223410e-01,  1.02932162e+00,
        1.48326159e-01,  6.90296301e-01, -9.10967536e-01, -1.92673815e-01,
       -6.04407779e-03, -2.61148821e-01, -1.14869105e+00,  4.73430223e-01,
        1.50235114e+00, -7.35833936e-01, -1.58051030e-02, -1.03311560e+00,
       -3.26033239e-01, -1.50657787e-01, -8.35836665e-01, -9.39832181e-01,
        7.81051321e-01, -1.12009602e-01,  3.18527123e-01, -8.42612416e-01,
        7.74522928e-01, -8.24563491e-01,  2.54958320e+00,  7.26541176e-01,
        4.05946240e-01, -1.88907649e-01, -6.37599513e-02, -4.67954304e-01,
       -1.54052177e-01, -2.31679862e-01, -5.63158371e-01,  2.05224445e+00,
        7.48625211e-01,  1.06974400e+00, -5.96465385e-01, -8.20185798e-01,
        9.31041818e-02,  3.34880736e-01,  4.34889470e-02,  2.08715829e+00,
       -8.98876070e-01, -5.77175378e-01,  5.76404209e-01, -1.72388778e+00,
        1.72946230e-01,  1.06213858e+00,  1.13029845e+00, -5.54838400e-01,
        5.07398009e-01,  7.83189070e-01, -2.58984527e+00, -7.41285566e-01,
        3.85810353e-01,  6.02175909e-01, -6.38740438e-01,  1.10571702e+00,
       -1.04512234e+00, -2.27408730e-01,  7.05587259e-01,  1.39424805e+00,
       -4.87076202e-01, -3.73567559e-01,  8.25776099e-01,  1.39063112e-01,
        1.23013207e+00,  2.97406116e-02,  1.57891994e-01,  2.49226179e-01,
       -5.05316919e-01, -1.63364137e+00, -4.86372875e-01, -5.86138068e-02,
       -6.03487956e-01, -2.64061368e-01, -8.63930994e-01, -3.33703660e-01,
        1.10046688e-01, -1.35697059e+00, -1.93279827e-01, -9.64497754e-01,
       -2.48391485e-01,  4.96065658e-01,  9.55104497e-01,  5.28932278e-01,
        6.16309708e-01, -8.13521325e-01,  3.33279630e-01, -1.11479881e+00,
       -8.47700645e-01,  1.93083510e+00,  5.34248293e-02, -4.67048721e-01,
        7.21030869e-01,  5.67722619e-01,  2.05686946e+00,  1.16570443e+00,
       -1.20059026e+00, -1.14212456e+00,  1.04585578e+00, -8.75412941e-01,
       -7.69671239e-01, -4.53009975e-01, -2.50954291e-01, -1.03670388e+00,
       -1.69391283e+00, -1.29476905e+00, -2.26797416e-01, -3.66970971e-01,
        8.22377032e-01, -5.29721112e-01, -6.39290797e-01, -9.64980099e-02,
        5.46983862e-01,  2.75171650e-01, -3.57772515e-01, -4.35683706e-01,
       -2.92249984e-02, -1.33860355e+00,  1.98365702e+00, -7.56165983e-01,
       -2.70143522e-01,  1.41132311e+00, -3.43760776e-02,  5.38439827e-01,
        1.28747088e+00,  9.91717313e-01,  8.50426205e-01,  6.19514753e-01,
       -7.48254849e-01, -1.32225641e-01, -7.23713227e-01,  1.68613111e-01,
       -1.70745791e-01, -5.10494895e-01,  3.71025850e-01, -1.92404784e-01,
        1.55707243e+00, -2.00020793e-01, -2.34619401e-03, -6.43855709e-01,
       -5.04503849e-01,  1.95837219e+00, -6.22930933e-01, -1.17479176e+00,
       -1.03226847e+00, -1.19885526e+00,  1.28594537e+00, -3.50359473e-01,
       -5.73690929e-01,  1.75236542e-01, -1.35735150e+00, -7.69537247e-01,
       -2.28962196e+00,  9.56017608e-01, -1.35616423e+00,  8.60384702e-01,
        3.16222550e-01, -7.96653231e-01,  2.05687308e+00,  1.09395349e+00])

2. White noise

If \({X_t}\) is a sequence of uncorrelated random variables, each with zero mean and variance \(\sigma^2\), then such a sequence is referred to as white noise.

Note: Every \((IID(0, \sigma^2)\) sequence is \((WN(0, \sigma^2)\) but not conversely.

3. Random walk

A random walk process is obtained by cumulatively summing iid random variables. If \({S_t, t=0, 1, 2, ...}\) is a random walk process, then \(S_0 =0\)

\(S_1=0+X_1\)

\(S_2=0+X_1+X_2\)

\(...\)

\(S_t=X_1+X_2+...+X_t.\)

Question

Is \({S_t, t=0, 1, 2, ...}\) a weak stationary process?

Identifying non-stationarity in the mean

• Using time series plot

• ACF plot

  • ACF of stationary time series will drop to relatively quickly.

  • The ACF of non-stationary series decreases slowly.

  • For non-stationary series, the ACF at lag 1 is often large and positive.

Elimination of Trend and Seasonality by Differencing

• Differencing helps to stabilize the mean.

Backshift notation:

\[BX_t=X_{t-1}\]

Ordinary differencing

The first-order differencing can be defined as

\[\nabla X_t = X_t-X_{t-1}=X_t-BX_t=(1-B)X_t\] where \(\nabla=1-B\).

The second-order differencing

\[\nabla^2X_t=\nabla(\nabla X_t)=\nabla(X_t-X_{t-1})=\nabla X_t - \nabla X_{t-1}\]

\[\nabla X_t - \nabla X_{t-1}=(X_t-X_{t-1})-(X_{t-1}-X_{t-2})\] - In practice, we seldom need to go beyond second order differencing.

Seasonal differencing

• differencing between an observation and the corresponding observation from the previous year.

\[\nabla_mX_t=X_t-X_{t-m}=(1-B^m)X_t\] where \((m)\) is the number of seasons. For monthly, \((m=12)\), for quarterly \((m=4)\).

For monthly series

\[\nabla_{12}X_t=X_t-X_{t-12}\]

Twice-differenced series

\[\nabla^2_{12}X_t=\nabla_{12}X_t-\nabla_{12}X_{t-1}\] \[\nabla_{12}X_t-\nabla_{12}X_{t-1}=(X_t-X_{t-12})-(X_{t-1}-X_{t-13})\] If seasonality is strong, the seasonal differencing should be done first.

Linear filter model

A linear filter is an operation \(L\) which transform the white noise process into another time series \({X_t}\).

White noise \((\epsilon_t)\) \(\to\) Linear Filter \((\psi(B))\) \(\to\) Output \((X_t)\)

Autoregressive models

current value = linear combination of past values + current error

An autoregressive model of order \(p\), \(AR(P)\) model can be written as

\[x_t= c+ \phi_1x_{t-1}+\phi_2x_{t-2}+...+\phi_px_{t-p}+\epsilon_t,\] where \(\epsilon_t\) is white noise.

• Similar to multiple linear regression model but with lagged values of \((x_t)\) as predictors.

Question

Show that \((AR(P)\) is a linear filter with transfer function \(\phi^{-1}(B)\), where \((\phi(B)=1-\phi_1B-\phi_2B^2-...-\phi_pB^p.)\)

Stationary condition for AR(P)

The roots of \((\phi(B)=0)\) (characteristic equation) must lie outside the unit circle.

Stationarity Conditions for AR(1)

Let’s consider \((AR(1))\) process,

\[x_t=\phi_1x_{t-1}+\epsilon_t.\]

Then,

\[(1-\phi_1 B)x_t=\epsilon_t.\]

This may be written as, \[x_t=(1-\phi_1 B)^{-1}\epsilon_t=\sum_{j=0}^{\infty}\phi_1^j\epsilon_{t-j}.\]

Hence,

\[\psi(B)=(1-\phi_1 B)^{-1}=\sum_{j=0}^{\infty}\phi_1^jB^j\]

If \((\phi_1| < 1,)\) the \((AR(1))\) process is stationary.

This is equivalent to saying the roots of \((1-\phi_1B=0)\) must lie outside the unit circle.

Geometric series

\[a+ar+ar^2+ar^3+...=\frac{a}{1-r}\]

for \((|r| < 1.)\)

AR(1) process

\[x_t=c+\phi_1x_{t-1}+\epsilon_t,\]

• when \(\phi_1=0\) - equivalent to white noise process

• when \(\phi_1=1\) and \(c=0\) - random walk

• when \(\phi_1=1\) and \(c \neq 0\) - random walk with drift

• when \(\phi_1 < 0\) - oscillates around the mean

slides: https://tsforecasting-thiyanga.netlify.app/slides/timeseries2.html#27