Time Series Clustering
Is the following series dissimilar or similar?
What is time series clustering?
Group similar time series
What we meant by similar?
similar profiles
similar features
same best forecasting models
similar data generating process
Before clustering think what is the purpose of grouping?
Source: Montero, P., & Vilar, J. A. (2015). TSclust: An R package for time series clustering. Journal of Statistical Software, 62, 1-43.
Time Series Clustering
Model-free approaches
Model-based approaches
Complexity-based approaches
Prediction-based approaches
Model-free approaches
XT=(X1,X2,...,XT)T
YT=(Y1,Y2,...,YT)T
Minkowski distance
dLq(XT,YT)=[T∑t=1(Xt,Yt)q](1q)
When q=1 - Manhattan distance
When q=2 - Euclidean distance
Observations are treated as independent. (dLq is invariant to permutation over time)
Very sensitive to time scaling
Frechet distance
Source: https://pure.tue.nl/ws/portalfiles/portal/93882810/Thesis_Tom_van_Diggelen.pdf
Source: https://pure.tue.nl/ws/portalfiles/portal/93882810/Thesis_Tom_van_Diggelen.pdf
Frechet distance
The standard Fréchet distance is the minimum leash length required for the person to walk the dog without backtracking.
Account the ordering of the observations
Can be computed on series of different lengths
Dynamic time warping distance (DTW)
https://www.youtube.com/watch?v=ERKDHZyZDwA
Correlation-based distance
Pearson’s correlation between X_T and Y_T: Cor(XT,YT)
dcor1=√2(1−Cor(XT,YT))
dcor2=((1−Cor(XT,YT)1+Cor(XT,YT))β)12
where β≥0.
Autocorrelation-based distance
Distance is based on estimated autocorrelation function.
Case 1: Uniform weight
d(XT,YT)=√L∑i=1(ˆρi,X−ˆρi,Y)2
Case 2: Geometric weights decaying with autocorrelation lag
d(XT,YT)=√L∑i=1w(1−w)i(ˆρi,X−ˆρi,Y)2
Periodogram-based distance
A periodogram is used to identify the dominant periods (or frequencies) of a time series.
Euclidean distance between the periodogram ordinates (Gives more weight to the shape of the curve)
Euclidean distance between the normalized periodogram ordinates (Consider the scale)
Reading: https://online.stat.psu.edu/stat510/lesson/6/6.1
Other model-free measures
Dissimilarity measures based on nonparametric spectral estimators
Dissimilarity measure based on wavelet transformation
Dissimilarity measure based on the symbolicrepresentation SAX
Model-based approaches
Piccolo distance
Invertible ARIMA processes
Euclidean distance between the AR(∞)
In-class explanations.
Invertible condition
Backshift notation:
BXt=Xt−1 ARMA(p, q)
xt=c+ϕ1xt−1+ϕ2xt−2+...+ϕpxt−p+θ1ϵt−1+θ2ϵt−2+...+θqϵt−q+ϵt,
xt=c+ϕ1Bxt+ϕ2B2xt+...+ϕpBpxt+θ1Bϵt+θ2B2ϵt+...+θqBqϵt+ϵt, xt−ϕ1Bxt−ϕ2B2xt−...−ϕpBpxt=c+θ1Bϵt+θ2B2ϵt+...+θqBqϵt+ϵt,
Φ(B)xt=c+Θ(B)ϵt
Φ(B)xt=c+Θ(B)ϵt
Φ(B)=1−ϕ1B−ϕ2B2−...−ϕpBp
Θ(B)=1+θ1B+θ2B2t+...+θqBq
This process is called invertible if the modulus of all the roots of Θ(B)=0 are greater than one.
Demo
https://github.com/thiyangt/AR-infinite-coefficients/blob/master/ARinfinite-code.md
Maharaj distance
For the class of invertible and stationary ARMA processes
Based on hypotheses testing to determine whether or not two time series have significantly different generating processes
Two dissimilarity measures: one is based on test statistics, other one is based on the associated p-value
Prediction-based approaches
Two time series are similar if their forecasts at s specific future time are close.
Feature-based time series clustering
Link to slides: https://thiyangt.github.io/whyR2021keynote/#1
tsfeatures package in Python: https://pypi.org/project/tsfeatures/
Data
Additive and multiplicative seasonality – can you identify them correctly? https://kourentzes.com/forecasting/2014/11/09/additive-and-multiplicative-seasonality/
Time series features
Talagala, T. S., Hyndman, R. J., & Athanasopoulos, G. (2018). Meta-learning how to forecast time series. Monash Econometrics and Business Statistics Working Papers, 6(18), 16.
https://www.monash.edu/business/ebs/research/publications/ebs/wp06-2018.pdf
Complexity-based approaches
Compression-based dissimilarity measures
Permutation distribution clustering
Complexity-invariant dissimilarity measure