Time Series Analysis and Forecasting

Guide to AI and Data Courses · Part IV: Data
Trend and seasonStationarityARIMA and smoothingDeep forecastingEvaluationNeighboring courses
Sercan KÜLCÜ · Giresun University · Computer Engineering

Introduction

What Question Does It Ask?

How do we find structure in measurements over time and forecast the future?

Time series analysis studies measurements whose order matters: yesterday's value affects today's. The course teaches how to split a series into trend, seasonality and noise, how to model autocorrelation, and how to forecast the future together with its uncertainty.

At a glance

Approach

time-dependent statistical models

Input

timestamped measurements

Output

forecast, trend, seasonality

Success measure

forecast error (MAE, RMSE)

Math load

medium-high

Data needs

long history

Introduction

A Short History

1927

Yule explains sunspot numbers with an autoregressive (AR) model.

1960

The Kalman filter is published: it estimates a hidden state step by step from noisy measurements.

1970

Box and Jenkins's book establishes ARIMA models and how to build them.

1982

Engle proposes the ARCH model, in which volatility changes over time.

1997

Hochreiter and Schmidhuber introduce the LSTM; long dependencies become learnable.

2017

Facebook open-sources Prophet, which separates seasonality and holidays.

The Course

Core Concepts

Trend and seasonality

Long-run direction and regularly repeating cycles.

Stationarity

Mean and variance do not change over time.

Autocorrelation (ACF)

Correlation of a series with its own past.

Differencing

Removing the trend to make a series stationary.

ARIMA

Autoregression, differencing and moving average together.

Exponential smoothing

A forecast that weights the recent past more.

Prediction interval

A forecast with its uncertainty, not a single number.

Time-based split

The test set always comes after training; no shuffling.

The Course

A Typical Workflow

Visualizeplots,decompositionTransformdifferencing,logarithmChoose modelACF, PACF,AICRollingvalidationtime-basedsplitForecastpoint andinterval
Example: A power company plots hourly consumption, separates the daily and weekly cycles, builds a model that also takes temperature as input, backtests it on past weeks, and gives the next day's load with a 95% prediction interval.

The Course

What Does the Course Cover?

  1. Components of a series: trend, seasonality, noise
  2. Visualization and STL decomposition
  3. Stationarity, differencing and unit root tests
  4. Autocorrelation and partial autocorrelation
  5. Exponential smoothing and Holt-Winters
  6. AR, MA and ARMA models
  7. ARIMA and seasonal ARIMA (SARIMA)
  8. Regression with external regressors, dynamic models
  9. Forecast evaluation and rolling-origin validation
  10. Volatility models: ARCH and GARCH
  11. State space models and the Kalman filter
  12. Forecasting with machine learning and deep nets

Skills you will gain

date-indexed data in pandasfitting ARIMA with statsmodelsProphet and sktimedesigning rolling-origin validationproducing and reading prediction intervals

A typical semester; the order and coverage vary between universities.

Try It

Planning Next Summer's Ice Cream

An ice cream maker plans the coming months' production from three years of monthly sales. Pick a method and a horizon; the last year's real sales are held out and the forecast is scored on them.

Naive and moving average forecasts cannot see the season: they draw a flat line through summer. Seasonal methods carry last year's shape forward; Holt-Winters also adds the growth trend.

Context

Prerequisites

Courses worth knowing before this one, and why they are needed.

CourseWhy it is needed
Probability and Statisticsstationarity, autocorrelation, ARIMA
Calculusseries, derivatives and integrals: the language of models and filters
Programmingreading files, handling dates and plotting in Python

Context

Are You Ready for This Course?

Ask yourself; for every "no", the right-hand column shows where to start.

Ask yourselfIf not, start with
Can you explain mean, variance and the correlation coefficient?Probability and Statistics
Can you fit a linear regression and interpret its coefficients?Machine Learning
Can you compute the sum of a geometric series?Sequences and series from calculus
Can you read a CSV file with a date column in Python and plot it?An intro programming course, then a short pandas tutorial
If you can say "yes" to most of these, you are ready; one or two gaps can be closed during the semester.

Context

Neighbouring Courses

subfieldbuilds onshares methodsapplication

Overlap

Overlapping Courses

RelationCourseShared ground
builds onThis course builds on Probability and Statisticsstationarity, autocorrelation, ARIMA
shares methodsShares methods with Deep Learningrecurrent networks, attention-based forecasting
shares methodsShares methods with Digital Signal Processingfrequency analysis, filtering
shares methodsShares methods with Remote Sensingmultitemporal image series

Differences

Where It Differs (1/2)

CriterionTime SeriesProbability & StatisticsSignal Processing
Core questionHow do we find structure in measurements over time and forecast the future?How is uncertainty measured, and how do we draw reliable conclusions from limited data?How are sampled signals analyzed, filtered and transformed?
Approachtime-dependent statistical modelsprobability models and inferencemathematical transforms and filters
Inputtimestamped measurementssample datasampled signal
Outputforecast, trend, seasonalityestimate, confidence interval, test resultfiltered signal, spectrum
Success measureforecast error (MAE, RMSE)unbiasedness, power, significancesignal-to-noise ratio, frequency response
Math loadmedium-highthe theory itselfhigh: complex numbers, Fourier
Data needslong historya sampleno learning; the signal suffices
  • Probability & Statistics: Statistics usually assumes independent observations; this course deals with ordered data whose observations depend on each other.
  • Signal Processing: Signal processing filters a signal and looks at its frequencies; this course uses the same tools to forecast the future.

Differences

Where It Differs (2/2)

CriterionTime SeriesDeep LearningMachine Learning
Core questionHow do we find structure in measurements over time and forecast the future?How do many-layered networks learn useful representations from raw data?How does a program generalize from examples?
Approachtime-dependent statistical modelsmultilayer representation learningstatistical learning
Inputtimestamped measurementsraw data: images, text, audiolabeled or unlabeled examples
Outputforecast, trend, seasonalitylearned representation and predictiona model that makes predictions
Success measureforecast error (MAE, RMSE)accuracy, loss, scalingerror on unseen data
Math loadmedium-highhigh: derivatives, matrices, probabilityhigh: linear algebra, probability, optimization
Data needslong historylarge; needs GPUsmoderate to large
  • Deep Learning: Deep learning offers a powerful family of forecasting models; this course also teaches when a simple model is enough.
  • Machine Learning: ML shuffles examples and splits them at random; here time order is kept and the model is trained only on the past.

Practice

Applications

Demand forecasting

Planning sales, stock and production.

Energy load

Power consumption and renewable output.

Finance

Volatility, risk and return models.

Weather and climate

Temperature, rainfall and CO₂ series.

Anomaly detection

Deviations in server and sensor metrics.

Economics

Forecasting inflation, unemployment and growth.

Practice

Risks and Ethics

Leaking the future

A random split, or scaling with statistics of the whole series, carries future values into training; test scores become fake.

Structural breaks

In the 2020 pandemic, demand and traffic series changed overnight; models that fit the past well were wrong for weeks.

Forecasts without intervals

A single number suggests certainty; stock or energy decisions get made without knowing the uncertainty.

Forecasts that change behaviour

When everyone acts on the same price or demand forecast, the series moves; a forecast can fulfil or defeat itself.

In depth: Explainable AI and AI Ethics.

Practice

Tools and Datasets

Tools and libraries

pandasstatsmodelsProphetsktimeDartsR forecast / fable

Well-known datasets

Airline passengers (Box-Jenkins)Sunspot numbersM4 and M5 competitionsUCI Electricity Load DiagramsMauna Loa CO₂ measurementsJena climate data

Practice

Common Misconceptions

Common beliefIn fact
Time series data can be tested with a random split.Future values leak into training; testing always follows time order.
Deep networks beat classic models on every series.In the M competitions, simple and statistical methods have often been very strong.
A good model gives one correct number.Forecasts are uncertain; for decisions the prediction interval matters at least as much as the point forecast.
A model that worked in the past always works.Structural breaks (a pandemic, a crisis) change the series; models must be monitored and refreshed.

Next Steps

After This Course

Courses that build on or extend this one

This course is mostly an end point: it builds on the others.

Related careers

Forecasting analystData scientistQuantitative analystDemand plannerEnergy data analyst
After this course you can move on to frequency analysis with signal processing, transformer-based forecasting with deep learning, and multi-temporal satellite images with remote sensing. Econometrics and state space models are natural next steps too.

Next Steps

Further Reading

  • Hyndman and Athanasopoulos, Forecasting: Principles and Practice, 3rd ed., OTexts, 2021.
  • Box, Jenkins, Reinsel and Ljung, Time Series Analysis: Forecasting and Control, 5th ed., Wiley, 2015.
  • Shumway and Stoffer, Time Series Analysis and Its Applications, 4th ed., Springer, 2017.
  • Brockwell and Davis, Introduction to Time Series and Forecasting, 3rd ed., Springer, 2016.

The End

Chapter 29: Time Series Analysis and Forecasting
← Previous chapter · Map of the field · Next chapter →
1 / 1 Sercan KÜLCÜ. All rights reserved.