Preface

Mankind has always sought the ability to predict the future. Since the earliest civilizations, people have tried to predict the future. Shamans, oracles, and prophets used anything ranging from astrology and palmistry to numerology to satisfy the human need to see into the future. In the last century, with the developments in IT, the mantle of predicting the future landed on data analysts and data scientists. And how do we predict the future? It’s not by examining the lines and creases on our hands or the positions of the stars anymore but by using data that has been generated in the past. And instead of prophecies, we now have forecasts.

Time, being the fourth dimension in our world, makes all the data generated in the world time series data. All the data that is generated in the real world has an element of time associated with it. Whether the temporal aspect is relevant to the problem or not is another question altogether. However, to be more concrete and immediate, we can find time series forecasting use cases in many industries, such as retail, energy, healthcare, and finance. We might want to know how many units of a particular product are to be dispatched to a particular store, or we might want to know how much electricity is to be produced to meet demand.

In this book, using a real-world dataset, you will learn how to handle and visualize time series data using pandas and plotly, generate baseline forecasts using darts, and use machine learning and deep learning for forecasting, using popular Python libraries such as scikit-learn and PyTorch. We conclude the book with a few chapters that cover seldom-touched aspects, such as multi-step forecasting, forecast metrics and cross validation for time series.

The book will enable you to build real-world time series forecasting systems that scale to millions of time series by mastering and applying modern concepts in machine learning and deep learning.

Who this book is for

The book is ideal for data scientists, data analysts, machine learning engineers, and Python developers who want to build industry-ready time series models. Since the book explains most concepts from the ground up, basic proficiency in Python is all you need. A prior understanding of machine learning or forecasting would help speed up the learning. For seasoned practitioners in machine learning and forecasting, the book has a lot to offer in terms of advanced techniques and traversing the latest research frontiers in time series forecasting.

What this book covers

Part 1—Getting Familiar with Time Series

Chapter 1, Introducing Time Series, is all about introducing you to the world of time series. We lay down a definition of time series and talk about how it is related to a Data Generating Process (DGP). We will also talk about the limits of forecasting and talk about what we cannot forecast, and then we finish off the chapter by laying down some terminology that will help you understand the rest of the book.

Chapter 2, Acquiring and Processing Time Series Data, covers how you can process time series data. You will understand how different forms of time series data can be represented in a tabular form. You will learn different date-time-related functionalities in pandas and learn how to fill in missing data using techniques suited for time series. Finally, using a real-world dataset, you will go through a step-by-step journey in processing time series data using pandas.

Chapter 3, Analyzing and Visualizing Time Series Data, furthers your introduction to time series by learning how to visualize and analyze time series. You will learn different visualizations that are commonly used for time series data and then learn how to go one level deeper by decomposing time series into its components. To wrap it up, you will also look at ways to identify and treat outliers in time series data.

Chapter 4, Setting a Strong Baseline Forecast, gets right to the topic of time series forecasting as we use tried and tested methods from econometrics, such as ARIMA and exponential smoothing, to generate strong baselines. These efficient forecasting methods will provide strong baselines so that we can go beyond these classical techniques and learn modern techniques, such as machine learning. You will also get an introduction to another key topic—assessing forecastability using techniques such as spectral entropy and coefficient of variation.

Part 2—Machine Learning for Time Series

Chapter 5, Time Series Forecasting as Regression, starts our journey into using machine learning for forecasting. A short introduction to machine learning lays down the foundations of what is to come in the next chapters. You will also understand, conceptually, how we can cast a time series problem as a regression problem so that we can use machine learning for it. To close off the chapter, we tease you with the possibility of global forecasting models.

Chapter 6, Feature Engineering for Time Series Forecasting, shifts gear into a more practical lesson. Using a real-world dataset, you will learn about different feature engineering techniques, such as lag features, rolling features, and Fourier terms, which help us formulate a time series problem as a regression problem.

Chapter 7, Target Transformations for Time Series Forecasting, continues the practice of exploring different target transformations to accommodate non-stationarity in time series. You will learn techniques such as the augmented Dickey–Fuller test and Mann–Kendall test to identify and treat non-stationarity.

Chapter 8, Forecasting Time Series with Machine Learning Models, continues from where the last chapter left off to start training machine learning models on the dataset we have been working on. Using the standard code framework present in the book, you will train models such as linear regression, random forest, and gradient-boosted decision trees on our dataset.

Chapter 9, Ensembling and Stacking, takes a step back and explores how we can use multiple forecasts and combine them to create a better forecast. You will explore popular techniques such as best fit, different versions of the hill-climbing algorithm, simulated annealing, and stacking to combine the different forecasts we have generated to get a better one.

Chapter 10, Global Forecasting Models, concludes your guided journey into machine learning-enabled forecasting to an exciting and new paradigm—global forecasting models. You will learn how to use global forecasting models and industry-proven techniques to improve their performance, which finally lets you develop scalable and efficient machine learning forecasting systems for thousands of time series.

Part 3—Deep Learning for Time Series

Chapter 11, Introduction to Deep Learning, we switch tracks and start with a specific type of machine learning—deep learning. In this chapter, we lay the foundations of deep learning by looking at different topics such as representation learning, linear transformations, activation functions, and gradient descent.

Chapter 12, Building Blocks of Deep Learning for Time Series, continues the journey into deep learning by making it specific to time series. Keeping in mind the compositionality of deep learning systems, you will learn about different building blocks with which you can construct a deep learning architecture. The chapter starts off by establishing the encoder-decoder architecture and then talks about different blocks such as feed forward networks, recurrent neural networks, and convolutional neural networks.

Chapter 13, Common Modeling Patterns for Time Series, strengthens the encoder-decoder architecture that you saw in the previous chapter by showing you a few concrete and common patterns in which you can arrange building blocks to generate forecasts. This is a hands-on chapter where you will be creating forecasts using deep learning-based tabular regression and different sequence-to-sequence models.

Chapter 14, Attention and Transformers for Time Series, covers the contemporary topic of using attention to improve deep learning models. The chapter starts off by talking about a generalized attention model with which you will learn different types of attention schemes, such as scaled dot product and additive. You will also tweak the sequence-to-sequence models from the previous chapter to include attention and then train those models to generate a forecast. The chapter then talks about transformer models, which is a deep learning architecture that relies solely on attention, and then you will use that to generate forecasts as well.

Chapter 15, Strategies for Global Deep Learning Forecasting Models, tackles yet another important aspect of deep learning-based forecasting. Although the book talked about global forecasting models earlier, there are some differences in how it is implemented for deep learning models. In this chapter, you will learn how to implement global deep learning models and techniques on how to make those models better. You will also see them working in the hands-on section, where we will be generating forecasts using the real-world dataset we have been working with.

Chapter 16, Specialized Deep Learning Architectures for Forecasting, concludes your journey into deep learning-based time series forecasting by talking about a few popular, specialized deep learning architectures for time series forecasting. Using the concepts and building blocks you have learned through the previous chapters, this chapter takes you to the cutting edge of research and exposes the leading state-of-the-art models in time series forecasting such as N-BEATS, N-HiTS, Informer, Autoformer, and Temporal Fusion Transformer. In addition to understanding them, you will also learn how to use these models to generate forecasts using a real-world dataset.

Chapter 17, Probabilistic Forecasting and More, take you into the realm of probabilistic forecasting with techniques like Conformal Prediction, Monte Carlo Dropout, Quantile Functions, and Probability Density Functions and enable you to practically implement these using popular open-source frameworks. The chapter also takes the road less travelled in time series forecasting and talk about Intermittent Forecasting, Interpretability, Cold Start Forecasting, and Hierarchical forecasting at a high level to serve as a starting point in your journey into those.

Part 4—Mechanics of Forecasting

Chapter 18, Multi-Step Forecasting, tackles the rarely talked-about but highly relevant topic of multi-step forecasting. You will learn about different strategies for generating forecasts for more than one time step into the future, such as Recursive, Direct, DirRec, RecJoint, and Rectify. The book also talks about the merits and demerits of each of them and helps you choose the right strategy for your problem.

Chapter 19, Evaluating Forecast Errors—A Survey of Forecast Metrics, traverses yet another topic that is rarely talked about and rife with controversy, with many opinions from different quarters. You will learn about different ways to measure the goodness of a forecast and through experiments, which you can run, expose the strengths and weaknesses of different metrics. The chapter concludes by laying down some guidelines that can help you choose the correct metric for your problem.

Chapter 20, Evaluating Forecasts—Validation Strategies, concludes the evaluation of forecasts and the book by talking about different validation strategies we can use for time series. You will learn different validation strategies such as hold-out, cross-validation, and their variations. The chapter also touches upon aspects to keep in mind while designing validation strategies for global settings as well. At the conclusion of the chapter, you will come across a few guidelines for choosing your validation strategies and answers to questions such as can we use cross-validation for time series?

To get the most out of this book

The book has to be considered bundled with the notebooks and associated code base in GitHub and is best used when both are used together. There are three levels of learning that you can do with the book. The first one is enabled by the text in the book alone and it will take you through the theory, build you intuitions and go through basic codes to get something implemented fast. To solidify the learning, we recommend you take the next step and use the provided notebooks and experiment with them. In most notebooks, we show how to do one thing in a particular way. But there are many levers, like hyperparameters, that you can tweak and run to understand how the output changes with these changes. This level of learning will make you understand the code and have a deeper understanding of the concepts you learning from the book.

And lastly, we have abstracted some code into src folder in the repository which is used in the notebooks. Your last and final level of learning is to go through and understand those so that you know how it works under the hood. Most of the code is well commented so that its easier for you to understand what’s happening under the hood of the functions and classes you used in the notebooks. This will elevate your learning to a level where you can confidently apply these techniques to other use-cases like a boss.

You should have basic familiarity with Python programming, as the entire code that we use for the practical sections is in Python. Familiarity with major libraries in Python, such as pandas and scikit-learn, are not essential (because the book covers some basics) but will help you get through the book much faster. Familiarity with PyTorch, the framework the book uses for deep learning, is also not essential but would accelerate your learning. Any of the software requirements shouldn’t stop you because, in today’s internet-enabled world, the only thing that is standing between you and a world of knowledge is the search bar in your favorite search engine.

Setting up an environment

Setting up an environment, preferably a separate one, for the book is highly recommended. There are two main ways we suggest to create the environment—Anaconda/Mamba or a Python virtual environment.

Using Anaconda/Miniconda/Mamba

The easiest way to set up an environment is by using Anaconda, a distribution of Python for scientific computing. You can use Miniconda, a minimal installer for Conda, as well if you do not want the pre-installed packages that come with Anaconda. And you can also use Mamba, a reimplementation of the conda package manager in C++. It is much faster than conda and is a drop-in replacement for conda. Mamba is the recommended way because it has much less chances of getting stuck at the dreaded Resolving dependencies… screen in Anaconda. If you are using Anaconda version 23.10 or above, then you need not worry about Mamba that much because the fast and efficient package resolver is part of anaconda by default.

1. Install Anaconda/Miniconda/Mamba/MicroMamba: Anaconda can be installed from https://www.anaconda.com/products/distribution. Depending on your operating system, choose the corresponding file and follow the instructions. Alternatively, you can install Miniconda from here: https://docs.anaconda.com/miniconda/. You can install Mamba and MicroMamba from here: https://mamba.readthedocs.io/en/latest/. If you are using Mamba, in all the instructions below replace “conda" with “mamba".
2. Open conda prompt: To open Anaconda Prompt (or Terminal on Linux or macOS), do the following:
   • Windows: Open the Anaconda Prompt (Start | Anaconda Prompt)
   • macOS: Open Launchpad and then open Terminal. Type conda activate.
   • Linux: Open Terminal. Type conda activate.
3. Create a new environment: Use the following command to create a new environment of your choice. For instance, to create an environment named modern_ts_2E with Python 3.10 (recommended to use 3.10 or above), use the following command:

   conda create -n modern_ts_2E python=3.10
   

4. Activate the environment: Use the following command to activate the environment:

   conda activate modern_ts_2E
   

5. Install PyTorch from the official website: PyTorch is best installed from the official website. Go to https://pytorch.org/get-started/locally/ and select the appropriate options for your system. You can replace conda with mamba if you want to use Mamba to install.
6. Navigate to the downloaded code: Use operating systeg-specific commands to navigate to the folder where you have downloaded the code. For instance, in Windows, use cd.
7. Install the required libraries: Use the provided anaconda_env.yml file to install all the required libraries. Use the following command:

   conda env update --file anaconda_env.yml
   

This will install all the required libraries in the environment. This can take a while.

8. Checking the installation: We can check if all the libraries required for the book is installed properly by executing a script in the downloaded code folder, python test_installation.py. If the GPU is not showing up, install PyTorch again on top of the environment.
9. Activating the environment and Running Notebooks: Every time you want to run the notebooks, first activate the environment using the conda activate modern_ts_2E command and then use Jupyter Notebook (jupyter notebook) or Jupyter Lab (jupyter lab) according to your preference.

Using Python Virtual environments and pip

If you prefer to stick to native Python for environment management, we have provided an alternate requirements.txt as well, which should help you with that:

1. Install Python: You can download Python from https://www.python.org/downloads/. Recommended to use 3.10 or above.
2. Create a virtual environment: Use the following command to create a virtual environment named modern_ts_2E:

   python -m venv modern_ts_2E
   

3. Activate the environment: Use the following command to activate the environment:
   • Windows: modern_ts_2E\Scripts\activate
   • macOS/Linux: source modern_ts_2E/bin/activate
4. Install PyTorch: PyTorch is best installed from the official website. Go to https://pytorch.org/get-started/locally/ and select the appropriate options for your system.
5. Navigate to the downloaded code: Use operating-system-specific commands to navigate to the folder where you have downloaded the code. For instance, in Windows, use cd.
6. Install the required libraries: Use the provided requirements.txt file to install all the required libraries. Use the following command:

   pip install -r requirements.txt
   

This will install all the required libraries in the environment. This can take a while.

7. Checking the installation: We can check if all the libraries required for the book is installed properly by executing a script in the downloaded code folder

   python test_installation.py
   

8. Activating the environment and Running Notebooks: Every time you want to run the notebooks, first activate the environment using the command modern_ts_2E\Scripts\activate (Windows) or source modern_ts_2E/bin/activate (macOS/Linux) and then use Jupyter Notebook (jupyter notebook) or Jupyter Lab (jupyter lab) according to your preference.

What to do when environment creation throws an error?

Considering the wide variety of computers around the world and ever-changing library dependencies, there is a very real chance that the environment setup we are providing with the book (which is tested and working as of Sept 2024) may not stand the test of time. In such cases, we recommend you open the requirements.txt and see which libraries we are installing and try to figure out if downgrading or upgrading versions will help you along.

Another way to tackle this will be to just install the packages that you need for a notebook when you are using those. Mostly conflicts occur when we try to install different libraries into a same environment. For instance, one of the libraries needs PyTorch < 1.0.0, but some other library needs PyTorch > 1.0.0. In such cases, it makes sense to separate the two libraries into two environments or find another version of the library that is compatible with others.

Your best friend in such situations is a Google search and subsequent trawling of GitHub comments that complain of the same issue. As a last resort, you can also raise an issue in the book repository, and we can try to help you out.

Download the data

You are going to be using a single dataset throughout the book. The book uses London Smart Meters dataset from Kaggle for this purpose. Many of the notebooks from early chapters are dependencies for some of the later chapters. As such, to remove this dependency if you want to run the notebooks out of order, we have included a data.zip file with all the required datasets.

To set up, follow these steps:

1. Download the data from AWS: https://packt-modern-time-series-py.s3.eu-west-1.amazonaws.com/data.zip.
2. Unzip the content.
3. Copy over the data folder to the Modern-Time-Series-Forecasting-with-Python-2E folder you pull from GitHub.

That’s it! You are now ready to start running the code.

If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book’s GitHub repository. Doing so will help you avoid any potential errors related to the copying and pasting of code.

The code that is provided along with the book is in no way a library but more of a guide for you to start experimenting on. The amount of learning you can derive from the book and code is directly proportional to how much you experiment with the code and stray outside your comfort zone. So, go ahead and start experimenting and putting the skills you pick up in the book to good use.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Modern-Time-Series-Forecasting-with-Python-2E. If there’s an update to the code, it will be updated in the GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots and diagrams used in this book. You can download it here: https://packt.link/gbp/9781835883181.

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “statsmodels.tsa.seasonal has a function called seasonal_decompose.”

A block of code is set as follows:

#Does not support missing values, so using imputed ts instead
res = seasonal_decompose(ts, period=7*48, model="additive", extrapolate_trend="freq")

Any command-line input or output is written as follows:

conda env create -f anaconda_env.yml

Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: “But if you look at the Time Elapsed column, it stands out.”

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at customercare@packtpub.com and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.