Skip to content

Moirai forecasting model

Moirai forecasting model

Since a while Large Language Models have been taking over the modeling scene. Among the classical NLP related ones, those dedicated for time series processing started to emerge. Moirai model is one of those. It is a foundational language model for time series forecasting developed by Salesforce. It is a Masked Encoder-based Universal Time Series Forecasting Transformer model pre-trained on LOTSA data.

You may find online a few tutorials showing how to interact with MOIRAI, however I still struggled to understand what is happening on different steps. Let me therefore break it down for you.

Contents

Dataset

MOIRAI model operates on PandasDataset structure from Gluonts package. It can be easily created from a regular pandas data frame, providing that it contains identifier column for your time series (cause more than one series can be ingested into the object).

For the demonstration purposes let’s use airpassangers dataset which can be downloaded i.e. from Kaggle. I changed column names to “date” and “value”.

 

Now let’s add artificial id column:

Then we need to set data frame index using date column:

And now we can create our data object:

 

Model setup

Now having the data ready, we can move to modeling stage. I will show how to perform train-testing modeling as well as prediction forward.

Let’s start by defining model parameters.

Let’s consider prediction horizon of 3.

Model weights are downloaded from huggingface hub.

Within we provide information about the number of exogenous features, that can be included in the data in the form of additional columns.

 

Train testing forecast

To perform train testing prediction we need to specify the testing set. Contrary to regular train-testing splits, were we cut the series at some point and use training part for model training and testing set for evaluation – we do here zero shot learning. Which basically means, that we enhance part of the series by providing prediction windows.

Let’s say that we want to perform testing using crossvalidation – for that let’s mark our test set to have 12 data points. With our prediction horizon of 3, it gives us 4 prediction windows.

To understand what happens, we need to look into gluonts documentation.

Created training set is exactly the same as our full dataset.

As we can see, we have monthly series starting on Jan 1949, containing values from 112 to 432.

Test dataset contains 4 elements: repeated training data, enhanced by specification of our four 3-elements prediction windows: starting on Jan, Apr, Jun and Sep 1960.

However to obtain prediction, we provide only the “input” part of the testing set, which simply is a training set repeated 4 times, with last observations removed, but with additional metadata.

Now when running the training prediction, we obtain four 3-element predictions, each sampled 20 times, as we specified during model fetching. This nicely simulates crossvalidation scenario.

“samples” property allows you to access those values to calculate average predictions, or required percentiles.

 

Prediction ahead

To perform forecast ahead, we do not need a testing dataset. And as in previous case, we perform prediction using the proper part of the training set sample, we now provide the full sample instead.

What we get is a 3 points prediction for Jan-Mar 1961, each sampled 20 times.

And this is how this model works.

«

Leave a Reply

Your email address will not be published. Required fields are marked *