gorold commited on
Commit
76e2adf
1 Parent(s): 1a9d410

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +138 -0
README.md CHANGED
@@ -1,3 +1,141 @@
1
  ---
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+
5
+ # Moirai-R-Small
6
+
7
+ Moirai, the Masked Encoder-based Universal Time Series Forecasting Transformer is a Large Time Series Model pre-trained on [LOTSA data](https://huggingface.co/datasets/Salesforce/lotsa_data).
8
+ For more details on the Moirai architecture, training, and results, please refer to the [paper](https://arxiv.org/abs/2402.02592).
9
+
10
+ <p align="center">
11
+ <img src="figures/architecture.png" width="100%">
12
+ <br />
13
+ <span>
14
+ Fig. 1: Overall architecture of Moirai. Visualized is a 3-variate time series, where variates 0 and 1 are target variables (i.e. to be forecasted, and variate 2 is a dynamic covariate (values in forecast horizon known). Based on a patch size of 64, each variate is patchified into 3 tokens. The patch embeddings along with sequence and variate id are fed into the Transformer. The shaded patches represent the forecast horizon to be forecasted, whose corresponding output representations are mapped into the mixture distribution parameters.
15
+ </span>
16
+ </p>
17
+
18
+ ## Usage
19
+
20
+ To perform inference with Moirai, install the uni2ts library from our [GitHub repo](https://github.com/SalesforceAIResearch/uni2ts).
21
+
22
+ 1. Clone repository:
23
+ ```shell
24
+ git clone https://github.com/SalesforceAIResearch/uni2ts.git
25
+ cd uni2ts
26
+ ```
27
+
28
+ 2) Create virtual environment:
29
+ ```shell
30
+ virtualenv venv
31
+ . venv/bin/activate
32
+ ```
33
+
34
+ 3) Build from source:
35
+ ```shell
36
+ pip install -e '.[notebook]'
37
+ ```
38
+
39
+ 4) Create a `.env` file:
40
+ ```shell
41
+ touch .env
42
+ ```
43
+
44
+ A simple example to get started:
45
+
46
+ ```python
47
+ import torch
48
+ import pandas as pd
49
+ from gluonts.dataset.pandas import PandasDataset
50
+ from gluonts.dataset.split import split
51
+ from huggingface_hub import hf_hub_download
52
+
53
+ from uni2ts.eval_util.plot import plot_single
54
+ from uni2ts.model.moirai import MoiraiForecast
55
+
56
+
57
+ SIZE = "small" # model size: choose from {'small', 'base', 'large'}
58
+ PDT = 20 # prediction length: any positive integer
59
+ CTX = 200 # context length: any positive integer
60
+ PSZ = "auto" # patch size: choose from {"auto", 8, 16, 32, 64, 128}
61
+ BSZ = 32 # batch size: any positive integer
62
+ TEST = 100 # test set length: any positive integer
63
+
64
+ # Read data into pandas DataFrame
65
+ url = (
66
+ "https://gist.githubusercontent.com/rsnirwan/c8c8654a98350fadd229b00167174ec4"
67
+ "/raw/a42101c7786d4bc7695228a0f2c8cea41340e18f/ts_wide.csv"
68
+ )
69
+ df = pd.read_csv(url, index_col=0, parse_dates=True)
70
+
71
+ # Convert into GluonTS dataset
72
+ ds = PandasDataset(dict(df))
73
+
74
+ # Split into train/test set
75
+ train, test_template = split(
76
+ ds, offset=-TEST
77
+ ) # assign last TEST time steps as test set
78
+
79
+ # Construct rolling window evaluation
80
+ test_data = test_template.generate_instances(
81
+ prediction_length=PDT, # number of time steps for each prediction
82
+ windows=TEST // PDT, # number of windows in rolling window evaluation
83
+ distance=PDT, # number of time steps between each window - distance=PDT for non-overlapping windows
84
+ )
85
+
86
+ # Prepare pre-trained model by downloading model weights from huggingface hub
87
+ model = MoiraiForecast.load_from_checkpoint(
88
+ checkpoint_path=hf_hub_download(
89
+ repo_id=f"Salesforce/moirai-R-{SIZE}", filename="model.ckpt"
90
+ ),
91
+ prediction_length=PDT,
92
+ context_length=CTX,
93
+ patch_size=PSZ,
94
+ num_samples=100,
95
+ target_dim=1,
96
+ feat_dynamic_real_dim=ds.num_feat_dynamic_real,
97
+ past_feat_dynamic_real_dim=ds.num_past_feat_dynamic_real,
98
+ map_location="cuda:0" if torch.cuda.is_available() else "cpu",
99
+ )
100
+
101
+ predictor = model.create_predictor(batch_size=BSZ)
102
+ forecasts = predictor.predict(test_data.input)
103
+
104
+ input_it = iter(test_data.input)
105
+ label_it = iter(test_data.label)
106
+ forecast_it = iter(forecasts)
107
+
108
+ inp = next(input_it)
109
+ label = next(label_it)
110
+ forecast = next(forecast_it)
111
+
112
+ plot_single(
113
+ inp,
114
+ label,
115
+ forecast,
116
+ context_length=200,
117
+ name="pred",
118
+ show_label=True,
119
+ )
120
+ ```
121
+
122
+ ## The Moirai Family
123
+
124
+ | # Model | # Parameters |
125
+ | :---: | :---: |
126
+ | [Moirai-R-Small](https://huggingface.co/Salesforce/moirai-R-small) | 14m |
127
+ | [Moirai-R-Base](https://huggingface.co/Salesforce/moirai-R-base) | 91m |
128
+ | [Moirai-R-Large](https://huggingface.co/Salesforce/moirai-R-large) | 311m |
129
+
130
+ ## Citation
131
+
132
+ If you're using Uni2TS in your research or applications, please cite it using this BibTeX:
133
+
134
+ ```markdown
135
+ @article{woo2024unified,
136
+ title={Unified Training of Universal Time Series Forecasting Transformers},
137
+ author={Woo, Gerald and Liu, Chenghao and Kumar, Akshat and Xiong, Caiming and Savarese, Silvio and Sahoo, Doyen},
138
+ journal={arXiv preprint arXiv:2402.02592},
139
+ year={2024}
140
+ }
141
+ ```