[[Attachments]]
关键信息
核心贡献
-
Autoformer as a novel decomposition architecture with an Auto-Correlation mechanism
ls-type:: annotation
hl-page:: 1
hl-color:: yellow
[[Auto-Correlation Mechanism]] 自相关机制,代替点向连接的注意力机制,实现序列级连接和较低复杂度
- 序列级别依赖发现以及表示聚合 conducts the dependencies discovery and representation aggregation at the sub-series level.
ls-type:: annotation
hl-page:: 1
hl-color:: yellow
-
Decomposition Architecture 深度分解架构,从复杂时间模式种分解出可预测性更强的成分
核心问题
相关工作
-
之前方法集中在 recurrent connections, temporal attention or causal convolution.
ls-type:: annotation
hl-page:: 2
hl-color:: blue
-
Transformer 类方法
-
Decomposition of Time Series
-
将原始时间序列分解成多个序列,新序列更容易预测
- each representing one of the underlying categories of patterns that are more predictable.
ls-type:: annotation
hl-page:: 3
hl-color:: blue
-
[[Prophet]] with trend-seasonality decomposition
-
[[N-BEATS]] with basis expansion
-
[[DeepGLO]] with matrix decomposition
-
缺点
-
简单分解限制
- limited by the plain decomposition effect of historical series
ls-type:: annotation
hl-page:: 3
hl-color:: yellow
-
忽视层次交互
-
overlooks the hierarchical interaction between the underlying patterns of series in the long-term future.
ls-type:: annotation
hl-page:: 3
hl-color:: yellow
-
预测问题未来的不可知性,通常方法先对过去序列进行分解,再分别预测,这会造成预测结果受限于分解效果,并且忽视了未来各个组分之间的相互作用。
解决方法
tags:: #[[Model Architecture]] [[Encoder-Decoder]]
+ series decomposition block
ls-type:: annotation
hl-page:: 3
hl-color:: yellow
保留周期部分
+ 序列分解成趋势项和周期项部分 separate the series into trend-cyclical and seasonal parts.
ls-type:: annotation
hl-page:: 3
hl-color:: yellow
+ 在预测过程中,模型交替进行预测结果优化和序列分解,从隐藏变量中逐步分离趋势项与周期项
+ 逐步从预测的中间隐藏变量中提取长期稳定的趋势 xtract the long-term stationary trend from predicted intermediate hidden variables progressively.
ls-type:: annotation
hl-page:: 3
hl-color:: yellow
+ 使用 [[Moving Average]] 平滑周期性、突出趋势项 smooth out periodic fluctuations and highlight the long-term trends
ls-type:: annotation
hl-page:: 3
hl-color:: yellow
+ $\mathcal{X}_{\mathrm{s}}, \mathcal{X}_{\mathrm{t}}=\operatorname{SeriesDecomp}(\mathcal{X})$
+ $\begin{aligned} & \mathcal{X}_{\mathrm{t}}=\operatorname{Avg} \operatorname{Pool}(\operatorname{Padding}(\mathcal{X})) \\ & \mathcal{X}_{\mathrm{s}}=\mathcal{X}-\mathcal{X}_{\mathrm{t}}\end{aligned}$
+ xs seasonal
+ xt trend-cyclial part
+ [[Encoder]]
+ Encoder 输入过去 I 步 $\mathcal{X}_{\mathrm{en}} \in \mathbb{R}^{I \times d}$
+ 建模周期性部分,逐步消除趋势项(在 decoder 中通过累积得到) focuses on the seasonal part modeling
ls-type:: annotation
hl-page:: 4
hl-color:: yellow
+ 当成 decoder 的交叉信息 be used as the cross information to help the decoder refine prediction results
ls-type:: annotation
hl-page:: 4
hl-color:: yellow
+ 流程
+ _ the eliminated trend part
ls-type:: annotation
hl-page:: 4
hl-color:: red
消除趋势项
+ $\begin{aligned} & \mathcal{S}_{\text {en }}^{l, 1},_{-}=\operatorname{SeriesDecomp}\left(\text { Auto-Correlation }\left(\mathcal{X}_{\text {en }}^{l-1}\right)+\mathcal{X}_{\text {en }}^{l-1}\right) \\ & \mathcal{S}_{\text {en }}^{l, 2},_{-}=\operatorname{SeriesDecomp}\left(\text { FeedForward }\left(\mathcal{S}_{\text {en }}^{l, 1}\right)+\mathcal{S}_{\text {en }}^{l, 1}\right)\end{aligned}$
+ [[Decoder]] 分解对趋势项与周期项建模
+ 一半过去信息 + 填充
+ $\begin{aligned} \mathcal{X}_{\text {ens }}, \mathcal{X}_{\text {ent }} & =\operatorname{SeriesDecomp}\left(\mathcal{X}_{\text {en } \frac{I}{2}: I}\right) \\ \mathcal{X}_{\text {des }} & =\operatorname{Concat}\left(\mathcal{X}_{\text {ens }}, \mathcal{X}_0\right) \\ \mathcal{X}_{\text {det }} & =\operatorname{Concat}\left(\mathcal{X}_{\text {ent }}, \mathcal{X}_{\text {Mean }}\right),\end{aligned}$
+ seasonal part $\mathcal{X}_{\mathrm{des}} \in \mathbb{R}^{\left(\frac{1}{2}+O\right) \times d}$
+ trend-cyclical part $\mathcal{X}_{\mathrm{det}} \in \mathbb{R}^{\left(\frac{1}{2}+O\right) \times d}$
+ 趋势-周期累积结构 the accumulation structure for trend-cyclical components
ls-type:: annotation
hl-page:: 4
hl-color:: yellow
+ 从中间隐变量提取潜在趋势,使得模型可以逐步改进趋势预测并且消除干扰信息,以便于在自相关性中发现基于周期的依赖关系。
+ 其中,对于周期项,自相关机制利用序列的周期性质,聚合不同周期中具有相似过程的子序列;
+ Note that the model extracts the potential trend from the intermediate hidden variables during the decoder, allowing Autoformer to progressively refine the trend prediction and eliminate interference information for period-based dependencies discovery in Auto-Correlation.
ls-type:: annotation
hl-page:: 4
hl-color:: red
+ the stacked Auto-Correlation mechanism for seasonal component
ls-type:: annotation
hl-page:: 4
hl-color:: yellow
+ 流程
+ $\begin{aligned} \mathcal{S}_{\mathrm{de}}^{l, 1}, \mathcal{T}_{\mathrm{de}}^{l, 1} & =\operatorname{SeriesDecomp}\left(\text { Auto-Correlation }\left(\mathcal{X}_{\mathrm{de}}^{l-1}\right)+\mathcal{X}_{\mathrm{de}}^{l-1}\right) \\ \mathcal{S}_{\mathrm{de}}^{l, 2}, \mathcal{T}_{\mathrm{de}}^{l, 2} & =\operatorname{SeriesDecomp}\left(\text { Auto-Correlation }\left(\mathcal{S}_{\mathrm{de}}^{l, 1}, \mathcal{X}_{\mathrm{en}}^N\right)+\mathcal{S}_{\mathrm{de}}^{l, 1}\right) \\ \mathcal{S}_{\mathrm{de}}^{l, 3}, \mathcal{T}_{\mathrm{de}}^{l, 3} & =\operatorname{SeriesDecomp}\left(\text { FeedForward }\left(\mathcal{S}_{\mathrm{de}}^{l, 2}\right)+\mathcal{S}_{\mathrm{de}}^{l, 2}\right) \end{aligned}$
+ 趋势项,通过累积的方式逐步从预测的隐变量中提取出趋势信息
+ ${\mathcal{T}_{\mathrm{de}}^l =\mathcal{T}_{\mathrm{de}}^{l-1}+\mathcal{W}_{l, 1} * \mathcal{T}_{\mathrm{de}}^{l, 1}+\mathcal{W}_{l, 2} * \mathcal{T}_{\mathrm{de}}^{l, 2}+\mathcal{W}_{l, 3} * \mathcal{T}_{\mathrm{de}}^{l, 3}}$
实验结论
读后总结
[[Autoformer Code]]