L1和L2正则的先验分布

[[MAP]] 公式 → ${\log P(x,w)P(w) = \log P(x,w) + \log P(w)}$

• {{c1 [[L2 Regularization]]}} 中参数先验分布 {{c2 [[Normal Distribution]]}}
  • $P(w_j)$ → $\frac{1}{\sigma \sqrt{2 \pi}} e^{-\frac{(w_j)^{2}}{2 \sigma^{2}}}$
  • $\log P(w)=\log \prod_{j} P\left(w_{j}\right)= \log \prod_{j}\left[\frac{1}{\sqrt{2 \pi} \sigma} e^{-\frac{\left(w_{j}\right)^{2}}{2 \sigma^{2}}}\right]$ ↔ $-\frac{1}{2 \sigma^{2}} \sum_{j} w_{j}^{2}+C$
• {{c1 [[L1 Regularization]]}} 中参数先验分布 {{c2 [[Laplace Distribution]]}}
  • $P\left(w_{j}\right)$ → $\frac{1}{\sqrt{2 a}} e^{\frac{\left|w_{j}\right|}{a}}$
  • $\log P(w)=\log \prod_{j} P\left(w_{j}\right)=\log \prod_{j}\left[\frac{1}{\sqrt{2 a} \sigma} e^{-\frac{w_{j}}{a}}\right]$ ↔ $-\frac{1}{2 a} \sum_{j}\left|w_{j}\right|+C$

网络回响