mirror of
https://github.com/skindhu/Build-A-Large-Language-Model-CN.git
synced 2026-07-16 01:00:11 +08:00
Add appendixD
This commit is contained in:
@@ -194,7 +194,11 @@ $$|v|_{2}=\sqrt{v_{1}^{2}+v_{2}^{2}+\ldots+v_{n}^{2}}$$
|
||||
|
||||
这种计算方法也适用于矩阵。例如,考虑以下梯度矩阵:
|
||||
|
||||
$$|v|_{2}=\sqrt{v_{1}^{2}+v_{2}^{2}+\ldots+v_{n}^{2}}G=\left[\begin{array}{ll}
|
||||
$$G=\left[\begin{array}{ll}
|
||||
1 & 2 \\
|
||||
2 & 4
|
||||
\end{array}\right]$$
|
||||
\end{array}\right]$$
|
||||
|
||||
如果我们旨在将这些梯度裁剪到最大范数 1,我们首先计算这些梯度的 L2 范数,即为:
|
||||
|
||||
$$|G|_{2}=\sqrt{1^{2}+2^{2}+2^{2}+4^{2}}=\sqrt{25}=5$$
|
||||
Reference in New Issue
Block a user