fz-mirrors/Build-A-Large-Language-Model-CN
1
0
Fork 0
mirror of https://github.com/skindhu/Build-A-Large-Language-Model-CN.git synced 2026-07-29 07:10:24 +08:00
Code Issues Packages Projects Releases Wiki Activity

Update 5.在无标记数据集上进行预训练.md

Browse Source
This commit is contained in:
yuhui
2025-05-09 19:59:55 +08:00
committed by GitHub
parent 5ed44ac984
commit f399a51b79
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
cn-Book/5.在无标记数据集上进行预训练.md
View File

@@ -1069,8 +1069,9 @@ def generate(model, idx, max_new_tokens, context_size,
idx_next = torch.argmax(logits, dim=-1, keepdim=True) idx_next = torch.argmax(logits, dim=-1, keepdim=True)
if idx_next == eos_id: #E if idx_next == eos_id: #E
break break
idx_next = idx_next.unsqueeze(1)
idx = torch.cat((idx, idx_next), dim=1) idx = torch.cat((idx, idx_next), dim=1)
return idx return idx
#A For循环与之前相同：获取logits，仅关注最后的时间步 #A For循环与之前相同：获取logits，仅关注最后的时间步