mirror of
https://github.com/skindhu/Build-A-Large-Language-Model-CN.git
synced 2026-07-29 15:20:24 +08:00
@@ -422,7 +422,7 @@ tensor(10.7940)
|
||||
```python
|
||||
file_path = "the-verdict.txt"
|
||||
with open(file_path, "r", encoding="utf-8") as file:
|
||||
text_data = file.read()
|
||||
text_data = file.read()
|
||||
```
|
||||
|
||||
加载数据集后,我们可以查看其中的字符数和 token 数:
|
||||
|
||||
Reference in New Issue
Block a user