Skip to content

Commit

Permalink
Merge pull request #429 from adambala/fixes
Browse files Browse the repository at this point in the history
Open "shakespeare" data in UTF-8 in "prepare.py"
  • Loading branch information
karpathy authored Feb 27, 2024
2 parents f68ac22 + f35dc82 commit a022d02
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 2 deletions.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,5 @@ __pycache__/
*.pt
*.pyc
input.txt
env/
venv/
4 changes: 2 additions & 2 deletions data/shakespeare/prepare.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@
input_file_path = os.path.join(os.path.dirname(__file__), 'input.txt')
if not os.path.exists(input_file_path):
data_url = 'https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt'
with open(input_file_path, 'w') as f:
with open(input_file_path, 'w', encoding='utf-8') as f:
f.write(requests.get(data_url).text)

with open(input_file_path, 'r') as f:
with open(input_file_path, 'r', encoding='utf-8') as f:
data = f.read()
n = len(data)
train_data = data[:int(n*0.9)]
Expand Down

0 comments on commit a022d02

Please sign in to comment.