Testing the Division Capabilities of Small Transformers

This code tests the division capabilities of small transformers, and is based off the work of Andrej Karpathy's NanoGPT, and https://github.com/lee-ny/teaching_arithmetic.

Name		Name	Last commit message	Last commit date
Latest commit History 9 Commits
__pycache__		__pycache__
config		config
config2		config2
config_gpt2/addition		config_gpt2/addition
data		data
matrix_completion		matrix_completion
out2		out2
plots		plots
prompts		prompts
run		run
run_gpt2		run_gpt2
utils		utils
wandb		wandb
.DS_Store		.DS_Store
.env		.env
LICENSE		LICENSE
README.md		README.md
bench.py		bench.py
configurator.py		configurator.py
evaluate_additions.py		evaluate_additions.py
evaluate_models.py		evaluate_models.py
main_utils.py		main_utils.py
meta_all_ascii_chars.pkl		meta_all_ascii_chars.pkl
model.py		model.py
run.ipynb		run.ipynb
sample.py		sample.py
sample_addition.py		sample_addition.py
sample_addition_fewshot.py		sample_addition_fewshot.py
sample_addition_fewshot2.py		sample_addition_fewshot2.py
test_addition.py		test_addition.py
train.py		train.py
train_mixed.py		train_mixed.py
train_multi_task.py		train_multi_task.py
train_no527.py		train_no527.py

Provide feedback