07-transformers

Language Models

• LLM / transformer slides

Sequential Data and Recurrent Neural Networks

• 📚 The Unreasonable Effectiveness of RNNs and Visualizing and Understanding Recurrent Networks by by Andrei Karpathy
• 🍿 Sunspring
• 🎨 Double Agent by Simon Biggs (Drawing)
• 🎨 Four Experiments in Handwriting with a Neural Network (Drawing)
• 📖 10 things artificial intelligence did in 2018 by Janelle Shane (Text)
• 📖 Writing with the Machine
• 🍿 Interactive Drawing with SketchRNN

Transformers and Large Language Models

among the reasons I use large pre-trained language models sparingly in my computer-generated poetry practice is that being able to know whose voices I'm speaking with is... actually important, as is being understanding how the output came to have its shape - @aparrish, full thread

• 📚 Watch an A.I. Learn to Write by Reading Nothing but ____ by Aatish Bhatia
• 📚 Attention is All You Need - Original "Transformer" paper from 2017, also Neural Machine Translation by Jointly Learning to Align and Translate -- Attention paper from 2014
• 📚 The Illustrated Transformer
• 📚 What Are Transformer Models and How Do They Work?

LLM Training

• 🦙 LLaMA: Open and Efficient Foundation Language Models
• 🦙 Llama 2: Open Foundation and Fine-Tuned Chat Models
• 🦜 On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
• 🔍 Introducing The Foundation Model Transparency Index

Datasets for LLMs

• 🔢 Common Crawl
• 🔢 The Pile

Climate Impact

• 🌏 The Internet’s Next Great Power Suck
• ⚡️ Carbon Emissions and Large Neural Network Training

Node.js with p5.js + Replicate

• 🎥 Workflow: Terminal, Shell, Node.js, VSCode
• 🎥 How to Set Up a Node.js Project
• 🎥 Server Side / Express with node.js
• 🎥 HTTP "POST" request with fetch

Node.js + p5.js Template

• 💻 Hello World node.js + express + p5 example

Replcate Examples

• 🎨 Hello World p5.js + Replicate web app
• ⌨️ Streaming results from Replicate model to p5.js
• 💬 ChatBot Conversations with Llama via Replicate
• 💻 Misc Models via Replicate

Fine-tuning with Replicate

• Official Replicate Documentation
• Repo with Instructions and Code Example for fine-tuning

Other Resources and Examples

• 💬 ChatGPT Clone with OpenAI API
• 🦙 Ollama: Run Llama locally

Assignment

• Read On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 and review the The Foundation Model Transparency Index. What questions arise for you about using LLMs in your work at ITP?
• Experiment with prompting a large language model in some way other than the ChatGPT interface and document the results in a blog post. Consider how working with an LLM compares to generating text from the previous exerc ise working with markov chains and context free grammars. Here are some options:
  • Run the ChatBot with Llama via Replicate example or ChatGPT Clone with OpenAI API and adjust the prompts, interaction, or visual design of the examples. There is also a new simpler Replicate + p5.js example you can try.
  • Sign up for the OpenAI API and try creating a "custom assistant" with a system prompt and knowledge base.
  • If you have the GPT Editor enabled in your OpenAI account, try making a custom chatbot.
  • Try running Llama locall with Ollama
  • Can you connect an LLM to a Discord Bot!?!
  • Invent your own idea!

Add your assignment below via Pull Request

(Please note you are welcome to post under a pseudonym and/or password protect your published assignment. For NYU blogs, privacy options are covered in the NYU Wordpress Knowledge Base. Finally, if you prefer not to post your assignment at all here, you may email the submission.)