What is ChatGPT Doing? Episode 5: Inside ChatGPT

A conversation about large language models, specifically why and how ChatGPT works. Read Stephen Wolfram's blog: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work Check out Wolfram Machine Learning-a Core Part of Wolfram Language: https://wolfr.am/ml Chapters 0:00 Intro 0:55 Beyond just Numbers 1:28 Images: LeNet trained on MNIST data 6:06 Text: GPT2 Transformer Trained on WebText Data 9:45 What Happens After Tokenization? 12:18 How Do We Get the Final Output? 15:22 Why Is It Called Attention? 17:56 Continuing Sentences 20:50 Training ChatGPT Follow us on our official social media channels. Twitter: https://twitter.com/WolframResearch Facebook: https://www.facebook.com/wolframresearch Instagram: https://www.instagram.com/wolframresearch LinkedIn: https://www.linkedin.com/company/wolfram-research Contribute to the official Wolfram Community: https://community.wolfram.com Stay up-to-date on the latest interest at Wolfram Research through our blog: https://blog.wolfram.com