AI Debates, Reinforcement Learning, & The Power of Generative Models | Yilun Du | Eye on AI #147

This episode is sponsored by Crusoe. Crusoe Cloud is a scalable, clean, high-performance cloud, optimized for AI and HPC workloads, and powered by wasted, stranded or clean energy. Crusoe offers virtualized compute and storage solutions for a range of applications - including generative AI, computational biology, and rendering. Visit https://crusoecloud.com/ to see what climate-aligned computing can do for your business This episode is sponsored by Celonis ,the global leader in process mining. AI has landed and enterprises are adapting. To give customers slick experiences and teams the technology to deliver. The road is long, but you’re closer than you think. Your business processes run through systems. Creating data at every step. Celonis recontrusts this data to generate Process Intelligence. A common business language. So AI knows how your business flows. Across every department, every system and every process. Go to https:/celonis.com/eyeonai/ to find out more. Welcome to episode 147 of the Eye on AI podcast. In this episode, host Craig Smith sits down with Yilun Du, a final year PhD student at MIT EECS with a background in research at leading institutions like OpenAI, FAIR, and Google Deepmind. Yilun's extensive expertise spans generative models, decision making, robot learning, and embodied agents, making him a valuable voice in the AI domain. Our conversation kicks off with a brief on Yilun's academic journey, leading into a deep dive into Reinforcement Learning with AI feedback (RLHF) - its history, inception, and challenges. We then touch upon the effectiveness of RLHF, the intriguing concept of multi-agent debate, and the PAPES procedure. Craig and Yilun further explore the vast realm of AI, debating the gaps between open-source and proprietary models, the need for more compute resources, and the future of robotics interlaced with AI. Yilun provides a glimpse into his vision of decentralized AI systems, contrasting the industry's commercial trajectory with academia. Tune in for an enlightening episode that traverses the intricacies of AI, its nuances, and the future of robotics. Craig Smith Twitter: https://twitter.com/craigss Eye on A.I. Twitter: https://twitter.com/EyeOn_AI (00:00) Preview, Celonis and Crusoe Ad (04:06) Yilun's Academic Background (05:52) Origin and Applications of RLHF (12:16) ROHF and the Multi-Agent Debate Method (17:32) AI Model Interaction without Human Intervention? (20:41) Applicability and Inconsistency Detection (28:43) The Future of AI Training (45:26) Robotics and Decentralized AI Systems