Yannic Kilcher's Traditional Holiday Live Stream: Minecraft & AI Discussions
0:00-5:03 Stream Setup: The stream starts with some technical difficulties in setting up the chat window.
5:03 - 8:03 Starting a New World: Yannic creates a new Minecraft world and checks the game sound. He mentions living in Zurich, Switzerland and briefly touches upon a paper on Swiss German audio domain transfer for AI.
8:03 - 9:29 AI Going Mainstream: Yannic talks about how AI, including ChatGPT, is becoming mainstream, with family members using it for gift ideas. He acknowledges the hype around AGI and expresses his belief in fundamental limits to AI capabilities.
9:36 - 12:08 Exploring New Minecraft Features: He explores the Minecraft world, encountering new underwater features and a shipwreck.
12:08 - 13:16 Benchmark Gaming: Yannic discusses how AI companies are heavily focused on benchmark performance due to the financial incentives. He highlights how this focus can lead to gaming the benchmarks and the limitations of benchmarks like ARC in measuring true intelligence.
13:41 - 15:55 Arc and Test Time Compute: He mentions that the AI community has figured out how to use test-time compute, which involves the AI doing extensive internal searches, drawing parallels with how AlphaGo works, and highlighting that more computation leads to better results. He stresses that achieving a high score on ARC does not equal to AGI, but solving it is still a significant achievement.
15:55 - 19:18 Getting Stone Tools: Yannic focuses on getting basic stone tools and shares that he has not played Minecraft in around three years.
19:18 - 22:10 Large Concept Model and Verifiers: The conversation shifts to the challenge of verifying model outputs in discrete spaces. Yannic suggests that AI companies might be using methods like majority voting or self-verification with meta-prompting in order to improve the accuracy of outputs.
22:10 - 23:37 The Problem with Incorrect Verifiers: Yannic notes the issue of having slightly incorrect verifiers, stressing how that limits progress. He points out that in many problems, verifiers are only somewhat correct.
23:49 - 27:30 Test Time Compute Assumptions: The conversation shifts to the fundamental assumption behind test time compute. He states that for these AI models to work, the knowledge must already be inside the LLM. He also states that this is the same assumption that allows fine-tuning and instruction-tuning.
27:30 - 35:15 Skepticism about Simulation Data: Yannic expresses some skepticism of simulation-based data for AI training, as one cannot learn more than what is baked into the simulation. He is more open to training models on natural data such as internet text.
35:15 - 39:07 Exploring the World and Finding Sheep: Yannic explores the Minecraft world, getting some food along the way and trying to find some sheep.
39:07 - 41:18 Discovering a Village: Yannic finds a village near the side of a mountain, and explores the various houses and villagers.
41:18 - 45:21 Identity as an ML Researcher & the AI Bubble: Yannic discusses how his identity as a researcher is now tied to the AI field due to recent popularity. He states that the mainstream businesses are still implementing chat bots, and the AI "bubble" has not yet burst.
45:21 - 47:30 The Mission for the Day: Yannic states that his mission for the day is to get a cat. He also mentions being open to surprises with regards to timelines to achieve AGI.
47:30 - 50:26 Test Time Search: Yannic returns to test time compute, noting that they have figured out how to do a test time search, which will lead to advancements.
50:26 - 54:21 Limits of LLLMs and Gary Marcus: Yannic discusses how he believes that some things are unachievable with current LLMs as some knowledge is not encapsulated inside them. He also discusses the public persona of Gary Marcus, a known AI skeptic.
54:21 - 57:26 Measuring AI with Moving Goal Posts: The stream moves onto AI measurement and the selling of tokens. Yannic highlights that test time compute is perfect for selling tokens, as one can say that the more compute you invest, the smarter the model becomes.
57:26 - 59:03 Taming a Cat: Yannic struggles with taming a cat with fish, but eventually succeeds with the guidance of chat.
59:03 - 1:01:06 Current Tasks: He states that he has been busy lately, working 12-15 hours a day.
1:01:06 - 1:03:33 More Minecraft: Yannic gets a pickaxe and states that he is going to go do something cool with the cat. He also speaks about his early experience multiplying redstone in Minecraft.
1:03:33 - 1:07:07 More Minecraft and the Idea of Infinite Memory: He gets salmon, and cooks potatoes, and then sets off into a new adventure, leading into a discussion about infinite memory claims, and whether tokenization is ideal for LLMs.
1:07:07 - 1:09:24 The Evaluator Problem: Yannic notes that the key problem is having a good evaluator for LLMs. He adds that it is possible that if one could build a good enough evaluator, then infinite AGI would be possible using the techniques that are already in place.
1:09:24 - 1:13:09 Text vs. Vision: The conversation moves into whether vision and video would be better mediums for teaching LLMs, and Yannic states that text is far superior due to its compression ratio.
1:13:09 - 1:21:44 Random Minecraft Adventures: Yannic goes on several random adventures, including an attempt to dig down and is attacked by several Minecraft mobs.
1:21:44 - 1:27:23 Human Level Minecraft Yannic states that if he is the "human" in question, then human level minecraft is easy to achieve. He also mentions that the world is procedurally generated, making it different every time.
1:27:23 - 1:28:16 Stream Conclusion: Yannic concludes the stream, mentioning his poor performance and his hopes to do another stream where he does something Redstone-related.