- Synthetic
- Posts
- GPT-Next, Colossus, and How AI Models Store Facts
GPT-Next, Colossus, and How AI Models Store Facts
Plus, Mushroom-Powered Robots!
Steve Brown
September 05, 2024
In partnership with
Subscribe to Synthetic
The week’s most interesting and relevant AI news and analysis
This Week in AI
At the KDDI event in Japan, an OpenAI representative showed a roadmap slide that featured “GPT-Next,” which he said would be 100 times more powerful than GPT-4 and feature early reasoning capabilities with a cut-down version of the long-rumored “Strawberry” technology. Another more powerful model, Orion, will follow in 2025. The OpenAI rep also shared that ChatGPT has over 200 million daily active users. Note: Use your browser’s language translation feature to read this article. Also, consume with a grain of salt.
Elon Musk announced that xAI’s new AI supercomputer, Colossus, has begun training the next version of their frontier model, Grok. The machine is built from more than 100,000 Nvidia H100 GPUs, which makes it the largest AI training cluster on the planet. Colossus consumes over 150MW of power and took Dell about four months to build, which is FAST. xAI plans to double Colossus’s size in the next few months, adding more H100 and H200 chips. The AI infrastructure wars continue to rage. Microsoft plans to build a $100 billion mega AI data center project codenamed Stargate, which will consume 5 Gigawatts and begin operations in 2028.
China is a leader in the design and manufacture of industrial robots. More than half of all industrial robots are installed in China. Nearly 30 Chinese robotics companies showed off their latest creations at the Beijing World Robotic Conference this week. Chinese robotics leaders include Astribot, UbTech, Agibot, Unitree, Galbot, and Siasun. Goldman Sachs expects the global market for humanoid robots to reach $38 billion by 2035. (Note: Synthetic believes this significantly undercalls the opportunity). Tesla is expected to produce about 1000 of its Optimus robots next year and underwhelmed attendees. Synthetic’s money is still on Figure.
China makes far greater use of industrial robots than any other country
Quick Hits
Nvidia Gets Antitrust Subpoena - As predicted by Synthetic a few weeks ago, Nvidia has been hit with a subpoena from the U.S. Department of Justice for anti-competitive behavior that locks in buyers and penalizes them for using companies that use alternative solutions. NVDA stock tumbled on the news, wiping $280 billion (about 3.5 Intels) off their market cap.
Intel Launches Impressive New AI PC Chip - First laptops with the Intel Core Ultra 200V (formerly known as “Lunar Lake”) will ship on September 24th. Intel claims incredible battery life, snazzy ray-traced graphics, and leading AI performance compared to AMD and Qualcomm. Good timing for gamers as reports swirl of major compatibility problems for Copilot+ PCs based on Qualcomm Snapdragon chips.
Sutskever Raises $1 Billion for Superintelligence - Former OpenAI cofounder and chief scientist Ilya Sutskever has raised $1 billion in funding on a $5 billion valuation for his three-month-old, pre-revenue startup, Safe Superintelligence. Nice funding if you can get it.
OpenAI AI Chip Plans Revealed - Early details have emerged on OpenAI’s plans to build AI training and inference chips with the first chip said to boost video generation performance for their video model, Sora. Chips will be built on TSMC’s 1.6nm A16 process and designed by Marvell or Broadcom.
Video: How LLMs Store Facts
We have another great video this week from 3Blue1Brown, known for their excellent AI explainers. If you’ve ever wondered how AI models store all the facts they learn during training, this video is for you. It’s a great way to learn about vectorization, attention, and the key technology mechanisms behind the remarkable abilities of large language models. (23 mins)
AI Tech and Innovation
Cornell University researchers have built two proof-of-concept robots powered by fungal mycelia. These hybrid electrical/living systems respond to UV light and use the mycelia as a controller that mimics a neural circuit. In the research paper published in Science, the scientists imagine future robots that sense and respond to chemical reactions, making them ideal for deployment at scale in agricultural applications. Fans of The Last of Us probably won’t sleep well tonight.
Chinese researchers at Beijing’s Peking University have developed an AI accelerator chip—A tensor processing unit, or TPU—that uses carbon-based transistors instead of silicon. Google designed the first TPU in 2015 to reduce the energy consumption of AI projects. The scientists claim the new carbon nanotube-based TPU consumes just 295 microwatts of power and can deliver 1 trillion operations per watt. The new technology is nowhere near ready for prime time but illustrates promise for future chip designers if they can scale it up. The research was published in Nature Electronics.
AI Insights
“With AI automating many routine tasks, uniquely human skills like creativity are now more crucial than ever.”
A new report on creativity in education found that despite impressive AI capabilities, creativity remains crucial for career success. The Harris poll, commissioned by Canva, surveyed over 3000 educators, recent college grads (RCGs), and hiring managers. 75% of hiring managers consider creativity an important skill when making hiring decisions. On the flipside, managers think only 56% of graduates are prepared for the workforce.
A new report by the Data Provenance Initiative attempts to draw attention to what it sees as a crisis in AI—a lack of training data for tomorrow’s AI models. As companies wall off their data and restrict access to crawler bots, generative AI companies are finding it harder and harder to source quality data to train models.
Venture capitalist Andreeson Horowitz argues that the “next Pixar” will be a company that uses AI to blend the storytelling and powerful visuals of movies with the interactivity of video games. Today’s best video-generating AIs output video at a rate of 5 seconds per second of video. Soon, high-quality (read: watchable) video will be generated from prompts in less time than it takes to play that video, making interactive video possible. Google’s AI Doom project, described in last week’s issue of Synthetic, hints at the possibilities.
“One day, with the help of AI, we might start the creative process by crafting a storyworld – an IP universe we envision fully formed with characters, narrative arcs, visuals, etc. – and then generate the individual media products we want for an audience or situation. This will be the final evolution of transmedia storytelling, fully blurring the lines between traditional forms of media.”
Toolkit for the Future
💥 Use AI to 10X your productivity & efficiency at work (free bonus) 🤯
Still struggling to achieve work-life balance and manage your time efficiently?
Join this 3 hour Intensive Workshop on AI & ChatGPT tools (usually $399) but FREE for first 100 readers.
Save your free spot here (seats are filling fast!) ⏰
An AI-powered professional will earn 10x more. 💰 An AI-powered founder will build & scale his company 10x faster 🚀 An AI-first company will grow 50x more! 📊 |
Want to be one of these people & be a smart worker?
Free up 3 hours of your time to learn AI strategies & hacks that less than 1% people know!
🗓️ Tomorrow | ⏱️ 10 AM EST
In this workshop, you will learn how to:
✅ Make smarter decisions based on data in seconds using AI
✅ Automate daily tasks and increase productivity & creativity
✅ Skyrocket your business growth by leveraging the power of AI
✅ Save 1000s of dollars by using ChatGPT to simplify complex problems
Invite the HireLogic AI to your next phone, video, or in-person interview. It analyzes the conversation, provides specific insights, and highlights the interviewee’s strengths, skills, and any potential concerns. It creates a full interview transcript so you can focus on the conversation. Free to try. Awesome service.
Make a logo in minutes—no design skills required! Auto-create a brand kit with fonts and colors to complement your new logo and create a brand identity. Instantly create branded marketing materials, then use over 300 templates to customize branded business cards, social profiles, email signatures, and more.
Notion helps you focus on the things that actually matter. Organize information, thoughts, to-do lists, calendars, articles, tasks, notes, and team discussions in one well-organized space. Use Notion AI to create content, translate languages, and ask your notes questions.
Melio is a free and easy-to-use payment solution that helps U.S. small businesses pay any expense by bank transfer or credit card - even where cards are not accepted. No setup or monthly fees.
Focus on the email that matters. Deep Clean automatically identifies old, unimportant emails so you can delete them. Snooze emails until a better time and easily track emails that need a response. Automatically filters unimportant emails into specific folders. It works across every device where you check your email. Get to inbox zero with the award-winning SaneBox.
Recording, editing, and refining audio demands more time than most entrepreneurs can spare. Generate voices, high-quality music, and sound effects efficiently and cost-effectively with a single click. Create multilingual voice tracks in up to 29 languages to reach new markets. Try it for free.
Find phone numbers, emails, and company org charts to reach the people you need to. Fill information gaps in your contacts, auto-generate follow-up emails, and identify new target markets. Get 50 free credits and discover why over 500,000 companies use Seamless to grow their business.