— Elon Musk (@elonmusk) July 1, 2024 Musk is referring to Nvidia's H100 graphics processing unit, also known as Hopper, which is an AI hip that helps handle data processing for large language models (LLMs). The chips are a key component of AI development and a hot commodity in Silicon Valley as tech companies race to build ever-smarter AI products. Each Nvidia H100 GPU chip is estimated to cost around $30,000, although some estimates place the cost as high as $40,000. Volume discounts may also be possible. Advertisement Based on those estimates, that would mean Grok 3 is being trained on $3 billion to $4 billion worth of AI chips — but it's not clear if those chips were purchased outright by Musk's company. It's also possible to rent GPU compute from cloud service providers, and The Information reported in May that Musk's xAI startup was in talks with Oracle to spend $10 billion over multiple years to rent cloud servers. Related stories But we do know that Musk's companies have purchased a hefty amount of H100s outright in recent years. The Tesla CEO reportedly diverted a $500 million shipment of Nvidia H100s intended for Tesla to X instead, for example. Training based on 100,000 GPUs would be a big step up from Grok 2. Musk said in an interview in April with the head of Norway's sovereign fund Nicolai Tangen that Grok 2 would take around 20,000 H100s to train. xAI has so far released Grok-1 and Grok-1.5, with the latest only available to early testers and existing users on X, formerly known as Twitter. Musk said in a post on X Monday that Grok 2 is set to launch in August and indicated in the other post about GPUs that Grok 3 will come out at the end of the year. Advertisement