As artificial intelligence continues to evolve, the demand for specialized hardware like Graphics Processing Units (GPUs) is skyrocketing. OpenAI, the company behind the popular ChatGPT, is finding itself in a crunch for these valuable resources. According to recent reports, the company is exploring the possibility of manufacturing its own AI chips to alleviate the shortage and “eye-watering” costs.
The GPU Conundrum
GPUs are integral to the operation of machine learning models, and OpenAI is no exception. The company has been vocal about its struggles with the scarcity of these chips. Sam Altman, OpenAI’s CEO, has expressed concern over two primary issues: the shortage of advanced processors and the enormous costs of running the necessary hardware. Given that Nvidia dominates more than 80% of the global GPU market, the shortage has become a critical bottleneck for the company.
A Strategic Shift?
OpenAI is contemplating various solutions, including building its own AI chips and diversifying its suppliers beyond Nvidia. It has even evaluated a potential acquisition target in this space, although the identity remains undisclosed. Building custom chips could be a game-changer but would require a significant investment, potentially running into hundreds of millions of dollars per year.
What’s at Stake?
According to an analysis by Bernstein analyst Stacy Rasgon, each ChatGPT query costs roughly 4 cents. If the service were to scale to even a tenth of Google Search, OpenAI would need about $48.1 billion worth of GPUs initially and another $16 billion annually to maintain operations. It’s a daunting figure that underscores the severity of the situation.
Industry Implications
If OpenAI decides to produce its own chips, it would join tech giants like Google and Amazon who have taken similar steps. While it could accelerate OpenAI’s projects, it’s worth noting that custom chip production is a complicated and risky venture. Companies like Meta have faced issues in their custom chip efforts, which have led to scrapping some of their AI chips.
OpenAI and Microsoft
OpenAI’s primary backer, Microsoft, is also developing a custom AI chip. This raises questions about the future relationship between the two companies, especially considering that OpenAI’s plans could signal a distancing from Microsoft.
Sam Altman Speaks
In June, Sam Altman participated in a discussion with developers, shedding light on OpenAI’s immediate challenges and future plans. Altman confirmed that the company is severely GPU-limited, affecting the reliability and speed of the API. Furthermore, he shared OpenAI’s near-term roadmap, which includes making GPT-4 cheaper and faster and extending the finetuning API to the latest models.
The Road Ahead
Even if OpenAI decides to go forward with its custom chip plans or an acquisition, the process will take years, leaving the company dependent on commercial providers like Nvidia and AMD in the meantime. Nevertheless, the strategic move could reshape the AI hardware landscape and potentially create a ripple effect across the industry.
