In the AI gold rush, chip-makers are selling shovels 鈥� and they are in short supply. As the latest generation of artificial intelligence models like ChatGPT look set to transform our lives, the hardware that makes it all possible is becoming a strategic asset, with countries, companies and researchers all scrabbling to get hold of supplies. With talk of shortages lasting until at least next year, some would-be AI developers face being left behind.
At the centre of this is Nvidia, which joined the earlier this year, sitting alongside the likes of Apple and Google owner Alphabet. Nvidia, based in California, made its name聽creating hardware to run聽computer games, but these graphics processing units (GPUs) turned out to be extremely capable at churning through the vast amounts of data required to train an AI. Reports suggest that Nvidia now controls .
But there are signs that this huge demand is and creating a bottleneck for those wanting to get their hands on Nvidia鈥檚 most powerful AI GPUs, the H100 and A100. This聽is a predictable result of the聽current AI boom, with recent advances being closely linked to scale and therefore the number of聽GPUs you have.
Advertisement
The company hasn鈥檛 confirmed聽a shortage and declined an interview with New 杏吧原创, but any new orders are unlikely to be fulfilled until 2024 鈥� a huge stretch of time聽when聽it comes to the pace of聽AI development.
鈥淣VIDIA allegedly has sold out聽its whole supply through the聽year,鈥� Suhail Doshi, founder of AI image creator Playground AI, on聽25 July in response to conversations he said聽he has had with vendors. 鈥淚鈥檇聽forecast a minimum spend of聽$10m+ to play right now if you can even get GPUs.鈥�
On top of this seeming shortage, AI teams in China are feeling the bite of US restrictions that came into force in September last year. The US has banned exports of high-end GPUs, like the H100, to聽Russia and China in order to .
Nvidia has attempted to skirt this export ban by developing versions of the chips with reduced data transfer rates that comply with the ban, and is seeing healthy sales. TikTok owner ByteDance has so far聽this year, perhaps aiming to get ahead of聽any . ByteDance didn鈥檛 respond to聽a聽request for comment.
Meanwhile, other Chinese firms聽are suffering. Tech giant Alibaba recently told investors it聽had been unable to meet customer demand for training AIs聽due to a shortage of GPUs.
Worst hit by the global GPU shortage is academia, given its limited research budgets. 鈥淭he hype around machine learning has led to a lot of competition for聽kit and brains,鈥� says Tobias Weinzierl at Durham University, UK. 鈥淓ven if we order [GPUs], we don鈥檛 get kit in a timely fashion.鈥� Meanwhile, higher salaries and better access to resources are tempting many academic researchers to leave for the private sector, says Mhairi Aitken at the Alan Turing Institute in London.
at Liverpool John Moores University, UK, says that industry is taking AI ideas, architectures and research from academia and applying massive scale 鈥� which university budgets can鈥檛 stretch to 鈥� to leverage better results. What is needed, he says, is聽more collaboration 鈥� to bring together the experience and talent of academia with the budgets and experience of commercialisation from big tech companies.
鈥淐ompanies look for money, they don鈥檛 care about human development. As academics, we聽work hard to serve human beings,鈥� says Al-Jumeily. 鈥淲hatever AI has reached now,聽it鈥檚聽because of academia. Without our research, they cannot push those boundaries.鈥�
, also at Durham University, says that tech聽companies working on exceedingly large data-driven AI聽models, equipped with vast amounts of hardware, are leaving academic researchers behind.
鈥淐ompanies can exploit extremely large corpora of data [that] we will never be able to host聽or handle given our limited storage capabilities,鈥� he says. 鈥淥n聽top of that, large corporations now actively involved in applied and more theoretical research have stormed into the international conference and scientific journal landscape, making it very hard for聽small academic institutions to聽compete.鈥�
at聽chip-maker AMD says that academics can apply for grants to聽get GPUs, or buy time on cloud-based services, but concedes that the costs can rise quickly. He says he is seeing a trend for some researchers to focus on smaller AI聽models that are trained to do specific tasks, leaving research on the gigantic models that require computing hardware worth millions to the big technology companies. 鈥淭hat鈥檚 a lot more economical,鈥� he says.
Fine-tuned model
The other approach academics are聽taking is to focus on a different part of the process of training an AI model. The biggest AI systems are built from scratch using huge datasets to create software that has broad abilities. Then a fine-tuning stage occurs to tweak the聽model and hone its abilities on聽a single task. This stage uses a聽smaller but more focused dataset and needs fewer computing resources.
鈥淔or researchers that may not聽have many GPU resources, they can still do many things in the fine-tuning stage and they can聽try聽to improve this work or improve this field,鈥� says 聽at Nanyang Technological University in Singapore.
One issue with this approach is聽that the researchers often don鈥檛 have full details of how the model was trained and what data was used, making it harder to carry out聽open and repeatable science.
Can anything be done to increase the supply of chips for AI?聽Numerous companies are racing to launch their own GPUs to聽rival Nvidia, while tech giants like Google, Amazon, Microsoft and Meta are turning to custom chips to run their own AI tools as聽efficiently as possible.
But switching chips isn鈥檛 easy. Nvidia鈥檚 GPUs utilise proprietary software called CUDA that squeezes the most out of them when applied to AI-related tasks, and this isn鈥檛 available on chips made by the likes of Intel and AMD.
George Hotz, the founder of AI company Comma, said in a recent video posted to YouTube that he聽is聽鈥溾�� after encountering problems trying to write and run custom AI software on the company鈥檚 GPUs. James Knight at the University of Sussex, UK, says that Hotz鈥檚 complaint was聽a 鈥渧ersion of the experience everyone who has tried to use AMD hardware has suffered鈥�.
Balasubramanian says it is true聽that some users have found problems making the switch from聽Nvidia to AMD because the platforms are slightly different. This can be solved, he says, if developers invest more time learning the new environment.
鈥淧eople who have gotten into the CUDA ecosystem early, for them it聽is this system of familiarity. And聽the expectation is everyone matches that familiarity,鈥� says Balasubramanian. 鈥淚t requires a little extra tweaking because, at the end of the day, the architecture for Nvidia and AMD are not the same.鈥�
That said, there are software tools available to help make switching between chips a smoother experience. If researchers build their AI using聽software such as PyTorch (originally developed by Meta) or聽TensorFlow (originally created by Google) they can theoretically switch from Nvidia GPUs to AMD GPUs with minimal upheaval. This聽means they can change their supplier based on performance, cost or supply chain issues聽鈥� a benefit also enjoyed by the likes of聽Meta and Google.
鈥淭hat鈥檚 what these big AI players want,鈥� says Balasubramanian. 鈥淎nd that鈥檚 part of the reason why they developed this聽鈥� they want to make sure that they鈥檙e not locked into a single [GPU] vendor.鈥�
There are other positive signs, too. Supply chain problems during the covid-19 pandemic are resolving, the race to buy hardware to capitalise on a new trend is slowing down and the聽number of GPU manufacturers is expanding. New entrants are working to catch up and the market is widening, while existing players step up production.
Ultimately, if advanced AI models are going to be fully integrated into society, we need a wide range of people, from academics to activists, to be able to explore and interrogate them. Only then will the wealth of the AI gold rush be shared by all, rather than be limited to a few firms.
