Prices of server-grade GPUs are soaring as demand for machine-learning accelerators continues to rise due to the current AI buzz.
This trend is anticipated to result in higher cloud GPU prices and possibly resource shortages. In the hardware industry, there has been far too much demand and far too little supply. The AI hype train is reported to be fueling hardware demand, while manufacturers are content to keep selling their inventories without refilling.
Top-end Nvidia hardware, for example, has been observed selling for an astounding $40,000 per unit, creating a scenario analogous to the overpriced used car market in the United States. Hyperscalers and cloud providers are already limiting GPU access to their own engineers. The demand for GPU computation has reached unprecedented levels due to the fast rise in AI training for generative AI applications based on huge language models.
The rush to include generative AI into workloads is reminiscent of the late 1990s and early 2000s, when firms hurried to upgrade their internet-based systems. However, data centers are not changing into general-purpose X86 computing substrates at this moment. Instead, they are growing into ecosystems comprised of different and intertwined architectures with the goal of delivering maximum performance and value across a wide variety of workloads.
The sources for this piece include an article in TheRegister.
