Samsung claims to have built a novel supercomputer using AMD datacenter GPUs coupled with its processing-in-memory chips, which can significantly improve the efficiency and energy efficacy of training massive AI models.
By putting more processing power into memory, the supercomputer attempts to relieve devices of the tedious task of moving data out of memory and into a processor. It will do so with 96 AMD Intuition MI100 GPUs, each outfitted with a processing-in-memory (PIM) chip, a brand new type of memory know-how that reduces the amount of data that must be transferred between the CPU and DRAM.
The PIM technology has significant implications for energy consumption and the environment, reducing a cluster’s annual energy use by 2,100 Gigawatt hours and, as a result, reducing 960,000 tons of carbon emissions.
When compared to the same cluster configuration that did not use the PIM chips, the cluster was able to train the Text-to-Test Transfer Transformer (T5) language model developed by Google 2.5 times faster while using 2.7 times less power.
Furthermore, Samsung Electronics is looking into ways to automatically combine the know-how contained in the notes of engineers working on-site with inspection data and use it in production processes to improve the current method of checking yields in semiconductors.
The sources for this piece include an article in TheRegister.