Could a new Generative AI Architecture Crash Demand for Nvidia Chips?

Brett Winton, the Chief Futurist at Ark Invest has called out a potential risk for Nvidia that investors may be overlooking. Ark is a firm focused on disruptive innovation.

What you may find interesting about Ark is that the firm slashed its stake in Nvidia last spring before the stock went on a tear.

Why would a firm that is focused on disruptive innovation sell what many on Wall Street now proclaim to be the enabler of the next great economic and business revolution?

Perhaps, the ultra-GPU-intensive architecture of today’s generative AI models may evolve to require less compute.

Winton explained just such a risk in a post on Ark’s website. New research published by Rui-Jie Zhu and colleagues challenges the long-standing dominance of the “attention” mechanism in AI models, particularly the transformer architecture, which has underpinned the success of large language models.

Transformers, the backbone of many advanced AI models, rely on an “attention” mechanism, which is computationally intensive and necessitates substantial parallel processing power. Nvidia GPUs, designed for such tasks, have enjoyed a boom due to this requirement. However, Zhu’s groundbreaking approach proposes a sequential relationship computation method that is far less resource intensive.

This new methodology, paired with simplified model parameters, offers significant performance gains. Models using this architecture not only train more efficiently but also operate faster and more cost-effectively post-training. These models require a fraction of the memory and deliver responses at 5X the speed, even when running on existing Nvidia hardware.

The implications for Nvidia are profound. Custom computational hardware tailored to this new approach, like the FPGA chips highlighted by the authors, could surpass current GPU capabilities. These chips can achieve human-like reading speeds with minimal energy consumption, potentially transforming AI’s hardware needs.

The reliance on GPUs for AI could wane as more efficient and cost-effective architectures emerge potentially hurting demand for Nvidia chips.

With Nvidia shares earning a C rating on my Retirement Stock Ratings system today, the shares are best avoided by investors saving for retirement or already enjoying retirement.