Samsung Electronics was once the dominant player in a type of semiconductor known as memory, putting it in a great position to capitalize on the boom of artificial intelligence.

But the South Korea electronics giant has now fallen behind its long-time rival SK Hynix in next-generation chips that have been key component for AI silicon leader Nvidia. The result? Samsung’s profit has plunged, around $126 billion has been wiped off its market value, according to data from S&P Capital IQ, and an executive issued a rare public apology about the company’s recent financial performance.

Memory is a critical type of chip used to store data, and it can be found in a plethora of devices from smartphones to laptops. For years, Samsung was the undeniable leader in this technology, ahead of South Korean rival SK Hynix and U.S. competitor Micron.

But as AI applications such as OpenAI’s ChatGPT rose in popularity, the underlying infrastructure required to train the huge models they rely on became a bigger focus. Nvidia has emerged as the top player in this space with its graphics processing units (GPUs) that have become the gold standard used by tech giants for AI training.

A crucial part of that semiconductor architecture is high-bandwidth memory, or HBM. This next generation of memory involves stacking multiple dynamic random access memory (DRAM) chips, but it had a small market before the AI boom.

That’s where Samsung got caught out and failed to invest.

“HBM has been a very niche product … for a long time and Samsung has not focused its resources on its development,” Kazunori Ito, director of equity research at Morningstar, told CNBC by email.

“Due to the difficulty of the technology involved in stacking DRAMs and the small size of the addressable market, it was believed that the high development costs were not justified.”

SK Hynix saw this opportunity. The company aggressively launched HBM chips which were approved for use in Nvidia architecture and, in the process, the South Korean firm established a close relationship with the U.S. giant. Nvidia’s CEO even asked the company to speed up supply of its next generation chip, underscoring the importance of HBM to its products.

SK Hynix posted record quarterly operating profit in the September quarter.

“With strong R&D (research and development) investments and established industry partnerships, SK Hynix maintains an edge in both HBM innovation and market penetration,” Brady Wang, associated director at Counterpoint Research, told CNBC by email.

Samsung told CNBC that, in the third quarter, total HBM sales grew more than 70% quarter-on-quarter. The tech giant added that the current product known as HBM3E is in mass production and generating sales.

The South Korean tech company noted that development for its next-generation HBM4 is “underway according to plan” and that the company is targeting starting “mass production” in the second half of 2025.

Analysts said that Samsung is lagging behind competitors for a number of reasons, including underinvestment in HBM and the fact that it is not a first-mover.

“It is fair to say that Samsung has not been able to close the gap with SK Hynix on the HBM development roadmap,” Morningstar’s Ito said.

Samsung’s ability to make a comeback in the short term appears to be closely linked to Nvidia.

A company must pass a strict qualification process before Nvidia approves it as a HBM supplier — and Samsung has not yet completed this verification. But a green light from Nvidia could open the door for Samsung to return to growth and compete more effectively with SK Hynix, according to analysts.

“Since NVIDIA holds more than 90% of the AI chip market, where most HBMs are used, NVIDIA’s approval is critical for Samsung to benefit from the robust demand for AI servers,” Ito said.

A Samsung spokesperson said that the company has made “meaning progress” regarding HBM3E and has “completed an important phase in the qualification process.”

“We expect to start expanding sales in the fourth quarter,” the spokesperson said.

Meanwhile, Wang noted that Samsung’s strength in research and development, as well as the company’s semiconductor manufacturing capacity that could help it catch up to SK Hynix.