The Real AI Chip Shortage Isn't the GPU. It's What's Stacked on Top of It

Quick Facts

What You'll Learn

By the end of this article, you will understand:

• Why the "AI chip shortage" most investors picture, Nvidia simply not making enough GPUs, is not actually where the constraint sits in 2026
• What HBM memory is, in plain terms, and why an AI chip is functionally useless without it
• What TSMC's CoWoS packaging process actually does, and why it became the industry's tightest chokepoint through 2024 and 2025
• How the bottleneck is shifting in 2026, from packaging capacity toward HBM allocation itself, and what Nvidia is doing about it
• Exactly which companies capture the profit in this layer, and at what margins, with real disclosed numbers
• Why this is the one part of the AI buildout where an Indian investor currently has close to zero direct exposure, and what the honest workarounds are
• From zero: what a chip actually is, what "design," "fabrication," "packaging" and "memory" each mean, which companies do which job, and how they all connect into one supply chain, so the rest of this article makes sense even if you have never thought about hardware manufacturing before

This piece sits one layer beneath our AI data center economics deep dive, which covers what a data center costs and who profits from building it, and it extends a single line from our AI picks-and-shovels map, which never made it past cooling, power and connectors. This is the layer underneath the GPU itself.

The Shortage Nobody Is Actually Measuring Correctly

Ask most people why AI chips are hard to get, and you will hear some version of "Nvidia can't make enough GPUs." That framing made sense in 2023. It stopped being the accurate story by 2026.

The compute dieDieA single rectangular piece of silicon cut from a wafer, etched with one complete chip's circuits. A wafer holds many identical dies printed side by side, sliced apart once fabrication finishes. The 'compute die' is the piece containing an AI processor's logic, as distinct from the separate memory dies packaged next to it.See all terms in the glossary at the center of an Nvidia GB200 or the newer Vera Rubin platform is, at this point, a solved manufacturing problem for TSMC. What has not been solved is everything that has to happen after that die comes off the fabricationFabricationThe physical manufacturing of a chip design onto silicon, done in specialised factories called foundries. Companies that design chips but own no factories, like Nvidia and AMD, are called 'fabless'; TSMC alone fabricates close to 70% of the world's advanced logic chips.See all terms in the glossary line, before it can ship as a working AI accelerator. Two things have to happen to a bare compute die before it is a product: it has to be physically bound to a stack of specialised memory chips that can feed it data fast enough to stay busy, and enough of those specialised memory chips have to exist in the first place. Both of those steps, not the compute die itself, are where 2026's real scarcity lives.

Part 1: How a Chip Actually Gets Made, From Sand to Server

Before any of that makes sense, it helps to have the full picture, because "chip" is doing a lot of hidden work as a word. No single company designs, manufactures, packages and assembles an AI chip. It passes through a chain of specialists, each of whom does one job so well, and so few other companies can do it at all, that the whole global AI buildout can be gated by any single link. Here is that chain, start to finish.

What a chip actually is. At the most basic level, a chip is a thin wafer of silicon with billions of microscopic switches, transistors, etched into it, wired together to perform calculations. A modern AI processor packs tens of billions of these transistors into an area smaller than a fingernail. Everything else in this section is about who builds which part of that, and how those parts reach the reader as a finished, working machine.

Design, the "fabless" model. Nvidia does not own a single chip factory. Neither does AMD, nor Google, which designs its own AI chips called TPUs. All three are fablessFablessA chip company that designs processors but owns no factories, paying a foundry like TSMC to fabricate them instead. Nvidia, AMD and Google's chip division are all fabless; it lets a company focus capital on design talent rather than tens of billions of dollars of factory construction.See all terms in the glossary: they design the blueprint, a precise digital map of every transistor and every connection, using specialised design software from a small handful of companies, chiefly Synopsys, Cadence and Siemens EDA, which together control roughly three-quarters of that software market. The finished blueprint is not a physical chip. It is closer to an extraordinarily detailed architectural drawing, handed off to someone else to actually build.

Fabrication, the foundry. Turning that blueprint into working silicon is a separate, staggeringly capital-intensive business called a foundry, and almost nobody does it at the leading edge. TSMC alone held close to 70% of the global foundry market in the first quarter of 2026; Samsung Foundry was a distant second at roughly 6.5%, with Intel Foundry pushing to catch up. This is also where a bottleneck-beneath-the-bottleneck lives, worth knowing about even though it is not this article's main subject: TSMC's own factories depend entirely on a single Dutch company, ASML, the sole maker on earth of the EUV lithography machines needed to etch these designs at this precision. Each machine costs hundreds of millions of dollars and takes a long time to build; ASML's newest "High-NA" generation prices out around $400 million apiece, expensive enough that TSMC has reportedly chosen to delay adopting it until closer to 2029 rather than pay ASML's asking price today (reported by Bloomberg and Tom's Hardware). Even TSMC, the company everyone else in this article is worried about being squeezed by, is itself squeezed by its own supplier.

Memory fabrication, a separate chain entirely. While all this is happening, a completely different set of companies, SK Hynix, Samsung and Micron, are running their own separate factories to fabricate memory chips, including the HBM covered in depth in Part 2 below. This is not a side business of the logic foundries above; it is its own supply chain, with its own factories, its own equipment, and its own scarcity dynamics, and it does not meet the compute die at all until the next step.

Packaging, where the two chains meet. This is the step this entire article is built around. The compute die coming out of TSMC's fabrication line and the memory stack coming out of SK Hynix's, Samsung's or Micron's fabrication line are physically joined together at this stage, TSMC's CoWoSCoWoS (Chip-on-Wafer-on-Substrate)TSMC's advanced packaging technique for binding an AI processor and its memory onto a single package. The industry's tightest bottleneck through 2024 and 2025, with CEO C.C. Wei describing capacity as sold out through 2026.See all terms in the glossary process, covered in full in Part 3. Until this point, a finished AI chip does not exist. It is two separate pieces built by two separate companies in two separate supply chains.

System assembly, the ODMs. A packaged chip is still not a server. Companies like Foxconn, Quanta, Supermicro and Dell, known in the industry as ODMs (original design manufacturers), take finished, packaged chips and build them into the actual server boards, racks and cooling systems that ship to a data center.

Deployment. The finished server is installed in a data center, whether a hyperscaler's own facility or a colocation site, which is where the story picked up in our AI data center economics deep dive.

Part 2: What HBM Memory Actually Is, From Zero

With the full chain from Part 1 in view, zoom into the second lane, the memory fabrication chain, and the specific product it makes for AI chips. Start with the problem HBM solves. A modern AI accelerator has thousands of processing cores working in parallel, and every one of them needs a constant, enormous stream of data to stay busy. Ordinary computer memory, the kind in a laptop or an office server, connects to its processor over a relatively narrow electrical pathway. That pathway is fine for a CPU running a handful of tasks at once. It is nowhere near fast enough for a GPU trying to feed thousands of cores simultaneously. Starve the cores of data and you have paid for an enormously expensive chip that spends most of its time idle, waiting.

HBMHBM (High-Bandwidth Memory)Specialised memory stacked next to an AI accelerator to feed it data fast enough to keep it busy. Genuinely scarce and highly profitable: SK hynix holds a majority global share and reported operating margins above 70% through 2026 as HBM demand surged.See all terms in the glossary, high-bandwidth memory, solves this with two changes. First, instead of a single flat layer of memory chips, HBM stacks multiple memory dies directly on top of each other, connected internally through thousands of microscopic vertical wires. Second, that entire stack sits immediately next to the compute die, on the same package, instead of across the board on a separate chip connected by ordinary circuit-board wiring. The combination, stacking plus proximity, is what delivers the bandwidth a modern AI chip actually needs. Where regular DRAM might deliver data at tens of gigabytes per second, current-generation HBM delivers it at multiple terabytes per second, an order-of-magnitude difference.

That last row is the entire reason this layer of the supply chain has pricing power nobody else in electronics manufacturing enjoys right now. You cannot design around HBM the way you might design around a shortage of, say, a particular capacitor. If your AI chip does not have adequate high-bandwidth memory attached to it, it is not a competitive AI chip, full stop.

Part 3: What TSMC's CoWoS Actually Does, and Why It Became the Bottleneck First

This is the Packaging stage from the chain in Part 1, the point where the two separate lanes, logic and memory, physically become one chip. Having HBM memory chips and a compute die is not enough. Something has to physically join them together into a single, working package, precisely enough that the thousands of microscopic connections between them all function correctly. That process is TSMC's CoWoS, short for Chip-on-Wafer-on-Substrate: the compute die and the HBM stacks are mounted onto an intermediate silicon layer, which is itself mounted onto a substrate, creating one finished, functional chip out of what started as several separate pieces.

This is not a new idea in chip manufacturing, but doing it at the size, complexity and volume that modern AI accelerators require is genuinely difficult, and TSMC essentially had a global monopoly on doing it well. For years leading into 2024, nobody, including TSMC itself, had built anywhere near enough of this specific manufacturing capacity, because nobody had correctly forecast how fast AI demand would grow. The result was that CoWoS, not the compute die itself, became the tightest constraint in the entire AI chip supply chain through 2024 and 2025.

TSMC has been racing to close that gap ever since, and the numbers on that race are genuinely striking. The company's CoWoS capacity went from roughly 35,000 wafers a month at the end of 2024 to a targeted 130,000 to 140,000 wafers a month by the end of 2026, a near-fourfold increase in about two years, a pace of expansion with little precedent for a manufacturing process this precise (TrendForce). And it still is not enough. TSMC CEO C.C. Wei told shareholders directly at the company's annual meeting on June 4, 2026 that CoWoS capacity remains "extremely tight and sold out through 2026" (reported across trade press including TechTimes).

Even at that expanded scale, one customer is absorbing most of it. Trade press coverage citing supply chain analysts estimates Nvidia alone has reserved roughly 60% of TSMC's 2026 advanced packaging capacity, on the order of 595,000 wafers, with 510,000 of those specifically allocated to the CoWoS-L variant used for its newest Rubin-generation chips (Astute Group). AMD, Broadcom, Google and Amazon are reported to be competing for what is left of the remaining 40%. Treat the precise wafer counts as trade-press estimates rather than a company-disclosed figure, since neither TSMC nor Nvidia has published an official allocation breakdown, but the direction is not in dispute: one customer, absorbing more advanced packaging capacity than everyone else combined, is itself a form of scarcity.

Part 4: The 2026 Shift, From Packaging to Memory Itself

Here is where the story gets more interesting for 2026 specifically. TSMC's aggressive capacity expansion is genuinely working: the CoWoS supply-demand gap is reported to be narrowing from roughly 20% earlier in 2026 to about 10% by the end of the year, as new capacity comes online (TrendForce, citing Economic Daily News). Packaging is not solved, TSMC's own CEO still calls it sold out, but the gap is closing.

As that happens, the binding constraint is relocating one layer further upstream, to the HBM memory chips themselves. You can build more packaging lines faster than you can build the specialised fabs and yield expertise needed to make HBM at scale, because HBM manufacturing is its own multi-year capital and know-how problem, not a capacity dial TSMC's customers can simply turn up.

Nvidia's own response to this is the clearest signal of how seriously it takes the risk. For years, SK Hynix supplied the overwhelming majority of Nvidia's HBM needs, essentially a single-supplier relationship for the industry's single most critical component. On June 5, 2026, at an event in Seoul, Nvidia CEO Jensen Huang confirmed that Samsung Electronics, SK Hynix and Micron Technology have all now been certified as qualified HBM4 suppliers for Nvidia's next-generation Vera Rubin AI platform, with all three in production ahead of a third-quarter 2026 commercial ship date (reported by multiple outlets including TechTimes and Yahoo Finance, corroborating each other). Industry analysts estimate the resulting allocation at roughly 60% to 70% of Vera Rubin's HBM4 volume to SK Hynix, 25% to 30% to Samsung, and the remainder to Micron, though Nvidia itself has not published an official split.

The pricing evidence backs this up directly. Multiple industry reports through 2026 describe Samsung and SK Hynix repricing HBM and broader DRAM contracts upward, with some reports describing increases of 20% or more, as demand continues to outrun even the expanded supply base.

Part 5: Who Actually Captures the Profit Here

This is, without question, one of the most profitable layers in the entire AI value chain right now, and unlike a lot of AI-adjacent claims, the numbers here come directly from audited quarterly results rather than press releases.

Sources: SK hynix's own Q1 2026 and Q2 2026 results, Micron's Q3 FY2026 guidance as reported by IG International, and TSMC's Q2 2026 earnings call.

SK Hynix's own newsroom does not break out HBM's exact share of that headline revenue, but it explicitly credits "sales of high-value-added products, including HBM, high-capacity server DRAM modules, and eSSDs" for the record quarter. Independent market research fills the gap: research firms IDC and Counterpoint both placed SK Hynix's global HBM market share at roughly 56% to 58% in the first quarter of 2026, making it comfortably the market leader ahead of Samsung and Micron. In a further sign of how central this business has become, Nvidia and SK Group announced a long-term AI memory supply partnership reportedly worth more than $500 billion, spanning years of future HBM commitments.

TSMC's position is structurally different and arguably safer. It does not depend on a single product category the way a memory maker does; advanced packaging is one input into a foundry business that also fabricates the compute dies themselves, at margins the company has never disclosed separately for packaging alone, though trade press has previously cited advanced packaging margins in the neighbourhood of 80%, well above TSMC's already-high corporate average. TSMC's overall Q2 2026 revenue reached a record $40.2 billion, up 36% year on year, and the company raised its full-year 2026 revenue growth guidance to slightly above 40% in dollar terms, driven overwhelmingly by AI-linked demand.

Part 6: The India Angle, and Why It's Genuinely Different Here

Every other layer of the AI infrastructure story covered on this site so far has at least some Indian listed exposure. Colocation data centers have Bharti Airtel's Nxtra, Adani's AdaniConneX, and Sify. Power and utilities have Tata Power and NTPC. Even IT services, as we covered in our piece on AI and Indian IT, sits adjacent to this buildout as a services supplier.

This layer is different. There is currently no Indian company that designs, fabricates, or packages HBM memory, and no Indian company that operates CoWoS-class advanced packaging at any scale. This is not an oversight or a missed opportunity investors should expect to be corrected soon. It reflects the nature of the business itself: HBM fabrication and CoWoS packaging both require decades of accumulated, closely guarded manufacturing know-how, tens of billions of dollars of committed capital, and access to a genuinely small global pool of the specialised engineering talent capable of running these processes at acceptable yields. Exactly three companies on earth currently ship qualified HBM at meaningful volume, and exactly one company operates CoWoS at meaningful scale. That is not a market structure India, or almost anywhere else, can simply decide to enter.

The Micron Sanand plant is genuinely worth understanding in detail, because it is close enough to this story to cause confusion. It is India's first commercial semiconductor assembly and test facility, and at full capacity is expected to account for up to 10% of Micron's global output, dispatching its first products to customers including Dell, Asus and Qualcomm (TrendForce; Computer Weekly). But its actual product mix, as reported, is conventional DDR5 DRAM modules, stacked GDDR (graphics memory) and enterprise SSDs, not HBM. India's first entry into this industry is real, and it is a meaningful milestone for the country's broader semiconductor ambitions, but it currently sits one layer away from the specific bottleneck this article is about, not inside it.

So what is an honest, indirect way for an Indian investor to think about this layer? There are two answers, and neither is a stock tip. First, global technology and semiconductor funds or ETFs accessible through international investing platforms hold Samsung, SK Hynix (listed in Seoul) and Micron (Nasdaq-listed), which gives genuine, if diluted, exposure to this specific profit pool. Second, and more usefully for readers who already hold Indian AI-infrastructure names, treat this bottleneck as a variable that affects the timing of everything built on top of it: a memory-constrained AI buildout is a slower AI buildout, which has knock-on effects for the data center capex numbers, the colocation absorption rates, and the power demand growth already covered on this site. This layer does not need to be in your portfolio directly for it to matter to what already is.

Part 7: The Honest Risk, Because Every Bull Case Deserves Skepticism

This site does not exist to tell you a profitable-sounding story is risk-free, and this one genuinely is not. The memory chip industry has a three-decade history that should make anyone cautious about extrapolating today's 72% to 76% operating margins indefinitely.

The pattern has repeated multiple times: a genuine shortage drives prices and margins to extreme highs, manufacturers respond by pouring capital into new capacity, and two to three years later, once that capacity finally comes online, supply overshoots demand and prices crash. The 2017-to-2018 DRAM supercycle is the clearest recent example: memory margins reached levels the industry called previously unattainable for what is fundamentally a commodity product, and spot prices then fell more than 50% into 2019 as the resulting overbuild worked through the system. Historically, this boom-bust pattern has run on a roughly four-to-seven-quarter upswing followed by a four-to-eight-quarter downswing, with revenue declines in the 25% to 40% range during the bust phase.

The single most useful thing to track, if you want to judge this for yourself over time rather than trust either the bull or bear framing, is the same discipline this site recommends everywhere else: watch capital expenditure announcements from SK Hynix, Samsung and Micron over the next several quarters. Large, sustained capex increases aimed specifically at HBM capacity are the leading indicator of a future supply glut, exactly as they were in 2017. The absence of that capex signal is the leading indicator that today's scarcity, and today's margins, might actually hold.

Key Takeaways

1. A finished AI chip is really two supply chains that meet only at packaging. Logic design and fabrication (Nvidia designs, TSMC or Samsung Foundry etches) run entirely separately from memory fabrication (SK Hynix, Samsung, Micron), and scarcity compounds at every layer along the way, from ASML's monopoly on the lithography machines TSMC itself depends on, down to the specific bottleneck this article is about.

2. The popular "GPU shortage" story is outdated. By 2026, the binding constraints on AI chip supply sit one and two layers beneath the compute die itself: TSMC's CoWoS advanced packaging, and the HBM memory that gets packaged onto every AI accelerator.

3. TSMC's own CEO has called CoWoS capacity "extremely tight and sold out through 2026," even after the company raced to nearly quadruple monthly capacity, from about 35,000 wafers at the end of 2024 to a targeted 130,000 to 140,000 by the end of 2026.

4. The bottleneck is actively shifting from packaging to HBM memory itself. Nvidia's decision to certify Samsung, SK Hynix and Micron as three parallel HBM4 suppliers for its Vera Rubin platform, rather than relying on SK Hynix alone, is the clearest evidence of how seriously it is managing this risk.

5. This is one of the most profitable layers in the entire AI value chain, with numbers to prove it. SK Hynix posted a 76% operating margin in Q2 2026 on record revenue; Micron guided 81% gross margin for Q3 FY2026; TSMC's high-performance computing segment, including AI packaging, made up 66% of a record $40.2 billion quarterly revenue.

6. India has essentially no direct exposure to this specific layer, unlike colocation, power, or IT services. Micron's own semiconductor assembly plant in Sanand, Gujarat, is a genuine milestone for India's chip industry, but as of this writing it packages conventional DRAM and SSDs, not HBM, making it adjacent to this bottleneck rather than inside it.

7. The closest exposure an Indian investor has is indirect, through global technology funds holding Samsung, SK Hynix or Micron, or simply by understanding that this bottleneck sets the pace for the broader AI buildout covered elsewhere on this site.

8. The memory industry's three-decade boom-bust history is a real risk, not a footnote. The bull case that "this time is structurally different" is genuine and worth taking seriously, but it is also precisely the argument every memory supplier makes at the top of every cycle, and the honest tracking signal, supplier capex, is one every investor can watch for themselves each quarter.

Sources

• Samsung Semiconductor, HBM4 product page. Source of the embedded I/O pin count visualisation in Part 2, and Samsung's own technical description of HBM4.
• Tom's Hardware, "ASML's Planned Low-NA EUV Machine Price Hikes Reportedly Frustrate TSMC," 2026, citing Bloomberg. ASML's sole-supplier position, High-NA EUV machine pricing, and TSMC's decision to delay adoption.
• Foundry market share (TSMC roughly 70%, Samsung Foundry roughly 6.5%, Q1 2026) and the EDA software market structure (Synopsys, Cadence, Siemens EDA), as reported across semiconductor industry trade press including SemiWiki and SemiAnalysis's EDA market coverage in 2026.
• TrendForce, "TSMC CoWoS Supply-Demand Gap Reportedly Seen Narrowing From 20% to 10% by End-2026," June 2026. CoWoS capacity figures and the narrowing supply-demand gap, citing Economic Daily News.
• TechTimes, "TSMC Q2 Earnings July 16: Three CoWoS Signals That Test AI's Spending Ceiling," July 2026. TSMC CEO C.C. Wei's "sold out through 2026" comments.
• TechTimes, "TSMC Posts Record Quarter as AI Chip Demand Pushes Full-Year Growth Outlook Past 40%," July 2026. TSMC Q2 2026 revenue, gross margin, HPC segment share, and revised full-year guidance.
• Astute Group, "Advanced Packaging Demand Soars: Nvidia Secures 60% of CoWoS Capacity." Nvidia's estimated share of TSMC's 2026 advanced packaging capacity.
• TechTimes, "Nvidia Vera Rubin HBM4: Jensen Huang Confirms All Three Suppliers in Production for Q3 Ship," June 2026. Nvidia's certification of Samsung, SK Hynix and Micron as HBM4 suppliers, and estimated allocation splits.
• SK hynix, official Q1 2026 business results and Q2 2026 results as reported by Investing.com. Revenue, operating profit and operating margin figures, directly from the company.
• The Motley Fool, "SK Hynix Supplies More Than Half the World's HBM Memory. Its Stock Costs Under 4 Times Next Year's Earnings," August 2026. HBM market share (citing Counterpoint Research), the Nvidia-SK Group supply partnership, and valuation context.
• IG International, "Micron Q3 Earnings: Record Margins and HBM4 in Focus," 2026. Micron's gross margin guidance and HBM4 production ramp.
• TrendForce, "Inside Look at Micron's Newly Opened Sanand Plant: 10% of Global Output, Key Products and Clients," March 2026, and Computer Weekly, "Micron Opens $2.75bn Chip Assembly Plant in India." Details of the Sanand, Gujarat facility and its actual product mix.
• SemiAnalysis, "Memory Mania: How a Once-in-Four-Decades Shortage Is Fueling a Memory Boom." Structural context on the 2026 memory cycle versus historical DRAM cycles.