GPUs, TPUs, and AI accelerators are subjected to loads that are orders of magnitude greater than those on conventional logic.

Requirements:

・Large numbers of arithmetic units (MACs) operating simultaneously

・Ultra-high current density (on the order of 10 MA/cm²)

・Extremely dense wiring (more layers)

・A sharp rise in power density (W/mm²)

・Huge die sizes (the H100 is 814 mm²)

As a result, heat, current, and reliability become the most important themes in the process.

AI chips are an all-round contest of “scaling technology × power control technology × packaging technology.”

In AI chips, rather than the logic circuits, the wiring tends to become the bottleneck.

Challenges

・Increasing wiring resistance (shifting from Cu to new materials such as Ru/Co)

・Increasing RC delay (limits of Low-k, fragility of Ultra Low-k)

・Limits of adding more layers (12 → 15 → 20 layers)

・Wiring degradation due to heat (electromigration)

Latest Countermeasures

・Barrierless metals (Ru, Mo, Co)

・Hybrid Low-k materials

・ALD barriers for high aspect ratios

・Accelerated adoption of new ALD technologies in BEOL

Wiring has become so important that the AI era is said to be the era of wiring materials.

An AI chip uses several hundred watts on a single server board, and that power is concentrated on a die of a few hundred mm².

Challenges

・Local hot spots

・Thermo-mechanical stress

・Limits of the heat dissipation path from the package to the outside

・Insufficient power delivery network (PDN) capacity

Process-Side Countermeasures

・Optimization of the metal stack

・Thick copper wiring (top metal)

・3D heat dissipation through TSVs and micro-bumps

・Backside power delivery network

Intel and TSMC in particular are introducing backside PDN, supplying power lines from the back side to reduce resistance.

Because AI chips handle enormous currents, reliability failures become pronounced.

Typical reliability problems:

Electromigration (EM)

・A phenomenon in which current causes metal atoms to migrate, leading to open circuits and voids

・Especially serious in AI chips, where wiring is thin and currents are large

TDDB (Time Dependent Dielectric Breakdown)

・Insulating properties gradually deteriorate as an electric field is continuously applied to the insulating film

・Pronounced in areas where High-k is used

BTI (Bias Temperature Instability)

・Vth shift in transistors at high temperatures

・Problems increase in combination with the high heat generation of AI chips

AI chips can be described as devices that battle reliability.

AI chips are manufactured as “huge dies” of around 800 mm². However, if even a single defect on the wafer lands on a die, that die becomes defective.

→ Yield becomes very poor

For this reason, the following have become essential:

・Chiplets

・Multi-die packages

・Stacked structures with HBM (CoWoS, InFO, FO-PLP)

The fact that TSMC and NVIDIA have grown together is due to the combination of leading-edge processes × packaging (CoWoS).

AI chips cannot exist without packaging.

Advantages of Chiplets

・Less waste when defects occur (improved yield)

・Large chips can be divided into smaller pieces for manufacturing

・Heat density is distributed

・HBM (high-bandwidth memory) can be placed close by

Representative Technologies

・TSMC CoWoS

・Samsung I-Cube

・Intel EMIB / Foveros

・FO-WLP (fan-out)

For AI chips, we are in an era in which packaging technology, not just the process, determines performance.

Backside Power Delivery (BSPDN)

→ Supplies power from the back side, greatly reducing wiring resistance

PowerVia (Intel)

→ Eliminates wiring congestion and raises clock frequency

3D Logic × HBM Integration

→ Eliminates the memory bandwidth bottleneck

Dedicated Processes for AI Accelerators

→ Higher heat resistance only in specific areas, localized material changes

Advanced Heat Dissipation Materials (Graphite, Evolving TIMs, Micro Liquid Cooling)

→ Solving the heat problem is the biggest theme

In AI chip manufacturing, the key is the trinity of “scaling × power delivery × thermal management.”

・AI chips operate under harsh conditions of high current, high density, and high heat

・Increasing wiring resistance, RC delay, and EM are major challenges

・With rising heat density, thermal design is more important than in logic

・Lower yield of huge dies → chiplets are essential

・Backside PDN and advanced packaging determine performance

・AI processes are a field where process, wiring, and packaging converge

1.Why does wiring become a bottleneck in AI chips?

2.Explain why backside power delivery is important.

3.Why is EM (electromigration) serious in AI chips?

 

Column Supervisor: Koji Kakumoto (President, OTIS Co., Ltd.)

After studying abroad and working in planning and development at a trading company, he joined OTIS Co., Ltd. in 2011. While primarily working in the Corporate Planning Department, he has also served concurrently in manufacturing and technical divisions, and since 2018 has served as President, working to drive business growth and strengthen the organization.

 

This article is a general technical explanation intended for educational purposes and does not refer to any specific company, product, or technology.