■ Why Do GPUs Get Hot?
Inside a GPU, an enormous number of transistors operate at high speed.
Current flows through them.
Where current flows, there is resistance.
Where there is resistance, heat is generated.
In other words, heat generation in GPUs is not a special phenomenon;
it is a phenomenon that always occurs as long as electricity is used to compute.
■ In the AI Era, the Load on GPUs Increases by Orders of Magnitude
Conventional GPUs were often used for games and image processing.
In the AI era, however, things are different.
GPUs are used for extremely heavy computation, such as
● Training generative AI
● Inference for large-scale AI models
● Image and video generation
● Autonomous driving AI
● Robot control
In other words, the GPU is no longer a component that works temporarily;
it has become a component that keeps working under high load at all times.
Naturally, heat problems become greater.
■ The Smaller Semiconductors Become, the Hotter They Get
For many years, semiconductors have evolved through miniaturization.
By making them smaller, they have achieved
● Higher speed
● Lower power consumption
● Higher density
In the AI era, however, simple miniaturization alone is no longer enough.
More computing circuits are packed into smaller areas.
As a result, heat density rises sharply.
■ The Problem Is Heat Density, Not Temperature
This is the important point.
The GPU heat problem is not simply a matter of high temperature.
The real problem is that a large amount of heat is concentrated in a small area.
For example, even with the same 100 W of heat generation, the difficulty is completely different between a case where it is spread across a large plate and a case where it is concentrated on a small chip.
In AI semiconductors, this heat density becomes extremely high.
■ Hot Spots Stop Performance
Inside a semiconductor, the whole does not heat up uniformly.
There are places where only some parts become extremely hot.
These are called hot spots.
When hot spots occur,
● Reduced computing performance
● Errors
● Degradation
● Shorter lifespan
● Performance limits due to protective control
occur.
In other words, in AI semiconductors, localized heat concentration, rather than average temperature, becomes the problem.
■ GPUs That Cannot Be Cooled Cannot Deliver Performance
When GPUs get hot, they may reduce their own performance.
This is so-called thermal throttling.
It is a control in which the GPU lowers its operating speed to prevent failure.
In other words, even if a high-performance GPU is installed,
if heat cannot be released, it cannot deliver its catalog performance.
■ Heat Problems Become Even More Difficult with 3D Semiconductors
In the AI era, not just the GPU alone,
but structures in which high-performance memory such as HBM is placed close around it are increasing.
Furthermore, in the future, 3D semiconductors, in which semiconductors are stacked vertically, will also become important.
However, stacking makes the escape routes for heat more complex.
There are heat sources both above and below.
You want to release heat, yet the components that generate heat are stacked on top of each other.
In other words, 3D semiconductors are a technology for higher performance and, at the same time,
a structure that tends to trap heat.
■ The Entire Semiconductor Package Becomes the Target of Thermal Design
In the past, attention was focused on the performance of the chip alone.
Today, however, things are different.
In AI semiconductors, thermal design must be considered including
● Chips
● Memory
● Substrates
● TIMs
● Heat spreaders
● Heat sinks
● Liquid cooling plates
In other words, the heat problem is no longer a problem of the semiconductor alone,
but a problem of the entire package.
■ Heat Tends to Stop at Interfaces
What matters here is the contact thermal resistance discussed in the previous chapter.
To release heat from a semiconductor, the heat passes through several materials.
For example,
Chip
↓
TIM
↓
Heat spreader
↓
Cooling plate is the path.
At this point, if there are, at the boundaries between materials,
● Air layers
● Lifting
● Insufficient pressure
● Surface roughness
● Uneven thickness
heat has difficulty flowing.
In other words, in heat problems for GPUs and semiconductors,
interface performance can become more dominant than material performance.
■ Why It Does Not Cool Even with High-Performance Materials
It may seem that using materials with high thermal conductivity solves everything.
In reality, however, thermal performance changes greatly just because
● It is not in close contact
● The thickness varies
● The pressure is insufficient
● There are fine steps
● Contact changes due to thermal expansion
In other words, using good materials and
being able to use them in good condition are different things.
If you get this wrong, thermal management for AI semiconductors will not work.
■ The Researcher's Perspective: Structures That Can Release Heat Become the Axis of Competition for Semiconductors
In research and development, progress is being made in
● High thermal conductivity materials
● Next-generation TIMs
● Liquid cooling plates
● Heat spreading structures
● Thermal analysis of 3D packaging
● Low-power design
However, the essence is, from semiconductors that have become higher-performance,
how to release heat to the outside.
■ The Shop-Floor Perspective: In the End, It Falls Apart in Volume Production
Even if it cools in prototyping, the results can change in volume production.
The reasons include
● Lamination misalignment
● Pressure variation
● Variation in material thickness
● Differences in surface condition
● Deformation due to thermal expansion
● Unit-to-unit differences in the assembly process
In other words, in heat problems for GPUs and semiconductors, cooling in theory is not enough.
It must cool even in volume production.
■ OTIS's Perspective
From OTIS’s perspective, what matters in heat problems for GPUs and semiconductors is
● High-precision processing of heat dissipation materials
● Processing of insulating heat dissipation materials
● Thin-film processing
● High-precision lamination
● Lamination of dissimilar materials
● Die-cutting of fine shapes
● Dimensional stability in volume production
In the world of AI semiconductors, not only the thermal management materials themselves,
but also how those materials are processed, how they are applied, and how they are mass-produced becomes important.
■ What OTIS Can Do
At OTIS, through
● Graphite sheet processing
● Heat dissipation sheet processing
● Insulating material processing
● High-precision die-cutting
● Microfabrication
● High-precision lamination
● Stacking of irregular shapes
● Support for reel supply, and so on,
we may be able to contribute to bringing thermal management components around GPUs and semiconductors, with reproducibility,
closer to a form that can be used in volume production.
■ Outside OTIS's Expertise
On the other hand, OTIS is not a company that specializes in
● GPU design
● Semiconductor circuit design
● Semiconductor front-end processes
● AI chip development
● Design of liquid cooling equipment itself
However, in the area of making thermal management components work in volume production processes, we may be able to play an important role.
■ Industries Where This Technology Will Be Important
★★★★★ AI semiconductors
★★★★★ GPUs and accelerators
★★★★★ Data centers
★★★★★ 3D semiconductors
★★★★☆ High-performance servers
★★★★☆ Edge AI devices
■ Summary
Semiconductors in the AI era will be decided by whether they can release heat
GPUs and semiconductors are the core components of the AI era.
However, the higher their performance, the higher the heat density.
The higher the heat density, the more likely hot spots are to occur.
And if heat cannot be released, no matter how high-performance a semiconductor is, it cannot deliver its true performance.
In other words, semiconductor competition in the AI era is a competition in computing performance and, at the same time, a competition in releasing heat.
And behind the scenes, processing, contact, lamination, and interface design for making high-performance materials work in volume production
will become increasingly important.
Column Supervisor: Koji Kakumoto (President, OTIS Co., Ltd.)
After studying abroad and working in planning and development at a trading company, he joined OTIS Co., Ltd. in 2011. While primarily working in the Corporate Planning Department, he has also served concurrently in manufacturing and technical divisions, and since 2018 has served as President, working to drive business growth and strengthen the organization.
This article is a general technical explanation intended for educational purposes and does not refer to any specific company, product, or technology.
