■ AI Is a Giant Heat-Generating Device
AI looks like software.
In reality, however, it runs on hardware such as
● GPUs
● CPUs
● Memory
● Communication equipment
And these execute enormous amounts of computation.
The more computation increases, the more power consumption increases.
And almost all of the power consumed turns into heat.
In other words, AI is a technology that converts electricity into intelligence and, at the same time,
a technology that converts electricity into heat.
Perhaps it is something like the fever humans get from thinking too hard.
■ The Smarter AI Gets, the More Heat Increases
In the AI industry, larger models,
more data,
and faster inference are being demanded.
In other words, the more AI evolves, the more the amount of computation increases.
As the amount of computation increases, the amount of heat generated also increases.
This cannot be avoided.
■ Why Do GPUs Get Hot?
Inside a GPU, an enormous number of transistors are operating.
When current flows, heat is generated by resistance.
This is the same principle as Joule heating, which we learn in junior high school science.
In other words, a hot GPU also means that it is doing that much work.
Humans should obviously not work when they have a fever; I can’t help feeling sorry for GPUs.
■ The Real Challenge of AI Servers
When people hear “AI competition,” they tend to focus on which company will create superior AI.
However, data center operators and server manufacturers are looking at a different problem.
That is, how to handle this heat.
■ AI Servers Are Determined by Cooling Capacity
For example, no matter how high-performance the GPUs installed are,
if they cannot be cooled, performance must be reduced.
This is called thermal throttling.
In other words, whether they can actually be cooled matters more than theoretical performance.
■ The Limits of Air Cooling Are Beginning to Show
Conventional servers mainly used air cooling with fans.
In AI servers, however, the amount of heat generated is rising sharply.
Then air alone can no longer carry away all the heat.
That is why, in recent years, the shift to liquid cooling has been advancing rapidly.
■ Why Liquid Cooling?
The reason is simple.
Water carries heat
far more easily than air.
That is why structures such as
GPU
↓
Cold plate
↓
Coolant
↓
Heat exchanger are increasing.
In other words, today’s AI servers are becoming less like computers
and more like cooling facilities.
Indeed, humans too want to go to the sea or the pool when it is hot.
■ AI Servers Are a Battle Against Heat Density
In the past, with servers, the whole room was hot.
Today, however, things are different.
What becomes a problem is a state in which only some parts are abnormally hot.
This is called a hot spot.
It is especially likely to occur in
● GPUs
● HBM memory
● Areas around power supplies
■ Why Is Heat Density a Problem?
For example, even 100 W of heat generation is not a problem if it is spread over an area the size of a desk.
However, if it is concentrated in an area the size of a postage stamp, the temperature rises sharply.
In other words, the challenge for AI servers is
not the amount of heat generated but heat density.
■ Heat Problems Will Get Even Worse
The AI industry will continue to aim for even higher performance.
In other words,
● Larger GPUs
● Increasing computation
● Increasing power consumption
will continue.
Then heat problems will become even more serious.
In the future, it is even possible that AI competition = cooling competition.
■ The Researcher's Perspective: Thermal Design Supports the Evolution of AI
What is currently being researched includes
● Liquid cooling
● Immersion cooling
● Vapor chambers
● High thermal conductivity materials
● Next-generation TIMs
However, the essence is where to release heat.
■ The Shop-Floor Perspective: In the End, It Comes Down to Contact
No matter how excellent the heat dissipation material used,
if there is
● Lifting
● Air layers
● Insufficient pressure
it will not cool.
In other words, in AI servers, contact thermal resistance becomes important.
■ OTIS's Perspective
In thermal management for AI servers,
● Heat dissipation material processing
● Insulating heat dissipation material processing
● Microfabrication
● High-precision lamination
● Joining of dissimilar materials
● High-precision die-cutting, and so on become important.
In particular, it is important to make high-performance materials work in volume production.
■ What OTIS Can Do
At OTIS, through
● Heat dissipation sheet processing
● Graphite processing
● High-precision lamination
● Microfabrication
● Stacking of irregular shapes
● Support for automated supply, and so on,
we may be able to contribute to making thermal management components for AI servers work in volume production.
■ Outside OTIS's Expertise
OTIS is not a company that specializes in
● GPU design
● AI model development
● Data center operation
● Liquid cooling system development
However, in the area of making thermal management materials work in volume production, we may be able to play an important role.
■ Industries Where This Technology Will Be Important
★★★★★ AI servers
★★★★★ Data centers
★★★★☆ Cloud services
★★★★☆ Edge AI
★★★★☆ Supercomputers
■ Summary
Many people think that AI competition is a competition of intelligence.
In reality, however, behind the scenes, a battle against heat is taking place.
The more AI evolves, the more heat it generates.
And from now on, not only how smart an AI can be built,
but also how much heat can be handled may become what determines competitiveness.
Column Supervisor: Koji Kakumoto (President, OTIS Co., Ltd.)
After studying abroad and working in planning and development at a trading company, he joined OTIS Co., Ltd. in 2011. While primarily working in the Corporate Planning Department, he has also served concurrently in manufacturing and technical divisions, and since 2018 has served as President, working to drive business growth and strengthen the organization.
This article is a general technical explanation intended for educational purposes and does not refer to any specific company, product, or technology.
