■ The Heat Problems of AI Servers Are Not Only About "Chips"
Even if the GPU is cooled, the server as a whole does not necessarily work.
For example, power circuits.
Because AI servers handle large amounts of power, heat is also generated by power supply components.
Thermal management is also needed for
● MOSFETs
● Inductors
● Transformers
● Capacitors
Here, materials such as
● Heat dissipation sheets
● Insulating materials
● Gap fillers
● Thermally conductive adhesives
may be used.
■ VRMs Are Small but Tend to Generate a Lot of Heat
Because they supply large currents to the GPU, VRMs are also heavily loaded.
VRM components are not as large as GPUs, but their heat generation density can be high.
Moreover, there are many components around them.
That is why it is hard to place large heat sinks directly on them.
Here,
● Thin TIM
● Small heat dissipation sheets
● Insulating heat dissipation materials
become important.
In other words, AI servers need local thermal management not only for large heat sources,
but also for small, high-heat components.
■ Optical Communication Components Also Generate Heat
AI data centers communicate large amounts of data at high speed.
As communication speeds increase,
● Optical modules
● DSPs
● Driver ICs
● Lasers
also generate heat.
In other words, thermal management for AI servers extends not only to computation, but also to communication.
For the AI server as a whole,
GPUs,
memory,
power supplies,
and communications
all need their heat handled.
■ The Suitable Cooling Method Differs by Component
Liquid cooling for GPUs.
Heat dissipation sheets for VRMs.
TIM for communication components.
Graphite for enclosures.
In this way, the entire AI server is not necessarily cooled by a single method.
What matters is
using cooling methods selectively according to heat generation and mounting conditions.
In other words, the AI server becomes a collection of thermal management technologies.
■ Cooling Equipment Also Needs Power
AI servers use large amounts of power.
And to handle that heat, power is also needed for
● Fans
● Pumps
● Chillers
● Air conditioning
In other words, it becomes a structure of
Using power for computation.
↓
Heat is generated.
↓
Using even more power to handle that heat.
That is why, in AI data centers, cooling efficiency itself is directly linked to economics.
■ The Idea of Reusing Heat Also Emerges
If large amounts of heat can be recovered with liquid cooling,
it is also conceivable to use that heat for
● Building heating
● Hot water supply
● Factories
● Agricultural facilities
In other words, the heat problems of AI servers expand from a problem of cooling
to an energy problem of where to take the heat.
■ Optimizing Cooling with AI Control
AI servers do not always run at the same load.
Times when GPU load is high.
Times when it is low.
Times when communication volume is high.
Standby states.
Heat generation changes accordingly.
So
● Temperature
● Power consumption
● GPU load
● Coolant flow
● Fan speed
are monitored, and cooling capacity is raised only where needed.
In other words, control that cools AI with AI also becomes important.
■ However, Even AI Control Cannot Fix Poor Contact
AI judges that “the GPU is hot” and raises fan speed.
It increases coolant flow.
Even so, if the TIM is lifting, heat does not transfer to the cold plate.
In other words, no matter how advanced control technology becomes, it will not cool unless
the physical heat path
is correct.
Here too, the importance of interface components does not change.
■ Volume Production Capability Is Also Important in AI Servers
If a skilled person carefully applies just one high-performance TIM, good thermal performance can be obtained.
However, when mass-producing AI servers,
● TIM
● Insulating materials
● Gaskets
● Graphite
● Adhesives
must be incorporated in large quantities, in the same positions, in the same state.
With manual labor alone, variation arises in
● Application position
● Pressure
● Air bubbles
● Foreign matter
That is why thermal management components themselves must also be made into forms suited to automation.
■ The Concept of Reel Supply
Supplying small thermal management components
as loose pieces one at a time.
Rather than that,
if they can be arranged at a constant pitch on a carrier
and supplied in reel form,
it is easier to connect to
● Automated peeling
● Automated pick-up
● Automated application
In other words, not only material performance, but even the supply format becomes part of the component specification.
■ Integration Can Reduce Customer Processes
For example, suppose a customer performs three processes:
1.Applying insulating film
2.Applying TIM
3.Applying graphite
If these are laminated with high precision in advance and supplied as a single component, application processes may be reduced.
Furthermore,
● Misalignment
● Foreign matter contamination
● Work time
can also be reduced.
In other words, in thermal management for AI servers, there is value not only in selling high-performance materials,
but in making them into functional components customers can incorporate as is.
■ However, Over-Integration Also Increases Thermal Resistance
Integrating components reduces processes.
However, as
adhesive layers,
insulating layers,
and protective layers
increase, thermal resistance may also increase.
Furthermore, problems of
● Warping
● Thickness
● Peelability
also increase.
That is why the goal is not minimizing the number of components.
It is necessary to choose the configuration in which thermal performance and volume production capability are most stable.
■ Processing Accuracy Becomes Thermal Performance
In components for AI servers,
● Burrs of several tens of μm
● Application misalignment of 0.1 mm
● Small air bubbles
● Slight warping
may affect thermal performance.
In other words,
processing accuracy is not just drawing quality.
It becomes cooling performance itself.
This is a major point of contact between AI servers and precision processing.
■ Cooling in Prototyping and Continuing to Cool in Volume Production Are Different Things
At the development stage, thermal evaluation can be done in an ideal state.
In volume production, however,
● Material lots
● Die wear
● Processing speed
● Tension
● Temperature and humidity
● Application equipment
change.
Even if a good temperature can be achieved with just one piece,
reproducing the same temperature performance across tens of thousands of pieces is a different technology.
In AI servers, the higher the performance, the smaller the allowable temperature margin tends to become.
That is why reproducibility of thermal performance is important.
■ The Researcher's Perspective: The Heat Problems of AI Servers Cannot Be Solved with One Material
The heat path of an AI server contains many materials, such as
● Silicon
● TIM
● Metals
● Graphite
● Air
● Coolant
In other words,
just “choosing the material with the highest thermal conductivity” does not solve it.
What is needed is
lowering the total thermal resistance from the heat source to the final heat discharge destination.
Good TIM alone will not cool it.
High-performance liquid cooling alone will not cool it.
High-performance graphite alone will not cool it.
The entire heat path must be connected.
■ The Shop-Floor Perspective: For AI Server Heat Problems, Search for "Where the Heat Is Stopping"
The GPU temperature is high.
At this point,
one must not jump to the conclusion of
insufficient fans,
or insufficient liquid cooling.
The cause could be anywhere, such as
● TIM
● Contact pressure
● Air bubbles
● Heat spreader
● Coolant
● Airflow
● Thermal interference from power supply components
What matters is checking in order from the heat source to final heat discharge.
■ Things to Check in Thermal Management for AI Servers
GPUs and semiconductors
● How much heat is generated?
● How high is the heat generation density?
● Where are the hot spots?
● Are there effects from surrounding heat sources such as HBM?
Interface
● Is the TIM appropriate?
● Is the TIM thickness appropriate?
● Are there air bubbles or lifting?
● Is contact pressure appropriate?
● Can package warping be absorbed?
Heat spreading
● Is it necessary to spread heat sideways?
● Is graphite needed?
● Are copper or aluminum suitable?
● Is a vapor chamber needed?
Cooling
● Does air cooling work?
● Is liquid cooling needed?
● Should only high-heat components be liquid-cooled?
● Where is heat ultimately discarded?
Insulation and sealing
● Is insulation from metal components sufficient?
● Are the seals in the liquid cooling section appropriate?
● Are there effects from burrs or foreign matter?
● Can performance be maintained after temperature cycling?
Volume production
● Can it be applied automatically?
● Can it be supplied on reels?
● Is component pitch stable?
● Can multiple components be integrated?
● Can the same contact state be reproduced in volume production?
In other words, thermal management for AI servers is not material selection, but
designing the heat path and the volume production process at the same time.
■ OTIS's Perspective: Even Without Making AI Servers Themselves, We Can Enter the Heat Bottleneck
OTIS is
not a GPU manufacturer.
Not an AI server manufacturer.
Not a liquid cooling equipment manufacturer.
Not a vapor chamber manufacturer.
However, inside AI servers,
between high-performance semiconductors and high-performance cooling equipment, there are large numbers of thin functional components.
TIM.
Graphite.
Copper foil.
Aluminum foil.
Insulating films.
Thermally conductive adhesives.
Seal materials.
EMI materials.
These are the parts that connect heat sources and cooling equipment.
This is OTIS’s main battlefield.
■ The Challenges OTIS Should Target in AI Servers
We do not target the finished AI server itself.
We target
the common processing challenges
that exist within it.
For example,
● High-precision processing of TIM around GPUs
● Thin-film insulation around HBM
● Irregular-shape processing of graphite
● Heat-spreading components of copper and aluminum foil
● Seal materials around cold plates
● Insulating components for liquid cooling sections
● Heat dissipation and insulating components for VRMs
● TIM for optical modules
● High-precision lamination of multiple materials
● Reel supply for automated application
These are challenges that tend to arise in common even when AI server models change.
■ Not "Selling Materials," but "Making Interfaces Work"
OTIS does not need to become a TIM manufacturer.
Nor does it need to become a graphite manufacturer.
We procure high-performance materials from material manufacturers
and process them into the required
● Shapes
● Thicknesses
● Laminations
● Positions
● Supply formats
In other words, we turn high-performance materials into a state that can actually be used in AI servers.
This is OTIS’s value.
■ AI Servers Are a Good Match for Integration Proposals
For example,
TIM
＋
Insulating material
＋
Adhesive
or
Graphite
＋
Insulating film
＋
Fixing material
are laminated in advance.
The customer only needs to apply it once.
This may lead to
● Fewer processes
● Stable application position
● Less foreign matter
● Automation
The further the density of AI servers advances,
the more difficult processes in which people apply sheets one at a time become.
That is why making composite functional components is an important direction.
■ What OTIS Can Do
At OTIS, through
● TIM processing
● Heat dissipation sheet processing
● Graphite processing
● Copper foil processing
● Aluminum foil processing
● Insulating film processing
● Thermally conductive adhesive processing
● Seal material and foam processing
● EMI material processing
● High-precision die-cutting
● Microfabrication
● High-precision lamination
● Lamination of dissimilar materials
● Partial lamination
● Half-cutting
● Reel-to-reel processing
● Supply formats for automated application
● Integration of multiple functional materials
● Volume production process design including image inspection
we may be able to contribute to
turning thermal management materials for AI servers into components that can actually be used in volume production processes.
■ Outside OTIS's Expertise
OTIS is not a company that specializes in
● GPU design
● HBM design
● AI server design
● Data center design
● Design of liquid cooling equipment itself
● Cold plate flow path design
● Design of vapor chambers themselves
● Dedicated CFD analysis
● Formulation development of thermal materials themselves
However, by working with
● Semiconductor manufacturers
● Material manufacturers
● Server manufacturers
● Cooling equipment manufacturers
● Equipment manufacturers
we may be able to contribute in the area of making the functional components that connect heat sources and cooling equipment work in volume production.
■ Markets Where This Technology Will Be Important
★★★★★ AI servers
★★★★★ AI data centers
★★★★★ GPUs and AI accelerators
★★★★★ HBM and advanced packaging
★★★★★ Optical communication and high-speed communication equipment
★★★★☆ Edge AI
★★★★☆ High-performance computing
★★★★☆ AI robots
■ Summary
The AI server performance race is also a race in cooling capacity
AI servers use large amounts of power.
As a result, large amounts of heat are generated.
Furthermore,
GPUs,
HBM,
power supplies,
and communications:
these high-heat components gather in a narrow space.
That is why, in AI servers, making semiconductors higher-performance is not enough.
To maintain that performance, heat must continue to be sent outside.
For that, a long heat path must be made to work:
receiving heat with TIM,
spreading heat with graphite and metals,
diffusing heat with vapor chambers,
carrying heat with air or liquid cooling,
and finally discarding heat outside the data center.
And along that heat path,
there are inconspicuous components of a few tens of μm to a few mm.
TIM.
Insulating materials.
Adhesives.
Graphite.
Metal foils.
Seal materials.
These small components have
a 0.1 mm lift.
Air bubbles.
Burrs.
Shifted positions.
That alone can make it impossible to fully use the capacity of high-performance cooling equipment.
In other words, what becomes important in the AI server era is not only
choosing the highest-performance cooling equipment.
It is connecting the entire heat path from the heat source to the final heat discharge destination.
And what OTIS should target is
not the AI server itself.
Not the GPU itself.
Not the cooling equipment itself.
It is the common challenges in between:
heat, insulation, fixing, sealing, thin films and automation.
The more AI performance rises,
the more severe heat becomes.
The more advanced cooling technology becomes,
the more important the accuracy of the thin components in between also becomes.
That is why, in the AI era,
the technology to make “the last 1 mm of heat” work in volume production
is becoming more important than ever.
Column Supervisor: Koji Kakumoto (President, OTIS Co., Ltd.)
After studying abroad and working in planning and development at a trading company, he joined OTIS Co., Ltd. in 2011. While primarily working in the Corporate Planning Department, he has also served concurrently in manufacturing and technical divisions, and since 2018 has served as President, working to drive business growth and strengthen the organization.
This article is a general technical explanation intended for educational purposes and does not refer to any specific company, product, or technology.
