The conventional computer configuration was a structure in which the CPU was at the center and controlled peripheral devices.

In AI computation, however, there are more and more cases in which GPUs and AI accelerators take the leading role.

Typical configuration:

・CPU (control and general processing)

・GPU (parallel computing)

・AI accelerator (processing dedicated to machine learning)

GPUs in particular deliver performance suited to AI computation thanks to a parallel structure with thousands to tens of thousands of compute units.

Semiconductors for AI have already been developed by many companies.

Main examples:

・NVIDIA: GPU (the mainstream for AI training)

・Google: TPU (Tensor Processing Unit)

・Apple: Neural Engine

・AMD: AI accelerators

These chips are designed to accelerate AI-specific computations such as matrix operations and tensor operations.

AI processing has characteristics that differ from conventional computation.

Main characteristics:

・Centered on parallel computing

・Memory bandwidth is important

・High power consumption

In AI chips in particular, not only computing performance but also the data transfer speed to and from memory determines performance.

This is one of the reasons for the accelerating adoption of HBM (High Bandwidth Memory).

In AI servers, system configurations that combine multiple chips have become common.

Typical configuration:

・CPU

・GPU / AI accelerator

・HBM memory

・High-speed interconnect

In this configuration, not only the performance of individual chips but also packaging technology and interconnect technology become very important.

In semiconductor design for the AI era, the following directions are strengthening.

・More dedicated chips (ASICs)

・Adoption of chiplet structures

・Use of advanced packaging technology

・Design focused on power efficiency

In other words, semiconductors are evolving from a performance race of single chips toward optimization of the entire system.

・AI is the biggest driver of semiconductor evolution

・From CPU-centric to AI accelerator-centric

・Dedicated chips such as GPUs and TPUs are increasing

・HBM and advanced packaging are becoming important

1.Why are GPUs and dedicated accelerators important for AI computation?

2.Why is HBM important for AI chips?

3.How is semiconductor design changing in the AI era?

 

Column Supervisor: Koji Kakumoto (President, OTIS Co., Ltd.)

After studying abroad and working in planning and development at a trading company, he joined OTIS Co., Ltd. in 2011. While primarily working in the Corporate Planning Department, he has also served concurrently in manufacturing and technical divisions, and since 2018 has served as President, working to drive business growth and strengthen the organization.

 

This article is a general technical explanation intended for educational purposes and does not refer to any specific company, product, or technology.