The 'Meteor Lake' NPU: Meet Intel's Dedicated Silicon for Local AI Processing

19.09.2023 - 16:21 / pcmag.com / Meteor Lake / Ai

When Intel’s “Meteor Lake” processors launch, they’ll feature not just CPU cores spread across two on-chip tiles, alongside an on-die GPU portion, but also the company’s first-ever Neural Processing Unit (NPU) devoted to AI workloads. 

According to Intel, the NPU means generative AI programs such as Stable Diffusion, or today's chatbots working off of a locally hosted language model, can run natively on Meteor Lake laptops, without suffering from slow performance, poor quality, or crippling power demands. 

“At Intel, our goal is to democratize AI, building it around standardized interfaces, making it accessible to anyone, anywhere in the world,” said Tim Wilson, Intel’s general manager for SOC design. 

The resulting NPU promises to offer as much as an eight-times power-efficiency improvement over Intel’s previous chip generation when it comes to AI-based workloads. The company previewed the technology in action back at Computex 2023, at which time the NPU was alternately referred to as the VPU, or “Versatile Processing Unit.” Now the company is sharing more details, including the architecture.

You can already use a PC's GPU to power AI workloads, but doing so can guzzle electricity, which isn't ideal for a computing environment like a battery-constrained laptop. (It's also potentially wasteful in a desktop environment, of course.) Intel's solution is the NPU, which has been "optimized for long-sustained, power moderate, intensive applications,” according to Tom Petersen, an Intel Fellow specializing in computer graphics. "So if you’re doing a sustained workload, then the NPU is for you." An example of this kind of NPU-optimal task might be using an AI image generator to churn out numerous pictures from a locally hosted model.

The NPU itself is attached to the Meteor Lake processor through a chiplet or tile (specifically, the SoC Tile, one of four on the die), which also contains a section dubbed “Low Power Island.” The Island is home to a special subset of Efficient cores (E-cores) meant for low-power workloads. The Compute Tile is home to the ordinary Performance cores and E-cores familiar from previous-generation Intel processors. (Wander over to our main Meteor Lake explainer for more on the architecture of the new chips, including a tile-by-tile breakdown.)

The NPU will operate over two Neural Compute engines, which can each work on its own AI workload, or join forces to tackle a single workload. Each engine contains an inference pipeline that can process incoming data and transform it.

“These things are all programmed using our software, and provide industry-standard APIs at the top to make it very easy to adopt in software,” Petersen added. The NPU is also separate to the point that it will