The Edge Efficiency Pivot: Analyzing DeepX's Push to Slash LLM TCO
AI-generated image · US National Wire
By leveraging custom PIM technology and a 2-nm process, DeepX aims to move generative AI from power-hungry GPUs to battery-constrained edge hardware.
For enterprise leaders deploying large-scale language models (LLMs) at the edge, the primary bottleneck is rarely the software architecture—it is the brutal reality of the power budget. When deploying AI into robotics, drones, or automotive systems, the total cost of ownership (TCO) is inextricably linked to thermal management and battery longevity.
As EE Times first reported, South Korean startup DeepX is targeting this specific pain point with the tape-out of its second-generation AI accelerator, the DX-M2. The value proposition is clear: the ability to run generative AI on-device within a power envelope of less than 5 W. For operations leaders, this represents a critical shift from the energy-intensive GPU paradigms that currently dominate the prototyping phase of edge AI.
### The Mechanism: Breaking the Memory Wall
As Lokwon Kim, founder and CEO of DeepX, explained to EE Times, the leap from the first-generation DX-M1 to the DX-M2 centers on memory bandwidth, which has tripled from 50 GB/s to 150 GB/s. However, the more significant architectural pivot is the integration of processor-in-memory (PIM) technology.
Standard AI inference is often throttled by the movement of weights between memory and compute, particularly during general matrix-vector multiplication (GEMV) operations. To solve this, DeepX has collaborated with Samsung Memory for 1.5 years to integrate the DX-M2 with Samsung's custom LPDDR5x-PIM. By computing memory-bound operations directly within the memory chip, DeepX reduces access times and increases effective bandwidth.
According to DeepX simulations cited by EE Times, offloading GEMV operations to the LPDDR5x-PIM results in a 6.7× increase in tokens per second for the Llama3.1-8B model compared to the DX-M2 operating alone. While Kim notes that this performance boost requires 2.7× the power consumption, the company believes it can achieve a 2× performance uplift while remaining within the same power envelope as non-PIM operations.
### Hardware ROI: 2-nm Process and Power Efficiency
From an operational lens, the DX-M2 is designed to make local AI a practical reality for devices with strict thermal budgets. The chip is engineered to support models ranging from 20 billion to 100 billion parameters—with active parameters between 3 billion and 20 billion—and can deliver 10 to 30 tokens per second.
DeepX attributes this efficiency to three primary drivers: 1. **Process Node:** Power efficiency was doubled by moving to a 2-nm process node. 2. **NPU Design:** Design improvements further doubled the NPU's power efficiency. 3. **Compute Output:** The system can now achieve 80 TOPS.
Kim told EE Times that a 5 W power consumption limit allows the technology to be deployed in virtually any battery-powered device, including drones.
### The Competitive Pivot: Migrating from Nvidia
Perhaps the most significant enterprise implication is the direct challenge to the current hardware status quo. Tim Park, director of strategic marketing at DeepX, told EE Times that robotics companies often prototype using GPUs, such as Nvidia Thor, but are forced to seek alternatives due to power and thermal constraints.
To facilitate this migration, DeepX is positioning its "Newton" platform as a direct alternative to the Nvidia Isaac robotics platform. Park notes that customers can migrate by changing their APIs to call Newton instead of Isaac, allowing them to run the same pipeline on DeepX hardware. DeepX currently maintains 17 compatible libraries to support this transition.
According to Kim, the ROI for customers switching from Nvidia to DeepX is substantial, claiming that users can achieve 20 times longer battery life and lower costs.
### Market Application and Future Roadmap
DeepX is targeting several high-stakes enterprise sectors: * **Robotics & Automotive:** Providing low-power chips for in-vehicle agents and robotic systems. * **AI PCs:** Collaborating with Taiwanese OEMs for laptops and workstations. * **IoT:** Working with Samsung on a next-generation smart hub and joining the AWS IoT ecosystem to utilize Greengrass for remote model updates.
Looking forward, the company is preparing the DX-M3, which Kim told EE Times will support official LPDDR6 PIM technology. This is expected to yield a theoretical performance advancement of 5× to 8×.
While the current LPDDR5x-PIM implementation requires "substantial manual effort" in the compiler—which is currently in prototype form based on simulations—the trajectory suggests a concerted effort to move LLMs out of the data center and into the hardware-constrained edge without sacrificing the operational viability of the device.