AI accelerator module (edge AI M.2 card and AI system-on-module)

01

Pick an accelerator card or a compute module

Geniatech's AIM-M-H8 is an M.2 AI acceleration module built on the Hailo-8 NPU, connecting to a host processor through PCIe Gen3 on an M.2 Key M slot; it lists up to 26 TOPS (INT8), 3.3 W power consumption, −40 to 85 °C operation, 22 × 42, 22 × 60 or 22 × 80 mm sizes, Linux and Windows support, model support for TensorFlow, TensorFlow Lite, Keras, PyTorch and ONNX, and host platforms including NXP, Renesas, TI, Rockchip, SocioNext and Xilinx. [2]

NVIDIA's Jetson Orin family is a set of compute modules with identical architecture, each combining an Ampere GPU with Tensor cores, Arm Cortex-A78AE CPU cores and LPDDR5 memory: AGX Orin modules up to 275 TOPS at 15–60 W on a 699-pin connector (100 × 87 mm), Orin NX modules up to 157 TOPS at 10–40 W and Orin Nano modules up to 67 TOPS at 7–25 W, both on a 260-pin SO-DIMM connector (69.6 × 45 mm). [1]

A second M.2 card shows that some accelerator cards carry their own memory. DEEPX lists its DX-M1 M.2 module at 25 TOPS (INT8) over PCIe Gen3 x4 (supporting Gen 1/2/3 and x1/x2/x4), with 4 GB LPDDR5 at 5,600 MT/s on the module, 1 W minimum to 5 W maximum power, −40 to 85 °C operation, an optional heatsink, and the M.2 2280 M-key form factor (22 × 80 mm); it lists Windows 11/10, Debian-based Linux (Ubuntu 24.04/22.04/20.04 LTS), Yocto and Docker, the Ultralytics, PyTorch, TensorFlow, ONNX and Keras frameworks, and x86 and Arm host systems. The DX-M1 chip itself is a 17 × 17 mm 625-ball FC-BGA supporting up to 8 GB of LPDDR4x or LPDDR5, offered in commercial (0 to 70 °C), industrial (−40 to 85 °C) and AEC-Q100 Grade 3 grades, with Grade 2 (−40 to 105 °C) on request. [3]

Editorial note: an M.2 card needs a host CPU, memory, operating system and an M.2 slot with the right key and PCIe lanes; a system-on-module needs a carrier board but brings its own CPU, memory and I/O. [2] [1]

Give the supplier the model types and input sizes, required throughput (frames or inferences per second) and precision, host platform or carrier board, interface and connector, power and thermal budget, memory needs, camera and video I/O, operating temperature, operating system and AI framework, supply life and quantity.

02

Size AI performance, power and memory

NVIDIA's comparison table lists AI performance of 275 TOPS (AGX Orin 64GB), 248 TOPS (AGX Orin Industrial), 241 TOPS (AGX Orin 32GB), 157 TOPS (Orin NX 16GB), 117 TOPS (Orin NX 8GB), 67 TOPS (Orin Nano 8GB) and 34 TOPS (Orin Nano 4GB), with power modes from 7–25 W (Orin Nano) to 15–75 W (AGX Orin Industrial) and memory from 4 GB 64-bit LPDDR5 at 51 GB/s to 64 GB 256-bit LPDDR5 at 204.8 GB/s, the industrial version adding ECC. [1]

The same table lists 2× NVDLA v2 deep-learning accelerators on AGX Orin, video encode up to 2× 4K60 H.265 and decode up to 1× 8K30, up to 16 lanes of MIPI CSI-2 camera input, PCIe Gen4 lanes and up to 10 GbE networking on the AGX Orin module. [1]

Editorial calculation: 26 TOPS at 3.3 W is about 7.9 TOPS/W for the Hailo-8 card; 157 TOPS at the 40 W top power mode is about 3.9 TOPS/W for Orin NX 16GB — but peak TOPS and maximum power are not necessarily measured together, so ask for performance at your power mode. 204.8 GB/s on a 256-bit bus is 204.8 ÷ 32 bytes = 6.4 GT/s. [2] [1]

Editorial calculation: the DX-M1 M.2 module's 25 TOPS at its 5 W maximum is 5 TOPS/W, against about 7.9 TOPS/W for the Hailo-8 card at its listed 3.3 W; as with the Jetson figures, these divide peak TOPS by a maximum or listed power and are not measured efficiency on your model. [3] [2]

03

Check that the M.2 slot can power and cool the card

MemryX's datasheet for its MX3 M.2 AI Accelerator Module, which carries four MX3 chips that appear to the software as one logical unit, gives the numbers a host design must meet. It lists a 3.3 V ± 5% input, PCIe Gen 3 with 2 lanes, the M.2-2280-D5-M (Socket 3) form factor at 22 × 80 mm, ARM, x86 and RISC-V host support, −40 to +85 °C operation and CE / FCC Class A and RoHS compliance. Citing the PCI Express M.2 Specification Rev 3.0, it gives two power limits: 3,500 mA, or 11.55 W, in the power rating table, and 4,500 mA (500 mA per pin), or 14.85 W, in the adaptor power section. [4]

The datasheet's board power table puts the module's total maximum load power at 12.97 W and maximum input power at 14.85 W, and states that the module is designed to fulfil the maximum power the host can provide. Its thermal simulation table gives a system TDP of 9.27 W in the worst case at 70 °C ambient with a heatsink and at least 3 CFM of airflow, 7.7 W at 70 °C with 2 CFM, 7.7 W at 50 °C with 1 CFM, and 5.15 W at 25 °C without a heatsink and with 1 CFM. [4]

Editorial calculation: 3.3 V × 3.5 A = 11.55 W and 3.3 V × 4.5 A = 14.85 W, matching the two limits above. The Hailo-8 card's listed 3.3 W and the DX-M1's 5 W maximum sit well inside 11.55 W, while a 14.85 W maximum input does not, so for a higher-power card confirm that the host's M.2 slot or adaptor can deliver the current, and that the enclosure gives the airflow the maker's thermal case assumes at your ambient temperature. [4] [2] [3]

04

Compare what suppliers offer

Suppliers linked to this page list AI accelerator modules; each card links to the page reviewed.

The listing reviewed covers M.2 NPU accelerator cards. Ask each supplier for benchmark results on your model at your power mode, the thermal design, driver and SDK versions and the product's supply life. [2]

05

Trace what each part controls

This is a functional reference built from the sources cited; it is not one maker's specification.

06

Fill the specification fields with values for your device

An AI accelerator module quotation needs values for your device. The table explains each field, gives published values where a reviewed source states them, and says what to obtain from the supplier.

07

Send the model, host and power data together

Paste it into your enquiry below or into an email to a maker, then replace every bracket.

Models: [type; input size; fps]
Architecture: [M.2 card; SoM]
Host/carrier: [platform; slot; connector]
Power/thermal: [W; cooling]
Memory: [GB; ECC]
I/O: [cameras; video; network]
Environment: [°C]
Software/supply: [OS; SDK; years; quantity]

See manufacturer records Send an enquiry through CNFX