Generative compilation supports four times more models, reduces implementation time by orders of magnitude

TORONTO: Untether AI®, a leader in energy-centric AI inference acceleration today introduced a breakthrough in AI model support and developer velocity for users of the imAIgine® Software Development Kit (SDK). Using a breakthrough generative compiler technology, the upcoming release the imAIgine SDK will support 4 times more AI models than the previous releases. Additionally, for new neural networks users may architect, the generative compiler creates new kernels for these layers automatically, reducing development time to just minutes, increasing developer velocity by orders of magnitude.

Generative compiler technology speeds development for true push-button deployment

Kernel-based compilers require hand-coded c/c++ programs for each layer in a neural network. With potentially thousands of possible kernels required to support the exploding number of neural networks, these kernels become a bottleneck in lowering neural networks to hardware implementations. To resolve this bottleneck, Untether AI has developed a generative compiler technology, which ingests neural networks and applies a series of specialized compiler transformations that lowers high-level neural network operators into fundamental computational primitives. These primitives are then recombined into a hardware-optimized representation, maximizing throughput, minimizing latency, and boosting efficiency on Untether AI’s At-Memory Compute architecture. With the push of a button, neural networks can be lowered to Untether AI’s speedAI® devices and cards in a matter of seconds.

Out of the box, this new technology increases the number of models supported by the imAIgine SDK to over 300, spanning neural networks performing object detection, semantic segmentation, classification, error detection, and many other functions. This breadth of support builds on Untether AI’s validated performance and energy efficiency advantages shown in the last MLPerf inference submissions.

For new neural networks or modified layers of neural networks, the generative compiler can create new kernels on-the-fly, optimized for the over 1,400 RISC-V processors and At-Memory Compute processing elements on speedAI-based inference acceleration solutions.

Kernel-based optimizations available for utmost performance

The new compiler retains the ability to use hand-crafted, optimized kernels. The compiler has the option to use existing kernels, but if a kernel doesn’t exist in the library, the generative technology is used to construct new kernels on-the-fly. This ability to mix and match pre-existing, optimized kernels with generated kernels speeds the time to implement highly performant neural networks.

“This new compiler technology provides the ultimate in productivity and flexibility,” said Alex Grbic, PhD and VP of Software Engineering at Untether AI. “By introducing our generative compiler technology, customers gain an accelerated path to market, quickly achieving a deployment of their neural networks on Untether AI’s architecture.”

This new generative compiler technology will be in the imAIgine SDK version 25.04 release, scheduled for availability in early April of 2025. To obtain early access to the SDK, please visit https://www.untether.ai/imaigine-sdk-early-access-program/

About Untether AI

Untether AI® provides energy-centric AI inference acceleration from the edge to the cloud, supporting any type of neural network model. With its At-Memory compute architecture, Untether AI has solved the data movement bottleneck that costs energy and performance in traditional CPUs and GPUs, resulting in high-performance, low-latency neural network inference acceleration without sacrificing accuracy. Untether AI’s technology ships today as speedAI® devices, acceleration cards, and its imAIgine® Software Development Kit. More information can be found at www.untether.ai.

All references to Untether AI trademarks are the property of Untether AI. All other trademarks mentioned herein are the property of their respective owners.

Fonte: Business Wire