▾ G11 Media Network: | ChannelCity | ImpresaCity | SecurityOpenLab | Italian Channel Awards | Italian Project Awards | Italian Security Awards | ...
InnovationOpenLab

MangoBoost Launches Mango LLMBoost™: AI Inference Optimization Software with Up to 12.6x Relative Performance Improvement and 92% Cost Savings

#AI--MangoBoost, a provider of cutting-edge system solutions designed to maximize AI data center efficiency, is announcing the launch of Mango LLMBoost™, system optimization software providing unpar...

Business Wire

BELLEVUE, Wash.: #AI--MangoBoost, a provider of cutting-edge system solutions designed to maximize AI data center efficiency, is announcing the launch of Mango LLMBoost™, system optimization software providing unparalleled performance improvement and efficiency for AI inference.

Through effective system schedule coordination, kernel optimization and its proprietary prefetching mechanism, reinforced by model quantization that fully takes advantage of modern GPUs, Mango LLMBoost™ achieves up to 12.6x boost in relative performance improvement and 92% cost savings compared to other popular LLM inference engines.

Mango LLMBoost™ is currently available through AWS Marketplace, with expansion to other major cloud service providers and support for on-prem deployments in the horizon.

Product Highlights:

  • GPU Flexibility: Mango LLMBoost™ is compatible with all popular NVIDIA and AMD GPUs.
  • Multi-Model Deployment and Management: Mango LLMBoost™ is validated across a diverse range of chat-based and multi-modal models, including Llama, Mixtral, Gemma, Qwen2, Llava, Phi3, Chameleon, MiniCPM, and GLM-v4, which can be deployed and managed on a single inference server with automated resource allocation.
  • Hassle-Free Deployment: Mango LLMBoost™ provides end-to-end deployment option with MangoBoost’s web-serving and streaming APIs, and intelligently selects the best performing configuration given the GPU and the running models.
  • OpenAI API Compatibility: Mango LLMBoost™ can be easily integrated into existing AI applications utilizing OpenAI’s API.

"The launch of Mango LLMBoost™ represents a significant step in MangoBoost's continued dedication to enhancing systems-level performance and efficiency. Our expertise in DPUs has been central to our mission of improving data center efficiency, and Mango LLMBoost™ expands that focus to deliver optimization at both hardware and software levels. By addressing the critical need for performance and efficiency in AI inference workloads, we’re enabling businesses to achieve more with their existing infrastructure," says Jangwoo Kim, CEO of MangoBoost.

For information on Mango LLMBoost™, visit the product webpage or reach out to contact@mangoboost.io.

About MangoBoost

MangoBoost delivers cutting-edge, full-stack system solutions designed to maximize AI data center efficiency. At the heart of these solutions is the MangoBoost DPU, which ensures full compatibility with general-purpose GPUs, accelerators, and storage products, allowing for cost-efficient, standardized AI infrastructure. Founded in 2022 on a decade of research presented at top-tier computer systems conferences such as OSDI and ISCA, MangoBoost’s core DPU technology is protected by over 30 patents. The company has raised over $60 million in funding and is rapidly expanding its operations in the U.S., Canada, and Korea. MangoBoost’s team includes more than 100 experts, many of whom hold PhDs from world-class research institutions. For more information, visit MangoBoost’s website and LinkedIn page.

Fonte: Business Wire

If you liked this article and want to stay up to date with news from InnovationOpenLab.com subscribe to ours Free newsletter.

Related news

Last News

RSA at Cybertech Europe 2024

Alaa Abdul Nabi, Vice President, Sales International at RSA presents the innovations the vendor brings to Cybertech as part of a passwordless vision for…

Italian Security Awards 2024: G11 Media honours the best of Italian cybersecurity

G11 Media's SecurityOpenLab magazine rewards excellence in cybersecurity: the best vendors based on user votes

How Austria is making its AI ecosystem grow

Always keeping an European perspective, Austria has developed a thriving AI ecosystem that now can attract talents and companies from other countries

Sparkle and Telsy test Quantum Key Distribution in practice

Successfully completing a Proof of Concept implementation in Athens, the two Italian companies prove that QKD can be easily implemented also in pre-existing…

Most read

Chris Catanzaro of Halcyon Recognized on the 2025 CRN® Channel Chiefs…

Halcyon, a leading platform designed from day one to defeat ransomware, today announced that CRN®, a brand of The Channel Company, has named Chris Catanzaro,…

Northern Trust Universe Data: Strong 2024 Performance for U.S. Institutional…

#assetmanagement--Strong U.S. equity market returns were offset by weakness in the bond market and global stocks in the fourth quarter of 2024, generating…

SurePeople Democratizes Workplace Assessments—Now Free for All Employees,…

For decades, workplace assessments have been reserved for leaders and managers, until now. Today, SurePeople announces free access to Prism, a proprietary…

Cheesecake Labs Opens San Francisco Office, Secures #1 Spot in Mobile…

Cheesecake Labs, a global leader in mobile and web app development, has announced significant milestones in its 2025 expansion strategy, including the…

Newsletter signup

Join our mailing list to get weekly updates delivered to your inbox.

Sign me up!