Aiven Targets Token-Cost Arbitrage With New LLM Gateway

Aiven, the Finland-based open-source database service provider, is launching a gateway service designed to help developers manage and reduce their large language model (LLM) token costs. The move signals a shift toward treating AI spend as a primary operational expense that can be optimized through model arbitrage.

As The Register first reported, Aiven has already applied this strategy internally, achieving an 80 percent reduction in token costs for its heaviest users. Ville Lehto, Aiven’s VP of strategy, told The Register that the company faced a choice between imposing strict token budgets or finding a more efficient usage method. Aiven opted for the latter, implementing an internal gateway to allow developers to seamlessly switch between frontier models and open-weight alternatives.

Open-weight models—which include Google's Gemma, Meta's Llama family, Alibaba Cloud's Qwen family, Mistral AI's Devstral and Medium 3.5, and China's DeepSeek—can be hosted at a lower cost or run locally compared to frontier models from Anthropic and OpenAI. Lehto noted that while frontier models remain essential for planning critical work, the majority of actual tasks are now handled by open-weight models.

To facilitate this, Aiven built its gateway using the open-source LiteLLM framework, adding proprietary features for access management and budgeting. The company hosts these models on rented GPUs and through self-hosting. Lehto emphasized that because performance varies across models, Aiven utilizes metrics such as cost per task to determine the most efficient path.

Aiven now plans to offer this gateway as a service to its enterprise customers, which include communications firm Avaya, cybersecurity vendor Sophos, UK payment processor Dojo, and German ecommerce firm Conrad. The goal is to provide a frictionless way for organizations to swap models in and out of their workflows to control their AI usage destiny.