Kog wants to boost agent inference on GPUs

An economic constraint at the heart of agentic AI

French start-up Kog wants to tackle one of the most tangible constraints of the new wave of artificial intelligence: the cost of GPU inference. In an article about the company, TechCrunch sums up its ambition with an explicit headline, “Kog is going deeper to squeeze more inference out of GPUs”. In other words, Kog is seeking to get more inference from already available graphics accelerators, rather than assuming that the answer to growing needs must necessarily involve continually buying new chips.

The issue is particularly strategic as companies experiment with AI agents. A conventional conversational assistant can provide an answer to a question in a relatively linear exchange. An agent, by contrast, is generally designed to break down a task, reason through several steps, call tools, consult data, verify a result and then rerun a model if necessary. This sequence of operations can multiply inference requests. The functional promise of agents is therefore inseparable from a more prosaic question: how much does each task actually completed cost?

In this equation, the GPU occupies a central place. These processors, initially designed for parallel graphics computations, have become the most widely used accelerators for training and serving large AI models. NVIDIA largely dominates this market with its generations of data-center GPUs, but the issue extends beyond a single manufacturer or particular chip reference. Organizations operating language models must balance available capacity, response speed, energy consumption, cloud instance costs, sovereignty constraints and the expected level of service.

Kog's positioning therefore stands out through its starting point. According to information reported by TechCrunch, the company challenges the idea that GPUs are inherently poorly suited to agentic workflows. This debate matters. The rise of agents has indeed fueled demand for specialized hardware architectures, alternative chips and infrastructure designed around inference. Kog argues instead that there remains substantial room for optimization in the use of existing GPUs.

This approach does not mean denying the physical limitations of the hardware. GPUs have a defined amount of memory, a given compute capacity and communication constraints between machines. But the actual efficiency of infrastructure also depends on how requests are organized, distributed, queued, grouped or executed. The gap between the theoretical power of a GPU fleet and the performance an application actually achieves can be significant. It is in this gap, often less visible than the launch of a new chip, that Kog intends to position itself.

For businesses, the question is far from abstract. An agent demonstration may work with a few users and an experimental budget. Scaling up is much more demanding: more employees or customers being served, maintaining low latency, processing longer documents, using multiple models, confidentiality requirements and continuous availability. At that point, the marginal cost of an AI task becomes an operational indicator as important as the quality of the answers generated.

The scope of the issue thus goes beyond technical optimization. If agentic deployments are to become production tools, they will have to demonstrate sustainable economics. A company cannot merely measure the time saved by an agent; it must also account for the infrastructure that runs it, calls to third-party services, observability, security, integration with internal systems and human oversight. By targeting GPU inference, Kog is positioning itself on one of the links that determines the profitability of this chain.

Kog is betting on the use of already installed GPUs

The TechCrunch article presents Kog as a French start-up deepening its work on inference in order to derive more value from GPUs. The wording reflects a shift in priorities in the AI ecosystem. For several years, attention was heavily focused on model training: the size of datasets, the number of parameters, the compute capabilities required and investments in data centers. Inference, meaning the execution of a trained model to answer real requests, has now become an equally decisive battleground.

This shift is easily explained. Training a leading model can be extremely costly, but it remains a one-off or periodic operation. Inference accompanies every user interaction, every text generation, every agent step and every automated process. When a product finds its market, this expenditure becomes recurring. Reducing the usage needed to complete a request, or increasing the number of requests handled with the same GPU fleet, can therefore have significant effects on total operating cost.

The thesis advanced by Kog aligns with a well-known infrastructure reality: having the hardware is not enough to fully exploit its potential. A GPU may be underused if requests arrive irregularly, if their sizes vary widely, if the needed data is not available at the right time, if memory is poorly managed, or if one part of the system is waiting for another's results. Agentic applications are likely to exacerbate these imbalances because they do not always behave like a homogeneous stream of identical requests.

An agentic workflow may, for example, alternate between generation, search, classification, information extraction or calls to external software. The model is not necessarily used continuously in the same way. Some steps are fast, while others require more context. Some requests may be interrupted or reformulated. Compute demand is therefore more fragmented than it is in a simple service where each user sends a short question to a single model.

Kog's point is not that GPUs alone would solve all these problems. Rather, it is that they should not be dismissed simply because agents introduce complex workflows. The question then becomes as much about software as hardware: how can inference processing be adapted to the characteristics of these tasks without having to completely rebuild the infrastructure? This is an attractive proposition for companies that already own accelerators, rent GPU capacity in the cloud, or have built their applications around the dominant GPU software ecosystem.

It also fits within supply and timing constraints. New generations of accelerators attract attention, but they are not an immediate solution for all players. Companies do not have the same budgets, deployment timelines or capacity needs. Some must optimize infrastructure that has already been depreciated; others have existing cloud contracts; still others are seeking to avoid increasing their operating expenditure too quickly. In these cases, improving the efficiency of GPUs already in use may be more directly useful than waiting for a hardware upgrade.

This logic is also relevant for organizations seeking to retain greater control over their AI environment. Using hosted model APIs can simplify a product launch, but it leaves less control over infrastructure and certain cost parameters. Conversely, serving a model in one's own environment entails managing GPUs, software, security and operations. A layer that improves the efficiency of this execution can therefore reinforce the appeal of autonomous or hybrid deployment, without eliminating the technical requirements that come with it.

Inference: a less visible battle than that of models

The generative AI race is often told through new models, their evaluation results or their multimodal capabilities. Yet the market depends just as much on the infrastructure that makes these systems available to millions of users. Advances in models generally increase their memory and compute requirements; meanwhile, users expect fast, reliable answers at a reasonable cost. This tension structures the entire value chain, from chip designers to cloud operators and inference software publishers.

GPUs have become dominant because they can efficiently execute many mathematical operations in parallel. Their role in training neural networks has helped create a vast software and hardware environment around them. Developers, researchers and businesses have libraries, deployment tools, monitoring systems and teams trained in this environment. Replacing this installed base with a different technology is not just a matter of raw performance; it is also a matter of compatibility, skills, availability and operational risk.

This is precisely what makes Kog's position notable. The sector is seeing the emergence or strengthening of offerings specialized in inference, sometimes based on hardware architectures distinct from general-purpose GPUs. These approaches address a real need: inference for large language models has characteristics that differ from training, particularly because it often must prioritize latency, throughput or energy efficiency. But the presence of specialized solutions does not automatically mean that GPUs cease to be relevant for agents.

Major cloud providers, processor manufacturers and AI infrastructure companies are all working to reduce the costs associated with serving models. Amazon Web Services offers its own chips intended in particular for inference, while Google has been developing its TPUs for several years. GPU providers are also seeking to improve their platforms for AI workloads. These initiatives reflect a shared certainty: as uses move from prototype to production, inference efficiency becomes a criterion for commercial differentiation.

Kog does not present itself, in the information highlighted by TechCrunch, as a chipmaker seeking to replace GPUs. Its angle is different: optimizing the use of these accelerators more deeply. This difference is essential. Developing a new hardware component entails long cycles, industrial capacity, complex validation and access to the semiconductor manufacturing chain. An approach centered on software and orchestration can potentially be deployed faster, provided it demonstrates concrete and reproducible gains in production environments.

Any benefits of such optimization must nevertheless be assessed beyond an isolated performance figure. A company may increase a system's overall throughput while at the same time degrading the latency experienced by some users. It may reduce GPU spending but increase operational complexity. It may improve average accelerator utilization but encounter difficulties during traffic peaks. In agentic AI, where tasks can be heterogeneous and automated decisions may interact with business systems, predictability matters as much as speed.

The value of a proposition such as Kog's will therefore depend on its ability to operate within real-world enterprise constraints. It is not just a matter of processing more tokens or requests. Customers will look for integration with their models, deployment tools, security mechanisms, monitoring practices and governance rules. They will also need to verify that optimization preserves the expected quality of service. Reducing the bill cannot come at the cost of more errors, insufficient availability or difficulty auditing an agent's behavior.

Why agents put infrastructure under pressure

Agentic AI has become a widely used term, sometimes referring to different technical realities. In its most common sense, it refers to systems able to chain together several actions in pursuit of a goal: interpreting an instruction, preparing a plan, querying a database, using a tool, generating a document, requesting validation or modifying an action based on the previous result. These systems are not autonomous in the general sense of the term, and their reliability remains a major issue. But they are already changing the workload profiles that infrastructure must support.

A conversational model designed to answer a single question offers a computational path that is relatively easy to anticipate. An agent may generate several calls to the same model or to multiple models, retain longer context, use internal data and trigger successive operations. Each additional step potentially increases inference consumption. In some cases, an agent may also perform verification iterations, potentially improving the result but increasing the total amount of required computation.

For a CIO or innovation leader, the issue is therefore not simply choosing the best model. They must define which use cases justify higher compute costs, which tasks must remain under human validation, which volumes are predictable and which budget limits can be applied. An agent tasked with preparing an initial internal summary does not have the same requirements as a system that assists a customer or interacts with a business tool. Granularity of control becomes essential.

Cost is also influenced by variability in demand. A company may experience seasonal peaks, periods of high activity or unexpected workloads linked to the rapid adoption of a new tool. In infrastructure sized to meet these peaks, there is a risk of underutilization under normal conditions. In infrastructure overly optimized for average cost, conversely, there is a risk of saturation. The promise of doing more with existing GPUs directly addresses this tension, even if it does not eliminate the need for rigorous capacity planning.

Agents add another difficulty: wait times do not come only from the GPU. They may depend on access to a database, a search tool, an ERP, customer relationship management software or an external API. Optimizing inference does not solve all these bottlenecks. However, it can reduce the share of cost and delay that comes from model execution, which is crucial when that execution is repeated across many steps.

This observation requires distinguishing between a technological demonstration and economic impact. It is possible to build an impressive agent within a limited scope while having a cost structure that is difficult to reconcile with thousands of users. The maturity of an agentic product will increasingly be measured by its ability to deliver a useful result with a controlled level of resources. This is where infrastructure stops being a subject reserved for technical teams: it becomes a product, pricing and business strategy variable.

In this context, Kog's approach may interest companies seeking to avoid binary reasoning between two options: retaining a GPU fleet while accepting its inefficiencies, or migrating to a new hardware architecture. Reality is more nuanced. Some workloads can remain on GPUs, others can be distributed differently, and execution policies can evolve depending on applications. The ability to precisely measure usage and allocate the right level of resources to each step will be at least as important as the initial hardware choice.

• For product teams, greater efficiency can expand the number of economically viable use cases.
• For infrastructure teams, it can postpone or reduce certain additional capacity needs.
• For finance departments, it reinforces the need to link GPU cost to a measurable business unit.
• For security and compliance teams, it is a reminder that optimization must not compromise auditability or control over processing.

Particular resonance for France and Europe

The fact that Kog is a French start-up gives this announcement particular resonance. Europe is seeking to strengthen its capabilities in AI, across research, models, applications and infrastructure. In this landscape, discussions of technological sovereignty often focus on access to models or the availability of compute. Optimizing existing compute is a less spectacular subject, but it directly affects the ability of European organizations to deploy AI within local budgets and constraints.

French and European companies do not all start from the same position as major U.S. technology groups. Many must operate with limited compute resources, AI projects distributed across multiple departments and significant data protection requirements. They may use cloud services, hybrid environments or internal infrastructure. In all these scenarios, limiting GPU capacity waste can improve a project's feasibility without requiring an immediate transformation of the entire information system.

European regulation also makes the operational question more complex, without it being solely a regulatory matter. For certain uses, companies must pay heightened attention to data location, security, system traceability and responsibilities related to AI-assisted decisions. Hosting or serving models in a more controlled environment may be a response for some players, but that makes infrastructure efficiency all the more important. An organization that itself assumes part of model serving must bear its costs and operation on a daily basis.

The ability to use already deployed GPUs more efficiently may also be relevant for sectors where generative AI cannot be adopted without precautions: banking, insurance, healthcare, industry, energy, public services or large public administrations. In these environments, projects are often deployed progressively, with proofs of value, security validations and lengthy integrations. Buying large amounts of new hardware is not always the simplest or most urgent step. Gains achieved at the execution level can then support more pragmatic scaling.

For the French ecosystem, the emergence of companies focused on infrastructure also serves as a reminder that competition is not limited to creating general-purpose models. The software layers that make models faster, less costly, more observable or easier to deploy constitute a market in their own right. They can serve international customers while drawing on the very concrete needs of local organizations. In AI, companies that capture value are not only those that create the model; they are also those that reduce friction in its use.

This position is nevertheless difficult to win. Infrastructure customers are demanding, sales cycles can be long and established providers have powerful ecosystems. A young company must prove that its approach works with existing tools, that it does not create difficult-to-manage dependency, and that its results withstand the diversity of workloads. The inference market is also crowded, with cloud players, hardware manufacturers, software publishers and specialized start-ups all seeking to improve the performance-to-cost ratio.

Kog's proposition is therefore part of a broader European dynamic: turning compute capacity constraints into a field for software innovation. Access to GPUs remains decisive, but their efficiency is just as important. For French companies, this distinction has a practical consequence: an AI project is not limited to reserving compute power. It requires knowing how that power will be consumed, measured, governed and, if possible, optimized as uses evolve.

The next step: demonstrating lasting gains in production

Kog's challenge will be to turn its assessment into measurable results in real deployments. The assertion that GPUs can better support agentic workflows is relevant in a market where inference spending is growing with usage. But the value of an infrastructure solution is judged over time: performance under load, stability, compatibility, ease of maintenance, behavior in the face of model updates and the ability to meet stringent security requirements.

Customers will not only seek a better theoretical utilization rate. They will want to know whether a platform makes it possible to serve more users with the same fleet, reduce the cost of a task, maintain acceptable latency or more easily manage unexpected growth. They will also examine trade-offs. A system highly optimized for a specific model or a given request type may lose its appeal if the company quickly changes models, providers or product strategy. Flexibility will therefore be as important a factor as raw performance.

Competition could become more intense as agents move from the status of demonstration to that of a standard component of enterprise software. Model providers will have an interest in offering ever more efficient inference. Hyperscalers will seek to draw workloads onto their own infrastructure. Chipmakers will defend their architectures and software. In this environment, a company such as Kog must demonstrate specific expertise: making available GPUs more productive without requiring customers to completely rewrite their systems.

The debate is unlikely to be resolved by the total victory of a single category of hardware. GPUs will retain a major role thanks to their installed base and ecosystem. Specialized chips may gain ground where they offer a clear advantage for certain workloads. Companies will likely use combinations of infrastructure depending on their cost, latency, data location and availability constraints. In this hybrid world, software able to efficiently orchestrate available resources will become increasingly important.

The bet highlighted by TechCrunch is less about replacing GPUs than challenging their insufficiently efficient use for agentic workflows.

This direction may prove particularly important if agentic AI delivers on its promises in business processes. The more steps agents perform, the more central the economic discipline of inference will become. Companies will have to balance the quality of reasoning, the number of checks, response time and the cost of each action. They will not be able to treat infrastructure as a mere technical detail behind the conversational interface.

For Kog, the opportunity lies in this transition. The industry has long valued access to ever more compute. The next phase may place equal value on the ability to make better use of compute already available. For the French and European market, where spending control, integration with existing systems and governance constraints play a decisive role, this promise addresses a concrete expectation. The long-term issue will be whether GPU optimization can become a lasting lever of competitiveness for AI agents, rather than a one-off improvement in a hardware race that is also continuing to accelerate.