French startup Kog is betting that the next leap in AI inference will come from understanding existing GPUs more deeply, not replacing them with specialised chips. Its pitch is especially focused on single-request decoding: making one demanding user’s response arrive faster rather than maximising aggregate throughput across a large batch.
The company attracted attention with a technical preview that ran on conventional data-center hardware, including AMD MI300X and Nvidia H200 GPUs. CEO Gaël Delalleau told TechCrunch that the demonstration produced 200 tangible business leads, reflecting how strongly professional AI users feel latency when coding agents or generation tools take minutes—or hours—to complete work.
Kog advertises the possibility of 30-times faster large-language-model inference. Its evidence remains narrower. The preview reached 3,000 tokens per second for one request using Laneformer 2B, a purpose-built model with roughly two billion parameters. Moving that approach to the larger models customers actually want is the company’s central technical and commercial test.
Latency is becoming a product constraint
AI infrastructure is often optimised for throughput: serving as many requests as possible at an acceptable average speed. That works for providers managing large fleets, but it does not eliminate the wait experienced by an individual developer whose agent is running a long task.
Kog believes software engineering will be an early market because experienced coding-agent users already understand the cost of delay. Faster responses can shorten iteration cycles and keep developers engaged with a workflow. Companies that generate games or applications from prompts have a similar incentive, according to Delalleau, because a quicker result can translate directly into more completed sessions and revenue.
The startup initially expected customers to work with smaller, fine-tuned models. Feedback suggested that prospective buyers were not prepared to make that compromise. Kog has consequently shifted its attention toward accelerating larger models, where memory use and decoding behaviour create a much harder engineering problem.
The demand signal is real in the form of leads, but it is not yet customer traction. Kog still needs to demonstrate repeatable gains on widely used models and workloads, including the trade-offs between speed, output quality, cost and concurrent users.
Deep optimisation creates its own scaling problem
Delalleau argues that modern GPUs have enough memory bandwidth to be strong decoding machines if software can use the hardware efficiently. Kog’s approach goes below conventional application optimisation, drawing on low-level GPU engineering, assembly code and reverse-engineering techniques.
That intensity is both the differentiation and the limitation. Delalleau said the team may spend weeks or months studying each new GPU. With 11 employees, Kog can support only a limited selection of chips and models for the foreseeable future.
Other projects are also trying to extract more inference performance through software. TechCrunch points to French company ZML, which offers hardware-agnostic tooling that bypasses Nvidia’s CUDA ecosystem. Delalleau instead compares Kog with Stanford’s Hazy Research, emphasising its deeper focus on the behaviour of individual processors.
Kog hopes eventually to put its optimisation methodology into agent-based pipelines that can help adapt the system across more hardware. That is a future ambition rather than a demonstrated automation capability. Until then, every supported GPU adds demanding manual research.
Larger models will decide whether the claim travels
Kog has open-sourced Laneformer 2B, making its initial result easier to examine. The company now aims to implement its first major model at ten-times speed by September. Delalleau expects that milestone to help demonstrate customer traction and support a Series A fundraising effort.
The startup has institutional support from Scaleway, Bpifrance and the French Tech 2030 programme, giving its story a European sovereignty angle. Software that makes several vendors’ GPUs more productive could reduce dependence on any single accelerator roadmap, although Kog has not yet proved broad hardware coverage.
Its opportunity rests on a useful asymmetry: replacing data-center chips is expensive, while deploying better software can be much faster. Its risk is that optimisation tuned to one model and one processor may not generalise as architectures change.
For now, Kog has shown an unusually fast small-model demonstration and found businesses interested in lower latency. The decisive evidence will be performance on production-scale models under realistic concurrency. Only then will “30x faster” move from an attention-grabbing ceiling to a claim customers can plan around.