Google has introduced multi-token prediction drafters for Gemma 4, a technique that accelerates inference speed by enabling the model to generate multiple tokens simultaneously rather than one at a time.
Multi-token prediction represents a shift in how language models generate text. Traditional inference processes tokens sequentially—the model generates one token, then uses that output to predict the next. This sequential dependency creates a bottleneck, especially for longer outputs.
Gemma 4's new approach uses a drafter model that speculates on multiple future tokens in parallel. A verifier then validates these predictions, accepting correct tokens and only recomputing when necessary. This speculative decoding technique reduces the number of forward passes required, lowering overall latency.
The speed improvements are substantial in practical scenarios. For tasks requiring longer text generation, the technique delivers 2-3x faster inference on standard hardware. This acceleration comes without sacrificing output quality—the model produces identical results to standard sequential generation.
The development aligns with broader industry efforts to optimize inference efficiency. As AI models grow larger and deployment costs increase, inference optimization has become critical for commercial viability. Similar approaches have gained traction across competing implementations.
Google's implementation in Gemma 4 is particularly significant because it demonstrates the technique's effectiveness in a production-ready model. Developers using Gemma 4 can access these improvements through Google's standard deployment channels.
The multi-token prediction method works best for longer outputs and is particularly effective on modern accelerators. For shorter completions, gains are more modest, but the approach maintains consistent quality across all scenarios.
This advancement addresses a core challenge in deploying large language models at scale. By reducing inference time while maintaining quality, the technique makes real-time AI applications more feasible and cost-effective. The approach is generalizable, suggesting similar optimizations could benefit other model architectures.
Israel-based Hemispheric secured $52 million in funding for its AI model that analyzes non-invasive brain activity measurements and converts them into quantitative diagnostic metrics.
Anthropic and Blackstone are backing Ode, a new venture that embeds AI engineers directly inside enterprises. The bet signals a shift in where the next trillion dollars in AI value may be created: not in building models, but in implementing them.
Spectro Cloud, an AI infrastructure company focused on managing token costs, secured $100 million in Series D funding at a valuation exceeding $1 billion. The raise marks significant growth from the company's $750 million valuation in 2024.
Startups like Altur are deploying AI chatbots to handle debt collection calls, automating a process traditionally done by humans. Y Combinator has backed six debt collection and settlement startups over the past six years.