Tencent has released Hy3, an open-source language model using mixture-of-experts architecture that activates only 21 billion parameters at runtime. The company claims Hy3 matches the performance of models two to five times larger while reducing hallucination rates to 5.4 percent.
Hy3 represents an approach to scaling language models through parameter efficiency rather than raw size. The 295-billion-parameter model uses a mixture-of-experts (MoE) architecture, a technique that selectively activates different subsets of parameters depending on the input, rather than using all parameters for every computation.
The active parameter count of 21 billion during inference allows Hy3 to operate with reduced computational overhead compared to dense models of equivalent or larger sizes. Tencent's benchmarks indicate the model performs comparably to models containing 105 billion to 150 billion active parameters.
Reducing hallucination rates—instances where models generate false or fabricated information—remains a priority in language model development. Tencent reports Hy3 achieves a 5.4 percent hallucination rate, approximately half that of comparable models.
The open-source release makes Hy3 available for researchers and developers to implement, evaluate, and build upon. This aligns with a broader industry trend of releasing capable models under open licenses, contrasting with closed commercial approaches.
Mixture-of-experts architectures have gained traction in recent years as organizations seek efficiency improvements. Other major model developers have incorporated similar approaches, though implementing and deploying MoE models effectively presents technical challenges around memory management and training stability.
The efficiency gains demonstrated by Hy3 could reduce the computational resources required for deployment, lowering inference costs and enabling broader access to capable language models. The specifics of Hy3's architecture, training data, and performance metrics across different benchmark suites will shape its adoption and impact on the field.
Tencent's release contributes to ongoing development of parameter-efficient language models, a key research direction as organizations balance model capability with deployment practicality.
Nvidia is investing $3.5 billion in MediaTek while Anthropic prepares for a potentially record-breaking IPO. Forrester Research CEO George Colony assesses whether the AI boom's hype matches reality.
An industry-backed advocacy group is launching a multimillion-dollar advertising blitz in swing states to counter growing public opposition to data center expansion. The effort targets concerns about the construction wave driving AI infrastructure growth.
Researchers have developed an AI system that detects heart disease from electrocardiograms (ECGs) faster and more accurately than human doctors. The tool, trained on millions of patient records, extracts diagnostic information invisible to the human eye.
A Guardian Australia investigation has uncovered dozens of policy submissions to government inquiries that contain AI-generated misinformation, including fabricated studies and false citations. The problem spans across the political spectrum.