Mixtral 8x22B Pushes Sparse Open Models Forward
Mistral AI’s Mixtral 8x22B activates only part of its 141-billion-parameter architecture, pairing open access with multilingual, coding and long-context capabilities.
Share
Posts by tag
MoE
4 posts
Databricks DBRX Raises the Bar for Enterprise Open Models
DBRX combines a 132B sparse mixture-of-experts architecture with curated training data and Databricks tooling for custom, governed enterprise AI.
Share
xAI Opens Grok-1’s 314B Mixture-of-Experts Model
Grok-1’s base weights and architecture are available under Apache 2.0, but the release excludes its training data and real-time data connections.
Share
Gemini 1.5 Pro Tests a One-Million-Token Context Window
Google introduced Gemini 1.5 Pro with an experimental one-million-token context window and a Mixture-of-Experts architecture.
Share