ETH Zurich and EPFL are set to release a fully public open-weight LLM trained on a carbon-neutral supercomputer. This model, referred to as 'Switzerland’s open LLM,' provides a transparent alternative to closed systems that lack inspectability. With configurations of 8 billion and 70 billion parameters, it boasts multilingual training on 15 trillion tokens across 1,500 languages. Key features include open-by-design architecture, enabling users to fine-tune and deploy without restrictions, and compliance with EU regulations, positioning it favorably against models like GPT-4. The model encourages ethical data practices and supports onchain inference and tokenized data marketplaces. It highlights the potential for building applications within the blockchain space while addressing the pitfalls linked to open-source LLMs, such as legal uncertainties and performance gaps compared to proprietary models. Overall, it aims to empower researchers and developers with accessible technology while promoting sustainability and transparency.

Source 🔗