Apple has released its own language model that can run locally on its own devices. OpenELM is a public model that allows developers to create generative AI. It is a small model, which is necessary for local operation.
OpenELM, or Open-source Efficient Language Models, is a small language model that is mainly suitable for generating texts by artificial intelligence. Apple does not explicitly say what it wants to use the tool for, but it is likely that the company wants to build AI applications that can run locally on iPhones or Macbooks. Apple has previously said that it would also join the battle around generative AI, but was always vague about this.
The OpenELM model consists of four variants. They all have a different size. The sizes of the models range from 270 million, 450 million, 1.1 billion and 3 billion parameters. For comparison: the recent large language model that Meta wants to put in WhatsApp will have between 8 billion and 400 billion parameters. This is possible if the models work cloud-based, but smaller models are required for local processing. More and more companies are not only focusing on large models, but also on smaller models. Microsoft, for example, did that earlier this week.
Apple has written a paper about how the model works and it says that the model will be available open source. OpenELM was trained on the RefinedWeb dataset and on part of RedPajama, a collection of largely English-language articles, books and information from platforms such as GitHub, Wikipedia and StackExchange. In addition, the model was trained on data from Reddit and Project Gutenberg.