OpenAI is dedicated to making artificial intelligence widely accessible. Today, we’re excited to introduce GPT-4o mini, our most cost-efficient small model yet. This new model is expected to greatly expand the range of applications built with AI by making advanced intelligence more affordable. GPT-4o mini scores 82% on the MMLU benchmark and outperforms GPT-41 on the LMSYS leaderboard for chat preferences. Priced at just 15 cents per million input tokens and 60 cents per million output tokens, it is significantly cheaper than previous models, including GPT-3.5 Turbo.
Affordable and Versatile AI for Various Tasks
GPT-4o mini’s low cost and quick response times make it perfect for a broad range of tasks. These include applications that require multiple model calls, such as calling multiple APIs, handling large volumes of context like full code bases or conversation histories, and interacting with customers through real-time text responses, such as customer support chatbots.
Currently, GPT-4o mini supports text and vision in the API, with plans to add support for text, image, video, and audio inputs and outputs in the future. The model has a context window of 128K tokens and supports up to 16K output tokens per request, with knowledge up to October 2023. The improved tokenizer, shared with GPT-4o, makes handling non-English text even more cost-effective.
Superior Textual and Multimodal Intelligence
GPT-4o mini surpasses GPT-3.5 Turbo and other small models on academic benchmarks in both textual intelligence and multimodal reasoning. It supports the same range of languages as GPT-4o and demonstrates strong performance in function calling, enabling developers to create applications that fetch data or perform actions with external systems. Its long-context performance is also improved compared to GPT-3.5 Turbo.
Key benchmarks highlight GPT-4o mini’s capabilities:
- Reasoning tasks: Scores 82.0% on MMLU, compared to 77.9% for Gemini Flash and 73.8% for Claude Haiku.
- Math and coding proficiency: Scores 87.0% on MGSM for math reasoning, and 87.2% on HumanEval for coding performance, outperforming Gemini Flash and Claude Haiku.
- Multimodal reasoning: Scores 59.4% on MMMU, better than Gemini Flash and Claude Haiku.
Collaborations and Practical Use Cases
To better understand GPT-4o mini’s capabilities and limitations, we partnered with companies like Ramp and Superhuman. They found GPT-4o mini significantly better than GPT-3.5 Turbo for tasks such as extracting structured data from receipts and generating high-quality email responses when provided with thread history.
Built-in Safety Measures
Safety is a priority for OpenAI. From the beginning, our models are designed with built-in safety measures, and these are reinforced throughout the development process. During pre-training, we filter out inappropriate content like hate speech, adult content, and spam. Post-training, we align the model’s behavior with our policies using techniques like reinforcement learning with human feedback (RLHF) to improve the accuracy and reliability of the models’ responses.
GPT-4o mini includes the same safety mitigations as GPT-4o, carefully assessed using automated and human evaluations according to our Preparedness Framework. More than 70 external experts tested GPT-4o to identify potential risks, which we addressed. Insights from these evaluations helped improve the safety of both GPT-4o and GPT-4o mini.
We have also developed new techniques to improve the safety of GPT-4o mini, such as the instruction hierarchy method, which helps the model resist jailbreaks, prompt injections, and system prompt extractions. This enhances the reliability of the model’s responses and makes it safer for large-scale applications.
Availability and Pricing
GPT-4o mini is now available as a text and vision model in the Assistants API, Chat Completions API, and Batch API. Developers pay just 15 cents per million input tokens and 60 cents per million output tokens. Fine-tuning for GPT-4o mini will be available soon.
In ChatGPT, Free, Plus, and Team users will have access to GPT-4o mini starting today, replacing GPT-3.5. Enterprise users will gain access next week, aligning with our mission to make AI benefits accessible to everyone.
The Future of AI
In recent years, we’ve seen significant advancements in AI intelligence alongside substantial cost reductions. For example, the cost per token of GPT-4o mini has dropped by 99% since the introduction of text-davinci-003 in 2022. OpenAI is committed to continuing this trend, enhancing model capabilities while driving down costs.
We envision a future where AI models are seamlessly integrated into every app and website. GPT-4o mini is paving the way for developers to build and scale powerful AI applications more efficiently and affordably. The future of AI is becoming more accessible, reliable, and embedded in our daily digital experiences, and OpenAI is excited to lead the way forward.