Gemini 2.0 Flash Thinking: Complete Benchmark Performance Guide

google 2.0-flash-thinking 2024-12-19 Proprietary

In-depth analysis of Gemini 2.0 Flash Thinking performance on MMLU, HumanEval, GSM8K, MATH, BBH benchmarks, including pricing and use cases.

Key Specifications

Vendorgoogle
Version2.0-flash-thinking
Release Date2024-12-19
Context Window1.048576e+06 tokens
Input Modalitiestext, image
Output Modalitiestext
LicenseProprietary
Documentationhttps://ai.google.dev/gemini-api/docs

Benchmark Performance

BenchmarkScoreUnitEvaluated AtNotesSource
MMLU86.5%2024-12-195-shotview
HUMANEVAL87.2pass@12024-12-19—view
GSM8K91.3%2024-12-190-shot CoTview
MATH56.6%2024-12-190-shot CoTview
BBH87.9%2024-12-193-shot CoTview
GPQA61%2024-12-190-shotview
IFEVAL82.8%2024-12-19prompt_strictview
ARC95%2024-12-19challengeview
MUSR70.8%2024-12-190-shotview
WINOGRANDE88.1%2024-12-190-shotview

Pricing

TierPriceCurrency
Input$0.1 / MtokUSD
Output$0.4 / MtokUSD
Cache Read$0 / MtokUSD
Cache Write$0 / MtokUSD

Source: https://ai.google.dev/pricing · as of 2024-12-19

Compliance

  • Data Residency: US
  • SOC2: ✓
  • HIPAA: ✗
  • GDPR: ✓
  • ISO 27001: ✓

Gemini 2.0 Flash Thinking

Tổng quan mô hình

Google Gemini 2.0 Flash Thinking 实验版, 1M 上下文, 链式思维推理, 在数学与编码上接近 o1。

Thông số kỹ thuật cốt lõi

Nhà cung cấpPhiên bảnNgày phát hànhCửa sổ ngữ cảnhPhương thức đầu vàoPhương thức đầu raGiấy phép
Google2.0-flash-thinking2024-12-191048Ktext, imagetextProprietary

Hiệu suất benchmark

BenchmarkĐiểmĐơn vịGhi chú
MMLU (Massive Multitask Language Understanding)86.5%5-shot
HumanEval87.2pass@1—
GSM8K (Grade School Math 8K)91.3%0-shot CoT
MATH56.6%0-shot CoT
BBH (BIG-Bench Hard)87.9%3-shot CoT
GPQA61.0%0-shot
IFEval82.8%prompt_strict
ARC95.0%challenge
MUSR70.8%0-shot
WinoGrande88.1%0-shot

Giá

Đầu vàoĐầu raĐọc bộ nhớ đệmGhi bộ nhớ đệm
————

mỗi triệu token

Điểm mạnh

  • MMLU score 86.5, strong knowledge reasoning.
  • HumanEval 87.2, excellent code generation.
  • GSM8K 91.3, robust math reasoning.
  • 支持文本、图像、音频多模态输入。

Điểm yếu

  • 闭源专有模型，不支持自托管。

Trường hợp sử dụng

  • 代码生成与调试
  • 视觉与图像理解
  • Agent 工作流与工具调用

Tham chiếu