Key Specifications

Benchmark Performance

Benchmark	Score	Unit	Evaluated At	Notes	Source
MMLU	63.4	%	2024-02-19	5-shot	view
HUMANEVAL	59.2	pass@1	2024-02-19	—	view
GSM8K	59.6	%	2024-02-19	0-shot CoT	view
MATH	37.7	%	2024-02-19	0-shot CoT	view
BBH	71.8	%	2024-02-19	3-shot CoT	view
GPQA	35.9	%	2024-02-19	0-shot	view
IFEVAL	55.6	%	2024-02-19	prompt_strict	view
ARC	85.9	%	2024-02-19	challenge	view
MUSR	39.6	%	2024-02-19	0-shot	view
WINOGRANDE	74.4	%	2024-02-19	0-shot	view

Name: Nous Hermes 2 Solar 10.7B
Brand: other

NousResearch Hermes 2 Solar 10.7B 微调模型, 4K 上下文, 基于 Solar 10.7B, 性能接近 Llama 2 70B。

Vendor	Versi	Tanggal Rilis	Jendela Konteks	Modalitas Input	Modalitas Output	Lisensi
Other	nous-hermes-2-solar-10-7b	2024-02-19	4K	text	text	Apache 2.0

Benchmark	Skor	Satuan	Catatan
MMLU (Massive Multitask Language Understanding)	63.4	%	5-shot
HumanEval	59.2	pass@1	—
GSM8K (Grade School Math 8K)	59.6	%	0-shot CoT
MATH	37.7	%	0-shot CoT
BBH (BIG-Bench Hard)	71.8	%	3-shot CoT
GPQA	35.9	%	0-shot
IFEval	55.6	%	prompt_strict
ARC	85.9	%	challenge
MUSR	39.6	%	0-shot
WinoGrande	74.4	%	0-shot

Input	Output	Baca Cache	Tulis Cache
—	—	—	—

per juta token