MedAgentsBench基准测试数据集
MedAgentsBench医疗推理评测基准，包含多个医疗问答数据集，专注于复杂医疗推理任务，用于评测模型和智能体框架。
基本信息
资源简介
MedAgentsBench评测基准，专注于复杂医疗推理场景下的思考模型与AI智能体框架性能评测。包含MedQA、PubMedQA、MedMCQA等10个医疗问答数据集，每个数据集提供高难度测试集（模型准确率低于50%）和完整测试集。数据模态为文本，任务为医疗问答推理，用于评估模型在复杂医疗问题上的推理能力。
下载信息
注册下载
Tips: 该数据集需要在对应的数据源网站注册通过后，才能进行数据下载，注册有对应要求，或者需要收费。
暂未开放公开下载
Tips: 该数据集属于公开下载，应该可以免费公开下载。
免登录有偿下载
Tips: 该数据集 Qianfanghub 可以协助提供有偿下载服务，注意，服务不针对数据相关产权，只是技术服务费。
提供高速下载与技术交付服务（收技术服务费，非数据销售）
暂未开放千方医数集，医疗数据集部分，是为社区服务的公开医疗数据集搜索引擎，并不存储或者下载原始的任何数据。 如果您有其他医疗数据需求，可以和客服联系，或者下工单。我们有强大的三甲医疗机构帮助您提供个性化的医疗数据定制、采集、标注服务。
使用方式
数据集获取（ModelScope）
方式一：MsDataset（Python）
# 前置依赖: pip install modelscope
from modelscope.msdatasets import MsDataset
ds = MsDataset.load("AI-Bench/medagents-benchmark", subset_name="default", split="train")
print(ds)
方式二：CLI（命令行）
pip install modelscope
modelscope download dataset AI-Bench/medagents-benchmark
数据集卡片摘录（源站）
MedAgentsBench Dataset
Overview
This dataset is part of the MedAgentsBench, which focuses on benchmarking thinking models and agent frameworks for complex medical reasoning. The benchmark contains challenging medical questions specifically selected where models achieve less than 50% accuracy.
Dataset Structure
The benchmark includes the following medical question-answering datasets:
| Dataset | Description |
|---|---|
| MedQA | Medical domain question answering dataset |
| PubMedQA | Questions based on PubMed abstracts |
| MedMCQA | Multiple-choice questions from medical entrance exams |
| MedBullets | Clinical case-based questions |
| MMLU | Medical subset from Massive Multitask Language Understanding |
| MMLU-Pro | Advanced version of MMLU with more complex questions |
| AfrimedQA | Medical questions focused on African healthcare contexts |
| MedExQA | Expert-level medical questions |
| MedXpertQA-R | Medical expert reasoning questions |
| MedXpertQA-U | Medical expert understanding questions |
Dataset Splits
Each dataset contains:
test_hard: Specifically curated hard questions (accuracy <50%)test: Complete test set
Citation
If you use this dataset in your research, please cite the MedAgentsBench paper:
@inproceedings{tang2025medagentsbench,
title={MedAgentsBench: Benchmarking Thinking Models and Agent Frameworks for Complex Medical Reasoning},
author = {Tang, Xiangru and Shao, Daniel and Sohn, Jiwoong and Chen, Jiapeng and Zhang, Jiayi and Xiang, Jinyu and Wu, Fang and Zhao, Yilun and Wu, Chenglin and Shi, Wenqi and Cohan, Arman and Gerstein, Mark},
journal = {arXiv preprint arXiv:2503.07459},
year = {2025},
}
License
Please refer to the repository for license information.
Additional Information
For more details, visit the MedAgentsBench repository or the paper.
许可
Apache License 2.0
数据加载示例（表格/文本类）
import pandas as pd, glob, os
files = (glob.glob(os.path.join(path, "**", "*.csv"), recursive=True)
+ glob.glob(os.path.join(path, "**", "*.tsv"), recursive=True)
+ glob.glob(os.path.join(path, "**", "*.xlsx"), recursive=True))
print("数据文件:", files)
df = pd.read_csv(files[0])
print(df.shape); print(df.columns.tolist()); print(df.head(3))
精度瓶颈？数据缺失？
当前公开数据无法满足您的算法精度？千方提供针对 其他 的高质量、多模态真实临床数据定制解决方案。
