Data Sheet 2：使用共识大语言模型方法提高结构化数据提取的可靠性和准确性——多发性硬化症用例描述

数据集获取（Figshare）

• 源站标题：Data Sheet 2_Improving reliability and accuracy of structured data extraction using a consensus large-language model approach–a use case description in multiple sclerosis.pdf
• DOI：10.3389/frai.2026.1658575.s003
• 发布日期：2026-02-13
• 文件数：1 个
• 文件总体积：207.45 KB
• 源站页面：https://frontiersin.figshare.com/articles/dataset/Data_Sheet_2_Improving_reliability_and_accuracy_of_structured_data_extraction_using_a_consensus_large-language_model_approach_a_use_case_description_in_multiple_sclerosis_pdf/31330897

命令行下载

curl -L -o "Data Sheet 2_Improving reliability and accuracy of structured data extraction using a consensus large-language model approach–a use case description i.pdf" "https://ndownloader.figshare.com/files/61861231"

文件清单

• Data Sheet 2_Improving reliability and accuracy of structured data extraction using a consensus large-language model approach–a use case description i.pdf（207.45 KB）

文件总体积 207.45 KB（源站 API 实测）。

数据加载示例（图像类）

from PIL import Image
import glob, os

files = (glob.glob(os.path.join(path, "**", "*.png"), recursive=True)
       + glob.glob(os.path.join(path, "**", "*.jpg"), recursive=True)
       + glob.glob(os.path.join(path, "**", "*.tif"), recursive=True))
print("图像文件数:", len(files))
img = Image.open(files[0]); print("尺寸/模式:", img.size, img.mode)

# torchvision Dataset 方式:
# from torchvision import datasets
# ds = datasets.ImageFolder(path)  # 要求 子目录=类别

目录组织与标注格式以源站说明和下载后实际文件为准。

数据说明

BackgroundThe absence of standardization in the documentation of routine clinical data complicates research usage of retrospective data on a large-scale basis. Medically trained personnel is required for interpretation and conversion into a structured format making it time and cost intensive and creating a potential bias of such data. To address these challenges, we have developed a semi-automated approach for evaluating Multiple Sclerosis (MS) outpatients reports that utilizes different large-language models (LLM) and their consensus in comparison to manual evaluation.
MethodsWe used several commercially available LLMs by OpenAI, Anthropic and Google to create a structured output of several variables with differing complexity of 30 anonymized outpatient reports with zero-shot-learning. We added a consensus output by combining the results of three different LLMs. Over several runs, we adapted the prompt, compared the results with a reference and assessed the error rate. Any deviation from the reference was considered an error. A true-error rate was determined for the LLM consensus output and the neurology specialist output, where only content deviations are counted as errors.
ResultsThrough 9 iterations of improving the structure and content of the prompt, we have seen a clear reduction in the error rate of the various LLMs. By creating an LLM consensus with the final prompt design, we were able to overcome a ceiling effect in reducing the error rate. With a true-error rate of 1.48%, the LLM consensus shows a similar error rate as neurologists (around 2%) in the creation of structured data.
DiscussionOur method enables fast and reliable LLM-based analysis of large clinical routine data sets of varying complexity with a low technical barrier to entry. By generating an LLM consensus, we were able to considerably improve the quality of the output making it comparable to data created by neurology specialists. This approach allows large amounts of unstructured data to be analyzed in a time and cost-efficient manner. Nevertheless, the evaluation of errors in results produced by LLM remains difficult. Scientific work using such methods must continue to be subject to st

许可

CC BY 4.0

引用

学术使用请引用 DOI 10.3389/frai.2026.1658575.s003（source: https://frontiersin.figshare.com/articles/dataset/Data_Sheet_2_Improving_reliability_and_accuracy_of_structured_data_extraction_using_a_consensus_large-language_model_approach_a_use_case_description_in_multiple_sclerosis_pdf/31330897）。