对解释的忠实度

数据集获取（Figshare）

• 源站标题：degree of faithfulness to the explanation.
• DOI：10.1371/journal.pone.0322529.t005
• 发布日期：2025-05-28
• 文件数：1 个
• 文件总体积：5.63 KB
• 源站页面：https://plos.figshare.com/articles/dataset/degree_of_faithfulness_to_the_explanation_/29173305

命令行下载

curl -L -o "Table 5.xls" "https://ndownloader.figshare.com/files/54900252"

文件清单

• Table 5.xls（5.63 KB）

文件总体积 5.63 KB（源站 API 实测）。

数据加载示例（图像类）

from PIL import Image
import glob, os

files = (glob.glob(os.path.join(path, "**", "*.png"), recursive=True)
       + glob.glob(os.path.join(path, "**", "*.jpg"), recursive=True)
       + glob.glob(os.path.join(path, "**", "*.tif"), recursive=True))
print("图像文件数:", len(files))
img = Image.open(files[0]); print("尺寸/模式:", img.size, img.mode)

# torchvision Dataset 方式:
# from torchvision import datasets
# ds = datasets.ImageFolder(path)  # 要求 子目录=类别

目录组织与标注格式以源站说明和下载后实际文件为准。

数据说明

Introduction
In this study, we aim to evaluate the ability of large language models (LLM) to generate questions and answers in oral and maxillofacial surgery.
Methods
ChatGPT4, ChatGPT4o, and Claude3-Opus were evaluated in this study. Each LLM was instructed to generate 50 questions about oral and maxillofacial surgery. Three LLMs were asked to answer the generated 150 questions.
Results
All 150 questions generated by the three LLMs were related to oral and maxillofacial surgery. Each model exhibited a correct answer rate of over 90%. None of the three models were able to answer correctly all the questions they generated themselves. The correct answer rate was 97.0% for questions with figures, significantly higher than the 88.9% rate for questions without figures. The analysis of problem-solving by the three LLMs showed that each model generally inferred answers with high accuracy, and there were few logical errors that could be considered controversial. Additionally, all three scored above 88% for the fidelity of their explanations.
Conclusion
This study demonstrates that while LLMs like ChatGPT4, ChatGPT4o, and Claude3-Opus exhibit robust capabilities in generating and solving oral and maxillofacial surgery questions, their performance is not without limitations. None of the models were able to answer correctly all the questions they generated themselves, highlighting persistent challenges such as AI hallucinations and contextual understanding gaps. The results also emphasize the importance of multimodal inputs, as questions with annotated images achieved higher accuracy rates compared to text-only prompts. Despite these shortcomings, the LLMs showed significant promise in problem-solving, logical consistency, and response fidelity, particularly in structured or numerical contexts.

许可

CC BY 4.0

引用

学术使用请引用 DOI 10.1371/journal.pone.0322529.t005（source: https://plos.figshare.com/articles/dataset/degree_of_faithfulness_to_the_explanation_/29173305）。