医疗转录指令数据集

数据集获取（ModelScope）

方式一：MsDataset（Python）

# 前置依赖: pip install modelscope
from modelscope.msdatasets import MsDataset

ds = MsDataset.load("AI-ModelScope/medical-transcription-instruct", subset_name="default", split="train")
print(ds)

方式二：CLI（命令行）

pip install modelscope
modelscope download dataset AI-ModelScope/medical-transcription-instruct

数据集卡片摘录（源站）

<a href=“https://www.datafog.ai”>
<img src=“https://www.datafog.ai/colorlogo.png” alt=“DataFog logo” width=“300”>
</a>

About

This dataset consists of 38,924 samples of instruct-input-output data, most helpfully for training instruction-following models tailored to the medical field

Dataset Summary

• Source: Original medical transcriptions with added instruction-output pairs
• Size: 38,924 instruction-output pairs
• Format: CSV file
• Domain: Medical / Healthcare
• Language: English
• Last Updated: 08-20-2024

Dataset Structure

Each row in the dataset represents a unique instruction-output pair based on a medical transcription. The columns are organized as follows:

1. instruction: The task or question to be performed on the medical text
2. task_output: The expected output or answer for the given instruction
3. transcription: The original medical transcription text
4. description: A brief description or summary of the transcription
5. medical_specialty: The medical specialty associated with the transcription
6. sample_name: A name or identifier for the transcription sample
7. keywords: Original keywords associated with the transcription (if available)
8. derived_keywords: Automatically extracted keywords using TF-IDF
9. transcription_length: The character count of the transcription
10. normalized_length: The transcription length normalized to a 0-1 scale
11. complexity_score: A measure of the transcription’'s textual complexity

Task Types

The dataset includes various instruction types, such as:

1. Identifying medical specialties
2. Summarizing transcriptions
3. Extracting keywords
4. Assessing text complexity
5. Determining relative transcription length
6. Suggesting follow-up questions

Intended Uses

This dataset is suitable for:

• Fine-tuning language models for medical text analysis
• Developing instruction-following models in the healthcare domain
• Research in medical natural language processing
• Exploring various aspects of medical transcriptions

Ethical Considerations

• This dataset contains medical information. While it has been de-identified, users should be cautious about potential privacy concerns.
• The data should not be used for making real-world medical decisions without proper validation and expert oversight.
• Biases may exist in the original transcriptions or de

许可

Apache License 2.0

数据加载示例（图像类）

from PIL import Image
import glob, os

files = (glob.glob(os.path.join(path, "**", "*.png"), recursive=True)
       + glob.glob(os.path.join(path, "**", "*.jpg"), recursive=True)
       + glob.glob(os.path.join(path, "**", "*.tif"), recursive=True))
print("图像文件数:", len(files))
img = Image.open(files[0]); print("尺寸/模式:", img.size, img.mode)

# torchvision Dataset 方式:
# from torchvision import datasets
# ds = datasets.ImageFolder(path)  # 要求 子目录=类别

目录组织与标注格式以源站说明和下载后实际文件为准。

源站：modelscope.cn/datasets/AI-ModelScope/medical-transcription-instruct