数据集获取（ModelScope）

方式一：MsDataset（Python）

# 前置依赖: pip install modelscope
from modelscope.msdatasets import MsDataset

ds = MsDataset.load("histai/SPIDER-skin", subset_name="default", split="train")
print(ds)

方式二：CLI（命令行）

pip install modelscope
modelscope download dataset histai/SPIDER-skin

数据集卡片摘录（源站）

SPIDER-SKIN Dataset

SPIDER is a collection of supervised pathological datasets covering multiple organs, each with comprehensive class coverage. These datasets are professionally annotated by pathologists.

If you would like to support, sponsor, or obtain a commercial license for the SPIDER data and models, please contact us at models@hist.ai.

For a detailed description of SPIDER, methodology, and benchmark results, refer to our research paper:

SPIDER: A Comprehensive Multi-Organ Supervised Pathology Dataset and Baseline Models
View on arXiv

This repository contains the SPIDER-skin dataset. To explore datasets for other organs, visit the Hugging Face HistAI page or GitHub. SPIDER is regularly updated with new organs and data, so follow us on Hugging Face to stay updated.

Overview

SPIDER-skin is a supervised dataset of image-class pairs for the skin organ. Each data point consists of:

• A central 224×224 patch with a class label
• 24 surrounding context patches of the same size, forming a composite 1120×1120 region
• Patches are extracted at 20X magnification

We provide a train-test split for consistent benchmarking. The split is done at the slide level, ensuring that patches from the same whole slide image (WSI) do not appear in both training and test sets. Users can also merge and re-split the data as needed.

How to Use

Downloading the Dataset

Option 1: Using huggingface_hub

from huggingface_hub import snapshot_download

snapshot_download(repo_id="histai/SPIDER-skin", repo_type="dataset", local_dir="/local_path")

Option 2: Using git

# Ensure you have Git LFS installed (https://git-lfs.com)
git lfs install
git clone https://huggingface.co/datasets/histai/SPIDER-skin

Extracting the Dataset

The dataset is provided in multiple tar archives. Unpack them using:

cat spider-skin.tar.* | tar -xvf -

Using the Dataset

Once extracted, you will find:

• An images/ folder
• A metadata.json file

You can process and use the dataset in two ways:

1. Directly in Code (Recommended for PyTorch Training)

Use the dataset class provided in scripts/spider_dataset.py. This class takes:

• Path to the dataset (folder containing metadata.json and

许可

Apache License 2.0

数据加载示例（图像类）

from PIL import Image
import glob, os

files = (glob.glob(os.path.join(path, "**", "*.png"), recursive=True)
       + glob.glob(os.path.join(path, "**", "*.jpg"), recursive=True)
       + glob.glob(os.path.join(path, "**", "*.tif"), recursive=True))
print("图像文件数:", len(files))
img = Image.open(files[0]); print("尺寸/模式:", img.size, img.mode)

# torchvision Dataset 方式:
# from torchvision import datasets
# ds = datasets.ImageFolder(path)  # 要求 子目录=类别

目录组织与标注格式以源站说明和下载后实际文件为准。

源站：modelscope.cn/datasets/histai/SPIDER-skin