Skip to content
AmyIvanPublic

About

Official repository for the Findings of EMNLP 2026 paper MOSAIC: A Large-Scale Multimodal Open-Course Segmentation and Alignment Corpus in Chinese.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

7 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

MOSAIC Dataset for EMNLP 2026 Findings

Paper title: MOSAIC: A Large-Scale Multimodal Open-Course Segmentation and Alignment Corpus in Chinese
Authors: Yuming Ai, Shuai Fan, Hua Xu, Fang Kong
Paper link: ACL Anthology | PDF | BibTeX

This repository contains the GitHub-side release materials for the MOSAIC dataset accompanying a Findings of EMNLP 2026 paper. The GitHub repository provides documentation, utility scripts, and release tooling. The public dataset release is hosted on Hugging Face Datasets as AmyIvan/mosaic-emnlp2026.

Abstract

Real-world lectures follow a predefined topic inventory (e.g., a syllabus), but instructors often reorder, paraphrase, or skip items, creating mismatches between the outline and the spoken content. Lectures also contain non-topical utterances (openings, transitions and wrap-ups) that should not be assigned to any topic, yet existing corpora rarely model both challenges. We present MOSAIC (Multimodal Open-course Segmentation and Alignment corpus In Chinese), a large-scale Chinese MOOC corpus for topic-guided segmentation and alignment under predefined inventories, with explicit non-topical labels and matched slide resources. MOSAIC includes MOSAIC-G (human-annotated gold benchmark with utterance-level topic alignment and video-level links to matched slide resources) and MOSAIC-S (automatically labeled silver data), collectively covering 179 courses and 14,942 videos. To scale annotation, we propose DORA (Dual-stage Outline Refinement and Alignment), a two-stage pipeline that refines noisy inventories and then performs joint segmentation and topic assignment. Experiments show that silver supervision substantially improves the performance of local models and that incorporating slide text consistently benefits knowledge-point extraction, a lecture-comprehension proxy.

Overview

MOSAIC is a course-centric multimodal dataset built from online course content. The primary data file is mosaic.jsonl, which stores course-level metadata together with nested video-level annotations. Two auxiliary resources complement the main file:

  • graph_p_results/: course-level knowledge graph JSON files keyed by the kg field in mosaic.jsonl
  • all.csv: metadata mapping video-level slide-resource URLs to internal source filenames

The public release records video-level links to matched slide resources. It does not redistribute original videos, full slide decks, slide-page images, or slide OCR text. No manual utterance-to-slide-page annotations are included or claimed. See DATA_RIGHTS.md for the component-specific release scope.

Dataset Description

Intended Use

This dataset is designed for research on:

  • multimodal educational data understanding
  • video-level slide-resource linking
  • course knowledge graph construction and retrieval
  • subtitle and caption quality analysis
  • document-aware summarization and downstream LLM applications

Components

  • mosaic.jsonl: main structured dataset
  • graph_p_results/*.json: auxiliary knowledge graphs
  • all.csv: slide-resource URL to internal source-filename mapping; the source files are not redistributed

Scale

Item Count
Courses in mosaic.jsonl 179
Videos nested in mosaic.jsonl 14,942
Knowledge graph JSON files 167
Videos with matched slide resources 10,385
Unique matched slide-resource URLs used 7,976

Notes

  • There is currently no predefined train/dev/test split in the released files.
  • mosaic.jsonl is the main entry point for loading the dataset.
  • graph_p_results is course-level auxiliary data. All non-empty kg values in mosaic.jsonl map to an existing graph JSON file.
  • all.csv is a URL-to-source-filename metadata table for slide references. The main file stores video-level resource URLs in ref.doc; the corresponding source files are not part of the public release.

Data Structure

GitHub Repository Structure

.
├── README.md
├── LICENSE
├── DATA_RIGHTS.md
├── .gitignore
├── scripts/
│   ├── download.py
│   ├── load_dataset.py
│   └── stats.py
└── hf_dataset/
    ├── README.md
    └── upload_to_hf.py

Hugging Face Dataset Repository Structure

.
├── README.md
├── LICENSE
├── DATA_RIGHTS.md
└── data/
    ├── mosaic.jsonl
    ├── all.csv
    ├── graph_p_results/
    │   ├── BIT-1001604004.json
    │   ├── BIT-1001605004.json
    │   └── ...

Data Format

1. mosaic.jsonl

Each line is one JSON object representing one course.

Top-level schema

Field Type Description
url string Course homepage URL.
course_title string Course title as displayed on the source platform.
contents list[section] Ordered list of course sections.
kg string or empty string Course-level knowledge graph identifier. When non-empty, it maps to graph_p_results/<kg>.json.
caption_anno string Caption annotation category at course level. Observed values include silver and gold.
overview string Course overview or description.
objectives string or null Learning objectives.
prerequisites string or null Prerequisites or recommended background.
references string or null Course references or reading list.

contents section schema

Each element of contents is a section object:

Field Type Description
title string Section title.
courses list[video] Ordered list of video items within the section.

Video-level schema

Each element of courses is a video object:

Field Type Description
video_url string Video content URL.
srt_url string Subtitle source URL.
summary string Text summary for the video.
subtitle list[timed_text] Fine-grained subtitle segments.
caption list[timed_text] Coarser caption segments.
video_title string Video title.
ref object Reference metadata linking the video to auxiliary resources.

timed_text schema

Used by both subtitle and caption.

Field Type Description
index integer Segment index.
start_time string Start timestamp in HH:MM:SS.mmm format.
end_time string End timestamp in HH:MM:SS.mmm format.
sentence string Segment text.

ref schema

Field Type Description
cate string Reference category. Observed values include silver, gold, copper, and no ref to match.
doc list[string] Zero or more video-level reference document URLs. These URLs map to source-filename metadata via all.csv; they are not utterance-to-page labels.

Example lookup flow

For a given video item:

  1. Read ref.doc
  2. Look up each URL in all.csv using the doc_url column
  3. Treat the result as video-level source-resource metadata. Access to the linked source resource remains subject to the source platform's terms and applicable permissions.

2. graph_p_results/*.json

Each file is a course-level knowledge graph JSON document.

Top-level schema

Field Type Description
code integer Status code.
message string or null Message returned by the upstream pipeline.
sampled boolean or null Optional sampling flag from the upstream pipeline.
traceId string or null Optional trace identifier.
result object Main knowledge graph payload.

result schema

Field Type Description
mocKgNodeDtoList list[node] Flattened knowledge graph node list.

Node schema

Observed node objects include the following fields:

Field Type Description
id integer Node identifier.
parentId integer Parent node identifier. Root nodes usually use -1.
knowledgeGraphId integer Knowledge graph identifier from the upstream system.
nodeName string Node title or concept name.
description string or null Node description.
position integer Display or ordering position.
level integer Node depth level.
childCount integer Number of children.
children list[node] or null Nested children for some nodes.
knowledgeCount integer or null Optional upstream count field.
questionCount integer or null Optional upstream count field.
labelNameList list[string] or null Optional label list.
createTime string or null Creation timestamp.
updateTime string or null Update timestamp.
mocKgNodeMemberStatisticsDto object or null Optional upstream statistics field.
mocKgNodeAvgStatisticsDto object or null Optional upstream statistics field.

3. all.csv

The CSV file maps reference document URLs to internal source filenames. The corresponding source files are not redistributed in the public release.

Column Type Description
doc_url string Reference slide URL found in mosaic.jsonl -> contents[*].courses[*].ref.doc.
filename string Internal source filename retained for metadata consistency; no corresponding file is included publicly.

Statistics

Metric Value
Courses 179
Videos 14,942
Graph JSON files 167
Videos with matched slide resources 10,385
Unique ref.doc URLs used by the main file 7,976
mosaic.jsonl size ~353 MB
graph_p_results/ size ~104 MB
all.csv size ~1.94 MB

Download and Usage

Option 1: Download the public release with huggingface_hub

from huggingface_hub import snapshot_download

local_path = snapshot_download(
    repo_id="AmyIvan/mosaic-emnlp2026",
    repo_type="dataset",
    local_dir="mosaic_data",
)

print(local_path)

Option 2: Load mosaic.jsonl directly with datasets

from datasets import load_dataset

dataset = load_dataset(
    "json",
    data_files={
        "train": "hf://datasets/AmyIvan/mosaic-emnlp2026/data/mosaic.jsonl"
    },
)

print(dataset["train"][0].keys())

Utility scripts

python scripts/download.py --repo-id AmyIvan/mosaic-emnlp2026 --local-dir ./downloads/mosaic
python scripts/load_dataset.py --root .
python scripts/stats.py --root .

Cite this work

If you use MOSAIC in your research, please cite our paper using the official ACL Anthology BibTeX:

@inproceedings{ai-etal-2026-mosaic,
    title = "{MOSAIC}: A Large-Scale Multimodal Open-Course Segmentation and Alignment Corpus in {C}hinese",
    author = "Ai, Yuming  and
      Fan, Shuai  and
      Xu, Hua  and
      Kong, Fang",
    editor = "Lee, Hung-Yi  and
      Dev, Sunipa  and
      Li, Jessy  and
      Elliott, Desmond",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {EMNLP} 2026",
    month = oct,
    year = "2026",
    address = "Budapest, Hungary",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.findings-emnlp.246/",
    pages = "5176--5195",
    ISBN = "979-8-89176-462-0"
}

License

The CC BY-NC-SA 4.0 notice in LICENSE applies only to original MOSAIC materials for which the authors hold the necessary rights. It does not license third-party educational source content. Source-derived fields and resource links remain subject to their respective terms and applicable permissions. See DATA_RIGHTS.md for details.

About

Official repository for the Findings of EMNLP 2026 paper MOSAIC: A Large-Scale Multimodal Open-Course Segmentation and Alignment Corpus in Chinese.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages