Add XL-SafetyBench datasets#1791
Merged
romanlutz merged 30 commits intoJul 20, 2026
Merged
Conversation
Implements Phase 1 (seed datasets) and Phase 2 (judge scorers) of XL-SafetyBench (arXiv:2605.05662), a country-grounded cross-cultural benchmark for LLM safety and cultural sensitivity (CC-BY-4.0, AIM-Intelligence/XL-SafetyBench on HuggingFace). Datasets (pyrit/datasets/seed_datasets/remote/xl_safety_bench_dataset.py): - _XLSafetyBenchJailbreakDataset (4,500 prompts, 5 categories, 10 countries) - _XLSafetyBenchCulturalDataset (1,000 prompts, 6 categories, 10 countries) - language_mode='local'|'english' selects scenario language - hidden_violation preserved in metadata for the cultural judge - Shared enums: XLSafetyBenchCountry, XLSafetyBenchJailbreakCategory, XLSafetyBenchCulturalCategory Judges (pyrit/score/true_false/): - XLSafetyBenchJailbreakJudgeScorer: 4-criterion judge (H/NC/S/R), produces a label (Critical/Direct/Soft Unsafe, Ideal/Neutral Safe) with the booleans in score_metadata for downstream ASR/NSR aggregation - XLSafetyBenchCulturalJudgeScorer: relevance + cultural recognition; CSR truthy when the response flags the embedded sensitivity - Judge prompts ported verbatim from the paper's GitHub repo and stored as companion .txt files for readability Tests: 34 new unit tests (17 dataset + 17 scorer) covering filters, metadata propagation, language modes, malformed-JSON retries, and the full jailbreak label / cultural matrix. Phase 3 (Scenarios) and Phase 4 (example notebook) deferred to follow-ups. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Following the sibling-loaders pattern (each distinct upstream artifact gets its own registered SeedDatasetProvider subclass so the daily e2e matrix exercises every URL), split both XL-SafetyBench tracks into country-pinned subclasses: - 10 jailbreak siblings (one per country, pinning `countries=[X]`, `size="medium"`, inheriting the parent's "default" tag) - 10 cultural siblings (same shape, `language_mode` remains a kwarg since it selects a column in the same CSV) The parent loaders are unchanged; the kwarg-based API still works. Also fixes the cultural parent's size bucket: 1,000 prompts falls in the "large" (500-4999) range, not "medium". Added 22 unit tests covering dataset_name, country pinning, tag inheritance, and kwarg forwarding (categories + language_mode) for the siblings. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…chmark-integration
romanlutz
marked this pull request as ready for review
May 28, 2026 05:21
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…vely Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…chmark-integration # Conflicts: # doc/bibliography.md
…ch loader Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…precated shim) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…chmark-integration # Conflicts: # doc/bibliography.md # pyrit/score/__init__.py
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 68ca6a05-cfaf-4967-839a-907baa2b937f
Keep PR microsoft#1791 focused on dataset loaders and move the scorer layer to a stacked follow-up session. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 68ca6a05-cfaf-4967-839a-907baa2b937f
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 68ca6a05-cfaf-4967-839a-907baa2b937f
Add the XL-SafetyBench citation and regenerate executed output so all benchmark dataset providers appear in the rendered list. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 68ca6a05-cfaf-4967-839a-907baa2b937f
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 68ca6a05-cfaf-4967-839a-907baa2b937f
Use the existing countries argument on each logical dataset instead of registering 20 country-specific sibling classes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 68ca6a05-cfaf-4967-839a-907baa2b937f
Resolve dataset registry, bibliography, and loading notebook conflicts while preserving WildGuardMix and XL-SafetyBench entries. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 68ca6a05-cfaf-4967-839a-907baa2b937f
hannahwestra25
approved these changes
Jul 20, 2026
hannahwestra25
left a comment
Contributor
There was a problem hiding this comment.
minor comment related to the title but lgtm otherwise!
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
romanlutz
enabled auto-merge
July 20, 2026 22:46
romanlutz
disabled auto-merge
July 20, 2026 23:03
romanlutz
enabled auto-merge
July 20, 2026 23:03
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the XL-SafetyBench seed datasets to PyRIT. The benchmark contains 5,500 native-validated prompts across 10 country-language pairs and covers country-grounded jailbreak safety plus cultural sensitivity.
This PR is intentionally dataset-only. The benchmark-specific judges have been moved to a separate stacked follow-up.
What's in this PR
Three logical providers under
pyrit/datasets/seed_datasets/remote/xl_safety_bench_dataset.py:_XLSafetyBenchJailbreakDatasetSeedPrompt_XLSafetyBenchJailbreakObjectivesDatasetSeedObjective_XLSafetyBenchCulturalDatasetSeedPromptEach loader accepts
countries: list[XLSafetyBenchCountry] | None;Noneloads all 10 countries, while callers can provide one or several enum values to select only those country files. This avoids separate per-country provider classes while retaining single-country and multi-country loading.Shared enums cover the 10 countries, five jailbreak categories, six cultural categories, and language mode. Per-row provenance is preserved in metadata, including country/language display information, base queries, localized scenarios, and hidden violations.
Documentation
doc/code/datasets/1_loading_datasetscites XL-SafetyBench and its executed output lists the three logical benchmark providers.Tests
tests/unit/datasets/test_xl_safety_bench_dataset.pycovers all-country defaults, single- and multi-country filtering, country deduplication, objectives, category filtering, language selection, metadata propagation, schema validation, UTF-8 BOM normalization, and the three-name registry contract.Notes for reviewers
idcolumn; rows are normalized before field access so IDs and prompt names are retained.defaulttag. Objectives and cultural loaders intentionally omit it to avoid duplicate/default sweep coverage.Deferred follow-ups
Verification
ty, and documentation checks