Skip to content
BranchesTags

Repository files navigation

Deconvolution Benchmarking Pipeline

Pipeline for benchmarking a variety of bulk RNA‑seq deconvolution tools using single‑cell references, orchestrated with Nextflow.

Overview

  1. PREPARE (subworkflow)
    1. INGEST - Standardize inputs, apply gene intersection, optionally: duplicate handling and variance filtering
    2. QC_FILTER - Gene/sample/cell filtering (skippable)
    3. MERGE_FILTERED_GENES - Merge ingest + QC filtered-gene reports
    4. MARKERS - Copy/derive marker file (if marker-based tools enabled)
    5. NORMALIZE - RNAnorm layer creation (cpm, tpm, fpkm, uq, cuf, tmm, ctf)
    6. PUBLISH_PREP - Standardize outputs
  2. DECONVOLVE (subworkflow)
    1. DECONVOLUTION per selected tool
      1. Runs tool wrapper with --bulk and --sc (and --markers for TOAST-P/GLDADec)
      2. Writes predictions .csv (one per tool)
  3. ASSESS (subworkflow)
    1. EVALUATE
      1. Merges predictions
      2. Evaluates vs ground truth (if available)
    2. VISUALIZE
      1. Creates figures from evaluations
      2. Creates figures using ground truth (if available)

Data Inputs

Format

  • Required: a samplesheet .csv with header bulk,sc_ref,marker_list,gtf,truth; marker_list and truth are optional, gtf is required.
  • Data expectations:
    • bulk: samples x genes matrix (.csv, .tsv, or .h5ad)
    • sc_ref: .h5ad with obs['cell_type']
    • Optional GrooD features: may use obs['individual'] and/or obs['condition'] if present and target set in its wrapper
  • Required for TOAST-P and GLDADec: marker_list file, leave empty to derive markers using RectanglePy

Samplesheet Parameters

Parameter Required Types Notes
bulk yes .csv/.tsv/.h5ad samples x genes
sc_ref yes .h5ad must contain obs['cell_type']
marker_list optional .csv/.tsv columns: cell_type, gene; leave empty to skip/derive
gtf yes .gtf required by normalization (tpm/fpkm gene lengths)
truth optional .csv/.tsv samples x proportions

Samplesheet Examples

With all input files:

bulk,sc_ref,marker_list,gtf,truth
/path/to/bulk1.h5ad,/path/to/sc_ref.h5ad,/path/to/markers.tsv,/path/to/genes.gtf,/path/to/ground_truth.tsv
/path/to/bulk2.h5ad,/path/to/sc_ref.h5ad,/path/to/markers.tsv,/path/to/genes.gtf,/path/to/ground_truth.tsv

Without markers & ground-truth files:

bulk,sc_ref,marker_list,gtf,truth
/path/to/bulk1.h5ad,/path/to/sc_ref.h5ad,,/path/to/genes.gtf,
/path/to/bulk2.h5ad,/path/to/sc_ref.h5ad,,/path/to/genes.gtf,

Marker List Format (TOAST-P/GLDADec)

Marker list must be a .csv/.tsv with columns cell_type and gene:

cell_type	gene
Tcell	CD3D
Tcell	CD3E
Bcell	MS4A1

GLDADec Setup Note

GLDADec expects the repository to be available, wrapper searches GLDADEC_DIR or defaults to /opt/GLDADec.

Pseudobulk benchmarking note

Pseudobulks are often supplied as CPM normalized. To avoid poor performance on pseudobulks, make sure to change tools requiring TPM bulk input to CPM. For this you can edit the nextflow.config changing the tools.config to the tools_pseudobulk_input.config.

Usage

Example

nextflow run main.nf \
  --input /path/to/samplesheet.csv \
  --output_dir /path/to/output \
  --tools adroit,autogenes,bayesprism,dwls,gldadec,grood,music,rnasieve,scaden,scdc,toastp

Parameters

Parameter Default Description
input ./samplesheet.csv Path to samplesheet .csv
output_dir ./results/ Output directory root
tools all enabled tools from registry Omit to run all enabled tools; case-insensitive.

Tools can be disabled in configs/tools.config by setting enabled: false in the tool registry entry. When --tools is omitted, the pipeline runs all tools with enabled: true.

Scale Configuration

Normalization is always executed in PREPARE and creates the same RNAnorm layers for both bulk and sc:

  • cpm, tpm, fpkm, uq, cuf, tmm, ctf
  • X remains on the counts scale
  • tpm and fpkm require gene lengths resolved from the provided gtf

Tools select their input scale through configs/tools.config (scale_bulk, scale_sc), which maps to AnnData X (counts) or one of the normalized layers above.

Ingestion Parameters

Parameter Default Description
ingest.cell_type_col cell_type Column name for cell type annotation in single-cell .obs
ingest.duplicate_handling none How to handle duplicate gene names: none, sum, mean, first
ingest.remove_zero_variance true Remove genes with zero variance on common genes

QC/Filter Parameters

QC filtering is applied when qc.enable=true. Filters are disabled if set to 0.

Parameter Default Description
qc.enable true Enable QC filtering
qc.min_samples 0 Gene must be in >=N bulk samples
qc.min_cells 0 Gene must be in >=N sc cells
qc.drop_gene_prefixes [] Gene name prefixes to drop (e.g., ['MT-', 'tRNA:'])

Bulk Sample Filtering

Parameter Default Description
qc.bulk_min_genes 0 Filter samples with fewer detected genes
qc.bulk_max_genes 0 Filter samples with more detected genes

SC Cell Filtering

Parameter Default Description
qc.sc_min_genes 0 Filter cells with fewer detected genes
qc.sc_max_genes 0 Filter cells with more detected genes
qc.sc_max_mito_pct 0 Filter cells with mito% above threshold
qc.mt_prefixes ['mt:', 'MT-'] Prefixes to identify mitochondrial genes

Marker Parameters

Parameter Default Description
markers.derive_if_missing true Derive markers if not provided (requires ingest.sc_in_scale=counts)
markers.derive_bulk_guided true Derive markers with the help of bulk data

Getting Started

Prerequisites

  • Nextflow ≥ 23 (installed on host)
  • OpenJDK 17 (installed on host)
  • Conda/Mamba or Docker

Installation

From the benchmark/env directory, run:

cd benchmark/env
./install.sh

Then choose:

  • 1 for conda
  • 2 for mamba
  • 3 for docker

For local installs, activate the environment with:

conda activate deconv-benchmark

or

mamba activate deconv-benchmark

For Docker installs, run the pipeline with:

nextflow run main.nf -profile docker ...

Outputs

  • Results are grouped by a tag derived from input basenames: <bulkBase>_<scBase>_<markerBase>_<gtfBase|noGtf>_<truthBase|noTruth>.
  • Key files:
    • prep/bulk.ready.h5ad: Preprocessed bulk data
    • prep/sc.ready.h5ad: Preprocessed single-cell reference
    • prep/markers.csv: Marker genes (when marker generation is run)
    • logs/*.log: Processing logs
    • evaluation/evaluation.csv: long‑format table of metrics per tool/cell type
    • evaluation/observation_metrics.csv: per-tool, per-sample, per-cell-type truth/prediction/error table
    • evaluation/predictions_merged.csv: wide-format table of merged predictions
    • evaluation/summary_stats.csv: summary statistics across metrics
    • evaluation/sample_qc.csv: per-tool, per-sample composition QC summary
    • figures/summary/png/*: summary overview figures in PNG format
    • figures/binned/png/*: sample-level abundance-stratified figures in PNG format
    • figures/celltype/png/*: per-cell-type figures in PNG format
    • figures/summary/svg/*: summary overview figures in SVG format
    • figures/binned/svg/*: sample-level abundance-stratified figures in SVG format
    • figures/celltype/svg/*: per-cell-type figures in SVG format

Example Structure

results/
└── Bulk_SingleCell_Markers_GTF_GroundTruth/
    ├── logs/
    │   ├── ingest.log
    │   ├── qc_filter.log
    │   ├── normalize.log
    │   ├── markers.log
    │   └── filtered_genes.csv
    ├── prep/
    │   ├── bulk.ready.h5ad
    │   ├── sc.ready.h5ad
    │   └── markers.csv
    ├── evaluation/
    │   ├── binned_celltype_metrics.csv
    │   ├── evaluation.csv
    │   ├── observation_metrics.csv
    │   ├── predictions_merged.csv
    │   ├── sample_qc.csv
    │   ├── summary_stats.csv
    │   └── tool_summary.csv
    ├── figures/
    │   ├── summary/
    │   │   ├── png/
    │   │   │   ├── composition_summary.png
    │   │   │   ├── error_cdf.png
    │   │   │   ├── overall_mean_residual_error.png
    │   │   │   ├── overall_metrics_celltypes.png
    │   │   │   ├── overall_metrics_samples.png
    │   │   │   ├── qc_summary.png
    │   │   │   └── tool_ranking.png
    │   │   └── svg/
    │   │       └── ...
    │   ├── binned/
    │   │   ├── png/
    │   │   │   ├── binned_mean_residual_error.png
    │   │   │   ├── binned_metric_trends.png
    │   │   │   └── binned_metrics.png
    │   │   └── svg/
    │   │       └── ...
    │   └── celltype/
    │       ├── png/
    │       │   ├── bland_altman/
    │       │   │   ├── bland_altman_01.png
    │       │   │   └── ...
    │       │   ├── celltype_metrics/
    │       │   │   ├── celltype_metrics_01.png
    │       │   │   └── ...
    │       │   ├── celltype_mean_residual_error/
    │       │   │   ├── celltype_mean_residual_error_01.png
    │       │   │   └── ...
    │       │   ├── celltype_error_distribution/
    │       │   │   ├── celltype_error_distribution_01.png
    │       │   │   └── ...
    │       │   ├── scatter/
    │       │   │   ├── scatter_01.png
    │       │   │   └── ...
    │       │   └── ...
    │       └── svg/
    │           └── ...
    └── predictions/
        ├── adroit.csv
        ├── autogenes.csv
        ├── bayesprism.csv
        ├── dwls.csv
        ├── gldadec.csv
        ├── grood.csv
        ├── music.csv
        ├── rnasieve.csv
        ├── scaden.csv
        ├── scdc.csv
        └── toastp.csv

Citations

Tools

Method License Citation
AdRoit Non-commercial Yang, T. et al. "AdRoit is an accurate and robust method to infer complex transcriptome composition." (AdRoit GitHub: https://github.com/TaoYang-dev/AdRoit)
AutoGeneS free (MIT) Aliee, Hassan, et al. "AutoGeneS: Automatic gene selection for bulk deconvolution of single-cell data." Bioinformatics (2020). https://doi.org/10.1093/bioinformatics/btaa1043
BayesPrism free (GPL) Chu, Tinyi et al. "Cell type and gene expression deconvolution with BayesPrism enables Bayesian integrative analysis across bulk and single-cell RNA sequencing in oncology." Nature cancer vol. 3,4 (2022). https://doi.org/10.1038/s43018-022-00356-3
DWLS free (GPL) Tsoucas, Daphne et al. "Accurate estimation of cell-type composition from gene expression data." Nature communications vol. 10,1 2975. 5 Jul. 2019. https://doi.org/10.1038/s41467-019-10802-z
GLDADec free (MIT) Azuma, Iori, Tadahaya Mizuno, and Hiroyuki Kusuhara. "GLDADec: marker-gene guided LDA modeling for bulk gene expression deconvolution." Briefings in Bioinformatics 25(4) (2024). https://doi.org/10.1093/bib/bbae315
GrooD TBD TBD
MuSiC free (GPL) Wang, Xuran et al. "Bulk tissue cell type deconvolution with multi-subject single-cell expression reference." Nature communications vol. 10,1 380. 22 Jan. 2019. https://doi.org/10.1038/s41467-018-08023-x
RNA-Sieve free (GPL) Erdmann-Pham, Dan D et al. "Likelihood-based deconvolution of bulk gene expression data using single-cell references." Genome research vol. 31,10 (2021). https://doi.org/10.1101/gr.272344.120
Scaden free (MIT) Menden, Kevin et al. "Deep learning-based cell composition analysis from tissue expression profiles." Science advances vol. 6,30 eaba2619. 22 Jul. 2020. https://doi.org/10.1126/sciadv.aba2619
SCDC free (GPL) Dong, Meichen et al. "SCDC: bulk gene expression deconvolution by multiple single-cell RNA sequencing references." Briefings in Bioinformatics (2021). https://meichendong.github.io/SCDC/articles/SCDC.html
TOAST-P free (GPL) Li, Ziyi, et al. "Robust partial reference-free cell composition estimation from tissue expression." Bioinformatics 36(11):3431-3438 (2020). https://doi.org/10.1093/bioinformatics/btaa184

Other

About

No description, website, or topics provided.

Resources

Readme
MIT license
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages