diff --git a/CHANGELOG.md b/CHANGELOG.md index fb2c964c..f6abb45a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### `Changed` +- [#207](https://github.com/IntGenomicsLab/lrsomatic/pull/207) - Aligned and haplotagged reads are now CRAM 3.0 with the reference embedded instead of BAM, from `MINIMAP2_ALIGN` onwards; `bamfiles/` holds `_.cram` + `.crai`. The new `--aligned_format` parameter (`cram` by default, or `bam`) keeps BAM throughout for tools that cannot read CRAM. On chr20 of three real samples CRAM is 33 % (ONT), 52 % (older PacBio) and 69 % (Revio) smaller, with identical records. Version 3.0 because Clair3 and ClairS-TO's Verdict bundle an htslib that cannot decode 3.1 and then call nothing without failing; the embedded reference (1-7 % of the file) because ClairS, ClairS-TO, Severus and SAVANA open the reads without the FASTA. `CRAMINO_POST` and `SAMTOOLS_MERGE` now get the FASTA (`ASCAT` does not: its image has no `ref.fasta` argument, and it reads the embedded reference); `CLAIRSTO` deletes its intermediate haplotagged copy of the tumour reads; the uBAM's own `RG` tag is dropped before alignment, since CRAM keeps one read group per record and kept the basecaller's. That change alters `MINIMAP2_ALIGN` in both formats, so a run resumed from an earlier version realigns every sample, also with `--aligned_format bam` (@ljwharbers). - [#201](https://github.com/IntGenomicsLab/lrsomatic/pull/201) - `CLAIRSTO` and `CLAIRSTO_VERDICT_TAG` now pull the fork image from Docker Hub: `oras://docker.io/ljwharbers/clairs-to-sif:0.5.1-verdict-chm13-c0687e8-flat` under Singularity/Apptainer and `docker.io/ljwharbers/clairs-to:0.5.1-verdict-chm13-c0687e8-flat` otherwise, instead of `ghcr.io/ljwharbers/clairs-to`. The `-cpu` SIF on ghcr failed with `PROTOCOL_ERROR` on slow links: ghcr redirects every blob download to an Azure URL that expires at the next 5-minute mark and resets a stream still open then, and Apptainer resumes neither an `oras://` nor a `docker://` download. Docker Hub's download URLs are valid for 50 minutes and only checked when the request starts. `-flat` is the same software copied into an empty image in a few layers (3.3 GB instead of 7 GB); the software and its outputs are unchanged. `docs/usage.md` describes `pullTimeout`, Docker Hub's anonymous pull limit, pre-pulling, and how to recover the remaining `oras://ghcr.io` SIFs resumably (@ljwharbers). - [#199](https://github.com/IntGenomicsLab/lrsomatic/pull/199) - `CLAIRSTO` and `CLAIRSTO_VERDICT_TAG` now run the `-cpu` rebuild of the fork image (`0.5.1-verdict-chm13-c0687e8-cpu`), which swaps PyTorch's CUDA build for the CPU build of the same version. The software is otherwise unchanged, but the Apptainer SIF drops from 6.53 GB to 3.46 GB. The old image could not be pulled on a normal VSC link: Apptainer fetches an `oras://` SIF as a single unresumable stream, and the signed blob URL ghcr redirects to expires on a 15-minute wall-clock boundary, so 6.53 GB needed 7.3 MB/s sustained and was otherwise cut mid-transfer with `PROTOCOL_ERROR` (@ljwharbers). - [#197](https://github.com/IntGenomicsLab/lrsomatic/pull/197) - `CLAIRSTO` now runs `ghcr.io/ljwharbers/clairs-to:0.5.1-verdict-chm13-c0687e8` (ClairS-TO 0.5.1) instead of `docker.io/hkubal/clairs-to:v0.4.2`: a fork that lets Verdict read its CNA resources from `--cna_resource_dir`, fixes four places where Verdict's Python port of ASCAT departed from R, and disables Verdict with a warning when its resources cannot be read. **GRCh38 results move as well as CHM13 ones.** Revert to the upstream image once HKU-BAL/ClairS-TO carries these changes. The module also selects the SIF under `-profile apptainer` and sets explicit output prefixes (@ljwharbers). diff --git a/conf/modules.config b/conf/modules.config index 54494857..12ab1304 100644 --- a/conf/modules.config +++ b/conf/modules.config @@ -306,6 +306,13 @@ process { "-R '${rg}'" ].join(' ').trim() } + // --aligned_format cram: CRAM 3.0, not 3.1, because the htslib bundled in Clair3 and in ClairS-TO's + // Verdict (1.11) cannot decode 3.1 slices and then calls nothing without failing; embed_ref=1 because + // ClairS, ClairS-TO, Severus and SAVANA's read counting open the reads without the reference (1-7 % larger) + ext.args2 = { params.aligned_format == 'cram' ? '--output-fmt-option version=3.0 --output-fmt-option embed_ref=1' : '' } + // Drop the uBAM's own RG tag: -T '*' would carry it through -y next to the @RG from -R, and a record + // with two RG tags keeps only the last one in CRAM -- the basecaller's, which has no @RG line here + ext.args3 = "-x RG" ext.args4 = "-T '*'" publishDir = [ enabled: false @@ -314,6 +321,7 @@ process { withName: '.*:SAMTOOLS_MERGE' { ext.prefix = { "${meta.id}_${meta.type}_mapped" } + ext.args = { params.aligned_format == 'cram' ? '--output-fmt-option version=3.0 --output-fmt-option embed_ref=1' : '' } publishDir = [ enabled: false ] @@ -416,6 +424,9 @@ process { params.longphase_tag_supplementary ? "--tagSupplementary" : '' ].join(' ').trim() } + // --aligned_format cram: CRAM through a FIFO, so the reference can be embedded (longphase's own --cram cannot) + ext.cram = { params.aligned_format == 'cram' } + ext.args2 = '--output-fmt-option version=3.0 --output-fmt-option embed_ref=1' publishDir = [ path: { "${params.outdir}/${meta.id}/bamfiles" }, mode: params.publish_dir_mode, diff --git a/docs/output.md b/docs/output.md index 54fb647c..28bc8ec1 100644 --- a/docs/output.md +++ b/docs/output.md @@ -152,18 +152,22 @@ The pipeline produces per-sample output directories. Two modes exist depending o ``` ├── bamfiles -│ ├── sample_normal.bam -│ ├── sample_normal.bam.bai -│ ├── sample_tumor.bam -│ ├── sample_tumor.bam.bai -``` - -| File | Description | -| ----------------------- | ---------------------------------------------------------------------------------------------------- | -| `sample_normal.bam` | Aligned and haplotagged bam file (with methylation and nucleosome predictions) for the normal sample | -| `sample_normal.bam.bai` | index file for the normal bam file | -| ` sample_tumor.bam` | Aligned and haplotagged bam file (with methylation and nucleosome predictions) for the tumor sample | -| `sample_tumor.bam.bai` | index file for the tumor bam file | +│ ├── sample_normal.cram +│ ├── sample_normal.cram.crai +│ ├── sample_tumor.cram +│ ├── sample_tumor.cram.crai +``` + +| File | Description | +| ------------------------- | ----------------------------------------------------------------------------------------------------- | +| `sample_normal.cram` | Aligned and haplotagged CRAM file (with methylation and nucleosome predictions) for the normal sample | +| `sample_normal.cram.crai` | index file for the normal CRAM file | +| `sample_tumor.cram` | Aligned and haplotagged CRAM file (with methylation and nucleosome predictions) for the tumor sample | +| `sample_tumor.cram.crai` | index file for the tumor CRAM file | + +The CRAM files are version 3.0 with the reference embedded, so they can be read without the reference FASTA +(`samtools view sample_tumor.cram`); `samtools view -b` turns one back into a BAM. With `--aligned_format bam` +this directory holds `sample_{normal,tumor}.bam` and `.bam.bai` instead. diff --git a/docs/usage.md b/docs/usage.md index 7d822ecb..85a6959c 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -242,6 +242,13 @@ opt-in. See [VEP plugins](#vep-plugins) for sizes, licence terms and per-assembl | `--minimap2_ont_model` | specifies which model to use minimap2 with for ONT samples. Default = `null` | | `--minimap2_pb_model` | specifies which model to use minimap2 with for PacBio samples. Default = `null` | | `--save_secondary_alignment` | A boolean to specify if secondary alignments are kept in aligned bam file. Default = `true` | +| `--aligned_format` | `cram` or `bam`: format of the aligned and haplotagged reads. Default = `cram` (see below) | + +`--aligned_format cram` writes CRAM 3.0 with the reference embedded, from alignment onwards and in `bamfiles/`. On +long reads this is a third (ONT) to two thirds (PacBio Revio) smaller than BAM, with identical records and the same +calls from every tool in the pipeline. Because the reference is embedded, the files can be read without the FASTA. +Use `--aligned_format bam` if a tool you run on the results cannot read CRAM, or convert afterwards with +`samtools view -b -o sample_tumor.bam sample_tumor.cram`. #### ASCAT Options diff --git a/modules.json b/modules.json index ab8746ac..111b1396 100644 --- a/modules.json +++ b/modules.json @@ -87,7 +87,8 @@ "longphase/haplotag": { "branch": "master", "git_sha": "b8d30a43f33aee3148b0e9e9f00587984a4ac195", - "installed_by": ["modules"] + "installed_by": ["modules"], + "patch": "modules/nf-core/longphase/haplotag/longphase-haplotag.diff" }, "longphase/phase": { "branch": "master", diff --git a/modules/local/clairsto/main.nf b/modules/local/clairsto/main.nf index 05cb64d4..3313f59a 100644 --- a/modules/local/clairsto/main.nf +++ b/modules/local/clairsto/main.nf @@ -71,6 +71,11 @@ process CLAIRSTO { cp -- "\$src" "${prefix}_Tumor_\${table}.txt" fi done + + # The intermediate haplotagged copy of the tumor reads (phased_bam_output/, always BAM, so larger than a + # CRAM input) is unused once ClairS-TO exits; --remove_intermediate_dir would also take cna_output above. + # 0.5.1 names the directory tmp_. + rm -rf tmp*/phasing_output/phased_bam_output """ stub: diff --git a/modules/local/cramino/main.nf b/modules/local/cramino/main.nf index e93d100f..5e68a3d8 100644 --- a/modules/local/cramino/main.nf +++ b/modules/local/cramino/main.nf @@ -9,6 +9,7 @@ process CRAMINO { input: tuple val(meta), path(bam) + tuple val(meta2), path(fasta) // [[:], []] for BAM/uBAM input output: tuple val(meta), path("*.txt"), emit: txt @@ -21,9 +22,10 @@ process CRAMINO { script: def args = task.ext.args ?: '' def prefix = task.ext.prefix ?: "${meta.id}" + def reference = fasta ? "--reference ${fasta}" : '' """ - cramino $args $bam --arrow ${prefix}.arrow > ${prefix}_cramino.txt + cramino $args $reference $bam --arrow ${prefix}.arrow > ${prefix}_cramino.txt """ stub: diff --git a/modules/local/cramino/meta.yml b/modules/local/cramino/meta.yml index 93fbea6c..6a52d6f4 100644 --- a/modules/local/cramino/meta.yml +++ b/modules/local/cramino/meta.yml @@ -33,6 +33,15 @@ input: - edam: "http://edamontology.org/format_25722" - edam: "http://edamontology.org/format_2573" - edam: "http://edamontology.org/format_3462" + - - meta2: + type: map + description: | + Groovy Map containing reference information + e.g. `[ id:'genome' ]` + - fasta: + type: file + description: Reference FASTA to decode CRAM input; empty for BAM/uBAM + pattern: "*.{fa,fasta,fna}" output: txt: diff --git a/modules/nf-core/longphase/haplotag/environment.yml b/modules/nf-core/longphase/haplotag/environment.yml index f436bdae..6d2cb363 100644 --- a/modules/nf-core/longphase/haplotag/environment.yml +++ b/modules/nf-core/longphase/haplotag/environment.yml @@ -7,3 +7,4 @@ channels: dependencies: - bioconda::htslib=1.23.1 - bioconda::longphase=2.0.1 + - bioconda::samtools=1.23.1 diff --git a/modules/nf-core/longphase/haplotag/longphase-haplotag.diff b/modules/nf-core/longphase/haplotag/longphase-haplotag.diff new file mode 100644 index 00000000..e18b54b6 --- /dev/null +++ b/modules/nf-core/longphase/haplotag/longphase-haplotag.diff @@ -0,0 +1,65 @@ +Changes in component 'nf-core/longphase/haplotag' +Changes in 'longphase/haplotag/environment.yml': +--- modules/nf-core/longphase/haplotag/environment.yml ++++ modules/nf-core/longphase/haplotag/environment.yml +@@ -7,3 +7,4 @@ + dependencies: + - bioconda::htslib=1.23.1 + - bioconda::longphase=2.0.1 ++ - bioconda::samtools=1.23.1 + +'modules/nf-core/longphase/haplotag/meta.yml' is unchanged +Changes in 'longphase/haplotag/main.nf': +--- modules/nf-core/longphase/haplotag/main.nf ++++ modules/nf-core/longphase/haplotag/main.nf +@@ -23,11 +23,27 @@ + + script: + def args = task.ext.args ?: '' ++ def args2 = task.ext.args2 ?: '' + def prefix = task.ext.prefix ?: "${meta.id}" + def sv_file = svs ? "--sv-file ${svs}" : "" + def mod_file = mods ? "--mod-file ${mods}" : "" ++ // ext.cram: longphase writes its BAM into a FIFO that samtools encodes as CRAM (options from ext.args2, ++ // e.g. embed_ref=1, which longphase's own --cram cannot set); no intermediate BAM touches the disk ++ def cram_out = task.ext.cram ?: false ++ def cram_start = cram_out ? """ ++ mkfifo ${prefix}.bam ++ samtools view --threads ${task.cpus} -C -T ${fasta} ${args2} -o ${prefix}.cram ${prefix}.bam & ++ conv=\$! ++ trap 'kill \$conv 2>/dev/null || true' EXIT ++ """ : '' ++ def cram_end = cram_out ? """ ++ wait \$conv ++ trap - EXIT ++ rm ${prefix}.bam ++ """ : '' + + """ ++ ${cram_start} + longphase \\ + haplotag \\ + $args \\ +@@ -38,7 +54,7 @@ + --bam ${bam} \\ + ${sv_file} \\ + ${mod_file} +- ++ ${cram_end} + if [ -f "${prefix}.out" ]; then + mv ${prefix}.out ${prefix}.log + fi +@@ -47,7 +63,7 @@ + stub: + def args = task.ext.args ?: '' + def prefix = task.ext.prefix ?: "${meta.id}" +- def suffix = args.contains('--cram') ? "cram" : "bam" ++ def suffix = args.contains('--cram') || task.ext.cram ? "cram" : "bam" + def log = args.contains('--log') ? "touch ${prefix}.log" : '' + """ + touch ${prefix}.${suffix} + +'modules/nf-core/longphase/haplotag/tests/nextflow.config' is unchanged +'modules/nf-core/longphase/haplotag/tests/main.nf.test' is unchanged +'modules/nf-core/longphase/haplotag/tests/main.nf.test.snap' is unchanged +************************************************************ diff --git a/modules/nf-core/longphase/haplotag/main.nf b/modules/nf-core/longphase/haplotag/main.nf index 7eb84669..55862c98 100644 --- a/modules/nf-core/longphase/haplotag/main.nf +++ b/modules/nf-core/longphase/haplotag/main.nf @@ -23,11 +23,27 @@ process LONGPHASE_HAPLOTAG { script: def args = task.ext.args ?: '' + def args2 = task.ext.args2 ?: '' def prefix = task.ext.prefix ?: "${meta.id}" def sv_file = svs ? "--sv-file ${svs}" : "" def mod_file = mods ? "--mod-file ${mods}" : "" + // ext.cram: longphase writes its BAM into a FIFO that samtools encodes as CRAM (options from ext.args2, + // e.g. embed_ref=1, which longphase's own --cram cannot set); no intermediate BAM touches the disk + def cram_out = task.ext.cram ?: false + def cram_start = cram_out ? """ + mkfifo ${prefix}.bam + samtools view --threads ${task.cpus} -C -T ${fasta} ${args2} -o ${prefix}.cram ${prefix}.bam & + conv=\$! + trap 'kill \$conv 2>/dev/null || true' EXIT + """ : '' + def cram_end = cram_out ? """ + wait \$conv + trap - EXIT + rm ${prefix}.bam + """ : '' """ + ${cram_start} longphase \\ haplotag \\ $args \\ @@ -38,7 +54,7 @@ process LONGPHASE_HAPLOTAG { --bam ${bam} \\ ${sv_file} \\ ${mod_file} - + ${cram_end} if [ -f "${prefix}.out" ]; then mv ${prefix}.out ${prefix}.log fi @@ -47,7 +63,7 @@ process LONGPHASE_HAPLOTAG { stub: def args = task.ext.args ?: '' def prefix = task.ext.prefix ?: "${meta.id}" - def suffix = args.contains('--cram') ? "cram" : "bam" + def suffix = args.contains('--cram') || task.ext.cram ? "cram" : "bam" def log = args.contains('--log') ? "touch ${prefix}.log" : '' """ touch ${prefix}.${suffix} diff --git a/modules/nf-core/minimap2/align/main.nf b/modules/nf-core/minimap2/align/main.nf index df55071a..d16a0262 100644 --- a/modules/nf-core/minimap2/align/main.nf +++ b/modules/nf-core/minimap2/align/main.nf @@ -19,7 +19,8 @@ process MINIMAP2_ALIGN { output: tuple val(meta), path("*.paf") , optional: true, emit: paf tuple val(meta), path("*.bam") , optional: true, emit: bam - tuple val(meta), path("*.bam.${bam_index_extension}"), optional: true, emit: index + tuple val(meta), path("*.cram") , optional: true, emit: cram + tuple val(meta), path("*.{bam,cram}.${bam_index_extension}"), optional: true, emit: index tuple val("${task.process}"), val("minimap2"), eval("minimap2 --version"), topic: versions, emit: versions_minimap2 when: @@ -31,8 +32,11 @@ process MINIMAP2_ALIGN { def args3 = task.ext.args3 ?: '' def args4 = task.ext.args4 ?: '' def prefix = task.ext.prefix ?: "${meta.id}" - def bam_index = bam_index_extension ? "${prefix}.bam##idx##${prefix}.bam.${bam_index_extension} --write-index" : "${prefix}.bam" - def bam_output = bam_format ? "-a | samtools sort -@ ${task.cpus-1} -o ${bam_index} ${args2}" : "-o ${prefix}.paf" + // bam_format 'cram' writes a reference-compressed CRAM instead of a BAM + def out_ext = bam_format == 'cram' ? 'cram' : 'bam' + def cram_opts = bam_format == 'cram' ? "-O cram --reference ${reference}" : '' + def bam_index = bam_index_extension ? "${prefix}.${out_ext}##idx##${prefix}.${out_ext}.${bam_index_extension} --write-index" : "${prefix}.${out_ext}" + def bam_output = bam_format ? "-a | samtools sort -@ ${task.cpus-1} ${cram_opts} -o ${bam_index} ${args2}" : "-o ${prefix}.paf" def cigar_paf = cigar_paf_format && !bam_format ? "-c" : '' def set_cigar_bam = cigar_bam && bam_format ? "-L" : '' def bam_input = "${reads.extension}".matches('sam|bam|cram') @@ -53,8 +57,9 @@ process MINIMAP2_ALIGN { stub: def prefix = task.ext.prefix ?: "${meta.id}" - def output_file = bam_format ? "${prefix}.bam" : "${prefix}.paf" - def bam_index = bam_index_extension ? "touch ${prefix}.bam.${bam_index_extension}" : "" + def out_ext = bam_format == 'cram' ? 'cram' : 'bam' + def output_file = bam_format ? "${prefix}.${out_ext}" : "${prefix}.paf" + def bam_index = bam_index_extension ? "touch ${prefix}.${out_ext}.${bam_index_extension}" : "" def bam_input = "${reads.extension}".matches('sam|bam|cram') if(bam_input && !reference) { error("Error: minimap2/align BAM input mode requires reference!") diff --git a/modules/nf-core/minimap2/align/minimap2-align.diff b/modules/nf-core/minimap2/align/minimap2-align.diff index 967bb654..feef5f4b 100644 --- a/modules/nf-core/minimap2/align/minimap2-align.diff +++ b/modules/nf-core/minimap2/align/minimap2-align.diff @@ -1,4 +1,5 @@ Changes in component 'nf-core/minimap2/align' +'modules/nf-core/minimap2/align/environment.yml' is unchanged 'modules/nf-core/minimap2/align/meta.yml' is unchanged Changes in 'minimap2/align/main.nf': --- modules/nf-core/minimap2/align/main.nf @@ -11,8 +12,43 @@ Changes in 'minimap2/align/main.nf': // Note: the versions here need to match the versions used in the mulled container below and minimap2/index conda "${moduleDir}/environment.yml" +@@ -19,7 +19,8 @@ + output: + tuple val(meta), path("*.paf") , optional: true, emit: paf + tuple val(meta), path("*.bam") , optional: true, emit: bam +- tuple val(meta), path("*.bam.${bam_index_extension}"), optional: true, emit: index ++ tuple val(meta), path("*.cram") , optional: true, emit: cram ++ tuple val(meta), path("*.{bam,cram}.${bam_index_extension}"), optional: true, emit: index + tuple val("${task.process}"), val("minimap2"), eval("minimap2 --version"), topic: versions, emit: versions_minimap2 + + when: +@@ -31,8 +32,11 @@ + def args3 = task.ext.args3 ?: '' + def args4 = task.ext.args4 ?: '' + def prefix = task.ext.prefix ?: "${meta.id}" +- def bam_index = bam_index_extension ? "${prefix}.bam##idx##${prefix}.bam.${bam_index_extension} --write-index" : "${prefix}.bam" +- def bam_output = bam_format ? "-a | samtools sort -@ ${task.cpus-1} -o ${bam_index} ${args2}" : "-o ${prefix}.paf" ++ // bam_format 'cram' writes a reference-compressed CRAM instead of a BAM ++ def out_ext = bam_format == 'cram' ? 'cram' : 'bam' ++ def cram_opts = bam_format == 'cram' ? "-O cram --reference ${reference}" : '' ++ def bam_index = bam_index_extension ? "${prefix}.${out_ext}##idx##${prefix}.${out_ext}.${bam_index_extension} --write-index" : "${prefix}.${out_ext}" ++ def bam_output = bam_format ? "-a | samtools sort -@ ${task.cpus-1} ${cram_opts} -o ${bam_index} ${args2}" : "-o ${prefix}.paf" + def cigar_paf = cigar_paf_format && !bam_format ? "-c" : '' + def set_cigar_bam = cigar_bam && bam_format ? "-L" : '' + def bam_input = "${reads.extension}".matches('sam|bam|cram') +@@ -53,8 +57,9 @@ + + stub: + def prefix = task.ext.prefix ?: "${meta.id}" +- def output_file = bam_format ? "${prefix}.bam" : "${prefix}.paf" +- def bam_index = bam_index_extension ? "touch ${prefix}.bam.${bam_index_extension}" : "" ++ def out_ext = bam_format == 'cram' ? 'cram' : 'bam' ++ def output_file = bam_format ? "${prefix}.${out_ext}" : "${prefix}.paf" ++ def bam_index = bam_index_extension ? "touch ${prefix}.${out_ext}.${bam_index_extension}" : "" + def bam_input = "${reads.extension}".matches('sam|bam|cram') + if(bam_input && !reference) { + error("Error: minimap2/align BAM input mode requires reference!") -'modules/nf-core/minimap2/align/environment.yml' is unchanged 'modules/nf-core/minimap2/align/tests/main.nf.test' is unchanged 'modules/nf-core/minimap2/align/tests/main.nf.test.snap' is unchanged ************************************************************ diff --git a/nextflow.config b/nextflow.config index 628b5ca3..79d26527 100644 --- a/nextflow.config +++ b/nextflow.config @@ -102,6 +102,7 @@ params { minimap2_ont_model = null minimap2_pb_model = null save_secondary_alignment = true + aligned_format = 'cram' // Fibertools options autocorrelation = null diff --git a/nextflow_schema.json b/nextflow_schema.json index 47356879..079591f4 100644 --- a/nextflow_schema.json +++ b/nextflow_schema.json @@ -191,6 +191,14 @@ "type": "boolean", "default": true, "description": "Whether to save secondary alignments" + }, + "aligned_format": { + "type": "string", + "default": "cram", + "enum": ["cram", "bam"], + "description": "File format of the aligned and haplotagged reads, in the work directory and in `bamfiles/`.", + "help_text": "`cram` writes CRAM 3.0 with the reference embedded: 33-69 % smaller than BAM on long reads, readable without the FASTA, and decoded by every tool in the pipeline. `bam` writes BAM with `.bai` indexes, for downstream tools that cannot read CRAM.", + "fa_icon": "fas fa-file-archive" } } }, diff --git a/nf-test.config b/nf-test.config index d1220b10..7cbb6dd1 100644 --- a/nf-test.config +++ b/nf-test.config @@ -34,7 +34,7 @@ config { // load the necessary plugins plugins { load "nft-utils@0.0.3" - // bam() assertions for sample4's merged BAM (see tests/.nftignore) + // bam() assertions for sample4's merged CRAM (see tests/.nftignore) load "nft-bam@0.7.0" } } diff --git a/subworkflows/local/phasing_haplotyping.nf b/subworkflows/local/phasing_haplotyping.nf index e80561c8..868db60f 100644 --- a/subworkflows/local/phasing_haplotyping.nf +++ b/subworkflows/local/phasing_haplotyping.nf @@ -392,13 +392,13 @@ workflow PHASING_HAPLOTYPING { // // MODULE: SAMTOOLS_INDEX (label: process_medium) // Input: [meta, bam] -- haplotagged BAM - // Output: .bai -- [meta, bai] + // Output: .bai / .crai -- [meta, index] -- whichever matches the haplotag output format // SAMTOOLS_INDEX ( tumor_normal_hapbams_ch ) tumor_normal_hapbams_ch - .join(SAMTOOLS_INDEX.out.bai) + .join(SAMTOOLS_INDEX.out.bai.mix(SAMTOOLS_INDEX.out.crai)) .set{ tumor_normal_hapbams_ch } // tumor_normal_hapbams_ch (final): [meta, bam, bai] -- haplotagged BAM with index diff --git a/tests/.nftignore b/tests/.nftignore index dfc63dec..aea9eed9 100644 --- a/tests/.nftignore +++ b/tests/.nftignore @@ -31,6 +31,6 @@ pipeline_info/*.{html,json,txt,yml} */signatures/matrices/** */signatures/assignment/** */report/*.html -# samtools merge gives sample4's colliding @PG IDs a random suffix, so this BAM's md5 differs every run (reads asserted in tests/clair_only.nf.test) -sample4/bamfiles/sample4_tumor.bam -sample4/bamfiles/sample4_tumor.bam.bai +# samtools view --threads lays CRAM containers out differently every run, so the CRAM/CRAI md5s never repeat (reads covered by the flagstat/idxstats/stats md5s; sample4's merged reads asserted in tests/clair_only.nf.test) +*/bamfiles/*.cram +*/bamfiles/*.cram.crai diff --git a/tests/clair_only.nf.test b/tests/clair_only.nf.test index 13f6d77c..20395500 100644 --- a/tests/clair_only.nf.test +++ b/tests/clair_only.nf.test @@ -62,19 +62,19 @@ nextflow_pipeline { } }, - // ── BAM files exist ────────────────────────────────────────── + // ── CRAM files exist ───────────────────────────────────────── { - // Paired samples have both tumor + normal BAMs + // Paired samples have both tumor + normal CRAMs ['sample1', 'sample2'].each { s -> - assert file("$launchDir/output/${s}/bamfiles/${s}_tumor.bam").exists() - assert file("$launchDir/output/${s}/bamfiles/${s}_tumor.bam.bai").exists() - assert file("$launchDir/output/${s}/bamfiles/${s}_normal.bam").exists() - assert file("$launchDir/output/${s}/bamfiles/${s}_normal.bam.bai").exists() + assert file("$launchDir/output/${s}/bamfiles/${s}_tumor.cram").exists() + assert file("$launchDir/output/${s}/bamfiles/${s}_tumor.cram.crai").exists() + assert file("$launchDir/output/${s}/bamfiles/${s}_normal.cram").exists() + assert file("$launchDir/output/${s}/bamfiles/${s}_normal.cram.crai").exists() } - // Tumor-only samples have only tumor BAM + // Tumor-only samples have only tumor CRAM; sample4 merges two replicates, so this covers the CRAM merge ['sample3', 'sample4', 'sample5'].each { s -> - assert file("$launchDir/output/${s}/bamfiles/${s}_tumor.bam").exists() - assert file("$launchDir/output/${s}/bamfiles/${s}_tumor.bam.bai").exists() + assert file("$launchDir/output/${s}/bamfiles/${s}_tumor.cram").exists() + assert file("$launchDir/output/${s}/bamfiles/${s}_tumor.cram.crai").exists() } }, @@ -138,12 +138,13 @@ nextflow_pipeline { } }, - // ── sample4: merged BAM content ────────────────────────────── + // ── sample4: merged CRAM content ───────────────────────────── { - // sample4's merged BAM has an unstable md5 (see tests/.nftignore), so - // assert the reads instead + // sample4's merged CRAM has an unstable md5 (see tests/.nftignore), so + // assert the reads instead. htsjdk decodes it without a FASTA (embed_ref=1), + // and the value is the one the merged BAM gave under --aligned_format bam assert snapshot( - bam("$launchDir/output/sample4/bamfiles/sample4_tumor.bam", stringency: 'silent').getReadsMD5() + bam("$launchDir/output/sample4/bamfiles/sample4_tumor.cram", stringency: 'silent').getReadsMD5() ).match("sample4_merged_reads") }, diff --git a/tests/clair_only.nf.test.snap b/tests/clair_only.nf.test.snap index ec1ff735..35fcc0dc 100644 --- a/tests/clair_only.nf.test.snap +++ b/tests/clair_only.nf.test.snap @@ -3,11 +3,11 @@ "content": [ "88c8d3cf9cb49fdbc53372b2275d5f3e" ], + "timestamp": "2026-09-02T10:24:37.511974063", "meta": { "nf-test": "0.9.4", "nextflow": "26.04.3" - }, - "timestamp": "2026-09-02T10:24:37.511974063" + } }, "-profile test, clair only, extended samplesheet": { "content": [ @@ -229,10 +229,10 @@ "pipeline_info/lrsomatic_software_mqc_versions.yml", "sample1", "sample1/bamfiles", - "sample1/bamfiles/sample1_normal.bam", - "sample1/bamfiles/sample1_normal.bam.bai", - "sample1/bamfiles/sample1_tumor.bam", - "sample1/bamfiles/sample1_tumor.bam.bai", + "sample1/bamfiles/sample1_normal.cram", + "sample1/bamfiles/sample1_normal.cram.crai", + "sample1/bamfiles/sample1_tumor.cram", + "sample1/bamfiles/sample1_tumor.cram.crai", "sample1/qc", "sample1/qc/normal", "sample1/qc/normal/cramino_aln", @@ -357,10 +357,10 @@ "sample1/vep/somatic/sample1_SOMATIC_VEP.vcf.gz_summary.html", "sample2", "sample2/bamfiles", - "sample2/bamfiles/sample2_normal.bam", - "sample2/bamfiles/sample2_normal.bam.bai", - "sample2/bamfiles/sample2_tumor.bam", - "sample2/bamfiles/sample2_tumor.bam.bai", + "sample2/bamfiles/sample2_normal.cram", + "sample2/bamfiles/sample2_normal.cram.crai", + "sample2/bamfiles/sample2_tumor.cram", + "sample2/bamfiles/sample2_tumor.cram.crai", "sample2/qc", "sample2/qc/normal", "sample2/qc/normal/cramino_aln", @@ -484,8 +484,8 @@ "sample2/vep/somatic/sample2_SOMATIC_VEP.vcf.gz_summary.html", "sample3", "sample3/bamfiles", - "sample3/bamfiles/sample3_tumor.bam", - "sample3/bamfiles/sample3_tumor.bam.bai", + "sample3/bamfiles/sample3_tumor.cram", + "sample3/bamfiles/sample3_tumor.cram.crai", "sample3/qc", "sample3/qc/tumor", "sample3/qc/tumor/cramino_aln", @@ -569,8 +569,8 @@ "sample3/vep/somatic/sample3_SOMATIC_VEP.vcf.gz_summary.html", "sample4", "sample4/bamfiles", - "sample4/bamfiles/sample4_tumor.bam", - "sample4/bamfiles/sample4_tumor.bam.bai", + "sample4/bamfiles/sample4_tumor.cram", + "sample4/bamfiles/sample4_tumor.cram.crai", "sample4/qc", "sample4/qc/tumor", "sample4/qc/tumor/cramino_aln", @@ -664,8 +664,8 @@ "sample4/vep/somatic/sample4_SOMATIC_VEP.vcf.gz_summary.html", "sample5", "sample5/bamfiles", - "sample5/bamfiles/sample5_tumor.bam", - "sample5/bamfiles/sample5_tumor.bam.bai", + "sample5/bamfiles/sample5_tumor.cram", + "sample5/bamfiles/sample5_tumor.cram.crai", "sample5/qc", "sample5/qc/tumor", "sample5/qc/tumor/cramino_aln", @@ -749,90 +749,78 @@ "sample5/vep/somatic/sample5_SOMATIC_VEP.vcf.gz_summary.html" ], [ - "sample1_normal.bam:md5,772e41f7cd86c03a22afbe5ec0592a6b", - "sample1_normal.bam.bai:md5,1b501f6a11efe5d2e6f47b7f1523220b", - "sample1_tumor.bam:md5,c8315c80dc92dfb5d874aef3f5dd46fb", - "sample1_tumor.bam.bai:md5,bc35f807be4b93fc795a14d701469367", "sample1_normal.flagstat:md5,1c41ea9923945501eb7e41f83a90502d", "sample1_normal.idxstats:md5,902e503387799123ea59255e3fca172c", - "sample1_normal.stats:md5,a8b3fba9c54efbc0934d6eacc1807140", + "sample1_normal.stats:md5,1fee9dfe18bc8b1c3f62bec6d54ae0b7", "sample1_tumor.flagstat:md5,8ff32d733c62c4910bf185ef24bf27cf", "sample1_tumor.idxstats:md5,2de140e61f9e86c9c10af20dd565cc93", - "sample1_tumor.stats:md5,1c60a1d249d2e503b0678c72e851ea93", + "sample1_tumor.stats:md5,8b33272d01216f2cb0d9039d0176ee1c", "sample1_whatshap_stats.gtf:md5,eff050a68e36e778b06e0ec19435c569", "sample1_whatshap_stats.log:md5,76b73731f74fe32ef2d11f6bb0a0f71a", "sample1_whatshap_stats.tsv:md5,f566ae25b3c5a8f7e94b3d6c1b0417f8", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "breakpoints_double.csv:md5,47cb0e0bbe71abdbf4f40217dfda43f9", + "breakpoints_double.csv:md5,3690b22cd4a7da90ed2f69042db45802", "read_qual.txt:md5,78247dfa2ea336eac0e128eba5e9eef4", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "sample2_normal.bam:md5,3157bd11ba095a884c7951aafcfcfb1c", - "sample2_normal.bam.bai:md5,edebda44c4383173caea728acde4ac43", - "sample2_tumor.bam:md5,47b2c5f86e0493ba94ff72cea77eeae3", - "sample2_tumor.bam.bai:md5,abf2c290c815f54c2b3f8179f717d9bd", "sample2_normal.flagstat:md5,714d0cc0c213e2640e54a16f3d0e6e7e", "sample2_normal.idxstats:md5,72eb83bb11748dc863fef1a0a5497e4b", - "sample2_normal.stats:md5,20c47cb94f9ac739d69c57be6daf82c5", + "sample2_normal.stats:md5,b00725632868d7181d67c23b8cc1d4af", "sample2_tumor.flagstat:md5,4344a8745efef9cc2a017024218d61c6", "sample2_tumor.idxstats:md5,69467fc02c83a30084736aeea8b785fb", - "sample2_tumor.stats:md5,8635df10132c85a13f2d9878b7cf90a2", + "sample2_tumor.stats:md5,154024d3791a0e2b804ae0cf78ff740e", "sample2_whatshap_stats.gtf:md5,4d8f4393e3aebe4e945c0b8236cf3b3e", "sample2_whatshap_stats.log:md5,10bba7bae6dd99b989ece5e5dac7a8f9", "sample2_whatshap_stats.tsv:md5,bb46226e486af9026ab76e014624e903", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "breakpoints_double.csv:md5,48baac86492026a4a7947bc708c47e6e", + "breakpoints_double.csv:md5,64b72dd7f32cc27c7138830a00695f16", "read_qual.txt:md5,8b92ff7dc4536188be159b95525511cd", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "sample3_tumor.bam:md5,3c995151bdd974df5bf61d08606d54d3", - "sample3_tumor.bam.bai:md5,9d5964ef8f44127a7d9162d628e2407d", "sample3_tumor.flagstat:md5,8ff32d733c62c4910bf185ef24bf27cf", "sample3_tumor.idxstats:md5,2de140e61f9e86c9c10af20dd565cc93", - "sample3_tumor.stats:md5,ecd5ea4fee37379dd5c5ae3e89dfddda", + "sample3_tumor.stats:md5,2535ddb28ef378016f7b4cfa9042c72a", "sample3_whatshap_stats.gtf:md5,46a97067376b06b476d180709bc9e3d8", "sample3_whatshap_stats.log:md5,376254ec9c98f9ba204895e7085516ed", "sample3_whatshap_stats.tsv:md5,f7cc79156f23e884ead18e50b8434dbf", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "breakpoints_double.csv:md5,56e899f85876cee082788927d0f89c5f", + "breakpoints_double.csv:md5,6ef9c12b0311d3a4a23cab394abfd389", "read_qual.txt:md5,b918430d35354dad1d7f02f21e4cd4ed", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", "sample4_tumor.flagstat:md5,5710382ba31b23172ca19f9407f689b4", "sample4_tumor.idxstats:md5,3bf4793a1667f41f0b31d578f99e3955", - "sample4_tumor.stats:md5,b998982ea897721c529959e39b693ec6", + "sample4_tumor.stats:md5,8d6846e65e9ac5ead88ed6f1ac4ead09", "sample4_whatshap_stats.gtf:md5,9ca85642243042d3580d14a1d54b3d6a", "sample4_whatshap_stats.log:md5,b453653f5d83c4aa408862e7141f0965", "sample4_whatshap_stats.tsv:md5,8830262298440c4bd8675d1ff3f601e8", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "breakpoints_double.csv:md5,e3beaa65b40a5093a7e74cecb0b2cfd2", + "breakpoints_double.csv:md5,4f67549ba79d56fb0c8141cd06cdfa99", "read_qual.txt:md5,42eb25878cc6b823a562406db56b29d7", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "sample5_tumor.bam:md5,e3b7a4c9d391e9ddda187d7d6bcd1403", - "sample5_tumor.bam.bai:md5,d3ba11453e0eefda8d602e7b4883e82c", "sample5_tumor.flagstat:md5,8ff32d733c62c4910bf185ef24bf27cf", "sample5_tumor.idxstats:md5,2de140e61f9e86c9c10af20dd565cc93", - "sample5_tumor.stats:md5,3f1429e9b2379bf282299d71a8c5d22c", + "sample5_tumor.stats:md5,628088d29060c1aa8cd466e930d68ea4", "sample5_whatshap_stats.gtf:md5,d02a3be9b50b953b32b42e2fedabc0e5", "sample5_whatshap_stats.log:md5,a2e4ed9edca8609fc947041a9fead794", "sample5_whatshap_stats.tsv:md5,860ad30057b783e24dcb41ec581c8b68", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "breakpoints_double.csv:md5,cc06f0726375cc45f29875b5053831e9", + "breakpoints_double.csv:md5,b12ed2cb2111ae41dd44b7e92c5ed6b9", "read_qual.txt:md5,b918430d35354dad1d7f02f21e4cd4ed", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50" ] ], + "timestamp": "2026-09-28T20:10:51.882257334", "meta": { - "nf-test": "0.9.0", - "nextflow": "26.04.6" - }, - "timestamp": "2026-09-21T11:27:45.728407577" + "nf-test": "0.9.4", + "nextflow": "26.04.3" + } } } \ No newline at end of file diff --git a/tests/consensus.nf.test b/tests/consensus.nf.test index 9689f212..efdfe685 100644 --- a/tests/consensus.nf.test +++ b/tests/consensus.nf.test @@ -66,13 +66,13 @@ nextflow_pipeline { // ── BAM files ──────────────────────────────────────────────── { ['sample1', 'sample2'].each { s -> - assert file("$launchDir/output/${s}/bamfiles/${s}_tumor.bam").exists() - assert file("$launchDir/output/${s}/bamfiles/${s}_tumor.bam.bai").exists() - assert file("$launchDir/output/${s}/bamfiles/${s}_normal.bam").exists() - assert file("$launchDir/output/${s}/bamfiles/${s}_normal.bam.bai").exists() + assert file("$launchDir/output/${s}/bamfiles/${s}_tumor.cram").exists() + assert file("$launchDir/output/${s}/bamfiles/${s}_tumor.cram.crai").exists() + assert file("$launchDir/output/${s}/bamfiles/${s}_normal.cram").exists() + assert file("$launchDir/output/${s}/bamfiles/${s}_normal.cram.crai").exists() } - assert file("$launchDir/output/sample3/bamfiles/sample3_tumor.bam").exists() - assert file("$launchDir/output/sample3/bamfiles/sample3_tumor.bam.bai").exists() + assert file("$launchDir/output/sample3/bamfiles/sample3_tumor.cram").exists() + assert file("$launchDir/output/sample3/bamfiles/sample3_tumor.cram.crai").exists() }, // ── Severus ────────────────────────────────────────────────── diff --git a/tests/consensus.nf.test.snap b/tests/consensus.nf.test.snap index 6bf3a01d..1b6fddc8 100644 --- a/tests/consensus.nf.test.snap +++ b/tests/consensus.nf.test.snap @@ -249,10 +249,10 @@ "pipeline_info/lrsomatic_software_mqc_versions.yml", "sample1", "sample1/bamfiles", - "sample1/bamfiles/sample1_normal.bam", - "sample1/bamfiles/sample1_normal.bam.bai", - "sample1/bamfiles/sample1_tumor.bam", - "sample1/bamfiles/sample1_tumor.bam.bai", + "sample1/bamfiles/sample1_normal.cram", + "sample1/bamfiles/sample1_normal.cram.crai", + "sample1/bamfiles/sample1_tumor.cram", + "sample1/bamfiles/sample1_tumor.cram.crai", "sample1/qc", "sample1/qc/normal", "sample1/qc/normal/cramino_aln", @@ -383,10 +383,10 @@ "sample1/vep/somatic/sample1_SOMATIC_VEP.vcf.gz_summary.html", "sample2", "sample2/bamfiles", - "sample2/bamfiles/sample2_normal.bam", - "sample2/bamfiles/sample2_normal.bam.bai", - "sample2/bamfiles/sample2_tumor.bam", - "sample2/bamfiles/sample2_tumor.bam.bai", + "sample2/bamfiles/sample2_normal.cram", + "sample2/bamfiles/sample2_normal.cram.crai", + "sample2/bamfiles/sample2_tumor.cram", + "sample2/bamfiles/sample2_tumor.cram.crai", "sample2/qc", "sample2/qc/normal", "sample2/qc/normal/cramino_aln", @@ -516,8 +516,8 @@ "sample2/vep/somatic/sample2_SOMATIC_VEP.vcf.gz_summary.html", "sample3", "sample3/bamfiles", - "sample3/bamfiles/sample3_tumor.bam", - "sample3/bamfiles/sample3_tumor.bam.bai", + "sample3/bamfiles/sample3_tumor.cram", + "sample3/bamfiles/sample3_tumor.cram.crai", "sample3/qc", "sample3/qc/tumor", "sample3/qc/tumor/cramino_aln", @@ -607,64 +607,54 @@ "sample3/vep/somatic/sample3_SOMATIC_VEP.vcf.gz_summary.html" ], [ - "sample1_normal.bam:md5,93dd8ac8b67eb4eb4bf27e09c8f5f99b", - "sample1_normal.bam.bai:md5,75402ef1cc35229cc131155d9ec973e0", - "sample1_tumor.bam:md5,69cba03cad51bcc1d1ee8c48da042527", - "sample1_tumor.bam.bai:md5,5d633ed05021ad81ce24b1f18cbf38b4", "sample1_normal.flagstat:md5,1c41ea9923945501eb7e41f83a90502d", "sample1_normal.idxstats:md5,902e503387799123ea59255e3fca172c", - "sample1_normal.stats:md5,a8b3fba9c54efbc0934d6eacc1807140", + "sample1_normal.stats:md5,1fee9dfe18bc8b1c3f62bec6d54ae0b7", "sample1_tumor.flagstat:md5,8ff32d733c62c4910bf185ef24bf27cf", "sample1_tumor.idxstats:md5,2de140e61f9e86c9c10af20dd565cc93", - "sample1_tumor.stats:md5,1c60a1d249d2e503b0678c72e851ea93", + "sample1_tumor.stats:md5,8b33272d01216f2cb0d9039d0176ee1c", "sample1_whatshap_stats.gtf:md5,9f09f9ad1a788384cb8e46a933f77b3b", "sample1_whatshap_stats.log:md5,20135b4e9965a31d3f9bb0df7d2cec90", "sample1_whatshap_stats.tsv:md5,264d2d76a9b8d34ea4933aee325ce36e", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "breakpoints_double.csv:md5,47cb0e0bbe71abdbf4f40217dfda43f9", + "breakpoints_double.csv:md5,3690b22cd4a7da90ed2f69042db45802", "read_qual.txt:md5,78247dfa2ea336eac0e128eba5e9eef4", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "sample2_normal.bam:md5,f2ba30d007c521d479c6158e1e22367a", - "sample2_normal.bam.bai:md5,c3096f52115ec1e24c46fedc41f1f3d3", - "sample2_tumor.bam:md5,1c0287d24fa5b25b86e48024f2f55031", - "sample2_tumor.bam.bai:md5,62849cea5a005e3d8dbe8f9edcefaf60", "sample2_normal.flagstat:md5,714d0cc0c213e2640e54a16f3d0e6e7e", "sample2_normal.idxstats:md5,72eb83bb11748dc863fef1a0a5497e4b", - "sample2_normal.stats:md5,20c47cb94f9ac739d69c57be6daf82c5", + "sample2_normal.stats:md5,b00725632868d7181d67c23b8cc1d4af", "sample2_tumor.flagstat:md5,4344a8745efef9cc2a017024218d61c6", "sample2_tumor.idxstats:md5,69467fc02c83a30084736aeea8b785fb", - "sample2_tumor.stats:md5,8635df10132c85a13f2d9878b7cf90a2", + "sample2_tumor.stats:md5,154024d3791a0e2b804ae0cf78ff740e", "sample2_whatshap_stats.gtf:md5,f15fb43f0af73d02fc73b66fdc12d5d8", "sample2_whatshap_stats.log:md5,ca87088fc2f11665eca3fb9c80489085", "sample2_whatshap_stats.tsv:md5,ca53f81e39bf5d46aa4f604216add1f6", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "breakpoints_double.csv:md5,48baac86492026a4a7947bc708c47e6e", + "breakpoints_double.csv:md5,64b72dd7f32cc27c7138830a00695f16", "read_qual.txt:md5,8b92ff7dc4536188be159b95525511cd", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "sample3_tumor.bam:md5,3c995151bdd974df5bf61d08606d54d3", - "sample3_tumor.bam.bai:md5,9d5964ef8f44127a7d9162d628e2407d", "sample3_tumor.flagstat:md5,8ff32d733c62c4910bf185ef24bf27cf", "sample3_tumor.idxstats:md5,2de140e61f9e86c9c10af20dd565cc93", - "sample3_tumor.stats:md5,ecd5ea4fee37379dd5c5ae3e89dfddda", + "sample3_tumor.stats:md5,2535ddb28ef378016f7b4cfa9042c72a", "sample3_whatshap_stats.gtf:md5,46a97067376b06b476d180709bc9e3d8", "sample3_whatshap_stats.log:md5,376254ec9c98f9ba204895e7085516ed", "sample3_whatshap_stats.tsv:md5,f7cc79156f23e884ead18e50b8434dbf", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "breakpoints_double.csv:md5,56e899f85876cee082788927d0f89c5f", + "breakpoints_double.csv:md5,6ef9c12b0311d3a4a23cab394abfd389", "read_qual.txt:md5,b918430d35354dad1d7f02f21e4cd4ed", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50" ] ], + "timestamp": "2026-09-25T17:03:30.854619726", "meta": { - "nf-test": "0.9.0", - "nextflow": "26.04.6" - }, - "timestamp": "2026-09-21T11:40:36.642665738" + "nf-test": "0.9.4", + "nextflow": "26.04.3" + } } } \ No newline at end of file diff --git a/tests/deep_only.nf.test b/tests/deep_only.nf.test index 5da64375..ffa6a6c7 100644 --- a/tests/deep_only.nf.test +++ b/tests/deep_only.nf.test @@ -12,6 +12,9 @@ nextflow_pipeline { outdir = "$outputDir" germline_var_keep = 'deepvariant' somatic_var_keep = 'deepsomatic' + // The one test on BAM (the others run the CRAM default); DeepVariant/DeepSomatic make_examples are + // 13-14x slower on CRAM on the thin chr19 test data only, so this also keeps the test short + aligned_format = 'bam' } } diff --git a/tests/deep_only.nf.test.snap b/tests/deep_only.nf.test.snap index 7887aaca..611d5881 100644 --- a/tests/deep_only.nf.test.snap +++ b/tests/deep_only.nf.test.snap @@ -606,10 +606,10 @@ "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50" ] ], + "timestamp": "2026-09-28T18:26:06.724830269", "meta": { - "nf-test": "0.9.0", - "nextflow": "26.04.6" - }, - "timestamp": "2026-09-21T11:46:42.562085661" + "nf-test": "0.9.4", + "nextflow": "26.04.3" + } } } \ No newline at end of file diff --git a/tests/default.nf.test b/tests/default.nf.test index 7c473eba..fb39bfe9 100644 --- a/tests/default.nf.test +++ b/tests/default.nf.test @@ -39,10 +39,10 @@ nextflow_pipeline { assert file("$launchDir/output/sample2/variants/clairs/snvs.vcf.gz.tbi").exists() assert file("$launchDir/output/sample2/variants/severus/somatic_SVs/severus_somatic.vcf.gz").exists() assert file("$launchDir/output/sample2/variants/savana/sample2.classified.somatic.vcf").exists() - assert file("$launchDir/output/sample1/bamfiles/sample1_normal.bam").exists() - assert file("$launchDir/output/sample1/bamfiles/sample1_tumor.bam").exists() - assert file("$launchDir/output/sample1/bamfiles/sample1_normal.bam.bai").exists() - assert file("$launchDir/output/sample1/bamfiles/sample1_tumor.bam.bai").exists() + assert file("$launchDir/output/sample1/bamfiles/sample1_normal.cram").exists() + assert file("$launchDir/output/sample1/bamfiles/sample1_tumor.cram").exists() + assert file("$launchDir/output/sample1/bamfiles/sample1_normal.cram.crai").exists() + assert file("$launchDir/output/sample1/bamfiles/sample1_tumor.cram.crai").exists() assert file("$launchDir/output/sample3/variants/clairsto/indel.vcf.gz").exists() assert file("$launchDir/output/sample3/variants/clairsto/snv.vcf.gz").exists() assert file("$launchDir/output/sample3/variants/clairsto/somatic.vcf.gz").exists() diff --git a/tests/default.nf.test.snap b/tests/default.nf.test.snap index 22e72e6f..a890840b 100644 --- a/tests/default.nf.test.snap +++ b/tests/default.nf.test.snap @@ -213,10 +213,10 @@ "pipeline_info/lrsomatic_software_mqc_versions.yml", "sample1", "sample1/bamfiles", - "sample1/bamfiles/sample1_normal.bam", - "sample1/bamfiles/sample1_normal.bam.bai", - "sample1/bamfiles/sample1_tumor.bam", - "sample1/bamfiles/sample1_tumor.bam.bai", + "sample1/bamfiles/sample1_normal.cram", + "sample1/bamfiles/sample1_normal.cram.crai", + "sample1/bamfiles/sample1_tumor.cram", + "sample1/bamfiles/sample1_tumor.cram.crai", "sample1/qc", "sample1/qc/normal", "sample1/qc/normal/cramino_aln", @@ -341,10 +341,10 @@ "sample1/vep/somatic/sample1_SOMATIC_VEP.vcf.gz_summary.html", "sample2", "sample2/bamfiles", - "sample2/bamfiles/sample2_normal.bam", - "sample2/bamfiles/sample2_normal.bam.bai", - "sample2/bamfiles/sample2_tumor.bam", - "sample2/bamfiles/sample2_tumor.bam.bai", + "sample2/bamfiles/sample2_normal.cram", + "sample2/bamfiles/sample2_normal.cram.crai", + "sample2/bamfiles/sample2_tumor.cram", + "sample2/bamfiles/sample2_tumor.cram.crai", "sample2/qc", "sample2/qc/normal", "sample2/qc/normal/cramino_aln", @@ -468,8 +468,8 @@ "sample2/vep/somatic/sample2_SOMATIC_VEP.vcf.gz_summary.html", "sample3", "sample3/bamfiles", - "sample3/bamfiles/sample3_tumor.bam", - "sample3/bamfiles/sample3_tumor.bam.bai", + "sample3/bamfiles/sample3_tumor.cram", + "sample3/bamfiles/sample3_tumor.cram.crai", "sample3/qc", "sample3/qc/tumor", "sample3/qc/tumor/cramino_aln", @@ -553,64 +553,54 @@ "sample3/vep/somatic/sample3_SOMATIC_VEP.vcf.gz_summary.html" ], [ - "sample1_normal.bam:md5,772e41f7cd86c03a22afbe5ec0592a6b", - "sample1_normal.bam.bai:md5,1b501f6a11efe5d2e6f47b7f1523220b", - "sample1_tumor.bam:md5,c8315c80dc92dfb5d874aef3f5dd46fb", - "sample1_tumor.bam.bai:md5,bc35f807be4b93fc795a14d701469367", "sample1_normal.flagstat:md5,1c41ea9923945501eb7e41f83a90502d", "sample1_normal.idxstats:md5,902e503387799123ea59255e3fca172c", - "sample1_normal.stats:md5,a8b3fba9c54efbc0934d6eacc1807140", + "sample1_normal.stats:md5,1fee9dfe18bc8b1c3f62bec6d54ae0b7", "sample1_tumor.flagstat:md5,8ff32d733c62c4910bf185ef24bf27cf", "sample1_tumor.idxstats:md5,2de140e61f9e86c9c10af20dd565cc93", - "sample1_tumor.stats:md5,1c60a1d249d2e503b0678c72e851ea93", + "sample1_tumor.stats:md5,8b33272d01216f2cb0d9039d0176ee1c", "sample1_whatshap_stats.gtf:md5,eff050a68e36e778b06e0ec19435c569", "sample1_whatshap_stats.log:md5,76b73731f74fe32ef2d11f6bb0a0f71a", "sample1_whatshap_stats.tsv:md5,f566ae25b3c5a8f7e94b3d6c1b0417f8", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "breakpoints_double.csv:md5,47cb0e0bbe71abdbf4f40217dfda43f9", + "breakpoints_double.csv:md5,3690b22cd4a7da90ed2f69042db45802", "read_qual.txt:md5,78247dfa2ea336eac0e128eba5e9eef4", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "sample2_normal.bam:md5,3157bd11ba095a884c7951aafcfcfb1c", - "sample2_normal.bam.bai:md5,edebda44c4383173caea728acde4ac43", - "sample2_tumor.bam:md5,47b2c5f86e0493ba94ff72cea77eeae3", - "sample2_tumor.bam.bai:md5,abf2c290c815f54c2b3f8179f717d9bd", "sample2_normal.flagstat:md5,714d0cc0c213e2640e54a16f3d0e6e7e", "sample2_normal.idxstats:md5,72eb83bb11748dc863fef1a0a5497e4b", - "sample2_normal.stats:md5,20c47cb94f9ac739d69c57be6daf82c5", + "sample2_normal.stats:md5,b00725632868d7181d67c23b8cc1d4af", "sample2_tumor.flagstat:md5,4344a8745efef9cc2a017024218d61c6", "sample2_tumor.idxstats:md5,69467fc02c83a30084736aeea8b785fb", - "sample2_tumor.stats:md5,8635df10132c85a13f2d9878b7cf90a2", + "sample2_tumor.stats:md5,154024d3791a0e2b804ae0cf78ff740e", "sample2_whatshap_stats.gtf:md5,4d8f4393e3aebe4e945c0b8236cf3b3e", "sample2_whatshap_stats.log:md5,10bba7bae6dd99b989ece5e5dac7a8f9", "sample2_whatshap_stats.tsv:md5,bb46226e486af9026ab76e014624e903", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "breakpoints_double.csv:md5,48baac86492026a4a7947bc708c47e6e", + "breakpoints_double.csv:md5,64b72dd7f32cc27c7138830a00695f16", "read_qual.txt:md5,8b92ff7dc4536188be159b95525511cd", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "sample3_tumor.bam:md5,3c995151bdd974df5bf61d08606d54d3", - "sample3_tumor.bam.bai:md5,9d5964ef8f44127a7d9162d628e2407d", "sample3_tumor.flagstat:md5,8ff32d733c62c4910bf185ef24bf27cf", "sample3_tumor.idxstats:md5,2de140e61f9e86c9c10af20dd565cc93", - "sample3_tumor.stats:md5,ecd5ea4fee37379dd5c5ae3e89dfddda", + "sample3_tumor.stats:md5,2535ddb28ef378016f7b4cfa9042c72a", "sample3_whatshap_stats.gtf:md5,46a97067376b06b476d180709bc9e3d8", "sample3_whatshap_stats.log:md5,376254ec9c98f9ba204895e7085516ed", "sample3_whatshap_stats.tsv:md5,f7cc79156f23e884ead18e50b8434dbf", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "breakpoints_double.csv:md5,56e899f85876cee082788927d0f89c5f", + "breakpoints_double.csv:md5,6ef9c12b0311d3a4a23cab394abfd389", "read_qual.txt:md5,b918430d35354dad1d7f02f21e4cd4ed", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50" ] ], + "timestamp": "2026-09-25T16:06:17.446805978", "meta": { - "nf-test": "0.9.0", - "nextflow": "26.04.6" - }, - "timestamp": "2026-09-21T11:10:56.589241561" + "nf-test": "0.9.4", + "nextflow": "26.04.3" + } } } \ No newline at end of file diff --git a/tests/union.nf.test b/tests/union.nf.test index 2fe8925b..a5627b9e 100644 --- a/tests/union.nf.test +++ b/tests/union.nf.test @@ -66,13 +66,13 @@ nextflow_pipeline { // ── BAM files ──────────────────────────────────────────────── { ['sample1', 'sample2'].each { s -> - assert file("$launchDir/output/${s}/bamfiles/${s}_tumor.bam").exists() - assert file("$launchDir/output/${s}/bamfiles/${s}_tumor.bam.bai").exists() - assert file("$launchDir/output/${s}/bamfiles/${s}_normal.bam").exists() - assert file("$launchDir/output/${s}/bamfiles/${s}_normal.bam.bai").exists() + assert file("$launchDir/output/${s}/bamfiles/${s}_tumor.cram").exists() + assert file("$launchDir/output/${s}/bamfiles/${s}_tumor.cram.crai").exists() + assert file("$launchDir/output/${s}/bamfiles/${s}_normal.cram").exists() + assert file("$launchDir/output/${s}/bamfiles/${s}_normal.cram.crai").exists() } - assert file("$launchDir/output/sample3/bamfiles/sample3_tumor.bam").exists() - assert file("$launchDir/output/sample3/bamfiles/sample3_tumor.bam.bai").exists() + assert file("$launchDir/output/sample3/bamfiles/sample3_tumor.cram").exists() + assert file("$launchDir/output/sample3/bamfiles/sample3_tumor.cram.crai").exists() }, // ── Severus ────────────────────────────────────────────────── diff --git a/tests/union.nf.test.snap b/tests/union.nf.test.snap index 654a82d7..a74edaad 100644 --- a/tests/union.nf.test.snap +++ b/tests/union.nf.test.snap @@ -249,10 +249,10 @@ "pipeline_info/lrsomatic_software_mqc_versions.yml", "sample1", "sample1/bamfiles", - "sample1/bamfiles/sample1_normal.bam", - "sample1/bamfiles/sample1_normal.bam.bai", - "sample1/bamfiles/sample1_tumor.bam", - "sample1/bamfiles/sample1_tumor.bam.bai", + "sample1/bamfiles/sample1_normal.cram", + "sample1/bamfiles/sample1_normal.cram.crai", + "sample1/bamfiles/sample1_tumor.cram", + "sample1/bamfiles/sample1_tumor.cram.crai", "sample1/qc", "sample1/qc/normal", "sample1/qc/normal/cramino_aln", @@ -383,10 +383,10 @@ "sample1/vep/somatic/sample1_SOMATIC_VEP.vcf.gz_summary.html", "sample2", "sample2/bamfiles", - "sample2/bamfiles/sample2_normal.bam", - "sample2/bamfiles/sample2_normal.bam.bai", - "sample2/bamfiles/sample2_tumor.bam", - "sample2/bamfiles/sample2_tumor.bam.bai", + "sample2/bamfiles/sample2_normal.cram", + "sample2/bamfiles/sample2_normal.cram.crai", + "sample2/bamfiles/sample2_tumor.cram", + "sample2/bamfiles/sample2_tumor.cram.crai", "sample2/qc", "sample2/qc/normal", "sample2/qc/normal/cramino_aln", @@ -516,8 +516,8 @@ "sample2/vep/somatic/sample2_SOMATIC_VEP.vcf.gz_summary.html", "sample3", "sample3/bamfiles", - "sample3/bamfiles/sample3_tumor.bam", - "sample3/bamfiles/sample3_tumor.bam.bai", + "sample3/bamfiles/sample3_tumor.cram", + "sample3/bamfiles/sample3_tumor.cram.crai", "sample3/qc", "sample3/qc/tumor", "sample3/qc/tumor/cramino_aln", @@ -607,64 +607,54 @@ "sample3/vep/somatic/sample3_SOMATIC_VEP.vcf.gz_summary.html" ], [ - "sample1_normal.bam:md5,cbfc940a38c74cbe8435c18b9da5dd32", - "sample1_normal.bam.bai:md5,c1498328929d45b2898fa2265b0d617c", - "sample1_tumor.bam:md5,b78866edf991393806d37505d16f7e3d", - "sample1_tumor.bam.bai:md5,f613de14ab19fc3a85403661a4f6188c", "sample1_normal.flagstat:md5,1c41ea9923945501eb7e41f83a90502d", "sample1_normal.idxstats:md5,902e503387799123ea59255e3fca172c", - "sample1_normal.stats:md5,a8b3fba9c54efbc0934d6eacc1807140", + "sample1_normal.stats:md5,1fee9dfe18bc8b1c3f62bec6d54ae0b7", "sample1_tumor.flagstat:md5,8ff32d733c62c4910bf185ef24bf27cf", "sample1_tumor.idxstats:md5,2de140e61f9e86c9c10af20dd565cc93", - "sample1_tumor.stats:md5,1c60a1d249d2e503b0678c72e851ea93", + "sample1_tumor.stats:md5,8b33272d01216f2cb0d9039d0176ee1c", "sample1_whatshap_stats.gtf:md5,9ae556e13516dd47d4108acf2104bddb", "sample1_whatshap_stats.log:md5,eaddcf6a1666d4a3c1ad3316dac24139", "sample1_whatshap_stats.tsv:md5,c2773e011c2781160fd9a7741b10546b", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "breakpoints_double.csv:md5,47cb0e0bbe71abdbf4f40217dfda43f9", + "breakpoints_double.csv:md5,3690b22cd4a7da90ed2f69042db45802", "read_qual.txt:md5,78247dfa2ea336eac0e128eba5e9eef4", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "sample2_normal.bam:md5,f2ba30d007c521d479c6158e1e22367a", - "sample2_normal.bam.bai:md5,c3096f52115ec1e24c46fedc41f1f3d3", - "sample2_tumor.bam:md5,1c0287d24fa5b25b86e48024f2f55031", - "sample2_tumor.bam.bai:md5,62849cea5a005e3d8dbe8f9edcefaf60", "sample2_normal.flagstat:md5,714d0cc0c213e2640e54a16f3d0e6e7e", "sample2_normal.idxstats:md5,72eb83bb11748dc863fef1a0a5497e4b", - "sample2_normal.stats:md5,20c47cb94f9ac739d69c57be6daf82c5", + "sample2_normal.stats:md5,b00725632868d7181d67c23b8cc1d4af", "sample2_tumor.flagstat:md5,4344a8745efef9cc2a017024218d61c6", "sample2_tumor.idxstats:md5,69467fc02c83a30084736aeea8b785fb", - "sample2_tumor.stats:md5,8635df10132c85a13f2d9878b7cf90a2", + "sample2_tumor.stats:md5,154024d3791a0e2b804ae0cf78ff740e", "sample2_whatshap_stats.gtf:md5,f15fb43f0af73d02fc73b66fdc12d5d8", "sample2_whatshap_stats.log:md5,a6767b3490cafdcbaf3b7114644028de", "sample2_whatshap_stats.tsv:md5,570796e5e291229e8872733425e0b133", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "breakpoints_double.csv:md5,48baac86492026a4a7947bc708c47e6e", + "breakpoints_double.csv:md5,64b72dd7f32cc27c7138830a00695f16", "read_qual.txt:md5,8b92ff7dc4536188be159b95525511cd", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "sample3_tumor.bam:md5,116b6944da4aa833a8d21c46b5f5ecfe", - "sample3_tumor.bam.bai:md5,ea9eca53bbaba26d40b791a2ea1aadf6", "sample3_tumor.flagstat:md5,8ff32d733c62c4910bf185ef24bf27cf", "sample3_tumor.idxstats:md5,2de140e61f9e86c9c10af20dd565cc93", - "sample3_tumor.stats:md5,ecd5ea4fee37379dd5c5ae3e89dfddda", + "sample3_tumor.stats:md5,2535ddb28ef378016f7b4cfa9042c72a", "sample3_whatshap_stats.gtf:md5,f47156e18c490ff9a4e6efd04d43acc5", "sample3_whatshap_stats.log:md5,679dcfa209888a9e69a07e4c4e4b049e", "sample3_whatshap_stats.tsv:md5,035d5aa0425ba3fc32d65268b793b424", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50", - "breakpoints_double.csv:md5,56e899f85876cee082788927d0f89c5f", + "breakpoints_double.csv:md5,6ef9c12b0311d3a4a23cab394abfd389", "read_qual.txt:md5,b918430d35354dad1d7f02f21e4cd4ed", "breakpoint_clusters.tsv:md5,d36a70de292ee130ef30da4a58bced18", "breakpoint_clusters_list.tsv:md5,0c0ce62e329f8de492487e8414c30a50" ] ], + "timestamp": "2026-09-25T17:03:32.469972501", "meta": { - "nf-test": "0.9.0", - "nextflow": "26.04.6" - }, - "timestamp": "2026-09-21T11:54:50.88115842" + "nf-test": "0.9.4", + "nextflow": "26.04.3" + } } } \ No newline at end of file diff --git a/workflows/lrsomatic.nf b/workflows/lrsomatic.nf index 37bd1c4c..1a743132 100644 --- a/workflows/lrsomatic.nf +++ b/workflows/lrsomatic.nf @@ -316,7 +316,7 @@ workflow LRSOMATIC { // Output: cramino_pre.out.arrow -- [meta, arrow_file] (feather format stats) // - CRAMINO_PRE( ch_samplesheet ) + CRAMINO_PRE( ch_samplesheet, [[:], []] ) if (!params.skip_nanoplot) { @@ -504,22 +504,27 @@ workflow LRSOMATIC { // MODULE: MINIMAP2_ALIGN (label: process_high) -- once per replicate; @RG encodes sample, type and replicate // Input: [meta_with_replicate, bam] -- unaligned BAM per replicate // ch_fasta -- [[:], fasta] - // sort_bam=true, cigar_paf_format='bai', cigar_bam='', split_prefix='' - // Output: .bam -- [meta_with_replicate, bam] -- coordinate-sorted aligned BAM - // .index -- [meta_with_replicate, bai] -- BAM index + // bam_format=params.aligned_format ('cram' or 'bam'), bam_index_extension='crai' or 'bai' + // Output: .cram / .bam -- [meta_with_replicate, alignments] -- coordinate-sorted; CRAM is reference-compressed + // .index -- [meta_with_replicate, crai or bai] // + def aligned_index = params.aligned_format == 'cram' ? 'crai' : 'bai' + MINIMAP2_ALIGN ( ch_ubams, ch_fasta, - true, - 'bai', + params.aligned_format, + aligned_index, "", "" ) - // Join BAM with index, drop the replicate field, group per sample; single-replicate samples skip SAMTOOLS_MERGE + // Join alignments with index, drop the replicate field, group per sample; single-replicate samples skip SAMTOOLS_MERGE. + // Downstream tuples keep the [meta, bam, bai] names whichever format --aligned_format picks; every consumer + // reads CRAM through the same slots. MINIMAP2_ALIGN.out.bam + .mix(MINIMAP2_ALIGN.out.cram) .join(MINIMAP2_ALIGN.out.index) .map { meta, bam, bai -> def new_meta = meta.subMap('id', @@ -555,22 +560,23 @@ workflow LRSOMATIC { // // MODULE: SAMTOOLS_MERGE (label: process_low) -- replicate identity survives via the unique @RG lines - // Input: [meta, [bam...], [bai...]] -- grouped replicate BAMs + indices - // Output: .bam -- [meta, bam] -- merged BAM + // Input: [meta, [cram...], [crai...]] -- grouped replicate alignments + indices + // [meta, fasta, fai, gzi] -- reference to decode CRAM inputs and encode a merged CRAM + // Output: .cram / .bam -- [meta, alignments] -- merged (output format follows the input extension) // SAMTOOLS_MERGE( ch_aligned_split.multiple, - [[],[],[],[]] + ch_fasta.join(ch_fai).map { meta, fasta, fai -> [meta, fasta, fai, []] }.first() ) - // Index the merged BAM to produce a BAI (SAMTOOLS_MERGE does not create BAI inline) - SAMTOOLS_INDEX_MERGE(SAMTOOLS_MERGE.out.bam) + // Index the merged file (SAMTOOLS_MERGE does not index inline) + SAMTOOLS_INDEX_MERGE(SAMTOOLS_MERGE.out.bam.mix(SAMTOOLS_MERGE.out.cram)) - // Combine single-replicate and merged paths into a unified [meta, bam, bai] channel + // Combine single-replicate and merged paths into a unified [meta, alignments, index] channel ch_single_indexed .mix( - SAMTOOLS_MERGE.out.bam - .join(SAMTOOLS_INDEX_MERGE.out.bai) + SAMTOOLS_MERGE.out.bam.mix(SAMTOOLS_MERGE.out.cram) + .join(SAMTOOLS_INDEX_MERGE.out.bai.mix(SAMTOOLS_INDEX_MERGE.out.crai)) ) .set { ch_index_minimap } // ch_index_minimap: [meta, bam, bai] -- one aligned BAM + index per sample (all replicates merged) @@ -681,7 +687,7 @@ workflow LRSOMATIC { allele_files, loci_files, [], - [], + [], // no fasta: the image's ASCAT has no ref.fasta argument; the CRAMs embed their reference gc_file, rt_file ) @@ -1028,7 +1034,7 @@ workflow LRSOMATIC { // Output: .arrow -- [meta, arrow_file] -- alignment statistics in feather format // - CRAMINO_POST ( ch_minimap_bam ) + CRAMINO_POST ( ch_minimap_bam, ch_fasta ) ch_cramino_post_txt = CRAMINO_POST.out.txt