-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Add custom/taxonomytree module and taxonomy2phylogeny subworkflow #12977
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
8cfacc7
ec60f0d
e3050fc
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| --- | ||
| # yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/environment-schema.json | ||
| channels: | ||
| - conda-forge | ||
| - bioconda | ||
| dependencies: | ||
| - conda-forge::python=3.11.10 |
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,36 @@ | ||||||
| process RAXMLNG_TAXONOMYTREE { | ||||||
| tag "$meta.id" | ||||||
| label 'process_single' | ||||||
|
|
||||||
| conda "${moduleDir}/environment.yml" | ||||||
| // quay.io/biocontainers/python:3.11 has no build-hash-suffixed tag to pin to (unlike | ||||||
| // real bioconda-recipe images); pin by digest instead so the underlying image can't | ||||||
| // silently drift between runs. | ||||||
| container "${ workflow.containerEngine in ['singularity', 'apptainer'] && !task.ext.singularity_pull_docker_container ? | ||||||
| 'https://depot.galaxyproject.org/singularity/python:3.11' : | ||||||
| 'quay.io/biocontainers/python@sha256:b322907f8e52b2055ccad4e46848d28a4a5631b403116cc80ddf61ec8601e05e' }" | ||||||
|
|
||||||
| input: | ||||||
| tuple val(meta), path(taxonomy) | ||||||
|
|
||||||
| output: | ||||||
| tuple val(meta), path("*.guide.nwk"), emit: guide_tree | ||||||
| path "versions.yml" , emit: versions_python, topic: versions | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| when: | ||||||
| task.ext.when == null || task.ext.when | ||||||
|
|
||||||
| script: | ||||||
| template 'taxonomytree.py' | ||||||
|
|
||||||
| stub: | ||||||
| def prefix = task.ext.prefix ?: "${meta.id}" | ||||||
| """ | ||||||
| touch ${prefix}.guide.nwk | ||||||
|
|
||||||
| cat <<-END_VERSIONS > versions.yml | ||||||
| "${task.process}": | ||||||
| python: \$(python3 --version | sed 's/Python //') | ||||||
| END_VERSIONS | ||||||
| """ | ||||||
| } | ||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| name: "raxmlng_taxonomytree" | ||
| description: | | ||
| Build a multifurcating guide tree from a flat taxonomy: one polytomy per shared | ||
| taxonomic rank, resolved only as far as the taxonomy itself resolves it. Intended | ||
| as a topology constraint for a downstream ML tree search (e.g. raxmlng/search), | ||
| so the resulting phylogeny stays consistent with the declared taxonomy. | ||
| keywords: | ||
| - taxonomy | ||
| - phylogenetics | ||
| - newick | ||
| - guide tree | ||
| tools: | ||
| - "custom": | ||
| description: | | ||
| Build a multifurcating guide (Newick) tree from a tab-separated taxonomy file. | ||
| homepage: "https://github.com/nf-core/modules" | ||
| documentation: "https://github.com/nf-core/modules" | ||
| licence: | ||
| - "MIT" | ||
| identifier: "" | ||
|
|
||
| input: | ||
| - - meta: | ||
| type: map | ||
| description: Groovy Map containing sample information. e.g. `[ id:'sample1' ]` | ||
| - taxonomy: | ||
| type: file | ||
| description: | | ||
| Tab-separated taxonomy labels (seq_name<TAB>rank1;rank2;...), one record per | ||
| sequence. | ||
| pattern: "*.{tax,tsv}" | ||
| ontologies: | ||
| - edam: http://edamontology.org/format_3475 # TSV | ||
| output: | ||
| guide_tree: | ||
| - - meta: | ||
| type: map | ||
| description: Groovy Map containing sample information. e.g. `[ id:'sample1' ]` | ||
| - "*.guide.nwk": | ||
| type: file | ||
| description: | | ||
| The multifurcating guide tree, one leaf per taxonomy record, grouped by | ||
| shared taxonomic rank. | ||
| pattern: "*.guide.nwk" | ||
| ontologies: | ||
| - edam: http://edamontology.org/format_1910 # Newick | ||
| versions_python: | ||
| - versions.yml: | ||
| type: file | ||
| description: File containing software versions. | ||
| pattern: "versions.yml" | ||
| ontologies: | ||
| - edam: http://edamontology.org/format_3750 | ||
| topics: | ||
| versions: | ||
| - versions.yml: | ||
| type: file | ||
| description: File containing software versions. | ||
| pattern: "versions.yml" | ||
| ontologies: | ||
| - edam: http://edamontology.org/format_3750 | ||
|
|
||
| authors: | ||
| - "@erikrikarddaniel" | ||
| maintainers: | ||
| - "@erikrikarddaniel" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| #!/usr/bin/env python3 | ||
| import sys | ||
|
|
||
| prefix = "${meta.id}" if "${task.ext.prefix}" == "null" else "${task.ext.prefix}" | ||
|
|
||
|
|
||
| def parse_taxonomy(filepath): | ||
| tab = chr(9) | ||
| entries = [] | ||
| with open(filepath) as fh: | ||
| for line in fh: | ||
| line = line.strip() | ||
| if not line: | ||
| continue | ||
| seq_name, _, tax_str = line.partition(tab) | ||
| ranks = [r.strip() for r in tax_str.split(";") if r.strip()] | ||
| entries.append((seq_name, ranks)) | ||
| return entries | ||
|
|
||
|
|
||
| def build_trie(entries): | ||
| root = {"children": {}, "leaves": []} | ||
| for seq_name, ranks in entries: | ||
| node = root | ||
| for rank in ranks: | ||
| if rank not in node["children"]: | ||
| node["children"][rank] = {"children": {}, "leaves": []} | ||
| node = node["children"][rank] | ||
| node["leaves"].append(seq_name) | ||
| return root | ||
|
|
||
|
|
||
| def node_to_newick(node): | ||
| parts = [] | ||
| for child in node["children"].values(): | ||
| parts.append(node_to_newick(child)) | ||
| parts.extend(node["leaves"]) | ||
| if len(parts) == 1: | ||
| return parts[0] | ||
| return "(" + ",".join(parts) + ")" | ||
|
|
||
|
|
||
| def main(taxonomy_file, output_file): | ||
| entries = parse_taxonomy(taxonomy_file) | ||
| if not entries: | ||
| sys.exit("No entries found in: " + taxonomy_file) | ||
| root = build_trie(entries) | ||
| newick = node_to_newick(root) + ";" | ||
| with open(output_file, "w") as fh: | ||
| print(newick, file=fh) | ||
|
|
||
|
|
||
| main("${taxonomy}", prefix + ".guide.nwk") | ||
|
|
||
| with open("versions.yml", "w") as fh: | ||
| print('"${task.process}":', file=fh) | ||
| print(" python: " + sys.version.split()[0], file=fh) |
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,71 @@ | ||||||
| nextflow_process { | ||||||
|
|
||||||
| name "Test Process RAXMLNG_TAXONOMYTREE" | ||||||
| script "../main.nf" | ||||||
| process "RAXMLNG_TAXONOMYTREE" | ||||||
|
|
||||||
| tag "modules" | ||||||
| tag "modules_nfcore" | ||||||
| tag "raxmlng" | ||||||
| tag "raxmlng/taxonomytree" | ||||||
|
|
||||||
| test("taxonomy tsv") { | ||||||
|
|
||||||
| when { | ||||||
| process { | ||||||
| """ | ||||||
| input[0] = [ | ||||||
| [ id:'test' ], | ||||||
| file(params.modules_testdata_base_path + 'generic/tax/taxonomytree_test.tax', checkIfExists: true), | ||||||
| ] | ||||||
| """ | ||||||
| } | ||||||
| } | ||||||
|
|
||||||
| then { | ||||||
| assert process.success | ||||||
|
|
||||||
| def newick = path(process.out.guide_tree[0][1]).text | ||||||
|
|
||||||
| assertAll( | ||||||
| { assert newick.trim().endsWith(';') }, | ||||||
| { assert newick.contains('sample1') }, | ||||||
| { assert newick.contains('sample2') }, | ||||||
| { assert newick.contains('sample3') }, | ||||||
| { assert newick.contains('sample4') }, | ||||||
| { assert newick.contains('sample5') }, | ||||||
| { assert newick.contains('sample6') }, | ||||||
| // sample1/sample2 share every rank down to genus -- must nest together. | ||||||
| { assert newick =~ /\([^)]*sample1[^)]*sample2[^)]*\)/ || | ||||||
| newick =~ /\([^)]*sample2[^)]*sample1[^)]*\)/ }, | ||||||
| { assert process.out.versions_python }, | ||||||
| ) | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we just |
||||||
| } | ||||||
|
|
||||||
| } | ||||||
|
|
||||||
| test("taxonomy tsv - stub") { | ||||||
|
|
||||||
| options "-stub" | ||||||
|
|
||||||
| when { | ||||||
| process { | ||||||
| """ | ||||||
| input[0] = [ | ||||||
| [ id:'test' ], | ||||||
| file(params.modules_testdata_base_path + 'generic/tax/taxonomytree_test.tax', checkIfExists: true), | ||||||
| ] | ||||||
| """ | ||||||
| } | ||||||
| } | ||||||
|
|
||||||
| then { | ||||||
| assert process.success | ||||||
| assertAll( | ||||||
| { assert snapshot(process.out).match() } | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| ) | ||||||
| } | ||||||
|
|
||||||
| } | ||||||
|
|
||||||
| } | ||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| { | ||
| "taxonomy tsv - stub": { | ||
| "content": [ | ||
| { | ||
| "0": [ | ||
| [ | ||
| { | ||
| "id": "test" | ||
| }, | ||
| "test.guide.nwk:md5,d41d8cd98f00b204e9800998ecf8427e" | ||
| ] | ||
| ], | ||
| "1": [ | ||
| "versions.yml:md5,47c92df7066dad8ee7db1a047cf4c806" | ||
| ], | ||
| "guide_tree": [ | ||
| [ | ||
| { | ||
| "id": "test" | ||
| }, | ||
| "test.guide.nwk:md5,d41d8cd98f00b204e9800998ecf8427e" | ||
| ] | ||
| ], | ||
| "versions_python": [ | ||
| "versions.yml:md5,47c92df7066dad8ee7db1a047cf4c806" | ||
| ] | ||
| } | ||
| ], | ||
| "timestamp": "2026-09-16T17:04:38.571532164", | ||
| "meta": { | ||
| "nf-test": "0.9.4", | ||
| "nextflow": "26.04.6" | ||
| } | ||
| } | ||
| } |
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this enough to be considered for a subworkflow on nf-core?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It will feed another subworkflow --
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think for now I would prefer that it stay pipeline local. I'm of the opinion that nf-core/subworkflows should be a bit more substantial and effectively become troublesome to keep in sync across multiple pipelines if not included here. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| include { RAXMLNG_TAXONOMYTREE } from '../../../modules/nf-core/raxmlng/taxonomytree/main' | ||
| include { RAXMLNG_SEARCH } from '../../../modules/nf-core/raxmlng/search/main' | ||
|
|
||
| workflow TAXONOMY2PHYLOGENY { | ||
|
|
||
| take: | ||
| ch_taxonomy_alignment // channel: [ val(meta), path(taxonomy), path(alignment), val(raxmlng_model) ] | ||
|
|
||
| main: | ||
| RAXMLNG_TAXONOMYTREE( | ||
| ch_taxonomy_alignment.map { meta, taxonomy, alignment, raxmlng_model -> [ meta, taxonomy ] } | ||
| ) | ||
|
|
||
| // RAXMLNG_SEARCH's tree_constraint is a bare path, not a [meta, path] tuple, so it | ||
| // can't be paired with the right alignment by a plain .map() off RAXMLNG_TAXONOMYTREE's | ||
| // own output -- .join() first, on the shared meta, then re-project the single joined | ||
| // channel into RAXMLNG_SEARCH's separate positional inputs, so both stay correctly | ||
| // paired regardless of the two processes' relative completion order. | ||
| def ch_search_input = ch_taxonomy_alignment | ||
| .map { meta, taxonomy, alignment, raxmlng_model -> [ meta, alignment, raxmlng_model ] } | ||
| .join(RAXMLNG_TAXONOMYTREE.out.guide_tree) | ||
| // ch_search_input: [ meta, alignment, raxmlng_model, guide_tree ] | ||
|
|
||
| RAXMLNG_SEARCH( | ||
| ch_search_input.map { meta, alignment, raxmlng_model, _guide_tree -> [ meta, alignment, raxmlng_model ] }, | ||
| [], | ||
| ch_search_input.map { meta, alignment, raxmlng_model, guide_tree -> guide_tree }, | ||
| [], | ||
| [] | ||
| ) | ||
|
|
||
| emit: | ||
| tree = RAXMLNG_SEARCH.out.phylogeny // channel: [ val(meta), path(tree) ] ML phylogeny, consistent with the taxonomy constraint | ||
| model = RAXMLNG_SEARCH.out.best_model // channel: [ val(meta), path(model) ] RAxML-NG model file, consistent with tree + alignment | ||
| guide_tree = RAXMLNG_TAXONOMYTREE.out.guide_tree // channel: [ val(meta), path(guide_tree) ] the multifurcating taxonomy-only guide tree | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| # yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/subworkflows/yaml-schema.json | ||
| name: "taxonomy2phylogeny" | ||
| description: | | ||
| Build a bifurcating ML phylogeny from an alignment, constrained to be consistent | ||
| with a taxonomy: a multifurcating guide tree built from the taxonomy strings | ||
| (raxmlng/taxonomytree) feeds RAxML-NG (raxmlng/search) as a topology constraint, | ||
| and RAxML-NG's own integrated automatic model testing (MOOSE) selects the | ||
| substitution model. | ||
| keywords: | ||
| - phylogenetics | ||
| - taxonomy | ||
| - raxml-ng | ||
| - newick | ||
| - guide tree | ||
| components: | ||
| - raxmlng/taxonomytree | ||
| - raxmlng/search | ||
| input: | ||
| - ch_taxonomy_alignment: | ||
| type: file | ||
| description: | | ||
| Taxonomy, alignment, and the RAxML-NG model (a fixed model string, or | ||
| "DNA"/"AA"/"auto" to trigger RAxML-NG's automatic model testing) to build | ||
| the phylogeny under. | ||
| Structure: [ val(meta), path(taxonomy), path(alignment), val(raxmlng_model) ] | ||
| pattern: "*.{fa,fasta,fna,mfa,phy}" | ||
| output: | ||
| - tree: | ||
| type: file | ||
| description: | | ||
| ML phylogeny, topologically consistent with the taxonomy constraint. | ||
| Structure: [ val(meta), path(tree) ] | ||
| pattern: "*.raxml.bestTree" | ||
| - model: | ||
| type: file | ||
| description: | | ||
| RAxML-NG model file, consistent with tree + alignment. | ||
| Structure: [ val(meta), path(model) ] | ||
| pattern: "*.raxml.bestModel" | ||
| - guide_tree: | ||
| type: file | ||
| description: | | ||
| The multifurcating, taxonomy-only guide tree used as the topology constraint. | ||
| Structure: [ val(meta), path(guide_tree) ] | ||
| pattern: "*.guide.nwk" | ||
| authors: | ||
| - "@erikrikarddaniel" | ||
| maintainers: | ||
| - "@erikrikarddaniel" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we use seqera containers instead?