diff --git a/docs/data/README.md b/docs/data/README.md index a4d4f8e..9fdaae8 100644 --- a/docs/data/README.md +++ b/docs/data/README.md @@ -1,26 +1,26 @@ # Example data -The models that the pages of the [user guide](../guide/index.md) load are copied here and not read from the `RAVEN` submodule, so that every snippet can use a short, stable path (`smallYeast.yml`) that works the same way for a reader who downloads the file and for the example runner in CI. +The models that the pages of the [user guide](../guide/index.md) load are copied here and are not read from the `RAVEN` submodule. Every snippet can therefore use a short path (`smallYeast.yml`), which works the same way for a reader who downloads the file and for the example runner in CI. | File | Size | Source | Used by | |---|---|---|---| -| [`smallYeast.yml`](smallYeast.yml) | 45 kB | `RAVEN/tutorials/data/models/smallYeast.yml` | the default example model for most guide pages | -| [`empty.xml`](empty.xml) | 4 kB | `RAVEN/tutorials/data/empty.xml` | [Building a model from scratch](../guide/building.md): an empty SBML model with compartments defined, to build a model from scratch | +| [`smallYeast.yml`](smallYeast.yml) | 45 kB | `RAVEN/tutorials/data/models/smallYeast.yml` | the default example model of most guide pages | +| [`empty.xml`](empty.xml) | 4 kB | `RAVEN/tutorials/data/empty.xml` | [Building a model from scratch](../guide/building.md): an empty SBML model with compartments, the start of a model built from scratch | | [`smallYeastBad.yml`](smallYeastBad.yml) | 45 kB | `RAVEN/tutorials/data/models/smallYeastBad.yml` | [Quality control](../guide/quality-control.md) and [Comparing models](../guide/comparing.md), which need a model with known errors | -| [`yeast-GEM.yml`](yeast-GEM.yml) | 3.6 MB | yeast-GEM v9.1.0, `model/yeast-GEM.yml` | pages that need a genome-scale model: [Reading and writing models](../guide/io.md), [Simulating growth with FBA](../guide/fba.md), [Growth media and conditions](../guide/media.md), [Biomass composition and annotation](../guide/biomass.md) | +| [`yeast-GEM.yml`](yeast-GEM.yml) | 3.6 MB | yeast-GEM v9.1.0, `model/yeast-GEM.yml` | the pages that need a genome-scale model: [Reading and writing models](../guide/io.md), [Simulating growth with FBA](../guide/fba.md), [Growth media and conditions](../guide/media.md) and [Biomass composition and annotation](../guide/biomass.md) | | [`yeast-GEM.xml`](yeast-GEM.xml) | 11.6 MB | yeast-GEM v9.1.0, `model/yeast-GEM.xml` | the same model as SBML, for [Reading and writing models](../guide/io.md) | -| [`anaerobic.yml`](anaerobic.yml) | 2 kB | transcribed from yeast-GEM's `anaerobicModel.m` | [Growth media and conditions](../guide/media.md): a condition file, yeast-GEM under anaerobic conditions | +| [`anaerobic.yml`](anaerobic.yml) | 2 kB | transcribed from `anaerobicModel.m` of yeast-GEM | [Growth media and conditions](../guide/media.md): a condition file, yeast-GEM under anaerobic conditions | | [`tasks.txt`](tasks.txt) | 0.5 kB | written for this guide | [Metabolic tasks](../guide/tasks.md) and [Gap-filling](../guide/gap-filling.md): a task list for `smallYeast.yml` | | [`curation-mets.tsv`](curation-mets.tsv) | 0.3 kB | written for this guide | [Table-driven curation](../guide/curation.md): a metabolite curation table | -| [`hpa-sample.tsv`](hpa-sample.tsv) | 1 kB | a ten-row excerpt in Human Protein Atlas format | [Context-specific models with ftINIT](../guide/init.md) | +| [`hpa-sample.tsv`](hpa-sample.tsv) | 1 kB | an excerpt of ten rows in Human Protein Atlas format | [Context-specific models with ftINIT](../guide/init.md) | -The two small yeast models are part of the RAVEN Toolbox (MIT licence). yeast-GEM is the consensus genome-scale model of *Saccharomyces cerevisiae* ([SysBioChalmers/yeast-GEM](https://github.com/SysBioChalmers/yeast-GEM), CC-BY-4.0). It is pinned here at release v9.1.0, so the numbers printed in the guide stay reproducible while the model keeps developing. +The two small yeast models are part of the RAVEN Toolbox (MIT licence). yeast-GEM is the consensus genome-scale model of *Saccharomyces cerevisiae* ([SysBioChalmers/yeast-GEM](https://github.com/SysBioChalmers/yeast-GEM), CC-BY-4.0). It is pinned at release v9.1.0, so the numbers printed in the guide stay reproducible while the model develops. -The `.yml` files are RAVEN YAML, read by `readYAMLmodel` in MATLAB and `read_yaml_model` in Python. `yeast-GEM.xml` is SBML, read by `importModel` and cobrapy's `read_sbml_model`. +The `.yml` files are RAVEN YAML, read by `readYAMLmodel` in MATLAB and `read_yaml_model` in Python. `yeast-GEM.xml` is SBML, read by `importModel` and by `read_sbml_model` of cobrapy. ## Refreshing the copies -The two RAVEN tutorial copies are deliberate duplicates and do not follow the `RAVEN` submodule automatically. After a submodule bump that changes either model, refresh the copies and re-run the examples: +The two RAVEN tutorial copies are deliberate duplicates and do not follow the `RAVEN` submodule automatically. After a submodule bump that changes either model, the copies are refreshed and the examples are run again: ```bash cp "RAVEN/tutorials/data/models/smallYeast.yml" docs/data/smallYeast.yml @@ -29,7 +29,7 @@ cp "RAVEN/tutorials/data/models/smallYeastBad.yml" docs/data/smallYeastBad.yml python scripts/run_examples.py ``` -A change of the pinned tag is a deliberate act, and it changes every number that the guide prints from the model. Bump the tag, re-download both files and re-generate the affected output blocks in one commit: +A change of the pinned tag is deliberate, and it changes every number that the guide prints from the model. The tag is bumped, both files are downloaded again, and the affected output blocks are regenerated in one commit: ```bash YEAST_GEM=v9.1.0 # the release the guide currently documents @@ -38,20 +38,20 @@ curl -sSL -o docs/data/yeast-GEM.xml "https://raw.githubusercontent.com/SysBioCh python scripts/run_examples.py --update ``` -Do not edit the files in place. Any change here makes the guide's numbers unreproducible, without a warning, for a reader who takes the model from RAVEN. +The files are never edited in place, and a change is made upstream and copied again. Any change here makes the numbers of the guide unreproducible without a warning for a reader who takes the model from RAVEN. ## Proteomes for homology reconstruction -`hanpo.faa` and `sce-template.faa` back [Reconstruction from homology](../guide/homology.md). +`hanpo.faa` and `sce-template.faa` serve [Reconstruction from homology](../guide/homology.md). -| file | what it is | +| File | Content | |---|---| -| [`hanpo.faa`](hanpo.faa) | the *Hansenula polymorpha* proteome, 5177 sequences, copied unchanged from the `hanpo-GEM` submodule (`data/genomes/hanpo.faa`) | -| [`sce-template.faa`](sce-template.faa) | 61 *S. cerevisiae* sequences (exactly the genes of `smallYeast.yml`) extracted from the same submodule's `data/genomes/sce.faa` | +| [`hanpo.faa`](hanpo.faa) | the *Hansenula polymorpha* proteome with 5177 sequences, copied unchanged from the `hanpo-GEM` submodule (`data/genomes/hanpo.faa`) | +| [`sce-template.faa`](sce-template.faa) | 61 *S. cerevisiae* sequences (the genes of `smallYeast.yml`), extracted from `data/genomes/sce.faa` of the same submodule | -The template proteome is restricted to the genes of the model. BLAST against 61 subjects keeps the example of the homology page to about fifteen seconds per language, and a template protein that no reaction uses cannot contribute to the draft. +The template proteome is restricted to the genes of the model. A BLAST against 61 subjects keeps the example of the homology page at about fifteen seconds per language, and a template protein that no reaction uses cannot contribute to the draft. -Regenerate `sce-template.faa` after a change to `smallYeast.yml`: +`sce-template.faa` is regenerated after a change to `smallYeast.yml`: ```bash python - <<'EOF' diff --git a/docs/guide/biomass.md b/docs/guide/biomass.md index c345880..beb3553 100644 --- a/docs/guide/biomass.md +++ b/docs/guide/biomass.md @@ -1,23 +1,23 @@ # Biomass composition and annotation -The biomass pseudoreaction states what a cell is made of and sets the units of every growth rate the model reports. The composition can be read, one part of it can be changed without breaking the rest, and the annotation makes a model interpretable to something other than a solver. +The biomass pseudoreaction states the composition of a cell and sets the units of every growth rate that the model reports. The composition can be read, one component can be changed without unbalancing the rest, and annotation makes a model interpretable for software other than a solver. ## Functions | MATLAB | Python | Purpose | |---|---|---| -| `getBiomassFractions` | `sum_biomass` | mass fraction per biomass component | +| `getBiomassFractions` | `sum_biomass` | mass fraction of each biomass component | | `scaleBiomassFraction` | `scale_biomass` | rescale one component to a target | | `scaleBiomassPseudoreaction` | `rescale_pseudoreaction` | rescale a pseudoreaction and rebalance it | | `assignSBOterms` | `add_sbo_terms` | label reactions and metabolites with SBO terms | -| `loadDeltaGCSV`, `saveDeltaGCSV` | `load_delta_g_csv`, `save_delta_g_csv` | thermodynamic data through CSV | +| `loadDeltaGCSV`, `saveDeltaGCSV` | `load_delta_g_csv`, `save_delta_g_csv` | thermodynamic data in CSV files | | `extractMiriam`, `editMiriam` | `Object.annotation` {bdg-link-secondary}`cobrapy ` | read and edit database cross-references | ## Setup -The setup uses `yeast-GEM.yml` from [`docs/data/`](../data/README.md). Its biomass is split across one pseudoreaction per macromolecule class, which is the layout both toolboxes expect to be told about. +The model is `yeast-GEM.yml` from [`docs/data/`](../data/README.md). Its biomass is split over one pseudoreaction per class of macromolecules, and both toolboxes need that layout as a configuration. -Neither toolbox guesses that layout: it differs per organism, so it is passed in as a configuration. Each component names the metabolite its pseudoreaction produces, the reaction's name, and how to turn the reaction's substrates into a mass. +The layout differs between organisms, so neither toolbox infers it. Each component names the metabolite that its pseudoreaction produces, the name of the reaction and the way in which the substrates of the reaction are converted to a mass. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -91,9 +91,14 @@ print(len(config.components), "components configured") ::: :::: -The component `name` is used differently on the two sides. MATLAB returns the fractions as a struct keyed by that name, so it has to be a valid field name and cannot contain a space; the lipid backbone is therefore `lipidBackbone` in the configuration. `scaleBiomassPseudoreaction` and `rescale_pseudoreaction` also use the name to identify the metabolite the pseudoreaction produces, which they leave unscaled, so it matches the model's own wording. In MATLAB a name that differs from the metabolite name, such as `lipidBackbone`, leaves the product unprotected. `pseudoreaction_name` locates the reaction in both, and it is the same string either way. +The component `name` has different roles in the two toolboxes. MATLAB returns the fractions as a struct keyed by the name, so the name is a valid field name and contains no space, and the lipid backbone is `lipidBackbone` in the configuration. `scaleBiomassPseudoreaction` and `rescale_pseudoreaction` use the name to identify the metabolite that the pseudoreaction produces and leave it unscaled, so the name matches the wording of the model. In MATLAB, a name that differs from the metabolite name, such as `lipidBackbone`, leaves the product unprotected. `pseudoreaction_name` locates the reaction in both toolboxes, with the same string. -The `mass_strategy` says how a pseudoreaction's substrates become grams. `mw` multiplies each coefficient by the metabolite's molecular weight. `mw_minus_2h` reduces each substrate's molecular weight by two hydrogens (2.016 g/mol), the protons a charged tRNA releases in the protein pseudoreaction; `mw_minus_water` reduces it by one water (18.015 g/mol), the mass lost per polymerisation step in RNA and DNA. A protein therefore weighs less than the sum of its amino acids. `grams` takes the coefficients as already in g/gDW, the form in which the lipid backbone is written. +`mass_strategy` defines how the substrates of a pseudoreaction become grams: + +- `mw` multiplies each coefficient by the molecular weight of the metabolite. +- `mw_minus_2h` reduces the molecular weight of each substrate by two hydrogens (2.016 g/mol), the protons that a charged tRNA releases in the protein pseudoreaction. A protein therefore weighs less than the sum of its amino acids. +- `mw_minus_water` reduces it by one water (18.015 g/mol), the mass lost in each polymerisation step of RNA and DNA. +- `grams` takes the coefficients as g/gDW already, the form of the lipid backbone. ## Composition of the biomass @@ -144,13 +149,13 @@ for name, value in fractions.items(): ::: :::: -The total is the number to check. A biomass pseudoreaction is written so that one unit of flux consumes one gram of cell, which gives the growth rate the unit `/h` rather than an arbitrary unit, so the fractions should sum to about 1 g/gDW. A total that deviates from 1 means a component was edited without rebalancing, and every growth rate the model reports is off by that factor. +The total is the value to check. One unit of flux through the biomass pseudoreaction consumes one gram of cell, which gives the growth rate the unit `/h`, so the fractions sum to about 1 g/gDW. A total far from 1 points to a component that was edited without rebalancing or to a wrong mass strategy, and every growth rate of the model is off by the same factor. The unedited yeast-GEM total is 0.9954. -The lipid chain pseudoreaction is left out of the configuration on purpose. yeast-GEM represents lipids with a backbone and a chain that are two views of the same mass, so counting both would double it; the [lipid curation](../protocol/lipid-curation.md) step in the protocol explains the representation. +The lipid chain pseudoreaction is not in the configuration. yeast-GEM represents lipids with a backbone and a chain, which are two views of the same mass, and counting both would double it. The step [lipid curation](../protocol/lipid-curation.md) of the protocol explains the representation. ## Changing one component -To apply a different measured protein content, set it and specify which other component absorbs the change. A biomass that does not sum to 1 is worse than one with the old number in it, so both functions can balance a second component to absorb the difference. +A different measured protein content is applied by setting it and naming the component that absorbs the change. A biomass that does not sum to 1 is worse than one with the old number, so both functions can balance a second component. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -188,15 +193,15 @@ protein 0.5000, carbohydrate 0.3481, total 1.0000 ::: :::: -`scale_biomass` edits the model in place and returns `None`, while `scaleBiomassFraction` returns a new struct and leaves its input alone. That is the same split as in [Combining and simplifying](combining.md); the Python tab therefore copies the model first. +`scale_biomass` edits the model in place and returns `None`, and `scaleBiomassFraction` returns a new struct and leaves its input unchanged. The same split applies in [Combining and simplifying models](combining.md), and the Python example copies the model first. -`balanceOut` names the component that absorbs the change: it is rescaled so the components sum to 1 g/gDW (`targetTotal` or `target_total`), whatever the total was before (0.9954 becomes 1.0000); the H+ rebalance of a rescaled pseudoreaction can leave the total slightly above it. The balancing component must be large enough to absorb the difference; if it would have to go negative, both functions raise an error, so its pseudoreaction is not turned around. Omit `balanceOut` and the component is rescaled on its own, which is correct only if some other step restores the total. +`balanceOut` names the component that absorbs the change. That component is rescaled so that the components sum to `targetTotal` (`target_total` in Python), 1 g/gDW, whatever the total was before; here 0.9954 becomes 1.0000. The proton rebalance of a rescaled pseudoreaction can leave the total slightly above the target. The balancing component must be large enough to absorb the difference. When it would have to become negative, both functions raise an error, so the pseudoreaction does not turn around. Without `balanceOut`, the component is rescaled alone, which is correct when another step restores the total. -`scaleBiomassPseudoreaction` and `rescale_pseudoreaction` work one level down, on a single pseudoreaction, and rebalance the protons afterwards using the `proton_met` from the configuration. The reaction charge therefore stays balanced when its coefficients move. +`scaleBiomassPseudoreaction` and `rescale_pseudoreaction` work on one pseudoreaction. They rebalance the protons afterwards with the `proton_met` of the configuration, so the charge of the reaction stays balanced when its coefficients change. ## SBO term annotation -An SBO term states what a reaction or metabolite is: a transport reaction, an exchange or a metabolite as opposed to a pseudo-species. Solvers ignore SBO terms; everything that reads a model afterwards does not. +An SBO term states what a reaction or metabolite is: a transport reaction, an exchange reaction, or a metabolite as opposed to a pseudo-species. Solvers ignore SBO terms, and the software that reads a model afterwards uses them. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -216,7 +221,7 @@ glcIN: SBO:0000628 PGI: SBO:0000176 ``` -RAVEN keeps the term as a MIRIAM entry with namespace `sbo` in `rxnMiriams` and `metMiriams`, so `extractMiriam` reads it out, and `exportModel` writes it to SBML as the `sboTerm` attribute. +RAVEN stores the term as a MIRIAM entry with the namespace `sbo` in `rxnMiriams` and `metMiriams`. `extractMiriam` reads it, and `exportModel` writes it to SBML as the attribute `sboTerm`. ::: :::{tab-item} 🐍 Python :sync: python @@ -235,30 +240,36 @@ glcIN: SBO:0000628 PGI: SBO:0000176 ``` -cobrapy keeps the term in each object's `annotation["sbo"]`, as a string, and writes it to SBML as `sboTerm` too. +cobrapy stores the term as a string in `annotation["sbo"]` of each object and writes it to SBML as `sboTerm`. ::: :::: -The terms are assigned from what the model already says. A reaction with a single metabolite is an exchange (SBO:0000627) when that metabolite is in an extracellular compartment (`e`, or one named `extracellular`), and otherwise a sink (SBO:0000632) when it consumes the metabolite or a demand (SBO:0000628) when it produces it; smallYeast has no extracellular compartment, so its uptakes are demands and its outputs sinks. A reaction in which one metabolite name occurs in more than one compartment is a transport (SBO:0000655), so `GROWTH`, which uses acetyl-CoA in both `c` and `m`, is labelled one. Everything else is a metabolic reaction (SBO:0000176), unless its name marks it as the biomass, maintenance or another pseudoreaction. Nothing is inferred that the stoichiometry does not already support, so running it on a model with mislabelled compartments propagates that mistake and does not catch it. +The terms are assigned from the content of the model: + +- A reaction with a single metabolite is an exchange (SBO:0000627) when the metabolite is in an extracellular compartment (`e`, or a compartment named `extracellular`). Otherwise it is a sink (SBO:0000632) when it consumes the metabolite and a demand (SBO:0000628) when it produces it. smallYeast has no extracellular compartment, so its uptake reactions are demands and its output reactions are sinks. +- A reaction in which one metabolite name occurs in more than one compartment is a transport (SBO:0000655). `GROWTH`, which uses acetyl-CoA in `c` and `m`, is labelled as one. +- Every other reaction is a metabolic reaction (SBO:0000176), unless its name marks it as the biomass, maintenance or another pseudoreaction. + +Nothing is inferred beyond what the stoichiometry supports. On a model with wrong compartments, the assignment carries the error forward and does not detect it. ## Cross-reference and thermodynamic annotation -Database cross-references are stored in MIRIAM form: a namespace and an identifier, such as `chebi/CHEBI:15589`. RAVEN keeps them in `metMiriams` and `rxnMiriams` and provides `extractMiriam` to read them out and `editMiriam` to change one without disturbing the rest. cobrapy keeps the same information in each object's `annotation` dictionary, so it is read and written as an ordinary dict, and [Reading and writing models](io.md) covers how both survive a round trip. +Database cross-references are stored in MIRIAM form as a namespace and an identifier, for example `chebi/CHEBI:15589`. RAVEN keeps them in `metMiriams` and `rxnMiriams`. `extractMiriam` reads them, and `editMiriam` changes one without affecting the others. cobrapy keeps the same information in the `annotation` dictionary of each object, which is read and written as an ordinary dictionary. [Reading and writing models](io.md) covers how both forms survive a round trip. -Thermodynamic data is handled in a separate file. `loadDeltaGCSV` and `load_delta_g_csv` read CSVs of standard Gibbs free energies onto the metabolites and reactions, and `saveDeltaGCSV` and `save_delta_g_csv` write it back out. Keeping it in CSV means a thermodynamics run can be versioned and reviewed separately from the model, and re-applied after the model changes. +Thermodynamic data is kept in a separate file. `loadDeltaGCSV` and `load_delta_g_csv` read CSV files of standard Gibbs free energies onto the metabolites and reactions, and `saveDeltaGCSV` and `save_delta_g_csv` write them. Keeping the data in a CSV file lets a thermodynamics run be versioned and reviewed apart from the model and be applied again after the model changes. :::{admonition} Troubleshooting :class: warning -- A configuration that does not match the model. Components are matched on the pseudoreaction's name and not its id. A component whose pseudoreaction is missing contributes zero without failing, so a mistyped name shows as a total that is short. -- The wrong mass strategy. Using `mw` where the convention is `mw_minus_2h` or `mw_minus_water` inflates every polymer by the mass of the bonds, and the total shows it: the total is not near 1. -- Rescaling without balancing. Setting protein to a measured value and leaving the rest alone changes the total, and therefore rescales every growth rate the model produces. -- Counting a lipid twice. A backbone-and-chain representation states the same mass two ways. Include one. -- Trusting SBO terms as validation. They record what the model already implies. They are useful downstream and are no check on the model. +- The configuration does not match the model. Components are matched by the name of the pseudoreaction. A component whose pseudoreaction is missing contributes zero without an error, and a mistyped name appears as a total that is too small. +- The mass strategy is wrong. `mw` where the convention is `mw_minus_2h` or `mw_minus_water` increases the mass of every polymer by the mass of the bonds, and the total is then not close to 1. +- A component is rescaled without balancing. Setting the protein content to a measured value and leaving the rest unchanged alters the total and rescales every growth rate of the model. +- A lipid is counted twice. A representation with a backbone and a chain states the same mass in two ways, and one of them is included. +- SBO terms are used as a validation. They record what the model already implies. Downstream software uses them, and they do not check the model. ::: ## See also -- [Simulating growth with FBA](fba.md), where the growth rate these units belong to comes from. -- [Editing an existing model](editing.md), changing a pseudoreaction by hand. -- [GEM reconstruction protocol: biomass composition](../protocol/biomass.md), building the pseudoreactions from measurements for a new organism. +- [Simulating growth with FBA](fba.md): the source of the growth rate that these units belong to. +- [Editing an existing model](editing.md): changing a pseudoreaction by hand. +- [GEM reconstruction protocol: biomass composition](../protocol/biomass.md): building the pseudoreactions from measurements for a new organism. diff --git a/docs/guide/building.md b/docs/guide/building.md index ef2349e..0bbed7b 100644 --- a/docs/guide/building.md +++ b/docs/guide/building.md @@ -1,6 +1,6 @@ # Building a model from scratch -A hand-built model starts with metabolites, a reaction, a gene association and the exchanges that let material in and out, and ends with a model that carries flux. The same steps test an idea before an automated reconstruction is involved. +A model built by hand needs metabolites, a reaction, a gene association and exchange reactions that let material enter and leave. The same steps test an idea before an automated reconstruction is run. ## Functions @@ -8,15 +8,15 @@ A hand-built model starts with metabolites, a reaction, a gene association and t |---|---|---| | `addMets` | `Metabolite` {bdg-link-secondary}`cobrapy ` | add metabolites | | `addRxns` | `add_reactions_from_equations` | add reactions from equation strings | -| `addGenesRaven` | auto-created from the GPR {bdg-link-secondary}`cobrapy ` | add genes | +| `addGenesRaven` | created from the gene rule {bdg-link-secondary}`cobrapy ` | add genes | | `addExchangeRxns` | `Model.add_boundary` {bdg-link-secondary}`cobrapy ` | add exchange reactions | | `constructEquations` | `Reaction.reaction` {bdg-link-secondary}`cobrapy ` | read the equations back | -In both toolboxes, write a reaction as an equation string and let the toolbox derive the stoichiometry, not a column of the stoichiometric matrix by hand. +Both toolboxes take a reaction as an equation string and derive the stoichiometry from it, which avoids writing a column of the stoichiometric matrix by hand. ## Setup -[`empty.xml`](../data/empty.xml) is RAVEN's starter model: four metabolites (sucrose, glucose, fructose and water) and one reaction, invertase, which splits sucrose into glucose and fructose. Everything is in one compartment, `e`. +[`empty.xml`](../data/empty.xml) is the starter model of RAVEN. It has four metabolites (sucrose, glucose, fructose and water) and one reaction, invertase, which splits sucrose into glucose and fructose. All of them are in one compartment, `e`. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -58,7 +58,7 @@ for reaction in model.reactions: ## Adding metabolites -Hexokinase is added next, so the model needs ATP, ADP and glucose-6-phosphate first. +The next reaction is hexokinase, which needs ATP, ADP and glucose-6-phosphate in the model. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -95,13 +95,13 @@ print(len(model.metabolites), "metabolites") 7 metabolites ``` -Give every metabolite a formula and a compartment when it is added. [Quality control](quality-control.md) checks against both, and adding them later means revisiting every reaction written in between. +A metabolite is given a formula and a compartment when it is added. [Quality control](quality-control.md) checks both, and adding them later means revisiting every reaction written in between. ::: :::: ## Adding a reaction -The equation string is the same in both toolboxes: `<=>` for a reversible reaction, `=>` for an irreversible one. +The equation string is the same in both toolboxes, with `<=>` for a reversible reaction and `=>` for an irreversible reaction. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -128,9 +128,9 @@ glucose[e] + ATP[e] => ADP[e] + glucose-6-phosphate[e] bounds: [0 Inf], genes: YFR053C ``` -`eqnType` specifies how the equation is written: `1` or `"id"` matches metabolites against `model.mets`, `2` or `"name"` against `model.metNames` with `compartment` deciding where they go, and `3` or `"name[comp]"` reads the compartment from the token itself. +`eqnType` specifies how the equation is written. The value `1` or `"id"` matches the metabolites against `model.mets`. The value `2` or `"name"` matches against `model.metNames` and uses `compartment` for the location. The value `3` or `"name[comp]"` reads the compartment from the token. -`allowNewGenes` is needed because `YFR053C` is not in the model. Both `allowNewGenes` and `allowNewMets` default to `false`, so `addRxns` refuses anything it does not recognise unless the option is set, and a mistyped identifier is an error, not a new entity. `allowNewMets` also accepts a string, used as the prefix for the ids it generates. Adding metabolites with `addMets` first carries more annotation than `addRxns` can infer. +`allowNewGenes` is required because `YFR053C` is not in the model. `allowNewGenes` and `allowNewMets` default to `false`, so `addRxns` raises an error for an unknown identifier unless the option is set, and a mistyped identifier does not create a new entity. `allowNewMets` also accepts a string, which becomes the prefix of the generated identifiers. Metabolites added with `addMets` first carry more annotation than `addRxns` can infer. ::: :::{tab-item} 🐍 Python :sync: python @@ -157,15 +157,15 @@ m2 + m5 --> m6 + m7 bounds: (0.0, 1000.0) genes: YFR053C ``` -The arrow sets the bounds, so an irreversible reaction needs no `bounds` key; passing `bounds` overrides it. `<=>` is reversible, and `-->`, `->` and `=>` are accepted for an irreversible reaction. +The arrow sets the bounds, so an irreversible reaction needs no `bounds` key, and `bounds` overrides the arrow. `<=>` is reversible, and `-->`, `->` and `=>` are irreversible. -`allow_new_genes` and `allow_new_mets` both default to `True` here, the opposite of the MATLAB defaults, so a gene named in the rule is created without a separate step and without a message. That removes the `addGenesRaven` call MATLAB needs, and a typo becomes a new gene without an error. Set them to `False` once the model's metabolites and genes are all defined. +`allow_new_genes` and `allow_new_mets` default to `True`, the opposite of the MATLAB defaults. A gene named in the rule is created without a separate step and without a message, which makes the `addGenesRaven` call of MATLAB unnecessary, and a typo creates a new gene without an error. Both should be set to `False` once the metabolites and genes of the model are defined. ::: :::: ## Adding exchanges -Without boundary reactions nothing can enter or leave, so there is no net uptake, secretion or growth. Internal cycles can carry flux even without boundary reactions (see [Flux variability and alternative optima](fva.md#loops-in-flux-ranges)), and under the steady-state assumption a metabolite that appears in only one reaction blocks that reaction. Exchange reactions are the boundary. +Without boundary reactions nothing enters or leaves the model, so there is no net uptake, secretion or growth. Internal cycles can carry flux without boundary reactions ([Flux variability and alternative optima](fva.md#loops-in-flux-ranges)). Under the steady-state assumption, a reaction with a metabolite that occurs in no other reaction is blocked. Exchange reactions form the boundary. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -197,13 +197,13 @@ print(sorted(rxn.id for rxn in model.exchanges)) ['EX_m1', 'EX_m2', 'EX_m3', 'EX_m4', 'EX_m5', 'EX_m6', 'EX_m7'] ``` -`add_boundary` works out that `e` is the external compartment. It refuses when nothing looks external, which is the usual reason it fails on a small hand-built model: give a compartment a recognisable name, or build the exchange as an ordinary reaction with a single metabolite. `type="demand"` and `type="sink"` add the other two kinds of boundary reaction, which cobrapy tracks in separate collections. +`add_boundary` determines that `e` is the external compartment. It raises an error when no compartment looks external, which is the usual cause of a failure in a small hand-built model. A compartment with a recognisable name, or an exchange built as an ordinary reaction with a single metabolite, avoids the error. `type="demand"` and `type="sink"` add the other two kinds of boundary reactions, which cobrapy keeps in separate collections. ::: :::: ## Flux check -Solving the model shows whether the reactions assembled so far can carry flux at all. +A solve shows whether the reactions assembled so far carry flux. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -237,13 +237,13 @@ HEX1 flux: 1000.00 :::{admonition} Troubleshooting :class: warning -- A typo creates a metabolite. The defaults differ: `add_reactions_from_equations` creates unrecognised metabolites and genes, while `addRxns` refuses them unless `allowNewMets` and `allowNewGenes` are set. So `m7` and `M7` become two different things in Python and an error in MATLAB. Set `allow_new_mets=False` and `allow_new_genes=False` once everything is defined. -- No exchange reactions. Nothing can be taken up or secreted, so growth is zero, with no error to explain why. -- A model with no external compartment. `add_boundary` cannot guess one, and RAVEN's `addExchangeRxns` adds exchanges for internal metabolites without warning, opening the model to uptake and secretion of intermediates. +- A typo creates a metabolite. The defaults differ: `add_reactions_from_equations` creates unknown metabolites and genes, and `addRxns` raises an error unless `allowNewMets` and `allowNewGenes` are set. `m7` and `M7` therefore become two metabolites in Python and an error in MATLAB. `allow_new_mets=False` and `allow_new_genes=False` are set once everything is defined. +- The model has no exchange reactions. Nothing is taken up or secreted, so growth is zero and no error explains it. +- The model has no external compartment. `add_boundary` cannot determine one, and `addExchangeRxns` of RAVEN adds exchange reactions for internal metabolites without a warning, which opens the model to the uptake and secretion of intermediates. ::: ## See also -- [Editing an existing model](editing.md), changing the contents of a model. -- [Quality control](quality-control.md), checking a newly built model. -- [Reading and writing models](io.md), saving it. +- [Editing an existing model](editing.md): changing the contents of a model. +- [Quality control](quality-control.md): checking a model after it is built. +- [Reading and writing models](io.md): saving the model. diff --git a/docs/guide/combining.md b/docs/guide/combining.md index da905e3..20950f2 100644 --- a/docs/guide/combining.md +++ b/docs/guide/combining.md @@ -1,6 +1,6 @@ # Combining and simplifying models -Merging and simplifying are two operations that look like simple bookkeeping but carry real risk. **Merging** puts models together: a draft and a template, two organisms, a curated core and an extension. The result is only as sound as the assumption that a metabolite in one model is the same molecule as in the other. **Simplifying** takes a model apart, dropping what cannot carry flux, and the risk is deleting something that was needed. +Merging and simplifying look like bookkeeping and carry risk. Merging puts models together, such as a draft and a template, two organisms, or a curated core and an extension. The result is sound if a metabolite in one model is the same molecule as the metabolite of that name in the other. Simplifying takes a model apart and drops what cannot carry flux, and its risk is the deletion of something that is needed. ## Functions @@ -9,12 +9,12 @@ Merging and simplifying are two operations that look like simple bookkeeping but | `mergeModels` | `merge_models` | combine models into one | | `contractModel` | `remove_duplicate_reactions` | collapse identical reactions | | `simplifyModel` | `simplify_model` | drop what cannot carry flux | -| `expandModel` | `expand_model` | split reactions with `or` in the GPR | +| `expandModel` | `expand_model` | split reactions with `or` in the gene rule | | `mergeCompartments` | `merge_compartments` | collapse the model to one compartment | ## Setup -`smallYeast.yml`, and a second model to merge it with. Taking a copy and giving it a new id makes the comparison unambiguous: whatever the merge does to two identical models shows what it matches on. +The examples use `smallYeast.yml` and a second model to merge it with. The second model is a copy with a new identifier. The merge of two identical models then shows what the merge matches on. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -89,21 +89,21 @@ merged: 106 rxns, 52 mets, 61 genes ::: :::: -The reactions double and the metabolites do not: two models that describe the same 52 metabolites end up sharing them, while every reaction is carried over from both sources and kept. +The number of reactions doubles and the number of metabolites does not. The two models describe the same 52 metabolites, which the merged model shares, and every reaction of both sources is carried over. -The asymmetry follows from the matching rules. Metabolites are matched on name and compartment (`metaboliteName[comp]`, not the identifier) and genes on name, so anything the two models call by the same name becomes one entity. Reactions are matched on nothing: every one is carried over, and the warning reports this. An id that exists in both models gets the source model's id appended (`HXK` and `HXK_second`), which keeps the identifiers unique; the two reactions are not reconciled. Python renames the same way, without a warning. +The asymmetry follows from the matching rules. Metabolites are matched by name and compartment (`metaboliteName[comp]`) and genes by name, so anything that the two models call by the same name becomes one entity. Reactions are matched on nothing, and the warning reports this. An identifier that exists in both models receives the identifier of the source model as a suffix (`HXK` and `HXK_second`), which keeps the identifiers unique, and the two reactions are not reconciled. Python renames in the same way and gives no warning. -`mergeModels` takes `metParam` to choose what metabolites are matched on: `"metNames"`, the default used in the example, or `"mets"` to match on identifier. Matching on id is the right choice when both models come from the same database, where the ids mean the same thing and the names may not. +`mergeModels` takes `metParam`, which selects what metabolites are matched on. The value `"metNames"` is the default and is used in the example, and `"mets"` matches on identifier. Matching on identifier suits two models from the same database, where the identifiers have the same meaning and the names may differ. -:::{admonition} Merging matches on names, not identifiers +:::{admonition} Name-based merging :class: warning -Two models built from different databases usually share almost no metabolite names either, in which case merging produces a model with two disconnected halves in the same struct. Merging assumes that the naming is consistent; check the naming before merging. +Two models built from different databases usually share almost no metabolite names either. The merge then gives a model with two disconnected halves in one struct. Merging assumes consistent naming, and the naming should be checked before the merge. ::: ## Contraction and simplification -Merging does not de-duplicate. Every reaction from both models is kept, so the merged model describes each conversion twice. Collapsing those is a separate step. +Merging does not remove duplicates. Every reaction of both models is kept, so the merged model describes each conversion twice. Collapsing them is a separate step. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -134,24 +134,24 @@ after contracting: 53 rxns ::: :::: -`simplifyModel` does little unless the flags say what to remove: every flag is off by default except `deleteUnconstrained`. `deleteMinMax` is the expensive one used in the example, minimising and maximising each reaction to find out whether it can carry flux; `deleteInaccessible` drops dead ends structurally, `deleteZeroInterval` drops reactions pinned to zero, and `deleteDuplicates` does what `contractModel` does. `reservedRxns` protects reactions from all of them. In Python the same choices are `delete_no_flux` and its siblings, with `open_exchanges` deciding whether the test runs on the medium as it stands or with the boundary opened first. +`simplifyModel` removes nothing unless the flags say what to remove, as every flag is off by default except `deleteUnconstrained`. `deleteMinMax` is the expensive flag used in the example. It minimises and maximises each reaction to find out whether the reaction can carry flux. `deleteInaccessible` drops dead ends structurally, `deleteZeroInterval` drops reactions that are fixed at zero, and `deleteDuplicates` does what `contractModel` does. `reservedRxns` protects reactions from `deleteZeroInterval`, `deleteInaccessible` and `deleteMinMax`. In Python, the same choices are `delete_no_flux` and its related arguments. `open_exchanges` decides whether the test runs on the medium as it is or with the boundary opened first. -:::{admonition} Which functions mutate +:::{admonition} Functions that change the model in place :class: warning Every RAVEN function here returns a new model struct and leaves its input unchanged. The Python functions differ: | Python | Behaviour | |---|---| -| `remove_duplicate_reactions`, `simplify_model` | change the model in place, return `None` | +| `remove_duplicate_reactions`, `simplify_model` | change the model in place and return `None` | | `merge_models`, `merge_compartments` | return a new model | -`model = simplify_model(model)` is correct MATLAB and sets `model` to `None` in Python. Copy the model first to keep the original. +`model = simplify_model(model)` is correct in MATLAB and sets `model` to `None` in Python. The model should be copied first when the original is needed. ::: ## Dropping what cannot carry flux -`smallYeast` ships with its medium shut, so almost nothing in it can carry flux, the state [Quality control](quality-control.md) measures. That makes it a subject for simplification and an illustration of its hazard. +`smallYeast` ships with a shut medium, so almost nothing in it can carry flux, which is the state that [Quality control](quality-control.md) measures. This makes it a subject for simplification and shows its hazard. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -189,11 +189,11 @@ simplified: 2 rxns, 4 mets ::: :::: -A model that has been simplified against a shut medium is a model of that medium, not of the organism. Open the conditions to be simulated before simplifying, otherwise the pathways to be studied are deleted; see [Growth media and conditions](media.md). +A model that is simplified against a shut medium describes that medium. The conditions to be simulated should be opened before simplifying, because the pathways of interest are otherwise deleted ([Growth media and conditions](media.md)). ## Collapsing compartments -Sometimes the compartments are the problem: a draft with unreliable localisation, or a comparison against a model that has none. +Collapsing compartments applies to a draft with unreliable localisation and to a comparison with a model that has no compartments. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -232,28 +232,28 @@ print(f"{len(dropped)} transport reactions dropped") ::: :::: -Seven metabolites in this model exist in both compartments, so 52 collapse to 45; the three reactions that moved something across the mitochondrial membrane become `A -> A`, carry no information, and are dropped. +Seven metabolites of this model exist in both compartments, so 52 metabolites collapse to 45. The three reactions that moved a metabolite across the mitochondrial membrane become `A -> A`, carry no information and are dropped. -The merged metabolite is one of its copies, with every field of that copy (id, name, formula, charge, annotation); raven-toolbox also adds `notes["merged_base_id"]`. The copy is the one in the merged compartment if there is one, otherwise the copy whose id sorts first (`ACCOA_c` before `ACCOA_m`). The id can therefore name a compartment that is not in the merged model. +The merged metabolite is one of its copies, with all fields of that copy (identifier, name, formula, charge, annotation), and raven-toolbox also adds `notes["merged_base_id"]`. The copy is the one in the merged compartment if there is one, and otherwise the copy whose identifier sorts first (`ACCOA_c` before `ACCOA_m`). The identifier can therefore name a compartment that the merged model does not contain. -The two toolboxes count the removals differently. Both end at 50 reactions, but the second line disagrees: RAVEN reports 0 dropped where raven-toolbox reports 3 (`CAT2`, `CO2TRANS`, `ShuttleX`). The same three reactions go in both cases. RAVEN clears them when it removes reactions the merge left empty, and its `deletedRxns` counts only what the `deleteRxnsWithOneMet` path deleted, which is nothing, since that flag is `false` by default. +The two toolboxes count the removals differently. Both end at 50 reactions, and the second line differs: RAVEN reports 0 dropped reactions and raven-toolbox reports 3 (`CAT2`, `CO2TRANS`, `ShuttleX`). The same three reactions go in both cases. RAVEN clears them when it removes the reactions that the merge left empty, and its `deletedRxns` counts the deletions of the `deleteRxnsWithOneMet` path alone, which are none because that flag is `false` by default. -A reaction that holds a single metabolite after merging is kept or removed by the `deleteRxnsWithOneMet` flag in MATLAB and by `drop_single_metabolite_reactions` in Python, which is `false` or `False` by default in both. An exchange has one metabolite by design (`=> glucose`) and is kept whatever the flag is, so the boundary reactions and the objective survive. +A reaction with a single metabolite after the merge is kept or removed by the flag `deleteRxnsWithOneMet` in MATLAB and by `drop_single_metabolite_reactions` in Python, which is `false` or `False` by default in both. An exchange reaction has one metabolite by design (`=> glucose`) and is kept whatever the flag is, so the boundary reactions and the objective remain. -Collapsing compartments discards information. A model that distinguishes mitochondrial from cytosolic acetyl-CoA cannot be recovered from the flattened one, and the flattened model permits a pathway to run on a pool that no membrane separates. Flatten a copy, for a specific question, and keep the original. +Collapsing compartments discards information. A model that distinguishes mitochondrial from cytosolic acetyl-CoA cannot be recovered from the flattened model, and the flattened model allows a pathway to run on a pool that no membrane separates. A copy is flattened for a specific question, and the original is kept. :::{admonition} Troubleshooting :class: warning -- Merging on inconsistent names. The match is on name and compartment. Different naming conventions give two disconnected networks in one model, and the merge raises no error; MATLAB only warns about the metabolite ids it renames. -- Expecting de-duplication. Merging keeps every reaction from every source. Contract afterwards, and check what got collapsed. -- Simplifying against the wrong condition. Reactions are removed because they cannot carry flux under the current bounds. Change the medium first. -- In place or a copy. The Python functions mutate; the MATLAB ones return. Mixing the conventions discards either the original or the result. -- `deleteUnconstrained` has no Python counterpart. RAVEN models mark boundary metabolites with an `unconstrained` field; cobra models use explicit boundary reactions, and that flag has nothing to act on. +- The names are inconsistent. The match uses name and compartment. Different naming conventions give two disconnected networks in one model without an error, and MATLAB warns about the metabolite identifiers that it renames. +- Duplicates are expected to disappear. Merging keeps every reaction of every source. Contract the model afterwards and check the collapsed reactions. +- The model is simplified against the wrong condition. Reactions are removed because they cannot carry flux under the current bounds, so the medium is changed first. +- The function works in place or returns a copy. The Python functions mutate, and the MATLAB functions return a new struct. Mixing the conventions discards either the original or the result. +- `deleteUnconstrained` has no Python counterpart. RAVEN models mark boundary metabolites with an `unconstrained` field, and cobra models use explicit boundary reactions, so the flag has nothing to act on. ::: ## See also -- [Quality control](quality-control.md), deciding what should be removed. -- [Gap-filling](gap-filling.md), the opposite operation, and the usual reason for having a template model to merge from. -- [Context-specific models](init.md), cutting a model down by evidence, not by connectivity. +- [Quality control](quality-control.md): deciding what should be removed. +- [Gap-filling](gap-filling.md): the opposite operation, and the usual reason for a template model to merge from. +- [Context-specific models](init.md): reducing a model by evidence. diff --git a/docs/guide/comparing.md b/docs/guide/comparing.md index 047f3b5..ef759b3 100644 --- a/docs/guide/comparing.md +++ b/docs/guide/comparing.md @@ -1,18 +1,18 @@ # Comparing models -Curation produces several versions of one model: a draft and a curated version, the model before and after gap-filling, a local build and a published one. Two different questions follow: what changed, answered by an exact entry-by-entry diff, and how alike the models are, answered by an overview across whole sets. The first is for review and for continuous integration (CI); the second is for deciding whether two models describe the same organism at all. +Curation produces several versions of one model, such as a draft and a curated version, the model before and after gap-filling, or a local build and a published one. Two questions follow. What changed is answered by an exact entry-by-entry difference, which serves review and continuous integration (CI). How alike the models are is answered by an overview over whole sets, which serves to decide whether two models describe the same organism. ## Functions | MATLAB | Python | Purpose | |---|---|---| | `diffModels` | `diff_models` | every semantic difference between two models | -| `compareMultipleModels` | `compare_models` | overlap in reactions, metabolites, genes, across two or more models | -| `compareFluxes` | `compare_fluxes` | which fluxes changed between two solutions of the same model | +| `compareMultipleModels` | `compare_models` | overlap in reactions, metabolites and genes across two or more models | +| `compareFluxes` | `compare_fluxes` | fluxes that changed between two solutions of the same model | ## Setup -`smallYeast.yml` and `smallYeastBad.yml`: the same model, one of them carrying deliberate errors. The examples compare these two models. +The examples compare `smallYeast.yml` and `smallYeastBad.yml`, where the second file is a copy of the first with deliberate errors. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -50,7 +50,7 @@ good 53 rxns, bad 54 rxns ## Entry-by-entry differences -`diffModels` matches by identifier and compares what it finds: stoichiometry, bounds, objective coefficients, gene rules, formulas, charges. It first answers yes or no on whether the models are the same, and then states the differences. +`diffModels` matches entries by identifier and compares stoichiometry, bounds, objective coefficients, gene rules, formulas and charges. It first reports whether the models are the same and then lists the differences. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -94,11 +94,11 @@ equal: False, 11 differences ::: :::: -This is the comparison for a test. `DiffReport` is falsy when the models differ, so `assert diff_models(before, after)` is a working regression test for a curation script, and in MATLAB, `report.equal` does the same job. +This comparison suits a test. `DiffReport` is falsy when the models differ, so `assert diff_models(before, after)` is a regression test for a curation script, and in MATLAB `report.equal` serves the same purpose. ## Similarity across models -The second question is coarser: the overlap of two models across the whole reaction set. `compare_models` builds a presence matrix, one row per identifier, one column per model, and reduces it to a Jaccard similarity. `compareMultipleModels` takes two models or twenty and reports the overlap on seven identity notions at once (reactions, metabolites, genes, EC numbers, metabolite names, equations with and without compartments), plus a similarity matrix. Pass `'printResults', true` for the full breakdown on screen. +The second question is coarser and concerns the overlap of two models over the whole reaction set. `compare_models` builds a presence matrix with one row per identifier and one column per model and reduces it to a Jaccard similarity. `compareMultipleModels` takes two or twenty models and reports the overlap for seven notions of identity (reactions, metabolites, genes, EC numbers, metabolite names, and equations with and without compartments) and a similarity matrix. `'printResults', true` prints the complete breakdown. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -140,15 +140,15 @@ similarity: 0.945 ::: :::: -:::{admonition} A high similarity does not mean the models agree +:::{admonition} A high similarity does not mean that the models agree :class: warning -These two models are 0.945 alike on the reaction set, and one of them is broken. The differences listed in [Entry-by-entry differences](#entry-by-entry-differences) are a doubled stoichiometric coefficient and a reaction made irreversible, changes that alter what the model predicts while barely moving a set-overlap score. Similarity is for grouping models, not for validating one. +The two models have a similarity of 0.945 on the reaction set, and one of them is broken. The differences in [Entry-by-entry differences](#entry-by-entry-differences) are a doubled stoichiometric coefficient and a reaction made irreversible. These changes alter the predictions of the model and hardly move a set-overlap score. Similarity serves to group models and does not validate one. ::: ## Comparison by behaviour -Two models with the same reactions can behave differently, and two models with different reactions can behave identically. What settles it is a simulation of both under the same constraints: growth first, then flux ranges or tasks, not a single flux vector, which is one optimum among many (see [Flux variability and alternative optima](fva.md)). +Two models with the same reactions can behave differently, and two models with different reactions can behave identically. A simulation of both under the same constraints decides. Growth is compared first, then flux ranges or tasks, and a single flux vector is not compared, because it is one optimum among many ([Flux variability and alternative optima](fva.md)). ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -185,24 +185,24 @@ growth: good 0.1222, bad 0.0000 ::: :::: -The errors in `smallYeastBad` change the prediction. On the same medium, with the same objective, the good model grows and the bad one does not grow at all, from a model that a set-overlap score called 94.5 % similar. A diff shows that the models differ; only a simulation shows what effect the difference has on growth. (MATLAB reports that zero as `-0.0000`; the minus sign is a formatting artefact of a zero objective, not a negative growth rate.) +The errors in `smallYeastBad` change the prediction. On the same medium and with the same objective, the good model grows and the bad model does not grow at all, although a set-overlap score called them 94.5 % similar. A difference shows that the models differ, and a simulation shows the effect of the difference on growth. MATLAB reports the zero as `-0.0000`, and the minus sign is a formatting artefact of a zero objective. -RAVEN can also run this comparison. `compareMultipleModels` takes `funcCompare` and a `taskFile`, and then compares the models by which metabolic tasks each one passes and not by which reactions each one has. That is the same shift from contents to behaviour made by hand in the examples, and [Metabolic tasks](tasks.md) is where the task file comes from. `groupVector` labels the models, by tissue or by condition, so the similarity matrix is reported per group. +RAVEN can run this comparison as well. `compareMultipleModels` takes `funcCompare` and a `taskFile` and compares the models by the metabolic tasks that each passes. This is the shift from content to behaviour that the examples make by hand. The task file is described in [Metabolic tasks](tasks.md). `groupVector` labels the models, for example by tissue or by condition, and the similarity matrix is then reported per group. -The two tabs also show what `compareMultipleModels` and `compare_models` count differently. RAVEN's matrix has one row per combination of models, in-first-only, in-second-only, in-both, with `nElements` counting each; raven-toolbox's has one row per identifier with a column per model. Both answer the same question, but they are indexed in opposite directions. +The two examples also show what `compareMultipleModels` and `compare_models` count differently. The matrix of RAVEN has one row per combination of models (in the first, in the second, in both) with `nElements` counting each. The matrix of raven-toolbox has one row per identifier and one column per model. Both answer the same question and are indexed in opposite directions. :::{admonition} Troubleshooting :class: warning -- Comparing on identifiers alone. Both functions match by id. Two models from different databases share few ids and look unrelated even when they describe the same metabolism; see [Combining and simplifying models](combining.md), where merging matches on names instead. -- Reading similarity as quality. It measures overlap, not correctness. -- Comparing one flux vector per model. An FBA solution is one of many optima, so two models, or two solvers on one model, can return different vectors with the same growth. Check that both solves succeeded and compare growth first, then ranges or tasks. `compareFluxes` and `compare_fluxes` compare two solutions of one model, so they need every reaction in both. -- Forgetting the medium. A flux comparison compares conditions as much as models. Set the same bounds on both, explicitly, before drawing any conclusion. -- Diffing a model against a reloaded copy of itself. Writing and reading a model can change formatting, ordering and rounding; a diff that reports differences after a round trip may reflect the file format; see [Reading and writing models](io.md). +- Models are compared by identifier alone. Both functions match by identifier. Two models from different databases share few identifiers and look unrelated although they describe the same metabolism. [Combining and simplifying models](combining.md) merges by name instead. +- Similarity is read as quality. It measures overlap. +- One flux vector per model is compared. An FBA solution is one of many optima, so two models, or two solvers on one model, can return different vectors with the same growth. Check that both solves succeeded, and compare growth first, then ranges or tasks. `compareFluxes` and `compare_fluxes` compare two solutions of one model, so every reaction must be in both. +- The medium is forgotten. A flux comparison compares conditions as much as models. Set the same bounds explicitly on both models before drawing a conclusion. +- A model is compared with a reloaded copy of itself. Writing and reading a model can change formatting, order and rounding, and differences after a round trip can come from the file format ([Reading and writing models](io.md)). ::: ## See also -- [Quality control](quality-control.md), checking a single model, not a pair. -- [Combining and simplifying models](combining.md), putting models together, and what has to line up first. -- [Simulating growth with FBA](fba.md), the simulation a flux comparison depends on. +- [Quality control](quality-control.md): checking one model. +- [Combining and simplifying models](combining.md): putting models together and what must line up first. +- [Simulating growth with FBA](fba.md): the simulation that a flux comparison depends on. diff --git a/docs/guide/curation.md b/docs/guide/curation.md index c1e7fe9..3a08025 100644 --- a/docs/guide/curation.md +++ b/docs/guide/curation.md @@ -1,17 +1,17 @@ # Table-driven curation -Curation done one function call at a time is hard to review and harder to repeat. Both toolboxes can take the curation as tables: one row per metabolite, gene or reaction, in tab-separated files that a spreadsheet can edit and a pull request can diff. +Curation by single function calls is hard to review and to repeat. Both toolboxes accept curation as tables, with one row per metabolite, gene or reaction, in tab-separated files that a spreadsheet edits and a pull request shows as a difference. ## Functions | MATLAB | Python | Purpose | |---|---|---| | `curateModelFromTables` | `batch_curate_from_tsv` | apply curation tables from files | -| `curateModelFromTables` | `batch_curate` | the same, from DataFrames already in memory | +| `curateModelFromTables` | `batch_curate` | apply curation tables from DataFrames in memory | ## The tables -Four tables, each optional, except that the two reaction tables go together: +There are four tables. Each is optional, except that the two reaction tables are used together. | Table | Columns | |---|---| @@ -20,11 +20,11 @@ Four tables, each optional, except that the two reaction tables go together: | reactions | `rxnIdx`, `rxnNames`, `grRules`, `lb`, `ub`, `rev`, `subSystems`, `eccodes`, `rxnNotes`, `rxnReferences`, `rxnConfidenceScores`, then MIRIAM | | reaction coefficients | `rxnIdx`, `rxnNames`, `metNames`, `comps`, `coefficient`, one row per reaction and metabolite | -Columns are found by their header, so their order does not matter and any of the optional ones can be left out. A `grRules` cell is read with the grammar of `parseGrRule` in both tools, so `G1&G2` is one gene id, while `G1 & G2` is two genes joined by `and`. Entities are matched by name, not by identifier: metabolites on `metaboliteName[comp]`, genes on the gene name, reactions on the stoichiometry of their reactants and products. A row that matches nothing adds a new entity; a row that matches an existing one overwrites it. `rxnIdx` links the two reaction tables and is local to the files, not an identifier in the model. +Columns are found by their header, so the order does not matter and the optional columns can be omitted. Both toolboxes read a `grRules` cell with the grammar of `parseGrRule`, so `G1&G2` is one gene identifier and `G1 & G2` is two genes joined by `and`. Entities are matched by name: metabolites by `metaboliteName[comp]`, genes by gene name, and reactions by the stoichiometry of their reactants and products. A row that matches nothing adds a new entity, and a row that matches an existing entity overwrites it. `rxnIdx` links the two reaction tables and is local to the files, without meaning in the model. ## Setup -`smallYeast.yml` and [`curation-mets.tsv`](../data/curation-mets.tsv) from [`docs/data/`](../data/README.md). The model carries formulas but no charges, so the charge half of a mass balance is undecidable in [Getting started](getting-started.md). The table supplies four of them, with the formulas of the charged species to match: a formula and a charge have to describe the same species, or the charge balance this is for cannot work. The rest of `smallYeast` keeps its neutral formulas, so the reactions that use these four metabolites balance again only once their partners are curated the same way; [Quality control](quality-control.md) finds them. +The files are `smallYeast.yml` and [`curation-mets.tsv`](../data/curation-mets.tsv) from [`docs/data/`](../data/README.md). The model has formulas and no charges, which leaves the charge balance undecidable in [Getting started](getting-started.md). The table supplies four charges, together with formulas of the charged species, because a formula and a charge must describe the same species for the charge balance to work. The other metabolites of `smallYeast` keep their neutral formulas, so the reactions of these four metabolites balance after their partners are curated in the same way. [Quality control](quality-control.md) finds them. ```text title="curation-mets.tsv" metNames comps formula charge metNotes @@ -59,9 +59,9 @@ charges before: 0 (S)-malate charge -2, note: charge assigned at pH 7.3 ``` -A model with no charges has no `metCharges` field, and no field full of `NaN`. RAVEN's optional fields work this way throughout, so code that reads one should check with `isfield` first. `curateModelFromTables` creates a field the table supplies and the model lacks, giving every metabolite the table does not name the field's default (`NaN` for a charge, an empty string for a note). The warning lists the four metabolites whose values were overwritten. +A model without charges has no `metCharges` field, and no field filled with `NaN`. The optional fields of RAVEN work this way throughout, and code that reads one checks with `isfield` first. `curateModelFromTables` creates a field that the table supplies and the model lacks, and gives every metabolite that the table does not name the default of the field (`NaN` for a charge, an empty string for a note). The warning lists the four metabolites whose values were overwritten. -`curateModelFromTables` returns a new struct and leaves its input unchanged, as the rest of RAVEN does. +`curateModelFromTables` returns a new struct and leaves its input unchanged. ::: :::{tab-item} 🐍 Python :sync: python @@ -88,37 +88,41 @@ updated: ['MAL_m', 'AKG_m', 'P2G_c', 'P3G_c'] (S)-malate charge -2, note: charge assigned at pH 7.3 ``` -`batch_curate_from_tsv` edits the model in place and returns a `CurationResult` listing what was added and what was updated, split by entity type. It warns when a row overwrites an existing entity, naming the ids, so an addition and a replacement are distinguishable without inspecting the model afterwards. +`batch_curate_from_tsv` edits the model in place and returns a `CurationResult` that lists the added and the updated entities per entity type. It warns when a row overwrites an existing entity and names the identifiers, so an addition and a replacement can be told apart without inspecting the model. -`batch_curate` takes the same four tables as DataFrames, for a pipeline that builds them and does not read them from disk. +`batch_curate` takes the same four tables as DataFrames, for a pipeline that builds the tables and does not read them from disk. ::: :::: ## Adding entities and overwriting values -A row whose name matches nothing in the model adds a new entity, and the identifier is generated and not taken from the file. Both toolboxes find the largest existing zero-padded number carrying the prefix and continue from it, so the new ids continue the model's own scheme. +A row whose name matches nothing in the model adds a new entity. The identifier is generated. Both toolboxes find the largest zero-padded number that carries the prefix and continue from it, so the new identifiers continue the scheme of the model. -The prefixes are arguments: `metPrefix` and `rxnPrefix` in MATLAB, `met_id_prefix` and `rxn_id_prefix` in Python. Their defaults are `M_` and `R_`, the cobrapy and BiGG convention. A yeast-GEM-derived model should use `s_` and `r_`, and passing the wrong prefixes gives a model with two id schemes in it. +The prefixes are arguments: `metPrefix` and `rxnPrefix` in MATLAB, `met_id_prefix` and `rxn_id_prefix` in Python. The defaults are `M_` and `R_`, the BiGG convention. A model derived from yeast-GEM uses `s_` and `r_`, and other prefixes give a model with two identifier schemes. -Adding a reaction needs both reaction tables. The row in the reactions table carries the bounds, the gene rule and the annotation; the rows in the coefficients table carry the stoichiometry, one per metabolite, negative for substrates and positive for products, linked by `rxnIdx`. Any metabolite named there that the model does not have is created, so the metabolite table is usually applied in the same pass. +Adding a reaction needs both reaction tables. The row in the reactions table holds the bounds, the gene rule and the annotation. The rows in the coefficients table hold the stoichiometry, one row per metabolite, negative for substrates and positive for products, linked by `rxnIdx`. A metabolite named there that the model lacks is created, so the metabolite table is usually applied in the same pass. -:::{admonition} Blank cells overwrite values +:::{admonition} Blank cells :class: warning -In both toolboxes an empty cell is an instruction and not an omission: by default it overwrites the model's existing value, so a table written to fix one charge erases the text annotations in every column that is present but blank in that row. Setting `overwriteWithEmpty` to false, or `overwrite_with_empty=False`, keeps the existing values instead. Empty `grRules` and bounds never overwrite, an empty charge or confidence score becomes 0 when another row has a value, and a charge, confidence-score or annotation (MIRIAM) column with no value at all is not written. In Python, an all-blank text column that is present still clears the stored values. For both toolboxes, include only the columns being curated, and keep the values that should survive in the file and do not rely on the blank, which keeps the value only when `overwriteWithEmpty` is false. +An empty cell is an instruction. By default, it overwrites the existing value of the model, so a table written to correct one charge erases the text annotations in every column that is present and blank in that row. `overwriteWithEmpty` set to false, or `overwrite_with_empty=False` in Python, keeps the existing values. + +Some cells never overwrite: an empty `grRules` cell, and empty bounds. An empty charge or confidence score becomes 0 when another row has a value. A charge, confidence-score or annotation (MIRIAM) column without any value is not written. In Python, a text column that is present and blank in every row also clears the stored values. + +The table includes the columns that are curated, and every value that must survive is written in the file. A blank cell keeps the existing value when `overwriteWithEmpty` is false. ::: :::{admonition} Troubleshooting :class: warning -- Matching on names that do not match. A trailing space, a different capitalisation, or `name[comp]` written with the wrong compartment makes the row add a duplicate entity and not curate the intended one. -- Reactions matched on stoichiometry. Two reactions with the same reactants and products, differing only in direction or bounds, are one reaction to the matcher. -- The wrong id prefix. The default is `M_` and `R_`. On a model using `s_` and `r_` the additions are correct, but their ids do not look like anything else in the model. -- No record of what changed. The tables are the record. Commit them alongside the model, or the curation is as unreviewable as the function calls it replaced. +- The names do not match. A trailing space, a different capitalisation or `name[comp]` with the wrong compartment makes the row add a duplicate entity and leaves the intended entity uncurated. +- Reactions are matched by stoichiometry. Two reactions with the same reactants and products that differ in direction or bounds alone are one reaction to the matcher. +- The identifier prefix is wrong. The default is `M_` and `R_`. On a model with `s_` and `r_`, the additions are correct and their identifiers differ from all others in the model. +- Changes are not recorded. The tables are the record. Commit them together with the model, as the curation is otherwise as difficult to review as the function calls it replaced. ::: ## See also -- [Editing an existing model](editing.md), the same changes one call at a time. -- [Quality control](quality-control.md), checking what a curation pass did. -- [Reading and writing models](io.md), the Excel format, which is the other tabular format for reading and writing models. +- [Editing an existing model](editing.md): the same changes, one call at a time. +- [Quality control](quality-control.md): checking the result of a curation pass. +- [Reading and writing models](io.md): the Excel format, the other tabular format for models. diff --git a/docs/guide/deletions.md b/docs/guide/deletions.md index 11ef42d..949298f 100644 --- a/docs/guide/deletions.md +++ b/docs/guide/deletions.md @@ -1,22 +1,25 @@ # Deletions and essentiality -Gene deletion analysis tests which genes can be deleted while the organism grows: each gene is knocked out, the model is solved again and the result is compared with the wild type. The same machinery answers which reactions are essential, what happens if two things are deleted at once, and what flux distribution a knockout has right after the gene is removed. +A gene deletion analysis finds the genes that can be deleted while the organism grows. Each gene is knocked out, the model is solved again, and the result is compared with the wild type. The same procedure identifies essential reactions, the effect of two simultaneous deletions, and the flux distribution of a knockout directly after the gene is lost. ## Functions | MATLAB | Python | Purpose | |---|---|---| -| `findGeneDeletions` | `single_gene_deletion` {bdg-link-secondary}`cobrapy ` | knock out every gene in turn | -| `findGeneDeletions` (`'sgd'`/`'dgd'`) | `double_gene_deletion` {bdg-link-secondary}`cobrapy ` | pairs of genes | -| `getEssentialRxns` | `find_task_essential_reactions` | reactions a task requires | -| `removeGenes` | `remove_genes` | remove genes, not knock them out | -| no equivalent | `moma` {bdg-link-secondary}`cobrapy ` | fluxes of a knockout, kept near the wild type | +| `findGeneDeletions` | `single_gene_deletion` {bdg-link-secondary}`cobrapy ` | knock out each gene in turn | +| `findGeneDeletions` (`'sgd'`/`'dgd'`) | `double_gene_deletion` {bdg-link-secondary}`cobrapy ` | knock out pairs of genes | +| `getEssentialRxns` | `single_reaction_deletion` {bdg-link-secondary}`cobrapy ` | reactions that the model requires (feasibility in MATLAB, growth in Python) | +| `checkTasks` (`getEssential`) | `find_task_essential_reactions` | reactions that a task requires | +| `removeGenes` | `remove_genes` | remove genes from the model, which differs from a knockout | +| no equivalent | `moma` {bdg-link-secondary}`cobrapy ` | fluxes of a knockout, kept close to the wild type | -`removeGenes` and `remove_genes` rewrite each affected GPR without the removed gene. A reaction left without a catalyst stays in the model with its rule dropped and its bounds set to `(0, 0)` (the default of `remove_genes`, and of `removeGenes` with `removeBlockedRxns` false), which is the bound state `knock_out_model_genes` leaves a blocked reaction in, so the flux consequence of the default equals that of a knockout. `removeGenes` and `remove_genes` also remove the gene and its occurrences in the reaction rules from the model, which a knockout does not; with `blocked_reactions="keep"` the blocked reactions stay at their bounds, and `removeBlockedRxns` true and `blocked_reactions="remove"` delete them. When at least one requested gene is in the model, genes that no remaining reaction used are removed as well, also when they were not requested; when none is in the model, no gene is removed (with `standardizeRules` true, RAVEN standardises the rules in that case too). +`removeGenes` and `remove_genes` rewrite each affected gene rule without the removed gene. A reaction that has no catalyst left stays in the model with its rule dropped and its bounds set to `(0, 0)`. This is the default of `remove_genes`, the behaviour of `removeGenes` with `removeBlockedRxns` false, and the bound state in which `knock_out_model_genes` leaves a blocked reaction, so the flux consequence of the default equals that of a knockout. Removal also takes the gene and its occurrences in the reaction rules out of the model, which a knockout does not. With `blocked_reactions="keep"` the blocked reactions keep their bounds, and `removeBlockedRxns` true or `blocked_reactions="remove"` deletes them. + +When at least one requested gene is in the model, the genes that no remaining reaction used before the removal are removed as well, including genes that were not requested. When no requested gene is in the model, no gene is removed, and RAVEN standardises the rules in this case too when `standardizeRules` is true. ## Setup -`smallYeast.yml` is used with glucose and oxygen open, so the model grows. Uptake in this model is a positive flux through a `=> metabolite` reaction, so the upper bound opens it. +The model is `smallYeast.yml` with glucose and oxygen open, and the model grows. Uptake in this model is a positive flux through a `=> metabolite` reaction, and the upper bound opens it. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -55,7 +58,7 @@ wild type: 0.1222 /h ## Single gene knockout -Every deletion study asks whether the model reaches its objective with a given gene deleted. A gene knockout is not a reaction knockout. The GPR decides how a gene knockout affects the reactions. Removing one of two isozymes changes nothing; removing a subunit of a complex blocks the reaction. +Every deletion study asks whether the model reaches its objective with a given gene deleted. A gene knockout differs from a reaction knockout, and the gene rule determines how the knockout affects the reactions. Removing one of two isozymes changes nothing, and removing a subunit of a complex blocks the reaction. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -89,7 +92,7 @@ PGI1 knockout: 0.0000 /h wild type again: 0.1222 /h ``` -`knock_out_model_genes` sets the bounds of every reaction whose GPR cannot be satisfied to zero and leaves the others unchanged. Inside `with model:` the change is undone when the block exits, so the knockout does not persist beyond the block. +`knock_out_model_genes` sets the bounds of every reaction whose gene rule cannot be satisfied to zero and leaves the other reactions unchanged. Inside `with model:` the change is undone when the block ends. ::: :::: @@ -109,7 +112,13 @@ fprintf('%d genes tested, %d essential\n', numel(genes), sum(growth < 1e-6)); 61 genes tested, 9 essential ``` -`fluxes` is sparse and has one column per deletion, so the growth row is selected by the objective's position (`fluxes(logical(model.c), :)`). `findGeneDeletions` also returns a `details` vector saying what happened to each gene: 1 when the deletion was solved, 2 when the problem had no optimal solution (infeasible or failed, with fluxes reported as zero), and 4 when the gene was skipped because it only appears on dead-end reactions and deleting it could not change the answer. A gene whose deletion solves to zero growth gets 1, not 2, so lethality is read from the growth row. +`fluxes` is sparse and has one column per deletion, so the growth row is selected by the position of the objective (`fluxes(logical(model.c), :)`). `findGeneDeletions` also returns the vector `details`, which states what happened to each gene: + +- `1`: the deletion was solved. +- `2`: the problem had no optimal solution (infeasible or failed), and the fluxes are reported as zero. +- `4`: the gene was skipped because all its reactions are dead ends, so deleting it cannot change the result. + +A deletion that is solved with zero growth gets `1`, so lethality is read from the growth row. ::: :::{tab-item} 🐍 Python :sync: python @@ -128,13 +137,13 @@ print(sorted(next(iter(ids)) for ids in essential.ids)[:5]) ['YBR196C', 'YCR012W', 'YDL066W', 'YDR050C', 'YKL060C'] ``` -`single_gene_deletion` returns a DataFrame with one row per deletion: `ids` (the set of deleted gene ids, a set so that the double-deletion case has the same shape), `growth` and `status`. A gene is looked up with `results.knockout["YBR196C"]`. A knockout that makes the problem infeasible gets `growth` NaN and `status` `'infeasible'`, and `growth < 1e-6` does not select it. To count those as lethal, use `results[(results.growth < 1e-6) | (results.status != "optimal")]`; the MATLAB filter does this, since `findGeneDeletions` reports zero fluxes for them. Here every solve is optimal, so both filters find the same 9. +`single_gene_deletion` returns a DataFrame with one row per deletion and the columns `ids`, `growth` and `status`. `ids` is the set of deleted gene identifiers, a set so that the double deletion has the same shape. A gene is looked up with `results.knockout["YBR196C"]`. A knockout that makes the problem infeasible gets `growth` NaN and `status` `'infeasible'`, and the filter `growth < 1e-6` does not select it. The selection `results[(results.growth < 1e-6) | (results.status != "optimal")]` counts these knockouts as lethal, as the MATLAB filter does, because `findGeneDeletions` reports zero fluxes for them. Every solve here is optimal, so both filters find the same 9 genes. ::: :::: ## Essential reactions -Reaction essentiality asks the same question for reactions. `getEssentialRxns` and `single_reaction_deletion` ask different questions, so their answers differ by 24. +Reaction essentiality asks the same question for reactions. `getEssentialRxns` and `single_reaction_deletion` answer different questions, and their answers differ by 24 reactions. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -149,7 +158,7 @@ fprintf('%d essential reactions\n', numel(essential)); 0 essential reactions ``` -`getEssentialRxns` zeroes the objective before it starts and calls a reaction essential when constraining it to zero makes the problem infeasible. It determines what the model needs to have any solution at all, with no requirement to grow. With the medium open, nothing in `smallYeast.yml` is required for feasibility, so the answer is zero. `ignoreRxns` excludes reactions from the search. +`getEssentialRxns` sets the objective to zero before it starts, and it calls a reaction essential when constraining the reaction to zero makes the problem infeasible. The function determines what the model needs to have any solution, with no requirement to grow. With the medium open, nothing in `smallYeast.yml` is needed for feasibility, so the result is zero. `ignoreRxns` excludes reactions from the search. ::: :::{tab-item} 🐍 Python :sync: python @@ -166,15 +175,15 @@ print(f"{len(essential)} essential reactions") 24 essential reactions ``` -`single_reaction_deletion` keeps the objective and counts the reactions whose removal drops growth below the threshold applied afterwards, `1e-6` here. That is essentiality with respect to growth, which is the question most deletion studies mean, so this function returns 24 where the MATLAB tab returns none. +`single_reaction_deletion` keeps the objective and counts the reactions whose removal lowers the growth below the threshold that is applied afterwards, `1e-6` here. This is essentiality with respect to growth, which is the question of most deletion studies, and the function returns 24 reactions where the MATLAB example returns none. -For essentiality with respect to a task, not the objective, raven-toolbox has `find_task_essential_reactions`; see [Metabolic tasks](tasks.md). +`find_task_essential_reactions` in raven-toolbox gives essentiality with respect to a task ([Metabolic tasks](tasks.md)). ::: :::: ## Double deletions -Double deletions find the redundancy that single deletions miss: two genes that each look dispensable but cannot both be deleted. The number of pairs to test is quadratic in the number of genes, so run time grows sharply with model size. +Double deletions find redundancy that single deletions miss, as two genes can each be dispensable and still not be deletable together. The number of pairs grows with the square of the number of genes, so the run time increases sharply with the size of the model. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -210,15 +219,15 @@ lowest growth: 0.0000 /h ## Fluxes of a knockout -FBA assumes that the knockout re-optimises perfectly, which a cell directly after losing a gene does not do. MOMA (minimisation of metabolic adjustment) looks for the flux distribution closest to a wild-type reference that the mutant can achieve. It models the state right after the knockout, before the cell adapts, and gives a different answer from FBA. +FBA assumes that the knockout re-optimises perfectly, which a cell directly after the loss of a gene does not do. MOMA (minimisation of metabolic adjustment) finds the flux distribution that the mutant can achieve and that is closest to a wild-type reference. It models the state directly after the knockout, before the cell adapts, and gives a different answer from FBA. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB :sync: matlab -RAVEN has no MOMA. RAVEN 3 removed `qMOMA`, which solved the quadratic problem with `quadprog` from MATLAB's Optimization Toolbox, as the last function that depended on a paid MATLAB toolbox. Nothing in RAVEN replaces it, so a MOMA prediction has to come from Python or from the COBRA Toolbox, which has its own. +RAVEN has no MOMA. RAVEN 3 removed `qMOMA`, which solved the quadratic problem with `quadprog` from the Optimization Toolbox of MATLAB, as the last function that depended on a paid MATLAB toolbox. No function of RAVEN replaces it, and a MOMA prediction comes from Python or from the COBRA Toolbox, which has its own. -The FBA half of the comparison runs in RAVEN: +The FBA part of the comparison runs in RAVEN: ```matlab modelKO = removeGenes(model, {'YER178W'}, 'removeBlockedRxns', true); % PDA1 @@ -230,7 +239,7 @@ fprintf('FBA growth: %.4f /h\n', solKO.f); FBA growth: 0.1053 /h ``` -For the same question, what changed in the mutant and not only how much growth fell, RAVEN has `compareFluxes`, which compares two flux vectors from the same model; see [Comparing models](comparing.md). +`compareFluxes` compares two flux vectors of the same model and shows what changed in the mutant, beyond how much the growth fell ([Comparing models](comparing.md)). ::: :::{tab-item} 🐍 Python :sync: python @@ -252,24 +261,24 @@ FBA growth: 0.1053 /h MOMA growth: 0.0869 /h ``` -The reference has to be computed before the knockout. Without `solution=`, cobrapy takes pFBA of the model it is given, which inside the `with` block is the mutant, and MOMA returns the FBA answer. `PDA1` is used as a test because it is not lethal: the PDH bypass (`PDC`, `ALD6`, `ACS`) keeps the mutant growing, so FBA and MOMA can differ. +The reference is computed before the knockout. Without `solution=`, cobrapy takes the pFBA solution of the model it receives, which inside the `with` block is the mutant, and MOMA returns the FBA answer. `PDA1` is the test gene because it is not lethal: the PDH bypass (`PDC`, `ALD6`, `ACS`) keeps the mutant growing, so FBA and MOMA can differ. -`linear=False` is the quadratic MOMA of Segrè et al. (2002): it minimises the squared distance to the reference, a quadratic program (QP). GLPK cannot solve QPs, so cobrapy switches to a QP-capable solver for the call (Gurobi or CPLEX) and raises `SolverNotFound` when none is installed. `pip install gurobipy` installs a size-limited Gurobi licence that is enough for a model this small. `linear=True` minimises the summed absolute distance, an LP; it is a different method with different answers. MOMA's prediction depends on the reference flux vector, which pFBA picks from many alternative optima. +`linear=False` is the quadratic MOMA of Segrè et al. (2002), which minimises the squared distance to the reference as a quadratic program (QP). GLPK cannot solve QPs, so cobrapy switches to a solver that supports QPs (Gurobi or CPLEX) for the call and raises `SolverNotFound` when none is installed. `pip install gurobipy` installs a Gurobi licence with a size limit that suffices for a model of this size. `linear=True` minimises the summed absolute distance as a linear program, which is a different method with different answers. The MOMA prediction depends on the reference flux vector, which pFBA selects from many alternative optima. ::: :::: :::{admonition} Troubleshooting :class: warning -- Confusing gene and reaction knockouts. Deleting a gene silences a reaction only when the GPR says so. Isozymes mask single knockouts, which is what the double deletions test. -- Comparing essentiality numbers from the two toolboxes. `getEssentialRxns` measures feasibility and `single_reaction_deletion` measures growth; they are different questions. -- A knockout that looks lethal because the medium is wrong. Essentiality is relative to the medium and the objective. State both when reporting it; see [Growth media and conditions](media.md). -- Reading essentiality off a threshold. `growth < 1e-6` is a numerical cut-off, not a biological one. A mutant at 1 % of wild-type growth is not dead. -- Double deletions on a genome-scale model. The run time is quadratic in the gene count; restrict the lists, or accept the wait. +- Gene and reaction knockouts are confused. Deleting a gene silences a reaction when the gene rule says so. Isozymes mask single knockouts, which the double deletions test. +- The essentiality numbers of the two toolboxes are compared. `getEssentialRxns` measures feasibility and `single_reaction_deletion` measures growth, which are different questions. +- A knockout looks lethal because the medium is wrong. Essentiality is relative to the medium and the objective, and both are stated with the result ([Growth media and conditions](media.md)). +- Essentiality is read from a threshold. `growth < 1e-6` is a numerical limit. A mutant at 1 % of the wild-type growth is not dead. +- A double deletion runs on a genome-scale model. The run time grows with the square of the number of genes, so restrict the lists or accept the wait. ::: ## See also -- [Simulating growth with FBA](fba.md), the solve underneath all of this. -- [Metabolic tasks](tasks.md), essentiality with respect to a task. -- [Quality control](quality-control.md), before trusting any of it. +- [Simulating growth with FBA](fba.md): the solve that underlies all of this. +- [Metabolic tasks](tasks.md): essentiality with respect to a task. +- [Quality control](quality-control.md): the checks before any of this. diff --git a/docs/guide/editing.md b/docs/guide/editing.md index eac3b9a..40a3a66 100644 --- a/docs/guide/editing.md +++ b/docs/guide/editing.md @@ -1,22 +1,22 @@ # Editing an existing model -Curation is mostly editing: a stoichiometry that was wrong, a gene association that has been superseded, a reaction that should never have been there. Each of those edits has a safe procedure in both toolboxes. +Curation consists mostly of edits: a wrong stoichiometry, a superseded gene association or a reaction that does not belong in the model. Both toolboxes have a procedure for each of these edits. ## Functions | MATLAB | Python | Purpose | |---|---|---| -| `changeRxns` | `change_reaction_equations` | replace a reaction's stoichiometry | +| `changeRxns` | `change_reaction_equations` | replace the stoichiometry of a reaction | | `changeGrRules` | `change_gene_reaction_rules` | set or extend gene associations | -| `standardizeGrRules` | `gpr_to_dnf` | normalise a GPR | -| `setParam` | `Reaction.bounds` {bdg-link-secondary}`cobrapy ` | change bounds, objective | +| `standardizeGrRules` | `gpr_to_dnf` | normalise a gene rule | +| `setParam` | `Reaction.bounds` {bdg-link-secondary}`cobrapy ` | change bounds and objective | | `removeReactions` | `Model.remove_reactions` {bdg-link-secondary}`cobrapy ` | delete reactions | | `removeMets` | `remove_metabolites` | delete metabolites | | `removeGenes`, `deleteUnusedGenes` | `remove_genes` | delete genes | ## Setup -`smallYeast.yml` from [`docs/data/`](../data/README.md). +The model is `smallYeast.yml` from [`docs/data/`](../data/README.md). ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -39,7 +39,7 @@ model = read_yaml_model("smallYeast.yml") ## Fixing a stoichiometry -Give the whole equation, not a coefficient: the toolbox rewrites the reaction's column in the stoichiometric matrix, which keeps everything consistent. +The whole equation is given. The toolbox rewrites the column of the reaction in the stoichiometric matrix, which keeps all fields consistent. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -70,9 +70,9 @@ print(model.reactions.get_by_id("PGI").reaction) G6P_c <=> F6P_c ``` -`changeRxns` takes two matching lists, reactions and equations, and `change_reaction_equations` a mapping of reaction to equation, so a batch of curated reactions can be applied in one call, which is what a curation spreadsheet turns into. Every id must exist in the model; these functions change reactions and do not add them. +`changeRxns` takes two matching lists of reactions and equations, and `change_reaction_equations` takes a mapping from reaction to equation. A batch of curated reactions, which is what a curation spreadsheet turns into, is therefore applied in one call. Every identifier must exist in the model, because these functions change reactions and do not add them. -Bounds are left unchanged. Rewriting an equation from `=>` to `<=>` does not make the reaction reversible, because `lb` and `ub` set the direction and the arrow does not; [Changing bounds and the objective](#changing-bounds-and-the-objective) sets those separately. `changeRxns` takes the `eqnType`, `compartment` and `allowNewMets` options of `addRxns`, and `change_reaction_equations` the `mets_by`, `compartment`, `allow_new_mets` and `new_met_prefix` options of `add_reactions_from_equations`; both match metabolites by id by default. `allowNewMets` defaults to `false` and `allow_new_mets` to `True`. +The bounds are not changed. Rewriting an equation from `=>` to `<=>` does not make the reaction reversible, because `lb` and `ub` set the direction and the arrow does not. [Changing bounds and the objective](#changing-bounds-and-the-objective) sets them separately. `changeRxns` takes the options `eqnType`, `compartment` and `allowNewMets` of `addRxns`, and `change_reaction_equations` takes the options `mets_by`, `compartment`, `allow_new_mets` and `new_met_prefix` of `add_reactions_from_equations`. Both match metabolites by identifier by default. `allowNewMets` defaults to `false` and `allow_new_mets` to `True`. ::: :::: @@ -108,13 +108,13 @@ YBR196C or YLR354C genes now: ['YBR196C', 'YLR354C'] ``` -A gene that is not in the model is created. Pass `replace=False` to append an isozyme to the rule, giving `(old) or (new)`, which records an additional catalyst and not a correction; the default overwrites the rule. A reaction with no rule gets the new rule alone, without empty brackets. +A gene that is not in the model is created. `replace=False` appends an isozyme to the rule and gives `(old) or (new)`, which records an additional catalyst. The default overwrites the rule. A reaction without a rule receives the new rule alone, without empty brackets. ::: :::: -## Normalising a GPR +## Normalising a gene rule -Rules that mix `and` and `or` in nested brackets are hard to compare and hard to score. Both toolboxes rewrite them into disjunctive normal form, a list of alternative complexes. `is_dnf` takes a rule string; `gpr_to_dnf` takes cobrapy's parsed `GPR` object and returns the complexes as lists. +Rules that mix `and` and `or` in nested brackets are difficult to compare and to score. Both toolboxes rewrite them in disjunctive normal form, a list of alternative complexes. `is_dnf` takes a rule string, and `gpr_to_dnf` takes the parsed `GPR` object of cobrapy and returns the complexes as lists. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -129,7 +129,7 @@ fprintf('%s\n', grRules{idx}); YBR196C or YLR354C ``` -`standardizeGrRules` returns the rules and a matching `rxnGeneMat`, not a model, so the results have to be assigned back into the struct to persist. Disjunctive normal form is a flat `or` of `and` groups, one group per alternative complex, which is the form the scoring in [Context-specific models](init.md) expects. +`standardizeGrRules` returns the rules and a matching `rxnGeneMat` and no model, so the results are assigned back into the struct. Disjunctive normal form is a flat `or` of `and` groups, one group per alternative complex, which is the form that the scoring in [Context-specific models](init.md) expects. ::: :::{tab-item} 🐍 Python :sync: python @@ -181,13 +181,13 @@ print("bounds:", model.reactions.get_by_id("PGI").bounds) bounds: (0, 1000) ``` -`setParam` takes `'lb'`, `'ub'`, `'eq'`, `'obj'`, `'rev'`, `'var'` and `'unc'`, and accepts a list of reactions with a list of values, or one value for all, applying them in one call. +`setParam` takes `'lb'`, `'ub'`, `'eq'`, `'obj'`, `'rev'`, `'var'` and `'unc'`. It applies a list of values to a list of reactions, or one value to all, in a single call. ::: :::: ## Deleting metabolites, reactions and genes -Deletion is where the two designs differ most. RAVEN must remove the matching row or column from every field, so deletion has its own functions; cobrapy's objects record what they are attached to. +The toolboxes differ most in deletion. RAVEN removes the matching row or column from every field, and deletion therefore has its own functions. The objects of cobrapy record what they are attached to. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -205,7 +205,7 @@ fprintf('%d -> %d reactions, %d -> %d genes\n', before, numel(reduced.rxns), ... 53 -> 52 reactions, 61 -> 60 genes ``` -Both flags default to `false`, so a plain `removeReactions` leaves metabolites and genes that nothing refers to, which later appear as gaps. `removeUnusedComps` does the same for compartments. `deleteUnusedGenes` performs the gene half on its own. +Both flags default to `false`, so `removeReactions` without flags leaves metabolites and genes that nothing refers to, and they appear later as gaps. `removeUnusedComps` does the same for compartments when `removeUnusedMets` is true, and `deleteUnusedGenes` performs the gene part alone. ::: :::{tab-item} 🐍 Python :sync: python @@ -223,13 +223,13 @@ with model: 53 -> 52 reactions, 61 -> 60 genes ``` -`remove_orphans=True` is cobrapy's equivalent of those two flags. Deletions inside `with model:` are rolled back on exit, so the effect of a deletion can be measured without keeping it. +`remove_orphans=True` corresponds to those two flags. A deletion inside `with model:` is undone when the block ends, so the effect of a deletion can be measured without keeping it. ::: :::: ## Deleting a gene -Deleting a gene differs from deleting its reactions. Removing a gene takes the gene and its occurrences in the reaction rules out of the model, and the reactions stay with rewritten GPRs. With the default policy for reactions whose last catalyst is removed, the flux state equals that of a knockout (bounds `(0, 0)`); the model differs in that the gene is removed. +Deleting a gene differs from deleting its reactions. The gene and its occurrences in the reaction rules are removed, and the reactions stay with rewritten rules. With the default policy for a reaction that has no catalyst left, the flux state equals that of a knockout (bounds `(0, 0)`), and the model differs because the gene is gone. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -265,19 +265,21 @@ PGI rule: 'YLR354C' ::: :::: -`removeGenes` and `remove_genes` rewrite each affected GPR without the removed gene. A reaction whose last catalyst is removed stays in the model with its rule dropped and its bounds set to `(0, 0)`: this is the default of `remove_genes` (`blocked_reactions="constrain"`), while `removeGenes` does it when `removeBlockedRxns` is false, its default, and it is the bound state that `knock_out_model_genes` leaves. With `blocked_reactions="keep"` the blocked reactions keep their bounds; setting `removeBlockedRxns` to true, or `blocked_reactions="remove"`, deletes them instead, which turns a gene deletion into a reaction deletion; [Deletions and essentiality](deletions.md) covers when that is the question being asked. When at least one requested gene is in the model, genes that no remaining reaction used are removed as well, also when they were not requested; when none is in the model, no gene is removed. In `removeGenes`, `standardizeRules`, on by default, normalises the rewritten rules. +`removeGenes` and `remove_genes` rewrite each affected rule without the removed gene. A reaction that has no catalyst left stays in the model with its rule dropped and its bounds set to `(0, 0)`. This is the default of `remove_genes` (`blocked_reactions="constrain"`), the behaviour of `removeGenes` with `removeBlockedRxns` at its default of false, and the bound state that `knock_out_model_genes` leaves. With `blocked_reactions="keep"`, the blocked reactions keep their bounds. `removeBlockedRxns` set to true, or `blocked_reactions="remove"`, deletes them, which turns a gene deletion into a reaction deletion. [Deletions and essentiality](deletions.md) covers when that is the question. + +When at least one requested gene is in the model, the genes that no remaining reaction uses are removed as well, including genes that were not requested. When no requested gene is in the model, no gene is removed. In `removeGenes`, `standardizeRules` is on by default and normalises the rewritten rules. :::{admonition} Troubleshooting :class: warning -- Editing `model.S` by hand. It leaves `model.rev`, the bounds and the gene matrix describing a different model than the matrix does. Use `changeRxns`. -- Deleting a reaction and leaving its metabolites without a reaction. Without the orphan flags, metabolites that nothing produces or consumes remain in the model and appear as gaps. See [Quality control](quality-control.md). -- Overwriting a GPR that was meant to be extended. `changeGrRules` and `change_gene_reaction_rules` replace by default. -- Python edits in place. MATLAB copies the struct on assignment, so `modelB = removeReactions(modelA, ...)` leaves `modelA` unchanged. In Python, use `model.copy()` or `with model:`. +- `model.S` is edited by hand. `model.rev`, the bounds and the gene matrix then describe a different model than the matrix does. `changeRxns` edits a reaction consistently. +- A deleted reaction leaves metabolites without a reaction. Without the orphan flags, metabolites that nothing produces or consumes remain in the model and appear as gaps ([Quality control](quality-control.md)). +- A gene rule that was meant to be extended is overwritten. `changeGrRules` and `change_gene_reaction_rules` replace the rule by default. +- Python edits the model in place. MATLAB copies the struct on assignment, so `modelB = removeReactions(modelA, ...)` leaves `modelA` unchanged. In Python, `model.copy()` or `with model:` keeps the original. ::: ## See also -- [Building a model from scratch](building.md), the same operations, in the other direction. -- [Model structure and identifiers](model-structure.md), why RAVEN needs dedicated deletion functions. -- [Quality control](quality-control.md), checking what an edit did. +- [Building a model from scratch](building.md): the same operations in the other direction. +- [Model structure and identifiers](model-structure.md): the reason RAVEN needs dedicated deletion functions. +- [Quality control](quality-control.md): checking the result of an edit. diff --git a/docs/guide/engineering.md b/docs/guide/engineering.md index ecc4523..2421821 100644 --- a/docs/guide/engineering.md +++ b/docs/guide/engineering.md @@ -1,17 +1,17 @@ # Engineering targets -Two questions underlie most strain-design work: which reactions would have to change for the cell to make more of something, and which metabolites are most affected by a transcriptional response. FSEOF (flux scanning based on enforced objective flux) answers the first from stoichiometry alone; reporter metabolites answer the second from expression data. +Two questions underlie most strain design: which reactions have to change for the cell to make more of a product, and which metabolites are most affected by a transcriptional response. FSEOF (flux scanning based on enforced objective flux) answers the first from the stoichiometry alone. Reporter metabolites answer the second from expression data. ## Functions | MATLAB | Python | Purpose | |---|---|---| -| `FSEOF` | `fseof` | reactions whose flux tracks enforced product formation | -| `reporterMetabolites` | `reporter_metabolites` | metabolites surrounded by transcriptional change | +| `FSEOF` | `fseof` | reactions whose flux follows an enforced product formation | +| `reporterMetabolites` | `reporter_metabolites` | metabolites around which transcription changes | ## Setup -`smallYeast.yml` is used with glucose and oxygen open, so the model grows. Uptake in this model is a positive flux through a `=> metabolite` reaction, so the upper bound opens it. +The model is `smallYeast.yml` with glucose and oxygen open, and the model grows. Uptake in this model is a positive flux through a `=> metabolite` reaction, and the upper bound opens it. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -48,9 +48,9 @@ growth 0.1222 /h ::: :::: -## Reactions that track the product +## Reactions that follow the product -FSEOF forces the product exchange to carry progressively more flux, maximising growth at each step, and records what every other reaction does. A reaction whose flux rises with the enforced product is an amplification candidate; one that falls is a knockdown candidate. Ethanol, `ethOUT`, is the product here. +FSEOF forces the product exchange to carry more flux in each step, maximises growth at each step, and records the flux of every other reaction. A reaction whose flux rises with the enforced product is a candidate for amplification, and a reaction whose flux falls is a candidate for knockdown. The product here is ethanol, `ethOUT`. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -66,9 +66,9 @@ fprintf('%d amplification target(s) of %d reactions\n', ... 14 amplification target(s) of 53 reactions ``` -The scan runs `iterations` steps, ten by default, up to `coefficient` times the theoretical maximum product flux, `0.9` by default. Writing to `outputFile` keeps the per-reaction table out of the command window; without `outputFile` the table prints to the command window. +The scan runs for `iterations` steps, ten by default, up to `coefficient` times the theoretical maximum product flux, `0.9` by default. `outputFile` keeps the per-reaction table out of the command window, and the table is printed there without it. -`corrThreshold` is the filter that decides what counts as a target: a reaction is kept only if the absolute Pearson correlation between its flux and the enforced product flux is at least this value, `0.9` by default. Alternative optima make individual fluxes change erratically between steps, and requiring a nearly linear response keeps that noise out of the target list. +`corrThreshold` decides what counts as a target. A reaction is kept when the absolute Pearson correlation between its flux and the enforced product flux is at least this value, `0.9` by default. Alternative optima make individual fluxes change erratically between steps, and the requirement of a nearly linear response keeps this noise out of the target list. ::: :::{tab-item} 🐍 Python :sync: python @@ -89,19 +89,19 @@ print(f"enforced flux from {result.enforced[0]:.3f} to {result.enforced[-1]:.3f} enforced flux from 0.180 to 1.800 ``` -The two examples count different things. `targets.logical` in MATLAB flags amplification targets only (as `targets.amplify` does; `targets.knockdown` and `targets.knockout` hold the others), while `result.targets` holds every classified reaction, with a `target_type` column of `amplify`, `knockdown` or `knockout`; `result.amplification` selects the amplification targets. Both scans use the same defaults: ten steps, the top of the scan at 0.9 (`coefficient` in MATLAB, `max_fraction` in Python) of the target flux in the pFBA solution that maximises the target, correlation at least 0.9. Both impose the enforced flux as a lower bound on the target reaction by default; `equality=True` in Python fixes the flux at each level instead, as the GECKO function `ecFSEOF` does. In Python, targets of equal score are listed in model order. +The two examples count different things. `targets.logical` in MATLAB flags the amplification targets, as does `targets.amplify`, and `targets.knockdown` and `targets.knockout` hold the others. `result.targets` holds every classified reaction, with a `target_type` column of `amplify`, `knockdown` or `knockout`, and `result.amplification` selects the amplification targets. Both scans use the same defaults: ten steps, a top of the scan at 0.9 of the target flux in the pFBA solution that maximises the target (`coefficient` in MATLAB, `max_fraction` in Python), and a correlation of at least 0.9. Both impose the enforced flux as a lower bound on the target reaction by default. `equality=True` in Python fixes the flux at each level instead, as the GECKO function `ecFSEOF` does. In Python, targets of equal score are listed in model order. -Both tabs find 14 amplification targets here. `fseof` classifies the fluxes of a reaction by the same fixed thresholds as `FSEOF` (`1e-10` for a track above zero, `1e-8` for a final flux of zero). +Both examples find 14 amplification targets. `fseof` classifies the fluxes of a reaction with the same fixed thresholds as `FSEOF`: `1e-10` for a track above zero and `1e-8` for a final flux of zero. -`fseof` returns the whole scan rather than only its conclusion. `scan` is the reactions by enforced-flux-level matrix of fluxes, `enforced` the levels themselves, and `targets` the classified per-reaction table sorted by score. `gene_targets` aggregates the same result to genes, which is the form a strain-design list usually takes. +`fseof` returns the whole scan in addition to its conclusion. `scan` is the matrix of fluxes by reaction and enforced flux level, `enforced` holds the levels, and `targets` is the classified table per reaction, sorted by score. `gene_targets` aggregates the result to genes, which is the usual form of a strain-design list. -`correlation_threshold` matches MATLAB's `corrThreshold`, and `flux_eps` is the tolerance of the check against the theoretical maximum flux (on iJO1366 with target `EX_succ_e` the counts are the same at `1e-8`, `1e-7` and `1e-6`, see [Benchmarks of `fseof`, `remove_genes` and other defaults](../parameter-tuning/benchmarks.md)). `min_target` sets the floor of the enforced range, by default one step (the top of the scan divided by `n_steps`); passing the target reaction's flux at the growth optimum starts the scan at the level the cell already has. +`correlation_threshold` corresponds to `corrThreshold` of MATLAB. `flux_eps` is the tolerance for the check against the theoretical maximum flux. On iJO1366 with the target `EX_succ_e`, the counts are the same at `1e-8`, `1e-7` and `1e-6` ([Benchmarks of `fseof`, `remove_genes` and other defaults](../parameter-tuning/benchmarks.md)). `min_target` sets the lower end of the enforced range, by default one step (the top of the scan divided by `n_steps`). Passing the flux of the target reaction at the growth optimum as `min_target` starts the scan at the level that the cell already has. ::: :::: ## Reading the scan -A target's reliability depends on the trend behind it. The scan matrix distinguishes a reaction that rises steadily from one that changes once and then stops changing. +The reliability of a target depends on the trend behind it. The scan matrix separates a reaction that rises steadily from a reaction that changes once and then stops changing. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -142,13 +142,13 @@ print(result.targets.head(5).to_string()) ::: :::: -The slope is the change in a reaction's flux per unit of enforced product. Its sign gives the direction of the change to apply, and its magnitude the strength. Neither shows that the change is achievable: FSEOF works on the stoichiometry and the bounds, and contains no information on regulation, enzyme capacity or toxicity. +The slope is the change of the flux of a reaction per unit of enforced product. Its sign gives the direction of the change to apply, and its magnitude gives the strength. Neither shows that the change is achievable. FSEOF works on the stoichiometry and the bounds and has no information on regulation, enzyme capacity or toxicity. ## Reporter metabolites -Reporter metabolites start from expression data. Given a differential-expression result, the analysis identifies the metabolites with the most transcriptionally responsive neighbourhood. The algorithm scores each metabolite from the p-values of the genes catalysing the reactions it takes part in, and corrects for how many genes that is, so a metabolite in a large subsystem does not rank high by size alone. +Reporter metabolites start from expression data. Given the result of a differential-expression analysis, the reporter-metabolite method identifies the metabolites whose neighbourhood responds most strongly at the transcript level. The algorithm scores each metabolite from the p-values of the genes that catalyse its reactions and corrects for the number of these genes. A metabolite in a large subsystem does not rank high by size alone. -Real p-values come from a differential-expression analysis. The p-values in the examples are derived from the gene identifiers to keep the results reproducible, so both tabs compute the same input. +Real p-values come from a differential-expression analysis. The p-values in the examples are derived from the gene identifiers so that the results are reproducible, and both examples compute the same input. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -176,7 +176,7 @@ end F16P_c z 1.126 p 0.1301 genes 4 ``` -`printResults` prints the top 20, and `outputFile` writes them. Supplying `geneFoldChanges` adds two further results alongside the full test, one for the up-regulated genes and one for the down-regulated, so a metabolite that is only interesting in one direction can be told apart from one that responds either way. +`printResults` prints the top 20, and `outputFile` writes them. `geneFoldChanges` adds two results next to the full test, one for the up-regulated genes and one for the down-regulated genes. A metabolite of interest in a single direction can then be told apart from a metabolite that responds in either direction. ::: :::{tab-item} 🐍 Python :sync: python @@ -202,26 +202,26 @@ metabolite name z_score p_value n_genes m F16P_c beta-D-fructofuranose 1,6-bisphosphate 1.126055 0.130071 4 0.089746 0.020478 ``` -`reporter_metabolites` returns one result per gene set. Without fold changes that is a single `"all"` result; with `gene_fold_changes` it is three, adding `"up"` and `"down"` for the two subsets. Each carries a `table` with the z-score, p-value and the number of genes behind each metabolite. +`reporter_metabolites` returns one result per gene set. Without fold changes there is one result, `"all"`, and with `gene_fold_changes` there are three, with `"up"` and `"down"` for the two subsets. Each result has a `table` with the z-score, the p-value and the number of genes behind each metabolite. -Genes the model does not have and missing (NaN) p-values are dropped; a p-value outside `[0, 1]` raises an error, because one invalid value would otherwise turn the whole result into `NaN`. +Genes that the model does not have and missing (NaN) p-values are dropped. A p-value outside `[0, 1]` raises an error, because one invalid value would turn the whole result into `NaN`. ::: :::: -Both toolboxes compute the background correction in closed form and report the same z-scores and one-sided p-values, sorted by z-score. RAVEN returns them as parallel arrays in a struct, raven-toolbox as one table per gene set. +Both toolboxes compute the background correction in closed form and report the same z-scores and one-sided p-values, sorted by z-score. RAVEN returns them as parallel arrays in a struct, and raven-toolbox returns one table per gene set. :::{admonition} Troubleshooting :class: warning -- Reading FSEOF as a prediction. It reports what the stoichiometry allows when product formation is forced, which is a hypothesis to test, not an expected yield. -- A target list full of noise. With a lowered correlation threshold, alternative optima appear as targets. If a reaction's flux is not close to linear in the enforced product, its slope is not meaningful. -- Forcing a product the model cannot make. If the target exchange cannot carry flux, every step of the scan is infeasible and there is nothing to correlate. Check it with a single solve first. -- Reporter metabolites on an unfiltered gene list. The correction handles neighbourhood size, not a p-value distribution that is uniform because nothing is differentially expressed. Look at the input distribution before trusting the ranking. -- Currency metabolites at the top. ATP, NADH and water take part in a large share of all reactions, so they tend to appear at the top. Excluding them, or reading past them, is usually necessary. +- FSEOF is read as a prediction. It reports what the stoichiometry allows when product formation is forced, which is a hypothesis to test. +- The target list contains noise. With a lowered correlation threshold, alternative optima appear as targets. When the flux of a reaction is not close to linear in the enforced product, its slope has no meaning. +- A product that the model cannot make is forced. When the target exchange cannot carry flux, every step of the scan is infeasible and there is nothing to correlate. A single solve checks this first. +- Reporter metabolites are computed on an unfiltered gene list. The correction handles the size of the neighbourhood and does not handle a uniform p-value distribution that arises because nothing is differentially expressed. Inspect the distribution of the input before trusting the ranking. +- Currency metabolites rank at the top. ATP, NADH and water take part in many reactions and therefore tend to rank at the top. Exclude them, or read the ranking beyond them. ::: ## See also -- [Simulating growth with FBA](fba.md), the solve that each FSEOF step performs. -- [Random sampling](sampling.md), the other way to ask what a network can do, not what one optimum says. -- [Deletions and essentiality](deletions.md), which covers knockouts, not amplification. +- [Simulating growth with FBA](fba.md): the solve that each FSEOF step performs. +- [Random sampling](sampling.md): the other way to ask what a network can do, which a single optimum does not show. +- [Deletions and essentiality](deletions.md): knockouts. diff --git a/docs/guide/fba.md b/docs/guide/fba.md index ac0099b..12f6abd 100644 --- a/docs/guide/fba.md +++ b/docs/guide/fba.md @@ -1,21 +1,21 @@ # Simulating growth with FBA -Flux balance analysis (FBA) sets an objective, constrains the uptake rates, solves and reads the fluxes back. Every other analysis on this site is built from this loop. +Flux balance analysis (FBA) sets an objective, limits the uptake rates, solves the linear program and reads the fluxes. The other analyses on this site repeat this sequence with changes. ## Functions | MATLAB | Python | Purpose | |---|---|---| -| `setParam` | `Reaction.bounds`, `Model.objective` {bdg-link-secondary}`cobrapy ` | set bounds and the objective | -| `solveLP` | `Model.optimize` {bdg-link-secondary}`cobrapy ` | solve the LP | +| `setParam` | `Reaction.bounds`, `Model.objective` {bdg-link-secondary}`cobrapy ` | set the bounds and the objective | +| `solveLP` | `Model.optimize` {bdg-link-secondary}`cobrapy ` | solve the linear program | | `printFluxes` | `Model.summary` {bdg-link-secondary}`cobrapy ` | show the fluxes that carry material | -| `solveLP` (`minFlux`) | `pfba` {bdg-link-secondary}`cobrapy ` | pick a parsimonious solution among the optima | +| `solveLP` (`minFlux`) | `pfba` {bdg-link-secondary}`cobrapy ` | select a parsimonious solution among the optima | -Every Python simulation step on this page is cobrapy, marked {bdg-link-secondary}`cobrapy ` in the functions table. In MATLAB, `solveLP` needs neither the COBRA Toolbox nor anything else outside RAVEN. +Every Python simulation step on this page comes from cobrapy and carries the {bdg-link-secondary}`cobrapy ` badge in the functions table. In MATLAB, `solveLP` does not need the COBRA Toolbox or any other package outside RAVEN. ## Setup -The model is `yeast-GEM.yml` from [`docs/data/`](../data/README.md), yeast-GEM v9.1.0, which has a growth objective and an aerobic glucose medium set. +The model is `yeast-GEM.yml` from [`docs/data/`](../data/README.md), yeast-GEM v9.1.0, with a growth objective and an aerobic glucose medium. ## Solving @@ -37,18 +37,18 @@ status: 1 growth: 0.0809 /h ``` -The objective is `model.c`, a vector with one entry per reaction, so `model.c == 1` finds the reaction being maximised. +The objective is `model.c`, a vector with one entry per reaction, and `model.c == 1` finds the reaction that is maximised. -`sol` holds the whole answer. `sol.f` is the objective value with its natural sign: RAVEN minimises internally and negates the result, so no sign has to be undone by the caller. `sol.x` is the flux vector, in the order of `model.rxns`, and `sol.stat` reports how the solve ended: +`sol` holds the result. `sol.f` is the objective value with its natural sign, because RAVEN minimises internally and negates the result before returning it. `sol.x` is the flux vector in the order of `model.rxns`. `sol.stat` reports how the solve ended: | `stat` | Meaning | |---|---| | `1` | solved to optimality | -| `0` | a feasible solution was found, but not proven optimal | +| `0` | a feasible solution was found without a proof of optimality | | `-1` | no feasible solution exists | -| `-2` | solved, but the second optimisation set by `minFlux` failed | +| `-2` | solved, and the second optimisation requested by `minFlux` failed | -Anything other than `1` makes the fluxes provisional at best, so check `stat` before reading `x`. A plain solve (`minFlux` left at its default) also returns `sol.sPrice` and `sol.rCost`, the shadow prices and reduced costs from the LP dual. +The fluxes are provisional for every value other than `1`, so `stat` should be checked before `x` is read. A solve with `minFlux` at its default also returns `sol.sPrice` and `sol.rCost`, the shadow prices and the reduced costs from the dual of the linear program. ::: :::{tab-item} 🐍 Python :sync: python @@ -70,15 +70,15 @@ status: optimal growth: 0.0809 /h ``` -`optimize` maximises by default and returns a `Solution` whose `fluxes` is a pandas Series indexed by reaction id. +`optimize` maximises by default and returns a `Solution`, whose `fluxes` is a pandas Series indexed by reaction identifier. -The objective prints as a difference of two terms because cobrapy gives every reaction a forward and a reverse variable, both non-negative, and reports the net flux as forward minus reverse. The suffix on the reverse variable is a hash, so it differs between models. Only the expression is split; `r_2111` remains one reaction, and `solution.fluxes['r_2111']` is the net value. +The objective is printed as a difference of two terms, because cobrapy gives every reaction a forward and a reverse variable, both non-negative, and reports the net flux as forward minus reverse. The suffix of the reverse variable is a hash and differs between models. The expression is split, and the reaction is not. `r_2111` remains one reaction, and `solution.fluxes['r_2111']` is its net flux. ::: :::: ## Changing the objective -The objective is a reaction to maximise: growth, a product exchange, an ATP demand. +The objective is a reaction that is maximised, for example growth, a product exchange or an ATP demand. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -93,7 +93,7 @@ disp(model.rxnNames{getIndexes(model, 'r_2111', 'rxns')}); growth ``` -`setParam` writes the coefficient into `model.c` and zeroes every other entry, so setting an objective replaces the previous one. A negative coefficient minimises that reaction. +`setParam` writes the coefficient into `model.c` and sets every other entry to zero, so a new objective replaces the previous one. A negative coefficient minimises the reaction. ::: :::{tab-item} 🐍 Python :sync: python @@ -107,13 +107,13 @@ print(model.reactions.get_by_id("r_2111").name) growth ``` -Assigning a reaction or its id replaces the objective. `model.objective` also accepts an expression, so a weighted combination of reactions is written as one expression, and `model.objective_direction` switches between maximising and minimising. +Assigning a reaction or its identifier replaces the objective. `model.objective` also accepts an expression, which defines a weighted combination of reactions, and `model.objective_direction` switches between maximisation and minimisation. ::: :::: -## Constraining an uptake rate +## Limiting an uptake rate -yeast-GEM writes its exchange reactions as `metabolite <=>`, so uptake is a negative flux and the lower bound limits it. A model that writes uptake as `=> metabolite`, like smallYeast in [Solvers](solvers.md), has the opposite sign. yeast-GEM has glucose capped at 1 mmol/gDW/h, which determines the growth rate of the first solve; ten times the glucose gives roughly ten times the growth. +yeast-GEM writes its exchange reactions as `metabolite <=>`, so uptake is a negative flux and the lower bound limits it. A model that writes uptake as `=> metabolite`, such as smallYeast in [Solvers](solvers.md), has the opposite sign. In yeast-GEM, glucose uptake is limited to 1 mmol/gDW/h, which sets the growth rate of the first solve. Ten times the glucose gives roughly ten times the growth. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -156,11 +156,11 @@ growth on 10 mmol glucose: 0.8370 /h ::: :::: -The upper bound of `1000` on the same reaction is the model's convention for "unbounded", and it allows secretion of glucose, which the model has no reason to do. Bounds are a pair, and only the one facing the direction of interest constrains the answer. [Growth media and conditions](media.md) sets all of them at once and covers the opposite sign convention that `model.medium` uses. +The upper bound of `1000` on the same reaction is the convention of the model for an unbounded flux, and it allows glucose secretion, which the model has no reason to use. A bound is one half of a pair, and the half that faces the direction of interest limits the result. [Growth media and conditions](media.md) sets all bounds of a medium at once and covers the opposite sign convention of `model.medium`. ## Temporary changes -A knockout or a tighter bound is usually a question to test, not a change to keep. cobrapy's model is a context manager: changes made inside `with model:` are rolled back on leaving the block. MATLAB has no equivalent; copy the struct, change the copy, and let it go out of scope. +A knockout or a tighter bound is usually tested and then discarded. The cobrapy model is a context manager, and the changes made inside `with model:` are undone when the block ends. MATLAB has no equivalent. The struct is copied, the copy is changed, and the copy is discarded. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -181,7 +181,7 @@ anaerobic: 0.0000 /h back to aerobic: 0.0809 /h ``` -`'eq'` sets both bounds at once, so a single call closes the reaction in both directions. MATLAB structs are value types, so `modelKO` is a copy from the moment it is assigned and `model` cannot be reached through it. +`'eq'` sets both bounds, so one call closes the reaction in both directions. MATLAB structs are value types, so `modelKO` is a copy from the assignment, and `model` cannot be reached through it. ::: :::{tab-item} 🐍 Python :sync: python @@ -199,15 +199,15 @@ anaerobic: -0.0000 /h back to aerobic: 0.0809 /h ``` -Without `with`, the bound change would persist and every later result of these examples would be an anaerobic one. The block undoes the edits cobrapy records: bounds, the objective, reactions and metabolites added or removed, stoichiometry changed with `add_metabolites`, gene rules and gene knockouts. Plain attribute edits are not recorded and survive the block: names, formulas, charges, `subsystem`, the model id and anything written into `annotation` or `notes`. Make those on `model.copy()`. +Without `with`, the bound change would remain, and every later result of the examples would be an anaerobic result. The block undoes the edits that cobrapy records: bounds, the objective, added or removed reactions and metabolites, stoichiometry changed with `add_metabolites`, gene rules and gene knockouts. Plain attribute edits are not recorded and remain after the block. These are names, formulas, charges, `subsystem`, the model identifier and the contents of `annotation` and `notes`. They are made on `model.copy()`. -The anaerobic growth rate is zero. Closing the oxygen exchange is not enough to make yeast-GEM grow fermentatively; growth also needs sterol and fatty-acid uptake and a different biomass composition, which a condition provides (see [Growth media and conditions](media.md)). The minus sign on that zero is solver noise and is not a negative growth rate. Compare the value against a tolerance, not against `0`. +The anaerobic growth rate is zero. A closed oxygen exchange does not make yeast-GEM grow by fermentation. Growth also needs the uptake of sterols and fatty acids and a different biomass composition, which a condition provides ([Growth media and conditions](media.md)). The minus sign on that zero is solver noise. A value is compared with a tolerance around `0`. ::: :::: ## Inspecting the fluxes -An FBA solution has thousands of numbers, most of them zero. Both toolboxes provide a filtered view. +An FBA solution has thousands of fluxes, and most are zero. Both toolboxes show a filtered view. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -229,7 +229,7 @@ end ethanol: 0.0000 ``` -`printFluxes(model, sol.x)` does the same filtering without a list of ids: by default it prints every exchange reaction whose absolute flux is at least `1e-8`. Its `cutOffFlux` raises that floor, `onlyExchange` set to `false` includes internal reactions, `metaboliteList` restricts the print-out to reactions touching named metabolites, and `outputString` controls the columns, with `%eqn`, `%lower` and `%upper` available alongside `%flux`. +`printFluxes(model, sol.x)` filters without a list of identifiers. By default it prints every exchange reaction with an absolute flux of at least `1e-8`. `cutOffFlux` raises that limit, `onlyExchange` set to `false` includes internal reactions, and `metaboliteList` restricts the output to reactions with the named metabolites. `outputString` sets the columns, and `%eqn`, `%lower` and `%upper` are available next to `%flux`. ::: :::{tab-item} 🐍 Python :sync: python @@ -253,15 +253,15 @@ for rxn_id, label in exchanges.items(): ethanol: 0.0000 ``` -`model.summary()` prints the same picture (uptake, secretion and the objective) as a table, and `solution.fluxes` is a pandas Series, so ordinary filtering works: `solution.fluxes[solution.fluxes.abs() > 1e-6]`. +`model.summary()` prints the same view of uptake, secretion and objective as a table. `solution.fluxes` is a pandas Series, so ordinary filtering applies: `solution.fluxes[solution.fluxes.abs() > 1e-6]`. ::: :::: -In yeast-GEM's orientation the signs say which way material moves: negative through an exchange is uptake, positive is secretion. Glucose and oxygen enter, CO2 leaves, and ethanol is zero because the model has enough oxygen to respire everything it takes up. +With the orientation of yeast-GEM, a negative flux through an exchange reaction is uptake and a positive flux is secretion. Glucose and oxygen enter and CO2 leaves. Ethanol is zero because the model has enough oxygen to respire all the glucose that it takes up. ## Parsimonious solution -An FBA optimum is rarely unique: many flux distributions reach the same growth rate, and a plain solve returns an arbitrary one, often with internal loops that carry flux without contributing to the objective. Parsimonious FBA (pFBA) keeps the objective at its optimum and then minimises the total flux, on the reasoning that a cell does not run reactions it gains nothing from. +An FBA optimum is rarely unique. Many flux distributions reach the same growth rate, and a plain solve returns an arbitrary one, which often contains internal loops that carry flux without contributing to the objective. Parsimonious FBA (pFBA) keeps the objective at its optimum and minimises the total flux, on the assumption that a cell does not run reactions that give it no gain. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -278,7 +278,7 @@ growth: 0.0809 /h total flux: 100.6 ``` -`minFlux` selects the second optimisation. `1` minimises the sum of absolute fluxes, which requires one further LP. `3` minimises the number of active reactions, which is a mixed-integer problem: the result is easier to read as a pathway, and the solve is far slower. Leaving `minFlux` at `0` skips the second solve and is the only setting that reports shadow prices. +`minFlux` selects the second optimisation. The value `1` minimises the sum of absolute fluxes and needs one further linear program. The value `3` minimises the number of active reactions, which is a mixed-integer problem. Its result is easier to read as a pathway, and the solve takes much longer. The value `0` skips the second solve and is the sole setting that reports shadow prices. ::: :::{tab-item} 🐍 Python :sync: python @@ -296,25 +296,24 @@ growth: 0.0809 /h total flux: 100.6 ``` -`pfba` is the equivalent of `minFlux` set to `1`. It takes `fraction_of_optimum`, which relaxes the objective before minimising flux, so `0.95` asks for the leanest distribution that still reaches 95 % of the optimum. +`pfba` corresponds to `minFlux` set to `1`. `fraction_of_optimum` relaxes the objective before the flux is minimised, so `0.95` gives the smallest total flux at 95 % of the optimum. -The growth rate is unchanged and the total flux is the smallest that achieves it. The number of active reactions varies between solves: pFBA pins the total flux and leaves open which reactions carry it, so a test on that count fails intermittently. +The growth rate does not change, and the total flux is the smallest that achieves it. The number of active reactions varies between solves, because pFBA fixes the total flux and leaves open which reactions carry it. A test on that number fails intermittently. ::: :::: :::{admonition} Troubleshooting :class: warning -- The status is `infeasible`: something is over-constrained, most often an uptake that is closed, or a bound set to the wrong sign. See [Growth media and conditions](media.md). -- Growth is zero but the solve succeeded: the model is feasible and the optimum is zero, because a nutrient is missing or a gap blocks the biomass pseudoreaction. -- The same model gives different flux distributions: this is expected, as alternative optima exist. Use pFBA, or compare ranges with flux variability analysis (FVA), not one solution. -- Reading `sol.f` as a negated objective: RAVEN minimises `-c'x` internally and negates the result before returning, so `sol.f` is the growth rate, not its negative. -- A model whose coefficients span many orders of magnitude: ill-scaled stoichiometry makes the solver report an optimum that shifts between solvers. `solveLP` takes `params.maxRatio`, which splits any reaction whose coefficients span more than that ratio through auxiliary metabolites before solving; the feasible region is unchanged. +- The status is `infeasible`. A constraint is too tight, most often a closed uptake or a bound with the wrong sign. [Growth media and conditions](media.md) covers the media. +- The solve succeeds and growth is zero. The model is feasible and its optimum is zero, because a nutrient is missing or a gap blocks the biomass pseudoreaction. +- The same model gives different flux distributions. Alternative optima exist. pFBA selects one, and flux variability analysis (FVA) compares ranges of fluxes. +- `sol.f` is read as a negated objective. RAVEN minimises `-c'x` internally and negates the result before returning it, so `sol.f` is the growth rate itself. +- The coefficients of a model span many orders of magnitude. Badly scaled stoichiometry makes the optimum differ between solvers. `params.maxRatio` of `solveLP` splits every reaction whose coefficients span a larger ratio through auxiliary metabolites before solving, and the feasible region does not change. ::: ## See also -- [Getting started](getting-started.md), loading and inspecting a model. -- [Growth media and conditions](media.md), what the exchange bounds mean and how - to define a medium. -- [MATLAB vs Python](../matlab-vs-python.md), the full function mapping. +- [Getting started](getting-started.md): loading and inspecting a model. +- [Growth media and conditions](media.md): the meaning of the exchange bounds and the definition of a medium. +- [MATLAB vs Python](../matlab-vs-python.md): the full function mapping. diff --git a/docs/guide/fva.md b/docs/guide/fva.md index 71c7539..3f0408c 100644 --- a/docs/guide/fva.md +++ b/docs/guide/fva.md @@ -1,13 +1,13 @@ # Flux variability and alternative optima -A flux balance analysis (FBA) solution is one point in a space of equally optimal answers. Flux variability analysis (FVA) asks how much each reaction's flux can vary, given the objective. It distinguishes a flux the model requires from one it happened to report. +A flux balance analysis (FBA) solution is one point in a space of equally optimal solutions. Flux variability analysis (FVA) determines how far the flux of each reaction can vary within the feasible space, optionally with the objective held at a fraction of its optimum. It separates a flux that the model requires from a flux that the solver happened to return. ## Functions | MATLAB | Python | Purpose | |---|---|---| -| `getAllowedBounds` | `flux_variability_analysis` {bdg-link-secondary}`cobrapy ` | the range each reaction can take | -| `haveFlux` | `find_blocked_reactions` {bdg-link-secondary}`cobrapy ` | reactions that can carry no flux at all | +| `getAllowedBounds` | `flux_variability_analysis` {bdg-link-secondary}`cobrapy ` | the range of each reaction | +| `haveFlux` | `find_blocked_reactions` {bdg-link-secondary}`cobrapy ` | reactions that cannot carry any flux | | `solveLP` (`minFlux`) | `pfba` {bdg-link-secondary}`cobrapy ` | one representative optimum | | no equivalent | `loopless_solution` {bdg-link-secondary}`cobrapy ` | an optimum without thermodynamically infeasible loops | | no equivalent | `add_loopless` {bdg-link-secondary}`cobrapy ` | the constraints behind `loopless="fastSNP"` | @@ -15,7 +15,7 @@ A flux balance analysis (FBA) solution is one point in a space of equally optima ## Setup -The model is `smallYeast.yml`, growing on glucose and oxygen. Uptake in this model is a positive flux through a `=> metabolite` reaction, so the upper bound opens it. +The model is `smallYeast.yml`, growing on glucose and oxygen. Uptake in this model is a positive flux through a `=> metabolite` reaction, and the upper bound opens it. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -54,7 +54,7 @@ growth: 0.1222 /h ## Range of every reaction -With no further constraint, this asks how far each flux can move anywhere in the feasible space; the objective is free. +Without further constraints, the analysis determines how far each flux can move anywhere in the feasible space, and the objective is free. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -71,7 +71,7 @@ fprintf('%d reactions, %d fixed, widest span %.1f\n', ... 53 reactions, 1 fixed, widest span 1000.0 ``` -`getAllowedBounds` solves two linear programmes (LPs) per reaction, one minimising and one maximising, and runs them in parallel, so the first call in a session opens a parallel pool and reports how many workers it got. Pass `'runParallel', false` to keep it in the one process, which is quicker on a model this size and the only option without the Parallel Computing Toolbox. +`getAllowedBounds` solves two linear programmes (LPs) per reaction, one that minimises and one that maximises, and runs them in parallel. The first call in a session therefore opens a parallel pool and reports the number of workers. `'runParallel', false` keeps the calculation in one process, which is quicker for a model of this size and the sole option without the Parallel Computing Toolbox. ::: :::{tab-item} 🐍 Python :sync: python @@ -89,13 +89,13 @@ print(f"{len(span)} reactions, {(span < 1e-9).sum()} fixed, " 53 reactions, 1 fixed, widest span 1000.0 ``` -`fraction_of_optimum=0.0` makes this the same question that `getAllowedBounds` asks. Without it, cobrapy defaults to 1.0, which asks a different question (see [Range at the optimum](#range-at-the-optimum)). +`fraction_of_optimum=0.0` gives the question that `getAllowedBounds` answers. The default of cobrapy is 1.0, which asks a different question ([Range at the optimum](#range-at-the-optimum)). ::: :::: ## Range at the optimum -With growth held at its maximum, the analysis determines which fluxes are free to move, and how many become free again if growth only has to reach 90 % of it. +With the growth at its maximum, the analysis determines which fluxes can move. It also determines how many fluxes are free again when growth has to reach 90 % of the maximum. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -115,7 +115,7 @@ growth >= 100% of optimum: 51 reactions fixed growth >= 90% of optimum: 1 reactions fixed ``` -RAVEN has no `fraction_of_optimum` argument: constrain the objective reaction, then ask for the bounds. A lower bound is a floor, so `0.9 * sol.f` allows growth to take any value from 90 % of the optimum up to the optimum. +RAVEN has no `fraction_of_optimum` argument. The objective reaction is constrained first, and the bounds are requested afterwards. A lower bound is a floor, so `0.9 * sol.f` allows any growth from 90 % of the optimum up to the optimum. ::: :::{tab-item} 🐍 Python :sync: python @@ -134,13 +134,13 @@ growth >= 90% of optimum: 1 reactions fixed ::: :::: -At the optimum only `FRDS2` and `SDH` can move (see [Loops in flux ranges](#loops-in-flux-ranges)); everything else is determined. With growth allowed to drop to 90 %, only `ethIN`, which is closed, stays fixed: a 10 % margin frees nearly every flux. +At the optimum, `FRDS2` and `SDH` are the sole fluxes that can move ([Loops in flux ranges](#loops-in-flux-ranges)), and every other flux is determined. When growth may drop to 90 %, `ethIN` is the sole flux that stays fixed, and it is closed. A margin of 10 % frees nearly every flux. -Reactions whose span is zero at the optimum are the ones for which the model has no choice. Those are the predictions the model determines; a flux with a wide range at the optimum is an artefact of which vertex the solver returned. +A reaction with a span of zero at the optimum is a reaction for which the model has no choice, so these fluxes are the predictions that the model determines. A flux with a wide range at the optimum depends on the vertex that the solver returned. ## Loops in flux ranges -A reaction can show a wide range purely because it is part of a thermodynamically infeasible cycle, flux going round a loop with no net driving force. The widest span in [Range of every reaction](#range-of-every-reaction) was 1000, the model's default bound. That span is such a cycle. +A reaction can show a wide range because it is part of a thermodynamically infeasible cycle, a flux that goes around a loop with no net driving force. The widest span in [Range of every reaction](#range-of-every-reaction) was 1000, the default bound of the model, and it is such a cycle. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -166,7 +166,7 @@ widest: FRDS2 (1000.0) and SDH (1000.0) with FRDS2 shut: growth 0.1222 /h, widest span 4.0 ``` -RAVEN has no loopless FVA. The practical check is to shut one reaction of a suspected cycle and see whether anything downstream moves. Growth is unchanged, so those 1000 units of flux do no work. +RAVEN has no loopless FVA. The check is to close one reaction of a suspected cycle and to see whether anything downstream moves. Growth does not change, so the 1000 units of flux do no work. ::: :::{tab-item} 🐍 Python :sync: python @@ -190,13 +190,13 @@ inflated by loops: ['FRDS2', 'SDH'] ::: :::: -Both tabs reach the same number by different methods: shutting one arm of the cycle and adding loopless constraints each cut the widest range in the model from 1000 to 4. `FRDS2` and `SDH` catalyse the same interconversion in opposite directions. Together they form a cycle that can carry arbitrary flux while consuming and producing nothing. The pair therefore reaches the default bound of 1000, and the model's largest reported range is an artefact of stoichiometry and does not describe the organism. +The two methods give the same number. Closing one arm of the cycle and adding loopless constraints each reduce the widest range in the model from 1000 to 4. `FRDS2` and `SDH` catalyse the same interconversion in opposite directions. Together they form a cycle that carries any flux while it consumes and produces nothing. The pair therefore reaches the default bound of 1000, and the largest range of the model follows from the stoichiometry and does not describe the organism. -The `loopless` argument names the algorithm that forbids the cycle. `"fastSNP"` adds loopless constraints to the model and gives optimal bounds; `"cycleFreeFlux"` removes loops from each solution, which is quicker but is not guaranteed to find the tightest bounds. `loopless=True` works and means `"cycleFreeFlux"`, but it is deprecated; name the algorithm. `find_good_reactions` uses the same idea to decide which reactions to sample over, keeping a reaction only if its loopless range is non-trivial. +The `loopless` argument names the algorithm that forbids the cycle. `"fastSNP"` adds loopless constraints to the model and gives optimal bounds. `"cycleFreeFlux"` removes loops from each solution, which is faster and does not guarantee the tightest bounds. `loopless=True` works and means `"cycleFreeFlux"`, and it is deprecated, so the algorithm is named. `find_good_reactions` applies the same idea to select the reactions for sampling and keeps a reaction when its loopless range is not trivial. ## Representative solution -When a single flux distribution is needed (for a figure, or to compare two conditions), take one from parsimonious FBA (pFBA), not whatever the solver returns first. It carries no loop flux and is less arbitrary than the first vertex found, but it need not be unique either: where two pathways cost the same total flux, pFBA can return either. Report any single flux together with its range. +A single flux distribution is needed for a figure or for a comparison of two conditions. It is taken from parsimonious FBA (pFBA). The pFBA solution carries no loop flux and is less arbitrary than the first vertex that is found, and it need not be unique: when two pathways cost the same total flux, pFBA can return either. A single flux is reported together with its range. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -212,7 +212,7 @@ fprintf('growth: %.4f /h, total flux: %.1f\n', solPars.x(idx), sum(abs(solPars.x growth: 0.1222 /h, total flux: 20.9 ``` -`minFlux` minimises total absolute flux subject to the objective, which is parsimonious FBA. It is also the closest option in RAVEN to excluding loops: a cycle adds flux, so a parsimonious solution has no reason to carry one. +`minFlux` minimises the total absolute flux subject to the objective, which is parsimonious FBA. It is also the closest option in RAVEN to excluding loops, because a cycle adds flux and a parsimonious solution has no reason to carry one. ::: :::{tab-item} 🐍 Python :sync: python @@ -231,22 +231,22 @@ loopless 20.9 pFBA 20.9 ``` -All three agree here: this particular optimum happens to carry no loop flux, even though [Loops in flux ranges](#loops-in-flux-ranges) showed that the cycle exists. Nothing guaranteed that: the solver could equally have returned a vertex with 1000 units going round `FRDS2` and `SDH`, the failure that `loopless_solution` exists to prevent. +All three methods agree here, because this optimum carries no loop flux, although [Loops in flux ranges](#loops-in-flux-ranges) showed that the cycle exists. This was not guaranteed: the solver could have returned a vertex with 1000 units around `FRDS2` and `SDH`, which `loopless_solution` prevents. ::: :::: :::{admonition} Troubleshooting :class: warning -- `fraction_of_optimum` is not set: cobrapy defaults to `1.0`, ranges at the optimum. `getAllowedBounds` has no such notion and answers for the whole feasible space. The same call in the two toolboxes therefore asks different questions unless the call states which one is meant. -- A flux with a wide range is reported: if the range at the optimum is wide, the number in a results table is one of many equally good answers. -- A loop is mistaken for capacity: wide ranges on internal cycles are a property of the stoichiometry and not of the organism; see [Loops in flux ranges](#loops-in-flux-ranges). -- Loopless FVA is a mixed-integer linear programme (MILP): `loopless="fastSNP"` adds binary variables, so it is far slower than plain FVA and wants a good solver on anything larger than a toy model. -- FVA on a genome-scale model needs two LPs per reaction. cobrapy parallelises it, so `processes` matters; where spawning is blocked, set `Configuration().processes = 1` and expect it to take longer. +- `fraction_of_optimum` is not set. cobrapy defaults to `1.0` and returns the ranges at the optimum, and `getAllowedBounds` answers for the whole feasible space. The same call in the two toolboxes asks different questions unless the call states which one is meant. +- A flux with a wide range is reported. When the range at the optimum is wide, the number in a results table is one of many equally good solutions. +- A loop is mistaken for capacity. Wide ranges on internal cycles follow from the stoichiometry and do not describe the organism ([Loops in flux ranges](#loops-in-flux-ranges)). +- Loopless FVA is a mixed-integer linear programme (MILP). `loopless="fastSNP"` adds binary variables, so it is much slower than plain FVA and needs a good solver for anything larger than a small model. +- FVA on a genome-scale model needs two LPs per reaction. cobrapy parallelises it, so `processes` matters. Where spawning processes is blocked, set `Configuration().processes = 1`, and the run takes longer. ::: ## See also -- [Simulating growth with FBA](fba.md), the single solve this qualifies. -- [Deletions and essentiality](deletions.md), the other way to ask what the model depends on. -- [Quality control](quality-control.md), blocked reactions, the degenerate case of a zero range. +- [Simulating growth with FBA](fba.md): the single solve that FVA qualifies. +- [Deletions and essentiality](deletions.md): the other way to ask what the model depends on. +- [Quality control](quality-control.md): blocked reactions, the case of a range of zero. diff --git a/docs/guide/gap-filling.md b/docs/guide/gap-filling.md index f345c80..0349a82 100644 --- a/docs/guide/gap-filling.md +++ b/docs/guide/gap-filling.md @@ -1,23 +1,23 @@ # Gap-filling -A draft model always has holes: reactions that cannot carry flux because something upstream is missing, metabolites nothing produces, a biomass component the network cannot make. Gap-filling adds reactions from a template until the holes close. Deciding which holes are worth closing is a judgement. +A draft model has holes: reactions that cannot carry flux because an upstream step is missing, metabolites that nothing produces, or a biomass component that the network cannot make. Gap-filling adds reactions from a template until the holes close. Which holes are worth closing is a decision for the curator. ## Functions | MATLAB | Python | Purpose | |---|---|---| | `gapReport` | `check_model`, `analyse_topology` | find the holes first | -| `fillGaps` | `connect_blocked_reactions` | add template reactions so blocked reactions can carry flux | -| `gapFillFastLP` | `fill_gaps_fast_lp` | the linear programming (LP) formulation (fastGapFill / swiftGapFill) | -| `gapFillMILP` | `fill_gaps_milp` | the mixed-integer linear programming (MILP) formulation, when the LP is not enough | -| `gapFillTopological` | `analyse_topology` | connectivity, without solving an LP | -| `fitTasks` | `fill_tasks` | fill until a task passes; see [Metabolic tasks](tasks.md) | +| `fillGaps` | `connect_blocked_reactions` | add template reactions so that blocked reactions can carry flux | +| `gapFillFastLP` | `fill_gaps_fast_lp` | the linear programming (LP) formulation (fastGapFill and swiftGapFill) | +| `gapFillMILP` | `fill_gaps_milp` | the mixed-integer linear programming (MILP) formulation, for the cases where the LP is not sufficient | +| `gapFillTopological` | `analyse_topology` | connectivity without solving an LP | +| `fitTasks` | `fill_tasks` | fill until a task passes ([Metabolic tasks](tasks.md)) | -`gapFillTopological` and `analyse_topology` take their defaults from the model. The seeds are the metabolites of every reaction that has only substrates and a negative lower bound, or only products and a positive upper bound, whatever the number of metabolites; when the model has no such reaction, RAVEN and the toolbox warn that there is no seed. Reactions without substrates fire in the scope computation whatever their bounds, so closing an uptake reaction does not remove its metabolite from the scope. The targets are the metabolites of the first objective reaction, substrates and products. Without targets, a model with no objective reaction raises `RAVEN:badInput` in MATLAB and `ValueError` in Python. +`gapFillTopological` and `analyse_topology` take their defaults from the model. The seeds are the metabolites of every reaction that has substrates alone and a negative lower bound, or products alone and a positive upper bound, whatever the number of metabolites. When the model has no such reaction, RAVEN and the toolbox warn that there is no seed. Reactions without substrates fire in the scope computation whatever their bounds, so closing an uptake reaction does not remove its metabolite from the scope. The targets are the substrates and products of the first objective reaction. Without targets, a model without an objective reaction raises `RAVEN:badInput` in MATLAB and `ValueError` in Python. ## Setup -A gap-filling example needs a draft with holes and a template to fill them from. `smallYeast.yml` is both: its medium is opened, a copy is taken and one reaction is removed. The reaction that gap-filling adds back can be checked directly: it should be the reaction that was removed. +A gap-filling example needs a draft with holes and a template to fill them from. `smallYeast.yml` is both: its medium is opened, a copy is taken and one reaction is removed from the copy. Gap-filling should add back the reaction that was removed, which can be checked directly. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -58,13 +58,13 @@ print(f"draft {len(draft.reactions)} rxns, template {len(template.reactions)} rx draft 52 rxns, template 53 rxns ``` -With the medium as shipped (closed), `smallYeast` has 51 of its 53 reactions blocked, because nothing can get in and not because of gaps. Gap-filling a model in that state asks the wrong question. +With the medium as shipped (closed), 51 of the 53 reactions of `smallYeast` are blocked, because nothing can enter. Gap-filling a model in that state asks the wrong question. ::: :::: ## Finding holes -A model acquires reactions that nobody can justify when gap-filling starts without a check of the holes. Start from the reactions that cannot carry flux. +A model that is gap-filled without a check of its holes acquires reactions that nobody can justify. The check starts from the reactions that cannot carry flux. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -99,7 +99,7 @@ print(sorted(blocked)[:5]) ## Connectivity gap-filling -Connectivity gap-filling is structural: it finds the template reactions that, added to the draft, let a blocked reaction carry flux. It uses no objective and no growth, only connectivity. +Connectivity gap-filling is structural. It finds the template reactions that let a blocked reaction carry flux when they are added to the draft. It uses no objective and no growth, and it considers connectivity. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -132,19 +132,19 @@ print(f"{len(result.added_reactions)} added: " 1 added: ADH1 ``` -The reaction that is returned is the one that was taken out, in both toolboxes. That identity is the result to report. `fillGaps` also returns the newly connected reactions and those that cannot be connected, and their numbers are not stable: it solves a MILP, several solutions are equally optimal, and which one is returned can change with the machine or the thread count. Two CI runs on identical input reported 11 and 9 newly connected. +Both toolboxes return the reaction that was removed, and this identity is the result to report. `fillGaps` also returns the newly connected reactions and the reactions that cannot be connected. Their numbers are not stable, because the problem is a MILP with several equally optimal solutions, and the returned solution can change with the machine or the number of threads. Two CI runs on identical input reported 11 and 9 newly connected reactions. -Both are MILPs. `connect_blocked_reactions` runs on the GLPK that ships with cobrapy; `fillGaps` runs on RAVEN's bundled GLPK too, with a warning that recommends Gurobi (`setRavenSolver('gurobi')`). +Both are MILPs. `connect_blocked_reactions` runs on the GLPK that ships with cobrapy. `fillGaps` runs on the GLPK that ships with RAVEN and warns that Gurobi is recommended (`setRavenSolver('gurobi')`). -`allow_net_production` decides how strict the test is. A reaction can be blocked because a substrate is unavailable or because a product has nowhere to go; with net production allowed, only the first counts. Turning it off asks the harder question and usually adds more reactions. +`allow_net_production` sets how strict the test is. A reaction can be blocked because a substrate is unavailable or because a product has nowhere to go. With net production allowed, the first case counts. Turning it off asks the harder question and usually adds more reactions. ::: :::: ## LP formulation -`fillGaps` takes a cell array of template models or a model structure, while `gapFillFastLP` and `gapFillMILP` take a single struct. Passing a cell to these two fails with `Dot indexing is not supported for variables of this type`. +`fillGaps` takes a cell array of template models or a model structure, and `gapFillFastLP` and `gapFillMILP` take one struct. A cell array passed to these two fails with `Dot indexing is not supported for variables of this type`. -`fill_gaps_fast_lp` (fastGapFill, and its `swift` variant) solves one LP per blocked reaction: activate this reaction using as few template reactions as possible. It is the fast option, and the one that scales to a genome-scale draft. +`fill_gaps_fast_lp` (fastGapFill and its `swift` variant) solves one LP per blocked reaction. Each LP activates that reaction with as few template reactions as possible. It is fast and scales to a genome-scale draft. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -179,13 +179,13 @@ print(f"{len(result.added_reactions)} reactions added") 1 reactions added ``` -The two tabs add 5 and 1 (MATLAB adds 8 with Gurobi: the LP has alternative optima, and the solver picks one). They apply the same method to different inputs: the MATLAB call merges the whole template into the draft first, renaming every id it already has, which the rename warning in the MATLAB output lists. Its universal database therefore holds a second copy of the draft's own reactions, and the LP is free to rescue a blocked reaction through those duplicates. `fill_gaps_fast_lp` matches by id and treats the shared reactions as already present, so only the missing one is a candidate. On two models that differ, which is the real case, the gap between the counts closes. +The two examples add 5 and 1 reactions (MATLAB adds 8 with Gurobi: the LP has alternative optima, and the solver selects one). They apply the same method to different inputs. The MATLAB call first merges the whole template into the draft and renames every identifier that the draft already has, which the rename warning in the MATLAB output lists. Its universal database therefore holds a second copy of the reactions of the draft, and the LP can rescue a blocked reaction through those duplicates. `fill_gaps_fast_lp` matches by identifier and treats the shared reactions as present already, so the missing reaction is the sole candidate. When the template holds reactions that the draft lacks, which is the usual case, the gap between the counts closes. ::: :::: ## MILP formulation -When the LP formulation adds too much, or when the provably smallest set is wanted, the MILP formulation is the alternative. GLPK solves it at this size in both toolboxes (RAVEN warns and recommends Gurobi); on anything genome-scale it is slower by a wide margin, and Gurobi is the better choice. +The MILP formulation applies when the LP formulation adds too much or when a set that is provably the smallest is wanted. At this size GLPK solves it in both toolboxes (RAVEN warns and recommends Gurobi). On a genome-scale model it is much slower, and Gurobi is preferable. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -228,7 +228,7 @@ fill_gaps_milp: added 0 template reaction(s). ::: :::: -Zero is the right answer here: without `ADH1` the draft grows at 0.1222 /h, so it meets the default floor (10 % of the merged model's maximum, 0.02443 /h in both tools). The template repeats the draft's reactions, its uptake reactions included, so the merged model can take up twice the glucose and its maximum is 0.2443 /h. The MILP has nothing to add. To see a repair, remove a reaction that growth needs, such as hexokinase: +Zero is the correct result here. Without `ADH1` the draft grows at 0.1222 /h, which meets the default lower limit of 10 % of the maximum of the merged model (0.02443 /h in both tools). The template repeats the reactions of the draft, including its uptake reactions, so the merged model can take up twice the glucose and its maximum is 0.2443 /h. The MILP has nothing to add. A repair appears when a reaction that growth needs is removed, such as hexokinase: ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -276,11 +276,11 @@ filled growth 0.1222 ::: :::: -Hexokinase is added back. Both functions only add template reactions: the draft's own reactions keep their bounds, forced fluxes such as ATP maintenance included. A draft reaction that runs in the wrong direction is therefore not reversed to close a gap; it is a curation finding, so correct its bounds in the draft (see [Editing an existing model](editing.md)) and fill again. +Hexokinase is added back. Both functions add template reactions, and the reactions of the draft keep their bounds, including forced fluxes such as the ATP maintenance. A draft reaction that runs in the wrong direction is therefore not reversed to close a gap. It is a curation finding: the bounds are corrected in the draft ([Editing an existing model](editing.md)) and the filling is repeated. ## Filling towards a task -Connectivity gap-filling asks whether a reaction can carry flux. The other question, whether the model can perform a given function, is answered by filling against a task list, which is the form the question usually takes: a model that grows, or that produces a particular compound. +Connectivity gap-filling asks whether a reaction can carry flux. The question whether the model can perform a function, such as growing or producing a compound, is answered by filling against a task list. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -304,25 +304,25 @@ tasks = parseTaskList('tasks.txt'); from raven_toolbox.init import fill_tasks ``` -See [Metabolic tasks](tasks.md) for the task list itself, and [Context-specific models](init.md) for `fill_tasks` in its usual role, repairing a model that ftINIT has reduced. +[Metabolic tasks](tasks.md) describes the task list, and [Context-specific models](init.md) describes `fill_tasks` in its usual role, the repair of a model that ftINIT has reduced. -`fitTasks` fills one task at a time, so the order of the task list can change the result. Its `rxnScores` weights the candidates, taking negative scores only and maximising their sum, so a reaction with a score closer to zero is cheaper to add; this is how ftINIT prefers reactions its expression data supports. `gapFillMode` selects the back-end: `'merge'` combines the reference model with the draft for every task, and `'preMerged'` expects a reference that already contains the draft, which is the form ftINIT passes. +`fitTasks` fills one task at a time, so the order of the task list can change the result. `rxnScores` weights the candidates. It accepts negative scores and maximises their sum, so a reaction with a score closer to zero is cheaper to add. ftINIT uses this to prefer reactions that its expression data support. `gapFillMode` selects the back-end: `'merge'` combines the reference model with the draft for every task, and `'preMerged'` expects a reference that already contains the draft, which is the form that ftINIT passes. ::: :::: :::{admonition} Troubleshooting :class: warning -- Identifiers that do not line up. Gap-filling can only add what it can match. Two models built from different databases share almost no metabolite ids, and the result is a gap-filler that adds nothing, or adds duplicates. -- Filling every gap. Some blocked reactions are blocked because the organism cannot perform them. Each addition is a claim about biology, and `fillGaps` makes hundreds of them in one call. -- Net production hiding the problem. `allowNetProduction` makes more reactions look connectable by ignoring where the products go. It helps in early stages and misleads in later ones. -- MILP on a genome-scale model. Expect it to be slow, on GLPK most of all, and give it a time limit; see [Solvers and configuration](solvers.md). -- A draft reaction in the wrong direction. `gapFillMILP` and `fill_gaps_milp` do not change the draft's own reactions, so a wrong direction shows up as a fill that fails or adds a detour around it. -- Comparing added-reaction counts across toolboxes. The two sides differ in what they treat as a candidate, so the counts are only comparable when the template holds reactions that the draft lacks. +- The identifiers do not line up. Gap-filling adds what it can match. Two models built from different databases share almost no metabolite identifiers, and gap-filling then adds nothing or adds duplicates. +- Every gap is filled. Some reactions are blocked because the organism cannot perform them. Each addition is a claim about biology, and `fillGaps` makes hundreds of them in one call. +- Net production hides the problem. `allowNetProduction` makes more reactions look connectable because it ignores where the products go. It helps in early stages and misleads in later stages. +- A MILP runs on a genome-scale model. It is slow, most of all on GLPK, so set a time limit ([Solvers and configuration](solvers.md)). +- A draft reaction runs in the wrong direction. `gapFillMILP` and `fill_gaps_milp` do not change the reactions of the draft, so the wrong direction appears as a failed fill or as a detour around the reaction. +- The numbers of added reactions of the two toolboxes are compared. The toolboxes treat different reactions as candidates, so the counts are comparable when the template holds reactions that the draft lacks. ::: ## See also -- [Quality control](quality-control.md), finding the holes, and deciding which matter. -- [Metabolic tasks](tasks.md), the other way to say what the model must do. -- [Reading and writing models](io.md), saving the filled model with a record of what was added. +- [Quality control](quality-control.md): finding the holes and deciding which matter. +- [Metabolic tasks](tasks.md): the other way to state what the model must do. +- [Reading and writing models](io.md): saving the filled model with a record of what was added. diff --git a/docs/guide/getting-started.md b/docs/guide/getting-started.md index 43241b0..32c6c88 100644 --- a/docs/guide/getting-started.md +++ b/docs/guide/getting-started.md @@ -1,6 +1,6 @@ # Getting started -Load a model, see how large it is, and inspect a single reaction, metabolite and gene. Every later page assumes these operations. +The page covers loading a model, reporting its size and inspecting one reaction, one metabolite and one gene. The later pages use these operations. ## Functions @@ -8,16 +8,16 @@ Load a model, see how large it is, and inspect a single reaction, metabolite and |---|---|---| | `readYAMLmodel` | `read_yaml_model` | read a RAVEN YAML model | | `importModel` | `read_sbml_model` {bdg-link-secondary}`cobrapy ` | read an SBML model | -| `printModelStats` | model attributes {bdg-link-secondary}`cobrapy ` | how big the model is | -| `getIndexes` | `get_by_id` {bdg-link-secondary}`cobrapy ` | look something up by identifier | -| `constructEquations` | `Reaction.reaction` {bdg-link-secondary}`cobrapy ` | a reaction as a readable string | -| `getElementalBalance` | `check_mass_balance` {bdg-link-secondary}`cobrapy ` | is a reaction balanced | +| `printModelStats` | model attributes {bdg-link-secondary}`cobrapy ` | size of the model | +| `getIndexes` | `get_by_id` {bdg-link-secondary}`cobrapy ` | find an object by identifier | +| `constructEquations` | `Reaction.reaction` {bdg-link-secondary}`cobrapy ` | reaction equation as a string | +| `getElementalBalance` | `check_mass_balance` {bdg-link-secondary}`cobrapy ` | mass balance of a reaction | ## Setup -The examples use `smallYeast.yml`, a model of central carbon metabolism in yeast that ships with RAVEN. Download it from [`docs/data/smallYeast.yml`](../data/smallYeast.yml) and run everything from the directory that contains it. +The examples use `smallYeast.yml`, a model of the central carbon metabolism of yeast that ships with RAVEN. The file is [`docs/data/smallYeast.yml`](../data/smallYeast.yml), and the examples run in the directory that contains it. -Install one of the toolboxes first ([Installation](../installation/index.md)). Every page lists the files it needs under its Setup heading; they are all in [`docs/data/`](../data/README.md), or clone the repository and work in `raven-docs/docs/data`. The blocks on a page build on each other: run them in order in one MATLAB or Python session. +One of the toolboxes must be installed first ([Installation](../installation/index.md)). The Setup section of most pages names the files that the page needs. All of them are in [`docs/data/`](../data/README.md), which is also the working directory when the `raven-docs` repository is cloned. The blocks of a page build on each other and run in order in one MATLAB or Python session. ## Loading the model @@ -34,7 +34,7 @@ disp(model.id); smallYeast ``` -A RAVEN model is a MATLAB struct of parallel arrays. `model.rxns`, `model.mets` and `model.genes` hold the identifiers, and every other field lines up with one of them by position: `model.lb(4)` is the lower bound of the reaction named in `model.rxns{4}`. The stoichiometric matrix `model.S` has one row per metabolite and one column per reaction, in those same orders. Working with the model means working with indices into these arrays, so looking an identifier up comes first. +A RAVEN model is a MATLAB struct of parallel arrays. `model.rxns`, `model.mets` and `model.genes` hold the identifiers, and every other field is aligned with one of them by position: `model.lb(4)` is the lower bound of the reaction in `model.rxns{4}`. The stoichiometric matrix `model.S` has one row per metabolite and one column per reaction, in the order of `model.mets` and `model.rxns`. Most operations work on indices into these arrays, so an identifier is first converted to an index. ::: :::{tab-item} 🐍 Python :sync: python @@ -50,11 +50,11 @@ print(model.id) smallYeast ``` -In Python a RAVEN model is a `cobra.Model`. raven-toolbox defines no model class of its own, so every cobrapy tool, and anything else in the Python COBRA ecosystem, operates on it directly. Reactions, metabolites and genes are objects held in collections. Each object records what it is connected to, so there are no indices to keep aligned. +A RAVEN model in Python is a `cobra.Model`. raven-toolbox defines no model class, so cobrapy and the other tools of the Python COBRA ecosystem operate on the model directly. Reactions, metabolites and genes are objects in collections, and each object holds references to the objects it is connected to. No indices need to be kept aligned. ::: :::: -For SBML use `importModel` in MATLAB and cobrapy's `read_sbml_model` in Python; [Reading and writing models](io.md) covers every supported format. +SBML models are read with `importModel` in MATLAB and with `read_sbml_model` of cobrapy in Python. [Reading and writing models](io.md) covers all supported formats. ## Model size @@ -86,7 +86,7 @@ Unique metabolites 45 ** Unique reactions are defined as being biochemically unique (no compartmentalization) ``` -The per-compartment counts do not sum to the total, because a reaction or gene is counted in every compartment in which one of its metabolites is located; a transport reaction appears under both. Unique means biochemically unique, ignoring compartments: 52 metabolites reduce to 45 because seven of them exist in both the cytosol and the mitochondrion. The same 45 is what [Combining and simplifying models](combining.md) arrives at when it flattens the compartments. +A reaction or gene is counted in every compartment in which one of its metabolites is located, and a transport reaction is therefore counted under two compartments. The per-compartment counts do not add up to the total for this reason. A unique metabolite is counted once across compartments: 52 metabolites reduce to 45, because 7 of them exist in both the cytosol and the mitochondrion. [Combining and simplifying models](combining.md) arrives at the same 45 when it merges the compartments. ::: :::{tab-item} 🐍 Python :sync: python @@ -99,13 +99,13 @@ print(len(model.reactions), len(model.metabolites), len(model.genes)) 53 52 61 ``` -There is no equivalent of `printModelStats`. The collections are attributes, so their lengths are the totals, and per-compartment counts come from filtering them. After a solve, `model.summary()` prints the exchange fluxes and the objective. +Python has no equivalent of `printModelStats`. The collections are attributes, so their lengths are the totals, and the counts per compartment are obtained by filtering the collections. After a solve, `model.summary()` prints the exchange fluxes and the objective. ::: :::: ## Inspecting a reaction -Glucose-6-phosphate isomerase, `PGI`, shows most of what a reaction carries: it is reversible, it has a gene association, and it should be mass balanced. +Glucose-6-phosphate isomerase (`PGI`) is reversible, has a gene association and is expected to be mass balanced, which covers most of what a reaction carries. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -127,7 +127,7 @@ bounds: [-1000 1000] YBR196C ``` -`getIndexes` turns an identifier into the position used by every other field. `constructEquations` builds the readable equation from the stoichiometric column, substituting metabolite names for identifiers, so the equation reads in words. +`getIndexes` converts an identifier to the position used by every other field. `constructEquations` builds the equation from the stoichiometric column of that reaction and replaces the metabolite identifiers by metabolite names. ::: :::{tab-item} 🐍 Python :sync: python @@ -147,11 +147,11 @@ bounds: (-1000.0, 1000.0) reversible: True YBR196C ``` -`Reaction.reaction` substitutes metabolite identifiers, so the same reaction reads differently from the MATLAB equation above. Both describe the same stoichiometry. +`Reaction.reaction` writes metabolite identifiers with their compartment suffix, where the MATLAB equation above uses metabolite names. Both describe the same stoichiometry. ::: :::: -The bounds carry the direction. A lower bound of −1000 means the reaction may run backwards; a lower bound of zero means it may not. cobrapy derives `reversibility` from the bounds each time it is asked, so the two cannot disagree. RAVEN stores an explicit `model.rev` field alongside the bounds, so one can be set without the other; [Model structure and identifiers](model-structure.md) covers what to do about that. +The bounds define the direction. A lower bound of −1000 allows the reaction to run backwards, and a lower bound of zero does not. cobrapy derives `reversibility` from the bounds whenever it is read, so the two cannot disagree. RAVEN stores a separate `model.rev` field next to the bounds, and one can be changed without the other. [Model structure and identifiers](model-structure.md) describes how to handle this. ## Inspecting a metabolite @@ -171,7 +171,7 @@ alpha-D-glucose 6-phosphate (C6H13O9P) in compartment c takes part in 4 reactions ``` -The reaction count comes from counting non-zero entries in the metabolite's row of `model.S`. `model.metComps` holds an index into `model.comps`, not the compartment letter itself, so the compartment needs two lookups. +The number of reactions is the number of non-zero entries in the row of the metabolite in `model.S`. `model.metComps` holds an index into `model.comps`, so the compartment takes two lookups. ::: :::{tab-item} 🐍 Python :sync: python @@ -187,11 +187,11 @@ alpha-D-glucose 6-phosphate (C6H13O9P) in compartment c takes part in 4 reactions ``` -`Metabolite.reactions` is maintained by the model as reactions are added and removed, so it needs no matrix lookup. +The model updates `Metabolite.reactions` when reactions are added or removed, so no matrix lookup is needed. ::: :::: -The formula makes a mass-balance check possible. A metabolite without one leaves every reaction it appears in undecidable, which is the distinction drawn in [Mass balance of a reaction](#mass-balance-of-a-reaction). +The formula of a metabolite is required for a mass-balance check. A reaction that contains a metabolite without a formula cannot be decided, and it is a different case from an unbalanced reaction ([Mass balance of a reaction](#mass-balance-of-a-reaction)). ## Inspecting a gene @@ -209,7 +209,7 @@ fprintf('%s -> %s\n', model.geneShortNames{gi}, strjoin(rxnIds, ', ')); PGI1 -> PGI ``` -`model.rxnGeneMat` is a reactions-by-genes incidence matrix; a non-zero entry means that gene appears in that reaction's rule. It records which genes are involved; how they combine is recorded only in `model.grRules`, as `and` and `or` in text. +`model.rxnGeneMat` is an incidence matrix of reactions by genes. A non-zero entry means that the gene appears in the rule of the reaction. The matrix does not record how the genes combine; `model.grRules` is the sole field that records that, as `and` and `or` in text. ::: :::{tab-item} 🐍 Python :sync: python @@ -225,11 +225,11 @@ PGI1 -> ['PGI'] ::: :::: -Both toolboxes distinguish the identifier from the name. `YBR196C` is the systematic identifier and `PGI1` the standard gene name; the model is keyed on the former. +Both toolboxes distinguish the identifier of a gene from its name. `YBR196C` is the systematic identifier and `PGI1` is the standard gene name. The model is keyed on the identifier. ## Mass balance of a reaction -Draft models routinely contain reactions that do not balance. Checking one is the same operation in both toolboxes; doing it for a whole model is covered in [Quality control](quality-control.md). +Draft models contain reactions that do not balance. The check of one reaction is the same in both toolboxes. [Quality control](quality-control.md) checks a whole model. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -244,7 +244,7 @@ fprintf('elemental %d, charge %d\n', balance.balanceStatus, balance.chargeStatus elemental 1, charge -1 ``` -The two statuses are reported separately, and each has three values: `1` balanced, `0` unbalanced, `-1` undecidable. `PGI` balances elementally and its charge is undecidable: the metabolites in this model carry formulas but no charges, so there is nothing to sum. That is a different statement from "the charges do not balance": one says the check could not run, the other says it failed. +The element balance and the charge balance are reported separately, each as `1` (balanced), `0` (unbalanced), `-1` (undecidable) or `-2` (formula that cannot be parsed). `PGI` is balanced elementally, and its charge balance is undecidable because the metabolites of this model have formulas but no charges, so there is nothing to sum. An undecidable balance means that the check could not run, and a failed charge balance means that the charges differ. ::: :::{tab-item} 🐍 Python :sync: python @@ -257,20 +257,20 @@ print(pgi.check_mass_balance()) {} ``` -An empty dictionary means the reaction balances. Anything listed is an element, or `charge`, mapped to the amount by which the two sides differ, so the result says both whether and by how much. cobrapy reports charge in the same dictionary, not separately, and omits what it cannot decide, so an empty result covers both "balanced" and "nothing to check". +An empty dictionary means that the reaction balances. Each entry maps an element, or `charge`, to the difference between the two sides, so the result gives the size of an imbalance as well as its presence. cobrapy reports the charge in the same dictionary and omits what it cannot decide. An empty dictionary therefore covers both a balanced reaction and a reaction with nothing to check. ::: :::: :::{admonition} Troubleshooting :class: warning -- A lookup fails. Identifiers are case-sensitive and carry the compartment suffix (`G6P_c`, not `G6P`). Neither toolbox returns a sentinel for a name it cannot find: `getIndexes` errors with `Could not find object 'X' in the model` and `get_by_id` raises `KeyError`, so a typo stops the script; nothing else is indexed without warning. -- The model loads but nothing grows. In `smallYeast.yml` every uptake reaction is closed (`glcIN` has bounds `[0 0]`). Opening a medium is covered in [Growth media and conditions](media.md). -- Gene identifiers differ between model and FASTA. Systematic names (`YBR196C`) and standard names (`PGI1`) are not interchangeable; RAVEN stores the systematic name as the identifier and the standard name as the gene name. +- A lookup fails. Identifiers are case-sensitive and carry the compartment suffix (`G6P_c`). Neither toolbox returns a placeholder for an identifier that is absent: `getIndexes` raises the error `Could not find object 'X' in the model`, and `get_by_id` raises `KeyError`. +- The model loads and does not grow. In `smallYeast.yml` every uptake reaction is closed (`glcIN` has bounds `[0 0]`). [Growth media and conditions](media.md) opens a medium. +- Gene identifiers differ between the model and a FASTA file. RAVEN stores the systematic name (`YBR196C`) as the identifier and the standard name (`PGI1`) as the gene name, and the two are not interchangeable. ::: ## See also -- [Guide overview](index.md), the other pages, and the planned pages. -- [Comparison of RAVEN and raven-toolbox](../raven3-vs-raven-toolbox.md), what each toolbox has, and where cobrapy takes over. -- [API reference](../api/index.md), every function in both toolboxes. +- [Guide overview](index.md): the other pages. +- [Comparison of RAVEN and raven-toolbox](../raven3-vs-raven-toolbox.md): the features of each toolbox and the functions that cobrapy provides. +- [API reference](../api/index.md): all functions of both toolboxes. diff --git a/docs/guide/homology.md b/docs/guide/homology.md index efba081..4a29186 100644 --- a/docs/guide/homology.md +++ b/docs/guide/homology.md @@ -1,23 +1,23 @@ # Reconstruction from homology -Homology-based reconstruction builds a draft model for an organism that has none, using a curated model of a related organism as the source of reactions. Genes in the new organism are matched to genes in the template by sequence similarity, and each template reaction whose genes have an accepted match is copied across. +Homology-based reconstruction builds a draft model for an organism without one, with a curated model of a related organism as the source of reactions. Genes of the new organism are matched to genes of the template by sequence similarity, and a template reaction is copied when its genes have an accepted match. -The work happens in two steps: a sequence search that produces a table of hits, and a transfer step that decides which of those hits are good enough to carry a reaction. The cut-offs used by the second step determine the size and the reliability of the draft and encode the method's assumptions. +The reconstruction has two steps. A sequence search produces a table of hits, and a transfer step selects the hits that are sufficient to carry a reaction. The cut-offs of the transfer step determine the size and the reliability of the draft. ## Functions | MATLAB | Python | Purpose | |---|---|---| | `getBlast` | `run_blast` | bidirectional BLASTP between two proteomes | -| `getDiamond` | `run_diamond` | the same, with DIAMOND: faster, less sensitive | -| `getModelFromHomology` | `get_model_from_homology` | carry reactions across on the hits | -| `makeFakeBlastStructure` | `make_ortholog_hits` | supply orthologs that are already available | +| `getDiamond` | `run_diamond` | the same search with DIAMOND, which is faster and less sensitive | +| `getModelFromHomology` | `get_model_from_homology` | transfer reactions on the basis of the hits | +| `makeFakeBlastStructure` | `make_ortholog_hits` | supply orthologs that are available already | ## Setup -The setup uses two proteomes and a template model. The template is `smallYeast.yml` (53 reactions, 61 genes), and `sce-template.faa` holds the sequences of exactly those 61 *S. cerevisiae* genes. The organism being reconstructed is *Hansenula polymorpha*, whose full 5177-protein proteome is in `hanpo.faa`. All three files are in [`docs/data/`](../data/README.md): [`smallYeast.yml`](../data/smallYeast.yml), [`sce-template.faa`](../data/sce-template.faa) and [`hanpo.faa`](../data/hanpo.faa). +The setup uses two proteomes and a template model. The template is `smallYeast.yml` with 53 reactions and 61 genes, and `sce-template.faa` holds the sequences of these 61 *S. cerevisiae* genes. The organism to reconstruct is *Hansenula polymorpha*, whose proteome of 5177 proteins is in `hanpo.faa`. The three files are in [`docs/data/`](../data/README.md): [`smallYeast.yml`](../data/smallYeast.yml), [`sce-template.faa`](../data/sce-template.faa) and [`hanpo.faa`](../data/hanpo.faa). -The template model's id must match the id used for the BLAST. The transfer step looks up each hit's source organism by that id to find which template model the reaction should come from, so a mismatch leaves every hit unattributable. +The identifier of the template model must match the identifier used for the BLAST. The transfer step looks up the source organism of each hit by that identifier to find the template model of the reaction, and a mismatch leaves every hit without a source. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -55,9 +55,9 @@ template: 53 rxns, 61 genes ## BLAST in both directions -Both toolboxes run BLASTP twice: the new organism's proteome against the template's, and the template's against the new organism's. +Both toolboxes run BLASTP twice: the proteome of the new organism against the proteome of the template, and the proteome of the template against the proteome of the new organism. -The second direction makes orthology testable. A one-directional search gives, for each new gene, the template gene it resembles most, but the most similar sequence is not necessarily the corresponding one. A gene that has been duplicated in the template, or a conserved domain shared across a family, attracts hits from genes that do a different job. Searching both ways lets the transfer step ask whether two genes pick each other, which is a much stronger claim than either picking the other alone. +The second direction makes orthology testable. A search in one direction gives for each new gene the template gene that it resembles most, and the most similar sequence is not always the corresponding gene. A gene duplicated in the template, or a conserved domain shared across a family, attracts hits from genes with a different function. With both directions, the transfer step tests whether two genes select each other, which supports orthology more strongly than a selection in one direction. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -98,17 +98,17 @@ sce hanpo 159 ::: :::: -The two directions return different counts because they ask different questions, and neither count is of genes: each line of BLAST output is one alignment. The 159 alignments in the template-to-*H. polymorpha* direction come from 60 of the 61 template genes, and the 178 in the other direction from 106 *H. polymorpha* genes. Neither number is the number of orthologs, which the next step decides from the pairs that appear in both directions. +The two directions return different counts because they answer different questions, and neither count is a count of genes, as each line of BLAST output is one alignment. The 159 alignments from the template to *H. polymorpha* come from 60 of the 61 template genes, and the 178 alignments in the other direction come from 106 *H. polymorpha* genes. Neither number is the number of orthologs. The next step decides that from the pairs that occur in both directions. -The shape of the result differs between the toolboxes. RAVEN returns a struct array with one entry per direction, each carrying `fromId`, `toId`, `fromGenes`, `toGenes` and per-hit `evalue`, `aligLen` and `identity` vectors. raven-toolbox returns a single `pandas` DataFrame with `from_id` and `to_id` columns, so both directions are in one table and can be filtered with ordinary DataFrame operations before being passed on. +The result has a different shape in the two toolboxes. RAVEN returns a struct array with one entry per direction. Each entry has `fromId`, `toId`, `fromGenes`, `toGenes` and per-hit vectors `evalue`, `aligLen` and `identity`. raven-toolbox returns one `pandas` DataFrame with the columns `from_id` and `to_id`, which holds both directions and can be filtered with DataFrame operations before it is passed on. -Both call the same BLAST+ executables with the same parameters, so the hit counts agree. Neither ships those executables. RAVEN downloads them per platform on first use and caches them; raven-toolbox uses `RAVEN_PYTHON_BLASTP` if set, then a `blastp` on the `PATH`, and downloads only when neither exists. A first reconstruction that downloads needs network access. +Both call the same BLAST+ executables with the same parameters, so the hit counts agree. Neither toolbox ships the executables. RAVEN downloads them for the platform on first use and caches them. raven-toolbox uses `RAVEN_PYTHON_BLASTP` if it is set, then a `blastp` on the `PATH`, and downloads when neither exists. A first reconstruction that downloads needs network access. -Without `threads`, `run_blast`, `run_diamond` and `run_supporth` use every CPU the process may run on. In MATLAB, `getBlast`, `getDiamond` and `runSuppOrth` use the cores assigned to MATLAB (`feature('numcores')`); `runSuppOrth` is the one of the three with a `threads` option. +Without `threads`, `run_blast`, `run_diamond` and `run_supporth` use every CPU that the process can run on. In MATLAB, `getBlast`, `getDiamond` and `runSuppOrth` use the cores that are assigned to MATLAB (`feature('numcores')`), and `runSuppOrth` is the sole function with a `threads` option. -`getDiamond` and `run_diamond` are drop-in alternatives that search with DIAMOND instead. DIAMOND ran with `--more-sensitive` (the `run_diamond` default) and was 11.7 and 14.3 times faster than BLASTP on two full proteome pairs (12 s against 140 s and 14 s against 200 s in [Homology cut-off calibration](../parameter-tuning/studies/homology-cutoff-calibration.md)). It returned fewer hits, 32,192 against 54,513 for *K. lactis* against *S. cerevisiae*. After the default filters (identity 40, length 200, E-value 1e-30) the two aligners kept 87.9 to 92.3 % of the same pairs on four targets (Jaccard index). +`getDiamond` and `run_diamond` are alternatives that search with DIAMOND. With `--more-sensitive`, the default of `run_diamond`, DIAMOND was 11.7 and 14.3 times faster than BLASTP on two complete proteome pairs (12 s against 140 s, and 14 s against 200 s, in [Homology cut-off calibration](../parameter-tuning/studies/homology-cutoff-calibration.md)). DIAMOND returned fewer hits: 32,192 against 54,513 for *K. lactis* against *S. cerevisiae*. After the filters identity 40, length 200 and E-value 1e-30, the two aligners kept 87.9 to 92.3 % of the same pairs on four targets (Jaccard index). -`makeFakeBlastStructure` and `make_ortholog_hits` wrap orthology assignments from another source (OrthoFinder, OMA, a published table) in the structure the transfer step expects, so no search is run. +`makeFakeBlastStructure` and `make_ortholog_hits` put orthology assignments from another source (OrthoFinder, OMA or a published table) into the structure that the transfer step expects, and no search is run. ## From hits to a draft @@ -145,25 +145,31 @@ draft: 37 rxns, 49 mets, 54 genes ::: :::: -The transfer works reaction by reaction. For each reaction in the template, the genes named in its gene-reaction rule are looked up in the hit table; a template gene is replaced by its accepted counterpart in the new organism, and the reaction is carried over if the rule still resolves to something satisfiable after the substitution. A reaction requiring two subunits is dropped when only one of them has a counterpart; a reaction with two isozymes survives on either. The metabolites a carried reaction needs come with it. The draft therefore has 49 metabolites against the template's 52, because the three that appear only in dropped reactions are not created. +The transfer works reaction by reaction. For each reaction of the template, the genes in its gene rule are looked up in the hit table. A template gene is replaced by its accepted counterpart in the new organism, and the reaction is carried over when the rule can be satisfied after the substitution. A reaction that needs two subunits is kept with a placeholder gene (`OLD__`) for the missing subunit under the default `complexPolicy` (`complex_policy` in Python) of `flag`, and the policy `drop` removes it. A reaction with two isozymes is kept when either has a counterpart. The metabolites that a carried reaction needs are carried with it. The draft has 49 metabolites against 52 in the template, because the three metabolites that occur in dropped reactions alone are not created. -The `Standardizing grRules` line is RAVEN rewriting the template's rules into a canonical form before substituting into them; the same rule written two ways is then handled identically. +The line `Standardizing grRules` shows RAVEN rewriting the rules of the template in a canonical form before the substitution. One rule written in two ways is then handled identically. -`get_model_from_homology` returns a `HomologyResult`, which holds more than the model. `.model` is the draft; `.gene_map` maps, per template model, each template gene to the new genes that replace it (`{model_id: {template_gene: [new_gene, ...]}}`), which traces a reaction back to the evidence that put it there; and `.candidates`, populated when `review_identity=` is given, collects reactions that failed the identity cut-off but came within the value given. Those are near-misses that a threshold rejected, not reactions without a hit. RAVEN returns the draft and its hit genes as two separate outputs. +`get_model_from_homology` returns a `HomologyResult`, which holds more than the model: -The draft is smaller than the template because reactions whose genes have no accepted counterpart are not carried over. That is the intended behaviour and also the main source of error: a reaction left out because no hit passed the cut-offs looks the same as a reaction the organism lacks. +- `.model` is the draft. +- `.gene_map` maps, per template model, each template gene to the new genes that replace it (`{model_id: {template_gene: [new_gene, ...]}}`). It traces a reaction back to the evidence that placed it in the draft. +- `.candidates` is filled when `review_identity=` is given. It collects the reactions that failed the identity cut-off and came within the given value, so these are near misses that a threshold rejected, and they are not reactions without a hit. + +RAVEN returns the draft and its hit genes as two separate outputs. + +The draft is smaller than the template, because reactions whose genes have no accepted counterpart are not carried over. This is the intended behaviour and also the main source of error, as a reaction left out because no hit passed the cut-offs looks the same as a reaction that the organism lacks. ## Hit cut-offs -Three cut-offs control which hits are accepted, and tightening any of them shrinks the draft. +Three cut-offs select the accepted hits, and tightening any of them reduces the size of the draft. -`maxE` / `max_evalue` (`1e-30`) is the maximum BLAST E-value, the number of hits of at least this quality expected by chance in a database this size. The default is strict by BLAST standards, where `1e-5` is a common threshold, because transferring a reaction on a marginal hit adds a claim about metabolism that nothing downstream questions. In four organisms the F0.5 score against KEGG was identical from `1e-4` to `1e-30`, was lower by up to 0.006 at `1e-50` and by 0.020 to 0.063 at `1e-100` ([Homology cut-off calibration](../parameter-tuning/studies/homology-cutoff-calibration.md)). +`maxE` / `max_evalue` (`1e-30`) is the maximum BLAST E-value, the number of hits of at least this quality that is expected by chance in a database of this size. The default is strict for BLAST, where `1e-5` is a common threshold, and it keeps the value of RAVEN 2. In four organisms, the F0.5 score against KEGG was identical from `1e-4` to `1e-30`, up to 0.006 lower at `1e-50` and 0.020 to 0.063 lower at `1e-100` ([Homology cut-off calibration](../parameter-tuning/studies/homology-cutoff-calibration.md)). -`minLen` / `min_align_len` (`100`) is the minimum aligned length in residues. It rejects hits that align well over a short conserved domain while the rest of the protein is unrelated. The value was measured against KEGG orthology assignments across four organisms ([Homology cut-off calibration](../parameter-tuning/studies/homology-cutoff-calibration.md)): 50 and 100 gave the same score, 150 a score at most 0.003 lower, and 200 and 300 lower scores. At 200 the loss came mainly from recall; precision and recall were not reported for 300. +`minLen` / `min_align_len` (`100`) is the minimum aligned length in residues. It rejects hits that align over a short conserved domain while the rest of the protein is unrelated. The value was measured against KEGG orthology assignments in four organisms ([Homology cut-off calibration](../parameter-tuning/studies/homology-cutoff-calibration.md)). The values 50 and 100 gave the same score, 150 gave a score that was at most 0.003 lower, and 200 and 300 gave lower scores. At 200 the loss came mainly from recall, and precision and recall were not reported for 300. -`minIde` / `min_identity` (`40`) is the minimum percentage identity across the alignment. +`minIde` / `min_identity` (`40`) is the minimum percentage of identical residues across the alignment. -A fourth parameter, `strictness`, decides how much agreement between the two BLAST directions is required, from accepting any hit that passes the cut-offs up to requiring reciprocal best hits. At its strictest setting each gene keeps only its best hits by bitscore in each direction, all of them when several tie; bitscore, unlike the E-value, does not depend on the size of the database searched, so the choice does not shift when a proteome is updated. +A fourth parameter, `strictness`, sets how much agreement between the two BLAST directions is required. It has three levels. Level 1, the default, requires a hit that passes the cut-offs in both directions. Level 2 requires a hit in one direction, and level 3 keeps the best hit per gene in each direction. At the strictest setting, each gene keeps its best hits by bitscore in each direction, and all of them when several tie. The bitscore, unlike the E-value, does not depend on the size of the searched database, so the choice does not shift when a proteome is updated. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -195,17 +201,17 @@ strict draft: 32 rxns, 46 genes ::: :::: -Raising the two thresholds removes five reactions and eight genes. Whether that is an improvement depends on what the draft is for: a model that is curated by hand benefits from the extra candidates, since a wrong reaction is easier to spot than a missing one, while a model used directly for prediction is better with fewer and better-supported reactions. +The two stricter thresholds remove five reactions and eight genes. Whether this is an improvement depends on the purpose of the draft. A draft that is curated by hand benefits from the additional candidates, as a wrong reaction is easier to notice than a missing one. A draft that is used for prediction directly is better with fewer and better supported reactions. :::{admonition} Reproducing a RAVEN 2 reconstruction :class: note -Some of these defaults differ from the values RAVEN 2 used, so the same script can produce a different draft under RAVEN 3. Set them explicitly, or see [Migrating from RAVEN 2](../raven3-migration.md). +Some of these defaults differ from the values of RAVEN 2, so the same script can give a different draft in RAVEN 3. The values should be set explicitly, or [Migrating from RAVEN 2](../raven3-migration.md) consulted. ::: ## Contents of a draft -What comes out of this step has reactions, metabolites and genes, and nothing else. There is no biomass reaction unless a template reaction happened to carry one, no exchange reactions, and no guarantee that anything can carry flux. +The result of this step has reactions, metabolites and genes. It has no biomass reaction unless a template reaction carried one, no exchange reactions, and no guarantee that any reaction carries flux. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -246,21 +252,21 @@ reactions that can carry flux: 0 of 37 ::: :::: -None of the 37 reactions can carry flux. This is not a defect in the draft: with no exchange reactions nothing can enter or leave, so the only flux left is internal cycling, and this draft has none. The number says nothing about the quality of the reconstruction and becomes informative only once a medium is defined. +None of the 37 reactions carries flux. This does not indicate a defect of the draft. Without exchange reactions nothing enters or leaves the model, and the draft has no internal cycle, so no flux is possible. The number says nothing about the quality of the reconstruction, and it becomes informative after a medium is defined. -A homology draft is a set of claims about which reactions the organism has. Turning it into a model means giving it a medium ([Growth media and conditions](media.md)), closing the gaps that stop it producing biomass ([Gap-filling](gap-filling.md)), and checking it against what the organism is known to do ([Metabolic tasks](tasks.md)). The [GEM reconstruction protocol](../protocol/index.md) follows that path for *H. polymorpha* at full scale. +A homology draft is a set of claims about the reactions of the organism. Three steps turn it into a model: a medium ([Growth media and conditions](media.md)), gap-filling to close the gaps that prevent biomass production ([Gap-filling](gap-filling.md)), and a check against the known behaviour of the organism ([Metabolic tasks](tasks.md)). The [GEM reconstruction protocol](../protocol/index.md) follows this path for *H. polymorpha* at full scale. :::{admonition} Troubleshooting :class: warning -- Identifiers that do not match. The FASTA headers must carry the same gene ids as the template model's `genes`. A mismatch produces a draft with no reactions and no error. -- One template, one organism's biases. Every reaction in the draft comes from the template, so anything the template lacks the draft cannot have. Several templates, with `preferredOrder`, spread that risk. -- Reading absence as evidence. A reaction left out means no acceptable hit was found, which does not show that the organism lacks the capability. Sequencing gaps, divergent sequences and short proteins all look the same here. -- Full proteomes are slow. The example finishes in seconds because the template proteome has 61 sequences. Two complete proteomes take minutes to hours with BLASTP; `getDiamond` and `run_diamond` are the faster alternative. +- The identifiers do not match. The headers of the FASTA file carry the gene identifiers of the `genes` field of the template model. A mismatch gives a draft without reactions and without an error. +- One template carries the bias of one organism. Every reaction of the draft comes from the template, so the draft cannot have anything that the template lacks. Several templates, ordered with `preferredOrder`, spread the risk. +- Absence is read as evidence. A missing reaction means that no acceptable hit was found, and it does not show that the organism lacks the capability. Sequencing gaps, divergent sequences and short proteins look the same. +- Complete proteomes are slow. The example finishes in seconds because the template proteome has 61 sequences. Two complete proteomes take minutes to hours with BLASTP, and `getDiamond` and `run_diamond` are faster. ::: ## See also -- [Gap-filling](gap-filling.md), the usual next step, and the one that decides what the draft is missing. -- [Context-specific models](init.md), cutting a model down by evidence, where homology builds one up. -- [Comparing models](comparing.md), checking a draft against a curated model of the same organism. +- [Gap-filling](gap-filling.md): the next step, which decides what the draft lacks. +- [Context-specific models](init.md): reducing a model by evidence, where homology builds a model up. +- [Comparing models](comparing.md): comparing a draft with a curated model of the same organism. diff --git a/docs/guide/index.md b/docs/guide/index.md index 5c749b6..05c38ec 100644 --- a/docs/guide/index.md +++ b/docs/guide/index.md @@ -1,84 +1,84 @@ # Guide -The guide consists of short, task-focused pages: one job per page, three to eight functions, with MATLAB and Python side by side. Start at [Getting started](getting-started.md) and continue in order, or go directly to the page for a task. +Each page of the guide covers one task, with MATLAB and Python examples side by side. The pages can be read in the order below or one at a time; [Getting started](getting-started.md) is the first. -The guide ends with two older sets of material: a complete [reconstruction protocol](../protocol/index.md) for *H. polymorpha*, followed from start to finish, and the [legacy tutorials](../tutorials/index.md) from the original RAVEN paper. Both are MATLAB only. +The [reconstruction protocol](../protocol/index.md) for *H. polymorpha* and the [legacy tutorials](../tutorials/index.md) from the original RAVEN paper are separate from the task pages. Both use MATLAB. ## Functions from cobrapy -Functions that come from cobrapy, not from raven-toolbox, carry a {bdg-link-secondary}`cobrapy ` badge, their import line shows where they live (`from cobra.io import read_sbml_model`), and the badge links to the cobrapy documentation. The [function mapping](../matlab-vs-python.md) lists every such function. +A function that comes from cobrapy carries a {bdg-link-secondary}`cobrapy ` badge that links to the cobrapy documentation, and its import line shows where it lives (`from cobra.io import read_sbml_model`). The [function mapping](../matlab-vs-python.md) lists all of these functions. ## Executed examples -The code on the task pages is run, MATLAB and Python separately, by the [Guide examples](https://github.com/CompMetEng/raven-docs/actions/workflows/examples.yml) workflow whenever a guide page, the example data or the runner changes, and once a week against the latest RAVEN and raven-toolbox. A run fails when a block errors or prints something other than the output shown beneath it. The blocks on a page run in order in one session, so later blocks use what earlier ones made: start from the Setup section of the page. +The [Guide examples](https://github.com/CompMetEng/raven-docs/actions/workflows/examples.yml) workflow runs the code on the task pages, MATLAB and Python separately. It starts when a guide page, the example data or the runner changes, and once a week against the latest RAVEN and raven-toolbox. A run fails when a block raises an error or prints output that differs from the output shown beneath it. The blocks of a page run in order in one session, so a block uses what the preceding blocks made. Reading a page starts at its Setup section. -Not executed: [Reconstruction from KEGG](kegg.md), which downloads tens of megabytes and takes minutes, and the Human-GEM steps of [Context-specific models with ftINIT](init.md), which take hours. A few other blocks are marked on their page as not run, with the reason. Both pages quote the run their numbers came from. The protocol and the legacy tutorials are MATLAB-only and are not executed. +One page and part of another are not executed. [Reconstruction from KEGG](kegg.md) downloads tens of megabytes and takes minutes, and the Human-GEM steps of [Context-specific models with ftINIT](init.md) take hours. Both pages quote the run that their numbers come from. A few further blocks are marked as not run on their own page, with the reason. The protocol and the legacy tutorials are not executed. ## Example data -| File | What it is | +| File | Content | |---|---| -| [`smallYeast.yml`](../data/smallYeast.yml) | central carbon metabolism in yeast; the default example | -| [`smallYeastBad.yml`](../data/smallYeastBad.yml) | the same model with deliberate errors, for the curation pages | -| [`empty.xml`](../data/empty.xml) | an empty model with compartments, for building one from scratch | -| [`yeast-GEM.yml`](../data/yeast-GEM.yml) | yeast-GEM v9.1.0, the consensus *S. cerevisiae* model, for pages that need a genome-scale one | -| [`yeast-GEM.xml`](../data/yeast-GEM.xml) | the same model as SBML | -| [`anaerobic.yml`](../data/anaerobic.yml) | a condition file: yeast-GEM under anaerobic conditions | +| [`smallYeast.yml`](../data/smallYeast.yml) | Central carbon metabolism of yeast; the default example | +| [`smallYeastBad.yml`](../data/smallYeastBad.yml) | The same model with deliberate errors, used on the curation pages | +| [`empty.xml`](../data/empty.xml) | A starter model with four metabolites and one reaction, the start of a model built from scratch | +| [`yeast-GEM.yml`](../data/yeast-GEM.yml) | yeast-GEM v9.1.0, the consensus *S. cerevisiae* model, for pages that need a genome-scale model | +| [`yeast-GEM.xml`](../data/yeast-GEM.xml) | yeast-GEM v9.1.0 as SBML | +| [`anaerobic.yml`](../data/anaerobic.yml) | A condition file: yeast-GEM under anaerobic conditions | ## Pages ### Foundations -1. [Getting started](getting-started.md), load a model and inspect it. -2. [Model structure and identifiers](model-structure.md), how the RAVEN struct and `cobra.Model` correspond, field by field. -3. [Reading and writing models](io.md), SBML, RAVEN YAML, Excel, text, and the layout a model repository expects. +1. [Getting started](getting-started.md): load a model and inspect it. +2. [Model structure and identifiers](model-structure.md): how the RAVEN struct and `cobra.Model` correspond, field by field. +3. [Reading and writing models](io.md): SBML, RAVEN YAML, Excel and text, and the layout of a model repository. ### Simulation -4. [Simulating growth with FBA](fba.md), objective, bounds, solve, read the fluxes. -5. [Growth media and conditions](media.md), define a medium, keep a condition as data. -6. [Solvers and configuration](solvers.md), which solver, and how to read what it returns. +4. [Simulating growth with FBA](fba.md): objective, bounds, solving and reading the fluxes. +5. [Growth media and conditions](media.md): defining a medium and keeping a condition as data. +6. [Solvers and configuration](solvers.md): selecting a solver and reading what it returns. ### Building and curating -7. [Building a model from scratch](building.md), metabolites, reactions, genes, exchanges. -8. [Editing an existing model](editing.md), stoichiometry, GPRs, bounds, deletions. -9. [Quality control](quality-control.md), structure, mass balance, blocked reactions, mass from nothing. -10. [Table-driven curation](curation.md), applying curation as reviewable tables, in place of a script of function calls. -11. [Biomass composition and annotation](biomass.md), what the cell is made of, rescaling it, and the annotation that remains useful after the solve. +7. [Building a model from scratch](building.md): metabolites, reactions, genes and exchange reactions. +8. [Editing an existing model](editing.md): stoichiometry, gene-reaction rules, bounds and deletions. +9. [Quality control](quality-control.md): structure, mass balance, blocked reactions and mass produced from nothing. +10. [Table-driven curation](curation.md): curation kept as reviewable tables. +11. [Biomass composition and annotation](biomass.md): the composition of the biomass reaction, rescaling it, and the annotation that the model keeps after the solve. ### Reconstruction -12. [Reconstruction from homology](homology.md), BLAST a proteome against a template model's genes, and what the cut-offs decide. -13. [Reconstruction from KEGG](kegg.md), a draft without a template model, and what such a draft is missing. -14. [Context-specific models with ftINIT](init.md), omics integration and extracting a cell-line model from Human-GEM and RNA-seq, end to end. +12. [Reconstruction from homology](homology.md): aligning a proteome against the genes of a template model, and the effect of the cut-offs. +13. [Reconstruction from KEGG](kegg.md): a draft without a template model, and what such a draft lacks. +14. [Context-specific models with ftINIT](init.md): integrating omics data and extracting a cell-line model from Human-GEM and RNA-seq data. ### Analysis and repair -15. [Metabolic tasks](tasks.md), defining what a model must be able to do, and checking that it keeps that ability. -16. [Deletions and essentiality](deletions.md), knockouts, essential genes and reactions, MOMA. -17. [Gap-filling](gap-filling.md), closing the gaps in a draft model. -18. [Flux variability and alternative optima](fva.md), how much each flux can vary, and which predictions the model fixes. -19. [Random sampling](sampling.md), the distribution behind those ranges, and how to condition it on a particular state. +15. [Metabolic tasks](tasks.md): defining what a model must be able to do and checking that it keeps that ability. +16. [Deletions and essentiality](deletions.md): knockouts, essential genes and reactions, and MOMA. +17. [Gap-filling](gap-filling.md): closing the gaps of a draft model. +18. [Flux variability and alternative optima](fva.md): the range of each flux and the predictions that the model fixes. +19. [Random sampling](sampling.md): the flux distribution behind those ranges and its conditioning on a state. ### Comparing and engineering -20. [Combining and simplifying models](combining.md), merging models on metabolite names, and taking one apart. -21. [Comparing models](comparing.md), what changed between two models, and what effect that difference has. -22. [Engineering targets](engineering.md), FSEOF for over-production targets and reporter metabolites for the metabolites a transcriptional response surrounds. +20. [Combining and simplifying models](combining.md): merging models on metabolite names and taking a model apart. +21. [Comparing models](comparing.md): the differences between two models and their effect on predictions. +22. [Engineering targets](engineering.md): FSEOF for over-production targets and reporter metabolites for the metabolites around a transcriptional response. ## Protocol and legacy tutorials -| Resource | Description | +| Resource | Content | |---|---| -| [GEM reconstruction protocol, *H. polymorpha*](../protocol/index.md) | homology-based reconstruction of `hanpo-GEM`, end to end, from template models to a growing methylotrophic draft. A published pipeline followed from start to finish. MATLAB only, not executed here. | -| [Legacy tutorials](../tutorials/index.md) | five exercises from the original RAVEN paper (Agren et al., 2013), updated to run with current RAVEN but otherwise unchanged. MATLAB only. | +| [GEM reconstruction protocol, *H. polymorpha*](../protocol/index.md) | Homology-based reconstruction of `hanpo-GEM`, from the template models to a draft that grows on methanol. MATLAB; not executed. | +| [Legacy tutorials](../tutorials/index.md) | Five exercises from the original RAVEN paper (Agren et al., 2013), adapted to run with current RAVEN and otherwise unchanged. MATLAB. | -The task pages are for looking something up; the protocol shows a whole reconstruction in order. +The task pages are for looking up a single task. The protocol follows one reconstruction in order. -## Planned pages +## Compartment localisation -Compartment localisation has no page; the feature is in development and not at parity between the two toolboxes. +No page covers it. The feature is in development and the two toolboxes differ in it. ```{toctree} :hidden: diff --git a/docs/guide/init.md b/docs/guide/init.md index 5954ebb..7806171 100644 --- a/docs/guide/init.md +++ b/docs/guide/init.md @@ -1,6 +1,6 @@ # Context-specific models with ftINIT -A genome-scale model describes what an organism can do. ftINIT reduces it to what a particular sample (a tissue, a cell line, a condition) appears to be doing, from expression data plus a list of metabolic tasks that the result must be able to perform. The legacy method tINIT, which ftINIT replaces, does the same for models that were built with it. +A genome-scale model describes everything that an organism can do. ftINIT reduces it to what a given sample (a tissue, a cell line or a condition) appears to do. The input is expression data and a list of metabolic tasks that the result must be able to perform. The legacy method tINIT, which ftINIT replaces, extracts models in the same way for models that were built with it. The workflow uses Human-GEM and the RNA-seq data that Human-GEM provides, in both toolboxes. @@ -8,13 +8,13 @@ The workflow uses Human-GEM and the RNA-seq data that Human-GEM provides, in bot ftINIT is the recommended method and replaces the legacy method tINIT. The workflow uses ftINIT. -Use tINIT (`getINITModel`) to reproduce a model that was built with it. tINIT is supported and is not scheduled for removal. It issues a `RAVEN:legacyMethod` notice, which states that it is not intended for new work. Silence the notice with `warning('off','RAVEN:legacyMethod')`. +tINIT (`getINITModel`) reproduces a model that was built with it. It is supported and remains in RAVEN. It issues the notice `RAVEN:legacyMethod` once per session, which states that the method is not intended for new work, and `warning('off','RAVEN:legacyMethod')` silences the notice. -The legacy method tINIT is MATLAB-only. RAVEN includes it for the models already built with it; raven-toolbox, an implementation without such an installed base, includes ftINIT only. A tINIT model has to be reproduced in MATLAB. +tINIT exists in MATLAB. RAVEN includes it for the models that were built with it. raven-toolbox has no such installed base and includes ftINIT, so a tINIT model is reproduced in MATLAB. ## Relation between tINIT and ftINIT -The two methods are separate implementations that share no algorithm code. In the table, the reaction-scoring and task gap-filling entries differ between the two methods only because the two entry points call their shared functions with different settings, not because two implementations exist. +The two methods are separate implementations that share no algorithm code. In the table, the entries for reaction scoring and task gap-filling differ because the two entry points call shared functions with different settings. Two implementations do not exist for these steps. | Aspect | tINIT | ftINIT | |---|---|---| @@ -24,35 +24,35 @@ The two methods are separate implementations that share no algorithm code. In th | Task gap-filling | `fitTasks` | `fitTasks` (`gapFillMode` `'preMerged'`), `ftINITFillGaps` | | Gene pruning | inline in `getINITModel` | `removeLowScoreGenes` | -What they share is general RAVEN machinery, nothing specific to the method: `checkTasks` and `getEssentialRxns` decide which tasks are feasible and which reactions they need, and below these are `parseTaskList`, `simplifyModel`, the solver layer and the model-manipulation and I/O functions. +The methods share general RAVEN functions that are not specific to either. `checkTasks` and `getEssentialRxns` decide which tasks are feasible and which reactions they need. Below these are `parseTaskList`, `simplifyModel`, the solver layer, and the functions for model manipulation and input and output. -`scoreModel` is one function for both. `getINITModel` calls it with a single operator for both `and` and `or` in a grRule and `dataPrecedence` `'reaction'` (the settings the original tINIT algorithm needs), converting an unmeasured gene's score from `NaN` to `-Inf` afterward. `ftINIT` calls it with the general defaults. Task gap-filling is likewise one `fitTasks` loop for both. `getINITModel` uses the default `gapFillMode` (`'merge'`, backed by `fillGaps`). `ftINIT` passes `'preMerged'` (backed by `ftINITFillGaps`, since its reference model already contains the sample's own reactions and needs no per-task merge). The MILP formulation (`fillGaps` merging per task, `ftINITFillGaps` expecting a pre-merged model) is the one place where gap-filling differs, not `getINITModel` against `ftINIT` themselves. +`scoreModel` is one function for both methods. `getINITModel` calls it with a single operator for `and` and `or` in a gene rule and with `dataPrecedence` `'reaction'`, the settings that the original tINIT algorithm needs, and afterwards converts the score of an unmeasured gene from `NaN` to `-Inf`. `ftINIT` calls it with the general defaults. Task gap-filling is one `fitTasks` loop for both. `getINITModel` uses the default `gapFillMode` (`'merge'`, backed by `fillGaps`). `ftINIT` passes `'preMerged'`, backed by `ftINITFillGaps`, because its reference model already contains the reactions of the sample and needs no merge per task. The MILP formulation is therefore the one place where gap-filling differs: `fillGaps` merges per task, and `ftINITFillGaps` expects a model that is merged already. The difference does not lie between `getINITModel` and `ftINIT` as such. ## Reference run -The outputs in this workflow come from one run made by hand. Other guide pages are re-executed by the [Executed examples](index.md#executed-examples) workflow whenever a guide page, the example data, the runner, the workflow or a submodule changes, and once a week; the outputs here are not, because preparing Human-GEM took 47 minutes in MATLAB and 37 minutes in Python, and the MATLAB run produced a 174 MB artefact, which no documentation build should attempt. The numbers come from one real run (Human-GEM `main` at commit `6812be2a`, RAVEN commit `2d0630e3`, raven-toolbox commit `94f39407`, Gurobi (MATLAB links the 13.0.1 module library; Python most likely loads the library of gurobipy 13.0.3), four cores, 2026-10-30) and are quoted with their wall-clock times for planning. +The outputs in this workflow come from one run made by hand. The [Executed examples](index.md#executed-examples) workflow runs the other guide pages again whenever a guide page, the example data, the runner, the workflow or a submodule changes, and once a week. It does not run the outputs here, because preparing Human-GEM took 47 minutes in MATLAB and 37 minutes in Python, and the MATLAB run produced an artefact of 174 MB, which a documentation build should not attempt. The numbers come from one run (Human-GEM `main` at commit `6812be2a`, RAVEN commit `2d0630e3`, raven-toolbox commit `94f39407`, Gurobi (MATLAB links the 13.0.1 module library; Python most likely loads the library of gurobipy 13.0.3), four cores, 2026-10-30) and are quoted with their wall-clock times for planning. -The same MATLAB workflow, as the model's own documentation presents it and including the comparison of the extracted models, is in the [Human-GEM guide](https://sysbiochalmers.github.io/Human-GEM-guide/gem_extraction/), which is maintained alongside the model. +The [Human-GEM guide](https://sysbiochalmers.github.io/Human-GEM-guide/gem_extraction/) presents the same MATLAB workflow, including the comparison of the extracted models. It is maintained together with the model. ## Functions | MATLAB | Python | Purpose | |---|---|---| -| `prepINITModel` | `prep_init_model` | the once-per-template preparation | -| `parseTaskList` | `parse_task_list` | the tasks the extracted model must satisfy | +| `prepINITModel` | `prep_init_model` | preparation of the template, once | +| `parseTaskList` | `parse_task_list` | the tasks that the extracted model must satisfy | | `getINITSteps` | `get_init_steps` | the step definitions (`1+0`, `1+1`, …) | | `ftINIT` | `ftinit` | the staged extraction | -| `scoreModel` | `score_reactions_from_genes` | gene scores → reaction scores | -| no equivalent | `gene_scores_from_expression` | expression → gene scores | +| `scoreModel` | `score_reactions_from_genes` | gene scores to reaction scores | +| no equivalent | `gene_scores_from_expression` | expression to gene scores | | `runINIT`, `getINITModel` | no equivalent | the legacy tINIT method, for reproducing models built with it | -| `removeLowScoreGenes` | `remove_low_score_genes` | prune negative-scoring genes from gene-protein-reaction rules (GPRs) | -| `parseHPA`, `parseHPArna` | `parse_hpa`, `parse_hpa_rna` | read Human Protein Atlas (HPA) dumps | +| `removeLowScoreGenes` | `remove_low_score_genes` | remove negative-scoring genes from gene-protein-reaction rules (GPRs) | +| `parseHPA`, `parseHPArna` | `parse_hpa`, `parse_hpa_rna` | read Human Protein Atlas (HPA) files | | no equivalent | `hpa_gene_scores`, `rna_gene_scores` | HPA levels or transcripts per million (TPM) to gene scores | | `checkTasks` | `check_tasks` | confirm that the result performs the tasks | ## Setup -Human-GEM ships everything needed except the toolbox: +Human-GEM provides everything except the toolbox: @@ -60,18 +60,20 @@ Human-GEM ships everything needed except the toolbox: git clone --depth=1 https://github.com/SysBioChalmers/Human-GEM.git ``` -| File | What it is | +| File | Content | |---|---| | `model/Human-GEM.mat` (or `.xml`, `.yml`) | the template: 12 748 reactions, 2 870 genes | | `model/reactions.tsv` | reaction annotations, including which reactions are spontaneous | -| `data/metabolicTasks/metabolicTasks_Essential.txt` | 57 tasks the extracted model must pass | +| `data/metabolicTasks/metabolicTasks_Essential.txt` | 57 tasks that the extracted model must pass | | `data/datasets/Hart2015_RNAseq.txt` | TPM for five cell lines: DLD1, GBM, HCT116, HELA, RPE1 | -Both extractions are mixed-integer problems. On Human-GEM, GLPK completed a benchmark extraction in 2182 s with a time-limited result (Jaccard 0.892 against Gurobi), so set Gurobi up first ([Solvers and configuration](solvers.md)). That benchmark ([ftINIT on different solvers](../parameter-tuning/studies/init-solver-benchmark.md)) used Human-GEM `38567ab8` (cleaned), prepared without a task list and with scaled stoichiometry, series `1+1` and `time_limit=900`, a different input from the `6812be2a` template with task preparation used here. In raven-toolbox, without `time_limit`, the scheduled runs of a series keep their own limits (120 s and 5000 s, plus a 5 s first run in the last step of `1+1` and `2+1`; 10000 s for `full`, which runs once), for GLPK as for Gurobi. With `setRavenSolver('glpk')`, RAVEN does not pass the `TimeLimit` of the schedule to GLPK: `optimizeProb` sets a 1000 s limit on every GLPK MILP. +Both extractions are mixed-integer problems. On Human-GEM, GLPK completed a benchmark extraction in 2182 s with a time-limited result (Jaccard 0.892 against Gurobi), so Gurobi should be set up first ([Solvers and configuration](solvers.md)). That benchmark ([ftINIT on different solvers](../parameter-tuning/studies/init-solver-benchmark.md)) used Human-GEM `38567ab8` (cleaned), prepared without a task list and with scaled stoichiometry, series `1+1` and `time_limit=900`. The input differs from the `6812be2a` template with task preparation used here. + +In raven-toolbox without `time_limit`, the scheduled runs of a series keep their own limits, for GLPK as for Gurobi. The limits are 120 s and 5000 s, plus a first run of 5 s in the last step of `1+1` and `2+1`, and 10000 s for `full`, which runs once. With `setRavenSolver('glpk')`, RAVEN does not pass the `TimeLimit` of the schedule to GLPK, and `optimizeProb` sets a limit of 1000 s on every GLPK MILP. ## Template preparation -The preparation finds the task-essential reactions, classifies every reaction into omics-independent categories, merges linear stretches, and rescales the stoichiometry to a narrower coefficient range, which is not needed for feasibility at the default `big_m`. In the single runs of [ftINIT parameter calibration and input robustness](../parameter-tuning/studies/init-param-calibration.md), the rescaling shortened the run (83 s against 157 s). It depends only on the template and the task list, and not on the expression data, so it is done once and reused for every sample. +The preparation finds the task-essential reactions, classifies every reaction into categories that do not depend on omics data, merges linear stretches, and rescales the stoichiometry to a narrower range of coefficients. The rescaling is not needed for feasibility at the default `big_m`. In the single runs of [ftINIT parameter calibration and input robustness](../parameter-tuning/studies/init-param-calibration.md), it shortened the run (83 s against 157 s). The preparation depends on the template and the task list, so it is done once and reused for every sample. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -95,7 +97,7 @@ save('prepData.mat', 'prepData', '-v7.3'); prepData.mat: 174 MB ``` -Human-GEM also has a wrapper, `prepHumanModelForftINIT`, which reads those two files; it needs Human-GEM 2.0.1 or later. +Human-GEM has a wrapper, `prepHumanModelForftINIT`, which reads those two files. It needs Human-GEM 2.0.1 or later. ::: :::{tab-item} 🐍 Python :sync: python @@ -128,22 +130,22 @@ load 11s prep_init_model 2216s ``` -Reading Human-GEM from the Systems Biology Markup Language (SBML) file alone takes about 11 seconds, and the preparation itself 37 minutes in Python and 47 minutes in MATLAB. The two preparations were separate Slurm jobs on the same node type with four cores each. MATLAB used a pool of 4 workers and Python ran with `processes = 1`, so the two times compare the whole runs, not the code alone. Two things affect the run: +Reading Human-GEM from the Systems Biology Markup Language (SBML) file alone takes about 11 seconds, and the preparation takes 37 minutes in Python and 47 minutes in MATLAB. The two preparations were separate Slurm jobs on the same node type with four cores each. The MATLAB run used the default of `prepINITModel` for `runParallel`, which is serial, and the Python run used `processes = 1`, so the two times compare the complete runs. Two settings affect the run: -- `prep_init_model` runs cobrapy's flux variability analysis (FVA), which spawns worker processes by default; the Python run above used `processes = 1`. Where worker processes are not permitted (a locked-down Windows machine, some continuous integration (CI) runners), the default fails with `PermissionError: [WinError 5] Access is denied`, and `processes = 1` removes the parallelism and lets the run finish. -- `essential_cache_path` caches the slow task-essential discovery, so a second preparation of the same template skips it. +- `prep_init_model` takes the argument `processes`, which defaults to 1, and its `simplify_model` passes follow `cobra.Configuration().processes`, which starts worker processes by default. The Python run above used `processes = 1`. Where worker processes are not permitted (a locked-down Windows machine, some continuous integration (CI) runners), the default fails with `PermissionError: [WinError 5] Access is denied`, and `processes = 1` removes the parallelism and lets the run finish. +- `essential_cache_path` caches the slow search for task-essential reactions, so a second preparation of the same template skips it. ::: :::: :::{admonition} Spontaneous reactions :class: note -Pass the list of spontaneous reactions. ftINIT keeps every reaction passed as spontaneous in every extracted model, whatever the data say. Other reactions without a gene rule are kept only where the optimisation needs them, so they can differ from sample to sample. In Human-GEM, `spontaneous` marks reactions that need no enzyme. That includes diffusion: transport between cytosol and nucleus through the nuclear pores, and gases and water (O2, CO2, H2O, NH3, H2O2, NO, H2S) crossing membranes. Leaving out the list lets ftINIT drop such transports in some samples. The model then uses other routes. For example, without the dCTP transport into the nucleus, deoxycytidine kinase (DCK) becomes essential for DNA synthesis, which is an artefact of the extraction and not biology. +The list of spontaneous reactions is passed to the preparation. ftINIT keeps every reaction in the list in every extracted model, whatever the data say. Other reactions without a gene rule are kept where the optimisation needs them, so they can differ between samples. In Human-GEM, `spontaneous` marks reactions that need no enzyme. These include diffusion: transport between cytosol and nucleus through the nuclear pores, and gases and water (O2, CO2, H2O, NH3, H2O2, NO, H2S) crossing membranes. Without the list, ftINIT can remove such transports in some samples, and the model then uses other pathways. Without the transport of dCTP into the nucleus, for example, deoxycytidine kinase (DCK) becomes essential for DNA synthesis, which is an artefact of the extraction. ::: ## Expression data -RAVEN takes one struct: the genes, the sample names, and a genes × samples matrix of levels. `threshold` is the level above which a gene counts as expressed; when it is left out, the mean across samples is used per gene. +RAVEN takes one struct with the genes, the sample names, and a matrix of levels with genes in the rows and samples in the columns. `threshold` is the level above which a gene counts as expressed. Without it, the mean across the samples is used for each gene. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -182,9 +184,9 @@ gene_scores = gene_scores_from_expression(tpm["HCT116"].to_dict(), reference=1.0 rxn_scores = score_reactions_from_genes(model, gene_scores) ``` -The two steps are separate in Python: `gene_scores_from_expression` applies RAVEN's rule (5·ln(level / reference), clamped to [−5, 10]) and `score_reactions_from_genes` maps the result through the GPRs. Because the steps are separate, any other source of gene scores (HPA via `hpa_gene_scores`, proteomics, a curated list) feeds the same second step. +The two steps are separate in Python. `gene_scores_from_expression` applies the rule of RAVEN (5·ln(level / reference), clamped to [−5, 10]), and `score_reactions_from_genes` maps the result through the gene rules. Any other source of gene scores (HPA through `hpa_gene_scores`, proteomics or a curated list) can therefore feed the second step. -The clamp explains why two samples that differ widely can produce nearly the same model: +The clamp explains why two samples that differ widely can give nearly the same model: @@ -196,13 +198,13 @@ print(gene_scores_from_expression({"a": 12.0, "b": 0.5, "c": 3.0}, reference=3.0 {'a': 6.93, 'b': -5.0, 'c': 0.0} ``` -`b` computes to −8.96 and is returned at the floor; a gene exactly at its reference scores zero, which is neither in nor out. +`b` computes to −8.96 and is returned at the floor. A gene at its reference scores zero, which is neither in nor out. ::: :::: ### Scores from the Human Protein Atlas -A transcript table is one source of gene scores. The Human Protein Atlas is another, and both toolboxes read it directly. [`hpa-sample.tsv`](../data/hpa-sample.tsv) is a ten-row excerpt in HPA's proteomics format: the six columns the parsers expect, five human genes across two tissues, with the levels chosen so each category appears. A real dump is `normal_tissue.tsv` from [proteinatlas.org](https://www.proteinatlas.org/about/download). +A transcript table is one source of gene scores, and the Human Protein Atlas is another. Both toolboxes read it directly. [`hpa-sample.tsv`](../data/hpa-sample.tsv) is an excerpt of ten rows in the proteomics format of HPA, with the six columns that the parsers expect and five human genes in two tissues. The levels are chosen so that every category occurs. A complete file is `normal_tissue.tsv` from [proteinatlas.org](https://www.proteinatlas.org/about/download). ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -220,9 +222,9 @@ fprintf('levels: %s\n', strjoin(hpaData.levels, ', ')); levels: High, Low, Medium, Not detected ``` -`parseHPA` returns a struct, not a table. `genes` and `geneNames` hold the Ensembl ids and the symbols, `tissues` and `celltypes` are parallel arrays with one entry per tissue and cell-type combination, not per tissue, and `gene2Level` is a sparse genes-by-combination matrix whose values index into `levels`. Reading a level therefore takes two steps, the same indirection `model.metComps` uses in [Model structure and identifiers](model-structure.md). +`parseHPA` returns a struct. `genes` and `geneNames` hold the Ensembl identifiers and the symbols. `tissues` and `celltypes` are parallel arrays with one entry per combination of tissue and cell type. `gene2Level` is a sparse matrix of genes by combination, and its values are indices into `levels`. A level is therefore read in two steps, the same indirection that `model.metComps` uses in [Model structure and identifiers](model-structure.md). -The `version` argument is accepted and ignored: the format is inferred from the column headers. `hpaData` is what `ftINIT` takes as its fourth positional argument, in place of `transcrData`. +The argument `version` is accepted and ignored, because the format is inferred from the column headers. `hpaData` is what `ftINIT` takes as its fourth positional argument, in place of `transcrData`. ::: :::{tab-item} 🐍 Python :sync: python @@ -244,15 +246,15 @@ cell types in liver: ['bile duct cells', 'hepatocytes'] levels: ['High', 'Low', 'Medium', 'Not detected'] ``` -`parse_hpa` returns an `HPAData` wrapping a tidy pandas DataFrame on `.df`, one row per gene, tissue and cell type, with the columns renamed to `gene_id`, `gene_name`, `tissue`, `celltype`, `level` and `reliability`, so ordinary grouping and filtering apply before any of it becomes a score. `tissues()` and `celltypes()` save writing the corresponding queries. +`parse_hpa` returns an `HPAData` that wraps a pandas DataFrame in `.df`, with one row per gene, tissue and cell type and the columns `gene_id`, `gene_name`, `tissue`, `celltype`, `level` and `reliability`. Ordinary grouping and filtering apply before anything becomes a score. `tissues()` and `celltypes()` return the corresponding lists. -MATLAB's parallel arrays plus a sparse index matrix answer "what is the level of gene i in combination j" directly; the DataFrame answers "which rows belong to this tissue". Neither holds anything the other does not. +The parallel arrays and the sparse index matrix of MATLAB answer the question for the level of gene i in combination j directly, and the DataFrame answers which rows belong to a tissue. Neither holds information that the other lacks. ::: :::: ### Levels to numbers -HPA reports a category and not a quantity, so a level has to become a number before it can be used in scoring. +HPA reports a category, so a level becomes a number before it is used in scoring. ::::{tab-set} :::{tab-item} 🐍 Python @@ -277,15 +279,15 @@ for gene, score in sorted(scores.items()): ENSG00000159399 -8.0 ``` -The mapping is exposed as `HPA_LEVEL_SCORES` and is replaced by passing `level_scores=`. Both vocabularies are covered: `High`, `Medium`, `Low` and `Not detected` for expression, and `Strong`, `Moderate`, `Weak` and `Negative` for antibody staining, scoring the same. +The mapping is available as `HPA_LEVEL_SCORES`, and `level_scores=` replaces it. Both vocabularies are covered, `High`, `Medium`, `Low` and `Not detected` for expression and `Strong`, `Moderate`, `Weak` and `Negative` for antibody staining, and the corresponding terms score the same. -In MATLAB there is no separate step: `scoreModel` takes `hpaData` and applies the mapping internally. +MATLAB has no separate step: `scoreModel` takes `hpaData` and applies the mapping internally. ::: :::: -A negative score is a statement and not a missing value. `Not detected` scores -8, which pushes a reaction towards exclusion. A gene absent from the tissue is omitted from the result, and the reaction scorer then uses its `no_gene_score`. The two cases produce different models. +A negative score is a statement. `Not detected` scores -8, which pushes a reaction towards exclusion. A gene that is absent from the tissue is omitted from the result, and the reaction scorer then uses its `no_gene_score`. The two cases give different models. -### One gene, several cell types +### One gene in several cell types A tissue has several cell types, and a gene can be measured differently in each. `PKM` in the excerpt is `High` in hepatocytes and `Low` in bile duct cells. @@ -304,11 +306,11 @@ print(f"PKM best {best['ENSG00000067225']:+.1f}, " PKM best +20.0, average +15.0 ``` -`"best"` takes the maximum and is the default, matching RAVEN. `"average"` takes the mean. The maximum states what the tissue is capable of, the mean what it does across the cells in it. Passing `celltype=` selects one cell type, so the choice does not arise. +`"best"` takes the maximum and is the default, as in RAVEN. `"average"` takes the mean. The maximum describes the capacity of the tissue, and the mean describes its activity across cells. `celltype=` selects one cell type, and the choice then does not arise. ::: :::: -`parseHPArna` and `parse_hpa_rna` read the RNA-seq dump, whose header is `Gene`, `Gene name`, `Tissue` followed by the TPM columns. Those are quantities already, so `rna_gene_scores` applies the same logarithmic rule as `gene_scores_from_expression` and not a level mapping. Both sources end at a gene-to-score mapping, which `score_reactions_from_genes` applies to the GPRs. +`parseHPArna` and `parse_hpa_rna` read the RNA-seq file, whose header is `Gene`, `Gene name`, `Tissue` followed by the TPM columns. These values are quantities, so `rna_gene_scores` applies the logarithmic rule of `gene_scores_from_expression`. Both sources end as a mapping from genes to scores, which `score_reactions_from_genes` applies to the gene rules. ## Extraction for one sample @@ -328,10 +330,10 @@ contextModel = ftINIT(prepData, 'HCT116', [], [], ... 9544 rxns, 1785 genes ``` -Two argument errors fail without naming the argument at fault: +Two argument errors fail without naming the argument: -- `hpaData` is the fourth positional argument. When it is skipped, the string `'transcrData'` is taken as `hpaData` and the expression struct is assigned to `transcrData` by position. -- the step list is `'INITSteps'` and not `'steps'`. An unrecognised name is taken as a positional value, and the misleading error message `Metabolomics contains the same metabolite multiple times` appears. +- `hpaData` is the fourth positional argument. When it is skipped, the string `'transcrData'` is taken as `hpaData`, and the expression struct is assigned to `transcrData` by position. +- The name of the step list is `'INITSteps'`. An unrecognised name is taken as a positional value, and the misleading error message `Metabolomics contains the same metabolite multiple times` appears. ::: :::{tab-item} 🐍 Python :sync: python @@ -345,17 +347,17 @@ context = ftinit(prep, rxn_scores=rxn_scores, series="1+0", gene_scores=gene_sco print(len(context.reactions), "reactions,", len(context.genes), "genes") ``` -`ftinit` takes the reaction scores directly and not the expression struct, so the scoring is a separate step. `gene_scores` is optional and prunes negative-scoring genes from the GPRs that survive; `fill_gaps=False` skips the task repair. +`ftinit` takes the reaction scores, so the scoring is a separate step. `gene_scores` is optional and removes negative-scoring genes from the gene rules that remain. `fill_gaps=False` skips the task repair. ::: :::: ### Series `1+0` and `1+1` -`1+0` leaves out most reactions that have no gene rule; in this sample it ran in 124 s in Python and 797 s in MATLAB. `1+1` adds a second optimisation over those reactions. The run shown uses `1+0`: 12 748 → 9 543 reactions and 2 870 → 1 783 genes in Python (124 seconds), 12 748 → 9 544 reactions and 2 870 → 1 785 genes in MATLAB (797 seconds). +`1+0` leaves out most reactions without a gene rule, and in this sample it ran in 124 s in Python and in 797 s in MATLAB. `1+1` adds a second optimisation over those reactions. The run shown uses `1+0`. It gives 12 748 → 9 543 reactions and 2 870 → 1 783 genes in Python (124 seconds), and 12 748 → 9 544 reactions and 2 870 → 1 785 genes in MATLAB (797 seconds). ## Checking the extracted model -An extracted model is a hypothesis. The first check is the tasks the model was built to satisfy, which is also the least expensive check. +An extracted model is a hypothesis. The first check is the set of tasks that the model was built to satisfy, which is also the cheapest check. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -381,19 +383,19 @@ print(sum(1 for result in report if result.passed), "of", len(report), "tasks pa ::: :::: -Compare the result against the template and against the other samples: [Quality control](quality-control.md) covers the structural checks, and the Human-GEM guide's [GEM comparison](https://sysbiochalmers.github.io/Human-GEM-guide/gem_comparison/) covers comparing many extracted models at once. +The result is compared with the template and with the other samples. [Quality control](quality-control.md) covers the structural checks, and the [GEM comparison](https://sysbiochalmers.github.io/Human-GEM-guide/gem_comparison/) of the Human-GEM guide covers the comparison of many extracted models. :::{admonition} Troubleshooting :class: warning -- Identifiers that do not match. Human-GEM uses ENSEMBL identifiers, and so does `Hart2015_RNAseq.txt`, so no mapping step appears in the workflow. Expression data with symbols or systematic names needs a mapping; score a few genes and check that they are not all at the floor before spending 37 to 47 minutes on the preparation. -- Re-preparing per sample. The preparation depends only on the template and the tasks. Do it once, save it, and reuse it; preparation and extraction are separate, which makes that possible. -- No Gurobi. Both extractions are mixed-integer problems; on Human-GEM, GLPK completed a benchmark extraction in 2182 s with a time-limited result (Jaccard 0.892 against Gurobi; Human-GEM `38567ab8` prepared without a task list and with scaled stoichiometry, series `1+1`, `time_limit=900`, a different input from the one in this workflow). -- A model without capabilities assumed of it. Without a task list there is nothing to repair against, and the extraction is free to remove capabilities that were never checked. +- The identifiers do not match. Human-GEM and `Hart2015_RNAseq.txt` both use ENSEMBL identifiers, so the workflow has no mapping step. Expression data with symbols or systematic names needs a mapping. A few genes are scored and checked against the floor before the preparation of 37 to 47 minutes starts. +- The template is prepared again for every sample. The preparation depends on the template and the tasks. It is done once, saved and reused, which is possible because preparation and extraction are separate. +- Gurobi is not available. Both extractions are mixed-integer problems. On Human-GEM, GLPK completed a benchmark extraction in 2182 s with a time-limited result (Jaccard 0.892 against Gurobi; Human-GEM `38567ab8` prepared without a task list and with scaled stoichiometry, series `1+1`, `time_limit=900`, a different input from the one in this workflow). +- The task list is missing. Without a task list there is nothing to repair against, and the extraction can remove capabilities that were never checked. ::: ## See also -- [Human-GEM guide: GEM extraction using ftINIT](https://sysbiochalmers.github.io/Human-GEM-guide/gem_extraction/), the same workflow in MATLAB, maintained with the model. +- [Human-GEM guide: GEM extraction using ftINIT](https://sysbiochalmers.github.io/Human-GEM-guide/gem_extraction/): the same workflow in MATLAB, maintained with the model. - [Human-GEM guide: extraction from single-cell data](https://sysbiochalmers.github.io/Human-GEM-guide/gem_extraction_sc/). -- [Quality control](quality-control.md), checking the model that comes out. +- [Quality control](quality-control.md): checking the model that results. diff --git a/docs/guide/io.md b/docs/guide/io.md index c1613b5..048cc97 100644 --- a/docs/guide/io.md +++ b/docs/guide/io.md @@ -1,6 +1,6 @@ # Reading and writing models -Get a model in and out of both toolboxes: SBML, RAVEN YAML, Excel, and the directory layout a Git-maintained model repository expects. +The page covers reading and writing of SBML, RAVEN YAML and Excel files, and writing of the directory layout of a Git-maintained model repository. ## Functions @@ -13,7 +13,7 @@ Get a model in and out of both toolboxes: SBML, RAVEN YAML, Excel, and the direc | `exportToExcelFormat` | `export_to_excel` | write the RAVEN Excel format | | `exportForGit` | `export_for_git` | write a Standard-GEM repository layout | -The function needed depends on the file, not on the model; all of them produce the same in-memory model, and none of them is the canonical format. What differs is what survives a round trip and how readable the file is; [Identifier mangling](#identifier-mangling) and [Spreadsheets](#spreadsheets) show this with examples. +The function is chosen by the file format. All readers produce the same content in memory, and no format is canonical. The formats differ in what survives a round trip and in how readable the file is, as [Identifier mangling](#identifier-mangling) and [Spreadsheets](#spreadsheets) show. ## Setup @@ -40,9 +40,9 @@ smallYeast 53 reactions yeastGEM_v9.1.0 4102 reactions ``` -Both warnings are informational and neither stops the read. The first says `importModel` stripped the `R_`, `M_`, `G_` and `C_` prefixes SBML requires on identifiers, which it does only when every identifier of that type carries one; pass `removePrefix` as `false` to keep them, which is what a comparison against the file's own ids requires. The second reports that the model annotates several differently-named metabolites with the same database identifier. The duplication is in yeast-GEM itself, and it makes those annotations unusable as a key for matching metabolites across models. +Both warnings are informational, and the read continues. The first reports that `importModel` removed the prefixes `R_`, `M_`, `G_` and `C_` that SBML requires on identifiers. It removes them when every identifier of that type has one, and `removePrefix` set to `false` keeps them, which a comparison with the identifiers in the file requires. The second warning reports that the model gives the same database identifier to several metabolites with different names. The duplication is in yeast-GEM, and it makes these annotations unusable for matching metabolites across models. -`importModel` accepts SBML Level 3 Version 1 with FBC version 2 and errors on anything older; see the [RAVEN 3 migration guide](../raven3-migration/formats-and-reconstruction.md#sbml-io) if a file is rejected. +`importModel` reads SBML Level 3 Version 1 with FBC version 2 and raises an error for older files. The [RAVEN 3 migration guide](../raven3-migration/formats-and-reconstruction.md#sbml-io) covers files that are rejected. ::: :::{tab-item} 🐍 Python :sync: python @@ -64,15 +64,15 @@ smallYeast 53 reactions yeastGEM_v9__46__1__46__0 4102 reactions ``` -`read_sbml_model` is cobrapy's own reader and strips the same SBML prefixes. `read_yaml_model` is raven-toolbox's, and returns a plain `cobra.Model`, so the models the two readers return are interchangeable from this point. It reads gzipped files directly and accepts both the current YAML layout and the older RAVEN one, so an archived model does not need converting first. +`read_sbml_model` is the reader of cobrapy and removes the same SBML prefixes. `read_yaml_model` is the reader of raven-toolbox. Both return a `cobra.Model`, and the models are interchangeable afterwards. `read_yaml_model` reads gzipped files directly and accepts the current YAML layout and the older RAVEN layout, so an archived model needs no conversion. -The yeast-GEM id printed here differs from the one in the MATLAB output; [Identifier mangling](#identifier-mangling) covers the difference. +The yeast-GEM identifier in this output differs from the one in the MATLAB output; [Identifier mangling](#identifier-mangling) explains the difference. ::: :::: ## Identifier mangling -SBML identifiers must be valid XML names, so they cannot contain a dot. Model, reaction and metabolite ids that do are encoded when the file is written, as `__` around the character's ASCII code: a `.` becomes `__46__`. The encoding is in the file, so what differs between the toolboxes is whether it is undone on the way back in. +SBML identifiers are XML names and cannot contain a dot. When a file is written, every character in the identifiers other than a letter, digit or underscore is encoded as its ASCII code between double underscores, and a dot becomes `__46__`. The code is stored in the file, and the toolboxes differ in whether they decode it on reading. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -89,7 +89,7 @@ yeastGEM_v9.1.0 yeastGEM_v9.1.0 ``` -`importModel` decodes every `__NN__` back to its character, across reactions, metabolites, compartments, genes, gene rules and the model id, so a round trip through SBML returns the original ids. +`importModel` decodes every `__NN__` in reactions, metabolites, compartments, genes, gene rules and the model identifier, so a round trip through SBML returns the original identifiers. ::: :::{tab-item} 🐍 Python :sync: python @@ -108,13 +108,13 @@ from YAML: yeastGEM_v9.1.0 same size: True ``` -cobrapy decodes the reaction, metabolite and gene ids but leaves the model id encoded, so the same release read from the two formats reports two different model ids while being the same model. `model.name` and the annotations are unchanged. +cobrapy decodes the identifiers of reactions, metabolites and genes and leaves the model identifier encoded. The same release read from SBML and from YAML therefore reports two model identifiers. `model.name` and the annotations are the same. ::: :::: ## Read time by format -Each toolbox is fast in one format and slow in the other. Reading yeast-GEM takes about 17 s from SBML and 73 s from YAML in Python, and about 78 s from SBML and 17 s from YAML in MATLAB, so the ranking is reversed. RAVEN parses YAML itself and goes through libSBML for SBML; cobrapy has the opposite balance. The format therefore follows from what is needed from it, a readable diff or the RAVEN-specific fields YAML preserves, and a script whose run time is dominated by reading the file often runs faster in the other format. +Each toolbox reads one format quickly and the other slowly, and the ranking is reversed between them. Reading yeast-GEM takes about 17 s from SBML and 73 s from YAML in Python, and about 78 s from SBML and 17 s from YAML in MATLAB. The format follows from what the file is needed for: a readable difference between versions, or the RAVEN-specific fields that YAML preserves. A script that spends most of its time reading the file runs faster with the format that its toolbox reads quickly. ## Writing a model @@ -138,9 +138,9 @@ Document written 53 52 61 ``` -The two warnings are the prefixing rule in both directions: `exportModel` adds a prefix to a whole field as soon as one id in it starts with something other than a letter or underscore, and `importModel` takes it off again. Pass `neverPrefix` as `true` to suppress the addition, but the resulting file is not valid SBML. +The two warnings show the prefix rule in both directions. `exportModel` adds a prefix to a whole field when one identifier in it starts with a character other than a letter or an underscore, and `importModel` removes it again. `neverPrefix` set to `true` suppresses the prefix, and the file is then not valid SBML. -Both `writeYAMLmodel` and `exportModel` take `sortIds`, which sorts a copy before writing and leaves the caller's model untouched. +`writeYAMLmodel` and `exportModel` take `sortIds`, which sorts a copy before writing and leaves the model of the caller unchanged. ::: :::{tab-item} 🐍 Python :sync: python @@ -161,15 +161,15 @@ print(len(roundtrip.reactions), len(roundtrip.metabolites), len(roundtrip.genes) 53 52 61 ``` -`write_yaml_model` and `export_to_excel` take `sort_ids`, which sorts what is written without touching the model. cobrapy's `write_sbml_model` has no such argument, so a stable SBML diff needs the sort applied first, on a copy: `write_sbml_model(sort_identifiers(small.copy()), path)`. +`write_yaml_model` and `export_to_excel` take `sort_ids`, which sorts the written content and leaves the model unchanged. `write_sbml_model` of cobrapy has no such argument, so a stable SBML file needs the sort on a copy first: `write_sbml_model(sort_identifiers(small.copy()), path)`. ::: :::: -Both toolboxes write reactions in whatever order the model holds them, so inserting one reaction near the front shifts every line after it. Sorting first confines the diff of a one-reaction change to that reaction. +Both toolboxes write the reactions in the order of the model, so a reaction inserted near the start shifts every line after it. After sorting, the difference of a one-reaction change is confined to that reaction. ## Spreadsheets -The Excel format holds the model as five sheets, covering reactions, metabolites, compartments, genes and the model's own metadata, in a form that can be edited without a toolbox. +The Excel format holds the model in five sheets (reactions, metabolites, compartments, genes and the metadata of the model), which can be edited without a toolbox. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -184,7 +184,7 @@ fprintf('%s\n', strjoin(sheetnames('smallYeast.xlsx'), ', ')); RXNS, METS, COMPS, GENES, MODEL ``` -`exportToExcelFormat` writes `.xlsx` and errors on any other extension, so a bare directory path is rejected without being interpreted. The [RAVEN 3 migration guide](../raven3-migration/formats-and-reconstruction.md#excel-io) covers scripts that pass one. There is no matching importer: `curateModelFromTables` reads a curated spreadsheet back by applying tabular edits to an existing model; it does not build one from scratch. +`exportToExcelFormat` writes `.xlsx` files and raises an error for any other extension, including a bare directory path. The [RAVEN 3 migration guide](../raven3-migration/formats-and-reconstruction.md#excel-io) covers scripts that pass a directory. No importer exists for the format. `curateModelFromTables` reads a curated spreadsheet and applies the edits to an existing model, and it does not build a model from scratch. ::: :::{tab-item} 🐍 Python :sync: python @@ -203,13 +203,13 @@ print(load_workbook("smallYeast.xlsx").sheetnames) ['RXNS', 'METS', 'COMPS', 'GENES', 'MODEL'] ``` -`export_to_excel` needs the `excel` extra (`pip install raven-toolbox[excel]`) and writes the same five sheets, plus ENZYMES and ENZRXNS for an enzyme-constrained model. Neither toolbox exports a whole model as a set of tab-delimited files; the single reaction table that `exportForGit` writes as its `txt` format is the only such output, and pandas over the model's collections covers the rest. +`export_to_excel` needs the `excel` extra (`pip install --pre raven-toolbox[excel]`) and writes the same five sheets, with the sheets ENZYMES and ENZRXNS added for an enzyme-constrained model. Neither toolbox exports a whole model as tab-delimited files. The sole such output is the reaction table that `exportForGit` writes as its `txt` format, and pandas over the collections of the model covers other tables. ::: :::: ## Exporting for a model repository -A Git-maintained model repository (yeast-GEM, Human-GEM, and the models built from `standard-GEM`) keeps the same model in several formats under `model/`, so that a release is usable without a toolbox and a diff is readable in a pull request. Both toolboxes write that layout directly, one subdirectory per format plus a `dependencies.txt` recording the versions the files were written with. +A Git-maintained model repository (yeast-GEM, Human-GEM and the models built from `standard-GEM`) keeps the model in several formats under `model/`. A release is then usable without a toolbox, and a pull request shows a readable difference. Both toolboxes write that layout: one subdirectory per format and a `dependencies.txt` with the versions of the software that wrote the files. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -230,7 +230,7 @@ Document written dependencies.txt, smallYeast.xml, smallYeast.yml ``` -`dir` reports names, so the subdirectories that contain the files are not visible in the output: the layout written is `repo/model/yml/smallYeast.yml` and `repo/model/xml/smallYeast.xml`. Set `subDirs` to `false` to put everything in one folder. Identifiers are sorted before writing, with no option to skip it, so successive releases diff cleanly. By default `exportForGit` writes all five formats, `mat`, `txt`, `xlsx`, `xml` and `yml`; `formats` narrows that. `COBRAtext` switches the `txt` table from metabolite names to metabolite ids, and `mainBranchFlag` makes the export fail unless RAVEN itself (and GECKO, for an enzyme-constrained model) is on its main branch, which pins a release to a released toolbox version. +`dir` lists names, so the subdirectories with the files do not appear in the output. The files are `repo/model/yml/smallYeast.yml` and `repo/model/xml/smallYeast.xml`. `subDirs` set to `false` puts all files in one folder. The identifiers are always sorted before writing, and successive releases give a clean difference. `exportForGit` writes the five formats `mat`, `txt`, `xlsx`, `xml` and `yml` by default, and `formats` selects a subset. `COBRAtext` writes metabolite identifiers in the `txt` table, where the default is metabolite names. `mainBranchFlag` makes the export fail unless RAVEN, and GECKO for an enzyme-constrained model, is on its main branch, which ties a release to a released toolbox version. ::: :::{tab-item} 🐍 Python :sync: python @@ -248,22 +248,22 @@ print(sorted(p.relative_to(root).as_posix() for p in root.rglob("*") if p.is_fil ['dependencies.txt', 'xml/smallYeast.xml', 'yml/smallYeast.yml'] ``` -`export_for_git` returns the `model/` directory it wrote, and sorts a copy of the model first, so the files are diff-friendly and the model passed in is unchanged. Its default `formats` are all five, `yml`, `xml`, `mat`, `xlsx` and `txt`; `formats` narrows that. `sub_dirs=False` flattens the layout, and `varname` sets the variable name inside the `.mat` file for repositories that pin one. +`export_for_git` returns the `model/` directory that it wrote. It sorts a copy of the model, so the files give a clean difference and the model that was passed in is unchanged. The default `formats` are `yml`, `xml`, `mat`, `xlsx` and `txt`, and `formats` selects a subset. `sub_dirs=False` writes a flat layout, and `varname` sets the variable name inside the `.mat` file for repositories that fix one. ::: :::: :::{admonition} Troubleshooting :class: warning -- The SBML writer refuses an identifier. SBML ids must start with a letter or underscore. Both toolboxes add a prefix to a whole field when any id in it does not, so `importModel` and `read_sbml_model` strip prefixes on the way back. -- The model id differs between two copies of the same release. cobrapy decodes `__46__`-style escapes in reaction, metabolite and gene ids, but not in the model id, so yeast-GEM read from SBML reports `yeastGEM_v9__46__1__46__0`. Decode it with `re.sub(r"__(\d+)__", lambda m: chr(int(m.group(1))), model.id)`, or compare `model.name`. Do not match entities on annotations: some database ids are shared by differently named metabolites. -- Excel export fails with a missing module. The Python side needs `openpyxl`; install `raven-toolbox[excel]`. -- A YAML round trip loses a field neither format defines. RAVEN YAML carries the RAVEN model fields; anything a plugin added outside them is not guaranteed to survive. Check with a diff, not by eye. -- A release written with `write_sbml_model` diffs badly. `exportForGit` and `export_for_git` sort identifiers, but a direct `write_sbml_model` does not, so a file written that way reorders whenever the model does. +- The SBML writer rejects an identifier. SBML identifiers start with a letter or an underscore. Both toolboxes add a prefix to a whole field when one identifier in it does not, and `importModel` and `read_sbml_model` remove the prefixes on reading. +- The model identifier differs between two copies of the same release. cobrapy decodes `__46__` in the identifiers of reactions, metabolites and genes and leaves the model identifier encoded, so yeast-GEM read from SBML reports `yeastGEM_v9__46__1__46__0`. The identifier is decoded with `re.sub(r"__(\d+)__", lambda m: chr(int(m.group(1))), model.id)`, or `model.name` is compared. Matching entities on annotations fails because some database identifiers are shared by metabolites with different names. +- The Excel export fails with a missing module. The Python export needs `openpyxl`, which `raven-toolbox[excel]` installs. +- A YAML round trip drops a field. RAVEN YAML holds the RAVEN model fields, and a field that a plugin added outside them is not guaranteed to survive. A text difference of the two files shows the loss. +- A release written with `write_sbml_model` shows large differences. `exportForGit` and `export_for_git` sort the identifiers, and `write_sbml_model` called directly does not, so the file is reordered whenever the model is. ::: ## See also -- [Getting started](getting-started.md), what to do with the model once it is loaded. -- [Guide overview](index.md), the other pages and the planned pages. -- [MATLAB vs Python](../matlab-vs-python.md), the full function mapping, including everything that resolves to cobrapy. +- [Getting started](getting-started.md): the next steps with a loaded model. +- [Guide overview](index.md): the other pages. +- [MATLAB vs Python](../matlab-vs-python.md): the full function mapping, including the functions that resolve to cobrapy. diff --git a/docs/guide/kegg.md b/docs/guide/kegg.md index 47629c7..03f4a75 100644 --- a/docs/guide/kegg.md +++ b/docs/guide/kegg.md @@ -2,22 +2,22 @@ # Reconstruction from KEGG -[Reconstruction from homology](homology.md) needs a template model of a related organism. KEGG needs none: its orthology groups (KOs) are already linked to reactions, so annotating a genome with KOs gives a draft directly. Which method applies depends on one thing: whether the organism is already in KEGG. +[Reconstruction from homology](homology.md) needs a template model of a related organism. KEGG needs none, because its orthology groups (KOs) are linked to reactions, and annotating a genome with KOs gives a draft directly. The method depends on whether the organism is in KEGG. -The documentation build does not run these examples. A KEGG reconstruction downloads tens of megabytes and takes minutes, so the page is not executed with the others. The Python numbers come from raven-toolbox 3.0.0b1 with the `kegg118` artefacts. +The documentation build does not run these examples. A KEGG reconstruction downloads tens of megabytes and takes minutes. The Python numbers come from raven-toolbox 3.0.0b1 with the `kegg118` artefacts. ## Functions | MATLAB | Python | Purpose | |---|---|---| -| `getKEGGModelForOrganism` | `get_kegg_model_for_organism_from_artefacts` | draft from an organism's KEGG annotation | +| `getKEGGModelForOrganism` | `get_kegg_model_for_organism_from_artefacts` | draft from the KEGG annotation of an organism | | `getKEGGModelForOrganism` (with `fastaFile`) | `get_kegg_model_from_sequences_with_artefacts` | draft from an HMM search of the query proteins | | `getModelFromKEGG` | `build_reference_model` | assemble and cache the global KEGG model | -| `getPhylDist` | `PhylDist` | phylogenetic distance between KEGG organisms; MATLAB uses it to select one domain's genes | +| `getPhylDist` | `PhylDist` | phylogenetic distance between KEGG organisms; MATLAB uses it to select the genes of one domain | ## Organisms in KEGG -If KEGG has the species (`sce` for *S. cerevisiae*), its gene-to-KO assignments are already made, and no sequence search is needed. This method is the faster one, and the one to prefer when it applies. +When KEGG has the species (`sce` for *S. cerevisiae*), the assignments of genes to KOs exist and no sequence search is needed. This method is faster, and it applies when the organism is in KEGG. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -28,9 +28,9 @@ model = getKEGGModelForOrganism('sce', ... 'dataDir', fullfile(tempdir, 'kegg118_eukaryotes')); ``` -Without a `fastaFile`, the reconstruction runs from the organism abbreviation alone and no sequence search happens, so the HMM library is not fetched; the download is guarded on a FASTA being supplied. The reconstruction needs the global KEGG model, which `getModelFromKEGG` supplies. On first use there is no `keggModel.mat` to load, so it downloads the published artefacts, the gene-free reference model plus the KO, reaction and organism-gene tables, assembles them, and caches the result. That first build takes a while and needs a few hundred MB of disk; later calls load the cached file directly. +Without a `fastaFile`, the reconstruction runs from the organism abbreviation alone. No sequence search takes place, and the HMM library is not fetched, because the download takes place when a FASTA file is supplied. The reconstruction needs the global KEGG model, which `getModelFromKEGG` supplies. On first use there is no `keggModel.mat` to load. The function downloads the published artefacts (the reference model without genes and the tables of KOs, reactions and organism genes), assembles them and caches the result. The first build takes a while and needs a few hundred MB of disk, and later calls load the cached file. -This method does not read `dataDir`, `cutOff`, `minScoreRatioKO`, `minScoreRatioG` and `outDir`, which apply only to a sequence search. `maxPhylDist` applies only here, with the organism id `eukaryotes` or `prokaryotes`: `-1` keeps the genes of that domain, and any other value keeps the genes of every organism. The settings `keepSpontaneous`, `keepUndefinedStoich`, `keepIncomplete` and `keepGeneral` apply. +This method does not read `dataDir`, `cutOff`, `minScoreRatioKO`, `minScoreRatioG` and `outDir`, which apply to a sequence search. `maxPhylDist` applies to this method, with the organism identifier `eukaryotes` or `prokaryotes`: `-1` keeps the genes of that domain, and any other value keeps the genes of every organism. The settings `keepSpontaneous`, `keepUndefinedStoich`, `keepIncomplete` and `keepGeneral` apply. ::: :::{tab-item} 🐍 Python :sync: python @@ -49,13 +49,13 @@ print(f"{len(model.reactions)} rxns, {len(model.metabolites)} mets, " 1357 rxns, 1502 mets, 838 genes ``` -The artefacts (a reference model and three tables, about 49 MB) are fetched from the `kegg118` raven-data release on first use and cached; see [Download data and binaries](../installation/data-and-binaries.md) for where, and how to fetch them ahead of a batch run. The first run takes a few minutes and later runs take much the same time, because assembling the draft from the tables is the slow part and the download is not. +The artefacts (a reference model and three tables, about 49 MB) are fetched from the `kegg118` release of `toolbox-assets` on first use and cached. [Download data and binaries](../installation/data-and-binaries.md) gives the location and describes how to fetch them before a batch run. The first run takes a few minutes, and later runs take about as long, because assembling the draft from the tables is slow and the download is not. ::: :::: ## Organisms absent from KEGG -For an organism KEGG has never seen, the KO assignments have to be made from sequence. Both toolboxes search the organism's proteins against a library of profile HMMs, one per KO, trained on either prokaryotic or eukaryotic sequences. +For an organism that is not in KEGG, the KO assignments are made from sequence. Both toolboxes search the proteins of the organism against a library of profile HMMs, one per KO, trained on prokaryotic or on eukaryotic sequences. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -81,15 +81,15 @@ hanpo_model = get_kegg_model_from_sequences_with_artefacts("hanpo.faa", domain=" ::: :::: -This method is expensive. The eukaryotic HMM library alone is 135 MB compressed, and one timed `hmmsearch` of a proteome (*S. cerevisiae*, 6021 sequences) against the eukaryotic library (2698 profiles) took 89.7 s on 16 threads and 88.6 s on 12 threads (AMD EPYC 9354). It also needs HMMER, which has no bundle for every platform; [Download data and binaries](../installation/data-and-binaries.md) has the matrix. Without `threads`, `run_hmmsearch`, `get_kegg_model_from_sequences`, `build_ko_hmm` and `build_hmm_library` use every CPU the process may run on; `getKEGGModelForOrganism` runs `hmmsearch` on the cores assigned to MATLAB (`feature('numcores')`) and has no `threads` option. +This method is expensive. The eukaryotic HMM library is 135 MB compressed. One timed `hmmsearch` of a proteome (*S. cerevisiae*, 6021 sequences) against the eukaryotic library (2698 profiles) took 89.7 s on 16 threads and 88.6 s on 12 threads (AMD EPYC 9354). The method also needs HMMER, which is not available as a bundle for every platform, and [Download data and binaries](../installation/data-and-binaries.md) has the matrix. Without `threads`, `run_hmmsearch`, `get_kegg_model_from_sequences`, `build_ko_hmm` and `build_hmm_library` use every CPU that the process can run on. `getKEGGModelForOrganism` runs `hmmsearch` on the cores assigned to MATLAB (`feature('numcores')`) and has no `threads` option. -With a `fastaFile`, the organism id only names the model; the search uses no phylogenetic distance and ignores `maxPhylDist`. +With a `fastaFile`, the organism identifier names the model. The search uses no phylogenetic distance and ignores `maxPhylDist`. -Three cut-offs shape the assignment. `cutOff` is the HMMER significance a hit needs at all. `minScoreRatioKO` drops genes that fit a KO far worse than the best gene in it, and `minScoreRatioG` drops KOs that fit a gene far worse than that gene's best KO; the two prune the same matrix from opposite sides. All three were measured in [KEGG HMM cut-off calibration](../parameter-tuning/studies/kegg-hmm-cutoff-calibration.md); `cutOff` and `minScoreRatioG` changed from the RAVEN 2 values. At `minScoreRatioG` 0.8 and `cutOff` 1e-50, raising `minScoreRatioKO` from 0.0 to 0.5 raised precision by 0.00 to 0.03 and lowered recall by 0.00 to 0.01, and it keeps the RAVEN 2 value of 0.3. +Three cut-offs shape the assignment. `cutOff` is the HMMER significance that a hit needs. `minScoreRatioKO` removes genes that fit a KO worse than the best gene of that KO, and `minScoreRatioG` removes KOs that fit a gene worse than the best KO of that gene. The two prune the same matrix from opposite sides. [KEGG HMM cut-off calibration](../parameter-tuning/studies/kegg-hmm-cutoff-calibration.md) measured all three, and `cutOff` and `minScoreRatioG` differ from the RAVEN 2 values. At `minScoreRatioG` 0.8 and `cutOff` 1e-50, raising `minScoreRatioKO` from 0.0 to 0.5 raised the precision by 0.00 to 0.03 and lowered the recall by 0.00 to 0.01. The default keeps the RAVEN 2 value of 0.3. ## Contents of a KEGG draft -A KEGG draft is genome-scale in size and differs from a working model in every other respect. +A KEGG draft has the size of a genome-scale model, and it differs from a working model in every other respect. ```text title="the sce draft, measured" reactions: 1357 @@ -100,9 +100,9 @@ exchange rxns: 0 objective: none ``` -There are no compartments. KEGG describes reactions and not cell biology, so every metabolite is placed in one undifferentiated space. A KEGG draft cannot distinguish mitochondrial from cytosolic anything until it is localised; [Combining and simplifying](combining.md) covers the reverse operation and what is lost by it. +The draft has no compartments. KEGG describes reactions, so every metabolite is in one undifferentiated space. A KEGG draft cannot distinguish mitochondrial and cytosolic reactions until it is localised. [Combining and simplifying models](combining.md) covers the reverse operation and what it discards. -The flags decide how inclusive the draft is. `keepIncomplete` / `keep_incomplete` keeps reactions that KEGG labels incomplete, erroneous or unclear; `keepUndefinedStoich` / `keep_undefined_stoich` keeps reactions with `n` or `x` in their stoichiometry, which cannot be balanced as written. With both turned off: +The flags decide how inclusive the draft is. `keepIncomplete` / `keep_incomplete` keeps reactions that KEGG labels incomplete, erroneous or unclear. `keepUndefinedStoich` / `keep_undefined_stoich` keeps reactions with `n` or `x` in the stoichiometry, which cannot be balanced as written. With both turned off: ```python strict = get_kegg_model_for_organism_from_artefacts( @@ -115,19 +115,19 @@ print(f"{len(strict.reactions)} rxns, {len(strict.genes)} genes " 1348 rxns, 827 genes (from 1357 and 838) ``` -That is a change of nine reactions on this organism. The size of the effect differs per organism. Measure it for each organism. Unlike the other flags, `keepGeneral` / `keep_general` is off by default: general reactions are placeholders like "an alcohol + NAD+", and admitting them produces a network that appears to do far more than it can. +This removes nine reactions from this organism, and the effect differs between organisms and should be measured for each. `keepGeneral` / `keep_general` is off by default, unlike the other flags. General reactions are placeholders such as "an alcohol + NAD+", and admitting them produces a network that appears to do more than it can. :::{admonition} Troubleshooting :class: warning -- Expecting a model. No biomass reaction, no exchanges, no compartments, no medium. The draft is a reaction inventory with gene associations. -- Trusting the KO assignment. A KO is an orthology group and not a demonstrated activity in the organism, and a partial HMM hit is weaker still. -- Undefined stoichiometry. Reactions with `n` or `x` coefficients cannot be mass-balanced. They are kept by default because dropping them removes real chemistry, but they interfere with [Quality control](quality-control.md) later. -- Version drift. The artefacts are built from a specific KEGG release, `kegg118` here. Rebuilding a year later with a different release gives a different model, so record the release used. +- The draft is expected to be a model. It has no biomass reaction, no exchange reactions, no compartments and no medium. It is an inventory of reactions with gene associations. +- The KO assignment is trusted. A KO is an orthology group, which does not demonstrate an activity in the organism, and a partial HMM hit is weaker. +- The stoichiometry is undefined. Reactions with `n` or `x` coefficients cannot be mass-balanced. They are kept by default because removing them removes real chemistry, and they affect [Quality control](quality-control.md). +- The KEGG release changes. The artefacts are built from one KEGG release, `kegg118` here. A rebuild with another release gives a different model, so record the release. ::: ## See also -- [Reconstruction from homology](homology.md), the same goal with a template model in place of an orthology database. -- [Gap-filling](gap-filling.md), what to do with 1357 reactions that cannot carry flux until the draft has exchange reactions. -- [Legacy tutorial 5](../tutorials/tutorial5.md), the original KEGG reconstruction exercise from the RAVEN paper. +- [Reconstruction from homology](homology.md): the same goal with a template model in place of an orthology database. +- [Gap-filling](gap-filling.md): the step for the 1357 reactions of a draft without exchange reactions, none of which can carry flux. +- [Legacy tutorial 5](../tutorials/tutorial5.md): the original KEGG reconstruction exercise from the RAVEN paper. diff --git a/docs/guide/media.md b/docs/guide/media.md index 6536101..e9731e5 100644 --- a/docs/guide/media.md +++ b/docs/guide/media.md @@ -1,24 +1,24 @@ # Growth media and conditions -A flux balance analysis (FBA) result is a statement about a medium. The medium is set deliberately by choosing which exchange reactions are open and how wide, and a condition keeps a medium as reviewable data and not as a paragraph of bound-setting code. +A flux balance analysis (FBA) result applies to one medium. The medium is the set of open exchange reactions and the width of their bounds. A condition stores a medium as reviewable data, where a script of bound-setting calls hides it. ## Functions | MATLAB | Python | Purpose | |---|---|---| | `getExchangeRxns` | `Model.exchanges` {bdg-link-secondary}`cobrapy ` | find the exchange reactions | -| `setExchangeBounds` | `Model.medium` {bdg-link-secondary}`cobrapy ` | set a whole medium, closing the rest | -| `setParam` | `set_reaction_bounds` | set one reaction's bounds | -| `getMinimalMedium` | `minimal_medium` {bdg-link-secondary}`cobrapy ` | the smallest medium that still supports growth | +| `setExchangeBounds` | `Model.medium` {bdg-link-secondary}`cobrapy ` | set a whole medium and close all other uptakes | +| `setParam` | `set_reaction_bounds` | set the bounds of one reaction | +| `getMinimalMedium` | `minimal_medium` {bdg-link-secondary}`cobrapy ` | find the smallest medium that supports growth | | `applyCondition` | `apply_condition`, `load_condition` | apply a condition file | ## Setup -The model is `yeast-GEM.yml` from [`docs/data/`](../data/README.md), which has an aerobic minimal glucose medium applied. +The model is `yeast-GEM.yml` from [`docs/data/`](../data/README.md), with an aerobic minimal glucose medium applied. ## Open exchanges -An exchange reaction connects a boundary metabolite to nothing, so its flux is the rate at which that metabolite enters or leaves the system. yeast-GEM writes its exchanges as `metabolite <=>`, so entering is a negative flux and leaving a positive one, and the medium is the set of exchanges with a negative lower bound. A model that writes uptake as `=> metabolite` has the opposite sign; see [Solvers](solvers.md). +An exchange reaction connects a boundary metabolite to nothing, so its flux is the rate at which the metabolite enters or leaves the system. yeast-GEM writes its exchange reactions as `metabolite <=>`, so a negative flux is entry and a positive flux is exit. The medium is the set of exchange reactions with a negative lower bound. A model that writes uptake as `=> metabolite` has the opposite sign ([Solvers](solvers.md)). ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -56,9 +56,9 @@ end r_4600 Ca(2+) exchange 1000.0 ``` -`getExchangeRxns` identifies an exchange by its stoichiometry, as a reaction with no substrates or no products, so it does not depend on a naming convention. Its `reactionType` argument can return a subset chosen by bounds: `'uptake'`, `'excrete'`, `'reverse'` and `'blocked'` classify a reaction by what its bounds permit, while `'in'` and `'out'` classify by which side the boundary metabolite is on and ignore the bounds. +`getExchangeRxns` identifies an exchange reaction by its stoichiometry (a reaction without substrates or without products), so it does not depend on a naming convention. `reactionType` selects a subset. The values `'uptake'`, `'excrete'`, `'reverse'` and `'blocked'` classify a reaction by what its bounds permit. The values `'in'` and `'out'` classify by the side of the boundary metabolite and ignore the bounds. -The hand-written filter in the code and `'uptake'` ask different questions. `'uptake'` returns reactions that can only take up; every open exchange in yeast-GEM also has an upper bound of 1000, so it can secrete as well and is classified `'reverse'`. Selecting on `model.lb < 0` is what "the medium" means here. +The filter in the code and `'uptake'` answer different questions. `'uptake'` returns the reactions that can take up and cannot secrete. Every open exchange reaction in yeast-GEM also has an upper bound of 1000, so it can secrete and is classified `'reverse'`. The medium is the selection `model.lb < 0`. ::: :::{tab-item} 🐍 Python :sync: python @@ -94,11 +94,11 @@ for rxn_id, uptake in sorted(model.medium.items()): r_4600 Ca(2+) exchange 1000.0 ``` -`model.medium` is a dict of `{exchange id: maximum uptake rate}`, given as a positive number; cobrapy flips the sign, so a medium entry of `1.0` means a lower bound of `-1.0`. It lists only the exchanges that are open, so the loop prints 16 rows out of 270 reactions. +`model.medium` is a dictionary of `{exchange identifier: maximum uptake rate}`. The rate is a positive number, and cobrapy flips the sign, so an entry of `1.0` means a lower bound of `-1.0`. The dictionary lists the open exchange reactions, and the loop prints 16 of 270 reactions. ::: :::: -The two toolboxes count exchange reactions differently, and both counts are correct. cobrapy sorts single-metabolite reactions into `exchanges`, `demands` and `sinks`; `getExchangeRxns` returns all of them together. In yeast-GEM the difference is three reactions: `r_2111` (growth) and the sinks `r_4062` and `r_4064`, so MATLAB reports 273 where `model.exchanges` reports 270. +The toolboxes count the exchange reactions differently, and both counts are correct. cobrapy sorts reactions with a single metabolite into `exchanges`, `demands` and `sinks`, and `getExchangeRxns` returns all of them. In yeast-GEM the difference is three reactions, `r_2111` (growth) and the sinks `r_4062` and `r_4064`, so MATLAB reports 273 where `model.exchanges` reports 270. ## Changing one nutrient @@ -117,7 +117,7 @@ fprintf('[%g %g] -> %.4f /h\n', model.lb(idx), model.ub(idx), sol.f); [-10 1000] -> 0.8370 /h ``` -`setParam` takes a reaction id, a list of ids, or indices, and applies one value to all of them or one value each. `'lb'`, `'ub'` and `'eq'` set bounds; `'obj'` sets the objective coefficient. +`setParam` takes a reaction identifier, a list of identifiers or indices, and applies one value to all of them or one value to each. `'lb'`, `'ub'` and `'eq'` set bounds, and `'obj'` sets the objective coefficient. ::: :::{tab-item} 🐍 Python :sync: python @@ -134,15 +134,15 @@ print(glucose.bounds, "->", f"{model.slim_optimize():.4f} /h") (-10.0, 1000.0) -> 0.8370 /h ``` -`reaction.bounds = (-10.0, 1000.0)` does the same thing for ordinary edits. `set_reaction_bounds` covers the case that cobrapy refuses: it writes the underlying attributes directly, so a condition can reach `lb > ub`, which cobrapy's own setter rejects as invalid. Conditions use that to force flux through a reaction with a sentinel bound. +`reaction.bounds = (-10.0, 1000.0)` has the same effect for ordinary edits. `set_reaction_bounds` writes the underlying attributes directly, so it also accepts `lb > ub`, which the setter of cobrapy rejects as invalid. A condition uses this to force flux through a reaction with a sentinel bound. -`slim_optimize` returns the objective value alone, without building a full `Solution`, which is the cheaper call inside a loop over many bound settings. +`slim_optimize` returns the objective value without building a `Solution`, which is cheaper in a loop over many bound settings. ::: :::: ## Defining a whole medium -Assigning to the medium replaces the whole medium: everything not in the dict is closed. That makes it the right tool for "this exact recipe and nothing else", and the wrong one for "the shipped medium, but with more glucose". +Assigning to the medium replaces it, and every exchange reaction that is not in the dictionary is closed. The assignment suits an exact recipe. A change to one entry edits a copy of `dict(model.medium)`. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -166,9 +166,9 @@ growth with nothing to eat: /h complete medium: 0.8370 /h ``` -The first line prints no number. With every uptake closed the linear programme (LP) is infeasible, and `solveLP` returns `sol.f` as an empty array with `sol.stat` set to `-1`; `fprintf` prints nothing for an empty argument. Check `sol.stat`, not the value, as in [Simulating growth with FBA](fba.md). +The first line prints no number. With every uptake closed, the linear programme (LP) is infeasible, `solveLP` returns an empty array in `sol.f` and sets `sol.stat` to `-1`, and `fprintf` prints nothing for an empty argument. `sol.stat` is checked before the value is read ([Simulating growth with FBA](fba.md)). -`setExchangeBounds` is the closer counterpart to assigning `model.medium`. It addresses metabolites and not reaction ids, sets their exchange bounds, and with its default `closeOthers` shuts every other uptake, which is the replace-the-whole-recipe behaviour. `mediaOnly` restricts it to the extracellular compartment, so intracellular sink reactions are left alone. +`setExchangeBounds` corresponds to the assignment of `model.medium`. It takes metabolite names, sets their exchange bounds, and with the default `closeOthers` closes every other uptake. `mediaOnly` restricts it to the extracellular compartment, which leaves intracellular sink reactions unchanged. ::: :::{tab-item} 🐍 Python :sync: python @@ -193,17 +193,17 @@ glucose and oxygen alone: 0.0000 /h complete medium: 0.8370 /h ``` -Glucose and oxygen alone are not enough; yeast-GEM also needs nitrogen, phosphate, sulphate and a handful of ions. Building a medium from `dict(model.medium)` and editing individual entries therefore leaves the rest of the recipe intact, which writing it out in full does not. +Glucose and oxygen are not sufficient. yeast-GEM also needs nitrogen, phosphate, sulphate and several ions. A medium built from `dict(model.medium)` with individual entries edited keeps the rest of the recipe, and a medium written out in full does not. -The three results are three different states. `nan` means the LP had no solution; `0.0000` means it solved and the optimum is zero; `0.8370` means it solved and the model grows. `slim_optimize` returns `nan` for the first case and does not raise, so check `model.optimize().status` when a number is unexpected. +The three results are three different states. `nan` means that the LP had no solution, `0.0000` means that it was solved with an optimum of zero, and `0.8370` means that it was solved and the model grows. `slim_optimize` returns `nan` in the first case and does not raise. `model.optimize().status` shows the cause of an unexpected number. ::: :::: ## Anaerobic growth and conditions -Switching yeast-GEM to anaerobic conditions takes more than closing the oxygen exchange. Without oxygen the model cannot make sterols or unsaturated fatty acids, so those have to be supplied; heme a leaves the cofactor pseudoreaction; the biomass gains an FADH2 term; and two reactions that are repressed on glucose are blocked. Those edits belong together, so both toolboxes read them from one YAML file (a **condition**), which can be reviewed as data and is not buried in a script. +An anaerobic yeast-GEM needs more than a closed oxygen exchange. Without oxygen the model cannot make sterols and unsaturated fatty acids, so they are supplied. Heme a leaves the cofactor pseudoreaction, the biomass gains an FADH2 term, and two reactions that glucose represses are blocked. These edits belong together, and both toolboxes read them from one YAML file, a condition, which is reviewed as data. -[`anaerobic.yml`](../data/anaerobic.yml) is that file, transcribed from yeast-GEM's own `anaerobicModel.m`: +[`anaerobic.yml`](../data/anaerobic.yml) is that file, transcribed from `anaerobicModel.m` of yeast-GEM, without its step for the amino-acid ratios: ```yaml title="anaerobic.yml (abridged)" cofactor_pseudoreaction: @@ -227,9 +227,9 @@ bounds: - { rxn: r_0714, lb: 0, ub: 0 } # MDH2, repressed on glucose ``` -The schema is narrow on purpose: an optional `prelude` that resets the exchanges, an optional cofactor-pseudoreaction edit, an optional biomass-stoichiometry delta, a list of per-reaction `bounds`, and an `expected_uptake_count` that triggers a warning when the condition does not match the model. +The schema has five parts: an optional `prelude` that resets the exchange reactions, an optional edit of the cofactor pseudoreaction, an optional change of the biomass stoichiometry, a list of `bounds` per reaction and an `expected_uptake_count`, which triggers a warning when it differs from the number of `bounds` entries with a lower bound of -1000. -Removing a metabolite from the cofactor pseudoreaction leaves it charge imbalanced, so `charge_balance_met` names the metabolite whose coefficient absorbs the difference. Both toolboxes recompute it and do not store the new value, so the file is readable as a statement of intent. +Removing a metabolite from the cofactor pseudoreaction leaves its charge unbalanced, and `charge_balance_met` names the metabolite whose coefficient absorbs the difference. Both toolboxes recompute the coefficient and do not store it, so the file states the intent. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -265,15 +265,15 @@ bounds changed: 12 anaerobic growth: 0.1615 /h ``` -`apply_condition` edits the model in place and returns it, so pass a copy to keep the aerobic model as well. It also takes the path directly: `apply_condition(model, "anaerobic.yml")`. `load_condition` is the parse step on its own, for inspecting or editing a condition before applying it. +`apply_condition` edits the model in place and returns it, so a copy should be passed when the aerobic model is needed. It also accepts the path: `apply_condition(model, "anaerobic.yml")`. `load_condition` parses the file without applying it, which allows a condition to be inspected or edited before it is applied. -Compare 0.1615 /h with the 0.8370 /h that the same model reached aerobically in [Defining a whole medium](#defining-a-whole-medium): both are on 10 mmol glucose, because the condition is applied to the model as that section left it. Fermentation yields far less ATP per glucose than respiration; the ratio of the two growth rates is what the comparison is for. +The growth rate of 0.1615 /h compares with the 0.8370 /h that the same model reached aerobically in [Defining a whole medium](#defining-a-whole-medium). Both are on 10 mmol glucose, because the condition is applied to the model as that section left it. Fermentation yields much less ATP per glucose than respiration, and the ratio of the two growth rates is the comparison. ::: :::: ## Minimal medium -A medium copied from a paper usually contains more than the model needs. `getMinimalMedium` and cobrapy's `minimal_medium` search for the smallest set of uptakes that supports a given growth rate, which identifies which component is necessary for growth, and finds a nutrient the model does not need because a gap-filled reaction produces it internally. +A medium copied from a paper often contains more than the model needs. `getMinimalMedium` and `minimal_medium` of cobrapy search for the smallest set of uptakes that supports a given growth rate. The result shows which components are necessary, and it shows a nutrient that the model does not need because a gap-filled reaction produces it internally. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -309,9 +309,9 @@ medium = getMinimalMedium(model, 'minGrowth', 0.9 * sol.f); ============================================================== ``` -Candidates are the exchanges that have `lb < 0`; a nutrient the model cannot take up is never proposed. `minGrowth` defaults to 10 % of the optimum under the current bounds, which is loose enough that most components drop out, so state it explicitly, as here, when the question is about a particular growth rate. `verbose` set to `false` suppresses the table and returns the ids alone. +The candidates are the exchange reactions with `lb < 0`, so a nutrient that the model cannot take up is never proposed. `minGrowth` defaults to 10 % of the optimum under the current bounds, a limit loose enough that most components drop out, and it is set explicitly when the question concerns a given growth rate. `verbose` set to `false` suppresses the table and returns the identifiers. -Of the 16 open uptakes, 14 are kept: water and H+ are dropped because other reactions supply them. `getMinimalMedium` solves a mixed-integer linear programme (MILP) that minimises the number of open uptakes. RAVEN's bundled GLPK solves it too, with a warning that recommends Gurobi. cobrapy's `minimal_medium` by default solves an LP that minimises the summed uptake flux (in mmol/gDW/h, all metabolites weighted equally), so it returns rates. +Of the 16 open uptakes, 14 are kept. Water and H+ are dropped because other reactions supply them. `getMinimalMedium` solves a mixed-integer linear programme (MILP) that minimises the number of open uptakes. The GLPK that ships with RAVEN solves it as well, with a warning that recommends Gurobi. `minimal_medium` of cobrapy solves by default an LP that minimises the summed uptake flux (in mmol/gDW/h, all metabolites weighted equally), so it returns rates. ::: :::{tab-item} 🐍 Python :sync: python @@ -340,22 +340,21 @@ r_4597 0.001 r_4600 0.000 ``` -The values are uptake rates and not bounds, and the zeros are components that the solution needs at a rate below the printed precision, not components the solution does without. `minimize_components=True` solves the component-count MILP (it runs on GLPK too). It returns the import fluxes of one feasible solution, whose index is the medium; the rates are not minimal. Treat values near the solver tolerance as zero: with Gurobi, a heme a uptake of 1e-6 appears in this model's answer. An integer value returns alternative media as columns. +The values are uptake rates. A zero is a component that the solution needs at a rate below the printed precision and is no component that the solution does without. `minimize_components=True` solves the MILP for the number of components, which also runs on GLPK. It returns the import fluxes of one feasible solution, which define the medium, and the rates are not minimal. Values close to the solver tolerance count as zero, and with Gurobi the answer for this model contains a heme a uptake of 1e-6. An integer value returns alternative media as columns. ::: :::: :::{admonition} Troubleshooting :class: warning -- `infeasible` right after setting a medium: something essential is closed. Reopen the medium one component at a time, or start from `minimal_medium` on the working model and compare. -- A sign error from mixing conventions: for an exchange written `metabolite <=>`, as in yeast-GEM, uptake is a negative flux, but `model.medium` takes positive numbers. Both are correct; mixing them is not. -- The model grows without a carbon source: usually a leak, where some reaction produces carbon from nothing. [Quality control](quality-control.md) covers the diagnosis, with `canExchange` and `analyse_topology`. -- Results that cannot be reproduced: if a medium is defined inside a script, the next person runs a different one. A condition file is data, and a diff shows its changes. +- The model is infeasible after setting a medium. An essential nutrient is closed. Reopen the components one at a time, or run `minimal_medium` on the working model and compare. +- The signs of two conventions are mixed. For an exchange written `metabolite <=>`, as in yeast-GEM, uptake is a negative flux, and `model.medium` takes positive numbers. Both are correct, and mixing them gives wrong bounds. +- The model grows without a carbon source. A reaction usually produces carbon from nothing. [Quality control](quality-control.md) describes the diagnosis with `canExchange` and `gap_report`. +- A result cannot be reproduced. A medium defined inside a script differs between users who edit the script. A condition file is data, and a text difference shows its changes. ::: ## See also -- [Simulating growth with FBA](fba.md), the solve that uses the medium. -- [Reading and writing models](io.md), keeping a model and its conditions in a - repository. -- [MATLAB vs Python](../matlab-vs-python.md), the full function mapping. +- [Simulating growth with FBA](fba.md): the solve that uses the medium. +- [Reading and writing models](io.md): keeping a model and its conditions in a repository. +- [MATLAB vs Python](../matlab-vs-python.md): the full function mapping. diff --git a/docs/guide/model-structure.md b/docs/guide/model-structure.md index 17a83ef..bcdca30 100644 --- a/docs/guide/model-structure.md +++ b/docs/guide/model-structure.md @@ -1,16 +1,16 @@ # Model structure and identifiers -The same model is a struct of parallel arrays in MATLAB and a graph of objects in Python. Knowing which field corresponds to which attribute is most of what translating a script between the two requires. +A model is a struct of parallel arrays in MATLAB and a graph of objects in Python. A script is translated between the two by mapping each field to its attribute. ## Functions | MATLAB | Python | Purpose | |---|---|---| | `checkModelStruct` | `check_model` | report problems in a model | -| `sortIdentifiers` | `sort_identifiers` | sort reactions, metabolites and genes by id | -| `getIndexes` | `DictList.get_by_id` {bdg-link-secondary}`cobrapy ` | look up a reaction, metabolite or gene by id | -| `getIndexes` (`'metcomps'`) | `parse_name_comp`, then a name and compartment filter | resolve a `name[comp]` token | -| no equivalent | `subsystem_to_str` | one subsystem string, whatever the source stored | +| `sortIdentifiers` | `sort_identifiers` | sort reactions, metabolites and genes by identifier | +| `getIndexes` | `DictList.get_by_id` {bdg-link-secondary}`cobrapy ` | find a reaction, metabolite or gene by identifier | +| `getIndexes` (`'metcomps'`) | `parse_name_comp`, then a filter on name and compartment | resolve a `name[comp]` token | +| no equivalent | `subsystem_to_str` | one subsystem string from any stored form | | `addIdentifierPrefix`, `removeIdentifierPrefix` | handled on read and write {bdg-link-secondary}`cobrapy ` | SBML identifier prefixes | | `ravenCobraWrapper` | not needed | convert between RAVEN and COBRA structs | @@ -18,27 +18,27 @@ The same model is a struct of parallel arrays in MATLAB and a graph of objects i | RAVEN field | cobrapy | Note | |---|---|---| -| `model.rxns` | `model.reactions` (ids) | a `DictList`, indexable by id or position | +| `model.rxns` | `model.reactions` (identifiers) | a `DictList`, indexable by identifier or position | | `model.rxnNames` | `reaction.name` | | | `model.mets`, `model.metNames` | `model.metabolites`, `metabolite.name` | | | `model.metFormulas` | `metabolite.formula` | | | `model.genes` | `model.genes` | | | `model.grRules` | `reaction.gene_reaction_rule` | same Boolean syntax | -| `model.rxnGeneMat` | `reaction.genes`, `gene.reactions` | the mapping is navigable from either side | +| `model.rxnGeneMat` | `reaction.genes`, `gene.reactions` | navigable from both sides | | `model.lb`, `model.ub` | `reaction.bounds` | | | `model.rev` | derived from the bounds | `reaction.reversibility` is read-only | -| `model.S` | `create_stoichiometric_matrix(model)` | built on demand, not stored | +| `model.S` | `create_stoichiometric_matrix(model)` | built on demand | | `model.comps`, `model.metComps` | `metabolite.compartment`, `model.compartments` | | -| `model.subSystems` | `reaction.subsystem` | RAVEN allows several per reaction | -| `model.c` | `model.objective` | an expression, not a coefficient vector | +| `model.subSystems` | `reaction.subsystem` | RAVEN allows several subsystems per reaction | +| `model.c` | `model.objective` | an expression | -In MATLAB the caller keeps parallel arrays aligned, and in Python the objects hold their own references. Deleting a reaction in RAVEN means removing the same row from every reaction-length field, and the matching column of `model.S`; `removeReactions` exists for this, so a deletion is not a one-line edit. In cobrapy the object holds its own links, so `model.remove_reactions([...])` is enough and there is no state left behind to go stale. +In MATLAB the caller keeps the parallel arrays aligned. `removeReactions` removes the same position from every reaction-length field and the matching column of `model.S`, so a manual deletion takes more than one line. In cobrapy each object holds its own links, and `model.remove_reactions([...])` leaves no stale state. -That difference also decides where mistakes become visible. A RAVEN model can be left in a state no function rejects but later functions misread, and RAVEN ships a validator for this ([Checking a model](#checking-a-model)). A `cobra.Model` cannot reach most of those states, because the class maintains the structure. +The difference also decides where errors become visible. A RAVEN model can be left in a state that no function rejects and that later functions misread, which `checkModelStruct` detects ([Checking a model](#checking-a-model)). A `cobra.Model` cannot reach most of these states, because the class maintains the structure. ## Setup -`smallYeast.yml` from [`docs/data/`](../data/README.md). +The examples use `smallYeast.yml` from [`docs/data/`](../data/README.md). ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -76,9 +76,9 @@ PGI: Glucose-6-phosphate isomerase genes: YBR196C ``` -Most RAVEN functions take or return indices, and `getIndexes` converts between identifiers and indices. The third argument names the field to search, because the same string can be a reaction id in one field and nothing in another. It takes `'rxns'`, `'mets'`, `'genes'`, `'metnames'` or `'metcomps'`, plus `'ecrxns'`, `'ecenzymes'` and `'ecgenes'` on a GECKO model that carries a `model.ec` structure. +Most RAVEN functions take or return indices, and `getIndexes` converts between identifiers and indices. The third argument names the field to search, because one string can be a reaction identifier in one field and absent from another. It takes `'rxns'`, `'mets'`, `'genes'`, `'metnames'` or `'metcomps'`, and on a GECKO model with a `model.ec` structure also `'ecrxns'`, `'ecenzymes'` and `'ecgenes'`. -A miss is an error, not a zero: `getIndexes` raises `Could not find object 'X' in the model` in every mode except `'metnames'`, which returns an empty vector. That exception exists because a metabolite name can occur in several compartments, so `'metnames'` returns every matching index. In `'metcomps'` mode a token that matches more than one metabolite (the same name twice in one compartment) is an error as well, naming the ids of the metabolites it matched. Asking for several names at once gives back a cell array with one index vector per name; asking for exactly one name unwraps that to a plain vector. +An identifier that is not found raises the error `Could not find object 'X' in the model` in every mode except `'metnames'`, which returns an empty vector. A metabolite name can occur in several compartments, and `'metnames'` returns every matching index. In `'metcomps'` mode, a token that matches more than one metabolite (the same name twice in one compartment) raises an error that names the matched metabolites. A request for several names returns a cell array with one index vector per name, and a request for one name returns a plain vector. ::: :::{tab-item} 🐍 Python :sync: python @@ -96,7 +96,7 @@ genes: YBR196C also reachable as: PGI ``` -There is no index to carry around: `DictList` looks up by id, and the object is the handle. `get_by_id` raises `KeyError` on a miss. The attribute form `model.reactions.PGI` is the same lookup, and works only for ids that are valid Python names, so `model.reactions.r_0001` resolves but an id containing a dot or a hyphen has to go through `get_by_id`. Both forms return the same object, and keeping a reference to it stays valid across edits elsewhere in the model. +`DictList` looks up an object by identifier and the object itself is the handle, so no index is carried around. `get_by_id` raises `KeyError` when the identifier is absent. The attribute form `model.reactions.PGI` is the same lookup, and it works for identifiers that are valid Python names: `model.reactions.r_0001` resolves, and an identifier with a dot or a hyphen needs `get_by_id`. Both forms return the same object, and a reference to it stays valid across edits elsewhere in the model. ::: :::: @@ -119,7 +119,7 @@ disp(model.comps{model.metComps(i)}); % the compartment of one metabolite c ``` -`model.metComps` holds an index into `model.comps`, not the compartment letter, so reading a metabolite's compartment always takes the two steps above. The indirection keeps compartment names editable in one place: renaming a compartment is a single edit to `model.compNames`, with nothing to update per metabolite. +`model.metComps` holds an index into `model.comps`, so the compartment of a metabolite takes the two steps above. A compartment is renamed with one edit to `model.compNames`, and no metabolite needs to be updated. ::: :::{tab-item} 🐍 Python :sync: python @@ -134,11 +134,11 @@ print(model.metabolites.get_by_id("G6P_c").compartment) c ``` -`model.compartments` maps id to name and is derived from the metabolites, so a compartment exists exactly as long as something is in it. +`model.compartments` maps identifier to name and is derived from the metabolites, so a compartment exists while at least one metabolite is in it. ::: :::: -RAVEN also writes metabolite names as `name[comp]` in Excel and text exports, and accepts that form on input. Both toolboxes split the token on the last bracketed group, so a name that itself contains brackets resolves. +RAVEN writes metabolite names as `name[comp]` in Excel and text exports and accepts that form on input. Both toolboxes split the token at the last bracketed group, so a name that itself contains brackets resolves. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -174,13 +174,13 @@ print([m.id for m in matches]) ['G6P_c'] ``` -`parse_name_comp` returns the compartment as `None` when there is no trailing bracket, so one call handles both forms and the caller decides what a missing compartment means. It splits the string and never sees a model; the `query` after it is the lookup that `getIndexes` performs. An empty list is a miss, and more than one entry means the name is duplicated in that compartment. The function is imported from `raven_toolbox.utils.parse`; the `raven_toolbox.utils` package namespace does not re-export it. +`parse_name_comp` splits the string without access to a model, and the `query` after it is the lookup that `getIndexes` performs. The compartment is `None` when the string has no trailing bracket, so one call handles both forms and the caller decides what a missing compartment means. An empty list means that the name is absent, and more than one entry means that the name is duplicated in that compartment. The function is imported from `raven_toolbox.utils.parse`, and the namespace `raven_toolbox.utils` does not re-export it. ::: :::: ## Subsystems -RAVEN lets a reaction belong to several subsystems, so an entry of `model.subSystems` is either a single string or a cell array of them. The field as a whole must have one of the two shapes; `checkModelStruct` reports mixing them ([Checking a model](#checking-a-model)). cobrapy stores one string per reaction, and `subsystem_to_str` normalises whichever form a model arrived with. +RAVEN allows several subsystems per reaction, so an entry of `model.subSystems` is a string or a cell array of strings. The field as a whole has one of the two shapes, and `checkModelStruct` reports a mixture ([Checking a model](#checking-a-model)). cobrapy stores one string per reaction, and `subsystem_to_str` converts either form to a string. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -215,13 +215,13 @@ Glycolysis;Pentose phosphate pathway Glycolysis ``` -cobrapy declares `Reaction.subsystem` a plain string but does not enforce it, so a model that came through RAVEN, or through hand-written YAML, can hold a list there, and no error is raised until something tries to concatenate it. `subsystem_to_str` joins the parts with `;` and does not take only the first, so no name is dropped, and returns `""` for an empty or absent subsystem. Use it wherever a subsystem is printed, compared or written out. +cobrapy declares `Reaction.subsystem` as a string and does not enforce it. A model that came through RAVEN or through hand-written YAML can hold a list there, and the error appears when code concatenates it. `subsystem_to_str` joins the parts with `;`, so no name is dropped, and returns `""` for an empty or absent subsystem. `subsystem_to_str` is applied wherever a subsystem is printed, compared or written. ::: :::: ## Checking a model -Both toolboxes provide a validator, and they check different things, because the two representations fail in different ways. +Each toolbox has a validator, and the two check different things because the two representations fail in different ways. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -244,9 +244,9 @@ fprintf('%s\n', issues(1).message); The "subSystems" field must be a cell array of chars, *or* a cell array of cell arrays of chars ``` -`checkModelStruct` validates the struct itself: that required fields are present, that each holds the type and length it should, and that identifiers are non-empty, unique and legal. On top of that it reports advisory findings: metabolites and genes nothing uses, bounds that contradict each other, a missing or ambiguous objective, malformed gene rules and formulas, and cross-references that point nowhere. Every finding carries a `category` (`missing_field`, `wrong_type`, `empty_id`, `duplicate`, `invalid_id`, `invalid_bounds`, `unused`, `objective`, `gpr`, `invalid_formula`, `cross_reference` or `other`), a `target` naming the field or identifier involved, and the full `message`. +`checkModelStruct` validates the struct: required fields are present, each field has the expected type and length, and identifiers are non-empty, unique and valid. It also reports advisory findings: metabolites and genes that nothing uses, contradicting bounds, a missing or ambiguous objective, malformed gene rules and formulas, and cross-references that point nowhere. A finding has a `category` (`missing_field`, `wrong_type`, `empty_id`, `duplicate`, `invalid_id`, `invalid_bounds`, `unused`, `objective`, `gpr`, `invalid_formula`, `cross_reference` or `other`), a `target` with the field or identifier concerned, and the full `message`. -Asking for an output argument returns the findings and neither prints nor throws. Called without one it reports as it goes, and `throwErrors` decides whether a structural problem stops the script; advisory findings are warnings either way. +With an output argument, the function returns the findings and neither prints nor throws. Without one, it reports while it checks, and `throwErrors` decides whether a structural problem stops the script. Advisory findings are warnings in both cases. ::: :::{tab-item} 🐍 Python :sync: python @@ -264,15 +264,15 @@ for issue in issues[:5]: 0 issue(s) ``` -`check_model` has no field types to check, because a `cobra.Model` cannot hold a misshapen field. It checks the curation layer that cobrapy leaves to the caller: metabolites and genes no reaction uses, reactions with no metabolites, metabolites without a name, two metabolites sharing a name inside one compartment, and an objective that is missing or spread over several reactions. Each `ModelIssue` carries `category`, `object_id` and `message`, and the function returns a list and does not raise, so a script can filter by category and decide for itself what is fatal. +`check_model` does not check field types, because a `cobra.Model` cannot hold a misshapen field. It checks what cobrapy leaves to the caller: metabolites and genes that no reaction uses, reactions without metabolites, metabolites without a name, two metabolites with the same name in one compartment, and an objective that is missing or spread over several reactions. A `ModelIssue` has `category`, `object_id` and `message`. The function returns a list and does not raise, so a script filters by category and decides which findings are fatal. -A `0` here means no curation problem was found; it does not imply that the model satisfies `checkModelStruct`. The two lists overlap only on the unused-element and objective checks. +A result of `0` means that no curation problem was found. It does not mean that the model satisfies `checkModelStruct`, because the two lists overlap in the checks for unused elements and for the objective. ::: :::: -## Sorting identifiers before a commit +## Sorting identifiers -Sorting makes the diff between two versions of a model readable, and `exportForGit` and `export_for_git` offer it too. Without it, adding one reaction can shift everything after it and turn a one-line change into a whole-file diff. +A sorted model gives a readable difference between two versions, as adding one reaction otherwise shifts everything after it and turns a one-line change into a whole-file difference. `exportForGit` and `export_for_git` sort as well. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -287,7 +287,7 @@ fprintf('%s\n', strjoin(sortedModel.rxns(1:5)', ', ')); ACO, ACS, ADH1, ALD6, ATPX ``` -`sortIdentifiers` returns a new struct and leaves the original alone. It permutes reactions, metabolites, genes and compartments together with every field indexed by them, `model.S` and `model.rxnGeneMat` included, so the model stays internally consistent. +`sortIdentifiers` returns a new struct and leaves the original unchanged. It reorders reactions, metabolites, genes and compartments together with every field indexed by them, including `model.S` and `model.rxnGeneMat`, so the model stays consistent. ::: :::{tab-item} 🐍 Python :sync: python @@ -305,26 +305,26 @@ print("same object:", sorted_model is model) same object: True ``` -`sort_identifiers` sorts in place and returns the model it was given, so the name on the left is not a second copy. Pass `model.copy()` when the original order matters. Compartments are a plain dict and are left alone here; the writers emit them in order where it is relevant. +`sort_identifiers` sorts in place and returns the model it received, and the returned name refers to the same model. `model.copy()` should be passed when the original order is needed. Compartments are stored as a dictionary and are not sorted here; the writers emit them in order where it matters. ::: :::: -## Converting to and from the COBRA Toolbox +## Conversion to and from the COBRA Toolbox -RAVEN and the COBRA Toolbox use different field names for the same model. In MATLAB, `ravenCobraWrapper` converts a struct in either direction, inferring which direction from the fields it finds. There is no Python equivalent because there is nothing to convert: a raven-toolbox model is a `cobra.Model`, which every Python COBRA tool takes. +RAVEN and the COBRA Toolbox use different field names for the same model. In MATLAB, `ravenCobraWrapper` converts a struct in either direction and infers the direction from the fields it finds. Python needs no conversion, because a raven-toolbox model is a `cobra.Model`, which every Python COBRA tool accepts. :::{admonition} Troubleshooting :class: warning -- Fields drift out of alignment. In MATLAB, editing one reaction-length field by hand without editing the others leaves a model that looks fine and fails later, often far from the edit. `checkModelStruct` detects it. -- A field ends up with mixed contents. `model.subSystems` takes strings or cell arrays of strings, but not both in the same model. Convert the whole field, not one entry. -- `model.rev` disagrees with the bounds. RAVEN stores reversibility explicitly, so it can contradict `lb` and `ub` after a manual edit, and different functions consult different ones. Python cannot get into that state, because `reaction.reversibility` is computed from the bounds on every read. -- Subsystems come back as a list. Code that assumes `reaction.subsystem` is a string breaks on a model imported from RAVEN. Use `subsystem_to_str`. -- A name lookup returns several indices. `getIndexes` with `'metnames'` matches a name in every compartment it occurs in. Use `'metcomps'` and the `name[comp]` form when one specific metabolite is meant. +- Fields are out of alignment. In MATLAB, a hand edit of one reaction-length field without the others leaves a model that works and fails later, often far from the edit. `checkModelStruct` detects it. +- A field has mixed contents. `model.subSystems` holds strings or cell arrays of strings, one kind per model. The whole field is converted. +- `model.rev` disagrees with the bounds. RAVEN stores the reversibility explicitly, so it can contradict `lb` and `ub` after a manual edit, and different functions read different fields. Python cannot reach this state, because `reaction.reversibility` is computed from the bounds on every read. +- Subsystems are lists. Code that assumes a string in `reaction.subsystem` fails on a model imported from RAVEN. `subsystem_to_str` handles both forms. +- A name lookup returns several indices. `getIndexes` with `'metnames'` matches the name in every compartment. `'metcomps'` with the `name[comp]` form selects one metabolite. ::: ## See also -- [Getting started](getting-started.md), loading a model and inspecting it. -- [Reading and writing models](io.md), where identifier prefixes come from. -- [MATLAB vs Python](../matlab-vs-python.md), the full function mapping. +- [Getting started](getting-started.md): loading and inspecting a model. +- [Reading and writing models](io.md): the origin of identifier prefixes. +- [MATLAB vs Python](../matlab-vs-python.md): the full function mapping. diff --git a/docs/guide/quality-control.md b/docs/guide/quality-control.md index c6c351e..889e067 100644 --- a/docs/guide/quality-control.md +++ b/docs/guide/quality-control.md @@ -1,21 +1,21 @@ # Quality control -A model that loads and solves can be wrong. The checks that apply before a result can be trusted are: whether the structure is sound, whether the reactions balance, whether anything is disconnected, and whether the model can make something out of nothing. +A model that loads and solves can be wrong. Four checks come before a result is trusted: the structure is sound, the reactions balance, no part of the network is disconnected, and the model cannot make mass from nothing. ## Functions | MATLAB | Python | Purpose | |---|---|---| | `checkModelStruct` | `check_model` | structural problems | -| `getElementalBalance` | `get_elemental_balance` | mass balance, reaction by reaction | +| `getElementalBalance` | `get_elemental_balance` | mass balance of each reaction | | `haveFlux` | `find_blocked_reactions` {bdg-link-secondary}`cobrapy ` | reactions that cannot carry flux under the current bounds | -| `canExchange` | `analyse_topology` | which metabolites the model can make, or consume, given its medium | -| `findLeakMetabolite` | `find_leak_metabolite`, `gap_report` | can the model make something from nothing | -| `gapReport` | `check_model` + `analyse_topology` | one summary of the gaps | +| `canExchange` | `gap_report` | metabolites that the model can make or consume with its medium | +| `findLeakMetabolite` | `find_leak_metabolite`, `gap_report` | mass produced from nothing | +| `gapReport` | `gap_report` | one summary of the gaps | ## Setup -`smallYeastBad.yml` is the small yeast model used throughout this guide, with deliberate errors, which are what these checks detect. +`smallYeastBad.yml` is the small yeast model of this guide with deliberate errors, which the checks detect. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -50,7 +50,7 @@ print(len(model.reactions), "reactions,", len(model.metabolites), "metabolites") ## Structural soundness -The first pass checks the model as a data structure: metabolites that no reaction uses, reactions with no metabolites, genes that nothing refers to and a missing objective. The two validators do not check the same list, so their counts differ; [Model structure and identifiers](model-structure.md) sets out what each one covers. +The first pass checks the model as a data structure: metabolites that no reaction uses, reactions without metabolites, genes that nothing refers to and a missing objective. The two validators check different lists, so their counts differ. [Model structure and identifiers](model-structure.md) describes what each covers. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -65,7 +65,7 @@ fprintf('%d issue(s)\n', numel(issues)); 2 issue(s) ``` -The two are a missing objective, reported as `invalid_id` because an objective-less model cannot be written as FBC v2 SBML, and a `cross_reference` finding: one InChI string matches two different metabolite names. The second finding is a consequence of the corrupted formula on `F6P_c` that [Comparison with a trusted model](#comparison-with-a-trusted-model) finds by comparison, as it appears in the annotations. +The two findings are a missing objective, reported as `invalid_id` because a model without an objective cannot be written as FBC version 2 SBML, and a `cross_reference` finding: one InChI string matches two metabolite names. The second finding follows from the corrupted formula of `F6P_c`, which [Comparison with a trusted model](#comparison-with-a-trusted-model) finds by comparison, and which appears in the annotations. ::: :::{tab-item} 🐍 Python :sync: python @@ -88,7 +88,7 @@ for issue in issues[:5]: ## Reaction balance -An unbalanced reaction can create matter, and one that creates ATP or a redox carrier inflates every prediction the model makes. +An unbalanced reaction can create matter. A reaction that creates ATP or a redox carrier inflates every prediction of the model. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -151,17 +151,17 @@ for balance in balances: ATPX: {'H': 2.0, 'O': 1.0} ``` -Exchange reactions read as unbalanced by design: they are where mass enters and leaves, so filter them out before judging the number. +Exchange reactions are unbalanced by design, because they are where mass enters and leaves. They are filtered out before the number is judged. -`getElementalBalance` returns a struct and prints nothing by default. Its `printUnbalanced` and `printUnparsable` options add warnings for the two failure kinds separately, which distinguishes a reaction that is wrong from one that is undecidable. `rxns` restricts the check to a subset. +`getElementalBalance` returns a struct and prints nothing by default. `printUnbalanced` and `printUnparsable` add warnings for the two failure kinds separately, which separates a wrong reaction from an undecidable one. `rxns` restricts the check to a subset. ::: :::: ## Reactions that cannot carry flux -A reaction that cannot carry flux under the model's current bounds is either a gap, a mistake, or waiting for a medium that is not open. Both functions test the bounds as they are; neither explores other media. It is also the cheapest of these checks to run. +A reaction that cannot carry flux under the current bounds is a gap, an error, or a reaction that waits for a medium that is not open. Both functions test the bounds as they are and do not explore other media. This is the cheapest of the checks. -`haveFlux` counts a reaction as active when it carries more than its `cutOff`, `1e-6` by default, which keeps solver noise from reading as flux. It shuffles the order in which reactions are tested, and takes a `seed` so that order is reproducible. +`haveFlux` counts a reaction as active when its flux exceeds `cutOff`, which is `1e-6` by default and keeps solver noise from counting as flux. It tests the reactions in shuffled order, and `seed` makes the order reproducible. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -195,11 +195,11 @@ blocked: ['ACO', 'ACS', 'ADH1', 'ADH2', 'ALD6'] ::: :::: -`smallYeastBad` has glucose and oxygen uptake open, and the same test on `smallYeast` with that medium finds 52 of 53 reactions active: the 52 blocked here are blocked by the errors that the other checks find. The two that do carry flux, `SDH` and `FRDS2`, are each other's reverse and only cycle (see [Loops in flux ranges](fva.md#loops-in-flux-ranges)). +`smallYeastBad` has glucose and oxygen uptake open. The same test on `smallYeast` with that medium finds 52 of 53 reactions active, and the 52 blocked reactions here are blocked by the errors that the other checks find. The two reactions that carry flux, `SDH` and `FRDS2`, are the reverse of each other and form a cycle ([Loops in flux ranges](fva.md#loops-in-flux-ranges)). -## Making something from nothing +## Mass from nothing -With every uptake closed, a correct model can produce nothing at all. If it makes a metabolite, some reaction is unbalanced or a loop is creating mass: the most damaging class of error, because such a model grows without a carbon source. +A correct model with every uptake closed produces nothing. When it makes a metabolite, a reaction is unbalanced or a loop creates mass. This is the most damaging class of error, because such a model grows without a carbon source. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -225,7 +225,7 @@ closed model feasible: 1 one leak: F6P_c ``` -`getExchangeRxns` returns every reaction with metabolites on one side only, so sinks and demands are closed along with the exchanges. A model that is infeasible once closed, for instance because of a forced maintenance flux, cannot show a leak, so check that first. `canExchange` counts the leaks; `findLeakMetabolite` finds one, using as few reactions as possible, which gives a starting point. Both take the direction as their second argument, and `'consume'` asks the mirror-image question: what this model can take up without limit. +`getExchangeRxns` returns every reaction with metabolites on one side, so the sinks and demands are closed together with the exchange reactions. A model that is infeasible when closed, for instance because of a forced maintenance flux, cannot show a leak. The feasibility of the closed model should be checked before the leak test. `canExchange` counts the leaks. `findLeakMetabolite` finds one leak with the smallest total flux, and `minNrFluxes` minimises the number of reactions instead. The leak is a starting point. Both take the direction as the second argument, and `'consume'` asks the reverse question: which metabolites the model can take up without limit. ::: :::{tab-item} 🐍 Python :sync: python @@ -250,19 +250,23 @@ closed model: optimal one leak: ['F6P_c', 'GP_c', 'PYR_c'] ``` -`gap_report` closes every boundary reaction itself, adds an output for every metabolite, and lists in `can_produce_without_input` the ones whose output can carry flux, as `canExchange` does. If the model is infeasible under its own bounds, or once its boundary is closed, it stops with an error that names the reactions with a forced flux; set ATP maintenance to zero first. `gapReport` does the same. `find_leak_metabolite` is the port of `findLeakMetabolite` and does not close anything, so give it the closed copy. It returns a list: more than one id means those metabolites could only be exchanged together. For a manual closure, close `model.boundary`; `model.exchanges` omits sinks and demands because cobrapy does not count them as exchanges, and an open sink is an input. `analyse_topology` answers the connectivity half of the same question without solving an LP. Without `seeds` it takes as seeds the metabolites of every reaction with only substrates and a negative lower bound or only products and a positive upper bound, and without `targets` the metabolites of the first objective reaction; a model with no objective reaction and no `targets` raises `ValueError`. +`gap_report` closes every boundary reaction, adds an output reaction for every metabolite, and lists in `can_produce_without_input` the metabolites whose output can carry flux, as `canExchange` does. When the model is infeasible with its own bounds or with the closed boundary, the function stops with an error that names the reactions with a forced flux. ATP maintenance should be set to zero before the call. `gapReport` behaves the same way. + +`find_leak_metabolite` is the port of `findLeakMetabolite` and closes nothing, so it receives the closed copy. It returns a list, and more than one identifier means that the metabolites can be exchanged together. A manual closure closes `model.boundary`. `model.exchanges` omits sinks and demands, because cobrapy does not count them as exchanges, and an open sink is an input. + +`analyse_topology` answers the connectivity part of the question without solving a linear program, and it requires a `templates` argument. Without `seeds`, the seeds are the metabolites of every reaction that has substrates alone and a negative lower bound or products alone and a positive upper bound. Without `targets`, the targets are the metabolites of the first objective reaction. A model without an objective reaction and without `targets` raises `ValueError`. ::: :::: :::{admonition} Outlets for by-products in leak tests :class: note -The leak-test functions assume every metabolite can be excreted while the test runs: `canExchange` and `gap_report` add an output reaction for each metabolite they check, and `findLeakMetabolite` and `find_leak_metabolite` take `allowExcretion` (`allow_excretion` in Python) as true by default. A hand-written loop has to add the outlets explicitly. Testing one demand reaction at a time, with no outlet for the by-products, reports zero leaks in this model instead of 33, because a leaking reaction is then blocked by its own stoichiometry. +The leak-test functions assume that every metabolite can be excreted during the test. `canExchange` and `gap_report` add an output reaction for each metabolite that they check, and `findLeakMetabolite` and `find_leak_metabolite` take `allowExcretion` (`allow_excretion` in Python), which is true by default. A hand-written loop adds the outlets explicitly. Testing one demand reaction at a time without outlets for the by-products reports zero leaks in this model where 33 are present, because the stoichiometry of a leaking reaction blocks it. ::: ## Comparison with a trusted model -When a check reports something, comparing against the last version that passed shows whether the finding is new. +When a check reports a finding, a comparison with the last version that passed shows whether the finding is new. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -308,18 +312,18 @@ Models differ (11 differences): ::: :::: -`diffModels` compares two models entry by entry: which reactions are unique to each, and where a shared reaction differs in stoichiometry, bounds, objective coefficient, gene rule or metabolite properties. [Comparing models](comparing.md) uses the same function, here applied to a model and its last good version. +`diffModels` compares two models entry by entry. It reports the reactions that are unique to each model and the shared reactions that differ in stoichiometry, bounds, objective coefficient, gene rule or metabolite properties. [Comparing models](comparing.md) uses the same function, and here it compares a model with its last good version. :::{admonition} Troubleshooting :class: warning -- Judging a model by whether it solves. An unbalanced reaction makes it solve better. Balance first, then believe the growth rate. -- Treating every unbalanced reaction as a bug. Exchange, demand, sink and biomass pseudo-reactions are unbalanced on purpose. -- `unknown` instead of `unbalanced`. A metabolite with no formula, or a polymer formula cobrapy cannot parse, makes the balance undecidable, not wrong. Fill in the formula and check again. +- A model is judged by whether it solves. An unbalanced reaction can improve the solve. Check the balance first, and trust the growth rate afterwards. +- Every unbalanced reaction is treated as an error. Exchange, demand, sink and biomass pseudoreactions are unbalanced by design. +- The result is `unknown`. A metabolite without a formula, or a polymer formula that cobrapy cannot parse, makes the balance undecidable. Add the formula and repeat the check. ::: ## See also -- [Editing an existing model](editing.md), fixing what these checks find. -- [Model structure and identifiers](model-structure.md), what the structural checks are checking. -- [Growth media and conditions](media.md), closing the medium, as in [Making something from nothing](#making-something-from-nothing). +- [Editing an existing model](editing.md): fixing what the checks find. +- [Model structure and identifiers](model-structure.md): what the structural checks cover. +- [Growth media and conditions](media.md): closing the medium, as in [Mass from nothing](#mass-from-nothing). diff --git a/docs/guide/sampling.md b/docs/guide/sampling.md index 95011b7..ea5c2f0 100644 --- a/docs/guide/sampling.md +++ b/docs/guide/sampling.md @@ -1,6 +1,6 @@ # Random sampling -Flux variability gives the extremes a reaction can reach. Sampling draws many flux vectors from the feasible space and gives the **distribution** in between: how the vectors are spread. A range of 0–10 does not say whether the flux is usually 0, usually 10 or evenly spread; a sample does. +Flux variability gives the extremes that a reaction can reach. Sampling draws many flux vectors from the feasible space and gives the distribution between the extremes, which is how the vectors are spread. A range of 0 to 10 does not show whether the flux is mostly 0, mostly 10 or evenly spread, and a sample does. ## Functions @@ -10,11 +10,11 @@ Flux variability gives the extremes a reaction can reach. Sampling draws many fl | `sampleACHR` | `random_sampling` (`method='achr'`) | hit-and-run Markov chain Monte Carlo (MCMC), the default | | `sampleCHRR` | `random_sampling` (`method='chrr'`) | hit-and-run with rounding, for thin polytopes | | no equivalent | `find_good_reactions` | reactions usable as random objectives | -| `getAllowedBounds` | `flux_variability_analysis` {bdg-link-secondary}`cobrapy ` | the ranges sampling fills in | +| `getAllowedBounds` | `flux_variability_analysis` {bdg-link-secondary}`cobrapy ` | the ranges that sampling fills in | ## Setup -The model is the glucose-limited `smallYeast.yml` of [Flux variability](fva.md), so the two pages can be read against each other. +The model is the glucose-limited `smallYeast.yml` of [Flux variability](fva.md), so the two pages can be compared. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -51,9 +51,9 @@ model.objective = "biomassOUT" ## First sample -Both toolboxes take a `seed`. A seed should always be set: without it a chain differs in every run and no reported result can be reproduced. +Both toolboxes take a `seed`. A seed is set in every run, because a chain without a seed differs in every run and a result cannot be reproduced. -A seed does not guarantee identical numbers. The default `achr` sampler starts from warmup points, and those come from LPs: two per reaction. A different solver, solver version or platform, and with GLPK even a solve of the same model before the sampling (which leaves a different starting basis), returns different warmup vertices, so the same seed explores the same distribution along a different chain. With seed 1, MATLAB gives `FRDS2`'s sampled minimum as 18.8 with GLPK and as 4.5 with Gurobi. The sampled values in the examples are printed to few decimals, because the distribution is the result and the individual draws are not. +A seed does not guarantee identical numbers. The default `achr` sampler starts from warmup points that come from LPs, two per reaction. A different solver, solver version or platform returns different warmup vertices, and so does a solve of the same model before the sampling when GLPK is used, because it leaves a different starting basis. The same seed then explores the same distribution along a different chain. With seed 1, MATLAB gives a sampled minimum of `FRDS2` of 18.8 with GLPK and of 4.5 with Gurobi. The sampled values in the examples are printed with few decimals, because the distribution is the result and the single draws are not. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -89,12 +89,12 @@ method: achr :::{admonition} Transposed layouts :class: warning -RAVEN returns reactions × samples; raven-toolbox returns a DataFrame of samples × reactions, the `cobra.sampling` layout. Every mean, histogram and correlation has to use the right axis, and a wrong axis gives no error, only numbers other than the intended ones. +RAVEN returns reactions by samples. raven-toolbox returns a DataFrame of samples by reactions, which is the layout of `cobra.sampling`. Every mean, histogram and correlation must use the correct axis, because a wrong axis raises no error and gives other numbers than intended. ::: ## Distribution and range -[Flux variability](fva.md) found that `FRDS2` and `SDH` have the widest range in this model, 1000 units, and that all of it is a thermodynamically infeasible cycle. Sampling does not remove the cycle; it shows the share of the space that the cycle occupies. +[Flux variability](fva.md) showed that `FRDS2` and `SDH` have the widest range in this model, 1000 units, and that all of it comes from a thermodynamically infeasible cycle. Sampling does not remove the cycle and shows the share of the space that it occupies. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -131,15 +131,15 @@ growth sampled 0.01 to 0.09, mean 0.05 ::: :::: -The two tabs are separate implementations with separate random number generators, so their numbers are not expected to match draw for draw. They describe the same distribution, and the agreement of the distributions is the check to make. +The two implementations are separate and use separate random number generators, so their numbers do not match draw by draw. They describe the same distribution, and the agreement of the distributions is the check. -`FRDS2` is sampled across nearly its whole 1000-unit range, so the loop is most of the space, and most of the draws fall in it. Growth is the opposite. Its range runs from 0 to 0.1222, while the samples fall between about 0.01 and 0.1 and average about 0.05, because near-uniform sampling almost never reaches a vertex and the optimum is a corner with no volume around it. +`FRDS2` is sampled across nearly its whole range of 1000 units, so the loop forms most of the space and most draws fall in it. Growth shows the opposite. Its range runs from 0 to 0.1222, and the samples fall between about 0.01 and 0.1 with a mean of about 0.05. Near-uniform sampling almost never reaches a vertex, and the optimum is a corner without volume around it. -A sampled mean describes the shape of the feasible space and is no prediction: the mean sampled growth of this model is less than half its optimum, and biology does not cause the difference. +A sampled mean describes the shape of the feasible space and does not predict anything. The mean sampled growth of this model is less than half of its optimum, and biology does not cause the difference. ## Choosing a method -`achr` and `chrr` both draw the near-uniform interior distribution. `randomObjective` (`random_objective` in Python) maximises a small random objective each time, so it returns vertices; the samples of `achr` and `chrr` are interior points. Vertices are what FBA gives, which makes that method a way to survey alternative optima, not to describe the space. +`achr` and `chrr` both draw the near-uniform interior distribution. `randomObjective` (`random_objective` in Python) maximises a small random objective each time, so it returns vertices, while the samples of `achr` and `chrr` are interior points. FBA also gives vertices, so this method surveys alternative optima and does not describe the space. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -173,13 +173,13 @@ chrr: 9 dimensions, MVE converged: False, 1 fixed ::: :::: -The dimension is a property of the polytope, not of the sampler: it is how many degrees of freedom the network has once the implicitly determined reactions are eliminated. A model with 53 reactions has far fewer than 53, and the two toolboxes, which implement CHRR separately, both compute 9. +The dimension is a property of the polytope. It is the number of degrees of freedom of the network after the implicitly determined reactions are eliminated. A model with 53 reactions has far fewer than 53, and the two toolboxes, which implement CHRR separately, both compute 9. -`MVE converged: False` is a warning, not a failure. MVE is the maximum-volume ellipsoid used for rounding. The rounding step stopped before reaching its tolerance, so the last ellipsoid is a valid rounding and the samples are usable, but on a polytope this elongated mixing is slower than the defaults assume, and more thinning helps if the distribution is uneven. +`MVE converged: False` is a warning. MVE is the maximum-volume ellipsoid that is used for the rounding. The rounding stopped before it reached its tolerance. The last ellipsoid is a valid rounding and the samples are usable. On a polytope this elongated, mixing is slower than the defaults assume, and more thinning helps when the distribution is uneven. ## Sampling a constrained state -Sampling an unconstrained model answers "what could this network do?", which is rarely the question of interest. Constrain first (hold growth near its optimum, fix a measured flux) and sample the space that is left. +Sampling an unconstrained model answers what the network could do, which is rarely the question. The model should be constrained first, by holding growth near its optimum or by fixing a measured flux, and the remaining space is sampled. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -217,7 +217,7 @@ ethanol: free 0.73, growth >= 90% 0.03 ## Reactions to sample over -The `randomObjective` method needs reactions that can carry flux outside loops to use as objectives. Reactions that carry flux only through a loop are of no use as objectives, and the two toolboxes screen them out differently. raven-toolbox runs a loopless FVA, the test that [Flux variability](fva.md) applied manually. RAVEN runs a plain FVA (`getAllowedBounds`) and drops the reactions that reach 0.999 of the model's largest bound, which is where loop flux ends up. Here both keep the same 50. +The `randomObjective` method needs reactions that carry flux outside loops, as objectives. Reactions that carry flux through a loop alone are unsuitable, and the two toolboxes screen them out differently. raven-toolbox runs a loopless FVA, the test that [Flux variability](fva.md) applied by hand. RAVEN runs a plain FVA (`getAllowedBounds`) and drops the reactions that reach 0.999 times the upper bound of the loop (1000 in this model), which is where loop flux ends up. Both keep the same 50 reactions here. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -246,23 +246,23 @@ print(f"{len(good)} of {len(model.reactions)} reactions usable as objectives") 50 of 53 reactions usable as objectives ``` -Passing the list back as `good_reactions=` on a later call skips the FVA, the expensive part of the run on a genome-scale model. +The list passed back as `good_reactions=` on a later call skips the FVA, which is the expensive part of the run on a genome-scale model. ::: :::: :::{admonition} Troubleshooting :class: warning -- No seed. The numbers change in every run. Set one, and report it with the solver: the warmup points come from LPs, so another solver (or, with GLPK, a solve of the same model before the sampling) gives another chain (see [First sample](#first-sample)). Report the distribution, not the draws. -- Too little thinning. Consecutive MCMC steps are correlated, so only every 100th is recorded. That is enough for this model, not for a genome-scale one: on yeast-GEM, samples 100 steps apart are 0.93 correlated and independent chains disagree on most reactions (see [Flux sampling convergence and settings](../parameter-tuning/studies/sampling-convergence-calibration.md)). For results that are reported, run chains with different seeds and check that they agree. -- Reading a mean as a prediction. The mean of a sample describes the feasible space and the sampler, not the cell. It is a feasible state, but it is located where most of the volume is: in [Distribution and range](#distribution-and-range) that puts mean growth below half the optimum. With `randomObjective` the samples are vertices, so a reaction can average 5 while every sample has it at 0 or 10. -- Sampling a model that is wide open. With an unconstrained medium the space is enormous and the distribution says nothing. Constrain first. -- Loops. They inflate the space being sampled, and every sample drawn inside a cycle is wasted. See [Flux variability](fva.md). -- Genome-scale runtime. `achr` starts with two LPs per reaction for its warmup points and `chrr` with an FVA and a rounding step; the sampling steps after that solve no LP. `randomObjective` solves one LP per sample. Start with a few hundred samples to see the shape, not tens of thousands. +- No seed is set. The numbers change in every run. Set a seed and report it together with the solver, because the warmup points come from LPs and another solver gives another chain, as does a solve of the same model before the sampling with GLPK ([First sample](#first-sample)). The distribution is reported. +- Thinning is too small. Consecutive MCMC steps are correlated, so every 100th step is recorded. This suffices for this model and falls short for a genome-scale model. On yeast-GEM, samples 100 steps apart have a correlation of 0.93, and independent chains disagree on most reactions ([Flux sampling convergence and settings](../parameter-tuning/studies/sampling-convergence-calibration.md)). Reported results need chains with different seeds that agree. +- A mean is read as a prediction. The mean of a sample describes the feasible space and the sampler. It is a feasible state located where most of the volume is, which puts the mean growth below half of the optimum in [Distribution and range](#distribution-and-range). With `randomObjective` the samples are vertices, and a reaction can average 5 while every sample has it at 0 or 10. +- A model with a wide-open medium is sampled. With an unconstrained medium the space is enormous and the distribution says nothing, so constrain the model first. +- Loops are present. They inflate the sampled space, and every sample drawn inside a cycle is wasted ([Flux variability](fva.md)). +- The runtime on a genome-scale model is long. `achr` starts with two LPs per reaction for its warmup points, and `chrr` starts with an FVA and a rounding step. The sampling steps after that solve no LP. `randomObjective` solves one LP per sample. A few hundred samples show the shape, and tens of thousands are not needed for that. ::: ## See also -- [Flux variability](fva.md), the ranges that sampling fills in. -- [Simulating growth with FBA](fba.md), the single point that sampling surrounds. -- [Deletions and essentiality](deletions.md), the other way to ask what the model depends on. +- [Flux variability](fva.md): the ranges that sampling fills in. +- [Simulating growth with FBA](fba.md): the single point that sampling surrounds. +- [Deletions and essentiality](deletions.md): the other way to ask what the model depends on. diff --git a/docs/guide/solvers.md b/docs/guide/solvers.md index 8008530..064f60b 100644 --- a/docs/guide/solvers.md +++ b/docs/guide/solvers.md @@ -1,21 +1,21 @@ # Solvers and configuration -Most simulations on this site solve a linear program (LP); gap-filling, minimum-component media, `solveLP` with `minFlux` 3 and ftINIT solve mixed-integer linear programs (MILPs), and the quadratic MOMA in [Deletions and essentiality](deletions.md) solves a quadratic program (QP). Each of them needs a solver, and which solver can do which depends on the solver and the toolbox. +Most simulations on this site solve a linear program (LP). Gap-filling, minimal media with a minimum number of components, `solveLP` with `minFlux` set to 3, and ftINIT solve mixed-integer linear programs (MILPs). The quadratic MOMA in [Deletions and essentiality](deletions.md) solves a quadratic program (QP). The solvers that can handle each problem type depend on the solver and on the toolbox. ## Functions | MATLAB | Python | Purpose | |---|---|---| -| `setRavenSolver` | `Configuration` {bdg-link-secondary}`cobrapy ` | choose the solver | -| `checkRaven` | `Configuration` {bdg-link-secondary}`cobrapy ` | check the solver works | -| `solveLP` | `Model.optimize` {bdg-link-secondary}`cobrapy ` | solve, and get a solution object | -| `optimizeProb` | `Model.solver` {bdg-link-secondary}`cobrapy ` | solve a problem the toolbox built | +| `setRavenSolver` | `Configuration` {bdg-link-secondary}`cobrapy ` | select the solver | +| `checkRaven` | no equivalent | test the solver | +| `solveLP` | `Model.optimize` {bdg-link-secondary}`cobrapy ` | solve and return a solution object | +| `optimizeProb` | `Model.solver` {bdg-link-secondary}`cobrapy ` | solve a problem that the toolbox built | ## Setup -`smallYeast.yml` from [`docs/data/`](../data/README.md), with glucose and oxygen opened so there is something to solve. +The model is `smallYeast.yml` from [`docs/data/`](../data/README.md), with glucose and oxygen opened so that there is a problem to solve. -In this model the uptake reactions are written as `=> metabolite` and supplied closed at `[0 0]`, so uptake is a positive flux and opening one means raising its upper bound. Lowering the lower bound instead gives a model that solves, reports success, and grows at exactly zero. +In this model, the uptake reactions are written as `=> metabolite` and are closed at `[0 0]`. Uptake is a positive flux, and an uptake reaction is opened by raising its upper bound. Lowering the lower bound instead gives a model that solves, reports success and grows at zero. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -39,7 +39,7 @@ model.reactions.get_by_id("o2IN").upper_bound = 1000.0 ::: :::: -The sign convention here is the opposite of yeast-GEM's, where uptake is a negative flux through an exchange reaction. Neither is more correct; a model carries whichever its author chose, and the direction the equation is written in decides which bound opens it. [Growth media and conditions](media.md) works through the other convention. +The sign convention is the opposite of the one in yeast-GEM, where uptake is a negative flux through an exchange reaction. A model carries the convention of its author, and the direction in which the equation is written decides which bound opens the reaction. [Growth media and conditions](media.md) works with the other convention. ## Solver in use @@ -55,7 +55,7 @@ fprintf('RAVEN solver preference: %s\n', getpref('RAVEN', 'solver')); RAVEN solver preference: ... ``` -RAVEN keeps the choice in MATLAB's preferences, so it survives restarts, and the answer is whatever this installation was last set to, so the output is elided. The preference is global to the installation, not a property of the model, so a script that changes it changes it for everything that follows. `checkRaven` prints it along with a test solve. +RAVEN stores the choice in the MATLAB preferences, so it persists across sessions. The answer is the last setting of the installation, and the output is therefore elided. The preference belongs to the installation, so a script that changes it changes it for everything that runs afterwards. `checkRaven` prints the solver together with a test solve. ::: :::{tab-item} 🐍 Python :sync: python @@ -72,13 +72,13 @@ default: optlang.glpk_interface this model: optlang.glpk_interface ``` -cobrapy has two levels: `Configuration()` is the default applied to models created from then on, and `model.solver` is the interface this model is using. Because the choice is stored on the model, two models in one session can use different solvers. +cobrapy has two levels. `Configuration()` holds the default for models created afterwards, and `model.solver` is the interface that a model uses. The choice is stored on the model, so two models in one session can use different solvers. ::: :::: ## Changing the solver -Both toolboxes ship with GLPK, which solves LPs and MILPs, so it runs every example on this site except the quadratic MOMA. RAVEN prints a warning when it sends a MILP to GLPK, recommending Gurobi: GLPK handles problems the size of these examples, but on a genome-scale model gap-filling and ftINIT can take far longer than with Gurobi, which is free for academic use. A QP needs a QP-capable solver, such as Gurobi or CPLEX; RAVEN has no functions that solve one. +Both toolboxes ship with GLPK, which solves LPs and MILPs and therefore runs every example on this site except the quadratic MOMA. RAVEN warns when it sends a MILP to GLPK and recommends Gurobi. GLPK handles problems of the size of these examples, but gap-filling and ftINIT on a genome-scale model take much longer than with Gurobi, which is free for academic use. A QP needs a solver that supports QPs, such as Gurobi or CPLEX. RAVEN has no function that solves a QP. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -88,7 +88,7 @@ Both toolboxes ship with GLPK, which solves LPs and MILPs, so it runs every exam setRavenSolver('glpk'); % 'gurobi', 'scip', or 'cobra' to hand over to the COBRA Toolbox ``` -The accepted values are `'glpk'`, `'gurobi'`, `'scip'` and `'cobra'`. `'glpk'` uses the binaries RAVEN ships, so it works with no further installation. `'gurobi'` needs Gurobi on the MATLAB path, and `'scip'` the SCIP MATLAB interface. `'cobra'` translates the problem into COBRA Toolbox form and uses whatever `changeCobraSolver` last selected, which requires the COBRA Toolbox to be initialised first; `setRavenSolver` checks that and errors without storing a preference that cannot work. +The accepted values are `'glpk'`, `'gurobi'`, `'scip'` and `'cobra'`. `'glpk'` uses the binaries that ship with RAVEN and needs no further installation. `'gurobi'` needs Gurobi on the MATLAB path, and `'scip'` needs the SCIP MATLAB interface. `'cobra'` translates the problem to the format of the COBRA Toolbox and uses the solver last selected with `changeCobraSolver`. It requires an initialised COBRA Toolbox, which `setRavenSolver` checks, and it raises an error without storing a preference that cannot work. ::: :::{tab-item} 🐍 Python :sync: python @@ -100,7 +100,7 @@ from cobra import Configuration Configuration().solver = "glpk" # the default for models loaded later ``` -Assigning to `model.solver` rebuilds this model's problem; assigning to `Configuration().solver` changes the default for models created afterwards and leaves existing ones alone. The available names are whichever optlang interfaces are installed. +Assigning to `model.solver` rebuilds the problem of this model. Assigning to `Configuration().solver` changes the default for models created afterwards and leaves existing models unchanged. The available names are the optlang interfaces that are installed. ::: :::: @@ -110,15 +110,15 @@ The two solution objects carry the same information under different names. | MATLAB `solveLP` | cobrapy `Solution` | Meaning | |---|---|---| -| `sol.f` | `solution.objective_value` | objective value, same sign in both | -| `sol.x` | `solution.fluxes` | fluxes: a vector in `model.rxns` order, or a Series by id | +| `sol.f` | `solution.objective_value` | objective value, with the same sign in both | +| `sol.x` | `solution.fluxes` | fluxes: a vector in the order of `model.rxns`, or a Series by identifier | | `sol.stat` | `solution.status` | how the solve ended | -| `sol.msg` | `solution.status` | what the solver said | -| `sol.sPrice`, `sol.rCost` | `solution.shadow_prices`, `solution.reduced_costs` | duals | +| `sol.msg` | `solution.status` | message of the solver | +| `sol.sPrice`, `sol.rCost` | `solution.shadow_prices`, `solution.reduced_costs` | dual values | -`sol.stat` takes four values: `1` solved to optimality, `0` feasible but not proven optimal, `-1` infeasible, and `-2` solved but with the flux minimisation (`minFlux`) failing afterwards. Only `1` means the fluxes are the answer to the question that was asked. +`sol.stat` takes four values: `1` for solved to optimality, `0` for feasible without a proof of optimality, `-1` for infeasible, and `-2` for solved with a failure of the flux minimisation (`minFlux`) afterwards. Only `1` means that the fluxes answer the question. -The duals are the exception to the correspondence: RAVEN reports `sPrice` and `rCost` only when `solveLP` runs without flux minimisation, because the second optimisation replaces the problem whose duals they describe. +The dual values do not correspond in all cases. RAVEN reports `sPrice` and `rCost` when `solveLP` runs without flux minimisation, because the second optimisation replaces the problem that the duals describe. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -136,7 +136,7 @@ stat 1: Optimal solution found growth 0.1222 /h ``` -`solveLP` builds a COBRA-style problem struct and hands it to `optimizeProb`, which is where the selected solver is called. Calling `optimizeProb` directly is the way to solve a problem the toolbox built, such as the MILPs inside gap-filling. It takes two name-value arguments: `'params'`, a struct forwarded to the solver, and `'verbose'`, which shows MILP progress. One field of `params` is handled by RAVEN and not passed on: `maxRatio`, which, for LPs only, splits badly-scaled columns through auxiliary metabolites before solving and leaves the feasible region unchanged. +`solveLP` builds a problem struct in the COBRA format and passes it to `optimizeProb`, which calls the selected solver. `optimizeProb` is called directly to solve a problem that the toolbox built, such as the MILPs inside gap-filling. It takes two name-value arguments: `'params'`, a struct that is passed to the solver, and `'verbose'`, which shows the progress of a MILP. RAVEN handles one field of `params` itself and does not pass it on. `maxRatio` applies to LPs, splits badly scaled columns through auxiliary metabolites before solving and leaves the feasible region unchanged. ::: :::{tab-item} 🐍 Python :sync: python @@ -153,13 +153,13 @@ status optimal growth 0.1222 /h ``` -`model.solver` is the optlang problem itself, so the underlying solver is reachable for anything cobrapy does not expose: `model.solver.problem` is the native object, and `model.solver.configuration` carries tolerances, time limits and thread counts. +`model.solver` is the optlang problem itself, so the underlying solver is accessible for anything that cobrapy does not expose. `model.solver.problem` is the native object, and `model.solver.configuration` holds tolerances, time limits and the number of threads. ::: :::: ## Failed solves -An infeasible problem is the most common outcome of a bad edit, and the two toolboxes report it differently: RAVEN sets `sol.stat` to `-1` and returns an empty flux vector, while cobrapy raises nothing; `optimize` returns a solution with status `infeasible`, and `slim_optimize` returns `nan`. +An infeasible problem is the usual result of a bad edit, and the toolboxes report it differently. RAVEN sets `sol.stat` to `-1` and returns an empty flux vector. cobrapy raises no error: `optimize` returns a solution with the status `infeasible`, and `slim_optimize` returns `nan`. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -176,7 +176,7 @@ fprintf('stat %d: %s\n', sol.stat, sol.msg); stat -1: The problem is infeasible ``` -`sol.f` and `sol.x` are empty here, not zero, so arithmetic on them produces an empty result and no error. An unchecked infeasible solve therefore turns into a blank in a report. +`sol.f` and `sol.x` are empty, so arithmetic on them returns an empty result without an error. An infeasible solve that is not checked appears as a blank in a report. ::: :::{tab-item} 🐍 Python :sync: python @@ -194,23 +194,23 @@ status: infeasible slim_optimize: nan ``` -Check the status before using a number. `slim_optimize` skips building a full solution, so it is the fast option inside a loop, but it returns `nan` and gives no reason. +The status should be checked before a number is used. `slim_optimize` does not build a full solution, which makes it fast inside a loop, and it returns `nan` without a reason. ::: :::: -An infeasible model is over-constrained: the constraints as written have no solution. That is different from a feasible model whose optimum is zero, which is a statement about biology, not about the constraints. +An infeasible model is over-constrained, and no solution satisfies the constraints as written. A feasible model with an optimum of zero is a different case, and it describes the biology. :::{admonition} Troubleshooting :class: warning -- A solver that is set but not installed. `setRavenSolver('gurobi')` stores the preference whether or not Gurobi is there; the failure appears at the next solve. `checkRaven` tests it directly. -- A large MILP on GLPK. Both toolboxes send MILPs to GLPK, and RAVEN warns when it does. On a genome-scale model, gap-filling and ftINIT can run for hours on GLPK where Gurobi takes minutes; use Gurobi for those. -- Reading fluxes after a non-optimal solve. `sol.stat` of `0` means a feasible point was found but not proven optimal, and `-2` means the parsimonious step failed after an optimal first solve. Both return numbers. -- Tiny differences between solvers. Alternative optima mean two solvers can return different flux distributions for the same objective value. Pin one solver for anything that is to be compared. +- A solver is set and missing. `setRavenSolver('gurobi')` stores the preference whether or not Gurobi is present, and the failure appears at the next solve. `checkRaven` tests the solver directly. +- A large MILP runs on GLPK. Both toolboxes send MILPs to GLPK, and RAVEN warns when it does. Gap-filling and ftINIT on a genome-scale model run for hours on GLPK where Gurobi takes minutes. +- Fluxes are read after a solve that is not optimal. `sol.stat` of `0` means that a feasible point was found without a proof of optimality, and `-2` means that the parsimonious step failed after an optimal first solve. Both return numbers. +- Solvers give slightly different results. Alternative optima allow two solvers to return different flux distributions with the same objective value. Fix one solver for any comparison. ::: ## See also -- [Simulating growth with FBA](fba.md), the solve itself. -- [Growth media and conditions](media.md), the constraints that decide whether a problem is feasible. -- [Installation](../installation/index.md), installing and testing a solver. +- [Simulating growth with FBA](fba.md): the solve. +- [Growth media and conditions](media.md): the constraints that decide whether a problem is feasible. +- [Installation](../installation/index.md): installation and test of a solver. diff --git a/docs/guide/tasks.md b/docs/guide/tasks.md index 96113c8..50a9c4c 100644 --- a/docs/guide/tasks.md +++ b/docs/guide/tasks.md @@ -1,6 +1,6 @@ # Metabolic tasks -A metabolic task is a statement of something the model must be able to do: "given glucose and oxygen, produce biomass", "make ATP", "do not produce something from nothing". Tasks test a reconstruction against biology and not against the reconstruction itself, and ftINIT repairs against them after cutting a model down. +A metabolic task states something the model must be able to do: "given glucose and oxygen, produce biomass", "make ATP", or "do not produce something from nothing". Tasks test a reconstruction against biology. ftINIT repairs against them after it reduces a model. ## Functions @@ -8,12 +8,12 @@ A metabolic task is a statement of something the model must be able to do: "give |---|---|---| | `parseTaskList` | `parse_task_list` | read a task-list file | | `checkTasks` | `check_tasks` | run the tasks against a model | -| `checkTasks` (`getEssential`) | `find_task_essential_reactions` | reactions a task requires | +| `checkTasks` (`getEssential`) | `find_task_essential_reactions` | reactions that a task requires | | `fitTasks` | `fill_tasks` | add reactions until the tasks pass | ## The file format -A task list is a tab-separated table, one row per task, with columns for the inputs, the outputs, any equations that must carry flux, and the bounds on each. [`tasks.txt`](../data/tasks.txt) holds two tasks for `smallYeast.yml`: +A task list is a tab-separated table with one row per task. The columns hold the inputs, the outputs, the equations that must carry flux, and the bounds of each. [`tasks.txt`](../data/tasks.txt) holds two tasks for `smallYeast.yml`: ```text title="tasks.txt, abridged; the real file keeps every column" ID DESCRIPTION SHOULD FAIL IN IN LB IN UB OUT OUT LB OUT UB … @@ -23,18 +23,18 @@ A task list is a tab-separated table, one row per task, with columns for the inp LEAK Biomass from nothing 1 biomass[c] 0.01 1000 … ``` -Two things about the format have to be right the first time, because both fail in MATLAB with the same error `Index in position 2 is invalid`: +Two requirements of the format apply to MATLAB: -- Every row starts with an empty cell: `parseTaskList` discards any row whose first cell is non-empty (that is how it skips comments and section headers), so in a file without the leading tab every row is discarded. -- Keep the full column set, even where the columns are empty: `parseTaskList` matches all seventeen known headers and then indexes the numeric ones positionally, so omitting `EQU LB` or `CHANGED LB` leaves an index of zero. +- The header row starts with an empty cell. `parseTaskList` treats the leading rows whose first cell is not empty as comments and takes the first row with an empty first cell as the header, so a file without the leading tab has no header row. +- The header row has a column named `ID`, which is the sole required column. A file without a row that starts with an empty cell raises `RAVEN:badInput` with the message that the TASKS sheet must have a column named ID. -raven-toolbox's `parse_task_list` accepts the file with or without either, so both are easy to miss until the MATLAB tab runs. The file `tasks.txt` has the full set; the columns after `OUT UB` are blank. +`parse_task_list` of raven-toolbox accepts the file with or without the leading empty cell, so a missing cell is noticed when the MATLAB example runs. `tasks.txt` has the full set of columns, and the columns after `OUT UB` are blank. -Metabolites are named `name[compartment]`, matching the metabolite names and not the identifiers. Several go in one cell, separated by `;`, and the bounds columns apply to all of them. To give a metabolite bounds of its own, put it on a continuation row: a row with no `ID` adds its inputs and outputs to the task above it. `GROWTH` uses that to ask for at least 0.01 units of biomass while leaving CO₂ and acetate free; with a lower bound of 0 on biomass, the all-zero flux would pass the task. The 0.01 is illustrative; any positive floor has the same effect. +Metabolites are given as `name[compartment]` with the metabolite name. Several metabolites go in one cell, separated by `;`, and the bound columns apply to all of them. A metabolite with its own bounds goes on a continuation row, which is a row without an `ID` that adds its inputs and outputs to the task above it. `GROWTH` uses this to require at least 0.01 units of biomass while CO₂ and acetate are unrestricted. With a lower bound of 0 on biomass, the zero flux would pass the task. The value 0.01 is illustrative, and any positive lower bound has the same effect. -`SHOULD FAIL` marks a task the model must fail: the second asks for biomass with nothing supplied, which a correct model cannot do, so failing it is a pass. +`SHOULD FAIL` marks a task that the model must fail. The second task asks for biomass with nothing supplied, which a correct model cannot do, so failing it is a pass. -Once the model's own exchanges are closed (see [Task-defined medium](#task-defined-medium)), `GROWTH` has to list glucose and oxygen as inputs, and biomass, CO₂ and acetate as outputs, because growth has to release its carbon: as CO₂ and, in this small model, some acetate. A task without CO₂ is infeasible, which looks like a broken model and is a broken task. +When the exchange reactions of the model are closed ([Task-defined medium](#task-defined-medium)), `GROWTH` lists glucose and oxygen as inputs and biomass, CO₂ and acetate as outputs. Growth must release its carbon, as CO₂ and, in this small model, as acetate. A task without CO₂ is infeasible, which looks like a broken model and is a broken task. ## Reading and running tasks @@ -61,11 +61,11 @@ end LEAK pass ``` -`report.ok` is true for both, and for opposite reasons: `GROWTH` was required to pass and did, `LEAK` was required to fail and did. +`report.ok` is true for both tasks, for opposite reasons: `GROWTH` had to pass and did, and `LEAK` had to fail and did. -`checkTasks` prints its own verdicts unless `printOutput` is off, as here, and `printOnlyFailed` narrows that to the failures. Reading the verdicts from `report` gives a structure to test against, not text to read. +`checkTasks` prints its verdicts unless `printOutput` is off, as here, and `printOnlyFailed` limits the print-out to the failures. The verdicts in `report` form a structure that a test can use. -Passing the parsed tasks as `taskStructure` makes the second argument, the task file, redundant. `runParallel` evaluates the tasks in parallel workers, and defaults to `false`, because starting a pool takes longer than the time it saves on a short task list; on a genome-scale list it is the setting that matters. +The parsed tasks passed as `taskStructure` make the second argument, the task file, unnecessary. `runParallel` evaluates the tasks in parallel workers. It defaults to `false`, because starting a pool takes longer than a short task list saves, and it matters for a genome-scale list. ::: :::{tab-item} 🐍 Python :sync: python @@ -89,13 +89,13 @@ for result in check_tasks(model, tasks, close_boundaries=True): # see Task-def LEAK pass (feasible: False) ``` -`close_boundaries=True` closes the model's own exchange, sink and demand reactions first, so the inputs and outputs are what the task says they are. A task list is therefore portable between models, which a script full of `setParam` calls is not. +`close_boundaries=True` first closes the exchange, sink and demand reactions of the model, so the inputs and outputs are those that the task defines. A task list is therefore portable between models, which a script of `setParam` calls is not. ::: :::: ## Task-defined medium -By default neither `checkTasks` nor `check_tasks` closes the model's own exchange reactions: a task's inputs and outputs are added to whatever the model allows. To make the task the whole boundary, close the model first: `closeModel` in RAVEN (what ftINIT does before collecting task-essential reactions), `close_boundaries=True` in raven-toolbox. The task's inputs and outputs are then the only way anything enters or leaves, which makes a task list portable between models and makes an incomplete task look like a broken model. +By default, neither `checkTasks` nor `check_tasks` closes the exchange reactions of the model, and the inputs and outputs of a task are added to what the model allows. The task becomes the whole boundary when the model is closed first, with `closeModel` in RAVEN (which ftINIT does before it collects the task-essential reactions) or with `close_boundaries=True` in raven-toolbox. The inputs and outputs of the task are then the sole way for anything to enter or leave. A task list is then portable between models, and an incomplete task looks like a broken model. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -157,11 +157,11 @@ for close in (False, True): ::: :::: -The only difference between the two tasks is the CO₂ output. With the model's own exchanges open, both are feasible: the carbon leaves through the model's `co2OUT`, whatever the task says. With them closed, the task without CO₂ is infeasible. A task list written for one reading gives different answers under the other, so check which one it was curated against. +The tasks differ in the CO₂ output. With the exchange reactions of the model open, both are feasible, because the carbon leaves through `co2OUT` of the model whatever the task says. With them closed, the task without CO₂ is infeasible. A task list written for one reading gives different answers under the other, so the boundary that the list was curated against is checked. ## Task-essential reactions -Essentiality here is defined with respect to a task, not to the objective: remove each candidate reaction and see whether the task passes. ftINIT uses it to decide which reactions must survive an extraction regardless of their expression score. +Essentiality here refers to a task. Each candidate reaction is removed, and the task is tested again. ftINIT uses the result to decide which reactions must remain in an extraction regardless of their expression score. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -177,7 +177,7 @@ fprintf('%d reactions essential for the task\n', sum(essentialRxns)); 28 reactions essential for the task ``` -Task-essential reactions come from `checkTasks` with its `getEssential` flag, returned as a reactions-by-tasks logical matrix and not as a list. Failed tasks and `SHOULD FAIL` tasks are left out of it, since a task that does not pass has no reactions it depends on. `getEssentialRxns` answers for the model's own objective and takes no task. +`checkTasks` with its `getEssential` flag returns the task-essential reactions as a logical matrix of reactions by tasks. Failed tasks and `SHOULD FAIL` tasks are left out, because a task that does not pass has no reactions that it depends on. `getEssentialRxns` answers for the objective of the model and takes no task. ::: :::{tab-item} 🐍 Python :sync: python @@ -199,7 +199,7 @@ print(sorted(result.reactions)[:5]) ## Failing tasks -A failing task is a gap: something the model should be able to do and cannot. Both toolboxes can add reactions from a template until the task passes, which is [Gap-filling](gap-filling.md) applied to tasks. +A failing task is a gap, something that the model should be able to do and cannot. Both toolboxes add reactions from a template until the task passes, which is [Gap-filling](gap-filling.md) applied to tasks. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -219,20 +219,20 @@ template = readYAMLmodel('smallYeast.yml'); % where the reactions come from from raven_toolbox.init import fill_tasks ``` -`fill_tasks` is the same operation inside the ftINIT pipeline: given the tasks and a source of reactions, it adds back the minimum needed to make every task feasible again. +`fill_tasks` is the same operation inside the ftINIT pipeline. Given the tasks and a source of reactions, it adds the reactions of lowest cost that make each task feasible again. ::: :::: :::{admonition} Troubleshooting :class: warning -- A metabolite given by its identifier: task files address metabolites by name, not by identifier, as `name[compartment]`. A task that matches nothing without an error is a task that passes for the wrong reason. -- Missing `SHOULD FAIL` tasks: the tasks that matter most are often the ones the model must not satisfy; without them, a leaking model passes everything. -- The boundary a task ran against is unknown: with the model closed, a task defines its own inputs and does not inherit the model's medium, so a task passing says nothing about whether the model grows on the medium it has. Left open (the default in both toolboxes), the model's own exchanges take part, and a task can pass because of an uptake or secretion it never lists. +- A metabolite is given by its identifier. Task files address metabolites by name, as `name[compartment]`. A task that matches nothing and raises no error passes for the wrong reason. +- `SHOULD FAIL` tasks are missing. The tasks that matter most are often those that the model must not satisfy, and a leaking model passes every other task. +- The boundary of a task run is unknown. With the model closed, a task defines its own inputs and does not inherit the medium of the model, so a passed task does not show that the model grows on its medium. With the model open, which is the default in both toolboxes, the exchange reactions of the model take part, and a task can pass because of an uptake or secretion that it does not list. ::: ## See also -- [Gap-filling](gap-filling.md), which makes a failing task pass. -- [Context-specific models](init.md), tasks that an extraction must preserve. -- [Quality control](quality-control.md), the checks that do not need a task list. +- [Gap-filling](gap-filling.md): making a failing task pass. +- [Context-specific models](init.md): the tasks that an extraction must preserve. +- [Quality control](quality-control.md): the checks that need no task list. diff --git a/docs/index.md b/docs/index.md index a59b7c5..3ae21f2 100644 --- a/docs/index.md +++ b/docs/index.md @@ -9,7 +9,7 @@

A toolkit for building, curating, and simulating genome-scale metabolic models, available as a MATLAB toolbox and a Python package built on cobrapy.

-## Install +## Installation ::::{tab-set} :::{tab-item} Python (pip) @@ -18,12 +18,6 @@ pip install --pre raven-toolbox ``` ::: -:::{tab-item} MATLAB (Add-Ons) - -Home → Add-Ons → Get Add-Ons → search "RAVEN Toolbox". - -Then run `checkRaven` to verify. -::: :::{tab-item} Python (git) ```bash @@ -37,7 +31,11 @@ pip install -e raven-toolbox/ git clone --depth=1 https://github.com/CompMetEng/RAVEN.git ``` -Add the folder to the MATLAB path, then run `checkRaven` to verify. +Add the folder to the MATLAB path and run `checkRaven`, which reports whether the installation and the solver work. +::: +:::{tab-item} MATLAB (Add-Ons) + +The Add-Ons manager installs the latest stable release, RAVEN 2.x. RAVEN 3.0.0b1, which this site documents, is installed from git or from a source archive; see [RAVEN (MATLAB)](installation/raven.md). ::: :::: @@ -46,33 +44,35 @@ Add the folder to the MATLAB path, then run `checkRaven` to verify. ::::{grid} 1 2 3 3 :::{grid-item-card} Homology reconstruction -Build draft models by transferring reactions from template models using BLAST+, DIAMOND, or HMMER. +Transfers reactions from template models to a new genome, using BLAST+, DIAMOND or HMMER to find the homologous genes. ::: :::{grid-item-card} KEGG-based reconstruction -Reconstruct metabolic networks from KEGG organism annotations and pathway databases. +Builds a draft network from the KEGG annotation of an organism or from KEGG pathway and reaction entries. ::: :::{grid-item-card} Flux analysis -FBA, FVA, gene knockouts, and flux sampling with Gurobi or GLPK solvers. +Runs flux balance analysis (FBA), flux variability analysis (FVA), gene and reaction deletions and flux sampling. The MATLAB toolbox solves with GLPK, Gurobi, SCIP or a COBRA Toolbox solver; raven-toolbox uses the solvers of cobrapy. ::: :::{grid-item-card} ftINIT -Fast task-and-data-driven INIT for extracting context-specific models from transcriptomics data. +Extracts a context-specific model from gene expression data. ftINIT is the successor of the legacy tINIT method. ::: :::{grid-item-card} Gap-filling -Identify and fill stoichiometric gaps by LP to restore connectivity or enable predicted growth. +Adds reactions from a template model so that blocked reactions carry flux, a metabolic task passes or the model grows. ::: :::{grid-item-card} Model curation -Check mass and charge balance, dead-end metabolites, and metabolic task fulfilment. +Checks the structure, the mass and charge balance, the connectivity and the metabolic tasks of a model. ::: :::: ## Quick start +The example loads yeast-GEM, sets growth as the objective, limits glucose uptake to 1 mmol/gDW/h and runs FBA. + ::::{tab-set} :::{tab-item} 🐍 Python :sync: python @@ -80,16 +80,13 @@ Check mass and charge balance, dead-end metabolites, and metabolic task fulfilme ```python from raven_toolbox.io import read_yaml_model -# load yeast-GEM from RAVEN YAML -- returns a plain cobra.Model +# read_yaml_model returns a cobra.Model model = read_yaml_model("yeast-GEM.yml") -# set growth as the objective model.objective = "r_2111" - -# constrain glucose uptake to 1 mmol/gDW/h model.reactions.get_by_id("r_1714").lower_bound = -1.0 -# run FBA -- simulation comes from cobrapy, unchanged +# optimize is the cobrapy method sol = model.optimize() print(f"Growth rate: {sol.objective_value:.4f} h⁻¹") ``` @@ -98,16 +95,11 @@ print(f"Growth rate: {sol.objective_value:.4f} h⁻¹") :sync: matlab ```matlab -% load yeast-GEM from RAVEN YAML model = readYAMLmodel('yeast-GEM.yml'); -% set growth as the objective model = setParam(model, 'obj', 'r_2111', 1); - -% constrain glucose uptake to 1 mmol/gDW/h model = setParam(model, 'lb', 'r_1714', -1); -% run FBA sol = solveLP(model); fprintf('Growth rate: %.4f h-1\n', sol.f); ``` @@ -122,28 +114,28 @@ fprintf('Growth rate: %.4f h-1\n', sol.f); :link: guide/index :link-type: doc -Twenty-two task-focused pages, MATLAB and Python side by side, with the examples executed and their output checked. +One page per task, with MATLAB and Python examples side by side. Most examples are executed, and their output is checked against the text. ::: :::{grid-item-card} API reference :link: api/index :link-type: doc -Function reference for both MATLAB and Python. +Reference of the MATLAB functions and the Python functions. ::: :::{grid-item-card} Installation :link: installation/index :link-type: doc -Set up RAVEN in MATLAB or raven-toolbox in Python with a solver. +Installation of RAVEN or raven-toolbox, and the choice of a solver. ::: :::{grid-item-card} Comparison of RAVEN and raven-toolbox :link: raven3-vs-raven-toolbox :link-type: doc -Use of each implementation, features present in only one of them, and where the same function gives a different answer. +The features that exist in one implementation alone, and the functions that return a different result in each. ::: :::: @@ -158,15 +150,15 @@ Cite the following publication when using RAVEN in research: > coelicolor*.** PLoS Computational Biology 14(10): e1006541. > -[Citations](references.md) holds the full list, including the first RAVEN paper (2013) and the GEM reconstruction protocol. +[Citations](references.md) lists further publications, among them the first RAVEN paper (2013) and the protocol for genome-scale model reconstruction. ## Source code -RAVEN is publicly available under the GPLv3 licence at [CompMetEng/RAVEN](https://github.com/CompMetEng/RAVEN){.github}, archived at [https://doi.org/10.5281/zenodo.3689518](https://doi.org/10.5281/zenodo.3689518). raven-toolbox, the Python port, is under the MIT licence at [CompMetEng/raven-toolbox](https://github.com/CompMetEng/raven-toolbox){.github}. +RAVEN is released under the GPLv3 licence at [CompMetEng/RAVEN](https://github.com/CompMetEng/RAVEN){.github}, archived at [https://doi.org/10.5281/zenodo.3689518](https://doi.org/10.5281/zenodo.3689518). raven-toolbox, the Python port, is released under the MIT licence at [CompMetEng/raven-toolbox](https://github.com/CompMetEng/raven-toolbox){.github}. ## Issues and discussions -Bug reports and feature requests go to the tracker of the toolbox concerned: [RAVEN issues](https://github.com/CompMetEng/RAVEN/issues){.github} for the MATLAB toolbox, [raven-toolbox issues](https://github.com/CompMetEng/raven-toolbox/issues){.github} for the Python package. Questions and general discussion go to [RAVEN discussions](https://github.com/CompMetEng/RAVEN/discussions){.github} or [raven-toolbox discussions](https://github.com/CompMetEng/raven-toolbox/discussions){.github}. +Bug reports and feature requests go to the issue tracker of the toolbox concerned: [RAVEN issues](https://github.com/CompMetEng/RAVEN/issues){.github} for the MATLAB toolbox, [raven-toolbox issues](https://github.com/CompMetEng/raven-toolbox/issues){.github} for the Python package. Questions go to [RAVEN discussions](https://github.com/CompMetEng/RAVEN/discussions){.github} or [raven-toolbox discussions](https://github.com/CompMetEng/raven-toolbox/discussions){.github}. ```{toctree} :hidden: diff --git a/docs/installation/data-and-binaries.md b/docs/installation/data-and-binaries.md index 97b41cf..f7b6572 100644 --- a/docs/installation/data-and-binaries.md +++ b/docs/installation/data-and-binaries.md @@ -1,25 +1,25 @@ # Download data and binaries -Neither toolbox ships the large files it needs. The KEGG reference data, the profile hidden Markov model (HMM) libraries and the BLAST+, DIAMOND and HMMER executables are fetched on first use from a shared, checksummed release and cached locally. This page covers what is fetched, where it is kept and how to override each step. +Neither toolbox ships the large files that it needs. The KEGG reference data, the libraries of profile hidden Markov models (HMMs) and the executables of BLAST+, DIAMOND and HMMER are fetched on first use from a shared, checksummed release and cached locally. The sections below describe what is fetched, where it is kept and how each step is overridden. -Both toolboxes read from the same [`toolbox-assets`](https://github.com/CompMetEng/toolbox-assets) repository, so the files RAVEN downloads and the files raven-toolbox downloads are the same files. +Both toolboxes read from the same [`toolbox-assets`](https://github.com/CompMetEng/toolbox-assets) repository, so RAVEN and raven-toolbox download the same files. ## Downloaded artefacts -| Artefact | Size | Needed for | +| Artefact | Download size | Needed for | |---|---|---| -| KEGG core bundle (reference model plus the KO, reaction and organism-gene tables) | about 47 MB | [Reconstruction from KEGG](../guide/kegg.md), either method | -| KEGG HMM library, one per domain | 129 MB compressed, eukaryotes | a KEGG draft for an organism KEGG has never seen | -| KEGG taxonomy | small | phylogenetic weighting of KO assignments | -| BLAST+ (`blastp`, `makeblastdb`) | small | [Reconstruction from homology](../guide/homology.md) | -| DIAMOND | small | the faster homology method | -| HMMER (`hmmsearch`) | small | searching the KEGG HMM library | +| KEGG core bundle (reference model and the tables of KOs, reactions and organism genes) | 49 MB | [Reconstruction from KEGG](../guide/kegg.md), both methods | +| KEGG HMM library, one per domain | 135 MB for eukaryotes, 155 MB for prokaryotes | a KEGG draft for an organism that is not in KEGG | +| KEGG taxonomy | 0.2 MB | phylogenetic weighting of KO assignments | +| BLAST+ (`blastp`, `makeblastdb`) | 13 to 35 MB per platform | [Reconstruction from homology](../guide/homology.md) | +| DIAMOND | 3 to 9 MB per platform | the faster homology method | +| HMMER (`hmmsearch`) | 0.2 to 1.8 MB per platform | searching the KEGG HMM library | -Nothing is downloaded until a function needs it. Loading, editing, simulating and comparing models require none of it. +Nothing is downloaded until a function needs it. Loading, editing, simulating and comparing models need none of these files. ## Cache locations -raven-toolbox caches under `$XDG_CACHE_HOME`, or `~/.cache` when that is unset, using the same rule on every platform: +raven-toolbox caches under `$XDG_CACHE_HOME`, or under `~/.cache` when that variable is not set, with the same rule on every platform: ::::{tab-set} :::{tab-item} Linux @@ -45,15 +45,15 @@ raven-toolbox caches under `$XDG_CACHE_HOME`, or `~/.cache` when that is unset, ::: :::: -The path `~/.cache` is used literally on every platform, and the platform conventions (`%LOCALAPPDATA%` on Windows, `~/Library/Caches` on macOS) do not apply. +The path `~/.cache` is used literally on every platform. The platform conventions (`%LOCALAPPDATA%` on Windows and `~/Library/Caches` on macOS) do not apply. -The version is part of the path, so two KEGG releases coexist and a pinned run fetches the release it was pinned to when that release is missing from the cache. +The version is part of the path, so two KEGG releases coexist. A pinned run fetches the pinned release when the cache lacks it. -In MATLAB the KEGG artefacts go to the directory `dataDir`, which `getKEGGModelForOrganism` requires as an argument. The executables are stored in `src/external/` (`blast+`, `diamond`, `hmmer`) inside the RAVEN installation directory on every platform. +In MATLAB the KEGG artefacts go to the directory `dataDir`, which `getKEGGModelForOrganism` requires as an argument. The executables are stored in `src/external/` (`blast+`, `diamond`, `hmmer`) in the RAVEN installation directory on every platform. ## Tool resolution -raven-toolbox resolves an executable in a fixed order and reaches the network only at the download step: +raven-toolbox resolves an executable in a fixed order and reaches the network at the download step: ```text explicit binary= argument @@ -63,11 +63,11 @@ explicit binary= argument → an error naming the conda package and the manual alternative ``` -An existing installation therefore takes precedence, and the bundled copy is used only on a machine without one. The environment variable per tool is `RAVEN_PYTHON_` followed by the tool name: `RAVEN_PYTHON_BLASTP`, `RAVEN_PYTHON_MAKEBLASTDB`, `RAVEN_PYTHON_DIAMOND`, `RAVEN_PYTHON_HMMSEARCH`, `RAVEN_PYTHON_HMMBUILD`, `RAVEN_PYTHON_HMMPRESS`, `RAVEN_PYTHON_HMMSCAN`, `RAVEN_PYTHON_MAFFT`, `RAVEN_PYTHON_CDHIT`. +An existing installation therefore takes precedence, and the bundled copy is used on a machine without one. The environment variable of a tool is `RAVEN_PYTHON_` followed by the tool name: `RAVEN_PYTHON_BLASTP`, `RAVEN_PYTHON_MAKEBLASTDB`, `RAVEN_PYTHON_DIAMOND`, `RAVEN_PYTHON_HMMSEARCH`, `RAVEN_PYTHON_HMMBUILD`, `RAVEN_PYTHON_HMMPRESS`, `RAVEN_PYTHON_HMMSCAN`, `RAVEN_PYTHON_MAFFT` and `RAVEN_PYTHON_CDHIT`. ## Fetching before first use -Downloading on first use puts the download inside the first run, which is not always where it should be. Both toolboxes can fetch the files before that run. +A download on first use takes place inside the first run. Both toolboxes can fetch the files before that run. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -77,7 +77,7 @@ Downloading on first use puts the download inside the first run, which is not al downloadRavenBinaries ``` -`downloadRavenBinaries` fetches the executables that RAVEN needs into the `src/external/` directory of the RAVEN installation. The KEGG artefacts are fetched separately, on the first `getKEGGModelForOrganism` call, into the `dataDir` given there. +`downloadRavenBinaries` fetches the executables that RAVEN needs into the `src/external/` directory of the RAVEN installation. The KEGG artefacts are fetched separately by the first call of `getKEGGModelForOrganism`, into the `dataDir` given there. ::: :::{tab-item} 🐍 Python :sync: python @@ -88,21 +88,21 @@ raven-toolbox-binaries --set runtime # blastp, makeblastdb, diamond, hmmsearc raven-toolbox-binaries --set build # hmmbuild, mafft, cd-hit ``` -The `runtime` set covers an ordinary reconstruction. The `build` set is for rebuilding the KEGG HMM libraries, which end users do not do. +The `runtime` set covers an ordinary reconstruction. The `build` set serves to rebuild the KEGG HMM libraries, which end users do not do. -The command skips anything already on the `PATH`, verifies every download against its checksum and reports tools with no bundle for the platform without failing. +The command skips anything that is on the `PATH` already, verifies every download against its checksum and reports tools without a bundle for the platform without failing. ::: :::: ## Working offline -Setting `RAVEN_PYTHON_AUTOFETCH` to `0`, `false`, `no` or `off` stops raven-toolbox from reaching the network. Resolution then stops at the `PATH` and raises an error that names the missing tool, without downloading it. The explicit `raven-toolbox-binaries` command fetches when run. An air-gapped setup fetches once on a connected machine, copies the cache and sets the variable. +`RAVEN_PYTHON_AUTOFETCH` set to `0`, `false`, `no` or `off` stops raven-toolbox from reaching the network. Resolution then stops at the `PATH` and raises an error that names the missing tool, without downloading it. The explicit command `raven-toolbox-binaries` fetches when it is run. An air-gapped setup fetches once on a connected machine, copies the cache and sets the variable. -For data artefacts, an explicit directory replaces the cache: `get_kegg_model_for_organism_from_artefacts` and its siblings take an `artefact_dir=` argument, and MATLAB's `dataDir` is already explicit. +For data artefacts, an explicit directory replaces the cache. `get_kegg_model_for_organism_from_artefacts` and the related functions take an `artefact_dir=` argument, and `dataDir` of MATLAB is explicit already. ## Platform support -Every tool is invoked as a subprocess, so the availability of a native build for the platform decides support. +Every tool is invoked as a subprocess, so the availability of a native build for the platform decides the support. The bundles in `toolbox-assets` are built for Linux on x86-64, macOS on Apple silicon (arm64) and Windows on x86-64. Intel Macs and Linux on arm64 have no bundle, and the table applies to the three bundle platforms. | Tool | Linux | macOS | Windows | |---|---|---|---| @@ -113,22 +113,22 @@ Every tool is invoked as a subprocess, so the availability of a native build for | MAFFT | yes | yes | no | | CD-HIT | yes | yes | no | -On Windows, homology reconstruction and the KEGG query path run natively, since they need only the first three tools. Building an HMM library needs MAFFT and CD-HIT, which have no Windows builds. Use Windows Subsystem for Linux 2 (WSL2) for that and keep the whole stack inside it, since raven-toolbox calls the resolved executable directly and does not translate paths between Windows and WSL. +On Windows, homology reconstruction and the KEGG query path run natively, because they need the first three tools. Building an HMM library needs MAFFT and CD-HIT, which have no Windows builds. Building runs in Windows Subsystem for Linux 2 (WSL2), with Python and the binaries both inside it, because raven-toolbox calls the resolved executable directly and does not translate paths between Windows and WSL. :::{admonition} Troubleshooting :class: warning | Problem | Possible reason | Solution | |---|---|---| -| A machine with no network | The first reconstruction needs to download files, which fails without a network. | Fetch on a connected machine, copy `~/.cache/raven_toolbox` and set `RAVEN_PYTHON_AUTOFETCH=0`. | -| An unexpected version of a tool | The `PATH` takes precedence over the pinned bundle, so a conda environment carrying an old BLAST+ supplies it without a message. | Point the `RAVEN_PYTHON_*` variable at the intended binary. | +| A machine without network | The first reconstruction needs to download files, which fails without a network. | Fetch on a connected machine, copy `~/.cache/raven_toolbox` and set `RAVEN_PYTHON_AUTOFETCH=0`. | +| An unexpected version of a tool | The `PATH` takes precedence over the pinned bundle, so a conda environment with an old BLAST+ supplies it without a message. | Point the `RAVEN_PYTHON_*` variable at the intended binary. | | A read-only or unusual home directory | The cache follows `XDG_CACHE_HOME`. | Set `XDG_CACHE_HOME` to a writable location on a shared or containerised machine. | -| Building HMM libraries on Windows | MAFFT and CD-HIT have no Windows builds. | Use WSL2 and keep Python and the binaries both inside it. | -| Disk space | The HMM libraries are the large item, and the first KEGG model build needs a few hundred MB of working space beyond the download. | | +| Building HMM libraries on Windows | MAFFT and CD-HIT have no Windows builds. | Use WSL2 with Python and the binaries both inside it. | +| Disk space | The HMM libraries are the largest item, and the first KEGG model build needs a few hundred MB of working space beyond the download. | | ::: ## See also -- [Reconstruction from KEGG](../guide/kegg.md), the workflow that fetches the KEGG artefacts. -- [Reconstruction from homology](../guide/homology.md), the workflow that needs BLAST+ or DIAMOND. +- [Reconstruction from KEGG](../guide/kegg.md): the workflow that fetches the KEGG artefacts. +- [Reconstruction from homology](../guide/homology.md): the workflow that needs BLAST+ or DIAMOND. - [RAVEN (MATLAB)](raven.md) and [raven-toolbox (Python)](python.md). diff --git a/docs/installation/index.md b/docs/installation/index.md index b887ea1..fb4da60 100644 --- a/docs/installation/index.md +++ b/docs/installation/index.md @@ -1,6 +1,6 @@ # Installation -RAVEN is available as a MATLAB toolbox and as the Python package raven-toolbox. Both build the same models with the same algorithms, and the choice depends on the workflow. +RAVEN is available as a MATLAB toolbox and as the Python package raven-toolbox. Both build models for the same workflows, some algorithms differ between them ([Tuned parameter defaults](../tuned-parameters.md)), and the workflow decides which to use. ::::{grid} 1 2 2 2 @@ -8,27 +8,27 @@ RAVEN is available as a MATLAB toolbox and as the Python package raven-toolbox. :link: raven :link-type: doc -Installation of the MATLAB toolbox through the MATLAB Add-Ons manager, a release download or `git`, with upgrading and removal. +Installation of the MATLAB toolbox with the Add-Ons manager, a release download or `git`, and its upgrade and removal. ::: :::{grid-item-card} raven-toolbox (Python) :link: python :link-type: doc -`pip install raven-toolbox`, development installs, upgrading and removal. +`pip install --pre raven-toolbox`, a development installation, and the upgrade and removal of the package. ::: :::: ## Choosing a solver -Both versions need a linear-programming solver. The options are the same for either language: +Both versions need a linear-programming solver: | Solver | Licence | Suited to | |---|---|---| -| Gurobi | Free academic | Genome-scale models, ftINIT and the legacy tINIT method, gap-filling (recommended) | -| GLPK | Open source | Small to medium models, getting started | -| SCIP | Open source | MILP problems (MATLAB); `setRavenSolver('scip')` downloads the binary on Windows only | +| Gurobi | Free academic | Genome-scale models, ftINIT, gap-filling, and in MATLAB the legacy tINIT method (recommended) | +| GLPK | Open source | Small and medium models, and getting started | +| SCIP | Open source | MILP problems (MATLAB); `setRavenSolver('scip')` downloads the binary on Windows | In MATLAB, `setRavenSolver('gurobi')` selects the active solver. In raven-toolbox, the solver is configured through cobrapy (`cobra.Configuration`). diff --git a/docs/installation/python.md b/docs/installation/python.md index 319ecb2..46041fc 100644 --- a/docs/installation/python.md +++ b/docs/installation/python.md @@ -4,33 +4,35 @@ raven-toolbox is the Python implementation of RAVEN, built on [cobrapy](https:// ## Requirements -- Python ≥ 3.11 -- cobrapy, installed automatically as a dependency -- A solver: GLPK (bundled with cobrapy) works for small and medium models, and [Gurobi](https://www.gurobi.com/) is recommended for genome-scale work. See [Choosing a solver](index.md#choosing-a-solver). +- Python 3.11 or later. +- cobrapy, which is installed as a dependency. +- A solver. GLPK, which is bundled with cobrapy, works for small and medium models, and [Gurobi](https://www.gurobi.com/) is recommended for genome-scale work ([Choosing a solver](index.md#choosing-a-solver)). -A virtual environment is recommended. Creation and activation: +A virtual environment is recommended. It is created and activated with: ```bash python -m venv .venv source .venv/bin/activate # Windows: .venv\Scripts\activate ``` -raven-toolbox fetches KEGG data and reconstruction binaries (BLAST+, DIAMOND, HMMER) on first use. [Download data and binaries](data-and-binaries.md) covers preparing a machine with no network access ahead of time. +raven-toolbox fetches KEGG data and the reconstruction binaries (BLAST+, DIAMOND and HMMER) on first use. [Download data and binaries](data-and-binaries.md) describes how to prepare a machine without network access in advance. --- ## Installation +The release documented on this site, 3.0.0b1, is a pre-release. PyPI also holds the stable release 0.4.0, and `pip` selects that release unless `--pre` is given. + ::::{tab-set} :::{tab-item} {octicon}`package;1em` From PyPI ```bash -pip install raven-toolbox +pip install --pre raven-toolbox ``` ::: :::{tab-item} {octicon}`git-branch;1em` From source -For the latest unreleased code or to contribute: +The source installation provides the latest unreleased code and is the route for contributors: ```bash git clone https://github.com/CompMetEng/raven-toolbox.git @@ -49,7 +51,7 @@ import raven_toolbox print(raven_toolbox.__version__) ``` -The import package is `raven_toolbox` (underscore) and the PyPI distribution name is `raven-toolbox` (hyphen). +The import package is `raven_toolbox` with an underscore, and the PyPI distribution is `raven-toolbox` with a hyphen. --- @@ -59,7 +61,7 @@ The import package is `raven_toolbox` (underscore) and the PyPI distribution nam :::{tab-item} {octicon}`package;1em` From PyPI ```bash -pip install --upgrade raven-toolbox +pip install --pre --upgrade raven-toolbox ``` ::: :::{tab-item} {octicon}`git-branch;1em` From source @@ -82,7 +84,7 @@ pip uninstall raven-toolbox ## Solver configuration -raven-toolbox uses cobrapy's solver interface: +raven-toolbox uses the solver interface of cobrapy: ```python import cobra diff --git a/docs/installation/raven.md b/docs/installation/raven.md index dc8ba48..305361e 100644 --- a/docs/installation/raven.md +++ b/docs/installation/raven.md @@ -1,12 +1,12 @@ # RAVEN (MATLAB) -The RAVEN Toolbox runs in MATLAB and works independently. It does not require the COBRA Toolbox, although it can interoperate with it. +The RAVEN Toolbox runs in MATLAB and works independently. It does not require the COBRA Toolbox and can interoperate with it. ## Requirements -- MATLAB, with no minimum release stated. The code has no check that rejects an older release. `readYAMLmodel` has a fallback for releases before R2020b, `parseTaskList` calls `readcell` (R2019a) for Excel task lists and `addIdentifierPrefix` calls `regexpPattern` (R2020b). Continuous integration tests R2024b only; older releases were not tested. No additional MathWorks toolboxes are required. -- A linear-programming solver: [Gurobi](https://www.gurobi.com/) (free academic licence, recommended) or GLPK, bundled with RAVEN itself and needing no separate install. See [Choosing and configuring a solver](#choosing-a-solver-matlab). -- RAVEN bundles `libSBML` and the GLPK mex files for Windows, macOS and Linux. `BLAST+`, `DIAMOND` and `HMMER` are not bundled: RAVEN fetches the build for the platform on first use, so the reconstruction functions need internet access the first time they run. `downloadRavenBinaries` fetches them ahead of time. [Download data and binaries](data-and-binaries.md) lists the full set, where it is cached and how to prepare an offline machine. +- MATLAB. No minimum release is stated, and the code has no check that rejects an older release. `readYAMLmodel` has a fallback for releases before R2020b, `parseTaskList` calls `readcell` (R2019a) for Excel task lists, and `addIdentifierPrefix` calls `regexpPattern` (R2020b). Continuous integration tests R2024b, and older releases are not tested. No additional MathWorks toolboxes are required. +- A linear-programming solver: [Gurobi](https://www.gurobi.com/) (free academic licence, recommended) or GLPK, which is bundled with RAVEN and needs no separate installation. See [Choosing and configuring a solver](#choosing-a-solver-matlab). +- RAVEN bundles `libSBML` and the GLPK mex files for Windows, macOS and Linux. `BLAST+`, `DIAMOND` and `HMMER` are not bundled. RAVEN fetches the build for the platform on first use, so the reconstruction functions need internet access the first time they run. `downloadRavenBinaries` fetches them in advance. [Download data and binaries](data-and-binaries.md) lists the full set, the cache location and the preparation of an offline machine. --- @@ -16,11 +16,11 @@ The RAVEN Toolbox runs in MATLAB and works independently. It does not require th | Solver | LP | MILP | Relative speed | Licence | |---|---|---|---|---| | GLPK | yes | yes (RAVEN warns; slow on large problems) | 1x (baseline) | open source, bundled with RAVEN | -| SCIP | yes | yes | 0.33x | open source; `setRavenSolver('scip')` downloads the MEX file on Windows (from the RAVEN v2.8.6 GitHub release), a separate install on macOS or Linux | +| SCIP | yes | yes | 0.33x | open source; `setRavenSolver('scip')` downloads the MEX file on Windows (from the RAVEN v2.8.6 GitHub release), and macOS and Linux need a separate installation | | Gurobi | yes | yes | 4.2x | free academic licence | | COBRA Toolbox | yes | depends on the COBRA solver configured | depends on the COBRA solver configured | depends on the COBRA solver configured | -MILP-solving functions (`getMinimalMedium`, ftINIT, some gap-filling algorithms) also run on the bundled GLPK, with a warning that recommends Gurobi. On genome-scale models GLPK can be slower than Gurobi, which runs at 4.2 times the speed of GLPK in the table. +The functions that solve MILPs (`getMinimalMedium`, ftINIT and some gap-filling algorithms) also run on the bundled GLPK and give a warning that recommends Gurobi. On genome-scale models GLPK can be slower than Gurobi, which runs at 4.2 times the speed of GLPK in the table. `setRavenSolver` sets the active solver and takes `'gurobi'`, `'glpk'`, `'scip'` or `'cobra'`: @@ -31,21 +31,21 @@ setRavenSolver('gurobi'); ### Gurobi 1. Install Gurobi 7.5 or later. -2. Request and download a licence (a free academic licence covers most research use). -3. Place the licence file where Gurobi's own installer says to. -4. Follow Gurobi's MATLAB integration instructions, then run `savepath` to keep the change after a restart. +2. Request and download a licence. A free academic licence covers most research use. +3. Place the licence file where the installer of Gurobi says. +4. Follow the MATLAB integration instructions of Gurobi, then run `savepath` to keep the change after a restart. 5. Run `setRavenSolver('gurobi')`. ### Solving through the COBRA Toolbox -RAVEN can solve through the solver that the COBRA Toolbox has configured, in place of its own bundled options: +RAVEN can solve with the solver that the COBRA Toolbox has configured, in place of the bundled solvers: ```matlab changeCobraSolver('glpk'); % or any solver the COBRA Toolbox supports setRavenSolver('cobra'); ``` -`ravenCobraWrapper` converts a model between the RAVEN struct and the COBRA Toolbox structure; see [COBRA Toolbox conversion](../raven3-vs-raven-toolbox/what-each-has.md#cobra-toolbox-conversion). +`ravenCobraWrapper` converts a model between the RAVEN struct and the COBRA Toolbox structure ([COBRA Toolbox conversion](../raven3-vs-raven-toolbox/what-each-has.md#cobra-toolbox-conversion)). --- @@ -54,31 +54,31 @@ setRavenSolver('cobra'); :::{admonition} Installing 3.0.0b1 :class: warning -This site documents RAVEN 3.0.0b1, a pre-release. The Add-Ons manager and the normal releases page of the Release download offer only the latest stable release (RAVEN 2.x), so neither method installs 3.0.0b1. Use the Clone with git tab, or the direct archive link in the Release download tab. +This site documents RAVEN 3.0.0b1, a pre-release. The Add-Ons manager and the normal releases page of the release download offer the latest stable release (RAVEN 2.x), so neither installs 3.0.0b1. The Clone with git tab or the archive link of the Release download tab installs it. ::: ::::{tab-set} :::{tab-item} {octicon}`plug;1em` Add-Ons manager -The installation runs from within MATLAB with no separate download. The installed version is the latest stable release (RAVEN 2.x), not 3.0.0b1; see the warning at the start of the [Installation](#installation) section. +The installation runs from within MATLAB and needs no separate download. The installed version is the latest stable release (RAVEN 2.x), which is older than 3.0.0b1, as the warning at the start of the [Installation](#installation) section states. -1. Open the **Home** tab and click **Add-Ons → Get Add-Ons**. -2. Search for **RAVEN Toolbox** and click **Add → Add to MATLAB**. +1. Open the Home tab and select Add-Ons, then Get Add-Ons. +2. Search for RAVEN Toolbox and select Add, then Add to MATLAB. 3. [Verify the installation](#verification). -If MATLAB does not pick up the toolbox after step 2, `matlab.addons.enableAddon("RAVEN")` enables it. +When MATLAB does not pick up the toolbox after step 2, `matlab.addons.enableAddon("RAVEN")` enables it. ::: :::{tab-item} {octicon}`download;1em` Release download -This method suits offline or managed environments. The [RAVEN releases page](https://github.com/CompMetEng/RAVEN/releases) lists only stable releases, and 3.0.0b1 is a git tag with no packaged release, so the archive is downloaded directly: +This method suits offline and managed environments. The [RAVEN releases page](https://github.com/CompMetEng/RAVEN/releases) lists stable releases, and 3.0.0b1 is a git tag without a packaged release, so the archive is downloaded directly: -1. Download [the 3.0.0b1 archive](https://github.com/CompMetEng/RAVEN/archive/refs/tags/3.0.0b1.zip). This works without git. +1. Download [the 3.0.0b1 archive](https://github.com/CompMetEng/RAVEN/archive/refs/tags/3.0.0b1.zip), which works without git. 2. Extract it to any location. -3. In MATLAB, add the RAVEN folder to the path (`pathtool`), then [verify](#verification). +3. In MATLAB, add the RAVEN folder to the path (`pathtool`) and [verify](#verification). ::: :::{tab-item} {octicon}`git-branch;1em` Clone with git -The commands clone the repository and check out the `3.0.0b1` tag. A plain `git clone` tracks the `develop` branch, which can contain changes made after this site's pre-release. +The commands clone the repository and check out the `3.0.0b1` tag. A plain `git clone` follows the `develop` branch, which can contain changes made after the pre-release documented on this site. ```bash git clone --depth=1 https://github.com/CompMetEng/RAVEN.git @@ -87,7 +87,7 @@ git fetch --depth=1 origin tag 3.0.0b1 git checkout 3.0.0b1 ``` -Add the folder to the MATLAB path and [verify](#verification). +Afterwards, add the RAVEN folder to the MATLAB path and [verify](#verification). ::: :::: @@ -96,13 +96,13 @@ Add the folder to the MATLAB path and [verify](#verification). (verify)= ## Verification -From the MATLAB command window: +In the MATLAB command window: ```matlab checkRaven ``` -The output of a successful run has the form: +A successful run prints output of this form: ```text *** THE RAVEN TOOLBOX *** @@ -139,7 +139,7 @@ The output of a successful run has the form: *** checkRaven complete *** ``` -MATLAB can report that it could not save the path, which is common on shared or managed installations without write access to MATLAB's own `pathdef.m`. In that case, run `addRavenToUserPath`, which writes a `startup.m` that adds RAVEN to the path on every MATLAB start and leaves the shared path file unchanged. `addRavenToUserPath('overwrite', false)` appends to an existing `startup.m`, which is replaced otherwise. +MATLAB can report that it could not save the path, which is common on shared or managed installations without write access to the `pathdef.m` of MATLAB. `addRavenToUserPath` then writes a `startup.m` that adds RAVEN to the path on every MATLAB start and leaves the shared path file unchanged. `addRavenToUserPath('overwrite', false)` appends to an existing `startup.m`, which is replaced otherwise. --- @@ -148,11 +148,11 @@ MATLAB can report that it could not save the path, which is common on shared or ::::{tab-set} :::{tab-item} {octicon}`plug;1em` Add-Ons manager -In MATLAB, open **Help → Check for Updates**, click **Update** for RAVEN and run `checkRaven` again. +In MATLAB, open Help, then Check for Updates, select Update for RAVEN and run `checkRaven` again. ::: :::{tab-item} {octicon}`download;1em` Release download -Close MATLAB, delete the old RAVEN folder, download and extract the new release and run `checkRaven`. +MATLAB should be closed, the old RAVEN folder deleted, the new release downloaded and extracted, and `checkRaven` run. ::: :::{tab-item} {octicon}`git-branch;1em` Clone with git @@ -163,7 +163,7 @@ git checkout develop git pull origin develop ``` -`develop` can contain unreleased changes. Afterwards, run `checkRaven`. +`develop` can contain unreleased changes. `checkRaven` should be run afterwards. ::: :::: @@ -174,7 +174,7 @@ git pull origin develop ::::{tab-set} :::{tab-item} {octicon}`plug;1em` Add-Ons manager -Open **Add-Ons → Manage Add-Ons** and remove RAVEN from the list. A following call of `removeRavenFromPath` should report an unrecognised-function error, which confirms that RAVEN is off the path. +RAVEN is removed from the list under Add-Ons, then Manage Add-Ons. A following call of `removeRavenFromPath` should report an unrecognised-function error, which confirms that RAVEN is off the path. ::: :::{tab-item} {octicon}`download;1em` Release download @@ -183,7 +183,7 @@ which removeRavenFromPath % locate the installation removeRavenFromPath % clear RAVEN from the MATLAB path ``` -Then delete the RAVEN folder from disk. +The RAVEN folder is then deleted from disk. ::: :::{tab-item} {octicon}`git-branch;1em` Clone with git @@ -192,7 +192,7 @@ which removeRavenFromPath % locate the installation removeRavenFromPath % clear RAVEN from the MATLAB path ``` -Then delete the cloned RAVEN folder from disk. +The cloned RAVEN folder is then deleted from disk. ::: :::: diff --git a/docs/protocol/biomass.md b/docs/protocol/biomass.md index 6cf3f4f..41c05dc 100644 --- a/docs/protocol/biomass.md +++ b/docs/protocol/biomass.md @@ -1,18 +1,18 @@ # Biomass composition -Metabolism must be able to synthesise the macromolecules that make up biomass. GEMs encode this in **biomass pseudoreactions** that specify which macromolecules (DNA, RNA, protein, lipid, carbohydrate) are required and in what amounts. Defining the biomass composition early ensures that the model can biosynthesise everything it needs. +Metabolism must be able to synthesise the macromolecules that make up the biomass. GEMs encode this in biomass pseudoreactions, which specify the required macromolecules (DNA, RNA, protein, lipid and carbohydrate) and their amounts. Defining the biomass composition early states what the model must be able to synthesise. In the template models the biomass is split over several pseudoreactions, for example the DNA pseudoreaction: $$ x\;\text{dAMP} + y\;\text{dCMP} + y\;\text{dGMP} + x\;\text{dTMP} \rightarrow \text{DNA} $$ -which then combine into biomass: +The pseudoreactions combine into biomass: $$ \text{DNA} + \text{RNA} + \text{protein} + \text{lipids} + \text{carbohydrate} \rightarrow \text{biomass} $$ ## Taking the pseudoreactions from a template -*H. polymorpha* has poly-unsaturated fatty acids, similar to *R. toruloides*, so the rhto-GEM biomass pseudoreactions are used as the starting point. They are recognised by reaction names ending in `pseudoreaction`: +*H. polymorpha* has poly-unsaturated fatty acids, as does *R. toruloides*, so the biomass pseudoreactions of rhto-GEM are the starting point. Their reaction names end in `pseudoreaction`: ```matlab biomassRxns = modelRhto.rxns(endsWith(modelRhto.rxnNames, 'pseudoreaction')); @@ -21,7 +21,7 @@ model = addRxnsGenesMets(model, modelRhto, biomassRxns); ## Updating the stoichiometric coefficients -A spreadsheet in the repository's `data/biomass/` folder details the calculations behind the coefficients; the script loads them and applies each group with `changeRxns`. The DNA pseudoreaction (`r_4050`), for example: +A spreadsheet in the `data/biomass/` folder of the repository documents the calculations behind the coefficients. The script loads the coefficients and applies each group with `changeRxns`. The DNA pseudoreaction (`r_4050`) is an example: ```matlab DNA.mets = {'s_0584', 's_0589', 's_0615', 's_0649', 's_3720'}; @@ -29,21 +29,21 @@ DNA.stoichCoeffs = [-0.00189, -0.00174, -0.00174, -0.00189, 1]; model = changeRxns(model, 'r_4050', DNA, 1); ``` -The same pattern updates RNA (`r_4049`), protein (`r_4047`), carbohydrate (`r_4048`) and the lipid backbone and chain pseudoreactions (`r_4063`, `r_4065`). +The same pattern updates RNA (`r_4049`), protein (`r_4047`), carbohydrate (`r_4048`) and the pseudoreactions of the lipid backbone and chain (`r_4063`, `r_4065`). ## Origin of the coefficients -Determining a coefficient combines a measured total fraction of a macromolecule with the ratio of its constituents: +A coefficient combines the measured total fraction of a macromolecule with the ratio of its constituents: | Component | Total fraction from | Constituent ratio from | |---|---|---| -| DNA | scaled from yeast-GEM (0.24 % for the smaller genome) | DNA FASTA nucleotide frequencies | -| RNA | 6 % of biomass (as in yeast-GEM) | coding-sequence nucleotide frequencies (from the GenBank file) | -| Protein | measured total protein content | amino-acid frequencies in the protein FASTA | +| DNA | scaled from yeast-GEM (0.24 % for the smaller genome) | nucleotide frequencies in the DNA FASTA file | +| RNA | 6 % of biomass (as in yeast-GEM) | nucleotide frequencies of the coding sequences (from the GenBank file) | +| Protein | measured total protein content | amino-acid frequencies in the protein FASTA file | | Carbohydrate | measured glucan, mannan, trehalose and glycogen | reported levels for *H. polymorpha* | -| Lipids | measured lipid classes and acyl chains | the SLIME approach (next section) | +| Lipids | measured lipid classes and acyl chains | the SLIME approach (next page) | -For nucleotides, the DNA FASTA gives the frequency of each base, counted with `sed` and `grep`: +For nucleotides, the frequency of each base is counted in the DNA FASTA file with `sed` and `grep`: ```bash sed 's/>.*$//' hpo.fna > hanpo_modified.fa @@ -52,8 +52,8 @@ for i in {A,T,G,C}; do done ``` -Together with the paired nature of A·T and C·G and the nucleotide molecular weights, this gives the amount of each deoxyribonucleotide needed per gram of biomass. Ribonucleotide and amino-acid ratios are derived analogously from the coding sequences and the protein FASTA. +The frequencies, the pairing of A·T and C·G and the molecular weights of the nucleotides give the amount of each deoxyribonucleotide per gram of biomass. The ribonucleotide ratios and the amino-acid ratios are derived in the same way from the coding sequences and the protein FASTA file. -Lipid representation depends on two measurement types: lipid classes (for example triacylglycerol, phosphatidylinositol) and acyl chains (for example 16:0, 18:1). This model uses the SLIME formalism (Split Lipids Into Measurable Entities), set up in the next section. +The lipid representation depends on two types of measurements: lipid classes (for example triacylglycerol and phosphatidylinositol) and acyl chains (for example 16:0 and 18:1). The model uses the SLIME formalism (Split Lipids Into Measurable Entities), which the next page sets up. -Next: [Curation of lipid reactions](lipid-curation.md). +The next page is [Curation of lipid reactions](lipid-curation.md). diff --git a/docs/protocol/gap-filling.md b/docs/protocol/gap-filling.md index c94ea76..5483b84 100644 --- a/docs/protocol/gap-filling.md +++ b/docs/protocol/gap-filling.md @@ -1,12 +1,12 @@ # Gap-filling -Homology-based draft models tend to have many **gaps**: missing reactions that break a pathway needed to produce biomass macromolecules. The pathway is likely functional in the template, but `getBlast` and `getModelFromHomology` cannot always find a homolog for every gene, leaving the pathway incomplete. Gap-filling identifies and resolves these gaps so the draft becomes a functional model. +Homology-based draft models tend to have many gaps, which are missing reactions that break a pathway needed to produce the biomass macromolecules. The pathway is likely functional in the template, and `getBlast` and `getModelFromHomology` cannot find a homolog for every gene, which leaves the pathway incomplete. Gap-filling identifies and resolves the gaps, and the draft becomes a functional model. -RAVEN's `fillGaps` adds the smallest set of reactions from the template models needed to support flux through the draft. +`fillGaps` of RAVEN adds the smallest set of reactions from the template models that supports flux through the draft. ## Forcing biomass production -Gap-filling is set up to support growth on glycerol as carbon source with biomass production as objective. The model is forced to carry a small non-zero biomass flux, and glycerol uptake is raised so the fluxes stay above the MILP solver tolerance: +Gap-filling is set up for growth on glycerol as the carbon source with biomass production as the objective. Force the model to carry a small non-zero biomass flux, and raise the glycerol uptake so that the fluxes stay above the tolerance of the MILP solver: ```matlab model = setParam(model, 'obj', 'r_4041', 1); @@ -16,7 +16,7 @@ model = setParam(model, 'lb', 'r_1808', -10); % allow glycerol uptake ## Preparing the template models -Gaps should be resolved with enzymatic and transport reactions, and not with new exchange reactions. New exchange reactions could, for example, introduce other carbon sources. All exchange reactions, and the biomass pseudoreactions, are therefore removed from the template copies used as the reaction source. Reactions already shared between the two templates are also dropped to shrink the MILP search space: +Gaps are resolved with enzymatic and transport reactions. New exchange reactions are excluded because they could introduce other carbon sources. All exchange reactions are therefore removed from the copies of the templates that are the reaction source, and the script also removes the biomass pseudoreactions. Reactions that both templates share are also dropped, to reduce the MILP search space: ```matlab modelSce2 = removeReactions(modelSce, getExchangeRxns(modelSce, 'both'), true, true, true); @@ -31,7 +31,7 @@ modelRhto2 = removeReactions(modelRhto2, rxns); [~, ~, addedRxns, model] = fillGaps(model, {modelSce2, modelRhto2}, false, true); ``` -Setting `allowNetProduction` to `false` prevents additional exchange reactions, and `useModelConstraints` to `true` ensures that the glycerol-to-biomass flux is gap-filled. Verify that the model grows, then scale the lipids and restore the carbon-uptake bounds: +`allowNetProduction` set to `false` prevents additional exchange reactions, and `useModelConstraints` set to `true` ensures that the flux from glycerol to biomass is gap-filled. Verify that the model grows, then scale the lipids and restore the bounds of the carbon uptake: ```matlab sol = solveLP(model, 1); @@ -43,10 +43,17 @@ model = setParam(model, 'lb', 'r_4041', 0); model = deleteUnusedGenes(model); ``` -After gap-filling, the model produces all the macromolecules that constitute *H. polymorpha* biomass. +The `solveLP` call verifies that the model grows after gap-filling. ## Semi-automatic gap analysis -RAVEN has functions to find gaps: `canExchange` (net synthesis or consumption of metabolites, `'produce'` or `'consume'`), `checkProduction` (smallest set of metabolites needing net synthesis), `getAllSubGraphs` (disconnected subnetworks) and `haveFlux` (reactions that can or cannot carry flux). Absence of a biomass macromolecule from `canExchange(model, 'produce')` indicates a gap. +RAVEN has functions that find gaps: -Next: [Saving to GitHub and simulating](simulation.md). +- `canExchange` finds the net synthesis or consumption of metabolites (`'produce'` or `'consume'`). +- `checkProduction` finds the smallest set of metabolites that need net synthesis. +- `getAllSubGraphs` finds disconnected subnetworks. +- `haveFlux` finds the reactions that can or cannot carry flux. + +A biomass macromolecule that is absent from `canExchange(model, 'produce')` indicates a gap. + +The next page is [Saving to GitHub and simulating](simulation.md). diff --git a/docs/protocol/homology.md b/docs/protocol/homology.md index c19f25c..08da291 100644 --- a/docs/protocol/homology.md +++ b/docs/protocol/homology.md @@ -1,24 +1,24 @@ # Draft from homology -The protein FASTA files of the target and template organisms are compared to build a draft *H. polymorpha* model based on homology. Two RAVEN functions do the work: `getBlast` aligns the protein sequences, and `getModelFromHomology` uses those results to assemble the first draft. +The protein FASTA files of the target and the template organisms are compared to build a draft *H. polymorpha* model on the basis of homology. Two RAVEN functions do the work: `getBlast` aligns the protein sequences, and `getModelFromHomology` assembles the first draft from the results. ## Cleaning and matching protein identifiers -The gene identifiers in each template model must match those in its protein FASTA (compare against, for example, *S. cerevisiae* `YML001W`). For the target organism it is also convenient to shorten the identifiers to a standard format such as `Hanpo2_12345`. The *H. polymorpha* FASTA from JGI needs some editing to strip annotations; see [Note 2](#notes). +The gene identifiers in each template model must match those in its protein FASTA file, for example `YML001W` for *S. cerevisiae*. For the target organism, the identifiers can be shortened to a standard format such as `Hanpo2_12345`. The *H. polymorpha* FASTA file from JGI needs editing to remove annotations ([Note 2](#notes)). ## Determining homology by BLAST -Evidence of homology is determined by bidirectional BLAST of the target proteome against both template proteomes (this can take some minutes): +Evidence of homology comes from a bidirectional BLAST of the target proteome against both template proteomes, which can take some minutes: ```matlab blast = getBlast('hanpo', 'hanpo.faa', {'sce', 'rhto'}, {'sce.faa', 'rhto.faa'}); ``` -`getBlast` takes the target organism id, the target protein FASTA, a cell array of template ids, and a cell array of the corresponding template FASTA files. +`getBlast` takes the identifier of the target organism, the protein FASTA file of the target, a cell array of template identifiers and a cell array of the corresponding template FASTA files. ## Building the first draft -The BLAST results then drive `getModelFromHomology`: +The BLAST results drive `getModelFromHomology`: ```matlab model = getModelFromHomology({modelSce, modelRhto}, blast, 'hanpo', ... @@ -26,11 +26,11 @@ model = getModelFromHomology({modelSce, modelRhto}, blast, 'hanpo', ... model = contractModel(model); % merge identical reactions with different grRules ``` -The cutoffs control which BLAST hits count as homology. The maximum E-value is `1e-30` (the default), the minimum alignment length 150 (default 100; values up to 150 give the same result) and the minimum identity 35 % (default 40 %) (see [Note 6](#notes)). +The cut-offs select the BLAST hits that count as homology. The maximum E-value is `1e-30` (the default), the minimum alignment length is 150 (default 100; in the [calibration study](../parameter-tuning/studies/homology-cutoff-calibration.md), 150 scored at most 0.003 lower than 100) and the minimum identity is 35 % (default 40 %) ([Note 6](#notes)). ## Adding exchange reactions for the growth medium -The homology approach can only add gene-annotated reactions. Exchange reactions are not gene-associated (they simulate flow of metabolites in and out of the system), so they must be added explicitly. Add the exchanges that correspond to the growth-medium components: +The homology approach adds reactions that have gene annotations, and no others. Exchange reactions simulate the flow of metabolites in and out of the system and have no gene association, so they are added explicitly. Add the exchanges of the growth-medium components: ```matlab mediumComps = {'r_1654', 'r_1672', 'r_1808', 'r_1832', 'r_1861', ... @@ -38,15 +38,15 @@ mediumComps = {'r_1654', 'r_1672', 'r_1808', 'r_1832', 'r_1861', ... model = addRxnsGenesMets(model, modelSce, mediumComps); ``` -These represent ammonium, CO₂, glycerol, H⁺, iron, O₂, phosphate, sulfate, water and biomass, everything that must be able to enter or leave the system to support growth. Inspect the first draft: +They represent ammonium, CO₂, glycerol, H⁺, iron, O₂, phosphate, sulfate, water and biomass, which are everything that must enter or leave the system to support growth. Inspect the first draft: ```matlab exportToExcelFormat(model, 'scrap/hanpo_step3.3.xlsx'); ``` -Next: [Biomass composition](biomass.md). +The next page is [Biomass composition](biomass.md). ## Notes -- **Note 2: matching identifiers.** Protein identifiers must match between the model and FASTA of each organism. One common fix is to discard everything after the first whitespace in each FASTA header with `sed -i 's/ .*$//' sce.faa`, turning a long annotated header into a bare identifier such as `>YAL001C`. -- **Note 6: `getModelFromHomology` arguments.** Up to thirteen input arguments can be given (three required; the optional ones by position or as name-value pairs). The first four are the template model(s), the BLAST structure, the target organism id (which must match the one given to `getBlast`), and the preferred order of templates (here *S. cerevisiae* is preferred). The next arguments tweak the inclusion criteria; the seventh to ninth are the BLAST cutoffs (maximum E-value, minimum alignment length, minimum identity). The defaults balance sensitivity and specificity. +- **Note 2: matching identifiers.** The protein identifiers of the model and the FASTA file of each organism must match. A common fix discards everything after the first whitespace in each FASTA header with `sed -i 's/ .*$//' sce.faa`, which turns a long annotated header into a bare identifier such as `>YAL001C`. +- **Note 6: `getModelFromHomology` arguments.** Up to thirteen input arguments can be given, three of them required. The optional arguments are passed by position or as name-value pairs. The first four are the template models, the BLAST structure, the identifier of the target organism (which must match the one given to `getBlast`) and the preferred order of templates (here *S. cerevisiae* is preferred). The next arguments adjust the inclusion criteria, and the seventh to ninth are the BLAST cut-offs (maximum E-value, minimum alignment length and minimum identity). The defaults balance sensitivity and specificity. diff --git a/docs/protocol/index.md b/docs/protocol/index.md index eb7a850..c494852 100644 --- a/docs/protocol/index.md +++ b/docs/protocol/index.md @@ -1,29 +1,29 @@ # GEM reconstruction protocol: *Hansenula polymorpha* -This protocol is a complete homology-based reconstruction of `hanpo-GEM`, followed from start to finish: it takes template models as input and produces a growing methylotrophic draft. It is the published pipeline, in order, and states the choices made. +The protocol is a complete homology-based reconstruction of `hanpo-GEM`, from the template models as input to a draft that grows on methanol. It follows the published pipeline in order and states the choices that were made. -It is the counterpart to the [user guide](../guide/index.md). Each numbered guide page covers one operation; this protocol covers a whole reconstruction in order. +The protocol complements the [user guide](../guide/index.md). Each page of the guide covers one operation, and the protocol covers a whole reconstruction in order. -The protocol predates the dual-language guide and its steps are MATLAB. The documentation build does not run its commands, unlike the numbered guide pages; several steps take hours and need data that is not shipped with this site. +The guide covers MATLAB and Python, and the protocol steps use MATLAB. The documentation build does not run its commands, unlike the guide pages. Several steps take hours and need data that this site does not ship. ## Protocol steps | Page | Content | |---|---| -| [Introduction](reconstruction.md) | what is being built, and from what | +| [Introduction](reconstruction.md) | what is built, and from what | | [Materials and installation](materials.md) | software, databases and input files | -| [Importing template models](template-models.md) | the *S. cerevisiae* and *R. toruloides* models the draft is built from | -| [Draft from homology](homology.md) | BLAST and `getModelFromHomology`; see also [Reconstruction from homology](../guide/homology.md) | +| [Importing template models](template-models.md) | the *S. cerevisiae* and *R. toruloides* models that the draft is built from | +| [Draft from homology](homology.md) | BLAST and `getModelFromHomology` ([Reconstruction from homology](../guide/homology.md)) | | [Biomass composition](biomass.md) | measuring and assembling the biomass equation | -| [Curation of lipid reactions](lipid-curation.md) | SLIME formalism for the lipid reactions | -| [Gap-filling](gap-filling.md) | making the draft able to produce biomass; see also [Gap-filling](../guide/gap-filling.md) | -| [Saving to GitHub and simulating](simulation.md) | writing the model out and checking it grows | -| [Manual curation](manual-curation.md) | fixing gene associations and adding methanol metabolism | +| [Curation of lipid reactions](lipid-curation.md) | the SLIME formalism for the lipid reactions | +| [Gap-filling](gap-filling.md) | making the draft able to produce biomass ([Gap-filling](../guide/gap-filling.md)) | +| [Saving to GitHub and simulating](simulation.md) | writing the model and checking that it grows | +| [Manual curation](manual-curation.md) | correcting gene associations and adding methanol metabolism | ## Related -- [Legacy tutorials](../tutorials/index.md), five shorter exercises from the original RAVEN paper. -- [Reconstruction from homology](../guide/homology.md), the same technique at small scale, in both languages, with every example executed. +- [Legacy tutorials](../tutorials/index.md): five shorter exercises from the original RAVEN paper. +- [Reconstruction from homology](../guide/homology.md): the same technique at small scale, in both languages, with every example executed. ```{toctree} :hidden: diff --git a/docs/protocol/lipid-curation.md b/docs/protocol/lipid-curation.md index fed5030..e9ef0b5 100644 --- a/docs/protocol/lipid-curation.md +++ b/docs/protocol/lipid-curation.md @@ -1,16 +1,16 @@ # Curation of lipid reactions -After adjusting the biomass composition, it is useful to curate reactions already known to differ in the target organism. Using the **SLIME** formalism (Split Lipids Into Measurable Entities) to describe lipid metabolism requires curating the lipid reactions to match the lipids in the biomass reaction. +After the biomass composition is adjusted, the reactions that are known to differ in the target organism are curated. The SLIME formalism (Split Lipids Into Measurable Entities) describes lipid metabolism, and it requires curated lipid reactions that match the lipids in the biomass reaction. :::{admonition} Order of curation and gap-filling :class: note -This step can also be done after gap-filling, but the curation may influence the gap-filling result, so it is done first here. +Curation can also follow gap-filling. Curated lipid reactions change the gap-filling result, so the protocol curates first. ::: ## The SLIME formalism -SLIME captures the flexibility of lipid metabolism while allowing the user to incorporate measurements of lipid classes and acyl chains. Each lipid species in the biomass composition is split into pseudometabolites for its backbone and acyl chains, which are then gathered into the measured entities: +SLIME captures the flexibility of lipid metabolism and accepts measurements of lipid classes and of acyl chains. Each lipid species in the biomass composition is split into pseudometabolites for its backbone and its acyl chains, and these are gathered into the measured entities: ```text phosphatidylcholine (1–16:0, 2–18:0) → phosphatidylcholine backbone @@ -22,23 +22,23 @@ lipid backbone + lipid chain → lipid ## Applying the template reactions -Template reactions are provided in the repository's `data/reconstruction/` folder. The protocol reads these templates and applies them with `addLipidReactions` and `addSLIMEreactions`. In outline, the calls add the lipid and transport reactions and the SLIME split reactions: +The `data/reconstruction/` folder of the repository provides template reactions. The protocol reads the templates and applies them with `addLipidReactions` and `addSLIMEreactions`. In outline, the calls add the lipid and transport reactions and the SLIME split reactions: ```matlab model = addLipidReactions(template, model, modelSce); % lipid + transport reactions model = addSLIMEreactions(template, model, modelSce); % SLIME split reactions ``` -In the full script, three template files are read (`lipidTemplates.txt`, `lipidTransport.txt`, `SLIMERtemplates.txt`), reorganised into a `template` structure, and any pre-existing matching or `SLIME rxn` reactions are removed first so the curated set replaces them. To reduce complexity, only a subset of possible acyl-chain distributions is included. +The full script reads three template files (`lipidTemplates.txt`, `lipidTransport.txt` and `SLIMERtemplates.txt`) and reorganises them into a `template` structure. It first removes the pre-existing matching reactions and `SLIME rxn` reactions, and the curated set replaces them. A subset of the possible acyl-chain distributions is included, to reduce complexity. ## Scaling the acyl chains -After gap-filling (next section) it becomes apparent that the provided acyl-chain and lipid-class levels are not fully compatible; one of the two must be scaled for a functional model. Assuming the lipid-class measurements are technically more reliable, the acyl-chain data is scaled: +After gap-filling, the provided levels of acyl chains and lipid classes turn out to be incompatible, and one of the two must be scaled for a functional model. The protocol assumes that the lipid-class measurements are technically more reliable. Scale the acyl-chain data: ```matlab model = scaleLipids(model, 'tails'); ``` -After scaling, the model has the lipid reactions needed to support a flux toward lipid biosynthesis. +After the scaling, the model has the lipid reactions that support a flux towards lipid biosynthesis. -Next: [Gap-filling](gap-filling.md). +The next page is [Gap-filling](gap-filling.md). diff --git a/docs/protocol/manual-curation.md b/docs/protocol/manual-curation.md index 8dc8c0a..8e2f7e8 100644 --- a/docs/protocol/manual-curation.md +++ b/docs/protocol/manual-curation.md @@ -1,10 +1,10 @@ # Manual curation -A draft GEM always requires manual curation to become a high-quality model. Curation can mean correcting gene associations, adding or removing reactions, and moving reactions between compartments, all aiming for a more accurate description of the in vivo network. (For comparison, the *S. cerevisiae* model has been continuously curated since 2003.) +A draft GEM needs manual curation to become a high-quality model. Curation corrects gene associations, adds or removes reactions, and moves reactions between compartments, all to describe the in vivo network more accurately. For comparison, the *S. cerevisiae* model is curated continuously. ## Replacing template genes with target homologs -Gap-filling adds reactions from the templates without searching for homology of the associated genes, so the draft contains *S. cerevisiae* and *R. toruloides* genes that should be replaced by *H. polymorpha* homologs. List the reactions that carry an *R. toruloides* gene: +Gap-filling adds reactions from the templates without a homology search for the associated genes, so the draft contains *S. cerevisiae* and *R. toruloides* genes that should be replaced by *H. polymorpha* homologs. List the reactions that carry an *R. toruloides* gene: ```matlab rhtoRxns = find(contains(model.grRules, 'RHTO')); @@ -12,7 +12,7 @@ model.rxnNames(rhtoRxns); model.grRules(rhtoRxns); ``` -By literature search and BLAST, gene `RHTO_03911` (oleoyl-CoA desaturase, reaction `y300009`) was found to have the homolog `Hanpo2_15704`. Curate the gene association: +BLAST identified that gene `RHTO_03911` (oleoyl-CoA desaturase, reaction `y300009`) has the homolog `Hanpo2_15704`. Curate the gene association: ```matlab model = changeGrRules(model, 'y300009', 'Hanpo2_15704', true); @@ -21,12 +21,12 @@ model = deleteUnusedGenes(model); ## Adding methanol assimilation -*H. polymorpha* is a methylotroph able to assimilate methanol, but the draft lacks the required pathway. Five reactions are needed; reactions 3–5 (formate dehydrogenase, peroxisomal catalase, dihydroxyacetone kinase) are already present, so two are added manually: +*H. polymorpha* is a methylotroph and assimilates methanol, and the draft lacks the required pathway. The pathway has five reactions. Reactions 3 to 5 (formate dehydrogenase, peroxisomal catalase and dihydroxyacetone kinase) are present already, so two reactions are added manually: 1. methanol oxidase: `methanol[p] + oxygen[p] => formaldehyde[p] + hydrogen peroxide[p]` 2. dihydroxyacetone synthase: `formaldehyde[p] + D-xylulose 5-phosphate[p] => glyceraldehyde 3-phosphate[p] + glycerone[p]` -A `rxnsToAdd` structure collects all the information for the new reactions, which are then added with `addRxns`: +A `rxnsToAdd` structure collects the information of the new reactions, and `addRxns` adds them: ```matlab rxnsToAdd.rxns = {'MOX', 'DAS'}; @@ -42,7 +42,7 @@ rxnsToAdd.rxnNotes = {'Methanol metabolism reaction added by manual curation', model = addRxns(model, rxnsToAdd, 3, '', true, true); ``` -A few transport and exchange reactions complete the pathway: methanol (and glucose) exchange, peroxisomal diffusion of O₂ and H₂O, and transport of the intermediates between cytoplasm and peroxisome: +A few transport and exchange reactions complete the pathway: the exchange of methanol (and glucose), the peroxisomal diffusion of O₂ and H₂O, and the transport of the intermediates between cytoplasm and peroxisome: ```matlab model = addRxnsGenesMets(model, modelSce, {'r_4494', 'r_4391', 'r_1714', 'r_1980', 'r_2098'}); @@ -52,7 +52,7 @@ model = addTransport(model, 'c', 'p', {'D-xylulose 5-phosphate' 'methanol' ... ## Confirming growth on methanol -Block other carbon sources, allow methanol uptake, and verify growth: +Block the other carbon sources, allow methanol uptake and verify growth: ```matlab model = setParam(model, 'eq', {'r_1808', 'r_1714'}, 0); % no glycerol/glucose @@ -65,4 +65,4 @@ newCommit(model); After these additions the draft model is methylotrophic, in agreement with the literature. -The full runnable script for the whole protocol is [`code/reconstructionProtocol.m`](https://github.com/SysBioChalmers/hanpo-GEM/blob/main/code/reconstructionProtocol.m). +[`code/reconstructionProtocol.m`](https://github.com/SysBioChalmers/hanpo-GEM/blob/main/code/reconstructionProtocol.m) is the complete runnable script of the protocol. diff --git a/docs/protocol/materials.md b/docs/protocol/materials.md index 38e81e9..5240098 100644 --- a/docs/protocol/materials.md +++ b/docs/protocol/materials.md @@ -2,7 +2,7 @@ ## Software -This protocol runs in MATLAB (R2013b or later; no extra MathWorks toolboxes required), together with software for reconstructing and analysing GEMs. Most of it is open source or available under an academic licence. +The protocol runs in MATLAB, together with software to reconstruct and analyse GEMs. The published protocol states MATLAB R2013b or later, and [RAVEN (MATLAB)](../installation/raven.md) describes the requirements of RAVEN 3. No extra MathWorks toolboxes are required. Most of the software is open source or available under an academic licence. | Software | Dependency | Source | |---|---|---| @@ -12,15 +12,15 @@ This protocol runs in MATLAB (R2013b or later; no extra MathWorks toolboxes requ | Gurobi | Licence | | | libSBML | MATLAB | | -- RAVEN is the toolbox that drives the reconstruction. -- libSBML enables import and export of the community-standard SBML (`.xml`) format. -- A linear-programming solver is needed for simulation. This protocol uses Gurobi (free academic licence); RAVEN can alternatively use the open-source GLPK solver bundled with COBRA. +- RAVEN drives the reconstruction. +- libSBML imports and exports the community-standard SBML (`.xml`) format. +- Simulation needs a linear-programming solver. The protocol uses Gurobi (free academic licence). RAVEN also runs on the open-source GLPK solver that it bundles. -See [Installation](../installation/raven.md) for the full setup. +[Installation](../installation/raven.md) describes the complete setup. ## Files -Homology-based reconstruction requires files for both the target organism and the template organisms. +Homology-based reconstruction needs files for the target organism and for the template organisms. | File type | Usage | Organism | Source | |---|---|---|---| @@ -32,34 +32,34 @@ Homology-based reconstruction requires files for both the target organism and th | DNA FASTA | Biomass curation | *H. polymorpha* | NCBI `GCF_001664045.1_Hanpo2` | | GenBank | Biomass curation | *H. polymorpha* | NCBI `GCF_001664045.1_Hanpo2` | -The target organism requires (1) a protein FASTA of all proteins in its genome, (2) a DNA FASTA for nucleotide ratios, and (3) a GenBank file for ribonucleotide ratios. Each template requires a protein FASTA and a model file in SBML format. +The target organism needs a protein FASTA file of all proteins in its genome, a DNA FASTA file for the nucleotide ratios and a GenBank file for the ribonucleotide ratios. Each template needs a protein FASTA file and a model file in SBML format. :::{admonition} Matching identifiers :class: important -For automatic gene matching, the protein FASTA and the model file of each template must use the same gene identifiers. +For automatic gene matching, the protein FASTA file and the model file of each template must use the same gene identifiers. ::: -All required files for *H. polymorpha* are provided in the [hanpo-GEM](https://github.com/SysBioChalmers/hanpo-GEM) repository; clone it to get a local copy. +The [hanpo-GEM](https://github.com/SysBioChalmers/hanpo-GEM) repository provides all required files for *H. polymorpha*, and a clone of it gives a local copy. ## Literature data -Model quality improves by using any known metabolic behaviour of the target organism: macromolecule fractions of biomass, essential and non-essential genes, usable nutrient sources, and growth rate. A short literature search yields total protein, lipid and carbohydrate content for *H. polymorpha*, used to set the biomass stoichiometric coefficients ([Biomass composition](biomass.md)). Where organism-specific data is missing, values from related organisms can be used. +Known metabolic behaviour of the target organism improves the quality of the model: the macromolecule fractions of the biomass, essential and non-essential genes, usable nutrient sources and the growth rate. A short literature search gives the total protein, lipid and carbohydrate content for *H. polymorpha*, which sets the stoichiometric coefficients of the biomass ([Biomass composition](biomass.md)). Values from related organisms can replace missing organism-specific data. ## Installing RAVEN -After obtaining the software and files, install RAVEN (see [Installing RAVEN](../installation/raven.md)). Use `pathtool` to add the RAVEN, libSBML and Gurobi subfolders to the MATLAB path, then verify with: +After obtaining the software and files, install RAVEN ([Installing RAVEN](../installation/raven.md)). `pathtool` adds the RAVEN folder to the MATLAB path, and RAVEN bundles libSBML. Gurobi is added to the path by its own MATLAB integration ([RAVEN (MATLAB)](../installation/raven.md#choosing-and-configuring-a-solver)), and verify the installation with: ```matlab checkRaven ``` -See [Installing RAVEN](#verify) for what a successful run looks like. +[Installing RAVEN](../installation/raven.md#verification) shows the output of a successful run. :::{admonition} Troubleshooting :class: warning - Problem: `checkRaven` reports Fail for Export Excel format. -- Possible reason: The protocol script reports that MATLAB's Text Analytics Toolbox can conflict with reading Excel files. -- Solution: uninstall the Text Analytics Toolbox. For support, see the [RAVEN issues](https://github.com/CompMetEng/RAVEN/issues) page. +- Possible reason: The Text Analytics Toolbox of MATLAB can conflict with Excel files. +- Solution: Uninstall the Text Analytics Toolbox. Support is available through the [RAVEN issues](https://github.com/CompMetEng/RAVEN/issues) page. ::: diff --git a/docs/protocol/reconstruction.md b/docs/protocol/reconstruction.md index 78f8e76..89e4a72 100644 --- a/docs/protocol/reconstruction.md +++ b/docs/protocol/reconstruction.md @@ -1,45 +1,45 @@ # GEM reconstruction protocol -This protocol is a complete, worked example of reconstructing a genome-scale metabolic model (GEM) from homology, using RAVEN. It follows the published protocol: +The protocol is a complete, worked example of the reconstruction of a genome-scale metabolic model (GEM) from homology with RAVEN. It follows the published protocol: > Zorrilla F, Kerkhoven EJ (2022). *Reconstruction of Genome-Scale Metabolic Model for* Hansenula polymorpha *Using RAVEN.* In: *Yeast Metabolic Engineering: Methods and Protocols*, Methods in Molecular Biology vol. 2513, pp. 271–290. -All scripts and data are hosted in the [hanpo-GEM](https://github.com/SysBioChalmers/hanpo-GEM) repository, where `hanpo-GEM` is a short name for the model. The complete, runnable version of every command on these pages is in [`code/reconstructionProtocol.m`](https://github.com/SysBioChalmers/hanpo-GEM/blob/main/code/reconstructionProtocol.m); sections 3.1 to 3.9 of the script correspond to the pages in the order of the stages below, with sections 3.7 and 3.8 on the page Saving to GitHub and simulating. The commands shown in these pages are occasionally more concise than in the script (mainly around file locations). +All scripts and data are in the [hanpo-GEM](https://github.com/SysBioChalmers/hanpo-GEM) repository, where `hanpo-GEM` is the short name of the model. [`code/reconstructionProtocol.m`](https://github.com/SysBioChalmers/hanpo-GEM/blob/main/code/reconstructionProtocol.m) holds the complete, runnable version of every command on these pages. Its sections 3.1 to 3.9 correspond to the pages in the order of the stages below, and sections 3.7 and 3.8 correspond to the page Saving to GitHub and simulating. The commands on these pages are in places more concise than in the script, mainly in the file locations. -The original, published protocol was written for RAVEN 2. This site applies minor modifications to the commands shown so the same protocol also works with RAVEN 3. +The published protocol was written for RAVEN 2. The commands on these pages differ in minor details from the published version, to match RAVEN 3. The documentation build does not run them. ## Genome-scale metabolic models -Cellular metabolism is a complex network of hundreds to thousands of reactions and metabolites. A GEM is a mathematical description of that network. At its core is the **stoichiometric matrix** (*S*-matrix) of dimensions *M* × *N*, where *M* is the number of metabolites and *N* the number of reactions. Each reaction is bounded by a lower (LB) and upper (UB) bound reflecting thermodynamics and reversibility, and gene–reaction associations are stored in `grRules`. +Cellular metabolism is a network of hundreds to thousands of reactions and metabolites, and a GEM is a mathematical description of that network. Its core is the stoichiometric matrix (*S*-matrix) of dimensions *M* × *N*, where *M* is the number of metabolites and *N* the number of reactions. Each reaction has a lower (LB) and an upper (UB) bound that reflect thermodynamics and reversibility, and the gene–reaction associations are stored in `grRules`. -Constraining the model with measurable rates (such as carbon uptake) allows estimation of all intracellular fluxes by **flux balance analysis (FBA)**, and so prediction of phenotypes. This supports applications from strain engineering to antibiotic discovery and microbial-community analysis. +Measurable rates, such as the carbon uptake, constrain the model. Flux balance analysis (FBA) then estimates all intracellular fluxes and predicts phenotypes. This supports applications from strain engineering to antibiotic discovery and the analysis of microbial communities. ## The homology-based approach -RAVEN can create a draft GEM for an organism of interest by using well-curated models of phylogenetically related organisms as templates. Whether a given reaction from a template is included in the new model is decided by homology between the protein sequences of the two organisms (determined by bidirectional BLAST). +RAVEN creates a draft GEM for an organism with well-curated models of phylogenetically related organisms as templates. The homology between the protein sequences of the two organisms, determined by bidirectional BLAST, decides whether a reaction of a template is included in the new model. -In this protocol a draft model is reconstructed for the industrially relevant methylotrophic yeast *Hansenula polymorpha* (also known as *Ogataea polymorpha*), which lacks a published high-quality GEM. Two highly curated yeast models are used as templates: +The protocol reconstructs a draft model for the industrially relevant methylotrophic yeast *Hansenula polymorpha* (also named *Ogataea polymorpha*), which has no published high-quality GEM. Two highly curated yeast models are the templates: | Template | Organism | Role | |---|---|---| -| [yeast-GEM](https://github.com/SysBioChalmers/yeast-GEM) | *Saccharomyces cerevisiae* (ascomycete) | Primary template | -| [rhto-GEM](https://github.com/SysBioChalmers/rhto-GEM) | *Rhodotorula toruloides* (basidiomycete) | Complementary template | +| [yeast-GEM](https://github.com/SysBioChalmers/yeast-GEM) | *Saccharomyces cerevisiae* (ascomycete) | primary template | +| [rhto-GEM](https://github.com/SysBioChalmers/rhto-GEM) | *Rhodotorula toruloides* (basidiomycete) | complementary template | :::{admonition} Same identifier namespace :class: warning -Multiple template models must use the same identifiers for metabolites, reactions and compartments. The yeast-GEM and rhto-GEM models used here share the same identifier style. +Multiple template models must use the same identifiers for metabolites, reactions and compartments. The yeast-GEM and rhto-GEM models used here share the identifier style. ::: ## Reconstruction stages -The reconstruction proceeds through these stages, each a page of the protocol: +Each stage is a page of the protocol: -1. [Materials and installation](materials.md), software, files and verifying RAVEN. -2. [Importing template models](template-models.md), load the yeast-GEM and rhto-GEM templates. -3. [Draft from homology](homology.md), BLAST and `getModelFromHomology`. -4. [Biomass composition](biomass.md), DNA, RNA, protein, carbohydrate and lipid pseudoreactions. -5. [Curation of lipid reactions](lipid-curation.md), the SLIME formalism. -6. [Gap-filling](gap-filling.md), make the draft able to produce biomass. -7. [Saving to GitHub and simulating](simulation.md), version control and FBA. -8. [Manual curation](manual-curation.md), fix gene associations, add methanol metabolism. +1. [Materials and installation](materials.md): software, files and verification of RAVEN. +2. [Importing template models](template-models.md): loading the yeast-GEM and rhto-GEM templates. +3. [Draft from homology](homology.md): BLAST and `getModelFromHomology`. +4. [Biomass composition](biomass.md): the pseudoreactions for DNA, RNA, protein, carbohydrate and lipid. +5. [Curation of lipid reactions](lipid-curation.md): the SLIME formalism. +6. [Gap-filling](gap-filling.md): making the draft able to produce biomass. +7. [Saving to GitHub and simulating](simulation.md): version control and FBA. +8. [Manual curation](manual-curation.md): correcting gene associations and adding methanol metabolism. diff --git a/docs/protocol/simulation.md b/docs/protocol/simulation.md index 94a4945..a9818ce 100644 --- a/docs/protocol/simulation.md +++ b/docs/protocol/simulation.md @@ -2,7 +2,7 @@ ## Saving to GitHub -The model is tracked and distributed through a GitHub repository. Before saving, add some metadata to the model structure: +The model is tracked and distributed through a GitHub repository. Add metadata to the model structure before saving it: ```matlab model.annotation.defaultLB = -1000; @@ -16,25 +16,25 @@ model.id = 'hanpo'; model.name = 'Hansenula polymorpha-GEM'; ``` -Every significant curation should be committed to the repository. To standardise this, hanpo-GEM provides helper functions: +Every significant curation should be committed to the repository. hanpo-GEM provides helper functions to standardise this: ```matlab newCommit(model); ``` -`newCommit` automatically generates the XML, YAML and text files that help identify differences between model versions; run it whenever a curation is committed. After several curations, the current state can be designated as a numbered release: +`newCommit` generates the XML, YAML and text files that show the differences between model versions, and it is run whenever a curation is committed. After several curations, the current state is designated as a numbered release: ```matlab newRelease(model); ``` -`newRelease` also generates MAT and XLSX files and records the changes between versions in the history file. Commits and releases become public only once pushed to the online repository. +`newRelease` also generates MAT and XLSX files and records the changes between versions in the history file. Commits and releases become public when they are pushed to the online repository. -hanpo-GEM follows a standardised layout: `model/` holds the latest model in several formats, while `code/` and `data/` contain everything needed to regenerate the model and run analyses. +hanpo-GEM follows a standardised layout. `model/` holds the latest model in several formats, and `code/` and `data/` contain everything needed to regenerate the model and to run analyses. ## Performing simulations -After gap-filling, the draft model can simulate biomass production. Set biomass formation as the objective, run FBA with `solveLP`, and inspect the active fluxes with `printFluxes`: +After gap-filling, the draft model can simulate biomass production. Set biomass formation as the objective, run FBA with `solveLP` and show the active fluxes with `printFluxes`: ```matlab model = setParam(model, 'obj', 'r_4041', 1); @@ -42,6 +42,6 @@ sol = solveLP(model, 1); printFluxes(model, sol.x, false); ``` -When available, constrain the model with measured uptake or excretion rates by setting the bounds of the relevant exchange reactions. This allows evaluation of the model against experimental data from the literature and identification of where further development is needed. +Constrain the model with measured uptake or excretion rates when they are available, by setting the bounds of the relevant exchange reactions. The model can then be evaluated against experimental data from the literature, and the result shows where further development is needed. -Next: [Manual curation](manual-curation.md). +The next page is [Manual curation](manual-curation.md). diff --git a/docs/protocol/template-models.md b/docs/protocol/template-models.md index 75eb451..0349ee9 100644 --- a/docs/protocol/template-models.md +++ b/docs/protocol/template-models.md @@ -1,6 +1,6 @@ # Importing template models -RAVEN creates a draft GEM by using models of phylogenetically related organisms as templates. For *H. polymorpha*, the *S. cerevisiae* model (yeast-GEM) is the primary template and the *R. toruloides* model (rhto-GEM) the complementary one. +RAVEN creates a draft GEM with models of phylogenetically related organisms as templates. For *H. polymorpha*, the *S. cerevisiae* model (yeast-GEM) is the primary template and the *R. toruloides* model (rhto-GEM) is the complementary template. ## Loading the primary template @@ -10,7 +10,7 @@ RAVEN creates a draft GEM by using models of phylogenetically related organisms modelSce = importModel('yeastGEM.xml'); ``` -yeast-GEM is written by the COBRA Toolbox, which appends the compartment to each metabolite ID. RAVEN keeps the compartment in a separate field (`.metComps`), so the redundant suffix is removed, and the model is given the id `sce` (used later by `getModelFromHomology` to match the model with its protein FASTA): +yeast-GEM was exported with the compartment appended to each metabolite identifier. RAVEN keeps the compartment in a separate field (`.metComps`), so the redundant suffix is removed. The model receives the identifier `sce`, which `getModelFromHomology` later uses to match the model with its protein FASTA file: ```matlab modelSce.mets = regexprep(modelSce.mets, '\[[a-z]+\]$', ''); @@ -19,15 +19,15 @@ modelSce.id = 'sce'; ## Inspecting the model -A model is often easier to browse as a spreadsheet. `exportToExcelFormat` writes an `.xlsx` with five sheets: `RXNS`, `METS`, `COMPS`, `GENES` and `MODEL`: +A spreadsheet is often easier for browsing a model. `exportToExcelFormat` writes an `.xlsx` file with five sheets: `RXNS`, `METS`, `COMPS`, `GENES` and `MODEL`: ```matlab exportToExcelFormat(modelSce, 'modelSce.xlsx'); ``` -Reconstruction often writes files that are not meant to be tracked in git. The hanpo-GEM repository keeps these in a `scrap/` folder that is excluded from version control. +Reconstruction often writes files that are not meant to be tracked in git. The hanpo-GEM repository keeps them in a `scrap/` folder that is excluded from version control. -Make changes to the model in MATLAB, not in the Excel file, and export to SBML when desired: +Make changes in MATLAB, and export the model to SBML when needed: ```matlab exportModel(modelSce, 'modelSce.xml'); @@ -35,18 +35,18 @@ exportModel(modelSce, 'modelSce.xml'); ## Loading the complementary template -The *R. toruloides* model is loaded the same way. It was written by RAVEN, so it has no redundant compartment suffix to clean up: +The *R. toruloides* model is loaded in the same way. RAVEN wrote it, so it has no redundant compartment suffix to remove: ```matlab modelRhto = importModel('rhto.xml', true); modelRhto.id = 'rhto'; ``` -It can be useful to store intermediate states of the MATLAB workspace to resume later: +Store intermediate states of the MATLAB workspace to resume later: ```matlab save('scrap/importModels.mat') % load('scrap/importModels.mat') ``` -Next: [Draft from homology](homology.md). +The next page is [Draft from homology](homology.md). diff --git a/docs/tuned-parameters.md b/docs/tuned-parameters.md index 29458e5..0ea2f96 100644 --- a/docs/tuned-parameters.md +++ b/docs/tuned-parameters.md @@ -1,27 +1,27 @@ # Tuned parameter defaults -Every RAVEN parameter with a non-trivial default has been through one of three processes: matched to a literature value from the original paper of the method, measured empirically on real or synthetic models, or kept for consistency with an existing convention (the default of a widely used underlying tool, or a value already established elsewhere in RAVEN). The short version (current default and a one-line reason) is grouped by capability and covers both the MATLAB and Python (raven-toolbox) implementations. The linked studies give the full measurements and methodology. +Each default listed here has one of three origins: a literature value from the original paper of the method, a measurement on real or synthetic models, or a convention taken over from a widely used tool or from another part of RAVEN. The tables are grouped by capability and cover the MATLAB implementation (RAVEN) and the Python implementation (raven-toolbox). Each row gives the default and the basis for it, and the linked studies give the measurements and methods. :::{admonition} Differences between the two implementations :class: note -The MATLAB implementation of RAVEN dates from 2013 and the Python implementation (raven-toolbox) from 2026. The parameters re-measured in one implementation have been ported to the other (the homology and KEGG cut-offs). Un-measured defaults of both implementations were not assumed correct; where the two disagreed, the default was measured, with no deference to whichever implementation came first. Every row states the current value(s); where the two differ, both are given. [Cross-toolbox parity decisions](#cross-toolbox-parity-decisions) gives the full accounting, including the cases that are deliberately kept different because the two implementations rest on different solvers or algorithms. +The MATLAB implementation dates from 2013 and the Python implementation from 2026. The homology and KEGG cut-offs were measured and then set to the same values in both. Defaults that were not measured were not assumed correct, and where the two implementations disagreed, the default was measured. A row gives both values where they differ. [Cross-toolbox parity decisions](#cross-toolbox-parity-decisions) lists the values that are equal and the values that differ on purpose because the implementations use different solvers or algorithms. ::: ## Flux sampling -RAVEN MATLAB: `randomSampling` (`method='achr'|'chrr'|'random_objective'`), `sampleACHR`, `sampleCHRR`, `sampleMaxVolEllipse`, `sampleChebyshevCenter`. raven-toolbox: `random_sampling`, `find_good_reactions`, `max_volume_ellipsoid`. Both implementations expose all three sampling methods through one entry point. +RAVEN MATLAB: `randomSampling` (`method='achr'|'chrr'|'randomObjective'`), `sampleACHR`, `sampleCHRR`, `sampleMaxVolEllipse`, `sampleChebyshevCenter`. raven-toolbox: `random_sampling`, `find_good_reactions`, `max_volume_ellipsoid`. Both implementations expose all three sampling methods through one entry point. | Parameter | Default | How determined | |---|---|---| -| `method` | `'achr'` (Python); `'achr'` (RAVEN 3, `randomSampling`) | Literature: ACHR gives near-uniform interior sampling, and the random-objective method draws polytope vertices. All three methods are selectable in both implementations (`'randomObjective'` in MATLAB, `'random_objective'` in Python). RAVEN 2.11.3 had only the random-objective method. | +| `method` | `'achr'` (Python); `'achr'` (RAVEN 3, `randomSampling`) | Literature: ACHR gives near-uniform interior sampling, and the random-objective method draws polytope vertices. All three methods are selectable in both implementations (`'randomObjective'` in MATLAB, `'random_objective'` in Python). RAVEN 2.11.3 had the random-objective method alone. | | `thinning` | `100` | Upstream convention (cobrapy's own ACHR default). Measured insufficient at genome scale: on yeast-GEM one ACHR chain (seed 0) had an effective sample size (ESS) of about 10 from 300 stored samples (9.9 to 63.1 over four seeds), and 4 chains disagreed (median Gelman-Rubin R-hat 1.1020, 50.7 % of scored reactions above 1.1). At `thinning=500` the lag-1 autocorrelation of one chain was 0.825 for seed 0 (0.644 to 0.825 over four seeds, ESS 28.8 to 64.9). Agreement between chains was tested at one other setting, `thinning=300` with `n_samples=100` (the same 30,000 steps per chain), and did not improve. See [Flux sampling convergence and settings](parameter-tuning/studies/sampling-convergence-calibration.md). | -| `warmup` | `1000` | Used by `method='chrr'` only (the steps discarded before the first stored sample); the ACHR sampler of cobrapy has no such parameter. Same as `nBurnin` of RAVEN's `sampleCHRR` and `randomSampling`, which also applies to CHRR only. Not measured. | +| `warmup` | `1000` | Used by `method='chrr'` (the steps discarded before the first stored sample); the ACHR sampler of cobrapy has no such parameter. Same as `nBurnin` of RAVEN's `sampleCHRR` and `randomSampling`, which also applies to CHRR. Not measured. | | `n_objectives` | `2` | Literature: [Bordel et al. 2010](https://doi.org/10.1371/journal.pcbi.1000859). | | `replace_max_bound` | `False` | Used by `method='random_objective'`. With `True` the run fails with an unbounded solver status: one probe of 50 objectives gave 49 optimal results and 1 unbounded result. From the cobra code, one unbounded objective stops all draws; the sample at which the run aborted was not recorded. With `False`, 200 samples completed. Kept different from MATLAB on purpose. Whether MATLAB's own solver path handles `True` was not tested. | -| `loopless_good_reactions` | `True` | Loopless FVA excludes only reactions that take part in infeasible cycles, at higher cost; the alternative fixed-threshold (±999) heuristic also excludes reactions that legitimately reach capacity. The loopless technique is implemented in Python only; porting it is a larger project than changing a default. | +| `loopless_good_reactions` | `True` | Loopless FVA excludes reactions that take part in infeasible cycles, at higher cost; the alternative fixed-threshold (±999) heuristic also excludes reactions that legitimately reach capacity. The loopless technique is implemented in Python; porting it is a larger project than changing a default. | | `n_samples`, `seed`, `min_flux`, `max_attempts` | `1000`, `None`, `False`, `100` | Same in both implementations or no evidence favoured a different value. | -| `flux_tol` (in `find_good_reactions`) | `1e-9` | Python-only helper (supports target selection of the random-objective method); no cross-implementation comparison. | +| `flux_tol` (in `find_good_reactions`) | `1e-9` | Python helper (supports target selection of the random-objective method); no cross-implementation comparison. | | `maxiter` (in `max_volume_ellipsoid`) | `150` | Literature: [Zhang & Gao 2003](https://doi.org/10.1137/S1052623401397230). | | `tol`, `reg` (in `max_volume_ellipsoid`) | `1e-6`, `1e-8` | Same in both implementations. | @@ -55,9 +55,8 @@ Only MATLAB has the legacy tINIT method (`runINIT`, `getINITModel`), which ftINI | `series` | `'1+1'` | Literature: [Gustafsson et al. 2023](https://doi.org/10.1073/pnas.2217868120) (the ftINIT paper). | | `force_on` | `0.1` | RAVEN's value; measured near-insensitive across the range 0.02–0.5. | | `big_m` | `1000.0` | The cap on each scored reaction's own big-M, as in `ftINITInternalAlg`: the big-M of a reaction is `max(min(ub, big_m), min(-lb, big_m), 0)`, raised to `force_on` where the "on" floor needs it, and an infinite bound counts as `big_m`. The cap limits the flux a reaction can carry when it is on (it is the largest big-M of a scored reaction in its on/off constraint) and sets no bound on the reaction in the model. | -| `eps` | `1.0` | Same in both implementations. | | `resolve_ties`, `prove_abs_gap` | `False`, `None`, both opt-in; MATLAB has them as `resolveTies` and `proveAbsGap` at the same defaults | `resolve_ties` fixes the answer among equal-score optima in phases: it holds the score objective at its optimum, minimises the count of kept removable (negative-score) reactions, then among those minimises the summed id rank of the kept ones (a merged reaction is ranked by the smallest original id of its group), then takes the fluxes with the least total absolute flux (relative and absolute gap 1e-6), then those with the least id-rank-weighted backward flux. The last ftINIT step skips the last two phases, because no later step reads its fluxes. Measured on Human-GEM v2.0.0-38 (one cell line, seeds 1234 and 7): with the default escalation the seed-to-seed difference is 110 reactions and 17 predicted essential genes. `resolve_ties=True` gives identical reaction sets at a fixed seed and reduces the seed-to-seed difference to 41 reactions and 1 gene, at about 3.6 times the run time (about 14 min against 4 min at `time_limit=1800`). With `prove_abs_gap=1.0` as well, the difference is 1 reaction and 0 genes, at about 26 times the baseline run time (about 100 min); in the four sweep builds with `prove_abs_gap=1.0`, step 1 took 3305 to 3650 s (59.0 to 66.3 % of a build), mostly in its two flux phases, and step 2 another 1650 to 2500 s; no build without `resolve_ties` was run with `prove_abs_gap`. The flux phases (`phase2c-minflux`, `phase2d-direction`) reach the time limit in most builds, and a warning names each phase adopted at the limit. A tie-break that finds no incumbent in phase 2a or 2b is dropped without a warning. The escalation raises no warning either: in the four `resolve_ties=True` runs without `prove_abs_gap` of the `prove_abs_gap` sweep, its step 1 objective ended 4.0 to 12.0 below the proven optimum. A build without a warning is therefore not necessarily proven or tie-broken. | -| `factor`, `max_score`, `min_score` (in `gene_scores_from_expression`) | `5.0`, `10.0`, `-5.0` | RAVEN's own formula. A 2012 paper (Wang et al. 2012, [mCADRE](https://doi.org/10.1186/1752-0509-6-153)) uses categorical scoring and is not the origin; no source in either implementation cites a paper for it. | +| `factor`, `max_score`, `min_score` (in `gene_scores_from_expression`) | `5.0`, `10.0`, `-5.0` | RAVEN's own formula. Wang et al. 2012 ([mCADRE](https://doi.org/10.1186/1752-0509-6-153)) use categorical scoring, and no source in either implementation cites a paper for these constants. | | `score_reactions_from_genes` and `classify_reactions` arguments | various | Same in both implementations or standard practice. | Full detail: [ftINIT parameter calibration and input robustness](parameter-tuning/studies/init-param-calibration.md) (the primary measurement, genome-scale Human-GEM) · [ftINIT on different solvers](parameter-tuning/studies/init-solver-benchmark.md) · [ftINIT reproducibility with `resolve_ties` and `prove_abs_gap`](parameter-tuning/studies/ftinit-determinism.md) (`resolve_ties=True` gives identical reaction sets at a fixed seed and a seed-to-seed difference of 41 reactions and 1 gene; adding `prove_abs_gap=1.0` reduces it to 1 reaction and 0 genes at about 26 times the run time) · [Human-GEM cell-type model validation against RAVEN](parameter-tuning/studies/humangem-validation.md) (Jaccard index against RAVEN across 5 cell lines of 0.964 to 0.982 for models extracted without tasks and 0.981 to 0.988 for task-constrained models) @@ -71,7 +70,7 @@ RAVEN MATLAB: `fillGaps` (connectivity mode), ported as `connect_blocked_reactio | `epsilon` (in `fill_gaps_fast_lp`) | `0.0001` | Literature: [Thiele et al. 2014](https://doi.org/10.1093/bioinformatics/btu321) (the fastGapFill paper). | | `threshold` (in `connect_blocked_reactions`) | `1.0` | The flux a reaction must be able to exceed to count as unblocked. Same as the fixed threshold of 1 in RAVEN's `fillGaps`, and suited to models in the RAVEN convention, where exchange reactions have bounds of ±1000. Measured on one synthetic network with a maximum flux of 0.5, one run per value: 0.01 and 0.1 filled the gap, 0.5 and 1.0 did not ([Benchmarks](parameter-tuning/benchmarks.md)). | | `penalty`, `allow_net_production` (in `connect_blocked_reactions`) | `1.0`, `False` | No measurement recorded. `penalty=1.0` makes the MILP minimise the number of added reactions. `allow_net_production=False` keeps the steady-state balance `S @ v == 0`, the stricter test; `True` relaxes it to `S @ v >= 0`. | -| `big_m` (in `fill_gaps_milp`) | `1000.0` | Matches RAVEN's ±1000 bound convention. It caps an unbounded template reaction only when the merged model has no finite bound; otherwise the largest finite bound does. | +| `big_m` (in `fill_gaps_milp`) | `1000.0` | Matches RAVEN's ±1000 bound convention. It caps an unbounded template reaction when the merged model has no finite bound; otherwise the largest finite bound does. | Full detail: [Benchmarks of `fseof`, `remove_genes` and other defaults](parameter-tuning/benchmarks.md) @@ -81,7 +80,7 @@ RAVEN MATLAB: `getModelFromHomology`, `getBlast`, `getDiamond`. raven-toolbox: ` | Parameter | Default | How determined | |---|---|---| -| `evalue` (`run_blast`) | `1e-4` | Matches MATLAB's `getBlast` (hardcodes `-evalue 10e-5`). On *H. polymorpha* against *S. cerevisiae*, loosening it from `1e-5` to `1e-4` added 5,200 raw hits (8.9 % of the table); 5,310 of the 5,314 rows found only at `1e-4` fail `max_evalue=1e-30` of `get_model_from_homology`; 4 have an E-value below `1e-30`. | +| `evalue` (`run_blast`) | `1e-4` | Matches MATLAB's `getBlast` (hardcodes `-evalue 10e-5`). On *H. polymorpha* against *S. cerevisiae*, loosening it from `1e-5` to `1e-4` added 5,200 raw hits (8.9 % of the table); 5,310 of the 5,314 rows found at `1e-4` alone fail `max_evalue=1e-30` of `get_model_from_homology`; 4 have an E-value below `1e-30`. | | `evalue` (`run_diamond`) | `1e-3` | Matches MATLAB's `getDiamond`, which passes no `-evalue` flag and so inherits DIAMOND's own default. The value is deliberately different from `run_blast`; each aligner follows its own MATLAB call. | | `threads` | every CPU the process may run on (Python); the cores assigned to MATLAB (MATLAB) | A 500-query BLAST subset returned the same 2,746 hits at one thread and at four, in 20.4 s and 5.9 s (3.5 times faster; *H. polymorpha* against *S. cerevisiae*, BLAST+ 2.17.0, 2026-11-01). Both sides use all available cores. | | `max_evalue` | `1e-30` | At identity 40 % and length 200 the score was identical from `1e-4` to `1e-30` in four organisms (BLAST, F0.5, KEGG reference). `1e-50` lowered it by 0.002 to 0.006 in three organisms and left it unchanged in *K. lactis*, and `1e-100` lowered it by 0.020 to 0.063. | @@ -92,6 +91,7 @@ RAVEN MATLAB: `getModelFromHomology`, `getBlast`, `getDiamond`. raven-toolbox: ` Scoring against curated GEMs (hanpo-GEM, rhto-GEM) built by the same reconstruction method is not part of the 2026-11-01 measurement. Full detail: [Homology cut-off calibration](parameter-tuning/studies/homology-cutoff-calibration.md) + ## KEGG-based reconstruction RAVEN MATLAB: `getKEGGModelForOrganism` (the exact internal name of the KO-assignment sub-step has not been independently confirmed). raven-toolbox: `assign_kos`, `run_hmmsearch`, `build_ko_hmm`, and the `get_kegg_model_*` family. @@ -116,9 +116,9 @@ Compartment assignment is in development in Python and is not at parity with MAT | Parameter | Default | How determined | |---|---|---| | `default_compartment` | `'c'` in Python; MATLAB requires the argument every call | Convenience default: cytosol is correct for most reactions. MATLAB has no default, so no values disagree. | -| `transport_cost` | `0.5` | Not independently benchmarked; same in both implementations. Calibration note: for real (non-integer-scale) predictor scores, the value typically needs lowering; see the linked study. | -| `time_limit` (Python) and `maxTime` (MATLAB) | `None` (uncapped) in Python; MATLAB's `maxTime` defaults to `15` minutes | The values are not unified: the MATLAB number is a simulated-annealing search budget (more time generally means a better heuristic answer, without a proof of optimality), and the Python number is a MILP solver cutoff (returns a proven-bounded incumbent). Python's `None` was validated at ~2.5 minutes on yeast-GEM, the primary development-scale model. Whether the MILP needs a cap for harder cases has not been stress-tested. | -| `mip_gap` | `None` | Python only; the MATLAB heuristic has no optimality gap. | +| `transport_cost` | `0.5` | Not independently benchmarked; same in both implementations. For real (non-integer-scale) predictor scores the value needs lowering in the studies that measured it; see the linked study. | +| `time_limit` (Python) and `maxTime` (MATLAB) | `None` (uncapped) in Python; MATLAB's `maxTime` defaults to `15` minutes | The values are not unified: the MATLAB number is a simulated-annealing search budget (more time can improve the heuristic answer, without a proof of optimality), and the Python number is a MILP solver cutoff (returns a proven-bounded incumbent). Python's `None` was validated at ~2.5 minutes on yeast-GEM, the primary development-scale model. Whether the MILP needs a cap for harder cases has not been stress-tested. | +| `mip_gap` | `None` | Python; the MATLAB heuristic has no optimality gap. | Full detail: the measurements behind these values (the yeast-GEM benchmark, the head-to-head against MATLAB's `predictLocalization`, the predictor benchmarks across four organisms, and the design documents) are in [raven-gecko-parity](https://github.com/CompMetEng/raven-gecko-parity/tree/develop/docs/localization), with the rest of the work in progress on this method. @@ -129,7 +129,7 @@ RAVEN MATLAB: `removeGenes`, `mergeModels`, `addRxns`, `addTransport`, `simplify | Parameter | Default | How determined | |---|---|---| | `blocked_reactions` (in `remove_genes`) | `'constrain'` in Python; MATLAB's `removeGenes` with `removeBlockedRxns` false does the same | Measured on e_coli_core: keeping a reaction unchanged after it has lost its last gene gives false-positive growth after an essential gene is removed (it survives with an empty gene rule). `'constrain'` sets the bounds of such a reaction to 0 and keeps the reaction; `'remove'` deletes it and is opt-in, for gene-essentiality or metabolic-engineering use where the reaction is to disappear; `'keep'` leaves it unchanged. | -| `eps` (in `constrain_reversible_reactions`) | `1e-9` | Matches Gurobi's own default primal feasibility tolerance. | +| `eps` (in `constrain_reversible_reactions`) | `1e-10` | The threshold of `constrainReversible` in RAVEN's `simplifyModel`. | | `match_by` (in `merge_models`) | `'name'` (Python) and `'metNames'` (MATLAB) | Same semantic field (metabolite display name) under a different schema field name; no values disagree. | | `remove_orphans`, `mets_by`, `ignore_direction`, `reversible`, `only_to_existing` | various | Python additions and refinements; no direct MATLAB equivalent exists for comparison. | @@ -147,17 +147,17 @@ Full detail: [Benchmarks of `fseof`, `remove_genes` and other defaults](paramete ## Cross-toolbox parity decisions -Policy: where Python and MATLAB RAVEN disagree on a default, one value is chosen for both, except where a real implementation difference forces the divergence (different solver stack, different algorithm, different data schema). In that case identical values would turn correct behaviour into incorrect behaviour. Every row states which side has to change or explains why the values stay split. +Where the Python and MATLAB implementations disagree on a default, one value is used for both. The exception is a difference between the implementations that requires different values (solver stack, algorithm or data schema), where identical values would give incorrect behaviour. The tables state which side changed, or why the values differ. ### Unified values -| Parameter | Python now | MATLAB now | Unify at | Side that changes | Confidence | Evidence | +| Parameter | Python | MATLAB | Common value | Side that changed | Confidence | Evidence | |---|---|---|---|---|---|---| -| `run_blast.evalue` | `1e-4` ✓ done | `1e-4` | `1e-4` | ~~Python~~ done | High | No downstream effect measured either way (`get_model_from_homology`'s `1e-30` dominates); matches `develop`'s independent PR #91 call | -| `get_model_from_homology.min_align_len` | `100` ✓ done | `100` ✓ done (`200` in RAVEN 2.11.3) | `100` | ~~Python and MATLAB~~ done | High | Directly measured: [Homology cut-off calibration](parameter-tuning/studies/homology-cutoff-calibration.md) | -| `assign_kos.cutoff` | `1e-30` ✓ done | `1e-30` ✓ done (`1e-50` in RAVEN 2.11.3) | `1e-30` | ~~MATLAB~~ done | High | Directly measured: [KEGG HMM cut-off calibration](parameter-tuning/studies/kegg-hmm-cutoff-calibration.md) | -| `assign_kos.min_score_ratio_g` | `0.9` ✓ done | `0.9` ✓ done (`0.8` in RAVEN 2.11.3) | `0.9` | ~~MATLAB~~ done | High | Directly measured, same study | -| `run_ftinit.allow_excretion` | `False` ✓ done | `False` | `False` | ~~Python~~ done (2026-06-20) | High | Recorded, without a measurement, as having no effect at the default `prod_weight`; no measurement is recorded on a page of [Parameter tuning](parameter-tuning/index.md). Unified to remove the inconsistency. Within `ftinit`, the step schedule sets the value (`allow_met_secr`) | +| `run_blast.evalue` | `1e-4` | `1e-4` | `1e-4` | Python | High | No downstream effect measured either way (`get_model_from_homology`'s `1e-30` dominates); matches `develop`'s independent PR #91 call | +| `get_model_from_homology.min_align_len` | `100` | `100` (`200` in RAVEN 2.11.3) | `100` | Python and MATLAB | High | Directly measured: [Homology cut-off calibration](parameter-tuning/studies/homology-cutoff-calibration.md) | +| `assign_kos.cutoff` | `1e-30` | `1e-30` (`1e-50` in RAVEN 2.11.3) | `1e-30` | MATLAB | High | Directly measured: [KEGG HMM cut-off calibration](parameter-tuning/studies/kegg-hmm-cutoff-calibration.md) | +| `assign_kos.min_score_ratio_g` | `0.9` | `0.9` (`0.8` in RAVEN 2.11.3) | `0.9` | MATLAB | High | Directly measured, same study | +| `run_ftinit.allow_excretion` | `False` | `False` | `False` | Python | High | Recorded, without a measurement, as having no effect at the default `prod_weight`; no measurement is recorded on a page of [Parameter tuning](parameter-tuning/index.md). The values were unified to remove the inconsistency. Within `ftinit`, the step schedule sets the value (`allow_met_secr`) | ### Settings resolved without unification @@ -166,22 +166,22 @@ The INIT parameters `mip_gap` and `time_limit` of `run_init` and `run_ftinit` ar - `mip_gap`: solve time is flat across the gap in the single first-step solve, so a tight gap is nearly free there. The docstring of `run_ftinit` attributes the flat time to model construction; the study did not measure the share of model construction. A named series such as `1+1` ignores `mip_gap` and `mip_gap_abs`, because every step runs its own schedule of MILP runs, so the full pipeline gives the same result at 0.001, 0.003 and 0.01. Neither 0.001 nor 0.01 is MATLAB's `0.0004`. Keep `None` (Gurobi's own `~1e-4` default is at least as tight as the measured-good `0.001`). - `time_limit`: RAVEN's `ftINIT` limits a MILP run to 5000 seconds when the run sets none, and its schedules use 120 s, 5000 s and 5 s runs. Python's `ftinit` with a named series caps each scheduled run by the same limits, and `time_limit` lowers the cap; `time_limit=None` leaves an ad-hoc step or a `run_ftinit` call uncapped, but no scheduled run. Measured first-step solves took 23 to 30 s, and full pipelines 81 to 119 s with scaling and 157 s without. In the robustness setup, the gap-fill of one 90 %-dropout model ran longer than 75 minutes; the gap-fill has its own fixed limit of 300 s per task and does not read `time_limit`. The study ran its full pipelines at 600 s; its robustness script sets 120 s, and the limit of the recorded robustness runs is not stated. Keep `None` as the code default. The docstring of `run_ftinit` recommends `120-600 s/step`, a range with no measured basis on this site. -[ftINIT on different solvers](parameter-tuning/studies/init-solver-benchmark.md) measured GLPK honouring the 900 s limit: both steps of the full pipeline hit it, the run took 2182 s and kept 6758 reactions (Jaccard 0.892 against Gurobi). The `scipy` interface (HiGHS) fails because it supports continuous variables only. Gurobi is the backend the genome-scale numbers above were measured on. It completed the run within the overall limit; one scheduled solve stopped at a limit (the warning in the `big_m=1000` run). +[ftINIT on different solvers](parameter-tuning/studies/init-solver-benchmark.md) measured GLPK honouring the 900 s limit: both steps of the full pipeline hit it, the run took 2182 s and kept 6758 reactions (Jaccard 0.892 against Gurobi). The `scipy` interface (HiGHS) fails because it supports continuous variables. Gurobi is the backend the genome-scale numbers above were measured on. It completed the run within the overall limit; one scheduled solve stopped at a limit (the warning in the `big_m=1000` run). ### Values kept different | Parameter | Python | MATLAB | Why | |---|---|---|---| | `random_sampling.replace_max_bound` | `False` | `True` | With every bound of 1000 replaced by infinity, the random-objective LP becomes unbounded, because the objective can grow without limit through any reaction that is then unconstrained ([Flux sampling convergence and settings](parameter-tuning/studies/sampling-convergence-calibration.md) gives this mechanism, read from the cobra code; the run with `True` failed with an unbounded status, the sample of the abort was not recorded, and the run with `False` completed 200 samples). Whether MATLAB's own solver path handles `True` was not tested. | -| `random_sampling.loopless_good_reactions` | `True` (loopless FVA) | heuristic (exclude FVA ≥ 999) | The two are different techniques. Loopless FVA excludes only reactions that take part in infeasible cycles, and the MATLAB heuristic also excludes reactions that legitimately reach capacity. Porting the loopless technique to MATLAB is a larger project than changing a default. | +| `random_sampling.loopless_good_reactions` | `True` (loopless FVA) | heuristic (exclude FVA ≥ 0.999 times the loop bound, 999 for a bound of 1000) | The two are different techniques. Loopless FVA excludes reactions that take part in infeasible cycles, and the MATLAB heuristic also excludes reactions that legitimately reach capacity. Porting the loopless technique to MATLAB is a larger project than changing a default. | | `merge_models.match_by` | `'name'` | `'metNames'` | Same semantic field (metabolite display name) under a different field name, which results from the cobrapy and COBRA Toolbox schemas; the field name is not tunable. | | `run_diamond.evalue` | `1e-3` | `1e-3` (implicit) | Both agree at `1e-3`, matching the DIAMOND default. `getDiamond` passes no `-evalue` flag, so it inherits the DIAMOND default and does not follow the explicit `1e-4` of `getBlast`. `run_blast` and `run_diamond` deliberately stay unpaired: each follows the MATLAB call of its own aligner. | -| `predict_localization.time_limit` | `None` | `maxTime=15` (minutes) | Confirmed by reading `localization/predictLocalization.m` (2026-08-28): MATLAB solves the problem with simulated annealing (`maxTime` is its search budget; more time generally improves the heuristic answer, with no optimality guarantee), and Python solves a deterministic MILP (`time_limit` is a solver cutoff; returns a proven-bounded incumbent). The same number has a different role in each, so the values are not unified. A tentative unification at `None` is not adopted, because it assumed that both sides solve the same kind of problem. | +| `predict_localization.time_limit` | `None` | `maxTime=15` (minutes) | `localization/predictLocalization.m` shows that MATLAB solves the problem with simulated annealing (`maxTime` is its search budget; more time can improve the heuristic answer, with no optimality guarantee), and Python solves a deterministic MILP (`time_limit` is a solver cutoff; returns a proven-bounded incumbent). The same number has a different role in each, so the values are not unified. A common value of `None` would assume that both sides solve the same kind of problem, which they do not. | | `predict_localization.default_compartment` | `'c'` | required arg (no default) | MATLAB has no default; Python's `'c'` is a convenience default for the near-universal correct choice and produces no output difference (a MATLAB user must already supply `'c'` explicitly in the common case). Optional, low priority: MATLAB could add the same default. | -## Changes needed in MATLAB RAVEN for parity +## Defaults that differ between RAVEN 2.11.3 and RAVEN 3 -No change is open. The changes to `getKEGGModelForOrganism` (`cutOff`, `minScoreRatioG`) and `getModelFromHomology` (`minLen`) are in RAVEN 3, checked against `src/` on RAVEN `develop`: +The RAVEN 3 defaults of `getKEGGModelForOrganism` (`cutOff`, `minScoreRatioG`) and `getModelFromHomology` (`minLen`) equal the raven-toolbox defaults. The values come from `src/` on RAVEN `develop`: | Function | RAVEN 2.11.3 | RAVEN 3 | |---|---|---| @@ -189,11 +189,11 @@ No change is open. The changes to `getKEGGModelForOrganism` (`cutOff`, `minScore | `getKEGGModelForOrganism`, `minScoreRatioG` | `0.8` | `0.9` | | `getModelFromHomology`, `minLen` | `200` | `100` | -The time-budget parameter of the localisation needs no port: `predictLocalization.m` solves the problem with simulated annealing (`maxTime`, default 15 minutes, a search budget), and raven-toolbox uses a deterministic MILP (`time_limit`, a solver cutoff). See the table of values kept different. +The time-budget parameter of the localisation is not unified: `predictLocalization.m` solves the problem with simulated annealing (`maxTime`, default 15 minutes, a search budget), and raven-toolbox uses a deterministic MILP (`time_limit`, a solver cutoff). See the table of values kept different. ## Open items -- Genome-scale ACHR sampling convergence has no cheap fix. At genome scale (yeast-GEM, default ACHR settings) the median reaction fails the R-hat>1.1 threshold. Neither follow-up fixes it cheaply: reallocating the same step budget onto a larger `thinning` makes no measurable difference, and `method='chrr'` converges on a small model, but its genome-scale cost makes it impractical as a drop-in fix: a probe of 2 chains × 20 samples on yeast-GEM took 136 min (8147.9 s) in total, and the share of the rounding step in that time was not measured. [Flux sampling convergence and settings](parameter-tuning/studies/sampling-convergence-calibration.md) describes the investigation, including a possible future unblock (caching the CHRR rounding transform across calls, an engineering change and not a parameter default, so out of scope here). +- Genome-scale ACHR sampling convergence has no cheap fix. At genome scale (yeast-GEM, default ACHR settings) the median reaction fails the R-hat>1.1 threshold. Neither follow-up fixes it cheaply: reallocating the same step budget onto a larger `thinning` makes no measurable difference, and `method='chrr'` converges on a small model, but its genome-scale cost makes it impractical as a drop-in fix: a probe of 2 chains × 20 samples on yeast-GEM took 136 min (8147.9 s) in total, and the share of the rounding step in that time was not measured. [Flux sampling convergence and settings](parameter-tuning/studies/sampling-convergence-calibration.md) describes the investigation, including a possible future unblock (caching the CHRR rounding transform across calls, an engineering change, so out of scope here). - The `gene_scores_from_expression` scoring constants (`factor=5.0`, `max_score=10.0`, `min_score=-5.0`) have no confirmed literature source on either side; see the INIT section. ## References @@ -227,4 +227,4 @@ The time-budget parameter of the localisation needs no port: `predictLocalizatio > filling in metabolic networks. Bioinformatics 30(17):2529–2531. > -Not listed: the `gene_scores_from_expression` constants (`factor`, `max_score`, `min_score`). Neither the source of RAVEN nor that of raven-toolbox cites a paper for them. A 2012 paper (Wang et al. 2012, [mCADRE](https://doi.org/10.1186/1752-0509-6-153)) uses categorical scoring and is not the origin. The constants are RAVEN's own choice with no confirmed literature source. +Not listed: the `gene_scores_from_expression` constants (`factor`, `max_score`, `min_score`). Neither the source of RAVEN nor that of raven-toolbox cites a paper for them. Wang et al. 2012 ([mCADRE](https://doi.org/10.1186/1752-0509-6-153)) use categorical scoring. The constants are a choice of RAVEN with no confirmed literature source. diff --git a/docs/tutorials/index.md b/docs/tutorials/index.md index f9c8dcb..9cb81b6 100644 --- a/docs/tutorials/index.md +++ b/docs/tutorials/index.md @@ -1,23 +1,23 @@ # Legacy tutorials -Five hands-on tutorials were part of the original RAVEN 1 paper (Agren et al., 2013). The scripts run with the current RAVEN version. The exercises are otherwise unchanged from the original. They use the MATLAB toolbox and build up from running a simulation on an existing model to reconstructing a genome-scale model from sequence data. The scripts and all the data files they use are in the [`tutorials/`](https://github.com/CompMetEng/RAVEN/tree/develop/tutorials) folder of the RAVEN repository. +Five tutorials belonged to the original RAVEN 1 paper (Agren et al., 2013). The scripts run with the current version of RAVEN, and the exercises are otherwise unchanged from the original. They use the MATLAB toolbox and progress from a simulation with an existing model to the reconstruction of a genome-scale model from sequence data. The scripts and their data files are in the [`tutorials/`](https://github.com/CompMetEng/RAVEN/tree/develop/tutorials) folder of the RAVEN repository. | # | Tutorial | Content | |---|---|---| -| 1 | [Importing a GEM and running FBA](tutorial1.md) | Load a model, set constraints and an objective, run FBA, visualise fluxes | -| 2 | [Constructing a functional small model](tutorial2.md) | Build a model from scratch; exchange reactions and the steady-state assumption | -| 3 | [Knockouts and omics data](tutorial3.md) | Gene deletions, and using a GEM as a scaffold for microarray data | -| 4 | [Fixing an erroneous model](tutorial4.md) | Systematic quality control: find and fix mass-balance and naming errors | -| 5 | [Reconstructing a GEM from KEGG](tutorial5.md) | De novo reconstruction from protein sequences using KEGG | +| 1 | [Importing a GEM and running FBA](tutorial1.md) | Load a model, set constraints and an objective, run FBA and visualise fluxes | +| 2 | [Constructing a functional small model](tutorial2.md) | Build a model from scratch, with exchange reactions and the steady-state assumption | +| 3 | [Knockouts and omics data](tutorial3.md) | Gene deletions, and a GEM as a scaffold for microarray data | +| 4 | [Fixing an erroneous model](tutorial4.md) | Quality control: find and fix mass-balance and naming errors | +| 5 | [Reconstructing a GEM from KEGG](tutorial5.md) | De novo reconstruction from protein sequences with KEGG | ## Prerequisites -- The tutorials use the MATLAB toolbox. RAVEN must be installed and `checkRaven` must pass; see [Installation](../installation/raven.md). -- Tutorial 2 loads its model with `importModel` (SBML), and Tutorials 3 and 4 with `readYAMLmodel` (YAML). `curateModelFromTables` curates an existing model from `.tsv` files; see [Excel I/O](../raven3-migration/formats-and-reconstruction.md#excel-io). -- To run a section of a script in MATLAB, highlight it, right-click, and choose "Evaluate selection". -- Tutorials 2, 3 and 4 ship with a `*_solutions.m` companion script containing the completed exercise. +- The tutorials use the MATLAB toolbox. RAVEN is installed and `checkRaven` passes ([Installation](../installation/raven.md)). +- Tutorials 1 and 2 load their model with `importModel` (SBML), and Tutorials 3 and 4 load theirs with `readYAMLmodel` (YAML). `curateModelFromTables` curates an existing model from `.tsv` files ([Excel I/O](../raven3-migration/formats-and-reconstruction.md#excel-io)). +- A section of a script runs in MATLAB when it is highlighted and "Evaluate selection" is chosen from the context menu. +- Tutorials 2, 3 and 4 have a companion script `*_solutions.m` with the completed exercise. -The reconstruction concepts carry over to raven-toolbox. The `snake_case` form of a RAVEN function name matches the raven-toolbox function name for some functions. Some RAVEN functions map to cobrapy (`solveLP` → `model.optimize()`), and some have no Python counterpart (there is no Excel reader). The [API reference](../api/index.md) lists the functions, and [Comparison of RAVEN and raven-toolbox](../raven3-vs-raven-toolbox.md) shows what maps where. +The reconstruction concepts apply to raven-toolbox. The `snake_case` form of a RAVEN function name is the name of the raven-toolbox function for some functions. Some RAVEN functions map to cobrapy (`solveLP` to `model.optimize()`), and some have no Python counterpart (there is no Excel reader). The [API reference](../api/index.md) lists the functions, and [Comparison of RAVEN and raven-toolbox](../raven3-vs-raven-toolbox.md) shows what maps where. ```{toctree} :hidden: diff --git a/docs/tutorials/tutorial1.md b/docs/tutorials/tutorial1.md index 82ee045..ed6af5d 100644 --- a/docs/tutorials/tutorial1.md +++ b/docs/tutorials/tutorial1.md @@ -1,16 +1,16 @@ # Tutorial 1: Import a GEM, set parameters and run FBA -This short introduction shows how to load a genome-scale metabolic model (GEM), set reaction constraints and an objective function, run an optimisation through flux balance analysis (FBA), and visualise the resulting fluxes. +The tutorial loads a genome-scale metabolic model (GEM), sets reaction constraints and an objective function, runs a flux balance analysis (FBA) and visualises the fluxes. -The example uses a GEM for the filamentous fungus *Penicillium chrysogenum* (`iAL1006`), provided as the SBML file `iAL1006 v1.00.xml`. +The example uses the GEM of the filamentous fungus *Penicillium chrysogenum* (`iAL1006`), which is the SBML file `iAL1006 v1.00.xml`. -Importing the model performs structural checks. With this model there is only one warning, that the formula for the metabolite LPE could not be parsed. This is expected and can be ignored. +The import performs structural checks. With this model there is one warning: the formula of the metabolite LPE cannot be parsed. The warning is expected and can be ignored. ## Step by step ### 1. Importing the model -`importModel` imports the model from SBML and performs structural checks (such as for incorrectly written equations or illegal characters). The `false` flag imports the model with exchange reactions in their "closed" form, which is unsuited for modelling and suited to quality-control steps. +`importModel` imports the model from SBML and performs structural checks, for example for incorrectly written equations and illegal characters. The `false` flag imports the exchange reactions in their closed form, which is not suitable for simulation and suits quality-control steps. ```matlab model = importModel('iAL1006 v1.00.xml', false); @@ -21,7 +21,7 @@ The model contains 1632 reactions, 1395 metabolites and 1006 genes. ### 2. Simplifying the model -Mass balancing is done around the internal metabolites. The "unconstrained" (boundary) metabolites that let the system take up or excrete compounds must be removed before simulating. `simplifyModel` is a general-purpose function for making models smaller, for example by grouping linear reactions and deleting reactions that cannot carry flux. The call below deletes the unconstrained metabolites, the reactions constrained to zero flux (mainly uptake of non-standard carbon sources) and the dead-end reactions, and leaves linear reactions ungrouped. +Mass balancing is done around the internal metabolites. Remove the unconstrained (boundary) metabolites that let the system take up or excrete compounds before simulation. `simplifyModel` makes models smaller, for example by grouping linear reactions and by deleting reactions that cannot carry flux. The call below deletes the unconstrained metabolites, the reactions constrained to zero flux (mainly the uptake of non-standard carbon sources) and the dead-end reactions, and leaves linear reactions ungrouped. ```matlab model = simplifyModel(model, true, false, true, true); @@ -31,7 +31,7 @@ The simplified model contains 1305 reactions, 1037 metabolites and 1006 genes. ### 3. Setting constraints and an objective, and solving -As a first validation, compute the theoretical yield of CO₂ from glucose. The supplied model already allows uptake of phosphate, sulfate, NH3, O2 and the co-factor precursors thiamin and pimelate. Use `setParam` to cap glucose uptake at 1 `mmol/gDW/h`, block ethanol uptake, and set CO₂ production as the objective. +The first validation is the theoretical yield of CO₂ from glucose. The supplied model allows the uptake of phosphate, sulfate, NH3, O2 and the cofactor precursors thiamin and pimelate. `setParam` limits glucose uptake to 1 `mmol/gDW/h`, blocks ethanol uptake and sets CO₂ production as the objective. ```matlab model = setParam(model, 'ub', {'glcIN' 'etohIN'}, [1 0]); @@ -40,11 +40,11 @@ sol = solveLP(model); printFluxes(model, sol.x, true, 10^-7); % exchange fluxes only ``` -The solution structure contains `.f` (the objective value), `.stat` (1 on success) and `.x` (the flux through each reaction). The system takes up 1 glucose and 6 O₂ while producing 6 CO₂ and 6 H₂O. +The solution structure contains `.f` (the objective value), `.stat` (1 on success) and `.x` (the flux through each reaction). The system takes up 1 glucose and 6 O₂ and produces 6 CO₂ and 6 H₂O. -### 4. Cleaning up loops +### 4. Removing loops -Printing all fluxes (set the third argument to `false`) shows many reactions at ±1000 because of loops in the solution. Passing `1` as the second argument to `solveLP` minimises the sum of fluxes for a more interpretable result. +Printing all fluxes (the third argument is `false`) shows many reactions at ±1000 because of loops in the solution. `1` as the second argument of `solveLP` minimises the sum of fluxes and gives a result that is easier to interpret. ```matlab printFluxes(model, sol.x, false, 10^-7); % all fluxes @@ -56,7 +56,7 @@ The solution that minimises the sum of fluxes has far fewer active reactions tha ### 5. Optimising for growth and comparing carbon sources -To study growth, change the objective to biomass production (`bmOUT`). The growth rate on glucose is 0.084/h, and the system also requires sulfate, phosphate, NH3, thiamin and pimelate. Compare with growth on ethanol, using three times the molar flux (ethanol has 2 carbons and glucose has 6). The `eq` flag sets the lower and upper bound to the same value. +For growth, change the objective to biomass production (`bmOUT`). The growth rate on glucose is 0.084/h, and the system also requires sulfate, phosphate, NH3, thiamin and pimelate. Compare with growth on ethanol at a molar flux three times as large, because ethanol has 2 carbons and glucose has 6. The `eq` flag sets the lower and the upper bound to the same value. ```matlab model = setParam(model, 'obj', {'bmOUT'}, 1); @@ -71,16 +71,16 @@ printFluxes(modelETH, solETH.x, true, 10^-7); ### 6. Investigating the changes -`compareFluxes` takes two flux distributions and reports every reaction whose flux changed, largest change first, labelling the ones that were turned on, turned off or reversed direction. The first call considers changes above 0.5 `mmol/gDW/h`; the second restricts the view to ATP metabolism by naming the metabolites of interest. +`compareFluxes` takes two flux distributions and reports every reaction whose flux changed, with the largest change first. It labels the reactions that were turned on, turned off or reversed in direction. The first call considers changes above 0.5 `mmol/gDW/h`, and the second restricts the view to ATP metabolism by naming the metabolites of interest. ```matlab res = compareFluxes(modelETH, sol.x, solETH.x, 'cutoff', 0.5); compareFluxes(modelETH, sol.x, solETH.x, 'cutoff', 0.4, 'metaboliteList', {'ATP'}); ``` -The printed table is capped at 20 rows; `res.changed` holds every changed reaction and `res.turnedOn`, `res.turnedOff` and `res.flipped` list those that switched state. +The printed table has at most 20 rows. `res.changed` holds every changed reaction, and `res.turnedOn`, `res.turnedOff` and `res.flipped` list the reactions that switched state. -Comparing the fluxes this way shows the flux redistributions that produce different phenotypes; for example, on glucose ATP is generated in glycolysis, whereas on ethanol it involves acetate. +The comparison shows the flux redistributions that produce different phenotypes. On glucose, ATP is generated in glycolysis, and on ethanol the generation involves acetate. ## Full script diff --git a/docs/tutorials/tutorial2.md b/docs/tutorials/tutorial2.md index 5f49cc0..9f47d13 100644 --- a/docs/tutorials/tutorial2.md +++ b/docs/tutorials/tutorial2.md @@ -1,10 +1,10 @@ # Tutorial 2: Construct a functional small model -This exercise covers a small glycolysis model in RAVEN format and the basic aspects of stoichiometric modelling: building a model from scratch, setting parameters and performing basic simulations. +The exercise builds a small model of glycolysis in RAVEN format and covers the basic aspects of stoichiometric modelling: building a model from scratch, setting parameters and running basic simulations. -Start from `empty.xml` (an incomplete model containing the first reaction of glycolysis) and extend it until it forms a functional network. The companion script `tutorial2_solutions.m` reads the provided solution model `small.yml` directly with `readYAMLmodel`. +The start is `empty.xml`, an incomplete model with the first reaction of glycolysis, which is extended to a functional network. The companion script `tutorial2_solutions.m` reads the solution model `small.yml` directly with `readYAMLmodel`. -Build a model of glycolysis and answer how many units of ATP can be generated from one unit of sucrose. +The task is to build a model of glycolysis and to determine how many units of ATP can be generated from one unit of sucrose. ## Step by step @@ -16,30 +16,30 @@ sol = solveLP(smallModel); printFluxes(smallModel, sol.x, true); ``` -Importing the (incomplete) model gives a RAVEN warning that some internal metabolites are used in only one reaction, so the only feasible solution is zero flux. If `sol.f` equals zero the problem is not solvable, and uptake and excretion of all necessary metabolites must be added to the model before running `solveLP` again. +The import of the incomplete model gives a RAVEN warning that some internal metabolites are used in one reaction, so the sole feasible solution is zero flux. When `sol.f` equals zero, the problem cannot be solved, and the uptake and excretion of all necessary metabolites must be added to the model before `solveLP` is run a second time. ### 2. Internal and external metabolites -Under the steady-state assumption, the production rate of each internal metabolite must equal its consumption rate. A metabolite that participates in only one reaction can therefore carry no flux. The fix is to introduce **external (boundary) metabolites**, which need not be mass-balanced. Reactions involving them are **exchange reactions**. +Under the steady-state assumption, the production rate of each internal metabolite equals its consumption rate. A metabolite that takes part in one reaction can therefore carry no flux. Exchange reactions, which involve external (boundary) metabolites that do not need to be mass-balanced, remove this restriction. ### 3. Adding the glycolysis reactions and exchanges -Add the remaining reactions of glycolysis, then add exchange and transport reactions so that the cell can take up sucrose and excrete pyruvate and water. +Add the remaining reactions of glycolysis. Then add exchange and transport reactions, so that the cell can take up sucrose and excrete pyruvate and water. -Re-import the model with `importModel` to check the structure during the build. +Run `importModel` on the file again to check the structure during the build. ### 4. Handling cofactors and defining the objective Two problems remain: -- Regenerating NAD⁺ from NADH. Either extend the model (for example ethanol production via pyruvate decarboxylase + alcohol dehydrogenase) or add "fake" exchange reactions for NAD⁺ and NADH. -- Measuring ATP yield. Add a "fake" ATP hydrolysis reaction and maximise its flux: because production must match consumption, this reports how much ATP the network can make. Check the directionality; a wrong direction allows free ATP synthesis. +- NAD⁺ must be regenerated from NADH. Extend the model, for example with ethanol production through pyruvate decarboxylase and alcohol dehydrogenase, or add exchange reactions for NAD⁺ and NADH as an approximation. +- The ATP yield must be measured. Add a "fake" ATP hydrolysis reaction and maximise its flux. Production must match consumption, so the flux reports how much ATP the network can make. Check the direction, because a wrong direction allows free ATP synthesis. -Constrain the sucrose uptake to one unit and set the ATP-hydrolysis reaction as the objective to maximise. +Limit the sucrose uptake to one unit, and set the ATP-hydrolysis reaction as the objective to maximise. ### 5. Solving -Once the model is complete, solve the linear programming problem and print the fluxes. The completed model is provided as `small.yml`, which the solutions script reads directly: +When the model is complete, solve the linear programming problem and print the fluxes. The completed model is `small.yml`, which the solutions script reads directly: ```matlab smallModel = readYAMLmodel('small.yml'); @@ -48,7 +48,7 @@ printFluxes(smallModel, sol.x, true); % printFluxes(smallModel, sol.x, false, 10^-5, [], '%rxnID (%rxnName):\n\t%eqn\n\t%flux\n'); % all fluxes ``` -The worked answer is `tutorial2_solutions.m` in RAVEN's `tutorials` folder. +The worked answer is `tutorial2_solutions.m` in the `tutorials` folder of RAVEN. ## Full script diff --git a/docs/tutorials/tutorial3.md b/docs/tutorials/tutorial3.md index 7901f0d..c881163 100644 --- a/docs/tutorials/tutorial3.md +++ b/docs/tutorials/tutorial3.md @@ -1,16 +1,16 @@ # Tutorial 3: Knockouts and omics data -This exercise shows how to run FBA simulations and gene deletion analysis, and how a GEM can be used as a scaffold for interpreting microarray data. It uses a simplified model of yeast metabolism (`smallYeast.yml`), imported with `readYAMLmodel`. +The exercise runs FBA simulations and a gene deletion analysis, and it uses a GEM as a scaffold for the interpretation of microarray data. The model is a simplified model of yeast metabolism (`smallYeast.yml`), which `readYAMLmodel` imports. -It is assumed that Tutorial 2 is completed. +Tutorial 2 must be completed first. -`tutorial3.m` is a skeleton script to work through. The completed answers are in the companion script `tutorial3_solutions.m`. The code excerpts follow the solutions; try to derive them before looking. +`tutorial3.m` is a skeleton script for the exercise. The completed answers are in the companion script `tutorial3_solutions.m`, and the code excerpts below follow the solutions. Write the code before reading the solutions. ## Step by step -### 1. Importing and checking ATP yield +### 1. Importing and checking the ATP yield -Import the model, restrict it to anaerobic glucose growth, and maximise ATP hydrolysis (`ATPX`): +Import the model, restrict it to anaerobic growth on glucose and maximise the ATP hydrolysis (`ATPX`): ```matlab model = readYAMLmodel('smallYeast.yml'); @@ -20,11 +20,11 @@ sol = solveLP(model); printFluxes(model, sol.x, false); ``` -The ATP production rate should be 2.0. It was 4.0 in Tutorial 2, where sucrose was the substrate. +The ATP production rate is 2.0. In Tutorial 2, where sucrose was the substrate, it was 4.0. ### 2. Checking product yields -Switch to fully aerobic conditions and check the yield of different products, maximising each in turn: +Switch the model to fully aerobic conditions and check the yield of each product by maximising it in turn: ```matlab model = setParam(model, 'ub', {'glcIN' 'o2IN'}, [1 1000]); @@ -33,11 +33,11 @@ sol = solveLP(model); fprintf(['Yield of ethanol is ' num2str(sol.f) ' mol/mol\n']); ``` -The same is repeated for acetate (`acOUT`), glycerol (`glyOUT`) and biomass (`biomassOUT`). +Repeat this for acetate (`acOUT`), glycerol (`glyOUT`) and biomass (`biomassOUT`). ### 3. Aerobic and limited oxygen conditions -Solve for biomass production under full aeration, then restrict the oxygen uptake upper bound to 0.5 and solve again: +Solve for biomass production with full aeration. Then restrict the upper bound of the oxygen uptake to 0.5 and solve a second time: ```matlab solA = solveLP(model); @@ -47,7 +47,7 @@ solB = solveLP(model); ### 4. Single gene deletions with FBA -Set the model to anaerobic growth maximising biomass, then scan single gene deletions. After running the deletion scan, keep only solutions that grow and look for the one giving the highest glycerol production: +Set the model to anaerobic growth with biomass as the objective, and scan single gene deletions. Keep the solutions that grow, and select the one with the highest glycerol production: ```matlab model = setParam(model, 'eq', {'o2IN'}, 0); @@ -65,7 +65,7 @@ fprintf(['Glycerol production is ' num2str(maxGlycerol) ... ' after deletion of ' originalGenes{genes(okSolutions(J),:)} '\n']); ``` -The model predicts a glycerol production of 0.23 `mmol/gDW/h` for the unperturbed anaerobic case. The best gene deletion corresponds to turning off the `ZWF` reaction (`ZWF1`, `YNL241C`). Compare the deletion strain to wild type and follow the changed fluxes around the redox metabolites: +The model predicts a glycerol production of 0.23 `mmol/gDW/h` for the unperturbed anaerobic case. The best gene deletion turns off the `ZWF` reaction (`ZWF1`, `YNL241C`). Compare the deletion strain with the wild type, and follow the changed fluxes around the redox metabolites: ```matlab model2 = setParam(model, 'eq', {'ZWF'}, 0); @@ -76,7 +76,7 @@ compareFluxes(model, sol.x, sol2.x, 'cutoff', 10^-2, ... ### 5. Reporter metabolites from microarray data -A GEM can highlight the metabolites around which significant transcriptional changes cluster. Reload the model first (the preceding steps changed its bounds and objective), then load the expression data and run the reporter-metabolites test: +A GEM shows the metabolites around which significant transcriptional changes cluster. Reload the model first, because the preceding steps changed its bounds and objective. Then load the expression data and run the reporter-metabolites test: ```matlab model = readYAMLmodel('smallYeast.yml'); diff --git a/docs/tutorials/tutorial4.md b/docs/tutorials/tutorial4.md index e468d05..db14715 100644 --- a/docs/tutorials/tutorial4.md +++ b/docs/tutorials/tutorial4.md @@ -1,18 +1,18 @@ # Tutorial 4: Fix an erroneous model -A GEM is powerful because of its size, and the same size makes errors almost inevitable, in a self-built model and in a model from another source. This exercise is a systematic round of quality control on a deliberately broken version of the small yeast model, `smallYeastBad.yml`. +A GEM covers metabolism at a large scale, and the size makes errors almost inevitable, in a self-built model and in a model from another source. The exercise is a systematic round of quality control on a deliberately broken version of the small yeast model, `smallYeastBad.yml`. -Model validation is iterative: some errors only appear once others are fixed. The RAVEN philosophy is that trying to make a model do something it should not (for example create mass from nothing) is more valuable than testing only for what it should do. +Model validation is iterative, as some errors appear after others are fixed. In the RAVEN approach, the user tries to make a model do something that it should not do, for example create mass from nothing. This finds errors that a check of the intended behaviour misses. -`tutorial4.m` leaves several gaps to fill; the completed version is in `tutorial4_solutions.m`. Many of these checks are also wrapped by the `gapReport` function, but doing them step by step is far more instructive. +`tutorial4.m` leaves several gaps to fill, and the completed version is `tutorial4_solutions.m`. The `gapReport` function wraps many of these checks, and doing them step by step shows how they work. -The changes are made here in code, which avoids having several model files. It is assumed that Tutorials 2–3 are completed and that the reader is familiar with linear programming. +The changes are made in code, which avoids several model files. Tutorials 2 and 3 must be completed first, and the exercise assumes familiarity with linear programming. ## Step by step ### 1. Detecting production from nothing -Import the broken model, close all uptake, and maximise the sum of the producing exchange reactions. Any non-zero solution is a problem. +Import the broken model, close all uptake and maximise the sum of the producing exchange reactions. Any non-zero solution is a problem. ```matlab model = readYAMLmodel('smallYeastBad.yml'); @@ -24,7 +24,7 @@ printFluxes(model, sol.x, true); %Nothing produced, good ### 2. Provoking hidden errors -An error may be masked because it costs energy or redox power. Add temporary "free" cofactor reactions (ATP ⇌ ADP + Pᵢ, and similar for NADH and NADPH) to provoke the model, then minimise the sum of fluxes for an interpretable solution: +An error can be masked because it costs energy or redox power. Add temporary "free" cofactor reactions (ATP ⇌ ADP + Pᵢ, and similar reactions for NADH and NADPH) to provoke the model, and minimise the sum of fluxes for an interpretable solution: ```matlab rxns.rxns = {'FREE_ATP'; 'FREE_NADH'; 'FREE_NADPH'}; @@ -34,12 +34,13 @@ sol = solveLP(model, 1); printFluxes(model, sol.x, false, [], [], '%rxnID (%rxnName):%flux\n\t%eqn\n'); ``` -??? success "Answer to Question 2: production of ethanol from nothing" - A large amount of ethanol is produced. `ADH1` should only produce one unit of ethanol. Change its equation with `model = changeRxns(model, 'ADH1', 'acetaldehyde[c] + NADH[c] => ethanol[c] + NAD(+)[c]', 3);` +:::{dropdown} Answer to step 2: production of ethanol from nothing +A large amount of ethanol is produced. `ADH1` should produce one unit of ethanol. Change its equation with `model = changeRxns(model, 'ADH1', 'acetaldehyde[c] + NADH[c] => ethanol[c] + NAD(+)[c]', 3);` +::: ### 3. Allowing every metabolite to be excreted -Add a second column to `model.b` so that RAVEN treats it as lower and upper bounds on the equality constraints and any metabolite can be excreted. This exposes errors that need a partner metabolite to be dumped. +Add a second column to `model.b`, and RAVEN then treats the columns as lower and upper bounds on the equality constraints and any metabolite can be excreted. This exposes errors that need a partner metabolite to be dumped. ```matlab model.b = [model.b inf(numel(model.b), 1)]; @@ -47,17 +48,18 @@ sol = solveLP(model, 1); printFluxes(model, sol.x, false, 10^-5, [], '%rxnID (%rxnName):\n\t%eqn\n\t%flux\n'); ``` -??? success "Answer to Question 3: two unbalanced reactions" - The unbalanced reactions in the flux list show that two reactions should each result in only one unit of product: +:::{dropdown} Answer to step 3: two unbalanced reactions +The unbalanced reactions in the flux list show that two reactions should each give one unit of product: - - `FBP` should give one unit of F6P: change it to `beta-D-fructofuranose 1,6-bisphosphate[c] => beta-D-fructofuranose 6-phosphate[c] + phosphate[c]`. - - `PFK` should give one unit of F16P: change it to `ATP[c] + beta-D-fructofuranose 6-phosphate[c] => ADP[c] + beta-D-fructofuranose 1,6-bisphosphate[c]`. +- `FBP` should give one unit of F6P. Change its equation to `beta-D-fructofuranose 1,6-bisphosphate[c] => beta-D-fructofuranose 6-phosphate[c] + phosphate[c]`. +- `PFK` should give one unit of F16P. Change its equation to `ATP[c] + beta-D-fructofuranose 6-phosphate[c] => ADP[c] + beta-D-fructofuranose 1,6-bisphosphate[c]`. - Apply each with `changeRxns(model, …, 3)`. +Apply each with `changeRxns(model, …, 3)`. +::: ### 4. `canExchange` -Set all uptakes and production to 0, drop the excretion column again, and check which metabolites can be consumed without any production: +Set all uptake and production to 0, drop the excretion column and determine the metabolites that can be consumed without any production: ```matlab model = setParam(model, 'eq', getExchangeRxns(model), 0); @@ -66,7 +68,7 @@ I = canExchange(model, 'consume'); disp(model.mets(I)); %These 12 metabolites can be consumed without any production ``` -`canExchange` with `'consume'` reports the metabolites the model can consume even when no production is allowed; `'produce'` asks the opposite. Allow all uptake again, then force the uptake of one of them (here CO2), and study the fluxes (a negative output means input): +`canExchange` with `'consume'` reports the metabolites that the model can consume when no production is allowed, and `'produce'` asks the opposite. Allow all uptake, force the uptake of one of the metabolites (here CO2) and study the fluxes (a negative output means input): ```matlab model.b = [ones(numel(model.b), 1) * -1000 model.b]; @@ -75,21 +77,22 @@ sol = solveLP(model); printFluxes(model, sol.x, false, 10^-5, [], '%rxnID (%rxnName):\n\t%eqn\n\t%flux\n'); %Now it works ``` -??? success "Answer to Question 4: `PDC` is missing a product" - `PDC` converts pyruvate (3 carbons) to acetaldehyde (2 carbons) without any other products: CO2 is missing. The change can be made with `changeRxns`; as an exercise, add the cytosolic CO2 coefficient directly to the stoichiometric matrix: +:::{dropdown} Answer to step 4: `PDC` lacks a product +`PDC` converts pyruvate (3 carbons) to acetaldehyde (2 carbons) without another product, and CO2 is missing. `changeRxns` makes the change. As an exercise, add the coefficient of cytosolic CO2 directly to the stoichiometric matrix: - ```matlab - Irxn = ismember(model.rxns, 'PDC'); - Imet = ismember(model.mets, 'CO2_c'); - model.S(Imet, Irxn) = 1; %The coefficient is 1.0 - constructEquations(model, Irxn) - ``` +```matlab +Irxn = ismember(model.rxns, 'PDC'); +Imet = ismember(model.mets, 'CO2_c'); +model.S(Imet, Irxn) = 1; %The coefficient is 1.0 +constructEquations(model, Irxn) +``` - The solution is infeasible after the change, meaning that forcing uptake of CO2 without any output is not possible. +The solution is infeasible after the change, because the forced uptake of CO2 without any output is not possible. +::: ### 5. Gap-filling against a reference model -`tutorial4.m` also imports a reference model to draw reactions from, allows all excretion, and reports gaps before filling them: +`tutorial4.m` also imports a reference model from which reactions are drawn, allows all excretion, and reports the gaps before filling them: ```matlab model = readYAMLmodel('smallYeastBad.yml'); @@ -107,7 +110,7 @@ disp(newConnected); disp(cannotConnect); ``` -`gapReport` automates this whole quality-control workflow, while `fillGaps` adds reactions from the reference model (here the Tutorial 3 small yeast model) to connect the remaining gaps. +`gapReport` automates this quality-control workflow, and `fillGaps` adds reactions from the reference model (here the small yeast model of Tutorial 3) to connect the remaining gaps. ## Full script diff --git a/docs/tutorials/tutorial5.md b/docs/tutorials/tutorial5.md index 7cc4995..864a764 100644 --- a/docs/tutorials/tutorial5.md +++ b/docs/tutorials/tutorial5.md @@ -1,46 +1,46 @@ # Tutorial 5: Reconstruct a GEM from KEGG -This exercise creates a model from KEGG, based on protein sequences in a FASTA file, and runs some functionality checks on the result. The example organism is *Saccharomyces cerevisiae*. +The exercise creates a model from KEGG on the basis of protein sequences in a FASTA file and runs functionality checks on the result. The example organism is *Saccharomyces cerevisiae*. -Unlike Tutorials 1–4, this tutorial is a demonstration. Its main purpose is to provide a scaffold that can be adapted to reconstruct a GEM for any organism. +Unlike Tutorials 1 to 4, this tutorial is a demonstration. It provides a scaffold that can be adapted to reconstruct a GEM for any organism. -De novo reconstruction performs sequence searches against large HMM databases. Building the model takes up to 20–35 minutes on macOS and Unix systems and 40–55 minutes on Windows, depending on the hardware and the size of the target organism's proteome. `gapReport` can take several to many hours, depending on the number of gaps in the model. +The de novo reconstruction searches large HMM databases. Building the model takes 20 to 35 minutes on macOS and Unix systems and 40 to 55 minutes on Windows, depending on the hardware and the size of the proteome of the target organism. `gapReport` can take several to many hours, depending on the number of gaps in the model. ## Step by step ### 1. Reconstructing from KEGG -`getKEGGModelForOrganism` needs trained Hidden Markov Models (HMMs) for eukaryotes. If the library file `.hmm` (or its `.gz` archive) is already present, for example because it was downloaded manually, it is used. Otherwise it is downloaded from the matching release of [toolbox-assets](https://github.com/CompMetEng/toolbox-assets); here the archive `kegg118_eukaryotes` is picked for automatic download. +`getKEGGModelForOrganism` needs trained hidden Markov models (HMMs) for eukaryotes. When the library file `.hmm` or its `.gz` archive exists, for example after a manual download, it is used. Otherwise it is downloaded from the matching release of [toolbox-assets](https://github.com/CompMetEng/toolbox-assets). Here the archive `kegg118_eukaryotes` is selected for automatic download. -The function then creates a model for *S. cerevisiae* from the whole-proteome FASTA (`sce.fa`). The parameters are set to exclude general or unclear reactions and reactions with undefined stoichiometry. Type `help getKEGGModelForOrganism` to see what the different parameters are for. +The function then creates a model for *S. cerevisiae* from the FASTA file of the whole proteome (`sce.fa`). The parameters exclude general or unclear reactions and reactions with undefined stoichiometry. `help getKEGGModelForOrganism` describes the parameters. ```matlab model=getKEGGModelForOrganism('sce','sce.fa','kegg118_eukaryotes','output',false,false,false,false,10^-30,0.8,0.3,-1); disp(model); ``` -The resulting model should contain around 1589 reactions, 1600 metabolites and 836 genes. Small variations are possible since it is a heuristic algorithm, and different KEGG versions give slightly different results. +The resulting model should contain around 1589 reactions, 1600 metabolites and 836 genes. Small variations are possible because the algorithm is heuristic, and different KEGG versions give slightly different results. ### 2. Removing reactions that make something from nothing -A first control is that the model should not be able to produce any metabolites without uptake of some metabolites. This often happens when metabolites have a different meaning in different reactions. `removeBadRxns` tries to find and remove such reactions automatically; type `help removeBadRxns` for details. +A first control is that the model cannot produce any metabolite without the uptake of some metabolites. A violation often arises when metabolites have a different meaning in different reactions. `removeBadRxns` finds and removes such reactions automatically, and `help removeBadRxns` gives the details. ```matlab [newModel, removedRxns]=removeBadRxns(model); ``` -This reports that H⁺ can be made even if no reactions were unbalanced. Protons are particularly problematic because the pH at which the formulas are written is rather arbitrary. For this analysis the protons can be ignored and fixed later, so rerun while allowing H⁺ to be produced. +The output reports that H⁺ can be made even when no reaction was unbalanced. Protons are problematic because the pH at which the formulas are written is arbitrary. For this analysis the protons can be ignored and corrected later, so repeat the call with the production of H⁺ allowed. ```matlab [newModel, removedRxns]=removeBadRxns(model,1,{'H+'},true); disp(removedRxns); ``` -One reaction was removed because it enabled the model to produce something from nothing. Because it is a single reaction, the reaction is examined in more detail. +One reaction was removed because it enabled the model to produce something from nothing. As it is a single reaction, examine it in more detail. ### 3. Investigating the problematic reaction -According to KEGG, the removed reaction is a general polymer reaction. Use `findLeakMetabolite` to look at the flux distributions in more detail and find out if there is a better alternative to delete. +According to KEGG, the removed reaction is a general polymer reaction. `findLeakMetabolite` shows the flux distributions in more detail and shows whether a better alternative exists for deletion. ```matlab [fluxes, metabolite]=findLeakMetabolite(model,'produce',{'H+'},true); @@ -48,7 +48,7 @@ model.metNames(metabolite) printFluxes(model, fluxes, false, [], [],'%rxnID (%rxnName):\n\t%eqn: %flux\n') ``` -This shows the model could produce H₂O, and returns more fluxes than can be read at once. Excluding the elementally balanced reactions narrows the list. Since water was produced, only look at the reactions unbalanced for oxygen (column 6 of the elemental balance). +The model could produce H₂O, and the output contains more fluxes than can be read at once. Excluding the elementally balanced reactions shortens the list. Water was produced, so the reactions that are unbalanced for oxygen (column 6 of the elemental balance) are of interest. ```matlab balanceStructure=getElementalBalance(model); @@ -56,13 +56,13 @@ goodOnes=balanceStructure.leftComp(:,6)==balanceStructure.rightComp(:,6); printFluxes(removeReactions(model,goodOnes), fluxes(~goodOnes), false, [], [],'%rxnID (%rxnName):\n\t%eqn: %flux\n'); ``` -Many reactions remain. Leave only the reactions which involve amylose or starch, from one of the problematic reactions identified in step 2. +Many reactions remain. The reactions that involve amylose or starch are kept, as one of the problematic reactions from step 2 involves them. ```matlab printFluxes(model, fluxes, false, [], [],'%rxnID (%rxnName):\n\t%eqn: %flux\n',{'Amylose';'Starch'}); ``` -There are two elementally unbalanced reactions, including the one identified by `removeBadRxns`. They contradict each other: the first shows that amylose and starch are interconvertible, while the second shows that amylose contains one less glucose unit than starch. Such general reactions should be fixed manually. Trusting `removeBadRxns`, delete `R02110`. +Two elementally unbalanced reactions remain, including the reaction identified by `removeBadRxns`. They contradict each other: the first shows that amylose and starch are interconvertible, and the second shows that amylose contains one glucose unit fewer than starch. General reactions of this kind are fixed manually. The result of `removeBadRxns` is trusted, and `R02110` is deleted. ```matlab model=removeReactions(model,'R02110'); @@ -70,32 +70,32 @@ model=removeReactions(model,'R02110'); ### 4. Checking consumption and adding uptakes -After removing `R02110` the model cannot make something from nothing. Check whether it can consume something without any output. +After the removal of `R02110`, the model cannot make something from nothing. The next check is whether it can consume something without any output. ```matlab [solution, metabolite]=findLeakMetabolite(model,'consume',{'H+'},true); model.metNames(metabolite) ``` -Nothing is consumed without output. Add some uptakes and check what the model can produce. +Nothing is consumed without output. Add some uptakes and check the products of the model. ```matlab [~, J]=ismember({'D-Glucose';'H2O';'Orthophosphate';'Oxygen';'NH3';'Sulfate'},model.metNames); [model, addedRxns]=addExchangeRxns(model,'in',J); ``` -`canExchange` reports which metabolites can be produced given these uptakes. It allows output of all metabolites, which does not happen in a real cell. This is useful for functionality testing. +`canExchange` reports the metabolites that can be produced with these uptakes. It allows the output of all metabolites, which does not happen in a real cell, and it serves functionality testing. ```matlab I=canExchange(model,'produce'); fprintf('%d%%\n', round(sum(I)/numel(model.mets)*100)); ``` -Around 31 % of the metabolites could be synthesised. The result does not show by itself whether this is high or low. Many metabolites are not expected to be synthesised from glucose, water, orthophosphate, oxygen, NH3 and sulfate alone. +About 31 % of the metabolites can be synthesised. The result does not show whether this is high or low. Many metabolites are not expected to be synthesised from glucose, water, orthophosphate, oxygen, NH3 and sulfate alone. -### 5. Filling gaps using the full KEGG model +### 5. Filling gaps with the full KEGG model -Gap-filling with the full KEGG model tests whether this gives a higher fraction of synthesisable metabolites. `getModelFromKEGG` retrieves the full KEGG model. It is associated with more than 6,400,000 genes that are not used for gap-filling, so they are removed to speed up the step. +Gap-filling with the full KEGG model tests whether it gives a higher fraction of synthesisable metabolites. `getModelFromKEGG` retrieves the full KEGG model. It is associated with more than 6,400,000 genes that gap-filling does not use, and they are removed to speed up the step. ```matlab keggModel=getModelFromKEGG(false,false,false,false); @@ -103,14 +103,14 @@ keggModel=rmfield(keggModel,'genes'); keggModel=rmfield(keggModel,'rxnGeneMat'); ``` -Some reactions in KEGG are known to be unbalanced. Only the balanced ones are used for gap-filling. +Some reactions in KEGG are known to be unbalanced, and the balanced reactions are used for gap-filling. ```matlab balanceStructure=getElementalBalance(keggModel); keggModel=removeReactions(keggModel,balanceStructure.balanceStatus~=1,true,true); ``` -`fillGaps` with these settings tries to include reactions that give flux through all reactions in the model. The first flag says that production of all metabolites should be allowed. +With these settings, `fillGaps` tries to include reactions that give flux through all reactions in the model. The first flag allows the production of all metabolites. ```matlab params.relGap=0.6; %Lower number for a more exhaustive search @@ -118,31 +118,31 @@ params.printReport=true; [newConnected, cannotConnect, addedRxns, newModel, exitFlag]=fillGaps(model,keggModel,true,false,false,[],params); ``` -The results show that `fillGaps` could connect around 29 reactions (`newConnected`) by including around 41 reactions from the KEGG model (`addedRxns`). These should be checked manually to confirm that they exist in yeast; here they are assumed to. +`fillGaps` connected around 29 reactions (`newConnected`) by including around 41 reactions from the KEGG model (`addedRxns`). These reactions should be checked manually to confirm that they exist in yeast, and here they are assumed to. ### 6. Reporting connectivity -Continue improving the connectivity of the model by identifying metabolites that should be connected. `gapReport` summarises how connected the model is, along with the supporting detail. +The connectivity of the model is improved further by identifying metabolites that should be connected. `gapReport` summarises how connected the model is, together with the supporting detail. ```matlab [noFluxRxns, noFluxRxnsRelaxed, subGraphs, notProducedMets, minToConnect,... neededForProductionMat]=gapReport(newModel); ``` -The results show that 532 from 1634 reactions cannot carry flux. Because output is allowed for all metabolites, it calculates 532 in both cases. 543 from 1617 metabolites cannot be synthesised from the supplied precursors. There are 6 subnetworks in the model, of which 1605 from 1617 metabolites belong to the first one. +532 of 1634 reactions cannot carry flux. Output is allowed for all metabolites, so the number is 532 in both cases. 543 of 1617 metabolites cannot be synthesised from the supplied precursors. The model has 6 subnetworks, and 1605 of 1617 metabolites belong to the first. -`gapReport` also prints the metabolites to connect: to enable net production of all metabolites, a total of 322 metabolites must be connected, with the top ranked one being Acyl-CoA (which connects 162 metabolites). Connecting only the top 10 in the list already connects 271/543 (50 %) of them. This is a useful way of directing the gap-filling tasks to where they are of greatest use. +`gapReport` also prints the metabolites to connect. Net production of all metabolites requires the connection of 322 metabolites, and the top-ranked one is Acyl-CoA, which connects 162 metabolites. The top 10 of the list connect 271 of 543 (50 %) of them. The list shows where the gap-filling has the greatest effect. ### 7. Adding minimal-media uptakes and iterating -The list reveals that yeast cannot grow on only the substrates defined so far; it requires some other precursors for co-factor synthesis as well. Add uptake reactions for the minimal-media constituents needed for yeast to grow. +The list shows that yeast cannot grow on the substrates defined so far and requires other precursors for cofactor synthesis. Add uptake reactions for the minimal-medium constituents that yeast needs to grow. ```matlab [~, J]=ismember({'4-Aminobenzoate';'Riboflavin';'Thiamine';'Biotin';'Folate';'Nicotinate';'Zymosterol';'Choline'},newModel.metNames); [newModel, addedRxns]=addExchangeRxns(newModel,'in',J); ``` -Rerun `gapReport` and use the output for targeting the gap-filling efforts. `gapReport` prints part of the information; most of it is available in the output structures. Work iteratively until the model is of sufficient quality. +Run `gapReport` again, and use its output to direct the gap-filling. `gapReport` prints part of the information, and the output structures hold the rest. Repeat these steps until the model meets the quality criteria of the project, for example a fraction of synthesisable metabolites. ## Full script diff --git a/docs/yaml-format.md b/docs/yaml-format.md index a607548..3154667 100644 --- a/docs/yaml-format.md +++ b/docs/yaml-format.md @@ -1,12 +1,12 @@ # YAML file format -The YAML format is produced and consumed by three tools: +Three tools produce and read the YAML format: - cobrapy ([`cobra.io.{load,save}_yaml_model`](https://github.com/opencobra/cobrapy)) -- raven-toolbox (`raven_toolbox.io.yaml.{read,write}_yaml_model`, see [API](api/index.md)) -- RAVEN MATLAB (`readYAMLmodel.m` and `writeYAMLmodel.m` in the [RAVEN repo](https://github.com/CompMetEng/RAVEN/tree/develop/io)) +- raven-toolbox (`raven_toolbox.io.yaml.{read,write}_yaml_model`, [API](api/index.md)) +- RAVEN MATLAB (`readYAMLmodel.m` and `writeYAMLmodel.m` in [`src/io`](https://github.com/CompMetEng/RAVEN/tree/develop/src/io) of the RAVEN repository) -The same file can be round-tripped through any of the three. cobrapy defines the core shape. raven-toolbox and RAVEN MATLAB add namespaced extensions (RAVEN curation fields, MIRIAM cross-references already covered by cobrapy's `annotation`, and the GECKO `ec-*` sections) that leave the cobra-known shape intact. +The same file can be read by any of the three, and a round trip through raven-toolbox or RAVEN MATLAB preserves all content. cobrapy defines the core shape. raven-toolbox and RAVEN MATLAB add namespaced extensions that leave the shape known to cobra intact: RAVEN curation fields, MIRIAM cross-references that the `annotation` of cobrapy already covers, and the GECKO `ec-*` sections. --- @@ -63,11 +63,11 @@ The same file can be round-tripped through any of the three. cobrapy defines the - e: extracellular ``` -Three structural rules apply throughout the field-by-field description: +Three structural rules apply to the field-by-field description: -1. The whole document is one ordered mapping (`!!omap`) at the root. Every nested map that preserves key order is also `!!omap` (metaData, each metabolite, reaction and gene entry, `annotation`, `metabolites`, `compartments`, and the ec sections). -2. Each metabolite, reaction, and gene is one `- !!omap` element of a list. Inside that mapping, every field is written as `- key: value`. This is cobrapy's native shape, and RAVEN MATLAB's reader depends on it. -3. Strings are unquoted by default. Quotes appear only when YAML would otherwise misparse the value (an empty string; a leading indicator character; `-`, `?` or `:` followed by whitespace; embedded `: ` or ` #`; values that read as `true`, `false`, `y`, `n`, `yes`, `no`, `on`, `off`, `null`, a number or a date). The section on numbers, strings and quoting lists the cases. +1. The whole document is one ordered mapping (`!!omap`) at the root. Every nested mapping that preserves key order is also an `!!omap`: `metaData`, each metabolite, reaction and gene entry, `annotation`, `metabolites`, `compartments` and the ec sections. +2. Each metabolite, reaction and gene is one `- !!omap` element of a list. Inside the mapping, every field is written as `- key: value`. This is the native shape of cobrapy, and the reader of RAVEN MATLAB depends on it. +3. Strings are unquoted by default. Quotes appear when YAML would otherwise misparse the value: an empty string, a leading indicator character, `-`, `?` or `:` followed by whitespace, an embedded `: ` or ` #`, and values that read as `true`, `false`, `y`, `n`, `yes`, `no`, `on`, `off`, `null`, a number or a date. The section on numbers, strings and quoting lists the cases. --- @@ -87,22 +87,22 @@ Three structural rules apply throughout the field-by-field description: | Key | Required | Source | Notes | |-----|----------|--------|-------| -| `metaData` | optional | RAVEN | Provenance block. Holds `id`, `name`, `version`, `date`, `taxonomy`, optionally `givenName`, `familyName`, `email`, `organization`, `note` and `sourceUrl`, plus `defaultLB` and `defaultUB`. Cobrapy ignores this block, which carries no semantics of the core model. | +| `metaData` | optional | RAVEN | Provenance block with `id`, `name`, `version`, `date`, `taxonomy`, optionally `givenName`, `familyName`, `email`, `organization`, `note` and `sourceUrl`, and `defaultLB` and `defaultUB`. cobrapy ignores the block, which carries no semantics of the core model. | | `metabolites` | yes | cobra core | Ordered list of `- !!omap` entries. | | `reactions` | yes | cobra core | Ordered list of `- !!omap` entries. | -| `genes` | yes | cobra core | Ordered list; may be `genes: []` for a model with no genes. | +| `genes` | yes | cobra core | Ordered list; `genes: []` for a model without genes. | | `compartments` | yes | cobra core | `!!omap` of `: `. | -| `gecko_light` | optional | GECKO | Scalar boolean. Cobrapy and raven-toolbox emit this at the top level; the spelling `geckoLight` inside `metaData` is accepted on read. | -| `ec-rxns` | optional | GECKO | Per-reaction coupling table of kcat, source and enzymes. | -| `ec-enzymes` | optional | GECKO | Per-enzyme table of MW, sequence and concentration. | +| `gecko_light` | optional | GECKO | Scalar boolean. cobrapy and raven-toolbox write it at the top level, and the spelling `geckoLight` inside `metaData` is accepted on reading. | +| `ec-rxns` | optional | GECKO | Table per reaction with kcat, source and enzymes. | +| `ec-enzymes` | optional | GECKO | Table per enzyme with MW, sequence and concentration. | -Cobrapy writes `id`, `name` and `version` at the root level and not inside `metaData`. The RAVEN readers accept both placements, and the RAVEN writers write the `metaData` form. +cobrapy writes `id`, `name` and `version` at the root level. The RAVEN readers accept both placements, and the RAVEN writers write the `metaData` form. --- ## Metabolite entry -Field order (cobra-core first, then RAVEN extensions): +The field order has the cobra-core fields first and the RAVEN extensions after them: ```yaml - !!omap @@ -124,9 +124,9 @@ Field order (cobra-core first, then RAVEN extensions): - metFrom: KEGG # RAVEN extension ``` -Cobrapy emits exactly the first seven keys (the cobra-core block). raven-toolbox and RAVEN MATLAB additionally emit `inchis`, `deltaG`, and `metFrom` when those fields are populated. On read, cobrapy puts the RAVEN extensions on the metabolite as attribute fall-through, raven-toolbox captures them into `metabolite.notes` (keyed by their YAML name), and RAVEN MATLAB stores them on `model.inchis`, `model.metDeltaG` and `model.metFrom`. +cobrapy writes the first seven keys, which are the cobra-core block. raven-toolbox and RAVEN MATLAB also write `inchis`, `deltaG` and `metFrom` when those fields are populated. On reading, cobrapy keeps the RAVEN extensions on the metabolite as attributes (attribute fall-through), raven-toolbox captures them in `metabolite.notes` under their YAML name, and RAVEN MATLAB stores them in `model.inchis`, `model.metDeltaG` and `model.metFrom`. -Annotation entries with multiple values are emitted as a YAML list (`chebi:` then several `-` items). Single-value entries are emitted inline (`kegg.compound: C00002`). SMILES strings are stored inside the annotation block under the `smiles` key. A `smiles` field at the top level of a metabolite entry is also accepted on read. +An annotation entry with several values is written as a YAML list (`chebi:` followed by several `-` items), and a single value is written inline (`kegg.compound: C00002`). SMILES strings are stored in the annotation block under the key `smiles`. A `smiles` field at the top level of a metabolite entry is also accepted on reading. --- @@ -159,17 +159,17 @@ Annotation entries with multiple values are emitted as a YAML list (`chebi:` the Some fields are conditional: -- `objective_coefficient` is only written when non-zero (cobrapy convention). -- The `metabolites` block uses `!!omap []` (flow-style empty omap) when the reaction has no metabolites, which keeps the file a valid YAML 1.2 document. -- `eccodes` is written inline (`eccodes: 2.7.1.1`) when there is one code, and as a list when there are several. The same applies to `references`. +- `objective_coefficient` is written when it is not zero (cobrapy convention). +- The `metabolites` block is `!!omap []` (a flow-style empty omap) when the reaction has no metabolites, which keeps the file a valid YAML 1.2 document. +- `eccodes` is written inline (`eccodes: 2.7.1.1`) for one code and as a list for several codes. The same applies to `references`. -Cobrapy and the raven-toolbox and RAVEN MATLAB writers use the key `notes`. Some yeast-GEM files use `rxnNotes`, a legacy alias that both readers accept. +cobrapy and the writers of raven-toolbox and RAVEN MATLAB use the key `notes`. Some yeast-GEM files use `rxnNotes`, a legacy alias that both readers accept. -A `notes` entry can be a string or a mapping. raven-toolbox writes one for notes that have no RAVEN field, for example the base id that `merge_compartments` stores on each merged metabolite, with the free-text note under the key `note`. raven-toolbox reads such a mapping back into `notes`. RAVEN shows it in `metNotes`, and a round trip through RAVEN YAML or a `.mat` file does not retain it. +A `notes` entry is a string or a mapping. raven-toolbox writes a mapping for notes that have no RAVEN field, for example the base identifier that `merge_compartments` stores on each merged metabolite, with the free-text note under the key `note`. raven-toolbox reads such a mapping back into `notes`. RAVEN shows it in `metNotes`, and a round trip through RAVEN YAML or a `.mat` file does not retain it. -raven-toolbox reads `gene_reaction_rule` with the grammar of RAVEN's `parseGrRule`. RAVEN's reader splits rules with `getGenesFromGrRules`, which needs a space on each side of a lowercase `and`, `or`, `&` or `|` unless the operator touches a bracket, so `G1&G2` is one gene id and `G1 & G2` is two genes joined by `and`. Brackets next to an operator make no difference, because `getGenesFromGrRules` inserts a space in `)and`, `and(`, `)or`, `or(`, `)&`, `&(`, `)|` and `|(`. raven-toolbox also accepts the spellings `AND`, `OR`, `&&` and `||`, which RAVEN's reader leaves inside a gene id. +raven-toolbox reads `gene_reaction_rule` with the grammar of `parseGrRule` of RAVEN. The reader of RAVEN splits rules with `getGenesFromGrRules`, which needs a space on each side of a lowercase `and`, `or`, `&` or `|` unless the operator touches a bracket. `G1&G2` is therefore one gene identifier, and `G1 & G2` is two genes joined by `and`. A bracket next to an operator makes no difference, because `getGenesFromGrRules` inserts a space in `)and`, `and(`, `)or`, `or(`, `)&`, `&(`, `)|` and `|(`. raven-toolbox also accepts the spellings `AND`, `OR`, `&&` and `||`, which the reader of RAVEN leaves inside a gene identifier. -Bounds are emitted as floats with an explicit decimal point (`1000.0`, `-1000.0`), matching Python's float repr and cobrapy's output. +Bounds are written as floats with an explicit decimal point (`1000.0`, `-1000.0`), which matches the float representation of Python and the output of cobrapy. --- @@ -185,7 +185,7 @@ Bounds are emitted as floats with an explicit decimal point (`1000.0`, `-1000.0` - protein: P04807 # RAVEN extension ``` -Empty names (`name: ''`) are not emitted (matches RAVEN MATLAB's behaviour). +Empty names (`name: ''`) are not written, as in RAVEN MATLAB. --- @@ -198,7 +198,7 @@ Empty names (`name: ''`) are not emitted (matches RAVEN MATLAB's behaviour). - m: mitochondrion ``` -An `!!omap` of `: ` pairs. Compartments carry no MIRIAM annotations of their own in the format. +The block is an `!!omap` of `: ` pairs. Compartments carry no MIRIAM annotations of their own in the format. --- @@ -221,9 +221,9 @@ An `!!omap` of `: ` pairs. Compartments carry n - sourceUrl: https://github.com/SysBioChalmers/yeast-GEM ``` -The block holds provenance. Cobrapy ignores the block. raven-toolbox keeps the verbatim dictionary on `model.notes['metaData']` and lifts `id`, `name` and `version` to `model.id`, `model.name` and `model.notes['version']`, where cobra-shape accessors find them. RAVEN MATLAB populates `model.id`, `model.name`, `model.version` and `model.annotation.*` from the same fields. +The block holds provenance. cobrapy ignores it. raven-toolbox keeps the verbatim dictionary in `model.notes['metaData']` and lifts `id`, `name` and `version` to `model.id`, `model.name` and `model.notes['version']`, where the accessors of cobra find them. RAVEN MATLAB fills `model.id`, `model.name`, `model.version` and `model.annotation.*` from the same fields. -`date` is preserved across round-trips when present on the model. Otherwise the writer fills in `YYYY-MM-DD` of the current date. +`date` is preserved across round trips when the model has it. Otherwise the writer fills in the current date as `YYYY-MM-DD`. --- @@ -251,20 +251,20 @@ Enzyme-constrained models carry the EC layer in three additional top-level keys: - concs: .nan ``` -The sections map onto `model.ec` in RAVEN MATLAB and `raven_toolbox.io.ec_data.EcData` (attached as `model.ec`) in raven-toolbox. Cobrapy ignores the sections. +The sections map to `model.ec` in RAVEN MATLAB and to `raven_toolbox.io.ec_data.EcData` (attached as `model.ec`) in raven-toolbox. cobrapy ignores them. -An `- enzymes: !!omap` line with nothing under the tag is read as a reaction with no enzymes. +An `- enzymes: !!omap` line with nothing under the tag is read as a reaction without enzymes. -The spelling `geckoLight` inside `metaData` is also accepted on read. +The spelling `geckoLight` inside `metaData` is also accepted on reading. --- ## Annotations -The `annotation` block uses MIRIAM-style namespace keys. Cobrapy treats the block as a free-form dictionary, raven-toolbox preserves it verbatim through `cobra.Metabolite.annotation`, `Reaction.annotation` and `Gene.annotation`, and RAVEN MATLAB maps it to `model.metMiriams`, `rxnMiriams` and `geneMiriams`. +The `annotation` block uses MIRIAM-style namespace keys. cobrapy treats the block as a free-form dictionary. raven-toolbox preserves it verbatim in `cobra.Metabolite.annotation`, `Reaction.annotation` and `Gene.annotation`, and RAVEN MATLAB maps it to `model.metMiriams`, `rxnMiriams` and `geneMiriams`. - A single value is written inline: `kegg.compound: C00002`. -- Multiple values are written as a YAML list: +- Several values are written as a YAML list: ```yaml - chebi: @@ -272,16 +272,16 @@ The `annotation` block uses MIRIAM-style namespace keys. Cobrapy treats the bloc - CHEBI:30616 ``` -- The `smiles` key inside a metabolite's `annotation` carries the SMILES string (cobrapy convention). Both readers also accept `smiles` as a metabolite top-level field, and the writers use the annotation block. -- The `sbo` key carries the Systems Biology Ontology term assigned by `assignSBOterms` / `add_sbo_terms`. +- The `smiles` key in the `annotation` of a metabolite holds the SMILES string (convention of cobrapy). Both readers also accept `smiles` as a top-level field of a metabolite, and the writers use the annotation block. +- The `sbo` key holds the Systems Biology Ontology term that `assignSBOterms` or `add_sbo_terms` assigns. --- ## Numbers, strings and quoting -Whole-number floats are written with an explicit `.0` (`1000.0`, `-1000.0`, `0.0`). Other floats use up to 15 significant digits (`-17.39`, `-2768.1`). `NaN` is encoded as `.nan`, and `+Inf` and `-Inf` as `.inf` and `-.inf` (YAML 1.2 conventions). +Floats with a whole-number value are written with an explicit `.0` (`1000.0`, `-1000.0`, `0.0`). Other floats use up to 15 significant digits (`-17.39`, `-2768.1`). `NaN` is encoded as `.nan`, and `+Inf` and `-Inf` as `.inf` and `-.inf` (YAML 1.2 conventions). -The default string style is bare (no quotes). raven-toolbox and RAVEN MATLAB apply the same rules and use double-quoted style when the value: +The default string style is bare, without quotes. raven-toolbox and RAVEN MATLAB apply the same rules and use the double-quoted style when the value: - is empty; - starts with one of `#`, `,`, `[`, `]`, `{`, `}`, `&`, `*`, `!`, `|`, `>`, `'`, `"`, `%`, `@` or `` ` ``; @@ -292,9 +292,9 @@ The default string style is bare (no quotes). raven-toolbox and RAVEN MATLAB app The characters `[`, `]`, `{` and `}` inside a value do not trigger quoting. -In a double-quoted string, only `\` and `"` are escaped. Other characters (including Unicode, and newlines if the underlying model permits them) are passed through. +In a double-quoted string, `\` and `"` are escaped. Other characters, including Unicode and newlines where the model permits them, are passed through. -The raven-toolbox reader repairs three quoted shapes that are not valid YAML: `""text""`, a double-quoted scalar with an unescaped inner quote (`"a "b" c"`), and a single-quoted scalar with an inner apostrophe (`'it's'`). The values are `"text"`, `a "b" c` and `it's`, as `readYAMLmodel` reads them. The repair runs when the strict parse fails and does not handle a comment after the closing quote. +The reader of raven-toolbox repairs three quoted shapes that are not valid YAML: `""text""`, a double-quoted scalar with an unescaped inner quote (`"a "b" c"`), and a single-quoted scalar with an inner apostrophe (`'it's'`). The values are `"text"`, `a "b" c` and `it's`, as `readYAMLmodel` reads them. The repair runs when the strict parse fails, and it does not handle a comment after the closing quote. --- @@ -302,21 +302,21 @@ The raven-toolbox reader repairs three quoted shapes that are not valid YAML: `" | File written by (rows) and reader (columns) | cobrapy | raven-toolbox | RAVEN MATLAB | |---|---|---|---| -| cobrapy (`save_yaml_model`) | full | full, with extras in `notes` via attribute fall-through | full, including root-level `id` / `name` / `version` | -| raven-toolbox (`write_yaml_model`) | core (no `metaData`-derived `id`); RAVEN extras are unknown top-level keys that do not break parsing | full | full | -| RAVEN MATLAB (`writeYAMLmodel`) | core (no `metaData`-derived `id`); RAVEN extras via attribute fall-through | full | full | +| cobrapy (`save_yaml_model`) | full | full, with the extras in `notes` | full, including root-level `id`, `name` and `version` | +| raven-toolbox (`write_yaml_model`) | core (no `metaData`-derived `id`); RAVEN extras are unknown keys inside the entries, and they do not break parsing | full | full | +| RAVEN MATLAB (`writeYAMLmodel`) | core (no `metaData`-derived `id`); RAVEN extras through attribute fall-through | full | full | -"Full" means that every field is read back into its canonical position on the model object. "Core" means that cobrapy-known fields are read and RAVEN extensions are ignored or kept on the object as attribute fall-through (for example `reaction.eccodes`, not re-emitted on save). A round-trip through cobrapy is therefore lossy for RAVEN extensions: only the core fields survive `cobrapy.load → cobrapy.save`. Round-trips through raven-toolbox or RAVEN MATLAB are lossless. +"Full" means that every field is read back into its canonical position on the model object. "Core" means that the fields known to cobrapy are read, and the RAVEN extensions are ignored or kept on the object as attributes (for example `reaction.eccodes`, which is not written again on saving). A round trip through cobrapy is therefore lossy for the RAVEN extensions: only the core fields survive `cobrapy.load` followed by `cobrapy.save`. Round trips through raven-toolbox or RAVEN MATLAB are lossless. -Neither writer reproduces cobrapy's own `ruamel`-default layout byte-for-byte (indentation, quoting style, line folding). RAVEN and raven-toolbox share a layout with each other. The shared layout follows the structure of a cobrapy file and differs from it in details such as indentation and quoting. [raven-gecko-parity's reconciliation record](https://github.com/CompMetEng/raven-gecko-parity/blob/develop/docs/yaml-reconciliation.md) gives the reason and the one scenario (`yaml_roundtrip_smallyeast`) that continuously checks that the two writers agree. +Neither writer reproduces the default layout of `ruamel` in cobrapy byte for byte (indentation, quoting style, line folding). RAVEN and raven-toolbox share a layout with each other. The shared layout follows the structure of a cobrapy file and differs from it in details such as indentation and quoting. The [reconciliation record of raven-gecko-parity](https://github.com/CompMetEng/raven-gecko-parity/blob/develop/docs/yaml-reconciliation.md) gives the reason and the one scenario (`yaml_roundtrip_smallyeast`) that continuously checks that the two writers agree. --- ## Round-trip content counts -Loading `yeast-GEM.yml` (2748 metabolites, 4102 reactions, 1143 genes) and re-writing it through any of the three tools preserves every documented piece of content: +`yeast-GEM.yml` (2748 metabolites, 4102 reactions, 1143 genes) is loaded and written again through raven-toolbox or RAVEN MATLAB. Every documented piece of content is preserved: -| Count | After round-trip | +| Count | After round trip | |---|---| | metabolites | 2748 / 2748 | | reactions | 4102 / 4102 | @@ -327,4 +327,4 @@ Loading `yeast-GEM.yml` (2748 metabolites, 4102 reactions, 1143 genes) and re-wr | metabolites with SMILES | 1788 | | reactions with notes (rxnNotes) | 1443 | -Cobrapy round-trips give 2748 / 4102 / 1143 for the core and drop the RAVEN extensions in the rightmost column, which is the documented loss. +Round trips through cobrapy give 2748 / 4102 / 1143 for the core and drop the RAVEN extensions in the rightmost column, which is the documented loss. diff --git a/scripts/known_names.txt b/scripts/known_names.txt index 77f5744..53ea1be 100644 --- a/scripts/known_names.txt +++ b/scripts/known_names.txt @@ -479,3 +479,6 @@ xml_unblank tBinaries tParseRAVENargs regexpPattern +removeUnusedMets +complex_policy +minNrFluxes