diff --git a/DESIGN.md b/DESIGN.md index f8de2cb..8c95313 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -36,7 +36,7 @@ One documentation site for **GECKO**, the MATLAB toolbox, and **geckopy**, its P - **Home** (`docs/index.md`): one sentence on GECKO and geckopy, the install tabs for both languages, key features, quick start, links to the papers. - **Installation** (`docs/installation/`). -- **Guide** (`docs/guide/`): *Start* (Introduction, Installation), *Guide* (ten task pages from getting started to enzyme usage and bottlenecks), *Background* (GECKO light, the YAML format, the files of the data folder, the model adapter parameters). The Introduction and the Background pages explain a concept or a format and are marked ``, so they need no function table. +- **Guide** (`docs/guide/`): *Start* (Introduction, Installation), *Quick path* (Build an ecModel: a working ecModel with the default settings), *Decisions* (one page per choice the user makes, in the order the user meets it: the ecModel structure, the kcat sources, custom kcats, the protein pool, the kcat tuning, the advanced kcat tuning, proteomics, relaxing proteomics, analysis, engineering targets, context-specific ecModels, enzyme usage and bottlenecks), *Background* (the YAML format, the files of the data folder, the model adapter parameters, basic model operations). A decision page sets out the options (what each needs and changes), gives a recommendation where a source supports one, and then the procedure of each option. The guide describes GECKO 4 only; the migration pages describe the changes against GECKO 3. The Introduction and the YAML, data folder and model adapter pages explain a concept or a format and are marked ``, so they need no function table. - **Migration**: `migrate.md`, `gecko3-to-gecko4.md` and `gecko-to-geckopy.md`. - **API reference** (`docs/api/`): a hand-written index, then generated MATLAB pages by folder and Python pages by package, and `docs/matlab-vs-python.md`. - **References** (`docs/references.md`): the papers to cite. @@ -58,7 +58,8 @@ One documentation site for **GECKO**, the MATLAB toolbox, and **geckopy**, its P - **Unresolved names.** `scripts/check_names.py` reports 121 distinct names on the guide pages that are neither a top-level function or class of the toolboxes nor in `scripts/known_names.txt`. Most are keyword arguments, struct fields, YAML keys and variables of the examples; four (`cap_usage`, `from_folder`, `shadow_price`, `upper_bound`) are methods or properties. Someone who knows the pages has to sort them, and allow-list the ones that are not functions, before the check can block CI. - **Link check.** `docs-build.yml` does not fail on the link check yet; the state of the external links was not known when the check was added. -- **Guide page sections.** Most guide pages have no "What can go wrong" section and no function table; `scripts/check_site.py` reports this as a warning. +- **Site scripts.** `scripts/check_site.py` and `scripts/autolink.py` differ from `docs-site/scripts` on main (the expected section names), and `check_style.py` and `run_examples.py` are missing. Until they are copied, `check_site.py` reports the section names "Functions on this page" and "What can go wrong" as missing on pages that use "Functions" and "Troubleshooting". +- **Examples are not run.** The code blocks of the guide are written from the source and have not been executed; the site has no `run_examples.py` and no examples workflow yet. ## 8. Decisions log @@ -68,3 +69,8 @@ One documentation site for **GECKO**, the MATLAB toolbox, and **geckopy**, its P - 2026-10-24: the language of the code tabs (MATLAB or Python) is remembered in `localStorage` by `docs/_static/language-sync.js`, so it applies to every tab on a page, to the next pages, to new browser tabs and to later visits, and install tabs follow it. sphinx-design alone keeps the choice in `sessionStorage` for one browser tab. - **Broken links and code owner:** `.github/workflows/link-issue.yml` checks the external links every Wednesday and keeps one open issue labelled `broken-links` with the broken ones; it closes the issue when none is broken. `.github/CODEOWNERS` names `@edkerk` as the owner of the repository, who is asked to review every pull request. - **Rules for every session:** `CLAUDE.md` holds the baseline rules of the `craft` plugin, `.claude/settings.json` enables the `craft` and `docs-site` plugins and turns on the hooks in `.githooks`, and `.github/workflows/repo-hygiene.yml` runs `.github/scripts/repo_hygiene.py` on every pull request. They refuse a tool author, trailer, mention, branch name or pull request footer. +- 2026-11-11: the guide no longer follows the order of the GECKO 3 Nature Protocols pipeline. GECKO 4 is configured by `model_adapter.toml`, and `assignKcats` replaces the manual steps of gathering, merging and applying kcats, so most of the work is choosing settings. The guide has one quick-path page and one page per decision; each page was written from the code, not edited from the earlier pages. +- 2026-11-11: the workflow diagrams (`docs/_static/workflow-*.pdf`, `workflow-overview.png`, the Mermaid page `guide/workflow.md`) and the table of approximate timings are removed. The diagrams showed the order of the GECKO 3 protocol, and the timings had no source. +- 2026-11-11: the guide calls `sensitivityTuning` the standard kcat tuning (it needs one maximum growth rate) and `cmaesKcatTuning` the advanced kcat tuning (it fits maximum growth rates and exchange fluxes of one or more conditions). They are alternatives; neither is called legacy. The API category of `src/kcat_tuning` is named kcat tuning. +- 2026-11-11: DLKcat predictions are described as coming from the DLKcat method of OpenKineticsPredictor, which `kcatsFromDLKcat` uses when `data/DLKcat.tsv` does not exist; the local Docker run is described as the older method in one admonition. +- 2026-11-11: the guide describes KinForm-H as the default OpenKineticsPredictor method and the presets' `okp` stage with KinForm-H alone, as GECKO and geckopy set it. Where the guide cites the measured comparison of the presets, it states that the comparison combined eight OKP methods. Functions that GECKO deprecates (`updateProtPool`, `mergeDLKcatAndFuzzyKcats`, adapter classes) appear on guide pages only in troubleshooting, and on the migration page. diff --git a/docs/_static/workflow-matlab.pdf b/docs/_static/workflow-matlab.pdf index fbbfba9..4349946 100644 Binary files a/docs/_static/workflow-matlab.pdf and b/docs/_static/workflow-matlab.pdf differ diff --git a/docs/_static/workflow-overview.png b/docs/_static/workflow-overview.png index 4820a80..0780c36 100644 Binary files a/docs/_static/workflow-overview.png and b/docs/_static/workflow-overview.png differ diff --git a/docs/_static/workflow-python.pdf b/docs/_static/workflow-python.pdf index 8536c1b..a96e2c9 100644 Binary files a/docs/_static/workflow-python.pdf and b/docs/_static/workflow-python.pdf differ diff --git a/docs/gecko-to-geckopy.md b/docs/gecko-to-geckopy.md index b09e86c..dbce057 100644 --- a/docs/gecko-to-geckopy.md +++ b/docs/gecko-to-geckopy.md @@ -49,8 +49,6 @@ The table is a selection; [MATLAB and Python](matlab-vs-python.md) lists all pai | `reviewKcatAssignment` | `review_assignment` | | `upgradeGECKOproject` | `upgrade_project`, command `geckopy upgrade` | | `createGECKOproject` | command `geckopy init` | -| `writeOpenKineticsPredictorInput` | `build_okp_input_csv` in `geckopy.gather_kcats.open_kinetics_predictor` | -| `readOpenKineticsPredictorOutput` | `parse_okp_output` in `geckopy.gather_kcats.open_kinetics_predictor` | ## Functions of one implementation @@ -71,7 +69,7 @@ The table is a selection; [MATLAB and Python](matlab-vs-python.md) lists all pai |---|---| | Result and data types | `TunedKcatsResult`, `FlexEnzResult`, `BrendaData`, `PhylDist`, `FluxData`, `Enzyme` | | Build steps of `make_ec_model` | `geckopy.ec_model.pipeline.add_protein_usage_reactions` and the other steps that `makeEcModel` runs inline | -| Algorithms | `relax_enz_concs` (least total fold increase of the enzyme bounds in one linear program), `level_complex_abundances` (one abundance per complex), `sweep_tunable_mask_size` and `recommend_target_impact_share` (share of kcats that `cmaes_kcat_tuning` tunes) | +| Algorithms | `level_complex_abundances` (one abundance per complex), `sweep_tunable_mask_size` and `recommend_target_impact_share` (share of kcats that `cmaes_kcat_tuning` tunes) | ## Differences in behaviour @@ -79,7 +77,7 @@ The scenarios of the [raven-gecko-parity](https://github.com/CompMetEng/raven-ge - **`ecFSEOF` and `ec_fseof`** return the same target reactions on ecTestGEM. On the ecYeastGEM of the tutorial the order of targets differs in 27 positions, and the set of targets depends on the solver pair, because the pFBA solution that the regression uses is not unique. On ecYeastGEM 9.1.1 the order is the same. - **`upgradeGECKOproject` and `upgrade_project`** differ for a project that has an adapter class and no `model_adapter.toml`. `upgradeGECKOproject` converts the class. `upgrade_project` lists the adapter for conversion by hand, because an adapter class is MATLAB code, and then skips the conversion of the custom kcats and the stored predictions. -- **`sigmaFitter` and `fit_sigma`** scan the same grid of 0.01 steps. `sigmaFitter` returns the smallest sigma within a relative 1e-9 of the smallest error, and `fit_sigma` takes the first grid point with the smallest error, so solver rounding noise on a growth plateau can move its result to a later grid point. `fit_sigma` with `method="bisect"` returns a continuous sigma. +- **`sigmaFitter` and `fit_sigma`** scan the same grid of 0.01 steps. `sigmaFitter` returns the smallest sigma whose growth rate differs from the target by at most the smallest difference plus 1e-9 times the growth rate at the smallest difference, and `fit_sigma` takes the first grid point with the smallest error, so solver rounding noise on a growth plateau can move its result to a later grid point. `fit_sigma` with `method="bisect"` returns a continuous sigma. - **`saveEcModel` and `save_ec_model`** write files that differ in charge, SBO terms, gene names and the subsystem of the protein usage reactions, because the models in memory differ in these fields; the layout of the files agrees. ## See also diff --git a/docs/gecko3-to-gecko4.md b/docs/gecko3-to-gecko4.md index c3ad91d..1a9c9f2 100644 --- a/docs/gecko3-to-gecko4.md +++ b/docs/gecko3-to-gecko4.md @@ -11,7 +11,9 @@ A GECKO 3 script keeps running after each change in this box and returns a diffe - `fuzzyKcatMatching` divides a BRENDA kcat by the coefficient of the substrate the kcat was measured on. GECKO 3 divided by the smallest coefficient in the reaction. - `assignKcats` and `collectKcats` use the preset `brenda_ec_tree` when `model_adapter.toml` sets no other. - `calculateFfactor` averages each protein over the samples in which it is measured. -- `sigmaFitter` returns the smallest sigma whose percent error is within a relative 1e-9 of the smallest error. The 1e-9 tolerance is not part of GECKO 3. geckopy `fit_sigma` takes the first grid point with the smallest error (default `method="grid"`) or a continuous sigma (`method="bisect"`). +- `sigmaFitter` returns the smallest sigma within a tolerance of the smallest error, and the returned ecModel has the protein pool of that sigma (see below the first table). +- `flexibilizeEnzConcs` keeps an upper bound that is higher than the new bound, and stops after a total number of iterations. + ::: ## Changes by consequence @@ -24,22 +26,45 @@ A GECKO 3 script keeps running after each change in this box and returns a diffe | Substrate of a BRENDA kcat | the kcat is divided by the smallest coefficient in the reaction | the kcat is divided by the coefficient of the substrate that matched | Rebuild the kcat list. Reactions that are not written per the measured substrate get a lower kcat | | Default kcat assignment | `selectKcatValue` (`assignKcatValues` in GECKO 4) merges a DLKcat list and a BRENDA list | `assignKcats` determines the kcat from the kcat source files in `data/kcats` with the preset `brenda_ec_tree` | Set `preset = "gecko_classic"` in `[kcat_assignment]` to keep the GECKO 3 order (see [Kcat assignment](gecko4-kcat-assignment.md)) | | Protein fraction `f` | `calculateFfactor` averages over all samples, so a protein missing in one sample is left out of both totals | the average is over the samples in which the protein is measured | Recalculate `f` when the proteomics file has missing values | -| Sigma fit | `sigmaFitter` returns the sigma with the smallest error | it returns the smallest sigma whose percent error is within a relative 1e-9 of the smallest error (geckopy `fit_sigma`: no tolerance in the default grid) | Refit when the growth rate is flat over a range of sigma | +| Enzyme bounds of `flexibilizeEnzConcs` | the new upper bound of an enzyme replaces the old bound, also when the old bound is higher; the iterations have no total limit | an upper bound is never lowered, and the iterations have a total limit (see below the table) | Check the warnings and `flexEnz` | +| File name of `loadConventionalGEM` | `.xml` is appended to every name | a name ending in `.yml` is loaded as YAML; `.xml` is appended to a name without `.xml` or `.yml` | none | +| Sigma fit | `sigmaFitter` returns the sigma with the smallest error, and an ecModel whose protein pool is set for sigma 1 | it returns the smallest sigma within a tolerance of the smallest error (see below the table), and an ecModel whose protein pool is set for that sigma | Refit when the growth rate is flat over a range of sigma; a script that called `setProtPoolSize` after `sigmaFitter` gets the same pool | + +`flexibilizeEnzConcs` stops and prints a message after a total of (`iterPerEnzyme` + 1) Γ— (number of measured enzymes) iterations, with 1000 for `iterPerEnzyme` when it is 0. On an infeasible input model, it warns with the identifier `flexibilizeEnzConcs:infeasible` and returns `flexEnz` with empty fields. `sigmaFitter` returns the smallest sigma whose growth rate differs from the target by at most the smallest difference plus 1e-9 times the growth rate at the smallest difference; geckopy `fit_sigma` has no tolerance in its default grid. ### Changes that raise an error +The changes in this table, and calls of removed functions (see [Removed](#removed)), stop a GECKO 3 script. + | Change | Condition | Action | | --- | --- | --- | | `loadProtData` checks the replicate count | the sum of `replPerCond` differs from the number of replicate columns of the file | Correct `replPerCond` | | `loadEcModel` checks the ecModel format | `gecko_format` in the file is higher than 4 | Update GECKO | | Adapter keys | `model_adapter.toml` has a key that is not in `adapterParameters.tsv` | Remove or rename the key, or place a key that GECKO uses and geckopy does not use under `[matlab.params]` | | Options of `assignKcats` and `upgradeGECKOproject` | an option name that the function does not have | Correct the option name | +| `runDLKcat`, `writeDLKcatInput` | every call in MATLAB: the functions are deprecated and raise an error | Use `kcatsFromDLKcat(ecModel, 'via', 'docker')` | +| `sigmaFitter` raises `sigmaFitter:growthRate` | the target growth rate is not above 0 | Give a growth rate above 0 | +| `setProtPoolSize` raises `setProtPoolSize:noPoolExchange` | the model has no `prot_pool_exchange` (GECKO 3 left the model unchanged) | Give an ecModel made by `makeEcModel` | +| `getConcControlCoeffs` raises `getConcControlCoeffs:infeasible` | the model has no optimal solution | Make the model feasible first | +| `collectKcats` and `assignKcats` raise `collectKcats:legacyCustomKcats` | a stage lists `custom`, `data/kcats/custom.tsv` does not exist, and `data/customKcats.tsv` does | Run `convertCustomKcats(ecModel)` | ### Changes of the project -A GECKO 3 project is configured by an adapter class and keeps its kcats in `data/DLKcat.tsv`, `data/customKcats.tsv` and in the variables of the script. A GECKO 4 project is configured by `model_adapter.toml`, keeps each kcat source in its own file in `data/kcats`, and records the ecModel format in the saved model. [Project format 4](gecko4-project-format.md) describes the files, and [Upgrade a GECKO 3 project](gecko3-upgrade-project.md) the steps that convert a project. +A GECKO 3 project is configured by an adapter class and keeps its kcats in `data/DLKcat.tsv`, `data/customKcats.tsv` and in the variables of the script. A GECKO 4 project is configured by `model_adapter.toml`, keeps each kcat source in its own file in `data/kcats`, and records the ecModel format in the saved model. `assignKcats` writes its decisions to `output/_kcatDecisions.tsv`, with column names in `snake_case` (`ec_rxn`, `spread_log10`, `sd_log10`); the table that `collectKcats` returns keeps `camelCase` names. [Project format 4](gecko4-project-format.md) describes the files, and [Upgrade a GECKO 3 project](gecko3-upgrade-project.md) the steps that convert a project. + +Functions with name-value arguments also take them positionally, in the GECKO 3 order: `loadEcModel(filename, modelAdapter)` and `loadEcModel('filename', filename)` are equivalent. A call uses one of the two forms for all its optional arguments. -Adapter classes keep working. Functions with name-value arguments also take them positionally, in the GECKO 3 order: `loadEcModel(filename, modelAdapter)` and `loadEcModel('filename', filename)` are equivalent. A call uses one of the two forms for all its optional arguments. +### Adapter classes + +A GECKO 3 adapter class keeps working. It sets `obj.params` with the MATLAB names of [Model adapter parameters](guide/model-adapter-parameters.md), and `ModelAdapterManager.setDefault` takes the path of its `.m` file. `convertAdapterToToml` writes the parameters of a class as `model_adapter.toml` and lists the methods that the class overrides and the file cannot express. The option `'adapterFormat','class'` of `createGECKOproject` creates a project with a class; both the option and adapter classes as project format are deprecated (see [Deprecated](#deprecated)). + +A class that needs code beyond the file can read its parameters from the file in the constructor: + +```matlab +obj.params = loadAdapterParams(fullfile(projectPath, 'model_adapter.toml')); +``` + +In geckopy, a subclass of `ModelAdapter` overrides `get_spontaneous_reactions`, `get_uniprot_compatible_genes`, `get_uniprot_ids_from_table`, `get_kegg_compatible_genes`, `get_brenda_db_folder` or `get_phyl_dist_path`. It is instantiated explicitly; `from_folder` does not discover it. ## New functions @@ -52,31 +77,51 @@ The table lists a selection. | Databases | `fetchBrendaAsset`, `loadBrendaMeasurements`, `resolveKcatAggregation` | | Project and model upgrade | `upgradeGECKOproject`, `upgradeEcModel`, `convertAdapterToToml` | | Adapter | `TomlAdapter`, `readAdapterToml`, `writeAdapterToml`, `loadAdapterParams` | +| Kcat tuning | `cmaesKcatTuning`, `screenKcatLeverage`, `selectTunableMask`, `reviewKcatAssignment`, `tunePriorPenaltyWeight`, `loadEvotuneData` | | Proteomics | `levelComplexConcs`, `getComplexFamilies`, and the option `perComplex` of `flexibilizeEnzConcs` | -| Analysis | `getEnzymeBottlenecks`, `relaxProteomicsGreedy` | +| Analysis | `getEnzymeBottlenecks`, `relaxProteomicsGreedy`, `relaxEnzConcs`, and the option `method` of `getConcControlCoeffs` | `getEnzymeBottlenecks` solves the model again after raising the usage bound of each enzyme that runs at its bound, and ranks the enzymes by the rise of the objective. The shadow price is reported and does not enter the order. +`relaxProteomicsGreedy` returns `[result, model]`, with the relaxed model as second output. `relaxEnzConcs` raises the enzyme bounds with the least total fold increase in one linear program. + `levelComplexConcs` gives the subunits of a multi-subunit complex one abundance. The complex is then not capped by its least abundant subunit. It runs after `fillEnzConcs` and before `constrainEnzConcs`. -kcat prediction goes through the hosted OpenKineticsPredictor service, in addition to a local DLKcat installation. The DLKcat Docker context is not part of the GECKO repository (`src/dlkcat-gecko` is removed). +kcat prediction goes through the hosted OpenKineticsPredictor (OKP) service, through `kcatsFromOKP`. The default OKP method (`[okp] method`) is KinForm-H, and the `okp` stage of the presets uses the methods of `[kcat_sources.okp] methods`, by default KinForm-H alone. `kcatsFromDLKcat` uses its DLKcat method unless `data/DLKcat.tsv` exists, or the option `'via', 'docker'` selects a local run in the Docker image. The DLKcat Docker context is not part of the GECKO repository (`src/dlkcat-gecko` is removed). ## Removed | Removed | Replacement | | --- | --- | | BRENDA files in `databases/` | `fetchBrendaAsset` downloads the files of the toolbox-assets release `brenda-2026.1-1` into `databases/brenda-2026.1-1` on first use. A project keeps its own files in `data/brenda` | +| `truncateValues` | `round(x, 6, 'significant')` | +| `findMaxValue` | `kcatsFromBrenda` or `kcatsFromECTree` | +| `bayesianSensitivityTuning` and its helpers `abc_max`, `getrSample`, `updateprior` (Bayesian ABC-SMC tuning of light ecModels) | `cmaesKcatTuning`, the advanced kcat tuning, with the data files `data/evotune*.tsv` (see [Tune kcats against experimental data](guide/advanced-tuning.md)) | +| `getcarbonnum` | `fillCarbonNum` writes the carbon number of each exchange reaction to `model.excarbon` | +| `anaerobicModel_GECKO`, `changeMedia` | copies in `tutorials/full_ecModel/code`, for *Saccharomyces cerevisiae* | + +The removed functions in the table, except `truncateValues`, `anaerobicModel_GECKO` and `changeMedia`, have a stub in `src/deprecated` that raises an error naming the replacement. `upgradeGECKOproject` lists the lines of a project's scripts that call them. + +## Deprecated + +`updateProtPool`, `applyCustomKcats`, `runDLKcat`, `writeDLKcatInput`, `readDLKcatOutput` and the option `'adapterFormat', 'class'` of `createGECKOproject` print a deprecation warning, and so does the first adapter class loaded in a MATLAB session, unless it subclasses `TomlAdapter`. Everything in the table is removed in a future release. + +| Deprecated | Replacement | +| --- | --- | +| `updateProtPool` | `setProtPoolSize` with the protein content of the condition. From GECKO 3.2.0 measured enzymes draw from the protein pool, and `updateProtPool` raises an error on such an ecModel | +| `applyCustomKcats` | `convertCustomKcats` writes `data/customKcats.tsv` as `data/kcats/custom.tsv`, which `collectKcats` and `assignKcats` read | +| `runDLKcat`, `writeDLKcatInput` (raise an error), `readDLKcatOutput` (still reads the file) | `kcatsFromDLKcat` with `'via', 'docker'` or `'via', 'file'` | +| Adapter classes as project format: `createGECKOproject(..., 'adapterFormat', 'class')` and `adapterTemplate.m` | `model_adapter.toml`. Existing adapter classes still load with a deprecation warning, and `convertAdapterToToml` converts them | ## Renamed functions -The old name of each function in the table calls the new function and prints a deprecation warning. Seven of the old names are in `src/deprecated`; `mergeDLKcatAndFuzzyKcats` is a wrapper of `mergeKcats` in `src/gather_kcats`. The old names are removed in a future release. +The old name of each function in the table calls the new function and prints a deprecation warning. The old names are in `src/deprecated` and are removed in a future release. | GECKO 3 name | GECKO 4 name | Reason | | --- | --- | --- | | `startGECKOproject` | `createGECKOproject` | | | `copyECtoGEM` | `applyECcodes` | | | `enzymeUsage` | `getEnzymeUsage` | | -| `pfbaEnzymes` | `getPfbaEnzymes` | | | `addCarbonNum` | `fillCarbonNum` | | | `updateGECKOdoc` | `buildGECKOdoc` | | | `selectKcatValue` | `assignKcatValues` | the function writes `ecModel.ec.kcat` | @@ -116,7 +161,7 @@ The Add-On manager distributes the latest release. Uninstall the Add-On (`GECKOI GECKO 1 ([SΓ‘nchez et al., 2017](references.md#gecko-1)) and GECKO 2 ([Domenzain et al., 2022](references.md#gecko-2)) share one ecModel structure, which differs from GECKO 3 and GECKO 4: - Enzyme and kcat information is spread over several model fields. From GECKO 3, `ecModel.ec` (Python: `ec_model.ec`) collects it. -- Enzymes enter the S-matrix as `1/kcat`, with the molecular weight in the protein exchange reactions. From GECKO 3 the coefficient is `MW/kcat` (see [Building an empty ecModel](guide/building-ec-model.md)). +- Enzymes enter the S-matrix as `1/kcat`, with the molecular weight in the protein exchange reactions. From GECKO 3 the coefficient is `MW/kcat` (see [Choose the ecModel structure](guide/ecmodel-structure.md)). - The YAML format of GECKO 3 and GECKO 4 retains the full model content. GECKO 1 and GECKO 2 have no such format. Functions of GECKO 3 and GECKO 4 do not work on a GECKO 1 or GECKO 2 ecModel, and the reverse. geckopy supports GECKO 4 ecModels. diff --git a/docs/gecko4-kcat-assignment.md b/docs/gecko4-kcat-assignment.md index a5d25a6..8d70e18 100644 --- a/docs/gecko4-kcat-assignment.md +++ b/docs/gecko4-kcat-assignment.md @@ -2,7 +2,7 @@ -GECKO 4 assigns the kcat of each reaction from kcat source files, in an order that `model_adapter.toml` sets. `collectKcats` determines the kcats and does not change the model. `assignKcats` applies the decisions and the kcat constraints. [Gathering kcats](guide/gathering-kcats.md) describes how the sources are gathered, and [Applying kcats](guide/applying-kcats.md) the steps after the assignment. +GECKO 4 assigns the kcat of each reaction from kcat source files, in an order that `model_adapter.toml` sets. `collectKcats` determines the kcats and does not change the model. `assignKcats` applies the decisions and the kcat constraints. [Choose the kcat sources](guide/kcat-sources.md) describes how the sources are gathered and assigned, and [Set the protein pool](guide/protein-pool.md) the step after the assignment. ## Functions @@ -17,12 +17,12 @@ GECKO 4 assigns the kcat of each reaction from kcat source files, in an order th ## Sources -A source is the name of a file in `data/kcats` (`custom`, `ec_tree`, `homologs`, `dlkcat`, `okp_`), or one of the BRENDA quality tiers `database_exact`, `database_top` and `database_bottom`, which read `brenda.tsv`. In a stage, `okp` stands for the OpenKineticsPredictor methods of `[kcat_sources.okp] methods`, which default to all methods that have combination parameters. +A source is the name of a file in `data/kcats` (`custom`, `ec_tree`, `homologs`, `dlkcat`, `okp_`), or one of the BRENDA quality tiers `database_exact`, `database_top` and `database_bottom`, which read `brenda.tsv`. In a stage, `okp` stands for the OpenKineticsPredictor (OKP) methods of `[kcat_sources.okp] methods`, which default to KinForm-H alone. | Source | Content | | --- | --- | | `custom` | curated kcats of the project | -| `brenda.tsv` | BRENDA kcats matched by EC number and substrate, with the origin of the match (1 to 6). A stage reads it through the tiers `database_exact`, `database_top` and `database_bottom`, not as `brenda` | +| `brenda.tsv` | BRENDA kcats matched by EC number and substrate, with the origin of the match (1 to 6). A stage reads it through the tiers `database_exact`, `database_top` and `database_bottom` | | `ec_tree` | the median of all wild-type BRENDA measurements of the EC number, shrunk towards the median of its 3-digit EC class, its 2-digit EC class and the global median (weight `n/(n+k)` for `n` measurements, `k = 5`) | | `homologs` | the kcat of measured homologs that act on the same substrate, found with DIAMOND (at least 40 % identity and 80 % coverage of both sequences) | | `dlkcat` | DLKcat predictions | @@ -32,7 +32,7 @@ A source is the name of a file in `data/kcats` (`custom`, `ec_tree`, `homologs`, A stage determines the kcat of the reactions that are still open. The action `select` takes the first listed source that has a value. The action `combine` takes all listed sources that have a value, corrects each for the bias of the source and weights it by 1 divided by its error variance. The error variance depends on the evidence of the source for the reaction: for the EC tree the measurements of the EC number and of its EC class, for an OKP method the similarity of the enzyme to the training sequences of the method. The parameters are in `kcatCombineParams.tsv` of the BRENDA files. `model.ec.source` reads `combined (sd=, n=)` for a combined value. -After the stages, `collectKcats` applies `data/customKcats.tsv` when a stage lists `custom` and `data/kcats/custom.tsv` does not exist. With `isozymes = true`, a reaction without a kcat in a full ecModel takes one from its isozymes. With `standard = true`, the remaining reactions get the standard kcat. +When a stage lists `custom`, `data/kcats/custom.tsv` does not exist and the GECKO 3 file `data/customKcats.tsv` does, `collectKcats` raises the error `collectKcats:legacyCustomKcats`; `convertCustomKcats` converts the file. After the stages, with `isozymes = true`, a reaction without a kcat in a full ecModel takes one from its isozymes. With `standard = true`, the remaining reactions get the standard kcat. ## Presets @@ -40,11 +40,11 @@ After the stages, `collectKcats` applies `data/customKcats.tsv` when a stage lis | Preset | Stages | | --- | --- | -| `brenda_ec_tree` | custom kcats; the BRENDA median of the measurements of the organism on the substrate of the reaction (origin 1, no EC wildcards); the EC tree; then the OKP methods combined for the reactions that the earlier stages left open | +| `brenda_ec_tree` | custom kcats; the BRENDA median of the organism on the substrate of the reaction (origin 1, no EC wildcards); the EC tree; then the OKP methods for the reactions still open | | `ec_tree_homologs` | custom kcats; the BRENDA value of the organism; the EC tree and the homologs combined; then the OKP methods combined | | `ec_tree_okp` | custom kcats; the BRENDA value of the organism; the EC tree, the homologs and the OKP methods combined; then the OKP methods combined | | `okp` | custom kcats; then the OKP methods combined | -| `gecko_median` | custom kcats; the BRENDA median of matches without EC wildcards (origin 1 to 4); DLKcat; then the other BRENDA matches (matches with EC wildcards, origin 3 to 6, and matches from specific activities without a wildcard, origin 5 and 6) | +| `gecko_median` | custom kcats; BRENDA median without EC wildcards (origin 1 to 4); DLKcat; then the other BRENDA matches (EC wildcards, origin 3 to 6; specific activities, origin 5 and 6) | | `gecko_classic` | as `gecko_median` with the BRENDA maximum and the specific-activity matches above DLKcat (origin up to 6), the order of the GECKO 3 tutorial | The tutorial `full_ecModel` uses `gecko_classic`. A project lists the stages in `[[kcat_assignment.stage]]` tables in place of a preset: @@ -63,7 +63,7 @@ top_origin_limit = 1 A stage has the options `action` (default `select`), `sources`, `criteria` (default `max`), `top_origin_limit` (default 4), `bottom_origin_limit` (default 6) and `wildcard_limit` (default 3). -Against measured kcats hidden from the search, the EC tree was closer than the BRENDA median at origin 2, at origin 4 and with EC wildcards, and as close at origin 1 and 3. The EC tree combined with the OKP methods was slightly closer than the EC tree, and the OKP methods combined were as close. In ecModels, values combined with predictions set high-turnover enzymes such as ATP synthase too low. The default therefore uses predictions where measurements give no value. +Against measured kcats hidden from the search, the EC tree was closer than the BRENDA median at origin 2, at origin 4 and with EC wildcards, and as close at origin 1 and 3. The EC tree combined with the OKP methods was slightly closer than the EC tree, and the OKP methods combined were as close. In ecModels, values combined with predictions set high-turnover enzymes such as ATP synthase too low. The default therefore uses predictions where measurements give no value. The OKP methods of this comparison were eight methods combined (CataPro, DLKcat, EITLEM, KinForm-H, KinForm-L, OmniESI, OmniESI-O2DENet and UniKP); the default of `[kcat_sources.okp] methods` is KinForm-H alone, which the comparison did not test separately. ## Provenance of a kcat @@ -77,7 +77,7 @@ Against measured kcats hidden from the search, the EC tree was closer than the B | `manual` | a kcat that `setKcatForReactions` (Python `set_kcat_for_reactions`) set | | `brenda (wc=0, origin=2) \| tuned=cmaes (prior=6.55)` | a kcat that tuning changed, with the kcat before the first tuning as `prior` | -The origin is the label of the kcat source in lower case with underscores (`brenda`, `ec_tree`, `okp_catapro`). The detail has the pairs `wc` and `origin` for a BRENDA match and `sd` and `n` for a combined value. The assignment writes a new origin and drops a tuning status. `sensitivityTuning` and `cmaesKcatTuning` (Python `sensitivity_tuning` and `cmaes_kcat_tuning`) keep the origin and detail of every kcat they change and add their method, `sensitivity` or `cmaes`, to the tuning status. The `prior` stays the kcat before the first tuning, and a second tuning with the same method adds nothing. +The origin is the label of the kcat source in lower case with underscores (`brenda`, `ec_tree`, `okp_kinform_h`). The detail has the pairs `wc` and `origin` for a BRENDA match and `sd` and `n` for a combined value. The assignment writes a new origin and drops a tuning status. `sensitivityTuning` and `cmaesKcatTuning` (Python `sensitivity_tuning` and `cmaes_kcat_tuning`) keep the origin and detail of every kcat they change and add their method, `sensitivity` or `cmaes`, to the tuning status. The `prior` stays the kcat before the first tuning, and a second tuning with the same method adds nothing. `parseKcatSource` and `formatKcatSource` read and write these entries. The parser returns the origin as it was written, the detail pairs (a number when the value is a plain decimal number, text otherwise), the tuning methods and the prior. An entry that does not follow the form is read as one origin without detail, so an entry that an earlier release wrote stays readable. The formatter writes the origin in lower case with underscores, a number with six significant digits and a prior only together with a tuning status. A detail pair without a value, with an infinite value or with a key that does not start with a letter is left out. An empty origin with a detail or a tuning status is written as `unknown`. @@ -135,7 +135,7 @@ decisions, report = assign_kcats(ec_model) # changes ec_model in place ::: :::: -`updateKcatSources` has the option `dryRun`, which reports what would be gathered without gathering it. In MATLAB it checks the requirements first: Docker for a DLKcat run and an OpenKineticsPredictor API key for OKP. In Python a source with a missing requirement gets the status `failed`. A file is outdated when its header records other settings than `[kcat_sources.]` asks for, or for `brenda` and `ec_tree` another BRENDA release than the files in use. `custom.tsv` and other files it cannot gather are left as they are. +`updateKcatSources` has the option `dryRun`, which reports what would be gathered without gathering it. A source that needs an OpenKineticsPredictor API key (`okp_`, and `dlkcat` when `data/DLKcat.tsv` does not exist) gets the status `failed` when it is gathered and no key is found. `dryRun` does not check the key. A `[kcat_sources.]` table overrides the keys that it sets and keeps the other settings of the preset for that source. A file is outdated when its header records other settings than the preset and `[kcat_sources.]` ask for, or for `brenda`, `ec_tree` and `homologs` another BRENDA release than the files in use. `custom.tsv` and other files it cannot gather are left as they are. ## Troubleshooting @@ -143,11 +143,12 @@ decisions, report = assign_kcats(ec_model) # changes ec_model in place | --- | --- | --- | | Error `kcatAssignment:config` | an empty preset, a criteria name in capitals, or a stage with a `sources` value of another type than list | Correct `[kcat_assignment]` | | OKP sources are pending | the OpenKineticsPredictor job is still running | Call `updateKcatSources` again later | -| Kcats differ from a GECKO 3 model | the default preset is not the GECKO 3 order | Set `preset = "gecko_classic"` | +| Error `collectKcats:legacyCustomKcats` | the project has the GECKO 3 file `data/customKcats.tsv` and no `data/kcats/custom.tsv` | Run `convertCustomKcats` | +| Kcats differ from a GECKO 3 model | the default preset `brenda_ec_tree` reads other sources than the GECKO 3 tutorial | Set `preset = "gecko_classic"` | | Rows of a source file are not applied | the row fits no reaction, or part of an enzyme set | Read the report of `loadKcatSource` | ## See also - [GECKO 3 to GECKO 4](gecko3-to-gecko4.md) - [Project format 4](gecko4-project-format.md) -- [Gathering kcats](guide/gathering-kcats.md) +- [Choose the kcat sources](guide/kcat-sources.md) diff --git a/docs/gecko4-project-format.md b/docs/gecko4-project-format.md index 48bac76..c29a348 100644 --- a/docs/gecko4-project-format.md +++ b/docs/gecko4-project-format.md @@ -30,7 +30,7 @@ The file holds the parameters that an adapter class sets in GECKO 3. Paths are r | `[evotune]` | the hyperparameters of the CMA-ES kcat tuning, with the source groups and the prior width per source | | `[kcat_assignment]` | a preset or stages, and the options `isozymes` and `standard` (see [Kcat assignment](gecko4-kcat-assignment.md)) | | `[kcat_sources.]` | settings of one source, for example `aggregation` of `brenda` and `methods` of `okp` | -| `[okp]` | the OpenKineticsPredictor settings `method` (default `CataPro`), `handle_long_sequences`, `include_similarity_columns` and `canonicalize_substrates` | +| `[okp]` | the OpenKineticsPredictor settings `method` (default `KinForm-H`), `handle_long_sequences`, `include_similarity_columns` and `canonicalize_substrates` | | `[matlab]` | names of MATLAB functions for the anaerobic model and the protein content of the biomass. GECKO alone reads this section | | `[matlab.params]` | parameters that GECKO uses and geckopy does not use | diff --git a/docs/guide/advanced-tuning.md b/docs/guide/advanced-tuning.md new file mode 100644 index 0000000..98775b1 --- /dev/null +++ b/docs/guide/advanced-tuning.md @@ -0,0 +1,327 @@ +# Tune kcats against experimental data + +The advanced kcat tuning fits many kcats at once to maximum growth rates and exchange fluxes, measured in one or more conditions, by minimising the root-mean-square error (RMSE) between simulated and measured values. It searches with CMA-ES (covariance matrix adaptation evolution strategy), and the trust in the source of each kcat limits how far the kcat moves. The steps are: prepare the data files, configure the tuning, rank the kcats by their effect on the fit, review the assignment of the top kcats, and tune. The result is an ecModel whose selected kcats fit the data, and a list of the changed kcats. [Choose a kcat tuning](kcat-tuning.md) compares this tuning with the standard tuning. + + +:::::{dropdown} Place in the workflow + +The diagram orders the functions of this page within the [workflow](workflow.md). Boxes with a dashed outline are optional steps, a dotted arrow is an optional path, and a diamond is a choice. Rounded boxes with a dotted outline are the context, other pages or the model to start from, and a click on one opens its page. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Previous: Choose a kcat tuning"]):::ctx + DF["prepare the data files
(read by loadEvotuneData, which the tuning calls)"] + CG["[evotune] in model_adapter.toml"] + SC["screenKcatLeverage"] + RV["reviewKcatAssignment
ecStatsFromBrenda"]:::opt + TP["tunePriorPenaltyWeight
seeds and reproducibility"]:::opt + CM["cmaesKcatTuning
selectTunableMask on the screen"] + PV --> DF --> CG --> SC --> CM + SC -.-> RV -.-> CM + SC -.-> TP -.-> CM + NP(["Next: Constrain enzymes with proteomics (optional)"]):::ctx + NA(["Next: Analyse an ecModel"]):::ctx + CM --> NP + CM --> NA + click PV href "kcat-tuning.html" + click NP href "proteomics.html" + click NA href "analysis.html" +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Previous: Choose a kcat tuning"]):::ctx + DF["prepare the data files
(read by load_evotune_data, which the tuning calls)"] + CG["[evotune] in model_adapter.toml"] + SC["screen_kcat_leverage"] + RV["review_assignment
ec_stats_from_brenda"]:::opt + TP["tune_prior_penalty_weight
seeds and reproducibility"]:::opt + CM["cmaes_kcat_tuning
select_tunable_mask on the screen"] + PV --> DF --> CG --> SC --> CM + SC -.-> RV -.-> CM + SC -.-> TP -.-> CM + NP(["Next: Constrain enzymes with proteomics (optional)"]):::ctx + NA(["Next: Analyse an ecModel"]):::ctx + CM --> NP + CM --> NA + click PV href "kcat-tuning.html" + click NP href "proteomics.html" + click NA href "analysis.html" +``` +::: +:::: +::::: + +## Functions + +| MATLAB | Python | Note | +| --- | --- | --- | +| `loadEvotuneData` | `load_evotune_data` | reads the three data files; the tuning functions call it when no data are given | +| `screenKcatLeverage` | `screen_kcat_leverage` | ranks the kcats by their effect on the fit, without tuning | +| `selectTunableMask` | `select_tunable_mask` | selects the top kcats of the screen for the search | +| `reviewKcatAssignment` | `review_assignment` | flags kcat assignments that may be wrong, independent of the data | +| `ecStatsFromBrenda` | `ec_stats_from_brenda` | summarises the BRENDA data per EC number for the review | +| `cmaesKcatTuning` | `cmaes_kcat_tuning` | searches the selected kcats for the best fit to the data | +| `tunePriorPenaltyWeight` | `tune_prior_penalty_weight` | runs the tuning for several `prior_penalty_weight` values and seeds, and reports fit and reproducibility | + +## Setup + +- An ecModel with assigned kcats (see [Choose the kcat sources](kcat-sources.md)) and a set protein pool (see [Set the protein pool](protein-pool.md)), in a project with `model_adapter.toml`. +- At least one of the files `evotuneMaxGrowth.tsv` and `evotuneFluxData.tsv` in the `data` folder of the project. The tutorial `full_ecModel` of GECKO has all three data files in its `data` folder. +- MATLAB: `nProc` above 1 needs the Parallel Computing Toolbox. `cmaesKcatTuning` has its own CMA-ES. Python: geckopy installs the CMA-ES package `cma`. + +## Prepare the data files + +`loadEvotuneData` reads three tab-separated files from `/data/`. A missing file is skipped. + +| File | Contents | +| --- | --- | +| `evotuneMaxGrowth.tsv` | maximum growth rates, one row per condition | +| `evotuneFluxData.tsv` | growth rates and exchange fluxes, one row per condition | +| `evotuneZeroExch.tsv` | reactions that carry no flux in any condition of `evotuneFluxData.tsv` | + +The first two files have the layout of the flux data file that `loadFluxData` reads: + +```text +Condition Ptot grRate glucose (r_1714) fructose (r_1709) oxygen (r_1992) ethanol (r_1761) evotuneRMSEweight source +glucose NaN 0.41 -1000 NaN NaN NaN 1 reference +fructose NaN 0.338 NaN -1000 NaN NaN 1 reference +``` + +| Column | Meaning | +| --- | --- | +| 1 (`Condition`) | the carbon source of the row. It must equal the metabolite name of one exchange column, for example `glucose` for `glucose (r_1714)` | +| 2 (`Ptot`) | total protein content in g/gDCW, or `NaN`. A value other than `NaN` or 0 changes the protein content of the biomass for that condition (see [Conditions that change the model](#conditions-that-change-the-model)) | +| 3 (`grRate`) | measured growth rate in 1/h | +| ` ()` | measured exchange flux in mmol/gDCW/h, or `NaN` when it was not measured. A column `oxygen` with the value 0 makes the condition anaerobic | +| `evotuneRMSEweight` | optional; the RMSE of the row is multiplied by this weight | +| `source` | optional; free text, not used in the fit | + +Each row is simulated with the uptake of its own carbon source open and the uptake of the carbon sources of the other rows of the same file closed (lower bound 0): + +- In `evotuneMaxGrowth.tsv`, the uptake of the carbon source of the row is set to -1000, and the growth rate is the sole compared value. The value in the carbon source column is not read. +- In `evotuneFluxData.tsv`, the lower bound of the carbon source uptake is set to the measured value. The growth rate and every exchange flux that is not `NaN` are compared, and so is every reaction of `evotuneZeroExch.tsv` that carries flux in the simulation. + +Fluxes are compared in carbon units: each flux is multiplied by the number of carbon atoms of the exchanged metabolite (1 for a metabolite without carbon), and the growth rate by 41 Cmmol/gDCW. A row whose simulation is infeasible has an RMSE of 99. The RMSE of a file is the mean over its rows, and the RMSE of the fit is `(rmse_flux + w * rmse_max_growth) / (w + 1)`, with `w` the setting `max_growth_weight`. + +`evotuneZeroExch.tsv` has the header `Rxns` and one reaction ID per line. + +## Configure the tuning + +The settings are in the `[evotune]` table of `model_adapter.toml`. Every setting has a default (see [Model adapter parameters](model-adapter-parameters.md#kcat-tuning)). The template of `createGECKOproject` writes these values: + +```toml +[evotune] +sigma0_log_default = 0.5 +max_growth_weight = 1.0 +prior_penalty_weight = 0.03 +tie_isozymes = true +rmse_threshold = 0.2 +max_generations = 150 + +[evotune.source_groups.dlkcat] +sources = ["dlkcat"] + +[evotune.source_groups.brenda] +sources = ["brenda"] + +[evotune.source_groups.custom] +sources = ["custom"] + +[evotune.sigma0_log_source] +dlkcat = 0.4 +brenda = 0.2 +custom = 0.1 +``` + +| Setting (MATLAB field) | Meaning | +| --- | --- | +| `source_groups` (`kcatSources`) | trust groups, each with the `sources` it covers | +| `sigma0_log_source` (`sigma0logSource`) | standard deviation in log space for each group; the keys are the group names | +| `sigma0_log_default` (`sigma0logDefault`) | standard deviation for a kcat whose source is in no group | +| `max_growth_weight` (`maxGrowthWeight`) | weight `w` of the maximum growth rates against the fluxes in the RMSE | +| `prior_penalty_weight` (`priorPenaltyWeight`) | weight `p` of a penalty for moving kcats; the search minimises `rmse + p * mean(((log k - log k0) / sigma0_log)^2)`. At 0 the objective is the RMSE | +| `tie_isozymes` (`tieIsozymes`) | isozyme copies of one reaction with the same kcat and the same group get one shared kcat | +| `rmse_threshold` (`rmseThreshold`) | the search stops when the best RMSE reaches this value; a negative value never stops the search early | +| `max_generations` (`maxGenerations`) | maximum number of CMA-ES generations | + +A smaller `sigma0_log` is more trust: the search moves such a kcat less, and the screen ranks it lower. A kcat belongs to a group when the origin of its `ec.source`, or the whole `ec.source`, equals one of the `sources` of the group, without regard to case (see [Provenance of a kcat](../gecko4-kcat-assignment.md#provenance-of-a-kcat)). `brenda (wc=0, origin=1)` is in the group that lists `brenda`, `combined (sd=0.10, n=2)` in the group that lists `combined`, and a tuned kcat such as `brenda (wc=0, origin=1) | tuned=cmaes (prior=2)` stays in the group of `brenda`. With `match_okp = true`, a group also covers the method of `[okp] method`. + +The groups depend on the origins in the model. With the default kcat assignment, `ec.source` also holds origins such as `ec_tree`, `combined` and `standard`, which the template groups do not list, so these kcats take `sigma0_log_default`. List the origins of `ec.source` (`parseKcatSource`, Python `parse_kcat_source`, returns the origin of each entry) and add a group for each origin that needs its own trust. MATLAB reads `source_groups` as a flat list of source labels (`kcatSources`) with one width per label (`sigma0logSource`), so in MATLAB the group of a kcat, in `screen.sourceGroup` and `result.groups`, is the matched source label; in Python it is the group name. When a label matches several groups, the first group applies. + +## Rank the kcats by their effect + +`screenKcatLeverage` multiplies and divides each kcat (or each group of tied isozyme copies) by `fold` (default 2) and records the largest change of the RMSE, the leverage. The rows are sorted by leverage multiplied by the `sigma0_log` of the kcat. The screen runs one simulation per condition for each of these probes and one baseline, so on a genome-scale model it takes about as long as a tuning run. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +ModelAdapterManager.setDefault('path/to/project'); +ecModel = loadEcModel(); + +screen = screenKcatLeverage(ecModel, 'nProc', 8); +removevars(screen(1:20, :), 'positions') +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import ModelAdapter, load_ec_model +from geckopy.kcat_tuning.evotune import screen_kcat_leverage + +adapter = ModelAdapter.from_folder("path/to/project") +ec_model = load_ec_model(adapter=adapter) # models/ecModel.yml + +screen = screen_kcat_leverage(ec_model, n_proc=8) +screen.drop(columns="_positions").head(20) +``` +::: +:::: + +The screen has one row per kcat or tied group: `rxnId` (Python `rxn_id`), `nIsozymes` (`n_isozymes`), `sourceGroup` (`source_group`), `kcat0`, `leverage`, `sigma0log` (`sigma0_log`), `rank` (leverage times `sigma0log`) and `cumLeverageShare` (`cum_leverage_share`), the sum of `rank` from the first row to this row as a fraction of the total. The column `positions` (Python `_positions`) holds the indices into `ec.rxns` of each row. The screen scores two kcat vectors for each kcat or tied group, so its time grows with the number of kcats screened; `tunableMask` (Python `tunable_mask`) restricts the screen to a subset. + +## Review the assignments of the top kcats + +`reviewKcatAssignment` checks each kcat for these problems: an EC number with specific-activity rows but no kcat rows in BRENDA (`no-ec-evidence`; an EC number that is not in BRENDA is not flagged), the same value in 5 or more reactions (`repeated-value`), the maximum of an EC number that is more than 3 times its median (`ec-maximum`), a value outside 0.01 to 10000 per s (`magnitude`), and a custom value that equals a BRENDA value of its EC number (`custom-duplicate`). The checks that need BRENDA statistics are skipped without `ecStats`. With the leverage of the screen the findings are sorted by leverage, and `coverage` or `top` shortens the list. Correct a wrong kcat in its source (see [Choose the kcat sources](kcat-sources.md) and [Set custom kcats](custom-kcats.md)); every model built from the same files then has the correction. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +[KCATcell, SAcell] = loadBRENDAdata('kcatAggregation', 'max'); +KCATmedianCell = loadBRENDAdata('kcatAggregation', 'median'); +ecStats = ecStatsFromBrenda(KCATcell, SAcell, KCATmedianCell); + +leverage = zeros(numel(ecModel.ec.rxns), 1); +for i = 1:height(screen) + leverage(screen.positions{i}) = screen.leverage(i); +end +findings = reviewKcatAssignment(ecModel.ec.rxns, ecModel.ec.kcat, ... + ecModel.ec.source, ecModel.ec.eccodes, 'ecStats', ecStats, ... + 'leverage', leverage, 'coverage', 0.9); +writetable(findings, 'findings.tsv', 'FileType', 'text', 'Delimiter', '\t') +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +import numpy as np +from geckopy import ec_stats_from_brenda, load_brenda_data, review_assignment +from geckopy.gather_kcats.review import findings_tsv + +ec_stats = ec_stats_from_brenda(load_brenda_data(adapter.get_brenda_db_folder())) + +leverage = np.zeros(len(ec_model.ec.rxns)) +for _, row in screen.iterrows(): + leverage[list(row["_positions"])] = row["leverage"] +findings = review_assignment( + ec_model.ec.rxns, ec_model.ec.kcat, ec_model.ec.source, ec_model.ec.eccodes, + ec_stats=ec_stats, leverage=leverage, coverage=0.9, +) +open("findings.tsv", "w").write(findings_tsv(findings)) +``` +::: +:::: + +Each finding has the reaction, its EC number, source, kcat and leverage, the number of isozyme copies, the number of other reactions that share an enzyme (with the option `enzymes`), the names of the checks and a `detail` in words. Tied isozyme copies are reported once. Kcats with the source `standard` are reported when `include` lists `standard-fallback`, and the check `tied-isozymes` is added when `include` lists it; the default `include` is empty. + +## Run the advanced tuning + +`cmaesKcatTuning` uses `selectTunableMask` to select the kcats to search: the top rows of the screen, up to the first row at which `cumLeverageShare` reaches `targetImpactShare` (default 0.9). With `screen` the function uses that screen; without it, the function runs a new screen. `tunableMask` (Python `tunable_mask`) gives the set of kcats directly and takes precedence over `screen`. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +[ecModel, result] = cmaesKcatTuning(ecModel, 'screen', screen, ... + 'nProc', 8, 'seed', 0); +saveEcModel(ecModel, 'ecModel_tuned.yml'); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import save_ec_model +from geckopy.kcat_tuning.evotune import cmaes_kcat_tuning + +result = cmaes_kcat_tuning(ec_model, screen=screen, n_proc=8, seed=0) +save_ec_model(ec_model, "ecModel_tuned.yml", adapter=adapter) +``` +::: +:::: + +Each candidate kcat lies between `min(0.01, k0/100)` and `max(10000, 100*k0)` per s, where `k0` is the kcat before tuning. After the run, the selected kcats hold the values of the best candidate, and their enzyme constraints are applied. Python changes `ec_model` in place. `popsize` sets the number of candidates per generation (default `4 + floor(3*log(n))` for `n` free kcats). In Python, the search can also stop earlier on a convergence criterion of the `cma` package, such as `tolx`. + +### Conditions that change the model + +A data row with oxygen uptake 0 or with a value of `Ptot` needs a function that changes the model. MATLAB runs the functions named by `make_model_anaerobic` and `change_protein_biomass` in the `[matlab]` table of `model_adapter.toml` (see [Model adapter parameters](model-adapter-parameters.md#matlab-functions)). Python takes these functions through the arguments `make_anaerobic(model)` and `change_protein_biomass(model, ptot)`; `tutorials/full_ecModel/code/anaerobic.py` in the geckopy repository has an example. Without a function, the condition is simulated without the change. + +## Seeds, reproducibility and parallel runs + +`seed` makes a run reproducible. MATLAB and Python have different CMA-ES implementations, so the same seed gives different results in the two toolboxes. In Python, a run with `n_proc` above 1 gives the same result as a run with `n_proc=1` and the same seed. `nProc` has the default 1 in MATLAB; `n_proc` has the default `cobra.Configuration().processes` in Python. + +Different kcat vectors can fit the data equally well, so two seeds can move a kcat in different directions. Run the tuning with a second seed and compare the large changes (above two-fold): they should be in the same direction and of similar size. If they are not, increase `prior_penalty_weight`. `tunePriorPenaltyWeight` (Python `tune_prior_penalty_weight`) runs the tuning for each value of `candidates` (default 0, 0.01, 0.03 and 0.1) and each of `seeds` (default 0 and 1), on the same set of kcats, and reports per value the fit, the number of changed kcats and the agreement between seeds. It costs one tuning run per value and seed, and the model is not changed. + +## Read the result + +`result` (Python `EvotuneResult`) has these fields: + +| MATLAB | Python | Meaning | +| --- | --- | --- | +| `rxns`, `oldKcat`, `newKcat`, `groups` | `rxns`, `old_kcat`, `new_kcat`, `groups` | the selected kcats before and after tuning, and the trust group of each; tied copies share one value | +| `rmseTrace`, `objectiveTrace` | `rmse_trace`, `objective_trace` | the best RMSE and the best value of the minimised objective, per generation; the two are equal when `prior_penalty_weight` is 0 | +| `nGenerations` | `n_generations` | number of generations that ran | +| `converged` | `converged` | whether the best RMSE reached `rmse_threshold` | + +`ec.source` of every kcat that changed gets the tuning status `tuned=cmaes`, with the kcat before the first tuning as `prior` (see [Provenance of a kcat](../gecko4-kcat-assignment.md#provenance-of-a-kcat)). `nChanged` (Python `n_changed`) counts the kcats that differ from their value before tuning by more than 2 %. `parsimonyFrontier` (Python `parsimony_frontier`) sets each kcat that moved less than a threshold, in units of its `sigma0_log`, back to the value before tuning, and scores the result for a series of thresholds with a scoring function that the caller gives. `bestParsimonious` (Python `best_parsimonious`) returns the point with the fewest changed kcats whose RMSE is within 2 % of the best RMSE. In Python, `result.corrections(leverage=...)` returns one row per changed kcat, sorted by the leverage of the screen; `leverage` has one value per entry of `result.rxns`; `corrections_tsv` in `geckopy.kcat_tuning` writes the rows as TSV. Check the tuned model against data that were not used in the fit: an improvement on the fitted conditions without an improvement on other data indicates overfitting. As with the standard tuning, check the large changes against the source of each kcat (see [Choose a kcat tuning](kcat-tuning.md#check-a-tuned-kcat-against-its-source)), and keep curated values in `data/kcats/custom.tsv`. + +## Troubleshooting + +| Problem | Possible reason | Solution | +| --- | --- | --- | +| Error `evotune:noData` (MATLAB), or `ValueError` that `evotune_data` has neither `flux_data` nor `max_grate` (Python) | Neither `evotuneFluxData.tsv` nor `evotuneMaxGrowth.tsv` is in `/data/` | Add one of the files | +| Error that a carbon source cannot be matched by name with an exchange reaction | The `Condition` of a row differs from every metabolite name in the exchange column headers | Use the metabolite name of the column header as the condition | +| Error `No tunable kcats` | No kcat is above 0, or the mask excludes every kcat | Assign kcats first, or widen the mask | +| Error `Only 1 free parameter(s) after masking and tying` | The mask and the isozyme tying leave fewer than two free kcats | Raise `targetImpactShare`, or give a wider `tunableMask` | +| Error that the keys of `sigma0_log_source` must be the group names of `source_groups` | A group has no width, or a width has no group | Give each group one value in `[evotune.sigma0_log_source]` | +| Most kcats take `sigma0_log_default` | The `sources` of the groups do not match the origins of `ec.source` in the model | List the origins that occur in `ec.source` (for example `ec_tree`, `combined` or `custom`) in `source_groups` | +| The large changes differ in direction between seeds | Several kcat vectors fit the data equally well | Increase `prior_penalty_weight`, and compare values with `tunePriorPenaltyWeight` | + +## See also + +- [Choose a kcat tuning](kcat-tuning.md), the standard tuning and the check of a tuned kcat. +- [Files in the data folder](data-folder-files.md), every file of the `data` folder. +- [Model adapter parameters](model-adapter-parameters.md#kcat-tuning), the `[evotune]` settings in both toolboxes. +- [API reference](../api/index.md), every function in both toolboxes. diff --git a/docs/guide/analysis.md b/docs/guide/analysis.md new file mode 100644 index 0000000..337aff4 --- /dev/null +++ b/docs/guide/analysis.md @@ -0,0 +1,332 @@ +# Analyse an ecModel + +An ecModel has the structure of a conventional GEM, so every method for a conventional GEM runs on it. GECKO adds functions that take the enzyme constraints into account and map the results to the reactions of the conventional GEM. The analyses below set an objective, compute flux ranges or samples, and map the results to the conventional GEM for comparison. + + +:::::{dropdown} Place in the workflow + +The diagram orders the functions of this page within the [workflow](workflow.md). Boxes with a dashed outline are optional steps, a dotted arrow is an optional path, and a diamond is a choice. Rounded boxes with a dotted outline are the context, other pages or the model to start from, and a click on one opens its page. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Start: a functional or relaxed ecModel"]):::ctx + OB["set the objective
setParam, solveLP"] + FV["ecFVA, plotEcFVA"] + RS["randomSampling (RAVEN)"] + MP["mapRxnsToConv"] + PV --> OB + OB --> FV + OB --> RS + OB --> MP + RS --> MP + NE(["Find engineering targets"]):::ctx + NU(["Enzyme usage and bottlenecks"]):::ctx + PV -.-> NE + PV -.-> NU + click PV href "first-ecmodel.html" + click NE href "engineering-targets.html" + click NU href "enzyme-usage.html" +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Start: a functional or relaxed ecModel"]):::ctx + OB["set the objective
model.objective, model.optimize"] + FV["ec_fva"] + RS["random_sampling (raven-toolbox)"] + MP["map_rxns_to_conv"] + PV --> OB + OB --> FV + OB --> RS + OB --> MP + RS --> MP + NE(["Find engineering targets"]):::ctx + NU(["Enzyme usage and bottlenecks"]):::ctx + PV -.-> NE + PV -.-> NU + click PV href "first-ecmodel.html" + click NE href "engineering-targets.html" + click NU href "enzyme-usage.html" +``` +::: +:::: +::::: + +## Functions + +| MATLAB | Python | Note | +| --- | --- | --- | +| `setParam(..., 'obj', ...)` and `solveLP` | `model.objective` and `model.optimize()` | set the objective and solve | +| `setProtPoolSize` | `set_prot_pool_size` | change the upper bound of the protein pool exchange | +| `mapRxnsToConv` | `map_rxns_to_conv` | map fluxes of the ecModel to the reactions of the conventional GEM | +| `ecFVA` | `ec_fva` | flux variability analysis, mapped to the reactions of the conventional GEM | +| `plotEcFVA` | (none) | plot the cumulative distribution of the flux ranges | +| `randomSampling` (RAVEN) | `random_sampling` (raven-toolbox) | draw flux vectors from the solution space | + +## Setup + +The examples use the model adapter parameters `params`, the ecModel, the conventional GEM that the ecModel was made from, and for some sections the flux data of `data/fluxData.tsv` ([Files in the data folder](data-folder-files.md)): + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +adapter = ModelAdapterManager.setDefault(pwd); +params = adapter.getParameters(); +model = loadConventionalGEM(); +ecModel = loadEcModel(); % models/ecModel.yml +fluxData = loadFluxData(); % data/fluxData.tsv +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import ModelAdapter, load_conventional_gem, load_ec_model, load_flux_data + +adapter = ModelAdapter.from_folder("ecMyModel") +params = adapter.params +model = load_conventional_gem(adapter) +ec_model = load_ec_model("ecModel.yml", adapter=adapter) +flux_data = load_flux_data(params.path / "data" / "fluxData.tsv") +``` +::: +:::: + +[Find engineering targets](engineering-targets.md), [Make a context-specific ecModel](context-ecmodel.md) and [Enzyme usage and bottlenecks](enzyme-usage.md) use the same setup. + +## Choose an analysis + +| Question | Analysis | Needs | Result | +| --- | --- | --- | --- | +| How fast can the cell grow under these constraints? | maximise growth | the biomass reaction of the model adapter | the maximum growth rate and one flux distribution | +| Which flux distribution uses the least protein at this growth rate? | minimise `prot_pool_exchange` at fixed growth | a growth rate, measured or from the first objective | the minimum protein pool usage in mg/gDCW and its flux distribution | +| How far can each reaction vary? | `ecFVA` | the conventional GEM that the ecModel was made from | the minimum and maximum flux of each reaction of the conventional GEM | +| How are fluxes distributed over the solution space? | random sampling | a convergence check across independent chains | a matrix of flux vectors | +| Which reactions and genes to change to make a product? | `ecFSEOF`, see [Find engineering targets](engineering-targets.md) | a production reaction and the carbon source uptake reaction | reaction and gene targets with a suggested action | +| How do the fluxes compare with those of the conventional GEM? | `mapRxnsToConv` | the conventional GEM that the ecModel was made from | a flux vector in the reaction order of the conventional GEM | +| What does a tissue, cell line or strain do? | `getSubsetEcModel`, see [Make a context-specific ecModel](context-ecmodel.md) | a context-specific GEM made from the same GEM as the ecModel | a context-specific ecModel | + +[Enzyme usage and bottlenecks](enzyme-usage.md) describes the analyses of single enzymes: which enzymes a solution uses, and which enzymes limit the objective. + +For flux ranges, use `ecFVA`. A [convergence study of the samplers of raven-toolbox](https://raven-docs.readthedocs.io/en/latest/parameter-tuning/studies/sampling-convergence-calibration.html) found no setting that gave converged samples on yeast-GEM, a conventional GEM of about 4,100 reactions. The study did not test an ecModel. + +## Set the objective + +### Maximise growth + +The biomass reaction of the model adapter is the objective. The protein pool exchange limits growth when its flux reaches the upper bound that `setProtPoolSize` sets ([Set the protein pool](protein-pool.md)). + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +ecModel = setParam(ecModel, 'obj', params.bioRxn, 1); +sol = solveLP(ecModel); +maxGrowth = sol.f; +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +ec_model.objective = params.bio_rxn +sol = ec_model.optimize() +max_growth = sol.objective_value +``` +::: +:::: + +To see the effect of the protein pool constraint, solve a copy of the ecModel with an unlimited pool: `setProtPoolSize(ecModel, Inf)` in MATLAB, and in Python `set_prot_pool_size(ec_model_copy, p_tot=math.inf)` on a copy made with `ec_model.copy()`, because the Python function changes the model in place. Each enzyme usage reaction has an upper bound of 1000 mg/gDCW unless proteomics data constrain it, so the copy has no effective enzyme constraint. The `full_ecModel` tutorial of GECKO makes this comparison for yeast: its function `code/plotCrabtree.m` simulates the exchange fluxes over a range of growth rates and plots them against measured fluxes, and the tutorial runs it on the ecModel and on a copy with an unlimited pool. + +### Minimise protein pool usage at fixed growth + +`prot_pool_exchange` runs forward: its flux is the protein that the solution draws from the pool, a positive number in mg/gDCW. Fix growth at a value slightly below the maximum, for example 99 %. The second problem then stays feasible within the tolerance of the solver. Then minimise the flux of `prot_pool_exchange`. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +`solveLP` maximises the objective, so the coefficient -1 minimises the flux, and `sol.f` is the negative of the pool usage. + +```matlab +ecModel = setParam(ecModel, 'lb', params.bioRxn, 0.99 * maxGrowth); +ecModel = setParam(ecModel, 'obj', 'prot_pool_exchange', -1); +sol = solveLP(ecModel); +poolUsage = -sol.f; % mg/gDCW +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +ec_model.reactions.get_by_id(params.bio_rxn).lower_bound = 0.99 * max_growth +ec_model.objective = "prot_pool_exchange" +ec_model.objective_direction = "min" +sol = ec_model.optimize() +pool_usage = sol.objective_value # mg/gDCW +``` +::: +:::: + +For a measured growth rate, call `constrainFluxData` with `maxMinGrowth` set to `'min'` (Python: `apply_flux_data_constraints` with `max_min_growth="min"`): the measured growth rate becomes the lower bound of the biomass reaction. [Constrain enzymes with proteomics](proteomics.md) describes the flux data file. + +If `prot_pool_exchange` or the `usage_prot_*` reactions carry negative flux, the model has these reactions in the reverse direction. `upgradeEcModel` (Python: `upgrade_ec_model`) turns them forward, and `loadEcModel` (Python: `load_ec_model`) does this with a warning when it loads such a model ([GECKO 3 to GECKO 4](../gecko3-to-gecko4.md)). + +## Compare the solution space with the conventional GEM + +`ecFVA` groups the reactions of the ecModel by identifier without the suffixes `_REV` and `_EXP_`: the isozyme copies and the reverse copy of a reaction form one group. For each group it minimises and maximises the combined flux, which takes two linear problems per group, the enzyme usage reactions and `prot_pool_exchange` included. The result has the reaction order of the conventional GEM and leaves out the enzyme usage reactions; for those, run a plain FVA on the ecModel (`getAllowedBounds` in RAVEN, `cobra.flux_analysis.flux_variability_analysis` in Python). The objective of the model is not used: each reaction varies within the bounds of the model, so set the bounds that define the condition (growth rate, uptake rates) on all models first. In MATLAB, a conventional GEM that does not match the ecModel raises its error after all problems are solved. + +The example gives the conventional GEM and the ecModel the same exchange constraints from `fluxData` (from `loadFluxData`, Python: `load_flux_data`) and compares the ranges: + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +model = constrainFluxData(model, fluxData, 1, 'max', 'loose'); +ecModel = constrainFluxData(ecModel, fluxData, 1, 'max', 'loose'); + +[minFlux(:,1), maxFlux(:,1)] = ecFVA(model, model); +[minFlux(:,2), maxFlux(:,2)] = ecFVA(ecModel, model); +plotEcFVA(minFlux, maxFlux); +``` + +`ecFVA` runs in parallel when the Parallel Computing Toolbox is installed (option `runParallel`, default `true`). `plotEcFVA` takes one column per model, leaves out the reactions whose minimum and maximum are both 0, and plots the cumulative distribution of the ranges (`maxFlux - minFlux`) on a log scale, with the median of each model in the legend. +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import apply_flux_data_constraints, ec_fva + +for m in (model, ec_model): + apply_flux_data_constraints( + m, flux_data, condition=0, max_min_growth="max", + loose_strict_flux="loose", bio_rxn=params.bio_rxn, c_source=params.c_source, + ) + +fva_gem = ec_fva(model, model) +fva_ec = ec_fva(ec_model, model) +ranges = (fva_ec["max_flux"] - fva_ec["min_flux"]).rename("ecModel").to_frame() +ranges["GEM"] = fva_gem["max_flux"] - fva_gem["min_flux"] +print(ranges.median()) +``` + +`ec_fva` returns a DataFrame indexed by the reaction identifiers of the conventional GEM, with the columns `min_flux` and `max_flux`. The option `n_proc` sets the number of worker processes (default `cobra.Configuration().processes`). With the default solver GLPK, `ec_fva` is slow on a genome-scale ecModel; a commercial solver (`ec_model.solver = "gurobi"`, with a licence) or more worker processes shorten the run. geckopy has no counterpart of `plotEcFVA`. +::: +:::: + +A smaller range in the ecModel than in the conventional GEM shows a reaction that the enzyme constraints restrict. A proteomics-constrained ecModel ([Constrain enzymes with proteomics](proteomics.md)) can be added as a third column. + +## Sample the solution space + +FVA gives the extremes of each reaction, one reaction at a time. A sampler draws flux vectors from the space that the bounds of the model define, so the samples depend on every bound of the model at the time of the call (growth rate, uptake rates, enzyme bounds). The samplers are part of RAVEN (MATLAB) and raven-toolbox (Python): + +| `method` | Samples | Note | +| --- | --- | --- | +| `'achr'` (default) | points inside the solution space | Artificially Centered Hit-and-Run | +| `'chrr'` | points inside the solution space | Coordinate Hit-and-Run with Rounding; the help of both toolboxes states that it mixes better in thin, ill-conditioned spaces such as those of enzyme-constrained models | +| `'randomObjective'` (Python: `"random_objective"`) | corners of the solution space | each sample optimises a random objective of a few reactions; in MATLAB, the `usage_prot_*` reactions of an ecModel are not used as objectives | + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +samples = randomSampling(ecModel, 1000, 'method', 'chrr', 'seed', 1); +mappedSamples = mapRxnsToConv(ecModel, model, samples); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import map_rxns_to_conv +from raven_toolbox.analysis import random_sampling + +result = random_sampling(ec_model, n_samples=1000, method="chrr", seed=1) +samples = result.samples[[r.id for r in ec_model.reactions]].T.to_numpy() +mapped_samples = map_rxns_to_conv(ec_model, model, samples).mapped_flux +``` + +`result.samples` has one sample per row and one reaction per column, so the matrix is transposed before mapping. +::: +:::: + +Run several chains with different seeds and compare them, for example with the R-hat statistic, before the samples are used. The [convergence study](https://raven-docs.readthedocs.io/en/latest/parameter-tuning/studies/sampling-convergence-calibration.html) describes the check and the settings it tested. + +## Map fluxes to the conventional GEM + +A flux vector of an ecModel has more reactions than one of the conventional GEM: isozyme copies, reverse copies and the enzyme usage reactions. `mapRxnsToConv` subtracts the flux of each reverse copy, adds the isozyme copies, and returns the fluxes in the reaction order of the conventional GEM. It also accepts a matrix with one flux vector per column, such as the samples above. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +sol = solveLP(ecModel); +[mappedFlux, enzUsageFlux, usageEnz] = mapRxnsToConv(ecModel, model, sol.x); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import map_rxns_to_conv + +sol = ec_model.optimize() +mapped = map_rxns_to_conv(ec_model, model, sol.fluxes) +mapped.mapped_flux, mapped.enz_usage_flux, mapped.usage_enz +``` +::: +:::: + +`enzUsageFlux` (Python: `enz_usage_flux`) holds the fluxes of the `usage_prot_*` reactions and of `prot_pool_exchange`, and `usageEnz` (Python: `usage_enz`) their labels: the protein identifier, or `pool` for the protein pool exchange. The function matches reactions by identifier. A reaction of the conventional GEM that is missing from the ecModel raises an error, but a reaction that was changed in the ecModel after `makeEcModel` (another stoichiometry or direction) is mapped without a warning. + +## Troubleshooting + +| Problem | Possible reason | Solution | +| --- | --- | --- | +| `prot_pool_exchange` or the `usage_prot_*` reactions carry negative flux | the model has these reactions in the reverse direction and was not loaded with `loadEcModel` | Run `upgradeEcModel` (Python: `upgrade_ec_model`) | +| `mapRxnsToConv` or `ecFVA` raises "Not all reactions from model.rxns can be found in the ecModel" | the conventional GEM differs from the GEM that the ecModel was made from | Pass the GEM that `makeEcModel` started from | +| `map_rxns_to_conv` or `ec_fva` raises a `ValueError` that lists conventional reactions missing from the ecModel | the conventional GEM differs from the GEM that the ecModel was made from | Pass the GEM that `makeEcModel` started from | +| The `ecFVA` ranges of the ecModel reach 1000 for many reactions | the bounds of the condition are missing, so the default bounds limit the fluxes | Set the growth rate and the uptake rates before `ecFVA` | +| Independent sampling chains give different flux distributions | the chains have not converged | Increase `thinning` and the number of samples, and check again | + +## See also + +- [Enzyme usage and bottlenecks](enzyme-usage.md): the enzymes that a solution uses and the enzymes that limit the objective. +- [Find engineering targets](engineering-targets.md): flux scanning with `ecFSEOF` for the production of a compound. +- [Make a context-specific ecModel](context-ecmodel.md): trim a generic ecModel to a tissue, cell line or strain. +- [Basic model operations](basic-model-operations.md): bounds, objectives and flux inspection. +- [Constrain enzymes with proteomics](proteomics.md): the flux data file and the proteomics-constrained ecModel. +- [Set the protein pool](protein-pool.md): the upper bound of `prot_pool_exchange`. +- [API reference](../api/index.md) and [MATLAB vs Python](../matlab-vs-python.md): the signature of each function in the table. diff --git a/docs/guide/applying-kcats.md b/docs/guide/applying-kcats.md deleted file mode 100644 index b84c393..0000000 --- a/docs/guide/applying-kcats.md +++ /dev/null @@ -1,391 +0,0 @@ -# Applying kcats - -The kcat source files gathered on [Gathering kcats](gathering-kcats.md) become one kcat per reaction in `ec.kcat`, the kcats become enzyme constraints in the stoichiometry, and the protein pool exchange reaction gets a bound. The product is a draft ecModel with active enzyme constraints. Its growth rate depends on the sigma, $P_{tot}$ and f parameters, and [Growth-rate tuning](growth-rate-tuning.md) covers making it functional. - - -:::::{dropdown} Place in the workflow - -The diagram orders the functions of this page within the [workflow](workflow.md). Dashed boxes are optional and dotted boxes are neighbouring pages. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - PV(["Previous: Gathering kcats"]):::ctx - CU["data/kcats/custom.tsv"]:::opt - AS["assignKcats
preview: collectKcats"] - PV --> AS - CU --> AS - AS --> IS["getKcatAcrossIsozymes
getStandardKcat (optional)"]:::opt --> AK["applyKcatConstraints
setProtPoolSize"] - NX(["Next: Growth-rate tuning"]):::ctx - AK --> NX -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - PV(["Previous: Gathering kcats"]):::ctx - CU["data/kcats/custom.tsv"]:::opt - AS["assign_kcats
preview: collect_kcats"] - PV --> AS - CU --> AS - AS --> IS["fill_kcats_from_isozymes
assign_standard_kcat (optional)"]:::opt --> AK["apply_kcat_constraints
set_prot_pool_size"] - NX(["Next: Growth-rate tuning"]):::ctx - AK --> NX -``` -::: -:::: -::::: - -## Functions - -| MATLAB | Python | Note | -|---|---|---| -| `assignKcats` | `assign_kcats` | determine one kcat per reaction from the kcat source files and apply the kcat constraints | -| `collectKcats` | `collect_kcats` | determine the kcats without changing the model | -| `convertCustomKcats` | `convert_custom_kcats` | convert `data/customKcats.tsv` into `data/kcats/custom.tsv` | -| `applyCustomKcats` | `apply_custom_kcats` | apply the kcats of `data/customKcats.tsv` | -| `getKcatAcrossIsozymes` | `fill_kcats_from_isozymes` | fill missing kcats with the mean of their isozymes | -| `getStandardKcat` | `assign_standard_kcat` | assign a standard kcat to reactions without a gene association | -| `assignKcatValues` | `apply_kcat_list` | write one kcat per reaction from a kcat list to `ec.kcat` | -| `applyKcatConstraints` | `apply_kcat_constraints` | apply `ec.kcat` to the stoichiometry of the ecModel | -| `setProtPoolSize` | `set_prot_pool_size` | set the upper bound of the protein pool exchange reaction | -| `calculateFfactor` | `calculate_f_factor` | compute the f factor from proteomics data | - -## Setup - -The page continues from an ecModel built as described on [Building an empty ecModel](building-ec-model.md), with the kcat source files in `data/kcats` (see [Gathering kcats](gathering-kcats.md)). The kcat order is set in `[kcat_assignment]` of `model_adapter.toml` (see [Kcat assignment](../gecko4-kcat-assignment.md) and [Model adapter parameters](model-adapter-parameters.md)). The standard kcat needs `data/uniprot.tsv`, and the f factor can use `data/paxDB.tsv` (see [Files in the data folder](data-folder-files.md)). - -## Provide custom kcat values - -Custom kcat values can be an alternative or sole source, based on manual curation, results from later in the reconstruction, or another prediction procedure such as in vivo apparent enzyme turnover numbers. The source file `data/kcats/custom.tsv` holds them. It is a TSV with `#` header lines and the columns `rxn`, `enzymes`, `substrate`, `kcat`, `log10_sd`, `source`, `detail`, `ec_code` and `note`. A row names a reaction in `rxn` (the identifier of the conventional reaction, with `_REV` for the reverse direction), the UniProt accessions of an enzyme set in `enzymes` (joined by ` + `), or both, and the kcat in s^-1. Every preset starts with the stage `custom`, so the custom kcats take precedence over all other sources. [Project format 4](../gecko4-project-format.md#kcat-source-files) describes the format of the kcat source files. - -A project of GECKO 3 documents the custom kcats in `data/customKcats.tsv` in the project folder. `convertCustomKcats` converts the file with the matching rules of `applyCustomKcats`, including which of two entries for the same reaction is kept, and writes `data/kcats/custom.tsv`. The ecModel must be a full ecModel; the `custom.tsv` written also applies to light ecModels. An existing `custom.tsv` is not overwritten. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -[file, report] = convertCustomKcats(ecModel); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import convert_custom_kcats - -path, report = convert_custom_kcats(ec_model) -``` -::: -:::: - -In MATLAB, `report` is a struct: `report.rows` is the number of rows written and `report.partial` lists the reactions whose enzymes match an entry in part. In Python, `report` is a dict with the number of rows written as `report["rows"]`, and has no `partial`. - -Without `data/kcats/custom.tsv`, `collectKcats` applies `data/customKcats.tsv` itself when a stage lists `custom`. The file can also be applied to an ecModel directly: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -[ecModel, rxnUpdated, notMatch] = applyCustomKcats(ecModel); -``` - -This overwrites existing values in `ecModel.ec.kcat` for matching reactions, on the assumption that custom values have higher confidence. The stoichiometry changes when `applyKcatConstraints` runs. - -Custom values that fail to match: if `notMatch` contains reactions, their values were not successfully applied, because the gene association between the model and the input file is a partial match: at least 50 %, but not 100 %. A gene association that matches less than 50 % is skipped and does not appear in `notMatch`. Inspect which enzymes are associated with the reaction in the ecModel and curate the input file to resolve it. -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import apply_custom_kcats - -apply_custom_kcats(ec_model, path=params.path / "data" / "customKcats.tsv") -``` - -The same overwrite behaviour and the same gene-match threshold of 50 % apply here; unmatched rows are logged and no `notMatch` list is returned. By default, `apply_custom_kcats` calls `apply_kcat_constraints` after updating `ec.kcat`; `apply=False` leaves the stoichiometry unchanged. -::: -:::: - -## Assign kcats from the source files - -`assignKcats` determines one kcat per reaction from the kcat source files, writes `ec.kcat` and `ec.source` (see [Provenance of a kcat](../gecko4-kcat-assignment.md#provenance-of-a-kcat)), and applies the kcat constraints with `applyKcatConstraints`. The order of the sources follows the preset or the stages of `[kcat_assignment]`; [Kcat assignment](../gecko4-kcat-assignment.md) describes the presets, the stages and the gathering of missing source files with `updateKcatSources`. Kcats that the model had are replaced, because the source files are the input of the assignment. After the stages, `assignKcats` fills reactions without a kcat from their isozymes for full ecModels when `isozymes` is true, and gives the remaining reactions the standard kcat when `standard` is true; both options are true by default. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -[ecModel, decisions] = assignKcats(ecModel); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import assign_kcats - -decisions, report = assign_kcats(ec_model) -``` -::: -:::: - -`decisions` has one row per reaction of `ec`: the stage that decided the kcat (a stage number, `custom`, `isozymes` (full ecModels) or `standard`, and empty when no kcat was found), the source as written to `ec.source`, the kcat, the number of candidate values and their range in log10 units. For a combined value it also has the uncertainty in log10 units. The table is written to `output/_kcatDecisions.tsv` unless `writeReport` (Python: `write_report`) is false. `report` records the preset, the source files that a stage names and that do not exist, and the reactions of each source file that were not applied. - -`collectKcats` makes the same decisions without changing the model and returns the ecModel with the decided kcats, before the kcat constraints. The reactions without a kcat are those with an empty stage: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -decisions = collectKcats(ecModel); -decisions(strcmp(decisions.stage, ''), :) -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import collect_kcats - -decisions, report, ec_copy = collect_kcats(ec_model) -decisions[decisions["stage"] == ""] -``` - -`collect_kcats` returns a copy of the ecModel. -::: -:::: - -`assignKcatValues` (Python: `apply_kcat_list`) writes one kcat per reaction from a kcat list in memory to `ec.kcat`, with the criteria `max`, `min`, `median` or `mean`. [GECKO 3 to GECKO 4](../gecko3-to-gecko4.md) describes how it relates to `assignKcats`. - -The functions of the following sections change `ec.kcat` of a model that has been assigned, or of a model whose kcats were set in other ways. `assignKcats` replaces all kcats, `applyCustomKcats` overwrites the matching reactions, and the isozyme and standard steps fill the reactions without a kcat. Each documents its source in `ec.source`, for example `custom`, `isozymes` or `standard`. - -## Mean kcat of isozymes (optional) - -When a kcat is not defined for every isozymatic variant of a reaction, the variant lacking a kcat gets no protein cost and is preferred over variants that do have one, since it looks free in the optimisation. Substituting missing values in `ec.kcat` with the mean kcat of their isozymes avoids this. Isozymes with a kcat above 0 enter the mean. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -ecModel = getKcatAcrossIsozymes(ecModel); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import fill_kcats_from_isozymes - -fill_kcats_from_isozymes(ec_model) -``` - -`get_kcat_across_isozymes` also exists as a deprecated alias. `fill_kcats_from_isozymes` updates `ec_model.ec.kcat`, as `getKcatAcrossIsozymes` does; `apply=True` also calls `apply_kcat_constraints` for the reactions it filled. The option `aggregate` (`mean`, `median` or `max`) sets how the kcats of the isozymes are combined; the default is `kcat_aggregate_isozymes` of the model adapter, which is `mean`. -::: -:::: - -This step does not apply to light ecModels, where the most efficient isozyme sets the cost; `getKcatAcrossIsozymes` raises an error on a light ecModel (Python: `fill_kcats_from_isozymes` raises `NotImplementedError`). - -:::{admonition} Example output -:class: note - -`ec.kcat` is updated so every isozymic reaction with at least one kcat has a value, with `ec.source` for those reactions reading `isozymes`. The stoichiometric coefficients are unaffected until the constraints are applied below. -::: - -## Assign a standard kcat value (optional) - -Reactions without genes associated in the starting GEM (empty gene-reaction rules) cannot get enzyme constraints, for lack of enzyme data. For such reactions, excluding exchange, spontaneous, transport and pseudo-reactions (reaction names that contain `pseudoreaction` or `SLIME rxn`, and the reactions in `data/pseudoRxns.tsv`), a standard kcat and a standard pseudo-enzyme (`prot_standard`) constrain catalytic capacity instead. The standard kcat is the median of the kcat values of the reactions in the same subsystem, when the subsystem has at least 10 reactions with a kcat (the option `threshold`), and the median of every kcat in `ec.kcat` otherwise. The standard molecular weight is the median across every protein in the organism, and a `usage_prot_standard` reaction is added. - -Reactions that are annotated with an enzyme but have a kcat of 0 or `NaN` receive the standard kcat as well and keep their enzyme. Earlier standard assignments (an `ec.source` with the origin `standard` and no tuning status) are removed when the function runs again, and a standard kcat that tuning has changed is kept. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -[ecModel, rxnsMissingGPR, standardMW, standardKcat, rxnsNoKcat] = getStandardKcat(ecModel); -``` - -The standard MW and the subsystem-agnostic standard kcat are reported in `standardMW` and `standardKcat`; `rxnsNoKcat` lists the reactions whose kcat of 0 or `NaN` was replaced with the standard value. -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import assign_standard_kcat - -assign_standard_kcat(ec_model, uniprot_db) -``` - -`get_standard_kcat` also exists as a deprecated alias of `assign_standard_kcat`. The function returns nothing; the reactions missing a GPR, the standard MW and the standard kcat are logged. -::: -:::: - -## Apply the enzyme constraints - -With `ec.kcat` populated from the sources above, apply the enzyme constraints. `assignKcats` has applied them already; this call re-applies them after a change of `ec.kcat`, `ec.mw` or the complex data: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -ecModel = applyKcatConstraints(ecModel); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import apply_kcat_constraints - -apply_kcat_constraints(ec_model) -``` -::: -:::: - -This modifies the stoichiometric coefficients of the ecModel (the S-matrix in MATLAB, the cobra reaction stoichiometries in Python) to directly include the protein cost, based on the kcat values from `ec.kcat`, the MWs from `ec.mw` and, if applied, the enzyme complex stoichiometry from `applyComplexData`/`apply_complex_data` (see [Building an empty ecModel](building-ec-model.md#apply-enzyme-complex-stoichiometry-optional)). Enzymes are directly involved as pseudo-substrates, reflecting enzyme usage. Existing enzyme constraints of the reactions are removed first, so the function can run again at any point. A reaction with a kcat of 0 or `NaN` gets no enzyme cost. Both toolboxes can restrict the update to some reactions (MATLAB: the option `updateRxns`, Python: the argument `update_rxns`). - -In a light ecModel, the cost of a reaction is the lowest MW/kcat across its isozymes, written to the `prot_pool` row of the reaction. - -:::{admonition} Example output -:class: note - -Every stoichiometric coefficient added or changed above is applied to the ecModel. The coefficient of an enzyme pseudo-metabolite is: - -$$ \frac{MW}{k_{cat} \times 3600} \times (\text{complex stoichiometry}) $$ - -where the factor 3,600 converts kcat from s^-1 to h^-1, matching the unit of metabolic rates. Each enzymatic reaction gets one kcat value, which for complexes can be the highest predicted value across subunits. For example, if reaction X to Y is catalysed by a complex of subunits A, B and C at a ratio 1:1:2, all with MW 27,000 Da and highest predicted kcat of 30 s^-1, the reaction becomes X + 0.25 A + 0.25 B + 0.5 C to Y. -::: - -## Constrain the protein pool exchange reaction - -Enzyme pseudo-metabolites are replenished from the total protein pool by default. The protein pool exchange reaction `prot_pool_exchange` runs forward, from nothing to `prot_pool`, with the bounds 0 and 1000. The `usage_prot_` reactions run forward as well, from `prot_pool` to `prot_`. `setProtPoolSize` sets the upper bound of `prot_pool_exchange` to $P_{tot} \times f \times \sigma \times 1000$ mg/gDCW, calculated from the sigma, $P_{tot}$ and f parameters in the model adapter (average enzyme saturation, total protein content and the fraction of proteins included in the model). An ecModel of GECKO 3 has the protein reactions in the reverse direction; `upgradeEcModel` converts it (see [GECKO 3 to GECKO 4](../gecko3-to-gecko4.md)). - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -ecModel = setProtPoolSize(ecModel); -``` - -If quantitative proteomics data are available, for example via PaxDB, store them in `data/paxDB.tsv` and calculate the f factor: - -```matlab -f = calculateFfactor(ecModel); -``` - -Then use the new f factor to set the protein pool constraint: - -```matlab -ecModel = setProtPoolSize(ecModel, [], f); -``` - -Without `data/paxDB.tsv` and without the option `protData`, `calculateFfactor` returns 0.5 and prints a warning. -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import set_prot_pool_size - -set_prot_pool_size(ec_model, p_tot=params.p_tot, f=params.f, sigma=params.sigma) -``` - -With no arguments beyond `ec_model`, `set_prot_pool_size` reads `p_tot`, `f` and `sigma` from `ec_model.adapter.params`; the explicit keyword arguments above override them. The function returns the bound it set. If quantitative proteomics data are available via PaxDB, store them in `data/paxDB.tsv` and calculate the f factor: - -```python -from geckopy import calculate_f_factor, load_pax_db - -pax_data = load_pax_db(params.path / "data" / "paxDB.tsv", uniprot_db) -f = calculate_f_factor(ec_model, pax_data) -``` - -Then use the new f factor to set the protein pool constraint: - -```python -set_prot_pool_size(ec_model, f=f) -``` - -`calculate_f_factor` has no default for the proteomics data; without measurements, `f = 0.5` is used directly. -::: -:::: - -:::{admonition} Values to define the total protein pool are missing -:class: note - -sigma, $P_{tot}$ and f each have the standard value 0.5 when no better estimate exists. If the resulting maximum growth rate is too small or too large, adjusting sigma manually is a reasonable way to reach a more plausible value; see [Growth-rate tuning](growth-rate-tuning.md#protein-pool-constraint). -::: - -:::{admonition} Example output -:class: note - -With $P_{tot}$ = 0.5, f = 0.5 and sigma = 0.5, the upper bound of `prot_pool_exchange` becomes 125 mg/gDCW. -::: - -## The ec structure - -Enzyme-related information is stored in `ecModel.ec` (MATLAB) or `ec_model.ec` (Python, an `EcData` instance), both referred to as `ec`. Field names match between languages except for `rxnEnzMat`/`rxn_enz_mat`. The structure is similar for full and light ecModels but differs in how the reaction-related fields are populated: in full ecModels the whole model is expanded, each isozyme gets a separate reaction, and each reaction maps to one entry in the structure; in light ecModels the reactions are not split per isozyme, and some starting-GEM reactions have several entities represented in the structure. - -| Field | Data type | Size | Description | -|-------|-----------|------|-------------| -| `rxns` | string array (Python: `list[str]`) | m | Reaction identifiers gathered from the ecModel, after expansion and making irreversible. | -| `rxnEnzMat` (Python: `rxn_enz_mat`) | matrix (Python: sparse `scipy.sparse.csr_matrix`) | m x n | Comparable to the model's reaction-gene matrix, but Enz refers to `ec.enzymes`. Positive integers give the number of enzyme subunits annotated to each reaction. | -| `kcat` | float vector (Python: `numpy.ndarray`) | m | One value per reaction-enzyme (complex) combination, in s^-1, gathered from various sources; `0` means no kcat assigned yet. | -| `source` | string array (Python: `list[str]`) | m | Where the kcat came from, for example `dlkcat`, `custom`, `brenda (wc=0, origin=1)`, `combined (sd=, n=)`, `isozymes` or `standard`. | -| `notes` | string array (Python: `list[str]`) | m | Free-text notes the user adds. | -| `eccodes` | string array (Python: `list[str]`) | m | EC numbers gathered from the ecModel and/or UniProt or KEGG, used for matching kcats by EC number. | -| `genes` | string array (Python: `list[str]`) | n | Gene identifiers, corresponding to the genes of the ecModel, matching the columns in `rxnEnzMat` (Python: `rxn_enz_mat`). | -| `enzymes` | string array (Python: `list[str]`) | n | UniProt protein identifiers derived from the matching entries in `genes`. | -| `mw` | float vector (Python: `numpy.ndarray`) | n | Molecular weight for each enzyme, in Dalton; unknown is `NaN`. | -| `sequence` | string array (Python: `list[str]`) | n | Amino acid sequence for each enzyme. | -| `concs` | float vector (Python: `numpy.ndarray`) | n | Measured concentration of each enzyme in mg/gDCW; unmeasured is `NaN`. | - -In Python, `ec.n_rxns` and `ec.n_enzymes` return the m and n sizes above, and `ec.gecko_light` is a boolean recording which layout the model uses; MATLAB has no fields with these names. - -## Troubleshooting - -| Problem | Possible reason | Solution | -|---|---|---| -| `No kcat source file ; it is left out` | a stage names a source that has no file in `data/kcats` | Gather the file with `updateKcatSources`, or remove the source from the stage | -| Kcats set with `setKcatForReactions` are gone after `assignKcats` | `assignKcats` replaces the kcats of the model | Place the kcats in `data/kcats/custom.tsv`, or set them after the assignment and apply the constraints again | -| `notMatch` of `applyCustomKcats` is not empty | the enzymes of an entry match the reaction in part | Curate `data/customKcats.tsv` or `custom.tsv` for the enzyme set of the reaction | -| `No kcat values are provided in model.ec.kcat` | every kcat is 0 when `getKcatAcrossIsozymes` or `applyKcatConstraints` runs; `applyKcatConstraints` also clears existing enzyme constraints, `getKcatAcrossIsozymes` leaves the model unchanged | Run `assignKcats` first | -| `convertCustomKcats` raises an error that `custom.tsv` exists | the target file is not overwritten | Remove the file or give another `outFile` | -| `getKcatAcrossIsozymes` raises an error on a light ecModel | isozymes are not split in a light ecModel | Skip the step; the constraint uses the most efficient isozyme | -| Growth rate far from the experimental value | sigma, $P_{tot}$ or f differs from the cell | See [Growth-rate tuning](growth-rate-tuning.md) | - -## See also - -- [Kcat assignment](../gecko4-kcat-assignment.md), the presets and stages of `assignKcats`. -- [Growth-rate tuning](growth-rate-tuning.md), the next step: diagnosing why growth is too low and tuning the model to a realistic rate. -- [Gathering kcats](gathering-kcats.md), where the kcat source files applied here come from. -- [API reference](../api/index.md), every function in both toolboxes. diff --git a/docs/guide/basic-model-operations.md b/docs/guide/basic-model-operations.md index d58ba81..4aafd56 100644 --- a/docs/guide/basic-model-operations.md +++ b/docs/guide/basic-model-operations.md @@ -57,7 +57,7 @@ ec_model.reactions.get_by_id("r_0003").lower_bound = 0 ::: :::: -Set the objective to maximize flux through the biomass reaction: +Set the objective to maximise flux through the biomass reaction: ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -97,7 +97,7 @@ sol = ec_model.optimize() ::: :::: -Perform parsimonious FBA, optimising the objective and minimizing the total sum of flux: +Perform parsimonious FBA, optimising the objective and minimising the total sum of flux: ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -118,10 +118,10 @@ sol = pfba(ec_model) ::: :::: -:::::{admonition} GECKO 4: enzyme-aware pFBA +:::::{admonition} Enzyme-aware pFBA :class: tip -Both languages also offer a variant that minimizes total *enzyme* usage, the L1 norm of the `usage_prot_*` fluxes, in place of the total flux of all reactions. It is not part of the original Nature Protocols procedure, and it serves the [enzyme-usage inspection](enzyme-usage-and-bottlenecks.md) done during simulation and analysis. It is not available for light ecModels, which have no `usage_prot_*` reactions to minimize. +`getPfbaEnzymes` (Python: `pfba_enzymes`) minimises the total enzyme usage, the L1 norm of the `usage_prot_*` fluxes, in place of the total flux of all reactions. [Enzyme usage and bottlenecks](enzyme-usage.md) uses it to find the flux distribution with the least enzyme mass. It is not available for light ecModels, which have no `usage_prot_*` reactions to minimise. ::::{tab-set} :::{tab-item} Ⓜ️ MATLAB @@ -223,12 +223,22 @@ import cobra cobra.io.write_sbml_model(ec_model, "filename.xml") ``` -As on [Building an empty ecModel](building-ec-model.md#save-the-ecmodel), this plain-cobrapy SBML export does not retain the `ec_model.ec` fields, the same caveat as MATLAB's `exportModel`. +As on [Build an ecModel](first-ecmodel.md#7-save-the-ecmodel), this plain-cobrapy SBML export does not retain the `ec_model.ec` fields, the same caveat as MATLAB's `exportModel`. ::: :::: +## Troubleshooting + +| Problem | Possible reason | Solution | +|---|---|---| +| Warning `Reactions not present in model, will be ignored` from `setParam` (Python: `KeyError` from `get_by_id`) | no reaction of the ecModel has this identifier; `makeEcModel` adds the suffixes `_REV` and `_EXP_` to split reactions | use the identifier of the ecModel, for example `r_0003_REV` for the reverse direction | +| Error `Invalid set of bounds for reaction(s)` from `setParam` (Python: `ValueError` from the bound setter) | the new lower bound is above the upper bound, or the reverse | change the other bound first: to raise both bounds, set the upper bound before the lower bound, and to lower both, the lower bound first (Python: set both at once with `Reaction.bounds = (lb, ub)`) | +| `sol.stat` is -1 and `sol.x` is empty (Python: `sol.status` is `infeasible`) | the bounds cannot be met together, for example a fixed growth rate above the growth that the protein pool allows | relax the fixed bounds one at a time to find the bound that makes the problem infeasible | +| Error from `getPfbaEnzymes` that a full ecModel is required (Python: `NotImplementedError`) | the ecModel is light and has no `usage_prot_` reactions | use `solveLP(ecModel, 1)` (Python: `pfba`), or build a full ecModel | +| An ecModel read from an SBML file has no `ec` structure | `exportModel` and `cobra.io.write_sbml_model` do not write the `ec` fields | save the ecModel with `saveEcModel` (Python: `save_ec_model`) as YAML for further GECKO work | + ## See also -- [Growth-rate tuning](growth-rate-tuning.md) -- [Simulation and analysis](simulation-and-analysis.md) -- [Enzyme usage and bottlenecks](enzyme-usage-and-bottlenecks.md) +- [Choose a kcat tuning](kcat-tuning.md) +- [Analyse an ecModel](analysis.md) +- [Enzyme usage and bottlenecks](enzyme-usage.md) diff --git a/docs/guide/building-ec-model.md b/docs/guide/building-ec-model.md deleted file mode 100644 index 520e9a7..0000000 --- a/docs/guide/building-ec-model.md +++ /dev/null @@ -1,394 +0,0 @@ -# Building an empty ecModel - -Building an ecModel expands a conventional GEM so that enzyme constraints can be added to it later. Reversible reactions split into forward and backward copies, because enzyme kinetics depend on direction. Reactions catalysed by isozymes split into one reaction per isozyme, because each has its own kinetics and molecular weight. The protein pool, together with the enzyme pseudo-metabolites that draw from it, is added to the network. The result is an **empty ecModel**: the structure that holds the enzyme constraints exists, and no constraints are applied yet. The empty ecModel is saved as a YAML file at the end. - - -:::::{dropdown} Place in the workflow - -The diagram orders the functions of this page within the [workflow](workflow.md). Dashed boxes are optional and dotted boxes are neighbouring pages. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - PV(["Previous: Getting started"]):::ctx - PV --> B1["ModelAdapterManager.setDefault
loadConventionalGEM"] --> B3{"Full or light?"} - B3 -->|full| B4["makeEcModel"] - B3 -->|light| B5["makeEcModel, light"] - B4 --> B6["getComplexData, applyComplexData (optional)
saveEcModel"] - B5 --> B6 - NX(["Next: Gathering kcats"]):::ctx - B6 --> NX -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - PV(["Previous: Getting started"]):::ctx - PV --> B1["ModelAdapter.from_folder
load_conventional_gem"] --> B3{"Full or light?"} - B3 -->|full| B4["make_ec_model"] - B3 -->|light| B5["make_ec_model, light"] - B4 --> B6["get_complex_data, apply_complex_data (optional)
save_ec_model"] - B5 --> B6 - NX(["Next: Gathering kcats"]):::ctx - B6 --> NX -``` -::: -:::: -::::: - -## Functions - -| MATLAB | Python | Note | -|---|---|---| -| `ModelAdapterManager.setDefault` | `ModelAdapter.from_folder` | load the model adapter of the project; MATLAB keeps it as the default of the session | -| `loadConventionalGEM` | `load_conventional_gem` | load the starting GEM from the path in the adapter | -| `makeEcModel` | `make_ec_model` | build the empty ecModel, full or light | -| `getComplexData` / `applyComplexData` | `get_complex_data` / `apply_complex_data` | add enzyme complex subunit stoichiometry | -| `saveEcModel` / `loadEcModel` | `save_ec_model` / `load_ec_model` | write the ecModel to YAML and read it back | - -## Setup - -The project folder with `model_adapter.toml` and the starting GEM exist (see [Getting started](getting-started.md)). `createGECKOproject` (MATLAB) and `geckopy init` (Python) create the folder. The adapter sets the path of the GEM (`conv_gem`), the compartment of the protein pseudo-metabolites (`enzyme_comp`, a compartment name), the UniProt and KEGG parameters and, for the complex data, `complex.taxonomic_id`. [Model adapter parameters](model-adapter-parameters.md) describes them. The examples use the `full_ecModel` tutorial of GECKO, and its data files are described in [Files in the data folder](data-folder-files.md). - -`makeEcModel` reads `data/uniprot.tsv` and `data/kegg.tsv` of the project and downloads a missing file when the identifier is not empty (`uniprot.id` and `kegg.id`, defaults 559292 and `sce`). An empty `uniprot.id` prints a warning, and the download is still attempted. An empty `kegg.id` prints a warning and skips the KEGG download. `make_ec_model` reads `data/uniprot.tsv` and raises a `FileNotFoundError` when it is missing, and it loads `data/kegg.tsv` when that file exists. - -In Python, `geckopy uniprot-download` takes the UniProt identifier and the output path, for example `geckopy uniprot-download 559292 data/uniprot.tsv`. The option `--id-type` sets the field of the UniProt query (default `taxonomy_id`; `proteome` for a proteome identifier such as `UP000002311`), and `--gene-id-field` sets the gene identifier field (default `gene_oln`). `geckopy kegg-download eco data/kegg.tsv` creates `data/kegg.tsv`, a comma-delimited file that `make_ec_model` loads under that name. - -## Set the default model adapter - -Most GECKO functions need the model adapter. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -Set a default with `ModelAdapterManager` to avoid passing the adapter to every call. `ModelAdapterManager.setDefault` takes the project folder, its `model_adapter.toml` or the file of an adapter class. The default then applies to every GECKO function for the rest of that MATLAB session, unless another adapter is given explicitly: - -```matlab -adapterLocation = fullfile(findGECKOroot, 'tutorials', 'full_ecModel'); -ModelAdapterManager.setDefault(adapterLocation); -``` - -Retrieve the default explicitly when needed, for example when simulating several ecModels in the same session: - -```matlab -ModelAdapter = ModelAdapterManager.getDefault(); -params = ModelAdapter.getParameters(); -``` - -The rest of this guide assumes a default model adapter is set. - -The adapter is read when it is set. After a change in `model_adapter.toml`, set it as default again: - -```matlab -ModelAdapterManager.setDefault(adapterLocation); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -geckopy has no session-wide default adapter. An `EcModel` holds its adapter as `model.adapter`, which functions use when no adapter is passed. Load the adapter once and pass it, or its `.params` attribute, to each call: - -```python -from pathlib import Path -from geckopy import ModelAdapter - -adapter = ModelAdapter.from_folder(Path("GECKO/tutorials/full_ecModel")) -params = adapter.params -``` - -If the adapter file changes, reload it by calling `from_folder` again; there is no separate "set as default" step to repeat. -::: -:::: - -## Load the conventional GEM - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -If the location of the model is set in the adapter (the parameter `convGEM`, `conv_gem` in `model_adapter.toml`): - -```matlab -model = loadConventionalGEM(); -``` - -Both YAML and XML files are supported. To load a model at a different location, or without a model adapter, use the RAVEN function directly, for XML: - -```matlab -model = importModel('path/to/modelFile.xml'); -``` - -or YAML: - -```matlab -model = readYAMLmodel('path/to/modelFile.yml'); -``` - -GECKO loads models through RAVEN by default. A model loaded through the COBRA Toolbox (recognizable by a `model.rules` field) needs converting to the RAVEN format (recognizable by a `model.metComps` field) first, with `ravenCobraWrapper()`. `makeEcModel` stops with an error for a model that has the field `rules` or `modelID`. - -Not every GEM in SBML format strictly follows the L3V1 FBCv2 standard; the [SBML Validator](https://sbml.bioquant.uni-heidelberg.de/validator_servlet/) reports what is wrong. In most cases the model still loads despite the warning. If it is later exported to SBML again, RAVEN avoids re-writing the same invalid content, so cycling through `importModel`, `exportModel` and `importModel` brings the loaded GEM closer to valid SBML. -::: -:::{tab-item} 🐍 Python -:sync: python - -If the location of the model is set in the adapter (`params.conv_gem`): - -```python -from geckopy import load_conventional_gem - -model = load_conventional_gem(adapter) -``` - -This returns a `cobra.Model` with the solver tolerance of RAVEN (1e-9). The extension of the file selects the reader: `.yml` and `.yaml` through raven-toolbox, `.json` and `.mat` through cobrapy, and any other extension as SBML. Cobrapy is the native format throughout geckopy, so no conversion step is needed. To load a model at a different location or without an adapter, use cobrapy directly: - -```python -import cobra - -model = cobra.io.read_sbml_model("path/to/modelFile.xml") -``` - -or, for a YAML file in the RAVEN and GECKO schema, raven-toolbox's reader (installed automatically with geckopy): - -```python -from raven_toolbox.io import read_yaml_model - -model = read_yaml_model("path/to/modelFile.yml") -``` -::: -:::: - -## Choose full or light, then build - -A full ecModel expands isozymes into separate reactions and adds each enzyme as its own pseudo-metabolite, which allows each enzyme to be constrained individually, for example from proteomics data. A light ecModel does neither: it keeps the isozymes of a reaction as rows of `ec` and applies the lowest-cost isozyme to the shared protein pool, which makes it smaller and faster to simulate. The two layouts are not interconvertible, and the choice affects every later step. See [GECKO light and full ecModels](gecko-light.md) for the differences. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -Full ecModel: - -```matlab -[ecModel, noUniprot] = makeEcModel(model); -``` - -Light ecModel: - -```matlab -[ecModel, noUniprot] = makeEcModel(model, true); -``` - -The optional argument is `geckoLight`, also given as `makeEcModel(model, 'geckoLight', true)`. - -`noUniprot` lists the model genes that match neither `data/uniprot.tsv` nor `data/kegg.tsv`. `makeEcModel` raises an error when no gene matches. If `noUniprot` contains many genes, for example more than ten, a different UniProt taxonomy or proteome identifier (see [The model adapter](getting-started.md#the-model-adapter)) may fit better. -::: -:::{tab-item} 🐍 Python -:sync: python - -Full ecModel (`gecko_light` defaults to `False`): - -```python -from geckopy import make_ec_model - -ec_model = make_ec_model(model, adapter) -``` - -Light ecModel: - -```python -ec_model = make_ec_model(model, adapter, gecko_light=True) -``` - -UniProt data loads automatically from `params.path / "data" / "uniprot.tsv"`. Pass a pre-loaded `uniprot_db=` to use a different file or to avoid re-reading it across multiple calls. KEGG data loads from `params.path / "data" / "kegg.tsv"` when that file exists; pass a pre-loaded `kegg_db=` to use another source. `make_ec_model` leaves the input `model` unchanged and returns the built `EcModel`. - -`make_ec_model` returns the built `EcModel` and no list of unmatched genes. Unmatched genes are logged as a warning summary and recorded on the affected reactions in `reaction.notes["geckopy_warning"]`. As in MATLAB, many unmatched genes are a reason to reconsider the UniProt taxonomy or proteome identifier. -::: -:::: - -The build consults the KEGG data for genes that UniProt does not match. `ec.enzymes` receives the UniProt accession on the matching KEGG row, or the KEGG gene identifier when that row has no accession; the build warns that the bare KEGG gene identifier is used in `ec.enzymes` for these genes. - -The result is an empty ecModel: the model structure allows enzyme constraints (see [the ec structure](applying-kcats.md#the-ec-structure)), and no constraints are applied yet. - -## Apply enzyme complex stoichiometry (optional) - -Building the ecModel assigns one copy of every subunit to each enzyme complex in `ec.rxnEnzMat` (Python: `ec.rxn_enz_mat`). In reality subunit copy numbers vary. GECKO adds this information from the [Complex Portal](https://www.ebi.ac.uk/complexportal/complex/organisms), using the taxonomic identifier in the model adapter parameter `complex.taxonomicID` (Python: `complex.taxonomic_id`); the Complex Portal covers a limited set of taxonomic identifiers. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -complexInfo = getComplexData(); -[ecModel, foundComplex, proposedComplex] = ... - applyComplexData(ecModel, complexInfo); -``` - -`getComplexData` takes the optional arguments `taxonomicID` and `modelAdapter`. Without them, the taxonomic identifier comes from the default model adapter, and the function writes `data/ComplexPortal.json` in the project folder. `applyComplexData` reads that file when called without `complexInfo`. `foundComplex` and `proposedComplex` are tables of the complexes with a full match and the proposed complexes. - -MATLAB prints the number of full and proposed matches: - -``` -A total of 238 complex have full match, and 22 proposed. -``` - -The counts depend on the Complex Portal release that `getComplexData` downloads. The `data/ComplexPortal.json` file included in the tutorial gives 208 full matches and 17 proposed complexes. -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import apply_complex_data - -apply_complex_data(ec_model, path=params.path / "data" / "ComplexPortal.json") -``` - -`apply_complex_data` changes `ec_model` in place and returns `None`. Proposed complexes are logged as warnings, and the number of full and proposed matches is logged at INFO level as `apply_complex_data: N full match(es), M proposed.`. The keyword argument `min_match_to_propose` (default 0.75) sets the minimum fraction of matching genes for a partial match to be proposed. With `apply=True` (default) the function runs `apply_kcat_constraints` afterwards when at least one complex matched. To (re)download the Complex Portal data first, use `get_complex_data`, which takes the taxonomic identifier and returns the complexes. `write_to` stores them as `ComplexPortal.json`: - -```python -from geckopy import get_complex_data - -get_complex_data( - params.complex.taxonomic_id, - write_to=params.path / "data" / "ComplexPortal.json", -) -``` -::: -:::: - -Complex data is integrated for a reaction when its ecModel gene association fully (100 %) matches a Complex Portal complex. `applyComplexData` changes `ec.rxnEnzMat`; the coefficients in the stoichiometric matrix follow from `applyKcatConstraints` (see [Applying kcat values](applying-kcats.md)). For a reaction with more than one enzyme and no full match, complexes with a partial match are proposed (MATLAB: in `proposedComplex`; Python: in the log). A complex is proposed when at least 75 % of the ecModel genes of the reaction match a Complex Portal complex that has no more subunits than the ecModel gene set, or when all genes match and the Complex Portal complex contains more subunits. Inspect the proposed complexes and consider whether curating the ecModel gene association is appropriate; this may need further literature study. - -:::{admonition} Example output -:class: note - -In the `full_ecModel` tutorial, applying the Complex Portal data file included in the tutorial finds 208 complexes with a full match and 17 proposed complexes (MATLAB and Python agree). One proposed match: reaction `r_0505_EXP_2` could match complex CPX-1293, the mitochondrial 2-oxoglutarate dehydrogenase complex. In the ecModel this reaction is annotated with proteins P20967 (KGD1, subunit E1), P19262 (KGD2, subunit E2) and P09624 (LPD1, subunit E3), while CPX-1293 also includes P19955 (YMR31), the 37S ribosomal protein. Literature confirms that P19955, also called KGD4 or subunit E4, is an essential subunit for a stable complex, so the gene association of reaction `r_0505` in the starting GEM could be curated to include it, which would give a full match. -::: - -:::{admonition} Complex data is missing for some or all complexes -:class: note - -The Complex Portal does not contain every complex; some are missing in most organisms. Uncertainty in kcat values is a larger source of error than uncertainty in complex stoichiometry, so skipping this step and keeping the default stoichiometry of one for every subunit is a reasonable option. `ComplexPortal.json` can be complemented manually with complex information from other databases or the literature. -::: - -## Save the ecModel - -The ecModel can be saved at any point in the procedure. YAML retains the full `ec` fields, so it is the format to use whenever the ecModel is modified further. The file structure is described in [ecModel YAML format](yaml-format.md). - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -`saveEcModel` writes to the `models` folder of the adapter folder and adds the keys `gecko_format` and `generator` to the file. The generic `writeYAMLmodel` writes anywhere and adds neither. `loadEcModel` or the generic `readYAMLmodel` read the model back. `loadEcModel` turns the protein reactions of an ecModel from GECKO 3 forward and stops for a file of a newer ecModel format: - -```matlab -saveEcModel(ecModel, 'ecModel.yml'); -writeYAMLmodel(ecModel, 'C:\path\to\ecModel.yml'); -ecModel = loadEcModel('ecModel.yml'); -ecModel = readYAMLmodel('C:\path\to\ecModel.yml'); -``` - -For constraint-based analysis in other software, SBML (with an XML extension) is often more suitable. `saveEcModel` writes SBML too, when `filename` ends in `.xml`; `exportModel` and `importModel` do the same for an arbitrary path. Either way, the file does not retain the `ecModel.ec` fields, so an SBML-saved ecModel cannot be loaded back into MATLAB for further GECKO functions: - -```matlab -saveEcModel(ecModel, 'ecModelFull.xml'); -exportModel(ecModel, 'C:\path\to\ecModelFull.xml'); -ecModel = importModel('C:\path\to\ecModelFull.xml'); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -`save_ec_model` writes YAML, either in the `models` folder of the adapter folder or at an arbitrary path, and adds the keys `gecko_format` and `generator`; `load_ec_model` reads it back and turns the protein reactions of an ecModel from GECKO 3 forward: - -```python -from geckopy import load_ec_model, save_ec_model - -save_ec_model(ec_model, "ecModel.yml", adapter=adapter) -save_ec_model(ec_model, r"C:\path\to\ecModel.yml") -ec_model = load_ec_model("ecModel.yml", adapter=adapter) -ec_model = load_ec_model(r"C:\path\to\ecModel.yml") -``` - -When `ec_model.adapter` is set, `save_ec_model` uses it and the argument `adapter` can be omitted. - -geckopy provides no separate SBML export for ecModels: YAML is the sole format that preserves the `ec` fields on writing and reading, and `save_ec_model` raises a `ValueError` for another extension and for a model whose `ec` is empty. Because `ec_model` is a `cobra.Model`, plain cobrapy SBML I/O still works for interoperating with other tools, with the same limit as MATLAB's `exportModel`: it does not retain the `ec` fields. - -```python -import cobra - -cobra.io.write_sbml_model(ec_model, r"C:\path\to\ecModelFull.xml") -model = cobra.io.read_sbml_model(r"C:\path\to\ecModelFull.xml") -``` -::: -:::: - -## Extension of a conventional GEM - -Converting a conventional GEM into an empty ecModel runs up to twelve operations in order. Three are skipped for light ecModels: the split by isozyme, the enzyme pseudo-metabolites and the usage reactions. A light ecModel therefore stays close to the size of the conventional GEM. - -:::{dropdown} All twelve steps, full and light -:open: - -| # | Operation | Full | Light | -|---|---|:---:|:---:| -| 1 | Remove gene associations from pseudo-reactions (name contains `pseudoreaction`, or listed in `data/pseudoRxns.tsv`) | βœ… | βœ… | -| 2 | Invert irreversible reactions that run backwards (lower bound < 0, upper bound = 0) | βœ… | βœ… | -| 3 | Determine which reactions are reversible from their bounds; a reaction with a single metabolite (exchange type) is also marked reversible (MATLAB stores this as the ecModel field `rev`; geckopy has no separate reversibility vector) | βœ… | βœ… | -| 4 | Split reversible reactions into forward and reverse copies (`_REV` suffix on the reverse copy; exchange reactions keep their original, still-reversible form) | βœ… | βœ… | -| 5 | Split isozyme-catalysed reactions (`or` in the gene-reaction rule) into one reaction per isozyme (`_EXP_1`, `_EXP_2`, …) | βœ… | -- | -| 6 | Build an empty `ec` structure (Python: an `EcData` instance) | βœ… | βœ… | -| 7 | Add enzyme MW and sequence to `ec`, from UniProt via the model adapter, and from KEGG for genes that UniProt does not match | βœ… | βœ… | -| 8 | Record reaction-enzyme associations in `ec.rxnEnzMat` (Python: `ec.rxn_enz_mat`) | βœ… | βœ… | -| 9 | Add each enzyme as a `prot_` pseudo-metabolite | βœ… | -- | -| 10 | Add the protein pool pseudo-metabolite | βœ… | βœ… | -| 11 | Add `usage_prot_` reactions for each enzyme pseudo-metabolite, from `prot_pool` to the enzyme (bounds 0 and 1000) | βœ… | -- | -| 12 | Add the `prot_pool_exchange` reaction, which supplies `prot_pool` (bounds 0 and 1000) | βœ… | βœ… | - -Step 5's new reaction identifiers: reaction `r_0001` catalysed by two isozymes becomes `r_0001_EXP_1` and `r_0001_EXP_2`; a backward reaction `r_0001_REV` becomes `r_0001_REV_EXP_1` and `r_0001_REV_EXP_2`. Light ecModels keep the original identifiers in the model, and the rows of `ec.rxns` have the prefix `001_`, `002_` and so on, one per isozyme. Step 9's example: *S. cerevisiae* enolase gene YHR174W, UniProt identifier P00925, appears as pseudo-metabolite `prot_P00925`. Step 11's identifiers are `usage_` followed by the enzyme metabolite identifier, for example `usage_prot_P00925`. See [the ec structure](applying-kcats.md#the-ec-structure) for what step 6 populates later, and [GECKO light and full ecModels](gecko-light.md) for what the skipped steps change. -::: - -:::{admonition} Usage and pool exchange reactions run forward -:class: tip - -`usage_prot_` consumes `prot_pool` to produce `prot_`, and `prot_pool_exchange` supplies `prot_pool`, both with bounds from 0 to 1000 at the build. An ecModel of GECKO 3 defines both reactions in the reverse direction with negative flux (see [GECKO 3 to GECKO 4](../gecko3-to-gecko4.md)). The direction determines which bound relaxes a constraint and which objective coefficient minimises usage; the worked example is in [Growth-rate tuning](growth-rate-tuning.md#protein-pool-constraint). -::: - -## Troubleshooting - -| Problem | Possible reason | Solution | -|---|---|---| -| Error that the identifiers in `model.mets` or `model.rxns` are not allowed (Python: `ValueError`) | A metabolite identifier starts with `prot_`, or a reaction identifier starts with `usage_prot_` or `prot_pool` or ends with `_REV` or `_EXP_` | Rename the identifiers in the conventional GEM | -| Error that a compartment cannot be found in `model.compNames` | `enzyme_comp` differs from every compartment name of the model | Set `enzyme_comp` to a compartment name | -| Error that no protein in `uniprot.tsv` or `kegg.tsv` matches the genes | The UniProt or KEGG parameters do not fit the gene identifiers | Adjust `uniprot` and `kegg` in `model_adapter.toml` (see [Getting started](getting-started.md#the-model-adapter)) | -| `FileNotFoundError` for `uniprot.tsv` (Python) | The file is not in `data/` | Run `geckopy uniprot-download 559292 data/uniprot.tsv` with the identifier of the organism | -| `ValueError` that the model already has a populated `ec` (Python) | `make_ec_model` was called on an ecModel | Call it on the conventional GEM | -| Warning `No taxonomicID specified` from `getComplexData` | `complex.taxonomicID` is not set | Set it in `model_adapter.toml` or give `taxonomicID` | - -## See also - -- [Gathering kcats](gathering-kcats.md), the next step: sourcing kcat values from BRENDA, DLKcat or OpenKineticsPredictor. -- [GECKO light and full ecModels](gecko-light.md), the differences between the two ecModel structures. -- [ecModel YAML format](yaml-format.md), the structure of the saved file. -- [API reference](../api/index.md), every function in both toolboxes. diff --git a/docs/guide/context-ecmodel.md b/docs/guide/context-ecmodel.md new file mode 100644 index 0000000..a033282 --- /dev/null +++ b/docs/guide/context-ecmodel.md @@ -0,0 +1,141 @@ +# Make a context-specific ecModel + +A context-specific GEM describes the metabolism of one tissue, cell line or strain. It is made from a generic GEM with a method such as ftINIT (`ftINIT` in RAVEN, `ftinit` in raven-toolbox). `getSubsetEcModel` trims a generic ecModel to the reactions and genes of a context-specific GEM, and `setProtPoolSize` sets the protein pool of the context. + + +:::::{dropdown} Place in the workflow + +The diagram orders the functions of this page within the [workflow](workflow.md). Boxes with a dashed outline are optional steps, a dotted arrow is an optional path, and a diamond is a choice. Rounded boxes with a dotted outline are the context, other pages or the model to start from, and a click on one opens its page. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Start: a generic ecModel without proteomics constraints"]):::ctx + OR{"Order"} + BE(["Build an ecModel
with the context-specific GEM"]):::ctx + GS["getSubsetEcModel
with the context-specific GEM"] + SP["setProtPoolSize
protein content of the context"]:::opt + PV --> OR + OR -->|context-specific GEM first| BE + OR -->|generic ecModel first| GS --> SP + NX(["Next: Constrain enzymes with proteomics of the context (optional)"]):::ctx + GS --> NX + SP --> NX + click PV href "first-ecmodel.html" + click BE href "first-ecmodel.html" + click NX href "proteomics.html" +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Start: a generic ecModel without proteomics constraints"]):::ctx + OR{"Order"} + BE(["Build an ecModel
with the context-specific GEM"]):::ctx + GS["get_subset_ec_model
with the context-specific GEM"] + SP["set_prot_pool_size
protein content of the context"]:::opt + PV --> OR + OR -->|context-specific GEM first| BE + OR -->|generic ecModel first| GS --> SP + NX(["Next: Constrain enzymes with proteomics of the context (optional)"]):::ctx + GS --> NX + SP --> NX + click PV href "first-ecmodel.html" + click BE href "first-ecmodel.html" + click NX href "proteomics.html" +``` +::: +:::: +::::: + +## Functions + +| MATLAB | Python | Note | +| --- | --- | --- | +| `getSubsetEcModel` | `get_subset_ec_model` | trim a generic ecModel to the reactions and genes of a context-specific GEM | +| `setProtPoolSize` | `set_prot_pool_size` | set the upper bound of the protein pool exchange | + +## Setup + +The examples use the generic ecModel and the model adapter parameters `params`, loaded as in the setup of [Analyse an ecModel](analysis.md#setup). The other names in the examples are placeholders for values of the project: + +| MATLAB | Python | Value | +| --- | --- | --- | +| `contextGEM` | `context_gem` | a context-specific GEM, made from the same GEM (and the same version) as the ecModel | +| `PtotContext` | `p_tot_context` | total protein content of the context, in g/gDCW | + +## Choose the order + +There are two ways to get an ecModel of a context: + +| Order | What it needs | Result | +| --- | --- | --- | +| make the context-specific GEM, then build an ecModel from it | the full ecModel workflow per context | each context has its own kcat assignment | +| build one generic ecModel, then trim it with `getSubsetEcModel` | the generic ecModel and the context-specific GEM | all contexts have the kcats of the generic ecModel | + +The second way builds the enzyme data once, which saves time when many contexts are made from one generic model, and gives all contexts the same kcats. The sections below describe the second way. For the first way, follow [Build an ecModel](first-ecmodel.md) with the context-specific GEM as the GEM of the project. + +## Trim the generic ecModel + +`getSubsetEcModel` removes the genes that the context-specific GEM does not have, with the enzymes and reactions that depend on them, and then the reactions whose identifier (without `_REV` and `_EXP_`) is not in the context-specific GEM. The `usage_prot_*` reactions of the remaining enzymes and `prot_pool_exchange` are kept, with their bounds. Both models must come from the same GEM: a reaction of the context-specific GEM that the ecModel does not have raises an error. The function warns when a `usage_prot_*` reaction of the generic ecModel has an upper bound other than 1000, because enzyme concentrations of the generic model may not apply to the context. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +ecContext = getSubsetEcModel(ecModel, contextGEM); +ecContext = setProtPoolSize(ecContext, PtotContext); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import get_subset_ec_model, set_prot_pool_size + +ec_context = get_subset_ec_model(ec_model, context_gem) +set_prot_pool_size(ec_context, p_tot=p_tot_context) +``` + +`get_subset_ec_model` returns a new `EcModel`; `ec_model` is not changed. +::: +:::: + +The upper bound of `prot_pool_exchange` is copied from the generic ecModel. Call `setProtPoolSize` with the total protein content of the context when it is different ([Set the protein pool](protein-pool.md)). Without the values `f` and `sigma`, `setProtPoolSize` reads them from the model adapter. + +## Troubleshooting + +| Problem | Possible reason | Solution | +| --- | --- | --- | +| `getSubsetEcModel` raises an error that lists reactions | the context-specific GEM was made from another GEM or another version of it | Make the context-specific GEM from the GEM of the ecModel | +| Warning that the ecModel is constrained by protein concentrations | the generic ecModel has measured enzyme concentrations as usage bounds | Trim an ecModel without proteomics constraints, and add the proteomics data of the context afterwards ([Constrain enzymes with proteomics](proteomics.md)) | + +## See also + +- [Analyse an ecModel](analysis.md): the analyses to run on the context-specific ecModel, and the shared setup. +- [Set the protein pool](protein-pool.md): `Ptot`, `f` and `sigma` of the protein pool. +- [API reference](../api/index.md) and [MATLAB vs Python](../matlab-vs-python.md): the signature of each function in the table. diff --git a/docs/guide/custom-kcats.md b/docs/guide/custom-kcats.md new file mode 100644 index 0000000..27dc77d --- /dev/null +++ b/docs/guide/custom-kcats.md @@ -0,0 +1,190 @@ +# Set custom kcats + +`data/kcats/custom.tsv` holds the curated kcats of a project, for example a value from the literature or a value kept after [kcat tuning](kcat-tuning.md). The result is a `custom.tsv` that `assignKcats` reads before all other sources, and an ecModel with the custom kcats applied. + + +:::::{dropdown} Place in the workflow + +The diagram orders the functions of this page within the [workflow](workflow.md). Boxes with a dashed outline are optional steps, a dotted arrow is an optional path, and a diamond is a choice. Rounded boxes with a dotted outline are the context, other pages or the model to start from, and a click on one opens its page. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Previous: Choose the ecModel structure"]):::ctx + PT(["Choose a kcat tuning: keep a tuned kcat"]):::ctx + G3["convertCustomKcats
required when the GECKO 3 file data/customKcats.tsv exists"]:::opt + WR["write data/kcats/custom.tsv"] + AS["assignKcats
custom kcats first"] + HB["setKcatForReactions
applyKcatConstraints
(model only, lost at the next assignKcats)"]:::opt + PV --> WR --> AS + PV --> G3 --> WR + PT -.-> WR + AS -.->|change by hand| HB + NX(["Next: Set the protein pool"]):::ctx + AS --> NX + click PV href "ecmodel-structure.html" + click PT href "kcat-tuning.html" + click NX href "protein-pool.html" +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Previous: Choose the ecModel structure"]):::ctx + PT(["Choose a kcat tuning: keep a tuned kcat"]):::ctx + G3["convert_custom_kcats
required when the GECKO 3 file data/customKcats.tsv exists"]:::opt + WR["write data/kcats/custom.tsv"] + AS["assign_kcats
custom kcats first"] + HB["set_kcat_for_reactions
apply_kcat_constraints
(model only, lost at the next assign_kcats)"]:::opt + PV --> WR --> AS + PV --> G3 --> WR + PT -.-> WR + AS -.->|change by hand| HB + NX(["Next: Set the protein pool"]):::ctx + AS --> NX + click PV href "ecmodel-structure.html" + click PT href "kcat-tuning.html" + click NX href "protein-pool.html" +``` +::: +:::: +::::: + +## Functions + +| MATLAB | Python | Note | +| --- | --- | --- | +| `assignKcats` | `assign_kcats` | determine the kcats, with the custom kcats first, and apply them; the report lists the rows that were not applied | +| `convertCustomKcats` | `convert_custom_kcats` | convert `data/customKcats.tsv` of a GECKO 3 project into `data/kcats/custom.tsv` | +| `setKcatForReactions` | `set_kcat_for_reactions` | change `ec.kcat` of named reactions in the model | +| `applyKcatConstraints` | `apply_kcat_constraints` | apply `ec.kcat` to the stoichiometry | + +## Setup + +The examples start from an ecModel made with `makeEcModel` (Python: `make_ec_model`), full or light. In MATLAB, the project is the default model adapter (`ModelAdapterManager.setDefault`); in Python, `ec_model.adapter` holds the project. The other source files of the preset are in `data/kcats` ([Choose the kcat sources](kcat-sources.md#gather-the-source-files)); `assignKcats` leaves a missing source file out. + +## Write custom.tsv + +Every preset reads `custom.tsv` in its first stage ([Choose the kcat sources](kcat-sources.md)), so a custom kcat is used before all other sources. `updateKcatSources` and `saveKcatSource` do not write this file. + +The file is tab-separated. It needs the columns `kcat` and `source`, and `rxn`, `enzymes` or both. The other columns of a kcat source file (`substrate`, `log10_sd`, `detail`, `ec_code`, `note`) are optional. Lines that start with `#` before the column header are header lines. + +| Column | Content | +| --- | --- | +| `rxn` | the identifier of the reaction in the conventional GEM, with `_REV` for the reverse direction. Without `enzymes`, the row applies to every isozyme of the reaction | +| `enzymes` | the UniProt accessions of the enzyme, joined by ` + ` in any order. Without `rxn`, the row applies to every reaction whose enzyme set is this set | +| `kcat` | the kcat in s^-1 | +| `source` | where the value comes from, for example `custom`; `ec.source` holds it in lower case, with each run of characters other than letters and digits replaced by `_` | +| `note` | free text, for example the reference | + +```text +rxn enzymes kcat source note +R1 P12345 120 custom measured value of the organism +R2_REV 35 custom value for all isozymes +``` + +`R1`, `R2` and `P12345` stand for reactions of the GEM and a UniProt accession of the organism. The rows are keyed by the conventional reaction and the enzyme set, which do not depend on how the ecModel is built. The file therefore stays valid when the ecModel is rebuilt, and it applies to full and light ecModels. [Project format 4](../gecko4-project-format.md#kcat-source-files) describes the format of all kcat source files. + +## Apply the custom kcats + +`assignKcats` reads `custom.tsv` with the other source files. A row whose kcat is empty, is text other than a number, or is 0 or below is not applied. A row that matches no reaction of the ecModel, or that matches part of an enzyme set, is not applied either. The report lists these rows per source file: + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +[ecModel, decisions, report] = assignKcats(ecModel); +report.sources.custom.invalid +report.sources.custom.unmatched +report.sources.custom.partial +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import assign_kcats + +decisions, report = assign_kcats(ec_model) +custom = report["sources"]["custom"] +print(custom["invalid"], custom["unmatched"], custom["partial"]) +``` +::: +:::: + +A row in `partial` shares at least half of its enzymes with a reaction of the ecModel. The table names that reaction and its enzymes, so the row can be corrected to the enzyme set of the model. + +## Convert the custom kcats of a GECKO 3 project + +`convertCustomKcats` (Python: `convert_custom_kcats`) writes the custom kcats of `data/customKcats.tsv` as `data/kcats/custom.tsv`; it needs the full ecModel and does not overwrite an existing `custom.tsv`. [Upgrade a GECKO 3 project](../gecko3-upgrade-project.md) describes this conversion with the other steps of an upgrade. + +## Change kcats by hand + +`setKcatForReactions` changes `ec.kcat` of the named reactions, and `applyKcatConstraints` applies the change to the stoichiometry. In a full ecModel, a reaction name without `_EXP_` changes all isozymes of the reaction. In a light ecModel, the names in `ec.rxns` have a number prefix (`001_R1`), and the name is given with this prefix. A name that matches no entry of `ec.rxns` is ignored without a message. `ec.source` of a changed reaction reads `manual`. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +ecModel = setKcatForReactions(ecModel, {'R1'}, 120); +ecModel = applyKcatConstraints(ecModel); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import apply_kcat_constraints, set_kcat_for_reactions + +set_kcat_for_reactions(ec_model, ["R1"], 120) +apply_kcat_constraints(ec_model) +``` +::: +:::: + +:::{warning} +`assignKcats` replaces all kcats of the model with the values from the source files. A kcat set with `setKcatForReactions` is lost when `assignKcats` runs again. Write a kcat that must stay in the model to `data/kcats/custom.tsv`. +::: + +## Troubleshooting + +| Problem | Possible reason | Solution | +| --- | --- | --- | +| A custom kcat is not used | the row matches no reaction, or part of an enzyme set | Read `report.sources.custom` of `assignKcats` and correct `rxn` or `enzymes` | +| A custom row is listed as invalid | the kcat is empty, 0, or not a plain number (for example `5,5` with a decimal comma) | Write the kcat as a number with a decimal point | +| Error `loadKcatSource:columns` (Python: `ValueError` with `needs the columns kcat and source`) | the file lacks `kcat`, `source`, or both `rxn` and `enzymes` | Add the columns to the header line | +| Error `collectKcats:legacyCustomKcats` (Python: `ValueError`) | the project has the GECKO 3 file `data/customKcats.tsv` and no `data/kcats/custom.tsv` | Run `convertCustomKcats` | +| Kcats set by hand are gone | `assignKcats` replaced them | Write them to `data/kcats/custom.tsv` | +| Error `convertCustomKcats:exists` (Python: `FileExistsError`) | the function does not overwrite the file | Remove the file, or give another `outFile` (Python: `out_file`) | +| Error `convertCustomKcats:light` (Python: `ValueError`) | the conversion needs the full ecModel | Convert with the full ecModel; the `custom.tsv` written also applies to the light ecModel | + +## See also + +- [Choose the kcat sources](kcat-sources.md): the presets and the order of the sources. +- [Choose a kcat tuning](kcat-tuning.md): tuned kcats to check and keep in `custom.tsv`. +- [Project format 4](../gecko4-project-format.md): the layout of `data/kcats` and of the kcat source files. diff --git a/docs/guide/data-folder-files.md b/docs/guide/data-folder-files.md index 8ef203d..e306de8 100644 --- a/docs/guide/data-folder-files.md +++ b/docs/guide/data-folder-files.md @@ -2,48 +2,48 @@ -The `data` folder of a GECKO 4 project holds the input data of the reconstruction, the kcat source files and the files of the kcat predictors. GECKO (MATLAB) and geckopy (Python) read and write the same files, so a `data` folder made with one toolbox is used by the other. [Getting started](getting-started.md) describes the layout of a project and [Project format 4](../gecko4-project-format.md) the folders outside `data`. +The `data` folder of a GECKO 4 project holds the input data of the reconstruction, the kcat source files and the files of the kcat predictors. GECKO (MATLAB) and geckopy (Python) read and write the same files, so a `data` folder made with one toolbox is used by the other. [Build an ecModel](first-ecmodel.md) describes the layout of a project and [Project format 4](../gecko4-project-format.md) the folders outside `data`. ## Databases and gene identifiers | File | Content | Functions | Page | |---|---|---|---| -| `uniprot.tsv` | UniProt data for the organism: protein identifier, gene identifier, EC number, MW and sequence. `loadDatabases` downloads it when it is missing | `loadDatabases` (Python: `geckopy uniprot-download`) writes; `makeEcModel`, `getStandardKcat` and `calculateFfactor` read | [Getting started](getting-started.md), [Building an empty ecModel](building-ec-model.md) | -| `kegg.tsv` | KEGG data for the organism: UniProt protein ID, gene, KEGG gene ID, EC number, MW, pathway and sequence. `loadDatabases` downloads it when it is missing | `loadDatabases` (Python: `geckopy kegg-download`) writes; `makeEcModel` (`make_ec_model`) reads | [Getting started](getting-started.md), [Gathering kcats](gathering-kcats.md) | -| `uniprotConversion.tsv` | Conversion table for models whose gene identifiers match no UniProt field. A header row, then the gene identifier and the UniProt identifier. A gene that is not listed keeps its identifier in geckopy and gets an empty UniProt identifier in MATLAB. Written by the user | `ModelAdapter.getUniprotIDsFromTable` (Python: `get_uniprot_ids_from_table`) | [Getting started](getting-started.md) | -| `pseudoRxns.tsv` | Optional. Reaction identifiers in the first column and reaction names, for reference, in the second. These reactions lose their gene association in `makeEcModel` (`make_ec_model`) and receive no standard kcat in `getStandardKcat` (`assign_standard_kcat`). Written by the user | `makeEcModel`, `getStandardKcat` | [Building an empty ecModel](building-ec-model.md), [Applying kcats](applying-kcats.md) | -| `ComplexPortal.json` | Enzyme complex information from the Complex Portal | `getComplexData` (Python: `get_complex_data`) writes; `applyComplexData` (`apply_complex_data`) reads | [Building an empty ecModel](building-ec-model.md) | -| `smilesDB.tsv` | SMILES for the metabolites of the ecModel, a cache of the PubChem queries | `findMetSmiles` (Python: `find_met_smiles`) | [Gathering kcats](gathering-kcats.md) | +| `uniprot.tsv` | UniProt data for the organism: protein identifier, gene identifier, EC number, MW and sequence. `loadDatabases` downloads it when it is missing | `loadDatabases` (Python: `geckopy uniprot-download`) writes; `makeEcModel`, `getStandardKcat` and `calculateFfactor` read | [Build an ecModel](first-ecmodel.md), [Choose the ecModel structure](ecmodel-structure.md) | +| `kegg.tsv` | KEGG data for the organism: UniProt protein ID, gene, KEGG gene ID, EC number, MW, pathway and sequence. `loadDatabases` downloads it when it is missing | `loadDatabases` (Python: `geckopy kegg-download`) writes; `makeEcModel` (`make_ec_model`) reads | [Build an ecModel](first-ecmodel.md), [Choose the kcat sources](kcat-sources.md) | +| `uniprotConversion.tsv` | Conversion table for models whose gene identifiers match no UniProt field. A header row, then the gene identifier and the UniProt identifier. A gene that is not listed keeps its identifier in geckopy and gets an empty UniProt identifier in MATLAB. Written by the user | `ModelAdapter.getUniprotIDsFromTable` (Python: `get_uniprot_ids_from_table`) | [Build an ecModel](first-ecmodel.md) | +| `pseudoRxns.tsv` | Optional. Reaction identifiers in the first column and reaction names, for reference, in the second. These reactions lose their gene association in `makeEcModel` (`make_ec_model`) and receive no standard kcat in `getStandardKcat` (`assign_standard_kcat`). Written by the user | `makeEcModel`, `getStandardKcat` | [Choose the ecModel structure](ecmodel-structure.md), [Choose the kcat sources](kcat-sources.md) | +| `ComplexPortal.json` | Enzyme complex information from the Complex Portal | `getComplexData` (Python: `get_complex_data`) writes; `applyComplexData` (`apply_complex_data`) reads | [Choose the ecModel structure](ecmodel-structure.md) | +| `smilesDB.tsv` | SMILES for the metabolites of the ecModel, a cache of the PubChem queries | `findMetSmiles` (Python: `find_met_smiles`) | [Choose the kcat sources](kcat-sources.md) | ## Kcat source files and BRENDA data | Path | Content | Functions | Page | |---|---|---|---| -| `kcats/.tsv` | One kcat source file per source: `custom`, `brenda`, `ec_tree`, `homologs`, `dlkcat` and `okp_`. The format is described in [Kcat source files](../gecko4-project-format.md#kcat-source-files) | `kcatsFromBrenda`, `kcatsFromECTree`, `kcatsFromHomologs`, `kcatsFromDLKcat` and `kcatsFromOKP` write; `loadKcatSource` and `collectKcats` read. `custom.tsv` is written by hand or by `convertCustomKcats` (`convert_custom_kcats`) | [Gathering kcats](gathering-kcats.md), [Applying kcats](applying-kcats.md) | -| `brenda/` | Optional. The BRENDA files of the project: `kcat.tsv`, `sa.tsv`, `mw.tsv`, `measurements.tsv` and the other files of the BRENDA release. Without the folder, the files of the toolbox-assets release are downloaded once | `ModelAdapter.getBrendaDBFolder` (Python: `get_brenda_db_folder`); `geckopy brenda-refresh` writes `kcat.tsv`, `sa.tsv` and `mw.tsv` to the folder, by default `data/brenda` of the current directory; it does not produce `measurements.tsv`, `sequences.fasta` or `substrates.tsv`, and `kcats_from_ec_tree` then falls back to the asset | [Gathering kcats](gathering-kcats.md) | -| `customKcats.tsv` | Curated kcats in the format of GECKO 3, with the columns `proteins`, `genes`, `gene_name`, `kcat`, `rxns`, `notes` and `stoicho`. `collectKcats` applies the file when a stage lists `custom` and `kcats/custom.tsv` does not exist | `applyCustomKcats` (Python: `apply_custom_kcats`) reads; `convertCustomKcats` (`convert_custom_kcats`) converts | [Applying kcats](applying-kcats.md) | -| `PhylDist.mat` | KEGG phylogenetic distances, which geckopy uses to match kcats of other organisms. Downloaded when it is missing. GECKO reads the corresponding file of RAVEN | `kcats_from_brenda` | [Gathering kcats](gathering-kcats.md) | +| `kcats/.tsv` | One kcat source file per source: `custom`, `brenda`, `ec_tree`, `homologs`, `dlkcat` and `okp_`. The format is described in [Kcat source files](../gecko4-project-format.md#kcat-source-files) | `kcatsFromBrenda`, `kcatsFromECTree`, `kcatsFromHomologs`, `kcatsFromDLKcat` and `kcatsFromOKP` write; `loadKcatSource` and `collectKcats` read. `custom.tsv` is written by hand or by `convertCustomKcats` (`convert_custom_kcats`) | [Choose the kcat sources](kcat-sources.md) | +| `brenda/` | Optional. The BRENDA files of the project: `kcat.tsv`, `sa.tsv`, `mw.tsv`, `measurements.tsv` and the other files of the BRENDA release. Without the folder, the files of the toolbox-assets release are downloaded once | `ModelAdapter.getBrendaDBFolder` (Python: `get_brenda_db_folder`); `geckopy brenda-refresh` writes `kcat.tsv`, `sa.tsv` and `mw.tsv` to the folder, by default `data/brenda` of the current directory; it does not produce `measurements.tsv`, `sequences.fasta` or `substrates.tsv`, and `kcats_from_ec_tree` then falls back to the asset | [Choose the kcat sources](kcat-sources.md) | +| `customKcats.tsv` | Curated kcats of a GECKO 3 project (columns `proteins`, `genes`, `gene_name`, `kcat`, `rxns`, `notes`, `stoicho`). Without `kcats/custom.tsv`, `collectKcats` raises `collectKcats:legacyCustomKcats` | `convertCustomKcats` (Python: `convert_custom_kcats`) converts it into `kcats/custom.tsv` | [Set custom kcats](custom-kcats.md) | +| `PhylDist.mat` | KEGG phylogenetic distances, which geckopy uses to match kcats of other organisms. Downloaded when it is missing. GECKO reads the corresponding file of RAVEN | `kcats_from_brenda` | [Choose the kcat sources](kcat-sources.md) | ## Predictor files | File | Content | Functions | Page | |---|---|---|---| -| `DLKcat.tsv` | Input of DLKcat and, after a run, its output | `writeDLKcatInput` (Python: `write_dlkcat_input`) and `runDLKcat` write; `kcatsFromDLKcat` and `readDLKcatOutput` read | [Gathering kcats](gathering-kcats.md) | -| `DLKcat_run.txt` | The number of reactions that the last DLKcat run covered, when it covered a subset of the reactions | `kcatsFromDLKcat` (Python: `kcats_from_dlkcat`) | [Gathering kcats](gathering-kcats.md) | -| `DLKcatIgnoreMets.tsv`, `DLKcatCurrencyMets.tsv` | Optional project copies of the lists of metabolites that are left out of the DLKcat and OKP input: names with SMILES, and pairs of currency metabolites. Without them, the files of the toolbox are used | `writeDLKcatInput`, `writeOpenKineticsPredictorInput` | [Gathering kcats](gathering-kcats.md) | -| `OKP.csv` | Input of an OpenKineticsPredictor job | `writeOpenKineticsPredictorInput`, `submitOpenKineticsPredictor` | [Gathering kcats](gathering-kcats.md) | -| `OKP_job_.txt`, `OKP_output_.csv` | The job and the result of one OKP method | `kcatsFromOKP` (Python: `kcats_from_okp`) | [Gathering kcats](gathering-kcats.md) | -| `OKP_job.txt`, `OKP_output.csv` | The job and the result of a job, written by `submitOpenKineticsPredictor` and `fetchOpenKineticsPredictor`. `kcatsFromOKP` adopts them for its method when no job of that method exists | `submitOpenKineticsPredictor`, `fetchOpenKineticsPredictor` | [Gathering kcats](gathering-kcats.md) | -| `okpApiKey.txt` | The API key of OpenKineticsPredictor, an alternative to the argument `apiKey` and to the environment variable `OKP_API_KEY` | `submitOpenKineticsPredictor`, `fetchOpenKineticsPredictor` | [Gathering kcats](gathering-kcats.md) | +| `DLKcat.tsv` | Input of a local DLKcat run in Docker and, after the run, its output | `kcatsFromDLKcat` (Python: `kcats_from_dlkcat`) with `via` `docker` writes; `kcatsFromDLKcat` with `via` `file` reads | [Choose the kcat sources](kcat-sources.md) | +| `DLKcat_run.txt` | The number of reactions that the last DLKcat run covered, when it covered a subset of the reactions | `kcatsFromDLKcat` (Python: `kcats_from_dlkcat`) | [Choose the kcat sources](kcat-sources.md) | +| `DLKcatIgnoreMets.tsv`, `DLKcatCurrencyMets.tsv` | Optional project copies of the metabolites left out of the DLKcat and OKP input and the homolog search (names with SMILES; pairs of currency metabolites). Default: the toolbox files | `kcatsFromOKP`, `kcatsFromDLKcat`, `kcatsFromHomologs` | [Choose the kcat sources](kcat-sources.md) | +| `OKP.csv` | Input of the last OpenKineticsPredictor job | `kcatsFromOKP` (Python: `kcats_from_okp`) | [Choose the kcat sources](kcat-sources.md) | +| `OKP_job_.txt`, `OKP_output_.csv` | The job and the result of one OKP method | `kcatsFromOKP` (Python: `kcats_from_okp`) | [Choose the kcat sources](kcat-sources.md) | +| `OKP_job.txt`, `OKP_output.csv` | The job and the result of a job, written by earlier versions of GECKO. `kcatsFromOKP` adopts them for its method when no job of that method exists | `kcatsFromOKP` (Python: `kcats_from_okp`) | [Choose the kcat sources](kcat-sources.md) | +| `okpApiKey.txt` | The API key of OpenKineticsPredictor, read when the environment variable `OKP_API_KEY` is not set | `kcatsFromOKP`, `kcatsFromDLKcat` (Python: `kcats_from_okp`, `kcats_from_dlkcat`) | [Choose the kcat sources](kcat-sources.md) | ## Proteomics, flux data and tuning data | File | Content | Functions | Page | |---|---|---|---| -| `paxDB.tsv` | Protein abundance data from [PaxDB](https://pax-db.org) for the f factor (the mass fraction of proteins accounted for in the ecModel). Header lines start with `#`, followed by three tab-separated columns: an internal identifier, the gene and the abundance. Without the file, `calculateFfactor` returns 0.5 | `calculateFfactor` (Python: `load_pax_db`, `calculate_f_factor`) | [Applying kcats](applying-kcats.md) | -| `proteomics.tsv` | Measured protein levels (mg protein/gDCW) from one or more experiments: UniProt identifiers in the first column, protein levels from individual replicates in each subsequent column | `loadProtData` (Python: `load_prot_data`) | [Proteomics integration](proteomics-integration.md) | -| `fluxData.tsv` | Experimental growth rate, protein content and carbon-source uptake, that constrain the ecModel during proteomics integration and in flux-variability comparisons. Exchange fluxes in mmol/gDCWh; uptake negative, excretion positive. Not required for the initial reconstruction | `loadFluxData` (Python: `load_flux_data`), `constrainFluxData` | [Proteomics integration](proteomics-integration.md), [Simulation and analysis](simulation-and-analysis.md) | -| `evotuneFluxData.tsv`, `evotuneMaxGrowth.tsv`, `evotuneZeroExch.tsv` | Optional data of the kcat tuning with CMA-ES: flux data, maximum growth rates and exchange reactions with zero flux | `loadEvotuneData` | [Tuning against experimental data](tuning-against-experimental-data.md) | +| `paxDB.tsv` | Protein abundance data from [PaxDB](https://pax-db.org) for the f factor (the mass fraction of proteins accounted for in the ecModel). Header lines start with `#`, followed by three tab-separated columns: an internal identifier, the gene and the abundance. Without the file, `calculateFfactor` returns 0.5 | `calculateFfactor` (Python: `load_pax_db`, `calculate_f_factor`) | [Set the protein pool](protein-pool.md) | +| `proteomics.tsv` | Measured protein levels (mg protein/gDCW) from one or more experiments: UniProt identifiers in the first column, protein levels from individual replicates in each subsequent column | `loadProtData` (Python: `load_prot_data`) | [Constrain enzymes with proteomics](proteomics.md) | +| `fluxData.tsv` | Experimental growth rate, protein content and carbon-source uptake, that constrain the ecModel during proteomics integration and in flux-variability comparisons. Exchange fluxes in mmol/gDCWh; uptake negative, excretion positive. Not required for the initial reconstruction | `loadFluxData` (Python: `load_flux_data`), `constrainFluxData` | [Constrain enzymes with proteomics](proteomics.md), [Analyse an ecModel](analysis.md) | +| `evotuneFluxData.tsv`, `evotuneMaxGrowth.tsv`, `evotuneZeroExch.tsv` | Optional data of the kcat tuning with CMA-ES: flux data, maximum growth rates and exchange reactions with zero flux | `loadEvotuneData` | [Tune kcats against experimental data](advanced-tuning.md) | ## Spontaneous reactions @@ -51,6 +51,6 @@ The file that `[spontaneous] file` in `model_adapter.toml` names is a TSV with a ## See also -- [Getting started](getting-started.md), where a project's `data` folder is created. +- [Build an ecModel](first-ecmodel.md), where a project's `data` folder is created. - [Model adapter parameters](model-adapter-parameters.md), the parameters of `model_adapter.toml`. - [Project format 4](../gecko4-project-format.md), the folder layout of a GECKO 4 project. diff --git a/docs/guide/ecmodel-structure.md b/docs/guide/ecmodel-structure.md new file mode 100644 index 0000000..60905e2 --- /dev/null +++ b/docs/guide/ecmodel-structure.md @@ -0,0 +1,313 @@ +# Choose the ecModel structure + +`makeEcModel` turns a conventional genome-scale metabolic model (GEM) into an enzyme-constrained model (ecModel) without kcats: reversible reactions are split by direction, the enzymes of each reaction are recorded in the `ec` structure, and the protein pool is added. The result is a full or a light ecModel without kcats, a list of the genes that have no protein data, and, for an organism in the Complex Portal, the subunit stoichiometry of the enzyme complexes. + + +:::::{dropdown} Place in the workflow + +The diagram orders the functions of this page within the [workflow](workflow.md). Boxes with a dashed outline are optional steps, a dotted arrow is an optional path, and a diamond is a choice. Rounded boxes with a dotted outline are the context, other pages or the model to start from, and a click on one opens its page. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Previous: Build an ecModel, step 1"]):::ctx + B1["ModelAdapterManager.setDefault
loadConventionalGEM"] + B3{"Full or light?"} + B4["makeEcModel
calls loadDatabases"] + B5["makeEcModel, light
calls loadDatabases"] + B6["getComplexData
applyComplexData"]:::opt + PV --> B1 --> B3 + B3 -->|full| B4 --> B6 + B3 -->|light| B5 --> B6 + NX(["Next: Choose the kcat sources"]):::ctx + B4 --> NX + B5 --> NX + B6 --> NX + click PV href "first-ecmodel.html" + click NX href "kcat-sources.html" +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Previous: Build an ecModel, step 1"]):::ctx + B1["ModelAdapter.from_folder
load_conventional_gem"] + B3{"Full or light?"} + B4["make_ec_model
downloads the UniProt and KEGG data;
load_uniprot_tsv, load_kegg_tsv read them"] + B5["make_ec_model, light
downloads the UniProt and KEGG data;
load_uniprot_tsv, load_kegg_tsv read them"] + B6["get_complex_data
apply_complex_data"]:::opt + PV --> B1 --> B3 + B3 -->|full| B4 --> B6 + B3 -->|light| B5 --> B6 + NX(["Next: Choose the kcat sources"]):::ctx + B4 --> NX + B5 --> NX + B6 --> NX + click PV href "first-ecmodel.html" + click NX href "kcat-sources.html" +``` +::: +:::: +::::: + +## Functions + +| MATLAB | Python | Note | +|---|---|---| +| `ModelAdapterManager.setDefault` | `ModelAdapter.from_folder` | read `model_adapter.toml` of the project | +| `loadDatabases` | `load_uniprot_tsv`, `load_kegg_tsv` | read `data/uniprot.tsv` and `data/kegg.tsv`, and download a missing file | +| `loadConventionalGEM` | `load_conventional_gem` | load the GEM named by `conv_gem` in the model adapter | +| `makeEcModel` | `make_ec_model` | build a full or a light ecModel | +| `getComplexData` | `get_complex_data` | download complex stoichiometries from the Complex Portal | +| `applyComplexData` | `apply_complex_data` | write the subunit stoichiometry of matching complexes to `ec.rxnEnzMat` | + +## Setup + +The project folder has a `model_adapter.toml` (see [Build an ecModel](first-ecmodel.md)). The build reads these adapter keys, described in [Model adapter parameters](model-adapter-parameters.md): + +- `conv_gem`: path of the conventional GEM. +- `enzyme_comp`: name of the compartment (an entry of `model.compNames`) that receives the protein pseudometabolites. +- `uniprot.id`, `uniprot.type`, `uniprot.gene_id_field`, `uniprot.reviewed`: the UniProt query that gives `data/uniprot.tsv`. +- `kegg.id`, `kegg.gene_id`: the KEGG organism and field that give `data/kegg.tsv`. +- `complex.taxonomic_id`: the taxonomy identifier for the Complex Portal. + +The data files are described in [Files in the data folder](data-folder-files.md). + +## Full or light ecModel + +Both structures add the enzyme cost of a reaction as protein mass drawn from the protein pool. They differ in how isozymes and single enzymes appear in the model: + +| Property | Full ecModel | Light ecModel | +|---|---|---| +| Isozymes | each isozyme gets its own copy of the reaction (`_EXP_1`, `_EXP_2`, ...) | the reaction is not copied; `ec.rxns` has one row per isozyme, with the prefix `001_`, `002_`, ... | +| Enzymes in the model | one pseudometabolite `prot_` and one reaction `usage_prot_` per enzyme | none | +| Enzyme cost of a reaction | the reaction consumes its enzymes; the enzymes draw on `prot_pool` | the reaction consumes `prot_pool` directly, with the cost of the isozyme with the smallest ratio of molecular weight (MW) to kcat | +| Size of the model | larger than the conventional GEM by the isozyme copies and one reaction per enzyme | the irreversible conventional GEM plus `prot_pool_exchange` | +| Needed for | proteomics constraints, enzyme usage and bottleneck analysis | | + +A light ecModel cannot be turned into a full one, or the reverse; the choice is made when `makeEcModel` runs, and it is stored in the ecModel (`ec.geckoLight` in MATLAB, `ec.gecko_light` in Python, `gecko_light` in the YAML file). + +Build a full ecModel unless the size of the full model makes the simulations too slow and no task in the next table is needed. These tasks need the per-enzyme pseudometabolites and usage reactions of a full ecModel: + +| Task | MATLAB | Python | +|---|---|---| +| Constrain enzymes with proteomics | `constrainEnzConcs` raises an error, so the functions that relax these constraints have nothing to act on; `relaxProteomicsGreedy` and `getIsozymeCoverage` raise an error themselves | `constrain_enz_concs` raises a `ValueError`; the flux, bound, shadow price and concentration setter of `Enzyme` raise a `NotImplementedError` | +| Enzyme usage and bottlenecks | `getEnzymeUsage`, `getEnzymeBottlenecks` and `getPfbaEnzymes` raise an error | `enzyme_usage` and `pfba_enzymes` raise a `NotImplementedError`; `get_enzyme_bottlenecks` raises a `KeyError`, because the `prot_` constraints it reads do not exist | +| Fill kcats from other isozymes | `getKcatAcrossIsozymes` raises an error | `fill_kcats_from_isozymes` raises a `NotImplementedError` | +| Add reactions to an ecModel | `addNewRxnsToEC` raises an error | `add_new_rxns_to_ec` raises a `NotImplementedError` | +| Convert `customKcats.tsv` to `custom.tsv` | `convertCustomKcats` raises an error | `convert_custom_kcats` raises a `ValueError` | + +On a light ecModel, `assignKcats` and `collectKcats` skip the step that fills kcats from other isozymes. A `custom.tsv` written from a full ecModel applies to a light ecModel too. `applyKcatConstraints`, `getStandardKcat`, `applyComplexData`, `sensitivityTuning` and `getSubsetEcModel` work on both structures. + +The tutorial script `tutorials/full_ecModel/code/plotlightVSfull.m` of GECKO builds a light and a full ecModel of yeast-GEM with the same kcats, maximises growth in both, and plots the two flux distributions mapped to the reactions of the GEM. + +## Build the ecModel + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +ModelAdapterManager.setDefault('path/to/project'); +model = loadConventionalGEM(); + +% Full ecModel +[ecModel, noUniprot] = makeEcModel(model); + +% Light ecModel +[ecModel, noUniprot] = makeEcModel(model, 'geckoLight', true); +``` + +`makeEcModel` needs a model in RAVEN format. It raises an error for a model with the field `rules` or `modelID`, which a model loaded with the COBRA Toolbox has; convert such a model with `ravenCobraWrapper`. +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import ModelAdapter, load_conventional_gem, make_ec_model + +adapter = ModelAdapter.from_folder("path/to/project") +model = load_conventional_gem(adapter) + +# Full ecModel +ec_model = make_ec_model(model, adapter) + +# Light ecModel +ec_model = make_ec_model(model, adapter, gecko_light=True) +``` + +`make_ec_model` returns a new `EcModel` and leaves `model` unchanged. It raises a `ValueError` when it is called on an ecModel whose `ec` is already filled. +::: +:::: + +The ecModel has no kcats yet: `ec.kcat` is 0 for every row, and no metabolic reaction consumes an enzyme (full ecModel) or `prot_pool` (light ecModel). [Choose the kcat sources](kcat-sources.md) fills the kcats and applies them. + +## Steps of makeEcModel + +`makeEcModel` (Python: `make_ec_model`) runs these steps in order: + +| Step | Full | Light | +|---|---|---| +| Remove the gene associations of pseudo-reactions: reactions whose name contains `pseudoreaction`, and the reactions listed in `data/pseudoRxns.tsv` | yes | yes | +| Reverse the reactions with a lower bound below 0 and an upper bound of 0 | yes | yes | +| Split each reversible reaction, except exchange reactions, into a forward reaction and a reverse reaction with the suffix `_REV` | yes | yes | +| Split each reaction with isozymes (`or` in the gene rule) into one reaction per isozyme, with the suffix `_EXP_` | yes | no | +| Create `ec` and fill the genes, UniProt identifiers, molecular weights and sequences from `data/uniprot.tsv`, and from `data/kegg.tsv` for genes that UniProt does not match | yes | yes | +| Record which enzymes each reaction needs in `ec.rxnEnzMat` (Python: `ec.rxn_enz_mat`), with 1 for every subunit | yes | yes | +| Add a pseudometabolite `prot_` for every enzyme | yes | no | +| Add the pseudometabolite `prot_pool` | yes | yes | +| Add a reaction `usage_prot_` for every enzyme, from `prot_pool` to `prot_`, with bounds 0 and 1000 | yes | no | +| Add `prot_pool_exchange`, which produces `prot_pool`, with bounds 0 and 1000 | yes | yes | + +The upper bound of `prot_pool_exchange` stays 1000 until [Set the protein pool](protein-pool.md) sets it. + +The identifiers that these steps add must be new, so `makeEcModel` raises an error when a metabolite identifier of the GEM starts with `prot_`, or when a reaction identifier starts with `usage_prot_` or `prot_pool`, or ends with `_REV` or `_EXP_`. + +## Match genes to UniProt and KEGG + +`makeEcModel` matches each gene of the model to a row of `data/uniprot.tsv`, on the field set by `uniprot.gene_id_field`. When `data/uniprotConversion.tsv` exists, the genes are first converted to UniProt identifiers with that table, and the match is on the UniProt identifier. A gene that UniProt does not match is looked up in `data/kegg.tsv`, on the field set by `kegg.gene_id`. For a gene found in KEGG, `ec.enzymes` gets the UniProt accession of the KEGG row, or the KEGG gene identifier when the row has no accession; a warning lists these genes. + +A gene that matches in neither file has no enzyme in the ecModel. A subunit of a complex without a match is left out of the complex, and the other subunits still give the reaction its enzyme cost. A reaction, or in a full ecModel an isozyme copy of a reaction, whose genes all have no match has an empty row in `ec.rxnEnzMat` and no enzyme cost. In a light ecModel, such an isozyme is left out when the isozyme with the smallest cost is selected. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +`noUniprot`, the second output of `makeEcModel`, lists the genes that match neither file, and a warning gives their number. + +`makeEcModel` downloads a missing `data/uniprot.tsv` or `data/kegg.tsv` through `loadDatabases`, with the `[uniprot]` and `[kegg]` values of the adapter. +::: +:::{tab-item} 🐍 Python +:sync: python + +`make_ec_model` returns no list. It logs a warning with the number of genes that match neither file and the first five, and adds a note under `rxn.notes["geckopy_warning"]` to every reaction that has such a gene. + +`make_ec_model` downloads a missing `data/uniprot.tsv` or `data/kegg.tsv` with the `[uniprot]` and `[kegg]` values of the adapter, through `load_uniprot_tsv` and `load_kegg_tsv`. It raises a `FileNotFoundError` when `data/uniprot.tsv` is missing and `uniprot.id` is empty. The arguments `uniprot_db` and `kegg_db` take databases loaded beforehand. + +The commands `geckopy uniprot-download` and `geckopy kegg-download` download the same files, with identifiers and options given on the command line; they do not read the adapter. +::: +:::: + +In both languages, a file that exists is not downloaded again: after a change of `uniprot.id`, delete `data/uniprot.tsv` before the build. + +Many genes without a match mean that the UniProt query or the gene identifier field does not fit the gene identifiers of the GEM. Compare a few gene identifiers of the GEM with the gene column of `data/uniprot.tsv`, then change `uniprot.id`, `uniprot.type` or `uniprot.gene_id_field`, or add `data/uniprotConversion.tsv` (see [Find the UniProt and KEGG parameters](#find-the-uniprot-and-kegg-parameters)). + +## Find the UniProt and KEGG parameters + +`data/uniprot.tsv` gives the molecular weight and the sequence of each enzyme, and a GEM without it has no enzyme data. `data/kegg.tsv` is optional: it fills genes that UniProt does not match, and EC numbers. The query must return gene identifiers in the same style as the genes of the GEM. + +- **UniProt proteome or taxonomy.** `uniprot.id` has no default: without it and without `data/uniprot.tsv`, the build raises an error that names the key (MATLAB: `loadDatabases:noUniprotID`). With `uniprot.type = "proteome"`, `uniprot.id` is a proteome identifier from [UniProt proteomes](https://www.uniprot.org/proteomes); choose the reference proteome when the organism has several proteomes. With `uniprot.type = "taxonomy"`, `uniprot.id` is a taxonomy identifier; choose the most specific one, for example the strain (559292 for *S. cerevisiae* S288C) in place of the species (4932). A proteome gives fewer duplicate matches between genes and proteins. `uniprot.reviewed = true` restricts the query to Swiss-Prot entries; the default `false` also includes unreviewed entries (`geckopy uniprot-download`: the option `--reviewed-only` restricts it). +- **UniProt gene identifier field.** `uniprot.gene_id_field` is the UniProt field whose values match the genes of the GEM. The page of one protein on uniprot.org shows its gene names under "Names & Taxonomy"; for yeast-GEM, which uses identifiers such as `YFL026W` ("Ordered locus names"), the field is `gene_oln`. +- **KEGG organism code.** `kegg.id` is the code from the [KEGG organism list](https://www.genome.jp/kegg/catalog/org_list.html), for example `sce` or `hsa`. The default is empty: the build then downloads no KEGG file, prints a warning and uses no KEGG data, which is the setting for an organism that is not in KEGG. +- **KEGG gene identifier.** With `kegg.gene_id = "kegg"`, the genes are matched on the KEGG gene identifier. Another value is read from the "Other DBs" links of each KEGG gene entry, for example `Ensembl` for the gene identifiers of Human-GEM. A KEGG entry without a UniProt link is not written to `data/kegg.tsv`. + +When no UniProt field matches the gene identifiers, `data/uniprotConversion.tsv` maps them: a header row, then one row per gene with the gene identifier and the UniProt identifier, tab-separated. `data/kegg.tsv` has both identifiers in its first and third columns (UniProt, gene, KEGG gene, comma-separated, no header), which gives the table for an organism whose GEM uses KEGG gene identifiers, such as *E. coli* iML1515 with `b0008`: + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +params = ModelAdapterManager.getDefault().getParameters(); +DB = loadDatabases('kegg'); +fID = fopen(fullfile(params.path, 'data', 'uniprotConversion.tsv'), 'w'); +fprintf(fID, '%s\t%s\n', 'genes', 'uniprot'); +output = transpose([DB.kegg.keggGene, DB.kegg.uniprot]); +fprintf(fID, '%s\t%s\n', output{:}); +fclose(fID); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +import pandas as pd + +data = adapter.params.path / "data" +columns = ["uniprot", "gene", "kegg_gene", "ec", "mw", "pathway", "sequence"] +kegg = pd.read_csv(data / "kegg.tsv", header=None, names=columns) +kegg[["kegg_gene", "uniprot"]].rename(columns={"kegg_gene": "genes"}).to_csv( + data / "uniprotConversion.tsv", sep="\t", index=False, +) +``` +::: +:::: + +For an organism in neither UniProt nor KEGG, write `data/uniprot.tsv` (and, if needed, `data/kegg.tsv`) by hand, with the columns of the files of the `full_ecModel` tutorial. The identifiers can be invented, but the molecular weights and sequences must be correct, because the enzyme costs and the kcat predictions use them. + +## Add the stoichiometry of enzyme complexes + +`makeEcModel` gives every subunit of an enzyme complex the stoichiometry 1. The [Complex Portal](https://www.ebi.ac.uk/complexportal/complex/organisms) has curated subunit stoichiometries for a limited set of organisms; the link lists them. For an organism in the list, set `complex.taxonomic_id` to its taxonomy identifier and add the data. For another organism, keep the stoichiometry of 1, or write `data/ComplexPortal.json` from other databases or the literature. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +complexInfo = getComplexData(); +[ecModel, foundComplex, proposedComplex] = applyComplexData(ecModel, complexInfo); +``` + +`getComplexData` reads the taxonomy identifier from the adapter, or takes it as the argument `taxonomicID`; 0 downloads the complexes of all organisms. Without a taxonomy identifier, it warns and returns an empty structure. It writes `data/ComplexPortal.json`. `applyComplexData` without `complexInfo` reads that file, and runs `getComplexData` when the file does not exist. `foundComplex` and `proposedComplex` are tables, and the number of each is printed. +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import apply_complex_data, get_complex_data + +json_path = adapter.params.path / "data" / "ComplexPortal.json" +get_complex_data(adapter.params.complex.taxonomic_id, write_to=json_path) +apply_complex_data(ec_model, path=json_path) +``` + +`get_complex_data` takes the taxonomy identifier (0 for all organisms) and returns the complexes; `write_to` saves them. Without a taxonomy identifier (`None`), it logs a warning and returns an empty list. `apply_complex_data` changes `ec_model` in place, logs the number of full and proposed matches, and logs each proposed complex as a warning. When at least one complex matches and `apply=True` (the default), it runs `apply_kcat_constraints` afterwards. +::: +:::: + +A complex is applied to a reaction when its proteins and the enzymes of the reaction are the same set. For a reaction with more than one enzyme and no exact match, a complex is proposed, not applied, in two cases: the complex contains all enzymes of the reaction and more subunits; or the complex contains at least 75 % (and less than 100 %) of the enzymes of the reaction and has no more subunits than the reaction has enzymes, so it can contain subunits that the reaction lacks (Python: the keyword `min_match_to_propose`, default 0.75). A reaction can have a proposal for each case, so two rows in `proposedComplex` (Python: two warnings). Read the proposed complexes: a missing subunit in the gene rule of the GEM can be the reason, and a curated gene rule then gives an exact match in the next build. + +The stoichiometry changes `ec.rxnEnzMat` (Python: `ec.rxn_enz_mat`). The enzyme cost in the reactions follows from it when the kcats are applied, so add the complex data before the kcats are assigned. + +## Troubleshooting + +| Problem | Possible reason | Solution | +|---|---|---| +| Error that identifiers in `model.mets` or `model.rxns` are not allowed | the GEM has identifiers that start with `prot_`, `usage_prot_` or `prot_pool`, or end with `_REV` or `_EXP_` | rename those identifiers in the GEM | +| Error that the compartment cannot be found in `model.compNames` | `enzyme_comp` differs from every compartment name of the GEM | set `enzyme_comp` to a name from `model.compNames`; a compartment identifier gives the same error | +| Error that none of the proteins in `uniprot.tsv` or `kegg.tsv` match the genes (Python: `ValueError`) | the UniProt and KEGG parameters do not fit the gene identifiers | see [Match genes to UniProt and KEGG](#match-genes-to-uniprot-and-kegg) | +| MATLAB error that the model was likely loaded with the COBRA Toolbox | the model has the field `rules` or `modelID` | load it with `importModel`, or convert it with `ravenCobraWrapper` | +| An error that names a light ecModel (Python: `NotImplementedError` or `ValueError`) | the function needs a full ecModel | build a full ecModel | +| Warning that no taxonomic identifier is specified, and no complex data | `complex.taxonomic_id` is not set; `getComplexData` returns an empty structure (Python: `get_complex_data` returns an empty list) | set it in `model_adapter.toml`, or give it as an argument | +| Error that no data could be gathered from the Complex Portal | the Complex Portal has no complexes for this taxonomy identifier | skip the complex data | +| The rows of `ec.rxns` do not match `model.rxns` in a light ecModel | the rows have the prefix `001_`, `002_`, ... | remove the first four characters | + +## See also + +- [Introduction to ecModels](introduction.md), for the enzyme constraint that both structures encode. +- [Choose the kcat sources](kcat-sources.md), the next step. +- [Constrain enzymes with proteomics](proteomics.md) and [Enzyme usage and bottlenecks](enzyme-usage.md), which need a full ecModel. +- [ecModel YAML format](yaml-format.md), for `gecko_light` and the `ec` fields in the saved file. diff --git a/docs/guide/engineering-targets.md b/docs/guide/engineering-targets.md new file mode 100644 index 0000000..311c986 --- /dev/null +++ b/docs/guide/engineering-targets.md @@ -0,0 +1,129 @@ +# Find engineering targets + +Flux scanning with enforced objective function (FSEOF) finds the reactions whose flux changes with the production of a compound. `ecFSEOF` returns reaction and gene targets for the production of a compound by an ecModel, each with a suggested action: over-express, knock down or knock out. + + +:::::{dropdown} Place in the workflow + +The diagram orders the functions of this page within the [workflow](workflow.md). Boxes with a dashed outline are optional steps, a dotted arrow is an optional path, and a diamond is a choice. Rounded boxes with a dotted outline are the context, other pages or the model to start from, and a click on one opens its page. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Start: a functional or relaxed ecModel"]):::ctx + TS["production reaction
and carbon source uptake"] + FS["ecFSEOF"] + PV --> TS --> FS + click PV href "first-ecmodel.html" +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Start: a functional or relaxed ecModel"]):::ctx + TS["production reaction
and carbon source uptake"] + FS["ec_fseof"] + PV --> TS --> FS + click PV href "first-ecmodel.html" +``` +::: +:::: +::::: + +## Functions + +| MATLAB | Python | Note | +| --- | --- | --- | +| `ecFSEOF` | `ec_fseof` | flux scanning with enforced objective function on an ecModel | + +## Setup + +The examples use the ecModel and the model adapter parameters `params`, loaded as in the setup of [Analyse an ecModel](analysis.md#setup). The biomass reaction is read from the model adapter. Set the bounds of the condition, such as the medium and the uptake rate of the carbon source, before the scan ([Basic model operations](basic-model-operations.md)). + +The other names in the examples are placeholders for values of the project: + +| MATLAB | Python | Value | +| --- | --- | --- | +| `prodTargetRxn` | `prod_target_rxn` | identifier of the production reaction, for example the exchange reaction of the product | +| `csRxn` | `cs_rxn` | identifier of the uptake reaction of the carbon source | + +## Target selection + +`ecFSEOF` first finds the production at maximum growth and the maximum production. It then fixes the production reaction at `nSteps` (Python: `n_steps`, default 16) evenly spaced values from the first to 90 % of the second. At each value it maximises growth and then minimises the total flux. A reaction is a target when the absolute Pearson correlation between its absolute flux and the enforced production is at least `corrThreshold` (Python: `correlation_threshold`, default 0.9). The scan covers the reactions with a gene rule, except those that use the standard enzyme ([Choose the kcat sources](kcat-sources.md)). The `usage_prot_*` reactions take part in the scan and are left out of the result. + +Each target reaction gets an action from the slope of its absolute flux: + +| Slope | Absolute flux at the last step | Action | +| --- | --- | --- | +| positive | any | `OE` (over-express) | +| negative | at most 1e-8 | `KO` (knock out) | +| negative | above 1e-8 | `KD` (knock down) | + +A gene gets one action from its target reactions. When these reactions have one action, the gene keeps it, also when the gene is essential. When they have different actions, the gene gets `OE` if one of them is `OE`, otherwise `KO` if the gene is not essential, otherwise `KD`. A gene is essential when growth is infeasible or below 1e-8 with the usage reactions of its enzymes fixed at 0 and the production fixed at the last value of the scan. + +When the production at maximum growth is equal to 90 % of the maximum production, there is no range to scan, and the function raises an error. + +## Run the scan + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +fseof = ecFSEOF(ecModel, prodTargetRxn, csRxn, 'nSteps', 16, 'outputFile', true); +fseof.geneTargets +``` + +`rxnTargets` and `transportTargets` list the target reactions with identifier, name, slope, gene rule and equation, ordered by the product of slope and correlation; they have no action column. `geneTargets` has one row per gene of the target reactions: gene, short name, mean slope, action and essentiality. With `outputFile` set to `true`, the function writes `ecFSEOF_genes.tsv`, `ecFSEOF_rxns.tsv` and `ecFSEOF_transporter.tsv` to the `output` folder of the project. The function warns when the lower bound of `csRxn` is below the uptake at maximum growth, that is, when the uptake does not reach its bound. +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import ec_fseof + +result = ec_fseof(ec_model, prod_target_rxn, cs_rxn, n_steps=16) +result.targets +result.gene_targets +``` + +`result.targets` is one DataFrame with the reaction targets first and the transport targets after them (column `transport`), each group ordered by `score`, the product of the absolute slope and the absolute correlation. The column `target_type` holds the action of each reaction. `result.gene_targets` has one row per gene of the target reactions, with the columns `gene`, `name`, `slope` (mean absolute slope), `action`, `essentiality` and `reactions`, ordered by slope. `result.scan` holds the fluxes at each step. `ec_fseof` writes no files; `result.targets.to_csv(...)` writes the table. The biomass reaction is read from `ec_model.adapter`, or from the argument `bio_rxn`. When `cs_rxn` is given, the function logs a warning when the lower bound of `cs_rxn` is below the uptake at maximum growth. +::: +:::: + +## Troubleshooting + +| Problem | Possible reason | Solution | +| --- | --- | --- | +| `ecFSEOF` or `ec_fseof` returns no targets | no reaction reaches `corrThreshold` (Python: `correlation_threshold`) | Lower the threshold, or check the bounds of the production and carbon source reactions | +| `ecFSEOF` raises `ecFSEOF:noRange`; `ec_fseof` raises a `ValueError` that says there is no range to scan | the production at maximum growth is equal to 90 % of the maximum production | Check the bounds of the production reaction and of the carbon source reaction | +| Warning about the lower bound of the carbon source | the uptake at maximum growth does not reach the lower bound of the carbon source reaction | Set the lower bound to the uptake rate of the condition | + +## See also + +- [Analyse an ecModel](analysis.md): the other analyses of an ecModel and the shared setup. +- [Enzyme usage and bottlenecks](enzyme-usage.md): the enzymes that a flux distribution uses and the enzymes that limit the objective. +- [API reference](../api/index.md) and [MATLAB vs Python](../matlab-vs-python.md): the signature of each function in the table. diff --git a/docs/guide/enzyme-usage-and-bottlenecks.md b/docs/guide/enzyme-usage-and-bottlenecks.md deleted file mode 100644 index 0980de1..0000000 --- a/docs/guide/enzyme-usage-and-bottlenecks.md +++ /dev/null @@ -1,260 +0,0 @@ -# Enzyme usage and bottlenecks - -A full ecModel simulation reports the amount of each enzyme that a flux distribution uses, and which enzyme would raise the objective most if more of it were available. These are three questions with three answers. The usage of every enzyme is ranked for a solved flux distribution, the enzymes that limit the objective are ranked independently of any particular solution, and the flux distribution with the smallest total enzyme mass is found. - -:::{warning} -`getEnzymeUsage` / `enzyme_usage`, `getPfbaEnzymes` / `pfba_enzymes` and `getEnzymeBottlenecks` require a full ecModel; see [GECKO light and full ecModels](gecko-light.md). -::: - - -:::::{dropdown} Place in the workflow - -The diagram orders the functions of this page within the [workflow](workflow.md). Dashed boxes are optional and dotted boxes are neighbouring pages. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - PV(["Previous: Simulation and analysis"]):::ctx - PV --> A["getEnzymeUsage
reportEnzymeUsage"] - PV --> B["getEnzymeBottlenecks"] - PV --> C["getPfbaEnzymes"] -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - PV(["Previous: Simulation and analysis"]):::ctx - PV --> A["enzyme_usage
report_enzyme_usage"] - PV --> B["get_enzyme_bottlenecks"] - PV --> C["pfba_enzymes"] -``` -::: -:::: -::::: - -## Functions - -| MATLAB | Python | Note | -|---|---|---| -| `getEnzymeUsage` | `enzyme_usage` | absolute and capacity usage of every enzyme, from a flux distribution | -| `reportEnzymeUsage` | `report_enzyme_usage` | rank that usage data into top-N and high-capacity tables | -| `getEnzymeBottlenecks` | `get_enzyme_bottlenecks` | rank enzymes by the gain of the objective when their usage bound is raised | -| `getPfbaEnzymes` | `pfba_enzymes` | the flux distribution that uses the least enzyme mass at a given objective | - -## Setup - -A full ecModel, built as described in [Building an empty ecModel](building-ec-model.md), with the constraints of the simulation applied (see [Simulation and analysis](simulation-and-analysis.md)). The usage reactions `usage_prot_` run forward, so the flux through them is not negative (see [GECKO 3 to GECKO 4](../gecko3-to-gecko4.md)). - -## Absolute and capacity usage - -The usage of an enzyme is read in two ways from a solved model: - -- **Absolute usage**, the flux through its `usage_prot_` reaction, in mg/gDCW. -- **Capacity usage**, absolute usage divided by the upper bound of that reaction, the fraction of the available enzyme that the solution uses. The upper bound is not always the concentration recorded in `ec.concs`; it can be a flexibilized value (see [Relaxing overconstrained proteomics](relaxing-constraints.md)). - -Enzyme usage is most informative when the total protein pool constraint is limiting, because it shows how the model allocates a scarce resource across the catalysed reactions. Usage is computed from any solved flux distribution, then ranked into the two summary tables: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -ecModel = setParam(ecModel, 'obj', 'r_1714', 1); -ecModel = setParam(ecModel, 'lb', params.bioRxn, 0.25); -sol = solveLP(ecModel, 1); -usageData = getEnzymeUsage(ecModel, sol.x); -usageReport = reportEnzymeUsage(ecModel, usageData); -usageReport.topAbsUsage -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -`r_1714` (glucose exchange) follows the cobrapy exchange-reaction convention (negative = uptake), so maximising it minimises uptake magnitude: - -```python -from cobra.flux_analysis import pfba -from geckopy import enzyme_usage, report_enzyme_usage - -ec_model.objective = "r_1714" -ec_model.reactions.get_by_id(params.bio_rxn).lower_bound = 0.25 -sol = pfba(ec_model) -usage = enzyme_usage(ec_model, sol.fluxes) -report = report_enzyme_usage(ec_model, usage) -print(report.top_abs_usage.head(10)) -``` -::: -:::: - -`reportEnzymeUsage` / `report_enzyme_usage` returns two tables and a total: - -- `topAbsUsage` / `top_abs_usage`, the top-N enzymes by absolute usage. The option `topAbsUsage` (Python: `top_abs_usage`) sets N, default 10, and 0 or `Inf` lists all enzymes. `.head(10)` in Python takes the top ten rows of the table. -- `highCapUsage` / `high_cap_usage`, every enzyme above a capacity-usage threshold, not limited to a top-N count. The option `highCapUsage` (Python: `high_cap_usage`) sets the threshold, default 0.9. -- `totalUsageFlux` / `total_usage_flux`, the current upper bound of the protein pool exchange, the denominator of the percentage column. -- `demandRatio` / `demand_ratio` and `demandRatioActive` / `demand_ratio_active`, the ratios of [Demand over the measured concentrations](#demand-over-the-measured-concentrations), copied from the usage data. They are NaN when the usage data has no `demandRatio` field. - -In `topAbsUsage` / `top_abs_usage`, enzymes whose reactions carry no flux are skipped after the top-N cut, so fewer than N enzymes can be listed. `highCapUsage` / `high_cap_usage` skips them in Python and lists them in MATLAB. An enzyme that catalyses several reactions with flux appears as a combined row marked `===`, followed by one row per reaction. - -An ecModel without individual concentrations in `ec.concs` shows an empty `highCapUsage` / `high_cap_usage` table: every enzyme draws from the shared protein pool through a usage reaction with the default upper bound of 1000 mg/gDCW, and capacity usage rarely comes close to that bound. Once individual concentrations constrain the usage reactions of some enzymes (see [Proteomics integration](proteomics-integration.md)), their upper bounds reflect the measured availability, and the capacity usage of those enzymes can approach or reach the threshold. - -### Example output - -The top ten enzyme usages at a fixed growth rate of 0.25 /h, minimising glucose uptake, in the tuned `full_ecModel` tutorial model (MATLAB and Python give the same values). An enzyme that catalyses several reactions has one summed row followed by one row per reaction in `topAbsUsage` (Python: `top_abs_usage`), which the table shows as one row: - -| Protein | Absolute usage (mg/gDCW) | % of pool | kcat (s⁻¹) | Source | Reaction | -|---|---|---|---|---|---| -| P07251 | 8.5 | 7.2 | 120 | custom | ATP synthase | -| P00830 | 7.9 | 6.7 | 120 | custom | ATP synthase | -| P12695 | 6.7 | 5.7 | 37.9 | brenda (wc=0, origin=2) | pyruvate dehydrogenase | -| P38858 | 5.9 | 5.0 | 10 | custom | 6-phosphogluconolactonase | -| P0CD90 | 4.9 | 4.2 | 12.3 | standard | water diffusion (reversible), summed over three reactions | -| P05694 | 4.0 | 3.4 | 0.33 | brenda (wc=0, origin=1) | methionine synthase | -| Q12482 | 3.8 | 3.2 | 2.04 | dlkcat | L-glutamate transport | -| P61829 | 3.7 | 3.2 | 120 | custom | ATP synthase | -| P38720 | 3.1 | 2.6 | 36.4 | custom | phosphogluconate dehydrogenase | -| P16387 | 2.4 | 2.1 | 37.9 | brenda (wc=0, origin=2) | pyruvate dehydrogenase | - -One entry (P0CD90) has the standard $k_{cat}$ value, the fallback that GECKO assigns to reactions without a gene association or without any kcat, as a median per subsystem or over all kcats (see [Applying kcat values](applying-kcats.md) and [Kcat assignment](../gecko4-kcat-assignment.md)). A standard $k_{cat}$ near the top of the usage ranking means that the model's estimate of the cost of that enzyme is a fallback, so a literature or BRENDA value for it supersedes a placeholder that has a large weight in the result. - -## Demand over the measured concentrations - -The usage data compares the solution with the proteomics that constrained the model. `demandRatio` (Python: `demand_ratio`) is the protein that the solution uses of the enzymes with a measured concentration in `ec.concs`, divided by the sum of those concentrations. A ratio above 1 means that the solution uses more of the measured enzymes than was measured, which happens when the usage bounds are above the measurements, for instance after `flexibilizeEnzConcs` (see [Relaxing overconstrained proteomics](relaxing-constraints.md)). A ratio at or below 1 does not exclude that a single enzyme exceeds its measurement, since the sum is over all measured enzymes; the capacity usage gives the ratio per enzyme. - -Enzymes without a measured concentration draw from the protein pool and are not counted. The ratio is NaN when no concentration is measured, and Inf when the measured concentrations sum to 0 while the demand is positive. An `ec.concs` that is missing or shorter than `ec.enzymes` counts as unmeasured. - -Measured enzymes that carry no flux raise the measured total and can hide a conflict among the active enzymes. `demandRatioActive` (Python: `demand_ratio_active`) restricts both sums to the enzymes with an absolute usage above 1e-7 mg/gDCW. The usage data holds the five values as `measuredProtein`, `measuredProteinActive`, `demandedProtein`, `demandRatio` and `demandRatioActive` (Python: `measured_protein`, `measured_protein_active`, `demanded_protein`, `demand_ratio` and `demand_ratio_active`), in mg/gDCW for the first three. They are computed over all enzymes that have a usage reaction, whatever the value of the option `zero` (Python: `include_zero`). - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -usageData = getEnzymeUsage(ecModel, sol.x); -usageData.demandRatio -usageData.demandRatioActive -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -usage = enzyme_usage(ec_model, sol.fluxes) -usage.demand_ratio -usage.demand_ratio_active -``` -::: -:::: - -## Ranking bottlenecks by gain of the objective - -Usage shows what the current solution uses the enzyme budget for. A heavily used enzyme can have spare capacity, and a lightly used enzyme at its bound can be the reason that the objective cannot rise. `getEnzymeBottlenecks` / `get_enzyme_bottlenecks` answers that question directly. It solves the model, and for each enzyme that runs at its usage bound (`upperBound` above 0 and `capUsage` at least 1 - 1e-6) it raises the upper bound of the `usage_prot_` reaction by `foldChange` (default 2, greater than 1; Python: `fold_change`) and solves again. The enzymes are ranked by `objectiveGain`, the rise of the objective after that solve. An enzyme that is not at its bound has `objectiveGain` 0 without a solve, because raising a bound that does not bind cannot change the optimum. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -bottlenecks = getEnzymeBottlenecks(ecModel, 'top', 10); -``` - -Returns a table with one row per enzyme, sorted by descending `objectiveGain`, with the columns `uniprot`, `gene`, `objectiveGain`, `shadowPrice`, `flux`, `capUsage`, `upperBound` and `multipleActive`. The table property `UserData.nResolved` holds the number of enzymes that were solved again, and `UserData.activeConstraints` the number of active capacity constraints. -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import get_enzyme_bottlenecks - -bottlenecks = get_enzyme_bottlenecks(ec_model, top=10) -``` - -Returns a pandas DataFrame indexed by uniprot id with the columns `gene`, `objective_gain`, `shadow_price`, `flux`, `cap_usage`, `upper_bound` and `multiple_active`. `bottlenecks.attrs["n_resolved"]` and `bottlenecks.attrs["active_constraints"]` hold the same two numbers. -::: -:::: - -The columns have these meanings: - -| Column (MATLAB / Python) | Meaning | -|---|---| -| `objectiveGain` / `objective_gain` | objective after raising the bound minus the objective before; 0 for enzymes not at their bound, gains below 1e-10 and non-optimal solves; positive for an improvement of a maximised objective (sign note below) | -| `shadowPrice` / `shadow_price` | dual of the `prot_` mass-balance constraint, reported for reference | -| `flux` | flux through the usage reaction | -| `capUsage` / `cap_usage` | absolute flux divided by the upper bound; NaN if the upper bound is 0 | -| `upperBound` / `upper_bound` | upper bound of the usage reaction | -| `multipleActive` / `multiple_active` | true when another capacity constraint is active at the same optimum: another enzyme at its bound, or `prot_pool_exchange` at its upper bound (within 1e-6 relative) | - -`getEnzymeBottlenecks` maximises the objective. `get_enzyme_bottlenecks` reports the gain in the direction of the objective: for a minimised objective the gain is the objective of the unchanged model minus the objective after raising the bound, so a positive value is an improvement in both cases. - -The shadow price does not enter the order. At a degenerate optimum, which is common in enzyme-constrained models, several constraints are active and the solver assigns the dual to one of them, so the dual depends on the basis. Where `multipleActive` is true, the dual is not informative about a single enzyme. Enzymes with equal `objectiveGain` (rounded to 9 decimals) keep their order in `ec.enzymes`. - -When no enzyme is at its usage bound, a warning (MATLAB identifier `getEnzymeBottlenecks:noEnzymeAtBound`; Python: logged) states that no single enzyme limits the objective, every `objectiveGain` is 0 and the rows are in model order. The warning names the protein pool exchange when that is at its bound. `getEnzymeBottlenecks` raises an error on a light ecModel, when `foldChange` is not greater than 1 and when the solve is not optimal. `get_enzyme_bottlenecks` raises a `ValueError` when `fold_change` is not greater than 1 and a `RuntimeError` when the solve is not optimal. - -The function is derived from `get_protein_bottlenecks` of the earlier geckopy package ([Carrasco et al., 2023](https://doi.org/10.1128/spectrum.01705-23)). [Relaxing overconstrained proteomics](relaxing-constraints.md) applies the shadow-price signal to a different task: `relaxProteomicsGreedy` / `relax_proteomics_greedy` uses it to select the next proteomics constraint to loosen, one relaxation step at a time. - -## The most parsimonious proteome - -Classical parsimonious FBA (pFBA) selects, among all flux distributions that reach the optimum of the objective, the one with the smallest total flux. `getPfbaEnzymes` / `pfba_enzymes` is the enzyme-aware analogue: among the same set of optimal flux distributions, it selects the one with the smallest total enzyme mass, the smallest sum of `usage_prot_*` fluxes. For an ecModel, that usually differs from the answer of classical pFBA, since it optimises the proteome that the flux distribution implies, where classical pFBA optimises the flux magnitudes. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -solution = getPfbaEnzymes(ecModel); -solution = getPfbaEnzymes(ecModel, 'fractionOfOptimum', 0.9); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import pfba_enzymes - -solution = pfba_enzymes(ec_model) -solution = pfba_enzymes(ec_model, fraction_of_optimum=0.9) -``` -::: -:::: - -`fractionOfOptimum` / `fraction_of_optimum` (default 1.0) allows the objective to fall from its optimum by `1 - fractionOfOptimum * 0.999999` times the magnitude of the optimum before minimising enzyme usage, for a negative optimum (such as the glucose exchange maximised above) as for a positive one; a value below 1.0 accepts a lower objective value in return for a lower total enzyme mass where one exists. The objective is the current objective of the model, or the reaction given by `rxnId` (Python: the argument `objective`). The MATLAB function returns a structure with the fields `x`, `enzymeUsage`, `objectiveValue` and `stat`; the Python function returns a `cobra.Solution`. - -Both implementations require the `usage_prot_` reactions to run forward (`lb = 0`, the layout of a GECKO 4 ecModel). A reaction that was changed to allow reverse flux is handled in Python (its reverse variable is included in the minimised sum) and raises an error in MATLAB, since minimising the raw flux there would not minimise `|flux|` for that reaction. - -## Troubleshooting - -| Problem | Possible reason | Solution | -|---|---|---| -| Warning that no enzyme is at its usage bound, and every `objectiveGain` is 0 | the objective is limited by the protein pool exchange or by constraints outside the enzymes | Read `multipleActive` and the usage tables; relax the pool, or inspect the exchange constraints | -| `multipleActive` is true for the top enzymes | several capacity constraints are active at the same optimum | Rank by `objectiveGain`; the shadow price does not identify a single enzyme here | -| `highCapUsage` is empty | no enzyme has a usage bound below 1000 mg/gDCW, or none runs above the threshold | Constrain enzymes with measured concentrations, or lower the threshold | -| An error that names light ecModels | the ecModel is a GECKO light ecModel, which has no `usage_prot_` reactions | Use a full ecModel | -| `getPfbaEnzymes` raises an error about a negative lower bound | a `usage_prot_` reaction allows reverse flux | Set the lower bound of the usage reactions to 0 | - -## See also - -- [Simulation and analysis](simulation-and-analysis.md), the flux distributions that the usage and bottleneck data are computed from. -- [Relaxing overconstrained proteomics](relaxing-constraints.md), where shadow prices determine which constraint to relax next. -- [GECKO light and full ecModels](gecko-light.md), why `getEnzymeUsage`, `getPfbaEnzymes` and `getEnzymeBottlenecks` do not work on a light ecModel. diff --git a/docs/guide/enzyme-usage.md b/docs/guide/enzyme-usage.md new file mode 100644 index 0000000..1cbd851 --- /dev/null +++ b/docs/guide/enzyme-usage.md @@ -0,0 +1,301 @@ +# Enzyme usage and bottlenecks + +A flux distribution of a full ecModel gives the amount of each enzyme that it uses. GECKO computes the usage of each enzyme in a solution, the enzymes that run at their limit, the enzymes that limit the objective, and the flux distribution that uses the least enzyme mass. These functions need a full ecModel: a light ecModel has no `usage_prot_` reactions, and the functions raise an error on it ([Choose the ecModel structure](ecmodel-structure.md)). + + +:::::{dropdown} Place in the workflow + +The diagram orders the functions of this page within the [workflow](workflow.md). Boxes with a dashed outline are optional steps, a dotted arrow is an optional path, and a diamond is a choice. Rounded boxes with a dotted outline are the context, other pages or the model to start from, and a click on one opens its page. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Start: a full ecModel"]):::ctx + ME{"Measure"} + US["getEnzymeUsage
reportEnzymeUsage
(of a solution)"] + BN["getEnzymeBottlenecks"] + CC["getConcControlCoeffs"] + PF["getPfbaEnzymes"] + PV --> ME + ME -->|absolute or capacity usage| US + ME -->|enzymes that limit the objective| BN + ME -->|rise per added enzyme| CC + ME -->|least enzyme mass| PF + PF -.->|its solution| US + click PV href "first-ecmodel.html" +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Start: a full ecModel"]):::ctx + ME{"Measure"} + US["enzyme_usage
report_enzyme_usage
(of a solution)"] + BN["get_enzyme_bottlenecks"] + CC["get_conc_control_coeffs"] + PF["pfba_enzymes"] + PV --> ME + ME -->|absolute or capacity usage| US + ME -->|enzymes that limit the objective| BN + ME -->|rise per added enzyme| CC + ME -->|least enzyme mass| PF + PF -.->|its solution| US + click PV href "first-ecmodel.html" +``` +::: +:::: +::::: + +## Functions + +| MATLAB | Python | Note | +| --- | --- | --- | +| `getEnzymeUsage` | `enzyme_usage` | absolute and capacity usage of each enzyme in a flux distribution | +| `reportEnzymeUsage` | `report_enzyme_usage` | tables of the enzymes with the highest absolute usage and with a high capacity usage | +| `getEnzymeBottlenecks` | `get_enzyme_bottlenecks` | rank the enzymes by the rise of the objective when their usage bound is raised | +| `getConcControlCoeffs` | `get_conc_control_coeffs` | rise of the objective per mg/gDCW of added enzyme | +| `getPfbaEnzymes` | `pfba_enzymes` | the flux distribution with the smallest total enzyme usage at the optimum | + +## Setup + +The examples use a full ecModel `ecModel` (Python: `ec_model`) and the model adapter parameters `params`, loaded as in the setup of [Analyse an ecModel](analysis.md#setup). `growthRate` (Python: `growth_rate`) is a growth rate in /hour at which the model is feasible, for example a measured growth rate or 99 % of the maximum. + +## Choose a measure + +Each enzyme draws its protein from the shared protein pool (`prot_pool_exchange`) through a usage reaction `usage_prot_`, which runs forward. Its flux is the usage of the enzyme in mg/gDCW. After `makeEcModel`, every usage reaction has an upper bound of 1000 mg/gDCW, so the pool is the one enzyme constraint that can bind. `constrainEnzConcs` sets the upper bound of the usage reaction of each enzyme with a measured concentration to that concentration; the enzyme still draws from the pool ([Constrain enzymes with proteomics](proteomics.md)). + +| Question | Analysis | Condition | +| --- | --- | --- | +| Which enzymes take most of the protein pool? | absolute usage (`getEnzymeUsage`, `reportEnzymeUsage`) | any solution; the ranking shows the allocation of the pool when the pool limits the solution | +| Which enzymes run at their measured concentration? | capacity usage (`getEnzymeUsage`, `reportEnzymeUsage`) | enzymes have measured concentrations as upper bounds | +| Which enzymes limit the objective, and by how much? | bottleneck ranking (`getEnzymeBottlenecks`) | enzymes have upper bounds that bind | +| How much does the objective rise per unit of added enzyme? | control coefficients (`getConcControlCoeffs`) | as for the bottleneck ranking | +| What is the smallest proteome that reaches the optimum? | `getPfbaEnzymes` | a flux distribution with the least enzyme mass is wanted | + +Usage describes one flux distribution. An LP often has several optimal flux distributions, and their enzyme usage can differ. `getPfbaEnzymes` selects one of them by a stated rule (least enzyme mass), so compute the usage from its solution to compare conditions. + +An enzyme with a high usage does not have to limit the objective, and an enzyme with a low usage that runs at its bound can. To find the enzymes that limit the objective, the bottleneck ranking raises the bound of one enzyme at a time and solves again. If no enzyme has a bound that binds, which is the normal case without proteomics data, the ranking and the control coefficients are 0 for all enzymes. The objective is then limited by the protein pool or by another bound of the model, such as an uptake rate; the warning of `getEnzymeBottlenecks` states whether the pool is at its bound. + +## Compute the enzyme usage + +`getEnzymeUsage` reads the flux of each usage reaction from a flux distribution. The example fixes the growth rate, minimises the protein pool usage, and reports the enzymes: + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +ecModel = setParam(ecModel, 'lb', params.bioRxn, growthRate); +ecModel = setParam(ecModel, 'obj', 'prot_pool_exchange', -1); +sol = solveLP(ecModel); +usageData = getEnzymeUsage(ecModel, sol.x); +usageReport = reportEnzymeUsage(ecModel, usageData); +usageReport.topAbsUsage +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import enzyme_usage, report_enzyme_usage + +ec_model.reactions.get_by_id(params.bio_rxn).lower_bound = growth_rate +ec_model.objective = "prot_pool_exchange" +ec_model.objective_direction = "min" +sol = ec_model.optimize() +usage = enzyme_usage(ec_model, sol.fluxes) +report = report_enzyme_usage(ec_model, usage) +report.top_abs_usage +``` +::: +:::: + +The usage data has one entry per enzyme of `ec.enzymes` that has a usage reaction (an enzyme without one is left out with a warning): + +| Field (MATLAB / Python) | Content | Unit | +| --- | --- | --- | +| `protID` / `prot_id` | UniProt identifier | (none) | +| `absUsage` / `abs_usage` | absolute usage: the absolute flux of `usage_prot_` | mg/gDCW | +| `UB` / `ub` | upper bound of `usage_prot_`; after `flexibilizeEnzConcs` this can be above the measured concentration | mg/gDCW | +| `capUsage` / `cap_usage` | capacity usage: `absUsage / UB`, the fraction of the available enzyme that the solution uses | fraction | +| `fluxes` | the flux distribution that was given | units of the reactions | +| `measuredProtein`, `demandedProtein`, `demandRatio` (Python: `measured_protein`, `demanded_protein`, `demand_ratio`) | sum of the measured concentrations in `ec.concs`, sum of the absolute usage of the same enzymes, and their ratio | mg/gDCW; ratio | +| `measuredProteinActive`, `demandRatioActive` (Python: `measured_protein_active`, `demand_ratio_active`) | the same, over the measured enzymes with an absolute usage above 1e-7 mg/gDCW | mg/gDCW; ratio | + +The option `zero` (Python: `include_zero`, default true) set to false leaves out the enzymes with zero usage. The demand fields are computed over all enzymes with a usage reaction, whatever its value. + +For an enzyme without a measured concentration, `UB` is 1000 mg/gDCW, so its capacity usage is close to 0 and has no meaning. Capacity usage applies to enzymes with a measured concentration. When `UB` is 0, the capacity usage is NaN in Python and NaN or Inf in MATLAB. + +A `demandRatio` above 1 means that the solution uses more of the measured enzymes than was measured. This happens when the usage bounds are above the measurements, for instance after `flexibilizeEnzConcs` ([Relax proteomics constraints](relaxing-proteomics.md)). A ratio at or below 1 does not exclude that one enzyme exceeds its measurement, because the sum is over all measured enzymes. Measured enzymes that carry no flux increase `measuredProtein`, so `demandRatioActive` is the ratio to read for a conflict among the active enzymes. The ratio is NaN when no concentration is measured. + +## Read the usage report + +`reportEnzymeUsage` returns: + +| Field (MATLAB / Python) | Content | Option, default | +| --- | --- | --- | +| `topAbsUsage` / `top_abs_usage` | the enzymes with the highest absolute usage, with the usage as a percentage of the upper bound of `prot_pool_exchange`, the kcat, its source and the reaction | `topAbsUsage` (Python: `top_abs_usage`), 10; 0 or `Inf` lists all enzymes | +| `highCapUsage` / `high_cap_usage` | the enzymes with a capacity usage above the threshold, with the same columns and the capacity usage | `highCapUsage` (Python: `high_cap_usage`), 0.9 | +| `totalUsageFlux` / `total_usage_flux` | the upper bound of `prot_pool_exchange`, in mg/gDCW | (none) | +| `demandRatio`, `demandRatioActive` (Python: `demand_ratio`, `demand_ratio_active`) | copied from the usage data | (none) | + +An enzyme that catalyses several reactions with flux has one row with the combined usage, marked `===` in the text columns, followed by one row per reaction. In `topAbsUsage`, enzymes whose reactions carry no flux are left out after the top N are selected, so the table can have fewer than N enzymes. In `highCapUsage`, MATLAB lists such an enzyme as a `===` row without reaction rows, and Python leaves it out. + +The `source` column shows where the kcat of the reaction came from. An enzyme near the top of `topAbsUsage` with the source `standard` has the standard kcat, a value that GECKO gives to reactions without a kcat from any source ([Choose the kcat sources](kcat-sources.md)). The usage of that enzyme depends on a value that is not specific to it; check the reaction for a measured kcat and put it in `data/kcats/custom.tsv` ([Set custom kcats](custom-kcats.md)). + +## Rank the enzymes that limit the objective + +`getEnzymeBottlenecks` solves the model with its current objective. For each enzyme whose usage is at its upper bound (upper bound above 0 and capacity usage at least 1 - 1e-6), it multiplies that bound by `foldChange` (Python: `fold_change`, default 2, must be greater than 1), solves again, and records the rise of the objective as `objectiveGain`. The enzymes are sorted by `objectiveGain`, highest first. An enzyme that is not at its bound gets 0 without a solve, because raising a bound that does not bind cannot change the optimum. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +ecModel = setParam(ecModel, 'obj', params.bioRxn, 1); +bottlenecks = getEnzymeBottlenecks(ecModel, 'top', 10); +``` + +The result is a table sorted by `objectiveGain`. `bottlenecks.Properties.UserData.nResolved` is the number of enzymes that were solved again, and `UserData.activeConstraints` the number of active capacity constraints. +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import get_enzyme_bottlenecks + +ec_model.objective = params.bio_rxn +ec_model.objective_direction = "max" +bottlenecks = get_enzyme_bottlenecks(ec_model, top=10) +``` + +The result is a DataFrame indexed by UniProt identifier. `bottlenecks.attrs["n_resolved"]` and `bottlenecks.attrs["active_constraints"]` hold the same two numbers as in MATLAB. +::: +:::: + +| Column (MATLAB / Python) | Content | +| --- | --- | +| `uniprot` (Python: the index), `gene` | the enzyme | +| `objectiveGain` / `objective_gain` | objective after raising the bound minus the objective before; 0 for enzymes not at their bound, for gains below 1e-10 and for solves that are not optimal | +| `shadowPrice` / `shadow_price` | the dual value of the `prot_` metabolite, for reference | +| `flux` | flux of the usage reaction, in mg/gDCW | +| `capUsage` / `cap_usage` | flux divided by the upper bound; NaN when the upper bound is 0 | +| `upperBound` / `upper_bound` | upper bound of the usage reaction, in mg/gDCW | +| `multipleActive` / `multiple_active` | true when another capacity constraint is active at the same optimum: another enzyme at its bound, or `prot_pool_exchange` at its upper bound | + +The shadow price is not used for the order. Enzyme-constrained models often have a degenerate optimum, where several constraints are active and the solver gives the dual value to one of them; the shadow price then depends on the basis that the solver returns. Where `multipleActive` is true, the shadow price does not describe the effect of the single enzyme. `objectiveGain` is computed from a new solve, so it does not depend on the dual values. + +`getEnzymeBottlenecks` maximises the objective. In Python, `objective_gain` is the improvement in the direction of the objective, so it is positive for a maximised objective that rises and for a minimised objective that falls. + +When no enzyme is at its usage bound, the function gives a warning that no single enzyme limits the objective (MATLAB identifier `getEnzymeBottlenecks:noEnzymeAtBound`; Python: a logged warning). Every `objectiveGain` is then 0, and the warning states whether the protein pool exchange is at its bound. + +## Compute control coefficients + +`getConcControlCoeffs` raises the upper bound of the usage reaction of each enzyme by `foldChange` (default 2), solves again, and divides the rise of the objective by the rise of the bound. The result is a rise of the objective per mg/gDCW of enzyme (for growth: per hour per mg/gDCW). It is not normalised to a relative (dimensionless) control coefficient. `flexibilizeEnzConcs` uses it to select the enzymes to relax ([Relax proteomics constraints](relaxing-proteomics.md)). + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +[evaluated, controlCoeffs] = getConcControlCoeffs(ecModel); +[evaluated, controlCoeffs] = getConcControlCoeffs(ecModel, 'method', 'shadow_price'); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import get_conc_control_coeffs + +evaluated, control_coeffs = get_conc_control_coeffs(ec_model) +evaluated, control_coeffs = get_conc_control_coeffs(ec_model, method="shadow_price") +``` +::: +:::: + +The option `method` selects how the coefficients are computed. The default `'finite_difference'` solves once per evaluated enzyme after raising its bound. `'shadow_price'` reads the reduced cost of each usage reaction from one solve and checks each nonzero value with one extra solve, because a reduced cost can overestimate the effect at a degenerate optimum; when the solver gives no reduced costs, the function uses `'finite_difference'`. Both functions raise an error when the first solve of the model is not optimal (MATLAB identifier `getConcControlCoeffs:infeasible`; Python: `ValueError`). + +Both outputs follow the order of the option `proteins`, which is `ec.enzymes` by default. The option `limit` (default 0) selects the enzymes whose usage divided by the upper bound is above that value; with 0, every enzyme with usage is evaluated. + +The two rankings differ in what they divide by. With the default `foldChange` of 2, the bound rises by the bound itself, so the control coefficient is `objectiveGain` divided by the upper bound. An enzyme with a small measured concentration therefore ranks higher by control coefficient than by `objectiveGain`. Use `objectiveGain` to find the enzyme whose doubling raises the objective most, and the control coefficient to find the enzyme where each added mg/gDCW raises the objective most. `getConcControlCoeffs` does not report whether other constraints are active. + +## Find the flux distribution with the least enzyme mass + +Parsimonious FBA (pFBA) selects, among the optimal flux distributions, the one with the smallest total flux. `getPfbaEnzymes` selects the one with the smallest sum of `usage_prot_*` fluxes, which is the total enzyme mass in mg/gDCW. It solves the model for the current objective (or for the reaction given by `rxnId`, Python: `objective`), keeps the objective at that optimum, and then minimises the sum of the usage fluxes. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +ecModel = setParam(ecModel, 'obj', params.bioRxn, 1); +solution = getPfbaEnzymes(ecModel); +solution = getPfbaEnzymes(ecModel, 'fractionOfOptimum', 0.9); +usageData = getEnzymeUsage(ecModel, solution.x); +``` + +`solution` is the result of `solveLP` with the added fields `enzymeUsage` (the minimised sum of the usage fluxes, mg/gDCW) and `objectiveValue` (the value of the original objective). +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import enzyme_usage, pfba_enzymes + +ec_model.objective = params.bio_rxn +ec_model.objective_direction = "max" +solution = pfba_enzymes(ec_model) +solution = pfba_enzymes(ec_model, fraction_of_optimum=0.9) +usage = enzyme_usage(ec_model, solution.fluxes) +``` + +`solution` is a `cobra.Solution`. +::: +:::: + +`fractionOfOptimum` (Python: `fraction_of_optimum`, default 1.0) lets the objective fall from its optimum by `1 - fractionOfOptimum * 0.999999` times the absolute value of the optimum. This also holds for a negative optimum, such as a maximised uptake flux. A value below 1 accepts a lower objective for a smaller enzyme mass. + +The usage reactions must have a lower bound of 0. MATLAB raises an error when a usage reaction allows reverse flux; Python includes the reverse flux in the minimised sum. + +## Troubleshooting + +| Problem | Possible reason | Solution | +| --- | --- | --- | +| Warning that no enzyme is at its usage bound, and every `objectiveGain` is 0 | no enzyme has a usage bound that binds; the protein pool or another bound of the model limits the objective, and the warning states whether the pool is at its bound | Read `topAbsUsage` for the enzymes that take most of the pool, or constrain enzymes with measured concentrations ([Constrain enzymes with proteomics](proteomics.md)) | +| `multipleActive` is true for the top enzymes | several capacity constraints are active at the same optimum | Use `objectiveGain`; the shadow price does not describe a single enzyme here | +| `highCapUsage` is empty | no enzyme has a measured concentration as its bound, or none is above the threshold | Constrain enzymes with measured concentrations, or lower `highCapUsage` | +| `demandRatio` is NaN | `ec.concs` has no measured concentration | Load proteomics data with `fillEnzConcs` | +| Error that names light ecModels | the ecModel is a light ecModel | Make a full ecModel | +| `getPfbaEnzymes` raises an error about a negative lower bound | a `usage_prot_` reaction allows reverse flux | Set the lower bound of the usage reactions to 0 | +| The `usage_prot_*` reactions carry negative flux | the model has these reactions in the reverse direction and was not loaded with `loadEcModel` | Run `upgradeEcModel` (Python: `upgrade_ec_model`) | + +## See also + +- [Analyse an ecModel](analysis.md): objectives, flux variability and the other analyses of the flux distribution. +- [Constrain enzymes with proteomics](proteomics.md): the measured concentrations that give the enzymes their own bounds. +- [Relax proteomics constraints](relaxing-proteomics.md): `flexibilizeEnzConcs` and `relaxProteomicsGreedy`, which use control coefficients and shadow prices to select the constraints to relax. +- [API reference](../api/index.md) and [MATLAB vs Python](../matlab-vs-python.md): the signature of each function in the table. diff --git a/docs/guide/first-ecmodel.md b/docs/guide/first-ecmodel.md new file mode 100644 index 0000000..c2e75a3 --- /dev/null +++ b/docs/guide/first-ecmodel.md @@ -0,0 +1,374 @@ +# Build an ecModel + +These steps build an enzyme-constrained model (ecModel) from a conventional genome-scale metabolic model (GEM) with the default settings of GECKO and geckopy. At the end, the project folder holds an ecModel with kcats and a protein pool, which reaches the growth rate of the organism, saved as a YAML file. Each step links to the page that describes the other options of that step. + + +:::::{dropdown} Place in the workflow + +The diagram orders the functions of this page within the [workflow](workflow.md). Boxes with a dashed outline are optional steps, a dotted arrow is an optional path, and a diamond is a choice. Rounded boxes with a dotted outline are the context, other pages or the model to start from, and a click on one opens its page. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Start: Installation"]):::ctx + S1["1. Create a project
createGECKOproject
fill model_adapter.toml"] + S2["2. Make the ecModel
ModelAdapterManager.setDefault
loadConventionalGEM
makeEcModel"] + S3["3. Assign kcats
getECfromGEM, then getECfromDatabase
findMetSmiles
updateKcatSources
assignKcats"] + S4["4. Set the protein pool
setProtPoolSize"] + S5["5. Check the growth rate
solveLP"] + S6["6. Tune kcats and fit sigma
sensitivityTuning (no change when growth reaches the target)
sigmaFitter"] + S7["7. Save the ecModel
saveEcModel"] + PV --> S1 --> S2 --> S3 --> S4 --> S5 --> S6 --> S7 + ND(["A decision page of the guide, to change a default"]):::ctx + NP(["Next: Constrain enzymes with proteomics (optional)"]):::ctx + NA(["Next: Analyse an ecModel"]):::ctx + S7 -.-> ND + S7 --> NP + S7 --> NA + click PV href "../installation/index.html" + click ND href "index.html" + click NP href "proteomics.html" + click NA href "analysis.html" +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Start: Installation"]):::ctx + S1["1. Create a project
geckopy init
fill model_adapter.toml"] + S2["2. Make the ecModel
ModelAdapter.from_folder
load_conventional_gem
make_ec_model"] + S3["3. Assign kcats
load_uniprot_tsv, load_kegg_tsv
fill_eccodes_from_gem, then fill_eccodes_from_database
find_met_smiles
update_kcat_sources
assign_kcats"] + S4["4. Set the protein pool
set_prot_pool_size"] + S5["5. Check the growth rate
model.slim_optimize"] + S6["6. Tune kcats and fit sigma
sensitivity_tuning (no change when growth reaches the target)
fit_sigma"] + S7["7. Save the ecModel
save_ec_model"] + PV --> S1 --> S2 --> S3 --> S4 --> S5 --> S6 --> S7 + ND(["A decision page of the guide, to change a default"]):::ctx + NP(["Next: Constrain enzymes with proteomics (optional)"]):::ctx + NA(["Next: Analyse an ecModel"]):::ctx + S7 -.-> ND + S7 --> NP + S7 --> NA + click PV href "../installation/index.html" + click ND href "index.html" + click NP href "proteomics.html" + click NA href "analysis.html" +``` +::: +:::: +::::: + +## Functions + +| MATLAB | Python | Note | +| --- | --- | --- | +| `createGECKOproject` | `geckopy init` | create a project folder with a template `model_adapter.toml` | +| `ModelAdapterManager.setDefault` | `ModelAdapter.from_folder` | read `model_adapter.toml` of the project | +| `loadConventionalGEM` | `load_conventional_gem` | load the GEM that `conv_gem` names | +| `makeEcModel` | `make_ec_model` | add the enzymes and the protein pool to the GEM | +| `loadDatabases` (called by `makeEcModel` and `getECfromDatabase`) | `load_uniprot_tsv`, `load_kegg_tsv` | read the UniProt and KEGG data of the organism | +| `getECfromGEM`, `getECfromDatabase` | `fill_eccodes_from_gem`, `fill_eccodes_from_database` | fill the EC numbers of the reactions | +| `findMetSmiles` | `find_met_smiles` | find the SMILES of the metabolites in PubChem | +| `updateKcatSources` | `update_kcat_sources` | make the kcat source files that the preset needs | +| `assignKcats` | `assign_kcats` | select one kcat per reaction and apply it | +| `setProtPoolSize` | `set_prot_pool_size` | set the upper bound of the protein pool | +| `setParam` | `reaction.lower_bound`, `model.objective` | set a bound or the objective | +| `solveLP` | `model.slim_optimize` | solve the model | +| `sensitivityTuning` | `sensitivity_tuning` | raise the kcats that limit growth (standard kcat tuning) | +| `sigmaFitter` | `fit_sigma` | fit the saturation factor $\sigma$ to the growth rate | +| `saveEcModel` | `save_ec_model` | save the ecModel as YAML in `models` | + +## Setup + +GECKO or geckopy must be installed (see [Installation](../installation/index.md)). The steps need an internet connection: the UniProt data of the organism, the BRENDA files and the SMILES of the metabolites are downloaded the first time they are used. The starting GEM is a curated model in SBML or YAML format. + +## 1. Create a project + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +createGECKOproject('name', 'ecMyModel', 'path', pwd); +``` + +`createGECKOproject` creates the folder `ecMyModel` with the subfolders `code`, `data`, `models` and `output`, and the template `model_adapter.toml`. It then changes the current folder of MATLAB to `ecMyModel`. +::: +:::{tab-item} 🐍 Python +:sync: python + +```bash +geckopy init ecMyModel +``` + +`geckopy init` creates the folder `ecMyModel` with the subfolders `data`, `models` and `output`, the template `model_adapter.toml`, and `adapter.py` for organism-specific code. The current folder does not change. +::: +:::: + +Copy the GEM into `models`, then set the parameters of the organism in `model_adapter.toml`. `conv_gem`, `org_name`, `c_source` and `bio_rxn` have no default value. `[uniprot] id` has no default either, and an empty `[kegg] id` means that KEGG is not used; set both for the organism. The example holds the values for [yeast-GEM](https://github.com/SysBioChalmers/yeast-GEM), in which `r_1714` is the glucose exchange and `r_4041` the biomass reaction: + +```toml +conv_gem = "models/yeast-GEM.yml" +org_name = "saccharomyces cerevisiae" + +sigma = 0.5 # average enzyme saturation +p_tot = 0.5 # total protein content [g protein/gDCW] +f = 0.5 # fraction of the protein that the enzymes of the model account for +gr_exp = 0.41 # growth rate without limitation of the nutrient uptake [1/h] + +c_source = "r_1714" +bio_rxn = "r_4041" +enzyme_comp = "cytoplasm" + +[uniprot] +type = "proteome" +id = "UP000002311" +gene_id_field = "gene_oln" +reviewed = false + +[kegg] +id = "sce" +gene_id = "kegg" +``` + +`[uniprot]` selects the UniProt data whose gene identifiers match those of the GEM, and `enzyme_comp` is the name of a compartment of the GEM. [Model adapter parameters](model-adapter-parameters.md) describes each parameter, and [Find the UniProt and KEGG parameters](ecmodel-structure.md#find-the-uniprot-and-kegg-parameters) describes how to find the values of an organism. + +## 2. Make the ecModel + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +adapter = ModelAdapterManager.setDefault(pwd); +params = adapter.getParameters(); +model = loadConventionalGEM(); +[ecModel, noUniprot] = makeEcModel(model); +``` + +`pwd` is the project folder, which `createGECKOproject` made the current folder. `noUniprot` lists the genes that neither database matched. +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import ModelAdapter, load_conventional_gem, make_ec_model + +adapter = ModelAdapter.from_folder("ecMyModel") +params = adapter.params +model = load_conventional_gem(adapter) +ec_model = make_ec_model(model, adapter) +``` +::: +:::: + +`makeEcModel` downloads the UniProt and KEGG data of the organism into `data/uniprot.tsv` and `data/kegg.tsv` when these files do not exist, with the settings of `[uniprot]` and `[kegg]`. With an empty `kegg.id`, it prints a warning and uses no KEGG data. `uniprot.id` has no default; without it and without `data/uniprot.tsv`, the build raises an error. + +The ecModel is a full ecModel: it has one pseudometabolite and one usage reaction per enzyme, and no kcats yet. [Choose the ecModel structure](ecmodel-structure.md) describes the light ecModel, the matching of genes to UniProt and KEGG, and the data of enzyme complexes. + +The model adapter holds the values that `model_adapter.toml` had when it was read (MATLAB: `ModelAdapterManager.setDefault`, Python: `ModelAdapter.from_folder`). After a change to the file, read it again with the same call. + +## 3. Assign kcats + +The kcat sources of the default preset (`brenda_ec_tree`) read the EC numbers of the reactions and the SMILES of the substrates, so these are filled first. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +ecModel = getECfromGEM(ecModel); +noEC = cellfun(@isempty, ecModel.ec.eccodes); +ecModel = getECfromDatabase(ecModel, 'ecRxns', noEC); +ecModel = findMetSmiles(ecModel); +updateKcatSources(ecModel); +[ecModel, decisions] = assignKcats(ecModel); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import ( + assign_kcats, fill_eccodes_from_database, fill_eccodes_from_gem, + find_met_smiles, load_uniprot_tsv, update_kcat_sources, +) +from geckopy.databases import load_kegg_tsv + +uniprot_db = load_uniprot_tsv(params.path / "data" / "uniprot.tsv") +kegg_db = load_kegg_tsv(params.path / "data" / "kegg.tsv", kegg_id=params.kegg.id) +fill_eccodes_from_gem(ec_model) +no_ec = [rxn for rxn, ec in zip(ec_model.ec.rxns, ec_model.ec.eccodes) if not ec] +fill_eccodes_from_database(ec_model, uniprot_db, kegg_db=kegg_db, ec_rxns=no_ec) +find_met_smiles(ec_model) +print(update_kcat_sources(ec_model)) +decisions, report = assign_kcats(ec_model) +``` + +`load_uniprot_tsv` and `load_kegg_tsv` read the files that `make_ec_model` downloaded; `load_kegg_tsv` returns `None` when `kegg.id` is empty and `data/kegg.tsv` does not exist. `assign_kcats` changes `ec_model` in place. +::: +:::: + +The EC numbers come from the annotation of the GEM, and from UniProt and KEGG for the reactions that the GEM does not annotate. `updateKcatSources` makes the kcat source files in `data/kcats` that the preset needs and prints a table with the status of each source. The BRENDA files are downloaded the first time they are used. `assignKcats` selects one kcat per reaction from the source files, applies the kcats to the model, and writes the decision for each reaction to `output/_kcatDecisions.tsv`. + +The default preset takes kcats from BRENDA measurements of the organism and from the measurements of the same EC number in other organisms. For the reactions that these leave without a kcat, it takes the predictions of the OpenKineticsPredictor (OKP) method KinForm-H. OKP is a web service that needs an API key, set as the environment variable `OKP_API_KEY` or in the file `data/okpApiKey.txt` (a key is generated at [predictor.openkinetics.org](https://predictor.openkinetics.org/)). The first call of `updateKcatSources` submits the OKP job and reports the source `okp_kinform_h` as `pending`; a later call fetches the result. Call `updateKcatSources` again until no source is `pending`, then run `assignKcats`. A source file that does not exist is skipped by `assignKcats`, and a reaction without a kcat from any source gets a kcat from an isozyme or the standard kcat (`getStandardKcat`, Python: `assign_standard_kcat`). + +[Choose the kcat sources](kcat-sources.md) describes the presets and the treatment of reactions without a kcat, and [Set custom kcats](custom-kcats.md) describes curated kcats. + +## 4. Set the protein pool + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +ecModel = setProtPoolSize(ecModel); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import set_prot_pool_size + +set_prot_pool_size(ec_model) +``` +::: +:::: + +`setProtPoolSize` sets the upper bound of `prot_pool_exchange` to $P_{tot} \cdot f \cdot \sigma \cdot 1000$ mg/gDCW, with the values of `p_tot`, `f` and `sigma` in `model_adapter.toml`. [Set the protein pool](protein-pool.md) describes how to choose the three values. + +## 5. Check the growth rate + +The growth rate of the ecModel is compared with `gr_exp`, the growth rate of the organism without a limitation of the nutrient uptake. The uptake of the carbon source is therefore opened (the uptake is a negative flux of the exchange reaction), and the biomass reaction is the objective. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +ecModel = setParam(ecModel, 'lb', params.c_source, -1000); +ecModel = setParam(ecModel, 'obj', params.bioRxn, 1); +sol = solveLP(ecModel); +growth = sol.x(strcmp(ecModel.rxns, params.bioRxn)) +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +ec_model.reactions.get_by_id(params.c_source).lower_bound = -1000 +ec_model.objective = params.bio_rxn +growth = ec_model.slim_optimize() +print(growth) +``` +::: +:::: + +[Basic model operations](basic-model-operations.md) describes bounds, objectives and the inspection of a solution. + +## 6. Tune kcats and fit sigma + +When the growth rate is lower than `gr_exp`, the standard kcat tuning raises the kcats that limit growth; when the model already reaches `gr_exp`, `sensitivityTuning` changes no kcat. Then `sigmaFitter` selects the saturation factor at which the growth rate of the model is closest to `gr_exp`, and sets the protein pool with that value. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +[ecModel, tunedKcats] = sensitivityTuning(ecModel); +[ecModel, sigma] = sigmaFitter(ecModel, 'makePlot', false); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import fit_sigma, sensitivity_tuning + +tuned_kcats = sensitivity_tuning(ec_model) +sigma = fit_sigma(ec_model).sigma +``` + +`sensitivity_tuning` and `fit_sigma` change `ec_model` in place. +::: +:::: + +`sensitivityTuning` selects the enzyme with the highest usage, multiplies the kcat of the reaction that uses the most of this enzyme by 10, and repeats this until the model reaches `gr_exp`. `tunedKcats` lists each changed reaction with its kcat before and after tuning, and `ec.source` of the reaction keeps the origin of the kcat, with the kcat before tuning as `prior`. [Choose a kcat tuning](kcat-tuning.md) describes how to check a tuned kcat and the advanced tuning against measured growth rates and fluxes, and [Set the protein pool](protein-pool.md) describes the fit of $\sigma$. + +## 7. Save the ecModel + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +saveEcModel(ecModel, 'ecMyModel.yml'); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import save_ec_model + +save_ec_model(ec_model, "ecMyModel.yml") +``` +::: +:::: + +The file is written to the `models` folder of the project. [ecModel YAML format](yaml-format.md) describes the format, which GECKO and geckopy both read. An ecModel can also be exported as SBML (MATLAB: `saveEcModel` with a file name that ends in `.xml`, Python: `cobra.io.write_sbml_model`), for use in other constraint-based tools; the SBML file does not keep the `ec` fields, so GECKO and geckopy cannot continue from it. + +## Next + +- [Choose the ecModel structure](ecmodel-structure.md): build a light ecModel, match the genes of the GEM to UniProt and KEGG, and add the subunit composition of enzyme complexes. +- [Choose the kcat sources](kcat-sources.md): select another preset and compare presets before they are applied. +- [Set custom kcats](custom-kcats.md): give reactions curated kcats from the literature or from measurements. +- [Set the protein pool](protein-pool.md): calculate $f$ from proteomics data and set the protein pool of a condition. +- [Choose a kcat tuning](kcat-tuning.md): select the standard or the advanced kcat tuning, and check the tuned kcats. +- [Tune kcats against experimental data](advanced-tuning.md): fit kcats to growth rates and exchange fluxes of one or more conditions. +- [Constrain enzymes with proteomics](proteomics.md): limit each measured enzyme to its measured concentration. +- [Relax proteomics constraints](relaxing-proteomics.md): loosen the proteomics constraints that prevent the measured growth rate. +- [Analyse an ecModel](analysis.md): simulate with other objectives, calculate flux ranges and sample solutions. +- [Enzyme usage and bottlenecks](enzyme-usage.md): report which enzymes a solution uses and which limit the objective. + +## Troubleshooting + +| Problem | Possible reason | Solution | +| --- | --- | --- | +| `makeEcModel` stops with an error that names `params.enzyme_comp` | `enzyme_comp` names no compartment of the GEM | Set `enzyme_comp` to a compartment name of the GEM, for example `cytoplasm` | +| `noUniprot` lists most genes of the GEM | the gene identifiers of the UniProt data do not match those of the GEM | Set `[uniprot]` as described in [Choose the ecModel structure](ecmodel-structure.md), delete `data/uniprot.tsv` and make the ecModel again | +| `updateKcatSources` reports the source `okp_kinform_h` as `failed`, with "No OpenKineticsPredictor API key found" | no OKP API key is set | Set `OKP_API_KEY` or write the key in `data/okpApiKey.txt`, then call `updateKcatSources` again | +| `updateKcatSources` reports a source as `pending` | the OKP job is still running | Call `updateKcatSources` again later | +| The growth rate does not change when the lower bound of `c_source` is changed | `makeEcModel` reverses a reaction with a negative lower bound and an upper bound of 0, so uptake through that exchange reaction is a positive flux in the ecModel | Set the upper bound of the exchange reaction | +| `sensitivityTuning` warns that the growth rate does not increase | the bound of an uptake reaction limits growth | Check the bounds of the uptake reactions | + +## See also + +- [Introduction to ecModels](introduction.md): the enzyme constraint and the protein pool. +- [Files in the data folder](data-folder-files.md): the files that the steps write in `data`. +- [Kcat assignment](../gecko4-kcat-assignment.md): the sources, stages and presets of `assignKcats`. +- [API reference](../api/index.md): the signatures of the functions in both toolboxes. diff --git a/docs/guide/gathering-kcats.md b/docs/guide/gathering-kcats.md deleted file mode 100644 index 1920fcf..0000000 --- a/docs/guide/gathering-kcats.md +++ /dev/null @@ -1,505 +0,0 @@ -# Gathering kcats - -An empty ecModel has the structure to hold enzyme constraints but no kcat values. Kcats are gathered from each source into a kcat source file in `data/kcats`: [BRENDA](https://www.brenda-enzymes.org/) by fuzzy matching, the EC tree, measured homologs, DLKcat and OpenKineticsPredictor. At the end, every source that the project uses has a source file that `assignKcats` can read. - -Not every enzymatic reaction has a reported measurement. BRENDA kcats are matched to a reaction by EC number and by the substrates of the reaction. DLKcat and OpenKineticsPredictor predict a kcat from the enzyme sequence and the substrate structure, independent of the EC number, which covers organisms with few measurements and reactions without an EC number. [Kcat assignment](../gecko4-kcat-assignment.md) describes how the source files are combined into one kcat per reaction, and [Applying kcats](applying-kcats.md) the steps after that. - - -:::::{dropdown} Place in the workflow - -The diagram orders the functions of this page within the [workflow](workflow.md). Dashed boxes are optional and dotted boxes are neighbouring pages. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - PV(["Previous: Building an empty ecModel"]):::ctx - PV --> A["EC numbers
getECfromGEM or getECfromDatabase"] --> A2["kcatsFromBrenda
kcatsFromECTree"] - PV --> C["Metabolite SMILES
findMetSmiles"] --> C2["kcatsFromDLKcat
kcatsFromOKP"] - PV --> E["kcatsFromHomologs"] - NX(["Next: Applying kcats"]):::ctx - A2 --> NX - C2 --> NX - E --> NX -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - PV(["Previous: Building an empty ecModel"]):::ctx - PV --> A["EC numbers
fill_eccodes_from_gem or fill_eccodes_from_database"] --> A2["kcats_from_brenda
kcats_from_ec_tree"] - PV --> C["Metabolite SMILES
find_met_smiles"] --> C2["kcats_from_dlkcat
kcats_from_okp"] - PV --> E["kcats_from_homologs"] - NX(["Next: Applying kcats"]):::ctx - A2 --> NX - C2 --> NX - E --> NX -``` -::: -:::: -::::: - -## Functions - -| MATLAB | Python | Note | -|---|---|---| -| `getECfromGEM` / `getECfromDatabase` | `fill_eccodes_from_gem` / `fill_eccodes_from_database` | populate EC numbers | -| `kcatsFromBrenda` | `kcats_from_brenda` | match kcats from BRENDA by EC number, substrate and organism, and write `brenda.tsv` | -| `fuzzyKcatMatching` | `fuzzy_kcat_matching` | the BRENDA matching alone, without a source file | -| `kcatsFromECTree` | `kcats_from_ec_tree` | estimate kcats from the EC tree, and write `ec_tree.tsv` | -| `kcatsFromHomologs` | `kcats_from_homologs` | estimate kcats from measured homologs, and write `homologs.tsv` | -| `findMetSmiles` | `find_met_smiles` | annotate metabolites with SMILES for DLKcat and OpenKineticsPredictor | -| `kcatsFromDLKcat` | `kcats_from_dlkcat` | run or read DLKcat, and write `dlkcat.tsv` | -| `writeDLKcatInput` / `runDLKcat` / `readDLKcatOutput` | `write_dlkcat_input` / `run_dlkcat` / `read_dlkcat_output` | the steps of a DLKcat run | -| `kcatsFromOKP` | `kcats_from_okp` | submit or fetch an OpenKineticsPredictor job, and write `okp_.tsv` | -| `submitOpenKineticsPredictor` / `fetchOpenKineticsPredictor` | `submit_open_kinetics_predictor` / `fetch_open_kinetics_predictor` | the steps of an OpenKineticsPredictor job | -| `saveKcatSource` / `loadKcatSource` | `save_kcat_source` / `load_kcat_source` | write a kcat list as a source file, and map a source file to the ecModel | -| `updateKcatSources` | `update_kcat_sources` | gather the sources that `model_adapter.toml` needs | -| `mergeKcats` | `merge_kcats` | merge kcat lists from multiple sources | - -## Setup - -The ecModel comes from [Building an empty ecModel](building-ec-model.md), and the project folder has a `model_adapter.toml` ([Project format 4](../gecko4-project-format.md)). The sources have these requirements: - -| Source file | Needs | -|---|---| -| `brenda.tsv` | EC numbers; the BRENDA files and the KEGG phylogenetic distances | -| `ec_tree.tsv` | EC numbers; the BRENDA files | -| `homologs.tsv` | enzyme sequences (`ec.sequence`); the BRENDA files; DIAMOND | -| `dlkcat.tsv` | SMILES of the metabolites; Docker for a DLKcat run | -| `okp_.tsv` | SMILES of the metabolites; an OpenKineticsPredictor API key | - -GECKO does not include BRENDA data. `fetchBrendaAsset` (Python: `fetch_brenda_asset`) downloads the files of the toolbox-assets release `brenda-2026.1-1` on first use, and a project uses its own files when it has the folder `data/brenda`. DIAMOND is the executable of RAVEN (`getDiamond`), which is downloaded when missing. Reactions that no source covers get custom kcats and the standard kcat, described in [Applying kcats](applying-kcats.md#provide-custom-kcat-values) and [Applying kcats](applying-kcats.md#assign-a-standard-kcat-value-optional). - -## Kcat source files - -Each `kcatsFrom*` function (Python: `kcats_from_*`) writes the kcats of one source to `data/kcats/.tsv` and returns the same kcat list (`kcatList`, `pandas.DataFrame` in Python). The option `save` set to false skips the file, and `saveAs` (Python: `save_as`) sets another name. The option `ecRxns` (Python: `ec_rxns`) restricts the reactions, except for `kcatsFromHomologs`: MATLAB takes a logical vector over `ecModel.ec.rxns` (`kcatsFromBrenda` and `kcatsFromECTree` also accept indices), Python a list of reaction identifiers. `kcatsFromDLKcat` takes `ecRxns` together with `'run', true` (Python: `run=True`), and `kcatsFromOKP` in a call that submits a job. With a subset and the default name, the file holds the reactions of the subset and no others. - -The rows of a source file are keyed by the conventional reaction and the enzyme set. The index in `ecModel.ec.rxns` is not part of the key. A file stays valid when the ecModel is rebuilt, and applies to a full and a light ecModel. `saveKcatSource` writes a kcat list made in another way, for example one that `fuzzyKcatMatching` returned, and `loadKcatSource` maps a file back to the ecModel and reports the rows that fit no reaction. A file with `kind: curated` in its header, such as `custom.tsv`, is never overwritten. - -`assignKcats` reads the files that `[kcat_assignment]` lists, so any combination of sources works. The hosted [OpenKineticsPredictor](https://predictor.openkinetics.org/) service offers several predictor methods and needs no Docker. DLKcat runs locally in a Docker image. - -## Assign EC numbers to reactions - -BRENDA matching and the EC tree use EC numbers. EC numbers can come from the starting GEM's EC number annotation, if it has one (MATLAB: the `model.eccodes` field; Python: each reaction's `annotation["ec-code"]`), or be gathered from UniProt and KEGG annotations. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -If the starting GEM has manually curated EC numbers, populate `ecModel.ec.eccodes` with model-derived EC numbers first, then fill the rest from the database. `getECfromDatabase` overwrites entries in `ecModel.ec.eccodes`, so mark which entries are still empty before calling it: - -```matlab -ecModel = getECfromGEM(ecModel); -noEC = cellfun(@isempty, ecModel.ec.eccodes); -ecModel = getECfromDatabase(ecModel, noEC); -``` - -If the starting GEM is not annotated with a `model.eccodes` field, does not contain standard EC numbers (four groups of digits separated by periods), or its EC annotations are not trusted, run `getECfromDatabase` alone: - -```matlab -ecModel = getECfromDatabase(ecModel); -``` - -`applyECcodes` transfers EC numbers derived from the database back into the `model.eccodes` fields, if desired. -::: -:::{tab-item} 🐍 Python -:sync: python - -`fill_eccodes_from_database` rewrites every entry that `ec_rxns` selects, all reactions by default, so the reactions without an EC number are selected explicitly. The UniProt data `uniprot_db` is loaded with `load_uniprot_tsv`: - -```python -from geckopy import fill_eccodes_from_database, fill_eccodes_from_gem - -fill_eccodes_from_gem(ec_model) -no_ec = [ - rxn for rxn, ec in zip(ec_model.ec.rxns, ec_model.ec.eccodes) if not ec -] -fill_eccodes_from_database(ec_model, uniprot_db, ec_rxns=no_ec) -``` - -If the starting GEM's EC annotations are not trusted, skip `fill_eccodes_from_gem` and call `fill_eccodes_from_database(ec_model, uniprot_db)`. With the argument `kegg_db` (loaded with `load_kegg_tsv`), KEGG supplies the EC number of a reaction for which UniProt has no EC number or an incomplete one (ending in `-`). `copy_ec_to_gem` is the equivalent of `applyECcodes`, writing into each reaction's `annotation["ec-code"]` (cobrapy's convention) while MATLAB writes a top-level `model.eccodes` cell array. -::: -:::: - -:::{admonition} Difficulties finding EC numbers for the reactions -:class: note - -BRENDA matching and the EC tree require EC numbers. EC numbers can be entered manually in the model's EC annotation field, based on any species-specific information available, and loaded with the functions above. DLKcat and OpenKineticsPredictor do not require EC numbers, so a predictor can be the sole kcat source when EC numbers are hard to find. -::: - -## Gather kcats from BRENDA - -Fuzzy matching queries BRENDA by EC number, substrate and organism. It is called fuzzy because it allows kcats from other organisms or substrates, and wildcards in the EC number, when no exact match exists. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -kcatList_fuzzy = kcatsFromBrenda(ecModel); -``` - -The BRENDA files come from `fetchBrendaAsset`, and the KEGG phylogenetic distances from RAVEN. -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import kcats_from_brenda - -kcat_list_fuzzy = kcats_from_brenda(ec_model) -``` - -`kcats_from_brenda` loads the BRENDA files from `adapter.get_brenda_db_folder()` and the KEGG phylogenetic distances from `data/PhylDist.mat`, which is downloaded when missing. Both can be passed as the arguments `brenda` (loaded with `load_brenda_data`) and `phyl_dist` (loaded with `load_phyl_dist`). -::: -:::: - -The option `aggregation` (`'max'` or `'median'`) of `kcatsFromBrenda` (Python: `kcats_from_brenda`) selects the BRENDA value per EC number, substrate and organism; the default is `kcat_aggregate_brenda` of `model_adapter.toml`, or `'max'`. `updateKcatSources` and `assignKcats` pass `aggregation` of `[kcat_sources.brenda]` instead, which the presets set (`median` for `brenda_ec_tree`, `ec_tree_homologs`, `ec_tree_okp` and `gecko_median`; `max` for `gecko_classic`). That setting overrides `kcat_aggregate_brenda` for a `brenda.tsv` gathered that way. The direct call names the option `kcatAggregation` in `fuzzyKcatMatching` and `aggregate` in `fuzzy_kcat_matching`. The option `forceWClvl` (Python: `force_wildcard_level`) replaces that number of EC levels, from the right, by a wildcard in every EC number. `fuzzyKcatMatching` (Python: `fuzzy_kcat_matching`, which takes the loaded `brenda` and `phyl_dist`) returns the same list without writing a file. - -Each BRENDA kcat is divided by the coefficient of the substrate that it was measured on, so a reaction that consumes two molecules of that substrate gets a kcat per molecule of the substrate ([GECKO 3 to GECKO 4](../gecko3-to-gecko4.md) describes the change from GECKO 3). - -The kcat list documents which kcat was assigned to each reaction and how precise the match is: the number of EC wildcards used, and whether the match agreed on substrate and organism. A full match often does not exist, even for model organisms such as *S. cerevisiae*, and a partial organism or substrate match is returned. For each EC number, the levels of organism and substrate are tried in the order of the origin values 1, 2, 3, 5, 4 and 6 (see the example output below). When no level matches, the rightmost EC level is replaced by a wildcard and the levels are tried again. EC 2.4.2.3 (uridine phosphorylase), for example, becomes 2.4.2.- (pentosyltransferase). With a wildcard, the levels that require a matching substrate (origin 1 and 2) are skipped, because the last digits of the EC number encode the substrate specificity. When a reaction has several EC numbers, the match with the fewest wildcards and the earliest level is selected, and among those the highest kcat. - -:::{admonition} Example output -:class: note - -In the `full_ecModel` tutorial, the fuzzy `kcatList` documents, per reaction: the substrates and EC numbers matched against BRENDA, the proposed kcat, `wildcardLvl` (Python: `wildcard_level`; 0: `w.x.y.z`, 1: `w.x.y.-`, 2: `w.x.-.-`, 3: `w.-.-.-`, 4: `-.-.-.-`), and `origin`, the specificity level of the match reached (1: correct organism and substrate; 2: closest related organism, correct substrate; 3: correct organism, any substrate; 4: closest related organism, any substrate; 5: correct organism, specific activity; 6: closest related organism, specific activity). -::: - -## Estimate kcats from the EC tree - -The EC tree takes the median of all wild-type BRENDA measurements of the EC number, from any organism and substrate, and shrinks it with the weight `n/(n+k)` for `n` measurements towards the estimate of the 3-digit EC class, which is shrunk in turn towards the 2-digit class and the global median. A reaction with several EC numbers uses the one with the most measurements. Reactions without an EC number, or with EC classes without measurements, get no value. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -kcatList_ecTree = kcatsFromECTree(ecModel); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import kcats_from_ec_tree - -kcat_list_ec_tree = kcats_from_ec_tree(ec_model) -``` -::: -:::: - -The option `k` sets the shrinkage constant (default 5). The `detail` column of the kcat list holds the number of measurements of the EC number (`n`) and the deepest EC level with measurements (`depth`). - -## Estimate kcats from measured homologs - -For each reaction, enzyme and substrate, DIAMOND searches the enzyme sequence against the sequences of the BRENDA measurements (at least 40 % identity and 80 % coverage of both sequences). The estimate is the mean of the `k` most identical homologs that were measured on the same substrate, weighted by identity; with homologs of at least 95 % identity, it is the median of those alone. For an enzyme complex, the estimate is the median over the subunits. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -kcatList_homologs = kcatsFromHomologs(ecModel); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import kcats_from_homologs - -kcat_list_homologs = kcats_from_homologs(ec_model) -``` -::: -:::: - -The options `k` (default 10), `minIdentity` (Python: `min_identity`, default 0.4), `coverage` (default 0.8) and `threads` set the number of homologs and the DIAMOND thresholds. Model metabolites are matched to the substrates of the measurements through their ChEBI annotations, then their MetaNetX annotations, then their names. Reactions without a homolog that was measured on one of their substrates get no value. - -## Gather SMILES annotations - -DLKcat and OpenKineticsPredictor take the substrates as SMILES. Starting GEMs rarely include SMILES annotation, so this queries PubChem, creating `data/smilesDB.tsv`, or fills that file manually. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -[ecModel, noSMILES] = findMetSmiles(ecModel); -``` - -`findMetSmiles` reports the percentage of unique metabolites assigned a SMILES annotation. If it is below 100 %, as is almost always the case, inspect the `noSMILES` list of metabolite names. In some models the metabolite name is suffixed with the metabolite formula, which prevents matching with PubChem; curating `ecModel.metNames` resolves such issues. - -MATLAB prints the progress and result: - -``` -Check for local SMILES database... done. -SMILES could be found for 66% of the unique metabolite names. -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import find_met_smiles - -find_met_smiles(ec_model, cache_path=params.path / "data" / "smilesDB.tsv") -``` - -`find_met_smiles` mutates `ec_model` in place and reads and writes through the `cache_path` TSV. No PubChem network access is needed when every metabolite name is in the cache, as with the cache included in the tutorial. Unmatched metabolites are logged rather than returned as a `noSMILES` list: the number of unique metabolite names with a SMILES and the number without are logged at INFO level, counting only the names looked up in PubChem, and each name without a SMILES at DEBUG level. As in MATLAB, curate metabolite names (`metabolite.name`) if a formula suffix or similar naming issue prevents matching. -::: -:::: - -:::{admonition} Example output -:class: note - -In the `full_ecModel` tutorial, a SMILES annotation is found for 66 % of the unique metabolite names, after checking the local SMILES database and querying PubChem for the rest. -::: - -## Predict kcats with DLKcat - -DLKcat predicts a kcat from enzyme sequence information and substrate structural information in SMILES format. `kcatsFromDLKcat` with the option `run` writes the DLKcat input, runs DLKcat in a Docker image and keeps the predictions as `dlkcat.tsv`. The run downloads and starts the Docker image `ghcr.io/sysbiochalmers/dlkcat-gecko:0.1` automatically, so Docker must be installed and running. A run of a subset of the reactions records their number in `data/DLKcat_run.txt`; a run of all reactions leaves the file empty. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -kcatList_DLKcat = kcatsFromDLKcat(ecModel, 'run', true); -``` - -Without `'run', true`, `kcatsFromDLKcat` reads the output of an earlier run from `data/DLKcat.tsv`. -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import kcats_from_dlkcat - -kcat_list_dlkcat = kcats_from_dlkcat(ec_model, run=True) -``` - -Without `run=True`, `kcats_from_dlkcat` reads the output of an earlier run from `data/DLKcat.tsv` and raises `FileNotFoundError` when the file is missing. -::: -:::: - -The kcat list is similar in shape to the BRENDA one, but has no EC numbers, wildcard levels or origin values, and instead includes the `genes` of each reaction. In MATLAB the fields `eccodes`, `wildcardLvl` and `origin` are absent; in Python the columns `eccode`, `wildcard_level` and `origin` are present but empty. - -### Write the DLKcat input file - -`kcatsFromDLKcat` with `'run', true` replaces `data/DLKcat.tsv`. The input file can also be written, run and read in separate steps. Currency metabolites that occur in pairs (for example ATP versus ADP, NADH versus NAD) are excluded, unless a reaction has no other reactants after removing them, as for ATP synthase. A selection of small molecules (for example Fe2+) is excluded in every reaction. Nonexhaustive exclusion lists are included in the toolbox as `DLKcatCurrencyMets.tsv` and `DLKcatIgnoreMets.tsv` (MATLAB: in `GECKO/databases`; Python: inside the `geckopy.data` package); an ecModel-specific override can be placed under `data/` in the adapter folder with the same filenames. The lists apply to the OpenKineticsPredictor input and to the substrates of the homolog search as well. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -writeDLKcatInput(ecModel); -``` - -`writeDLKcatInput` raises an error when `data/DLKcat.tsv` exists; the option `'overwrite', true` replaces the file and discards the predictions that it holds. - -`DLKcat.tsv` is specific to either the full or the light version of the ecModel, because -it holds reaction identifiers from `ec.rxns` needed when loading the predicted values back into the ecModel. Build separate `DLKcat.tsv` files for the two ecModel versions. The source file `dlkcat.tsv` is keyed by the conventional reaction and applies to both. -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import load_dlkcat_ignore_lists, write_dlkcat_input - -ignore_lists = load_dlkcat_ignore_lists(params.path / "data") -write_dlkcat_input( - ec_model, params.path / "data" / "DLKcat.tsv", ignore_lists, -) -``` - -`load_dlkcat_ignore_lists` reads the currency-metabolite and small-molecule exclusion lists (`DLKcatCurrencyMets.tsv`/`DLKcatIgnoreMets.tsv`, or their project-specific overrides under `data/`) that MATLAB's `writeDLKcatInput` reads implicitly. `write_dlkcat_input` raises `FileExistsError` when the file exists; `overwrite=True` replaces it. The same full-vs-light `DLKcat.tsv` caveat applies: the file encodes `ec_model.ec.rxns` identifiers, so a full-model file cannot be reused for a light model or the reverse. The source file `dlkcat.tsv` is keyed by the conventional reaction and applies to both. -::: -:::: - -### Run DLKcat and load its output - -In MATLAB no other input is given, because the functions assume the input file is at `data/DLKcat.tsv`. The Python functions take the path of the file: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -runDLKcat(); -kcatList_DLKcat = readDLKcatOutput(ecModel); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import read_dlkcat_output, run_dlkcat - -run_dlkcat(params.path / "data" / "DLKcat.tsv") -kcat_list_dlkcat = read_dlkcat_output( - ec_model, params.path / "data" / "DLKcat.tsv", -) -``` -::: -:::: - -These steps do not write `data/kcats/dlkcat.tsv`. `kcatsFromDLKcat(ecModel)` without `'run', true` reads `data/DLKcat.tsv` and writes the source file. `saveKcatSource(ecModel, kcatList_DLKcat, 'dlkcat')` (Python: `save_kcat_source(ec_model, kcat_list_dlkcat, "dlkcat")`) writes the list that `readDLKcatOutput` returned. - -## Predict kcats with OpenKineticsPredictor - -OpenKineticsPredictor (OKP) shares its input with DLKcat: protein sequences and single-substrate SMILES, gathered in the section on SMILES annotations. It needs no Docker. The method is set by the option `method` or, without it, by `[okp] method` of `model_adapter.toml` (default CataPro). The methods are CataPro, CatPred, CatRange, DLKcat, EITLEM, KinForm-H, KinForm-L, OmniESI, OmniESI-O2DENet, TurNup and UniKP. `kcatsFromOKP` submits enzyme-substrate pairs and raises an error for CatPred and TurNup, which read whole reactions. - -:::{admonition} Storage of the API key -:class: warning - -OKP requires a personal Bearer API key (looks like `ak_...`). Get one free, no registration required, at [predictor.openkinetics.org/api-docs](https://predictor.openkinetics.org/api-docs) (the "API key generator" section; the key is shown once, and can be revoked and regenerated from the same page). Keys are tied to the requesting IP and have a daily prediction quota that resets at midnight UTC. - -Because the model adapter is shared and committed to version control, the key is not stored there. Both languages resolve it the same way, checked in this order: an explicit function argument; the `OKP_API_KEY` environment variable; or a plain-text `data/okpApiKey.txt` file in the adapter folder (a filename already git-ignored by GECKO). -::: - -OKP is a web service with a job queue. `kcatsFromOKP` submits a job when the method has none, and fetches the result of the job on a later call. The file `okp_.tsv` is written once the job is done. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -kcatsFromOKP(ecModel, 'method', 'CataPro'); -[kcatList_OKP, done] = kcatsFromOKP(ecModel, 'method', 'CataPro', 'wait', true); -``` - -The first call submits the job. The second call waits for the result with `'wait', true`; without it, the call reports the status of the job and returns `done` as false, and `kcatList_OKP` is empty. -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import kcats_from_okp - -kcats_from_okp(ec_model, method="CataPro") -kcat_list_okp, done = kcats_from_okp(ec_model, method="CataPro", wait=True) -``` - -The first call submits the job. The second call waits for the result with `wait=True`, polling every 30 seconds; without it, the call returns `(None, False)` while the job runs. -::: -:::: - -Each method keeps its job in `data/OKP_job_.txt` and its result in `data/OKP_output_.csv`, so the jobs of several methods can run at the same time. The option `resubmit` submits a new job and removes the earlier result, and `useStored` (Python: `use_stored`) reads the stored result of the current job without contacting OKP. The `detail` column of the source file holds the similarity of the enzyme to the training sequences of the method. - -`submitOpenKineticsPredictor` and `fetchOpenKineticsPredictor` are the steps that `kcatsFromOKP` calls. They write the request to `data/OKP.csv` and the job identifier to `data/OKP_job.txt`, store the result in `data/OKP_output.csv`, and do not write a source file. `fetchOpenKineticsPredictor` returns `[done, kcatList]`; with `'wait', true` (Python: `wait=True`) it polls every `pollInterval` seconds (Python: `poll_interval`, default 30; `timeout`, default one hour). - -## Gather all configured sources - -`updateKcatSources` gathers the source files that `[kcat_assignment]` of `model_adapter.toml` uses and that are missing or outdated, with the functions above. A file is outdated when its header records other settings than `[kcat_sources.]` asks for, or for `brenda` and `ec_tree` another BRENDA release than the files in use. `custom.tsv` is left as it is. In MATLAB, the function checks Docker and the OKP API key first. In Python, a source whose requirement is missing gets the status `failed`. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -report = updateKcatSources(ecModel, 'dryRun', true); -report = updateKcatSources(ecModel); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import update_kcat_sources - -report = update_kcat_sources(ec_model, dry_run=True) -report = update_kcat_sources(ec_model) -``` -::: -:::: - -`report` has one row per source with the columns `source`, `status` (current, gathered, pending, outdated, missing, failed or skipped) and `note`. An OKP source is pending while its job runs, and a later call fetches the result. - -## Merge kcat lists - -`assignKcats` (Python: `assign_kcats`) reads the source files, merges them by the stages of the preset and writes the result to the ecModel. `mergeKcats` (Python: `merge_kcats`) merges kcat lists that are in memory, for example those returned in the sections above, by a priority order that the caller gives. It keeps the highest-priority source for each reaction. The same call works with `kcatList_OKP` / `kcat_list_okp` in place of the DLKcat list, since the function merges by structure: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -kcatList_merged = mergeKcats({kcatList_fuzzy, kcatList_DLKcat}, ... - {'database_top', 'dlkcat', 'database_bottom'}); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import merge_kcats - -kcat_list_merged = merge_kcats( - kcat_list_fuzzy, kcat_list_dlkcat, - source_priority=["database_top", "dlkcat", "database_bottom"], -) -``` -::: -:::: - -`mergeKcats` accepts any number of kcat lists, each assigned an explicit priority in `sourcePriority` (Python: `source_priority`). Sources that the priority does not list are dropped. Three reserved tier tokens rank BRENDA rows, and the position in the priority order sets the rank: - -| Token | Rows | -|---|---| -| `database_exact` | an exact experimental measurement without fuzzy metadata, which OpenKineticsPredictor can return directly | -| `database_top` | fuzzy BRENDA matches without EC wildcard and with an origin up to `topOriginLimit` (Python: `top_origin_limit`, default 4) | -| `database_bottom` | weaker fuzzy matches: origin up to `bottomOriginLimit` (default 6), and at most `wildcardLimit` (default 3) EC wildcards; also rows without EC wildcard whose origin is above `topOriginLimit` | - -Any other token (`'dlkcat'`, `'catapro'`, ...) matches the `source` value of a row, in lower case with other characters than letters and digits as `_`; a leading `okp_` of the source of an OpenKineticsPredictor row is dropped for the match. The default `topOriginLimit` of 4 ranks BRENDA matches from specific activities (origin 5 and 6) after the other sources, and the value 6 ranks them with the other `database_top` matches. All rows of the winning tier of a reaction are kept, and `assignKcatValues` (Python: `apply_kcat_list`) selects the value among them (default: the highest). - -The option `databaseSources` (Python: `database_sources`, default `{'brenda','sabio_rk'}`) lists the sources whose rows go to the `database_*` tiers; rows of other sources only match by their own token. - -The merge does not query a wildcard EC number itself. The wildcard rows exist in the BRENDA kcat list, and `database_bottom` admits them for a reaction that no source of a higher tier covers. The merged list has the fields of the fuzzy-matching list and a `kcatSource` field (Python: the `source` column, which every list has) naming which source contributed each value. - -`mergeDLKcatAndFuzzyKcats` (Python: `merge_dlkcat_and_fuzzy_kcats`) wraps `mergeKcats` with the priority `database_top`, `dlkcat`, `database_bottom` and prints a deprecation warning ([GECKO 3 to GECKO 4](../gecko3-to-gecko4.md)). - -## Troubleshooting - -| Problem | Possible reason | Solution | -|---|---|---| -| `kcatsFromBrenda` returns kcats for few reactions | many reactions have no EC number | Fill `ecModel.ec.eccodes` (section on EC numbers), or use a predictor | -| `Cannot find Docker` in `runDLKcat` | Docker is not installed or not running | Install and start Docker (start MATLAB from the command line when the launcher does not find Docker), or use OpenKineticsPredictor with the method DLKcat | -| `writeDLKcatInput` reports that `DLKcat.tsv` already exists | the file can hold predictions, so the function does not replace it | Delete the file, or set `'overwrite', true` | -| `kcatsFromOKP` returns `done` as false | the OpenKineticsPredictor job is still running | Call it again later, or with `'wait', true` | -| Error `kcatsFromOKP:method` | the method is CatPred or TurNup | Select another method | -| No OpenKineticsPredictor API key found | no key as argument, in `OKP_API_KEY` or in `data/okpApiKey.txt` | Generate a key and store it in one of the three places of the warning in the OpenKineticsPredictor section | -| Rows of a source file are not applied | the row fits no reaction, or part of an enzyme set | Read the report of `loadKcatSource` | -| `mergeKcats` drops a source | the source is not in the priority order | Add its token to `sourcePriority` | - -## See also - -- [Kcat assignment](../gecko4-kcat-assignment.md), how the source files become one kcat per reaction. -- [Applying kcats](applying-kcats.md), the next step: custom values, isozyme means, a standard fallback, and applying the resulting constraints to the ecModel. -- [Project format 4](../gecko4-project-format.md), the layout of `data/kcats` and the kcat source files. -- [Building an empty ecModel](building-ec-model.md), where the empty ecModel these kcats attach to comes from. -- [API reference](../api/index.md), every function in both toolboxes. diff --git a/docs/guide/gecko-light.md b/docs/guide/gecko-light.md deleted file mode 100644 index 128b7ec..0000000 --- a/docs/guide/gecko-light.md +++ /dev/null @@ -1,103 +0,0 @@ -# GECKO light and full ecModels - -GECKO builds two ecModel layouts from the same conventional GEM, **full** and **light**. A full ecModel has a pseudo-metabolite and a usage reaction for every enzyme, and a light ecModel has one shared protein pool. Full and light ecModels differ in layout and in the functions that they support. - -## Functions - -| MATLAB | Python | Note | -|---|---|---| -| `makeEcModel(model, true)` | `make_ec_model(model, adapter, gecko_light=True)` | build a light ecModel in place of a full one | -| `getSubsetEcModel` | `get_subset_ec_model` | build a context-specific ecModel from a general ecModel, full or light | - -## Setup - -A conventional GEM and a model adapter are needed, as described in [Building an empty ecModel](building-ec-model.md), which also shows the build of both layouts. The flag `geckoLight` (Python: `gecko_light`) is set when the ecModel is built, and it is stored in the ecModel file as `gecko_light` (see [ecModel YAML format](yaml-format.md)). - -## Layout of the two ecModels - -When a metabolic reaction is catalysed by an enzyme, GECKO encodes the protein cost of that catalysis in the model in one of two ways: - -- **Full ecModel.** For each enzyme, GECKO adds a pseudo-metabolite (`prot_`) and a usage reaction (`usage_prot_`). A reaction catalysed by several isozymes is split into one copy per isozyme. The solver reports the usage of each enzyme directly, for example of enzyme P00350. The LP is much larger than that of the conventional GEM: the `ecYeastGEM.yml` of the `full_ecModel` tutorial has 8001 reactions, of which 1145 are protein reactions, and a Human-GEM full ecModel runs into the tens of thousands. -- **Light ecModel.** The per-enzyme pseudo-metabolites and usage reactions are left out, and the cost is drawn from a single shared protein pool. Each catalysed reaction has one extra stoichiometric coefficient against that pool, $-MW_{sum} / (k_{cat} \times 3600)$, where $MW_{sum}$ is the molecular weight of the enzymes of the isozyme, summed with the subunit copies as weights, so the reaction consumes from the pool. Isozymes are not split into separate reactions. `applyKcatConstraints` selects, for each reaction, the isozyme with the smallest $MW_{sum} / k_{cat}$ among the isozymes that have a kcat and an enzyme, and writes that coefficient. A reaction without a usable isozyme gets no coefficient. The LP stays close to the size of the conventional GEM. - -A light ecModel suits genome-scale models for which the full layout solves slowly, such as Human-GEM or Recon3D. The full layout suits smaller models, such as yeast or *E. coli*. - -| Property | Full | Light | -|---|---|---| -| Isozyme expansion | reactions split, one copy per isozyme (`_EXP_` suffix) | reactions stay singular; `ec.rxns` has one row per isozyme with the prefix `_` | -| Per-enzyme pseudo-metabolite `prot_` | yes | no | -| Per-enzyme usage reaction `usage_prot_` | yes | no | -| Shared protein pool | yes | yes, and it is the sole enzyme constraint | -| LP size | much larger than the conventional GEM | close to the conventional GEM | - -## Functions that need a full ecModel - -A light ecModel has no per-enzyme pseudo-metabolites and no usage reactions. The functions that depend on them raise an error. - -| Capability | MATLAB | Python | -|---|---|---| -| Proteomics constraints | `constrainEnzConcs` raises an error, because the usage reactions are missing; `flexibilizeEnzConcs` acts on the same reactions and raises the error `Protein concentrations have not been defined` when no concentration is set | `constrain_enz_concs` raises `ValueError`; `flexibilize_enz_concs` acts on the same reactions | -| Greedy proteomics relaxation | `relaxProteomicsGreedy` raises an error | `relax_proteomics_greedy` raises `NotImplementedError` | -| Enzyme usage and bottlenecks | `getEnzymeUsage`, `getEnzymeBottlenecks` and `getPfbaEnzymes` raise an error; `reportEnzymeUsage` takes the output of `getEnzymeUsage` | `enzyme_usage`, `get_enzyme_bottlenecks` and `pfba_enzymes` raise `NotImplementedError`; `report_enzyme_usage` takes the output of `enzyme_usage` | -| Flux, capacity usage or shadow price of one enzyme | no `usage_prot_` reaction to read | `Enzyme.flux`, `Enzyme.cap_usage`, `Enzyme.upper_bound` and `Enzyme.shadow_price` raise `NotImplementedError` | -| Concentration or kcat of one enzyme | not applicable | the setters of `Enzyme.concentration` and `Enzyme.kcats` raise `NotImplementedError` | -| Isozyme-based functions | `getKcatAcrossIsozymes`, `addNewRxnsToEC` and `convertCustomKcats` raise an error | `fill_kcats_from_isozymes` and `add_new_rxns_to_ec` raise `NotImplementedError`, `convert_custom_kcats` raises `ValueError` | - -See [Proteomics integration](proteomics-integration.md), [Relaxing overconstrained proteomics](relaxing-constraints.md) and [Enzyme usage and bottlenecks](enzyme-usage-and-bottlenecks.md) for what these functions do on a full ecModel. - -`assignKcats` and `collectKcats` skip the filling of kcats from isozymes on a light ecModel and apply the standard kcat when it is configured. The `custom.tsv` file written for a full ecModel applies to a light ecModel too. - -## Functions that work on a light ecModel - -Functions that read `ec.kcat` and `ec.mw`, or edit the LP directly, work on a light ecModel. These are the kcat sources and their assignment, `applyCustomKcats`, `applyKcatConstraints`, `getStandardKcat`, the kcat sensitivity tuning (`sensitivityTuning`), `applyComplexData`, `getEcRxnKeys` and `getSubsetEcModel`. `getSubsetEcModel` removes the rows of `ec.rxns` of the reactions it removes, after it drops the prefix `_` of the row identifiers. See [Gathering kcat values](gathering-kcats.md), [Applying kcat values](applying-kcats.md) and [Growth-rate tuning](growth-rate-tuning.md). - -## Reconstruction time and simulation accuracy - -The comparison below builds a light and a full ecModel of yeast-GEM, populated with kcat values from BRENDA fuzzy matching alone, and runs FBA with growth maximised on each. Both toolboxes include the comparison script (`plotlightVSfull` and `plot_light_vs_full`). It sets the lower bound of the glucose uptake reaction `r_1714` to -1000 and the growth reaction `r_2111` as the objective: - -| Measure | Light | Full | -|---|---|---| -| ecModel reconstruction (s) | 39 | 41 | -| FBA (s) | 0.57 | 0.89 | -| Mapping fluxes (s) | 0.103 | 0.161 | -| Growth rate that is reached (/hour) | 0.0252 | 0.0252 | - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -The `plotlightVSfull` script of the `full_ecModel` tutorial prints this comparison: - -``` -Comparison of duration light vs. full ecModel -ecModel reconstruction: 95% (39 vs 41 seconds) -FBA: 64% (0.57 vs 0.89 seconds) -Mapping fluxes: 64% (0.103 vs 0.161 seconds) -Growth rate that is reached: 0.0252 vs 0.0252 -``` -::: -:::: - -Reconstruction time does not differ much between the two, because both require the same time-consuming BRENDA fuzzy matching step. FBA and flux mapping back to the conventional GEM (see [Simulation and analysis](simulation-and-analysis.md)) are substantially faster on the light ecModel. The difference matters most when many simulations are run, such as an extensive ecFVA scan or many single-cell ecModels built from the same generic model. - -The two ecModels reach the same growth rate, and the flux distributions they predict are near identical: the reactions that deviate by more than 0.1 % between the two are in lipid metabolism. Yeast-GEM's lipid metabolism follows the SLIMEr formulation, where reactions for each possible fatty-acyl-chain configuration are explicit and catalysed by the same enzyme, so the network has flexibility in which path it takes through lipid metabolism while reaching the same overall acyl-chain distribution. That flexibility comes from the conventional GEM and is independent of whether the enzyme constraints are recorded per enzyme or pooled. - -:::{tip} -When proteomics integration is not needed, a light ecModel gives comparable predictions to a full one at a fraction of the simulation cost. Proteomics integration, per-enzyme usage and bottleneck ranking need a full ecModel; there is no light equivalent for them. -::: - -## Troubleshooting - -| Problem | Possible reason | Solution | -|---|---|---| -| Error `This function does not work on GECKO light models.` from `getEnzymeUsage` | The ecModel is light | Build a full ecModel | -| Error from `getEnzymeBottlenecks`, `getPfbaEnzymes` or `relaxProteomicsGreedy` that a full ecModel is required | The ecModel is light | Build a full ecModel | -| `NotImplementedError` that names gecko-light (Python) | The function or property needs per-enzyme reactions | Build a full ecModel | -| The reaction identifiers of `ec.rxns` do not match `model.rxns` | A light ecModel prefixes the rows of `ec.rxns` with `_` | Remove the first four characters of the `ec.rxns` identifier | - -## See also - -- [Building an empty ecModel](building-ec-model.md), where the `geckoLight` (Python: `gecko_light`) choice is made. -- [Proteomics integration](proteomics-integration.md) and [Relaxing overconstrained proteomics](relaxing-constraints.md), which need a full ecModel. -- [Enzyme usage and bottlenecks](enzyme-usage-and-bottlenecks.md), which needs a full ecModel. -- [ecModel YAML format](yaml-format.md), for the `gecko_light` section and the identifiers of the rows of `ec.rxns`. diff --git a/docs/guide/getting-started.md b/docs/guide/getting-started.md deleted file mode 100644 index adaeb86..0000000 --- a/docs/guide/getting-started.md +++ /dev/null @@ -1,257 +0,0 @@ -# Getting started - -The reconstruction of an ecModel starts from a curated conventional genome-scale metabolic model (GEM) and a model adapter, the file that collects the organism-specific data and parameters that every later step reads. The steps below create a project folder with the starting GEM and a filled model adapter, and set the UniProt and KEGG parameters of the organism. - -:::{admonition} Based on the Nature Protocols pipeline -:class: note -This guide follows the structure of the published GECKO 3.0 Nature Protocols pipeline (Chen et al., 2024; see [Citations](../references.md)) and describes GECKO 4. The differences from GECKO 3 are in [GECKO 3 to GECKO 4](../gecko3-to-gecko4.md). -::: - - -:::::{dropdown} Place in the workflow - -The diagram orders the functions of this page within the [workflow](workflow.md). Dashed boxes are optional and dotted boxes are neighbouring pages. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - A1["createGECKOproject
Fill model_adapter.toml"] --> A2["loadDatabases('kegg')"]:::opt - NX(["Next: Building an empty ecModel"]):::ctx - A2 --> NX -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - A1["geckopy init
Fill model_adapter.toml"] --> A2["geckopy kegg-download"]:::opt - NX(["Next: Building an empty ecModel"]):::ctx - A2 --> NX -``` -::: -:::: -::::: - -## Functions - -| MATLAB | Python | | -|---|---|---| -| `createGECKOproject` | `geckopy init` | scaffold a new project folder with a template `model_adapter.toml` | -| `ModelAdapterManager.setDefault` | `ModelAdapter.from_folder` | load the model adapter of a project | -| `loadDatabases('kegg')` | `geckopy kegg-download` | download KEGG data for identifier conversion | - -## Setup - -GECKO (MATLAB) or geckopy (Python) must be installed (see [Installation](../installation/index.md)). The starting GEM is a curated model in a format that the toolbox reads. The example parameters are those of the `full_ecModel` tutorial, which uses [yeast-GEM](https://github.com/SysBioChalmers/yeast-GEM); the data files of that tutorial are described in [Files in the data folder](data-folder-files.md). - -## Start a new project - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -createGECKOproject(); -``` - -Two dialogs ask for the name of the project and for the location where `createGECKOproject` creates its folder. The options `name` and `path` replace the dialogs, as in `createGECKOproject('name', 'myproject', 'path', pwd)`. A folder that already exists at the location is not overwritten; the function prints a warning instead. - -`createGECKOproject` creates four subfolders: `code` and `data` hold custom code and data that the reconstruction and the analysis use, and `models` and `output` hold the reconstructed ecModels and the results of simulations. It writes the template `model_adapter.toml` in the project folder, with the project name in its first line, placeholders for the GEM and the organism, and the defaults of the other parameters. The option `'adapterFormat','class'` writes an adapter class `Adapter.m` instead (see [Project format 4](../gecko4-project-format.md)). -::: -:::{tab-item} 🐍 Python -:sync: python - -```bash -geckopy init myproject -``` - -`init` is a subcommand of the `geckopy` command-line tool that is installed with the package (see [Installation](../installation/index.md)). The command stops with an error when the folder exists and is not empty. - -`geckopy init` creates the subfolders `data`, `models` and `output`, which hold data for the reconstruction, the reconstructed ecModels and the results of simulations. It writes `adapter.py`, the counterpart of the `code` folder, in the project root, and the template `model_adapter.toml`. In the template, `conv_gem` and `org_name` are set to placeholders, and every other parameter is a comment line with its default value. -::: -:::: - -The template of both toolboxes has a section `[evotune]` with the hyperparameters of the CMA-ES kcat tuning (see [Tuning against experimental data](tuning-against-experimental-data.md)) and a section `[kcat_assignment]` for the kcat assignment (see [Kcat assignment](../gecko4-kcat-assignment.md)). The template of `geckopy init` lists the parameters of these sections as comments. The template of `createGECKOproject` sets `preset = "brenda_ec_tree"` and the other values of both sections. - -The starting GEM is stored under `models`. Other locations work, and `models` keeps the project self-contained. The folder that contains the model adapter is called the adapter folder. - -The files in `data` have the same formats (TSV, JSON) in both languages, so a `data` folder that one toolbox filled is usable in the other. [Files in the data folder](data-folder-files.md) lists every file that belongs there and where it is created. - -## The model adapter - -The model adapter is the file `model_adapter.toml`, which GECKO and geckopy read. Paths in it are relative to the project folder. A parameter that the file leaves out gets the default that is listed in `src/model_adapter/adapterParameters.tsv` of GECKO, and a key that is not in the list is an error in both toolboxes. [Model adapter parameters](model-adapter-parameters.md) describes every parameter. The values here are those of the `full_ecModel` tutorial, without the sections for kcat assignment and kcat tuning: - -```toml -conv_gem = "models/yeast-GEM.yml" -org_name = "saccharomyces cerevisiae" - -sigma = 0.5 -p_tot = 0.5 -f = 0.5 -gr_exp = 0.41 - -c_source = "r_1714" -bio_rxn = "r_4041" -enzyme_comp = "cytoplasm" - -[kegg] -id = "sce" -gene_id = "kegg" - -[uniprot] -type = "proteome" -id = "UP000002311" -gene_id_field = "gene_oln" -reviewed = true - -[complex] -taxonomic_id = 559292 -``` - -The adapter is loaded from the project folder: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -adapter = ModelAdapterManager.setDefault(projectPath); -params = adapter.getParameters(); -``` - -`ModelAdapterManager.setDefault` takes the project folder or the path of `model_adapter.toml`, and returns a `TomlAdapter`. The parameters are fields of `params` with the MATLAB names from `adapterParameters.tsv`, for example `params.Ptot` for `p_tot` and `params.uniprot.ID` for `uniprot.id`. Organism-specific behaviour that a value cannot express, for example a non-standard way to identify spontaneous reactions, is set in the sections `[spontaneous]` and `[matlab]` of the file, or in a subclass of `TomlAdapter`. -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import ModelAdapter - -adapter = ModelAdapter.from_folder("myproject") -params = adapter.params -``` - -`ModelAdapter.from_folder` reads `model_adapter.toml` from the folder. `adapter.py` of the project is not loaded by `from_folder`. A subclass of `ModelAdapter` in that file, for organism-specific behaviour that a value cannot express, is imported and instantiated explicitly, as in `MyAdapter.from_folder("myproject")`. Most projects edit the TOML file and leave `adapter.py` unchanged. -::: -:::: - -A GECKO 3 adapter class keeps working in MATLAB, and `convertAdapterToToml` writes it as `model_adapter.toml` (see [Upgrade a GECKO 3 project](../gecko3-upgrade-project.md)). - -A parameter that cannot be defined, because the information is absent or not relevant, is set to an empty string when it has a non-empty default. The default of `kegg.id` is `sce`, so a model of an organism that is not in KEGG has - -```toml -[kegg] -id = "" -``` - -### Querying UniProt and KEGG - -The UniProt database is the source of the molecular weights and the amino acid sequences of the enzymes and is essential. KEGG can assign EC number annotations and is optional. The query parameters select data with the same style of gene identifier as the starting GEM. A mismatch means that enzyme data cannot be matched to the genes of the model. - -:::{admonition} UniProt proteome and taxonomy identifiers -:class: warning - -UniProt data are identified by proteome or by taxonomy. The proteome is preferred, because it avoids redundant matches between gene and protein identifiers. -::: - -To find a UniProt proteome dataset, search [uniprot.org/proteomes](https://www.uniprot.org/proteomes) for the organism. If several proteome datasets exist for the organism, a [reference proteome](https://www.uniprot.org/proteomes?facets=proteome_type%3A1&query=%2A) is preferred. The UniProt page of one protein of the proteome shows under "Names & Taxonomy - Gene names" whether its gene identifiers have the same style as the starting GEM. - -*Saccharomyces cerevisiae* has 93 UniProt proteomes, and one of them is the reference proteome for strain S288c (`UP000002311`). A protein in that proteome has the gene identifier `YFL026W`, which matches the style of yeast-GEM. UniProt labels this style "Ordered locus name", and the API field is `gene_oln`. The model adapter accordingly sets `uniprot.type` to `proteome`, `uniprot.id` to `UP000002311` and `uniprot.gene_id_field` to `gene_oln`. - -The taxonomy is the alternative when no proteome fits. The identifier from the [taxonomy browser](https://www.uniprot.org/taxonomy/) should be as specific as possible. For *S. cerevisiae*, the taxonomy identifier 4932 covers 322 strains and 53,526 proteins in total, and 559292 refers to strain S288c alone, with 6,735 proteins. In that case `uniprot.type` is `taxonomy` and `uniprot.id` is `559292`. - -For KEGG, `kegg.id` is the three- or four-letter species identifier from the [KEGG organism list](https://www.genome.jp/kegg/catalog/org_list.html); for *Homo sapiens* it is `hsa`. The genome entry page (for example [genome.jp/entry/hsa](https://www.genome.jp/entry/hsa)) links to the list of KEGG genes and shows which gene identifier KEGG links to its genes. The KEGG genes of *H. sapiens* are numbered sequentially, which does not match the identifiers of human-GEM. The section "Other DBs" of the same page lists Ensembl gene identifiers (for example `ENSG00000236362`), so `kegg.gene_id` for human-GEM is `Ensembl`. - -## Handling unusual identifier formats - -If an organism is in UniProt and no `uniprot.gene_id_field` matches the gene identifiers of the model, a conversion table at `data/uniprotConversion.tsv` has columns of model genes and UniProt identifiers. - -The *Escherichia coli* model iML1515 has gene identifiers of the style `b0008`. UniProt does not have this style as a field, and KEGG links both identifiers. `uniprotConversion.tsv` is built from KEGG data (this needs a loaded model adapter; see [Building an empty ecModel](building-ec-model.md#set-the-default-model-adapter)): - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -DB = loadDatabases('kegg'); -fID = fopen(fullfile(params.path, 'data', 'uniprotConversion.tsv'), 'w'); -output = transpose([DB.kegg.keggGene, DB.kegg.uniprot]); -fprintf(fID, '%s\t%s\n', 'genes', 'uniprot'); -fprintf(fID, '%s\t%s\n', output{:}); -fclose(fID); -``` - -`loadDatabases('kegg')` takes the KEGG species from `kegg.id` of the adapter, so the adapter of iML1515 sets `kegg.id = "eco"` for this tab. -::: -:::{tab-item} 🐍 Python -:sync: python - -geckopy downloads KEGG data with the `geckopy` command-line tool, which has no Python function. The command writes a comma-delimited file without a header row, with the columns `uniprot`, `gene`, `kegg_gene` and further columns for EC numbers, molecular weights, pathways and sequences: - -```bash -geckopy kegg-download eco data/kegg.tsv -``` - -`make_ec_model` loads the file as `data/kegg.tsv`, so the command saves it under that name although the delimiter is a comma. The two relevant columns are written as `uniprotConversion.tsv`. The file has no header row, so `pandas.read_csv` receives the column names: - -```python -import pandas as pd - -columns = ["uniprot", "gene", "kegg_gene", "ec", "mw", "pathway", "sequence"] -kegg = pd.read_csv(params.path / "data" / "kegg.tsv", header=None, names=columns) -kegg[["kegg_gene", "uniprot"]].rename(columns={"kegg_gene": "genes"}).to_csv( - params.path / "data" / "uniprotConversion.tsv", sep="\t", index=False, -) -``` -::: -:::: - -Other organisms can need another source for the conversion table. For human-GEM, a conversion table is part of the starting GEM, and the `light_ecModel` tutorial (`protocol.m` and `protocol.py`) converts it to `uniprotConversion.tsv`. - -If an organism is in neither UniProt nor KEGG, the database files are built manually. `data/kegg.tsv` and `data/uniprot.tsv` of the `full_ecModel` tutorial are templates for the column layout. Artificial UniProt and KEGG identifiers work in these files, and the molecular weight and the sequence of each protein must be accurate for the rest of the reconstruction. - -:::{admonition} Example output -:class: note - -A new project has empty `data`, `models` and `output` subfolders (MATLAB also creates `code`) and a template `model_adapter.toml`. After the model adapter is filled in, MATLAB `loadDatabases` creates `data/uniprot.tsv` and `data/kegg.tsv` with queries of UniProt and KEGG. In Python, `geckopy uniprot-download` and `geckopy kegg-download` create the two files. When no UniProt or KEGG parameters are identified for the organism, `data/uniprotConversion.tsv`, `data/kegg.tsv` and `data/uniprot.tsv` are built by hand, as described above. -::: - -## Troubleshooting - -| Problem | Possible reason | Solution | -|---|---|---| -| Error that names a key of `model_adapter.toml` | the key is not in `adapterParameters.tsv`, or it is misspelled | Correct the key. A parameter that GECKO uses and geckopy does not use goes in `[matlab.params]` | -| UniProt or KEGG data of another organism | `uniprot.id` or `kegg.id` is not set, and the default (`559292`, `sce`) applies | Set the identifiers of the organism, or set `kegg.id = ""` | -| `geckopy init` stops with an error | the folder exists and is not empty | Use another folder name | -| Enzymes do not match the genes of the model | the gene identifier style of UniProt or KEGG differs from the starting GEM | Choose another `uniprot.gene_id_field` or `kegg.gene_id`, or build `data/uniprotConversion.tsv` | - -## See also - -- [Files in the data folder](data-folder-files.md), every file that a `data` folder can hold, and where it is used. -- [Model adapter parameters](model-adapter-parameters.md), every model adapter parameter in both languages. -- [Project format 4](../gecko4-project-format.md), the project folder and `model_adapter.toml`. -- [Building an empty ecModel](building-ec-model.md), the next step: loading the conventional GEM and expanding it into an ecModel. -- [API reference](../api/index.md), every function in both toolboxes. -- [MATLAB vs Python](../matlab-vs-python.md), what each toolbox has. diff --git a/docs/guide/growth-rate-tuning.md b/docs/guide/growth-rate-tuning.md deleted file mode 100644 index 7d563ca..0000000 --- a/docs/guide/growth-rate-tuning.md +++ /dev/null @@ -1,572 +0,0 @@ -# Growth-rate tuning - -A draft ecModel with active enzyme constraints usually predicts a growth rate far below what the organism reaches, because a maximum constraint has not yet been imposed on the protein pool and because some kcat values are uncertain or too low. Diagnosing which of the two limits growth, tuning the model against known physiological data until it reaches a realistic growth rate, and saving the result gives a functional ecModel. - -The maximum constraint of the protein pool is calculated as: - -$$ \sigma \times f \times P_{tot} $$ - -where sigma is the average in vivo saturation of every enzyme in the model, f is the mass fraction occupied by metabolic enzymes in the total proteome, and $P_{tot}$ is the total protein mass in grams per gram cell dry weight. When one or more factors are unknown, a default of 0.5 for each gives a default maximum constraint of $0.5 \times 0.5 \times 0.5 = 0.125$. - -:::{admonition} When no physiological data are available -:class: note - -Tuning requires physiological data, particularly growth rate. Without it, for example in large-scale reconstruction across many cell lines, the result is a draft ecModel: it may not simulate realistic fluxes, because the upper bound of the protein pool is undefined, but it can still compare cellular parameters such as protein costs. Even with no experimental data, a functional ecModel is reachable by assuming an arbitrary protein pool value and increasing low kcat values until simulations become reasonable. -::: - - -:::::{dropdown} Place in the workflow - -The diagram orders the functions of this page within the [workflow](workflow.md). Dashed boxes are optional and dotted boxes are neighbouring pages. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - PV(["Previous: Applying kcats"]):::ctx - PV --> SI["solveLP"] --> SE["sensitivityTuning
Evaluate the tuned kcats"] --> SG["sigmaFitter"] --> SV["saveEcModel"] - AL(["Alternative: Tuning against experimental data"]):::ctx - SI -.-> AL - AL -.-> SV - NX(["Next: Proteomics integration"]):::ctx - SV --> NX -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - PV(["Previous: Applying kcats"]):::ctx - PV --> SI["model.optimize"] --> SE["sensitivity_tuning
Evaluate the tuned kcats"] --> SG["fit_sigma"] --> SV["save_ec_model"] - AL(["Alternative: Tuning against experimental data"]):::ctx - SI -.-> AL - AL -.-> SV - NX(["Next: Proteomics integration"]):::ctx - SV --> NX -``` -::: -:::: -::::: - -## Functions - -| MATLAB | Python | Note | -| --- | --- | --- | -| `solveLP` | `Model.optimize` | runs FBA and predicts a growth rate | -| `setProtPoolSize` | `set_prot_pool_size` | sets the upper bound of the protein pool exchange reaction from Ptot, f and sigma | -| `sensitivityTuning` | `sensitivity_tuning` | iteratively raises the kcats that limit growth most | -| `sigmaFitter` | `fit_sigma` | fits sigma to the experimental growth rate | -| `setKcatForReactions` | `set_kcat_for_reactions` | sets a curated kcat value for specific reactions | -| `applyKcatConstraints` | `apply_kcat_constraints` | transfers the kcat values to the stoichiometric matrix | -| `saveEcModel` | `save_ec_model` | saves the functional ecModel | - -## Setup - -An ecModel with assigned kcats, the protein pool constraint of [Applying kcats](applying-kcats.md#constrain-the-protein-pool-exchange-reaction) and a model adapter with `gR_exp` (Python `gr_exp`), the experimental maximum growth rate. The code assumes the `full_ecModel` tutorial: `ecModel` (MATLAB) or `ec_model` (Python) is the ecModel and `params` holds the parameters of the model adapter. - -## Simulate growth rate - -Microorganisms reach their maximum growth rate when nutrient supply is not limiting, so the uptake of the carbon source is first set to be unconstrained: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -ecModel = setParam(ecModel, 'lb', params.c_source, -1000); -``` - -Or referring directly to the glucose exchange reaction in the *S. cerevisiae* ecModel: - -```matlab -ecModel = setParam(ecModel, 'lb', 'r_1714', -1000); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -geckopy/cobrapy has no `setParam` wrapper: bounds are attributes on the `cobra.Reaction` object itself. - -```python -ec_model.reactions.get_by_id(params.c_source).lower_bound = -1000 -``` - -Or directly: - -```python -ec_model.reactions.get_by_id("r_1714").lower_bound = -1000 -``` -::: -:::: - -The glucose exchange reaction, for example, is defined as `glucose[e] <=>`. A positive, forward flux represents dissipation from the system, so a negative flux signifies uptake into the cytoplasm, and a negative lower bound allows uptake. The direction of a reaction is queried with: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -constructEquations(ecModel, 'r_1714'); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -print(ec_model.reactions.get_by_id("r_1714").reaction) -``` -::: -:::: - -Growth is set as the objective, the flux distribution is predicted and the growth rate is printed: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -ecModel = setParam(ecModel, 'obj', params.bioRxn, 1); -sol = solveLP(ecModel); -bioRxnIdx = getIndexes(ecModel, params.bioRxn, 'rxns'); -fprintf('Growth rate: %f /hour\n', sol.x(bioRxnIdx)) -``` - -Output in the `full_ecModel` tutorial: - -``` -Growth rate: 0.116494 /hour -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -ec_model.objective = params.bio_rxn -sol = ec_model.optimize() -print(f"Growth rate: {sol.fluxes[params.bio_rxn]:.4f} /hour") -``` - -Setting `.objective` to a reaction id maximizes it by default; there is no separate "set objective coefficient" call, and no reaction-index lookup, since `sol.fluxes` is a pandas Series indexed by reaction id. -::: -:::: - -:::{admonition} Example output -:class: note - -In the `full_ecModel` tutorial, the growth rate right after applying enzyme constraints is about 0.12 h^-1. Because nutrient uptake is not limiting, this is the maximum the ecModel can reach. A rate substantially below 0.41 h^-1, the experimentally observed maximum growth rate of this organism (`params.gR_exp` in MATLAB, `params.gr_exp` in Python), indicates that the model needs tuning. -::: - -Three reasons can leave the predicted growth rate below the experimental maximum, and a systematic check distinguishes them: the stoichiometry of the metabolic network is not fully correct; the protein pool exchange reaction is too tightly constrained; or the kcat values assigned to reactions are too low. - -## Network stoichiometry - -The growth rate in the conventional GEM, without enzyme constraints, is predicted under the same exchange flux constraints: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -model = loadConventionalGEM(); -model = setParam(model, 'lb', params.c_source, -1000); -model = setParam(model, 'obj', params.bioRxn, 1); -sol = solveLP(model); -bioRxnIdx = getIndexes(model, params.bioRxn, 'rxns'); -fprintf('Growth rate: %f /hour\n', sol.x(bioRxnIdx)) -``` - -Output in the `full_ecModel` tutorial: - -``` -Growth rate: 19.944241 /hour -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import load_conventional_gem - -model = load_conventional_gem(adapter) -model.reactions.get_by_id(params.c_source).lower_bound = -1000 -model.objective = params.bio_rxn -sol = model.optimize() -print(f"Growth rate: {sol.fluxes[params.bio_rxn]:.4f} /hour") -``` -::: -:::: - -If the conventional GEM also cannot reach the maximum growth rate, the stoichiometry or the constraints of the starting GEM contain errors that should be resolved before an ecModel is reconstructed from it. - -:::{admonition} Example output -:class: note - -Without a constraint on carbon uptake, the conventional GEM in the `full_ecModel` tutorial reaches a growth rate of about 19.9 h^-1. This growth rate is biologically unrealistic given the unconstrained carbon uptake, but it confirms that the stoichiometry of the conventional GEM can reach 0.41 h^-1, so the enzyme constraints cause the shortfall above. -::: - -## Protein pool constraint - -The protein pool constraint from [Applying kcats](applying-kcats.md#constrain-the-protein-pool-exchange-reaction) is realistic when the sigma, $P_{tot}$ and f parameters in the model adapter are realistic; adjust the parameters if discrepancies show up. Regardless of those parameters, the simplest test of whether the protein pool limits growth is to relax the constraint of the pool usage reaction, which releases every enzyme constraint at once: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -ecModel = setParam(ecModel, 'ub', 'prot_pool_exchange', 1000); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -ec_model.reactions.prot_pool_exchange.upper_bound = 1000 -``` - -`set_prot_pool_size` narrows `prot_pool_exchange`'s upper bound to a realistic budget; relaxing it back to unconstrained means raising that upper bound back to geckopy's wide-open default (1000). -::: -:::: - -:::{admonition} GECKO 4: direction of prot_pool_exchange -:class: tip - -In GECKO 4 and geckopy, `prot_pool_exchange` is a forward reaction that produces `prot_pool`, with a lower bound of 0 and an upper bound of 1,000 by default (see [Building an empty ecModel](building-ec-model.md#extension-of-a-conventional-gem)). Relaxing the protein pool therefore raises its upper bound, and minimizing its usage is an ordinary minimization. The GECKO 3 protocol used the opposite (negative-flux) direction, where relaxing lowered the lower bound to -1,000 and minimizing usage required maximizing with coefficient `1`; see [GECKO 3 to GECKO 4](../gecko3-to-gecko4.md). -::: - -With neither a protein pool nor a nutrient constraint, the lowest protein pool usage that still supports the experimental maximum growth rate is predicted: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -ecModel = setParam(ecModel, 'lb', 'r_4041', 0.41); -ecModel = setParam(ecModel, 'obj', 'prot_pool_exchange', -1); -sol = solveLP(ecModel); -protPoolIdx = strcmp(ecModel.rxns, 'prot_pool_exchange'); -fprintf('Protein pool usage is: %.0f mg/gDCW\n', sol.x(protPoolIdx)) -``` - -`solveLP` maximizes the objective, so the coefficient `-1` minimizes the flux through `prot_pool_exchange`. - -Output in the `full_ecModel` tutorial: - -``` -Protein pool usage is: 407 mg/gDCW -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -cobrapy separates the objective reaction from the optimisation direction, so minimizing `prot_pool_exchange` usage is a plain positive objective with `objective_direction` set to `"min"`: - -```python -ec_model.reactions.get_by_id("r_4041").lower_bound = 0.41 -ec_model.objective = "prot_pool_exchange" -ec_model.objective_direction = "min" -sol = ec_model.optimize() -pool_usage = sol.fluxes["prot_pool_exchange"] -print(f"Protein pool usage is: {pool_usage:.0f} mg/gDCW") -``` -::: -:::: - -The predicted protein pool usage is set as the constraint on the protein pool exchange reaction, and the objective function and growth rate constraint that found it are reverted: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -ecModel = setParam(ecModel, 'ub', protPoolIdx, sol.x(protPoolIdx)); -ecModel = setParam(ecModel, 'lb', 'r_4041', 0); -ecModel = setParam(ecModel, 'obj', 'r_4041', 1); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -ec_model.reactions.prot_pool_exchange.upper_bound = pool_usage -ec_model.reactions.get_by_id("r_4041").lower_bound = 0 -ec_model.objective = "r_4041" -ec_model.objective_direction = "max" -``` -::: -:::: - -This yields an ecModel that reaches the intended growth rate whenever the conventional GEM can, but it often sets the protein pool usage to an unrealistic protein content. Increasing the kcat values of some reactions keeps the protein pool at the sigma, f and Ptot values of the model and is covered next. - -:::{admonition} Example output -:class: note - -Setting the upper bound of `prot_pool_exchange` to 1,000 mg/gDCW implies that the whole cell is 100 % protein. While unrealistic, this often avoids overconstraining. In the `full_ecModel` tutorial, the predicted protein pool usage is 407 mg/gDCW. With a total protein content (Ptot) of 500 mg/gDCW, this implies that the cell consists of 81.4 % protein. Assuming full enzyme saturation, a more realistic (lower) sigma would push the simulated total protein content above 100 %, so the recommended fix is a different one than releasing the protein pool constraint. -::: - -## Sensitivity tuning of kcat values - -Iteratively increasing the kcat values of the enzymes that limit growth until the desired growth rate is reached keeps the protein pool at the sigma, f and Ptot values of the model, which relaxing the pool does not. At this stage the enzyme constraints are dominated by the overarching protein pool constraint. Each deterministic iteration identifies the most limiting enzyme as the one that demands the largest fraction of the protein pool, and raises the kcat of the reaction that draws most on it. The raised value is the one that contributes most to overconstraining the model, which is not necessarily the one furthest from its true in situ value. - -The protein pool constraint is first reverted to a realistic value (see [Applying kcats](applying-kcats.md#constrain-the-protein-pool-exchange-reaction)), then the kcats are tuned and the changes reported: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -ecModel = setProtPoolSize(ecModel); -[ecModel, tunedKcats] = sensitivityTuning(ecModel); -struct2table(tunedKcats) -``` - -`sensitivityTuning` returns the updated ecModel and the `tunedKcats` structure, and with `verbose` at its default (`true`) prints the growth rate of each iteration. It also works on light ecModels. Output for the `full_ecModel` tutorial: - -``` -Iteration 1: Growth: 0.12508 -Iteration 2: Growth: 0.20879 -Iteration 3: Growth: 0.23956 -Iteration 4: Growth: 0.28248 -Iteration 5: Growth: 0.31398 -Iteration 6: Growth: 0.34943 -Iteration 7: Growth: 0.38066 -Iteration 8: Growth: 0.42157 -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import sensitivity_tuning, set_prot_pool_size - -set_prot_pool_size(ec_model) -tuning_result = sensitivity_tuning(ec_model) -``` - -`sensitivity_tuning` changes `ec_model` in place, with no separate returned model, and returns a `TunedKcatsResult` with the fields `rxns`, `rxn_names`, `enzymes`, `old_kcat`, `new_kcat` and `source`. It sets the growth objective from `params.bio_rxn` of the adapter and targets `params.gr_exp` when no growth rate is given. It also works on light ecModels. With `verbose=True` (the default) it logs the growth rate of each iteration at `INFO` level through the `logging` module. -::: -:::: - -The options are the same in both toolboxes (MATLAB name first): - -| MATLAB | Python | Default | Meaning | -| --- | --- | --- | --- | -| `desiredGrowthRate` | `desired_growth_rate` | `params.gR_exp` (Python `params.gr_exp`) | growth rate at which the tuning stops | -| `foldChange` | `fold_change` | `10` | factor by which each iteration raises a kcat | -| `protToIgnore` | `prot_to_ignore` | none | UniProt IDs whose enzymes are not selected as the limiting enzyme | -| `verbose` | `verbose` | `true` | reports the growth rate of each iteration | -| `modelAdapter` | | default model adapter | model adapter that supplies the parameters (Python reads `ec_model.adapter`) | -| | `max_iterations` | `1000` | upper limit of the iterations | -| | `bio_rxn` | `params.bio_rxn` | biomass reaction set as the objective | - -MATLAB accepts the options as name-value pairs (`sensitivityTuning(ecModel, 'foldChange', 5)`) or positionally (`sensitivityTuning(ecModel, desiredGrowthRate, modelAdapter, foldChange, protToIgnore, verbose)`); a call uses one of the two forms. Python takes keyword arguments. The tuning stops with a warning when an iteration raises the growth rate by less than 1e-9, which indicates that the uptake or pool constraints limit growth and that the kcats do not. - -Both toolboxes also provide a CMA-ES-based kcat tuning (`cmaesKcatTuning` in MATLAB, `cmaes_kcat_tuning` in Python), described in [Tuning against experimental data](tuning-against-experimental-data.md). - -`tunedKcats` (Python `tuning_result`) documents which kcat values were changed, with their previous value, source and the catalysed reaction. It is inspected to check whether a changed value was gathered incorrectly from a kcat source. `ec.source` of a tuned reaction keeps its origin and detail and gains the tuning status, for example `brenda (wc=0, origin=1) | tuned=sensitivity (prior=0.05)`, where the prior is the kcat before the first tuning (see [Provenance of a kcat](../gecko4-kcat-assignment.md#provenance-of-a-kcat)). A reaction that an earlier run has tuned keeps the prior of that run and adds no second `sensitivity`. A source that is empty is written as `unknown`. `ec.notes` is not changed. In a light ecModel the status is added to every ec row of the tuned reaction. Python `TunedKcatsResult.corrections()` returns the changes as the list of corrections that [Tuning against experimental data](tuning-against-experimental-data.md#read-the-result) describes. - -:::{admonition} Many kcat values are changed during tuning -:class: note - -Some kcat values changing is expected; the `full_ecModel` tutorial tunes 6. If a much larger fraction changes, check the following before assuming that the tuning itself is at fault: - -- No constraint is set on any nutrient uptake; the protein pool should be the sole limit. -- The maximum growth rate specified as `gR_exp`/`gr_exp` in the model adapter is realistic. -- The starting GEM can simulate the intended growth rate on its own, to exclude the network stoichiometry as the limiting factor (see [Network stoichiometry](#network-stoichiometry) above). -::: - -:::{admonition} Example output -:class: note - -In the `full_ecModel` tutorial, iterative tuning raises the growth rate from 0.125 h^-1 in the first iteration to 0.422 h^-1 in the eighth, above the target of 0.41 h^-1. - -The tuned `ec.kcat` entries appear in the tuned kcat output (see [Example tuned kcat output](#example-tuned-kcat-output) below). Transferring the changes into `data/kcats/custom.tsv` (see [Applying kcats](applying-kcats.md#provide-custom-kcat-values)) keeps them documented. -::: - -### Example tuned kcat output - -| rxns | rxnNames | enzymes | oldKcat | newKcat | source | -|------|----------|---------|---------|---------|--------| -| `r_0079` | 5'-phosphoribosylformyl glycinamidine synthetase | P38972 | 0.05 | 0.5 | brenda (wc=0, origin=4) | -| `r_0109` | acetyl-CoA carboxylase, reaction | P48445;Q00955 | 1.23 | 12.3 | brenda (wc=0, origin=2) | -| `r_0175_EXP_1` | aldehyde dehydrogenase (acetylaldehyde, NADP) | P40047 | 0.75 | 7.5 | brenda (wc=0, origin=2) | -| `r_0486_EXP_2` | glyceraldehyde-3-phosphate dehydrogenase | P00360 | 29 | 290 | custom | -| `r_0698` | lanosterol synthase | P38604 | 0.001937 | 0.1937 | brenda (wc=0, origin=5) | -| `r_1166_EXP_11` | glucose transport | P38695 | 19.26 | 192.6 | dlkcat | - -## Evaluate the tuned kcat values - -Tuning changes values automatically, based on which enzyme overconstrains the ecModel, without regard to whether the original value was itself unrealistic. Good evidence can exist that the tuned value reflects reality better than the source it replaced, but not every tuned value can be supported by literature data. The worked example covers one value from the example tuned kcat output, 5'-phosphoribosylformyl glycinamidine synthetase (reaction `r_0079`), which originates from BRENDA with a kcat of 0.05 s^-1 and which `sensitivityTuning` raises in the tutorial (to 0.5 s^-1). The same check is repeated for each value of interest. - -The match details behind the original BRENDA value are in the kcat list that `kcatsFromBrenda` (Python `kcats_from_brenda`) returns, which is kept as `data/kcats/brenda.tsv`: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -rxnIdx = find(strcmp(kcatList_fuzzy.rxns, 'r_0079')); -kcatList_fuzzy.wildcardLvl(rxnIdx) -kcatList_fuzzy.eccodes(rxnIdx) -kcatList_fuzzy.origin(rxnIdx) -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -kcat lists are `pandas.DataFrame`s in geckopy, one row per reaction, rather than a struct of parallel arrays, so this is a row lookup: - -```python -row = kcat_list_fuzzy.loc[kcat_list_fuzzy["rxn_id"] == "r_0079"].iloc[0] -print(row["wildcard_level"], row["eccode"], row["origin"]) -``` -::: -:::: - -In the `full_ecModel` tutorial, which assigns the kcats with the preset `gecko_classic` (see [Kcat assignment](../gecko4-kcat-assignment.md)), this yields a wildcard level of 0 (no EC wildcard), EC number 6.3.5.3 and origin 4 (any organism, any substrate, kcat). The levels are defined in the help of `fuzzyKcatMatching`. - -The specific [BRENDA entry](https://www.brenda-enzymes.org/enzyme.php?ecno=6.3.5.3) shows that the reported kcat of 0.05 comes from *E. coli* with NH4+ as substrate. The usual substrate is glutamine, so the value may not be a fair estimate for the reaction in the ecModel. The abstract of the original paper states that NH3 can replace glutamine as a nitrogen donor, with a Km of 1 M and a turnover of 3 min^-1 (2 % of the glutamine turnover). The same paper also reports a specific activity with glutamine as substrate, a more reasonable value to use here. The specific activity was reported as 2.15 micromol/min/mg protein, which equals mmol/min/g protein. Conversion to mol/s/g protein and multiplication with the molecular weight of the enzyme gives the kcat: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -convKcat = 2.15; -convKcat = convKcat / 1000; -convKcat = convKcat / 60; -enzMW = ecModel.ec.mw(strcmp(ecModel.ec.enzymes, 'P38972')); -convKcat = convKcat * enzMW -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -enz_idx = ec_model.ec.enzymes.index("P38972") -enz_mw = ec_model.ec.mw[enz_idx] -sa = 2.15 # umol/min/mg protein -conv_kcat = sa / 1000 / 60 * enz_mw -print(conv_kcat) -``` -::: -:::: - -The converted kcat is 5.3358, rounded to 5.34. The value in the ecModel is replaced with the new literature value, either documented in `data/kcats/custom.tsv` (see [Applying kcats](applying-kcats.md#provide-custom-kcat-values)) or applied directly. `setKcatForReactions` (Python `set_kcat_for_reactions`) sets `ec.kcat` of the reaction and `ec.source` to `manual`, and `applyKcatConstraints` (Python `apply_kcat_constraints`) transfers the new value to the stoichiometric matrix: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -ecModel = setKcatForReactions(ecModel, 'r_0079', 5.34); -ecModel = applyKcatConstraints(ecModel); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import apply_kcat_constraints, set_kcat_for_reactions - -set_kcat_for_reactions(ec_model, ["r_0079"], 5.34) -apply_kcat_constraints(ec_model) -``` - -`set_kcat_for_reactions` takes a list of reaction ids, even for a single reaction. The `kcat` argument is one value applied to all matched reactions, or a sequence with one value per matched reaction or one value per id. With its default `apply=False` it changes `ec.kcat` and `ec.source`, as in MATLAB; `apply=True` also applies the new kcat to the S-matrix immediately. -::: -:::: - -After this is repeated for each tuned value, the results are considered overall. When convincing evidence shows that the original value is more realistic than the tuned one, reactions or whole pathways may be missing from the network stoichiometry of the starting GEM. Such issues, as in [Network stoichiometry](#network-stoichiometry) above, are best resolved in the conventional GEM before the ecModel is reconstructed from it. - -## Fit the average saturation - -`sigmaFitter` (Python `fit_sigma`) sets sigma so that the ecModel reaches the experimental growth rate, after the kcats are tuned. It sizes the protein pool for a series of sigma values from 0.01 to 1 in steps of 0.01 (`n_sigma_steps = 100` in Python), solves the model for each and keeps the value whose growth rate is closest to the target growth rate. `sigmaFitter` returns the smallest sigma whose percent error is within a relative 1e-9 of the smallest error, so a growth plateau is returned at its start on every solver. `fit_sigma` with the default `method="grid"` takes the first grid point with the smallest error and has no tolerance, so solver rounding noise on a plateau can move the selected sigma to a later grid point. `fit_sigma` with `method="bisect"` finds a continuous sigma at 1e-6 resolution that is not on the grid and, for a target that growth cannot reach, the start of the plateau. The objective of the model must be growth. The bounds of the objective reaction are set to `[0, 1000]` before the scan and stay so on the returned model. The protein pool of the returned model is sized for the fitted sigma. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -[ecModel, sigma] = sigmaFitter(ecModel, 'makePlot', false); -``` - -`sigmaFitter` scans sigma from 0.01 to 1 in steps of 0.01. Ptot, f and the growth rate are read from the model adapter. The options `growthRate`, `Ptot`, `f`, `makePlot` (default `true`, a plot of the error against sigma) and `modelAdapter` are given as name-value pairs or positionally in this order; a call uses one of the two forms. -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import fit_sigma - -fit = fit_sigma(ec_model) -print(fit.sigma) -``` - -`fit_sigma` changes `ec_model` in place and returns a `SigmaFitterResult` with the fields `sigma`, `sigma_grid`, `growth_grid` and `error_grid` (the absolute relative error in percent). The keyword arguments `growth_rate`, `p_tot` and `f` default to the values of the model adapter. With `method="grid"` (the default) it scans `n_sigma_steps` (default 100) sigma values `i / n` and takes the first with the smallest error; growth rates that differ by solver noise on a plateau can move the selected sigma to a later grid point. With `method="bisect"` it uses the monotone growth in sigma to find sigma to a resolution of 1e-6 in about 20 solves, and for a growth rate that cannot be reached it returns the start of the plateau (growth within a relative 1e-9 of the growth at sigma 1). `sigma_fitter` is a deprecated alias of `fit_sigma` that raises a `DeprecationWarning`. -::: -:::: - -## Save the functional ecModel - -The tuned ecModel can simulate the physiological data it was tuned against. It is saved before [proteomics integration](proteomics-integration.md) or [simulation and analysis](simulation-and-analysis.md): - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -saveEcModel(ecModel); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import save_ec_model - -save_ec_model(ec_model, "ecYeastGEM.yml", adapter=adapter) -``` -::: -:::: - -## Troubleshooting - -| Problem | Possible reason | Solution | -| --- | --- | --- | -| Error that the FBA of the input model gives no valid result (`sensitivityTuning`), or `ValueError` that the FBA is not optimal (`sensitivity_tuning`) | The protein pool constraint or the exchange constraints make the model infeasible | Reduce the protein pool constraint with `setProtPoolSize` (Python `set_prot_pool_size`) and check the exchange constraints | -| Warning that growth does not increase from increased kcats | Uptake or pool constraints limit growth | Check the constraints on the uptake reactions; relax the protein pool as in [Protein pool constraint](#protein-pool-constraint) | -| Warning that no protein usage reaction carries flux, or that no reaction draws on the protein pool | The protein pool does not limit growth | Check the nutrient constraints and the network stoichiometry | -| `ValueError` that `prot_to_ignore` has no usage reaction for a protein | The UniProt ID is not an enzyme of the full ecModel | Correct the ID | -| Python `sensitivity_tuning` stops at `max_iterations` | The target growth rate is not reachable by raising kcats alone | Check the target growth rate and the constraints | -| `ValueError` that `model.adapter` is None | `fit_sigma` needs the adapter for `growth_rate`, `p_tot` or `f` | Attach an adapter or pass the three values | - -`getPfbaEnzymes` and `pfba_enzymes` need a full ecModel; a light ecModel uses `solveLP(ecModel, 1)` or `pfba` for parsimonious FBA. - -## See also - -- [Applying kcats](applying-kcats.md), where the enzyme constraints tuned here come from. -- [Basic model operations](basic-model-operations.md), bounds, objectives, FBA and exports. -- [Tuning against experimental data](tuning-against-experimental-data.md), the fit of kcats to measured data with CMA-ES in both toolboxes. -- [GECKO 3 to GECKO 4](../gecko3-to-gecko4.md), the changes of `sigmaFitter` and of the protein pool direction. -- [API reference](../api/index.md), every function in both toolboxes. diff --git a/docs/guide/index.md b/docs/guide/index.md index 83407a1..9e902fa 100644 --- a/docs/guide/index.md +++ b/docs/guide/index.md @@ -1,10 +1,10 @@ # Guide -The guide has one page per job, with MATLAB and Python side by side. [Introduction to ecModels](introduction.md) describes the ecModel concept, [Getting started](getting-started.md) scaffolds a project, and the following pages follow the order of the work from a conventional GEM to an analysed ecModel. Each page is also readable by itself. +The guide shows how to build an enzyme-constrained model (ecModel) from a conventional genome-scale metabolic model (GEM) with GECKO (MATLAB) or geckopy (Python), and how to analyse it. [Build an ecModel](first-ecmodel.md) makes a working ecModel with the default settings. Each step of that page links to a decision page, which describes the options of the step, what each option needs and what it changes in the model. The background pages describe the file formats and the parameters. :::{admonition} GECKO 4 pre-release :class: important -This guide documents GECKO 4, a pre-release. The pages follow the structure of the published GECKO 3.0 Nature Protocols pipeline and describe GECKO 4. [GECKO 3 to GECKO 4](../gecko3-to-gecko4.md) lists the changes against that protocol and describes how to stay on GECKO 3 for a workflow that depends on its behaviour. +GECKO 4 is a pre-release, and the guide describes GECKO 4. For GECKO 3 users, [GECKO 3 to GECKO 4](../gecko3-to-gecko4.md) lists the changes. ::: :::{admonition} MATLAB and Python @@ -14,66 +14,58 @@ Task pages show both languages in tabs: GECKO (MATLAB), built on RAVEN, and geck Text outside a MATLAB or Python tab reads the same in both languages. Where it refers to the enzyme data of a model, `ec.kcat` stands for `ecModel.ec.kcat` in MATLAB and `ec_model.ec.kcat` in Python. The field names are identical in both languages except `rxnEnzMat` (Python: `rxn_enz_mat`) and `ec.geckoLight` (Python: `gecko_light`). ::: -## Workflow - -[![Flowchart of the workflow: project, ecModel, kcat sources, assignment, growth-rate tuning, proteomics, relaxation and analysis](../_static/workflow-overview.png){width=200px .no-scaled-link}](workflow.md) +## Start -The [workflow overview](workflow.md) shows the order of the stages, the kcat sources that run in parallel and the alternative paths, with the functions of each stage in MATLAB and Python. +- [Introduction to ecModels](introduction.md): what an ecModel is, the enzyme constraint and the protein pool. +- [Installation](../installation/index.md): install GECKO or geckopy. -## Pages +## Quick path -Start: +- [Build an ecModel](first-ecmodel.md): create a project, make the ecModel, assign kcats, set the protein pool, check the growth rate and save the model, with the default settings. -- [Introduction to ecModels](introduction.md), what an ecModel is and how GECKO adds enzyme constraints to a conventional GEM. -- [Installation](../installation/index.md), set up GECKO or geckopy. +## Workflow -Steps, in the order of the work: +[![Flowchart of the guide pages: project, ecModel structure, kcat sources, protein pool, kcat tuning, proteomics, relaxation and analysis](../_static/workflow-overview.png){width=200px .no-scaled-link}](workflow.md) -1. [Getting started](getting-started.md), scaffold a project and fill the model adapter. -2. [Building an empty ecModel](building-ec-model.md), expand a conventional GEM into the structure of an ecModel. -3. [Gathering kcats](gathering-kcats.md), collect the kcats of each source. -4. [Applying kcats](applying-kcats.md), write kcats into the enzyme constraints of the ecModel. -5. [Growth-rate tuning](growth-rate-tuning.md), reduce the difference between the growth rate of a new ecModel and that of the organism. -6. [Tuning against experimental data](tuning-against-experimental-data.md), fit and curate the kcats that limit the model. -7. [Proteomics integration](proteomics-integration.md), constrain the concentrations of individual enzymes with proteomics data. -8. [Relaxing overconstrained proteomics](relaxing-constraints.md), loosen proteomics constraints that keep the ecModel from its growth rate. -9. [Simulation and analysis](simulation-and-analysis.md), objectives, flux variability, sampling, flux scanning, mapping to the conventional GEM and context-specific ecModels. -10. [Enzyme usage and bottlenecks](enzyme-usage-and-bottlenecks.md), which enzymes a solution uses and which enzymes limit the objective. +The [workflow overview](workflow.md) shows the order of the pages below, the steps of the quick path, the alternative paths and the functions of each page, in MATLAB and Python. -Background, the reference material that the steps link to: +## Decisions -- [GECKO light and full ecModels](gecko-light.md), the two ecModel layouts and the choice between them. -- [ecModel YAML format](yaml-format.md), the file format that both toolboxes read and write. -- [Files in the data folder](data-folder-files.md), every file that the `data` folder of a project can hold. -- [Model adapter parameters](model-adapter-parameters.md), every parameter of `model_adapter.toml` in both languages. -- [Basic model operations](basic-model-operations.md), bounds, objectives, FBA, flux inspection and export for conventional GEMs and ecModels. +The decision pages are in the order of the work. -Migration, pages for GECKO 3 projects and scripts: +| Page | Choice | +| --- | --- | +| [Choose the ecModel structure](ecmodel-structure.md) | full or light ecModel, matching of genes to UniProt and KEGG, enzyme complexes | +| [Choose the kcat sources](kcat-sources.md) | the kcat assignment preset, and the treatment of isozymes and of reactions without a kcat | +| [Set custom kcats](custom-kcats.md) | curated kcats that take precedence over the kcat sources | +| [Set the protein pool](protein-pool.md) | the values of the total protein content, the enzyme fraction $f$ and the saturation factor $\sigma$ | +| [Choose a kcat tuning](kcat-tuning.md) | standard or advanced kcat tuning, when the ecModel grows too slowly | +| [Tune kcats against experimental data](advanced-tuning.md) | the data and the settings of the advanced kcat tuning | +| [Constrain enzymes with proteomics](proteomics.md) | the integration of measured enzyme concentrations | +| [Relax proteomics constraints](relaxing-proteomics.md) | the method that loosens proteomics constraints that prevent growth | +| [Analyse an ecModel](analysis.md) | objectives, flux variability, sampling and the mapping of fluxes to the conventional GEM | +| [Find engineering targets](engineering-targets.md) | the reactions and genes to change for the production of a compound | +| [Make a context-specific ecModel](context-ecmodel.md) | an ecModel of a tissue, cell line or strain, trimmed from a generic ecModel | +| [Enzyme usage and bottlenecks](enzyme-usage.md) | which enzymes a solution uses and which enzymes limit the objective | -- [GECKO 3 to GECKO 4](../gecko3-to-gecko4.md), the changes of results, errors, functions and names. -- [Project format 4](../gecko4-project-format.md), the project folder, `model_adapter.toml`, kcat source files and the ecModel format. -- [Upgrade a GECKO 3 project](../gecko3-upgrade-project.md), the steps of `upgradeGECKOproject` and `upgradeEcModel`. -- [Kcat assignment](../gecko4-kcat-assignment.md), the kcat sources, stages and presets of `collectKcats` and `assignKcats`. +## Background -## Approximate timing +- [ecModel YAML format](yaml-format.md): the file format that both toolboxes read and write. +- [Files in the data folder](data-folder-files.md): the files that the `data` folder of a project can hold. +- [Model adapter parameters](model-adapter-parameters.md): every parameter of `model_adapter.toml` in both languages. +- [Basic model operations](basic-model-operations.md): bounds, objectives, flux balance analysis, flux inspection and export, for conventional GEMs and ecModels. -The table lists the wall-clock time of a first pass through each step on a genome-scale model such as yeast-GEM. The time depends on the size of the model, the internet connection and the configured solver. +## Migration -| Pages | Approximate time | Notes | -|---|---|---| -| Getting started, Building an empty ecModel | 15 min | | -| Gathering kcats, Applying kcats | 1 h | the prediction of kcats accounts for most of the time | -| Growth-rate tuning, Tuning against experimental data | 15 min | | -| Proteomics integration | 15 min | | -| Relaxing overconstrained proteomics | 15 min | longer with proteomics data of lower quality, which needs more enzymes to be flexibilised | -| Simulation and analysis | 3 h | `ecFVA` at genome scale accounts for most of the time, and it is much shorter with a commercial solver | -| Enzyme usage and bottlenecks | 15 min | | +- [GECKO 3 to GECKO 4](../gecko3-to-gecko4.md): the changes of results, errors, functions and names. +- [Project format 4](../gecko4-project-format.md): the project folder, `model_adapter.toml`, the kcat source files and the ecModel format. +- [Upgrade a GECKO 3 project](../gecko3-upgrade-project.md): the steps of `upgradeGECKOproject` and `upgradeEcModel`. +- [Kcat assignment](../gecko4-kcat-assignment.md): the kcat sources, stages and presets of `collectKcats` and `assignKcats`. ## See also -- [GECKO 3 to GECKO 4](../gecko3-to-gecko4.md), the changes of GECKO 4 against the published protocol. -- [API reference](../api/index.md), the generated documentation of the functions of both toolboxes. -- [MATLAB vs Python](../matlab-vs-python.md), the names of the functions in GECKO and geckopy. +- [API reference](../api/index.md): the generated documentation of the functions of both toolboxes. +- [MATLAB vs Python](../matlab-vs-python.md): the names of the functions in GECKO and geckopy. ```{toctree} :hidden: @@ -85,26 +77,40 @@ introduction ```{toctree} :hidden: -:caption: Guide +:caption: Quick path + +first-ecmodel +``` + +```{toctree} +:hidden: +:caption: Workflow workflow -getting-started -building-ec-model -gathering-kcats -applying-kcats -growth-rate-tuning -tuning-against-experimental-data -proteomics-integration -relaxing-constraints -simulation-and-analysis -enzyme-usage-and-bottlenecks +``` + +```{toctree} +:hidden: +:caption: Decisions + +ecmodel-structure +kcat-sources +custom-kcats +protein-pool +kcat-tuning +advanced-tuning +proteomics +relaxing-proteomics +analysis +engineering-targets +context-ecmodel +enzyme-usage ``` ```{toctree} :hidden: :caption: Background -gecko-light yaml-format data-folder-files model-adapter-parameters diff --git a/docs/guide/introduction.md b/docs/guide/introduction.md index 16b56cb..a1c1568 100644 --- a/docs/guide/introduction.md +++ b/docs/guide/introduction.md @@ -2,100 +2,88 @@ -An enzyme-constrained model (ecModel) is a genome-scale metabolic model (GEM) in which the capacity of each enzyme limits the rate of the reactions that the enzyme catalyses. GECKO builds an ecModel by expanding a conventional GEM with enzyme data: the molecular weight (MW) and the turnover number (kcat) of each enzyme, and a limit on the total protein mass that the enzymes use. Readers of the GECKO 3 protocol find the differences to GECKO 4 in [GECKO 3 to GECKO 4](../gecko3-to-gecko4.md). +An enzyme-constrained model (ecModel) is a genome-scale metabolic model (GEM) in which the amount of each enzyme limits the rate of the reactions that the enzyme catalyses. GECKO and geckopy build an ecModel from a conventional GEM. They add the molecular weight (MW) and the turnover number (kcat) of each enzyme, and a limit on the total mass of enzyme that the model can use. -## Conventional GEMs and enzyme constraints +## Conventional GEMs -A conventional GEM reconstructs the known metabolic reactions of an organism as a system of equations whose variables are reaction rates. The stoichiometric matrix (S-matrix) holds the stoichiometric coefficient of each metabolite in each reaction. Constraint-based methods impose bounds on the reaction rates and search the solution space, the set of all metabolic states that the model allows. Each added constraint shrinks the solution space and can sharpen the predictions of the model. +A conventional GEM describes the metabolic reactions of an organism as a system of linear equations. The stoichiometric matrix (S-matrix) holds the coefficient of each metabolite in each reaction, and the variables are the reaction rates (fluxes). Bounds on the fluxes, for example a measured uptake rate of glucose, limit the set of flux distributions that the model allows (the solution space). Constraint-based methods such as flux balance analysis search this space. -The most common constraints come from condition-dependent measurements, such as nutrient uptake rates. Constraints on the catalytic capacity of enzymes add a different kind of information. Several methods integrate such constraints into conventional models: metabolism and expression models, resource balance analysis, expression and thermodynamics flux, pcModel, flux balance analysis with molecular crowding, MOMENT, GECKO, sMOMENT and ECMpy. GECKO builds an ecModel by expanding the GEM of an organism and can in principle convert models of any size. This documentation addresses genome-scale GEMs. +A conventional GEM has no limit on the rate of a reaction other than its bounds, so a pathway with many reactions costs the model nothing more than a short pathway. An ecModel adds the protein cost of each reaction. -## The GECKO framework +## The enzyme constraint -GECKO converts a conventional GEM with the relation +The rate $v$ of a reaction is at most the turnover number of its enzyme times the amount of the enzyme: -$$ v \le k_{cat} \times e $$ +$$ v \le k_{cat} \cdot e $$ -where $v$ is the rate of an enzymatic reaction, $k_{cat}$ is the turnover number of the enzyme that catalyses it, and $e$ is the concentration of the enzyme. The minimum enzyme usage for a reaction rate is $v / k_{cat}$, and the enzyme mass that the rate requires is $v \times MW / k_{cat}$, where $MW$ is the molecular weight of the enzyme. +The enzyme mass that a flux $v$ needs is therefore at least $v \cdot MW / k_{cat}$. In a full ecModel each enzyme is a pseudometabolite, `prot_`, where `` is the UniProt identifier of the enzyme. Each reaction consumes the pseudometabolites of its enzyme with the coefficient -Each enzyme is a pseudometabolite (`prot_`) that the reactions it catalyses consume. The coefficient of the enzyme in a reaction is +$$ -\frac{n \cdot MW}{3600 \cdot k_{cat}} $$ -$$ -\frac{MW}{k_{cat}} $$ +where $MW$ is the molecular weight of the enzyme in g/mol, $k_{cat}$ is the kcat of the reaction in 1/s, and $n$ is the number of copies of the subunit in the enzyme complex (1 for an enzyme of one subunit). The factor 3600 converts the kcat to 1/h. With fluxes in mmol/gDCW/h, one unit of flux then uses the enzyme mass of the coefficient in mg/gDCW. MW and kcat form one coefficient, which is the protein cost of the reaction. `applyKcatConstraints` (Python: `apply_kcat_constraints`) writes these coefficients from `ec.kcat`, `ec.mw` and `ec.rxnEnzMat`. A reaction with kcat 0 gets no enzyme coefficient and has no protein cost. -with $k_{cat}$ in 1/h and, for a subunit of an enzyme complex, multiplied by the number of copies of the subunit. MW and kcat form one coefficient, which is the protein cost of the reaction: the protein mass that one unit of flux requires. A single coefficient avoids a coefficient so small that it falls below the numerical tolerance of a solver. The S-matrix of an ecModel is larger than that of the conventional GEM, and the model predicts enzyme usage together with reaction rates. `ec.kcat` stores the kcat in 1/s, and `applyKcatConstraints` (Python: `apply_kcat_constraints`) converts it to 1/h when it writes the coefficient. +A reaction that more than one enzyme can catalyse (isozymes) is split into one reaction per isozyme in a full ecModel, so each copy has one enzyme and one kcat. A reversible reaction, except an exchange reaction, is split into a forward and a reverse reaction, so these fluxes are positive. -### Enzyme usage reactions +## The protein pool -A full ecModel has one usage reaction per enzyme, `usage_prot_`, and one protein pool exchange, `prot_pool_exchange`. Both run forward. The pool exchange produces the pseudometabolite `prot_pool`, and its upper bound is the total protein budget that `setProtPoolSize` (Python: `set_prot_pool_size`) calculates from `p_tot`, `f` and `sigma`. Each usage reaction takes protein mass from the pool and produces the enzyme pseudometabolite. The usage of an enzyme is the positive flux of its usage reaction. +Each enzyme pseudometabolite is made by its usage reaction, `usage_prot_`, which takes protein mass from the pseudometabolite `prot_pool`. The flux of the usage reaction is the mass of the enzyme that the solution uses, in mg/gDCW: -Without proteomics data, the usage reactions are limited by the pool alone. A measured concentration of an enzyme sets the upper bound of its usage reaction (see [Proteomics integration](proteomics-integration.md)). For ecModels made with GECKO 3, in which both reactions run in reverse, see [Upgrade a GECKO 3 project](../gecko3-upgrade-project.md). +$$ e_i = \sum_j \frac{n_{ij} \cdot MW_i}{3600 \cdot k_{cat,j}} \, v_j $$ -### Structure of ecModel.ec +The reaction `prot_pool_exchange` produces `prot_pool`, so its flux is the sum of all enzyme usages. `setProtPoolSize` (Python: `set_prot_pool_size`) sets the upper bound of `prot_pool_exchange` to the protein pool, so the enzyme usages are limited by -The enzyme data are in the field `ecModel.ec` (Python: `ec_model.ec`) of the model. +$$ \sum_i e_i \le P_{tot} \cdot f \cdot \sigma \cdot 1000 $$ + +The three parameters are in `model_adapter.toml`: + +| Parameter | Key | Meaning | Default | +| --- | --- | --- | --- | +| $P_{tot}$ | `p_tot` | total protein content of the cell, in g protein/gDCW | 0.5 | +| $f$ | `f` | mass fraction of the total protein that the enzymes of the model account for | 0.5 | +| $\sigma$ | `sigma` | average saturation of the enzymes in vivo, between 0 and 1 | 0.5 | + +The factor 1000 converts g/gDCW to mg/gDCW. `calculateFfactor` (Python: `calculate_f_factor`) calculates $f$ from proteomics or PAXdb data, and `sigmaFitter` (Python: `fit_sigma`) selects the $\sigma$ at which the model reaches a given growth rate. [Set the protein pool](protein-pool.md) describes how to choose the three values. + +A measured concentration adds a second limit to an enzyme: `constrainEnzConcs` (Python: `constrain_enz_concs`) sets the upper bound of its usage reaction to the measured value. The usage reaction still takes its mass from the pool, so the measured enzymes count toward the protein pool as well (see [Constrain enzymes with proteomics](proteomics.md)). + +## Enzyme data in the model + +The enzyme data of an ecModel are in the field `ec` of the model (MATLAB: `ecModel.ec`, Python: `ec_model.ec`). | Field | Content | One entry per | | --- | --- | --- | -| `rxns` | reaction identifiers that correspond to `model.rxns` | reaction (isozymes have separate entries) | -| `kcat` | turnover number in 1/s; 0 marks a reaction without kcat | reaction | -| `source` | where the kcat comes from | reaction | -| `notes` | notes that downstream functions set | reaction | +| `rxns` | reaction identifiers that correspond to the reactions of the model | reaction (each isozyme has its own entry) | +| `kcat` | kcat in 1/s; 0 marks a reaction without a kcat | reaction | +| `source` | the source of the kcat | reaction | +| `notes` | notes that functions add, for example the kcat before tuning | reaction | | `eccodes` | EC numbers | reaction | | `genes` | the genes that code for the enzymes | enzyme | | `enzymes` | UniProt identifiers of the enzymes | enzyme | -| `mw` | molecular weights | enzyme | +| `mw` | molecular weights in g/mol | enzyme | | `sequence` | amino acid sequences | enzyme | -| `concs` | enzyme concentrations from proteomics | enzyme | +| `concs` | measured enzyme concentrations, NaN when not measured | enzyme | | `rxnEnzMat` (Python: `rxn_enz_mat`) | matrix that maps reactions to enzymes, with the number of copies of each subunit | reaction and enzyme | -`makeEcModel` (Python: `make_ec_model`) creates the structure, and the later steps of the workflow fill `kcat`, `source`, `eccodes` and `concs`. [ecModel YAML format](yaml-format.md) describes how a file stores the structure, and [Project format 4](../gecko4-project-format.md) describes the project folder and the ecModel format that `saveEcModel` writes. +`makeEcModel` (Python: `make_ec_model`) creates the structure with the enzyme data from UniProt (and KEGG, for genes that UniProt does not match), and `assignKcats` (Python: `assign_kcats`) fills `kcat` and `source`. [ecModel YAML format](yaml-format.md) describes how a file stores the structure. ## Full and light ecModels -GECKO builds two ecModel layouts from the same conventional GEM. A full ecModel has one pseudometabolite and one usage reaction per enzyme, so it constrains the concentration of each enzyme. A light ecModel writes the protein cost of an enzyme into the stoichiometry of the reaction that the enzyme catalyses and has no pseudometabolite or usage reaction per enzyme. It is smaller and faster to simulate, and it cannot represent individual enzyme concentrations. Proteomics integration, enzyme usage analysis and bottleneck analysis need a full ecModel. [GECKO light and full ecModels](gecko-light.md) describes both layouts and what each gives up. - -## Workflow - -The workflow from a conventional GEM to an analysed ecModel has six steps. Each step has its own page. - -1. Build: `makeEcModel` expands the GEM with enzyme information from UniProt and KEGG, using the parameters of the model adapter (see [Getting started](getting-started.md) and [Building an empty ecModel](building-ec-model.md)). -2. Gather kcats: `kcatsFromBrenda`, `kcatsFromECTree`, `kcatsFromHomologs`, `kcatsFromDLKcat` and `kcatsFromOKP` write one kcat source file per source in `data/kcats` (see [Gathering kcats](gathering-kcats.md)). -3. Assign kcats: `assignKcats` determines one kcat per reaction from the source files, in the order that `model_adapter.toml` sets, and constrains the reactions (see [Kcat assignment](../gecko4-kcat-assignment.md) and [Applying kcats](applying-kcats.md)). -4. Constrain with proteomics: `fillEnzConcs`, `levelComplexConcs` and `constrainEnzConcs` set the usage bounds of measured enzymes, and `flexibilizeEnzConcs` or `relaxProteomicsGreedy` loosen bounds that keep the model from its growth rate (see [Proteomics integration](proteomics-integration.md) and [Relaxing overconstrained proteomics](relaxing-constraints.md)). `levelComplexConcs` gives the subunits of a multi-subunit complex one abundance. The least abundant subunit then does not cap the complex. -5. Tune: `sensitivityTuning` raises single kcats until the model reaches a target growth rate, and the `evotune` functions fit many kcats against measured growth rates and fluxes (see [Growth-rate tuning](growth-rate-tuning.md) and [Tuning against experimental data](tuning-against-experimental-data.md)). -6. Simulate: flux balance analysis, flux variability analysis, sampling, flux scanning, enzyme usage analysis and bottleneck analysis run on the ecModel (see [Simulation and analysis](simulation-and-analysis.md) and [Enzyme usage and bottlenecks](enzyme-usage-and-bottlenecks.md)). - -## Predicted kcat values - -The kcat values determine the performance of an ecModel, and the availability and accuracy of kcat values limit the reconstruction. Measured kcat values come from public databases such as [BRENDA](https://www.brenda-enzymes.org/), and the coverage is incomplete, especially for less-studied organisms. This restricted the reconstruction of ecModels to relatively few species. - -GECKO fills the gaps with predicted kcat values. DLKcat is a deep-learning model that predicts the kcat of an enzyme-substrate pair, and GECKO 4 also reads predictions from the OpenKineticsPredictor service. Predicted values let GECKO assign a kcat to every enzymatic reaction and turn a conventional GEM into an ecModel without experimental kinetic data. [Gathering kcats](gathering-kcats.md) describes the sources, and [Kcat assignment](../gecko4-kcat-assignment.md) the order in which `assignKcats` uses them. - -## Applications - -Converting a conventional GEM to an ecModel reduces the solution space and the flux variability ranges of later simulations. The first GECKO study showed an improved prediction of phenotypes such as overflow metabolism for a yeast GEM (Sanchez et al., 2017; see [References](../references.md)). A conventional GEM predicts these phenotypes when the relevant enzymatic, transport or exchange reactions (such as oxygen uptake) are constrained by hand. - -An ecModel also supports calculations that a conventional GEM does not have: pathway protein costs, flux control coefficients, and enzyme usage and bottleneck analysis (see [Enzyme usage and bottlenecks](enzyme-usage-and-bottlenecks.md)). These quantities guide and help interpret metabolic engineering strategies for the overproduction of a target metabolite. - -A constraint-based method for conventional GEMs also runs on an ecModel, because GECKO adds pseudometabolites and pseudoreactions to the S-matrix and the system stays linear. These methods include flux balance analysis (FBA), flux variability analysis (FVA), flux scanning with enforced objective function (`ecFSEOF`) and context-specific reconstruction methods such as tINIT (see [Simulation and analysis](simulation-and-analysis.md)). - -## Comparison with other methods +A full ecModel has one pseudometabolite and one usage reaction per enzyme, so it can be constrained with the concentration of each enzyme and reports the usage of each enzyme. A light ecModel has no pseudometabolite per enzyme: each reaction takes protein directly from `prot_pool`, with the lowest protein cost of its isozymes. It is smaller, and it cannot use proteomics data or report the usage of an enzyme (see [Choose the ecModel structure](ecmodel-structure.md)). -sMOMENT ([Bekiaris and Klamt, 2020](https://doi.org/10.1186/s12859-019-3329-9)) and ECMpy ([Mao et al., 2022](https://doi.org/10.3390/biom12010065)) are other methods that add enzyme constraints. +## Kcat values -- **sMOMENT** is a simplified version of MOMENT. It gives the same predictions as MOMENT with fewer variables and includes the enzyme constraints in the standard representation of a constraint-based model. The AutoPACMEN toolbox constructs sMOMENT models from stoichiometric models. -- **ECMpy** is a Python-based workflow that adds a constraint on the total enzyme amount to a GEM, takes the protein subunit composition of a reaction into account and calibrates the enzyme kinetic parameters automatically. +The kcat values determine how much protein each reaction costs, so they determine the predictions of an ecModel. Measured kcats come from databases such as [BRENDA](https://www.brenda-enzymes.org/), which covers few enzymes of most organisms. GECKO fills the reactions without a measured kcat with values estimated from other measurements of the same EC number, or predicted from the enzyme sequence and the substrate structure by the methods of the OpenKineticsPredictor service. [Choose the kcat sources](kcat-sources.md) describes the sources and the presets that combine them. -GECKO integrates proteomics data, and its ecModel structures stay close to a conventional GEM. Analyses developed for conventional GEMs apply unchanged. GECKO 4 reads predicted kcat values (see [Predicted kcat values](#predicted-kcat-values)). +Most measured kcats are in vitro values, and predicted kcats are estimates. A new ecModel can therefore grow more slowly than the organism. [Choose a kcat tuning](kcat-tuning.md) describes how to find and correct the kcats that limit growth. ## Limitations -- An ecModel is much larger than its starting GEM, so large-scale simulations take more time. A light ecModel omits the bookkeeping per enzyme (see [GECKO light and full ecModels](gecko-light.md)). The trade-off is between the prediction of individual enzymes, including proteomics integration, and computation time. -- A smaller solution space improves predictions on average, and it can widen the gap to experimental observations where the applied constraints are uncertain. The kcat values, measured or predicted by DLKcat, can be inaccurate, particularly for enzyme complexes, and almost all available kinetic data are in vitro measurements that can differ from in vivo behaviour. GECKO tests an ecModel against known phenotype data and identifies the constraints to revisit (see [Growth-rate tuning](growth-rate-tuning.md) and [Tuning against experimental data](tuning-against-experimental-data.md)). Experimental data remain necessary for a high-quality reconstruction. -- An ecModel does not predict every phenotype, because constraints such as allosteric regulation and post-translational modification are not represented. Integrating these processes could shrink the solution space further. +- An ecModel has more reactions and metabolites than its GEM, so simulations take longer. A light ecModel reduces this. +- The enzyme constraint describes the capacity of each enzyme at a given kcat and saturation. Regulation such as allosteric control and post-translational modification is not part of the model. +- The predictions depend on the kcat values, which are uncertain for most reactions, in particular for enzyme complexes. Measured growth rates, fluxes and proteomics data are needed to test an ecModel. ## See also -- [Installation](../installation/index.md), set up GECKO or geckopy. -- [Getting started](getting-started.md), scaffold a project and load a first model. -- [GECKO 3 to GECKO 4](../gecko3-to-gecko4.md), the differences for readers of the GECKO 3 protocol. +- [Build an ecModel](first-ecmodel.md): the steps from a conventional GEM to an ecModel with the default settings. +- [Installation](../installation/index.md): install GECKO or geckopy. +- [Citations](../references.md): the publications of GECKO and of the methods it uses. diff --git a/docs/guide/kcat-sources.md b/docs/guide/kcat-sources.md new file mode 100644 index 0000000..a1b8cc9 --- /dev/null +++ b/docs/guide/kcat-sources.md @@ -0,0 +1,349 @@ +# Choose the kcat sources + +GECKO selects one kcat per enzymatic reaction from the kcat source files in `data/kcats`, in the order that `[kcat_assignment]` of `model_adapter.toml` defines. The result is a project with a preset (or its own stages) and the source files that the preset reads, and an ecModel with the assigned kcats applied as enzyme constraints. + + +:::::{dropdown} Place in the workflow + +The diagram orders the functions of this page within the [workflow](workflow.md). Boxes with a dashed outline are optional steps, a dotted arrow is an optional path, and a diamond is a choice. Rounded boxes with a dotted outline are the context, other pages or the model to start from, and a click on one opens its page. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Previous: Choose the ecModel structure"]):::ctx + PR["preset or stages
[kcat_assignment] in model_adapter.toml"] + EC["EC numbers
getECfromGEM, then getECfromDatabase"] + SM["metabolite SMILES
findMetSmiles"] + US["updateKcatSources
one file per source in data/kcats"] + CO["collectKcats
compare presets"]:::opt + AS["assignKcats"] + CU(["Set custom kcats"]):::ctx + PV --> PR + PR --> EC --> US + PR --> SM --> US + US --> CO --> AS + US --> AS + CU -.-> AS + NX(["Next: Set the protein pool"]):::ctx + AS --> NX + click PV href "ecmodel-structure.html" + click CU href "custom-kcats.html" + click NX href "protein-pool.html" +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Previous: Choose the ecModel structure"]):::ctx + PR["preset or stages
[kcat_assignment] in model_adapter.toml"] + EC["EC numbers
fill_eccodes_from_gem, then fill_eccodes_from_database"] + SM["metabolite SMILES
find_met_smiles"] + US["update_kcat_sources
one file per source in data/kcats"] + CO["collect_kcats
compare presets"]:::opt + AS["assign_kcats"] + CU(["Set custom kcats"]):::ctx + PV --> PR + PR --> EC --> US + PR --> SM --> US + US --> CO --> AS + US --> AS + CU -.-> AS + NX(["Next: Set the protein pool"]):::ctx + AS --> NX + click PV href "ecmodel-structure.html" + click CU href "custom-kcats.html" + click NX href "protein-pool.html" +``` +::: +:::: +::::: + +## Functions + +| MATLAB | Python | Note | +| --- | --- | --- | +| `getECfromGEM` / `getECfromDatabase` | `fill_eccodes_from_gem` / `fill_eccodes_from_database` | fill the EC numbers of `ec.eccodes` | +| `findMetSmiles` | `find_met_smiles` | find the SMILES of the metabolites, for the prediction methods | +| `updateKcatSources` | `update_kcat_sources` | gather the source files that the preset reads and that are missing or outdated | +| `collectKcats` | `collect_kcats` | determine the kcats without changing the model | +| `assignKcats` | `assign_kcats` | determine the kcats and apply them as enzyme constraints | + +## Setup + +The examples start from an ecModel made with `makeEcModel` (Python: `make_ec_model`), full or light ([Choose the ecModel structure](ecmodel-structure.md)). In MATLAB, the project is the default model adapter (`ModelAdapterManager.setDefault`); in Python, `ec_model.adapter` holds the project. The project folder has `data/uniprot.tsv` and `data/kegg.tsv` from making the ecModel. The OpenKineticsPredictor (OKP) methods and DLKcat without `data/DLKcat.tsv` need an OKP API key ([Prepare the inputs](#prepare-the-inputs)). The BRENDA download, the PubChem queries and OpenKineticsPredictor need internet access. + +## Presets + +`[kcat_assignment] preset` selects one of six presets. Each preset is a list of stages, and a stage gives a kcat to the reactions that the earlier stages left without one. Every preset starts with the custom kcats of the project. In the table, "the OKP methods" means the OKP methods of `[kcat_sources.okp] methods`; the default is KinForm-H alone, and a stage with one method uses its prediction with the bias correction of that method. + +| Preset | Kcat of a reaction, in this order | Needs | +| --- | --- | --- | +| `brenda_ec_tree` (default) | BRENDA median of measurements of the organism on the substrate of the reaction; the EC tree; the OKP methods combined | EC numbers, BRENDA files, SMILES, OKP API key | +| `ec_tree_homologs` | BRENDA value of the organism; the EC tree and the measured homologs combined; the OKP methods combined | as `brenda_ec_tree`, plus DIAMOND | +| `ec_tree_okp` | BRENDA value of the organism; the EC tree, the homologs and the OKP methods combined; the OKP methods combined | as `ec_tree_homologs` | +| `okp` | the OKP methods combined | SMILES, OKP API key | +| `gecko_median` | BRENDA median of matches without EC wildcards (origin 1 to 4); DLKcat; the other BRENDA matches | EC numbers, BRENDA files, SMILES, OKP API key or `data/DLKcat.tsv` | +| `gecko_classic` | the order of the GECKO 3 tutorial: as `gecko_median` with the BRENDA maximum, and with the matches from specific activities (origin 5 and 6) before DLKcat | as `gecko_median` | + +A "combined" value is the mean of the values of several sources in log10 units, each corrected for the bias of its source and weighted by the precision of the source for that reaction. The origin levels of BRENDA matches are explained in [BRENDA origin and wildcard levels](#brenda-origin-and-wildcard-levels). [Kcat assignment](../gecko4-kcat-assignment.md) describes each source and the exact stages of each preset. + +A project can list its own stages in `[[kcat_assignment.stage]]` tables in place of a preset. [Kcat assignment](../gecko4-kcat-assignment.md#presets) gives the TOML syntax and the stage options. + +### Preset to start with + +Against measured kcats hidden from the search, the EC tree was as close as or closer than the BRENDA median at origins 1 to 4 and with EC wildcards, and the EC tree combined with eight OKP methods (CataPro, DLKcat, EITLEM, KinForm-H, KinForm-L, OmniESI, OmniESI-O2DENet and UniKP) was slightly closer than the EC tree; in ecModels, however, values combined with predictions set high-turnover enzymes such as ATP synthase too low ([Kcat assignment](../gecko4-kcat-assignment.md#presets)). No other comparison of the presets has been measured, and KinForm-H alone, the default OKP method, was not compared separately. `brenda_ec_tree`, the default, uses predictions for the reactions without a value from measurements, and is the preset to start with. + +## BRENDA origin and wildcard levels + +A BRENDA match records how specific it is in two numbers, which `ec.source` shows as `brenda (wc=, origin=)` and which the presets use to rank BRENDA values. + +| Origin | BRENDA value | +| --- | --- | +| 1 | kcat of the organism, on the substrate of the reaction | +| 2 | kcat of the closest related organism (by KEGG phylogeny), on the substrate | +| 3 | kcat of the organism, on any substrate | +| 4 | kcat of the closest related organism, on any substrate | +| 5 | specific activity of the organism, converted with the molecular weight | +| 6 | specific activity of the closest related organism | + +| Wildcard level | EC number used | +| --- | --- | +| 0 | `w.x.y.z` | +| 1 | `w.x.y.-` | +| 2 | `w.x.-.-` | +| 3 | `w.-.-.-` | +| 4 | `-.-.-.-` (every EC number in BRENDA) | + +Matches at wildcard level 4 are kept in `brenda.tsv`, and no stage uses them: the `wildcard_limit` of a stage is at most 3. + +For each EC number, the origins are tried in the order 1, 2, 3, 5, 4, 6, so a specific activity of the organism is used before a kcat of another organism on another substrate. When no origin gives a value, the rightmost level of the EC number that is not yet a wildcard is replaced by `-`, and the origins are tried again. With a wildcard, origins 1 and 2 are skipped, because the last levels of an EC number define the substrate specificity. When a reaction has several EC numbers, the match with the fewest wildcards, then the earliest tried origin, then the highest kcat is used. + +## Prepare the inputs + +The sources read different parts of the ecModel and of the project: + +| Input | Read by | How to get it | +| --- | --- | --- | +| EC numbers in `ec.eccodes` | BRENDA, EC tree | `getECfromGEM` (from the GEM), `getECfromDatabase` (from UniProt and KEGG) | +| SMILES of the metabolites | OKP methods, DLKcat | `findMetSmiles`; substrates without a SMILES are left out of a prediction | +| enzyme sequences in `ec.sequence` | OKP methods, DLKcat, homologs | `makeEcModel` fills them from UniProt ([Choose the ecModel structure](ecmodel-structure.md)) | +| BRENDA files | BRENDA, EC tree, homologs, combined stages | `fetchBrendaAsset` downloads the release `brenda-2026.1-1` on first use. A project folder `data/brenda` takes precedence. MATLAB needs RAVEN 3 for the download | +| DIAMOND | homologs | the DIAMOND executable of RAVEN (MATLAB) or raven-toolbox (Python) | +| OpenKineticsPredictor API key | OKP methods, DLKcat without `data/DLKcat.tsv` | generate a key at [predictor.openkinetics.org](https://predictor.openkinetics.org/) | + +GECKO looks for the API key in the environment variable `OKP_API_KEY` and then in the file `data/okpApiKey.txt` of the project. The key is never read from `model_adapter.toml`. Do not commit `data/okpApiKey.txt` to a public repository. + +`getECfromDatabase` replaces the EC numbers of the reactions it is given, so the reactions that still have no EC number after `getECfromGEM` are selected. In Python, KEGG fills the reactions without a UniProt EC number when `kegg_db` is given: + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +ecModel = getECfromGEM(ecModel); +noEC = cellfun(@isempty, ecModel.ec.eccodes); +ecModel = getECfromDatabase(ecModel, noEC); +[ecModel, noSMILES] = findMetSmiles(ecModel); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import fill_eccodes_from_database, fill_eccodes_from_gem, find_met_smiles, load_uniprot_tsv +from geckopy.databases import load_kegg_tsv + +data = ec_model.adapter.params.path / "data" +fill_eccodes_from_gem(ec_model) +no_ec = [rxn for rxn, ec in zip(ec_model.ec.rxns, ec_model.ec.eccodes) if not ec] +uniprot_db = load_uniprot_tsv(data / "uniprot.tsv") +kegg_db = load_kegg_tsv(data / "kegg.tsv") +fill_eccodes_from_database(ec_model, uniprot_db, kegg_db=kegg_db, ec_rxns=no_ec) +find_met_smiles(ec_model) +``` +::: +:::: + +When the EC numbers of the GEM are not reliable, skip `getECfromGEM` and fill all reactions from the databases: `ecModel = getECfromDatabase(ecModel);` (Python: `fill_eccodes_from_database(ec_model, uniprot_db, kegg_db=kegg_db)`). + +`findMetSmiles` queries PubChem by metabolite name and keeps the results in `data/smilesDB.tsv`. A metabolite name that PubChem does not find gets no SMILES; correct the name in the model or add the SMILES to `data/smilesDB.tsv`. + +## Gather the source files + +`updateKcatSources` lists the source files that the preset reads and gathers each file that is missing or outdated. A file is outdated when the settings in its header differ from the settings of the preset and `[kcat_sources.]`, or when it was made from another BRENDA release than the BRENDA files in use. `custom.tsv` is never gathered or changed. The option `dryRun` reports the status of each file without gathering. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +report = updateKcatSources(ecModel, 'dryRun', true); +report = updateKcatSources(ecModel); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import update_kcat_sources + +report = update_kcat_sources(ec_model, dry_run=True) +report = update_kcat_sources(ec_model) +``` +::: +:::: + +`report` has one row per source file, with the columns `source`, `status` and `note`. The status is `current`, `gathered`, `pending`, `outdated`, `missing`, `failed` or `skipped`. OpenKineticsPredictor is a web service with a job queue: the first call submits one job per method and reports the source as `pending`, and a later call fetches the result and writes the file. Call `updateKcatSources` again until no source is `pending`. A source that is `failed` has the reason in `note`, for example a missing API key. + +`[kcat_sources.]` of `model_adapter.toml` holds the settings with which `updateKcatSources` gathers a source file. A preset sets `aggregation` of `brenda` and `k` of `ec_tree`. A `[kcat_sources.]` table in `model_adapter.toml` overrides the keys that it sets and keeps the other settings of the preset. A source file gathered before a change of these settings is then reported as `outdated`. + +| Table | Key | Default | +| --- | --- | --- | +| `[kcat_sources.brenda]` | `aggregation`: `max` or `median` of the BRENDA values per EC number, substrate and organism | set by the preset (`max` for `gecko_classic`, `median` for the others); `kcat_aggregate_brenda` with stages in place of a preset | +| `[kcat_sources.brenda]` | `force_wildcard_level`: number of EC levels replaced by a wildcard in every EC number | 0 | +| `[kcat_sources.ec_tree]` | `k`: shrinkage constant | 5 | +| `[kcat_sources.homologs]` | `k`: number of homologs | 10 | +| `[kcat_sources.homologs]` | `min_identity`: minimum sequence identity, as a fraction | 0.4 | +| `[kcat_sources.homologs]` | `coverage`: minimum coverage of both sequences, as a fraction | 0.8 | +| `[kcat_sources.okp_]` | `method`: the OpenKineticsPredictor method of the file | the method whose name gives the file name | +| `[kcat_sources.okp]` | `methods`: the methods that `okp` stands for in a stage | `["KinForm-H"]` | + +The source `dlkcat` (presets `gecko_median` and `gecko_classic`) reads `data/DLKcat.tsv` when the file exists. Without it, `dlkcat` is predicted with the DLKcat method of OpenKineticsPredictor. + +:::{admonition} DLKcat in Docker +:class: note + +DLKcat can also run on the local computer, in the Docker image `ghcr.io/sysbiochalmers/dlkcat-gecko:0.1`. This is the older method and needs Docker installed and running. `kcatsFromDLKcat(ecModel, 'via', 'docker')` (Python: `kcats_from_dlkcat(ec_model, via="docker")`) runs it and writes `data/kcats/dlkcat.tsv`; `updateKcatSources` then finds the file current. +::: + +## Compare presets before applying + +`collectKcats` determines the kcats as `assignKcats` does and does not change the model. It writes the result to `output/_kcatDecisions.tsv` unless `writeReport` (Python: `write_report`) is false. The file has one row per reaction of `ec.rxns` with these columns; the table that MATLAB returns has the names in parentheses: + +| Column | Content | +| --- | --- | +| `ec_rxn` (`ecRxn`) | the reaction of `ec.rxns` | +| `stage` | the number of the stage that gave the kcat (`1` for a kcat from `custom.tsv`), `isozymes` or `standard`; empty when the reaction has no kcat | +| `source` | the source as written to `ec.source` | +| `kcat` | the kcat in s^-1 | +| `candidates` | the number of values that the selected source had for the reaction, or the number of sources combined | +| `spread_log10` (`spreadLog10`) | the range of these values in log10 units | +| `sd_log10` (`sdLog10`) | the uncertainty of a combined value in log10 units | + +To compare two presets, change the preset of a copy of the adapter, gather its sources, and collect the decisions of both: + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +adapter = ModelAdapterManager.getDefault(); +other = adapter; +other.params.kcat_assignment.preset = 'ec_tree_okp'; +updateKcatSources(ecModel, 'modelAdapter', other); +a = collectKcats(ecModel, 'writeReport', false); +b = collectKcats(ecModel, 'modelAdapter', other, 'writeReport', false); +groupcounts(a, 'stage') +groupcounts(b, 'stage') +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import collect_kcats, update_kcat_sources + +settings = ec_model.adapter.params.kcat_assignment +a, _, _ = collect_kcats(ec_model, write_report=False) +default_preset = settings.preset +settings.preset = "ec_tree_okp" +update_kcat_sources(ec_model) +b, _, _ = collect_kcats(ec_model, write_report=False) +settings.preset = default_preset +print(a["stage"].value_counts()) +print(b["stage"].value_counts()) +``` +::: +:::: + +The MATLAB adapter is a value object, so `other` is a copy and the default adapter keeps its preset. In Python, the preset is changed on the adapter of the model and set back afterwards. To keep the other preset, write it in `model_adapter.toml`. + +Presets with other source settings share the source files: `gecko_classic` gathers `brenda.tsv` with the BRENDA maximum and the other presets with the median. Gathering the sources of one preset then makes the file outdated for the other. Compare such presets in a copy of the project folder, or run `updateKcatSources` with the preset of the project again before `assignKcats`. + +## Assign the kcats + +`assignKcats` determines the kcats, writes `ec.kcat` and `ec.source` (see [Provenance of a kcat](../gecko4-kcat-assignment.md#provenance-of-a-kcat)), applies the enzyme constraints and writes the decisions file. The kcats that the model had before are replaced. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +[ecModel, decisions, report] = assignKcats(ecModel); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import assign_kcats + +decisions, report = assign_kcats(ec_model) # changes ec_model in place +``` +::: +:::: + +`report` lists the source files that a stage names and that do not exist (`missing`), and per source file the rows that were not applied. A missing source file is left out with a message, and the other stages still give kcats. + +After the stages, two settings of `[kcat_assignment]` fill the reactions that are still without a kcat: + +- `isozymes` (default true): in a full ecModel, a reaction without a kcat gets the mean kcat of its isozymes. An isozyme reaction with a kcat of 0 has no enzyme cost after `applyKcatConstraints`, so an optimisation uses it in place of the isozymes that have a kcat. A light ecModel uses the most efficient isozyme and skips this step. +- `standard` (default true): the remaining reactions get the standard kcat, which is the median kcat of the subsystem of the reaction when the subsystem has at least 10 reactions with a kcat, and the median of all kcats otherwise. Reactions without a gene association get the pseudo-enzyme `prot_standard`, with the median molecular weight of the proteins in `data/uniprot.tsv`. Exchange, transport, spontaneous and pseudo-reactions get no standard kcat; list other reactions to leave out in `data/pseudoRxns.tsv` ([Files in the data folder](data-folder-files.md)). + +## Add custom kcats + +`data/kcats/custom.tsv` holds the curated kcats of the project. Every preset reads it in its first stage, so a custom kcat is used before all other sources. [Set custom kcats](custom-kcats.md) describes the file, the conversion of `data/customKcats.tsv` of a GECKO 3 project, and how to change a kcat by hand. + +## Troubleshooting + +| Problem | Possible reason | Solution | +| --- | --- | --- | +| OKP sources stay `pending` | the OpenKineticsPredictor jobs are still running | Call `updateKcatSources` again later | +| OKP sources are `failed` with `No OpenKineticsPredictor API key found` | no key in `OKP_API_KEY` or in `data/okpApiKey.txt` | Generate a key and store it in one of the two places | +| `No kcat source file ; it is left out` | a stage names a source that has no file in `data/kcats` | Run `updateKcatSources`, or remove the source from the stages | +| BRENDA and the EC tree give kcats for few reactions | many reactions have no EC number | Fill `ec.eccodes` (section [Prepare the inputs](#prepare-the-inputs)) | +| Few reactions get a predicted kcat | many metabolites have no SMILES | Read `noSMILES` of `findMetSmiles` (Python: the log of `find_met_smiles`, with the number at INFO level and the names at DEBUG level) and correct the metabolite names or `data/smilesDB.tsv` | +| Error `kcatAssignment:config` (Python: a validation error of the adapter) | an unknown preset, a preset and stages together, or a stage with a wrong option | Correct `[kcat_assignment]` | +| Kcats set by hand are gone | `assignKcats` replaced them | Write them to `data/kcats/custom.tsv` ([Set custom kcats](custom-kcats.md)) | + +## See also + +- [Kcat assignment](../gecko4-kcat-assignment.md): the sources, the stages and the presets in detail. +- [Project format 4](../gecko4-project-format.md): the layout of `data/kcats` and of the kcat source files. +- [Model adapter parameters](model-adapter-parameters.md): `[kcat_assignment]` and `[kcat_sources]`. +- [Set custom kcats](custom-kcats.md): the curated kcats of the project. +- [Set the protein pool](protein-pool.md): the next step after the kcats. +- [Choose a kcat tuning](kcat-tuning.md): when the model grows too slowly with the assigned kcats. diff --git a/docs/guide/kcat-tuning.md b/docs/guide/kcat-tuning.md new file mode 100644 index 0000000..875f924 --- /dev/null +++ b/docs/guide/kcat-tuning.md @@ -0,0 +1,276 @@ +# Choose a kcat tuning + +After the kcats are assigned and the protein pool is set, an ecModel can grow more slowly than the organism. A kcat tuning raises the kcats that limit growth until the ecModel reaches the maximum growth rate of the organism. The result is a tuned ecModel, a list of the changed kcats, and curated values for the changed kcats that the literature supports. + + +:::::{dropdown} Place in the workflow + +The diagram orders the functions of this page within the [workflow](workflow.md). Boxes with a dashed outline are optional steps, a dotted arrow is an optional path, and a diamond is a choice. Rounded boxes with a dotted outline are the context, other pages or the model to start from, and a click on one opens its page. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Previous: Set the protein pool"]):::ctx + DA{"Measured data"} + ST["sensitivityTuning
no change when growth reaches the target"] + SF["sigmaFitter"] + CK["check tuned kcats against their source
loadKcatSource
setKcatForReactions, applyKcatConstraints"]:::opt + KC(["Set custom kcats
to keep a curated kcat"]):::ctx + AT(["Tune kcats against experimental data"]):::ctx + PV --> DA + DA -->|one maximum growth rate| ST --> SF + ST -.-> CK -.-> KC + DA -->|measured fluxes or several growth rates| AT + NP(["Next: Constrain enzymes with proteomics (optional)"]):::ctx + NA(["Next: Analyse an ecModel"]):::ctx + SF --> NP + SF --> NA + click PV href "protein-pool.html" + click KC href "custom-kcats.html" + click AT href "advanced-tuning.html" + click NP href "proteomics.html" + click NA href "analysis.html" +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Previous: Set the protein pool"]):::ctx + DA{"Measured data"} + ST["sensitivity_tuning
no change when growth reaches the target"] + SF["fit_sigma"] + CK["check tuned kcats against their source
load_kcat_source
set_kcat_for_reactions, apply_kcat_constraints"]:::opt + KC(["Set custom kcats
to keep a curated kcat"]):::ctx + AT(["Tune kcats against experimental data"]):::ctx + PV --> DA + DA -->|one maximum growth rate| ST --> SF + ST -.-> CK -.-> KC + DA -->|measured fluxes or several growth rates| AT + NP(["Next: Constrain enzymes with proteomics (optional)"]):::ctx + NA(["Next: Analyse an ecModel"]):::ctx + SF --> NP + SF --> NA + click PV href "protein-pool.html" + click KC href "custom-kcats.html" + click AT href "advanced-tuning.html" + click NP href "proteomics.html" + click NA href "analysis.html" +``` +::: +:::: +::::: + +## Functions + +| MATLAB | Python | Note | +| --- | --- | --- | +| `sensitivityTuning` | `sensitivity_tuning` | standard kcat tuning: raises the kcat that limits growth most, one at a time, until the model reaches a growth rate | +| `cmaesKcatTuning` | `cmaes_kcat_tuning` | advanced kcat tuning: fits many kcats at once to measured growth rates and exchange fluxes, see [Tune kcats against experimental data](advanced-tuning.md) | +| `loadKcatSource` | `load_kcat_source` | reads the rows of a kcat source file in `data/kcats` for the reactions of an ecModel | +| `setKcatForReactions` | `set_kcat_for_reactions` | sets `ec.kcat` of reactions by hand | +| `applyKcatConstraints` | `apply_kcat_constraints` | writes `ec.kcat` into the enzyme constraints of the reactions | + +## Setup + +The examples need an ecModel with assigned kcats (`assignKcats`, see [Choose the kcat sources](kcat-sources.md)) and a set protein pool (`setProtPoolSize`, see [Set the protein pool](protein-pool.md)), in the variable `ecModel` (MATLAB) or `ec_model` (Python). The model adapter of the project defines `bio_rxn` and `gR_exp` (Python `gr_exp`); in MATLAB it is the default adapter (`ModelAdapterManager.setDefault`), and in Python it is attached to the model by `load_ec_model(adapter=...)`. The uptake of the carbon source is unconstrained. + +## Signs that the model needs a kcat tuning + +Maximise growth with flux balance analysis (FBA), with the uptake of the carbon source unconstrained (see [Basic model operations](basic-model-operations.md)). If the growth rate is far below the maximum growth rate of the organism (`gR_exp` in the model adapter, Python `gr_exp`), the enzyme constraints are too tight. Before kcats are changed, check the two other causes of a low growth rate: + +- The conventional GEM, with the same exchange bounds, reaches the maximum growth rate. If it does not, the stoichiometry or the bounds of the GEM limit growth, and the GEM is corrected first. +- Ptot, f and sigma of the protein pool are realistic for the organism (see [Set the protein pool](protein-pool.md)). + +If both hold, some kcats are too low for the enzymes that carry most of the flux, and a kcat tuning raises them. + +## Standard or advanced kcat tuning + +| Property | Standard: `sensitivityTuning` | Advanced: `cmaesKcatTuning` | +| --- | --- | --- | +| Data it needs | one maximum growth rate (`gR_exp` in the model adapter, or the option `desiredGrowthRate`) | maximum growth rates (`evotuneMaxGrowth.tsv`), exchange fluxes (`evotuneFluxData.tsv`), or both, measured in one or more conditions | +| What it changes | `ec.kcat` of the reactions it selects, and their enzyme constraints | `ec.kcat` of the kcats with the largest effect on the fit to the data, and their enzyme constraints | +| Which kcats | one reaction per iteration: the reaction with the largest use of the most used enzyme | the top kcats of a screen, up to 90 % of the weighted effect on the fit (option `targetImpactShare`) | +| How much | the kcat is multiplied by `foldChange` (default 10) per iteration; a kcat that is selected again is multiplied again | any value between `min(0.01, k0/100)` and `max(10000, 100*k0)` per s, where `k0` is the kcat before tuning | +| When it stops | the growth rate reaches the target, or growth stops rising | the best root-mean-square error (RMSE) reaches `rmse_threshold` (default 0.2), or after `max_generations` (default 150) generations | +| Cost in time | one FBA per iteration | one FBA per data row for each kcat vector that is scored: two per kcat in the screen, plus `popsize` per generation in the search; runs in parallel with `nProc` | +| Settings | the options of the function | the `[evotune]` table of `model_adapter.toml`, plus `targetImpactShare`, `popsize`, `seed` and `nProc` | + +In a full ecModel, each iteration of the standard tuning selects the enzyme whose usage reaction carries the most flux and raises the kcat of the reaction that uses most of that enzyme; in a light ecModel, it raises the kcat of the reaction that draws most from the protein pool. The standard tuning also stops when an iteration raises growth by 1e-9 per h or less, when the model becomes infeasible, or when no enzyme usage carries flux; Python also stops after `max_iterations` (default 1000). + +The advanced tuning weights the effect of each kcat on the fit by the `sigma0_log` of its source, so a kcat from a less trusted source has a larger weight. Python also stops on a convergence criterion of the `cma` package. The `[evotune]` settings set the trust in each kcat source, the weight of growth against flux data, the penalty for moving a kcat, isozyme tying and the stopping rules. + +The two tunings are alternatives. When the maximum growth rate of the organism is the sole measured value, use the standard tuning. With measured growth rates on several carbon sources or measured exchange fluxes, the advanced tuning uses those data, and the trust in each kcat source limits how far each kcat moves. + +## Run the standard tuning + +`sensitivityTuning` reads the target growth rate and the biomass reaction from the model adapter, and sets growth as the objective. The protein pool must already be set (`setProtPoolSize`, see [Set the protein pool](protein-pool.md)), and the uptake of the carbon source unconstrained. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +[ecModel, tunedKcats] = sensitivityTuning(ecModel); +struct2table(tunedKcats) +``` + +The options are name-value pairs (`sensitivityTuning(ecModel, 'foldChange', 5)`) or positional in the order `desiredGrowthRate, modelAdapter, foldChange, protToIgnore, verbose`. +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +import logging + +import pandas as pd +from geckopy import sensitivity_tuning + +logging.basicConfig(level=logging.INFO) # shows the growth rate of each iteration + +tuned = sensitivity_tuning(ec_model) +pd.DataFrame({"rxn": tuned.rxns, "enzymes": tuned.enzymes, + "old_kcat": tuned.old_kcat, "new_kcat": tuned.new_kcat, + "source": tuned.source}) +``` + +`sensitivity_tuning` changes `ec_model` in place and returns a `TunedKcatsResult`. It reads the target growth rate from `ec_model.adapter`. The options are keyword arguments. +::: +:::: + +| MATLAB | Python | Default | Meaning | +| --- | --- | --- | --- | +| `desiredGrowthRate` | `desired_growth_rate` | `gR_exp` of the model adapter (Python `gr_exp`) | growth rate at which the tuning stops | +| `foldChange` | `fold_change` | `10` | factor by which a selected kcat is multiplied | +| `protToIgnore` | `prot_to_ignore` | none | UniProt IDs of enzymes that are never selected | +| `verbose` | `verbose` | `true` | prints (Python: logs at `INFO`) the growth rate of each iteration | +| `modelAdapter` | | default model adapter | model adapter with the parameters; Python reads `ec_model.adapter` | +| | `max_iterations` | `1000` | maximum number of iterations | +| | `bio_rxn` | `bio_rxn` of the model adapter | biomass reaction that is set as the objective | + +The output (`tunedKcats`, Python `TunedKcatsResult`) has one entry per tuned reaction: `rxns`, `rxnNames` (Python `rxn_names`), `enzymes` (joined by `;`), `oldKcat` and `newKcat` (Python `old_kcat`, `new_kcat`), and `source`, the kcat source before tuning. `ec.source` of a tuned reaction keeps its origin and detail and gets the tuning status, for example `brenda (wc=0, origin=1) | tuned=sensitivity (prior=0.05)`, where the prior is the kcat before the first tuning (see [Provenance of a kcat](../gecko4-kcat-assignment.md#provenance-of-a-kcat)). A reaction that an earlier run already tuned keeps the prior of that run. In a light ecModel, every ec row of the tuned reaction gets the status. `ec.notes` does not change. In Python, `tuned.corrections()` returns the changes as a list of `Correction` rows, ordered by fold change. + +After the standard tuning, `sigmaFitter` (Python `fit_sigma`) fits sigma so that the model reaches the target growth rate (see [Set the protein pool](protein-pool.md#fit-sigma-to-a-growth-rate)). + +## Check a tuned kcat against its source + +A tuning raises the kcats that limit growth in the model. It does not check whether the value before tuning was wrong. For each tuned kcat, look up where the value before tuning came from, and search the literature for a measured value. + +`ec.source` of a tuned reaction gives the origin of the kcat and the kcat before tuning (`prior`). When the source is a file in `data/kcats` (for example `brenda` or `ec_tree`), `loadKcatSource` returns the rows of that file for the reaction; the `detail` column of a BRENDA row holds the EC wildcard level and the origin of the match (see [BRENDA origin and wildcard levels](kcat-sources.md#brenda-origin-and-wildcard-levels)). `output/_kcatDecisions.tsv` lists the stage of the assignment that set each kcat (see [Choose the kcat sources](kcat-sources.md)). + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +ecModel.ec.source(strcmp(ecModel.ec.rxns, 'r_0079')) +kcatList = loadKcatSource(ecModel, 'brenda'); +idx = strcmp(kcatList.rxns, 'r_0079'); +table(kcatList.kcats(idx), kcatList.eccodes(idx), kcatList.detail(idx)) +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import load_kcat_source + +print(ec_model.ec.source[ec_model.ec.rxns.index("r_0079")]) +kcat_list, _ = load_kcat_source(ec_model, "brenda") +kcat_list[kcat_list["rxn_id"] == "r_0079"] +``` +::: +:::: + +### Example: a kcat from a specific activity + +The GECKO tutorial `full_ecModel` builds an ecModel of yeast-GEM. With the preset `gecko_classic`, the order of the GECKO 3 tutorial, `sensitivityTuning` raised the kcat of `r_0079` (5'-phosphoribosylformyl glycinamidine synthetase, enzyme P38972). The kcat before tuning, 0.05 per s, came from a BRENDA match with wildcard level 0 and origin 4 (closest organism, any substrate; see [BRENDA origin and wildcard levels](kcat-sources.md#brenda-origin-and-wildcard-levels)) for EC 6.3.5.3. + +[BRENDA](https://www.brenda-enzymes.org/enzyme.php?ecno=6.3.5.3) lists this kcat for *Escherichia coli* with NH4+ as the substrate. The reaction in the model uses glutamine. The paper that reports the value ([Schendel et al. 1989](https://doi.org/10.1021/bi00432a017)) states in its abstract that "NH3 can replace glutamine as a nitrogen donor with a Km = 1 M and a turnover of 3 min-1 (2% glutamine turnover)", so 0.05 per s is the kcat with ammonia as the nitrogen donor. The paper also reports a specific activity, which BRENDA lists; the tutorial uses 2.15 Β΅mol/min/mg protein. A specific activity gives a kcat when it is multiplied by the molecular weight of the enzyme: + +- 2.15 Β΅mol/min/mg is 2.15 mmol/min/g; +- divided by 1000, it is 0.00215 mol/min/g; +- divided by 60, it is 3.583e-5 mol/s/g; +- multiplied by the molecular weight of P38972, 148905 g/mol (`ec.mw`), it is 5.336 per s. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +sa = 2.15; % umol/min/mg protein +mw = ecModel.ec.mw(strcmp(ecModel.ec.enzymes, 'P38972')); % g/mol +kcat = sa / 1000 / 60 * mw % 1/s +ecModel = setKcatForReactions(ecModel, 'r_0079', kcat); +ecModel = applyKcatConstraints(ecModel); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import set_kcat_for_reactions + +sa = 2.15 # umol/min/mg protein +mw = ec_model.ec.mw[ec_model.ec.enzymes.index("P38972")] # g/mol +kcat = sa / 1000 / 60 * mw # 1/s +set_kcat_for_reactions(ec_model, ["r_0079"], kcat, apply=True) +``` +::: +:::: + +The tutorial keeps this value as the row of `r_0079` in `tutorials/full_ecModel/data/kcats/custom.tsv`, so that `assignKcats` applies it (see [Keep a curated kcat](#keep-a-curated-kcat)). + +When a measured value supports the kcat before tuning and contradicts the tuned one, the network of the conventional GEM may lack a reaction or a pathway; correct the GEM first. + +## Keep a curated kcat + +`assignKcats` replaces every kcat of the model with the values selected from the files in `data/kcats`. A value set with `setKcatForReactions` is therefore lost at the next `assignKcats`. To keep it, add it to `data/kcats/custom.tsv`, which every preset reads first (see [Set custom kcats](custom-kcats.md)). A row for `r_0079`, with the columns `rxn`, `enzymes`, `kcat`, `source` and `note`: + +```text +rxn enzymes kcat source note +r_0079 P38972 5.3357625 custom specific activity 2.15 umol/min/mg, MW 148905 g/mol +``` + +## Troubleshooting + +| Problem | Possible reason | Solution | +| --- | --- | --- | +| Error that the FBA of the input model gives no valid result (MATLAB), or `ValueError` that the FBA of the input model is not optimal (Python) | The protein pool or the exchange bounds make the model infeasible | Check the exchange bounds and the protein pool (`setProtPoolSize`) | +| Warning that growth does not increase from increased kcats | An exchange bound or the protein pool limits growth | Check the bounds of the uptake reactions; check Ptot, f and sigma | +| Warning that no protein usage reaction carries flux, or that no reaction draws on the protein pool | The protein pool does not limit growth | Check the nutrient bounds and the stoichiometry of the GEM | +| Many kcats are tuned, or one kcat is raised several times | The target growth rate is too high, a nutrient uptake is constrained, or the GEM cannot reach the growth rate | Check `gR_exp`, the uptake bounds and the growth rate of the conventional GEM | +| Python `sensitivity_tuning` stops at `max_iterations` | Raising kcats alone does not reach the target growth rate | Check the target growth rate and the bounds | +| `ValueError` that `prot_to_ignore` has no usage reaction for a protein (Python, full ecModel) | The model has no usage reaction for this UniProt ID | Correct the ID | +| Error `sigmaFitter:growthRate` (Python: `ValueError`) | The target growth rate is not above 0 | Set `gR_exp` (Python `gr_exp`) in the model adapter, or give the growth rate | +| A curated kcat is gone after `assignKcats` | The value was set with `setKcatForReactions`, and `assignKcats` replaced it with the value from the kcat source files | Add the value to `custom.tsv` | + +## See also + +- [Tune kcats against experimental data](advanced-tuning.md), the advanced kcat tuning in detail. +- [Set the protein pool](protein-pool.md), Ptot, f, sigma and `sigmaFitter`. +- [Choose the kcat sources](kcat-sources.md) and [Set custom kcats](custom-kcats.md), where the kcats before tuning come from. +- [Basic model operations](basic-model-operations.md), bounds, objectives and FBA. +- [API reference](../api/index.md), every function in both toolboxes. diff --git a/docs/guide/model-adapter-parameters.md b/docs/guide/model-adapter-parameters.md index 3dd6c9c..879fe4a 100644 --- a/docs/guide/model-adapter-parameters.md +++ b/docs/guide/model-adapter-parameters.md @@ -69,7 +69,7 @@ name_contains = "spontaneous" | `sigma` | `sigma` | `0.5` | Average enzyme saturation factor | | `p_tot` | `Ptot` | `0.5` | Total protein content in the cell [g protein/gDW] | | `f` | `f` | `0.5` | Fraction of enzymes in the ecModel relative to all proteins (g enzyme/g protein). `calculateFfactor` (Python: `calculate_f_factor`) computes it from a proteomics data set | -| `gr_exp` | `gR_exp` | `0.41` | Reference (experimentally observed) maximum growth rate in h^-1, the default tuning target in [Growth-rate tuning](growth-rate-tuning.md#sensitivity-tuning-of-kcat-values) | +| `gr_exp` | `gR_exp` | `0.41` | Reference (experimentally observed) maximum growth rate in h^-1, the default tuning target in [Choose a kcat tuning](kcat-tuning.md#run-the-standard-tuning) | | `c_source` | `c_source` | `""` | Identifier of the exchange reaction of glucose or of another preferred carbon source | | `bio_rxn` | `bioRxn` | `""` | Identifier of the biomass or growth pseudoreaction, whose flux is the growth rate in h^-1 | | `enzyme_comp` | `enzyme_comp` | `"cytoplasm"` | Name of the compartment of the added protein pseudometabolites; must match one of the compartment names of the ecModel | @@ -88,10 +88,10 @@ GECKO accepts the two geckopy keys in the file and does not use them. `getKcatAc | TOML key | MATLAB parameter | Default | Description | |---|---|---|---| -| `kegg.id` | `kegg.ID` | `"sce"` | Organism code in the KEGG database, from the [KEGG organism list](https://www.genome.jp/kegg/catalog/org_list.html) | +| `kegg.id` | `kegg.ID` | `""` (KEGG is not used) | Organism code in the KEGG database, from the [KEGG organism list](https://www.genome.jp/kegg/catalog/org_list.html) | | `kegg.gene_id` | `kegg.geneID` | `"kegg"` | Field of the KEGG entry that matches the gene identifiers of the ecModel. `kegg` takes the KEGG entry identifier. Identifiers from the "Other DBs" section of the KEGG page, such as `NCBI-GeneID`, `UniProt` or `Ensembl` (for Human-GEM), are available for some organisms | | `uniprot.type` | `uniprot.type` | `"taxonomy"` | Whether `uniprot.id` is a taxonomy identifier (`taxonomy`) or a proteome identifier (`proteome`) | -| `uniprot.id` | `uniprot.ID` | `"559292"` | Identifier of the UniProt data set (by taxonomy or proteome) with complete coverage of the genes of the ecModel | +| `uniprot.id` | `uniprot.ID` | `""` (required unless `data/uniprot.tsv` exists) | Identifier of the UniProt data set (by taxonomy or proteome) with complete coverage of the genes of the ecModel | | `uniprot.gene_id_field` | `uniprot.geneIDfield` | `"gene_oln"` | UniProt field with the gene identifiers used in the ecModel; one of the "Returned Field" entries under "Names & Taxonomy" of the [UniProt return fields](https://www.uniprot.org/help/return_fields) | | `uniprot.reviewed` | `uniprot.reviewed` | `false` | Whether the data is limited to reviewed (Swiss-Prot) entries. Reviewed data has the highest confidence but can have low coverage for non-model organisms | | `complex.taxonomic_id` | `complex.taxonomicID` | none | Taxonomy identifier as available from the [Complex Portal](https://www.ebi.ac.uk/complexportal/home). Without it, no Complex Portal query is made | @@ -119,7 +119,7 @@ A stage has the keys `action` (`select`, the default, or `combine`), `sources`, ## Kcat tuning -The `[evotune]` section holds the hyperparameters of the CMA-ES kcat tuning described in [Tuning against experimental data](tuning-against-experimental-data.md). +The `[evotune]` section holds the hyperparameters of the CMA-ES kcat tuning described in [Tune kcats against experimental data](advanced-tuning.md). | TOML key | MATLAB parameter | Default | Description | |---|---|---|---| @@ -138,7 +138,7 @@ When `[evotune]` has no `source_groups`, the groups are `dlkcat`, `brenda` and ` | TOML key | MATLAB parameter | Default | Description | |---|---|---|---| -| `okp.method` | `okp.method` | `"CataPro"` | OpenKineticsPredictor method | +| `okp.method` | `okp.method` | `"KinForm-H"` | OpenKineticsPredictor method | | `okp.handle_long_sequences` | `okp.handleLongSequences` | `"truncate"` | Action of OpenKineticsPredictor on sequences that are too long | | `okp.include_similarity_columns` | `okp.includeSimilarityColumns` | `true` | Return the similarity to the training data of the method | | `okp.canonicalize_substrates` | `okp.canonicalizeSubstrates` | `true` | Canonicalise substrate SMILES before the prediction | @@ -155,17 +155,7 @@ The API key is no parameter. It is an argument of the functions, the environment geckopy ignores the `[matlab]` section. The functions are searched on the MATLAB path, to which the `code` folder of the project is added. -## Adapter classes - -A GECKO 3 adapter class keeps working. It sets `obj.params` with the MATLAB names of the tables above, and `ModelAdapterManager.setDefault` takes the path of its `.m` file. `convertAdapterToToml` writes the parameters of a class as `model_adapter.toml` and lists the methods that the class overrides and the file cannot express. The option `'adapterFormat','class'` of `createGECKOproject` creates a project with a class. - -A class that needs code beyond the file can read its parameters from the file in the constructor: - -```matlab -obj.params = loadAdapterParams(fullfile(projectPath, 'model_adapter.toml')); -``` - -In geckopy, a subclass of `ModelAdapter` overrides `get_spontaneous_reactions`, `get_uniprot_compatible_genes`, `get_uniprot_ids_from_table`, `get_kegg_compatible_genes`, `get_brenda_db_folder` or `get_phyl_dist_path`. It is instantiated explicitly; `from_folder` does not discover it. +A GECKO 3 adapter class sets the same parameters in MATLAB code; [GECKO 3 to GECKO 4](../gecko3-to-gecko4.md#adapter-classes) describes how such a class is used and converted. ## Troubleshooting @@ -179,5 +169,5 @@ In geckopy, a subclass of `ModelAdapter` overrides `get_spontaneous_reactions`, - [Project format 4](../gecko4-project-format.md), the sections of `model_adapter.toml`. - [Kcat assignment](../gecko4-kcat-assignment.md), the presets and stages of `[kcat_assignment]`. -- [Getting started](getting-started.md), where the model adapter is created and populated. +- [Build an ecModel](first-ecmodel.md), where the model adapter is created and populated. - [Files in the data folder](data-folder-files.md), the other project-wide reference table. diff --git a/docs/guide/protein-pool.md b/docs/guide/protein-pool.md new file mode 100644 index 0000000..1e4d734 --- /dev/null +++ b/docs/guide/protein-pool.md @@ -0,0 +1,310 @@ +# Set the protein pool + +The protein pool limits the total mass of enzymes that an enzyme-constrained model (ecModel) can use. The result is an ecModel whose `prot_pool_exchange` has an upper bound calculated from $P_{tot}$, $f$ and $\sigma$, with $\sigma$ fitted to the measured maximum growth rate after the kcats are tuned. + + +:::::{dropdown} Place in the workflow + +The diagram orders the functions of this page within the [workflow](workflow.md). Boxes with a dashed outline are optional steps, a dotted arrow is an optional path, and a diamond is a choice. Rounded boxes with a dotted outline are the context, other pages or the model to start from, and a click on one opens its page. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Previous: Choose the kcat sources"]):::ctx + AD["Ptot, f and sigma
of the model adapter"] + CF["calculateFfactor
f from PAXdb or proteomics"]:::opt + FD["loadFluxData
Ptot of a condition"]:::opt + SP["setProtPoolSize"] + TU(["Choose a kcat tuning: the standard tuning"]):::ctx + AT(["Tune kcats against experimental data"]):::ctx + SF["sigmaFitter
sigma to the growth rate, after the tuning"] + PV --> AD --> SP + CF -.-> SP + FD -.-> SP + SP --> TU --> SF + NP(["Next: Constrain enzymes with proteomics (optional)"]):::ctx + NA(["Next: Analyse an ecModel"]):::ctx + SF --> NP + SF --> NA + SP -.->|advanced tuning, no sigma fit| AT + AT --> NP + AT --> NA + click AT href "advanced-tuning.html" + click PV href "kcat-sources.html" + click TU href "kcat-tuning.html" + click NP href "proteomics.html" + click NA href "analysis.html" +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Previous: Choose the kcat sources"]):::ctx + AD["Ptot, f and sigma
of the model adapter"] + CF["calculate_f_factor
f from PAXdb or proteomics"]:::opt + FD["load_flux_data
Ptot of a condition"]:::opt + SP["set_prot_pool_size"] + TU(["Choose a kcat tuning: the standard tuning"]):::ctx + AT(["Tune kcats against experimental data"]):::ctx + SF["fit_sigma
sigma to the growth rate, after the tuning"] + PV --> AD --> SP + CF -.-> SP + FD -.-> SP + SP --> TU --> SF + NP(["Next: Constrain enzymes with proteomics (optional)"]):::ctx + NA(["Next: Analyse an ecModel"]):::ctx + SF --> NP + SF --> NA + SP -.->|advanced tuning, no sigma fit| AT + AT --> NP + AT --> NA + click AT href "advanced-tuning.html" + click PV href "kcat-sources.html" + click TU href "kcat-tuning.html" + click NP href "proteomics.html" + click NA href "analysis.html" +``` +::: +:::: +::::: + +## Functions + +| MATLAB | Python | Note | +|---|---|---| +| `setProtPoolSize` | `set_prot_pool_size` | set the upper bound of `prot_pool_exchange` from $P_{tot}$, $f$ and $\sigma$ | +| `calculateFfactor` | `calculate_f_factor` | calculate $f$ from `data/paxDB.tsv` or from proteomics data | +| `loadFluxData` | `load_flux_data` | read `data/fluxData.tsv`, which has a total protein content per condition | +| `sigmaFitter` | `fit_sigma` | fit $\sigma$ so that the ecModel reaches a growth rate | + +## Setup + +The ecModel has kcats applied (see [Choose the kcat sources](kcat-sources.md)). The adapter keys `sigma`, `p_tot`, `f` and `gr_exp` are described in [Model adapter parameters](model-adapter-parameters.md#model-and-protein-pool). + +## The three parameters + +`setProtPoolSize` (Python: `set_prot_pool_size`) sets the upper bound of `prot_pool_exchange` to + +$$ P_{tot} \cdot f \cdot \sigma \cdot 1000 $$ + +in mg/gDCW, the same unit as the enzyme cost of the reactions. + +| Parameter | TOML key (MATLAB name) | Meaning | Unit | Default | +|---|---|---|---|---| +| $P_{tot}$ | `p_tot` (`Ptot`) | total protein content of the cell | g protein per gDCW | 0.5 | +| $f$ | `f` | mass fraction of the total protein that the enzymes of the ecModel account for | g enzyme/g protein | 0.5 | +| $\sigma$ | `sigma` | average saturation of the enzymes: the fraction of their maximum rate at which the enzymes work in the cell | none, 0 to 1 | 0.5 | + +With the three defaults, the bound is 0.5 Γ— 0.5 Γ— 0.5 Γ— 1000 = 125 mg/gDCW. + +## Choose where the values come from + +| Value | Source | Needs | Section | +|---|---|---|---| +| $P_{tot}$, $f$, $\sigma$ | the model adapter | values from the literature, or the defaults | [Set the pool from the adapter](#set-the-pool-from-the-adapter) | +| $f$ | `calculateFfactor` with `data/paxDB.tsv` | a [PAXdb](https://pax-db.org) abundance file of the organism | [Calculate f](#calculate-f) | +| $f$ | `calculateFfactor` with proteomics data | absolute proteomics data, loaded with `loadProtData` | [Calculate f](#calculate-f) | +| $P_{tot}$ | `data/fluxData.tsv` | a measured total protein content for the condition | [Use the protein content of a condition](#use-the-protein-content-of-a-condition) | +| $\sigma$ | `sigmaFitter` | a measured maximum growth rate | [Fit sigma to a growth rate](#fit-sigma-to-a-growth-rate) | + +$P_{tot}$ and $f$ are measured properties of the cell; set them from data when data exist. $\sigma$ has no direct measurement, so it is the parameter that is fitted. + +## Set the pool from the adapter + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +ecModel = setProtPoolSize(ecModel); +ecModel = setProtPoolSize(ecModel, 'Ptot', 0.45, 'f', 0.4); +``` + +Each of `Ptot`, `f` and `sigma` that is not given is read from the model adapter (the default adapter, or the one given as `modelAdapter`). The arguments can also be given by position: `setProtPoolSize(ecModel, Ptot, f, sigma, modelAdapter)`. +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import set_prot_pool_size + +set_prot_pool_size(ec_model) +set_prot_pool_size(ec_model, p_tot=0.45, f=0.4) +``` + +Each of `p_tot`, `f` and `sigma` that is not given is read from `ec_model.adapter.params`. The function changes `ec_model` in place and returns the bound it set. It raises a `ValueError` when the ecModel has no `prot_pool_exchange`, or when a value is missing and the ecModel has no adapter. +::: +:::: + +The values given as arguments are not written to `model_adapter.toml`. The next call without arguments uses the adapter values again. + +## Calculate f + +$f$ is the mass of the measured proteins that are enzymes of the ecModel (in `ec.enzymes`), divided by the mass of all measured proteins. The abundances can be in any unit that is proportional to mass, because the unit cancels in the ratio. The abundances of `data/paxDB.tsv` are numbers of molecules (parts per million), so `calculateFfactor` multiplies them by the molecular weight (MW) of the protein from `data/uniprot.tsv`. The abundances of proteomics data are summed as they are; `loadProtData` gives them in mg/gDCW. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +From `data/paxDB.tsv`: + +```matlab +f = calculateFfactor(ecModel); +ecModel = setProtPoolSize(ecModel, 'f', f); +``` + +`calculateFfactor` maps the genes of `data/paxDB.tsv` to UniProt identifiers through `data/uniprot.tsv`. Without `data/paxDB.tsv`, it prints a warning and returns 0.5. + +From proteomics data loaded with `loadProtData` (see [Constrain enzymes with proteomics](proteomics.md)): + +```matlab +f = calculateFfactor(ecModel, 'protData', protData); +``` + +With several samples in `protData`, the abundance of a protein is averaged over the samples in which it is measured. The option `enzymes` replaces `ec.enzymes` as the list of proteins counted as enzymes. +::: +:::{tab-item} 🐍 Python +:sync: python + +From `data/paxDB.tsv`: + +```python +from geckopy import calculate_f_factor + +f = calculate_f_factor(ec_model) +set_prot_pool_size(ec_model, f=f) +``` + +Without `prot_data`, `calculate_f_factor` reads `data/paxDB.tsv` of the project of `ec_model.adapter` and maps its genes to UniProt identifiers through `data/uniprot.tsv`. Without `data/paxDB.tsv`, it logs a warning and returns 0.5. + +From proteomics data loaded with `load_prot_data` (see [Constrain enzymes with proteomics](proteomics.md)): + +```python +f = calculate_f_factor(ec_model, prot_data) +``` + +With several samples, the abundance of a protein is averaged over the samples in which it is measured. The keyword `enzymes` replaces `ec.enzymes` as the list of proteins counted as enzymes. +::: +:::: + +Both functions return 0 when no measured protein is an enzyme of the ecModel (Python: with a warning). An $f$ of 0, or close to 0, means that the protein identifiers of the data do not match `ec.enzymes`. + +## Use the protein content of a condition + +`data/fluxData.tsv` has one row per condition, with the total protein content in the column `Ptot` (g/gDCW), the growth rate and the exchange fluxes. The file is described in [Files in the data folder](data-folder-files.md). To set the pool for one condition, give its $P_{tot}$: + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +fluxData = loadFluxData(); +ecModel = setProtPoolSize(ecModel, 'Ptot', fluxData.Ptot(1)); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import load_flux_data + +flux_data = load_flux_data(ec_model.adapter.params.path / "data" / "fluxData.tsv") +set_prot_pool_size(ec_model, p_tot=float(flux_data.p_tot[0])) +``` +::: +:::: + +`constrainFluxData` (Python: `apply_flux_data_constraints`) sets the exchange fluxes and the growth rate of a condition, and does not change the protein pool. [Constrain enzymes with proteomics](proteomics.md) uses $P_{tot}$ and $f$ of the same condition. + +## Fit sigma to a growth rate + +`sigmaFitter` (Python: `fit_sigma`) sets the protein pool for a series of $\sigma$ values, maximises the objective for each, and selects the $\sigma$ whose growth rate is closest to the target. The target is `gr_exp` of the adapter unless it is given. $P_{tot}$ and $f$ are read from the adapter unless they are given. + +Before the scan, both functions set the bounds of the objective reaction to 0 and 1000, and these bounds stay on the returned ecModel. The objective must be the growth reaction. A $\sigma$ at which the model has no solution counts as growth 0. The returned ecModel has the protein pool set for the selected $\sigma$. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +[ecModel, sigma] = sigmaFitter(ecModel, 'makePlot', false); +``` + +`sigmaFitter` tests $\sigma$ = 0.01, 0.02, ..., 1 and calculates for each the error $100 \times |\mu_{target} - \mu| / \mu_{target}$ in percent. It selects the smallest $\sigma$ whose error is at most the smallest error plus $10^{-9} \times 100 \times \mu_{best} / \mu_{target}$, where $\mu_{best}$ is the growth rate at the $\sigma$ with the smallest error. This tolerance allows a difference in growth rate of a relative $10^{-9}$ of $\mu_{best}$, which keeps rounding differences of the solver from moving the result. When the target is above the growth rate at $\sigma$ = 1 and the growth rate is constant from some $\sigma$ on, the result is the first $\sigma$ of that plateau with every solver. + +`sigmaFitter` raises the error `sigmaFitter:growthRate` when the target growth rate is not above 0. The options are `growthRate`, `Ptot`, `f`, `makePlot` (default `true`, a plot of the error against $\sigma$) and `modelAdapter`, as name-value pairs or by position in this order. +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import fit_sigma + +fit = fit_sigma(ec_model) +print(fit.sigma) +``` + +`fit_sigma` changes `ec_model` in place and returns a `SigmaFitterResult` with `sigma`, and the arrays `sigma_grid`, `growth_grid` and `error_grid` (the error in percent) for a plot. The keywords `growth_rate`, `p_tot` and `f` replace the adapter values; a `growth_rate` that is not above 0 raises a `ValueError`. The keyword `method` selects the search: + +- `method="grid"` (default) tests $\sigma$ = 1/n, 2/n, ..., 1 with n = `n_sigma_steps` (default 100, the same values as `sigmaFitter`) and selects the first $\sigma$ with the smallest error. It has no tolerance: when the growth rates on a plateau differ by rounding errors of the solver, the result can be a later $\sigma$ of the plateau. +- `method="bisect"` uses that the growth rate does not decrease when $\sigma$ increases. It searches the smallest $\sigma$ at which the growth rate reaches the target, to a resolution of $10^{-6}$, so the result is in general not a multiple of 0.01. When the growth rate at $\sigma$ 1 is below the target, it searches the smallest $\sigma$ at which the growth rate is within a relative $10^{-9}$ of the growth rate at $\sigma$ 1, the start of the plateau. The arrays of the result contain the $\sigma$ values that were tested. +::: +:::: + +A $\sigma$ of 1, or a plateau that ends below the target growth rate, means that a larger protein pool does not let the ecModel reach the target. The kcats or the nutrient constraints limit growth. + +## Fit sigma or tune the kcats + +$\sigma$ changes the protein pool for all enzymes by the same factor. Tuning changes the kcats of the enzymes that limit growth (see [Choose a kcat tuning](kcat-tuning.md)). The order is: + +1. Set the pool with the adapter values of $P_{tot}$, $f$ and $\sigma$. +2. Tune the kcats. `sensitivityTuning` solves the model first and stops without a change when the growth rate is at or above the target. Otherwise it multiplies a kcat by `foldChange` (default 10) in each step until the growth rate reaches the target, so the growth rate after tuning is often above the target. +3. Fit $\sigma$. `sigmaFitter` selects the $\sigma$ at which the growth rate matches the target; after a tuning that ended above the target, this $\sigma$ is lower than the adapter value. + +A $\sigma$ fitted before the tuning, to an ecModel with kcats that are too low, is close to 1, or is the start of a plateau below the target, and makes the pool compensate for the kcats. + +A growth rate below the target can have three causes, checked in this order: the network (the conventional genome-scale metabolic model, GEM, does not reach the target either), the protein pool, and the kcats. To test the pool, remove its bound and compare the growth rate. The minimal protein pool usage at the target growth rate (see [Minimise protein pool usage at fixed growth](analysis.md#minimise-protein-pool-usage-at-fixed-growth)) shows how much enzyme mass the current kcats need. Setting the pool bound to that usage makes the ecModel reach the target, but that bound does not follow from $P_{tot}$, $f$ and $\sigma$; use it as a diagnostic and keep the pool calculated by `setProtPoolSize`. + +A fitted $\sigma$ close to 1 means that the enzymes work at their maximum rate on average; check the kcats that limit growth in that case. + +## Troubleshooting + +| Problem | Possible reason | Solution | +|---|---|---| +| Error `setProtPoolSize:noPoolExchange` (Python: `ValueError`) that the model has no `prot_pool_exchange` | the model is a conventional GEM, or the reaction was removed after `makeEcModel` | build the ecModel with `makeEcModel` | +| `calculateFfactor` prints a warning and returns 0.5 | there is no `data/paxDB.tsv` and no `protData` was given | add the file, give `protData`, or keep the adapter value of $f$ | +| $f$ is 0 or close to 0 | the protein identifiers of the data do not match `ec.enzymes` | map the identifiers of the data to UniProt identifiers | +| Error `sigmaFitter:growthRate` (Python: `ValueError`) | the target growth rate, given or read from `gr_exp`, is not above 0 | set `gr_exp`, or give the growth rate | +| `fit_sigma` raises a `ValueError` that `model.adapter` is None | the ecModel has no adapter attached, and `growth_rate`, `p_tot` or `f` is not given | attach the adapter, or give `growth_rate`, `p_tot` and `f` | +| The fitted $\sigma$ is 1, or the error is large at every $\sigma$ | the kcats, the nutrient constraints or the network limit growth | see [Fit sigma or tune the kcats](#fit-sigma-or-tune-the-kcats) | +| The growth rate is free after `sigmaFitter` or `fit_sigma`, although its bounds were fixed before | both set the bounds of the objective reaction to 0 and 1000, and the returned ecModel keeps them | set the bounds of the growth reaction again | +| `sigmaFitter` and `fit_sigma` give different $\sigma$ values for the same ecModel | on a growth plateau, `fit_sigma` with `method="grid"` has no tolerance for rounding errors of the solver | use `method="bisect"`, or compare the growth rates in `fit.growth_grid` | + +## See also + +- [Introduction to ecModels](introduction.md), for the enzyme constraint and the role of the protein pool. +- [Choose a kcat tuning](kcat-tuning.md), when the growth rate stays below the target. +- [Constrain enzymes with proteomics](proteomics.md), for $P_{tot}$ and $f$ of a condition with measured enzyme levels. diff --git a/docs/guide/proteomics-integration.md b/docs/guide/proteomics-integration.md deleted file mode 100644 index 44b54a9..0000000 --- a/docs/guide/proteomics-integration.md +++ /dev/null @@ -1,356 +0,0 @@ -# Proteomics integration - -Enzyme usage reactions let a full ecModel take proteomics data directly: individual enzymes with measured concentrations are bounded by those measurements, in addition to drawing from the shared total protein pool. The ecModel is then constrained by the proteomics and the exchange flux data of one experimental condition, and its growth rate is solved. - -:::{warning} -The functions below apply to full ecModels. A light ecModel has no per-enzyme usage reactions to constrain; see [GECKO light and full ecModels](gecko-light.md). -::: - - -:::::{dropdown} Place in the workflow - -The diagram orders the functions of this page within the [workflow](workflow.md). Dashed boxes are optional and dotted boxes are neighbouring pages. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - PV(["Previous: Growth-rate tuning"]):::ctx - PV --> A["loadProtData
fillEnzConcs"] --> B["levelComplexConcs
constrainEnzConcs"] --> I["getIsozymeCoverage
zeroUnmeasuredIsozymes"]:::opt --> C["calculateFfactor, setProtPoolSize"] --> D["loadFluxData, constrainFluxData"]:::opt - NX(["Next: Relaxing overconstrained proteomics"]):::ctx - D --> NX -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - PV(["Previous: Growth-rate tuning"]):::ctx - PV --> A["load_prot_data
fill_enz_concs"] --> B["level_complex_concs
constrain_enz_concs"] --> I["get_isozyme_coverage
zero_unmeasured_isozymes"]:::opt --> C["calculate_f_factor, set_prot_pool_size"] --> D["load_flux_data, apply_flux_data_constraints"]:::opt - NX(["Next: Relaxing overconstrained proteomics"]):::ctx - D --> NX -``` -::: -:::: -::::: - -## Functions - -| MATLAB | Python | Note | -|---|---|---| -| `loadProtData` | `load_prot_data` | read `proteomics.tsv` into a proteomics data structure and filter it | -| `fillEnzConcs` | `fill_enz_concs` | populate `ec.concs` with the measured concentrations | -| `levelComplexConcs` | `level_complex_concs` | give the subunits of each complex one abundance | -| `getComplexFamilies` | `complex_families` | group the enzymes of multi-subunit reactions into complexes | -| `constrainEnzConcs` | `constrain_enz_concs` | apply the concentrations as upper bounds of the `usage_prot_` reactions | -| `getIsozymeCoverage` | `get_isozyme_coverage` | list the reactions with a measured and an unmeasured isozyme | -| `zeroUnmeasuredIsozymes` | `zero_unmeasured_isozymes` | lower the usage upper bound of the unmeasured isozymes of such a reaction | -| `calculateFfactor` + `setProtPoolSize` | `calculate_f_factor` + `set_prot_pool_size` | recompute the protein pool constraint from the proteomics data | -| `loadFluxData` | `load_flux_data` | read `fluxData.tsv` (exchange fluxes and growth rate of the same experiment) | -| `constrainFluxData` | `apply_flux_data_constraints` | apply the exchange flux and growth rate constraints | - -## Setup - -The data are in `data/proteomics.tsv` and `data/fluxData.tsv` of the project (see [Files in the data folder](data-folder-files.md)). An ecModel is loaded or built as described in [Building an empty ecModel](building-ec-model.md). The order of the functions is `loadProtData`, `fillEnzConcs`, `levelComplexConcs` (optional), `constrainEnzConcs`, and optionally `zeroUnmeasuredIsozymes`. - -Constraining individual enzyme concentrations usually overconstrains the ecModel, because the uncertainty comes from two sources at once: the enzyme levels and the $k_{cat}$ values. Proteomics data need to be of high quality, with low-certainty measurements discarded, or the ecModel does not reach its intended growth rate. - -The data are prepared before loading: - -- Protein concentrations are in milligram per gram dry cell weight (mg/gDCW); relative proteomics cannot supply this unit (see the warning in [Apply the concentrations as bounds](#apply-the-concentrations-as-bounds)). -- `data/proteomics.tsv` holds the UniProt identifier in the first column and one column per replicate after it, for one or more experimental conditions. -- Low-certainty measurements are filtered out: low concentrations, proteins with a high relative standard deviation across replicates, and proteins absent from most replicates. - -:::{admonition} Absolute quantitative data is not the sole option -:class: tip - -Proteomics measurements are not always performed with external standards for absolute quantification. Label-free mass spectrometry needs neither external standards nor stable isotope labelling, and has been used with GECKO ecModels. -::: - -## Load the proteomics data - -`loadProtData` / `load_prot_data` takes the number of replicates of each condition, in the order of the columns of the file. The sum of these numbers must equal the number of replicate columns of the file; otherwise the function raises an error (`loadProtData:replicateMismatch` in MATLAB, `ValueError` in Python). It returns the mean over the replicates of each condition. An abundance of 0 is read as not measured (NaN) before any filter, also when `filterData` is false, unless `zerosAsMeasured` is true. The filtering is controlled by these parameters: - -| MATLAB | Python | Default | Effect | -|---|---|---|---| -| `filterData` | `filter_data` | true | when false, the other filtering parameters are not used and the mean of the replicates is returned | -| `maxMissing` | `max_missing` | 2/3 | minimum fraction of replicates with a value above zero; a protein with fewer is discarded for that condition. A vector gives one value per condition | -| `maxRSD` | `max_rsd` | 1 | maximum relative standard deviation across replicates; a protein above it is discarded for that condition | -| `addStdevs` | `add_stdevs` | 1 | number of standard deviations added to the mean of the replicates | -| `minVal` | `min_val` | 0 | minimum value after `addStdevs`; a protein below it is discarded for that condition | -| `cutLowest` | `cut_lowest` | 5 | percentage of the lowest values per condition that is discarded | -| `zerosAsMeasured` | `zeros_as_measured` | false | whether an abundance of 0 is a measurement (see [Measured zeros](#measured-zeros-and-unmeasured-enzymes)) | - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -% one experiment with three replicates -protData = loadProtData(3); - -% two experiments with three replicates each (a file with six replicate columns) -protData = loadProtData([3, 3]); -``` - -`loadProtData` also filters the dataset. When `protData` holds several experiments, select which one to use (experiment 1 here): - -```matlab -ecModel = fillEnzConcs(ecModel, protData, 1); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import fill_enz_concs, load_prot_data - -# one experiment with three replicates, as a one-element list -prot_data = load_prot_data( - params.path / "data" / "proteomics.tsv", repl_per_cond=[3], -) - -# two experiments with three replicates each: repl_per_cond=[3, 3] -``` - -`data_col` selects which experiment to use, 0-indexed (MATLAB's experiment index is 1-indexed), so the first experiment is `data_col=0`: - -```python -fill_enz_concs(ec_model, prot_data, data_col=0) -``` -::: -:::: - -`fillEnzConcs` / `fill_enz_concs` replaces `ec.concs` with the concentrations of the selected experiment. An enzyme without a measurement has `NaN`. A negative abundance of an enzyme of the model raises the error `fillEnzConcs:negativeAbundance` in MATLAB and a `ValueError` in Python. In Python, a `data_col` outside the columns of the abundances raises an `IndexError`. - -### Measured zeros and unmeasured enzymes - -`ec.concs` distinguishes two cases of an enzyme without a positive concentration: - -- **NaN:** the enzyme is not measured. Its usage reaction draws from the protein pool without a bound of its own. -- **0:** the enzyme was measured as absent. `constrainEnzConcs` sets the upper bound of its usage reaction to 0, which removes every flux through the enzyme. - -Proteomics reports a protein below the detection limit as 0 or as missing, so a 0 is not always an absent protein. By default (`zerosAsMeasured` false), `loadProtData` reads every 0 as NaN: the mean of a protein over its replicates leaves the zeros out, and a protein whose replicates are all 0 is removed. With `zerosAsMeasured` true, a 0 counts as a measured replicate and is kept in the mean. A protein whose replicates are all 0 is returned as 0, and neither `minVal` nor `cutLowest` removes it. A measured 0 constrains the model more than an unmeasured enzyme does and can lower the growth rate or make the model infeasible. `levelComplexConcs` keeps a subunit with a concentration of 0 at 0. - -An enzyme with a concentration of 0 is not raised by `flexibilizeEnzConcs`, because the bound is raised as a factor of the concentration (see [Relaxing overconstrained proteomics](relaxing-constraints.md)). - -## Level the concentrations of complexes - -Absolute proteomics recovers the subunits of a complex unevenly, so the subunits of one complex can disagree several fold about the amount of it that is present. When each subunit is capped by its own measurement, the complex carries no more flux than its least abundant subunit allows. `levelComplexConcs` / `level_complex_concs` gives each complex one abundance. It runs after `fillEnzConcs` and before `constrainEnzConcs`. - -`getComplexFamilies` / `complex_families` groups the enzymes of the multi-subunit reactions (`ec.rxnEnzMat`, Python: `ec.rxn_enz_mat`) into complexes. Two enzyme sets are one complex when they share at least `minShared` subunits (default 2) and at least a fraction `minOverlap` (default 0.5) of the larger set (Python: `min_shared`, `min_overlap`). A subunit that appears in two complexes belongs to the complex whose enzyme sets catalyse more reactions. A subunit in three or more complexes, in two complexes of equal weight, or that also catalyses a reaction on its own is shared and stays as measured. - -The concentrations (mg/gDCW) are converted to molar amounts with `ec.mw`. Per complex, the median (default) of the molar amount divided by the copies of the subunit, over the measured subunits that belong to the complex, is written back to every such subunit, multiplied by its copies and its molecular weight. The option `statistic` selects `'median'` or `'mean'`. With `fillMissing` true (default; Python: `fill_missing`), subunits of the complex without a measurement receive the concentration of the complex, and are constrained by `constrainEnzConcs` afterwards. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -[ecModel, levelResult] = levelComplexConcs(ecModel); -``` - -`levelResult` has the fields `adjusted` (measured subunits whose concentration was replaced), `added` (unmeasured subunits that received the concentration of the complex), `shared` (measured subunits left as measured) and `skipped` (complexes without a measured subunit that belongs to them). The option `families` takes the output of `getComplexFamilies`, to reuse a grouping with other parameters. -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import level_complex_concs - -level_result = level_complex_concs(ec_model) -``` - -`level_complex_concs` changes `ec_model.ec.concs` in place and returns a `LevelComplexResult` with `adjusted`, `added`, `shared` and `skipped` (a list of tuples of subunits, one per complex). -::: -:::: - -A model with levelled complexes is relaxed with the option `perComplex` of `flexibilizeEnzConcs` (see [Relaxing overconstrained proteomics](relaxing-constraints.md)). - -## Apply the concentrations as bounds - -Each `usage_prot_` reaction of an enzyme with a measured concentration receives that concentration as its upper bound. Every usage reaction keeps drawing from the protein pool, with or without a bound. A concentration of 0 gives an upper bound of 0 (see [Measured zeros](#measured-zeros-and-unmeasured-enzymes)), and an enzyme with a NaN concentration keeps the default upper bound. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -ecModel = constrainEnzConcs(ecModel); -``` - -A negative concentration in `ec.concs` raises the error `constrainEnzConcs:negativeConcentration`. -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import constrain_enz_concs - -constrain_enz_concs(ec_model) -``` - -Raises `ValueError` on a light ecModel: light ecModels have no `usage_prot_*` reactions to constrain. A negative concentration also raises `ValueError`. -::: -:::: - -:::{warning} -Enzyme concentrations must be in mg/gDCW. Relative proteomics data loaded with the wrong units produces an ecModel with nonsensical constraints and raises no error. -::: - -## Limit unmeasured isozymes - -A reaction with several isozymes can have a measured isozyme and an unmeasured one. The unmeasured isozyme then draws from the protein pool without a bound, although the reaction is already catalysed by the measured one. `getIsozymeCoverage` / `get_isozyme_coverage` lists these reactions without changing the ecModel. The entries of `ec.rxns` that share an identifier once the suffix `_EXP_` is removed are the isozymes of one reaction. An entry is measured when one of its enzymes has a concentration above 0, and unmeasured when all its enzymes are NaN; the kcat is not considered. An enzyme of an unmeasured entry that also catalyses an entry outside the unmeasured entries of partially covered reactions is listed as shared and is not changed. - -`zeroUnmeasuredIsozymes` / `zero_unmeasured_isozymes` lowers the upper bound of the `usage_prot_` reaction of the other enzymes of the unmeasured entries. It runs after `constrainEnzConcs`, which resets the bounds, and no other function calls it. It does not change `ec.concs` and never raises a bound. The `method` is `'zero'` (Python: `"zero"`, default), which sets the bound to 0, or `'detectionLimit'` (Python: `"detection_limit"`), which sets it to `limit` in mg/gDCW (default: the smallest concentration above 0 in `ec.concs`). A protein below the detection limit is missing from the data as well as an absent protein, so `'zero'` can remove a flux that exists and lower the growth rate. An enzyme that was lowered is a candidate for `relaxProteomicsGreedy` like any enzyme with a bound below the default. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -[coverage, zeroable] = getIsozymeCoverage(ecModel); -[ecModel, changed] = zeroUnmeasuredIsozymes(ecModel, 'method', 'detectionLimit'); -``` - -`coverage` is a table with one row per partially covered reaction and the variables `rxn`, `nEntries`, `nMeasured`, `nUnmeasured`, `measuredEnzymes`, `unmeasuredEnzymes` and `sharedEnzymes`. `zeroable` holds the enzymes that `zeroUnmeasuredIsozymes` acts on. `changed` is a table with one row per enzyme and the variables `uniprot` and `newUpperBound`. -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import get_isozyme_coverage, zero_unmeasured_isozymes - -coverage, zeroable = get_isozyme_coverage(ec_model) -changed = zero_unmeasured_isozymes(ec_model, method="detection_limit") -``` - -`coverage` is a `DataFrame` with the columns `rxn`, `n_entries`, `n_measured`, `n_unmeasured`, `measured_enzymes`, `unmeasured_enzymes` and `shared_enzymes`, and `zeroable` is a list of enzymes. `zero_unmeasured_isozymes` lowers the bounds of `ec_model` in place and returns a `DataFrame` with the columns `uniprot` and `new_upper_bound`. Both functions raise `NotImplementedError` on a light ecModel. -::: -:::: - -## Set the protein pool from the proteomics data - -A proteomics experiment may reflect a physiology different from the default assumption. The pool size is recomputed from a sample-specific total protein content (for example 0.5 g/gDCW) in place of the adapter default `Ptot` (Python: `p_tot`), and from the f-factor of the proteomics data, the fraction of the measured protein mass that is accounted for by enzymes of the model. With several experiments in `protData`, `calculateFfactor` averages the abundance of each protein over the experiments in which it is measured. `updateProtPool` is a function of GECKO 4 that raises an error when a usage reaction with an upper bound other than 1000 draws from the protein pool, which is the case for every ecModel with concentration-bounded usage reactions from GECKO 3.2 on; `setProtPoolSize` sizes the pool for such ecModels. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -f = calculateFfactor(ecModel, 'protData', protData); -ecModel = setProtPoolSize(ecModel, 'Ptot', 0.5, 'f', f); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import calculate_f_factor, set_prot_pool_size - -f = calculate_f_factor(ec_model, prot_data) -set_prot_pool_size(ec_model, p_tot=0.5, f=f) -``` -::: -:::: - -## Constrain with experimentally measured exchange fluxes - -Because the integrated enzyme concentrations come from one particular experiment, any exchange reaction for which that experiment has measured data, such as glucose uptake, ethanol production or CO2 exchange rate, is constrained too. `data/fluxData.tsv` can hold multiple experiments and as many reaction fluxes as needed; a flux of zero blocks the corresponding exchange reaction, and a flux of -1000 or 1000 leaves uptake or excretion unconstrained. Load it: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -fluxData = loadFluxData(); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import load_flux_data - -flux_data = load_flux_data(params.path / "data" / "fluxData.tsv") -``` -::: -:::: - -Two choices determine how the constraint is applied: - -1. **Loose or strict.** With loose constraints, the measured rate is set as the upper bound and the lower bound stays at zero (for a reaction with a negative rate, the lower bound gets the measured value and the upper bound stays at zero). With strict constraints, a percentage between 0 and 100 is passed in place of `'loose'` (Python: a number), and the lower and upper bounds are both set around the measured value; 10 gives 95 % and 105 % of it. Strict constraints force the ecModel to follow the measured fluxes closely, but imprecision in either the fluxes or the enzyme concentrations can then make the model infeasible; if that happens, loosen the constraint. -2. **Max or min growth.** With `max`, the measured growth rate is the upper bound, the ceiling the ecModel should reach; this suits growth as the objective function. With `min`, the measured growth rate is the lower bound, the floor the ecModel must reach; this suits minimising protein pool usage as the objective (see [Simulation and analysis](simulation-and-analysis.md)). - -Loose constraints for the metabolic exchange fluxes and `max` for growth make the ecModel least likely to be overconstrained. The following applies the data of the first experiment and solves: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -ecModel = constrainFluxData(ecModel, fluxData, 1, 'max', 'loose'); -sol = solveLP(ecModel); -fprintf('Growth rate that is reached: %f /hour\n', abs(sol.f)) -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import apply_flux_data_constraints - -apply_flux_data_constraints( - ec_model, flux_data, - condition=0, max_min_growth="max", loose_strict_flux="loose", -) -sol = ec_model.optimize() -print(f"Growth rate that is reached: {sol.objective_value:.4f} /hour") -``` - -`condition` is 0-indexed (MATLAB's experiment index is 1-indexed), so the first experiment is `condition=0`. -::: -:::: - -With `max` and loose constraints, FBA reaches a solution unless too many exchange fluxes were pinned to zero. If the reachable growth rate falls short of the intended one, the enzyme levels are overconstrained; relax them with the functions of [Relaxing overconstrained proteomics](relaxing-constraints.md). Applying the same flux constraints to the conventional GEM, with no enzyme constraints, shows which part is overconstrained: if the conventional GEM also falls short of the intended growth rate, the exchange flux data are the limit. - -## Troubleshooting - -| Problem | Possible reason | Solution | -|---|---|---| -| `loadProtData:replicateMismatch` (Python: `ValueError` from `load_prot_data`) | the sum of `replPerCond` differs from the number of replicate columns of the file | Correct `replPerCond` (Python: `repl_per_cond`) | -| `updateProtPool` raises an error | a usage reaction with an upper bound other than 1000 draws from the protein pool, as in every ecModel with concentration-bounded usage reactions | Calculate the f-factor and call `setProtPoolSize` | -| The growth rate that is reached is below the intended one | the enzyme levels, the $k_{cat}$ values or the exchange flux data are overconstrained | Relax the enzyme concentrations (see [Relaxing overconstrained proteomics](relaxing-constraints.md)), or loosen the flux constraints | -| `calculate_f_factor` returns 0 with a warning | none of the UniProt identifiers of the proteomics data matches an enzyme of the model | Use the same identifier namespace in `proteomics.tsv` and `ec.enzymes` | -| A complex is still limited by one subunit | the subunit is shared between complexes or catalyses a reaction on its own, and stays as measured | Check the `shared` field of the result of `levelComplexConcs` | -| The growth rate drops or the ecModel is infeasible after `constrainEnzConcs` | enzymes with a concentration of 0 (`zerosAsMeasured` true) or unmeasured isozymes limited to 0 (`zeroUnmeasuredIsozymes` with `'zero'`) have an upper bound of 0 | Load with `zerosAsMeasured` false, use `'detectionLimit'`, or relax with `relaxProteomicsGreedy` | -| A protein with a replicate of 0 is missing from `protData` or has a higher mean than in the file | the 0 is read as not measured by default | Set `zerosAsMeasured` (Python: `zeros_as_measured`) to true when the 0 is a measurement | -| `ValueError` from `constrain_enz_concs` | the ecModel is a light ecModel, or `ec.concs` has a negative value | Use a full ecModel; correct the proteomics data | - -## See also - -- [Relaxing overconstrained proteomics](relaxing-constraints.md), for an ecModel that cannot reach its intended growth rate after constraining. -- [GECKO light and full ecModels](gecko-light.md), why light ecModels cannot take proteomics data. -- [Simulation and analysis](simulation-and-analysis.md), the choice between maximising growth and minimising protein pool usage as the objective. -- [GECKO 3 to GECKO 4](../gecko3-to-gecko4.md), for the changes in `loadProtData` and `calculateFfactor`. diff --git a/docs/guide/proteomics.md b/docs/guide/proteomics.md new file mode 100644 index 0000000..587338c --- /dev/null +++ b/docs/guide/proteomics.md @@ -0,0 +1,355 @@ +# Constrain enzymes with proteomics + +Load the absolute proteomics data and the exchange fluxes of one experiment, bound the usage of each measured enzyme by its concentration, set the protein pool for the experiment, and constrain the exchange reactions and the growth rate. At the end, the ecModel describes the experiment, and solving it shows whether the ecModel reaches the measured growth rate. Proteomics data require a full ecModel, which has a usage reaction per enzyme; a light ecModel has none (see [Choose the ecModel structure](ecmodel-structure.md)). + + +:::::{dropdown} Place in the workflow + +The diagram orders the functions of this page within the [workflow](workflow.md). Boxes with a dashed outline are optional steps, a dotted arrow is an optional path, and a diamond is a choice. Rounded boxes with a dotted outline are the context, other pages or the model to start from, and a click on one opens its page. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Previous: a functional full ecModel"]):::ctx + LD["loadProtData
fillEnzConcs"] + LV["levelComplexConcs"]:::opt + CE["constrainEnzConcs"] + IZ["getIsozymeCoverage
zeroUnmeasuredIsozymes"]:::opt + FD["loadFluxData
Ptot and fluxes of the condition"] + PP["calculateFfactor (optional)
setProtPoolSize"] + FL["constrainFluxData
exchange fluxes and growth rate"] + GR{"Growth reaches the measured growth rate?"} + PV --> LD --> LV --> CE --> IZ --> FD --> PP --> FL --> GR + LD --> CE + CE --> FD + NX(["Next: Relax proteomics constraints"]):::ctx + NA(["Next: Analyse an ecModel"]):::ctx + GR -->|no| NX + GR -->|yes| NA + click PV href "first-ecmodel.html" + click NX href "relaxing-proteomics.html" + click NA href "analysis.html" +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Previous: a functional full ecModel"]):::ctx + LD["load_prot_data
fill_enz_concs"] + LV["level_complex_concs"]:::opt + CE["constrain_enz_concs"] + IZ["get_isozyme_coverage
zero_unmeasured_isozymes"]:::opt + FD["load_flux_data
Ptot and fluxes of the condition"] + PP["calculate_f_factor (optional)
set_prot_pool_size"] + FL["apply_flux_data_constraints
exchange fluxes and growth rate"] + GR{"Growth reaches the measured growth rate?"} + PV --> LD --> LV --> CE --> IZ --> FD --> PP --> FL --> GR + LD --> CE + CE --> FD + NX(["Next: Relax proteomics constraints"]):::ctx + NA(["Next: Analyse an ecModel"]):::ctx + GR -->|no| NX + GR -->|yes| NA + click PV href "first-ecmodel.html" + click NX href "relaxing-proteomics.html" + click NA href "analysis.html" +``` +::: +:::: +::::: + +## Functions + +| MATLAB | Python | Note | +| --- | --- | --- | +| `loadProtData` | `load_prot_data` | reads `proteomics.tsv`, filters it and averages the replicates of each condition | +| `fillEnzConcs` | `fill_enz_concs` | writes the concentrations of one condition to `ec.concs` | +| `levelComplexConcs` | `level_complex_concs` | gives the subunits of each complex one abundance | +| `constrainEnzConcs` | `constrain_enz_concs` | sets the concentrations as upper bounds of the `usage_prot_` reactions | +| `getIsozymeCoverage` | `get_isozyme_coverage` | lists the reactions with a measured and an unmeasured isozyme | +| `zeroUnmeasuredIsozymes` | `zero_unmeasured_isozymes` | lowers the usage bound of the unmeasured isozymes of these reactions | +| `calculateFfactor` | `calculate_f_factor` | calculates f from the proteomics data | +| `setProtPoolSize` | `set_prot_pool_size` | sets the upper bound of `prot_pool_exchange` to $P_{tot} \cdot f \cdot \sigma \cdot 1000$ mg/gDCW | +| `loadFluxData` | `load_flux_data` | reads `fluxData.tsv` | +| `constrainFluxData` | `apply_flux_data_constraints` | sets the bounds of the growth rate and the exchange reactions of one condition | + +## Setup + +A full ecModel with kcats and a protein pool (see [Build an ecModel](first-ecmodel.md)), in the variable `ecModel` (MATLAB) or `ec_model` (Python), and the files `data/proteomics.tsv` and `data/fluxData.tsv` of the project. In MATLAB, the model adapter of the project is the default adapter, and `params = ModelAdapterManager.getDefault().getParameters();` holds its parameters; in Python, `params = ec_model.adapter.params` holds them. The objective is the biomass reaction, maximised: `ecModel = setParam(ecModel, 'obj', params.bioRxn, 1);` in MATLAB, and `ec_model.objective = params.bio_rxn` with `ec_model.objective_direction = "max"` in Python (see [Basic model operations](basic-model-operations.md)). The examples assume a `proteomics.tsv` with two conditions of three replicates each, and select the first condition in both files. + +## Choose the options + +| Choice | Options | Criteria | +| --- | --- | --- | +| Complexes | level with `levelComplexConcs`, or keep each subunit as measured | Level when the subunits of a complex are measured at different amounts. | +| Unmeasured isozymes | no bound (default), a bound at the detection limit, or a bound of 0 | A protein that is missing from the data can be absent or below the detection limit. | +| f | the value in `model_adapter.toml`, or `calculateFfactor` on the proteomics of the condition | f from the same proteomics data describes the same proteome as the concentrations. | +| Exchange fluxes | `'loose'`, or a percentage around the measured value | A percentage forces a flux through each reaction. | +| Growth rate | `'max'` or `'min'` | `'max'` suits growth as objective; `'min'` suits the minimal protein pool usage as objective. | + +A complex that is bounded subunit by subunit carries no more flux than its least abundant subunit allows. Levelling uses the subunit copies in `ec.rxnEnzMat`, which come from the complex data (`applyComplexData`); without complex data, every subunit counts as one copy. + +An unmeasured isozyme without a bound can carry the flux of a measured one. A bound of 0 treats the protein as absent; a bound at the detection limit treats it as present at a low amount. + +The f in `model_adapter.toml` is often calculated from PaxDB data, or is the default 0.5 (see [Set the protein pool](protein-pool.md)). + +`'loose'` bounds each exchange reaction between 0 and the measured value. With a percentage, an error in the fluxes or in the enzyme data can make the ecModel infeasible. `'max'` sets the measured growth rate as upper bound and `'min'` as lower bound (see [Analyse an ecModel](analysis.md) for the objectives). + +Start with `'loose'` and `'max'`. The ecModel then has a solution also when the proteomics limit growth, which `flexibilizeEnzConcs` and `relaxProteomicsGreedy` need (see [Relax proteomics constraints](relaxing-proteomics.md)). With `'min'`, an ecModel that cannot reach the growth rate is infeasible. + +## Prepare the data files + +`data/proteomics.tsv` is tab-delimited with one header row. The first column holds the UniProt identifiers, and every following column holds one replicate, in mg/gDCW. The replicates of the first condition come first, then those of the second condition, and so on. The header must have one column more than the total number of replicates; otherwise `loadProtData` raises the error `loadProtData:replicateMismatch` (Python: `ValueError`). A cell with `NA`, `na`, `NaN`, `#VALUE!` or no value is not measured, and a row without an identifier is removed. Relative proteomics data cannot be used: a concentration in another unit gives wrong bounds and no error. + +`data/fluxData.tsv` is tab-delimited with one header row and one row per condition. The columns are the condition name, the total protein content Ptot (g/gDCW), the growth rate (1/h), and then one column per exchange reaction, in mmol/gDCW/h, negative for uptake and positive for excretion. The header of an exchange column has the form ` ()`. Optional columns `evotuneRMSEweight` and `source` can be at any position. A line with a different number of cells than the header raises an error. The example shows 7 of the 11 columns (the first six and the last one) of `fluxData.tsv` in the `full_ecModel` tutorial of GECKO, which uses yeast-GEM; `r_1714` is the glucose exchange of yeast-GEM: + +```text +Condition Ptot grRate glucose (r_1714) CO2 (r_1672) oxygen (r_1992) ethanol (r_1761) +Std 0.4228 0.1 -1.1 2.7 -2.5 1.00E-05 +``` + +The `proteomics.tsv` of the same tutorial has one condition with three replicates, so it is loaded with `loadProtData(3)`; the `[3, 3]` of the examples below raises `loadProtData:replicateMismatch` on it. See [Files in the data folder](data-folder-files.md) for the other files of the project. + +## Load the proteomics data + +`loadProtData` takes the number of replicates of each condition, in the order of the columns. It returns one column per condition. With the default settings, it filters each condition in this order: + +| MATLAB | Python | Default | Effect | +| --- | --- | --- | --- | +| `maxMissing` | `max_missing` | 2/3 | a protein measured in fewer than this fraction of the replicates is removed; a vector gives one value per condition | +| `maxRSD` | `max_rsd` | 1 | a protein with a higher relative standard deviation over the replicates is removed | +| `addStdevs` | `add_stdevs` | 1 | the concentration is the mean of the replicates plus this number of standard deviations | +| `minVal` | `min_val` | 0 | a protein with a lower concentration is removed | +| `cutLowest` | `cut_lowest` | 5 | this percentage of the lowest concentrations is removed | +| `filterData` | `filter_data` | true | false skips the filters above and returns the mean of the replicates | +| `zerosAsMeasured` | `zeros_as_measured` | false | whether a 0 is a measurement | + +A protein that is removed is NaN for that condition, and its enzyme draws from the protein pool without a bound of its own. By default a 0 is read as not measured, because proteomics reports a protein below the detection limit as 0 or as missing. With `zerosAsMeasured` true, a 0 counts as a measured replicate, and a protein measured as 0 in every replicate keeps the concentration 0. `constrainEnzConcs` then sets the usage bound of its enzyme to 0, which removes every flux through the enzyme. Set it to true when a 0 means that the protein is absent, for example in a deletion strain. + +`fillEnzConcs` writes the concentrations of one condition to `ec.concs` and sets every other enzyme to NaN. The condition index starts at 1 in MATLAB and at 0 in Python. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +% two conditions with three replicates each +protData = loadProtData([3, 3]); +ecModel = fillEnzConcs(ecModel, protData, 'dataCol', 1); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import fill_enz_concs, load_prot_data + +params = ec_model.adapter.params +# two conditions with three replicates each +prot_data = load_prot_data(params.path / "data" / "proteomics.tsv", [3, 3]) +fill_enz_concs(ec_model, prot_data, data_col=0) +``` +::: +:::: + +## Level the concentrations of complexes + +`levelComplexConcs` runs after `fillEnzConcs` and before `constrainEnzConcs`. It groups the enzymes of the reactions with several subunits into complexes (`getComplexFamilies`), converts the concentrations to molar amounts with `ec.mw`, and writes the median over the measured subunits, per copy, back to every subunit of the complex, multiplied by its copies. The option `statistic` selects `'median'` or `'mean'`. The option `families` takes the output of `getComplexFamilies` (Python: `complex_families`), for a grouping with other values of `minShared` and `minOverlap` (Python: `min_shared`, `min_overlap`). With `fillMissing` true (default), an unmeasured subunit of a complex receives the concentration of the complex. A subunit in two complexes belongs to the complex whose enzyme sets catalyse more reactions. A subunit in three or more complexes, in two complexes of equal weight, or that also catalyses a reaction alone keeps its measured value, and a subunit measured as 0 stays 0. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +[ecModel, levelResult] = levelComplexConcs(ecModel); +``` + +`levelResult` has the fields `adjusted` (measured subunits that received the concentration of the complex), `added` (unmeasured subunits that received it), `shared` (subunits kept as measured) and `skipped` (complexes without a measured subunit). +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import level_complex_concs + +level_result = level_complex_concs(ec_model) +``` + +`level_complex_concs` changes `ec_model.ec.concs` and returns a `LevelComplexResult` with the attributes `adjusted`, `added`, `shared` and `skipped`. +::: +:::: + +An ecModel with levelled complexes is relaxed per complex (see [Relax proteomics constraints](relaxing-proteomics.md)). + +## Apply the concentrations as bounds + +`constrainEnzConcs` sets the upper bound of the `usage_prot_` reaction of each enzyme with a concentration to that concentration, and the upper bound of each enzyme with NaN to 1000. Every usage reaction draws from the protein pool, also when it has a measured bound. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +ecModel = constrainEnzConcs(ecModel); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import constrain_enz_concs + +constrain_enz_concs(ec_model) +``` +::: +:::: + +The option `removeConstraints` (Python: `remove_constraints`) resets all usage bounds to 1000 and leaves `ec.concs` unchanged. The option `restrictTo` (Python: `restrict_to`) takes a list of UniProt identifiers; the usage reactions of the other enzymes keep their bounds. + +## Limit unmeasured isozymes + +A reaction can have a measured isozyme and an unmeasured one. The unmeasured isozyme then draws from the protein pool without a bound, and the ecModel can use it in place of the measured one. `getIsozymeCoverage` lists these reactions and does not change the ecModel. `zeroUnmeasuredIsozymes` runs after `constrainEnzConcs`, calls `getIsozymeCoverage`, and lowers the usage bound of the enzymes of the unmeasured isozymes. The method `'zero'` (default) sets the bound to 0; `'detectionLimit'` (Python: `"detection_limit"`) sets it to `limit` in mg/gDCW, by default the smallest concentration above 0 in `ec.concs`. An enzyme that also catalyses a reaction without a measured isozyme is not changed. `ec.concs` is not changed, and no other GECKO function calls `zeroUnmeasuredIsozymes`. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +[coverage, zeroable] = getIsozymeCoverage(ecModel); +[ecModel, changed] = zeroUnmeasuredIsozymes(ecModel, 'method', 'detectionLimit'); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import get_isozyme_coverage, zero_unmeasured_isozymes + +coverage, zeroable = get_isozyme_coverage(ec_model) +changed = zero_unmeasured_isozymes(ec_model, method="detection_limit") +``` +::: +:::: + +`coverage` has one row per reaction, with its measured and unmeasured enzymes. `changed` has one row per enzyme with its new upper bound. + +## Set the protein pool for the condition + +The upper bound of the protein pool exchange is $P_{tot} \cdot f \cdot \sigma \cdot 1000$ mg/gDCW (see [Set the protein pool](protein-pool.md)). Use the Ptot of the condition from `fluxData.tsv`. `calculateFfactor` divides the summed concentrations of the enzymes of the ecModel by the summed concentrations of all proteins in the data. It uses the proteins that `loadProtData` kept, so the filters change f. To calculate f over all measured proteins of one condition, load the data without filters and select the column of the condition. With several columns, `calculateFfactor` averages each protein over the columns in which it is measured. Sigma is read from `model_adapter.toml` when it is not given. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +fluxData = loadFluxData(); +protAll = loadProtData([3, 3], 'filterData', false); +protAll.abundances = protAll.abundances(:, 1); +f = calculateFfactor(ecModel, 'protData', protAll); +ecModel = setProtPoolSize(ecModel, 'Ptot', fluxData.Ptot(1), 'f', f); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import ( + ProtData, calculate_f_factor, load_flux_data, set_prot_pool_size, +) + +flux_data = load_flux_data(params.path / "data" / "fluxData.tsv") +prot_all = load_prot_data( + params.path / "data" / "proteomics.tsv", [3, 3], filter_data=False, +) +prot_cond = ProtData( + uniprot_ids=prot_all.uniprot_ids, abundances=prot_all.abundances[:, [0]], +) +f = calculate_f_factor(ec_model, prot_cond) +set_prot_pool_size(ec_model, p_tot=float(flux_data.p_tot[0]), f=f) +``` +::: +:::: + +## Constrain the exchange fluxes and the growth rate + +`constrainFluxData` sets the bounds of the carbon source uptake reaction of `model_adapter.toml` (`c_source`) to 0, then sets the growth rate and the exchange fluxes of the condition. The condition is an index (from 1 in MATLAB, from 0 in Python) or a name from the first column of `fluxData.tsv`. The proteomics column and the flux condition are selected separately; select the same experiment in both. + +- With `'loose'`, a negative flux becomes the lower bound with an upper bound of 0, and a positive flux becomes the upper bound with a lower bound of 0. A flux of 0 blocks the reaction. +- With a percentage `p`, the bounds are the measured value Γ— (1 Β± p/200); 10 gives 95 % and 105 % of the measured value. +- A flux of -1000 allows any uptake and 1000 allows any excretion, with both options. +- An empty cell leaves the bounds of the reaction unchanged. +- An exchange reaction that is not in the ecModel raises an error. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +ecModel = constrainFluxData(ecModel, 'fluxData', fluxData, 'condition', 1, ... + 'maxMinGrowth', 'max', 'looseStrictFlux', 'loose'); +sol = solveLP(ecModel); +fprintf('Growth rate: %g /h, measured: %g /h\n', sol.f, fluxData.grRate(1)) +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import apply_flux_data_constraints + +apply_flux_data_constraints( + ec_model, flux_data, condition=0, + max_min_growth="max", loose_strict_flux="loose", +) +growth = ec_model.slim_optimize() +print(f"Growth rate: {growth:g} /h, measured: {flux_data.gr_rate[0]:g} /h") +``` +::: +:::: + +With `'max'`, the growth rate cannot exceed the measured one. A growth rate below it means that the enzyme concentrations, the kcats, the protein pool or the exchange fluxes limit growth. [Relax proteomics constraints](relaxing-proteomics.md) raises the limiting enzyme bounds and shows how to find which of these limits growth. + +## Troubleshooting + +| Problem | Possible reason | Solution | +| --- | --- | --- | +| `loadProtData:replicateMismatch` (Python: `ValueError` from `load_prot_data`) | the replicate numbers do not add up to the number of replicate columns of `proteomics.tsv` | Correct `replPerCond` (Python: `repl_per_cond`) or the header of the file | +| `fillEnzConcs:negativeAbundance` (Python: `ValueError` from `fill_enz_concs`) | an enzyme of the ecModel has a negative concentration in the selected condition | Correct `proteomics.tsv` | +| `IndexError` from `fill_enz_concs` | `data_col` is outside the conditions of `prot_data`; it counts from 0 | Use a column index from 0 to the number of conditions minus 1 | +| `calculateFfactor` returns 0 (Python: with a warning) | the identifiers in `proteomics.tsv` do not match `ec.enzymes` | Use UniProt identifiers in `proteomics.tsv`, as in `ec.enzymes` | +| A protein with a replicate of 0 is missing or has a higher mean than expected | a 0 is read as not measured | Set `zerosAsMeasured` (Python: `zeros_as_measured`) to true if the 0 is a measurement | +| The ecModel is infeasible or grows much slower after `constrainEnzConcs` or `zeroUnmeasuredIsozymes` | enzymes with a bound of 0: measured zeros, or isozymes set to 0 | Load with `zerosAsMeasured` false, use `'detectionLimit'`, or relax with `relaxProteomicsGreedy` | +| A complex is still limited by one subunit after levelling | the subunit is in three or more complexes, in two complexes of equal weight, or catalyses a reaction alone, and keeps its measured value | Check the field `shared` of the result of `levelComplexConcs` | +| `constrainFluxData:missingReaction` (Python: `ValueError`) | a reaction identifier in the header of `fluxData.tsv` is not in the ecModel | Correct the header to ` ()` | +| `ValueError` from `constrain_enz_concs` | the ecModel is a light ecModel, or `ec.concs` has a negative value | Use a full ecModel; correct the proteomics data | +| `updateProtPool` raises an error | `updateProtPool` is deprecated and does not accept an ecModel in which measured enzymes draw from the protein pool | Use `setProtPoolSize` with the Ptot and f of the condition | + +## See also + +- [Relax proteomics constraints](relaxing-proteomics.md), for an ecModel that does not reach the measured growth rate. +- [Set the protein pool](protein-pool.md), for Ptot, f and sigma. +- [Analyse an ecModel](analysis.md), for the objectives that go with `'max'` and `'min'`. +- [Files in the data folder](data-folder-files.md). diff --git a/docs/guide/relaxing-constraints.md b/docs/guide/relaxing-constraints.md deleted file mode 100644 index c4ea008..0000000 --- a/docs/guide/relaxing-constraints.md +++ /dev/null @@ -1,257 +0,0 @@ -# Relaxing overconstrained proteomics - -Constraining individual enzyme concentrations from [proteomics integration](proteomics-integration.md) can leave the ecModel unable to reach its intended growth rate: measurement error in the proteomics data and uncertainty in the $k_{cat}$ values compound, and the model is more constrained than the cell. The usage bounds of the enzymes that limit growth are raised until the target growth rate is reachable. Two algorithms in both toolboxes and a third in geckopy answer the same question (which enzyme bounds are relaxed, and by how much), and they differ in the result they give. - -:::{warning} -All functions below apply to full ecModels; see [GECKO light and full ecModels](gecko-light.md). -::: - - -:::::{dropdown} Place in the workflow - -The diagram orders the functions of this page within the [workflow](workflow.md). Dashed boxes are optional and dotted boxes are neighbouring pages. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - PV(["Previous: Proteomics integration"]):::ctx - PV --> D{"Growth reaches the target?"} - D -->|no, gradual| FX["flexibilizeEnzConcs"] - D -->|no, fast| GR["relaxProteomicsGreedy"] - NX(["Next: Simulation and analysis"]):::ctx - D -->|yes| NX - FX --> NX - GR --> NX -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - PV(["Previous: Proteomics integration"]):::ctx - PV --> D{"Growth reaches the target?"} - D -->|no, gradual| FX["flexibilize_enz_concs"] - D -->|no, fast| GR["relax_proteomics_greedy"] - D -->|no, smallest total amount| SA["relax_enz_concs"] - NX(["Next: Simulation and analysis"]):::ctx - D -->|yes| NX - FX --> NX - GR --> NX - SA --> NX -``` -::: -:::: -::::: - -## Functions - -| MATLAB | Python | Note | -|---|---|---| -| `flexibilizeEnzConcs` | `flexibilize_enz_concs` | gradual relaxation ranked by control coefficient, with a tighten-back pass | -| `relaxProteomicsGreedy` | `relax_proteomics_greedy` | single-shot relaxation ranked by shadow price | -| not available | `relax_enz_concs` | smallest total relaxation that reaches the target, found in one LP | -| `getComplexFamilies` | `complex_families` | groups the enzymes of multi-subunit reactions into complexes | - -## Setup - -The ecModel has `ec.concs` filled and applied as bounds on the `usage_prot_` reactions (`fillEnzConcs` and `constrainEnzConcs`, see [Proteomics integration](proteomics-integration.md)), and an intended growth rate is known. An enzyme with a concentration of 0 (see [Measured zeros](proteomics-integration.md#measured-zeros-and-unmeasured-enzymes)) is not raised by `flexibilizeEnzConcs`, because the upper bound is raised as a factor of the concentration. The function warns and lists the enzymes (MATLAB warning `flexibilizeEnzConcs:measuredZero`, Python: logged warning). `relaxProteomicsGreedy` treats such an enzyme as any other enzyme with a bound below the default and can set its bound back to the default. `flexibilizeEnzConcs` reads the growth rate from `gR_exp` of the model adapter when `expGrowth` is not given (Python: `gr_exp` and `exp_growth`). - -## Choice of algorithm - -`flexibilizeEnzConcs` / `flexibilize_enz_concs` follows the procedure of the published GECKO pipeline. An iteration computes the control coefficient (the change in growth rate over the change in the enzyme concentration) of every measured enzyme whose usage runs at more than 75 % of its upper bound, and raises the usage upper bound of the enzyme with the largest coefficient. On the k-th selection of the same enzyme the upper bound is `original_concentration * (1 + fold_change * k)`. The loop ends when the target growth rate is reached. A tighten-back pass then finds the flux distribution that minimises protein pool usage at the target growth rate (within a band of 0.25 % growth either side), sets each relaxed bound to the usage in that distribution, and restores the original concentration where the usage is lower. The pass is skipped when the loop stopped on an iteration limit, and the target growth rate is then not reached. The result stays as close to the measured proteomics as the target growth rate allows. Computing the control coefficients re-solves the LP once per candidate enzyme, on every iteration. - -`relaxProteomicsGreedy` / `relax_proteomics_greedy` is not part of the published GECKO pipeline (see [GECKO 3 to GECKO 4](../gecko3-to-gecko4.md)). It solves the model once, reads the shadow price of every `prot_` mass-balance constraint from the LP dual, selects the still-constrained enzyme with the largest absolute shadow price and raises its usage upper bound to the default of 1000 in one step. It repeats until the target growth rate is reached. There is no tighten-back pass, so it tends to relax more than the target requires. Each step costs one solve; the control-coefficient algorithm costs one solve per candidate, which matters when one or two enzymes dominate the infeasibility. - -| Aspect | `flexibilizeEnzConcs` / `flexibilize_enz_concs` | `relaxProteomicsGreedy` / `relax_proteomics_greedy` | -|---|---|---| -| Selection signal | largest control coefficient (one re-solve per candidate) | largest absolute shadow price (read from one solve) | -| Relaxation step | gradual: the usage upper bound moves by a fold change that compounds on repeated selection of the same enzyme | the usage upper bound is set to the default (1000) | -| Tighten-back | yes, a final pass drops each relaxation that the flux distribution does not need | no, every relaxed enzyme stays unconstrained | -| Result stringency | tight, close to the proteomics data | loose, tends to relax more than necessary | -| Relaxes the protein pool itself | yes, when no single enzyme helps further | no | -| Non-convergence | returns a partial result with a warning | returns a partial result with a warning when the eligible candidates run out first; raises an error when the iteration cap is reached while candidates remain | - -`flexibilizeEnzConcs` / `flexibilize_enz_concs` keeps the result closest to the measured proteomics. `relaxProteomicsGreedy` / `relax_proteomics_greedy` gives a fast answer when the infeasibility is dominated by one or two enzymes, and its result is a looser bound on what the data support. - -## Gradual relaxation with tighten-back - -The usage upper bound of the enzyme with the largest control coefficient is raised, with a fold change of 10 per iteration, until the intended growth rate (0.1 /h) is reached or no enzyme limits growth any more. The tighten-back pass then finds the flux distribution that minimises protein pool usage at the target growth rate (within a band of 0.25 % growth either side), sets each raised bound to the usage in that distribution, and restores the original concentration where the usage is lower. When the loop stopped on an iteration limit, the pass is skipped, the relaxed bounds stay and the target growth rate is not reached. The call is: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -[ecModel, flexEnz] = flexibilizeEnzConcs(ecModel, 0.1, 10); -``` - -The relaxed levels are bounds of the affected `usage_prot_` reactions. `ecModel.ec.concs` is unchanged and keeps the measured values from `fillEnzConcs`. `flexEnz` has the fields `uniprotIDs`, `oldConcs`, `flexConcs`, `ratioIncr` and `frequence`, sorted by descending `ratioIncr`. The entry `prot_pool` is added when the protein pool exchange was relaxed. - -Output in the `full_ecModel` tutorial, whose proteomics and flux data reach 0.0129 /h before the relaxation (the last lines; eight enzymes are relaxed): - -``` -Protein P46367 UB adjusted. Grow: 0.032826 -No (more) limiting enzymes have been found. Attempting to increase protein pool exchange... -Protein pool exchange was also not limiting. Inability to reach growth rate is not related to -enzyme constraints. Maximum growth rate is 0.032826. -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import flexibilize_enz_concs - -flex_result = flexibilize_enz_concs(ec_model, exp_growth=0.1, fold_change=10.0) -``` - -`flexibilize_enz_concs` mutates `ec_model` in place and returns a `FlexEnzResult` with the parallel arrays `uniprot_ids`, `old_concs`, `flex_concs`, `ratio_incr` and `frequence`, the equivalent of `flexEnz` in MATLAB. `ec_model.ec.concs` is unchanged. In the `full_ecModel` tutorial, the function relaxes eight enzymes, logs `protein pool was not limiting either. Maximum growth reachable: 0.0328262` and leaves the model at 0.0327 /h, the same values as in MATLAB. -::: -:::: - -The enzymes in the result are the modified ones. Flexibilizing a concentration is a response to imprecise protein measurement. When the same enzyme is flexibilized repeatedly, its $k_{cat}$ may cause the shortfall; check the $k_{cat}$ and consider a custom value (see [Applying kcat values](applying-kcats.md) and [Tuning against experimental data](tuning-against-experimental-data.md)). An enzyme that is selected more often than `iterPerEnzyme` (Python: `iter_per_enzyme`, default 5) stops the loop with a warning. - -:::{admonition} Many enzymes flexibilized -:class: tip - -If the result contains many enzymes, lower-quality proteomics data is the likely cause, and constraining individual enzyme levels sharply reduces the solution space. The total protein amount should stay realistic, and low-quality concentrations should be filtered out before loading (`loadProtData` / `load_prot_data` has several filtering parameters, such as a maximum relative standard deviation). With stricter filtering, the highest-certainty enzyme levels are constrained directly, and the other enzymes draw from the overall protein pool. -::: - -### Target growth rate beyond the enzyme constraints - -Flexibilization does not always reach the target growth rate. When no limiting enzyme is left, the function raises the protein pool exchange and, if the growth rate does not rise either, reports the maximum growth rate that it reached. The shortfall is then not caused by the enzyme constraints. Applying the same exchange flux constraints to the conventional GEM, with no enzyme constraints, confirms this: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -model = constrainFluxData(model, fluxData); -sol = solveLP(model); -fprintf('Growth rate that is reached: %f /hour\n', abs(sol.f)) -``` - -Output in the `full_ecModel` tutorial: - -``` -Growth rate that is reached: 0.088037 /hour -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import apply_flux_data_constraints - -apply_flux_data_constraints( - model, flux_data, bio_rxn=params.bio_rxn, c_source=params.c_source, -) -sol = model.optimize() -print(f"Growth rate that is reached: {sol.objective_value:.4f} /hour") -``` -::: -:::: - -If the conventional GEM reaches the same ceiling, the exchange flux data are the overconstrained part; see [Proteomics integration](proteomics-integration.md) for the loose and strict choice on those constraints. - -## Relaxation of complexes - -When the concentrations were levelled with `levelComplexConcs` / `level_complex_concs` (see [Proteomics integration](proteomics-integration.md)), all subunits of a complex have one abundance and bind at once, so raising a single subunit does not raise growth. The option `perComplex` of `flexibilizeEnzConcs` (Python: `per_complex`, default false) treats every complex, as grouped by `getComplexFamilies` / `complex_families`, as one enzyme. Enzymes and complexes are ranked by the growth gained when all their usage upper bounds are raised together, every measured subunit of the selected complex is raised by the same factor, iterations are counted per complex, and the final pass keeps one factor per complex, which keeps levelled concentrations levelled. An upper bound is never lowered below its current value. The number of times that a complex was selected is recorded in `frequence` on its first subunit with a concentration above 0. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -[ecModel, flexEnz] = flexibilizeEnzConcs(ecModel, 'expGrowth', 0.1, ... - 'foldChange', 10, 'perComplex', true); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import flexibilize_enz_concs - -flex_result = flexibilize_enz_concs( - ec_model, exp_growth=0.1, fold_change=10.0, per_complex=True, -) -``` -::: -:::: - -## Single-shot shadow-price relaxation - -The usage bound of the enzyme with the largest absolute shadow price is raised to the default and the model is solved again, until the target growth rate is reached. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -result = relaxProteomicsGreedy(ecModel, 'minimalGrowth', 0.1); -``` - -`result.trace` has one row per relaxation step (`iteration`, `relaxedUniprot`, `growthBefore`, `growthAfter`, `shadowPrice`). `result.relaxed` maps each relaxed enzyme to its original `ecModel.ec.concs` value, which allows restoring it. `result.converged` is true when `result.finalGrowth` reaches `minimalGrowth`. -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import relax_proteomics_greedy - -result = relax_proteomics_greedy(ec_model, minimal_growth=0.1) -``` - -Returns a `GreedyRelaxResult` with `.trace`, `.relaxed`, `.final_growth` and `.converged`. -::: -:::: - -## Relaxation by the smallest total amount - -`relax_enz_concs` has no MATLAB counterpart. It sets the cap of every measured enzyme to `usage <= conc * (1 + x)` and finds, in one LP, the values of `x` with the smallest sum under the constraint that growth reaches `exp_growth`. The protein pool exchange may rise by a fold at `pool_weight` times the price of an enzyme fold. With `weight="relative"` (default) the summed fold increases are minimised; with `weight="absolute"` the mg/gDCW added. With `per_complex=True` the measured subunits of a complex share one `x`. It returns a `FlexEnzResult` as `flexibilize_enz_concs` does, with `frequence` 1 for every entry, and raises `ValueError` when no relaxation reaches `exp_growth`. - -```python -from geckopy import relax_enz_concs - -flex_result = relax_enz_concs(ec_model, exp_growth=0.1) -``` - -## Troubleshooting - -| Problem | Possible reason | Solution | -|---|---|---| -| `relaxProteomicsGreedy` returns `converged = false` with a warning (Python: `converged=False`) | the eligible enzymes ran out before the target growth rate was reached | Apply the exchange flux constraints to the conventional GEM; the target may be unreachable for reasons other than enzyme constraints | -| `relaxProteomicsGreedy` raises an error (Python: `RuntimeError`) | `maxIterations` (Python: `max_iterations`, default 100) was reached while candidates remained | Raise `maxIterations`, or use `flexibilizeEnzConcs` | -| The ecModel is less constrained than the proteomics data require | `relaxProteomicsGreedy` has no tighten-back pass | Use `flexibilizeEnzConcs` / `flexibilize_enz_concs` | -| Warning that an enzyme or complex reached its iteration limit (MATLAB: "seems to be problematic") | the same enzyme or complex was selected more often than `iterPerEnzyme` allows | Check the $k_{cat}$ of the enzyme | -| `flexibilizeEnzConcs` warns about enzymes with a concentration of 0 and the growth rate is not reached | the enzymes have an upper bound of 0 that is never raised | Use `relaxProteomicsGreedy`, or load the proteomics data with `zerosAsMeasured` false | -| `flexibilize_enz_concs` raises `ValueError` | no enzyme has a measured concentration, or `exp_growth` is not given and `gr_exp` is not set | Run `fill_enz_concs` and `constrain_enz_concs`, or pass `exp_growth` | - -## See also - -- [Proteomics integration](proteomics-integration.md), where the constraints that are relaxed here come from. -- [Enzyme usage and bottlenecks](enzyme-usage-and-bottlenecks.md), which ranks capacity bottlenecks and leaves the constraints unchanged. -- [Applying kcat values](applying-kcats.md), for a repeatedly flexibilized enzyme that points to a $k_{cat}$ problem. diff --git a/docs/guide/relaxing-proteomics.md b/docs/guide/relaxing-proteomics.md new file mode 100644 index 0000000..1773115 --- /dev/null +++ b/docs/guide/relaxing-proteomics.md @@ -0,0 +1,298 @@ +# Relax proteomics constraints + +Raise the usage bounds of the enzymes that keep an ecModel constrained with proteomics below the measured growth rate, until it reaches that growth rate. The result is an ecModel that reaches the growth rate, a list of the enzymes whose bounds were raised and by how much, and an indication of whether the shortfall comes from the proteomics data, the kcats, the protein pool or the metabolic network. + + +:::::{dropdown} Place in the workflow + +The diagram orders the functions of this page within the [workflow](workflow.md). Boxes with a dashed outline are optional steps, a dotted arrow is an optional path, and a diamond is a choice. Rounded boxes with a dotted outline are the context, other pages or the model to start from, and a click on one opens its page. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Previous: Constrain enzymes with proteomics"]):::ctx + GR{"Growth reaches the measured growth rate?"} + FX["flexibilizeEnzConcs (recommended)
perComplex after levelling"] + RG["relaxProteomicsGreedy"] + RE["relaxEnzConcs
compare with flexibilizeEnzConcs"] + IN["inspect the result"] + GM["constrainFluxData on the conventional GEM
when no method reaches the target"]:::opt + PV --> GR + GR -->|no, keep the measured bounds| FX --> IN + GR -->|no, enzymes at 0 limit growth| RG --> IN + GR -->|no, independent of the order| RE --> IN + IN -.-> GM + NX(["Next: Analyse an ecModel"]):::ctx + GR -->|yes| NX + IN --> NX + click PV href "proteomics.html" + click NX href "analysis.html" +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```{mermaid} +:align: left + +--- +config: + flowchart: + useMaxWidth: false +--- +flowchart TB + classDef opt stroke-dasharray: 5 4 + classDef ctx fill:transparent,stroke-dasharray: 2 3 + PV(["Previous: Constrain enzymes with proteomics"]):::ctx + GR{"Growth reaches the measured growth rate?"} + FX["flexibilize_enz_concs (recommended)
per_complex after levelling"] + RG["relax_proteomics_greedy"] + RE["relax_enz_concs
compare with flexibilize_enz_concs"] + IN["inspect the result"] + GM["apply_flux_data_constraints on the conventional GEM
when no method reaches the target"]:::opt + PV --> GR + GR -->|no, keep the measured bounds| FX --> IN + GR -->|no, enzymes at 0 limit growth| RG --> IN + GR -->|no, independent of the order| RE --> IN + IN -.-> GM + NX(["Next: Analyse an ecModel"]):::ctx + GR -->|yes| NX + IN --> NX + click PV href "proteomics.html" + click NX href "analysis.html" +``` +::: +:::: +::::: + +## Functions + +| MATLAB | Python | Note | +| --- | --- | --- | +| `flexibilizeEnzConcs` | `flexibilize_enz_concs` | raises the bound of the most limiting enzyme step by step, then lowers the bounds that the target does not need | +| `relaxProteomicsGreedy` | `relax_proteomics_greedy` | removes the bound of the enzyme with the largest shadow price, one enzyme at a time | +| `relaxEnzConcs` | `relax_enz_concs` | finds the smallest total relaxation in one linear program | +| `constrainFluxData` | `apply_flux_data_constraints` | applies the same exchange fluxes to the conventional GEM, for comparison | + +## Setup + +An ecModel with concentrations applied as bounds and the exchange fluxes of the same condition, as made in [Constrain enzymes with proteomics](proteomics.md). `flexibilizeEnzConcs` and `relaxProteomicsGreedy` start from a solution of the ecModel, so it must be feasible: apply the growth rate with `'max'`. The target growth rate is the measured growth rate of the condition. Pass it explicitly: by default `flexibilizeEnzConcs` reads `gr_exp` from `model_adapter.toml`, which is the maximum growth rate and can differ from the growth rate of the condition. + +## Choose a method + +| Method | Change to a bound | Enzymes it can raise | Solves | Returns | +| --- | --- | --- | --- | --- | +| `flexibilizeEnzConcs` | raised in steps, then lowered to the usage that the target needs | enzymes with a concentration above 0 | one per candidate enzyme on every iteration | the ecModel and `flexEnz` | +| `flexibilizeEnzConcs` with `perComplex` | as above, with one factor per complex | as above | one per iteration; one per candidate complex when the solver gives no reduced costs | the ecModel and `flexEnz` | +| `relaxProteomicsGreedy` | set to 1000, one enzyme per step | every enzyme with a usage bound below 1000 | one per relaxed enzyme | `result` and the ecModel | +| `relaxEnzConcs` | raised by the smallest sum of folds, in one linear program | enzymes with a concentration, including 0 | one | the ecModel and `flexEnz` | + +`flexibilizeEnzConcs` raises a bound as a factor of the concentration, so an enzyme measured as 0 stays at 0; the function warns and lists these enzymes. An isozyme lowered by `zeroUnmeasuredIsozymes` has no concentration in `ec.concs`, and `relaxProteomicsGreedy` is the method that raises it. `flexibilizeEnzConcs` and `relaxEnzConcs` can raise the protein pool when enzymes do not suffice; `relaxProteomicsGreedy` leaves the pool unchanged. + +Use `flexibilizeEnzConcs`, with `perComplex` when the concentrations were levelled with `levelComplexConcs`. Its result keeps the measured bound of every enzyme that the final solution does not need to raise. Use `relaxProteomicsGreedy` when enzymes with a bound of 0 limit growth; each enzyme that it relaxes has no bound of its own afterwards. `relaxEnzConcs` gives a result that does not depend on the order in which enzymes become limiting; compare it with the result of `flexibilizeEnzConcs`. Run each method on the same constrained ecModel: the examples keep `ecModel` (Python: `ec_model`) unchanged and relax a copy. An enzyme that both methods raise is limiting in the data; a conclusion that depends on an enzyme that one method raises and the other does not is uncertain. + +## Relax with flexibilizeEnzConcs + +On each iteration, `flexibilizeEnzConcs` computes the control coefficient (growth gained per mg/gDCW added) of every measured enzyme whose usage is above 75 % of its bound. It raises the bound of the enzyme with the largest coefficient to concentration Γ— (1 + `foldChange` Γ— k) on its k-th selection, or keeps the current bound when that is higher. With the default `foldChange` of 2, the bound is 3, 5, 7, 9 and 11 times the concentration on the first to fifth selection. + +The loop stops in one of four cases: + +- The target growth rate is reached. +- No enzyme limits growth. The function then raises the upper bound of `prot_pool_exchange` to the usage needed for the highest growth rate that the pool allows, and this growth rate becomes the target. +- An enzyme is selected more than `iterPerEnzyme` times (default 5; 0 removes the limit). The loop stops and prints a message (MATLAB: a printed line that starts with `WARNING:`, not a MATLAB warning; Python: a logged warning), and the target is not reached. +- The total number of iterations exceeds (`iterPerEnzyme` + 1) times the number of measured enzymes, with 1000 for `iterPerEnzyme` when it is 0. The loop stops and prints such a message, and the target is not reached. + +In the first two cases, a final pass minimises the protein pool usage at the target growth rate (Β± 0.25 %), sets each raised bound to the usage in that solution, and restores the measured concentration where the usage is lower. The function also raises the upper bound of the biomass reaction to the target when it is lower. `ec.concs` keeps the measured values. + +With `perComplex`, the measured subunits of a complex form one unit. Units are ranked by the reduced costs of their usage reactions, and the subunits of a unit are raised and lowered by the same factor, so levelled concentrations stay levelled. The limit of `iterPerEnzyme` counts per complex. + +When the ecModel has no solution before the relaxation, the function warns (MATLAB: `flexibilizeEnzConcs:infeasible`) and returns an empty result (MATLAB: a `flexEnz` structure with empty fields). + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +[flexModel, flexEnz] = flexibilizeEnzConcs(ecModel, 'expGrowth', fluxData.grRate(1)); + +% with complexes levelled by levelComplexConcs +[flexModel, flexEnz] = flexibilizeEnzConcs(ecModel, 'expGrowth', fluxData.grRate(1), ... + 'perComplex', true); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import flexibilize_enz_concs + +flex_model = ec_model.copy() +flex_result = flexibilize_enz_concs(flex_model, exp_growth=float(flux_data.gr_rate[0])) + +# with complexes levelled by level_complex_concs +flex_model = ec_model.copy() +flex_result = flexibilize_enz_concs( + flex_model, exp_growth=float(flux_data.gr_rate[0]), per_complex=True, +) +``` + +The copy `flex_model` is changed in place, and `ec_model` keeps the bounds from the proteomics. The progress messages are logged at the level INFO; `logging.basicConfig(level=logging.INFO)` shows them. +::: +:::: + +## Relax with relaxProteomicsGreedy + +`relaxProteomicsGreedy` solves the ecModel, takes the enzyme with the largest absolute shadow price of its `prot_` metabolite among the enzymes with a usage bound below 1000, sets that bound to 1000, and solves again. It stops in one of three cases: + +- The target is reached: `converged` is true. +- No candidate is left: a warning, and `converged` is false. +- `maxIterations` (default 100) relaxations are made, the target is not reached and candidates are left: an error. + +`enzymeSet` (Python: `enzyme_set`) limits the candidates to a list of UniProt identifiers. The MATLAB function returns `result` and the relaxed ecModel; the Python function changes the ecModel in place and returns `result`. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +[result, greedyModel] = relaxProteomicsGreedy(ecModel, 'minimalGrowth', fluxData.grRate(1)); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import relax_proteomics_greedy + +greedy_model = ec_model.copy() +result = relax_proteomics_greedy(greedy_model, minimal_growth=float(flux_data.gr_rate[0])) +``` +::: +:::: + +## Relax with relaxEnzConcs + +`relaxEnzConcs` writes the bound of every enzyme with a concentration as b Γ— (1 + x), where b is the current upper bound (the concentration, or a higher bound from an earlier relaxation), and minimises the sum of the x values under growth β‰₯ target, in one linear program. A bound raised earlier is therefore never lowered. + +- The protein pool can be raised too, at `poolWeight` (Python: `pool_weight`, default 1000) times the price of an enzyme. +- `weight` `'absolute'` minimises the mg/gDCW added; the default `'relative'` minimises the sum of the folds. +- `perComplex` (Python: `per_complex`) gives the measured subunits of a complex one x. +- An enzyme measured as 0 can be raised: its x is scaled by `floor` (default 1e-9 mg/gDCW), which makes it expensive with `'relative'`. +- When no relaxation reaches the target, the function raises the error `relaxEnzConcs:infeasible` (Python: `ValueError`), and the bounds of the ecModel are unchanged. + +The function changes the usage bounds and `prot_pool_exchange`, raises the upper bound of the biomass reaction to the target when it is lower, leaves `ec.concs` unchanged, and returns `flexEnz` (Python: `FlexEnzResult`) with `frequence` 1 for every enzyme. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +[lpModel, lpEnz] = relaxEnzConcs(ecModel, 'expGrowth', fluxData.grRate(1)); +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import relax_enz_concs + +lp_model = ec_model.copy() +lp_result = relax_enz_concs(lp_model, exp_growth=float(flux_data.gr_rate[0])) +``` +::: +:::: + +## Inspect the result + +`flexEnz` (Python: `FlexEnzResult`) lists the relaxed enzymes, sorted by the ratio of the new bound to the old one, largest first. Its fields are `uniprotIDs`, `oldConcs`, `flexConcs`, `ratioIncr` and `frequence` (the number of times the enzyme was selected); Python uses `uniprot_ids`, `old_concs`, `flex_concs`, `ratio_incr` and `frequence`. An entry `prot_pool` is the protein pool exchange, with its old and new upper bound. The `result` of `relaxProteomicsGreedy` has `trace`, with one step per relaxed enzyme (`relaxedUniprot`, `growthBefore`, `growthAfter`, `shadowPrice`), `relaxed`, which maps each relaxed enzyme to its value in `ec.concs`, `finalGrowth` and `converged`. The result of `relaxEnzConcs` has the fields of `flexEnz`. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +disp(struct2table(flexEnz)) +disp(struct2table(result.trace)) +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +import pandas as pd + +print(pd.DataFrame({ + "uniprot": flex_result.uniprot_ids, + "old": flex_result.old_concs, + "new": flex_result.flex_concs, + "ratio": flex_result.ratio_incr, + "selected": flex_result.frequence, +})) +print(pd.DataFrame([vars(step) for step in result.trace])) +``` +::: +:::: + +## Interpret a large relaxation + +A relaxation corrects for errors in the concentrations. The enzymes that are relaxed, and how much, show which part of the data or the ecModel limits growth: + +- **Many enzymes relaxed.** The proteomics data have many uncertain values. Stricter filters in `loadProtData` (for example a lower `maxRSD`) remove the uncertain concentrations, and their enzymes then draw from the protein pool without a bound of their own. +- **One enzyme selected many times**, or `flexibilizeEnzConcs` stops with the warning that the enzyme "seems to be problematic". Its kcat can be too low for this condition. Check the kcat against its source (see [Check a tuned kcat against its source](kcat-tuning.md#check-a-tuned-kcat-against-its-source) and [Choose the kcat sources](kcat-sources.md)). +- **The protein pool is raised.** The pool from $P_{tot}$, $f$ and $\sigma$ is smaller than the protein that the condition needs, for example when sigma was fitted to another condition (see [Set the protein pool](protein-pool.md)). +- **No method reaches the target.** `flexibilizeEnzConcs` reports that the protein pool exchange was also not limiting, `relaxProteomicsGreedy` returns `converged` false, or `relaxEnzConcs` raises `relaxEnzConcs:infeasible`. The enzyme constraints do not limit growth: the metabolic network or the exchange fluxes do. Apply the same exchange fluxes to the conventional GEM. If it does not reach the growth rate either, check the exchange fluxes of `fluxData.tsv` and the network. + +::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +model = loadConventionalGEM(); +model = constrainFluxData(model, 'fluxData', fluxData, 'condition', 1); +sol = solveLP(model); +fprintf('Growth rate of the GEM: %g /h\n', sol.f) +``` +::: +:::{tab-item} 🐍 Python +:sync: python + +```python +from geckopy import apply_flux_data_constraints, load_conventional_gem + +model = load_conventional_gem(ec_model.adapter) +apply_flux_data_constraints( + model, flux_data, condition=0, bio_rxn=params.bio_rxn, c_source=params.c_source, +) +print(f"Growth rate of the GEM: {model.slim_optimize():g} /h") +``` +::: +:::: + +## Troubleshooting + +| Problem | Possible reason | Solution | +| --- | --- | --- | +| `flexibilizeEnzConcs` warns about enzymes with a concentration of 0 and does not reach the target | a bound of 0 cannot be raised by a factor | Use `relaxProteomicsGreedy`, or load the data with `zerosAsMeasured` false | +| `flexibilizeEnzConcs:infeasible` (Python: a logged warning) and an empty result | the ecModel has no solution before the relaxation | Apply the growth rate with `'max'`; check the exchange fluxes | +| `flexibilize_enz_concs` raises `ValueError` | no enzyme has a concentration, or `exp_growth` is not given and `gr_exp` is not set | Run `fill_enz_concs` and `constrain_enz_concs`; pass `exp_growth` | +| `relaxProteomicsGreedy` raises an error (Python: `RuntimeError`) | `maxIterations` relaxations were made without reaching the target, and candidates are left | Raise `maxIterations`, or use `flexibilizeEnzConcs` | +| The growth rate after `flexibilizeEnzConcs` is slightly below the target | the final pass minimises the pool usage within 0.25 % of the target growth rate | The difference is within 0.25 % of the target | +| The relaxed ecModel is much less constrained than the data | `relaxProteomicsGreedy` sets each relaxed bound to 1000 | Use `flexibilizeEnzConcs` | + +## See also + +- [Constrain enzymes with proteomics](proteomics.md), where the bounds come from. +- [Enzyme usage and bottlenecks](enzyme-usage.md), to find the enzymes that limit growth without changing the bounds. +- [Choose a kcat tuning](kcat-tuning.md), for an enzyme whose kcat is too low. diff --git a/docs/guide/simulation-and-analysis.md b/docs/guide/simulation-and-analysis.md deleted file mode 100644 index c697157..0000000 --- a/docs/guide/simulation-and-analysis.md +++ /dev/null @@ -1,414 +0,0 @@ -# Simulation and analysis - -Light and full ecModels have the structure of a conventional GEM, an S-matrix with vectors of lower and upper bounds, so a method for conventional GEMs also runs on an ecModel in both languages. The tasks below are specific to ecModels: the protein pool constraint is varied, growth and protein pool usage are optimised, fluxes are mapped to the conventional GEM, the solution space is compared by FVA and sampled, engineering targets are scanned, and an ecModel is trimmed to a context-specific one. Media and solver setup are those of any model (see [Choosing a solver](../installation/index.md#choosing-a-solver)). [ecModel YAML format](yaml-format.md) describes the file format that this documentation uses. - - -:::::{dropdown} Place in the workflow - -The diagram orders the functions of this page within the [workflow](workflow.md). Dashed boxes are optional and dotted boxes are neighbouring pages. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - PV(["Previous: Relaxing overconstrained proteomics, or a finished ecModel"]):::ctx - PV --> A["ecFVA
randomSampling"] - PV --> B["ecFSEOF
getSubsetEcModel"] - PV --> C["mapRxnsToConv"] - NX(["Next: Enzyme usage and bottlenecks"]):::ctx - PV -.-> NX -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - PV(["Previous: Relaxing overconstrained proteomics, or a finished ecModel"]):::ctx - PV --> A["ec_fva
random_sampling"] - PV --> B["ec_fseof
get_subset_ec_model"] - PV --> C["map_rxns_to_conv"] - NX(["Next: Enzyme usage and bottlenecks"]):::ctx - PV -.-> NX -``` -::: -:::: -::::: - -## Functions - -| MATLAB | Python | | -|---|---|---| -| `setProtPoolSize` | `set_prot_pool_size` | change the protein pool constraint, including turning it off with `Inf` | -| `setParam(..., 'obj', ...)` | `model.objective = ...` | choose the objective: maximize growth, or minimize protein pool usage | -| `mapRxnsToConv` | `map_rxns_to_conv` | map an ecModel flux distribution back onto its starting conventional GEM | -| `constrainFluxData` | `apply_flux_data_constraints` | constrain exchange fluxes and growth rate from measured data | -| `getAllowedBounds` / `ecFVA` | `flux_variability_analysis` / `ec_fva` | flux variability, with `ecFVA` mapping the ranges back to the conventional GEM | -| `randomSampling` | `random_sampling` | draw feasible flux vectors and report their distribution | -| `ecFSEOF` | `ec_fseof` | flux scanning with enforced objective function, for metabolic engineering targets | -| `getSubsetEcModel` | `get_subset_ec_model` | trim a generic ecModel down to a context-specific gene and reaction set | - -## The protein pool constraint at different growth rates - -An ecModel without proteomics integration shows how the total protein pool constraint affects metabolism as the growth rate changes. At low growth rates, metabolic activity is low and enzymes are usually available in excess. At high growth rates, activity is high, and fluxes can be limited by the amount of enzyme. - -A copy of the ecModel with an unconstrained protein pool (`Inf`) isolates that effect. The copy behaves like a conventional GEM with respect to protein availability: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -ecModel_infProt = setProtPoolSize(ecModel, Inf); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -`set_prot_pool_size` changes the model in place and returns the new bound, so the copy is made first: - -```python -import math - -from geckopy import set_prot_pool_size - -ec_model_inf = ec_model.copy() -set_prot_pool_size(ec_model_inf, p_tot=math.inf) -``` - -An `EcModel` is a `cobra.Model`, so the methods of cobrapy apply to it directly. -::: -:::: - -Simulating the exchange fluxes over a range of growth rates on both models shows the difference. The copy with the unconstrained pool does not predict the shift from respiration to fermentation at high growth rate (the Crabtree effect in yeast), because no enzyme runs short in it. The comparison is plotted with project-specific code (`plotCrabtree` and `plot_crabtree` in the `full_ecModel` tutorial of each toolbox, outside the general API), because the plot depends on the exchange reactions and the reference dataset of the model. - -:::{admonition} Example output -:class: note -The kcat values affect this comparison (see [Growth-rate tuning](growth-rate-tuning.md)): an ecModel with untuned kcats can fail to reach realistic growth rates with the protein pool constraint in place. In the comparison of the published GECKO 3 protocol, 13 kcat values needed tuning before the model showed the Crabtree effect: yeast switched from respiration to fermentation when the protein pool was fully used. The simulated switch occurred earlier than the experimental one (around 0.2 /h simulated and around 0.28 /h observed), which indicates that a single enzyme dominated the constraint too early. The top enzyme usages show whether this is the case (see [Enzyme usage and bottlenecks](enzyme-usage-and-bottlenecks.md)). -::: - -## Choosing the objective function - -Maximising growth is the usual objective of FBA on a conventional GEM, particularly for microbes, and it applies to an ecModel. Once the growth rate is known, minimising the total protein pool usage is a second objective. It assumes that a cell uses its enzyme resources efficiently. - -The protein pool exchange `prot_pool_exchange` runs forward, so its flux is the protein pool usage as a positive number. Growth is maximised first: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -ecModel = setParam(ecModel, 'obj', params.bioRxn, 1); -sol = solveLP(ecModel); -fprintf('Growth rate that is reached: %.4f /hour\n', abs(sol.f)) -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -ec_model.objective = params.bio_rxn -sol = ec_model.optimize() -print(f"Growth rate that is reached: {sol.fluxes[params.bio_rxn]:.4f} /hour") -``` -::: -:::: - -:::{admonition} Example output -:class: note -Output in the `full_ecModel` tutorial, with the tuned kcats and the fitted sigma: - -``` -Growth rate that is reached: 0.4067 /hour -``` -::: - -Growth is then fixed at 99 % of that value, which avoids rounding issues, and the objective changes to minimising the protein pool usage: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -`solveLP` maximises the objective. The coefficient -1 in `setParam` minimises the flux of the reaction. - -```matlab -ecModel = setParam(ecModel, 'lb', params.bioRxn, 0.99*abs(sol.f)); -ecModel = setParam(ecModel, 'obj', 'prot_pool_exchange', -1); -sol = solveLP(ecModel); -fprintf('Minimum protein pool usage: %.2f mg/gDCW\n', abs(sol.f)) -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -The dict form of the objective in cobrapy takes a coefficient per reaction. Minimising `prot_pool_exchange` is a negative weight in a maximising objective: - -```python -max_growth = sol.fluxes[params.bio_rxn] -ec_model.reactions.get_by_id(params.bio_rxn).lower_bound = 0.99 * max_growth -ec_model.objective = { - ec_model.reactions.get_by_id("prot_pool_exchange"): -1.0, -} -sol = ec_model.optimize() -print(f"Minimum protein pool usage: {abs(sol.fluxes['prot_pool_exchange']):.2f} mg/gDCW") -``` -::: -:::: - -Output in the `full_ecModel` tutorial (MATLAB), with the tuned kcats and the fitted sigma: - -``` -Minimum protein pool usage: 116.33 mg/gDCW -``` - -The flux distribution has a growth rate of at least 99 % of the maximum for the condition, and it uses the enzyme resources as efficiently as the model allows. The sign of `prot_pool_exchange` and of the `usage_prot_*` fluxes is the opposite in an ecModel from GECKO 3 until `upgradeEcModel` runs (see [GECKO 3 to GECKO 4](../gecko3-to-gecko4.md)). - -## Comparing flux distributions of different models - -The flux distribution of an ecModel is not directly comparable to that of its starting conventional GEM. The construction of an ecModel changes the network (isozyme splitting, enzyme usage reactions), so both flux distributions are mapped to a common set of reactions. - -`mapRxnsToConv` maps a flux distribution of the ecModel to the starting conventional GEM. It combines the fluxes of isozymatic reactions and of inverted reversible reactions into the reaction they came from: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -sol = solveLP(ecModel); -[mappedFlux, enzUsageFlux, usageEnz] = mapRxnsToConv(ecModel, model, sol.x); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import map_rxns_to_conv - -sol = ec_model.optimize() -mapped = map_rxns_to_conv(ec_model, model, sol.fluxes) -mapped_flux, enz_usage_flux, usage_enz = ( - mapped.mapped_flux, mapped.enz_usage_flux, mapped.usage_enz, -) -``` - -`map_rxns_to_conv` returns a `MapRxnsResult` with the same three arrays under snake_case names. -::: -:::: - -`mappedFlux` (Python: `mapped_flux`) has the length and the order of a flux vector of the conventional GEM and is used as any other conventional flux vector. `enzUsageFlux` (Python: `enz_usage_flux`) holds the fluxes of the `usage_prot_*` reactions and of `prot_pool_exchange`, which are absent from `mappedFlux`, and `usageEnz` (Python: `usage_enz`) holds their labels (`pool` for the protein pool exchange). These fluxes are positive, because the usage reactions run forward. - -:::{admonition} Reaction mapping to the conventional GEM -:class: warning - -`mapRxnsToConv` matches reactions by identifier and does not check that the underlying chemical reaction is the same. A stoichiometric change beyond the enzyme and proteomics constraints, such as a flipped directionality or a new reaction, breaks the correspondence without an error. `mapRxnsToConv` raises an error when a reaction of the conventional GEM is missing from the ecModel. -::: - -## Evaluating the solution space - -An ecModel has a smaller solution space than a conventional GEM, which flux variability analysis (FVA) shows. FVA on an ecModel works as on any constraint-based model. `ecFVA` (Python: `ec_fva`) maps the ranges to the reaction identifiers of the conventional GEM, so the result is comparable to the FVA of the conventional GEM itself. - -The same exchange flux constraints are applied to three models: an ecModel with proteomics integrated (see [Proteomics integration](proteomics-integration.md)), the same ecModel without proteomics, and the underlying conventional GEM: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -ecModelProt = constrainFluxData(ecModelProt, fluxData, 1, 'max', 'loose'); -ecModel = constrainFluxData(ecModel, fluxData, 1, 'max', 'loose'); -model = constrainFluxData(model, fluxData, 1, 'max', 'loose'); - -minFlux = zeros(numel(model.rxns), 3); -maxFlux = minFlux; -[minFlux(:,1), maxFlux(:,1)] = ecFVA(model, model); -[minFlux(:,2), maxFlux(:,2)] = ecFVA(ecModel, model); -[minFlux(:,3), maxFlux(:,3)] = ecFVA(ecModelProt, model); -``` - -`getAllowedBounds` runs plain FVA on the reaction set of the ecModel, without mapping to the reaction identifiers of the conventional model. `ecFVA` runs in parallel when the Parallel Computing Toolbox is installed (option `runParallel`, default `true`). -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import apply_flux_data_constraints -from geckopy.utilities import ec_fva - -model.adapter = adapter -for m in (model, ec_model, ec_model_prot): - apply_flux_data_constraints( - m, flux_data, - condition=0, max_min_growth="max", loose_strict_flux="loose", - ) - -fva_bare = ec_fva(model, model) -fva_full = ec_fva(ec_model, model) -fva_prot = ec_fva(ec_model_prot, model) -``` - -`load_conventional_gem` returns a plain `cobra.Model` without the attribute `adapter`, which `apply_flux_data_constraints` reads for the biomass reaction, so the adapter is attached to it first. The ecModels hold their adapter. `bio_rxn` and `c_source` can be passed instead. - -`ec_fva` returns a DataFrame indexed by the reaction identifiers of the conventional model, with the columns `min_flux` and `max_flux`, so no matrix is pre-allocated as in MATLAB. The option `n_proc` sets the number of worker processes (default `cobra.Configuration().processes`). - -`ec_fva` solves two LPs per reaction and model, which is slow with the default open-source solver (GLPK); the run time is not measured here. A faster solver is configured first (`ec_model.solver = "gurobi"`, licence permitting) before the run at genome scale. -::: -:::: - -### Example output - -The ranges are visualised as a cumulative distribution (`plotEcFVA` or a project-specific `plot_ec_fva` helper). In the published GECKO 3 protocol, the median flux variability of the ecModel is more than fourfold lower than that of the conventional GEM, and more than 15 % of the reactions of the conventional GEM have a high flux variability (around 1,000 mmol/gDCWh) that neither ecModel has: - -| Model | Median flux variability | -|---|---| -| Conventional GEM | 3.104 | -| ecModel, no proteomics | 0.355 | -| ecModel, with proteomics | 0.00325 | - -## Sampling the solution space - -:::{admonition} Sampler choice -:class: tip - -FVA reports the extremes that each reaction reaches one at a time. Flux sampling draws many feasible flux vectors and reports their distribution, which shows what the model does as well as what it can do. RAVEN, the base of GECKO, and raven-toolbox, the base of geckopy, offer two Markov-chain Monte Carlo samplers through the entry point for conventional GEMs: ACHR (Artificially Centered Hit-and-Run) and CHRR (Coordinate Hit-and-Run with Rounding). The call is `randomSampling(model, ..., 'method', 'achr'|'chrr')` in MATLAB and `random_sampling(model, method="achr"|"chrr")` in Python. The choice of the sampler matters for an ecModel. -::: - -ACHR is the default. The documentation of the `chrr` method in both toolboxes (the help text of MATLAB `randomSampling` and the docstring of Python `random_sampling`) names CHRR for better mixing on thin, ill-conditioned polytopes such as enzyme-constrained models. - -The [convergence study of raven-docs](https://github.com/edkerk/raven-docs/blob/main/docs/parameter-tuning/studies/sampling-convergence-calibration.md) measured both methods on a conventional GEM; it has no ecModel. At genome scale on yeast-GEM, the default settings of ACHR were not converged: the median Gelman-Rubin R-hat was 1.17 across four chains, and two in three reactions failed even the loose threshold of 1.1. CHRR converged on a small model (`e_coli_core`, 95 reactions: the worst R-hat dropped from 1.30 to 1.02) and took about 20 times longer there. A bounded probe at yeast-GEM scale measured roughly 80 minutes of fixed rounding cost per chain before the first sample, comparable to a full ACHR run of 300 samples. The probe had too few samples for a stable R-hat, so the convergence of CHRR at genome scale is untested. - -:::{admonition} Sampled ranges at genome scale -:class: warning - -The study concludes that no cheap, validated solution exists yet at that scale. The `chrr` call below is therefore not a validated recipe: sampled flux ranges from either method are provisional at genome scale until they are checked, for example by an R-hat close to 1 on the reactions of interest across independent chains. The fixed rounding cost per chain of CHRR is part of the time budget. -::: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -solutions = randomSampling(ecModel, 1000, 'method', 'chrr'); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from raven_toolbox.analysis import random_sampling - -result = random_sampling(ec_model, n_samples=1000, method="chrr") -``` - -`random_sampling` accepts any `cobra.Model`, an ecModel included, so no conversion is needed before sampling. -::: -:::: - -## Scanning for engineering targets - -`ecFSEOF` (Python: `ec_fseof`) runs flux scanning with enforced objective function (FSEOF) on an ecModel. It enforces an increasing flux through a production reaction, maximises biomass at each step and then minimises the total flux (`ecFSEOF` calls `solveLP(model, 1)`; `ec_fseof` runs pFBA at each step), and classifies the reactions whose absolute flux correlates with the enforced flux as targets. A reaction is a target when the absolute correlation is at least `corrThreshold` (Python: `correlation_threshold`, default 0.9). The scan runs from the production at the optimum of biomass to 90 % of the maximum production. The reactions that draw enzyme usage (`usage_prot_*`) take part in the classification and are not reported. A target is over expressed (OE) when its flux increases with production, knocked out (KO) when it decreases and is at most 1e-8 at the last step, and knocked down (KD) when it decreases otherwise (MATLAB help of `ecFSEOF`). The function warns when the lower bound of the carbon source reaction is tighter than its uptake at maximum growth. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -fseof = ecFSEOF(ecModel, prodTargetRxn, csRxn); -fseof.geneTargets -``` - -`fseof.rxnTargets`, `fseof.transportTargets` and `fseof.geneTargets` are the lists of targets. The option `nSteps` sets the number of enforced production levels (default 16), and `outputFile` writes `ecFSEOF_rxns.tsv` and `ecFSEOF_genes.tsv` to the `output` folder of the project. -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import ec_fseof - -result = ec_fseof(ec_model, prod_target_rxn, cs_rxn) -result.targets -``` - -`result.targets` is a DataFrame with the reaction targets first and the transport targets after them, with a boolean column `transport`. The option `n_steps` sets the number of enforced production levels (default 10). -::: -:::: - -## Context-specific models - -ecModels of higher organisms, such as humans, start from a context-specific model that an omics-integration method such as tINIT builds. Contextualisation and enzyme constraints combine in two ways: - -1. The generic model is contextualised first, and the result is converted to an ecModel. -2. A generic ecModel is reconstructed first and then contextualised. - -The second path is faster. The contextualised ecModels that derive from one generic ecModel have the same kcat data, which matters for the reconstruction of hundreds of single-cell ecModels from one generic model. - -`getSubsetEcModel` trims a generic ecModel to the gene and reaction set of a smaller, contextualised conventional model. Both models derive from the same starting GEM. A warning appears when a `usage_prot_*` reaction of the generic ecModel has an upper bound other than 1000, because the protein concentration constraints of the generic ecModel can be irrelevant in the subset: - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -ecHT29 = getSubsetEcModel(ecModel, HT29); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import get_subset_ec_model - -ec_ht29 = get_subset_ec_model(ec_model, ht29) -``` - -The function matches the canonical reaction identifiers and returns a new `EcModel`; `ec_model` is not changed. -::: -:::: - -### Example output - -Contextualisation and enzyme constraints each lower the reachable growth rate, and the effects combine. The table gives the values of the published GECKO 3 protocol for the maximum growth rate of an unconstrained context-specific GEM, a generic ecModel, and a context-specific ecModel that derives from it: - -| Model | Maximum growth rate (/hour) | -|---|---| -| HT29-GEM (context-specific, unconstrained) | 149.97 | -| ecHuman-GEM (generic ecModel) | 0.122 | -| ecHT29-GEM (context-specific ecModel) | 0.086 | - -The growth rate of the unconstrained context-specific model (149.97 /h) has no biological meaning, because the carbon uptake is unconstrained. It shows that the stoichiometry of the network allows fast growth, so the lower growth rates of the ecModels result from the enzyme constraints and the contextualisation. A stoichiometric bottleneck is absent. - -## Troubleshooting - -| Problem | Possible reason | Solution | -|---|---|---| -| The sign of `prot_pool_exchange` or `usage_prot_*` fluxes is the opposite of the sign in the examples | the ecModel was made with GECKO 3 and these reactions run in reverse | Run `upgradeEcModel` (Python: `upgrade_ec_model`), or reload the model with `loadEcModel` | -| `mapRxnsToConv` raises "Not all reactions from model.rxns can be found in the ecModel" | the ecModel does not derive from the conventional GEM that was passed | Pass the GEM that `makeEcModel` started from | -| `ec_fva` runs for a long time | the default solver is GLPK | Configure a commercial solver, or set `n_proc` | -| `getSubsetEcModel` raises an error that lists reactions of the smaller model | the context-specific model and the ecModel derive from different starting GEMs | Contextualise the GEM that the ecModel is built from | - -## See also - -- [GECKO light and full ecModels](gecko-light.md), the runtime and accuracy trade-off between the two ecModel layouts. -- [Enzyme usage and bottlenecks](enzyme-usage-and-bottlenecks.md), the analyses of `getEnzymeUsage`, `getPfbaEnzymes` and `getEnzymeBottlenecks` that the usage reactions of a full ecModel allow. -- [Proteomics integration](proteomics-integration.md), the construction of the proteomics-constrained ecModel of the solution space comparison. -- [Growth-rate tuning](growth-rate-tuning.md), the kcat values that determine the growth rate of the simulation. -- [API reference](../api/index.md) and [MATLAB vs Python](../matlab-vs-python.md), the signature of every function in the table. diff --git a/docs/guide/tuning-against-experimental-data.md b/docs/guide/tuning-against-experimental-data.md deleted file mode 100644 index 4b53e5e..0000000 --- a/docs/guide/tuning-against-experimental-data.md +++ /dev/null @@ -1,304 +0,0 @@ -# Tuning against experimental data - -[Growth-rate tuning](growth-rate-tuning.md) raises kcats one at a time until a single target growth rate is reached. `cmaesKcatTuning` (Python `cmaes_kcat_tuning`) fits many kcats at once with CMA-ES against measured growth rates and fluxes of several conditions, weighted by how much the source of each kcat is trusted. The result is a ranking of the kcats that the data can inform, a review of the assignment behind the top kcats, a tuned ecModel and a list of the changes of the tuning for a curator to review. - - -:::::{dropdown} Place in the workflow - -The diagram orders the functions of this page within the [workflow](workflow.md). Dashed boxes are optional and dotted boxes are neighbouring pages. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - PV(["Previous: Applying kcats"]):::ctx - PV --> SC["screenKcatLeverage"] --> RV["reviewKcatAssignment"] --> CM["cmaesKcatTuning"] - NX(["Next: Proteomics integration"]):::ctx - CM --> NX -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```{mermaid} -:align: left - ---- -config: - flowchart: - useMaxWidth: false ---- -flowchart TB - classDef opt stroke-dasharray: 5 4 - classDef ctx fill:transparent,stroke-dasharray: 2 3 - PV(["Previous: Applying kcats"]):::ctx - PV --> SC["screen_kcat_leverage"] --> RV["review_assignment"] --> CM["cmaes_kcat_tuning"] - NX(["Next: Proteomics integration"]):::ctx - CM --> NX -``` -::: -:::: -::::: - -## Functions - -| MATLAB | Python | Note | -| --- | --- | --- | -| `loadEvotuneData` | `load_evotune_data` | loads the three data files; the tuning functions call it when no data are given | -| `screenKcatLeverage` | `screen_kcat_leverage` | reports which kcats the data can inform, without any optimisation | -| `selectTunableMask` | `select_tunable_mask` | turns the screen into the set of kcats that are searched | -| `reviewKcatAssignment` | `review_assignment` | flags kcat assignments that deserve a second look, independent of the data | -| `ecStatsFromBrenda` | `ec_stats_from_brenda` | summarises BRENDA per EC number for the assignment review | -| `cmaesKcatTuning` | `cmaes_kcat_tuning` | searches the tunable kcats with CMA-ES for the best fit to the data | -| `tunePriorPenaltyWeight` | `tune_prior_penalty_weight` | sweeps `priorPenaltyWeight` and reports fit against reproducibility across seeds | -| `nChanged`, `parsimonyFrontier`, `bestParsimonious` | `n_changed`, `parsimony_frontier`, `best_parsimonious` | count the changed kcats and trade their number against the fit | -| | `EvotuneResult.corrections` | turns the result into a reviewable list of changes (no MATLAB counterpart) | - -## Setup - -- An ecModel with a model adapter and the project folders `models/` and `data/` (see [Getting started](getting-started.md) and [Building an empty ecModel](building-ec-model.md)), with kcats assigned (see [Kcat assignment](../gecko4-kcat-assignment.md)). -- Python: the development branch of geckopy requires the CMA-ES package `cma`. A geckopy release whose extras list `evotune` needs `pip install --pre "geckopy[evotune]"`. MATLAB needs no additional package; `cmaesKcatTuning` contains its own separable (diagonal-covariance) CMA-ES. -- Experimental data in at least one of the files `evotuneFluxData.tsv` (measured exchange fluxes per condition) and `evotuneMaxGrowth.tsv` (measured maximum growth rates per carbon source). Both together pin the shape of metabolism (fluxes) and its rate (growth rate). Python raises a `ValueError` when neither file exists. - -## Prepare the experimental data - -Three tab-separated files are read from `/data/`. A missing file is skipped. The `full_ecModel` tutorial of GECKO has all three in its `data` folder. - -| File | Contents | Required | -| --- | --- | --- | -| `evotuneFluxData.tsv` | Measured exchange fluxes, one row per condition. | One of these two | -| `evotuneMaxGrowth.tsv` | Measured maximum growth rate, one row per condition, one active carbon source per row (at `-1000`, unconstrained). | One of these two | -| `evotuneZeroExch.tsv` | Reaction IDs assumed to carry zero flux in every condition. | No | - -The two flux files share the column layout of the regular flux data file, read by `loadFluxData` (Python `load_flux_data`): - -``` -Condition Ptot grRate glucose (r_1714) fructose (r_1709) ... evotuneRMSEweight source -glucose NaN 0.41 -1000 NaN ... 1 DLKcat -fructose NaN 0.338 NaN -1000 ... 1 DLKcat -``` - -`Condition` names the row, `grRate` is the measured growth rate and `Ptot` is the measured protein content (`NaN` if not measured). Each ` ()` column holds a measured or fixed flux of that exchange reaction. `evotuneRMSEweight` scales the contribution of the row to the RMSE (usually `1`), and `source` is free text. `evotuneZeroExch.tsv` has a `Rxns` header followed by one reaction ID per line. - -## Configure evotune in model_adapter.toml - -Every setting has a default, so the `[evotune]` table can be omitted. A configured example: - -```toml -[evotune] -sigma0_log_default = 0.3 # trust for any source not listed below -max_growth_weight = 2.0 # weight the growth-rate data double against flux data - -[evotune.source_groups.okp] -sources = ["CataPro"] -match_okp = true - -[evotune.source_groups.brenda] -sources = ["brenda"] - -[evotune.source_groups.custom] -sources = ["custom"] - -[evotune.sigma0_log_source] -okp = 0.25 -brenda = 0.2 -custom = 0.1 -``` - -The default template sets `sigma0_log_default = 0.5`, the groups `dlkcat`, `brenda` and `custom` with `sigma0_log_source` values 0.4, 0.2 and 0.1, `max_growth_weight = 1.0`, `prior_penalty_weight = 0.03`, `tie_isozymes = true`, `rmse_threshold = 0.2` and `max_generations = 150`. MATLAB reads the same keys under other names, for example `evotune.sigma0logDefault`, listed in `adapterParameters.tsv`. - -- **Trust tiers** (`source_groups`, `sigma0_log_source` and `sigma0_log_default`). Kcats are grouped by the origin of their `ec.source`, and each group has a standard deviation in log space. A smaller value means more trust, so the search needs stronger evidence before it moves such a kcat far. With the values above, a curated kcat (`custom`, 0.1) is three times more trusted than a kcat of an unlisted source (0.3). A kcat belongs to the group that lists the origin of its `ec.source` (the text before the detail and the tuning status), so `brenda (wc=0, origin=1)` is in the group that lists `brenda` and `combined (sd=0.10, n=2) | tuned=cmaes (prior=2)` in the group that lists `combined`. Entries of `sources` are matched case-insensitively, against the origin and against the whole `ec.source`. A group with `match_okp = true` also takes the kcats whose `ec.source` equals the configured OpenKineticsPredictor method (`CataPro` in the example, one of the methods named in the template), matched in the same way. When a label matches several groups, the first matching group applies in Python and the last matching group in MATLAB. A kcat in no group takes `sigma0_log_default`. The tiers depend on the source labels of the model, so no tier set applies to every model. -- **`max_growth_weight`**. The combined score is `(rmse_flux + w * rmse_max_growth) / (w + 1)`. At `2`, the growth-rate error counts double against the flux error, which suits growth rates measured on many carbon sources and fluxes measured mainly on one. -- **`prior_penalty_weight`**. Adds `w * mean(((log k - log k0) / sigma0)^2)` to the score, a charge for moving a kcat that is scaled by the trust in its prior. Without the term the fit is flat along many directions: several different kcat vectors fit equally well, and the search returns an arbitrary point that does not reproduce with another seed. The default of `0.03` keeps large corrections in the same direction and within a few fold of each other across seeds, at a small cost in fit. A value of `0` scores on the fit alone. The suitable strength depends on the model and the data, since it scales with `sigma0_log` and with the objective. `tunePriorPenaltyWeight` (Python `tune_prior_penalty_weight`) runs `cmaesKcatTuning` for each of `candidates` (default `0`, `0.01`, `0.03`, `0.1`) at each of `seeds` (default `0` and `1`) and reports the fit cost against the reproducibility across seeds. It costs one full tuning run per candidate and seed, which makes it an occasional calibration. -- **`tie_isozymes`**. Several ecModel reactions are one enzymatic reaction split over isozyme copies. When copies share a prior value and a source, no condition can distinguish them, so an untied search can assign them a difference without biological meaning. Tying gives such copies one shared kcat. -- **`max_generations`, `rmse_threshold`**. The stopping conditions: at most `max_generations` CMA-ES generations, or earlier once the best RMSE reaches `rmse_threshold`. A negative `rmse_threshold` never stops the search early. - -## Screen which kcats the data can inform - -The screen perturbs every tunable kcat up and down by a factor (`fold`, default `2`), moving isozyme copies together as `tie_isozymes` sets, and records how much the fit changes. No tuning happens. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -ModelAdapterManager.setDefault('path/to/project'); -ecModel = loadEcModel(); - -screen = screenKcatLeverage(ecModel, 'nProc', 8); -removevars(screen(1:20, :), 'positions') -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import ModelAdapter, load_ec_model -from geckopy.kcat_tuning.evotune import screen_kcat_leverage - -adapter = ModelAdapter.from_folder("path/to/project") -model = load_ec_model(adapter=adapter) # models/ecModel.yml by default - -screen = screen_kcat_leverage(model, adapter=adapter, n_proc=8) -screen.drop(columns="_positions").head(20) -``` -::: -:::: - -The result is a table with one row per kcat or tied group, ranked by leverage weighted by trust: the reaction ID (`rxnId`, Python `rxn_id`), the number of isozyme copies, the source group, the current value, the leverage, and `cumLeverageShare` (Python `cum_leverage_share`), the running fraction of the total combined ranking value (leverage weighted by trust) contributed by the row and all rows above it. Leverage is the largest change of the plain RMSE caused by one of the two perturbations. A hidden column (`positions`, Python `_positions`) holds the indices into `ec.rxns` that `selectTunableMask` needs. - -The top rows are what a curator can look at first, whether or not a tuning search follows. In one study on ecYeastGEM (not reproduced from the repository), a few kcats accounted for most of what a tuning run can achieve, and they tended to be implausible database values. - -The screen costs one simulation per condition and per kcat group (two probes per group plus a baseline), which is comparable to the budget of a full tuning run. On the `full_ecModel` tutorial the screen runs 8413 simulations (4206 kcat groups, two probes each, and a baseline). With `nProc` and `n_proc` set to 8, the Python screen took 2.8 h. In MATLAB a simulation took 3.6 s with eight processes, an estimate of 8.4 h for the full screen. Both runs used Gurobi; the difference between the two was not investigated. `nProc` (Python `n_proc`) parallelises it as it does the tuning; in MATLAB, values above 1 need the Parallel Computing Toolbox. - -## Check the assignments behind the top kcats - -`reviewKcatAssignment` (Python `review_assignment`) flags source-level problems of the assignment: a value without a kcat row for its EC number, one number reused across many unrelated reactions, the maximum of an EC number standing in for a distribution, an implausible magnitude, or a curated value that repeats what the database says. The corrections belong in the database or fuzzy-matching step in [Gathering kcats](gathering-kcats.md) that produced the value. A correction there reaches every model built from the same data, which a value moved by the tuner does not. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -% One struct per EC code queried while gathering kcats, summarising the -% kcat database (usually BRENDA) the assignment came from. For all EC -% codes of BRENDA: ecStatsFromBrenda(KCATcell, SAcell, KCATmedianCell). -ecStats = containers.Map('KeyType', 'char', 'ValueType', 'any'); -ecStats('1.1.1.1') = struct('nKcat', 12, 'kcatMax', 340, 'kcatMedian', 45, ... - 'values', [340 45 12], 'nSa', 0); - -findings = reviewKcatAssignment(ecModel.ec.rxns, ecModel.ec.kcat, ... - ecModel.ec.source, ecModel.ec.eccodes, 'ecStats', ecStats); -writetable(findings, 'findings.tsv', 'FileType', 'text', 'Delimiter', '\t') -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy.gather_kcats.review import EcStats, review_assignment, findings_tsv - -# One EcStats per EC code queried while gathering kcats, summarising -# whatever kcat database (usually BRENDA) the assignment came from. -ec_stats = { - "1.1.1.1": EcStats(n_kcat=12, kcat_max=340.0, kcat_median=45.0, - values=frozenset({340.0, 45.0, 12.0}), n_sa=0), - # ... -} - -findings = review_assignment( - model.ec.rxns, model.ec.kcat, model.ec.source, model.ec.eccodes, - ec_stats=ec_stats, -) -print(findings_tsv(findings)) -``` -::: -:::: - -Each finding names a reaction, its checks (`no-ec-evidence`, `repeated-value`, `ec-maximum`, `magnitude`, `custom-duplicate`) and a `detail` in words. Checks that need `ecStats` are skipped when it is absent. Isozyme copies that the assignment could not distinguish, the grouping of `tie_isozymes`, are reported once. Standard fallbacks (`standard-fallback`) and tied groups (`tied-isozymes`) are deliberate choices and are reported when `include` lists them. - -Findings are ranked by `leverage` when it is given: the per-kcat effect from the screen, expanded from tie groups to the individual positions of `ec.rxns` through the position column of the screen. Without `leverage` the findings stay in input order. `coverage` (a share of the total leverage of the flagged rows) or `top` (a number of rows) truncates the report. `coverage` adapts to the model, since it keeps as many rows as hold that share of the flagged leverage, where a fixed `top` does not. In the same study the checks flagged around 1,900 of 4,834 kcats before truncation, which is too long a report to read. Stricter check thresholds do not shorten it as well as truncation by leverage, since looking odd (what the checks measure) and mattering (what leverage measures) are unrelated, so a stricter threshold removes high-leverage entries as readily as low-leverage ones. - -## Tune - -`cmaesKcatTuning` builds the tunable set with `selectTunableMask`, which keeps the fewest highest-ranked kcats whose combined leverage reaches `targetImpactShare` (Python `target_impact_share`, default `0.9`) of the total. The cutoff is relative to the achievable improvement of the model, so the same value means a comparable set on another model. A given `screen` avoids a second screening run; without `screen`, `cmaesKcatTuning` computes one. `tunableMask` (Python `tunable_mask`) fixes the set by hand and takes precedence. - -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -[ecModel, result] = cmaesKcatTuning(ecModel, 'screen', screen, ... - 'nProc', 8, 'seed', 0); - -saveEcModel(ecModel, 'ecModel_tuned.yml'); -``` -::: -:::{tab-item} 🐍 Python -:sync: python - -```python -from geckopy import save_ec_model -from geckopy.kcat_tuning.evotune import cmaes_kcat_tuning - -result = cmaes_kcat_tuning( - model, adapter=adapter, screen=screen, n_proc=8, seed=0, -) - -save_ec_model(model, "ecModel_tuned.yml", adapter=adapter) -``` -::: -:::: - -After the call, every tunable kcat of the ecModel holds the best value that CMA-ES found, and the kcat constraints are applied. Python changes `model` in place; MATLAB copies structs on assignment, so the output `ecModel` has to be captured. The other options are `popsize` (the CMA-ES population size, default `4 + floor(3*log(n))` for `n` free parameters), `nProc` and `seed`. The remaining settings come from the `[evotune]` table. The API reference lists the remaining options (`verbose`, `evoData` and `modelAdapter` in MATLAB; `okp_method`, `bio_rxn`, `params` and `evotune_data` in Python). - -Growth that needs special handling for the organism, such as forced anaerobic conditions or a protein pool scaled with a condition-specific biomass composition, is set with functions. In Python these are the arguments `make_anaerobic` and `change_protein_biomass`, which default to `None`; `tutorials/full_ecModel/code/anaerobic.py` in the geckopy repository has an example. In MATLAB the `[matlab]` table of `model_adapter.toml` names the functions with the keys `make_model_anaerobic` and `change_protein_biomass`. - -## Read the result - -`result` is an `EvotuneResult` in Python and a struct in MATLAB, with these fields (Python name first): - -- `rmse_trace`, `objective_trace` (MATLAB `rmseTrace`, `objectiveTrace`): the best plain-fit RMSE so far and the best optimised objective so far, per generation. They are identical when `prior_penalty_weight` is 0. A trace that is flat for many generations before the end indicates a stalled run. -- `new_kcat`, `old_kcat`, `rxns`, `groups` (MATLAB `newKcat`, `oldKcat`, `rxns`, `groups`): the tuned and prior kcats of the searched set as parallel arrays, with the trust tier of each kcat. Tied isozyme copies share one value in `new_kcat`. -- `converged`: whether `rmse_threshold` was reached before `max_generations`. - -The tuned model has the tuning status `tuned=cmaes` in `ec.source` of every kcat that changed, with the kcat before the first tuning as `prior` (see [Provenance of a kcat](../gecko4-kcat-assignment.md#provenance-of-a-kcat)). -- `n_generations` (MATLAB `nGenerations`): the number of generations that ran. - -The final RMSE alone does not show whether a tuned kcat can be trusted: - -1. **Number and size of the changes.** A result that moves nearly every kcat by a little has not identified anything. `n_changed` (MATLAB `nChanged`) counts the kcats that differ from the prior by more than a relative tolerance (default 0.02). The module `geckopy.kcat_tuning.evotune.parsimony` also has `fold_change` and `source_movement`; `parsimony_frontier` and `best_parsimonious` (MATLAB `parsimonyFrontier`, `bestParsimonious`) trade the number of changed kcats against the fit. -2. **Impact share.** `parsimony.impact_share` reports the fraction of the total achievable improvement that the changed kcats account for. It separates few, large and consequential changes from sparsity. It is unweighted, unlike the trust-weighted `cum_leverage_share` of the screen. -3. **Reproducibility.** A run with another `seed` should give large corrections (beyond about two-fold) in the same direction and within a small factor of each other. Otherwise the search has found a flat direction, and `prior_penalty_weight` is the first setting to adjust. -4. **Generalisation.** Data that were not fitted score the tuned and the untuned model. A large improvement on the fitted conditions without an improvement elsewhere indicates overfitting to a thin dataset. - -`new_kcat` and `old_kcat` hold thousands of numbers, most of them unchanged, which a reviewer cannot read. In Python, `result.corrections()` turns them into one `Correction` row per changed kcat (reaction, EC code, source, prior and tuned value, fold change), ranked by leverage, which differs from the order of the distance a kcat moved. Parameters that the data cannot see are free to drift furthest and are the least reliable, so they sort to the bottom. MATLAB has no counterpart of `corrections`; `result.oldKcat` and `result.newKcat` hold the values. - -```python -from geckopy.kcat_tuning import annotate_from_model, corrections_tsv - -# screen is the table from "Screen which kcats the data can inform" above; reuse -# it rather than recomputing leverage. -# The screen has one row per tie group; every isozyme copy of a group takes -# the leverage of its group, through the positions column. -leverage_by_rxn = { - model.ec.rxns[pos]: row["leverage"] - for _, row in screen.iterrows() for pos in row["_positions"] -} -leverage = [leverage_by_rxn.get(r, 0.0) for r in result.rxns] - -annotations = annotate_from_model(model, result.rxns) -rows = result.corrections( - leverage=leverage, names=annotations["names"], ec_codes=annotations["ec_codes"], -) -open("corrections.tsv", "w").write(corrections_tsv(rows)) -``` - -`cumulative_share` of a row is the running fraction of the total leverage that the list has accounted for. The total is summed over the searched kcats, and each isozyme copy has the leverage of its group, so a tied group counts once per copy. The reading can stop once the value stops climbing. This is the tuning-time counterpart of `review_assignment`: the review flags the kcats that need curation before a run, and the corrections list the kcats that the run changed and how far each change can be trusted. - -## Troubleshooting - -| Problem | Possible reason | Solution | -| --- | --- | --- | -| Error that `evotune_data` has neither `flux_data` nor `max_grate` | Neither `evotuneFluxData.tsv` nor `evotuneMaxGrowth.tsv` is in `/data/` | Add one of the files, or pass the loaded data | -| Error `No tunable kcats` | `ec.kcat` is not positive anywhere, or `tunableMask` excludes every kcat with a value | Assign kcats first, or widen the mask | -| Error `Only 1 free parameter(s) after masking and tying` | The mask and `tie_isozymes` leave fewer than two free parameters | Widen `tunableMask` or raise `targetImpactShare` | -| Most kcats take `sigma0_log_default` | The labels in `source_groups` do not match the origins of `ec.source` in the model | List the origins that occur in `ec.source` (for example `ec_tree`, `combined` or `custom`) in `source_groups` | -| Large corrections differ in direction between seeds | The fit is flat along these kcats | Raise `prior_penalty_weight`, calibrated with `tunePriorPenaltyWeight` | - -## See also - -- [Growth-rate tuning](growth-rate-tuning.md), the iterative single-target tuning of kcats. -- [Kcat assignment](../gecko4-kcat-assignment.md) and [Applying kcats](applying-kcats.md), where the kcats and sources used here come from. -- [API reference](../api/index.md), every function in both toolboxes. diff --git a/docs/guide/workflow.md b/docs/guide/workflow.md index 989c145..909ef58 100644 --- a/docs/guide/workflow.md +++ b/docs/guide/workflow.md @@ -2,13 +2,12 @@ -The diagram orders the functions that build, parameterise, tune and analyse an ecModel, with the parts that run in parallel and the alternatives. Each stage is a guide page. A solid arrow connects functions that run one after another, a dashed arrow an optional preview or an alternative way to reach the next step, a diamond a choice between paths, and a dashed outline an optional step. The button in the corner of a diagram opens it in full screen, where the mouse wheel zooms and a drag pans. In the page, the wheel zooms with Ctrl (Cmd on macOS) held. The download icon at the top of each tab saves its diagram as a PDF. +The diagram shows the order of the guide pages, from a new project to the analysis of an ecModel, with the functions of each page. The boxes with a thick outline are the steps of [Build an ecModel](first-ecmodel.md), which makes a working ecModel with the default settings; that page skips the functions marked optional, and the steps from the ecModel structure to the kcat tuning each have a decision page with the options. A solid arrow connects pages that run one after another, a dotted arrow is an optional path, a diamond a choice between paths, and a dashed outline an optional page. A click on a box opens its page. The button in the corner of a diagram opens it in full screen, where the mouse wheel zooms and a drag pans. In the page, the wheel zooms with Ctrl (Cmd on macOS) held. The download icon at the top of each tab saves its diagram as a PDF. :::::{tab-set} ::::{tab-item} Ⓜ️ MATLAB :sync: matlab - :::{div} workflow-pdf {download}`Download the MATLAB diagram as PDF <../_static/workflow-matlab.pdf>` ::: @@ -17,67 +16,63 @@ The diagram orders the functions that build, parameterise, tune and analyse an e :config: {"flowchart": {"useMaxWidth": false}} flowchart TB + classDef quick stroke-width:3px classDef opt stroke-dasharray: 5 4 - A["Getting started
createGECKOproject
Fill model_adapter.toml"] - B["Building an empty ecModel
ModelAdapterManager.setDefault
loadConventionalGEM
makeEcModel (full or light)
getComplexData, applyComplexData (optional)
saveEcModel"] - A --> B - EC["EC numbers
getECfromGEM or getECfromDatabase"] - SM["Metabolite SMILES
findMetSmiles"] - HO["kcatsFromHomologs"] - BR["kcatsFromBrenda"] - TR["kcatsFromECTree"] - DL["kcatsFromDLKcat"] - OK["kcatsFromOKP"] - CU["data/kcats/custom.tsv"]:::opt - GA["Gathering kcats
any subset of the sources"] - B --> GA - GA --> EC - GA --> SM - GA --> HO - EC --> BR - EC --> TR - SM --> DL - SM --> OK - AS["Applying kcats
assignKcats (preview: collectKcats)
getKcatAcrossIsozymes, getStandardKcat (optional)
applyKcatConstraints
setProtPoolSize"] - BR --> AS - TR --> AS - HO --> AS - DL --> AS - OK --> AS - CU --> AS - TU{"solveLP: growth reaches the target?"} - AS --> TU - SE["Growth-rate tuning
sensitivityTuning
sigmaFitter"] - CM["Tuning against experimental data
screenKcatLeverage
reviewKcatAssignment
cmaesKcatTuning"] - TU -->|no| SE - TU -->|no, several conditions measured| CM - FM["Functional ecModel
saveEcModel"] - TU -->|yes| FM - SE --> FM - CM --> FM - PR["Proteomics integration
loadProtData, fillEnzConcs
levelComplexConcs, constrainEnzConcs
calculateFfactor, setProtPoolSize
loadFluxData, constrainFluxData"]:::opt - FM --> PR - RL{"Growth reaches the target?"} + P["Build an ecModel
createGECKOproject
fill model_adapter.toml"]:::quick + S["Choose the ecModel structure
ModelAdapterManager.setDefault
loadConventionalGEM
makeEcModel (full or light)
getComplexData, applyComplexData (optional)"]:::quick + K["Choose the kcat sources
getECfromGEM, then getECfromDatabase
findMetSmiles
updateKcatSources
collectKcats (optional preview)
assignKcats"]:::quick + C["Set custom kcats
data/kcats/custom.tsv
convertCustomKcats (required for a
GECKO 3 data/customKcats.tsv)"]:::opt + PP["Set the protein pool
setProtPoolSize
calculateFfactor (optional)"]:::quick + G{"Measured data"} + T["Choose a kcat tuning
standard: sensitivityTuning
(no change when growth reaches the target)
then sigmaFitter"]:::quick + A["Tune kcats against experimental data
screenKcatLeverage
reviewKcatAssignment (optional)
cmaesKcatTuning (selects with selectTunableMask)"] + F["Functional ecModel
saveEcModel"]:::quick + P --> S --> K --> PP --> G + C -.-> K + T -.->|keep a tuned kcat| C + G -->|one maximum growth rate| T + G -->|measured fluxes or several growth rates| A + T --> F + A --> F + PR["Constrain enzymes with proteomics (full ecModel)
loadProtData, fillEnzConcs
levelComplexConcs (optional), constrainEnzConcs
getIsozymeCoverage, zeroUnmeasuredIsozymes (optional)
loadFluxData, calculateFfactor (optional)
setProtPoolSize, constrainFluxData"]:::opt + F -.-> PR + F -.-> CX + CX -.->|proteomics of the context| PR + CX -.-> RD + RL{"Growth reaches the measured growth rate?"} PR --> RL - FX["Relaxing, gradual
flexibilizeEnzConcs"] - GR["Relaxing, fast
relaxProteomicsGreedy"] - RL -->|no, gradual| FX - RL -->|no, fast| GR + RX["Relax proteomics constraints
flexibilizeEnzConcs (recommended),
relaxProteomicsGreedy when enzymes at 0 limit,
relaxEnzConcs to compare"] + RL -->|no| RX RD["ecModel for simulation and analysis"] RL -->|yes| RD - FX --> RD - GR --> RD - FM -.->|without proteomics| RD - I1["Simulation
ecFVA, randomSampling
ecFSEOF
getSubsetEcModel"] - I2["Enzyme usage
getEnzymeUsage
getEnzymeBottlenecks"] - RD --> I1 - RD --> I2 + RX --> RD + F -.->|without proteomics| RD + AN["Analyse an ecModel
ecFVA, plotEcFVA
randomSampling (RAVEN)
mapRxnsToConv"] + EN["Find engineering targets
ecFSEOF"] + CX["Make a context-specific ecModel
getSubsetEcModel, before proteomics"]:::opt + EU["Enzyme usage and bottlenecks (full ecModel)
getEnzymeUsage, reportEnzymeUsage
getEnzymeBottlenecks, getConcControlCoeffs
getPfbaEnzymes"] + RD --> AN + RD --> EN + RD --> EU + click P href "first-ecmodel.html" + click S href "ecmodel-structure.html" + click K href "kcat-sources.html" + click C href "custom-kcats.html" + click PP href "protein-pool.html" + click T href "kcat-tuning.html" + click A href "advanced-tuning.html" + click F href "first-ecmodel.html" + click PR href "proteomics.html" + click RX href "relaxing-proteomics.html" + click AN href "analysis.html" + click EN href "engineering-targets.html" + click CX href "context-ecmodel.html" + click EU href "enzyme-usage.html" ``` :::: ::::{tab-item} 🐍 Python :sync: python - :::{div} workflow-pdf {download}`Download the Python diagram as PDF <../_static/workflow-python.pdf>` ::: @@ -86,111 +81,87 @@ flowchart TB :config: {"flowchart": {"useMaxWidth": false}} flowchart TB + classDef quick stroke-width:3px classDef opt stroke-dasharray: 5 4 - A["Getting started
geckopy init
Fill model_adapter.toml"] - B["Building an empty ecModel
ModelAdapter.from_folder
load_conventional_gem
make_ec_model (full or light)
get_complex_data, apply_complex_data (optional)
save_ec_model"] - A --> B - EC["EC numbers
fill_eccodes_from_gem or fill_eccodes_from_database"] - SM["Metabolite SMILES
find_met_smiles"] - HO["kcats_from_homologs"] - BR["kcats_from_brenda"] - TR["kcats_from_ec_tree"] - DL["kcats_from_dlkcat"] - OK["kcats_from_okp"] - CU["data/kcats/custom.tsv"]:::opt - GA["Gathering kcats
any subset of the sources"] - B --> GA - GA --> EC - GA --> SM - GA --> HO - EC --> BR - EC --> TR - SM --> DL - SM --> OK - AS["Applying kcats
assign_kcats (preview: collect_kcats)
fill_kcats_from_isozymes, assign_standard_kcat (optional)
apply_kcat_constraints
set_prot_pool_size"] - BR --> AS - TR --> AS - HO --> AS - DL --> AS - OK --> AS - CU --> AS - TU{"model.optimize: growth reaches the target?"} - AS --> TU - SE["Growth-rate tuning
sensitivity_tuning
fit_sigma"] - CM["Tuning against experimental data
screen_kcat_leverage
review_assignment
cmaes_kcat_tuning"] - TU -->|no| SE - TU -->|no, several conditions measured| CM - FM["Functional ecModel
save_ec_model"] - TU -->|yes| FM - SE --> FM - CM --> FM - PR["Proteomics integration
load_prot_data, fill_enz_concs
level_complex_concs, constrain_enz_concs
calculate_f_factor, set_prot_pool_size
load_flux_data, apply_flux_data_constraints"]:::opt - FM --> PR - RL{"Growth reaches the target?"} + P["Build an ecModel
geckopy init
fill model_adapter.toml"]:::quick + S["Choose the ecModel structure
ModelAdapter.from_folder
load_conventional_gem
make_ec_model (full or light)
get_complex_data, apply_complex_data (optional)"]:::quick + K["Choose the kcat sources
load_uniprot_tsv, load_kegg_tsv
fill_eccodes_from_gem, then fill_eccodes_from_database
find_met_smiles
update_kcat_sources
collect_kcats (optional preview)
assign_kcats"]:::quick + C["Set custom kcats
data/kcats/custom.tsv
convert_custom_kcats (required for a
GECKO 3 data/customKcats.tsv)"]:::opt + PP["Set the protein pool
set_prot_pool_size
calculate_f_factor (optional)"]:::quick + G{"Measured data"} + T["Choose a kcat tuning
standard: sensitivity_tuning
(no change when growth reaches the target)
then fit_sigma"]:::quick + A["Tune kcats against experimental data
screen_kcat_leverage
review_assignment (optional)
cmaes_kcat_tuning (selects with select_tunable_mask)"] + F["Functional ecModel
save_ec_model"]:::quick + P --> S --> K --> PP --> G + C -.-> K + T -.->|keep a tuned kcat| C + G -->|one maximum growth rate| T + G -->|measured fluxes or several growth rates| A + T --> F + A --> F + PR["Constrain enzymes with proteomics (full ecModel)
load_prot_data, fill_enz_concs
level_complex_concs (optional), constrain_enz_concs
get_isozyme_coverage, zero_unmeasured_isozymes (optional)
load_flux_data, calculate_f_factor (optional)
set_prot_pool_size, apply_flux_data_constraints"]:::opt + F -.-> PR + F -.-> CX + CX -.->|proteomics of the context| PR + CX -.-> RD + RL{"Growth reaches the measured growth rate?"} PR --> RL - FX["Relaxing, gradual
flexibilize_enz_concs"] - GR["Relaxing, fast
relax_proteomics_greedy"] - RL -->|no, gradual| FX - RL -->|no, fast| GR - SA["Relaxing, smallest total amount
relax_enz_concs"] - RL -->|no, smallest total amount| SA + RX["Relax proteomics constraints
flexibilize_enz_concs (recommended),
relax_proteomics_greedy when enzymes at 0 limit,
relax_enz_concs to compare"] + RL -->|no| RX RD["ecModel for simulation and analysis"] RL -->|yes| RD - FX --> RD - GR --> RD - SA --> RD - FM -.->|without proteomics| RD - I1["Simulation
ec_fva, random_sampling
ec_fseof
get_subset_ec_model"] - I2["Enzyme usage
enzyme_usage
get_enzyme_bottlenecks"] - RD --> I1 - RD --> I2 + RX --> RD + F -.->|without proteomics| RD + AN["Analyse an ecModel
ec_fva
random_sampling (raven-toolbox)
map_rxns_to_conv"] + EN["Find engineering targets
ec_fseof"] + CX["Make a context-specific ecModel
get_subset_ec_model, before proteomics"]:::opt + EU["Enzyme usage and bottlenecks (full ecModel)
enzyme_usage, report_enzyme_usage
get_enzyme_bottlenecks, get_conc_control_coeffs
pfba_enzymes"] + RD --> AN + RD --> EN + RD --> EU + click P href "first-ecmodel.html" + click S href "ecmodel-structure.html" + click K href "kcat-sources.html" + click C href "custom-kcats.html" + click PP href "protein-pool.html" + click T href "kcat-tuning.html" + click A href "advanced-tuning.html" + click F href "first-ecmodel.html" + click PR href "proteomics.html" + click RX href "relaxing-proteomics.html" + click AN href "analysis.html" + click EN href "engineering-targets.html" + click CX href "context-ecmodel.html" + click EU href "enzyme-usage.html" ``` :::: ::::: -## Stages, order and alternatives +## Pages, order and alternatives -Each row is a stage with the stage it runs after, its functions and what it leaves behind. "Any subset" marks functions that run independently of each other, "alternative" a path that replaces another, and "one of" a choice of algorithm. +Each row is a guide page with the page it runs after and what it leaves behind. The functions of each page are listed in the diagram and in the Functions table of the page. -::::{tab-set} -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -| Stage | Run after | Functions | Output | -| --- | --- | --- | --- | -| [Getting started](getting-started.md) | none | `createGECKOproject`, then edit `model_adapter.toml`; `loadDatabases('kegg')` (optional) | project folder | -| [Building an empty ecModel](building-ec-model.md) | project and model adapter | `ModelAdapterManager.setDefault`, `loadConventionalGEM`, `makeEcModel` (full or light), `getComplexData, applyComplexData` (optional), `saveEcModel` | ecModel file | -| [Gathering kcats](gathering-kcats.md) | built ecModel | any subset of `kcatsFromBrenda`, `kcatsFromECTree`, `kcatsFromHomologs`, `kcatsFromDLKcat`, `kcatsFromOKP`; `updateKcatSources` gathers all sources of `model_adapter.toml` | `data/kcats/.tsv` | -| [Applying kcats](applying-kcats.md) | at least one source file or `custom.tsv` | `assignKcats` (preview: `collectKcats`), `getKcatAcrossIsozymes` and `getStandardKcat` (optional), `applyKcatConstraints`, `setProtPoolSize` | `ec.kcat`, constrained ecModel | -| [Growth-rate tuning](growth-rate-tuning.md) | applied kcats, growth rate below the target | `sensitivityTuning`, then `sigmaFitter` | tuned `ec.kcat` | -| [Tuning against experimental data](tuning-against-experimental-data.md) | applied kcats, growth rates and fluxes of several conditions | alternative to sensitivity tuning: `screenKcatLeverage`, `reviewKcatAssignment`, `cmaesKcatTuning` | tuned `ec.kcat` | -| [Proteomics integration](proteomics-integration.md) | functional ecModel | `loadProtData`, `fillEnzConcs`, `levelComplexConcs`, `constrainEnzConcs`, `calculateFfactor, setProtPoolSize`, `loadFluxData, constrainFluxData` (optional) | proteomics-constrained ecModel | -| [Relaxing overconstrained proteomics](relaxing-constraints.md) | proteomics-constrained ecModel below the target growth rate | one of `flexibilizeEnzConcs`, `relaxProteomicsGreedy` | relaxed ecModel | -| [Simulation and analysis](simulation-and-analysis.md), [Enzyme usage and bottlenecks](enzyme-usage-and-bottlenecks.md) | any ecModel | independent of each other: `ecFVA`, `randomSampling`, `ecFSEOF`, `getSubsetEcModel`, `getEnzymeUsage`, `getEnzymeBottlenecks` | results | -::: -:::{tab-item} 🐍 Python -:sync: python - -| Stage | Run after | Functions | Output | -| --- | --- | --- | --- | -| [Getting started](getting-started.md) | none | `geckopy init`, then edit `model_adapter.toml`; `geckopy kegg-download` (optional) | project folder | -| [Building an empty ecModel](building-ec-model.md) | project and model adapter | `ModelAdapter.from_folder`, `load_conventional_gem`, `make_ec_model` (full or light), `get_complex_data, apply_complex_data` (optional), `save_ec_model` | ecModel file | -| [Gathering kcats](gathering-kcats.md) | built ecModel | any subset of `kcats_from_brenda`, `kcats_from_ec_tree`, `kcats_from_homologs`, `kcats_from_dlkcat`, `kcats_from_okp`; `update_kcat_sources` gathers all sources of `model_adapter.toml` | `data/kcats/.tsv` | -| [Applying kcats](applying-kcats.md) | at least one source file or `custom.tsv` | `assign_kcats` (preview: `collect_kcats`), `fill_kcats_from_isozymes` and `assign_standard_kcat` (optional), `apply_kcat_constraints`, `set_prot_pool_size` | `ec.kcat`, constrained ecModel | -| [Growth-rate tuning](growth-rate-tuning.md) | applied kcats, growth rate below the target | `sensitivity_tuning`, then `fit_sigma` | tuned `ec.kcat` | -| [Tuning against experimental data](tuning-against-experimental-data.md) | applied kcats, growth rates and fluxes of several conditions | alternative to sensitivity tuning: `screen_kcat_leverage`, `review_assignment`, `cmaes_kcat_tuning` | tuned `ec.kcat` | -| [Proteomics integration](proteomics-integration.md) | functional ecModel | `load_prot_data`, `fill_enz_concs`, `level_complex_concs`, `constrain_enz_concs`, `calculate_f_factor, set_prot_pool_size`, `load_flux_data, apply_flux_data_constraints` (optional) | proteomics-constrained ecModel | -| [Relaxing overconstrained proteomics](relaxing-constraints.md) | proteomics-constrained ecModel below the target growth rate | one of `flexibilize_enz_concs`, `relax_proteomics_greedy`, `relax_enz_concs` | relaxed ecModel | -| [Simulation and analysis](simulation-and-analysis.md), [Enzyme usage and bottlenecks](enzyme-usage-and-bottlenecks.md) | any ecModel | independent of each other: `ec_fva`, `random_sampling`, `ec_fseof`, `get_subset_ec_model`, `enzyme_usage`, `get_enzyme_bottlenecks` | results | -::: -:::: +| Page | Run after | Output | +| --- | --- | --- | +| [Build an ecModel](first-ecmodel.md) | none | a project folder with `model_adapter.toml`, and at the end a saved functional ecModel | +| [Choose the ecModel structure](ecmodel-structure.md) | project folder | ecModel with enzymes and protein pool, without kcats | +| [Choose the kcat sources](kcat-sources.md) | ecModel structure | `data/kcats/.tsv`, and `ec.kcat` applied as enzyme constraints | +| [Set custom kcats](custom-kcats.md) (optional) | the ecModel structure, or a kcat tuning whose result is kept; read by the first stage of every preset | `data/kcats/custom.tsv` | +| [Set the protein pool](protein-pool.md) | applied kcats | upper bound of the protein pool exchange | +| [Choose a kcat tuning](kcat-tuning.md) | protein pool; the standard tuning changes no kcat when the growth rate reaches the target | tuned `ec.kcat`, and $\sigma$ fitted to the growth rate | +| [Tune kcats against experimental data](advanced-tuning.md) | protein pool, and measured maximum growth rates, exchange fluxes or both, of one or more conditions; alternative to the standard tuning | tuned `ec.kcat` | +| [Constrain enzymes with proteomics](proteomics.md) (optional) | functional full ecModel | proteomics-constrained ecModel | +| [Relax proteomics constraints](relaxing-proteomics.md) | proteomics-constrained ecModel below the measured growth rate | relaxed ecModel | +| [Analyse an ecModel](analysis.md), [Find engineering targets](engineering-targets.md), [Enzyme usage and bottlenecks](enzyme-usage.md) | a functional, proteomics-constrained or relaxed ecModel (enzyme usage: full ecModel only); independent of each other | results | +| [Make a context-specific ecModel](context-ecmodel.md) (optional) | functional ecModel without proteomics constraints | a context-specific ecModel, to which the proteomics of the context can be added | ## Reading the paths -- **Kcat sources:** each `kcatsFrom*` function (Python `kcats_from_*`) writes its own file in `data/kcats`. `assignKcats` reads the files that `[kcat_assignment]` of `model_adapter.toml` lists, so any combination of sources works. The EC numbers feed the BRENDA, EC tree and homolog sources, and the metabolite SMILES feed DLKcat and OpenKineticsPredictor (see [Gathering kcats](gathering-kcats.md)). `updateKcatSources` (Python `update_kcat_sources`) gathers the sources that `model_adapter.toml` needs in one call. -- **Tuning:** sensitivity tuning raises kcats one at a time until one target growth rate is reached. Tuning against experimental data fits many kcats to the measured growth rates and fluxes of several conditions. -- **Relaxation:** `flexibilizeEnzConcs` and `relaxProteomicsGreedy` are alternatives with different stringency (see [Relaxing overconstrained proteomics](relaxing-constraints.md)). -- **Analysis:** the analysis functions work on the finished ecModel and do not depend on each other. +- **Quick path:** [Build an ecModel](first-ecmodel.md) runs the boxes with a thick outline in one go, without the functions marked optional and with the default of each decision page. A decision page is needed only to change a default. +- **Kcats:** `updateKcatSources` (Python: `update_kcat_sources`) makes the source files that the preset of `model_adapter.toml` reads, and `assignKcats` (Python: `assign_kcats`) selects one kcat per reaction from them, with the custom kcats first. +- **Tuning:** the standard tuning raises kcats one at a time until one target growth rate is reached, and `sigmaFitter` (Python: `fit_sigma`) then fits $\sigma$ to that growth rate. Tuning against experimental data fits many kcats to measured growth rates, exchange fluxes or both, of one or more conditions. +- **Relaxation:** `flexibilizeEnzConcs` is the method to start with; `relaxProteomicsGreedy` raises enzymes with a bound of 0, and `relaxEnzConcs` gives a result that does not depend on the order in which enzymes become limiting, to compare with ([Relax proteomics constraints](relaxing-proteomics.md)). +- **Analysis:** the analysis pages do not depend on each other. Enzyme usage and proteomics need a full ecModel. A context-specific ecModel is trimmed from an ecModel without proteomics constraints, and the proteomics of the context are added afterwards. ## See also diff --git a/docs/guide/yaml-format.md b/docs/guide/yaml-format.md index 929a97d..5155e85 100644 --- a/docs/guide/yaml-format.md +++ b/docs/guide/yaml-format.md @@ -105,7 +105,7 @@ The entries have these keys, in this order. A key is left out when its value is - Numbers have at least one decimal (`1000.0`). A string that YAML reads as a number, boolean, null or date is written in double quotes. - `compartments` is a flat mapping, for example `c: cytoplasm`. -The reactions of an ecModel have the identifiers of the build: `_REV` for the reverse direction and `_EXP_` for an isozyme (see [Building an empty ecModel](building-ec-model.md#extension-of-a-conventional-gem)). Once kcats are applied, the metabolites of a reaction include the enzymes (`prot_`) with the coefficient `-MW/(kcat*3600)`: +The reactions of an ecModel have the identifiers of the build: `_REV` for the reverse direction and `_EXP_` for an isozyme (see [Choose the ecModel structure](ecmodel-structure.md#steps-of-makeecmodel)). Once kcats are applied, the metabolites of a reaction include the enzymes (`prot_`) with the coefficient `-MW/(kcat*3600)`: ```yaml - reactions: @@ -167,7 +167,7 @@ The reactions `usage_prot_` and `prot_pool_exchange` run forward. `usage - Protein usage ``` -A light ecModel has `prot_pool` and `prot_pool_exchange` and no usage reactions (see [GECKO light and full ecModels](gecko-light.md)). +A light ecModel has `prot_pool` and `prot_pool_exchange` and no usage reactions (see [Choose the ecModel structure](ecmodel-structure.md)). GECKO 3 defines both reactions in the reverse direction, with negative flux (see [GECKO 3 to GECKO 4](../gecko3-to-gecko4.md)). `loadEcModel` and `load_ec_model` turn the protein reactions of such an ecModel forward in the loaded model. A `usage_prot_` reaction with a positive coefficient of `prot_pool`, and a `prot_pool_exchange` with a negative coefficient, change the sign of their stoichiometry, bounds and objective coefficient. `upgradeEcModel` (Python: `upgrade_ec_model`) applies the same rule to a model in memory and returns the reactions it turned. Both leave a forward model unchanged. The loaders warn with the number of reactions. The file stays as it is until `saveEcModel` stores the upgraded model, and `upgradeGECKOproject` upgrades all ecModel files of a project (see [Upgrade a GECKO 3 project](../gecko3-upgrade-project.md)). In MATLAB, `loadEcModel` turns the reactions and `readYAMLmodel` leaves them as they are. @@ -222,9 +222,9 @@ A reaction without enzymes has an empty `enzymes` entry or none; both are read a | `enzymes` | enzyme identifier: a UniProt accession or, for a gene that KEGG alone matches, the KEGG gene identifier | | `mw` | molecular weight in Da. The key is left out when unknown | | `sequence` | amino acid sequence. The default is an empty string | -| `concs` | measured concentration in mg/gDCW (see [Proteomics integration](proteomics-integration.md)). The key is left out when not measured | +| `concs` | measured concentration in mg/gDCW (see [Constrain enzymes with proteomics](proteomics.md)). The key is left out when not measured | -The pseudo-enzyme `standard` has `standard` as gene and enzyme identifier (see [Applying kcat values](applying-kcats.md)). +The pseudo-enzyme `standard` has `standard` as gene and enzyme identifier (see [Choose the kcat sources](kcat-sources.md)). ```yaml - ec-enzymes: @@ -262,7 +262,7 @@ geckopy repairs an entry `enzymes: !!omap` without items and a value in double q ## See also -- [Proteomics integration](proteomics-integration.md), where the values of `concs` come from. -- [Building an empty ecModel](building-ec-model.md) and [Applying kcat values](applying-kcats.md), where `ec-rxns` and `ec-enzymes` are populated. +- [Constrain enzymes with proteomics](proteomics.md), where the values of `concs` come from. +- [Choose the ecModel structure](ecmodel-structure.md) and [Choose the kcat sources](kcat-sources.md), where `ec-rxns` and `ec-enzymes` are populated. - [Project format 4](../gecko4-project-format.md), for the ecModel format and the project folder. - [API reference](../api/index.md), for the read and write functions. diff --git a/docs/index.md b/docs/index.md index 38a7564..3589b20 100644 --- a/docs/index.md +++ b/docs/index.md @@ -53,7 +53,7 @@ Convert a conventional GEM into an ecModel by bounding reaction rates with enzym ::: :::{grid-item-card} Proteomics integration -Constrain individual enzyme usage with absolute or relative proteomics measurements. +Constrain the usage of individual enzymes with absolute proteomics measurements. ::: :::{grid-item-card} Light ecModels @@ -65,6 +65,22 @@ A smaller, faster-simulating ecModel variant for when full proteome coverage is ## Quick start ::::{tab-set} +:::{tab-item} Ⓜ️ MATLAB +:sync: matlab + +```matlab +% load a previously built ecModel from a project folder +ModelAdapterManager.setDefault('path/to/project'); +model = loadEcModel('ecYeastGEM.yml'); + +% set growth as the objective +model = setParam(model, 'obj', 'r_2111', 1); + +% run FBA +sol = solveLP(model); +fprintf('Growth rate: %.4f h-1\n', sol.f); +``` +::: :::{tab-item} 🐍 Python :sync: python @@ -83,33 +99,24 @@ sol = model.optimize() print(f"Growth rate: {sol.objective_value:.4f} h⁻¹") ``` ::: -:::{tab-item} Ⓜ️ MATLAB -:sync: matlab - -```matlab -% load a previously built ecModel from a project folder -ModelAdapterManager.setDefault('path/to/project'); -model = loadEcModel('ecYeastGEM.yml'); - -% set growth as the objective -model = setParam(model, 'obj', 'r_2111', 1); - -% run FBA -sol = solveLP(model); -fprintf('Growth rate: %.4f h-1\n', sol.f); -``` -::: :::: ## Documentation ::::{grid} 1 2 2 2 +:::{grid-item-card} Build an ecModel +:link: guide/first-ecmodel +:link-type: doc + +A working ecModel with the default settings, from a project folder to a saved model, in MATLAB and Python. +::: + :::{grid-item-card} Workflow overview :link: guide/workflow :link-type: doc -The order of the functions from model construction to analysis, with the alternative paths, as MATLAB and Python diagrams. +The order of the guide pages from a new project to the analysis, with the alternative paths and the functions of each page, as MATLAB and Python diagrams. ::: :::{grid-item-card} Upgrade a GECKO 3 project @@ -123,7 +130,7 @@ Convert a GECKO 3 project and its ecModel to project format 4 with `upgradeGECKO :link: installation/index :link-type: doc -Set up GECKO in MATLAB or geckopy in Python, the solver and Docker for DLKcat. +Set up GECKO in MATLAB or geckopy in Python, and a linear-programming solver. ::: :::{grid-item-card} GECKO and geckopy diff --git a/docs/installation/index.md b/docs/installation/index.md index 71441d4..29d48d1 100644 --- a/docs/installation/index.md +++ b/docs/installation/index.md @@ -30,7 +30,7 @@ The starting point is a conventional constraint-based metabolic model, at genome ::: :::: -Both toolboxes also need a linear-programming solver (see [Choosing a solver](#choosing-a-solver) below) and, for DLKcat, [Docker](#docker). +Both toolboxes also need a linear-programming solver (see [Choosing a solver](#choosing-a-solver) below). Predicted kcats come from the hosted [OpenKineticsPredictor](#openkineticspredictor) service. ## RAVEN toolbox / raven-toolbox @@ -146,14 +146,14 @@ MATLAB reaches the solvers through RAVEN and the COBRA Toolbox, Python through c GLPK, the default solver of RAVEN, is not recommended for GECKO: ecModels are larger and more numerically demanding than the conventional GEMs for which GLPK is adequate. Configure Gurobi, or in MATLAB SCIP, before working with an ecModel of genome scale. ::: -## Docker +## OpenKineticsPredictor -`runDLKcat` (Python: `run_dlkcat`) runs DLKcat in a Docker container from the image `ghcr.io/sysbiochalmers/dlkcat-gecko:0.1`. Installation instructions are at [docs.docker.com/get-started/get-docker](https://docs.docker.com/get-started/get-docker/). +`kcatsFromOKP` and `kcatsFromDLKcat` (Python: `kcats_from_okp`, `kcats_from_dlkcat`) predict kcat values with the hosted [OpenKineticsPredictor](https://predictor.openkinetics.org/) service. The service needs an API key, which GECKO reads from the environment variable `OKP_API_KEY` or from `data/okpApiKey.txt` in the project folder. A project made with `createGECKOproject` (Python: `geckopy init`) has a `.gitignore` that keeps that file out of git. [Choose the kcat sources](../guide/kcat-sources.md) describes the methods and the jobs. -:::{admonition} kcat prediction without Docker -:class: tip +:::{admonition} DLKcat on the local computer +:class: note -`kcatsFromOKP` (Python: `kcats_from_okp`) predicts kcat values with the hosted [OpenKineticsPredictor](https://predictor.openkinetics.org/) service. The service needs an API key and no Docker installation (see [Kcat assignment](../gecko4-kcat-assignment.md)). +`kcatsFromDLKcat(ecModel, 'via', 'docker')` (Python: `kcats_from_dlkcat(ec_model, via="docker")`) runs DLKcat on the local computer in a Docker container from the image `ghcr.io/sysbiochalmers/dlkcat-gecko:0.1`. This is the older method; it needs [Docker](https://docs.docker.com/get-started/get-docker/) and no API key. ::: ## Tutorial code diff --git a/scripts/curated_pairs.yml b/scripts/curated_pairs.yml index b4a2b65..b4c2110 100644 --- a/scripts/curated_pairs.yml +++ b/scripts/curated_pairs.yml @@ -16,7 +16,6 @@ # not_ported -- deliberately absent from Python, with the reason aliases: - adapterTemplate: generate_template_toml assignKcatValues: apply_kcat_list constrainFluxData: apply_flux_data_constraints getECfromDatabase: fill_eccodes_from_database @@ -36,15 +35,3 @@ not_ported: plotEcFVA: >- Plots an ecFVA flux range directly from the result; geckopy returns the ec_fva DataFrame and leaves plotting to matplotlib or seaborn directly. - readOpenKineticsPredictorOutput: >- - Parses a downloaded OpenKineticsPredictor result file. geckopy replaced - the file round-trip with a REST client (submit_open_kinetics_predictor / - fetch_open_kinetics_predictor); the equivalent parsing is internal to - that client. - updateProtPool: >- - Obsolete since GECKO 3.2.0; setProtPoolSize (geckopy set_prot_pool_size) - supersedes it. - writeOpenKineticsPredictorInput: >- - Builds an OpenKineticsPredictor request file. geckopy replaced the file - round-trip with a REST client; the equivalent request-building is - internal to that client. diff --git a/scripts/known_names.txt b/scripts/known_names.txt index f70dd9c..813d423 100644 --- a/scripts/known_names.txt +++ b/scripts/known_names.txt @@ -289,3 +289,79 @@ database_sources plot_light_vs_full printFluxes exportToExcelFormat +zerosAsMeasured +zeros_as_measured +removeConstraints +remove_constraints +enzymeSet +enzyme_set +finalGrowth +vars +maxMinGrowth +fluxData +rxnTargets +transportTargets +geneTargets +csRxn +ftINIT +protID +prot_id +absUsage +abs_usage +measuredProtein +demandedProtein +demandRatio +measuredProteinActive +demandRatioActive +truncateValues +kcatSources +maxGenerations +maxGrowthWeight +nIsozymes +rmseThreshold +sigma0logDefault +sigma0logSource +sourceGroup +tieIsozymes +out_file +demanded_protein +demand_ratio +demand_ratio_active +include_zero +measured_protein +measured_protein_active +ecMyModel +ec_rxn +ecRxn +sd_log10 +sdLog10 +spread_log10 +spreadLog10 +rxnNames +source_group +context_gem +contextGEM +cs_rxn +prod_target_rxn +prodTargetRxn +target_type +prot_data +flex_model +get_by_id +restrictTo +restrict_to +poolWeight +okp_kinform_h +submitOpenKineticsPredictor +fetchOpenKineticsPredictor +writeOpenKineticsPredictorInput +readOpenKineticsPredictorOutput +bayesianSensitivityTuning +abc_max +getcarbonnum +getrSample +updateprior +findMaxValue +find_max_value +anaerobicModel_GECKO +changeMedia diff --git a/scripts/project.yml b/scripts/project.yml index 9cb8654..efd40a5 100644 --- a/scripts/project.yml +++ b/scripts/project.yml @@ -20,7 +20,7 @@ matlab: - folder: io title: Input / output - folder: kcat_tuning - title: kcat sensitivity tuning + title: kcat tuning - folder: limit_proteins title: Limit proteins - folder: model_adapter @@ -37,7 +37,7 @@ python: ec_model: Build & edit ecModel gather_kcats: Gather kcats get_enzyme_data: Enzyme & EC data - kcat_tuning: kcat sensitivity tuning + kcat_tuning: kcat tuning limit_proteins: Limit proteins adapter: Model adapter utilities: Utilities