Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/ci_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,7 @@ jobs:
--test sbom_page_drift \
--test sbom_schema \
--test skill_easyblock_fix \
--test solved_dot \
--test solver_completeness \
--test stack_files \
--test stack_order_filter \
Expand Down
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -387,6 +387,12 @@ All notable changes to this project are documented here.

### Added

- `stack solve --dot-out` and the MCP field `dot_out` write the solved
dependency DAG. A node is `name/version` plus the toolchain, and the
versionsuffix when the recipe has one. An edge points from a package to a
dependency it selected and is labelled with the requirement; a
builddependency is dashed. A `.svg` or `.png` path is rendered with
Graphviz `dot`. Any other suffix is the DOT source.
- `stack generation` and `eb_stack_generation` check each added entry's
toolchain generation against the generation the easystack file name names,
the token after `eb-<X.Y.Z>-` such as `-2025b.yml`. An entry whose
Expand Down
10 changes: 9 additions & 1 deletion docs/orgmode/reference/cli.org
Original file line number Diff line number Diff line change
Expand Up @@ -412,7 +412,8 @@ eb-stack stack solve \
[--build-list-out /tmp/build.list] [--build-list-excludes-baseline] \
[--jenkins-build-list-out /tmp/next.list [--jenkins-flags-from /tmp/previous.list]... \
[--jenkins-gpu-systems gpu-a,gpu-b] [--jenkins-roots-only]] \
[--stack-diff-out /tmp/stack.diff.md]
[--stack-diff-out /tmp/stack.diff.md] \
[--dot-out /tmp/stack.dot]

eb-stack stack sbom --lock /tmp/stack.lock.json [--out /tmp/stack.cdx.json]

Expand Down Expand Up @@ -536,6 +537,13 @@ preferences use the TOML schema documented with the package workflow; whole
stack solving uses the JSON ~Policy~ schema. The policy file is
~--stack-policy~, the same flag ~package plan~ and ~package bump~ take.

~--dot-out~ writes the solved dependency DAG. A node is ~name/version~ plus
the toolchain, and the versionsuffix when the recipe has one. An edge points
from a package to a dependency it selected and is labelled with the
requirement; a builddependency is dashed. A ~.svg~ or ~.png~ path is rendered
with the Graphviz ~dot~ program. Any other suffix, including ~.dot~, is the
DOT source. The MCP field ~dot_out~ writes the same file.

The baseline is either a previous generation (~--baseline-easyconfigs~, by
default the first ~--easyconfigs~ tree) or what a set of easystacks installs
(~--baseline-easystack~, repeatable). An easystack baseline is every entry
Expand Down
10 changes: 9 additions & 1 deletion llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4753,7 +4753,8 @@ eb-stack stack solve \
[--build-list-out /tmp/build.list] [--build-list-excludes-baseline] \
[--jenkins-build-list-out /tmp/next.list [--jenkins-flags-from /tmp/previous.list]... \
[--jenkins-gpu-systems gpu-a,gpu-b] [--jenkins-roots-only]] \
[--stack-diff-out /tmp/stack.diff.md]
[--stack-diff-out /tmp/stack.diff.md] \
[--dot-out /tmp/stack.dot]

eb-stack stack sbom --lock /tmp/stack.lock.json [--out /tmp/stack.cdx.json]

Expand Down Expand Up @@ -4877,6 +4878,13 @@ preferences use the TOML schema documented with the package workflow; whole
stack solving uses the JSON `Policy` schema. The policy file is
`--stack-policy`, the same flag `package plan` and `package bump` take.

`--dot-out` writes the solved dependency DAG. A node is `name/version` plus
the toolchain, and the versionsuffix when the recipe has one. An edge points
from a package to a dependency it selected and is labelled with the
requirement; a builddependency is dashed. A `.svg` or `.png` path is rendered
with the Graphviz `dot` program. Any other suffix, including `.dot`, is the
DOT source. The MCP field `dot_out` writes the same file.

The baseline is either a previous generation (`--baseline-easyconfigs`, by
default the first `--easyconfigs` tree) or what a set of easystacks installs
(`--baseline-easystack`, repeatable). An easystack baseline is every entry
Expand Down
3 changes: 2 additions & 1 deletion src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,8 @@ pub use sbom::{
artifact_facts_for_lock, build_dep_map_from_universe, dep_map_from_universe, dep_maps_for_lock,
host_build_environment, lock_to_bom, lock_to_bom_with_facts, lock_to_cyclonedx,
lock_to_cyclonedx_from_recipes, lock_to_cyclonedx_with_deps, lock_to_cyclonedx_with_facts,
lock_to_cyclonedx_with_runtime_and_build, ArtifactFacts, SbomFacts, SbomReadStats,
lock_to_cyclonedx_with_runtime_and_build, solved_dependency_dot, write_solved_dependency_graph,
ArtifactFacts, SbomFacts, SbomReadStats,
};
pub use select::{resolvo_resolve_dep_versions, select_stack, SelectError};

Expand Down
17 changes: 17 additions & 0 deletions src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -899,6 +899,15 @@ struct SolveArgs {
/// Markdown diff of this solve against the baseline.
#[arg(long)]
stack_diff_out: Option<PathBuf>,
/// Graphviz drawing of the solved dependency DAG.
///
/// A node is `name/version` plus the toolchain, and the versionsuffix
/// when the recipe has one. An edge points from a package to a
/// dependency it selected and is labelled with that requirement; a
/// builddependency is dashed. A `.svg` or `.png` path is rendered with
/// `dot`. Any other suffix is the DOT source.
#[arg(long)]
dot_out: Option<PathBuf>,
}

#[derive(clap::Args, Debug)]
Expand Down Expand Up @@ -2483,6 +2492,7 @@ fn run_stack(command: StackCommand) -> Result<()> {
jenkins_gpu_systems,
jenkins_roots_only,
stack_diff_out,
dot_out,
} = *args;
for path in &roots_from_build_list {
let text = std::fs::read_to_string(path)
Expand Down Expand Up @@ -2637,11 +2647,18 @@ fn run_stack(command: StackCommand) -> Result<()> {
error
}
})?;
if let Some(path) = &dot_out {
eb_stack::write_solved_dependency_graph(&lock, &lock_out, path)
.map_err(|error| anyhow::anyhow!(error))?;
}
println!(
"lock={} packages={}",
lock_out.display(),
lock.packages.len()
);
if let Some(path) = &dot_out {
println!("dot={}", path.display());
}
Ok(())
}
StackCommand::Order {
Expand Down
4 changes: 4 additions & 0 deletions src/mcp.rs
Original file line number Diff line number Diff line change
Expand Up @@ -338,6 +338,7 @@ fn tool_catalog() -> Vec<Value> {
("jenkins_gpu_systems", "array"),
("jenkins_roots_only", "boolean"),
("stack_diff_out", "string"),
("dot_out", "string"),
],
),
tool_with_optional(
Expand Down Expand Up @@ -2812,6 +2813,9 @@ fn stack_solve(arguments: &Value) -> Result<Value, String> {
},
)
.map_err(|error| error.to_string())?;
if let Some(path) = optional_path(arguments, "dot_out") {
crate::sbom::write_solved_dependency_graph(&lock, &lock_out, &path)?;
}
Ok(json!({
"lock": lock_out,
"packages": lock.packages.len(),
Expand Down
192 changes: 192 additions & 0 deletions src/sbom.rs
Original file line number Diff line number Diff line change
Expand Up @@ -651,6 +651,198 @@ pub fn closure_edges_for_lock(lock: &StackLock, lock_file: &Path) -> HashMap<Str
edges
}

/// Graphviz DOT for the dependency DAG of a solved lock.
///
/// A node is `name/version` plus the toolchain, and the versionsuffix when
/// the selected recipe has one. An edge points from a package to a
/// dependency it selected. The label is the requirement as the solver read
/// it, plus a versionsuffix or an explicit toolchain when the tuple carries
/// one. A builddependency that is not also a runtime dependency is dashed.
///
/// Edges come from the easyconfigs the lock names, the same walk as
/// [`closure_edges_for_lock`]. A dependency the lock did not select is left
/// out. Node order is the lock's package order made deterministic by name,
/// version, versionsuffix, and toolchain.
pub fn solved_dependency_dot(lock: &StackLock, lock_file: &Path) -> String {
let lock_dir = lock_directory(lock_file);
let mut nodes: Vec<&crate::domain::LockPackage> = lock.packages.iter().collect();
nodes.sort_by(|a, b| {
(
&a.name,
&a.version,
a.versionsuffix.as_deref().unwrap_or(""),
toolchain_text(&a.toolchain),
)
.cmp(&(
&b.name,
&b.version,
b.versionsuffix.as_deref().unwrap_or(""),
toolchain_text(&b.toolchain),
))
});
let mut id_of: HashMap<String, String> = HashMap::new();
for (index, package) in nodes.iter().enumerate() {
id_of.insert(package_identity(package), format!("n{index}"));
}

let mut edges: BTreeMap<(String, String), (String, bool)> = BTreeMap::new();
for package in &lock.packages {
let Some(resolved) = (!package.easyconfig_path.is_empty())
.then(|| {
crate::eb_parse::resolve_easyconfig_file(&resolve_lock_easyconfig(
lock_dir,
&package.easyconfig_path,
))
.ok()
})
.flatten()
else {
continue;
};
let candidate = resolved.to_candidate();
let from = package_identity(package);
for (dep, runtime) in candidate
.dependencies
.iter()
.map(|dep| (dep, true))
.chain(candidate.builddependencies.iter().map(|dep| (dep, false)))
{
let Some(target) = resolve_locked_dep(lock, package, dep) else {
continue;
};
let key = (from.clone(), package_identity(target));
let label = dependency_edge_label(dep);
if let Some(drawn) = edges.get_mut(&key) {
if runtime {
*drawn = (label, true);
}
} else {
edges.insert(key, (label, runtime));
}
}
}

let mut out = String::from("digraph solved {\n");
out.push_str(" rankdir=TB;\n");
out.push_str(" graph [fontname=\"Helvetica\"];\n");
out.push_str(" node [shape=box, fontname=\"Helvetica\", fontsize=11];\n");
out.push_str(" edge [fontname=\"Helvetica\", fontsize=9];\n");
for package in &nodes {
let id = &id_of[&package_identity(package)];
let version = format!(
"{}{}",
package.version,
package.versionsuffix.as_deref().unwrap_or("")
);
// Graphviz `\n`, written as two characters so a node stays one source line.
let label = format!(
"{}\\n{}",
dot_escape(&format!("{}/{}", package.name, version)),
dot_escape(&toolchain_text(&package.toolchain))
);
out.push_str(&format!(" {id} [label=\"{label}\"];\n"));
}
for ((from, to), (label, runtime)) in &edges {
let Some(from_id) = id_of.get(from) else {
continue;
};
let Some(to_id) = id_of.get(to) else {
continue;
};
let style = if *runtime { "" } else { ", style=dashed" };
out.push_str(&format!(
" {from_id} -> {to_id} [label=\"{}\", color=\"#004D40\"{style}];\n",
dot_escape(label)
));
}
out.push_str("}\n");
out
}

/// Write the solved dependency DAG.
///
/// `.svg` and `.png` are rendered with the Graphviz `dot` program. Any other
/// suffix, including `.dot`, is the DOT source.
pub fn write_solved_dependency_graph(
lock: &StackLock,
lock_file: &Path,
out: &Path,
) -> Result<(), String> {
let dot = solved_dependency_dot(lock, lock_file);
if let Some(parent) = out.parent() {
if !parent.as_os_str().is_empty() {
std::fs::create_dir_all(parent)
.map_err(|error| format!("create {}: {error}", parent.display()))?;
}
}
let kind = out
.extension()
.and_then(|ext| ext.to_str())
.unwrap_or("")
.to_ascii_lowercase();
if kind == "svg" || kind == "png" {
let mut child = std::process::Command::new("dot")
.arg(format!("-T{kind}"))
.arg("-o")
.arg(out)
.stdin(std::process::Stdio::piped())
.spawn()
.map_err(|error| {
format!(
"rendering {} needs the graphviz dot program: {error}",
out.display()
)
})?;
let stdin = child.stdin.as_mut().ok_or("dot stdin")?;
std::io::Write::write_all(stdin, dot.as_bytes())
.map_err(|error| format!("write dot source: {error}"))?;
// Drop stdin before waiting so dot sees EOF.
child.stdin.take();
let status = child
.wait()
.map_err(|error| format!("wait for dot: {error}"))?;
if !status.success() {
return Err(format!("dot exited {status}"));
}
return Ok(());
}
std::fs::write(out, dot).map_err(|error| format!("write {}: {error}", out.display()))?;
Ok(())
}

fn toolchain_text(toolchain: &crate::domain::Toolchain) -> String {
if crate::hierarchy::is_system_toolchain(toolchain) {
"system".to_string()
} else {
format!("{}-{}", toolchain.name, toolchain.version)
}
}

fn dependency_edge_label(dep: &crate::domain::DepReq) -> String {
let mut label = dep.version_req.clone();
if let Some(suffix) = dep.versionsuffix.as_deref() {
if !suffix.is_empty() {
label.push_str(" suffix ");
label.push_str(suffix);
}
}
if let Some(toolchain) = &dep.toolchain {
label.push_str(" toolchain ");
if crate::hierarchy::is_system_toolchain(toolchain) {
label.push_str("system");
} else {
label.push_str(&toolchain.name);
label.push('/');
label.push_str(&toolchain.version);
}
}
label
}

fn dot_escape(text: &str) -> String {
text.replace('\\', "\\\\").replace('"', "\\\"")
}

/// How many recipes a lock SBOM could read.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub struct SbomReadStats {
Expand Down
Loading
Loading