Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/cli-schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -3163,7 +3163,7 @@
"name": "files",
"type": "array",
"required": false,
"summary": "Filter by changelog YAML paths (comma-separated), or a path to a newline-delimited file containing changelog paths. Can be specified multiple times. Forces local entry sourcing. This option is not supported in profile-based commands; pass a path list file as the second or third positional argument instead.",
"summary": "Filter by changelog YAML paths (comma-separated), or a path to a newline-delimited file containing changelog paths. Can be specified multiple times. When entries are sourced from the CDN, paths are matched to pool entries by file name and do not need to exist locally; with local sourcing (--force-local, --directory, or bundle.use_local_changelogs) the paths must exist on disk. This option is not supported in profile-based commands; pass a path list file as the second or third positional argument instead.",
"repeatable": true,
"elementType": "string"
},
Expand Down
4 changes: 2 additions & 2 deletions docs/cli/changelog/cmd-bundle.md
Original file line number Diff line number Diff line change
Expand Up @@ -370,7 +370,7 @@ In profile mode, pass the same path list as a positional argument:
docs-builder changelog bundle serverless-release 2026-07-07 ./docs/temp/changelog_files.txt
```

`--files` / path-list selection always reads the named files from disk (local entry sourcing). It does not fetch entries from the CDN. `rules.bundle` still applies after selection.
`--files` / path-list selection follows the standard entry-sourcing rules. When entries are sourced from the CDN (the default when `bundle.repo` resolves), the listed paths are matched to CDN pool entries by file name and do not need to exist locally — useful for private repositories whose entries exist only in S3 and whose public copies have PR/issue references scrubbed, so PR-based filters cannot match. With local sourcing (`--force-local`, `--directory`, or `bundle.use_local_changelogs`), the listed files are read from disk and must exist. In either mode, a listed entry that cannot be found fails the run, and `rules.bundle` still applies after selection.

### Force local entry sourcing [changelog-bundle-force-local]

Expand All @@ -382,7 +382,7 @@ docs-builder changelog bundle serverless-release 2026-07-07 ./docs/temp/prs.txt
```

`--force-local` is allowed in both option-based and profile-based commands.
Path-list / `--files` filters already force local sourcing, so `--force-local` is optional in that case.
Use it with path-list / `--files` filters when the listed files should be read from disk instead of matched against the CDN pool.

### Hide features [changelog-bundle-hide-features]

Expand Down
7 changes: 7 additions & 0 deletions docs/data/release-notes/bundle.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,13 @@ For example, if the source of truth for what was shipped in each release is:
docs/changelog/1770424401-adhoc-feature.yaml
```

If you're bundling files from the CDN, use paths like this:

```txt
/changelog/elastic/kibana/main/1783971707-the-dashboards-and-visualizations-apis-are-now-gen.yaml
/changelog/elastic/kibana/main/247279.yaml
```

- automated release notes for GitHub releases:

```sh
Expand Down
6 changes: 3 additions & 3 deletions docs/data/release-notes/configure-ref.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,10 +72,10 @@ The authoring repo is resolved with the same precedence as `changelog upload`: `

Sourcing is decided per run:

- **Local folder.** Used when `bundle.use_local_changelogs: true`, when `--force-local` is passed, when `--files` / a path-list filter is used, when `--directory` is passed, or when the authoring repo cannot be resolved. The folder must contain the changelog files.
- **CDN (default when a repo resolves).** Used when the authoring repo resolves, local sourcing is not forced, and a CDN base URL is configured (`DOCS_BUILDER_CHANGELOG_CDN`, defaulting to the public distribution). The command fetches `changelog/{org}/{repo}/{branch}/registry.json` and the entries it lists, then applies the bundle's own product/PR/issue filters to the downloaded set.
- **Local folder.** Used when `bundle.use_local_changelogs: true`, when `--force-local` is passed, when `--directory` is passed, or when the authoring repo cannot be resolved. The folder must contain the changelog files.
- **CDN (default when a repo resolves).** Used when the authoring repo resolves, local sourcing is not forced, and a CDN base URL is configured (`DOCS_BUILDER_CHANGELOG_CDN`, defaulting to the public distribution). The command fetches `changelog/{org}/{repo}/{branch}/registry.json` and the entries it lists, then applies the bundle's own product/PR/issue/file filters to the downloaded set. Path-list / `--files` filters match pool entries by file name, so the listed paths do not need to exist locally.

Use `--force-local` for uncommon ad hoc runs that need the local folder without editing `changelog.yml`. Path-list / `--files` filters always force local sourcing because they select files by path on disk.
Use `--force-local` for uncommon ad hoc runs that need the local folder without editing `changelog.yml` — including path-list / `--files` runs that should read freshly authored files from disk instead of the CDN pool.

Because entries are org/repo/branch-scoped, one repository can produce a bundle for a shared product (for example, `cloud-serverless`) while sourcing its own entries from `changelog/{org}/{repo}/{branch}/`, without that product appearing in the repository's `docset.yml`. The `{changelog}` directive's `:cdn:` mode still consumes product-scoped *bundles*, so a repository that also renders its own release notes declares each product under `release_notes` as before.

Expand Down
109 changes: 76 additions & 33 deletions src/services/Elastic.Changelog/Bundling/ChangelogBundlingService.cs
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,9 @@ public record BundleChangelogsArguments

/// <summary>
/// Explicit changelog YAML paths (or a path-list file) for the <c>--files</c> filter.
/// Mutually exclusive with other filter sources. Forces local entry sourcing.
/// Mutually exclusive with other filter sources. Follows the standard entry-sourcing gate:
/// when entries are sourced from the CDN the paths are matched to pool entries by file name,
/// otherwise they must exist on the local filesystem.
/// </summary>
public string[]? Files { get; init; }

Expand Down Expand Up @@ -231,11 +233,13 @@ public async Task<bool> BundleChangelogs(IDiagnosticsCollector collector, Bundle
// an org/repo/branch pool (changelog/{org}/{repo}/{branch}/...), so CDN sourcing keys off the
// resolvable authoring repo (bundle.repo / --repo), with org and branch defaulting when unset —
// not the bundle's target products. Fall back to the local folder when the user forces it
// (bundle.use_local_changelogs / --force-local / --files / --directory), the repo is unresolvable,
// (bundle.use_local_changelogs / --force-local / --directory), the repo is unresolvable,
// or no CDN base is configured. This stays in lockstep with PlanBundleAsync's needs_network decision.
// The --files / path-list filter follows the same gate: in CDN mode the requested paths are
// matched to pool entries by file name, so private repos whose entries exist only in S3 (with
// PR/issue references scrubbed from the public copies) can still bundle by explicit selection.
var useLocalChangelogs = (config?.Bundle?.UseLocalChangelogs ?? false)
|| input.ForceLocal
|| input.Files is { Length: > 0 };
|| input.ForceLocal;
var authoringRepo = ChangelogRepoOwnerResolver.NormalizeRepo(input.Repo);
var authoringOwner = ChangelogRepoOwnerResolver.ResolveOwner(input.Owner, input.Repo, DefaultOwner);
var authoringBranch = string.IsNullOrWhiteSpace(input.Branch) ? DefaultBranch : input.Branch;
Expand All @@ -253,14 +257,27 @@ public async Task<bool> BundleChangelogs(IDiagnosticsCollector collector, Bundle
var prsToMatch = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
var issuesToMatch = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
IReadOnlyList<string>? explicitFilePaths = null;
IReadOnlyList<string>? requestedEntryNames = null;

if (input.Files is { Length: > 0 })
{
var fileFilterLoader = new FileFilterLoader(_fileSystem);
var fileFilterResult = await fileFilterLoader.LoadFilesAsync(collector, input.Files, input.Directory, ctx);
if (!fileFilterResult.IsValid)
return false;
explicitFilePaths = fileFilterResult.FilePaths;
if (useCdn)
{
// CDN mode: reduce the requested paths to entry file names (the pool is flat); the
// entries do not need to exist locally.
var namesResult = await fileFilterLoader.LoadFileNamesAsync(collector, input.Files, ctx);
if (!namesResult.IsValid)
return false;
requestedEntryNames = namesResult.FilePaths;
}
else
{
var fileFilterResult = await fileFilterLoader.LoadFilesAsync(collector, input.Files, input.Directory, ctx);
if (!fileFilterResult.IsValid)
return false;
explicitFilePaths = fileFilterResult.FilePaths;
}
}
else if (input.Prs is { Length: > 0 })
{
Expand Down Expand Up @@ -289,7 +306,8 @@ public async Task<bool> BundleChangelogs(IDiagnosticsCollector collector, Bundle
var filterCriteria = BuildFilterCriteria(input, prsToMatch, issuesToMatch);

// Source and match changelog entries — from the CDN (default) or the local folder.
// Explicit --files / path-list selection always loads the named local paths (IncludeAll).
// Explicit --files / path-list selection bypasses content filters (IncludeAll): locally it loads
// the named paths, in CDN mode it selects pool entries by file name.
var entryMatcher = new ChangelogEntryMatcher(_fileSystem, ReleaseNotesSerialization.GetEntryDeserializer(), _logger);
ChangelogMatchResult matchResult;
if (explicitFilePaths != null)
Expand All @@ -303,7 +321,18 @@ public async Task<bool> BundleChangelogs(IDiagnosticsCollector collector, Bundle
var contents = await FetchCdnEntriesAsync(collector, authoringOwner, authoringRepo, authoringBranch, ctx);
if (contents == null)
return false;
matchResult = entryMatcher.MatchChangelogContents(collector, contents, filterCriteria, ctx);
if (requestedEntryNames is not null)
{
var poolLabel = $"{authoringOwner}/{authoringRepo}/{authoringBranch}";
var selected = SelectRequestedCdnEntries(collector, contents, requestedEntryNames, poolLabel);
if (selected == null)
return false;
_logger.LogInformation("Matching {Count} explicitly selected changelog entries from the CDN", selected.Count);
var filesCriteria = filterCriteria with { IncludeAll = true };
matchResult = entryMatcher.MatchChangelogContents(collector, selected, filesCriteria, ctx);
}
else
matchResult = entryMatcher.MatchChangelogContents(collector, contents, filterCriteria, ctx);
}
else
{
Expand Down Expand Up @@ -684,9 +713,7 @@ private BundleChangelogsArguments ApplyConfigDefaults(BundleChangelogsArguments
// active when the authoring repo resolves (profile/config bundle.repo), the user has not forced
// local sourcing, and a CDN base is configured.
var useLocalChangelogs = (config?.Bundle?.UseLocalChangelogs ?? false)
|| input.ForceLocal
|| input.Files is { Length: > 0 }
|| await ProfileFilterForcesLocalAsync(input, ctx);
|| input.ForceLocal;
var explicitDirectory = !string.IsNullOrWhiteSpace(input.Directory);
var authoringRepo = ChangelogRepoOwnerResolver.NormalizeRepo(input.Repo ?? profileDef?.Repo ?? config?.Bundle?.Repo);
if (ShouldSourceFromCdn(authoringRepo, useLocalChangelogs: useLocalChangelogs, explicitDirectory: explicitDirectory))
Expand Down Expand Up @@ -824,7 +851,7 @@ private BundleChangelogsArguments ApplyConfigDefaults(BundleChangelogsArguments
return byName.Select(kv => (kv.Key, kv.Value)).ToList();
}

/// <summary>Gate for repo-scoped CDN entry sourcing: true when the authoring repo resolves, local sourcing is not forced (<c>bundle.use_local_changelogs</c>/<c>--force-local</c>/<c>--files</c>/<c>--directory</c>), and a CDN base is configured.</summary>
/// <summary>Gate for repo-scoped CDN entry sourcing: true when the authoring repo resolves, local sourcing is not forced (<c>bundle.use_local_changelogs</c>/<c>--force-local</c>/<c>--directory</c>), and a CDN base is configured.</summary>
private static bool ShouldSourceFromCdn(string? authoringRepo, bool useLocalChangelogs, bool explicitDirectory)
{
if (useLocalChangelogs || explicitDirectory || string.IsNullOrWhiteSpace(authoringRepo))
Expand All @@ -833,30 +860,46 @@ private static bool ShouldSourceFromCdn(string? authoringRepo, bool useLocalChan
}

/// <summary>
/// Detects whether a profile positional list file is a changelog path list (which forces local sourcing).
/// Used by <see cref="PlanBundleAsync"/> so <c>needs_network</c> matches run-mode without emitting filter diagnostics.
/// Selects the CDN-sourced entries whose file names were explicitly requested via <c>--files</c> / a
/// path list. Every requested name must exist in the pool: the registry is the source of truth for
/// what was uploaded, so a missing name means the entry never reached S3 (or the name is wrong) and
/// silently shipping an incomplete bundle is worse than failing the run. Returns <c>null</c> after
/// emitting an error when any requested name is missing.
/// </summary>
private async Task<bool> ProfileFilterForcesLocalAsync(BundleChangelogsArguments input, Cancel ctx)
private IReadOnlyList<(string FileName, string Content)>? SelectRequestedCdnEntries(
IDiagnosticsCollector collector,
IReadOnlyList<(string FileName, string Content)> contents,
IReadOnlyList<string> requestedEntryNames,
string poolLabel)
{
if (string.IsNullOrWhiteSpace(input.Profile))
return false;

var listPath = !string.IsNullOrWhiteSpace(input.ProfileReport)
? input.ProfileReport
: input.ProfileArgument;
if (string.IsNullOrWhiteSpace(listPath) || !_fileSystem.File.Exists(listPath))
return false;
var byName = new Dictionary<string, string>(StringComparer.Ordinal);
foreach (var (fileName, content) in contents)
byName[fileName] = content;

if (_fileSystem.Path.GetExtension(listPath).ToLowerInvariant() is ".html" or ".htm")
return false;
var selected = new List<(string FileName, string Content)>();
var missing = new List<string>();
var seen = new HashSet<string>(StringComparer.Ordinal);
foreach (var name in requestedEntryNames)
{
if (!seen.Add(name))
continue;
if (byName.TryGetValue(name, out var content))
selected.Add((name, content));
else
missing.Add(name);
}

var content = await _fileSystem.File.ReadAllTextAsync(listPath, ctx);
var lines = content
.Split('\n', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries)
.Where(l => !string.IsNullOrWhiteSpace(l))
.ToArray();
if (missing.Count > 0)
{
collector.EmitError(string.Empty,
$"Changelog entr{(missing.Count == 1 ? "y" : "ies")} not found in the CDN pool '{poolLabel}': {string.Join(", ", missing)}. " +
"Ensure the entries were uploaded (changelog upload), or pass --force-local / --directory to bundle local files instead.");
return null;
}

return lines.Length > 0 && lines.All(FileFilterLoader.IsYamlExtension);
_logger.LogInformation("Selected {Selected} of {Total} CDN entries by requested file name for {Pool}",
selected.Count, contents.Count, poolLabel);
return selected;
}

private bool ValidateInput(IDiagnosticsCollector collector, BundleChangelogsArguments input, bool requireDirectoryExists)
Expand Down
Loading
Loading