Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions workspaces/scorecard/.changeset/hungry-walls-burn.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
'@red-hat-developer-hub/backstage-plugin-scorecard': minor
'@red-hat-developer-hub/backstage-plugin-scorecard-backend': minor
'@red-hat-developer-hub/backstage-plugin-scorecard-common': minor
---

Skip scalar aggregation threshold coloring when no successful samples contributed (`total` is 0); return a null display color and keep the card grey fallback.
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ export const mockScalarAggregationResult: ScalarAggregationResult = {
timestamp: '2025-01-01T10:30:00.000Z',
entitiesConsidered: 2,
calculationErrorCount: 0,
aggregationChartDisplayColor: 'warning.main',
};

export const mockWeightedStatusScoreAggregationResult: WeightedStatusScoreAggregationResult =
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -269,23 +269,24 @@ Use this endpoint for all new integrations.
The response shape includes **`id`**, **`status`**, **`metadata`** (title, description, type, unit, visualization, aggregation type, and **`filter`** when configured), and **`result`**. The shape of **`result`** depends on the aggregation type:

- **`statusGrouped`**: counts per threshold rule, **`total`**, **`thresholds`**, **`entitiesConsidered`**, **`calculationErrorCount`**, **`timestamp`**.
- **`weightedStatusScore`**: same as status-grouped, plus **`weightedStatusScore`** (portfolio percentage in \[0, 100\], one decimal), **`weightedStatusSum`**, **`weightedStatusMaxPossible`**, and **`aggregationChartDisplayColor`** (see backend README). The homepage card shows a donut gauge for this type instead of a multi-slice status pie.
- **Scalar types** (`sum`, `average`, `max`, `min`, `count`): see [Scalar result fields](#scalar-result-fields) below. When **`filter.status`** is configured, **`metadata.filter`** is also returned.
- **`weightedStatusScore`**: same as status-grouped, plus **`weightedStatusScore`** (portfolio percentage in \[0, 100\], one decimal), **`weightedStatusSum`**, **`weightedStatusMaxPossible`**, and **`aggregationChartDisplayColor`** (color from the **first** matching rule in **`thresholds`** against **`weightedStatusScore`**; **`null`** when **`total`** is **0** — see backend README). The homepage card shows a donut gauge for this type instead of a multi-slice status pie.
- **Scalar types** (`sum`, `average`, `max`, `min`, `count`): see [Scalar result fields](#scalar-result-fields) below, including **`aggregationChartDisplayColor`**. When **`filter.status`** is configured, **`metadata.filter`** is also returned.

For a daily history of a **scalar** KPI over owned entities, see [`GET /aggregations/:aggregationId/time-series`](#get-aggregationsaggregationidtime-series).

### Scalar result fields

When **`metadata.aggregationType`** is one of **`sum`**, **`average`**, **`max`**, **`min`**, or **`count`**, **`result`** is a scalar aggregation payload:

| Field | Description |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **`value`** | Aggregated number from the KPI type (`sum` / `average` / `max` / `min` / `count`) over contributing latest non-null rows. Classified by **`options.thresholds`** when present. |
| **`total`** | How many latest rows contributed to **`value`** (non-null, calculation failures excluded, optionally narrowed by **`filter.status`**). For **`count`**, equals **`value`**. |
| **`entitiesConsidered`** | Owned entities in scope that have at least one latest row for this metric (includes calculation-error rows). |
| **`calculationErrorCount`** | Among **`entitiesConsidered`**, how many latest rows are metric calculation failures (`error_message` set and `value` null). |
| **`timestamp`** | Portfolio data freshness — ISO timestamp of the most recent latest row in scope (same merge rule as other aggregation types). |
| **`thresholds`** | Number-style rules for classifying **`value`**; from **`options.thresholds`** or **`DEFAULT_NUMBER_THRESHOLDS`** when omitted. |
| Field | Description |
| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **`value`** | Aggregated number from the KPI type (`sum` / `average` / `max` / `min` / `count`) over contributing latest non-null rows. Classified by **`options.thresholds`** when present. |
| **`total`** | How many latest rows contributed to **`value`** (non-null, calculation failures excluded, optionally narrowed by **`filter.status`**). For **`count`**, equals **`value`**. |
| **`entitiesConsidered`** | Owned entities in scope that have at least one latest row for this metric (includes calculation-error rows). |
| **`calculationErrorCount`** | Among **`entitiesConsidered`**, how many latest rows are metric calculation failures (`error_message` set and `value` null). |
| **`timestamp`** | Portfolio data freshness — ISO timestamp of the most recent latest row in scope (same merge rule as other aggregation types). |
| **`thresholds`** | Number-style rules for classifying **`value`**; from **`options.thresholds`** or **`DEFAULT_NUMBER_THRESHOLDS`** when omitted. |
| **`aggregationChartDisplayColor`** | Color from the **first** matching rule in **`thresholds`** against **`value`**. Standard keys (`success`, `warning`, `error`) use default theme colors when **`color`** is omitted. **`null`** when **`total`** is **0** (no contributing rows). |

Example scalar response with status filter:

Expand All @@ -306,6 +307,7 @@ Example scalar response with status filter:
"entitiesConsidered": 10,
"calculationErrorCount": 1,
"timestamp": "2026-02-17T10:30:00.000Z",
"aggregationChartDisplayColor": "rgb(224, 189, 108)",
"thresholds": {
"rules": [
{ "key": "success", "expression": "<100" },
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -211,7 +211,7 @@ These thresholds are **not** per-entity metric rules. They apply only to homepag

- **Configuration path:** `scorecard.aggregationKPIs.<aggregationId>.options.thresholds`

- **YAML shape:** Same as metric thresholds — a **`rules`** array of **`key`**, **`expression`**, and optional **`color`** (and optional **`icon`**, though icons are not used for the weightedStatusScore KPI donut). Custom keys in aggregation KPI thresholds **only require `color`** (not `icon`). Expressions are **number**-style and are evaluated against **`weightedStatusScore`**, the backend’s portfolio **percentage** in **`[0, 100]`** (one decimal; see [Entity Aggregation](./aggregation.md)). The **first** matching rule wins; its **`color`** is returned on the API as **`result.aggregationChartDisplayColor`**.
- **YAML shape:** Same as metric thresholds — a **`rules`** array of **`key`**, **`expression`**, and optional **`color`** (and optional **`icon`**, though icons are not used for the weightedStatusScore KPI donut). Custom keys in aggregation KPI thresholds **only require `color`** (not `icon`). Expressions are **number**-style and are evaluated against **`weightedStatusScore`**, the backend’s portfolio **percentage** in **`[0, 100]`** (one decimal; see [Entity Aggregation](./aggregation.md)). The **first** matching rule wins; its **`color`** is returned on the API as **`result.aggregationChartDisplayColor`** (or **`null`** when **`result.total`** is **0**).

- **Defaults:** If **`thresholds`** is omitted from app-config under **`options`**, it is not injected at config-parse time. **`WeightedStatusScoreAggregationStrategy`** applies **`DEFAULT_WEIGHTED_STATUS_SCORE_KPI_RESULT_THRESHOLDS`** from [`src/constants/aggregationKPIs.ts`](../src/constants/aggregationKPIs.ts) when serving an aggregation: **`<30`** → error, **`30-79`** → warning, **`>=80`** → success (higher percentage = better). When that default path is used, the strategy logs at **info** that the built-in 0–100% scale is in effect.

Expand All @@ -225,7 +225,7 @@ These thresholds are **not** per-entity metric rules. They apply to homepage agg

- **Configuration path:** `scorecard.aggregationKPIs.<aggregationId>.options.thresholds`

- **YAML shape:** Same as metric thresholds — a **`rules`** array of **`key`**, **`expression`**, and optional **`color`** (and optional **`icon`**). Expressions are **number**-style and are evaluated against **`result.value`**, the aggregated scalar from the KPI (see [Entity Aggregation — Scalar result fields](./aggregation.md#scalar-result-fields)). The **first** matching rule wins; its **`color`** and **`key`** can be used by custom UIs that render scalar KPIs.
- **YAML shape:** Same as metric thresholds — a **`rules`** array of **`key`**, **`expression`**, and optional **`color`** (and optional **`icon`**). Expressions are **number**-style and are evaluated against **`result.value`**, the aggregated scalar from the KPI (see [Entity Aggregation — Scalar result fields](./aggregation.md#scalar-result-fields)). The **first** matching rule wins; its **`color`** is returned on the API as **`result.aggregationChartDisplayColor`** (or **`null`** when **`result.total`** is **0**).

- **Defaults:** If **`thresholds`** is omitted from app-config under **`options`**, **`ScalarAggregationStrategy`** applies **`DEFAULT_NUMBER_THRESHOLDS`** from scorecard-common when serving an aggregation and includes them on the API as **`result.thresholds`**: **`<10`** → success, **`10-50`** → warning, **`>50`** → error.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ import type { AggregationStrategy } from './types';
import { isScalarAggregationConfig } from '../../../utils/aggregation/isScalarAggregationConfig';
import { classifyNumberAgainstThresholds } from '../../../utils/aggregation/classifyNumberAgainstThresholds';
import { ThresholdEvaluator } from '../../../threshold/ThresholdEvaluator';
import { getRequiredAggregationChartDisplayColor } from '../../../utils/aggregation/getAggregationChartDisplayColor';

export class ScalarAggregationStrategy implements AggregationStrategy {
constructor(
Expand Down Expand Up @@ -66,12 +67,23 @@ export class ScalarAggregationStrategy implements AggregationStrategy {
aggregationConfig.filter,
);

Comment thread
imykhno marked this conversation as resolved.
const aggregationChartDisplayColor =
total > 0
? getRequiredAggregationChartDisplayColor(
value,
headlineThresholds,
this.thresholdEvaluator,
`The color for value '${value}' metric '${metric.id}' is not configured. Check the 'scorecard.aggregationKPIs.${aggregationConfig.id}.options.thresholds' configuration.`,
)
: null;

const result = {
value,
total,
entitiesConsidered,
calculationErrorCount,
timestamp,
aggregationChartDisplayColor,
thresholds: headlineThresholds,
} satisfies ScalarAggregationResult;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ import {
type AggregatedMetric,
type WeightedStatusScoreAggregationResult,
type AggregatedMetricResult,
type ThresholdConfig,
ThresholdRule,
aggregationTypes,
type StatusScoreAggregationOption,
Expand All @@ -30,13 +29,15 @@ import type { AggregationOptions } from '../types';
import type { AggregationStrategy } from './types';
import { LoggerService } from '@backstage/backend-plugin-api';
import { ThresholdEvaluator } from '../../../threshold/ThresholdEvaluator';
import { getRequiredAggregationChartDisplayColor } from '../../../utils/aggregation/getAggregationChartDisplayColor';

export class WeightedStatusScoreAggregationStrategy
implements AggregationStrategy
{
constructor(
private readonly loader: AggregatedMetricLoader,
private readonly logger: LoggerService,
private readonly thresholdEvaluator: ThresholdEvaluator = new ThresholdEvaluator(),
) {}

async aggregate({
Expand Down Expand Up @@ -77,16 +78,15 @@ export class WeightedStatusScoreAggregationStrategy
weightedSum,
);

Comment thread
imykhno marked this conversation as resolved.
const aggregationChartDisplayColor = this.getAggregationChartDisplayColor(
weightedStatusScore,
headlineThresholds,
);

if (!aggregationChartDisplayColor) {
throw new Error(
`The color for percentage '${weightedStatusScore}' metric '${metric.id}' is not configured. Check the 'scorecard.aggregationKPIs.${aggregationConfig.id}.options.thresholds' configuration.`,
);
}
const aggregationChartDisplayColor =
aggregatedMetric.total > 0
? getRequiredAggregationChartDisplayColor(
weightedStatusScore,
headlineThresholds,
this.thresholdEvaluator,
`The color for percentage '${weightedStatusScore}' metric '${metric.id}' is not configured. Check the 'scorecard.aggregationKPIs.${aggregationConfig.id}.options.thresholds' configuration.`,
)
: null;

const result = {
total: aggregatedMetric.total,
Expand Down Expand Up @@ -131,21 +131,6 @@ export class WeightedStatusScoreAggregationStrategy
return weightedSum;
}

private getAggregationChartDisplayColor(
scorePercent: number,
thresholds: ThresholdConfig,
): string | undefined {
const thresholdEvaluator = new ThresholdEvaluator();

const matchedThresholdKey = thresholdEvaluator.getFirstMatchingThreshold(
scorePercent,
'number',
thresholds,
);

return thresholds.rules.find(r => r.key === matchedThresholdKey)?.color;
}

private prepareWeightedStatusScoreValues(
numberOfEntities: Pick<AggregatedMetric, 'total'>['total'],
statusScores: StatusScoreAggregationOption,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,11 @@ describe('ScalarAggregationStrategy', () => {

expect(spyMethods.toAggregatedMetricResultSpy).toHaveBeenCalledWith(
metric,
{ ...loadedScalarMetric, thresholds: DEFAULT_NUMBER_THRESHOLDS },
{
...loadedScalarMetric,
thresholds: DEFAULT_NUMBER_THRESHOLDS,
aggregationChartDisplayColor: 'error.main',
},
defaultAggregationConfig,
);
});
Expand All @@ -144,7 +148,64 @@ describe('ScalarAggregationStrategy', () => {

expect(spyMethods.toAggregatedMetricResultSpy).toHaveBeenCalledWith(
metric,
{ ...loadedScalarMetric, thresholds: mockHigherIsBetterThresholds },
{
...loadedScalarMetric,
thresholds: mockHigherIsBetterThresholds,
aggregationChartDisplayColor: 'green',
},
aggregationConfig,
);
});

it('should throw when aggregation chart display color is not configured', async () => {
const aggregationConfigWithoutColors = mockScalarAggregationConfig(
aggregationTypes.sum,
{
id: 'totalOpenPrs',
metricId: metric.id,
options: {
thresholds: {
rules: [{ key: 'success', expression: '<10' }],
},
},
},
);

await expect(() =>
strategy.aggregate({
metric,
entityRefs,
thresholds: mockHigherIsBetterThresholds,
aggregationConfig: aggregationConfigWithoutColors,
}),
).rejects.toThrow(
`The color for value '${loadedScalarMetric.value}' metric '${metric.id}' is not configured. Check the 'scorecard.aggregationKPIs.totalOpenPrs.options.thresholds' configuration.`,
);
});

it('should set aggregationChartDisplayColor to null when total is 0', async () => {
(loader.loadScalarMetricByEntityRefs as jest.Mock).mockResolvedValueOnce({
...loadedScalarMetric,
value: 0,
total: 0,
});

await strategy.aggregate({
metric,
entityRefs,
thresholds: mockHigherIsBetterThresholds,
aggregationConfig,
});

expect(spyMethods.toAggregatedMetricResultSpy).toHaveBeenCalledWith(
metric,
{
...loadedScalarMetric,
value: 0,
total: 0,
thresholds: mockHigherIsBetterThresholds,
aggregationChartDisplayColor: null,
},
aggregationConfig,
);
});
Expand Down Expand Up @@ -200,7 +261,11 @@ describe('ScalarAggregationStrategy', () => {

expect(spyMethods.toAggregatedMetricResultSpy).toHaveBeenCalledWith(
metric,
{ ...loadedScalarMetric, thresholds: mockHigherIsBetterThresholds },
{
...loadedScalarMetric,
thresholds: mockHigherIsBetterThresholds,
aggregationChartDisplayColor: 'green',
},
aggregationConfig,
);
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -113,26 +113,39 @@ describe('WeightedStatusScoreAggregationStrategy', () => {
});

it('should throw when aggregation chart display color is not configured', async () => {
const customStatusThresholds = {
rules: [
{ key: 'ok', expression: '>=80', color: 'green' },
{ key: 'notOk', expression: '<80', color: 'red' },
],
};
const aggregationConfigWithoutColors =
mockWeightedStatusScoreAggregationConfig({
id: 'weightedOpenPrs',
metricId: metric.id,
options: {
statusScores: { error: 0, warning: 50, success: 100 },
statusScores: { notOk: 0, maybe: 50, ok: 100 },
thresholds: {
rules: [
{ key: 'success', expression: '>=80' },
{ key: 'error', expression: '<80' },
{ key: 'ok', expression: '>=80' },
{ key: 'notOk', expression: '<80' },
],
},
},
});

(
loader.loadStatusGroupedMetricByEntityRefs as jest.Mock
).mockResolvedValueOnce({
...loadedStatusGroupedMetric,
values: { ok: 2 },
});

await expect(() =>
strategy.aggregate({
metric,
entityRefs,
thresholds: mockHigherIsBetterThresholds,
thresholds: customStatusThresholds,
aggregationConfig: aggregationConfigWithoutColors,
}),
).rejects.toThrow(
Expand Down Expand Up @@ -193,6 +206,40 @@ describe('WeightedStatusScoreAggregationStrategy', () => {
);
});

it('should set aggregationChartDisplayColor to null when total is 0', async () => {
(
loader.loadStatusGroupedMetricByEntityRefs as jest.Mock
).mockResolvedValueOnce({
...loadedStatusGroupedMetric,
values: {},
total: 0,
});

await strategy.aggregate({
metric,
entityRefs,
thresholds: mockHigherIsBetterThresholds,
aggregationConfig,
});

expect(spyMethods.toAggregatedMetricResultSpy).toHaveBeenCalledWith(
metric,
{
...mappedWeightedResult,
values: [
{ name: 'success', count: 0, score: 100 },
{ name: 'error', count: 0, score: 0 },
],
weightedStatusScore: 0,
weightedStatusSum: 0,
weightedStatusMaxPossible: 0,
aggregationChartDisplayColor: null,
total: 0,
},
aggregationConfig,
);
});

it('should get aggregation result', async () => {
const result = await strategy.aggregate({
metric,
Expand Down
Loading
Loading