Skip to content

Hatch only the UK areas a breakdown lists as withheld, and show others as no data - #1455

Draft
anth-volk wants to merge 5 commits into
mainfrom
feat/uk-hex-maps-listed-withheld
Draft

anth-volk wants to merge 5 commits into
mainfrom
feat/uk-hex-maps-listed-withheld

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #1454

Merge order

Merge this after the simulation API change, PolicyEngine/policyengine-sim-api#968, is deployed, and after the UK Budget on 28 October. It doesn't need the Public API change (PolicyEngine/policyengine-api#4091) first. Until the simulation API sends the lists, every breakdown reads as one without them, and the maps and note behave exactly as they do on main.

What changes

A completed breakdown=local_areas result from PolicyEngine/policyengine-sim-api#968 on carries two more fields:

  • constituency_withheld_codes: string[] | null
  • local_authority_withheld_codes: string[] | null

Each holds the ONS codes of that level's areas that the run estimated but disclosure control didn't publish, sorted. [] means none were withheld. null or an absent field means the result doesn't say.

When a level's list is present, the v2 and v3 hex maps:

  • hatch as "Not reported" only the listed codes that are in the report region's layout and that the result doesn't publish;
  • draw every other area of the region that the result lacks as "No data": the map's background colour inside the same grey edge as the hatched areas, with a "name: No data" tooltip. The colour scale never produces that fill (its lightest colour, at zero, is background.tertiary). These areas aren't in the colour range or the rankings, and can't be selected.

The note under the map counts the listed areas, by the same rule the map uses, so the note and the map still agree. An empty list hatches nothing and gives no withheld note.

The geographic map follows the same list: a boundary without a result reads "Not reported" in its tooltip only when the list names it, and "No data" otherwise. Its fill doesn't change: every boundary without a result stays grey, as on main.

When a level's list is null or absent, as in a breakdown computed before the change, the maps and the note behave exactly as they do today: every area of the region that the result lacks is hatched, counted, and reads "Not reported" on the geographic map. A result that can't be matched to the layout (an area with no code, or a code outside the region's layout) still gives no count and no hatched or no-data areas, with or without a list.

How

  1. HexagonalMap takes noDataAreas. They're drawn on the same grid as the valued and missing areas, count towards the map's bounds, and have no click handler.
  2. The shared rule takes the list. withheldUKAreaCodes in ukHexLayout.ts gets an optional listedWithheldCodes argument and returns noDataCodes beside codes. placeUKHexMap passes the list through and returns noDataAreas beside withheldAreas. countWithheldUKAreas counts from the list and carries it in its result as listedCodes.
  3. The breakdown's lists reach the maps and the note. LocalAreaBreakdown gains both fields. fetchLocalAreaBreakdown and normalizeSocietyWideOutput already pass top-level fields through unchanged, and a new test checks that they do. withheldUKAreasOf reads each level's list, treating a value that isn't a list as no list. v3's UKLocalAreaMaps and v2's four charts pass withheld.listedCodes to placeUKHexMap and noDataAreas to HexagonalMap.
  4. The geographic map reads the list too. This was found in review. UKAreaChoroplethMap takes missingAreaTextCodes, the codes that its missingAreaText describes; any other boundary without a result reads "No data". v3 and the four v2 charts pass the level's listed codes.
  5. One path for the hex map's withheld areas. withheldUKHexAreas had no caller left except tests, so it's removed, and its tests read placeUKHexMap instead. UKWithheldAreas documents why the note and the maps agree.

app/src/data/ukHexLayouts/README.md describes the new behaviour.

Not changed

  • A no-data boundary on the geographic map keeps the same grey fill as a withheld one; only its tooltip differs. Hatching the geographic map would be a separate change.
  • The note gives no count of the no-data areas. Their tooltips say "No data".

Overlap with #1436

#1436 also edits ukHexLayout.ts, HexagonalMap and the four v2 charts: it removes x and y and the untiled grid. This branch is based on main, not on #1436. Any overlap is left as a text conflict for whichever PR merges second.

Testing

The work was done red-green: each commit's tests were written first and seen failing for the expected reason, then the code was written, and both were committed together.

  • Let the hex map draw areas with no data…: six HexagonalMap tests for the fill, position, tooltip, selection and bounds failed with no no-data hexagon drawn.
  • Split a region's missing UK areas…: fifteen ukHexLayout and countWithheldUKAreas tests failed, because the list was ignored and there were no noDataCodes or noDataAreas.
  • Hatch only the UK areas a breakdown lists as withheld…: twenty-two tests for v3's maps, v2's four charts, withheldUKAreasOf, and the v3 section and v2 tabs end to end failed. Every absent area was hatched, none was drawn as no data, and the notes counted 649 and 360 where the lists gave 2 and 1.
  • Read "Not reported" on the geographic map only for a breakdown's listed areas: fourteen tests failed because every boundary without a result read "Not reported". They cover UKAreaChoroplethMap, v3's maps, v2's four charts, and, end to end, the v3 section and v2's constituency tab: "2 of 650 constituencies are not reported" with exactly two boundaries reading it.
  • Drop withheldUKHexAreas… is a refactor with documentation, so it has no red step. Its tests now read placeUKHexMap and still check the same things.

Each case has a test for v3 and for each v2 chart: a listed code is hatched; an absent area that isn't listed shows "No data"; [] hatches nothing; a null list behaves as today. The v3 section and the v2 constituency and local-authority tabs check end to end that the note counts the list.

Only the test files these changes touch were run locally, on Node 26, with make format, make lint and make typecheck. CI runs the full suite.

🤖 Generated with Claude Code

anth-volk and others added 3 commits October 28, 2026 15:33
HexagonalMap draws each of its missing areas hatched with a grey edge,
with a tooltip of its name and missingAreaText ("Not reported" on the UK
maps). It now also takes noDataAreas: areas it places but has no value
or reason for. Each is drawn on the same grid in the map's background
colour, which the colour scale never produces, inside the same grey
edge, with a tooltip of its name and "No data". Like missing areas, they
take no part in the colour range, cannot be selected, and count towards
the map's bounds. No map passes them yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…odes

withheldUKAreaCodes, the one rule behind the hex map's withheld areas
and the count in the note under it, took every area of the report's
region that a result leaves out as withheld. It now takes an optional
list of withheld ONS codes. With a list, even an empty one, an absent
area is withheld only when the list names it, and is returned among
noDataCodes otherwise; a listed code outside the region's layout, or
one the result publishes, is not withheld. Without a list (null or
absent) it behaves as before, with no no-data codes. A result that
cannot be matched to the layout still gives nothing.

placeUKHexMap passes the list through and returns noDataAreas beside
withheldAreas, on the tiled grid of a disclosure-controlled level only.
countWithheldUKAreas counts the listed areas and carries the list in its
result, as listedCodes, for the maps to draw from. No caller passes a
list yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s as no data

The v2 and v3 hex maps hatched every area of a breakdown's region that
the result leaves out as "Not reported", and the note under them counted
those areas, though an area can be missing for another reason, such as
having no households in the local-area dataset (#1454). From
PolicyEngine/policyengine-sim-api#968 on, a completed breakdown lists
each level's withheld ONS codes, in constituency_withheld_codes and
local_authority_withheld_codes.

LocalAreaBreakdown gains both fields, which the fetch and the normalizer
already pass through as they came. withheldUKAreasOf reads each level's
list (a value that is not a list reads as none) into countWithheldUKAreas,
so the note counts the listed areas, and the count carries the list as
listedCodes. v3's UKLocalAreaMaps and v2's four charts pass it to
placeUKHexMap, and draw its noDataAreas: each area the result leaves out
that the list does not name, in the map's background colour with a grey
edge, outside the colour scale, with a "No data" tooltip, not selectable
and not ranked. An empty list hatches nothing. A breakdown without the
lists, null or absent, draws and counts as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
policyengine-calculator-next Ready Ready Preview Oct 28, 2026 12:06pm UTC
policyengine-website Ready Ready Preview Oct 28, 2026 12:06pm UTC

Request Review

anth-volk and others added 2 commits October 28, 2026 16:01
…ed areas

Once a breakdown lists its withheld codes, the note under the UK maps
counts only the listed areas and the hex map hatches only those, but the
geographic map still read "Not reported" on every boundary without a
result: "2 of 650 constituencies are not reported" sat over 649
boundaries saying so (round 1 of the PR's review).

UKAreaChoroplethMap takes missingAreaTextCodes, the codes its
missingAreaText describes; any other boundary without a result reads
"No data". Without the codes, every one reads missingAreaText, as
before. v3's UKLocalAreaMaps and v2's four charts pass the breakdown
level's listed codes, so the geographic map, the hex map and the note
agree. A breakdown without the lists reads as it did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
placeUKHexMap no longer called withheldUKHexAreas, so only tests did:
the two ran the same rule down two paths. It is removed, and its tests,
and the adapter's tests that the note's count equals the hatched areas,
now read placeUKHexMap's withheldAreas, with a pointsWithCodes fixture
for points that carry only codes. placeUKHexMap's documentation takes
over the parts that described the areas themselves.

UKWithheldAreas now says why the note and the maps agree: the count and
the hex map pass the same codes, region and list to withheldUKAreaCodes,
and both give nothing for a result that cannot be matched to the
layout, so a null count needs no list. UK_WITHHELD_AREA_TEXT says that
with a list only the listed areas read it. A refactor and documentation:
the tests pass unchanged in what they check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

2 active deployments
Preview – policyengine-website — 9127c56f Deployed Oct 28, 2026 by vercel[bot]
Preview – policyengine-calculator-next — 9127c56f Deployed Oct 28, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Hatch only the UK areas a breakdown lists as withheld, and show other missing areas as no data

1 participant