[robbrad]

Please merge with a merge commit or rebase, not a squash

This branch carries 15 commits authored by the original contributors, cherry-picked without -n so each keeps their authorship. Squashing would re-author all of them to whoever merges (the problem described in the playbook after #2276). My follow-up commits are separate and labelled. Every commit title is conventional (fix:/test:/docs:), so the release triggers either way.

Included from contributor PRs

From	Author	Council	What
#2315	@InertiaUK	East Hampshire	adds the missing LAD24CD (E07000085)
#2314	@InertiaUK	Torfaen	sends a desktop User-Agent (the site 403s HeadlessChrome) and adds LAD24CD
#2313	@InertiaUK	Newcastle	address matching is whole-word and postcode-constrained, and raises instead of falling back to parcels[0]
#2323	@adammccormack162	Charnwood	restores legacy property-URL support, strictly validated
#2324	@craiggraham	Lancaster	splits the red and yellow recycling streams
#2336	@Russell-IO	East Herts	follows the council's new lookup id and reports the council's own error
#2308	@InertiaUK	Canterbury	moves to the council's new bin-dates endpoint (old one 403'd for everyone, see #2215)
#2300	@bfayers	East Suffolk	strips bracketed bin descriptors
#2295	@ryanocallaghan	Gedling	rewrites onto the council's new Netcall portal with plain requests
#2330	contributor	HA calendar	events overlapping the requested range are returned (was start-only)
#2310	@InertiaUK	Wolverhampton	handles the council's new page layout

My follow-up commits

I read every diff and added what I'd have asked the contributors for:

• Canterbury: a 403 now says what it usually means instead of a bare HTTPError; 6 tests (PR had none).
• East Suffolk: the bracket rule replaced the " - standard bin" rule, so old wording would regress. Both handled in a testable function; 7 tests.
• Newcastle: 12 tests pinning that "1" must not match 10, 11, 1A or a flat in No. 12 (the failure that silently returns another household's days).
• Gedling: two addresses starting the same way now raise and list them; a page with no collections raises instead of returning []; 2 tests.
• HA calendar: BST-aware test that fails on the old code.
• Wolverhampton: address matching is start-anchored and never guesses (it previously matched the number anywhere and fell back to the first address); only the innermost col-* columns are read (the outer wrapper repeated its first service); dates with/without a year; raises if no bins found. 25 tests, mutation-checked.
• Nuneaton & Bedworth (Targets #2334): five Exception(...) objects were built and never raised, so an unknown street returned an empty result and a failed street page crashed with UnboundLocalError. They now raise; an exact street name wins over longer ones containing it; ambiguity lists the candidates. 7 tests, all fail on the old code.
• Sevenoaks (Targets #2305): the dropdown was matched with paon in option.text, so "40" selected "140 ..." / "Flat 40" or nothing, and the page then timed out. Now start-anchored, whole-word, raises on none/ambiguous. 9 tests added; the existing test's mock dropdown had no "select an address" prompt, so I gave it one like the real page.
• docs: November notes in ISSUE_RESOLUTION_PROGRESS.md and a lessons section in the playbook.

Heads-up for the maintainer

• Lancaster is a breaking change for Home Assistant users. The single "Recycling" sensor becomes "Recycling Red" and "Recycling Yellow". Worth a line in the release notes.
• Gedling's test data looks residential. input.json gets NG5 6NQ / 11 (Gedling Road, Arnold). I haven't changed it because I can't verify a replacement offline. Decision for you.
• Live evidence is limited: this sandbox can't reach council sites, and CI can't reach Lancaster (Whitespace). The earlier CI run on the contributor batch: 6 of 8 council integration tests passed; Lancaster CI-blocked; Canterbury unverifiable (its API rejects unrecognised callers). Wolverhampton, Nuneaton and Sevenoaks have only offline tests so far; their CI run is on the latest head.
• Also noticed, not fixed here: Lancaster's address loop keeps the last matching address (its comment says "first").

Not included, with reasons

• fix: look up redditch uprn from postcode and house number #2316–fix: look up lambeth uprn from postcode and house number #2321 (postcode+house number → UPRN): all six paste a match_address whose third pattern matches the number anywhere in the label ("1" returns "Flat 1, 12 Main Street"). Several still require a UPRN in the HA form, and some test data looks residential. They need one shared, tested helper.
• fix: move east cambridgeshire to the council's new achieveforms form #2309 (East Cambridgeshire), feat: rewrite hyndburn as a plain http scraper #2312 (Hyndburn): same matching flaw plus unverifiable rewrites (28-day window down from 42, an API I can't check).
• fix: tighten south lanarkshire address matching #2311 (South Lanarkshire): breaks the repo's own unit tests.
• fix: Update Chelmsford Council to handle PDF instead of old ICS #2332 (Chelmsford): edits the auto-generated wiki/Councils.md, reads the PDF by hardcoded RGB colours, matches by substring.
• fix(swansea): search by postcode and never return another address's days #2252 (Swansea) is already fixed on master; dependabot PRs are in chore(deps): November dependency bumps (10 dependabot PRs, tested together) #2342.

Test plan

• make unit-tests on Python 3.14 with Home Assistant at the latest head: 630 passed; the 7 failures are the 3 sandbox-only network tests plus Orkney and Broxbourne ×3, all fixed by test: stop the Orkney and Broxbourne tests failing on the calendar and the network #2341.
• Every commit passes commitlint with the same rules as lint.yml.
• Offline parity: no missing or orphaned council keys.
• Contributor commits applied cleanly with authorship preserved.
• CI live results for the latest head: pending.

---

🤖 Generated with Claude Code

https://claude.ai/code/session_015MaCS29izthZp2dwtvLwf4

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 24 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 2 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 6c749914-1909-4b62-b49a-c977b90af3e1

Commits

Reviewing files that changed from the base of the PR and between 1fb4588 and a1c2617.

Files selected for processing: 26

• ISSUE_RESOLUTION_PROGRESS.md
• custom_components/uk_bin_collection/calendar.py
• custom_components/uk_bin_collection/tests/test_calendar.py
• docs/MAINTAINER_PLAYBOOK.md
• uk_bin_collection/tests/input.json
• uk_bin_collection/tests/test_canterbury_city_council.py
• uk_bin_collection/tests/test_charnwood_borough_council.py
• uk_bin_collection/tests/test_east_herts_council.py
• uk_bin_collection/tests/test_east_suffolk_council.py
• uk_bin_collection/tests/test_gedling_borough_council.py
• uk_bin_collection/tests/test_lancaster_city_council.py
• uk_bin_collection/tests/test_newcastle_city_council.py
• uk_bin_collection/tests/test_nuneaton_bedworth_borough_council.py
• uk_bin_collection/tests/test_sevenoaks_district_council.py
• uk_bin_collection/tests/test_wolverhampton_city_council.py
• uk_bin_collection/uk_bin_collection/councils/CanterburyCityCouncil.py
• uk_bin_collection/uk_bin_collection/councils/CharnwoodBoroughCouncil.py
• uk_bin_collection/uk_bin_collection/councils/EastHertsCouncil.py
• uk_bin_collection/uk_bin_collection/councils/EastSuffolkCouncil.py
• uk_bin_collection/uk_bin_collection/councils/GedlingBoroughCouncil.py
• uk_bin_collection/uk_bin_collection/councils/LancasterCityCouncil.py
• uk_bin_collection/uk_bin_collection/councils/NewcastleCityCouncil.py
• uk_bin_collection/uk_bin_collection/councils/NuneatonBedworthBoroughCouncil.py
• uk_bin_collection/uk_bin_collection/councils/SevenoaksDistrictCouncil.py
• uk_bin_collection/uk_bin_collection/councils/TorfaenCouncil.py
• uk_bin_collection/uk_bin_collection/councils/WolverhamptonCityCouncil.py

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[codecov]

❌ 2 Tests Failed:

Tests completed	Failed	Passed	Skipped
11	2	9	0

View the top 1 failed test(s) by shortest run time

uk_bin_collection.tests.step_defs.test_validate_council::test_scenario_outline[LancasterCityCouncil]

Stack Traces | 0.444s run time

fixturefunc = <function scrape_step at 0x738aa34e0250>
request = <FixtureRequest for <Function test_scenario_outline[LancasterCityCouncil]>>
kwargs = {'context': <test_validate_council.Context object at 0x738aa35470e0>, 'headless_mode': 'True', 'local_browser': 'False', 'selenium_url': 'http://localhost:4444'}

    def call_fixture_func(
        fixturefunc: _FixtureFunc[FixtureValue], request: FixtureRequest, kwargs
    ) -> FixtureValue:
        if inspect.isgeneratorfunction(fixturefunc):
            fixturefunc = cast(Callable[..., Generator[FixtureValue]], fixturefunc)
            generator = fixturefunc(**kwargs)
            try:
                fixture_result = next(generator)
            except StopIteration:
                raise ValueError(f"{request.fixturename} did not yield a value") from None
            finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)
            request.addfinalizer(finalizer)
        else:
            fixturefunc = cast(Callable[..., FixtureValue], fixturefunc)
>           fixture_result = fixturefunc(**kwargs)
                             ^^^^^^^^^^^^^^^^^^^^^

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.14/lib/python3.14.../site-packages/_pytest/fixtures.py:1005: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../tests/step_defs/test_validate_council.py:101: in scrape_step
    context.parse_result = CollectData.run()
                           ^^^^^^^^^^^^^^^^^
uk_bin_collection/uk_bin_collection/collect_data.py:109: in run
    return self.client_code(
uk_bin_collection/uk_bin_collection/collect_data.py:130: in client_code
    return get_bin_data_class.template_method(address_url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
uk_bin_collection/uk_bin_collection/get_bin_data.py:71: in template_method
    bin_data_dict = self.get_and_parse_data(this_url, **kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
uk_bin_collection/uk_bin_collection/get_bin_data.py:94: in get_and_parse_data
    bin_data_dict = self.parse_data("", url=address_url, **kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.../uk_bin_collection/councils/LancasterCityCouncil.py:53: in parse_data
    response = session.get(portal_link)
               ^^^^^^^^^^^^^^^^^^^^^^^^
../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.14/lib/python3.14........./site-packages/requests/sessions.py:671: in get
    return self.request("GET", url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.14/lib/python3.14........./site-packages/requests/sessions.py:635: in request
    prep = self.prepare_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.14/lib/python3.14........./site-packages/requests/sessions.py:541: in prepare_request
    p.prepare(
../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.14/lib/python3.14....../site-packages/requests/models.py:439: in prepare
    self.prepare_url(url, params)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <PreparedRequest [GET]>, url = '', params = OrderedDict()

    def prepare_url(
        self,
        url: _t.UriType,
        params: _t.ParamsType,
    ) -> None:
        """Prepares the given HTTP URL."""
        #: Accept objects that have string representations.
        #: We're unable to blindly call unicode/str functions
        #: as this will include the bytestring indicator (b'')
        #: on python 3.x.
        #: https://github..../requests/pull/2238
        if isinstance(url, bytes):
            url = url.decode("utf8")
        else:
            url = str(url)
    
        # Remove leading whitespaces from url
        url = url.lstrip()
    
        # Don't do any URL preparation for non-HTTP schemes like `mailto`,
        # `data` etc to work around exceptions from `url_parse`, which
        # handles RFC 3986 only.
        if ":" in url and not url.lower().startswith("http"):
            self.url = url
            return
    
        # Support for unicode domain names and paths.
        try:
            scheme, auth, host, port, path, query, fragment = parse_url(url)
        except LocationParseError as e:
            raise InvalidURL(*e.args)
    
        if not scheme:
>           raise MissingSchema(
                f"Invalid URL {url!r}: No scheme supplied. "
                f"Perhaps you meant https://{url}?"
            )
E           requests.exceptions.MissingSchema: Invalid URL '': No scheme supplied. Perhaps you meant https://?

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.14/lib/python3.14....../site-packages/requests/models.py:514: MissingSchema

View the full list of 1 ❄️ flaky test(s)

uk_bin_collection.tests.step_defs.test_validate_council::test_scenario_outline[CanterburyCityCouncil]

Flake rate in main: 99.71% (Passed 2 times, Failed 693 times)

Stack Traces | 0.382s run time

fixturefunc = <function scrape_step at 0x79fb070d4720>
request = <FixtureRequest for <Function test_scenario_outline[CanterburyCityCouncil]>>
kwargs = {'context': <test_validate_council.Context object at 0x79fb06f7bb60>, 'headless_mode': 'True', 'local_browser': 'False', 'selenium_url': 'http://localhost:4444'}

    def call_fixture_func(
        fixturefunc: _FixtureFunc[FixtureValue], request: FixtureRequest, kwargs
    ) -> FixtureValue:
        if inspect.isgeneratorfunction(fixturefunc):
            fixturefunc = cast(Callable[..., Generator[FixtureValue]], fixturefunc)
            generator = fixturefunc(**kwargs)
            try:
                fixture_result = next(generator)
            except StopIteration:
                raise ValueError(f"{request.fixturename} did not yield a value") from None
            finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)
            request.addfinalizer(finalizer)
        else:
            fixturefunc = cast(Callable[..., FixtureValue], fixturefunc)
>           fixture_result = fixturefunc(**kwargs)
                             ^^^^^^^^^^^^^^^^^^^^^

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.14/lib/python3.14.../site-packages/_pytest/fixtures.py:1005: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../tests/step_defs/test_validate_council.py:101: in scrape_step
    context.parse_result = CollectData.run()
                           ^^^^^^^^^^^^^^^^^
uk_bin_collection/uk_bin_collection/collect_data.py:109: in run
    return self.client_code(
uk_bin_collection/uk_bin_collection/collect_data.py:130: in client_code
    return get_bin_data_class.template_method(address_url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
uk_bin_collection/uk_bin_collection/get_bin_data.py:71: in template_method
    bin_data_dict = self.get_and_parse_data(this_url, **kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
uk_bin_collection/uk_bin_collection/get_bin_data.py:94: in get_and_parse_data
    bin_data_dict = self.parse_data("", url=address_url, **kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <CanterburyCityCouncil.CouncilClass object at 0x79fb06e2c2f0>, page = ''
kwargs = {'url': 'https://www.canterbury.gov.uk', 'postcode': None, 'paon': None, 'uprn': '10094583181', ...}

    def parse_data(self, page: str, **kwargs) -> dict:
        user_uprn = kwargs.get("uprn")
        check_uprn(user_uprn)
        bindata = {"bins": []}
    
        # The API keys on the UPRN; the USRN is required in the payload but
        # its value is not used for the lookup.
        data = {"uprn": str(user_uprn), "usrn": "1"}
    
        # The API Gateway rejects requests without a browser-like User-Agent.
        headers = {
            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)",
            "Accept": "application/json",
            "Origin": "https://www.canterbury.gov.uk",
            "Referer": "https://www.canterbury.gov.uk/",
        }
    
        response = requests.post(self.API_URL, json=data, headers=headers, timeout=30)
        if response.status_code == 403:
>           raise ConnectionError(
                "Canterbury's bin collection API returned 403 Forbidden. It "
                "rejects callers it doesn't recognise, so this is a block on "
                "your network or an outage on the council's side, not a bad UPRN."
            )
E           ConnectionError: Canterbury's bin collection API returned 403 Forbidden. It rejects callers it doesn't recognise, so this is a block on your network or an outage on the council's side, not a bad UPRN.

.../uk_bin_collection/councils/CanterburyCityCouncil.py:53: ConnectionError

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}