From f03fee5ab0373ed25c770acf7e99871b5c93ec94 Mon Sep 17 00:00:00 2001 From: reacher-z <68239571+reacher-z@users.noreply.github.com> Date: Tue, 17 Nov 2026 01:25:09 -0700 Subject: [PATCH] docs: name the website behind each task in tasks.md Task names do not always say which site they reproduce: 33 is Coursera, kanetix-ca is Kanetix / Rates.ca, and medium reads like a difficulty level. Add a Site column taken from each task.toml description, without the "Offline Clone" and "Offline Harbor" suffixes some descriptions carry. --- docs/tasks.md | 148 +++++++++++++++++++++++++------------------------- 1 file changed, 74 insertions(+), 74 deletions(-) diff --git a/docs/tasks.md b/docs/tasks.md index 695f93c8..0d4bd875 100644 --- a/docs/tasks.md +++ b/docs/tasks.md @@ -1,80 +1,80 @@ # Tasks -WebsiteBench has 70 tasks with 46,916 hidden test cases in total. Each row lists the task's case count and the size of the complete task download. Every task requests 8 CPUs, 8 GiB of memory and 20 GiB of disk for each agent and verifier environment, with a 20-minute build timeout. Every task gives the agent 16 hours and the verifier 8 hours. +WebsiteBench has 70 tasks with 46,916 hidden test cases in total. Each row lists the website the task reproduces, the task's case count and the size of the complete task download. Every task requests 8 CPUs, 8 GiB of memory and 20 GiB of disk for each agent and verifier environment, with a 20-minute build timeout. Every task gives the agent 16 hours and the verifier 8 hours. -Start with `medium`: it is small to download. +Start with `medium`, a reproduction of Medium: it is small to download. -| Task | Cases | Download | -| --- | ---: | ---: | -| `1password-web` | 679 | 8 MB | -| `33` | 765 | 73 MB | -| `ace-hardware` | 433 | 17 MB | -| `airtable` | 589 | 22 MB | -| `amazon` | 1,025 | 218 MB | -| `amazon-home-services` | 799 | 17 MB | -| `amc-theatres` | 369 | 17 MB | -| `angi` | 1,000 | 46 MB | -| `apple` | 700 | 421 MB | -| `article` | 1,759 | 254 MB | -| `asana` | 346 | 354 MB | -| `aspca-pet-insurance` | 1,019 | 145 MB | -| `autoslash` | 353 | 43 MB | -| `autotrader` | 487 | 9 MB | -| `backcountry` | 1,317 | 590 MB | -| `bean-box` | 987 | 5 MB | -| `beeradvocate` | 446 | 6 MB | -| `blinkist` | 437 | 4 MB | -| `blueapron` | 410 | 85 MB | -| `bluemercury` | 694 | 95 MB | -| `boardgamegeek` | 1,020 | 47 MB | -| `bookshop` | 632 | 63 MB | -| `breakoutgames-2` | 387 | 27 MB | -| `brooklinen` | 736 | 93 MB | -| `capterra` | 632 | 162 MB | -| `change` | 420 | 45 MB | -| `columbia` | 338 | 360 MB | -| `crumbl-cookies` | 531 | 179 MB | -| `crunchyroll` | 681 | 22 MB | -| `curology` | 539 | 34 MB | -| `dooney-bourke` | 754 | 10 MB | -| `eatthismuch` | 668 | 2.4 GB | -| `etsy` | 347 | 19 MB | -| `eventbrite` | 1,655 | 44 MB | -| `everydollar` | 394 | 11 MB | -| `everyplate` | 2,016 | 72 MB | -| `fandango` | 387 | 430 MB | -| `freshdirect` | 990 | 7 MB | -| `hipcamp` | 302 | 68 MB | -| `hmart` | 714 | 161 MB | -| `imdb` | 509 | 463 MB | -| `insureon` | 522 | 41 MB | -| `insurify` | 350 | 53 MB | -| `ioby` | 380 | 85 MB | -| `kanetix-ca` | 809 | 40 MB | -| `leetcode` | 326 | 14 MB | -| `legalnature` | 389 | 336 MB | -| `linkedin` | 565 | 42 MB | -| `llbean` | 423 | 149 MB | -| `lowes` | 494 | 26 MB | -| `medium` | 650 | 7 MB | -| `meetup` | 347 | 111 MB | -| `notion` | 393 | 12 MB | -| `ollie` | 394 | 56 MB | -| `ontario-parks` | 313 | 4 MB | -| `opentable` | 512 | 21 MB | -| `petco` | 609 | 24 MB | -| `petfinder` | 1,783 | 450 MB | -| `petsmart` | 837 | 42 MB | -| `ratemyprofessors` | 721 | 7 MB | -| `resy` | 519 | 111 MB | -| `simplify-jobs` | 752 | 5 MB | -| `target` | 537 | 193 MB | -| `taskrabbit` | 871 | 123 MB | -| `tripadvisor` | 625 | 17 MB | -| `tripit` | 478 | 49 MB | -| `ubereats` | 338 | 25 MB | -| `warbyparker` | 992 | 288 MB | -| `wayfair` | 1,163 | 7 MB | -| `zennioptical` | 558 | 146 MB | +| Task | Site | Cases | Download | +| --- | --- | ---: | ---: | +| `1password-web` | 1Password Web | 679 | 8 MB | +| `33` | Coursera | 765 | 73 MB | +| `ace-hardware` | Ace Hardware | 433 | 17 MB | +| `airtable` | Airtable | 589 | 22 MB | +| `amazon` | Amazon | 1,025 | 218 MB | +| `amazon-home-services` | Amazon Home Services | 799 | 17 MB | +| `amc-theatres` | AMC Theatres | 369 | 17 MB | +| `angi` | Angi | 1,000 | 46 MB | +| `apple` | Apple | 700 | 421 MB | +| `article` | Article | 1,759 | 254 MB | +| `asana` | Asana | 346 | 354 MB | +| `aspca-pet-insurance` | ASPCA Pet Health Insurance | 1,019 | 145 MB | +| `autoslash` | AutoSlash | 353 | 43 MB | +| `autotrader` | Autotrader | 487 | 9 MB | +| `backcountry` | Backcountry | 1,317 | 590 MB | +| `bean-box` | Bean Box | 987 | 5 MB | +| `beeradvocate` | BeerAdvocate | 446 | 6 MB | +| `blinkist` | Blinkist | 437 | 4 MB | +| `blueapron` | Blue Apron | 410 | 85 MB | +| `bluemercury` | bluemercury | 694 | 95 MB | +| `boardgamegeek` | BoardGameGeek | 1,020 | 47 MB | +| `bookshop` | Bookshop.org | 632 | 63 MB | +| `breakoutgames-2` | Breakout Games | 387 | 27 MB | +| `brooklinen` | Brooklinen | 736 | 93 MB | +| `capterra` | Capterra | 632 | 162 MB | +| `change` | Change.org | 420 | 45 MB | +| `columbia` | Columbia Sportswear | 338 | 360 MB | +| `crumbl-cookies` | Crumbl Cookies | 531 | 179 MB | +| `crunchyroll` | Crunchyroll | 681 | 22 MB | +| `curology` | Curology | 539 | 34 MB | +| `dooney-bourke` | Dooney & Bourke | 754 | 10 MB | +| `eatthismuch` | Eat This Much | 668 | 2.4 GB | +| `etsy` | Etsy | 347 | 19 MB | +| `eventbrite` | Eventbrite | 1,655 | 44 MB | +| `everydollar` | EveryDollar | 394 | 11 MB | +| `everyplate` | EveryPlate | 2,016 | 72 MB | +| `fandango` | Fandango | 387 | 430 MB | +| `freshdirect` | FreshDirect | 990 | 7 MB | +| `hipcamp` | Hipcamp | 302 | 68 MB | +| `hmart` | H Mart | 714 | 161 MB | +| `imdb` | IMDb | 509 | 463 MB | +| `insureon` | Insureon | 522 | 41 MB | +| `insurify` | Insurify | 350 | 53 MB | +| `ioby` | ioby | 380 | 85 MB | +| `kanetix-ca` | Kanetix / Rates.ca | 809 | 40 MB | +| `leetcode` | LeetCode | 326 | 14 MB | +| `legalnature` | LegalNature | 389 | 336 MB | +| `linkedin` | LinkedIn | 565 | 42 MB | +| `llbean` | L.L.Bean | 423 | 149 MB | +| `lowes` | Lowe's | 494 | 26 MB | +| `medium` | Medium | 650 | 7 MB | +| `meetup` | Meetup | 347 | 111 MB | +| `notion` | Notion | 393 | 12 MB | +| `ollie` | Ollie | 394 | 56 MB | +| `ontario-parks` | Ontario Parks Reservations | 313 | 4 MB | +| `opentable` | OpenTable | 512 | 21 MB | +| `petco` | Petco | 609 | 24 MB | +| `petfinder` | Petfinder | 1,783 | 450 MB | +| `petsmart` | PetSmart | 837 | 42 MB | +| `ratemyprofessors` | Rate My Professors | 721 | 7 MB | +| `resy` | Resy | 519 | 111 MB | +| `simplify-jobs` | Simplify Jobs | 752 | 5 MB | +| `target` | Target | 537 | 193 MB | +| `taskrabbit` | TaskRabbit | 871 | 123 MB | +| `tripadvisor` | Tripadvisor | 625 | 17 MB | +| `tripit` | TripIt | 478 | 49 MB | +| `ubereats` | Uber Eats | 338 | 25 MB | +| `warbyparker` | Warby Parker | 992 | 288 MB | +| `wayfair` | Wayfair | 1,163 | 7 MB | +| `zennioptical` | Zenni Optical | 558 | 146 MB | Total download: 9.7 GB.