-
Notifications
You must be signed in to change notification settings - Fork 4
feat: add Home Depot provider #77
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
fnziman
wants to merge
1
commit into
eshaffer321:main
Choose a base branch
from
fnziman:feat/homedepot-provider
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| # Home Depot provider | ||
|
|
||
| Syncs purchases from homedepot.com into Monarch, splitting each transaction by category. Wraps [`github.com/fnziman/homedepot-go`](https://github.com/fnziman/homedepot-go), which talks to the internal `/oms/customer/order/v1` endpoints homedepot.com itself uses. | ||
|
|
||
| **Unofficial API.** Home Depot doesn't publish this. Endpoint drift is possible; failures usually mean either the cookies expired or the schema changed. | ||
|
|
||
| ## One-time setup: export cookies | ||
|
|
||
| You need a JSON file at `~/.homedepot-api/cookies.json` containing your logged-in browser cookies. Override with `HOMEDEPOT_COOKIE_FILE=/path/to/cookies.json` if you'd rather put it somewhere else. | ||
|
|
||
| 1. Log in at [homedepot.com](https://www.homedepot.com). | ||
| 2. Open DevTools (`Cmd+Option+I` on Mac, `F12` on Windows/Linux), pick the **Console** tab. | ||
| 3. Paste and run: | ||
| ```js | ||
| copy(JSON.stringify(document.cookie.split('; ').map(c => { | ||
| const i = c.indexOf('='); | ||
| return { name: c.slice(0, i), value: c.slice(i + 1) }; | ||
| }), null, 2)); | ||
| console.log('Cookies copied to clipboard.'); | ||
| ``` | ||
| 4. Save clipboard to the file: | ||
| ```bash | ||
| mkdir -p ~/.homedepot-api | ||
| pbpaste > ~/.homedepot-api/cookies.json # macOS | ||
| ``` | ||
|
|
||
| The file must include a `THD_CUSTOMER` entry — that's the cookie the client decodes for the API auth token. | ||
|
|
||
| If the DevTools snippet doesn't grab enough (some anti-bot cookies are HTTP-only and won't appear in `document.cookie`), fall back to a browser extension like [EditThisCookie](https://www.editthiscookie.com/) to export cookies for `.homedepot.com`. See homedepot-go's README for the accepted JSON shapes. | ||
|
|
||
| ## Usage | ||
|
|
||
| ```bash | ||
| itemize homedepot -dry-run -days 14 -verbose # preview | ||
| itemize homedepot -days 14 # apply | ||
| itemize homedepot -days 14 -max 5 # cap at 5 orders | ||
| itemize homedepot -days 14 -force # reprocess already-processed orders | ||
| ``` | ||
|
|
||
| Config via `config.yaml`: | ||
|
|
||
| ```yaml | ||
| providers: | ||
| homedepot: | ||
| enabled: true | ||
| lookback_days: 14 | ||
| max_orders: 0 # 0 = no cap | ||
| cookie_file: "" # empty = ~/.homedepot-api/cookies.json | ||
| ``` | ||
|
|
||
| Or env vars: `HOMEDEPOT_LOOKBACK_DAYS`, `HOMEDEPOT_MAX_ORDERS`, `HOMEDEPOT_COOKIE_FILE`. | ||
|
|
||
| ## Both online and in-store orders are supported | ||
|
|
||
| Home Depot's API returns two response shapes: | ||
|
|
||
| - **Online** — `orderOrigin: "online"`, keyed by `orderNumber` (e.g. `WD00000000`). | ||
| - **In-store** — `orderOrigin: "instore"`, no `orderNumber`; the client uses a `hd-instore-{storeNumber}-{transactionId}` composite for dedup. | ||
|
|
||
| Both flow through the same categorizer + splitter downstream. | ||
|
|
||
| ## Merchant matching | ||
|
|
||
| itemize matches orders to Monarch transactions by substring on the merchant name. The Home Depot provider's `DisplayName()` is `"Home Depot"`, which case-insensitively substring-matches Monarch's canonical `"THE HOME DEPOT"`. No aliases required. | ||
|
|
||
| If your bank labels Home Depot differently in Monarch (e.g. `"HD SUPPLY"`), file an issue and we'll add an alias mechanism. | ||
|
|
||
| ## Known limitations | ||
|
|
||
| - **24-month history cap.** Home Depot's API only returns roughly the last 24 months regardless of `-days`. Older orders are unreachable. | ||
| - **No programmatic login.** homedepot.com is Akamai-protected and blocks headless browsers. Cookie replay works; automated login does not. | ||
| - **MFA step-up invalidates the cookie.** Any MFA event on your account (new-device flag, quarterly re-verify, etc.) invalidates the exported cookies. Symptom: `AuthError: home depot auth failed (status 401)`. Fix: re-export cookies from a fresh logged-in session. | ||
| - **`orginalOrderedQuantity` typo.** The upstream JSON key really is spelled that way (missing the "i"). Exposed via `LineItem.OriginalOrderedQty`. | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| | Symptom | Cause | Fix | | ||
| |---|---|---| | ||
| | `THD_CUSTOMER cookie not found in jar` | Cookies exported while logged out. | Log in, re-export. | | ||
| | `home depot auth failed (status 401 or 403)` | Cookies expired or MFA event invalidated the session. | Re-export from a fresh logged-in session. | | ||
| | `home depot API rate-limited the request` | Too many requests too fast. | Client already paces itself; back off and retry after a minute. | | ||
| | `home depot API returned status 5xx` | Home Depot backend issue or schema change. | Retry; if persistent, file an issue with a scrubbed reproduction. | | ||
| | Orders older than 24 months not returned | API's own cap. | Expected — cannot be worked around. | | ||
|
|
||
| ## Attribution | ||
|
|
||
| Schema mapping and endpoint discovery in the underlying homedepot-go client were reverse-engineered by [joshellissh/homedepot-history](https://github.com/joshellissh/homedepot-history) (MIT). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,137 @@ | ||
| package homedepot | ||
|
|
||
| import ( | ||
| "fmt" | ||
| "strings" | ||
| "time" | ||
|
|
||
| hdgo "github.com/fnziman/homedepot-go" | ||
|
|
||
| "github.com/eshaffer321/itemize/internal/adapters/providers" | ||
| ) | ||
|
|
||
| // Order wraps a Home Depot OrderDetail and its originating OrderSummary | ||
| // to implement providers.Order. The summary is retained because in-store | ||
| // orders don't populate orderNumber on the detail — the store/transaction | ||
| // composite from the summary is used as the stable ID instead. | ||
| type Order struct { | ||
| summary hdgo.OrderSummary | ||
| detail hdgo.OrderDetail | ||
| } | ||
|
|
||
| // GetID returns a stable unique identifier. Online orders use orderNumber | ||
| // directly; in-store orders use a composite of store number and transaction | ||
| // ID (both surfaced on the summary but empty on the detail). | ||
| func (o *Order) GetID() string { | ||
| if o.detail.OrderNumber != "" { | ||
| return o.detail.OrderNumber | ||
| } | ||
| if o.summary.TransactionID != "" { | ||
| return fmt.Sprintf("hd-instore-%s-%s", o.summary.StoreNumber, o.summary.TransactionID) | ||
| } | ||
| return "" | ||
| } | ||
|
|
||
| // GetDate parses the ISO-8601 salesDate the API returns. Home Depot has | ||
| // been observed to use full RFC3339 timestamps and date-only strings; | ||
| // try both. | ||
| func (o *Order) GetDate() time.Time { | ||
| raw := o.detail.SalesDate | ||
| if raw == "" { | ||
| raw = o.summary.SalesDate | ||
| } | ||
| if raw == "" { | ||
| return time.Time{} | ||
| } | ||
| for _, layout := range []string{time.RFC3339, "2006-01-02T15:04:05.000Z", "2006-01-02"} { | ||
| if t, err := time.Parse(layout, raw); err == nil { | ||
| return t | ||
| } | ||
| } | ||
| return time.Time{} | ||
| } | ||
|
|
||
| func (o *Order) GetTotal() float64 { return o.detail.GrandTotalAmount } | ||
| func (o *Order) GetSubtotal() float64 { return o.detail.SubTotalAmount } | ||
| func (o *Order) GetTax() float64 { return o.detail.TaxTotalAmount } | ||
|
|
||
| // GetTip returns 0 — Home Depot doesn't have driver tips. | ||
| func (o *Order) GetTip() float64 { return 0 } | ||
|
|
||
| // GetFees returns the sum of shipping and delivery charges. Home Depot | ||
| // reports these separately on the order detail. | ||
| func (o *Order) GetFees() float64 { | ||
| return o.detail.ShippingCharge + o.detail.DeliveryCharge | ||
| } | ||
|
|
||
| // GetItems flattens line items across every fulfillment group. | ||
| func (o *Order) GetItems() []providers.OrderItem { | ||
| all := o.detail.AllLineItems() | ||
| out := make([]providers.OrderItem, 0, len(all)) | ||
| for i := range all { | ||
| out = append(out, &OrderItem{item: all[i]}) | ||
| } | ||
| return out | ||
| } | ||
|
|
||
| // GetProviderName is the human-readable provider identifier the domain | ||
| // layer uses in log lines and prompts. | ||
| func (o *Order) GetProviderName() string { return "Home Depot" } | ||
|
|
||
| // GetRawData exposes the underlying detail for handlers that want to peek | ||
| // at provider-specific fields. | ||
| func (o *Order) GetRawData() interface{} { return o.detail } | ||
|
|
||
| // OrderItem wraps a single homedepot-go LineItem to implement | ||
| // providers.OrderItem. | ||
| type OrderItem struct { | ||
| item hdgo.LineItem | ||
| } | ||
|
|
||
| // GetName combines brand and description when both are present. Home Depot | ||
| // listings frequently split them ("RIDGID" + "18V Cordless Drill"), and | ||
| // concatenating gives the categorizer better signal than either alone. | ||
| func (i *OrderItem) GetName() string { | ||
| brand := strings.TrimSpace(i.item.BrandName) | ||
| desc := strings.TrimSpace(i.item.Description) | ||
| switch { | ||
| case brand != "" && desc != "": | ||
| return brand + " " + desc | ||
| case desc != "": | ||
| return desc | ||
| default: | ||
| return brand | ||
| } | ||
| } | ||
|
|
||
| // GetPrice returns the line total. | ||
| func (i *OrderItem) GetPrice() float64 { return i.item.TotalPrice } | ||
|
|
||
| // GetQuantity returns the purchased quantity (currentQuantity minus | ||
| // cancelledQuantity, clamped at zero). Cancellations happen fairly often | ||
| // on partial fulfillments; using CurrentQuantity as-is would over-report. | ||
| func (i *OrderItem) GetQuantity() float64 { return i.item.PurchasedQuantity() } | ||
|
|
||
| // GetUnitPrice returns the per-unit price. | ||
| func (i *OrderItem) GetUnitPrice() float64 { return i.item.UnitPrice } | ||
|
|
||
| // GetDescription returns the item description. | ||
| func (i *OrderItem) GetDescription() string { return i.item.Description } | ||
|
|
||
| // GetSKU prefers Home Depot's own thdSku (globally unique on | ||
| // homedepot.com) and falls back to skuNumber, then modelNumber. | ||
| func (i *OrderItem) GetSKU() string { | ||
| switch { | ||
| case i.item.THDSKU != "": | ||
| return i.item.THDSKU | ||
| case i.item.SKUNumber != "": | ||
| return i.item.SKUNumber | ||
| default: | ||
| return i.item.ModelNumber | ||
| } | ||
| } | ||
|
|
||
| // GetCategory returns "" — Home Depot's line-item response doesn't carry | ||
| // a category we can map cleanly to Monarch. The LLM categorizer does the | ||
| // mapping downstream. | ||
| func (i *OrderItem) GetCategory() string { return "" } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[P1] Please make the in-store dedup key include the full stable receipt identity. The companion client needs
registerNumber,salesDate,storeNumber, andtransactionIdto retrieve an in-store order, but this key drops register and date. Two summaries that differ only by those fields therefore produce the same ID;processing_records.order_idis globally unique andIsProcessedchecks only that ID, so a legitimate second purchase can be skipped permanently. Please include register and normalized sale date, reject incomplete identifiers, and add a test with same store/transaction but different register/date. This is safest to fix before release because changing persisted IDs later can cause reprocessing.