Get profile data from GraphQL instead of web_profile_info

[mifi]

Fixes #326 (follow-up to #324 / #325, related to #312)

Problem

After #325 the error became readable, and #326 shows what it actually is: web_profile_info responded with status 429 (text/html) on a clean install, on the very first profile. That endpoint is now blocked by Instagram rather than rate limited, so the bot can no longer get profile data (follower counts, private/verified flags, user id) from it.

Change

Instagram's own web client loads profile data through GraphQL (PolarisProfilePageContentQuery), either preloaded in the page HTML or via an XHR request, using a different shape (follower_count, following_count, is_business, hd_profile_pic_url_info, category).

• Add a page.on('response') listener that captures profile data from every GraphQL response the page makes, and a normalizer that converts both the new shape and the old web_profile_info shape into the existing InstagramUser type.
• Parse the page's embedded <script type="application/json"> preload blocks properly (instead of scraping raw HTML lines), including the stringified-response form the old code handled.
• Lookup order after navigating to a profile: intercepted GraphQL data → embedded page JSON → wait up to 10 s for an in-flight GraphQL request → web_profile_info as a last resort.

Testing

• tsc and eslint pass.
• The parsing helpers were tested against sample payloads: the new GraphQL shape, the old shape, a relay preload with a stringified response, streamed multi-object JSON responses, and a decoy owner object without follower counts (must not match).
• Not tested against the live site (no Instagram access from the sandbox). The assumption to verify on a real account is that the profile page still delivers the user object under a data.user key with follower_count / following_count. If it does not, the log shows "Waiting for user data from GraphQL request" and then falls through to the previous 429 error.

🤖 Generated with Claude Code

https://claude.ai/code/session_01BFDJvmX31k6Xd82sRL7meF

---

Generated by Claude Code

[trhuxtonr]

@mifi Hello, I ran version 1.11.15 again on macOS 26.6.2 on a Mac mini M4.

I performed the tests in the same way as last time: a clean .dmg installation. The application does not create the user’s .json files, so I created them manually afterward. The test result is always the same:

Failed to run
Try the troubleshooting button.
Error:
web_profile_info responded with status 429 (text/html; charset=utf-8).

I also performed the tests on Windows using the same version, 1.11.15, with the same results.

[mifi]

check newest version.

[trhuxtonr]

@mifi Hello, testing version 1.11.16. So far, everything is working fine with no errors. I’ll have to wait 667.22 seconds.

[trhuxtonr]

@mifi Well, after waiting all this time, I’ll let you know the result tomorrow. I’m sending you the log I have so far.

2:45 AMNo cookies found
2:45 AMSetting language to English (en)
2:45 AMWaiting 1.89 sec
2:45 AMWaiting 5.58 sec
2:46 AMFound language selector
2:46 AMAlready English language
2:46 AMWaiting 1.36 sec
2:46 AMElement not found for xpath: //button[contains(text(), "Accept")]
2:46 AMElement not found for xpath: //button[contains(text(), "Only allow essential cookies")]
2:46 AMElement not found for xpath: //button[contains(text(), "Allow essential and optional cookies")]
2:46 AMElement not found for xpath: //[@aria-label="Home"]
2:46 AMNo login page button, assuming we are on login form
2:46 AMElement not found for xpath: //button[contains(text(), "Log In")]
2:46 AMWaiting 1.98 sec
2:46 AMWaiting 0.57 sec
2:46 AMElement not found for xpath: //[@aria-label="Home"]
2:46 AMStill not logged in, trying to reload loading page
2:46 AMWaiting 7.21 sec
2:46 AMElement not found for xpath: //[@aria-label="Home"]
2:46 AMWARNING: Login has not succeeded. This could be because of an incorrect username/password, or a "suspicious login attempt"-message. You need to manually complete the process, or if really logged in, click the Instagram logo in the top left to go to the Home page.
2:46 AMWaiting 7.85 sec
2:46 AMElement not found for xpath: //[@aria-label="Home"]
2:46 AMWaiting 8.27 sec
2:46 AMElement not found for xpath: //[@aria-label="Home"]
2:46 AMWaiting 6.31 sec
2:46 AMElement not found for xpath: //[@aria-label="Home"]
2:46 AMWaiting 9.39 sec
2:47 AMElement not found for xpath: //*[@aria-label="Home"]
2:47 AMWaiting 9.94 sec
2:47 AMSetting language to English (en)
2:47 AMWaiting 1.57 sec
2:47 AMWaiting 3.45 sec
2:47 AMFound language selector
2:47 AMAlready English language
2:47 AMElement not found for xpath: //button[contains(text(), "Save Info")]
2:47 AMElement not found for xpath: //button[contains(text(), "Save info")]
2:47 AMElement not found for xpath: //button[contains(text(), "Not Now")]
2:47 AMSaving cookies
2:47 AMHave followed/unfollowed 0 in the last hour
2:47 AMHave followed/unfollowed 0 in the last 24 hours
2:47 AMHave liked 0 images in the last 24 hours
2:47 AMTrying to get user data from HTML
2:47 AMNavigating to user l_uismaria
2:47 AMGoto https://www.instagram.com/l_uismaria
2:47 AMPage loaded
2:47 AMWaiting 2.19 sec
2:47 AMUnfollowing currently followed users who were auto-followed more than 5 days ago (limit 100)...
2:47 AMUnfollowing users, up to limit 100
2:47 AMDone with unfollowing 0 0
2:47 AMFollowing up to 25 followers of beatriz.taboada.790
2:47 AMLiking images of up to 2 followers of beatriz.taboada.790
2:47 AMTrying to get user data from HTML
2:47 AMNavigating to user beatriz.taboada.790
2:47 AMGoto https://www.instagram.com/beatriz.taboada.790
2:47 AMPage loaded
2:47 AMWaiting 3.59 sec
2:47 AMUser followers batch
2:47 AMWaiting 667.22 sec
2:58 AMFollowing up to 25 followers of pablito_varela_varela
2:58 AMLiking images of up to 2 followers of pablito_varela_varela
2:58 AMTrying to get user data from HTML
2:58 AMNavigating to user pablito_varela_varela
2:58 AMGoto https://www.instagram.com/pablito_varela_varela
2:58 AMPage loaded
2:58 AMWaiting 2.53 sec
2:58 AMUser followers batch
2:58 AMWaiting 790.41 sec
3:12 AMFollowing up to 25 followers of kesarihomestay
3:12 AMLiking images of up to 2 followers of kesarihomestay
3:12 AMTrying to get user data from HTML
3:12 AMNavigating to user kesarihomestay
3:12 AMGoto https://www.instagram.com/kesarihomestay
3:12 AMPage loaded
3:12 AMWaiting 2.04 sec
3:12 AMUser followers batch
3:12 AMWaiting 1133.78 sec
3:31 AMFollowing up to 25 followers of mariovazqueztrabado1974
3:31 AMLiking images of up to 2 followers of mariovazqueztrabado1974
3:31 AMTrying to get user data from HTML
3:31 AMNavigating to user mariovazqueztrabado1974
3:31 AMGoto https://www.instagram.com/mariovazqueztrabado1974
3:31 AMPage loaded
3:31 AMWaiting 2.72 sec
3:31 AMUser followers batch
3:31 AMWaiting 1197.96 sec
3:51 AMFollowing up to 25 followers of joni_filipmarketing
3:51 AMLiking images of up to 2 followers of joni_filipmarketing
3:51 AMTrying to get user data from HTML
3:51 AMNavigating to user joni_filipmarketing
3:51 AMGoto https://www.instagram.com/joni_filipmarketing
3:51 AMPage loaded
3:51 AMWaiting 3.64 sec
3:51 AMUser followers batch
3:51 AMWaiting 1163.57 sec
4:10 AMFollowing up to 25 followers of jonimillet.dj
4:10 AMLiking images of up to 2 followers of jonimillet.dj
4:10 AMTrying to get user data from HTML
4:10 AMNavigating to user jonimillet.dj
4:10 AMGoto https://www.instagram.com/jonimillet.dj
4:10 AMPage loaded
4:10 AMWaiting 2.53 sec
4:10 AMUser followers batch
4:10 AMWaiting 930.35 sec
4:26 AMDone running
4:26 AMWaiting 50.83 sec
4:27 AMSleeping 5.328174986358175 hours (waiting until 10:00)...

[trhuxtonr]

@mifi Hi, I’ve been testing the application. It doesn’t show any errors, but it doesn’t work. The followed and unfollowed JSON files are still empty, both containing just [].

The Instagram interface only appears when the timeout finishes. It stays visible for 3–4 seconds and then goes back to this:"data":{"user":{"edge_followed_by":{"count":387,"page_info":{"has_next_page":false,"end_cursor":null},"edges":[]}}},"extensions":{"is_final":true,"server_metadata":{"request_start_time_ms":1789459139309,"time_at_flush_ms":1789459139348}},"status":"ok"}"