feat: suggest developers whose repositories you starred on GitHub

KNOWLEDGE_BASE.md

@@ -194,9 +194,11 @@ Sources: [badge definitions](app/Enums/Achievement.php), [unlock conditions and
 
 Follow or unfollow from a developer's profile or a follow control elsewhere in the app. If someone already follows you and you do not follow them yet, their profile, hover card, and connections lists offer **Follow back**. Following puts their commits in your Following log. Following or unfollowing from a log updates the controls, counts, and live-following sidebar without navigating away: Latest and Trending keep their loaded commits and ranking, while Following updates its commits and empty-state suggestions. You cannot follow yourself or follow across a block. GitHub follows are imported once: connections to developers already on /dev/null are added in both directions, with up to 1,000 GitHub connections checked each way. The organizations you follow on GitHub come along too: you follow the ones already on /dev/null, and each of the others as soon as it shows up here. This is not continuous synchronization, so unfollowing someone on /dev/null sticks, and following someone on GitHub later does not carry over.
 
-**Who to follow** suggests developers in the side column or within the log on smaller screens, each with the first line of their bio and why they are suggested (how many people you follow follow them, or their follower count). Follow from the icon next to each suggestion; for someone who already follows you, it follows back. The X icon marks a suggestion **Not interested**: that developer is never suggested to you again, without muting them, and the next suggestion takes their place. Follow buttons and dismissed suggestions also update in your other open tabs, keeping the other suggestions in place. Refresh for different suggestions, or use **… → Don't show again** to hide the block, with an immediate **Undo** option. Show or hide these suggestions in Settings → Appearance. Suggested developers must have a visible top-level commit. On profiles, suggestions prioritize the people that developer follows; on commit pages, they prioritize visible thread participants. Regular suggestions fill any remaining places. You can also follow back from the notifications list.
+**Who to follow** suggests developers in the side column or within the log on smaller screens, each with the first line of their bio and why they are suggested (how many of their repositories you starred on GitHub, how many people you follow follow them, or their follower count). Follow from the icon next to each suggestion; for someone who already follows you, it follows back. The X icon marks a suggestion **Not interested**: that developer is never suggested to you again, without muting them, and the next suggestion takes their place. Follow buttons and dismissed suggestions also update in your other open tabs, keeping the other suggestions in place. Refresh for different suggestions, or use **… → Don't show again** to hide the block, with an immediate **Undo** option. Show or hide these suggestions in Settings → Appearance. Suggested developers must have a visible top-level commit. On profiles, suggestions prioritize the people that developer follows; on commit pages, they prioritize visible thread participants. Regular suggestions fill any remaining places. You can also follow back from the notifications list.
 
-Sources: [follow import](app/Actions/CreateUserGitHubFollows.php), [suggestions](resources/js/components/who-to-follow.tsx), [suggestion eligibility](app/Queries/SuggestedUsersQuery.php), [appearance settings](resources/js/pages/appearance/update.tsx), [notifications page](resources/js/pages/notification/index.tsx).
+Regular suggestions also include the developers whose repositories you starred on GitHub, taking turns with the people your follows follow so they show up early, the ones you starred most first. Who owns your 300 most recently starred repositories is read once, when you sign in; repositories of organizations are left out. If that finishes after your suggestions are on screen, in onboarding or the side column, they are picked again right away. Developers who join later are suggested once they post a commit. Starring never follows anyone for you, and starring more on GitHub later does not carry over.
+
+Sources: [follow import](app/Actions/CreateUserGitHubFollows.php), [star import](app/Actions/CreateUserGitHubStars.php), [suggestions](resources/js/components/who-to-follow.tsx), [suggestion eligibility](app/Queries/SuggestedUsersQuery.php), [appearance settings](resources/js/pages/appearance/update.tsx), [notifications page](resources/js/pages/notification/index.tsx).
 
 ### Following, Latest, and Trending logs
 
@@ -650,6 +652,7 @@ Choose **Download JSON** in Settings → Profile to export your account data. Th
 - the days you used /dev/null (dates, not times), oldest first, and the last of them;
 - your commits and replies (including pending, rejected, and deleted entries still stored), their cached translations, polls, and your votes;
 - stars, followers and people you follow, mutes, blocks, and dismissed **Not interested** suggestions;
+- the GitHub IDs of the people whose repositories you starred on GitHub, and how many of each, as read for suggestions;
 - messages you sent, including links to your voice notes, and notifications;
 - SSH key fingerprints, approved devices, and your timeout history;
 - active-session metadata: IP address, browser information, and last activity.

app/Actions/CreateExport.php

@@ -52,6 +52,7 @@
  *     organizations: array<int, array{handle: string, role: string, joined_at: string}>,
  *     followed_organizations: array<int, array{handle: string, followed_at: string}>,
  *     pending_follows: list<string>,
+ *     starred_github_owners: list<array{github_id: string, repositories_count: int}>,
  *     messages: Generator<int, ExportMessage>,
  *     reports: Traversable<int, array<string, mixed>>,
  *     remote_pins: array<int, array{to: string|null, pinned_at: string|null}>,
@@ -129,6 +130,7 @@ public function handle(User $user): array
             'organizations' => $this->organizations($user),
             'followed_organizations' => $this->followedOrganizations($user),
             'pending_follows' => $this->pendingFollows($user),
+            'starred_github_owners' => $this->starredGitHubOwners($user),
             'messages' => $this->messages($user),
             'reports' => CommitReport::query()->where('reporter_id', $user->id)->oldest('id')->cursor()->map(fn (CommitReport $report): array => [
                 'commit_id' => $report->commit_id,
@@ -413,6 +415,22 @@ private function pendingFollows(User $user): array
         return DB::table('pending_follows')->where('user_id', $user->id)->orderBy('github_id')->pluck('github_id')->all();
     }
 
+    /**
+     * The GitHub IDs of the people whose repositories the user starred on GitHub, and how many, for suggestions.
+     *
+     * @return list<array{github_id: string, repositories_count: int}>
+     */
+    private function starredGitHubOwners(User $user): array
+    {
+        /** @var Collection<int, object{github_id: string, repositories_count: int}> $owners */
+        $owners = DB::table('starred_github_owners')->where('user_id', $user->id)->orderBy('github_id')->get(['github_id', 'repositories_count']);
+
+        return array_values($owners->map(fn (object $owner): array => [
+            'github_id' => $owner->github_id,
+            'repositories_count' => $owner->repositories_count,
+        ])->all());
+    }
+
     /**
      * Moderators stay anonymous, as they are on the notifications page, and level-ups are the user's own doing. An
      * organization's notifications name the organization the user heard about it for.

app/Actions/CreateUserGitHubStars.php

@@ -0,0 +1,61 @@
+<?php
+
+declare(strict_types=1);
+
+namespace App\Actions;
+
+use App\Contracts\Services\GitHubProvider;
+use App\Enums\UserUpdate;
+use App\Events\UsersUpdated;
+use App\Models\User;
+use Illuminate\Support\Facades\DB;
+
+/**
+ * Remembers whose repositories a user starred on GitHub, once, so Who to follow can suggest those people. Owners not
+ * on /dev/null yet are kept too, for when they join. It never follows anyone.
+ */
+final readonly class CreateUserGitHubStars
+{
+    /**
+     * The 300 repositories starred last, so someone who stars everything can't use up the rate limit on their own.
+     */
+    public const int MaxPages = 3;
+
+    public function __construct(private GitHubProvider $github)
+    {
+        //
+    }
+
+    public function handle(User $user): void
+    {
+        if ($user->github_stars_imported_at !== null) {
+            return;
+        }
+
+        $login = $this->github->login($user->github_id);
+        $ownerIds = $login === null ? [] : $this->github->starredOwnerIds($login, self::MaxPages);
+
+        $rows = collect($ownerIds)
+            ->reject(fn (string $id): bool => $id === $user->github_id)
+            ->countBy()
+            ->map(fn (int $count, int|string $id): array => [
+                'user_id' => $user->id,
+                'github_id' => (string) $id,
+                'repositories_count' => $count,
+                'created_at' => now(),
+            ])
+            ->values()
+            ->all();
+
+        DB::transaction(function () use ($user, $rows): void {
+            $saved = DB::table('starred_github_owners')->insertOrIgnore($rows);
+
+            $user->update(['github_stars_imported_at' => now()]);
+
+            // The suggestions on screen were picked without them, and are kept across visits, so they're picked again.
+            if ($saved > 0) {
+                event(new UsersUpdated([$user->id], UserUpdate::GitHubStars));
+            }
+        });
+    }
+}

app/Console/Commands/ImportGitHubStarsCommand.php

@@ -0,0 +1,83 @@
+<?php
+
+declare(strict_types=1);
+
+namespace App\Console\Commands;
+
+use App\Actions\CreateUserGitHubStars;
+use App\Contracts\Services\GitHubProvider;
+use App\Models\User;
+use Illuminate\Console\Attributes\Description;
+use Illuminate\Console\Attributes\Signature;
+use Illuminate\Console\Command;
+use Illuminate\Database\Eloquent\Builder;
+use Illuminate\Http\Client\ConnectionException;
+use Illuminate\Http\Client\RequestException;
+
+#[Signature('stars:import')]
+#[Description('Remember, for every user not imported yet, whose repositories they starred on GitHub, for Who to follow')]
+final class ImportGitHubStarsCommand extends Command
+{
+    /**
+     * Below this many requests left, the command stops, leaving the rest to the app until the rate limit resets, as
+     * follows:import does.
+     */
+    private const int RateLimitFloor = 1500;
+
+    public function handle(CreateUserGitHubStars $createUserGitHubStars, GitHubProvider $github): int
+    {
+        $imported = 0;
+        $failed = 0;
+
+        foreach ($this->users()->lazyById() as $user) {
+            $rateLimit = $github->rateLimit();
+
+            // A user takes one request for their login, and up to the maximum of pages of stars.
+            if ($rateLimit !== null && $rateLimit['remaining'] < self::RateLimitFloor + 1 + CreateUserGitHubStars::MaxPages) {
+                $this->components->warn("Stopped after {$imported} users, to keep ".self::RateLimitFloor." GitHub requests for the rest of the app: {$rateLimit['remaining']} left.");
+                $this->components->info("{$this->users()->count()} users left. Run the command again after {$rateLimit['resets_at']->utc()->toDateTimeString()} UTC ({$rateLimit['resets_at']->diffForHumans()}), when the GitHub rate limit resets.");
+
+                return self::FAILURE;
+            }
+
+            try {
+                $createUserGitHubStars->handle($user);
+
+                $imported++;
+            } catch (RequestException $exception) {
+                if (in_array($exception->response->status(), [403, 429], true)) {
+                    $this->components->error("GitHub rate limit reached after {$imported} users. Run the command again later.");
+
+                    return self::FAILURE;
+                }
+
+                $this->failed($user, $exception);
+
+                $failed++;
+            } catch (ConnectionException $exception) {
+                $this->failed($user, $exception);
+
+                $failed++;
+            }
+        }
+
+        $this->components->info("Imported the GitHub stars of {$imported} users.");
+
+        return $failed === 0 ? self::SUCCESS : self::FAILURE;
+    }
+
+    /**
+     * @return Builder<User>
+     */
+    private function users(): Builder
+    {
+        return User::query()->whereNull('github_stars_imported_at');
+    }
+
+    private function failed(User $user, RequestException|ConnectionException $exception): void
+    {
+        report($exception);
+
+        $this->components->warn("Could not import the GitHub stars of {$user->mention()}.");
+    }
+}

app/Contracts/Services/GitHubProvider.php

@@ -77,6 +77,14 @@ public function followingIds(string $login, int $maxPages): array;
      */
     public function followerIds(string $login, int $maxPages): array;
 
+    /**
+     * The IDs of the people owning the repositories the user starred, once per repository, most recently starred
+     * first, up to the given number of pages. The repositories of organizations are left out.
+     *
+     * @return list<string>
+     */
+    public function starredOwnerIds(string $login, int $maxPages): array;
+
     /**
      * The public SSH keys of the account, for authentication and signing, in OpenSSH format.
      *

app/Enums/UserUpdate.php

@@ -21,6 +21,7 @@ enum UserUpdate: string
     case Account = 'account';
     case Relationships = 'relationships';
     case Suggestions = 'suggestions';
+    case GitHubStars = 'github-stars';
     case Commits = 'commits';
     case Moderation = 'moderation';
     case Live = 'live';

app/Http/Controllers/OnboardingController.php

@@ -25,7 +25,7 @@ public function show(ShowOnboardingRequest $request, #[CurrentUser] User $user,
         return Inertia::render('onboarding/show', [
             'preferredLane' => $user->preferredLane(),
             'automaticLane' => $user->automaticLane(),
-            'suggestions' => fn (): Collection => $suggested->get($user, limit: 5),
+            'suggestions' => Inertia::live(fn (): Collection => $suggested->get($user, limit: 5), on: UserUpdate::GitHubStars->value, channel: new PrivateChannel($user)),
             'githubImport' => Inertia::live(fn (): ?array => $this->githubImport($request, $user), on: UserUpdate::Onboarding->value, channel: new PrivateChannel($user)),
             'commitScriptUrl' => route('cli-script.show'),
         ]);

app/Http/Controllers/SessionController.php

@@ -8,6 +8,7 @@
 use App\Actions\DeletePushSubscription;
 use App\Actions\UpdateOrCreateUser;
 use App\Jobs\ImportGitHubFollowsJob;
+use App\Jobs\ImportGitHubStarsJob;
 use App\Jobs\LookUpUserOrganizations;
 use App\Jobs\SyncGitHubSshKeysJob;
 use App\Models\User;
@@ -68,6 +69,12 @@ public function store(Request $request, UpdateOrCreateUser $action, CreateTomFri
         }
 
         dispatch(new SyncGitHubSshKeysJob($user));
+
+        // Ahead of the follows, since onboarding reloads its suggestions once those are in.
+        if ($user->github_stars_imported_at === null) {
+            dispatch(new ImportGitHubStarsJob($user));
+        }
+
         dispatch(new ImportGitHubFollowsJob($user, notify: $user->wasRecentlyCreated));
         LookUpUserOrganizations::dispatchFor($user);
 

app/Http/Resources/UserResource.php

@@ -49,6 +49,7 @@ public function toArray(Request $request): array
             'is_followed' => $this->whenHas('is_followed'),
             'follows_you' => $this->whenHas('follows_you'),
             'followed_by_following_count' => $this->whenHas('followed_by_following_count'),
+            'starred_repositories_count' => $this->whenHas('starred_repositories_count'),
         ];
     }
 

app/Jobs/ImportGitHubStarsJob.php

@@ -0,0 +1,29 @@
+<?php
+
+declare(strict_types=1);
+
+namespace App\Jobs;
+
+use App\Actions\CreateUserGitHubStars;
+use App\Models\User;
+use Illuminate\Contracts\Queue\ShouldQueue;
+use Illuminate\Foundation\Queue\Queueable;
+use Illuminate\Queue\Attributes\Backoff;
+use Illuminate\Queue\Attributes\Tries;
+
+#[Backoff([10, 60])]
+#[Tries(3)]
+final class ImportGitHubStarsJob implements ShouldQueue
+{
+    use Queueable;
+
+    public function __construct(public User $user)
+    {
+        //
+    }
+
+    public function handle(CreateUserGitHubStars $createUserGitHubStars): void
+    {
+        $createUserGitHubStars->handle($this->user);
+    }
+}

app/Models/User.php

@@ -66,6 +66,7 @@
  * @property-read CarbonInterface|null $onboarded_at
  * @property-read CarbonInterface|null $github_follows_imported_at
  * @property-read CarbonInterface|null $github_organization_follows_imported_at
+ * @property-read CarbonInterface|null $github_stars_imported_at
  * @property-read string|null $remember_token
  * @property-read CarbonInterface|null $github_synced_at
  * @property-read CarbonInterface|null $last_seen_at
@@ -122,6 +123,7 @@
     'onboarded_at',
     'github_follows_imported_at',
     'github_organization_follows_imported_at',
+    'github_stars_imported_at',
     'live_platform',
     'live_url',
     'live_ends_at',
@@ -228,6 +230,7 @@ public function casts(): array
             'onboarded_at' => 'datetime',
             'github_follows_imported_at' => 'datetime',
             'github_organization_follows_imported_at' => 'datetime',
+            'github_stars_imported_at' => 'datetime',
             'remember_token' => 'string',
             'live_platform' => LivePlatform::class,
             'live_url' => 'string',