Skip to content

fix: retry when the llm returns work experiences as strings - #5054

Merged
skwowet merged 1 commit into
mainfrom
fix/member-enrichment-work-experience-shape
Oct 20, 2026
Merged

skwowet merged 1 commit into
mainfrom
fix/member-enrichment-work-experience-shape

Conversation

@skwowet

@skwowet skwowet commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

Member enrichment fails when the LLM that merges work experiences returns each entry as a JSON-encoded string (["{\"name\":...}"]) instead of an object. The activity only checked that the result was an array, so the strings reached processMemberSources, which crashed with Cannot create property 'identities' on string. Temporal kept replaying the same recorded result until the 15-minute timeout, which failed enrichMember and the daily triggerMembersEnrichment run.

This happened for two members on Oct 19 and Oct 20, out of about 7k calls since Oct 4.

Changes

  • squashWorkExperiencesWithLLM now also checks that every entry is an object with a string name. If one isn't, the activity throws and Temporal retries it with a fresh LLM call, instead of passing bad data to the workflow.
  • The prompt now says each element must be a JSON object, never a JSON-encoded string.

Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 20, 2026 12:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused validation rejects the reported malformed payload within the activity, enabling existing retries without changing valid-result handling.

No open issues 🎉

What changed in this PR

Prevents malformed LLM work-experience entries from reaching member enrichment processing, allowing activity retries to request a fresh result.

Changes:

  • Validates that every returned entry is a plain object with a string name.
  • Clarifies that the prompt requires objects, not JSON-encoded strings.
File Description
services/​apps/​members_enrichment_worker/​src/​activities/​llm.ts Strengthens result validation and clarifies the LLM output format.

Review effort: Balanced

@skwowet skwowet self-assigned this Oct 20, 2026
@skwowet
skwowet merged commit d629a4d into main Oct 20, 2026
13 checks passed
@skwowet
skwowet deleted the fix/member-enrichment-work-experience-shape branch October 20, 2026 12:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants