fix(comparisons): reverify every profile and merge the compliance facts by j15z · Pull Request #7462 · simstudioai/sim

j15z · 2026-09-03T23:34:08Z

Summary

Recheck all 20 competitor profiles and Sim's own against public primary sources, correcting stale capability, pricing, licensing, and product lifecycle claims while keeping explicit uncertainty and scope qualifications intact.
Merge soc2 and additionalCompliance into a single compliance fact so a profile states its certifications once, with a migration test asserting no detail was dropped in the move.
Carry dated source metadata on prose links and profile descriptions rather than bare hrefs, so every linked claim can be traced to a page checked on a recorded date.

Scope

This branch is the data layer only, plus the three call sites that had to change to accept the new shape: the security rows in comparison-sections.ts, the compliance key in the security FAQ, and prose-text.tsx rendering a segment's source URL.

The comparison page redesign and the UI that renders these citations are a separate change on feat/redesign-comparison-page. Both branches are based on staging independently. Whichever lands second will need a small conflict resolution in utils.ts and [provider]/page.test.tsx.

Review

The facts went through two rounds of AI-assisted citation review (checker + independent fixer, per profile, batches of 5), and the corrections from those rounds are applied here. Claims that could not be confirmed are labeled as unknown or estimated rather than asserted — see Unverifiable claims below for the specific list. This is not a claim that every claim and link was independently confirmed — access limitations and unresolved items are recorded there.

Notable corrections from this pass: Flowise wound down operations and archived its GitHub repo (Aug 2026); LangChain achieved ISO 27001 certification that the profile said it lacked; OpenAI AgentKit's Zero-Data-Retention/background-mode compatibility was stated backwards; Gumloop repositioned entirely from visual workflow builder to chat-based agent product and dropped its free tier; Retool self-hosting is Enterprise-plan only, not available on all tiers as previously stated; Tines audit-log retention is now org-configurable (30 days–2 years), not fixed at 2 years; Power Automate is SOC 2 Type 2 in-scope for GCC High, contrary to the prior claim. Sim's own internal counts, cited across every competitor profile, were stale and refreshed: blocks 266→356, tool actions ~~3,900→~~5,500, LLM providers 21→25, knowledge-base connectors 51→65, trigger app coverage 61→64.

Unverifiable claims (confidence: `'unknown'`, or otherwise flagged as access-limited)

These rest on a source that could not be read (paywall, dead link, bot-blocked, or no longer contains the claim) after both a direct fetch and a browser retry. They are marked accordingly in the data rather than asserted as fact — do not treat them as confirmed:

claude-cowork.ts — Academy "three tracks" framing: anthropic.skilljar.com is now a bare login gate with no visible course content.
dust.ts — WAU/weekly-active-usage, churn, and net-revenue-retention figures from the Series B coverage: the Sifted article is paywalled/obfuscated for non-subscribers.
gumloop.ts — platform.versionControlDepth, observability.tracingDepth, observability.durabilityModel, observability.failureAlerting: all four cited doc pages now redirect to a generic overview page with none of the original content, and no replacement page was found.
microsoft-copilot.ts — the $0.01/Copilot-Credit pay-as-you-go rate: Azure's pricing page renders no static per-credit figure. support.sla's secondary citation (windowsnews.ai → windowsforum.com) still returns 403.
n8n.ts — the "~900–1,000 employees" figure: none of the four originally-named trackers (Tracxn, RocketReach, LeadIQ, Revelio) could be located as citable, addable sources; PitchBook (the one reachable source) shows a materially different count.
pipedream.ts — the "internal Postgres+pgvector RAG system" implementation detail behind Pipedream's support bot: the sole source was a blog post that Pipedream has since retired (its entire /blog section 302-redirects to /docs), and no independent corroboration exists.
workato.ts — support.companyMaturity's Crunchbase citation (crunchbase.com/organization/workato) returns 403 / a Cloudflare challenge; the fact is still backed by two other sources (TechCrunch, Tracxn) that were independently verified, so it isn't wholly unsupported, but that one citation itself couldn't be read.

Two additional flagged-but-resolved items worth noting, since they were close calls:

openclaw.ts — a follow-up task was filed (not applied here) to re-verify platform.subWorkflows and platform.customBlocks, which may repeat a stale "Lobster only has run/pipeline/approval steps" claim discovered as a side effect of fixing a different flag in the same file.
gumloop.ts — observability.executionLimits and support.supportChannels may still describe free-tier concurrency/support figures for a Free plan that Gumloop has since removed from its pricing page; this was noted but out of scope for the flags actually raised.

Type of Change

Bug fix

Testing

Comparison tests: page rendering, utils, and the compliance migration (bunx vitest run "app/(landing)/comparisons" "lib/compare")
tsc --noEmit clean across the app
Biome clean on every changed file

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

🤖 Generated with Claude Code

vercel · 2026-09-03T23:34:15Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
docs	Ready	Preview	Sep 16, 2026 3:10am UTC

cubic-dev-ai

All reported issues were addressed across 2 files

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

greptile-apps · 2026-09-03T23:36:54Z

Greptile Summary

The PR consolidates SOC 2 and other certification data into one Compliance fact across all comparison profiles.

Replaces the separate SOC 2 and Additional compliance rows with a single Compliance row.
Preserves each affected competitor’s SOC 2 status, scope qualifiers, and evidence in the consolidated fact.
Updates visible table values, FAQ copy, JSON-LD data, and Sim’s certification source.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
apps/sim/app/(landing)/comparisons/comparison-sections.ts	Replaces the two former certification rows with one type-checked Compliance row consumed by the table and JSON-LD builder.
apps/sim/app/(landing)/comparisons/utils.ts	Updates the security FAQ to read the consolidated compliance facts for Sim and each competitor.
apps/sim/lib/compare/data/types.ts	Replaces the obsolete security fact fields with the canonical compliance field.
apps/sim/lib/compare/data/sim.ts	Lists Sim’s SOC 2 Type II, ISO 27001:2022, and GDPR certifications under one Trust Center-backed fact.
apps/sim/lib/compare/data/competitors/stackai.ts	Preserves StackAI’s qualified SOC 2 status in both the full and visible consolidated compliance values.
apps/sim/lib/compare/data/competitors/tines.ts	Preserves Tines’s SOC 2 status in both the full and visible consolidated compliance values.
apps/sim/lib/compare/data/competitors/vellum.ts	Preserves Vellum’s SOC 2 status in both the full and visible consolidated compliance values.
apps/sim/lib/compare/data/competitors/power-automate.ts	Preserves Power Automate’s environment-qualified SOC 2 scope in both consolidated values.
apps/sim/lib/compare/data/competitors/microsoft-copilot.ts	Preserves Microsoft Copilot’s SOC 2 status and scope in both consolidated values.
apps/sim/lib/compare/data/competitors/crewai.ts	Preserves CrewAI AMP’s SOC 2 Type 1 status and scope in both consolidated values.
apps/sim/lib/compare/data/competitors/claude-cowork.ts	Preserves Anthropic’s company-wide SOC 2 status and Cowork-specific qualification in both consolidated values.

_{Reviews (4): Last reviewed commit: "improvement(comparisons): remove obsolet..." | Re-trigger Greptile}

j15z · 2026-09-03T23:42:57Z

@greptile

j15z · 2026-09-03T23:42:58Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-03T23:43:08Z

@cubic-dev-ai review this PR

@j15z I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 14 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Re-trigger cubic}

j15z · 2026-09-03T23:49:37Z

@greptile

j15z · 2026-09-03T23:49:38Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-03T23:49:44Z

@cubic-dev-ai review this PR

@j15z I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 24 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Re-trigger cubic}

j15z · 2026-09-04T00:20:34Z

@greptile

j15z · 2026-09-04T00:20:35Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-09-04T00:20:40Z

@cubic-dev-ai review this PR

@j15z I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 24 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Re-trigger cubic}

A profile stated its certifications twice: once as a bare SOC 2 yes/no and again under additional compliance. Merge them into one `compliance` fact that carries every certification, its report scope, and its access conditions in one place, keeping the long-form text from both. Add a `scim` row next to SSO. Sim ships SCIM 2.0 directory provisioning on Enterprise, and provisioning is a distinct buying question from authentication, so it deserves its own comparable row rather than a clause buried in the SSO cell. Where a profile described provisioning inside its SSO fact, that content moves to the new row and SSO keeps authentication. Every SCIM value is sourced to a vendor page fetched today. Where a product does not publish its provisioning support, the row says it is not documented and carries `confidence: 'unknown'` rather than asserting the product lacks it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Runs the comparison-page fact check end to end with no arguments: preflight that web fetching and the browser both work, one read-only checker per profile, then a fresh fixer that sees only that profile's flags and decides for itself which are real. Encodes what three review rounds taught us. Long-form value and detail are never shortened, because answer engines read them and a previous run cut 456 KB of sourced text. Absence of evidence is written as "not documented", never as absence of the feature. Pages that defeat fetching are retried in a browser and then reported as unverifiable rather than guessed. Dates refresh only for pages actually read. Nothing is written to the repo except edits to the data files. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Recheck all 20 competitor profiles and Sim's own against public primary sources in batches of 5 (checker + independent fixer per profile), correcting stale capability, pricing, licensing, and product lifecycle claims while preserving explicit uncertainty and scope qualifications. Notable corrections: Flowise wound down and archived its repo; LangChain now holds ISO 27001 certification the profile said it lacked; OpenAI AgentKit's ZDR/background-mode compatibility was stated backwards; Gumloop repositioned from visual builder to chat-agent product and dropped its free tier; Retool self-hosting is Enterprise-only, not all tiers; Tines audit-log retention is now configurable, not fixed; Power Automate is SOC 2 GCC High in-scope, contrary to the prior claim. Refreshed Sim's own internal counts cited across every profile: blocks 266->356, tool actions ~3,900->~5,500, providers 21->25, KB connectors 51->65, trigger apps 61->64. Claims that could not be independently confirmed are marked confidence: 'unknown' rather than asserted, and the sources arrays that support each claim were refreshed to the date they were actually re-read: docs.sim.ai, trust.sim.ai, and each competitor's own docs, security/trust center, and pricing pages, per the audit-comparisons skill. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

cubic-dev-ai Bot reviewed Sep 3, 2026

View reviewed changes

Comment thread apps/sim/app/(landing)/comparisons/comparison-sections.ts Outdated

greptile-apps Bot reviewed Sep 3, 2026

View reviewed changes

Comment thread apps/sim/app/(landing)/comparisons/comparison-sections.ts Outdated

j15z force-pushed the feat/update-comparison-compliance branch from c3825bf to f611370 Compare September 3, 2026 23:42

vercel Bot temporarily deployed to Preview September 3, 2026 23:42 Inactive

cubic-dev-ai Bot reviewed Sep 3, 2026

View reviewed changes

j15z force-pushed the feat/update-comparison-compliance branch from f611370 to 661020f Compare September 3, 2026 23:49

vercel Bot temporarily deployed to Preview September 3, 2026 23:49 Inactive

cubic-dev-ai Bot reviewed Sep 3, 2026

View reviewed changes

j15z force-pushed the feat/update-comparison-compliance branch from 661020f to 3ca2ce8 Compare September 4, 2026 00:20

vercel Bot temporarily deployed to Preview September 4, 2026 00:20 Inactive

cubic-dev-ai Bot reviewed Sep 4, 2026

View reviewed changes

vercel Bot temporarily deployed to Preview September 4, 2026 01:14 Inactive

vercel Bot temporarily deployed to Preview September 4, 2026 17:20 Inactive

vercel Bot temporarily deployed to Preview September 4, 2026 17:45 Inactive

vercel Bot temporarily deployed to Preview September 4, 2026 17:54 Inactive

j15z force-pushed the feat/update-comparison-compliance branch from 1a298a7 to 3aa79cf Compare September 5, 2026 02:39

vercel Bot temporarily deployed to Preview September 5, 2026 02:39 Inactive

j15z changed the title ~~fix(comparisons): update compliance certifications~~ fix(comparisons): refresh verified claims and citations Sep 5, 2026

j15z force-pushed the feat/update-comparison-compliance branch from 3aa79cf to 8926217 Compare September 15, 2026 23:24

vercel Bot temporarily deployed to Preview September 15, 2026 23:24 Inactive

j15z changed the title ~~fix(comparisons): refresh verified claims and citations~~ fix(comparisons): reverify every profile and merge the compliance facts Sep 15, 2026

j15z and others added 3 commits September 15, 2026 17:12

j15z force-pushed the feat/update-comparison-compliance branch from 8926217 to 17545d5 Compare September 16, 2026 03:07

vercel Bot deployed to Preview September 16, 2026 03:10 View deployment

Uh oh!

Conversation

j15z commented Sep 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Scope

Review

Unverifiable claims (confidence: 'unknown', or otherwise flagged as access-limited)

Type of Change

Testing

Checklist

Uh oh!

vercel Bot commented Sep 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

greptile-apps Bot commented Sep 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

Uh oh!

j15z commented Sep 3, 2026

Uh oh!

j15z commented Sep 3, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 3, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

j15z commented Sep 3, 2026

Uh oh!

j15z commented Sep 3, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 3, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

j15z commented Sep 4, 2026

Uh oh!

j15z commented Sep 4, 2026

Uh oh!

cubic-dev-ai Bot commented Sep 4, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

j15z commented Sep 3, 2026 •

edited

Loading

Unverifiable claims (confidence: `'unknown'`, or otherwise flagged as access-limited)

vercel Bot commented Sep 3, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

greptile-apps Bot commented Sep 3, 2026 •

edited

Loading