Skip to content

docs: update pipedream docs - #2945

Open
daveomri wants to merge 54 commits into
apify:masterfrom
daveomri:docs/pipedream-update-docs
Open

daveomri wants to merge 54 commits into
apify:masterfrom
daveomri:docs/pipedream-update-docs

Conversation

@daveomri

@daveomri daveomri commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Updates the Pipedream integration doc to match the connector changes from the retest, plus some housekeeping.

What changed

  • Copy: removed the deleted Run Task Synchronously action; updated Run Actor params (schema-driven input, Memory dropdown, Build accepts tag or number, Wait for Finish switch); Get Dataset Items now notes ID or name.
  • Screenshots: refreshed the Run Actor screenshot and converted all images to WebP (dropped the PNGs).
  • Logo + hub cards: added the official Pipedream SVG logo and a Pipedream card to both integration hubs (it was previously missing from the hubs; the old PNG logo was unreferenced).
  • AI usage: added a note pointing to the Apify MCP server for AI agents, and a caution that Pipedream's AI builder is token-heavy for this schema-driven connector.

Note

** Merge this alongside the connector release**, not before.

daveomri and others added 30 commits July 23, 2025 13:08
…le.md

Co-authored-by: Michał Olender <92638966+TC-MO@users.noreply.github.com>
…le.md

Co-authored-by: Michał Olender <92638966+TC-MO@users.noreply.github.com>
…le.md

Co-authored-by: Michał Olender <92638966+TC-MO@users.noreply.github.com>
…le.md

Co-authored-by: Michał Olender <92638966+TC-MO@users.noreply.github.com>
…le.md

Co-authored-by: Michał Olender <92638966+TC-MO@users.noreply.github.com>
…le.md

Co-authored-by: Michał Olender <92638966+TC-MO@users.noreply.github.com>
…le.md

Co-authored-by: Michał Olender <92638966+TC-MO@users.noreply.github.com>
…le.md

Co-authored-by: Michał Olender <92638966+TC-MO@users.noreply.github.com>
…le.md

Co-authored-by: Michał Olender <92638966+TC-MO@users.noreply.github.com>
…le.md

Co-authored-by: Michał Olender <92638966+TC-MO@users.noreply.github.com>
…le.md

Co-authored-by: Michał Olender <92638966+TC-MO@users.noreply.github.com>
…le.md

Co-authored-by: Michał Olender <92638966+TC-MO@users.noreply.github.com>
…le.md

Co-authored-by: Michał Olender <92638966+TC-MO@users.noreply.github.com>
…le.md

Co-authored-by: Michał Olender <92638966+TC-MO@users.noreply.github.com>
…le.md

Co-authored-by: Michał Olender <92638966+TC-MO@users.noreply.github.com>
…le.md

Co-authored-by: Michał Olender <92638966+TC-MO@users.noreply.github.com>
@daveomri daveomri self-assigned this Sep 10, 2026
@daveomri
daveomri marked this pull request as ready for review September 14, 2026 07:38
daveomri and others added 2 commits September 14, 2026 10:23
- Use the app's **Wait for Finish** casing, consistent with the other UI
  labels on the page
- Point the Actor task alternative at **Get dataset items**, since
  **Run task** returns the run without its output
- Make the large-output bullets consistently punctuated

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KjACXcv8gdDnNzCjsAs23M

@marcel-rbro marcel-rbro left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocked on the connector release.

- **Get key-value store record** - Retrieves a record from a [key-value store](/storage/key-value-store).
- **Run Actor** - Runs a selected Actor and, by default, waits for it to finish and returns the `OUTPUT` record. Input fields are generated from the Actor's input schema. Optional fields: **Build**, **Timeout (seconds)**, **Memory (MB)**, **Max Items**, **Max Total Charge USD**, **Webhook URL**, and **Output Record Key**.
- **Run task** - Runs a selected task and, by default, waits for it to finish and returns the run, without its output. Use **Override Input (JSON)** to replace the task's saved input for a single run, and leave it empty to use the saved input. Optional fields: **Build**, **Timeout (seconds)**, and **Memory (MB)**.
- **Scrape single URL** - Runs [Website Content Crawler](https://apify.com/apify/website-content-crawler) on a specified URL and returns its content as HTML. Use this for extracting content from a single page, e.g. in LLM workflows. **Crawler Type** selects the engine: **Firefox (Headless Browser)** renders JavaScript (the default), **Cheerio (Raw HTTP)** sends raw HTTP requests and renders none, and **Adaptive** switches per page.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you confirm the engine list? On master the crawlerType prop has four options and this covers three, missing playwright:chrome — the suggestion assumes it's still there and uses the app's own labels. The adaptive option's label is a full sentence in the app, so there's no short name to bold.

Suggested change
- **Scrape single URL** - Runs [Website Content Crawler](https://apify.com/apify/website-content-crawler) on a specified URL and returns its content as HTML. Use this for extracting content from a single page, e.g. in LLM workflows. **Crawler Type** selects the engine: **Firefox (Headless Browser)** renders JavaScript (the default), **Cheerio (Raw HTTP)** sends raw HTTP requests and renders none, and **Adaptive** switches per page.
- **Scrape single URL** - Runs [Website Content Crawler](https://apify.com/apify/website-content-crawler) on a specified URL and returns its content as HTML. Use this for extracting content from a single page, e.g. in LLM workflows. **Crawler Type** selects the engine: **Headless browser (stealthy Firefox+Playwright)** is the default, **Headless browser (Chrome+Playwright)** also renders JavaScript, **Raw HTTP client (Cheerio)** sends raw HTTP requests and renders none, and the adaptive option switches between raw HTTP and Chrome per page.


## Handle large Actor output

When **Run Actor** waits for a run, it returns the record named by **Output Record Key** (`OUTPUT` by default). If the record is over 256 KB, the step returns a reference instead of inline data.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does the 256 KB threshold come from, Pipedream's step output or Apify's key-value store? Readers need to know whose limit it is to know what to change, and naming the reference object's fields here grounds the recordUrl and recordKey used just below.

Suggested change
When **Run Actor** waits for a run, it returns the record named by **Output Record Key** (`OUTPUT` by default). If the record is over 256 KB, the step returns a reference instead of inline data.
When **Run Actor** waits for a run, it returns the record named by **Output Record Key** (`OUTPUT` by default). If the record is over 256 KB, the step returns a reference object with `recordUrl`, `keyValueStoreId`, and `recordKey` instead of inline data.

1. Select the trigger you want to use, e.g. **New Finished Actor Run**.
1. Configure the trigger by selecting the Actor or task to monitor.
1. Select the trigger you want to use, e.g. **New finished Actor run (instant)**.
1. Configure the trigger by selecting the Actor or task to monitor. Leave **Trigger on run states** empty to fire on every terminal state, or pick the states you care about.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better to name the terminal states here? They're only listed under Triggers further down, so at this point the reader can't tell what they'd be opting out of.

Suggested change
1. Configure the trigger by selecting the Actor or task to monitor. Leave **Trigger on run states** empty to fire on every terminal state, or pick the states you care about.
1. Configure the trigger by selecting the Actor or task to monitor. Leave **Trigger on run states** empty to fire on every terminal state (**Succeeded**, **Failed**, **Timed out**, and **Aborted**), or pick the states you care about.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants