Scrape

Task type: Scrape

Use Scrape to extract raw content from a known web page URL for later analysis.

Run behavior

• Side effects: This task is read-only or computation-only by default. It returns data for later steps rather than directly writing records or sending messages.
• Credit usage: This task is not classified as credit-billed by the workflow task registry.

When to use this task

• Fetch a known page before summarizing it with AI Prompt.
• Extract public page content from a URL stored on a record.
• Collect source material for research or content generation workflows.

Setup tips

• Use known URLs, not search queries.
• Follow with AI Prompt when you need structured extraction.
• Store or pass only the parts of the scrape that downstream steps need.

Examples

Summarize a pricing page

Scrape a known competitor pricing page, then use AI Prompt to summarize packaging and positioning.

Outputs

• Scraped page content and related metadata.

Common pitfalls

• Using Scrape to discover URLs instead of Search.
• Scraping pages that require authentication.
• Passing very large scraped content directly into every downstream prompt.

Reference details

These details are generated from the workflow task registry and match the fields available in the builder.

Required fields

• url

Variable-aware fields

Fields that accept variable references ({{ inputs.x }}, {{ steps.N.y }}) from the workflow's variable graph:

• url

Liquid template fields

Fields that are rendered through Liquid (variables plus filters and control flow):

• url

Builder guidance

When the Workflow Builder Genie configures this task, it follows this guidance:

Use only for known URLs; follow with AI Prompt for extraction if needed.

Default configuration

New instances of this task start with the following configuration:

{
  "name": "Scrape",
  "outputKey": "scrape",
  "taskType": "Scrape",
  "description": "",
  "url": ""
}