使用 Firecrawl Tools 执行 Firecrawl Crawl

Preparing Your Aident Workspace...

使用 Firecrawl Tools 执行 Firecrawl Crawl | Aident Loadout

输入

urlstring
必填
The base URL to start crawling from. This is the initial entry point for the web crawler.
delayinteger
Delay in milliseconds between requests to avoid overwhelming the server
limitinteger
Maximum number of pages to crawl. The crawl will stop once this limit is reached. Default is 10.
webhookstring
An optional webhook URL to receive real-time updates on the crawl job. Events include crawl start (`crawl.started`), page crawled (`crawl.page`), and crawl completion (`crawl.completed` or `crawl.failed`). The payload structure matches the `/scrape` endpoint response.
maxDepthinteger
Maximum depth of subpages to crawl relative to the entered URL (not the base domain). A depth of 0 crawls only the entered URL, 1 crawls the entered URL plus pages one path segment deeper, 2 adds two segments deeper, etc. For example, if URL is 'https://example.com/docs/api/', m…
excludePathsarray
A list of Regular Expression (regex) patterns for URL paths to exclude from the crawl. URLs whose paths match any of these patterns will be ignored. For example, `"blog/archive/.*"` would exclude all paths under `/blog/archive/`.
includePathsarray
A list of Regular Expression (regex) patterns for URL paths to include in the crawl. Only URLs whose paths match one of these patterns will be processed. For example, `"products/featured/.*"` would only include paths under `/products/featured/`.
ignoreSitemapboolean
If true, the crawler will ignore any sitemap.xml found on the website.
crawlEntireDomainboolean
If true, allows the crawler to follow internal links to sibling or parent URLs, not just child paths. This is the recommended replacement for 'allowBackwardLinks'.
maxDiscoveryDepthinteger
Maximum depth to crawl based on discovery order. The root site and sitemapped pages have a discovery depth of 0. For example, if you set it to 1 and set ignoreSitemap, you will only crawl the entered URL and all URLs that are linked on that page.
allowBackwardLinksboolean
DEPRECATED: Use 'crawlEntireDomain' instead. If true, allows the crawler to navigate to pages that were linked from pages already visited (i.e., navigate 'backwards').
allowExternalLinksboolean
If true, allows the crawler to follow links that lead to external websites (different domains).

Firecrawl Crawl

服务提供商说明

价格与限制

输入

可观察输出

使用此操作的技能

使用此操作的技能

相关使用场景

官方来源

使用此操作的技能