Skip to content

Web crawler

Updated View as Markdown

Reuses the original crawler chain: Firecrawl → Cloudflare Browser Rendering → Bright Data. Configure one or more providers; one page is fetched per request.

What you can do (1)

What you can do

Available commands: 1 (1 read, 0 write). Enable only the commands your application needs.

Command Effect
Scrape page text and links — crawler.crawl Read

Connect and use

Connect and use

Open the integration's connection form to check the available credential source. The source has not been resolved for this documentation export. The instructions below apply when using your own provider account.

Get your provider credentials

Follow these steps for Web crawler, then enter the issued values in the connection form below.

  1. Choose an upstream: copy a Firecrawl dashboard key into Firecrawl API key; Cloudflare Browser Rendering needs Account ID and a scoped Browser Rendering token; Bright Data needs API key and an enabled Web Unlocker zone. Fill only the fields for products you have enabled.

Official credential setup

Where to enter credentials

In the integration page, sign in, choose your workspace and open Connect. If a Credentials choice is shown, select Use my credentials. Fill the following connection fields and save. Provider credentials belong in this form, never in an MCP configuration or tool input.

Connection field Requirement
Firecrawl API Key Optional
Cloudflare Account ID Optional
Cloudflare Browser Rendering Token Optional
Bright Data API Key Optional
Bright Data Web Unlocker Zone Optional

Open API keys and create a Tooluse.net key for the enabled commands.

Set TOOLUSE_API_KEY in your client and use the shared MCP configuration.

Configure Claude Code, Codex or Cursor once

Permissions and results

If you have more than one workspace, check the selected workspace in the connection or API-key dialog before saving. Roles, enabled commands, expiry and revocation use Tooluse.net's existing permissions. Disabling a connection or revoking a key immediately prevents new calls.

Send one Claude tool_use block per request. Keep the same Idempotency-Key when retrying the same request; use a new key for a new call. The response includes the shared Tooluse.net receipt and Claude tool_result. Check both HTTP status and the returned tool result before treating a call as successful.

Your connection's daily allowance and key permissions also apply. Adding a connection does not make a provider request.

Calls return the shared execution receipt and a Claude tool_result. Provider output varies by operation; consult the provider documentation for its response fields. Unconfirmed calls should be checked before retrying.

Call from your client

Use the shared MCP configuration. Refresh your client's tool list after saving the connection or changing key permissions, then call the returned tool name with the example input below.

Configure Claude Code, Codex or Cursor once

Discover your enabled tools

List permitted tools in Claude format. Read every page using page and size (maximum 100) until all results have been received. Use the returned name, not a catalog capability identifier, in the tool_use block.

curl 'https://api.tooluse.net/v1/tools' \
  -H "Authorization: Bearer $TOOLUSE_API_KEY"

Find this command in the tool list permitted by your key. Replace svc_FROM_YOUR_TOOL_LIST with its returned name. Use a new Idempotency-Key for each new call and retain it when retrying that same call.

HTTP call

curl 'https://api.tooluse.net/v1/tool-calls' \
  -H "Authorization: Bearer $TOOLUSE_API_KEY" \
  -H 'Content-Type: application/json' \
  -H 'Idempotency-Key: replace-with-a-new-call-id' \
  --data '{
  "type": "tool_use",
  "id": "toolu_example",
  "name": "svc_FROM_YOUR_TOOL_LIST",
  "input": {
    "url": "https://example.com/"
  }
}'

Send one Claude tool_use block per request. Keep the same Idempotency-Key when retrying the same request; use a new key for a new call. The response includes the shared Tooluse.net receipt and Claude tool_result. Check both HTTP status and the returned tool result before treating a call as successful.

Commands

Scrape page text and links — crawler.crawl

Run Scrape page text and links with Web crawler. This is a read command; it does not grant access beyond the connected account's permissions.

This is a read operation. Provider rate limits and data access rules still apply.

Connect and use

Parameters

Requiredness is local to the containing object. A required child is required only when its parent object or array item is provided. Alternative schemas are identified in the Constraints column; the linked full JSON Schema is authoritative.

Parameter Type Required in parent Description Constraints
url string Required Target page URL minLength: 1; maxLength: 2000; format: "uri"

Full JSON Schema

Full JSON Schema

Example input

{
  "url": "https://example.com/"
}

At most 3 provider requests per invocation; timeout: 150 seconds.

Errors

Check the key's expiry, revocation, workspace and allowed operations.

Reuse the same idempotency key only for the same request.

The connection or provider limit has been reached. Check the receipt before retrying.

Test the connection

Open the integration's commands below to inspect parameters and examples. Start with a read command for an account or resource you can access. Refresh your client's tool list, use the returned tool name, and check the execution result and usage. Provider charges and permissions still apply. Never test a write against production data without reviewing it.

Official provider documentation

Type to search…

↑↓ navigate↵ selectEsc close