Name: scrape
Price: 0.04 USDC
Availability: InStock

$ man scrape

PRICE / CALL

$0.04

USDC · base mainnet · scheme: exact

METHOD

POST

CLUSTER

CATEGORY

utilities

STATUS

● live

NAME

scrape — web scraper / html extractor

SYNOPSIS

POST https://x402.org/v1/scrape
     Content-Type: application/json
     X-PAYMENT:    <signed-transferWithAuthorization>

     { ... }

↳ first call → 402 Payment Required. Sign USDCtransferWithAuthorization, retry with theX-PAYMENT header.

DESCRIPTION

Web scraper / HTML extractor. Title, OG/Twitter meta, body in text/HTML/markdown, optional links. Cheerio-based.

INPUT — request schema

property	type	description	req?
url	string	Public http/https URL to scrape.	required
format	string	'text' (default), 'html', or 'markdown'. enum: text · html · markdown	optional
include_links	boolean	Include array of unique links found on the page (max 500). Default false.	optional
user_agent	string	Override the User-Agent header.	optional

OUTPUT — response shape

field	type	description
url	string	Final URL fetched after following redirects.
final_url	string	Final URL after redirects.
status_code	number	HTTP status code returned by the upstream server.
title	string	Page title from the <title> tag.
description	string	Meta description from the page's <meta name="description"> or OG description tag.
canonical	string	Canonical URL from <link rel="canonical"> when present.
lang	string	Language code from the <html lang> attribute, like en or fr.
h1	string	Text of the first <h1> heading on the page.
og	object	All og:* meta tags.
twitter	object	All twitter:* meta tags.
text	string	Body when format=text.
html	string	Body when format=html.
markdown	string	Body when format=markdown.
format	string	Output format of the body content: text, html, or markdown.
links	array	Array of {href, text} when include_links=true.
body_chars	number	Character count of the extracted body content.

EXAMPLES — two ways to call

EXAMPLE 1 · curl

curl -X POST https://x402.org/v1/scrape \
  -H 'Content-Type: application/json' \
  -d '{ }'

first response = 402 Payment Required with payment requirements; sign + retry with X-PAYMENT.

EXAMPLE 2 · mcp

# install once
claude mcp add x402 --command "npx x402-deployer-mcp"

# then ask Claude Code:
# "use the scrape tool to ..."

MCP server handles payment automatically — your coding agent just calls the tool by name.

METADATA

ADJACENT — other endpoints in webprobe

endpoint	description	price
arxiv-summarize	arXiv paper summarizer / research-paper TLDR.	$0.04
rss-from-anything	RSS feed generator / HTML to RSS converter.	$0.04
scrape-website	Web scraper / HTML to text / website content extractor.	$0.04
screenshot	Website screenshot / URL to PNG/JPG.	$0.04
webpage-diff	Webpage change detection / website monitor / content diff.	$0.04
website-screenshot	URL to PNG / JPG / website screenshot tool.	$0.04
arxiv-search	arXiv full-text search.	$0.03
domain-suggest	Brandable domain suggester / startup-name brainstormer / product-name proposer / domain candidate generator.	$0.05

/scrape(1)