name: decodo-scraper description: Search Google, scrape web pages, Amazon product pages, YouTube subtitles, or Reddit (post/subreddit) using the Decodo Scraper OpenClaw Skill. homepage: https://decodo.com credentials:

• DECODO_AUTH_TOKEN env: required:
  • DECODO_AUTH_TOKEN

Decodo Scraper OpenClaw Skill

Use this skill to search Google, scrape any URL, or fetch YouTube subtitles via the Decodo Web Scraping API. Search outputs a JSON object of result sections; Scrape URL outputs plain markdown; Amazon and Amazon search output parsed product-page or search results (JSON). Amazon search uses --query. YouTube subtitles outputs transcript/subtitles. Reddit post and Reddit subreddit output post/listing content (JSON).

Authentication: Set DECODO_AUTH_TOKEN (Basic auth token from Decodo Dashboard → Scraping APIs) in your environment or in a .env file in the repo root.

Errors: On failure the script writes a JSON error to stderr and exits with code 1.

Tools

1. Search Google

Use this to find URLs, answers, or structured search results. The API returns a JSON object whose results key contains several sections (not all may be present for every query):

| Section | Description | |--------|--------------| | organic | Main search results (titles, links, snippets). | | ai_overviews | AI-generated overviews or summaries when Google shows them. | | paid | Paid/sponsored results (ads). | | related_questions | “People also ask”–style questions and answers. | | related_searches | Suggested related search queries. | | discussions_and_forums | Forum or discussion results (e.g. Reddit, Stack Exchange). |

The script outputs only the inner results object (these sections); pagination info (page, last_visible_page, parse_status_code) is not included.

Command:

python3 tools/scrape.py --target google_search --query "your search query"

Examples:

python3 tools/scrape.py --target google_search --query "best laptops 2025"
python3 tools/scrape.py --target google_search --query "python requests tutorial"

Optional: --geo us or --locale en for location/language.

2. Scrape URL

Use this to get the content of a specific web page. By default the API returns content as Markdown (cleaner for LLMs and lower token usage).

Command:

python3 tools/scrape.py --target universal --url "https://example.com"

Examples:

python3 tools/scrape.py --target universal --url "https://example.com"
python3 tools/scrape.py --target universal --url "https://news.ycombinator.com/"

3. Amazon product page

Use this to get parsed data from an Amazon product (or other Amazon) page. Pass the product page URL as --url. The script sends parse: true and outputs the inner results object (e.g. ads, product details, etc.).

Command:

python3 tools/scrape.py --target amazon --url "https://www.amazon.com/dp/PRODUCT_ID"

Examples:

python3 tools/scrape.py --target amazon --url "https://www.amazon.com/dp/B09H74FXNW"

4. Amazon search

Use this to search Amazon and get parsed results (search results list, delivery_postcode, etc.). Pass the search query as --query.

Command:

python3 tools/scrape.py --target amazon_search --query "your search query"

Examples:

python3 tools/scrape.py --target amazon_search --query "laptop"

5. YouTube subtitles

Use this to get subtitles/transcript for a YouTube video. Pass the video ID (e.g. from youtube.com/watch?v=VIDEO_ID) as --query.

Command:

python3 tools/scrape.py --target youtube_subtitles --query "VIDEO_ID"

Examples:

python3 tools/scrape.py --target youtube_subtitles --query "dFu9aKJoqGg"

6. Reddit post

Use this to get the content of a Reddit post (thread). Pass the full post URL as --url.

Command:

python3 tools/scrape.py --target reddit_post --url "https://www.reddit.com/r/SUBREDDIT/comments/ID/..."

Examples:

python3 tools/scrape.py --target reddit_post --url "https://www.reddit.com/r/nba/comments/17jrqc5/serious_next_day_thread_postgame_discussion/"

7. Reddit subreddit

Use this to get the listing (posts) of a Reddit subreddit. Pass the subreddit URL as --url.

Command:

python3 tools/scrape.py --target reddit_subreddit --url "https://www.reddit.com/r/SUBREDDIT/"

Examples:

python3 tools/scrape.py --target reddit_subreddit --url "https://www.reddit.com/r/nba/"

Summary

| Action | Target | Argument | Example command | |--------------------|----------------------|------------|-----------------| | Search | google_search | --query | python3 tools/scrape.py --target google_search --query "laptop" | | Scrape page | universal | --url | python3 tools/scrape.py --target universal --url "https://example.com" | | Amazon product | amazon | --url | python3 tools/scrape.py --target amazon --url "https://www.amazon.com/dp/B09H74FXNW" | | Amazon search | amazon_search | --query | python3 tools/scrape.py --target amazon_search --query "laptop" | | YouTube subtitles | youtube_subtitles | --query | python3 tools/scrape.py --target youtube_subtitles --query "dFu9aKJoqGg" | | Reddit post | reddit_post | --url | python3 tools/scrape.py --target reddit_post --url "https://www.reddit.com/r/nba/comments/17jrqc5/..." | | Reddit subreddit | reddit_subreddit | --url | python3 tools/scrape.py --target reddit_subreddit --url "https://www.reddit.com/r/nba/" |

Output: Search → JSON (sections). Scrape URL → markdown. Amazon / Amazon search → JSON (results e.g. ads, product info, delivery_postcode). YouTube → transcript. Reddit → JSON (content).

Decodo Scraper OpenClaw Skill

Overview

This OpenClaw skill integrates Decodo's Web Scraping API into any OpenClaw-compatible AI agent or LLM pipeline. It exposes these tools that agents can call directly:

| Tool | Description | Perfect for | | :---------------- | :------------------------------------------------------------------- | :--------------------------------------------------------------------------------------- | | google_search | Real-time Google Search (SERP) results as structured JSON. | Market research, competitor analysis, news monitoring, fact-checking, RAG pipelines. | | universal | Scrape & parse any public webpage into clean Markdown. | Summarizing articles, content aggregation, building custom datasets, general web browsing for AI agents. | | amazon | Parse Amazon product page data (price, reviews, specs, ASIN). | eCommerce monitoring, price tracking, competitive intelligence, product research. | | amazon_search | Search Amazon for products by keyword and get parsed results. | Discovering products, tracking trends, and broad market analysis. | | youtube_subtitles | Extract subtitles/transcripts from YouTube videos (by video ID). | Video summarization, content analysis, sentiment tracking, accessibility. | | reddit_post | Fetch a Reddit post's content, comments, and metadata (by post URL). | Social listening, community sentiment analysis, trend tracking, and gathering user feedback. | | reddit_subreddit | Scrape Reddit subreddit listings (by subreddit URL). | Monitoring specific communities, content discovery, niche market research. |

Backed by Decodo's residential and datacenter proxy infrastructure, the skill handles JavaScript rendering, bot detection bypass, and geo-targeting out of the box.

Why use Decodo for your OpenClaw agent?

• Zero blocks & CAPTCHAs. Backed by Decodo's proxy infrastructure from 125M+ locations, the skill automatically handles JavaScript rendering, bot detection, and CAPTCHA bypass.
• Real-time data. Access fresh, up-to-the-minute web data directly within your AI agent's workflow.
• LLM-optimized output. Data is returned in structured JSON or clean Markdown, making it easy for LLMs to understand and process.
• Scalability. Designed for high-volume data collection, ensuring your agent can scale from small tasks to complex projects.
• Minimal Friction. Easy setup with a single authentication token.

Features

• Real-time Google Search results scraping
• Universal URL scraping
• Amazon product page parsing (by URL)
• Amazon search (by query)
• YouTube subtitles/transcript by video ID
• Reddit post content by URL
• Reddit subreddit listing by URL
• Structured JSON or Markdown results
• Simple CLI interface compatible with any OpenClaw agent runtime
• Designed for scalable AI agent web scraping
• Minimal dependencies — just Python with Requests
• Authentication via a single Base64 token from the Decodo dashboard

Prerequisites

• Python 3.9 or higher
• Decodo account with access to the Web Scraping API
• OpenClaw installed on your machine

Setup

1. Clone this repo.

git clone https://github.com/Decodo/decodo-openclaw-skill.git

2. Install dependencies.

pip install -r requirements.txt

3. Set your Decodo auth token as an environment variable (or create a .env file in the project root):

# Linux/macOS Terminal
export DECODO_AUTH_TOKEN="your_base64_token"

# Windows (PowerShell)
$env:DECODO_AUTH_TOKEN="your_base64_token"

# .env file
DECODO_AUTH_TOKEN=your_base64_token

OpenClaw agent integration

This skill ships with a SKILL.md file that defines all tools in the OpenClaw skill format. OpenClaw-compatible agents automatically discover and invoke the tools from this file without additional configuration.

To register the skill with your OpenClaw agent, point it at the repo root — the agent will read SKILL.md and expose google_search, universal, amazon, amazon_search, youtube_subtitles, reddit_post, and reddit_subreddit as callable tools.

Usage

Google Search

Search Google and receive structured JSON. Results are grouped by type: organic (main results), ai_overviews (AI-generated summaries), paid (ads), related_questions, related_searches, discussions_and_forums, and others depending on the query.

python3 tools/scrape.py --target google_search --query "your query"

Scrape a URL

Fetch and convert any webpage to a clean Markdown file:

python3 tools/scrape.py --target universal --url "https://example.com/article"

Amazon product page

Fetch parsed data from an Amazon product page (e.g., ads, product details). Use the product URL:

python3 tools/scrape.py --target amazon --url "https://www.amazon.com/dp/B09H74FXNW"

Amazon search

Search Amazon and get parsed results (e.g., results list, delivery_postcode):

python3 tools/scrape.py --target amazon_search --query "laptop"

YouTube subtitles

Fetch subtitles/transcript for a YouTube video (use the video ID, e.g., from ?v=VIDEO_ID):

python3 tools/scrape.py --target youtube_subtitles --query "dFu9aKJoqGg"

Reddit post

Fetch a Reddit post’s content (use the full post URL):

python3 tools/scrape.py --target reddit_post --url "https://www.reddit.com/r/nba/comments/17jrqc5/serious_next_day_thread_postgame_discussion/"

Reddit subreddit

Fetch a Reddit subreddit listing (use the subreddit URL):

python3 tools/scrape.py --target reddit_subreddit --url "https://www.reddit.com/r/nba/"

Related resources

Decodo Web Scraping API documentation

OpenClaw documentation

ClaWHub – OpenClaw skill registry

License

All code is released under the MIT License.