Sign In Register
0 searches
+ 0 extra credits

Extract API

Release Notes
API uptime
83.494%

Our Extract API fetches a public web page and returns its content as structured JSON, or as Markdown with output=md. Map mode lists a website's URLs, and crawl mode extracts every page it visits on the same website.


Websites can opt out of the Extract API crawler by editing their robots.txt file.


The API endpoint is https://serpapi.com/search?engine=extract

Head to the playground for a live and interactive demo.

API Parameters

Search Query

url

Required

Parameter defines the URL of the page to extract. It must be an absolute http or https URL. Search pages that have a dedicated SerpApi engine, such as Google or Bing, are redirected to that engine.

Mode

mode

Optional

Parameter defines the extraction mode. It can be set to:
scrape (default) - Extract a single page.
map - List the website's URLs from the page links and sitemaps.
crawl - Follow links on the same website and extract every page visited.

Crawl mode consumes one search credit per successfully extracted page.

Crawl Settings

max_pages

Optional

Used with mode=crawl. Parameter defines the maximum number of pages to extract, including the starting page. Default: 10. Maximum: 100. Each successfully extracted page consumes one search credit.

max_depth

Optional

Used with mode=crawl. Parameter defines how many links deep the crawl follows from the starting page. 0 extracts only the starting page. Default: 2. Maximum: 3.

include_subdomains

Optional

Used with mode=map and mode=crawl. Parameter defines whether to include other subdomains of the same website, e.g. blog.example.com when starting from www.example.com. Default: false (same host only).

include_sitemap

Optional

Used with mode=map. Parameter defines whether to also read the website's sitemaps, from /sitemap.xml and /robots.txt. Default: true.

Fetch Settings

prefetch_delay

Optional

Parameter defines a delay in milliseconds before the first page is fetched (0–10,000). Default: 0. The Extract API doesn't run JavaScript, so this is not a rendering delay.

timeout

Optional

Parameter defines the HTTP request timeout in milliseconds (1,000–120,000). Default: 30,000 (30 seconds).

include_tags

Optional

Parameter defines a comma-separated list of CSS selectors to keep, e.g. article, main, .post-content. Everything else inside <body> is removed before parsing.

exclude_tags

Optional

Parameter defines a comma-separated list of CSS selectors to remove before parsing, e.g. nav, footer, .ads, #cookie-banner.

include_markdown

Optional

Parameter defines whether to add the page's main content as Markdown in text_content.markdown. Default: false.

include_inline_assets

Optional

Parameter defines whether to include inline <style> and <script> blocks, with their content, in stylesheets and scripts. Default: false.

Serpapi Parameters

engine

Required

Set parameter to extract to use the Extract API engine.

device

Optional

Parameter defines the device to use to get the results. It can be set to desktop (default) to use a regular browser, tablet to use a tablet browser (currently using iPads), or mobile to use a mobile browser.

no_cache

Optional

Parameter will force SerpApi to fetch the Extract results even if a cached version is already present. A cache is served only if the query and all parameters are exactly the same. Cache expires after 1h. Cached searches are free, and are not counted towards your searches per month. It can be set to false (default) to allow results from the cache, or true to disallow results from the cache. no_cache and async parameters should not be used together.

async

Optional

Parameter defines the way you want to submit your search to SerpApi. It can be set to false (default) to open an HTTP connection and keep it open until you got your search results, or true to just submit your search to SerpApi and retrieve them later. In this case, you'll need to use our Searches Archive API to retrieve your results. async and no_cache parameters should not be used together. async should not be used on accounts with Ludicrous Speed enabled.

zero_trace

Optional

Enterprise only. Parameter enables ZeroTrace mode. It can be set to false (default) or true. Enable this mode to skip storing search parameters, search files, and search metadata on our servers. This may make debugging more difficult.

api_key

Required

Parameter defines the SerpApi private key to use.

output

Optional

Parameter defines the format of the result. It can be set to json (default) to get a structured JSON of the results, html to get the raw HTML retrieved, or md to get a markdown-formatted version optimized for LLMs and AI agents.

json_restrictor

Optional

Parameter defines the fields you want to include in the output, reducing payload size for faster response. See the JSON Restrictor documentation for more details.

API Results

JSON Results

JSON output begins with search_metadata, search_parameters and search_information. In scrape mode (default), it continues with page_information, meta_tags, open_graph, twitter_card, dublin_core, json_ld, microdata, breadcrumbs, headings, inline_links, inline_images, inline_videos, inline_audios, inline_iframes, scripts, stylesheets, feeds, lists, tables, forms, sections, price_offers and text_content. Empty sections are omitted. Map mode returns a urls array, crawl mode returns a pages array.

Entries with a URL, such as inline_links, breadcrumbs, price_offers, urls and pages, include a serpapi_extract_link for extracting that URL next.

A search status is accessible through search_metadata.status. It flows this way: Processing -> Success || Error. If a search has failed, error will contain an error message. search_metadata.id is the search ID inside SerpApi.

HTML Results

HTML output returns the fetched HTML of the target page, up to 5 MB. Opening it requires your account session or api_key, e.g. /searches/SEARCH_ID.html?api_key=YOUR_API_KEY.

Markdown Results

Markdown output returns the page's main content as Markdown. See Markdown Output.

To access, use the output=md parameter, the /search.md endpoint, or request header Accept: text/markdown.

Note on Engine Redirects

Search pages that have a dedicated SerpApi engine, such as Google, Bing, eBay or Walmart, are redirected to that engine and return its JSON structure.

Following robots.txt

The Extract API follows each website's robots.txt. It uses the rules for the serpapi user agent, or the rules for * if there are none for serpapi.

For example, adding these lines to a website's robots.txt would fully block the Extract API, while still allowing other crawlers:

User-agent: serpapi
Disallow: /

If the URL, or a URL it redirects to, is disallowed, the search returns an error. Map mode leaves out disallowed URLs and sitemaps, and crawl mode doesn't follow disallowed links.

Crawl Credit Usage

Crawl mode consumes one search credit per successfully extracted page, including the starting page, so max_pages=10 can cost up to 10 credits. Failed pages and cached responses are not charged. search_metadata.credits_consumed shows the cost.

API Examples

Scrape a single page url: https://example.com/


JSON Example

{
  "search_metadata": {
    "id": "6a1234abc56789def0123456",
    "status": "Success",
    "credits_consumed": 1,
    "json_endpoint": "https://serpapi.com/searches/abcd1234ef567890/6a1234abc56789def0123456.json",
    "created_at": "2026-06-02 14:21:08 UTC",
    "processed_at": "2026-06-02 14:21:08 UTC",
    "extract_url": "https://example.com/",
    "raw_html_file": "https://serpapi.com/searches/abcd1234ef567890/6a1234abc56789def0123456.html",
    "total_time_taken": 0.58
  },
  "search_parameters": {
    "engine": "extract",
    "url": "https://example.com/",
    "device": "desktop"
  },
  "search_information": {
    "organic_results_state": "Results for exact URL"
  },
  "page_information": {
    "title": "Example Domain",
    "url": "https://example.com/",
    "final_url": "https://example.com/",
    "language": "en"
  },
  "meta_tags": {
    "viewport": "width=device-width, initial-scale=1"
  },
  "headings": [
    { "position": 1, "level": 1, "title": "Example Domain" }
  ],
  "inline_links": [
    {
      "position": 1,
      "title": "Learn more",
      "link": "https://iana.org/domains/example",
      "displayed_link": "iana.org",
      "internal": false,
      "serpapi_extract_link": "https://serpapi.com/search.json?device=desktop&engine=extract&url=https%3A%2F%2Fiana.org%2Fdomains%2Fexample"
    }
  ],
  "text_content": {
    "snippet": "Example Domain This domain is for use in documentation examples without needing permission. Avoid use in operations. Learn more",
    "excerpt": "Example Domain This domain is for use in documentation examples without needing permission. Avoid use in operations. Learn more",
    "full": "Example Domain This domain is for use in documentation examples without needing permission. Avoid use in operations. Learn more",
    "character_count": 127,
    "word_count": 19
  }
}

Map a site mode: map


JSON Example

{
  "search_metadata": {
    "id": "6a1234abc56789def0123457",
    "status": "Success",
    "credits_consumed": 1,
    "extract_url": "https://books.toscrape.com/",
    "total_time_taken": 0.62
  },
  "search_parameters": {
    "engine": "extract",
    "url": "https://books.toscrape.com/",
    "mode": "map",
    "device": "desktop"
  },
  "search_information": {
    "final_url": "https://books.toscrape.com/",
    "organic_results_state": "Map built"
  },
  "urls": [
    {
      "url": "https://books.toscrape.com/index.html",
      "title": "Books to Scrape",
      "source": "page",
      "serpapi_extract_link": "https://serpapi.com/search.json?device=desktop&engine=extract&url=https%3A%2F%2Fbooks.toscrape.com%2Findex.html"
    },
    {
      "url": "https://books.toscrape.com/catalogue/category/books_1/index.html",
      "title": "Books",
      "source": "page",
      "serpapi_extract_link": "https://serpapi.com/search.json?device=desktop&engine=extract&url=https%3A%2F%2Fbooks.toscrape.com%2Fcatalogue%2Fcategory%2Fbooks_1%2Findex.html"
    }
  ]
}

Crawl a site mode: crawl


JSON Example

{
  "search_metadata": {
    "id": "6a1234abc56789def0123458",
    "status": "Success",
    "credits_consumed": 3,
    "extract_url": "https://books.toscrape.com/",
    "total_time_taken": 0.85
  },
  "search_parameters": {
    "engine": "extract",
    "url": "https://books.toscrape.com/",
    "mode": "crawl",
    "max_pages": "3",
    "max_depth": "1",
    "device": "desktop"
  },
  "search_information": {
    "organic_results_state": "Crawl complete"
  },
  "pages": [
    {
      "position": 1,
      "url": "https://books.toscrape.com/",
      "depth": 0,
      "serpapi_extract_link": "https://serpapi.com/search.json?device=desktop&engine=extract&max_depth=1&max_pages=3&url=https%3A%2F%2Fbooks.toscrape.com%2F",
      "page_information": { "title": "All products | Books to Scrape - Sandbox" },
      "headings": [{ "position": 1, "level": 1, "title": "All products" }]
    },
    {
      "position": 2,
      "url": "https://books.toscrape.com/index.html",
      "depth": 1,
      "serpapi_extract_link": "https://serpapi.com/search.json?device=desktop&engine=extract&max_depth=1&max_pages=3&url=https%3A%2F%2Fbooks.toscrape.com%2Findex.html",
      "page_information": { "title": "All products | Books to Scrape - Sandbox" }
    },
    {
      "position": 3,
      "url": "https://books.toscrape.com/catalogue/category/books_1/index.html",
      "depth": 1,
      "serpapi_extract_link": "https://serpapi.com/search.json?device=desktop&engine=extract&max_depth=1&max_pages=3&url=https%3A%2F%2Fbooks.toscrape.com%2Fcatalogue%2Fcategory%2Fbooks_1%2Findex.html",
      "page_information": { "title": "Books | Books to Scrape - Sandbox" }
    }
  ]
}

Additional Parameters

Filter output to specific HTML tags include_tags


JSON Example

{
  ...
  "search_parameters": {
    "engine": "extract",
    "url": "https://example.com/",
    "device": "desktop",
    "include_tags": "h1,p,a"
  },
  "headings": [
    {
      "position": 1,
      "level": 1,
      "title": "Example Domain"
    }
  ],
  "inline_links": [
    {
      "position": 1,
      "title": "Learn more",
      "link": "https://iana.org/domains/example",
      "displayed_link": "iana.org",
      "internal": false,
      "serpapi_extract_link": "https://serpapi.com/search.json?device=desktop&engine=extract&include_tags=h1%2Cp%2Ca&url=https%3A%2F%2Fiana.org%2Fdomains%2Fexample"
    }
  ],
  "text_content": {
    "snippet": "Example Domain This domain is for use in documentation examples without needing permission. Avoid use in operations. Learn more",
    "excerpt": "Example Domain This domain is for use in documentation examples without needing permission. Avoid use in operations. Learn more",
    "full": "Example Domain This domain is for use in documentation examples without needing permission. Avoid use in operations. Learn more",
    "character_count": 127,
    "word_count": 19
  }
}

Exclude specific HTML tags exclude_tags


JSON Example

{
  ...
  "search_parameters": {
    "engine": "extract",
    "url": "https://example.com/",
    "device": "desktop",
    "exclude_tags": "nav,footer,aside,script"
  }
}

Delay before fetching prefetch_delay (milliseconds)


JSON Example

{
  ...
  "search_parameters": {
    "engine": "extract",
    "url": "https://example.com/",
    "device": "desktop",
    "prefetch_delay": "2000"
  }
}

Custom fetch timeout timeout (milliseconds)


JSON Example

{
  ...
  "search_parameters": {
    "engine": "extract",
    "url": "https://example.com/",
    "device": "desktop",
    "timeout": "30000"
  }
}

Mobile device device: mobile


JSON Example

{
  ...
  "search_parameters": {
    "engine": "extract",
    "url": "https://example.com/",
    "device": "mobile"
  }
}