Extract Markdown Output API
Use output=md, the /search.md endpoint or the Accept: text/markdown header
to get the page's main content as Markdown: its <main> element, or its only
<article>, otherwise the whole body. include_tags and exclude_tags
change which content is used.
To get the same Markdown in the JSON response, set include_markdown=true. It is returned in
text_content.markdown, up to 50,000 characters.
API Examples
Page Markdown output: md
---
search_metadata:
id: 6a1234abc56789def0123459
status: Success
...
search_parameters:
engine: extract
url: "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"
device: desktop
---
# A Light in the Attic
£51.77
In stock (22 available)
---
**Warning!** This is a demo website for web scraping purposes. Prices and ratings here were randomly assigned and have no real meaning.
## Product Description
It's hard to imagine a world without A Light in the Attic. This now-classic collection of poetry and drawings from Shel Silverstein celebrates its 20th anniversary with this special edition. ...
## Product Information
| UPC | a897fe39b1053632 |
| --- | --- |
| Product Type | Books |
| Price (excl. tax) | £51.77 |
| Price (incl. tax) | £51.77 |
| Tax | £0.00 |
| Availability | In stock (22 available) |
| Number of reviews | 0 |
Keep only the title and paragraphs include_tags: h1, p
---
search_metadata:
...
search_parameters:
engine: extract
url: "https://en.wikipedia.org/wiki/Coffee"
device: desktop
include_tags: h1, p
---
# Coffee
**Coffee** is a beverage [brewed](https://en.wikipedia.org/wiki/Coffee_brew) from roasted, ground [coffee beans](https://en.wikipedia.org/wiki/Coffee_bean). Dark-colored and [bitter](https://en.wikipedia.org/wiki/Bitterness), coffee has a [stimulating effect](https://en.wikipedia.org/wiki/Stimulant) on humans due to its [caffeine](https://en.wikipedia.org/wiki/Caffeine) content; however [decaffeinated](https://en.wikipedia.org/wiki/Decaffeinated) coffee is also commercially available. There are also various [coffee substitutes](https://en.wikipedia.org/wiki/Coffee_substitute).
Coffee production begins when the seeds from coffee cherries, the *[Coffea](https://en.wikipedia.org/wiki/Coffea)* plant's fruits, are separated to produce unroasted green coffee beans. The "beans" are [roasted](https://en.wikipedia.org/wiki/Coffee_roasting) and then ground into fine particles. ...
...
Markdown in the JSON response include_markdown: true
{
...
"text_content": {
"snippet": "Books to Scrape We love being scraped! Home Books Poetry A Light in the Attic A Light in the Attic £51.77 In stock (22 available) Warning! This is a demo website for web scraping purposes. Prices and ",
"excerpt": "Books to Scrape We love being scraped! Home Books Poetry A Light in the Attic A Light in the Attic £51.77 In stock (22 available) ...",
"full": "Books to Scrape We love being scraped! Home Books Poetry A Light in the Attic A Light in the Attic £51.77 In stock (22 available) ...",
"character_count": 1476,
"word_count": 239,
"markdown": "\n\n# A Light in the Attic\n\n£51.77\n\nIn stock (22 available)\n\n---\n\n**Warning!** This is a demo website for web scraping purposes. Prices and ratings here were randomly assigned and have no real meaning.\n\n## Product Description\n\n...\n\n## Product Information\n\n| UPC | a897fe39b1053632 |\n| --- | --- |\n| Product Type | Books |\n..."
}
}