subimpact WhatsApp us

Standby RAG Microservice API

Standby RAG Microservice API converts web pages into token-optimised Markdown and returns markdown, title, description, author, publishedTime, jsonLd entities, tokenEstimate, charCount, wordCount and crawledAt. It runs as a warm standby endpoint under a second on POST /convert, which also accepts raw html for instant conversion, or as a batch run over startUrls. It costs $0.001 per page ($1.00 per 1,000), billed only on successful conversion.

from $0.001 per clean markdown page converted, $1.00 per 1,000 · Updated Nov 2026, version 0.1.8

Key facts

Price
from $0.001 per clean markdown page converted, $1.00 per 1,000
Pricing model
Pay per event
Output
JSON, CSV, Excel, XML, RSS via Apify datasets
Access
Apify Console, REST API, Python and JS clients
Version
Updated Nov 2026, version 0.1.8
Actor
subimpact/standby-rag-api

Who it is for

  • RAG engineers who need clean Markdown instead of raw HTML
  • AI agent builders adding a fetch-and-convert tool
  • Data teams estimating token budgets before embedding
  • SEO and schema teams pulling JSON-LD into a graph

Use cases

Fetch a page inside an agent loop

Call the standby endpoint with a url and the response carries data.markdown, data.tokenEstimate and data.jsonLd in well under a second, because the container is already warm. That fits a tool call where a cold container start would blow the latency budget. Send includeJsonLd to get Schema.org entities in the same response instead of a second request.

Convert HTML you already hold

POST an html string instead of a url and the same parser strips the page in about 10 milliseconds with no network fetch at all, which is cheaper and faster than round-tripping through a URL. Useful when your crawler has already downloaded the page, or when you want to normalise archived HTML into Markdown for an index.

Estimate tokens before embedding

tokenEstimate, charCount and wordCount arrive with every conversion, so you can gate a document against a context window before chunking it. A page whose token count exceeds your budget can be trimmed or split deliberately rather than discovered at embedding time, which is the sort of failure that only shows up on the largest documents.

Batch a list of sources

In batch mode, list the pages in startUrls with maxPages as the cap, and each converted row lands in the dataset with Markdown and its metadata. The Actor fetches exactly the URLs you give it and does not follow links, so a run covers your curated list rather than an unbounded crawl. Bloat selectors such as a cookie banner are removed by class with removeSelectors.

Strip noise before indexing

HTML bloat is what makes retrieval noisy: navigation, newsletter popups, cookie banners. Pass those selectors in removeSelectors and the Markdown keeps the article body and headings, giving cleaner chunks and fewer wasted tokens. Pair that with the extracted jsonLd to keep the Article or Product entity alongside the text for structured filtering.

Output fields

FieldTypeDescription
url string Source URL of the converted webpage
title string Page title extracted from <title> or OpenGraph tags
status string Row status: absent for normal conversions, 'error' for uncharged status/error rows
error string Error explanation for uncharged status/error rows (e.g. blank or bot-challenge pages that yielded no usable content)
description string Page description extracted from meta tags
author string Author of the article or page if present
canonicalUrl string Canonical URL extracted from link rel=canonical or OpenGraph tags
rawHtml string Stripped HTML structure prior to Markdown conversion (present when includeHtml is true)
publishedTime string Publication timestamp if available
markdown string Clean, bloat-free, token-optimized Markdown ready for LLM context injection
jsonLd array Extracted Schema.org JSON-LD structured entities
tokenEstimate integer Estimated token budget consumed by the markdown content
Show all 15 fields
FieldTypeDescription
charCount integer Character count of the clean markdown content
wordCount integer Word count of the clean markdown content
crawledAt string ISO timestamp of extraction

Sample output

From one of our own test runs on 20 Nov 2026, trimmed. Personal data is redacted.

1 rows, first 3 shown

[
  {
    "url": "https://example.com",
    "title": "Example Domain",
    "canonicalUrl": "https://example.com",
    "markdown": "This domain is for use in documentation examples without needing permission. This is not a service; avoid relying on it for testing and monitoring purposes.",
    "charCount": 156,
    "wordCount": 25,
    "crawledAt": "2026-11-20T02:41:41.729Z"
  }
]

Input

FieldTypeRequiredDescriptionDefault
mode string No Choose 'auto' (detects Standby HTTP server when warm, batch otherwise), 'standby' (HTTP server for sub-second REST requests), or 'batch' (crawls startUrls). auto
startUrls array No Web page URLs to fetch, convert to token-optimized Markdown, and extract Schema.org entities in batch mode. -
maxPages integer No Maximum number of pages to process during a batch run. 10
removeSelectors array No Custom CSS selectors to strip from HTML before Markdown conversion (e.g. '.newsletter-popup', '#cookie-banner'). []
includeJsonLd boolean No Extract structured Schema.org JSON-LD entities (Articles, Products, FAQs, Organizations) for semantic graph enrichment. true
includeHtml boolean No Include the stripped HTML alongside clean Markdown in the dataset output (increases storage usage). false
proxyConfiguration object No Apify Proxy settings for accessing geo-blocked or protected websites. {"useApifyProxy":false}

Example input

{
  "startUrls": [
    {
      "url": "https://news.ycombinator.com"
    }
  ]
}

Pricing

EventWhat it meansPriceLowest tier
page-converted Charged each time a web page is fetched, stripped of HTML bloat, and converted into token-optimized Markdown with structured Schema.org JSON-LD entities. $0.001 $0.001
apify-actor-start Charged when the Actor starts running. Number of events charged depends on Actor memory (one event per GB, minimum one event). $0.00005 $0.00005

$1.00 for 1,000 clean markdown page converted units at the listed price.

Prices from the Apify Store on 20 November 2026. Apify shows the final price before each run.

How it compares with other Apify actors

Price per 1,000 pages at Apify Store list prices. Every row is a live listing on the Store, read on 20 Nov 2026, so a buyer can check each number at the source.

Actor Price per 1,000 pages (Free plan) Higher plans (Gold) Start fee per run Platform usage Users Rating Last build
Standby RAG Microservice API (Web-to-Markdown + JSON-LD) This API Pay per event $1.00 $1.00 $0.00005 Included 2 No reviews yet 18 Nov 2026
RAG Web Browser Direct match, pay per event $2.28 $1.33 $0.000225 Included 213,874 4.7 (35 reviews) 16 Nov 2026
RAG Web Browser — URL & Website to Markdown for LLM & RAG Direct match, pay per event $2.00 $2.00 None Included 13 No reviews yet 18 Nov 2026
RAG Web Browser Scraper Direct match, pay per event $2.00 $2.00 $0.00005 Paid separately 12 No reviews yet 11 Nov 2026
RAG Web Browser Direct match, pay per event $12.00 $8.00 $0.005 Included 20 No reviews yet 20 Oct 2026
Where RAG Web Browser is stronger

Where RAG Web Browser is stronger

  • Much larger user base and the best known RAG tool on the Apify Store
  • Search and fetch in one call: a query is resolved to pages and read for you
  • Full browser rendering for pages that need scripts to run

Where this API is stronger

  • Warm standby service, so a single page conversion answers without waiting for a cold container
  • Schema.org JSON-LD entities and a token budget returned with each page
  • Batch mode over URL lists from the same actor
Where RAG Web Browser — URL & Website to Markdown for LLM & RAG is stronger

Where RAG Web Browser — URL & Website to Markdown for LLM & RAG is stronger

  • Link crawling with depth limits, same domain and URL pattern filters
  • Choice of Markdown, plain text or HTML with page metadata
  • Reports which providers were tried and how long each fetch took

Where this API is stronger

  • JSON-LD entity extraction alongside the Markdown in the same row
  • Token estimate and character counts for LLM context budgeting
  • Warm standby endpoint for single page calls
Where RAG Web Browser Scraper is stronger

Where RAG Web Browser Scraper is stronger

  • Proxy group selection and query lists in one input
  • Captures language and HTTP status metadata per page

Where this API is stronger

  • Platform usage is included in the per page price
  • JSON-LD entities and a token budget with each converted page
  • Warm standby service for single page calls
Where RAG Web Browser is stronger

Where RAG Web Browser is stronger

  • Optional link extraction and HTML output as add-ons
  • Reading time, site name and image URL fields per page
  • Search rank recorded on rows that came from a query

Where this API is stronger

  • JSON-LD entities and a token budget included in the page charge
  • Warm standby service for single conversions
  • Platform usage included in the page charge

The RAG browser lane is busy and the Apify listing dominates it, with search plus fetch and browser rendering as its strengths. The s-r, datapilot and parseforge actors keep to page conversion with different amounts of metadata. Ours adds JSON-LD extraction and token budgets, and runs as a warm service so single calls come back quickly.

Prices are Apify Store list prices, checked 20 Nov 2026. Platform usage 'Paid separately' means Apify bills compute on top of the event price. Check each listing before you buy.

How to run it

  1. For a single page, POST to the standby /convert endpoint with a body such as {"url": "https://example.com/article", "includeJsonLd": true} and an Apify token in the Authorization header.
  2. Read data.markdown, data.tokenEstimate and data.jsonLd from the response, which also reports executionTimeMs and chargedUsd.
  3. To convert HTML you already have, send {"html": "<html>...</html>"} instead of a url for an instant parse.
  4. For a list of pages, run in batch mode with startUrls, for example [{"url": "https://news.ycombinator.com"}], and set maxPages.
  5. Add removeSelectors such as [".newsletter-banner", "#cookie-consent"] to strip boilerplate before conversion.
  6. Read the dataset rows in the Apify Console and export as JSON, or keep calling the endpoint from your app.

cURL

curl -X POST "https://api.apify.com/v2/acts/subimpact~standby-rag-api/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"startUrls":[{"url":"https://news.ycombinator.com"}]}'

Python

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("subimpact/standby-rag-api").call(run_input={"startUrls":[{"url":"https://news.ycombinator.com"}]})

for item in client.dataset(run["default_dataset_id"]).iterate_items():
    print(item)

JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_APIFY_TOKEN>' });
const run = await client.actor('subimpact/standby-rag-api').call({"startUrls":[{"url":"https://news.ycombinator.com"}]});
const { items } = await client.dataset(run.defaultDatasetId).listItems();

console.log(items);

n8n

Use the Apify node (Run an Actor).
Actor ID: subimpact/standby-rag-api
Input JSON: paste the example input above.

Limits

  • Batch mode does not follow links: only the URLs you list are fetched.
  • Blank pages and bot-challenge pages come back as uncharged error rows rather than retried endlessly.
  • A batch run fails with a status message when every requested URL fails or yields no usable content.
  • maxPages caps a batch run at 50,000 pages.

Questions people ask

What does a conversion cost?

$0.001 per page, which is $1.00 per 1,000 pages, plus $0.00005 per run. You are charged only for a successful conversion, platform usage is included rather than billed separately, and there is no monthly commitment.

What is the difference between standby and batch mode?

Standby runs the Actor as a continuous HTTP service behind a permanent URL, so POST /convert answers in well under a second from a warm container. Batch mode takes startUrls and writes converted rows into the dataset. mode set to auto picks standby when the service is warm and batch otherwise.

Does the response include Schema.org data?

Yes, when includeJsonLd is on. jsonLd holds the extracted entities such as Article, Product, FAQ and Organization, alongside the Markdown, so structured data and text come from one call rather than two.

Can I send HTML instead of a URL?

Yes. POST html rather than url and the conversion skips the network entirely, running in about 10 milliseconds. That is the cheap path when your own crawler has already fetched the page.

Does batch mode follow links?

No. Batch mode fetches exactly the URLs you list in startUrls, up to maxPages, with a ceiling of 50,000. It does not discover links on a page, so the run covers your curated set and stays predictable on cost.

How do I remove cookie banners and popups?

Pass extra CSS selectors in removeSelectors, for example .newsletter-popup and #cookie-banner. They are stripped from the HTML before the Markdown conversion, so the boilerplate never reaches your chunks and the token estimate reflects the cleaned text.

Can I call it from Python or TypeScript?

Yes. Any HTTP client can POST to the /convert endpoint with an Apify token, and the response is JSON with a data object holding markdown, tokenEstimate and jsonLd. That works the same from a LangChain loader, a Node agent tool or a curl command.

Need a data source that is not here?

We build and run scrapers for sites that have no public API. Tell us the site, the fields and how often you need them.