# Standby RAG Microservice API

Standby RAG Microservice API converts web pages into token-optimised Markdown and returns markdown, title, description, author, publishedTime, jsonLd entities, tokenEstimate, charCount, wordCount and crawledAt. It runs as a warm standby endpoint under a second on POST /convert, which also accepts raw html for instant conversion, or as a batch run over startUrls. It costs $0.001 per page ($1.00 per 1,000), billed only on successful conversion.

## Key facts

- Price: from $0.001 per clean markdown page converted, $1.00 per 1,000
- Pricing model: Pay per event
- Output: JSON, CSV, Excel, XML, RSS via Apify datasets
- Access: Apify Console, REST API, Python and JS clients
- Version: Updated Nov 2026, build 0.1.8
- Actor: subimpact/standby-rag-api

## Who it is for

- RAG engineers who need clean Markdown instead of raw HTML
- AI agent builders adding a fetch-and-convert tool
- Data teams estimating token budgets before embedding
- SEO and schema teams pulling JSON-LD into a graph

## Use cases

### Fetch a page inside an agent loop

Call the standby endpoint with a url and the response carries data.markdown, data.tokenEstimate and data.jsonLd in well under a second, because the container is already warm. That fits a tool call where a cold container start would blow the latency budget. Send includeJsonLd to get Schema.org entities in the same response instead of a second request.

### Convert HTML you already hold

POST an html string instead of a url and the same parser strips the page in about 10 milliseconds with no network fetch at all, which is cheaper and faster than round-tripping through a URL. Useful when your crawler has already downloaded the page, or when you want to normalise archived HTML into Markdown for an index.

### Estimate tokens before embedding

tokenEstimate, charCount and wordCount arrive with every conversion, so you can gate a document against a context window before chunking it. A page whose token count exceeds your budget can be trimmed or split deliberately rather than discovered at embedding time, which is the sort of failure that only shows up on the largest documents.

### Batch a list of sources

In batch mode, list the pages in startUrls with maxPages as the cap, and each converted row lands in the dataset with Markdown and its metadata. The Actor fetches exactly the URLs you give it and does not follow links, so a run covers your curated list rather than an unbounded crawl. Bloat selectors such as a cookie banner are removed by class with removeSelectors.

### Strip noise before indexing

HTML bloat is what makes retrieval noisy: navigation, newsletter popups, cookie banners. Pass those selectors in removeSelectors and the Markdown keeps the article body and headings, giving cleaner chunks and fewer wasted tokens. Pair that with the extracted jsonLd to keep the Article or Product entity alongside the text for structured filtering.

## Price events

- Clean Markdown Page Converted (`page-converted`): $0.001, lowest tier $0.001. Charged each time a web page is fetched, stripped of HTML bloat, and converted into token-optimized Markdown with structured Schema.org JSON-LD entities.
- Actor Start (`apify-actor-start`): $0.00005, lowest tier $0.00005. Charged when the Actor starts running. Number of events charged depends on Actor memory (one event per GB, minimum one event).

Prices from the Apify Store. Apify shows the final price before each run.

## How it compares with other Apify actors

Price per 1,000 pages at Apify Store list prices, checked 20 Nov 2026.

| Actor | Price per 1,000 pages (Free plan) | Higher plans (Gold) | Start fee per run | Platform usage | Users | Rating | Last build |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Standby RAG Microservice API (Web-to-Markdown + JSON-LD) (this API) | $1.00 | $1.00 | $0.00005 | Included | 2 | No reviews yet | 18 Nov 2026 |
| [RAG Web Browser](https://apify.com/apify/rag-web-browser) | $2.28 | $1.33 | $0.000225 | Included | 213,874 | 4.7 (35 reviews) | 16 Nov 2026 |
| [RAG Web Browser — URL & Website to Markdown for LLM & RAG](https://apify.com/s-r/rag-web-browser) | $2.00 | $2.00 | None | Included | 13 | No reviews yet | 18 Nov 2026 |
| [RAG Web Browser Scraper](https://apify.com/datapilot/rag-web-browser-scraper) | $2.00 | $2.00 | $0.00005 | Paid separately | 12 | No reviews yet | 11 Nov 2026 |
| [RAG Web Browser](https://apify.com/parseforge/rag-web-browser) | $12.00 | $8.00 | $0.005 | Included | 20 | No reviews yet | 20 Oct 2026 |

### Where RAG Web Browser is stronger

- Much larger user base and the best known RAG tool on the Apify Store
- Search and fetch in one call: a query is resolved to pages and read for you
- Full browser rendering for pages that need scripts to run

### Where this API is stronger

- Warm standby service, so a single page conversion answers without waiting for a cold container
- Schema.org JSON-LD entities and a token budget returned with each page
- Batch mode over URL lists from the same actor

### Where RAG Web Browser — URL & Website to Markdown for LLM & RAG is stronger

- Link crawling with depth limits, same domain and URL pattern filters
- Choice of Markdown, plain text or HTML with page metadata
- Reports which providers were tried and how long each fetch took

### Where this API is stronger

- JSON-LD entity extraction alongside the Markdown in the same row
- Token estimate and character counts for LLM context budgeting
- Warm standby endpoint for single page calls

### Where RAG Web Browser Scraper is stronger

- Proxy group selection and query lists in one input
- Captures language and HTTP status metadata per page

### Where this API is stronger

- Platform usage is included in the per page price
- JSON-LD entities and a token budget with each converted page
- Warm standby service for single page calls

### Where RAG Web Browser is stronger

- Optional link extraction and HTML output as add-ons
- Reading time, site name and image URL fields per page
- Search rank recorded on rows that came from a query

### Where this API is stronger

- JSON-LD entities and a token budget included in the page charge
- Warm standby service for single conversions
- Platform usage included in the page charge

The RAG browser lane is busy and the Apify listing dominates it, with search plus fetch and browser rendering as its strengths. The s-r, datapilot and parseforge actors keep to page conversion with different amounts of metadata. Ours adds JSON-LD extraction and token budgets, and runs as a warm service so single calls come back quickly.

Prices are Apify Store list prices, checked 20 Nov 2026. Platform usage 'Paid separately' means Apify bills compute on top of the event price. Check each listing before you buy.

## Input

- `mode` (string, optional): Choose 'auto' (detects Standby HTTP server when warm, batch otherwise), 'standby' (HTTP server for sub-second REST requests), or 'batch' (crawls startUrls).
- `startUrls` (array, optional): Web page URLs to fetch, convert to token-optimized Markdown, and extract Schema.org entities in batch mode.
- `maxPages` (integer, optional): Maximum number of pages to process during a batch run.
- `removeSelectors` (array, optional): Custom CSS selectors to strip from HTML before Markdown conversion (e.g. '.newsletter-popup', '#cookie-banner').
- `includeJsonLd` (boolean, optional): Extract structured Schema.org JSON-LD entities (Articles, Products, FAQs, Organizations) for semantic graph enrichment.
- `includeHtml` (boolean, optional): Include the stripped HTML alongside clean Markdown in the dataset output (increases storage usage).
- `proxyConfiguration` (object, optional): Apify Proxy settings for accessing geo-blocked or protected websites.

### Example input

```json
{
  "startUrls": [
    {
      "url": "https://news.ycombinator.com"
    }
  ]
}
```

## Output fields

- `url` (string): Source URL of the converted webpage
- `title` (string): Page title extracted from <title> or OpenGraph tags
- `status` (string): Row status: absent for normal conversions, 'error' for uncharged status/error rows
- `error` (string): Error explanation for uncharged status/error rows (e.g. blank or bot-challenge pages that yielded no usable content)
- `description` (string): Page description extracted from meta tags
- `author` (string): Author of the article or page if present
- `canonicalUrl` (string): Canonical URL extracted from link rel=canonical or OpenGraph tags
- `rawHtml` (string): Stripped HTML structure prior to Markdown conversion (present when includeHtml is true)
- `publishedTime` (string): Publication timestamp if available
- `markdown` (string): Clean, bloat-free, token-optimized Markdown ready for LLM context injection
- `jsonLd` (array): Extracted Schema.org JSON-LD structured entities
- `tokenEstimate` (integer): Estimated token budget consumed by the markdown content
- `charCount` (integer): Character count of the clean markdown content
- `wordCount` (integer): Word count of the clean markdown content
- `crawledAt` (string): ISO timestamp of extraction

## Sample output

Trimmed from one of our own test runs. Personal data is redacted.

```json
[
  {
    "url": "https://example.com",
    "title": "Example Domain",
    "canonicalUrl": "https://example.com",
    "markdown": "This domain is for use in documentation examples without needing permission. This is not a service; avoid relying on it for testing and monitoring purposes.",
    "charCount": 156,
    "wordCount": 25,
    "crawledAt": "2026-11-20T02:41:41.729Z"
  }
]
```

## How to run it

1. For a single page, POST to the standby /convert endpoint with a body such as {"url": "https://example.com/article", "includeJsonLd": true} and an Apify token in the Authorization header.
2. Read data.markdown, data.tokenEstimate and data.jsonLd from the response, which also reports executionTimeMs and chargedUsd.
3. To convert HTML you already have, send {"html": "<html>...</html>"} instead of a url for an instant parse.
4. For a list of pages, run in batch mode with startUrls, for example [{"url": "https://news.ycombinator.com"}], and set maxPages.
5. Add removeSelectors such as [".newsletter-banner", "#cookie-consent"] to strip boilerplate before conversion.
6. Read the dataset rows in the Apify Console and export as JSON, or keep calling the endpoint from your app.

### cURL

```bash
curl -X POST "https://api.apify.com/v2/acts/subimpact~standby-rag-api/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"startUrls":[{"url":"https://news.ycombinator.com"}]}'
```

### Python

```python
from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("subimpact/standby-rag-api").call(run_input={
      "startUrls": [
        {
          "url": "https://news.ycombinator.com"
        }
      ]
    })

for item in client.dataset(run["default_dataset_id"]).iterate_items():
    print(item)
```

### JavaScript

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_APIFY_TOKEN>' });
const run = await client.actor('subimpact/standby-rag-api').call({
  "startUrls": [
    {
      "url": "https://news.ycombinator.com"
    }
  ]
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();

console.log(items);
```

### n8n

Use the Apify node (Run an Actor).
Actor ID: subimpact/standby-rag-api
Input JSON: paste the example input below.

## Limits

- Batch mode does not follow links: only the URLs you list are fetched.
- Blank pages and bot-challenge pages come back as uncharged error rows rather than retried endlessly.
- A batch run fails with a status message when every requested URL fails or yields no usable content.
- maxPages caps a batch run at 50,000 pages.

## FAQ

### What does a conversion cost?

$0.001 per page, which is $1.00 per 1,000 pages, plus $0.00005 per run. You are charged only for a successful conversion, platform usage is included rather than billed separately, and there is no monthly commitment.

### What is the difference between standby and batch mode?

Standby runs the Actor as a continuous HTTP service behind a permanent URL, so POST /convert answers in well under a second from a warm container. Batch mode takes startUrls and writes converted rows into the dataset. mode set to auto picks standby when the service is warm and batch otherwise.

### Does the response include Schema.org data?

Yes, when includeJsonLd is on. jsonLd holds the extracted entities such as Article, Product, FAQ and Organization, alongside the Markdown, so structured data and text come from one call rather than two.

### Can I send HTML instead of a URL?

Yes. POST html rather than url and the conversion skips the network entirely, running in about 10 milliseconds. That is the cheap path when your own crawler has already fetched the page.

### Does batch mode follow links?

No. Batch mode fetches exactly the URLs you list in startUrls, up to maxPages, with a ceiling of 50,000. It does not discover links on a page, so the run covers your curated set and stays predictable on cost.

### How do I remove cookie banners and popups?

Pass extra CSS selectors in removeSelectors, for example .newsletter-popup and #cookie-banner. They are stripped from the HTML before the Markdown conversion, so the boilerplate never reaches your chunks and the token estimate reflects the cleaned text.

### Can I call it from Python or TypeScript?

Yes. Any HTTP client can POST to the /convert endpoint with an Apify token, and the response is JSON with a data object holding markdown, tokenEstimate and jsonLd. That works the same from a LangChain loader, a Node agent tool or a curl command.

## Links

- Page: https://subimpact.net/apis/standby-rag-api/
- Category: https://subimpact.net/apis/category/ai-developer/
- Run on Apify: https://apify.com/subimpact/standby-rag-api
- All data APIs: https://subimpact.net/apis/
- Catalogue JSON: https://subimpact.net/apis/catalogue.json
