API reference¶

Dead Simple Search exposes a JSON REST API. It listens on port 5555 by default, configurable with FLASK_PORT.

Authentication¶

Endpoints that change state require an API key, sent in the X-API-Key header:

curl -X POST http://localhost:5555/api/sites \
  -H "X-API-Key: your-key-here" \
  -H "Content-Type: application/json" \
  -d '{"domain": "example.com", "start_url": "https://example.com/"}'

Set the key with the API_KEY environment variable. Generate one with:

python -c "import secrets; print(secrets.token_urlsafe(32))"

Read endpoints — including search — are never authenticated. If your index is not meant to be public, restrict access at the reverse proxy or firewall.

Rate limits¶

Requests are rate limited per client IP. Exceeding a limit returns 429 Too Many Requests.

Counters live in memory by default, so they reset on restart and are not shared between processes.

Service info¶

GET /

A liveness check that answers without touching the database — useful for uptime monitoring.

{
  "service": "Dead Simple Search",
  "api": "/api/sites",
  "source": "https://codeberg.org/marcusosterberg/Dead-Simple-Search"
}

Sites¶

A site is a website you want to crawl and search, registered by domain and starting URL.

Register a new site¶

POST /api/sites

Request body:

{
  "domain": "example.com",
  "start_url": "https://example.com/"
}

Response (201 Created):

{
  "id": 1,
  "domain": "example.com",
  "start_url": "https://example.com/"
}

A domain that is already registered returns 409 Conflict.

List all sites¶

GET /api/sites

Returns an array of all registered sites.

Get site details¶

GET /api/sites/{id}

Returns the site's details, including a page_count of how many pages are currently indexed.

Delete a site¶

DELETE /api/sites/{id}

Removes the site along with its indexed pages and crawl history, via a cascading foreign key. Returns 204 No Content.

Crawling¶

Trigger a crawl¶

POST /api/sites/{id}/crawl

Starts a crawl in the background and responds immediately with 202 Accepted rather than waiting for it to finish.

{
  "message": "Crawl started.",
  "site_id": 1
}

Limited to 2 per hour by default. Two further conditions can refuse a crawl:

• 409 Conflict — a crawl is already running for this site.
• 429 Too Many Requests — CRAWL_MAX_CONCURRENT crawls are already running across all sites (3 by default).

To re-crawl every registered site in sequence, use the recrawl_all.py helper in the repository, which waits for each crawl to finish before starting the next.

Check crawl status¶

GET /api/sites/{id}/crawl/status

Returns the most recent crawl log entries.

[
  {
    "id": 1,
    "started_at": "2026-02-16T10:30:00",
    "finished_at": "2026-02-16T10:35:42",
    "pages_crawled": 127,
    "pages_failed": 3,
    "status": "completed"
  }
]

status is one of running, completed or failed.

Search¶

Search within a site¶

GET /api/sites/{id}/search?q=your+query

Response:

{
  "site_id": 1,
  "query": "python tutorial",
  "language_filter": null,
  "total": 42,
  "limit": 20,
  "offset": 0,
  "results": [
    {
      "page_id": 123,
      "url": "https://example.com/python-intro",
      "title": "Python Introduction",
      "meta_description": "Learn Python basics...",
      "h1": "Getting Started with Python",
      "language": "en",
      "page_published": "2019-03-04T00:00:00",
      "page_modified": "2025-12-01T10:00:00",
      "page_modified_source": "metadata",
      "page_last_modified": "2025-12-01T10:00:00",
      "relevance": 12.45,
      "snippet": "Python is a versatile programming language..."
    }
  ]
}

Page dates¶

Four date fields are returned per result, each null when the page provides nothing usable.

page_published comes only from the page's own markup — Schema.org JSON-LD, Microdata, Open Graph or Dublin Core. Nothing outside a page can know when it was created.

page_modified takes the first of three sources that speaks: the page's own dateModified, then the sitemap's <lastmod> for that URL, then the HTTP Last-Modified header. page_modified_source tells you which one was used, so a client that only trusts editorially declared dates can filter on it.

The sources are ranked rather than merged, and the newest value does not win — sitemaps and HTTP headers both skew toward the present, so preferring the most recent date would let the least reliable source decide almost every page. Values that look wrong are reported as null rather than passed on: dates before 1990 or more than a day ahead, sitemaps where a single <lastmod> covers most URLs, and a Last-Modified matching the response's own Date header.

Relevance and ranking¶

relevance comes from MySQL's full-text scoring; higher is better, and results are always sorted by it. Pages whose title, meta description or H1 match the query have their score multiplied by SEARCH_TITLE_BOOST (10 by default), because MySQL cannot weight individual fields within a single full-text index.

Search modes¶

The mode is chosen automatically from the query.

A query containing +, -, *, ", ~, <, > or parentheses is passed to MySQL's boolean mode verbatim:

Otherwise, with SEARCH_STEMMING on (the default), each term is expanded to its stemmed prefix and all terms are required. If that matches nothing, the same stemmed terms are retried as optional so partial matches still surface. Failing that, the query runs in plain natural-language mode.

Stemming covers Swedish, Danish, Norwegian, Finnish, Icelandic and English, so sökningar also finds sökning.

Error responses¶

Errors return JSON:

{
  "error": "Not Found",
  "message": "Site not found."
}