SapixDBSapixDB/Docs
Home

Indexes

Full-Text Search

SapixDB supports full-text search on any string field through an FTS index. Once the index is built, queries with the fts operator find every record whose field contains all of the query words — in sub-millisecond time regardless of strand size.

Creating an FTS Index

FTS indexes use the same endpoint as other index types. Add "type": "fts" to the request body and specify the field you want to search. In this example the articles agent indexes its content field.

HTTP
POST /v1/agents/articles/indexes
Authorization: Bearer spx_root_YOUR_ROOT_KEY
Content-Type: application/json

{
  "name":  "idx_fts_content",
  "field": "content",
  "type":  "fts"
}
Response (202 Accepted)
{
  "name":   "idx_fts_content",
  "field":  "content",
  "type":   "fts",
  "status": "building"
}

The build is asynchronous. Check GET /v1/agents/articles/indexes until status shows ready before relying on it in production queries.

The fts Filter Operator

Use op: "fts" in any filter to trigger a full-text search against an FTS-indexed field. The value is a plain string containing one or more search words separated by whitespace.

FTS query
POST /v1/agents/articles/query
Authorization: Bearer spx_root_YOUR_ROOT_KEY
Content-Type: application/json

{
  "type":  "scan",
  "limit": 10,
  "filter": {
    "field": "content",
    "op":    "fts",
    "value": "cryptographic signing"
  }
}

This returns every article whose content field contains both the word cryptographic and the word signing. The search is case-insensitive and strips punctuation — see How the Tokenizer Works below.

ℹ AND semantics — all words must matchWhen the query value contains multiple words, a record must contain every word to be returned. There is no OR mode. If you want to allow any word to match, run separate queries and merge the results in your application.

How the Tokenizer Works

When a record is written (or the index is built over existing records), SapixDB runs the field value through a tokenizer before storing it in the inverted index. The same tokenizer runs on query values at query time, so matching is always consistent.

StepWhat happensExample
Split on non-alphanumericThe input is split on any character that is not [a-zA-Z0-9]. Punctuation, spaces, hyphens, underscores, and slashes all act as delimiters."HIPAA-compliant" → ["HIPAA", "compliant"]
LowercaseEvery token is lowercased. Searches are always case-insensitive.["HIPAA", "compliant"] → ["hipaa", "compliant"]
Drop short tokensTokens shorter than 2 characters are discarded. Single-letter words and stray punctuation fragments are removed.["a", "hi"] → ["hi"] (the "a" is dropped)
✓ Case-insensitive by designQueries for HIPAA, hipaa, and Hipaa all return the same results because both the indexed tokens and the query tokens are lowercased before comparison.

Inverted Index Internals

Internally, SapixDB stores the FTS index as an inverted posting list in the agent graph using meta keys of the form _fts:<index_name>:<word>. Each key maps to the list of content hashes for records that contain that word.

At query time the engine looks up the posting list for each query word and intersects the lists. Only content hashes present in every posting list are returned — the AND semantics come directly from the set intersection.

Combining FTS with Structured Filters

FTS filters compose naturally with other filter types inside an AND block. This lets you combine keyword search with structured equality or range filters in a single query.

FTS + structured filter
POST /v1/agents/articles/query
Authorization: Bearer spx_root_YOUR_ROOT_KEY
Content-Type: application/json

{
  "type":  "scan",
  "limit": 10,
  "filter": {
    "AND": [
      { "field": "content", "op": "fts", "value": "HIPAA" },
      { "field": "year",    "op": "eq",  "value": 2026    }
    ]
  }
}

This returns articles from 2026 that mention HIPAA. The engine resolves the FTS filter using the inverted index and intersects those results with the equality filter on year. If a single-field or composite index exists on year, that index is used for the structured half of the filter.

ℹ Put the most selective filter firstIn compound filters, order the most selective clause first in the AND array. If the FTS term is rare (few results), put the fts filter first. If the structured filter is more selective, put it first. The engine intersects left-to-right and short-circuits as early as possible.

Deleting an FTS Index

Delete an FTS index the same way as any other index. All posting list meta keys are removed from the agent graph immediately.

HTTP
DELETE /v1/agents/articles/indexes/idx_fts_content
Authorization: Bearer spx_root_YOUR_ROOT_KEY
Response (200 OK)
{
  "deleted": "idx_fts_content"
}

Limitations

LimitationDetail
One FTS index per fieldYou cannot have two FTS indexes on the same field. Delete the existing one before recreating it with a different name.
No phrase searchThe tokenizer does not preserve word position. "data breach" matches records containing both words anywhere in the field, not necessarily adjacent.
No stemmingTokens are not stemmed. A query for sign will not match records containing only signing or signed.
String fields onlyFTS indexes only make sense on string-valued fields. Creating an FTS index on a numeric or boolean field will index the string representation.
ℹ Related pagesFor single-field equality indexes, see Indexes. To verify that the FTS index is being used by your query, use Query Explain.