> For the complete documentation index, see [llms.txt](https://cantonindexer.infradao.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://cantonindexer.infradao.com/server.md). # Tutorial: run the server The server is the `canton-indexer` node. It keeps every filter deployed to it up to date and writes the documents to its own Postgres database, the store. This part follows step 4 of the [Tutorial](/tutorial.md): you have `validator-licenses.toml` and it runs with `cinder run`. You will start a store and a node with Docker, deploy the filter, look at it with `cinder ls` and `cinder info`, read its documents with SQL, replace it, and read the node's metrics. {% hint style="info" %} The image `ghcr.io/infradao/canton-indexer-releases/canton-indexer` and the `cinder` commands `deploy`, `ls` and `info` are published from v0.10.0 on. {% endhint %} Before you start: * Docker. * `cinder` 0.10.0 or later ([Install](/install.md)). * A PQS URL the container can reach. `localhost` in `PQS_URL` is the container itself, not your machine. ## 1. Start a store The node needs a Postgres database of its own. For this tutorial, run one in a container on a network shared with the node: ```sh docker network create canton-indexer docker run -d --name store --network canton-indexer \ -e POSTGRES_USER=indexer -e POSTGRES_PASSWORD=secret -e POSTGRES_DB=indexer \ postgres:17 ``` `POSTGRES_USER` owns the database, so it needs no grants. On a Postgres server you already run, create the database and grant the node's user what [Install](/install.md#run-the-node) lists. ## 2. Start the node ```sh IMAGE='ghcr.io/infradao/canton-indexer-releases/canton-indexer:v0.10.0' # the release you want docker run -d --name canton-indexer --network canton-indexer \ -e DB_HOST=store -e DB_PASSWORD=secret \ -e PQS_URL='postgres://user:password@pqs-host:5432/pqs' \ -e NO_AUTH_GRPC_ADDR=0.0.0.0:8020 \ -p 127.0.0.1:8020:8020 -p 127.0.0.1:9464:9464 \ "$IMAGE" ``` * `DB_HOST=store` is the store's container name. `DB_USER` defaults to `indexer` and `DB_NAME` to the value of `DB_USER`, so both match the store above. * `PQS_URL` is the PQS every filter reads. * `NO_AUTH_GRPC_ADDR` is where the management service listens. Its default, `127.0.0.1:8020`, is the container's own loopback, which a published port cannot reach. In a container, set it to `0.0.0.0:8020`. * `-p 127.0.0.1:…` publishes both ports on your machine's loopback only. **The management service has no authentication.** Anyone who reaches port 8020 can deploy a filter, or replace one, which drops its documents. Keep it on localhost or a private network. The variable's name says so on purpose. `/metrics` on 9464 has no authentication either. The node runs its migrations at boot, before anything else: it creates the schema `indexer` in the store, or brings it up to date. Then it connects to PQS, binds both ports and starts every `active` filter. Watch the log: ```sh docker logs -f canton-indexer ``` It logs `indexer schema is current`, `connecting to PQS`, then `node started, serving gRPC and metrics`. If the store or PQS cannot be reached, the node exits with code 1. A store that a newer node has migrated has a migration this binary does not know, and the node refuses to boot on it. To roll back to an older image anyway, set `IGNORE_NEWER_MIGRATIONS=true`: the node skips that migration and logs a warning. Every other setting, with its default, is in [Install](/install.md#run-the-node). ## 3. Point cinder at it Add the node to `cinder.toml`: ```toml pqs_url = "postgres://user:password@pqs-host:5432/pqs" node = "http://127.0.0.1:8020" ``` `http://` is plain gRPC. `cinder` reads `node` only from this file. ## 4. Deploy, list, inspect ```sh cinder deploy validator-licenses.toml ``` ``` deployed validator_licenses as filter 1 (target1_v1, version 1) ``` The node gave the filter id `1` and the schema `target1_v1`, and starts indexing from the filter's `start_offset`, or from the oldest offset PQS holds. `version` counts the deploys of that name, not the `version` key in the file. A name can be deployed once; a second plain deploy is refused. ```sh cinder ls ``` ``` NAME ID VERSION SOURCE STATUS RUNNING FAILURES DOCS OFFSET validator_licenses 1 1 lifecycle active yes 0 42 113362 ``` One row per filter. `RUNNING` says whether a task runs now at this version. `FAILURES` counts transient failures in a row. `DOCS` is Postgres's live-row estimate for the current table, an estimate that lags writes; 0 before its statistics report any. `OFFSET` is the last committed offset, `-` until the first commit. Add `--json` for one JSON object per filter. ```sh cinder info validator_licenses ``` `info` shows the same fields for one filter, plus `schema_version`, `target_schema`, `failure`, `changed_at` and the filter's query indexes. With `--json` it also prints the manifest the node holds. ## 5. Read the documents Connect to the store with any Postgres client: ```sh docker exec -it store psql -U indexer -d indexer ``` A filter's documents are in `target_v.documents`, here `target1_v1.documents`. A `lifecycle` filter has one row per contract, with the columns `contract_id`, `template_fqn`, `archived_at_offset` and `document` (the JSON line as `jsonb`). The archive of a contract overwrites its row, so live contracts are the rows where `archived_at_offset` is null: ```sql select document->>'validator' as validator, document->>'last_active_at' as last_active_at from target1_v1.documents where archived_at_offset is null; ``` An `exercises` filter has one row per exercise, keyed on `(event_offset, event_node_id)`, with `contract_id` and `document`. `->>` gives text, so cast numbers before you sort or compare them: `(document->>'_offset')::numeric`. The SQL in each example's header comment reads one column per field; in the store, write `document->>'field'`. To sort, order by the cast field. `_offset` is the offset of the row's last event, its create while the contract is live, so this lists the newest live contracts first. One transaction's rows share an offset, so `_node_id` and `contract_id` break the ties, all three in the same direction: ```sql select document->>'validator' as validator, (document->>'_offset')::numeric as at_offset from target1_v1.documents where archived_at_offset is null order by (document->>'_offset')::numeric desc, (document->>'_node_id')::numeric desc, contract_id desc limit 10; ``` An `exercises` table sorts by `event_offset, event_node_id`. To check that a sort uses an index, see [Troubleshooting: A sort over the documents is slow](/troubleshooting.md#a-sort-over-the-documents-is-slow). The schema name changes with every replace. Read the current one from `cinder info` (`target_schema`) or from the store: ```sql select name, version, target_schema from indexer.filters; ``` ## 6. Replace a filter Edit the filter, then deploy it again with `--replace`: ```sh cinder deploy --replace validator-licenses.toml ``` ``` replaced validator_licenses as filter 1 (target1_v2, version 2) ``` What happens, in order: 1. In one transaction the node keeps the id, bumps the version to 2, creates the new, empty schema `target1_v2`, resets the cursor and sets the status `active`. 2. It stops the version 1 task, waiting for the read or commit in progress, and starts the version 2 task, which backfills `target1_v2` from `start_offset`, or from the oldest offset PQS holds. The runtime sets no deadline on that read or commit; `SHUTDOWN_GRACE_SECS` bounds a shutdown, not a replace. 3. It drops `target1_v1`, then answers `cinder`. So the old documents are gone when `replaced` prints, and `target1_v2` holds only what the new task has indexed so far. `OFFSET` and `DOCS` in `cinder ls` show it catching up. A query still on `target1_v1` fails. **Why the old documents are dropped.** Two reasons: * The old documents were written by the old `where` and `[select]`. A changed filter keeps other records or writes other columns, so those rows cannot stand for the new version (the developers, 2026-10-28). * Each version gets its own schema, so a reader sees one version whole: the node never truncates or rebuilds a table in place, and readers do not queue behind a lock on it. **To keep the old documents** while the changed filter backfills, deploy it under a new `name` instead, for example `validator_licenses_v2`. That is a plain deploy: the filter gets its own id and schema, `target_v1`, and both filters run and stay queryable. The old filter keeps indexing, since in v0.10.0 the node cannot pause or remove a filter. Unreleased code adds `cinder pause`, `resume` and `delete` (\[future release]). The drop waits at most 10 s for queries that read `target1_v1`. If it cannot finish, the node logs `dropping a version schema, left for the next sweep` and drops it at the next replace of that filter or the next boot. Unreleased code (\[future release]) has no boot sweep: the log reads `dropping a version schema, left for the next replace`, and the schema is dropped at the filter's next replace. A replace is also how a `failed` filter runs again. What PQS has pruned cannot be indexed again; see [Troubleshooting](/troubleshooting.md#a-deployed-filter-stopped-status-failed) before you replace a filter that failed on an offset. ## 7. Read the metrics ```sh curl -s http://127.0.0.1:9464/metrics | grep '^indexer_' ``` The node serves Prometheus metrics at `/metrics` on `METRICS_ADDR` (default `0.0.0.0:9464`) from the end of boot until the first `SIGTERM` or `SIGINT`, or until the filter runtime ends. By `filter_id` and `version`: `indexer_filter_running` (1 while the task runs), `indexer_filter_last_offset`, `indexer_filter_commits_total`, `indexer_filter_documents_total` and `indexer_filter_skipped_total` (records skipped under `on_error = "skip"`). By `filter_id`: `indexer_filter_failures` (in a row), `indexer_filter_respawns_total` and the histogram `indexer_filter_source_wait_seconds`. For the node: `indexer_tracker_restarts_total`, and `indexer_filters` by `status`. Counters start at zero when the node starts. ## 8. Stop the node ```sh docker stop -t 90 canton-indexer ``` On `SIGTERM` the node stops accepting gRPC connections, waits for open ones to close, then gives filter tasks `SHUTDOWN_GRACE_SECS` (default 60) to stop. Docker's default stop timeout is 10 s, hence `-t 90`. A kill past the grace loses nothing committed: the batch in progress is redone from the cursor at the next boot. Every filter stays `active` and resumes then. ## What the server does not do yet * No health endpoint. `/metrics` is the only HTTP route. * No authentication on the management service. * No pause or remove of a filter; only deploy and replace. * One node per store. Two nodes on one store are not supported. ## Sources The behaviour above is that of canton-indexer v0.10.0, released 2026-10-22 (): * The image and `cinder` `deploy`, `ls` and `info`, published from v0.10.0 on: the v0.10.0 release. * Settings and defaults: `crates/indexer/src/main.rs`; boot order and when metrics are served: `crates/indexer/src/node.rs`, README "Node". * Metrics: `crates/indexer/src/metrics.rs`. * `DOCS`: `crates/indexer-api/proto/canton/indexer/v1/management.proto`. * Replace waits with no deadline: `docs/runtime.md`. * Version schemas, replace and drop: `docs/pipeline.md`, `docs/runtime.md`, decision 0019. Why a changed filter's old rows cannot stand for the new version: the developers, 2026-10-28. * `documents` columns and the `_offset` field: the canton-indexer v0.10.0 source, and decision 0017. * A new `name` is a new filter, a taken one is refused without `--replace`: the unique `name` of `indexer.filters`. * No pause or remove in v0.10.0: decision 0009 (pause and resume) and decision 0010 (delete), designed in PR #278; built after v0.10.0 (#429), unreleased (\[future release]). * No boot sweep after v0.10.0: canton-indexer main, unreleased (\[future release]). * Loopback default and no auth: decision 0007, auth tracked as #281; one node per store: `rules.toml` `[deferrals]`. * `cinder ls` and `info` output: `crates/cinder/src/ls.rs`, `info.rs`. * Docker stop timeout: