Logging, metrics & rate limits

Vaultbase writes every request to a daily JSONL file and runs configurable rate limits before any handler executes. Both surface in the admin UI under Logs and Settings → Rate limits. A Prometheus exposition endpoint is also available — see Metrics below.

Request logs

One JSONL file per UTC day at <dataDir>/logs/YYYY-MM-DD.jsonl. Append-only. Files older than the data retention period (Settings → Data retention, maintenance.retention_days, default 30, 0 = keep forever) are deleted once a day. A path longer than 2 KB and a hook message longer than 8 KB are truncated in the log line.

A log write that fails (disk full, permissions, a missing folder) never fails the request. It is counted (vaultbase_log_write_failures_total and vaultbase_log_lines_lost_total in Metrics), reported on stderr at most once a minute, and marks /_/ready as degraded for five minutes.

What’s in each entry

{
  "id": "0811abc1-b2af-4951-900c-3d2966d25957",
  "ts": "2026-04-27T10:15:23.000Z",
  "created_at": 1777284923,
  "method": "GET",
  "path": "/api/v1/posts",
  "status": 200,
  "duration_ms": 4,
  "ip": "203.0.113.7",
  "auth_id": "u1",
  "auth_type": "user",
  "auth_email": "alice@x.com",
  "auth_impersonated_by": null,
  "rules": [
    {
      "rule": "list_rule",
      "collection": "posts",
      "expression": "@request.auth.id != \"\"",
      "outcome": "allow",
      "reason": "rule passed"
    }
  ]
}

• auth_impersonated_by is always present: the admin id when the JWT carries an impersonated_by claim, otherwise null.
• Requests made with an API token are logged as auth_id: "api:<token id>", auth_type: "admin".
• rules is present only when rules ran (records API).

rules[] is the per-rule audit trail — invaluable for debugging “why did this 403?”. For each rule that ran, you get:

• rule — list_rule / view_rule / create_rule / update_rule / delete_rule
• collection — collection name
• expression — the rule expression text as stored (null for a public rule, "" for admin only)
• outcome — allow, deny, or filter (list rule applied as SQL filter)
• reason — public, admin only, admin bypass, rule passed, rule failed, applied as SQL filter

Impersonation audit

When an admin uses POST /api/v1/admin/impersonate/:col/:userId, the minted user JWT carries an impersonated_by claim. Every request made with that token gets auth_impersonated_by: <admin_id> on its log entry.

Find every impersonated request via JSONPath in the Logs page or the admin API:

$[?(@.auth_impersonated_by)]

Or jq the file directly:

jq -c 'select(.auth_impersonated_by)' data/logs/2026-04-27.jsonl

Browsing logs in the admin

Logs page in the sidebar:

• Date picker — pick any past day, server reads the matching JSONL file.

• Search — JSONPath expression evaluated against each entry. Examples:

  • $.path — show every entry’s path
  • $.status — every status code
  • $.rules[*].outcome — every rule outcome
  • $.auth.email — emails of authenticated callers

• Filter — narrow by method, status range, path prefix, auth type.

• Rule outcome dropdown — narrow to entries whose rules[] evaluations match a given outcome:

  • All — no filter (default).
  • Any rule eval — only entries that ran any rule (records-API requests).
  • Allow — at least one rule passed.
  • Deny — at least one rule denied (403 responses live here).
  • Filter — list_rule was applied as a SQL filter.

  Backend equivalent: GET /api/v1/admin/logs?ruleOutcome=deny.

Since each entry is a self-contained JSON object, you can also jq the file directly outside the admin:

jq -r 'select(.status >= 400) | "\(.status) \(.method) \(.path)"' \
  data/logs/2026-04-27.jsonl

Programmatic access

GET /api/v1/admin/logs?page=1&perPage=50&status=4xx&ruleOutcome=deny    ← admin auth
   → { "data": [ {entry}, ... ], "page": 1, "perPage": 50, "totalItems": 42, "totalPages": 1 }

Param	Notes
page	Default 1.
perPage	Default 50, max 200.
method	GET, POST, … or all (default).
status	2xx, 4xx, 5xx or all (default).
search	Substring match on path and hook name / event / collection / message.
minDuration	Only entries at least this many ms.
ruleOutcome	all, any, allow, deny, filter.
includeAdmin	true to include /api/v1/admin/* requests (hidden by default).

JSONPath search is a separate endpoint:

POST /api/v1/admin/logs/search    ← admin auth
{ "jsonpath": "$[?(@.status >= 400)]", "from": "2026-04-27", "to": "2026-04-27", "limit": 500 }
   → { "data": { "matched": 18, "scanned": 120, "results": [ { "entry": {...}, "matches": [...] } ] } }

from / to are UTC dates (YYYY-MM-DD); limit caps the results returned (at most 5000, newest first), not the entries searched: matched counts every match, and scanned is the entries searched (up to the newest 100,000). The expression runs against each entry: a filter at the root ($[?(@.status >= 400)]) selects whole entries, and a plain path ($.status) reads a value from each entry. An entry matches when the expression returns anything — so $.auth_impersonated_by matches every entry (the field is always present, often null); use a filter instead.

Hook logs

ctx.helpers.log(...) writes a HOOK row into the same logs file. So afterCreate hooks, cron jobs, and custom routes all surface alongside HTTP traffic. The arguments are joined into one message string (cut at 8 KB).

ctx.helpers.log("Sending welcome email to", ctx.record.email);

{ "id": "...", "ts": "2026-04-27T10:15:23.000Z", "created_at": 1777284923,
  "method": "HOOK", "path": "welcome:users:afterCreate", "status": 200,
  "duration_ms": 0, "ip": null, "kind": "hook",
  "message": "Sending welcome email to alice@x.com",
  "hook_name": "welcome", "hook_collection": "users", "hook_event": "afterCreate" }

Rate limits

Token-bucket per IP, configurable per route. Master switch + ruleset both live in vaultbase_settings (see Settings keys).

Default rules

[
  { "label": "*:auth",   "max": 10,  "windowMs": 3000,  "audience": "all" },
  { "label": "*:create", "max": 60,  "windowMs": 5000,  "audience": "all" },
  { "label": "/api/*",   "max": 300, "windowMs": 10000, "audience": "all" }
]

Rules evaluate top-to-bottom; the first matching rule applies. Order specific → general (auth-only routes before catch-alls).

Label syntax

<path>[:<action>] where:

Component	Values
path	exact (/api/v1/posts), prefix (/api/*), or wildcard (*). /api/v1/... and /api/... are equivalent.
action (optional)	auth, create, list, view, update, delete
audience	all, guest (no token), auth (any token)

Examples:

[
  { "label": "/api/v1/auth/*:auth",     "max": 5,   "windowMs": 60000, "audience": "all" },
  { "label": "/api/v1/auth/*:create",   "max": 5,   "windowMs": 60000, "audience": "guest" },
  { "label": "/api/v1/posts:create",    "max": 30,  "windowMs": 60000, "audience": "auth" },
  { "label": "/api/v1/posts",           "max": 600, "windowMs": 60000, "audience": "all" },
  { "label": "*",                    "max": 1000,"windowMs": 60000, "audience": "all" }
]

When a limit triggers

Response is 429 Too Many Requests with a Retry-After header (seconds).

{ "error": "Rate limit exceeded (rule: /api/v1/posts)", "code": 429 }

Retry-After is the matched rule’s window, rounded up to whole seconds.

The bucket is keyed on the client IP (read from X-Forwarded-For only when the request comes through a trusted proxy — configure your reverse proxy to send it).

Disabling rate limits

PATCH /api/v1/admin/settings  { "rate_limit.enabled": "0" }

Useful for development or when you’ve put a dedicated rate-limit layer in front (a WAF or your reverse proxy’s rate limiting).

Tuning

• Make the first rule the most specific (e.g. /api/v1/auth/*:auth).
• Keep a generous catch-all * for general API health.
• For public endpoints with no auth, scope by audience: "guest" so signed-in users aren’t punished for guest spam.
• Token-bucket is per-process. Under vaultbase cluster each worker keeps its own buckets, and separate hosts share nothing. Put a dedicated rate-limit layer in front if you need limits across processes or nodes.

Metrics

Prometheus exposition at /api/v1/metrics. Off by default; enable in Settings → Health & metrics.

# With the scrape token from Settings → Health & metrics
curl -s -H "Authorization: Bearer $SCRAPE_TOKEN" http://localhost:8091/api/v1/metrics

# No scrape token set: an admin token is required (never anonymous)
curl -s -H "Authorization: Bearer $ADMIN_TOKEN" http://localhost:8091/api/v1/metrics

Format: text/plain; version=0.0.4 — the standard Prometheus exposition.

Metric	Type	Notes
vaultbase_uptime_seconds	gauge	Process uptime.
vaultbase_requests_total	counter	Finished requests since boot.
vaultbase_rps_1min	gauge	RPS over the last 60 s.
vaultbase_step_duration_microseconds	summary	Per-step request latency (p50/p90/p99/p99.9).
vaultbase_step_duration_microseconds_count	counter	Sample count per step.
vaultbase_step_duration_microseconds_sum	counter	Sum of values per step (for rate() math).
vaultbase_sqlite_page_count / page_size / wal_pages	gauge	Database size + WAL pressure.
vaultbase_responses_total{class}	counter	Finished requests by status class (2xx, 3xx, 4xx, 5xx).
vaultbase_jobs{status}	gauge	Queue jobs that are queued, running or dead.
vaultbase_jobs_oldest_due_seconds	gauge	How long the oldest due queued job has waited (0 when none).
vaultbase_webhook_deliveries{status}	gauge	Webhook deliveries that are pending or dead.
vaultbase_read_workers{state}	gauge	Read worker threads running and configured.
vaultbase_read_workers_dropped_total	counter	Read workers taken out of service since boot.
vaultbase_read_workers_restarted_total	counter	Read workers started to replace dropped ones.
vaultbase_log_write_failures_total	counter	Log writes that failed (disk full, permissions).
vaultbase_log_lines_lost_total	counter	Log lines lost to those failures.
vaultbase_database_bytes{driver}	gauge	Database size on disk.
vaultbase_database_connections{state}	gauge	Server connections to the database (all, active), when the driver has them.
vaultbase_realtime_connections	gauge	Open realtime connections (WebSocket and SSE).
vaultbase_realtime_queued_deliveries	gauge	Realtime deliveries waiting to be sent.
vaultbase_realtime_shed_connections_total	counter	Realtime connections dropped because delivery fell too far behind.

Steps cover the records hot path: route_match, auth_verify, collection_load, rule_compile, db_exec, record_write, row_decode, serialize, compress, log_write. Reads record collection_load and db_exec; creates, updates and deletes also record rule_compile and record_write (the write itself, including validation and hooks).

JSON snapshot and the Monitoring page

Admins can read a JSON snapshot at /api/v1/_/metrics (always on, admin auth required). Besides the per-step histograms it returns:

Field	Notes
total, reads, writes	End-to-end latency histograms; reads are GET/HEAD/OPTIONS, writes everything else.
status, methods	Counters by status class (2xx…5xx) and by HTTP method.
series	The last 120 seconds, one entry per second: requests, writes, 5xx, mean and max latency.
process	RSS and heap, CPU % since the previous read, worst event-loop lag since the previous read, realtime connections, pid and worker id.
sqlite	Page count and size, database and WAL file sizes, WAL frames pending, and checkpoint stats from the background checkpointer.

POST /api/v1/_/metrics/reset clears the histograms, counters and series.

The admin Monitoring page (under Data) charts this snapshot live, refreshing every 1, 2 or 5 seconds. All numbers are per process: under vaultbase cluster, each poll shows whichever worker answered.

Update checker

Vaultbase polls GitHub on boot (after a 30 s delay) and every 6 hours for the latest release tag. The Updates tab in Settings shows current vs latest with an Update available banner when behind. Disable the poller to silence the banner — vaultbase never auto-updates.