Security and operations
For an administrator, a DBA or a managed-PostgreSQL provider deciding
whether pg_warren is safe to install. This page collects what the rest of
the documentation and the code say about privileges, failures, resources,
testing and releases, and links to the details. It describes main; where
something is planned rather than built, it says so.
In short
Section titled “In short”- It is a compiled library with a fixed SQL surface. It creates one
schema,
warren, and runs no code a user supplies other than SQL that the user, or a graph’s owner, could run anyway. - No background workers, no network, no files of its own. It uses Postgres’s dynamic shared memory for a cache shared across backends, and threads inside a backend for computation. Only the backend’s main thread calls into Postgres.
- Non-superusers can install it (
trusted = true). The installer gets no privilege on pg_warren’s catalog tables. Every role, the installer included, reaches graphs only throughwarren.*functions, which check access on every call. - No function is
SECURITY DEFINER. pg_warren’s own catalog statements run as the catalog’s owner inside a narrow identity switch, under a pinnedsearch_path. User-written SQL never runs elevated. - Errors are SQL errors. Panics, cancellation, timeouts, memory limits
and allocation failures of large buffers end the statement with an
ERROR, not the backend. - Writes are refused on standbys with SQLSTATE 25006, and every read works there.
- Dumps without a superuser use Postgres’s predefined roles:
pg_read_all_datato dump graphs,pg_write_all_datato restore them. On Amazon RDS (PG 15 and later) the master user has both. A graph a superuser owns runs its source queries only for a superuser, so the catalog writespg_write_all_dataallows lead to no superuser’s rights. - Known gaps are listed at the end: per-session resource settings that any role can raise, and supply-chain items (reproducible builds, an SBOM, dependency audits, a security policy).
What it installs
Section titled “What it installs”Database objects
Section titled “Database objects”CREATE EXTENSION pg_warren runs one script that creates, all in schema
warren (the extension is not relocatable):
- Catalog tables:
graphs,versions,segments,vectors,vector_chunksandvector_quotas, their sequences and indexes, and three trigger functions with their triggers, which keepgraphs.current_versionandversions.label_vectorconsistent (DESIGN.md §5.1). Graph data is stored in these ordinary tables, so it is WAL-logged, backed up, replicated and transactional like any other data. - Functions,
\df warren.*lists them: graph management, traversal, compute, introspection. Most are C functions in the shared library. The library of algorithms (hits,louvain,label_propagation, …) is PL/pgSQL, and each such function setssearch_path = pg_catalog, pg_temp. - A composite type
warren.stepand a cast fromjsonbtowarren.step[], used bywarren.walk. Casts are not schema objects; this one converts only to pg_warren’s own type.
The script also runs REVOKE ALL … FROM PUBLIC on every warren.* table
and sequence, revokes EXECUTE on the trigger functions, grants USAGE on
the schema to PUBLIC, and registers every table and sequence with
pg_extension_config_dump so that pg_dump includes graph data
(crates/pg_warren/src/catalog/bootstrap.rs).
It does not create an event trigger, a role, a background worker or
anything outside schema warren other than the cast. The optional
DROP OWNED event trigger is something a superuser may create by hand
(semantics.md, “Access”).
Settings
Section titled “Settings”Six GUCs, registered when the library loads (_PG_init in
crates/pg_warren/src/lib.rs). Sizes are in kB.
| Setting | Context | Default | Range | Bounds |
|---|---|---|---|---|
warren.shared_cache_memory |
sighup |
4GB |
0 (off) to 2³¹−1 kB |
the cluster’s shared cache, all databases together |
warren.max_cache_memory |
user |
4GB |
64kB up |
what one backend’s private cache holds between statements |
warren.max_compute_memory |
user |
4GB |
64kB up |
what one warren.* call holds |
warren.max_build_memory |
user |
4GB |
1MB up |
what one refresh or contract holds |
warren.threads |
user |
half the cores, at least 1 | 1–1024 | compute threads per backend |
warren.max_walk_paths |
user |
10000000 |
1 up | rows a path-per-row walk may be asked for |
sighup settings are changed in the server’s configuration and take effect
on reload. user settings can be changed by any role, per session or
transaction (see Resource use and limits).
What each bounds is in limits.md, “Settings”.
Shared memory, hooks and callbacks
Section titled “Shared memory, hooks and callbacks”shared_preload_librariesis optional. pg_warren works without it on every supported major (15–18). On PG 15 and 16 the preload is what enables the shared cache and letswarren.backends()see other backends; without it each backend keeps its own copy of the graphs it reads. On PG 17 and 18 nothing needs configuring.- When preloaded on PG 15 and 16 it requests a small, fixed amount of
main shared memory (
shmem_request_hook,shmem_startup_hook): the shared cache’s registry (128 slots, a few tens of kB) and an activity table with one slot per backend (MaxBackends), and two named LWLock tranches. On PG 17 and 18 the same two structures are named DSM segments (GetNamedDSMSegment), created on first use. - Loaded graph versions are kept in one dynamic shared memory segment
each, bounded in total by
warren.shared_cache_memory. Withdynamic_shared_memory_type = posixon Linux these are files in/dev/shm, created by Postgres. Before creating one, pg_warren checks the free space there (statvfs) and keeps a quarter of it free, falling back to a private copy rather than fail (DESIGN.md §5.6, “Size checks”;crates/pg_warren/src/shared_memory.rs). A crash restart destroys every segment. - Hooks and callbacks: a
ProcessUtility_hook, installed when the library loads, that frees pg_warren’s session state afterDISCARD ALL(crates/pg_warren/src/discard.rs); transaction and subtransaction callbacks andbefore_shmem_exitcallbacks that give back shared-cache slots, activity slots and stop compute threads at exit; and memory-context reset callbacks that end per-statement pins. Each chains to any previous hook. - No background workers. None is registered, static or dynamic. A compute background worker is a future direction (DESIGN.md §6.9), not built.
Files, network and threads
Section titled “Files, network and threads”- Files. pg_warren writes no files. Everything it persists is rows in
its catalog tables. It reads file-system statistics (
statvfs) of/dev/shmorpg_dynshmem, and Postgres itself creates the DSM segments. Messages go to the client and server log through Postgres’s own error reporting (NOTICE,WARNING,DEBUG1). - Network. None. The library opens no sockets; its Rust dependencies
are
pgrx,libc,serde_json,thiserror,rayonandnalgebra(crates/*/Cargo.toml), none used for I/O. - Threads. Each backend that runs a compute kernel builds one thread
pool of
warren.threadsthreads (rayon), on first use. Only the main thread calls Postgres (DESIGN.md §6.9): the kernels live inwarren-core, a crate with no Postgres dependency (CI fails ifcargo tree -p warren-corementions pgrx or pg-sys), and take plain slices of memory pg_warren owns. Pool threads start with every signal blocked, so Postgres’s signal handlers run only on the main thread (crates/pg_warren/src/interrupts/pool.rs). The pool is stopped before Postgres unmaps shared memory at backend exit.
Who can do what
Section titled “Who can do what”The trusted install
Section titled “The trusted install”pg_warren.control sets trusted = true and superuser = true. Together
these mean: a superuser can install it, and so can any role with CREATE
on the database. In the second case Postgres runs the install script as the
bootstrap superuser, which then owns the schema and every catalog table,
while the installing role becomes the extension’s owner
(pg_extension.extowner). DESIGN.md §7, “Trusted install”, has the audit
of every install and upgrade script that this depends on: objects are
created only in warren, CREATE SCHEMA IF NOT EXISTS refuses to adopt a
schema created outside the extension (CVE-2022-2625 hardening), and
nothing evaluated at install time can be shadowed by a user’s objects.
What the installer can do as the extension’s owner: ALTER EXTENSION pg_warren UPDATE, and DROP EXTENSION, which drops every graph in the
database. What it cannot do: read or write the catalog tables directly, read
other roles’ graphs, or change who owns them. Granting it access to the
catalog was rejected because it could then rewrite a graph’s source query,
which refresh runs as that graph’s owner.
test/sql/trusted_install.sql checks these refusals, and CI runs the whole
SQL suite and the upgrade test with pg_warren installed by a non-superuser
(test/run.py --trusted, test/upgrade.py --trusted).
Graphs and their access
Section titled “Graphs and their access”Any role can call warren.create_graph; it becomes the graph’s owner.
A graph exposes what its owner could see when it was built, like a
materialised view, so source-table row-level security does not apply to
what a traversal returns. Access is per graph
(semantics.md, “Access”):
- The owner (and members of its role) may manage the graph:
refresh,promote,drop_*,compact,grant,revoke,set_owner, quotas. - Readers, roles added with
warren.grant(and their members), may query it and write vectors on it, within an optional per-role quota. - Anyone may call
warren.version(),warren.last_stats(),warren.evict()(which frees only the caller’s memory unless it owns the graph) and the introspection functions, which show only what the caller may see (below).
Every query and management function checks membership on every call, from
Postgres’s role caches, and management functions check ownership again
after taking the graph’s row lock (DESIGN.md §7). Others get
insufficient_privilege (42501).
Whose SQL runs as whom
Section titled “Whose SQL runs as whom”| SQL | Runs as | search_path |
|---|---|---|
a graph’s node_query and edge_query, at refresh |
the graph’s owner | the one captured at create_graph |
vector_from_query’s and gather’s queries, vector_map expressions |
the caller | the caller’s |
| pg_warren’s own statements on its catalog | the catalog owner (the role that owns the warren.* tables) |
pg_catalog, pg_temp |
All warren.* functions are SECURITY INVOKER; none is SECURITY DEFINER. Instead, each catalog statement runs inside a switch to the
catalog owner (SetUserIdAndSecContext with
SECURITY_LOCAL_USERID_CHANGE | SECURITY_RESTRICTED_OPERATION), and source
queries inside a switch to the graph’s owner, in the same restricted mode
(crates/pg_warren/src/access/identity.rs). The elevation therefore covers
only SQL that pg_warren wrote. An ERROR inside a switch is undone by the
transaction’s abort.
User-written SQL that pg_warren runs (source, value and message queries,
executor-path vector_map expressions) also runs in Postgres’s read-only
transaction state. A function it calls cannot write to tables (25006), and
a data-modifying CTE is refused. This protects what a build or a kernel is
reading; it is not a sandbox: writes to existing temporary tables,
pg_notify, LOCK and writes over another connection still work, with the
privileges of the role the SQL already runs as.
search_path pinning
Section titled “search_path pinning”Schema-qualifying table names is not enough to make extension SQL safe: an
operator such as = or a catalog function such as pg_has_role also
resolves through search_path, so a caller could otherwise put its own
ahead of pg_catalog and have it run as the catalog owner or answer an
access check. So every warren.* entry point runs its whole body under
access::entry, which pins search_path = pg_catalog, pg_temp at a new
GUC nest level, as Postgres does for SECURITY DEFINER … SET search_path,
and restores it on return or abort
(crates/pg_warren/src/access/guard.rs). The caller’s own search_path is
captured first and used only for the caller’s own SQL.
A graph’s source queries run under the search_path its creator had. If
that path includes a schema another role can create objects in (often
public when someone else owns the database), that role could add a
function or operator the queries call as the graph’s owner at every
refresh. create_graph raises a NOTICE naming such schemas; the remedy
is to schema-qualify functions and operators in source queries, or create
graphs under a path only their owner can write (README, “Install”;
test/sql/search_path_notice.sql).
What a non-superuser can see
Section titled “What a non-superuser can see”- Catalog tables: nothing directly. Their names appear in
pg_class, and every function’s definition inpg_proc, as for any extension. The exceptions are Postgres’s own: holders ofpg_read_all_dataread the catalog, and holders ofpg_write_all_datawrite it, as they do every table (below, “Dumps and restores”). - Graphs: there is no function that lists graphs. A role that names a graph it may not read is refused, and the refusal (42501) says the graph exists and names its owner role. Graph names share one namespace per database.
- Introspection:
warren.shared_cache()andwarren.cache()list only versions and vectors of graphs the caller may read, plus cluster totals (sizes and counts).warren.backends()andwarren.progress()followpg_stat_activity’s rule: details only for backends of roles whose privileges the caller has, or withpg_read_all_stats; a graph’s name only where the caller may also read that graph.warren.orphans()shows a non-owner only rows of graphs it owns (DESIGN.md §5.7).
Dumps and restores
Section titled “Dumps and restores”Graph data is extension configuration data, so a superuser’s pg_dump
includes it, and test/dump.py (CI, every major) checks that plain and
custom dumps restore through psql and pg_restore with versions, vectors
and answers intact. The roles that own and read graphs must exist first.
Without a superuser, dump with pg_read_all_data and restore with
pg_write_all_data (DESIGN.md §14 item 51). pg_warren grants no role of
its own access to the catalog, so a non-superuser needs Postgres’s
predefined roles:
- Dump: a role granted
pg_read_all_datarunspg_dumpas usual, and the dump holds every graph, version and vector. Without it,pg_dumpfails withpermission deniedon the first catalog table or sequence. - Restore: a role with
CREATEon the target database (to runCREATE EXTENSION, which is trusted) andpg_write_all_datarestores it withpsqlorpg_restore. Withoutpg_write_all_datathe restore fails atCOPY warren.graphs. Do not use--disable-triggers. - Roles first. Graph owners, readers, vector creators and quota roles
are stored as role names in the dump. If one does not exist on the
target, that catalog table’s data fails to load, and with it the rows
that depend on it: the graph is missing, never silently re-owned. Create
the roles first (
pg_dumpall --roles-only, with--no-role-passwordswhere passwords cannot be read, as on RDS). To move graphs to another role, restore under the old name (aNOLOGINrole will do), then runwarren.reassign_owned(old, new)andwarren.drop_owned(old). - Amazon RDS: from PG 15,
rds_superuserholds both predefined roles with the admin option, so the master user dumps and restores graphs without further grants, and can grantpg_read_all_datato a role that only takes backups. This is reported by RDS users and emulated in CI; it has not yet been run on a real RDS instance (#103).
What these roles also allow, which an administrator should know before granting them:
- A holder of
pg_read_all_datareads every graph’s segments and vectors directly, whateverwarren.grantsays. A graph over row-secured tables carries its owner’s view of them, which the holder could not read from the tables themselves. - A holder of
pg_write_all_datacan change any graph’s source queries and owner, and so have SQL run as that graph’s owner at the graph’s nextrefresh. A restore needs exactly this power: the restored rows decide every graph’s owner and queries. - It cannot, through a graph, run SQL as a superuser: a graph owned by
a superuser runs its source queries only when a superuser calls
refresh(orset_owner, which opens them as the new owner) in its own session. A role that is only a member of a superuser role (even afterSET ROLEto it), and a superuser’sSECURITY DEFINERfunction called from a non-superuser’s session, are refused with 42501. So on RDS, where the master user holdspg_write_all_databut onlyrdsadminis a superuser, pointing a graph atrdsadmingains nothing (test/dump.pychecks it). A superuser’s ownrefreshruns whatever the row holds: after a restore from an untrusted source, or after grantingpg_write_all_data, review the source queries of superusers’ graphs (inwarren.graphs) before a superuser refreshes them, or give those graphs to a non-superuser withwarren.set_ownerorwarren.reassign_owned(DESIGN.md §7, §14 item 51).
Grant them for the dump or the restore and revoke them afterwards; on RDS, treat the master user’s credentials accordingly. Restore only dumps you trust, as for any database.
Without either role, a non-superuser can dump everything but the graphs
with --exclude-table-data='warren.*', and create the graphs again with
create_graph and refresh after the restore, losing persisted vectors
and version history. test/dump.py (CI, PG 15–18) checks all three: the
superuser’s round trip, this one, and a round trip with no superuser at
all, where an emulated RDS master user installs pg_warren, dumps (as does a
role granted only pg_read_all_data), restores, compares every answer,
builds again, and then runs warren.drop_owned and DROP EXTENSION.
Connection poolers and DISCARD ALL
Section titled “Connection poolers and DISCARD ALL”DISCARD ALL (PgBouncer’s server_reset_query in session pooling) also
frees pg_warren’s session state: every role’s transient vectors on that
backend and last_stats(). Cached graph versions and persisted vectors are
not session state, and stay. In transaction pooling nothing is reset
between clients, so a transient vector left at the end of a transaction can
be read by another client of the same role on that backend (transient
vectors are private per role). Create, read and drop transient vectors
within one transaction behind a transaction pooler
(limits.md, “Known limitations”).
test/pooler_soak.py checks both pool modes in CI.
Dropped roles
Section titled “Dropped roles”Owners and readers are stored as role OIDs, which DROP ROLE does not see.
pg_warren provides warren.reassign_owned and warren.drop_owned for the
standard role-removal procedure, an optional event trigger a superuser may
add, and warren.orphans() / warren.drop_orphans() to sweep up after a
DROP ROLE that skipped them. Only a superuser may adopt a graph whose
owner was dropped (semantics.md, “Access”). Until the
sweep runs, a new role that is given a dropped role’s OID would inherit what
that OID still names.
Failure model
Section titled “Failure model”Errors and panics
Section titled “Errors and panics”- Errors are raised only through pgrx (
ereport!with a standard SQLSTATE), never by calling Postgres’s error functions directly. The SQLSTATEs pg_warren raises are documented per function. - A Rust panic is caught by pgrx’s guard at the function boundary and
raised as an
ERROR; the transaction aborts and the backend continues.unwrap,expectandpanic!are lint errors outside tests, so a panic is a bug, and the SQL fuzzer fails on any internal error. - pg_warren never catches an unwind around a Postgres call. The one place
it holds a caught Postgres
ERRORis the compute pool’s wait: when cancellation arrives while worker threads run a kernel, it raises the kernel’s cancellation flag, waits for the threads to stop, then resumes the unwind. Nothing calls Postgres in between (DESIGN.md §6.9). - Every Postgres function pg_warren calls goes through pgrx’s guarded
bindings, so an
ERRORinside one unwinds the Rust frames cleanly. Everyunsafeblock carries a// SAFETY:justification (a lint error otherwise);warren-corehas nounsafecode. - Cleanup that a transaction abort does itself (closing an SPI cursor,
popping a GUC nest level, restoring the user id) is left to the abort
while an
ERRORunwinds, so it happens in Postgres’s order.
Cancellation and timeouts
Section titled “Cancellation and timeouts”Every long call checks for interrupts: traversals every 65,536 units of
work; compute kernels from the main thread every millisecond, while worker
threads check a cancellation flag every task of at most 4,096 rows. So
statement_timeout, pg_cancel_backend and pg_terminate_backend stop a
call promptly, with query_canceled for a cancel. A backend terminated
mid-kernel stops its threads before exit
(test/sql/shared_cache_exit_mid_kernel.sql), and a backend that is
waiting for another to load a version into the shared cache wakes every
second to check for interrupts.
Memory
Section titled “Memory”- Graph-sized buffers are reserved with
try_reserve, so an allocation the system refuses isout_of_memory(53200), not an abort. Other, small allocations use Rust’s ordinary allocator; as with any C extension, a failure there ends the backend. - Within that, pg_warren bounds its own memory with the settings above.
A call that would exceed
warren.max_compute_memoryorwarren.max_build_memoryis refused withconfiguration_limit_exceeded(53400) before it allocates. A backend’s private memory stays withinwarren.max_cache_memorypluswarren.max_compute_memory(pluswarren.max_build_memoryduring arefresh), and the cluster’s shared graph memory withinwarren.shared_cache_memory. Not bounded by a setting: a traversal’s visited set and frontier, and several calls running at once in one statement (limits.md, “Memory model”).
Crashes and recovery
Section titled “Crashes and recovery”Graph data lives in ordinary WAL-logged tables, so crash recovery is
Postgres’s own: a refresh interrupted by a crash leaves no version, as its
transaction never committed. A crashed backend makes the postmaster
reinitialise shared memory, which destroys every shared-cache segment and
the registry, so there is nothing of pg_warren’s to repair. A backend that
exits with FATAL gives back its shared-cache slots in its exit callback.
test/crash_recovery.py stops refreshes mid-way by pg_terminate_backend,
SIGKILL and an immediate shutdown, and checks the catalog, the answers,
the shared cache and a new refresh afterwards.
Standbys and read-only transactions
Section titled “Standbys and read-only transactions”On a physical standby every query function answers as on the primary once
it has replayed the same WAL, and computations that persist nothing work.
Every function that would change the catalog fails with
read_only_sql_transaction (25006), with a hint that the server is a
standby; so does any such call in a read-only transaction on a primary.
All catalog writes go through one writable SPI connection
(read_only::writing) that refuses first, and a lint forbids the other
writable SPI calls outside tests. A promoted standby writes normally.
test/replicas.py checks this on a hot standby and a cascading standby,
including recovery conflicts (DESIGN.md §5.1, “Standbys”).
Parallel query
Section titled “Parallel query”Functions carry the parallel label their behaviour allows (DESIGN.md §9.3;
semantics.md, “Parallel query labels”):
writers and anything that may persist are PARALLEL UNSAFE; vector reads
and a few others are PARALLEL RESTRICTED; graph reads that use no vector,
and functions that read no graph, are PARALLEL SAFE. Reads assign no
transaction id, so they work in parallel mode (a lint enforces this). Where
a version is not shared, each parallel worker loads its own copy, within
its own memory limits; any role can avoid that with
max_parallel_workers_per_gather = 0.
Resource use and limits
Section titled “Resource use and limits”Today’s behaviour:
warren.shared_cache_memoryis set in the server’s configuration only, so it is the one cluster-wide bound a provider controls. Default4GB: shared versions stay until evicted, dropped or the server restarts, so an idle cluster can hold up to the cap. In containers,/dev/shmmust be sized for it (Docker’s default is 64 MB); pg_warren falls back to private copies rather than fail when it is short.- The other five settings are
usersettings: any role can raise them for its own sessions, up to 1,024 threads per backend and about 2 TB of memory per setting.ALTER ROLE … SETandALTER DATABASE … SETset defaults, not ceilings. So on a shared instance one role can run many threads, or large computations, in each of its connections. Postgres’s own limits (max_connections,statement_timeout, the operating system’s memory limits) still apply. - Per-database shares of the shared cache do not exist: one database’s loads can evict another’s unpinned versions.
Administrator-set ceilings on these settings and a cap on compute threads across backends are planned for 0.9.0 (#103, “Resource governance across the instance”). Sizing guidance is in limits.md.
Test evidence
Section titled “Test evidence”Workflows: ci.yml (every pull request,
every push to main, and nightly), corpus.yml
(nightly), release-publish.yml.
How to run each locally is in CONTRIBUTING.md.
| Evidence | What it checks | Where it runs |
|---|---|---|
warren-core unit and property tests |
kernels and data structures against reference implementations | every CI run |
#[pg_test] suite (cargo pgrx test) |
the SQL surface, SPI adapters, transactions, roles | PG 15, 16, 17, 18 on x86_64; PG 16 and 18 on arm64; PG 18 on macOS arm64 (reported, not yet gating) |
Hostile search_path (access::guard, every_warren_function_ignores_the_callers_search_path) |
every warren.* function, called with an attacker’s operators ahead of pg_catalog, never runs them and its access checks hold; fails if a new function is not in the list |
in the pgrx suite |
Parallel labels (parallel_labels, every_function_has_its_declared_parallel_label, every_read_runs_in_parallel_mode) |
each function’s label is the declared one, and every non-UNSAFE read runs in parallel mode and in a worker |
in the pgrx suite |
SQL harness (test/run.py, test/sql/) |
integration tests, races between sessions, the shared cache across backends, DISCARD ALL, the trusted install, documentation examples |
PG 16 (x86_64) at the default, trusted and preloaded; the shared-cache tests in three modes on PG 15–18; arm64 and macOS |
Oracle tests (test/sql/oracle_*.sql) |
results against NetworkX and recursive SQL | in the SQL harness |
Real graphs (bench/corpus) |
9 public datasets checked against NetworkX and igraph | nightly |
Segment decoder fuzzing (fuzz/, cargo-fuzz) |
the decoders of stored graph segments (CSR, id maps, tombstones, relation dictionary, membership) on arbitrary bytes, 30 s per target | every CI run |
SQL-surface fuzzer (test/sql_fuzz.py) |
every warren.* function with generated and hostile arguments, as four roles, under a hostile search_path, in parallel plans and rolled-back transactions; fails on crashes, internal errors, privilege leaks and leftover state |
PG 16 |
Miri (cargo miri test -p warren-core) |
undefined behaviour in the domain crate and its dependencies’ unsafe code | nightly on main, and on release pull requests |
Upgrades (test/upgrade.py) |
ALTER EXTENSION … UPDATE from every release gives the catalog of a fresh install, also by a non-superuser |
PG 15–18 |
Dumps (test/dump.py) |
plain and custom dumps restore with graphs, versions and vectors; a trusted installer’s dump without catalog data; the same round trip with no superuser, as on RDS (pg_read_all_data, pg_write_all_data), a missing role, a graph rewritten to a superuser owner refused to the non-superuser’s refresh, and the extension dropped by its non-superuser owner |
PG 15–18 |
Crash recovery (test/crash_recovery.py) |
refreshes stopped by termination, SIGKILL and immediate shutdown |
PG 16 |
Replicas (test/replicas.py) |
hot and cascading standbys read, refuse writes with 25006, and promote | PG 16 |
Concurrency (test/concurrent_refresh.py) |
many backends reading, computing and refreshing one graph | PG 16 |
Pooler soak (test/pooler_soak.py) |
PgBouncer in transaction and session mode, 2 minutes: memory growth, leftover state, session isolation | PG 16 (a 2.5-hour run is recorded in DESIGN.md §13, M11) |
| Lints | rustfmt; clippy and rustdoc with -D warnings; no unwrap/expect/panic!; documented unsafe; no SPI call that assigns a transaction id on a read path |
every CI run |
Supply chain and releases
Section titled “Supply chain and releases”- Toolchain pins. The Rust toolchain is pinned exactly in
rust-toolchain.toml;pgrxandcargo-pgrxare pinned to=0.19.3and upgraded together in their own change;Cargo.lockis committed; CI’s third-party actions are pinned by commit hash, and the Miri and fuzzing nightly by date. - Frozen schema. A released install script never changes: its SHA-256 is
recorded in
sql/released.sha256and verified before publishing, and each later schema change ships an upgrade script thattest/upgrade.pychecks. - Release builds (
release-publish.yml; CONTRIBUTING.md, “Packages”) run in jobs with read-only repository access and no secrets, in distribution containers pinned by digest. Each.debis installed into a clean container and smoke-tested. After a person approves the protectedreleaseenvironment, a publish job that runs no build code checks each package’s contents and metadata, pushes the annotated tag with a deploy key, writesSHA256SUMSand signs it (SHA256SUMS.asc), and assembles and signs the apt repository. The signing key’s fingerprint is stated in the README and checked by the job; a signed apt snapshot is then published to GitHub Pages behind its own approval. - What is published:
.debpackages for PG 15–18 on Debian 12 and 13 and Ubuntu 22.04 and 24.04 (amd64, arm64), tarballs, Docker images on GHCR, a Homebrew formula, the source tarball, a vendored source tarball and an SBOM (README, “Install”), all listed in the signedSHA256SUMS. - Dependency checks.
cargo-denyruns in CI (deny.toml): RustSec advisories (an exemption is by advisory id, with its reason), an allow-list of permissive licences, and crates.io as the only source. - SBOM. Each release attaches
pg_warren-X.Y.Z.cdx.json, a CycloneDX 1.5 SBOM of pg_warren and every crate it uses, with licences andCargo.lock’s checksums; the publish job checks it against the source. - Offline build. Each release attaches
pg_warren-X.Y.Z-vendored.tar.gz, the source with every locked crate, which builds withcargo build --frozenand no network; the publish job checks it is the source tarball plusCargo.lock’s crates and nothing else (CONTRIBUTING.md, “Supply chain”). - Reproducible builds. With the same toolchain and PostgreSQL
headers,
pg_warren.sois the same bytes wherever it is built and whether its crates come from crates.io or the vendored tarball, and two builds of a.debfrom the same source give the same bytes; CI checks both. The packages ship the reviewed install script thatsql/released.sha256freezes (CONTRIBUTING.md, “Reproducible builds”).
Not yet:
- Reproducible packages: the
.debbuild installs the current PGDG headers and Debian build tools without recording their versions, so a later rebuild of a release need not give the same.debbytes; the Homebrew bottle and the Docker images are not checked either. - RPM packages, a PGXN release, and inclusion in the PGDG repositories.
- macOS as a supported platform.
These are tracked in #103 (“Supply chain”).
Reporting a vulnerability
Section titled “Reporting a vulnerability”pg_warren does not yet publish a security policy (SECURITY.md) or a
private channel for reporting vulnerabilities. This is a known gap.
Known gaps, together
Section titled “Known gaps, together”- Resource settings other than
warren.shared_cache_memorycan be raised by any role; instance-wide ceilings are planned for 0.9.0. - Dumping and restoring graphs without a superuser needs
pg_read_all_dataandpg_write_all_data, which reach every graph directly; there is no narrower grant (§14 item 51). A holder ofpg_write_all_datacan make any graph that a non-superuser owns run SQL of its choosing as that owner at the nextrefresh(not a superuser’s: those run only for a superuser). The RDS route is emulated in CI, not yet run on RDS. - A role can learn that a graph name exists, and its owner’s role name, by calling a function on it.
- Small Rust allocations are not fallible; only graph-sized buffers are.
- No security policy yet, and the packages are not yet reproducible on a later rebuild (the library is, from the same toolchain and headers).