# sudohulk · full corpus > Vignesh Kumar A, a database engineer at Zoho, keeps a write-ahead log of what he learns: database internals and computer science, written down plainly and corrected in the open. Every public page of sudohulk.com, as markdown, in one file. Canonical URLs are given per entry. 12 entries and 2 tracks. Licence: CC BY 4.0. Cite the canonical URL. ============================================================================== # Glossary The terms the writing uses, each in one plain line, with an analogy and where that analogy breaks. > Canonical URL: https://sudohulk.com/glossary > 3 terms, A to Z. Each is a note on the site; the first time an entry says one, the word opens a card with these lines. > Author: Vignesh Kumar A > Licence: CC BY 4.0. Quote freely, cite the canonical URL. --- ## Full-page writes and torn pages Also: full page image, FPI, torn page, full-page write The first change to a page after a checkpoint puts the whole 8 KB page in WAL, so recovery can overwrite a half-written page instead of patching it. Analogy: Before the first edit to a form, photocopy the whole sheet. Later edits are pencil marks; if the sheet tears, you start again from the copy and redo the marks. Where the analogy breaks: The copy is taken once per checkpoint, not once per form. After the next checkpoint the first edit copies the sheet again, which is why WAL swells right after one. The note: [Full-page writes and torn pages](https://sudohulk.com/writing/full-page-writes-and-torn-pages.md) ## Lazy transaction id assignment Also: lazy xid, virtual transaction id, xid A transaction gets its 32-bit xid on its first write, not at BEGIN; until then it runs under a virtual id, so a read-only transaction never uses one up. Analogy: A counter that hands out numbered tickets only to people who order. Whoever reads the menu and leaves never takes a ticket, so the numbers last longer. Where the analogy breaks: A ticket, once taken, is never handed back. A transaction that rolls back has still used its xid; the counter only moves forward, and wraparound is measured against it. The note: [Lazy transaction id assignment](https://sudohulk.com/writing/lazy-xid-assignment.md) ## pd_lsn vs record LSN Also: page LSN, record LSN, LSN, pd_lsn An LSN is a byte address in the WAL stream. A record's LSN is where it sits; a page's pd_lsn is the address just past the last record applied to that page. Analogy: Page numbers in a diary. Each entry has its page, and each thing you own is marked with the page of the last entry about it, so you know which entries it already reflects. Where the analogy breaks: Two diaries never share page numbers. An LSN means something only inside the server it came from, so comparing LSNs across machines compares nothing. In the PostgreSQL source: `src/include/storage/bufpage.h` The note: [pd_lsn vs record LSN](https://sudohulk.com/writing/pd-lsn-vs-record-lsn.md) ============================================================================== # How PostgreSQL tells every connection its cache is stale Each PostgreSQL connection caches the catalog for itself. How a change is announced to all of them at commit through a shared ring, and what happens to a connection that falls behind. > Canonical URL: https://sudohulk.com/writing/postgres-cache-invalidation > Type: essay · Stream: Tech > Published: 2026-10-17 > Series: PostgreSQL internals, explained simply (part 5) > Tags: postgresql, databases, performance > Reading time: 19 min > Author: Vignesh Kumar A > Licence: CC BY 4.0. Quote freely, cite the canonical URL. --- ## What happened I once worked on a cache that sat in front of a database. It answered quickly and sometimes wrongly. I redesigned it as three isolated segments with **version-based invalidation**. Each entry remembers the version of the data it came from, and once the data has a newer version, the old entry stops counting. The hit rate went from 84.3% to 98.1%, and the database answered about 131 million fewer queries a week. Looking back, storing things was the easy part of that cache. Most of the effort went into deciding when to throw an entry away. Later, when I started reading PostgreSQL's source, I found the same problem inside every connection. **Each PostgreSQL connection keeps its own copy of the catalog. A change is announced at commit through one shared ring of messages, and each connection reads the ring before it trusts its copy.** ## Why every connection keeps its own cache Think of a library where every reader keeps a notebook of the facts they look up often, so they don't have to walk back to the shelves each time. PostgreSQL connections work the same way. Each connection gets its own server process, a **backend**, with private memory. What the database knows about itself is in the **system catalog**, a set of ordinary tables. `pg_class` has a row per table, index or view, and `pg_attribute` has a row per column. To run even `SELECT * FROM orders`, a backend has to know what `orders` is: its columns, their types, its indexes. It could read all of that from the catalog tables for every query, but that would be far too slow. Instead, each backend keeps three caches of its own: - the **catalog cache** (catcache) holds single catalog rows looked up by key, such as "the `pg_class` row of table 16384". Most code gets to it through a layer called the **syscache**. - the **relation cache** (relcache) holds one assembled description per table or index, with everything needed to open it. - the **plan cache** holds the saved plans of prepared statements, which depend on the tables they read. None of this is shared. When one connection changes the catalog, every other connection's caches are wrong, and nothing has told them yet. ## Collecting changes until commit When a backend inserts, updates or deletes a catalog row, it doesn't flush anything straight away. Under PostgreSQL's rules, the old row stays valid until the current command ends. What the backend does instead is record an **invalidation message** for each cache that could hold the row. It keeps that list until the change is final. An update counts as a delete plus an insert, and the two fold into one message when the cache's key didn't change. A change to a row of `pg_class`, `pg_attribute` or `pg_index`, or to a foreign key's row in `pg_constraint`, also records a relcache message for the table that row describes. A table gets at most one of those per command. What happens to the list depends on how things end: - **A command ends.** The backend applies that command's messages to its own caches and nobody else's ([CommandEndInvalidationMessages](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/utils/cache/inval.c#L1408)). That's why your next statement already sees the column you added. Other backends can't see your uncommitted rows, so there's nothing to tell them. - **The transaction commits.** The commit is recorded first, then every message goes into the shared queue ([AtEOXact_Inval](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/utils/cache/inval.c#L1198)), and only after that are the locks released. - **The transaction aborts.** Nothing is sent, and the backend undoes its own cache changes. An aborted subtransaction does the same for its part, and a committed one hands its list to its parent. The order at commit protects both sides. If the messages went out before the commit was recorded, another backend could reload the catalog and still find the old rows. If they went out after the locks were released, a backend waiting for the table could get them only after it had used its stale entry. The commit's WAL record carries the same messages, so a standby replays them for its own connections. ## The ring of messages The messages go into the **shared invalidation queue**, sinval for short, in shared memory. There's one for the whole server, covering every database. Each message carries a database id, and a backend skips messages for other databases. The code describes it as an infinite list of numbered messages. In memory it's a ring of 4,096 slots (`MAXNUMMESSAGES`), and message *n* sits in slot *n* modulo 4,096. A message is 16 bytes, so the whole ring is about 64 KB. Three numbers keep track of it: - `maxMsgNum`, the write position, where the next message goes; - `nextMsgNum`, one per backend, the next message that backend hasn't read; - `minMsgNum`, the oldest message anyone still needs. A new connection starts with its read position at the write position. Its caches are empty, so it has nothing to catch up on. ![A ring of 24 slots standing in for 4,096. The writer, W, is at the next free slot. Readers B1 and B2 are a little behind it. B3 is far behind, with most of the ring still unread.](https://sudohulk.com/_astro/sinval-ring.BXghl_4U_2dzIV9.svg "The ring drawn with 24 slots. W writes clockwise, and each reader has its own position.") The main kinds of message are: - **catcache**: rows with this hash may be stale in this cache; - **catalog**: forget everything cached from this whole catalog; - **relcache**: this table's description is stale, or every table's when the id is 0; - **smgr**: close your file handles for this table; this kind is never skipped by database; - **relmap** and **snapshot**: smaller kinds, for the map of a few catalogs' files and for the snapshot used to read the catalog. A catcache message doesn't carry the key, only a **hash** of it, because a table name wouldn't fit in 16 bytes. The receiver removes every entry in that cache with the same hash. Once in a while that catches an unrelated row whose key hashes the same, a false positive that costs one reload. Writers add messages under a lock, at most 64 at a time ([SIInsertDataEntries](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/storage/ipc/sinvaladt.c#L369)), so even a commit with thousands of messages never holds the lock for long. After each chunk, the writer sets a "you have messages" flag on every backend. ## Reading the ring before trusting the cache A backend has to read the ring before it trusts its caches. Reading the ring is one function, `AcceptInvalidationMessages()` ([inval.c:929](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/utils/cache/inval.c#L929)). It calls [ReceiveSharedInvalidMessages](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/storage/ipc/sinval.c#L68), which pulls messages 32 at a time from [SIGetDataEntries](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/storage/ipc/sinvaladt.c#L472) and applies each one to the local caches. A backend reads the ring at two moments: - **when a transaction starts**, so every transaction begins with checked caches; - **when it locks a table it did not already hold in that mode**, in [LockRelationOid](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/storage/lmgr/lmgr.c#L106) and its siblings in `lmgr.c`. `ALTER TABLE ... ADD COLUMN` shows why the second one matters. It takes an `ACCESS EXCLUSIVE` lock, and the committing backend writes its messages before it lets go of that lock. Whoever gets a lock on the table next reads the ring right after, and finds the messages already there. Once a backend holds the lock, it can trust its cache for that table. If the backend already held that kind of lock, it skips the read. Nobody could have changed the table in a conflicting way while it held the lock. Usually there's nothing new, and the read costs almost nothing. `SIGetDataEntries` first looks at the backend's own "has messages" flag without taking a lock, and returns at once when the flag isn't set. When there are messages, each one is applied in turn: - A catcache message removes the matching rows, plus every cached list in that cache. A row in use at that moment is marked dead, and removed once it's released. - A relcache message drops the table's description if nobody has it open, and rebuilds it if someone does. - Each message also runs the **callbacks** registered for it. The plan cache registers some, so a relcache message marks every saved plan that reads the table as invalid. The plan is built again the next time it's used. Within one transaction's messages, the catcache ones come first. That way a table's description is never rebuilt from catalog rows that are about to be thrown away. ## When the ring fills up A writer never waits for a slow reader, or one idle connection could hold up every DDL on the server. Instead, PostgreSQL cleans up the ring and nudges the reader furthest behind. A reader that falls too far behind gets reset. Cleanup ([SICleanupQueue](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/storage/ipc/sinvaladt.c#L576)) runs when the ring holds 2,048 messages, again every 256 after that, and whenever a write would not fit. It does three things: 1. It finds the oldest message anyone still needs, which frees the slots before it. 2. It marks for **reset** any backend so far behind that the writer is about to overwrite a message it hasn't read. 3. It signals at most one backend: of those more than 2,048 messages behind and not yet signalled, the one furthest behind. The signal asks it to **catch up**. ```mermaid flowchart TD W["A commit writes messages"] --> Q{"Past the cleanup threshold
(2,048, then every 256),
or no room left?"} Q -->|no| G["Write them and go on"] Q -->|yes| C["Clean up the ring"] C --> L{"Is a reader about to lose
a message it has not read?"} L -->|yes| R["Mark it for reset: at its next
read it drops its whole cache"] R --> S{"Any reader over 2,048 behind
and not yet signalled?"} L -->|no| S S -->|no| G S -->|yes| N["Signal the one furthest behind"] N --> G N -.-> K["The reader catches up
while waiting for its client"] K -.-> P["Then it runs cleanup itself"] P -.->|signal passes on| S ``` The signalled backend deals with it while it waits for its client. If it's inside an open transaction, it calls `AcceptInvalidationMessages()`. If not, it starts and commits an empty transaction, which reads the ring on the way in. Once it has caught up, it runs cleanup itself, and that cleanup signals the next backend furthest behind. The source calls this a **daisy chain**. Usually only one signal is in flight at a time, so idle connections wake one after another and not all at once. A reader that stays stuck after its signal is passed over. Later cleanups signal the next reader that hasn't had one. The last resort is the reset. A backend whose unread message has been overwritten can't know what that message said. The only safe thing is to throw away its whole cache and reload entries from the catalog as they're needed. ![The same ring after the writer has come all the way round. W has written into the slot B3 had not read, shown in red, so B3 is marked for reset. B1 and B2 are not affected.](https://sudohulk.com/_astro/sinval-reset.DfUPHrcm_ZH7clu.svg "The writer laps B3. B3 can't know what it missed, so it starts over.") At its next read, a backend marked for reset gets no messages. It jumps to the write position and resets instead, through [InvalidateSystemCachesExtended](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/utils/cache/inval.c#L784): - every catalog cache is emptied; - every table description nobody has open is dropped, and the open ones are rebuilt; - every catcache callback runs with hash 0, and every relcache callback with `InvalidOid`, both meaning "everything". A reset is always correct, but for a while it's slow. Every lookup in that connection goes back to the catalog tables until the caches fill up again. ## Keeping your own cache honest An extension may keep a cache of its own, with things like options worked out for a table, a parsed setting or a connection to another server. PostgreSQL doesn't know about that cache, so it can't flush it, but you can register a function for it to call when it applies a message. - `CacheRegisterSyscacheCallback(cacheid, func, arg)` ([inval.c:1815](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/utils/cache/inval.c#L1815)) runs `func` for each catcache message for that cache, with the hash. - `CacheRegisterRelcacheCallback(func, arg)` ([inval.c:1857](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/utils/cache/inval.c#L1857)) runs `func` for each relcache message, with the table's OID. The rules I follow: 1. **Register once per backend.** There's no function to unregister. The slots are fixed: 64 catcache callbacks and 10 relcache callbacks per backend, shared with PostgreSQL's own caches and every other extension that's loaded. Running out is `FATAL`, and the connection ends. 2. **Treat 0 and `InvalidOid` as "drop everything".** That's what a reset sends. 3. **In the callback, only mark entries stale.** Don't read the catalogs there; rebuild on the next lookup. The callback runs whenever the backend reads the ring, and that can be in the middle of your own catalog read. 4. **A catcache callback gets a hash, not a key.** Store each key's hash when you build an entry, or flush everything for that cache. 5. **Watch for entries born stale.** Building an entry reads the catalogs, reading takes locks, and a lock reads the ring. That can run your callback for the very entry you're building. PostgreSQL's relcache builds again when that happens; most other caches rely on DDL taking an `ACCESS EXCLUSIVE` lock. The sketch below shows the pattern. The `my_` names are made up, and I haven't compiled this exact code. ```c typedef struct MyEntry { Oid relid; /* hash key: the table */ bool valid; int setting; /* worked out from the catalogs */ } MyEntry; static HTAB *my_cache = NULL; static uint64 my_inval_count = 0; static void my_relcache_callback(Datum arg, Oid relid) { HASH_SEQ_STATUS status; MyEntry *entry; my_inval_count++; if (relid == InvalidOid) /* a reset: anything may be stale */ { hash_seq_init(&status, my_cache); while ((entry = hash_seq_search(&status)) != NULL) entry->valid = false; } else if ((entry = hash_search(my_cache, &relid, HASH_FIND, NULL)) != NULL) entry->valid = false; /* only mark it: no catalog reads here */ } static MyEntry * my_lookup(Oid relid) { MyEntry *entry; bool found; if (my_cache == NULL) { HASHCTL ctl = {.keysize = sizeof(Oid), .entrysize = sizeof(MyEntry)}; my_cache = hash_create("my cache", 64, &ctl, HASH_ELEM | HASH_BLOBS); CacheRegisterRelcacheCallback(my_relcache_callback, (Datum) 0); } entry = hash_search(my_cache, &relid, HASH_ENTER, &found); if (!found) entry->valid = false; while (!entry->valid) { uint64 before = my_inval_count; entry->setting = my_compute_setting(relid); /* reads the catalogs */ entry->valid = (my_inval_count == before); } return entry; } ``` The counter guards against an entry born stale: if any callback ran during the build, the loop builds it again. If the build fails with an error, the entry stays invalid and the next lookup tries again. `postgres_fdw` is a real example. Its connection cache registers catcache callbacks for `pg_foreign_server` and `pg_user_mapping` once, when it creates its hash table. Each connection it opens remembers the hash of its server's key and of its user mapping's key. Its callback ([connection.c:1295](https://github.com/postgres/postgres/blob/REL_18_3/contrib/postgres_fdw/connection.c#L1295)) treats hash 0 as "all", and otherwise compares hashes. A connection not used in the current transaction is closed at once. A connection in use is only marked, and closed when the transaction ends, because you can't pull a connection out from under a running query. Another cache in the same extension, the one that remembers which objects are safe to send to the remote server, does the simple thing. It empties itself whenever a foreign server's row changes ([shippable.c:64](https://github.com/postgres/postgres/blob/REL_18_3/contrib/postgres_fdw/shippable.c#L64)). ## A worked example: one ALTER TABLE Take two connections to the same database. Session B has been using `orders`: ```sql PREPARE get_order(int) AS SELECT id, total FROM orders WHERE id = $1; EXECUTE get_order(7); ``` B now holds the `pg_class` row of `orders`, a relcache entry for it, and a saved plan for `get_order`. Session A runs one statement: ```sql ALTER TABLE orders ADD COLUMN discount integer; ``` Inside A: 1. A takes an `ACCESS EXCLUSIVE` lock on `orders`. If B were inside a transaction that had read `orders`, A would wait here until it ended. 2. A inserts a `pg_attribute` row for `discount`. `pg_attribute` has two caches, by column name and by column number, so that's two catcache messages. The row belongs to `orders`, so there's one relcache message too. 3. A updates the `pg_class` row of `orders`, whose column count goes up by one. `pg_class` has two caches, by OID and by name. Neither key changed, so that's one message each, and `orders` already has its relcache message. 4. A ends an internal command and applies the messages to its own caches. The rest of the statement sees the new column. 5. A commits. The commit is recorded, the messages go into the ring, and the lock is released. | # | Kind | Cache or table | Why | |---|---|---|---| | 1 | catcache | `pg_attribute` by name | the new column's row | | 2 | catcache | `pg_attribute` by number | the same row | | 3 | catcache | `pg_class` by OID | the column count changed | | 4 | catcache | `pg_class` by name | the same row | | 5 | relcache | `orders` | its rows changed; listed once | By my reading of the source, five is the floor. A default, a constraint or an index touches rows in more catalogs, and each of those rows brings its own messages. I'd rather give you the kinds than a total I can't defend. Then B runs `EXECUTE get_order(7);`: 1. B starts a transaction and reads the ring, which now holds A's messages. 2. The `pg_class` messages remove B's cached row of `orders`. The `pg_attribute` messages name the new column, which B never cached. B's rows for the old columns didn't change, so they stay. 3. The relcache message drops B's description of `orders`, which nobody has open between transactions. The plan cache's callback marks `get_order` invalid. 4. `EXECUTE` finds the plan invalid. It locks `orders`, reads the ring again and finds nothing new, reloads `orders` from the catalog, and plans again. 5. B gets the same two columns, and it never runs the old plan against the changed table. The same steps as a sequence diagram. In it, B reads the ring when it takes its lock; in a new transaction it has already read it once at the start. ```mermaid sequenceDiagram participant A as Session A participant R as Shared ring participant B as Session B participant C as Catalog A->>C: ALTER TABLE orders ADD COLUMN discount Note over A: holds ACCESS EXCLUSIVE on orders A->>C: commit recorded A->>R: invalidation messages written Note over A: locks released B->>B: next query on orders B->>B: lock on orders B->>R: AcceptInvalidationMessages() R-->>B: the messages from A B->>B: stale entries for orders dropped B->>C: reload orders C-->>B: orders, now with discount ``` ## Gotchas 1. **A catcache message carries a hash, not a key.** Unrelated rows with the same hash are flushed too, which costs a reload and nothing else. Your own catcache callback gets the hash and no OID, so compare stored hashes or flush everything. 2. **Any message to a cache drops every list in it.** Some lookups ask a cache for all rows that match part of a key, and the answer is cached as a list; finding functions by name works this way. A list can't be checked against one hash, so it goes. A burst of `CREATE OR REPLACE FUNCTION` empties those lists in every connection. 3. **An idle connection lags.** It reads the ring at its next transaction or new lock, or when it's the one signalled. It acts on a signal only while it waits for its client, so a connection busy with one long query reads the ring only when it takes a new lock. 4. **One huge DDL can reset many connections at once.** Dropping a schema full of tables changes thousands of catalog rows, and the commit sends all their messages together. Connections that haven't read along by the time the writer comes round can be reset, and then they all reload their caches at about the same time. 5. **`debug_discard_caches` works only in assert builds.** It throws caches away at every chance, to catch code that holds on to an entry too long. At 1, the regression tests run about 100 times slower. It works in builds with assertions, or with `DISCARD_CACHES_ENABLED` defined; anywhere else it stays 0, and setting it to anything else fails. 6. **A prepared `SELECT *` fails after `ADD COLUMN`, on purpose.** The new plan would return one more column, and PostgreSQL won't change a prepared statement's result shape. Name the columns, or prepare the statement again. ```text ERROR: cached plan must not change result type ``` ## The whole loop in one picture ```text session A, the writer change a catalog row -> one message per cache that could hold it end of each command -> apply them to A's own caches COMMIT recorded -> all messages into the ring -> locks released the ring: one per server, 4,096 slots one write position one read position per backend session B, a reader transaction start, or a new lock -> read the ring -> drop stale rows, descriptions, plans next use -> reload from the catalog tables ring too full -> signal the furthest-behind reader not yet signalled -> a full ring behind: reset, reload all ``` ## Why this matters If you write an extension, PostgreSQL can't see any cache you keep, so follow the five rules in [Keeping your own cache honest](#keeping-your-own-cache-honest). If you run many connections, each one keeps its own caches and pays for its own reloads. After a wave of resets, every one of them reads the same catalog rows again. Huge catalog changes go down better at quiet times, or split into smaller transactions, so connections get a chance to catch up between commits. A connection pooler helps as well, since fewer backends means fewer caches to keep up to date. ## Dig deeper The long comment that opens [`inval.c`](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/utils/cache/inval.c) is the best explanation of all this that I know. [`sinvaladt.c`](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/storage/ipc/sinvaladt.c) opens with the design of the ring. Both links, and every trail below, are at PostgreSQL 18.3. **Code trail: From a commit to the next query** 1. `CommandEndInvalidationMessages`, in [inval.c, lines 1416–1427](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/utils/cache/inval.c#L1416-L1427): ```c if (transInvalInfo == NULL) return; ProcessInvalidationMessages(&transInvalInfo->ii.CurrentCmdInvalidMsgs, LocalExecuteInvalidationMessage); /* WAL Log per-command invalidation messages for wal_level=logical */ if (XLogLogicalInfoActive()) LogLogicalInvalidations(); AppendInvalidationMessages(&transInvalInfo->PriorCmdInvalidMsgs, &transInvalInfo->ii.CurrentCmdInvalidMsgs); ``` At the end of each command, the backend applies that command's messages to its own caches only. Nothing reaches the ring yet. 2. `CommitTransaction`, in [xact.c, lines 2427–2438](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/access/transam/xact.c#L2427-L2438): ```c * Make catalog changes visible to all backends. This has to happen after * relcache references are dropped (see comments for * AtEOXact_RelationCache), but before locks are released (if anyone is * waiting for lock on a relation we've modified, we want them to know * about the catalog change before they start using the relation). */ AtEOXact_Inval(true); AtEOXact_MultiXact(); ResourceOwnerRelease(TopTransactionResourceOwner, RESOURCE_RELEASE_LOCKS, ``` After the commit is recorded (`RecordTransactionCommit`, earlier in this function), the messages go out, and the comment explains why this happens before the locks are released. 3. `AtEOXact_Inval`, in [inval.c, lines 1222–1234](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/utils/cache/inval.c#L1222-L1234): ```c AppendInvalidationMessages(&transInvalInfo->PriorCmdInvalidMsgs, &transInvalInfo->ii.CurrentCmdInvalidMsgs); ProcessInvalidationMessagesMulti(&transInvalInfo->PriorCmdInvalidMsgs, SendSharedInvalidMessages); if (transInvalInfo->ii.RelcacheInitFileInval) RelationCacheInitFilePostInvalidate(); } else { ProcessInvalidationMessages(&transInvalInfo->PriorCmdInvalidMsgs, LocalExecuteInvalidationMessage); ``` On commit, every message goes to the shared ring. On abort, the `else` branch only undoes this backend's own caches. 4. `SIInsertDataEntries`, in [sinvaladt.c, lines 395–408](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/storage/ipc/sinvaladt.c#L395-L408): ```c * If the buffer is full, we *must* acquire some space. Clean the * queue and reset anyone who is preventing space from being freed. * Otherwise, clean the queue only when it's exceeded the next * fullness threshold. We have to loop and recheck the buffer state * after any call of SICleanupQueue. */ for (;;) { numMsgs = segP->maxMsgNum - segP->minMsgNum; if (numMsgs + nthistime > MAXNUMMESSAGES || numMsgs >= segP->nextThreshold) SICleanupQueue(true, nthistime); else break; ``` Before each chunk of at most 64 messages, the writer cleans the ring if it is full or past its next threshold. 5. `AtStart_Cache`, in [xact.c, lines 1166–1170](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/access/transam/xact.c#L1166-L1170): ```c static void AtStart_Cache(void) { AcceptInvalidationMessages(); } ``` Every transaction starts by reading the ring. This is the whole function. 6. `LockRelationOid`, in [lmgr.c, lines 119–137](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/storage/lmgr/lmgr.c#L119-L137): ```c * Now that we have the lock, check for invalidation messages, so that we * will update or flush any stale relcache entry before we try to use it. * RangeVarGetRelid() specifically relies on us for this. We can skip * this in the not-uncommon case that we already had the same type of lock * being requested, since then no one else could have modified the * relcache entry in an undesirable way. (In the case where our own xact * modifies the rel, the relcache update happens via * CommandCounterIncrement, not here.) * * However, in corner cases where code acts on tables (usually catalogs) * recursively, we might get here while still processing invalidation * messages in some outer execution of this function or a sibling. The * "cleared" status of the lock tells us whether we really are done * absorbing relevant inval messages. */ if (res != LOCKACQUIRE_ALREADY_CLEAR) { AcceptInvalidationMessages(); MarkLockClear(locallock); ``` The other moment: right after a lock on a table is granted, unless the backend already held one like it. 7. `AcceptInvalidationMessages`, in [inval.c, lines 938–949](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/utils/cache/inval.c#L938-L949): ```c ReceiveSharedInvalidMessages(LocalExecuteInvalidationMessage, InvalidateSystemCaches); /*---------- * Test code to force cache flushes anytime a flush could happen. * * This helps detect intermittent faults caused by code that reads a cache * entry and then performs an action that could invalidate the entry, but * rarely actually does so. This can spot issues that would otherwise * only arise with badly timed concurrent DDL, for example. * * The default debug_discard_caches = 0 does no forced cache flushes. ``` Reading the ring is one call, handed two functions: one applies a message, the other does a full reset. 8. `SIGetDataEntries`, in [sinvaladt.c, lines 515–526](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/storage/ipc/sinvaladt.c#L515-L526): ```c if (stateP->resetState) { /* * Force reset. We can say we have dealt with any messages added * since the reset, as well; and that means we should clear the * signaled flag, too. */ stateP->nextMsgNum = max; stateP->resetState = false; stateP->signaled = false; LWLockRelease(SInvalReadLock); return -1; ``` A backend marked for reset jumps to the write position and returns -1, which its caller turns into a reset. *Code from PostgreSQL 18.3. Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group; Portions Copyright (c) 1994, The Regents of the University of California. Quoted under the [PostgreSQL Licence](https://www.postgresql.org/about/licence/).* **Code trail: When the ring fills up** 1. `SICleanupQueue`, in [sinvaladt.c, lines 617–632](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/storage/ipc/sinvaladt.c#L617-L632): ```c if (n < lowbound) { stateP->resetState = true; /* no point in signaling him ... */ continue; } /* Track the global minimum nextMsgNum */ if (n < min) min = n; /* Also see who's furthest back of the unsignaled backends */ if (n < minsig && !stateP->signaled) { minsig = n; needSig = stateP; ``` For each backend: mark it for reset if it blocks the space a writer needs, and otherwise note the furthest behind that has not been signalled. 2. `ProcessCatchupInterrupt`, in [sinval.c, lines 191–200](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/storage/ipc/sinval.c#L191-L200): ```c if (IsTransactionOrTransactionBlock()) { elog(DEBUG4, "ProcessCatchupEvent inside transaction"); AcceptInvalidationMessages(); } else { elog(DEBUG4, "ProcessCatchupEvent outside transaction"); StartTransactionCommand(); CommitTransactionCommand(); ``` How a signalled, idle backend catches up: read the ring inside its transaction, or run an empty one. 3. `ReceiveSharedInvalidMessages`, in [sinval.c, lines 128–139](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/storage/ipc/sinval.c#L128-L139): ```c * We are now caught up. If we received a catchup signal, reset that * flag, and call SICleanupQueue(). This is not so much because we need * to flush dead messages right now, as that we want to pass on the * catchup signal to the next slowest backend. "Daisy chaining" the * catchup signal this way avoids creating spikes in system load for what * should be just a background maintenance activity. */ if (catchupInterruptPending) { catchupInterruptPending = false; elog(DEBUG4, "sinval catchup complete, cleaning queue"); SICleanupQueue(false, 0); ``` The daisy chain: once caught up, run cleanup, which signals the next backend furthest behind. 4. `InvalidateSystemCachesExtended`, in [inval.c, lines 789–804](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/utils/cache/inval.c#L789-L804): ```c InvalidateCatalogSnapshot(); ResetCatalogCachesExt(debug_discard); RelationCacheInvalidate(debug_discard); /* gets smgr and relmap too */ for (i = 0; i < syscache_callback_count; i++) { struct SYSCACHECALLBACK *ccitem = syscache_callback_list + i; ccitem->function(ccitem->arg, ccitem->id, 0); } for (i = 0; i < relcache_callback_count; i++) { struct RELCACHECALLBACK *ccitem = relcache_callback_list + i; ccitem->function(ccitem->arg, InvalidOid); ``` The reset: every catalog cache and table description, then every callback with 0 or `InvalidOid`. *Code from PostgreSQL 18.3. Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group; Portions Copyright (c) 1994, The Regents of the University of California. Quoted under the [PostgreSQL Licence](https://www.postgresql.org/about/licence/).* **Code trail: A callback of your own** 1. `CacheRegisterSyscacheCallback`, in [inval.c, lines 1822–1833](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/utils/cache/inval.c#L1822-L1833): ```c if (syscache_callback_count >= MAX_SYSCACHE_CALLBACKS) elog(FATAL, "out of syscache_callback_list slots"); if (syscache_callback_links[cacheid] == 0) { /* first callback for this cache */ syscache_callback_links[cacheid] = syscache_callback_count + 1; } else { /* add to end of chain, so that older callbacks are called first */ int i = syscache_callback_links[cacheid] - 1; ``` A fixed array, `FATAL` when it is full, and no way to take an entry out again. 2. `CacheRegisterRelcacheCallback`, in [inval.c, lines 1857–1868](https://github.com/postgres/postgres/blob/REL_18_3/src/backend/utils/cache/inval.c#L1857-L1868): ```c void CacheRegisterRelcacheCallback(RelcacheCallbackFunction func, Datum arg) { if (relcache_callback_count >= MAX_RELCACHE_CALLBACKS) elog(FATAL, "out of relcache_callback_list slots"); relcache_callback_list[relcache_callback_count].function = func; relcache_callback_list[relcache_callback_count].arg = arg; ++relcache_callback_count; } ``` Ten slots per backend, shared with PostgreSQL's own caches. 3. `pgfdw_inval_callback`, in [connection.c, lines 1311–1330](https://github.com/postgres/postgres/blob/REL_18_3/contrib/postgres_fdw/connection.c#L1311-L1330): ```c /* hashvalue == 0 means a cache reset, must clear all state */ if (hashvalue == 0 || (cacheid == FOREIGNSERVEROID && entry->server_hashvalue == hashvalue) || (cacheid == USERMAPPINGOID && entry->mapping_hashvalue == hashvalue)) { /* * Close the connection immediately if it's not used yet in this * transaction. Otherwise mark it as invalid so that * pgfdw_xact_callback() can close it at the end of this * transaction. */ if (entry->xact_depth == 0) { elog(DEBUG3, "discarding connection %p", entry->conn); disconnect_pg_server(entry); } else entry->invalidated = true; ``` Hash 0 means everything; otherwise match the stored hashes. An idle connection closes now, a busy one at the end of its transaction. *Code from PostgreSQL 18.3. Portions Copyright (c) 1996-2026, PostgreSQL Global Development Group; Portions Copyright (c) 1994, The Regents of the University of California. Quoted under the [PostgreSQL Licence](https://www.postgresql.org/about/licence/).* :::unsure - How often resets happen on a busy server. I know of no statistics view that counts them; the only sign I have found is a `DEBUG4` log line, "cache state reset". - Why the ring holds 4,096 messages, and not a number sized from `max_connections` or a setting. I have not found the discussion that chose it. - What a long `REPEATABLE READ` transaction sees when another session adds a column before the first one touches the table. Its data snapshot is fixed but its catalog reads are not, and I want to trace that case end to end. ::: ============================================================================== # Full-page writes and torn pages Why the first change to a page after a checkpoint carries the whole 8 KB page in WAL. > Canonical URL: https://sudohulk.com/writing/full-page-writes-and-torn-pages > Type: note · Stream: Tech > Maturity: evergreen. I would defend this. Revisited at least once. > Published: 2026-09-22 > Tags: postgresql, databases > Reading time: 1 min > Author: Vignesh Kumar A > Licence: CC BY 4.0. Quote freely, cite the canonical URL. --- A **torn page** is what you get when the operating system writes only part of an 8 KB PostgreSQL page before power is lost: the disk sector size is smaller than the page, so half the page can be new and half old. Replaying a normal WAL record against such a page would corrupt it further, because the record assumes the page looked exactly like it did before the change. PostgreSQL's answer is the **full-page write** (also called a full-page image, FPI): the first time a page is modified after a checkpoint, the WAL record carries the entire page, not just the delta. On recovery, that copy simply overwrites whatever is on disk, torn or not. Later changes to the same page before the next checkpoint carry only deltas, because the full copy in the log has already made the page's starting state safe. :::gotcha This is why WAL volume spikes right after a checkpoint, and why spreading checkpoints out (`checkpoint_completion_target`) or checkpointing less often reduces WAL, at the cost of longer recovery. ::: `full_page_writes` can be turned off on storage that guarantees atomic page writes. Most people should not. Related: [[pd_lsn vs record LSN]]. ============================================================================== # Lazy transaction id assignment A transaction does not get an xid at BEGIN; it gets one the first time it writes. > Canonical URL: https://sudohulk.com/writing/lazy-xid-assignment > Type: note · Stream: Tech > Maturity: seedling. Just planted. May be wrong. > Published: 2026-09-22 > Tags: postgresql > Reading time: 1 min > Author: Vignesh Kumar A > Licence: CC BY 4.0. Quote freely, cite the canonical URL. --- `BEGIN` does not hand out a **transaction id** (xid). A transaction that only reads never gets one; it runs under a lightweight **virtual transaction id** made of the backend id and a per-backend counter. The real 32-bit xid is assigned lazily, on the first operation that needs to mark a tuple: an `INSERT`, `UPDATE`, `DELETE`, or a lock that has to be recorded. Xids are scarce, and they wrap around. Every assigned xid must eventually be frozen by vacuum to avoid wraparound, so handing them to read-only transactions would burn through the space and create work for no reason. :::gotcha `txid_current()` forces an xid to be assigned. `pg_current_xact_id_if_assigned()` (PG 13+) tells you whether one exists without creating it. ::: Related: [[pd_lsn vs record LSN]], and a post on what happens between BEGIN and COMMIT once it is written. ============================================================================== # pd_lsn vs record LSN Two LSNs that are easy to mix up, and the rule recovery uses to compare them. > Canonical URL: https://sudohulk.com/writing/pd-lsn-vs-record-lsn > Type: note · Stream: Tech > Maturity: budding. Checked once; the structure holds. > Published: 2026-09-21 > Tags: postgresql > Reading time: 1 min > Author: Vignesh Kumar A > Licence: CC BY 4.0. Quote freely, cite the canonical URL. --- Every WAL record has an address in the log stream: its **record LSN**. Every 8 KB data page also carries an LSN in its header, `pd_lsn`: the **page LSN**, which is the address just past the most recent WAL record already applied to that page. The rule that makes crash recovery idempotent: when replaying a record for a page, PostgreSQL applies it only if `pd_lsn` is older than the record's LSN. If the page already shows a newer LSN, that change reached disk before the crash and is skipped. :::gotcha An LSN is meaningful only inside one server. Two nodes' `1/2D3E` have nothing to do with each other. Never compare LSNs across machines. ::: Related: [[full-page-writes-and-torn-pages]] (once written). ============================================================================== # Decentralized P2P Network A peer-to-peer messaging network in Rust on libp2p: messages arrive in under 50 ms across 100+ nodes, with Ed25519 peer identity and Noise-protocol encryption. > Canonical URL: https://sudohulk.com/projects/p2p-network > Status: paused > Tags: rust, libp2p, ed25519, noise > Author: Vignesh Kumar A > Licence: CC BY 4.0. Quote freely, cite the canonical URL. --- ## What it does A network of peers that send messages straight to each other, with no server in the middle. Every peer has an identity of its own, and every connection is encrypted before the first message. Across more than a hundred nodes, a message arrives in under 50 milliseconds. ## Why I built it Years of building the server side of an offline-first product made me wonder how much of that server was needed. Everything at work went through a central service that decided what happened. I wanted to know what is left of the problem when there is no centre. ## How it works It is written in Rust on **libp2p**. Each peer holds an **Ed25519** key pair, and its public key is its identity, so a peer proves who it is with a signature instead of an account. Every connection runs the **Noise** protocol: the two peers authenticate each other and agree on keys before anything else is sent, and everything after that is encrypted. ## Status and what's next Paused. Code not public yet. ============================================================================== # Hybrid Storage Engine A key-value engine in C that keeps values in an append-only log and keys in a B-tree, so rewriting a large value does not churn the index around it. > Canonical URL: https://sudohulk.com/projects/hybrid-storage-engine > Status: paused > Tags: c > Author: Vignesh Kumar A > Licence: CC BY 4.0. Quote freely, cite the canonical URL. --- ## What it does Stores keys and values, and looks them up again. The difference is where the two live: values go to the end of a log file and are never moved; only a small pointer to each value goes into the index. ## Why I built it I kept reading the same argument (B-tree against LSM-tree, read amplification against write amplification) and kept not believing either side, because I had never felt it. Reading about **write amplification** tells you a large value can be rewritten many times over before it settles. Writing an engine that does it tells you how much that costs and where. Building one was the cheapest way to test the argument myself. ## How it works Two files. Values are appended to a **value log** and never updated in place; a write is a sequential append plus a pointer. Keys live in a **B-tree**, and each leaf holds the key and the offset of its value, nothing more. That is the WiscKey idea, with Bitcask's append-only log underneath it: the index stays small because it carries pointers rather than payloads, so the tree's pages hold far more keys and the tree stops being rewritten every time a big value changes. The cost moves somewhere else. A read is now two reads (tree, then log), and the log fills with values nothing points at any more, so it needs a garbage collector that walks it, asks the tree whether each entry is still live, and copies the survivors forward. Most of the design work is in that collector. ## What I learned That the LSM-tree-versus-B-tree question is badly framed. The real axis is whether the index has to move your data when your data changes, and separating the values from the keys is a third answer to that question rather than a point between the two. I also learned how much of RocksDB and LevelDB is compaction policy, which is obvious in hindsight and not obvious from the outside. ## Status and what's next It works, which is a low bar: it stores and returns keys and values, and it survives a restart. Not benchmarked against anything yet, and the garbage collector is the naive version that holds a lock for far too long. Next: crash-consistency tests before any performance numbers, because I cannot trust speed numbers from an engine that might lose writes. ============================================================================== # Smart Video Analysis Event-processing microservices for video analysis events over Kafka and PostgreSQL, deployed with Docker, Terraform and GitHub Actions. > Canonical URL: https://sudohulk.com/projects/smart-video-analysis > Status: paused > Tags: kafka, postgresql, docker, terraform, github actions > Author: Vignesh Kumar A > Licence: CC BY 4.0. Quote freely, cite the canonical URL. --- ## What it does Takes a stream of events emitted by a video-analysis pipeline, processes them through a small set of Kafka consumers, and stores the results in PostgreSQL for querying. ## Why I built it To practise the full loop I only saw parts of at work: infrastructure as code, a CI/CD pipeline that tests and deploys on every push, dashboards for the running system. Nothing about it is novel; the point was owning every layer. ## How it works Producers publish analysis events to Kafka. Consumer services, one per concern, write to PostgreSQL. Terraform provisions the containers and networking; GitHub Actions runs tests and redeploys. In a local benchmark the consumers kept up with roughly a thousand events per second, which was enough to find the first real bottleneck (batching writes). ## Status and what's next Paused. The code is not public yet; I want to strip the credentials handling and add the benchmark script before it is. ============================================================================== # learn-rust My working notes and exercises while reading The Rust Programming Language, chapter by chapter. > Canonical URL: https://sudohulk.com/projects/learn-rust > Status: paused > Tags: rust > Author: Vignesh Kumar A > Licence: CC BY 4.0. Quote freely, cite the canonical URL. --- ## What it is A repository of small programs, one per chapter of the Rust book, with comments on what surprised me coming from Java: ownership, lifetimes, and how much the compiler is willing to argue. ## Why Rust is the language I reach for when I want control close to C, with the compiler catching my memory bugs. Reading the book properly, with code for every chapter, was the way to stop pretending I knew it. ## Status Paused. The C I read now is at work, so the evenings go to writing instead. I will pick up the next chapters later. ============================================================================== # grokking-algorithms Practice problems worked through while reading Grokking Algorithms, in Java. > Canonical URL: https://sudohulk.com/projects/grokking-algorithms > Status: done > Tags: java > Author: Vignesh Kumar A > Licence: CC BY 4.0. Quote freely, cite the canonical URL. --- ## What it is Implementations of every algorithm in Aditya Bhargava's book, written from the pictures rather than copied from the text, then checked against the text. ## Why Algorithms were the part of computer science I skipped on the way to a job. This was the cheapest way to fill that gap without a course. ============================================================================== # Distributed PostgreSQL The internals of a horizontally scalable distributed PostgreSQL: sharding, distributed query execution, two-phase commit, and moving shards while the cluster serves. > Canonical URL: https://sudohulk.com/projects/distributed-postgresql > Status: active > Tags: c, postgresql, libpq, distributed systems > Author: Vignesh Kumar A > Licence: CC BY 4.0. Quote freely, cite the canonical URL. --- ## What it is A database engine that spreads one logical table across many machines. Rows are assigned to **shards** by a partitioning rule, shards are placed on worker nodes, a coordinator plans and executes queries across them, and a transaction that touches shards on more than one node commits with **two-phase commit** so the nodes cannot disagree about whether it happened. The same public space as Citus, if you want a reference point. ## What I work on Described at the level I can talk about publicly. The shard lifecycle: the operations that extend, retire and rebalance ranges while the cluster is serving traffic. I rewrote that layer, folding a few thousand lines of user-defined functions into one operation model with cutoff expressions and scheduling that knows which kind of operation it is running. Every operation now has a dry run: an operator sees what it would do to live cluster metadata before a single shard is mutated, so a wrong plan is caught before it changes anything. A read-only consistency verifier for the cluster: fifty checks over shard placement, colocation, replication residue, two-phase-commit state and node health, fanning out to every node asynchronously with a deadline per node so one sick node cannot hang the run. Each finding is fingerprinted, so the same defect keeps the same identity across scheduled runs instead of looking new every night. It is modelled on PostgreSQL's `amcheck`, and like `amcheck` it is allowed to read and nothing else. A logical-decoding output plugin that reads the write-ahead log and emits each row change as a JSON record, honouring the row filters and column lists a publication declares, with a background worker beside it reporting throughput per replication slot. Underneath all of it: the write-ahead log, MVCC and tuple visibility, heap and B-tree storage, distributed transaction recovery, shard pruning and distributed query planning. Mostly C, mostly under LLDB across a multi-node cluster. ## What I have learned That the interesting cases are all failure cases. A distributed commit is easy to describe and hard to recover; the useful findings from a consistency checker are the ones that stay failing for weeks until someone works out why; and a dry run is worth more than a rollback, because by the time you want a rollback the metadata has already moved. ## Status and what's next Current work. The generic PostgreSQL knowledge it forces me to learn ends up in the *PostgreSQL internals, explained simply* series; the product-specific design does not appear here. ============================================================================== # ZohoPOS sync platform The server side that keeps offline point-of-sale registers consistent with a finance platform, which I worked on for four years. > Canonical URL: https://sudohulk.com/projects/zohopos-sync-platform > Status: done > Tags: java, kafka, redis, mysql, websocket > Author: Vignesh Kumar A > Licence: CC BY 4.0. Quote freely, cite the canonical URL. --- ## What it was The server side of a point-of-sale product: sales processing, and the sync layer that pushes items, prices, stock and settings out to Android, iOS and Windows registers that spend part of their day with no network. ## What I worked on Described at the level I can talk about publicly. Every number here is on my [resume](/resume). - Replaced per-device polling and M×N direct API calls with one event per change over Kafka, cutting upstream API load by 99% and isolating the platform from outages in the finance system it syncs with. New sync consumers are declared in YAML and onboard in minutes rather than weeks; several teams picked the framework up. - A transactional outbox: the outgoing event is written inside the same MySQL transaction as the sale, with a background retry behind it, so a crash between "sale committed" and "event published" cannot happen. 99.9% consistency at more than a thousand transactions a minute. - Split a single-segment JVM and Redis cache into three isolated segments with version-based invalidation: hit rate from 84.3% to 98.1%, about 131 million fewer database queries a week. - A three-level circuit breaker (global, per-organisation, per-register) with Redis-backed state and a sliding window tuned from p97 latency. It opens within fifteen seconds and queues rather than failing, so downstream slowness degrades gracefully for customers running 200 or more registers. - Real-time push to over a thousand concurrent WebSocket endpoints over Kafka and Redis, so price and inventory changes reach a register without anyone pressing refresh. Data conflicts fell by 98%. - Correlation-ID tracing across asynchronous NIO callback threads, which turned three-day root-cause hunts into minutes for organisations processing 80 or more invoices a minute. - A config-driven access-control framework with bitwise permission storage, adopted across the product's APIs. ## What I learned That almost every bug I could not explain turned out to be a question about durability, isolation or ordering, and that the answers were a layer below the code I was writing. That is what sent me into database internals. Several of these stories will become posts in the series *Things a POS platform taught me*. ============================================================================== # PostgreSQL internals, explained simply One mechanism per post, in the order a change flows through the server. Analogy first, then the real thing, then the gotchas. > Canonical URL: https://sudohulk.com/tracks/pg-internals-simply > Kind: series > Status: active > Started: 2026-10-17 > Progress: 1/5 parts (20%) > Last activity: 2026-10-17 > Feed: https://sudohulk.com/tracks/pg-internals-simply/rss.xml > Tags: postgresql, databases > Author: Vignesh Kumar A > Licence: CC BY 4.0. Quote freely, cite the canonical URL. --- ## Parts - [ ] WAL, explained simply - [ ] What actually happens between BEGIN and COMMIT - [ ] Checkpoints and the redo point - [ ] Two-phase commit in vanilla PostgreSQL - [x] How PostgreSQL tells every connection its cache is stale, done 2026-10-17 ([How PostgreSQL tells every connection its cache is stale](https://sudohulk.com/writing/postgres-cache-invalidation)) ============================================================================== # Building sudohulk.com Why this site exists, and the story behind it. > Canonical URL: https://sudohulk.com/tracks/building-sudohulk > Kind: series > Status: active > Started: 2026-09-21 > Progress: 1/1 part (100%) > Last activity: 2026-09-21 > Feed: https://sudohulk.com/tracks/building-sudohulk/rss.xml > Tags: meta > Author: Vignesh Kumar A > Licence: CC BY 4.0. Quote freely, cite the canonical URL. --- ## Parts - [x] Hi, I'm Vignesh. Start here, done 2026-09-21 ([Hi, I'm Vignesh. Start here](https://sudohulk.com/writing/hello-sudohulk)) ============================================================================== # Hi, I'm Vignesh. Start here I'm a database engineer at Zoho, working on database internals. Why I write here in public, and how to use the site. > Canonical URL: https://sudohulk.com/writing/hello-sudohulk > Type: essay · Stream: Meta > Published: 2026-09-21 > Updated: 2026-10-17 > Series: Building sudohulk.com (part 1) > Tags: meta, career > Reading time: 3 min > Author: Vignesh Kumar A > Licence: CC BY 4.0. Quote freely, cite the canonical URL. --- ## Who I am I'm a database engineer at Zoho. I work on database internals: how data is stored, how queries run, and how a transaction stays correct when something fails. I learn by doing. I don't feel I understand something until I've built it and broken it. I haven't done anything big yet. One day I want to build something bigger and better. Until then I'll keep challenging myself, learning a lot and trying the things that make me happy, and this site is where that gets written down. ## Why this site At Zoho I wrote tech posts so other developers could follow the engineering work in our product. This site is the same thing, in public. If I think something might be useful to someone, it goes here. Each post explains one thing in plain words. When a post is about PostgreSQL, I usually point to where it lives in the source and what tends to go wrong. Most posts take a few minutes to read, and many end with links if you want to go further. A lot of what goes here took me a while to work out, and a post should save you some of that time. Writing a post also shows me what I hadn't understood, and if I still get something wrong, you can tell me. ## What you will find here - **Tech**, at [/tech](/tech): computer science, databases first. For now that means short notes on how PostgreSQL works inside, each marked with how far along it is. Longer explainers are on the way. - **Bookshelf**, at [/bookshelf](/bookshelf): the books and papers I have read, am reading or mean to read, with a line on why each one is there. - **Tracks**, at [/tracks](/tracks): the things I'm working through in public, each showing how far along it is. This note is the first part of one. More sections will appear once there is something to put in them. Everything is also in one list, on [Writing](/writing). ## How to use it - **It is made for your phone.** Light or dark is under the menu on a phone and behind the theme icon on a wider screen. By default it follows your device. - **Underline a post** at its end to say it was worth your time. No account and no cookies. It tells me what to write next. - **Comment without an account.** A name and an email address are optional. I read every comment before it appears. - **Follow along** with the [RSS feed](/rss.xml), or see the [newsletter page](/newsletter) for email. No algorithm decides whether you see a post. - **Link to a paragraph.** Every paragraph has its own link, the ¶ after its last word. A short link in one of my LinkedIn posts opens the post, and its "In 30 seconds" box takes you to the part each point is about. - **Read it with an AI assistant.** Add `.md` to a page's address to get the page as plain markdown. [For AI](/for-ai) has the rest. - **Tell me when I am wrong.** Write to [vignesh@sudohulk.com](mailto:vignesh@sudohulk.com) or use the [contact form](/contact). I add a correction above the text, leave the text as it was, and log the correction on [/changes](/changes). ## One boundary My work at Zoho is not mine to publish. What I write about databases is general knowledge and my own projects, never how our product does it. The opinions are mine. Pick something from [Writing](/writing), or start with [/tech](/tech). Thanks for coming over.