Layers of sediment holding still fragments of data, with a branching network rising out of the top layer and lighting up as it grows.archiveLiving Software · 01
Living Software · 01

Oliver: a Living Database.

Databases were built as archives, where data settles into layers and waits for someone to dig. Oliver senses what arrives, reacts when it changes and remembers what it learns, fast enough to have reflexes.

The Oliver team26 October 20265 min read

Fossils don't answer back

Most databases are dig sites. Data arrives, settles into layers and waits. Nothing happens until somebody turns up with a question, which is to say a shovel, and starts digging. A dashboard is a museum label for something dug up earlier, and every investigation is an excavation that starts from bare ground.

That design made sense when the people asking were few and their questions were slow. A fossil record is a fine thing to own. It just cannot tell you what is happening now.

Nothing alive waits to be asked

Living things work the other way round. They sense what reaches them, react when something changes and remember what they learn, so the next encounter costs less. A living database should do the same, and Oliver is built to.

Start with the senses. When a new table arrives, Oliver works out what each field means: which one names a service, which one is a duration, which log line says a connection pool has run dry. Nobody has to write that down for it.

Knowing what the data means, it can react to it. When something drifts from normal, Oliver raises the alarm once for the incident rather than once a minute for the rest of the day, and several alarms about the same failure arrive as one.

What it learns, it keeps. Oliver holds on to what it has worked out about a table, so the next question about that data starts from knowledge instead of from a shovel.

Meet Cartographer

The part of Oliver that does the sensing has a codename for now: Cartographer. Point it at a table and it draws a map of what the data means, then uses that map to watch the data and explain what it sees. It works in four steps.

  1. ReadSamples the table until new fields stop turning up, including the ones buried inside JSON and log text.
  2. NameGives every field a meaning. Standard fields by rule, unfamiliar ones by asking a small model once each.
  3. WatchSets reflexes on what matters, such as errors, latency and traffic, and fires once per incident.
  4. ExplainWhen a reflex fires, follows the evidence one step at a time until it reaches a cause.
The study loop. The map it draws is kept, so the next table from the same system starts already named.

Naming is where most of the work goes. Fields that follow the observability standard are recognised by rule, with no model involved. The custom fields every team adds are the interesting part, and those are where a small model is asked one question per field.

Field as it arrivesWhat Cartographer calls itHow
service.namethe serviceby rule
http.response.status_codethe outcomeby rule
durationlatencyby rule
attrs.cart.itemsa measure: items per cartby judgment
attrs.payment.providera category to slice byby judgment
attrs.cache.hita yes-or-no flagby judgment
attrs.retry.after_msa measure, in millisecondsby judgment
Naming the fields. Every standard field was named by rule; custom fields were named correctly about 98 times in 100.

Log text gets the same treatment. Cartographer reads a sample, masks the parts that change from line to line, and keeps the shapes that are left. Each shape becomes something it can count and watch, and a model labels each shape once with what kind of event it describes.

PoolExhaustedError: no connection after 5003ms (pool=main, waiting=37) PoolExhaustedError: no connection after 5011ms (pool=main, waiting=41) charge ok order=88213 amount=41.20 in 212ms PoolExhaustedError: no connection after 5007ms (pool=main, waiting=39) charge ok order=88214 amount=9.99 in 198ms
PoolExhaustedError: no connection after <duration> (pool=<id>, waiting=<n>)resource exhausted
charge ok order=<id> amount=<number> in <duration>normal
Log lines into shapes (example lines). A day of logs was mapped from about one line in ten, and every line found its shape.

With fields named and log shapes labelled, Cartographer can explain an incident instead of just announcing one. Here is the incident planted in a full day of test traffic, from the first alarm to the cause.

  1. First signErrors start rising on the payment service.
  2. Reflex firesPayment errors at 19 times their normal rate. Four alarms over the next minutes arrive as one incident.
  3. What kind of problem? small model
    What the model readsPayment errors: 34.5% of requests, up from 1.5% the hour before. Most common error: PoolExhaustedError. Requests: in line with the hour before. Version: unchanged.
    resourcedependencytrafficchangedatanone
    A resource ran out.
  4. Which resource? small model
    pool exhaustedtimeoutslow querymemory or CPUclient errornoise
    The connection pool. The log shape PoolExhaustedError surges on payment during the incident.
  5. One incident, one causePayment, on POST /v1/charge, called by the API gateway: connection pool exhausted, first sign 10:12. Worked out in under a minute.
An incident, explained. The engine runs every check and writes the evidence out in words; a small model picks one answer from a short list at each step, and the engine decides where to look next. The model that answers is small enough to run on a laptop.

Fast enough to have reflexes

Being alive is mostly a matter of timing. A reflex that arrives a minute late describes the injury instead of preventing it.

That is why the rest of this blog has been about speed: charts that draw in under 10 milliseconds at any zoom, full-text search up to 714 times faster, thousands of questions answered in one pass on a GPU. Those are the nerve fibres. A living database is only possible on an engine that answers before the moment has passed.

The brain only gets the hard questions

Your hand leaves a hot stove before your brain has heard about it. The reflex runs through the spinal cord, which is why it is fast, and the brain is kept for the questions that need meaning.

Oliver divides the work the same way. The engine handles the reflexes: watching, counting, comparing and grouping. A model is consulted only for judgment, such as what an unfamiliar field means or what kind of failure a log line describes, which is why the payment incident above took under a minute to explain.

What frontier models want to work on

Nobody trained a frontier model so that it could write joins. Give a model an archive and it becomes an archaeologist, guessing where to dig and sifting through whatever comes back. Give it a living database and it gets to do what it is actually good at, which is judgment.

That is the other half of being a Frontier Database™. Oliver runs on the hardware frontier models run on, and it behaves the way they need their world to behave, noticing things on its own and bringing them only the questions worth thinking about.