# Unimeter — complete documentation for LLMs > Open source usage metering engine. Record events, aggregate in real time, query billing totals in microseconds. Alternative to Lago and Amberflo. Source-available under the O'Saasy License (MIT-style, SaaS rights reserved). Throughput on a 3-node bare-metal cluster (AMD Ryzen 7 7700X · NVMe · private VLAN): 4.01M events/sec async (batch=500), 1.70M events/sec durable (fsync + quorum-replicated). Durable write latency: 171µs p50, 224µs p99, 250µs p99.9. Aggregate queries answer from memory in microseconds. Per-node throughput saturates at 8 workers — bound by disk, not cores; scale by adding nodes, not by sizing up a single box. --- # What is Unimeter If you just landed here from the [home page](/), welcome. This page is the longer introduction for anyone deciding whether Unimeter is the right tool for their situation. If you already know it is and want to get hands-on, jump to the [Quickstart](/quickstart/). If you sell software or services and need to charge customers based on what they actually use, you need a way to count that usage reliably. Every API call, every gigabyte transferred, every active seat, every minute of compute. Unimeter is the piece of software that does this counting for you. You send it events, one by one or thousands at a time. Unimeter stores them safely on disk, groups them by customer and by billing period, and keeps a running total that your application can ask about at any moment. When the month ends and you need to generate invoices, you query Unimeter and get exact numbers back in a few milliseconds. ## Why you might want this Building a usage metering system inside your own application sounds simple until you actually try. You need to handle millions of events without dropping any. You need to survive a server crash without losing the last hour of usage. You need to query totals quickly even when a single customer has generated ten million events this month. You probably want to alert yourself when a customer crosses a threshold, like exceeding their free tier. And you want all of this to keep working when your traffic grows. Unimeter exists so you do not have to build any of that yourself. It is designed from the ground up for high throughput and for correctness, which means your usage counts match reality and invoices come out right. ## What it gives you When you send an event to Unimeter, you get a receipt telling you it was stored safely. Events are persisted to disk and replicated to other nodes in your cluster before Unimeter confirms receipt, so a crash right after the confirmation cannot lose the event. When you ask how much a specific customer has used this month, Unimeter answers in about a millisecond because it keeps the running totals in memory. The system is clustered by default. You run three or more nodes, they coordinate automatically, and if one dies the others keep accepting events without dropping traffic. You do not need a separate database, message broker, or orchestration system. Unimeter is a single compiled binary that speaks a compact binary protocol over TCP. For your application code, you talk to Unimeter through a client library. The Go library exists today and handles all the networking details for you. You call `client.Ingest` to record events, `client.Query` to read back totals, and that is almost the entire surface area you need to know. ## Is this the right tool for your situation Unimeter is a good fit when your pricing depends on how much customers use your product and you need those numbers to be correct. Think of per-call API billing, per-gigabyte storage fees, seat-based subscriptions with mid-cycle changes, or anything similar. It also works well when you need usage data for internal dashboards and you want one source of truth for how much each customer is doing. It is probably not the tool you want if your product has a simple flat subscription and you only need to know whether the customer is active or not. A boolean in your existing database is cheaper. It is also not a full billing platform. Unimeter tells you how much was used, but you still need something like Stripe or a home-grown system to turn those numbers into invoices, charge cards, and manage subscriptions. The two fit together naturally though. ## Next step Head to the [Quickstart](/quickstart/) to run Unimeter on your machine and record your first event. It takes about five minutes. --- # Quickstart This guide gets Unimeter running on your machine, then walks through defining a metric, sending a handful of events, and reading the usage total back. By the end you will have a working integration you can adapt to your real application. ## What you need You need Docker installed to run the Unimeter server, and either Go or Python to run the client code. ## Start Unimeter Pull the published image and start a single-node instance. ```bash docker run -d --name unimeter \ --security-opt seccomp:unconfined \ -p 7001:7001 -p 9090:9090 \ ghcr.io/unimeter/unimeter:latest ``` Unimeter listens on port 7001 for application traffic and port 9090 for health checks. Verify it is alive by asking for its health status. ```bash curl http://localhost:9090/health ``` A successful response looks like `{"status":"ok","uptime_seconds":3}`. If you see that, Unimeter is running and ready to accept events. ## Set up the client Create a new project and install the Unimeter client library. #### Go ```bash mkdir billing-demo && cd billing-demo go mod init billing-demo go get github.com/unimeter/go-unimeter@latest ``` #### Python ```bash mkdir billing-demo && cd billing-demo python -m venv .venv && source .venv/bin/activate pip install unimeter-python ``` ## Define a metric A metric describes something you want to count, like API calls or bytes transferred. Before sending events, you tell Unimeter what the metric is called and how it should count. In this quickstart we define a metric called `api_calls` that simply counts one unit per event. #### Go Create a file called `main.go` with the following content. ```go package main import ( "context" "fmt" "log" billing "github.com/unimeter/go-unimeter" ) func main() { client, err := billing.New([]string{"localhost:7001"}) if err != nil { log.Fatal(err) } defer client.Close() ctx := context.Background() err = client.Metrics.Create(ctx, billing.MetricSchema{ Code: "api_calls", AggType: billing.AggCount, }) if err != nil { log.Fatal(err) } fmt.Println("metric created") } ``` Run it with `go run .`. You should see `metric created`. #### Python Create a file called `main.py` with the following content. ```python import asyncio from unimeter import AsyncClient, MetricSchema, AggType async def main(): async with AsyncClient(["localhost:7001"]) as client: await client.metrics.create(MetricSchema( code="api_calls", agg_type=AggType.COUNT, )) print("metric created") asyncio.run(main()) ``` Run it with `python main.py`. You should see `metric created`. If the metric already exists from a previous run, you can ignore the error; the call is safe to repeat. ## Record some events Now that the metric exists, send a few events. Three for account 1 and one for account 2. #### Go Replace the body of `main` with the following, keeping the imports and the client setup at the top. ```go events := []billing.Event{ {AccountID: 1, MetricCode: "api_calls", Value: 1}, {AccountID: 1, MetricCode: "api_calls", Value: 1}, {AccountID: 1, MetricCode: "api_calls", Value: 1}, {AccountID: 2, MetricCode: "api_calls", Value: 1}, } result, err := client.Ingest(ctx, events) if err != nil { log.Fatal(err) } fmt.Printf("stored %d events\n", result.NStored) ``` Run it with `go run .`. You should see `stored 4 events`. #### Python Replace the body of `main()` with the following. ```python async with AsyncClient(["localhost:7001"]) as client: from unimeter import Event result = await client.ingest([ Event(account_id=1, metric_code="api_calls", value=1), Event(account_id=1, metric_code="api_calls", value=1), Event(account_id=1, metric_code="api_calls", value=1), Event(account_id=2, metric_code="api_calls", value=1), ]) print(f"stored {result.n_stored} events") ``` Run it with `python main.py`. You should see `stored 4 events`. Unimeter persisted the events to disk, replicated them within the cluster (a cluster of one in this case), and updated its in-memory totals. ## Read the usage back Querying is the reverse of recording. You tell Unimeter which account and metric you care about, and for what period, and it returns the total. #### Go Add this at the end of `main`. ```go usage, err := client.Query(ctx, billing.QueryRequest{ AccountID: 1, MetricCode: "api_calls", Period: billing.CurrentMonth(), }) if err != nil { log.Fatal(err) } fmt.Printf("account 1 used api_calls %d times this month\n", usage.Value.Count) ``` Run it again with `go run .`. #### Python Add this inside the `async with` block. ```python from unimeter import current_month usage = await client.query(1, "api_calls", current_month()) print(f"account 1 used api_calls {usage.value.count} times this month") ``` Run it again with `python main.py`. The first three events from account 1 produced a total of three, matching what you sent. Account 2 sent one event, so querying for account 2 would return one. This is the full loop. You defined a metric, sent events, and got a total back. Everything else you might want to do with Unimeter builds on this pattern. ## Where to go from here Browse the [examples repository](https://github.com/unimeter/examples) for complete, runnable scenarios in Go and Python: SaaS API metering, seat-based subscriptions, infrastructure billing with dimension filters, high-throughput ingest, alert thresholds, and Stripe integration. To understand the different counting modes (counting, summing, taking the maximum, tracking unique values), read [Metric types](/concepts/metric-types/). To see how to tag events with dimensions like region or feature name and query them separately, read [Filters and dimensions](/concepts/filters/). When you are ready to run Unimeter in production with multiple nodes and failover, see [Running a cluster](/operations/cluster/). ## Clean up When you are done experimenting, stop Unimeter and remove its container. ```bash docker stop unimeter && docker rm unimeter ``` --- # Events and metrics Everything in Unimeter revolves around two ideas. An event is something that happened in your application, and a metric is a rule that tells Unimeter what to do with events of a certain kind. Once you understand these two, the rest of the system falls into place. ## What an event is An event is a record of one thing happening to one customer at one moment. A user made an API call. A virtual machine ran for another minute. Someone uploaded a file. Your service returned a response containing a certain number of bytes. Each of these is one event. When your application sends an event, it is telling Unimeter four things. Who did this, what did they do, how much of it was there, and when did it happen. The who is represented by an account identifier, which is a number you choose to mean a specific customer or workspace in your system. The what is a metric code, a short string like `api_calls` or `compute_seconds` that picks out which metric this event belongs to. The how much is a value, a whole number that means whatever the metric says it means. For counting API calls the value is usually one. For measuring bytes transferred the value is the number of bytes. The when is a timestamp in nanoseconds since the Unix epoch, though Unimeter fills this in automatically if you leave it blank. Events are append-only. Once stored, they are not edited or deleted. If you made a mistake and want to correct usage for a customer, you send a compensating event rather than going back and changing history. This makes auditing simple and keeps the system honest. ## What a metric is A metric is the definition of a thing you want to measure and the rule for how to measure it. Before you can send events with a certain metric code, you register that metric with Unimeter so it knows what the events mean. A metric schema has a few fields. The code is the identifier your events reference, like `api_calls`. The aggregation type tells Unimeter how to combine events for the same customer over a billing period; it might count them, sum their values, remember the maximum, and so on. Optional pieces include filters, which let you break usage down by tag, and alert thresholds, which let you notice when a customer crosses a line. ### Billing periods Every metric has a period that defines how wide each billing window is. Unimeter supports two modes. **Fixed periods** divide time into equal-length windows. You choose a duration like 30 days, and Unimeter computes which window an event belongs to by simple division. This is the default and works well for testing or for billing models that do not align with calendar months. **Calendar periods** follow the actual calendar. Each month is its own period regardless of whether it has 28, 30, or 31 days. You can also set a custom billing cycle day between 1 and 28. For example, if your billing cycle starts on the 15th of each month, an event on April 20 belongs to the April 15 – May 14 period, while an event on April 10 belongs to the March 15 – April 14 period. Calendar periods are the right choice for most production billing because they match what customers expect to see on an invoice. When you register a metric, Unimeter remembers it across restarts. The same metric is available to every node in your cluster. If you change the metric definition later by registering it again with different settings, future events use the new rules while past events stay as they were. ## How they fit together Think of a metric as a column heading on a spreadsheet and events as rows you add over time. The metric decides what kind of number goes in the column and whether new values add up, replace, or track the largest. When you ask Unimeter how much a customer has used, it looks at every event that matches the account and metric for the period you care about, applies the aggregation rule, and hands you back the result. A concrete walkthrough makes this clearer. Imagine a metric called `api_calls` set to count events with a monthly period. Customer 42 makes a hundred API calls on the first day of the month, so your application sends a hundred events each with account 42, metric `api_calls`, value 1. Unimeter counts them and remembers that customer 42 is at 100 for this month. Customer 42 makes another fifty calls on the second day, and the total moves to 150. When the billing system asks for customer 42's `api_calls` usage for the current month, Unimeter returns 150 immediately. At the start of the next month the counter resets to zero because it is a new billing period. ## What comes next The aggregation types cover more than just counting. Head to [Metric types](/concepts/metric-types/) to see the five available modes and when to use each. If you want to break totals down by things like region or feature, read [Filters and dimensions](/concepts/filters/). And to get warned when a customer crosses a free-tier boundary or a usage cap, see [Alerts](/concepts/alerts/). --- # Metric types Every metric in Unimeter has an aggregation type that tells the system how to turn a stream of events into one number you can put on an invoice. There are five of them, and each answers a slightly different business question. Most real products use more than one. ## Count Count is the simplest type and the one you reach for first. It looks at all the events for a customer in a period and returns how many there were. The value field of each event is ignored. If you send ten events, the count is ten. This is the right choice when you charge per occurrence. Per API request, per message sent, per document processed, per notification delivered. Anything where each event represents one billable thing and they all cost the same. If you want to charge different prices for different kinds of requests, you split them into separate metrics rather than trying to make count do more than it was meant to. A small trap to avoid: count ignores the value you send, so putting anything other than 1 in there is meaningless. Send events with value 1, or leave it zero, and let the count do its job. ## Sum Sum adds up the value of every event in the period. If one customer sends events with values 100, 250, and 50, the sum for that period is 400. This is the workhorse when your unit of billing is not a whole event but a quantity inside the event. Use sum when you charge by volume. Bytes transferred, tokens consumed, seconds of compute, cents of revenue, grams of coffee shipped. Your application measures the quantity, and sum gives you the total for the period. A subtle point that matters at scale. The running total is a 128-bit number under the hood, which means even if each event value is as large as a 64-bit integer can hold, summing billions of them will not overflow. You do not need to worry about this boundary. ## Max Max keeps only the largest value it has seen in the period. If you send events with values 10, 50, 30, 50 for a customer, the max is 50. Later events with smaller values do not change the result. Only an event with a value greater than the current maximum moves the needle. This is how you measure peaks. Peak number of concurrent users in the hour. Highest bandwidth seen in the day. Largest file uploaded in the month. Anything where the business cares about the top of the curve and not the average or total. Max is useful for tiered pricing where customers pay based on which band their peak fell into. You send a measurement event whenever a potential new peak happens and Unimeter keeps only the highest, which is exactly what you query at the end of the period. ## Latest Latest keeps only the most recent value, where most recent is decided by the event timestamp rather than the arrival order. If events arrive out of order for any reason, Unimeter compares timestamps and holds on to the one with the newest time. Events with older timestamps are stored in history but do not overwrite a newer reading. Use latest for gauges that describe the current state. Current seat count on a workspace. Current level of a prepaid credit balance. Current active plan tier. When you want to know what the state is right now rather than how much happened in total, latest is the type that fits. Because latest depends on event ordering by time, it is important that your timestamps are correct. If your application has clock drift between servers, consider having Unimeter assign the timestamp by sending a zero value. Unimeter stamps events at the moment of ingestion, which gives you a consistent ordering even when the originating services disagree on time. ## Count unique Count unique is count with a twist. Instead of counting every event, it counts how many distinct values it has seen. If you send events with values 1, 2, 2, 3, 3, 3, the count is 3 because there are three distinct values. This is what you want for questions like how many unique users logged in this month or how many distinct machines connected to the service. Count unique has one extra wrinkle. Each event carries an operation type that says whether it is adding a value to the set or removing one. An add event with a value you have already seen does nothing because the set already contains it. A remove event with a value you have seen takes it out of the set and the count goes down by one. This lets you model things like active seats, where users come and go and you need the current count to reflect departures as well as arrivals. A pattern that comes up often is this. When a user logs in, send a count unique add with their user id as the value. When they cancel or are removed, send a remove event with the same id. Ask Unimeter how many unique users are active, and the answer reflects both joins and leaves. ## Choosing the right type Think of it this way. If you want to know how many times something happened, use count. If you want to know how much of something was used, use sum. If you want the biggest single occurrence, use max. If you want the current state, use latest. If you want how many different things, use count unique. Most real products combine several types. An API product might count successful calls with one metric, sum the megabytes transferred with another, track peak concurrent connections with max, and count unique active API keys with count unique. Each metric is independent, and customers can have many metrics at once. You pick the right tool for each number you need. --- # Filters and dimensions Sometimes a single total per customer per metric is not enough. You want to know how much compute ran in each cloud region, or how many API calls came from each feature, or how much data was transferred in each product tier. Filters are how you answer questions like these without creating a separate metric for every combination. ## The idea in one paragraph When you register a metric, you can declare one or more dimensions. A dimension is a name for a tag, like `region` or `feature`. Each event you send can carry properties, which are name and value pairs attached to that specific event. If an event has a property whose name matches a declared dimension, Unimeter keeps a second running total just for that dimension value. Later you query with the same property name and value, and Unimeter returns the slice of usage that matched. ## A concrete example Imagine you run a platform that lets customers spin up compute workloads across three clouds. You track compute seconds, and you want to know at the end of the month how many seconds ran on AWS versus GCP versus Azure for each customer. You define a single metric called `compute_seconds` with aggregation type sum and one dimension called `provider` whose allowed values are `aws`, `gcp`, and `azure`. Now every event you send for this metric carries the workload's duration as its value and includes a property named `provider` with whichever cloud ran the job. Unimeter keeps two kinds of totals internally. There is the total across all providers, which is what you get when you query without specifying any filter. And there is a per-provider total, one for each value you tagged. When you query for `provider=aws`, Unimeter returns just the AWS slice. When you query for `provider=gcp`, you get the GCP slice. When you query without a filter, you get the sum of all three. This works because Unimeter computes a hash of the property name and value when the event arrives and uses that hash as part of the storage key. Events that share the same tag end up in the same bucket. Queries compute the same hash and look in the same bucket. The math lines up. ## Multiple dimensions on the same metric A metric can have more than one dimension. You might register `compute_seconds` with both `provider` and `region` dimensions, where region can be values like `us-east`, `us-west`, or `europe`. Events can carry both properties, and Unimeter keeps separate totals per dimension value. You can query `provider=aws` and get all AWS compute across all regions, or query `region=us-east` and get all US-East compute across all providers. ## AND queries across dimensions When an event matches two or more dimensions, Unimeter stores an additional combined aggregate alongside the per-dimension ones. This lets you query for `provider=aws AND region=us-east` and get back a single number representing only the events that matched both conditions at once. Under the hood, each per-dimension aggregate has a hash computed from the dimension name and value. The combined aggregate has a hash that is the XOR of all per-dimension hashes. The query computes the same combined hash and looks it up directly, so AND queries are just as fast as single-dimension queries. For an event that carries `provider=aws` and `region=us-east` on a metric with both dimensions declared, Unimeter creates four aggregates: the unfiltered total, the `provider=aws` slice, the `region=us-east` slice, and the `provider=aws AND region=us-east` combined slice. Each is updated in a single pass when the event arrives. If an event matches only one of the declared dimensions, no combined aggregate is created for that event. This keeps storage overhead minimal when events do not carry all possible properties. ## What happens to unrecognized tags If an event has a property whose name is not declared as a dimension on its metric, Unimeter ignores it. The event is stored and counted toward the overall total, but no per-property slice is created. This means you can attach arbitrary metadata to events for audit purposes without having to commit to slicing by it up front. When you do decide a tag is interesting, update the metric schema to include that dimension and future events will start contributing to the sliced total. ## When to use filters versus separate metrics Filters shine when the different slices share the same unit and the same pricing logic, and you just want to know the breakdown. They are much cheaper than defining five separate metrics and much less painful to query because one metric code gives you both the overall total and the slices. Separate metrics are the right answer when the business treats the categories as genuinely different things. Inbound bandwidth and outbound bandwidth might have different prices and be billed as distinct line items. In that case they are two metrics, not one metric with a direction dimension. ## What comes next See [Send events](/sdk/go/#send-events) for the exact syntax of attaching properties to events in your Go code. For defining metric schemas that include dimensions, read [Define metrics](/sdk/go/#define-metrics). --- # Alerts Usage-based pricing invites surprises. A customer might accidentally leave a job running and run up a bill they did not mean to. A free-tier account might cross the boundary where they start owing money, and you want to nudge them toward a paid plan before they hit a harder cap. Alerts are the mechanism Unimeter offers for noticing these moments as they happen. ## How an alert works When you register a metric, you can attach one or more thresholds to it. A threshold is a named number and a flag that says whether it fires repeatedly or only once. If the running total for any customer crosses that number, Unimeter flips a bit in that customer's aggregate to say so and writes a record to a durable log on disk. Every event ingestion checks the updated total against every threshold on the metric. If the new total is at or above a threshold value and the corresponding bit was not already set, or if the threshold is set to recur, Unimeter records a crossing. If a later event pushes the total back below the threshold, the bit is cleared so that a future recrossing is recorded again. Nothing is sent out of Unimeter automatically. There is no webhook today, no email, no push notification. What you get is state inside Unimeter that your application can read and act on. This is a deliberate design choice. By making alerts a queryable state rather than a push event, Unimeter stays simple, and your application controls exactly how to react and when. ## A worked example Imagine a free-tier SaaS where each workspace gets ten thousand API calls per month free, then pays per call above that, and is cut off at one hundred thousand to prevent runaway bills. You define the `api_calls` metric with count aggregation, a monthly period, and two thresholds. The first is named `free_tier_exceeded` at value 10000, non-recurring, so it fires the moment a customer first goes over their free allowance. The second is named `hard_cap` at value 100000, also non-recurring, so it fires the moment a customer hits the cap. When a customer crosses 10000 calls, their aggregate's alert flags gain a bit. Your billing system polls Unimeter on a timer or checks the flags during the next query, and when it notices the free_tier_exceeded bit is set, it sends a friendly email offering to upgrade. When the customer crosses 100000 calls, the hard_cap bit is set and your ingress layer can refuse further calls until the next period begins. ## Recurring versus non-recurring Most thresholds are non-recurring. You want to notice the first time a customer enters a state and react to that event, not get pinged every time another event arrives while they are still in the state. Non-recurring thresholds fire once per period, and reset only if the customer's total drops below the line and then climbs back up. Recurring thresholds are for the case where you want ongoing confirmation. If a customer is continuously over their quota, a recurring threshold emits a new log entry with every event that keeps them above the line. This is less common but sometimes useful for accounting purposes, when you want a precise record of each event that occurred while the customer was in breach. ## The alert log Unimeter writes every threshold crossing to a durable append-only file called the alert log. Each entry records which account crossed which threshold on which metric, the value at the moment of crossing, and the timestamp. Your application can read the log from an offset you remember, process entries you have not yet seen, and remember where you stopped. This is how you build reliable alerting on top of Unimeter even when your consumer restarts. The log is local to the node that ingested the event. In a multi-node cluster you read from each node and merge, or you consolidate in your own downstream system. A future release will expose the log over the protocol directly, but today it is accessible through the query API on the node that wrote it. ## Thresholds and filters Thresholds are set on the overall total for a metric, not on per-dimension slices. If you want to alert on a specific dimension value, like warning when AWS compute alone crosses a number, the current workaround is to create a separate metric for that slice. Per-dimension thresholds are a natural extension and are planned for a future release. ## Live delivery to your application Alerts are delivered to your code through a Go channel. When you subscribe, the Go client opens a live push stream with every node in the cluster, catches up any alerts that happened while your process was offline, and then forwards every new crossing to the channel in real time. This is covered in detail in [Alert subscriptions](/sdk/go/#alert-subscriptions). For batch use cases where a small delay is acceptable, you can also poll `client.ListAlerts` on a timer and consume the log at whatever cadence suits you. The two approaches read from the same underlying durable log, so the data is identical; the only difference is how quickly you learn about new crossings. ## What comes next See [Define metrics](/sdk/go/#define-metrics) for the syntax that attaches thresholds to a metric in your Go code. Once alerts are firing, learn how to consume them in real time with [Alert subscriptions](/sdk/go/#alert-subscriptions). --- # Go SDK The Go client library handles the network protocol, connection pooling, cluster routing, and retries for you. Your application gets `Ingest` and `Query` methods and nothing else to learn. This page covers the full surface area of the library. ## Install You need Go 1.22 or newer. Inside your Go module, add the client: ```bash go get github.com/unimeter/go-unimeter@latest ``` Import it under a name of your choice. The convention in our examples is `billing`: ```go import billing "github.com/unimeter/go-unimeter" ``` ## Connect A client is created by calling `billing.New` with a list of seed node addresses. The client connects, downloads the cluster topology, and opens persistent connections to every node. ```go client, err := billing.New([]string{"localhost:7001"}) if err != nil { log.Fatal(err) } defer client.Close() ``` For a multi-node cluster, list two or three seeds so the client has fallbacks if the first is unavailable at startup. The client will discover the rest through the partition map. The library is concurrency-safe. One `*billing.Client` should be shared across all goroutines in your service, initialized at startup, closed at shutdown. Every method that does network IO takes a `context.Context` as its first argument: ```go ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) defer cancel() usage, err := client.Query(ctx, billing.QueryRequest{ AccountID: 42, MetricCode: "api_calls", Period: billing.CurrentMonth(), }) ``` If your Unimeter is running in Docker on your laptop and your Go program is running on the host, use `localhost:7001` as the seed, assuming you published the port with `-p 7001:7001`. If your Go program is running inside another container on the same Docker network, use the container name as the host, like `unimeter:7001`. ## Define metrics Before sending events for a metric code, register it with the server. Schemas are durable and cluster-wide. ```go err := client.Metrics.Create(ctx, billing.MetricSchema{ Code: "api_calls", AggType: billing.AggCount, }) ``` If the metric already exists, the call returns `billing.ErrAlreadyExists`. In most applications you want to treat this as success: ```go err := client.Metrics.Create(ctx, schema) if err != nil && !errors.Is(err, billing.ErrAlreadyExists) { log.Fatal(err) } ``` ### Aggregation types ```go billing.AggCount // count events billing.AggSum // sum values billing.AggMax // keep the maximum billing.AggLatest // keep the most recent by timestamp billing.AggCountUnique // count distinct values ``` ### Dimension filters If you want to slice usage by a tag at query time, add filters to the schema: ```go err := client.Metrics.Create(ctx, billing.MetricSchema{ Code: "compute_seconds", AggType: billing.AggSum, Filters: []billing.DimensionFilter{ {Key: "provider", Values: []string{"aws", "gcp", "azure"}}, {Key: "region", Values: []string{"us-east", "us-west", "europe"}}, }, }) ``` ### Billing periods By default metrics use fixed 30-day windows. For calendar-month billing, set the period type: ```go err := client.Metrics.Create(ctx, billing.MetricSchema{ Code: "api_calls", AggType: billing.AggCount, PeriodType: billing.PeriodCalendar, }) ``` If your billing cycle starts on a day other than the first, set `BillingCycleDay` (1–28): ```go err := client.Metrics.Create(ctx, billing.MetricSchema{ Code: "api_calls", AggType: billing.AggCount, PeriodType: billing.PeriodCalendar, BillingCycleDay: 15, // period runs from the 15th to the 14th of the next month }) ``` Period helpers for queries: ```go billing.CurrentMonth() // 1st to 1st of next month billing.LastMonth() // previous calendar month billing.CurrentBillingPeriod(15) // current period starting on the 15th billing.LastBillingPeriod(15) // previous period starting on the 15th ``` ### Alert thresholds A metric can declare up to eight thresholds that fire when the running aggregate crosses them: ```go err := client.Metrics.Create(ctx, billing.MetricSchema{ Code: "api_calls", AggType: billing.AggCount, Thresholds: []billing.AlertThreshold{ {Code: "free_tier_exceeded", Value: 10_000, Recurring: false}, {Code: "hard_cap", Value: 100_000, Recurring: false}, }, }) ``` ### Listing and deleting ```go schemas, err := client.Metrics.List(ctx) for _, s := range schemas { fmt.Printf("metric %q aggregates with %s\n", s.Code, s.AggType) } err := client.Metrics.Delete(ctx, "api_calls") ``` ## Send events Events are sent in batches via `Ingest`. The client groups them by partition and fans out to the correct nodes in parallel. ```go result, err := client.Ingest(ctx, []billing.Event{{ AccountID: 42, MetricCode: "api_calls", Value: 1, }}) ``` `AccountID` picks the customer. `MetricCode` references the metric you registered. `Value` is the quantity. Timestamp is optional; if zero, the server stamps the event at ingestion time. ### Batching Batches are much more efficient than one-event-per-call. Accumulate events for a short window (ten to fifty milliseconds) and send as a single batch: ```go events := make([]billing.Event, 0, 1000) for _, call := range recentApiCalls { events = append(events, billing.Event{ AccountID: call.AccountID, MetricCode: "api_calls", Value: 1, }) } result, err := client.Ingest(ctx, events) ``` ### Attaching properties If the metric has dimension filters, attach properties to events: ```go event := billing.Event{ AccountID: 42, MetricCode: "compute_seconds", Value: billing.Scale(45.0), Properties: map[string]string{ "provider": "aws", "region": "us-east", }, } ``` The `billing.Scale` helper converts a float to the integer representation the wire protocol uses, keeping six decimal places. Use `billing.Unscale` to convert back. ### Operations for count unique When the metric type is `AggCountUnique`, each event says whether it is adding or removing a value: ```go events := []billing.Event{ { AccountID: 100, MetricCode: "active_seats", Value: uint64(userID), OperationType: billing.OperationAdd, }, { AccountID: 100, MetricCode: "active_seats", Value: uint64(departedUserID), OperationType: billing.OperationRemove, }, } ``` ### Async and sync delivery By default `Ingest` returns as soon as the server has accepted the batch into its buffer (async). For stronger guarantees, set `DeliveryMode` to `DeliverySync`: ```go event := billing.Event{ AccountID: 42, MetricCode: "one_time_payment", Value: billing.Scale(9.99), DeliveryMode: billing.DeliverySync, } ``` Sync delivery waits for fsync and replica acknowledgement. The cost is a few milliseconds of latency. Sync and async events can coexist in the same batch; if any event is sync, the whole batch is treated as sync. ### Handling the result ```go type IngestResult struct { NStored uint32 NDuplicates uint32 Offsets []uint64 } ``` Most applications only need to check `err` and move on. ## Query usage ### Real-time totals `QueryRealtime` returns the running total from in-memory state in about a millisecond: ```go agg, err := client.QueryRealtime(ctx, 42, "api_calls") fmt.Printf("account 42 has made %d calls this period\n", agg.Count) ``` The returned `AggValue` contains every statistic: `Count`, `Sum`, `Max`, `LastValue`, `LastEventAt`, `AlertFlags`. ### Specific period `Query` takes a period and returns the total for that range: ```go usage, err := client.Query(ctx, billing.QueryRequest{ AccountID: 42, MetricCode: "api_calls", Period: billing.CurrentMonth(), }) fmt.Printf("account 42 made %d calls in %s\n", usage.Value.Count, usage.Period.Start.Format("January 2006")) ``` Period helpers: `CurrentMonth()`, `LastMonth()`, `CurrentPeriod(duration)`, `CurrentBillingPeriod(cycleDay)`, `LastBillingPeriod(cycleDay)`. Or construct a `Period` by hand with `Start` and `End` times. ### Filtering by dimension Query a single dimension: ```go usage, err := client.Query(ctx, billing.QueryRequest{ AccountID: 42, MetricCode: "compute_seconds", Period: billing.CurrentMonth(), Filters: map[string]string{ "provider": "aws", }, }) ``` Query multiple dimensions at once (AND). Pass two or more entries in the `Filters` map and Unimeter returns only events that matched all of them: ```go usage, err := client.Query(ctx, billing.QueryRequest{ AccountID: 42, MetricCode: "compute_seconds", Period: billing.CurrentMonth(), Filters: map[string]string{ "provider": "aws", "region": "us-east", }, }) ``` ### Listing raw events ```go events, err := client.ListEvents(ctx, 42, startTime, endTime) for _, e := range events { fmt.Printf("%s: %s value=%d\n", e.Timestamp, e.MetricCode, e.Value) } ``` ### Consuming alert history ```go alerts, err := client.ListAlerts(ctx, 42, lastSeenOffset) for _, a := range alerts { fmt.Printf("%s: %s crossed %s at value %d\n", a.TriggeredAt, a.MetricCode, a.ThresholdCode, a.ValueAtCross) } ``` ## Alert subscriptions When a metric has thresholds, the server broadcasts every crossing to subscribed clients. The Go client turns those into a channel: ```go sub, err := client.Alerts.Subscribe(ctx, alerts.Filter{ AccountIDs: []uint64{42, 43, 44}, }) if err != nil { log.Fatal(err) } defer sub.Close() for alert := range sub.C { fmt.Printf("account %d crossed %s on %s (value %d)\n", alert.AccountID, alert.ThresholdCode, alert.MetricCode, alert.ValueAtCross) } ``` ### Resuming after a restart Persist offsets from `sub.Offsets()` and pass them back via `alerts.Filter.SinceOffset`: ```go lastSeen := loadOffsetsFromDB() sub, err := client.Alerts.Subscribe(ctx, alerts.Filter{ AccountIDs: []uint64{42, 43, 44}, SinceOffset: lastSeen, }) go func() { ticker := time.NewTicker(30 * time.Second) defer ticker.Stop() for range ticker.C { saveOffsetsToDB(sub.Offsets()) } }() ``` At-least-once semantics mean you might see the same alert twice after a crash. Make handlers idempotent. ### Backpressure The channel has a buffer of 64 alerts. If your consumer falls behind, the SDK increments a counter readable via `sub.Dropped()`. Dropped alerts are not lost; they stay in the node's alert log and appear on the next catchup. ### A concrete example: billing enforcement ```go var ( capsMu sync.RWMutex overCap = make(map[uint64]bool) ) func consumeAlerts(ctx context.Context, client *billing.Client) { sub, err := client.Alerts.Subscribe(ctx, alerts.Filter{}) if err != nil { log.Fatal(err) } defer sub.Close() for alert := range sub.C { switch alert.ThresholdCode { case "free_tier_exceeded": go sendUpgradeEmail(alert.AccountID) case "hard_cap": capsMu.Lock() overCap[alert.AccountID] = true capsMu.Unlock() } } } func apiHandler(w http.ResponseWriter, r *http.Request) { account := accountFromRequest(r) capsMu.RLock() capped := overCap[account] capsMu.RUnlock() if capped { http.Error(w, "usage cap exceeded", http.StatusPaymentRequired) return } client.Ingest(r.Context(), []billing.Event{{ AccountID: account, MetricCode: "api_calls", Value: 1, }}) } ``` ## Examples The [unimeter/examples](https://github.com/unimeter/examples) repository collects working Go programs. Clone it, point `UNIMETER_NODES` at a running server, and run any example. - **saas-api** — HTTP middleware that calls `client.Ingest` per request. [Source](https://github.com/unimeter/examples/tree/main/saas-api) - **infra-metering** — `DimensionFilter` on compute seconds by provider and region. [Source](https://github.com/unimeter/examples/tree/main/infra-metering) - **seat-based** — `AggCountUnique` with `OperationAdd` and `OperationRemove`. [Source](https://github.com/unimeter/examples/tree/main/seat-based) - **high-throughput** — Buffered flush pattern from a queue like Kafka or SQS. [Source](https://github.com/unimeter/examples/tree/main/high-throughput) - **free-tier-alerts** — Two thresholds with live subscription and local-cache enforcement. [Source](https://github.com/unimeter/examples/tree/main/free-tier-alerts) ## What comes next For the operational side of running Unimeter in production, see [Running a cluster](/operations/cluster/). For the Python client, which offers the same capabilities with an async API, see [Python SDK](/sdk/python/). --- # Python SDK The Python client library talks to Unimeter over the same binary protocol as the Go client. It is built on asyncio and exposes an `AsyncClient` that you use with `async`/`await`. This page covers the full surface area of the library. ## Install ```bash pip install unimeter-python ``` Requires Python 3.11 or newer. ## Connect Create a client with one or more seed node addresses. The client fetches the cluster topology on connect, opens persistent connections to every node, and refreshes the topology every thirty seconds in the background. ```python from unimeter import AsyncClient async with AsyncClient(["localhost:7001"]) as client: # client is ready to use ... ``` If you prefer explicit lifecycle management over a context manager: ```python client = AsyncClient(["localhost:7001"]) await client.connect() # ... use client ... await client.close() ``` For a multi-node cluster, list two or three seed addresses so the client has fallbacks if the first seed is unavailable at startup. ## Define metrics Before sending events for a metric code, register the metric with the server. Schemas are durable and cluster-wide. ```python from unimeter import ( MetricSchema, AggType, PeriodType, DimensionFilter, AlertThreshold, ) await client.metrics.create(MetricSchema( code="api_calls", agg_type=AggType.COUNT, )) await client.metrics.create(MetricSchema( code="compute_seconds", agg_type=AggType.SUM, filters=[ DimensionFilter(key="provider", values=["aws", "gcp", "azure"]), ], thresholds=[ AlertThreshold(code="free_tier", value=10_000), ], )) ``` ### Calendar billing periods By default metrics use fixed 30-day windows. For calendar-month billing: ```python await client.metrics.create(MetricSchema( code="api_calls", agg_type=AggType.COUNT, period_type=PeriodType.CALENDAR, billing_cycle_day=1, # 1-28; day of month when the period starts )) ``` Period helpers for queries: ```python from unimeter import current_month, last_month, current_billing_period, last_billing_period current_month() # 1st to 1st of next month last_month() # previous calendar month current_billing_period(15) # current period starting on the 15th last_billing_period(15) # previous period starting on the 15th ``` List all registered metrics: ```python schemas = await client.metrics.list() for s in schemas: print(f"{s.code}: {s.agg_type.name}") ``` Delete a metric by code: ```python await client.metrics.delete("api_calls") ``` ## Send events Events are sent in batches. The client groups them by partition and fans out to the correct nodes in parallel. ```python from unimeter import Event, DeliveryMode, scale await client.ingest([ Event(account_id=42, metric_code="api_calls", value=1), Event(account_id=42, metric_code="api_calls", value=1), Event(account_id=42, metric_code="api_calls", value=1), ]) ``` Attach properties for dimension filtering: ```python await client.ingest([ Event( account_id=42, metric_code="compute_seconds", value=scale(45.0), properties={"provider": "aws"}, ), ]) ``` For critical events that must be fsynced and replicated before acknowledgement, use sync delivery: ```python result = await client.ingest( [Event(account_id=42, metric_code="payment", value=scale(9.99))], delivery=DeliveryMode.SYNC, ) print(f"stored {result.n_stored} events") ``` For count unique metrics, set the operation type: ```python from unimeter import OperationType await client.ingest([ Event( account_id=100, metric_code="active_seats", value=user_id, operation_type=OperationType.ADD, ), ]) ``` ## Query usage Read the total for a customer in a billing period: ```python from unimeter import current_month, unscale result = await client.query(42, "api_calls", current_month()) print(f"API calls this month: {result.value.count}") ``` Query with a dimension filter: ```python result = await client.query( 42, "compute_seconds", current_month(), filters={"provider": "aws"}, ) print(f"AWS compute: {unscale(result.value.sum)} seconds") ``` Query with multiple dimensions (AND). Pass two or more keys and Unimeter returns only events matching all of them: ```python result = await client.query( 42, "compute_seconds", current_month(), filters={"provider": "aws", "region": "us-east"}, ) ``` For real-time dashboards, use `query_realtime` which reads from memory without touching disk: ```python agg = await client.query_realtime(42, "api_calls") print(f"current count: {agg.count}") ``` ## Error handling The client raises typed exceptions for different failure modes: ```python from unimeter import BackpressureError, ServerError try: await client.ingest(events) except BackpressureError: # Server disk is full. Back off and retry. await asyncio.sleep(1) await client.ingest(events) except ServerError as e: print(f"Server error: {e}") ``` Redirect responses from the cluster are handled transparently. If a node is no longer the leader for a partition, the client updates its routing cache and retries automatically. ## What comes next For the operational side of running Unimeter in production, see [Running a cluster](/operations/cluster/). For the Go client, which offers the same capabilities with a synchronous API, see [Go SDK](/sdk/go/). --- # Stripe integration Unimeter records and aggregates usage events in real time. Stripe handles invoicing and payments. Together they give you a complete usage-based billing pipeline without building aggregation logic yourself. This guide walks through a working example: your application ingests usage events into Unimeter, and when Stripe creates an invoice it queries the usage total and adds a line item automatically. ## How it fits together The integration has two sides. On the recording side, your application sends usage events to Unimeter every time a billable action happens. Unimeter aggregates them in real time and keeps a running total per customer per billing period. On the invoicing side, Stripe sends your server a webhook when it creates a new invoice. Your webhook handler takes the customer ID from the invoice, maps it to a Unimeter account, queries the usage total for the last completed billing period, and creates a line item on that invoice with the metered amount. Stripe then finalizes the invoice and charges the customer as usual. Unimeter owns the usage data. Stripe owns the billing. Your webhook handler is the bridge between the two, and it only runs when Stripe asks for the numbers. ## Prerequisites - A **Stripe account** with an API key and webhook signing secret. - A running **Unimeter cluster** (single-node is fine for development). - The Unimeter SDK installed: #### Go ```bash go get github.com/unimeter/go-unimeter@latest ``` #### Python ```bash pip install unimeter-python ``` ## Step 1: Define your metric Register a metric that counts API calls per calendar month. A `BillingCycleDay` of 1 means each period starts on the first of the month, matching the default Stripe subscription cycle. #### Go ```go client, _ := billing.New([]string{"localhost:7001"}) err := client.Metrics.Create(ctx, billing.MetricSchema{ Code: "api_calls", AggType: billing.AggCount, PeriodType: billing.PeriodCalendar, BillingCycleDay: 1, }) ``` #### Python ```python from unimeter import AsyncClient, MetricSchema, AggType, PeriodType async with AsyncClient(["localhost:7001"]) as client: await client.metrics.create(MetricSchema( code="api_calls", agg_type=AggType.COUNT, period_type=PeriodType.CALENDAR, billing_cycle_day=1, )) ``` You only need to do this once. The metric definition persists across restarts. ## Step 2: Record usage Call `Ingest` every time a billable action happens. A common pattern is an HTTP middleware that fires after each request. #### Go ```go func usageMiddleware(next http.Handler, client *billing.Client) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { next.ServeHTTP(w, r) accountID, _ := strconv.ParseUint(r.Header.Get("X-Account-ID"), 10, 64) if accountID == 0 { return } client.Ingest(r.Context(), []billing.Event{{ AccountID: accountID, MetricCode: "api_calls", Value: 1, }}) }) } ``` #### Python ```python from unimeter import Event async def usage_middleware(request, client): """Call after each request to record one API call.""" account_id = int(request.headers.get("X-Account-ID", "0")) if account_id == 0: return await client.ingest([ Event(account_id=account_id, metric_code="api_calls", value=1), ]) ``` Events are deduplicated by their idempotency key, so retries are safe. ## Step 3: Handle the Stripe webhook When Stripe finalizes an invoice it sends an `invoice.created` webhook. Your handler queries Unimeter for the customer's usage during the last billing period and creates a line item on the invoice. #### Go ```go package main import ( "encoding/json" "fmt" "io" "log" "net/http" "os" "strconv" billing "github.com/unimeter/go-unimeter" "github.com/stripe/stripe-go/v82" "github.com/stripe/stripe-go/v82/invoiceitem" "github.com/stripe/stripe-go/v82/webhook" ) // Map Stripe customer IDs to Unimeter account IDs. // In production, store this in your database or in Stripe customer metadata. var customerToAccount = map[string]uint64{ "cus_demo_alice": 1, "cus_demo_bob": 2, } func main() { stripe.Key = os.Getenv("STRIPE_SECRET_KEY") webhookSecret := os.Getenv("STRIPE_WEBHOOK_SECRET") client, err := billing.New([]string{os.Getenv("BILLING_NODES")}) if err != nil { log.Fatal(err) } defer client.Close() http.HandleFunc("/webhook", func(w http.ResponseWriter, r *http.Request) { body, _ := io.ReadAll(r.Body) event, err := webhook.ConstructEvent( body, r.Header.Get("Stripe-Signature"), webhookSecret, ) if err != nil { http.Error(w, "invalid signature", http.StatusBadRequest) return } if event.Type != "invoice.created" { w.WriteHeader(http.StatusOK) return } var invoice stripe.Invoice json.Unmarshal(event.Data.Raw, &invoice) accountID, ok := customerToAccount[invoice.Customer.ID] if !ok { w.WriteHeader(http.StatusOK) return } // Query last completed billing period. usage, err := client.Query(r.Context(), billing.QueryRequest{ AccountID: accountID, MetricCode: "api_calls", Period: billing.LastBillingPeriod(1), }) if err != nil { http.Error(w, "query failed", http.StatusInternalServerError) return } // Create a line item on the Stripe invoice. period := billing.LastBillingPeriod(1) idempotencyKey := fmt.Sprintf( "unimeter-%d-api_calls-%s", accountID, period.Start.Format("2006-01"), ) _, err = invoiceitem.New(&stripe.InvoiceItemParams{ Customer: stripe.String(invoice.Customer.ID), Invoice: stripe.String(invoice.ID), Description: stripe.String("API calls"), Quantity: stripe.Int64(int64(usage.Value.Count)), UnitAmount: stripe.Int64(10), // $0.001 per call Currency: stripe.String("usd"), Params: stripe.Params{ IdempotencyKey: stripe.String(idempotencyKey), }, }) if err != nil { log.Printf("stripe error: %v", err) } w.WriteHeader(http.StatusOK) }) log.Println("listening on :4242") log.Fatal(http.ListenAndServe(":4242", nil)) } ``` #### Python ```python import json import os import stripe from aiohttp import web from unimeter import AsyncClient, last_billing_period stripe.api_key = os.environ["STRIPE_SECRET_KEY"] webhook_secret = os.environ["STRIPE_WEBHOOK_SECRET"] CUSTOMER_TO_ACCOUNT = { "cus_demo_alice": 1, "cus_demo_bob": 2, } client: AsyncClient # initialized at startup async def webhook_handler(request: web.Request) -> web.Response: body = await request.read() sig = request.headers.get("Stripe-Signature", "") try: event = stripe.Webhook.construct_event(body, sig, webhook_secret) except stripe.error.SignatureVerificationError: return web.Response(status=400, text="invalid signature") if event["type"] != "invoice.created": return web.Response(status=200) invoice = event["data"]["object"] customer_id = invoice["customer"] account_id = CUSTOMER_TO_ACCOUNT.get(customer_id) if account_id is None: return web.Response(status=200) # Query last completed billing period. period = last_billing_period(1) usage = await client.query( account_id, "api_calls", period, ) # Create a line item on the Stripe invoice. idempotency_key = f"unimeter-{account_id}-api_calls-{period.start:%Y-%m}" stripe.InvoiceItem.create( customer=customer_id, invoice=invoice["id"], description="API calls", quantity=usage.value.count, unit_amount=10, # $0.001 per call currency="usd", idempotency_key=idempotency_key, ) return web.Response(status=200) async def on_startup(app: web.Application) -> None: global client nodes = os.environ["BILLING_NODES"].split(",") client = AsyncClient(nodes) await client.connect() async def on_cleanup(app: web.Application) -> None: await client.close() app = web.Application() app.on_startup.append(on_startup) app.on_cleanup.append(on_cleanup) app.router.add_post("/webhook", webhook_handler) if __name__ == "__main__": web.run_app(app, port=4242) ``` The important mapping is between Stripe customer IDs and Unimeter account IDs. In the example this is a hardcoded map; in production, store the Unimeter account ID in Stripe customer metadata or in your database. The idempotency key includes the account, metric, and period start date. If Stripe retries the webhook, the invoice item creation is skipped because Stripe has already seen that key. ## Step 4: Run and test Set the required environment variables and start the server: #### Go ```bash export STRIPE_SECRET_KEY="sk_test_..." export STRIPE_WEBHOOK_SECRET="whsec_..." export BILLING_NODES="localhost:7001" go run main.go ``` #### Python ```bash export STRIPE_SECRET_KEY="sk_test_..." export STRIPE_WEBHOOK_SECRET="whsec_..." export BILLING_NODES="localhost:7001" python main.py ``` In a separate terminal, forward Stripe test webhooks to your local server using the Stripe CLI: ```bash stripe listen --forward-to localhost:4242/webhook ``` Create a test invoice in the Stripe dashboard or via the API and watch the webhook handler query Unimeter and attach a line item. ## What's next - [Go SDK reference](/sdk/go/) — full API documentation for the Go client. - [Python SDK reference](/sdk/python/) — full API documentation for the Python client. - [Events and metrics](/concepts/events-and-metrics/) — how events are recorded and aggregated. - [Metric types](/concepts/metric-types/) — COUNT, SUM, MAX, LATEST, and COUNT UNIQUE. - [Alerts](/concepts/alerts/) — get notified when usage crosses a threshold. --- # Running a cluster Running Unimeter in production means running more than one node. A single node is fine for development and staging, but for the real thing you want the guarantees that come from replication. This page covers what a production cluster looks like, how to operate it, and what to watch. ## Why run multiple nodes Unimeter is the system of record for how much your customers used your product, which means the cost of losing events is real money. A single server can lose its events if the disk fails, if the kernel panics at the wrong moment, or if the VM it runs on is terminated. None of these are rare in practice over a long enough timeline. A cluster of three or more nodes solves this by writing every event to a quorum of nodes before acknowledging the client. If one node dies, the remaining two have the data and keep serving requests. When the dead node comes back, it catches up from the others. At no point does a committed event vanish, even if the timing of the failure is unlucky. Three is the practical minimum because replication requires a majority to acknowledge. With two nodes a single failure leaves you without a majority and writes stall. Five or seven nodes gives you more failure tolerance at the cost of higher replication overhead, but three is the sweet spot for most deployments. ## What nodes do Every node in a Unimeter cluster runs the same binary with the same code. There is no leader process separate from worker processes, no coordinator that has to be running for others to work. Each node is responsible for a slice of the accounts and serves writes and reads for that slice directly. When a node receives a write for an account it does not own, it replicates the write to the owner and acknowledges only after the owner confirms. Account ownership is determined by hashing the account identifier and taking the remainder when divided by the number of partitions. A cluster has 256 partitions by default, and these are distributed across the nodes evenly. With three nodes each owns about 85 partitions. When a node dies, ownership of its partitions transfers to surviving nodes automatically, and clients learn about the change on the next request. This design means the Go client never talks to a proxy or a coordinator. It holds a cache of which node owns each partition and sends each request directly to the right node. The cache updates automatically when ownership changes. This is why your application code does not need any retry or reconnection logic; it is all in the client library. ## A production layout A typical production cluster is three Unimeter nodes running on three separate machines. Each node has its own disk for data, its own network interface, and its own Prometheus scrape target. The nodes are configured with each other's addresses so they can replicate events. From outside the cluster, your application's Unimeter clients are given all three addresses as seeds, so they can connect to whichever is reachable. The binary listens on two TCP ports. The application port, default 7001, speaks the binary protocol that the Go client uses. The replication port, which is the application port plus 1000, is for inter-node traffic. If your nodes are on a private network, only the application port needs to be exposed to the outside world. The replication port stays internal. An additional HTTP port, default 9090, serves health checks at `/health` and Prometheus metrics at `/metrics`. Put this behind your internal monitoring infrastructure. The HTTP port is not for application traffic; your application should always use the binary protocol through the Go client for ingest and query. ## Starting nodes Each node is the same `unimeter` binary started with flags that tell it who it is and who its peers are. You pick a node identifier, a data directory for its local state, the TCP port for the application protocol, and a list of peer addresses for replication. ```bash unimeter \ --node-id=0 \ --port=7001 \ --data-dir=/var/lib/unimeter \ --peers=1:node1.internal:8002,2:node2.internal:8003 ``` The replication port is conventionally the application port plus 1000, so node 0 listening on 7001 replicates on 8001. Each peer entry takes the form `node_id:host:replication_port`. Start the same binary on each machine with its own node identifier and data directory, and the cluster forms automatically once the processes can reach each other. Run the binary under a supervisor that restarts it on exit. Systemd, Kubernetes, or whichever orchestrator you normally use is fine. Unimeter has no special requirements beyond a writable data directory and network reachability to its peers. ## Monitoring Every node exposes a Prometheus scrape endpoint and a health check endpoint on its HTTP port, default 9090. Point your existing monitoring infrastructure at each node and the rest is normal Prometheus. The full list of metrics, what they mean, and suggested alerts is on the [Metrics and observability](/operations/observability/) page. ## Handling failures A node crash is not a crisis. The cluster continues to serve writes and reads, and clients do not need to be restarted. The surviving nodes take over the dead node's partitions and keep working. When the crashed node restarts, it rejoins the cluster and catches up from its peers. A disk failure on one node is similarly contained. The cluster continues serving traffic with the two remaining nodes. When you replace the disk, the new node resyncs its state from its peers. This works because every event is replicated across a majority of the cluster before acknowledgement, so losing one copy never loses the event. A network partition that splits the cluster is the trickiest case. If the partition leaves a majority on one side, that side keeps working and the minority side is unable to commit new writes. When connectivity is restored the minority side catches up. If the partition is a clean split into two equal groups, neither side has a majority and writes stall until the partition heals. This is a safety feature, because allowing both sides to continue accepting writes would lead to divergent state that cannot be reconciled. ## Upgrading Upgrades are done one node at a time. Stop a node, deploy the new binary, start the node, wait for it to rejoin and catch up, then move to the next. The cluster continues serving traffic during the whole process because the other nodes are still up. There is no requirement to run all nodes at the same version simultaneously; a mixed-version cluster is supported as long as the new version is at most one minor version ahead of the old. Before you upgrade, check the release notes for any specific migration instructions. Durable state on disk is forward-compatible within a major version, so you should not need to export and re-import data for routine upgrades. ## What is not yet in place A few operational conveniences are on the roadmap. Node rebalancing, meaning adding a node and having partitions automatically redistribute, requires a manual partition-map update today. Authentication using API keys is not yet enforced by the server, which means anyone who can reach the application port can send events. Rate limiting by customer is similarly unimplemented. Treat Unimeter the way you would treat an internal Postgres: put it behind your own edge and use network policy to control who can reach it. ## What comes next If you are curious about how Unimeter achieves its throughput and correctness claims, read [How it stays fast](/operations/architecture/) for the design rationale. --- # Metrics and observability Unimeter is opinionated about observability. Every node exposes a Prometheus scrape endpoint with the counters, gauges, and histograms needed to answer the questions that matter in production. The metric set is small on purpose. Instead of dozens of numbers that nobody reads, you get the dozen or so that actually tell you whether the system is healthy and where it hurts when it is not. ## Where metrics come from Every Unimeter node listens on a separate HTTP port, default 9090, for monitoring traffic. Two endpoints are exposed. Health checks live at `/health` and return a small JSON object with a status field and the process uptime in seconds. Prometheus metrics live at `/metrics` and return the standard text exposition format. Both endpoints are cheap to hit and safe to scrape as often as your collector prefers. Point your Prometheus server at each Unimeter node's HTTP port with a normal scrape job. A fifteen-second scrape interval works well for most deployments. Once the metrics are in Prometheus you can visualise them with whichever dashboarding tool your organisation already uses. Grafana is the most common choice, but the metrics are plain and well-documented Prometheus primitives, so any compatible backend works. ## What the metrics tell you ### Traffic volume The headline number is how many events are being ingested. `billing_events_ingested_total` is a counter with a `mode` label that is either `async` or `sync` depending on the delivery mode the client requested. Taking the per-second rate over this counter gives you events per second. The async side should be the overwhelming majority of traffic in most deployments; a sudden shift to sync usually means a client library change somewhere worth investigating. `billing_events_duplicate_total` counts events that the deduplication ring rejected because it had seen the same content recently. A small trickle of duplicates is normal and comes from client retries after timeouts. A large jump usually means a client is stuck in a retry loop, and the number here is how you notice. ### Latency Three histograms describe how long operations take, all measured in seconds. `billing_ingest_async_duration_seconds` tracks the time from request arrival to async acknowledgement. This is typically well under a millisecond because async does not wait for disk. `billing_ingest_sync_duration_seconds` tracks the sync path and includes the fsync and replica acknowledgement, so it is slower by a few milliseconds at p50. `billing_wal_sync_duration_seconds` isolates the fsync alone, which is useful for diagnosing when slow disk is the culprit behind a latency regression. Use these with Prometheus `histogram_quantile` to compute p50, p95, and p99 over any window. Reasonable alert thresholds are p99 under 10 milliseconds for async ingest and under 50 milliseconds for sync ingest. Sustained breaches of either usually mean either disk pressure or network congestion between replicas. ### Storage activity `billing_wal_writes_total` counts the number of write-ahead log append operations completed. Each ingest batch produces one append, so this is roughly proportional to ingest rate divided by batch size. `billing_wal_syncs_total` counts fsync completions, which happen for sync deliveries and at periodic flush intervals. `billing_wal_offset_bytes` is a gauge showing the current write offset in the log, which grows monotonically as events accumulate on disk. Watching `billing_wal_offset_bytes` per node tells you how fast you are burning through disk space. Compare the growth rate to available space and you have a rough runway estimate. ### Connection counts Two gauges track active connections. `billing_connections_active` is the number of open TCP connections on the binary protocol port, which is where application clients land. `billing_http_connections_active` is the number of open connections on the HTTP port, which is where monitoring tools land. Both should be a small number relative to your client pool size. A steady climb suggests a client is leaking connections and not closing cleanly. ### Cluster health `billing_view_changes_total` counts the number of times a node became leader of a partition. Every node increments its own counter when it takes over. In a healthy stable cluster this counter barely moves after startup, because leaders do not churn. A sudden increase is how you learn that something is making nodes flap, whether that is network problems, disk stalls, or a crash loop. For clusters with active-passive replication patterns you would also watch for asymmetry. If one node's view change counter is stuck at zero while the others keep climbing, that node is probably unreachable from a quorum of peers. ### Alert subscriptions Three metrics describe the live alert push flow. `billing_alert_subscribers` is a gauge of how many TCP connections currently have push enabled on this node. `billing_alerts_recorded_total` counts every threshold crossing appended to the alert log, including crossings that replicas record for durability. `billing_alerts_pushed_total` counts frames queued for live subscribers, which only the leader increments. In a healthy system the ratio of pushed to recorded should be about one per subscriber on the leader side. If recorded keeps growing while pushed stays flat, your subscribers may have dropped off without reconnecting. If subscribers is zero when you expect it to be non-zero, either no client is subscribed or the nodes are serving connections that never sent `ALERT_PUSH_ENABLE`. ## Quick reference These are the full metric names for easy copy-paste into Prometheus queries. | Name | Type | What it means | |------|------|---------------| | `billing_events_ingested_total{mode}` | counter | Events successfully persisted. `mode` is `async` or `sync`. | | `billing_events_duplicate_total` | counter | Events rejected because the dedup ring already had them. | | `billing_wal_writes_total` | counter | WAL append operations completed. | | `billing_wal_syncs_total` | counter | WAL fsync operations completed. | | `billing_view_changes_total` | counter | Times this node became leader of some partition. | | `billing_connections_active` | gauge | Open TCP connections on the binary protocol port. | | `billing_http_connections_active` | gauge | Open connections on the HTTP monitoring port. | | `billing_wal_offset_bytes` | gauge | Current WAL write offset in bytes. | | `billing_alert_subscribers` | gauge | Connections with live alert push enabled. | | `billing_alerts_recorded_total` | counter | Threshold crossings durably appended to the alert log. | | `billing_alerts_pushed_total` | counter | Alert frames queued for live subscribers (leader only). | | `billing_ingest_async_duration_seconds` | histogram | Async ingest latency in seconds. | | `billing_ingest_sync_duration_seconds` | histogram | Sync ingest latency in seconds. | | `billing_wal_sync_duration_seconds` | histogram | WAL fsync latency in seconds, isolated from the ingest path. | ## Suggested alerts A minimal alerting setup focuses on the signals that predict outages or user pain. A page-worthy alert fires when sync ingest p99 crosses 100 milliseconds for more than five minutes, because something is wrong with either disk or replication. A warning alert fires when `billing_view_changes_total` increases by more than three in a fifteen-minute window, because that points to a flapping cluster. A separate warning alert watches `billing_connections_active` against a static ceiling you choose based on your expected client pool, to catch runaway connection growth. These are starting points. Tune the thresholds to what your infrastructure actually does on a good day, then alert on deviations from that baseline rather than absolute numbers. ## A minimal scrape configuration A Prometheus job for a three-node Unimeter cluster looks like the following. Adjust the targets to match your deployment. ```yaml scrape_configs: - job_name: unimeter scrape_interval: 15s static_configs: - targets: - node0.internal:9090 - node1.internal:9090 - node2.internal:9090 ``` That is the entire integration. The metrics names do not clash with any other common exporter, so no relabeling is required. ## What comes next To see how these metrics connect to the overall design, read [How it stays fast](/operations/architecture/). If you are still getting a cluster up, see [Running a cluster](/operations/cluster/). --- # Operational runbook This page collects the procedures you need when operating Unimeter in production. Each section is a self-contained recipe you can follow under pressure without reading the rest of the documentation first. ## Backup Unimeter stores all durable state in the data directory. A backup is a copy of that directory. The files are append-only or atomically replaced, so you can copy them while the server is running without stopping traffic. The files that matter are the write-ahead log (`wal.log`), the segment files (`*.seg`, `*.idx`, `*.props`), the checkpoint (`checkpoint.bin`), the metric registry (`metric_registry.bin`), the alert log (`alert_log.bin`), and the partition map (`partition_map.bin`). Everything else is derived from these. ```bash rsync -a /var/lib/unimeter/ /backup/unimeter/$(date +%Y%m%d)/ ``` Run this on each node independently. In a three-node cluster you get three independent backups, any two of which are sufficient to restore the full dataset because every committed event exists on at least two nodes. ## Restore Stop the server, replace the data directory with the backup, and start the server again. ```bash systemctl stop unimeter rm -rf /var/lib/unimeter/* cp -a /backup/unimeter/20260415/* /var/lib/unimeter/ systemctl start unimeter ``` On startup Unimeter loads the checkpoint, replays the write-ahead log from the checkpoint offset, and rebuilds the in-memory aggregates. The server is ready to accept traffic once the replay completes, which typically takes a few seconds even for large datasets. If you are restoring a single node in a running cluster, the node will catch up any events it missed via replication from the other nodes after it rejoins. ## Scaling: adding a node Adding a fourth node to a three-node cluster requires three steps. First, start the new node with the correct `--node-id`, `--peers`, and an empty data directory. The node will connect to its peers but will not own any partitions yet. ```bash unimeter \ --node-id=3 \ --port=7001 \ --data-dir=/var/lib/unimeter \ --peers=0:node0.internal:8001,1:node1.internal:8002,2:node2.internal:8003 ``` Second, trigger a rebalance from any client. The rebalance command tells the cluster to redistribute partitions across the new node count. The server transfers the relevant event data to the new node automatically. Third, update your application's seed list to include the new node address so that new client connections can discover it. ## Scaling: removing a node To remove a node, trigger a rebalance with the reduced node count first. This moves the departing node's partitions to the remaining nodes. After the rebalance completes, stop the departing node and remove it from the `--peers` lists of the remaining nodes on their next restart. ## Failover Failover is automatic. When a leader node becomes unreachable, the replicas detect the absence of heartbeats within two seconds and run a view change to elect a new leader. Clients discover the change through redirect responses and update their partition map cache transparently. No manual intervention is required. You can verify that failover succeeded by checking the `billing_view_changes_total` Prometheus counter. A single increment per affected partition is normal. Sustained increments indicate flapping, which usually points to network instability between nodes. ## Rolling restart Restart nodes one at a time. Wait for each node to rejoin and catch up before restarting the next. ```bash for node in node0 node1 node2; do ssh $node "systemctl restart unimeter" sleep 10 # wait for rejoin and catchup curl -sf http://$node:9090/health || echo "WARNING: $node not healthy" done ``` The PodDisruptionBudget in the Helm chart enforces this automatically in Kubernetes: at least two out of three pods remain available during a rolling update. ## Disk space emergency When free disk space drops below 256 MB, the server stops accepting new events and returns a backpressure status to clients. Existing data is not affected. To recover, free space by reducing `--retention-days` and restarting the server. The retention sweep runs hourly and deletes segment files older than the configured threshold. Alternatively, move old segment files to another volume manually. Monitor `billing_wal_offset_bytes` growth rate to predict when you will run out of space. ## Retention policy Unimeter deletes segment files and evicts aggregate data older than `--retention-days` (default 90). The sweep runs once per hour. After deletion, a new checkpoint is written so that the freed memory is not reconstructed on restart. The alert log is not rotated automatically. It grows at roughly 40 bytes per threshold crossing, which is negligible for most deployments. If it grows large, you can safely truncate it while the server is stopped. ## Startup validation On startup the server runs integrity checks before accepting traffic. It verifies that the data directory is writable, validates the WAL's CRC chain, checks the checkpoint's magic number and checksum, and scans segment files for size consistency. Corrupt checkpoints are fatal; the server will not start. WAL corruption is handled by truncating at the last valid entry and logging a warning. ## Graceful shutdown Sending SIGTERM or SIGINT to the server triggers a graceful shutdown. The server stops accepting new connections, waits up to five seconds for in-flight sync writes to complete, saves a final checkpoint, flushes the WAL, and exits. The log message "graceful shutdown complete" confirms that all data was persisted. --- # Benchmarks Reproducible benchmarks on bare-metal servers. 3-node cluster with a dedicated Go load generator on a private VLAN. ## Throughput | Scenario | batch=500 | batch=1 | |----------|-----------|---------| | Async | **4.01M** evt/s | **266K** evt/s | | Sync | **1.70M** evt/s | **24.1K** evt/s | Async delivery responds before data is fsynced to disk. Sync delivery waits for WAL fsync and replication acknowledgement before responding — every event is durable on disk when the client receives "ok". ## Latency Round-trip latency for a single durable sync write, measured end-to-end from the load generator. | Percentile | Latency | |------------|---------| | p50 | **171 µs** | | p90 | 198 µs | | p99 | 224 µs | | p999 | 250 µs | ## Scaling Async throughput with batch=500, varying the number of concurrent workers. | Workers | Events/sec | |---------|-----------| | 1 | 1,442K | | 2 | 2,554K | | 8 | **4,011K** | | 16 | 4,034K | | 64 | 3,942K | The cluster saturates at 8 workers with approximately 4.0M events per second. Each node runs a single-threaded event loop — throughput scales with the number of nodes, not CPU cores. ## Hardware All numbers were collected on Cherry Servers (US-Chicago, hourly billing): - **CPU**: AMD Ryzen 7 7700X, 8 cores / 16 threads, 5.6 GHz boost - **RAM**: 64 GB DDR5 - **Disk**: 2× Micron 7450 NVMe, enterprise grade - **Network**: private VLAN, approximately 64 µs RTT between nodes - **OS**: Ubuntu 22.04, kernel 6.8 ## Reproduce Benchmark scripts and the Go load generator are available in the [benchmarks repository](https://github.com/unimeter/benchmarks). ```bash cd benchmarks make up # provision servers, build, start cluster make bench # run all scenarios with batch=500 make bench-single # run batch=1 scenarios make clean # destroy servers and stop billing ``` The [benchmarks README](https://github.com/unimeter/benchmarks#readme) covers setup, configuration, and scenario details. --- # How it stays fast Unimeter is meant to handle the event rates that real production billing systems throw at it, which is sometimes hundreds of thousands of events per second from a single customer during traffic peaks. It is also meant to never lose a committed event. Those two goals normally pull in opposite directions, because durability means syncing to disk and disk is slow. This page explains how we reconciled them, so you can judge whether the design fits your situation. ## One binary, written in Zig, no dependencies Unimeter is a single statically linked executable. It does not pull in a database, a message broker, or an orchestration framework. Everything it needs is compiled into the binary, which is a few megabytes and has no runtime dependencies beyond a modern Linux kernel. The language choice is Zig. We wanted a compiled, systems-level language with explicit memory management and no hidden control flow, which rules out Go and JVM languages. We wanted something that compiles quickly and produces a predictable binary, which ruled out Rust for our taste. Zig fits, and its standard library has first-class support for the Linux primitives we rely on. Being dependency-free has practical benefits beyond simplicity. There is no PostgreSQL to upgrade, no Kafka cluster to operate, no version-skew concerns between internal services. You run one binary on some machines and monitor it. That is the whole operational surface. ## Events never touch a database The hot path of Unimeter, the place where an event spends its milliseconds from arrival to acknowledgement, never talks to a database. It talks to the kernel through io_uring and to a few hundred megabytes of carefully laid out in-process memory. That is it. When an event arrives, it is appended to a write-ahead log on disk through an asynchronous io_uring submission. While the disk I/O is in flight, the event is also added to a running aggregate in memory, which is a hash map keyed by account and metric. When the kernel confirms the disk write, the client gets an acknowledgement. The whole path is zero-allocation on the critical path; buffers are preallocated at startup and reused forever. In-memory aggregates are what make reads fast. Every time a new event updates the running total for a customer, the in-memory record is updated in place. Answering a query is an O(1) hash lookup that returns the current total in about a microsecond. There is no aggregation query, no scan, no join. ## Write-ahead log plus checkpoints The in-memory aggregates are fast but volatile. If the process crashes, the memory is gone. The write-ahead log is what makes them durable. Every event is appended to the log with a checksum chain before the client is acknowledged, and the log is the source of truth. On restart, Unimeter replays the log to rebuild the in-memory aggregates and then continues serving traffic. To keep startup fast even after the log has grown large, Unimeter periodically writes a checkpoint to disk. A checkpoint is a snapshot of all the in-memory aggregates at a specific log position. On startup, Unimeter loads the latest checkpoint and then replays only the events that arrived after it. This keeps recovery time proportional to the amount of recent activity rather than the full history of the system. The log and checkpoints together mean that a crash never loses a committed event and never requires a long rebuild. In our tests, a server with a year of production data typically restarts and resumes serving within a few seconds. ## Replication through a proven protocol A single node's log can still be lost to hardware failure, so Unimeter replicates every event to a quorum of cluster nodes before acknowledging the client. The replication protocol is Viewstamped Replication, the same algorithm used in TigerBeetle and similar in spirit to Raft. We chose it because it is well-studied, it handles leader failure cleanly, and its correctness properties have been proven formally. In practice this means that every event your application sends is written to at least two out of three disks before your `client.Ingest` call returns when you use sync delivery. With async delivery the acknowledgement comes slightly earlier and the replication happens in the background within milliseconds, which is usually fine for usage metering and gives you higher throughput. Replication also gives you leader failover. If the leader for a partition crashes, the remaining nodes run an election and one of them becomes the new leader within a second or two. Clients notice the change through a redirect response and update their partition map automatically. No manual intervention is needed. ## Smart client, no gateway A lot of distributed systems solve the client-side routing problem by putting a proxy in front of the cluster. Every write and every read goes through a proxy that knows the topology. This adds a network hop and a point of failure, and it means the proxy itself needs to be load-balanced, highly available, and operated. Unimeter takes the approach used by ScyllaDB, Cassandra, and Redis Cluster instead. The client library is smart. It downloads the cluster topology on startup, keeps it cached, and sends each request directly to the node that owns the relevant data. When the topology changes, the cluster pushes an update to every connected client and they refresh. The client handles redirects gracefully, so even if a node's cached topology is stale, the worst case is one extra roundtrip. The benefit is dramatic. There is no proxy tier to operate. There is no extra network hop between your application and the storage node. Your application talks to Unimeter with one TCP roundtrip per request, and that roundtrip goes directly to the node that has the data in memory. ## Testing with a deterministic simulator Distributed systems are hard to test because failures that matter are rare in practice and happen in bad combinations. Unimeter is tested with a deterministic simulator (VOPR) that runs the full cluster logic in a single process with a seeded random number generator that controls when nodes crash, when messages are dropped, when the network partitions, and when disks corrupt writes. For every seed, the simulator checks a set of invariants after each simulated step. The invariants cover the properties that matter. No two nodes ever think they are leader for the same partition in the same view. A committed event is never lost after a view change. Aggregate sums always equal the sum of events that fed them. Filtered totals match the events that carry the matching tag. Running the simulator for ten thousand iterations across several seeds takes a minute and exercises millions of operations under chaos conditions. The simulator runs on every change to the code. A commit that breaks an invariant is caught immediately, well before reaching production. This is the main way we get to a system that does what it claims under all the awkward conditions that distributed systems get into. ## What we gave up No system gets everything. To keep Unimeter fast and simple, we chose not to support a few things that systems aimed at different tradeoffs would include. There is no ad-hoc query language. You cannot write SQL against Unimeter. The query surface is small and targeted at the billing use case: totals by account, metric, and period. For analytical queries over raw events, export the events to a system built for that and query there. There is no storage engine to tune. Unimeter keeps everything that matters in memory, so it is fast but the footprint scales with the number of unique customer-metric-period combinations you have. At a million customers times ten metrics, you use about half a gigabyte of memory, which is fine. At a hundred million customers you would feel the pressure and probably want a different system. There is no multi-region replication today. A Unimeter cluster is assumed to live within one datacenter or one availability zone. Multi-region active-active is conceptually straightforward given the replication protocol but it is not yet implemented. If your billing system spans continents, Unimeter is currently not the answer for the global layer. ## Ready to operate If you have read this far, you know enough about Unimeter's design to decide whether it fits. It is fast because the hot path avoids every avoidable cost. It is correct because every event is durable, replicated, and tested against a simulator. It is simple because it is one binary and it does one job. To put it into production, review [Running a cluster](/operations/cluster/). To understand how your application talks to it, start with the [Quickstart](/quickstart/). --- # External guides - Meter LLM usage like Anthropic — tokens, per-model rates, weekly and monthly caps: https://dev.to/0x180db/meter-llm-usage-like-anthropic-tokens-models-weekly-and-monthly-caps-4302 ---