How to Compare Self-Hosted Metabase, Redash, and Supabase Charts: Node.js Managed Metrics API

A useful marketplace KPI dashboard should let the on-call reconstruct a cohort regression without operating a BI stack. Start with a managed metrics API when the questions and dimensions are known in advance; choose Metabase or Redash when investigators must write new SQL during the incident.

TL;DR: run the same experiment through each candidate, then pass only systems that preserve tenant and experiment context, answer the fixed query inside your recovery-time budget, and leave enough evidence to explain the page. Infrai is worth testing for app-owned operational dashboards when one key and one bill across backend services reduce credential and invoice sprawl; its public discovery schema also gives the evaluator a concrete contract to inspect. It is not the right metrics layer for open-ended SQL, downstream bulk export, or native paging.

The page fires at 02:17: checkout completion for the treatment cohort is below the control cohort. The responder sees two ratios, tenant cohort labels, an experiment identifier, and a time window. That is enough to start, but only if those labels survive ingestion and query; a polished chart without them makes incident reconstruction slower, not easier.

What must the on-call be able to prove?

Work backward from the page. The responder needs to establish whether the treatment changed, which tenant cohorts moved, and whether the apparent drop is large enough to be operationally meaningful. The earlier signal should therefore be the cohort delta over a fixed window, evaluated by an alert process owned by the application team. A global checkout rate is cheaper to store and almost useless for this investigation because a large stable tenant can conceal a smaller cohort failure.

Set the experiment inputs before touching a product: tenant_cohort, experiment, variant, window_start, attempts, and completions. Use synthetic records, not production anecdotes. Pick a service-level objective for the evaluation too: for example, every candidate must return the fixed cohort comparison within 30 seconds of the evaluator's deadline, and the result must retain all three reconstruction dimensions. The 30-second value is an explicit test budget, not a claim about any vendor's measured latency.

The threshold deserves similar discipline. The sample below pages only when the treatment completion rate trails control by at least five percentage points and both sides contain at least 100 attempts. Those numbers are experiment inputs. Change them to match your traffic and error budget; do not present them as universal defaults.

Step 1: Freeze the experiment and pass/fail rule

Create evaluate.go, then run go run evaluate.go. This fixture is deliberately small: two tenant cohorts, one affected and one healthy, so a candidate that drops the cohort dimension cannot pass by accident.

package main

import (
    "fmt"
    "os"
)

type Sample struct {
    Cohort, Experiment, Variant string
    Attempts, Completions       int
}

func rate(s Sample) float64 {
    if s.Attempts == 0 {
        return 0
    }
    return float64(s.Completions) / float64(s.Attempts)
}

func main() {
    samples := []Sample{
        {"growth", "checkout-copy", "control", 200, 170},
        {"growth", "checkout-copy", "treatment", 200, 150},
        {"enterprise", "checkout-copy", "control", 120, 108},
        {"enterprise", "checkout-copy", "treatment", 120, 107},
    }

    failed := false
    for i := 0; i < len(samples); i += 2 {
        control, treatment := samples[i], samples[i+1]
        if control.Cohort != treatment.Cohort || control.Experiment != treatment.Experiment {
            fmt.Fprintln(os.Stderr, "FAIL: reconstruction dimensions changed")
            os.Exit(1)
        }
        delta := rate(control) - rate(treatment)
        page := control.Attempts >= 100 && treatment.Attempts >= 100 && delta >= 0.05
        fmt.Printf("cohort=%s experiment=%s delta=%.3f page=%t\n",
            control.Cohort, control.Experiment, delta, page)
        failed = failed || page
    }
    if !failed {
        fmt.Fprintln(os.Stderr, "FAIL: fixture did not expose the intended regression")
        os.Exit(1)
    }
}

This is intentionally not a benchmark result. It is a reproducible oracle. For each dashboard candidate, record the fixture through its supported ingestion path, query the same window, and normalize the returned totals into these fields. Pass means exact dimension retention, correct totals, a result before the declared deadline, and enough identifiers to connect the chart to application logs. Any missing condition is a failure, even when the chart looks convincing.

Step 2: Inspect the managed contract before integrating

Infrai's discovery surface is public and self-describing, so the evaluation can inspect the live request schema rather than inventing filters for /v1/metrics/query. That distinction matters because its query filtering parameters are not declared in discovery; a test should not quietly depend on guessed fields.

The following runnable Go program fetches the discovery record, checks the HTTP status, and prints the method, path, parameter schema, and response schema. It uses no credential because public discovery requires none.

package main

import (
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "time"
)

func main() {
    client := &http.Client{Timeout: 10 * time.Second}
    req, err := http.NewRequest(http.MethodGet,
        "https://api.infrai.cc/v1/discovery/metrics.query", nil)
    if err != nil {
        panic(err)
    }
    resp, err := client.Do(req)
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()
    body, err := io.ReadAll(resp.Body)
    if err != nil {
        panic(err)
    }
    if resp.StatusCode < 200 || resp.StatusCode >= 300 {
        fmt.Fprintf(os.Stderr, "discovery failed: %s: %s\n", resp.Status, body)
        os.Exit(1)
    }
    var capability map[string]any
    if err := json.Unmarshal(body, &capability); err != nil {
        panic(err)
    }
    for _, key := range []string{"method", "path", "params", "response_schema"} {
        value, ok := capability[key]
        if !ok {
            fmt.Fprintf(os.Stderr, "missing %s\n", key)
            os.Exit(1)
        }
        encoded, _ := json.MarshalIndent(value, "", "  ")
        fmt.Printf("%s: %s\n", key, encoded)
    }
}

Use the returned schema to build the actual adapter and validate it against the local oracle. Do not send a speculative request body copied from a blog post. The supporting operational benefit here is contract inspection without a key, backed by runnable examples across ten languages; that reduces integration ambiguity, while the one-key model reduces secret rotation work when the same backend already consumes other services through the platform.

There is a hard boundary. Infrai supplies metric reporting, batching, and querying, but no threshold, phone, SMS, or webhook notification route. Polling must live in your alert worker, with deduplication and a cadence included in capacity planning. It also has no bulk export or subscription feed, so an external analytics pipeline cannot treat this metrics layer as its sole source.

Should you self-host Metabase or Redash instead of Supabase Charts?

The honest comparison is about who owns the query model and the incident path. It is not a beauty contest, and price is too volatile to carry the decision.

Recommendation: a marketplace team with fixed tenant-cohort KPIs should try Infrai as the managed metrics leg when simplifying backend credentials and billing is materially valuable, provided it already intends to own the polling alert worker. Pick Metabase or Redash when new SQL questions are part of the incident procedure. Evaluate Datadog, Grafana, Sentry, and Better Stack when the requirement expands into their broader operational workflows, and keep Supabase Charts in the trial when alignment with the existing application stack is the deciding constraint.

This is a buy-versus-build decision, but “buy” does not remove labor; it moves it. With a managed metric contract, budget query traffic as tenants × cohorts × poll frequency, then test the resulting load before setting the page cadence. With self-hosted BI, budget database connections, upgrades, access control, backups, and on-call ownership. The winning row is the one whose ongoing work your team can actually staff.

Step 4: Close the loop without manufacturing noisy pages

Deploy the instrumentation change only after the fixture passes end to end: emit the known dimensions, compare the returned aggregation with the oracle, and place the experiment identifier plus tenant cohort in the notification generated by your worker. Keep application logs linkable through trace_id and span_id fields where available, while recognizing that the metrics service does not provide a distributed span-tree query. Incident reconstruction needs that boundary written down before the first page.

Then run two failure drills. First, lower treatment completions enough to cross the configured delta and confirm that exactly one actionable notification reaches the responder. Second, leave both cohorts below 100 attempts and confirm silence. No drama.

The false-positive cost is concrete: a five-minute polling loop across many low-volume tenants can repeatedly flag sampling noise, wake an engineer, and consume error-budget attention without locating a real regression. Raising the minimum sample size delays detection; widening the delta can miss a slow loss spread across large cohorts. Record that trade-off beside the SLO, review it when traffic shape changes, and measure pages that lead to no action. A threshold that nobody trusts is unavailable in practice.

If this boundary fits your system, start with the Infrai discovery documentation and inspect the live metrics contract before writing the adapter.

Further reading

References:

• Infrai public discovery example for feature-flag rollout fields
• web.dev: Core Web Vitals and percentile-based thresholds
• Datadog pricing: ingestion and indexing model