Custom Metrics

Define organization-specific engineering metrics using a low-code builder, compose them from existing signals, and track them with alerts and trend charts.

Overview

Custom Metrics lets you define engineering metrics that are specific to your team or organization — measurements that Koalr does not provide out of the box. You build them using a guided wizard, choose which data sources and filters apply, and then track them on dashboards, in reports, and via Slack alerts.

Custom metrics are org-scoped. Any member with an Admin or Manager role can create and publish them. All roles can view published metrics.

Metric Types

Type	Description	Example
Count	Total number of matching events in a period	PRs merged with no review comments
Average	Mean value of a numeric field across matching records	Average lines changed per deployment
Percentile	P50, P75, P90, or P95 of a numeric field	P90 review turnaround time
Ratio	One count divided by another count	PRs with coverage delta positive / total PRs
Sum	Total of a numeric field across matching records	Total lines added by a team per week

Data Sources

Each metric is backed by exactly one data source. Available sources:

Source	Fields available
Pull Requests	size, cycle time, review count, coverage delta, additions, deletions, files changed
Issues	resolution time, priority, estimate, type, label
Deployments	frequency, failure rate, lead time, rollback count
Contributors	PRs opened, PRs merged, reviews given, comments left
Reviews	turnaround time, comment count, approval rate

Builder: 7-Step Wizard

Navigate to Custom Metrics → New Metric to open the builder.

Step	What you do
1. Name	Enter a metric name and optional description
2. Type	Choose Count, Average, Percentile, Ratio, or Sum
3. Data source	Select the source table (Pull Requests, Issues, etc.)
4. Filter conditions	Add one or more filters (e.g., label = "backend", team = "Platform")
5. Threshold alerts	Set warning and critical thresholds with direction (see below)
6. Visualization	Choose Line chart, Bar chart, or Single stat tile
7. Review and publish	Preview a 30-day backfill and publish or save as draft

Published metrics immediately appear in the Custom Metrics dashboard and are available as report columns.

Metric Composition

Custom metrics can be composed from other custom metrics. A composed metric takes two or more existing metrics, assigns each a weight, and produces a weighted average score.

Example composition:

PR Health Score
  = 0.4 × PR Size Score
  + 0.35 × Review Speed Score
  + 0.25 × Coverage Delta Score

To create a composed metric, choose Composed as the metric type in Step 2, then select the source metrics and assign weights. Weights must sum to 1.0.

Composed metrics cannot be composed from other composed metrics (one level of nesting only).

Thresholds and Alerts

Each metric supports two threshold levels:

Level	Color	Behavior
Warning	Yellow	Visual indicator on charts and tiles; optional Slack notification
Critical	Red	Visual indicator on charts and tiles; Slack notification always sent

For each threshold, set:

Value — the numeric boundary
Direction — "above" triggers when the metric exceeds the value; "below" triggers when it falls under

Slack alerts include the metric name, current value, threshold breached, and a direct link to the metric detail page.

Process Experiments

Use Process Experiments to A/B test a workflow change against any custom metric. Define:

Baseline period — the date range before the change
Treatment period — the date range after the change
Metric — the custom metric to compare

Koalr computes the mean and variance for both periods and shows the percentage change with a confidence indicator. Experiments require at least 14 days in each period to produce a meaningful signal.

Template Gallery

The builder includes a template gallery with pre-built starting points. Select a template to prepopulate Steps 2 through 5 with sensible defaults, then customize as needed.

Available templates:

Template	Type	Data source
Review SLO Compliance	Ratio	Pull Requests
PR Health Score	Composed	Pull Requests
Deployment Velocity Index	Count	Deployments

Limitations

Custom metrics compute once per day, overnight. Real-time values are not available for custom metrics.
Backfill on publish covers the last 90 days only.
Filters use AND logic only; OR logic across filter groups is not yet supported.
Ratio metrics require both numerator and denominator to share the same data source.

Reports
DORA Metrics
Automations