Deploy Risk

Overview

Koalr's Deploy Risk page is the platform's core differentiator: every pull request receives a composite risk score before merge, computed from 36 research-validated signals. High-risk PRs are flagged in the GitHub PR check, in the dashboard, and via Slack alerts — giving teams a window to intervene before a deployment becomes an incident.

No other engineering metrics platform scores deployment risk pre-merge at this level of signal depth.

How risk is scored

Each PR receives a Risk Score from 0–100 composed of weighted signals across five categories:

1. Change characteristics (30% weight)

• Change size — lines added + deleted; PRs > 400 lines carry elevated risk
• Change entropy — how spread the diff is across files and directories; cross-cutting changes are riskier than focused ones
• File churn rate — how frequently the changed files have been modified in the past 90 days; high churn = unstable area
• DDL migration detected — schema changes (CREATE TABLE, ALTER TABLE, DROP COLUMN) carry a fixed high-risk penalty
• Dependency vulnerability risk — changed manifest files (package.json, go.mod, requirements.txt, and 7 others) are checked against deps.dev and the GitHub Advisory Database; CVE count and severity contribute to risk score (see Dependency Vulnerability)

2. Author expertise (25% weight)

• File familiarity — percentage of changed files the author has previously modified; first-time file edits are higher risk
• Module ownership — whether the author matches the CODEOWNERS entry for changed paths
• Historical rework rate — how often the author's past PRs have been reworked within 21 days

3. Review coverage (20% weight)

• Review depth — comment count and reviewer count relative to PR size
• Reviewer familiarity — whether reviewers have context in the changed files
• CODEOWNERS compliance — whether required owners have approved the PR
• Time-to-approval — PRs approved in < 5 minutes receive a scrutiny penalty (rubber-stamping signal)

4. Test coverage (15% weight)

• Coverage delta on changed files — drop in coverage on the specific files modified in this PR
• New code coverage — percentage of new lines covered by tests
• Test-to-code ratio — ratio of test additions to implementation changes in the PR

5. Deployment context (10% weight)

• Deployment timing — Friday afternoons and late-night deploys carry elevated risk
• Deployment frequency baseline — deploying significantly faster than team baseline
• SLO burn rate — deploying into a service already burning its error budget at elevated rate
• Concurrent open incidents — deploying while an active incident is in progress
• Rollback frequency — how often recent deploys to this service have been rolled back
• AI authorship — Co-authored-by trailers from Copilot, Cursor, Claude Code, or other AI assistants

Risk score thresholds

Score	Level	Default action
0–29	🟢 Low	No action — deploy freely
30–59	🟡 Moderate	Amber warning in PR check
60–79	🟠 High	Slack alert to team channel
80–100	🔴 Critical	GitHub Check Run fails, PR blocked

Block thresholds are configurable per team in Settings → Deploy Risk.

Dashboard sections

Organization risk summary

• Total PRs scored in the period
• High/critical risk PR count and percentage
• Risk reduction trend (week-over-week)
• Average risk score by team

PR risk table

Sortable table of all open and recently merged PRs with:

• Risk score badge (colored)
• Top contributing signals (up to 3 shown)
• PR author, repo, and age
• Link to GitHub PR

Signal breakdown chart

Bar chart showing which signals contribute most to risk scores across your org. Useful for identifying systemic issues (e.g., consistently low reviewer familiarity, or chronic coverage drops).

Risk trend over time

30-day line chart of average risk score per week. Upward trend = increasing systemic risk.

Team comparison

Per-team average risk score with breakdown by signal category.

GitHub integration

When GitHub is connected, Koalr creates a GitHub Check Run on every PR. The check:

• Shows the risk score and top signals in the PR status area
• Passes for Low/Moderate risk (green check)
• Creates a warning annotation for High risk (amber)
• Fails the check for Critical risk (red X) — blocks merge if branch protection requires all checks to pass

Configure branch protection in your GitHub repo settings: Settings → Branches → Branch protection rules → Require status checks to pass → koalr/deploy-risk.

CI Analytics

Koalr tracks GitHub Actions workflows to surface flaky jobs and CI duration trends. High flakiness in a repo is correlated with higher deploy risk — teams with flaky CI tend to approve PRs without green CI, which degrades the review coverage signal.

See CI Analytics for the full CI health dashboard.

Slack alerts

When a PR is scored High or Critical, Koalr can post to your Slack channel with:

• PR title, author, and link
• Risk score and level
• Top 3 contributing signals

Enable in Automations by creating a HIGH_RISK_PR automation rule with a Slack action.

Configuring risk thresholds

Navigate to Deploy Risk → Accuracy to:

• Enable/disable specific signals
• Set signal weights (advanced)
• View model accuracy against your historical deployments

Relationship to other metrics

Deploy risk is correlated with:

• DORA Change Failure Rate — teams that reduce average risk score see lower CFR within 30 days (DORA Metrics)
• Cycle time — high-risk PRs spend longer in review and accumulate more comments, adding to cycle time
• Incident correlation — the Incidents page shows which past incidents were preceded by high-risk deployments (Incidents)
• CI Analytics — CI flakiness rate correlates with deploy risk; flaky CI degrades review coverage signals (CI Analytics)