Engineering Analytics Documentation

Analytics

Review analytics Failure trend analysis AI quality metrics

Review Analytics#

Codluma tracks every review it performs and surfaces the data as actionable engineering metrics. Analytics are available to all team members and exportable as CSV or JSON for use in your own dashboards.

Review throughput

PRs reviewed per day/week/month, broken down by repository and team member.

Time to first review

Median and P95 time from PR open to first AI review comment.

Finding trends

Volume and severity distribution of findings over time — see if code quality is improving.

Fix rate

Percentage of Critical and High findings that are resolved before the PR is merged.

Available metrics

Metric

Description

Granularity

PRs reviewed

Total pull requests analysed

Daily / weekly / monthly

Time to first review

Duration from PR open to first comment posted

Per PR, median, P95

Findings by severity

Count of Critical / High / Medium / Info over time

Daily / weekly

Fix rate

% of Critical + High findings resolved before merge

Weekly / monthly

False positive rate

% of findings dismissed by developers

Monthly

Top recurring issues

Most frequently flagged finding types across your repos

Monthly

Team fix velocity

How quickly each team member resolves findings

Monthly

Repo health score

Rolling score based on finding volume and severity trend

Weekly

Exporting data

All analytics data can be exported from the dashboard under Analytics → Export. Available formats: CSV, JSON, and PDF report. Enterprise plans additionally have access to a read-only REST API:

analytics API (Enterprise)

GET https://api.codluma.ai/v1/analytics/reviews
Authorization: Bearer <API_KEY>
X-Org-ID: <YOUR_ORG_ID>

Query params:
  from=2025-01-01          ISO 8601 date
  to=2025-03-31            ISO 8601 date
  repo=acme/backend        filter by repository (optional)
  granularity=weekly       daily | weekly | monthly

Failure Trend Analysis#

Codluma aggregates failure data across all your CI/CD pipelines to surface recurring patterns, identify flaky tests, and track mean time to recovery (MTTR).

Available failure metrics

Metric

Description

Pipeline failure rate

Failed runs as a percentage of total runs per day

MTTR

Mean time from failure detected to next successful run

Top failure causes

Most common root-cause categories (dependency, test, config, infra)

Flaky test detection

Tests that alternate pass/fail across runs without code changes

Failure hotspots

Directories and files most frequently involved in failures

Failure by author

Correlates failures with the last committer (for coaching, not blame)

Provider comparison

Failure rate by CI/CD provider if you use multiple

Flaky test detection

Codluma automatically flags tests that fail intermittently without any code changes as Flaky. These are tracked separately from genuine failures so they don't inflate your failure rate metrics or create false urgency in Slack alerts.

Tip

Flaky test data is most useful after 2+ weeks of data collection, once Codluma has enough run history to distinguish genuine failures from intermittent ones with high confidence.

AI Quality Metrics#

We measure the quality of Codluma's reviews so you can track whether the tool is actually helping your team, not just generating noise.

Quality signals

Acceptance rate

Percentage of findings that developers acknowledge (leave a reaction or apply the fix) vs dismiss. High acceptance = high signal-to-noise.

Dismissal rate

Percentage of findings marked "not relevant" or hidden. We use this internally to improve the model. You can review dismissed findings in Analytics → Dismissed.

Severity accuracy

We track when developers escalate or downgrade a finding severity. Consistent escalation patterns feed back into calibration.

Coverage score

Ratio of files changed vs files analysed. Low scores indicate files being skipped due to size limits or exclusion rules.

Note

Quality metrics are visible under Analytics → AI Quality. We recommend reviewing the dismissal rate weekly. A dismissal rate above 40% on a specific rule type is a strong signal to suppress or tune that rule in your reviewops.yml.

PreviousSecurity NextConfiguration

Can't find what you need?

Contact support