Automated Build Failure Analysis with Log Detective in Packit

Overview

Starting this month, Packit integrates Log Detective—an AI-powered analysis tool that automatically examines failed Koji builds triggered by dist-git pull requests. This tutorial walks you through how the analysis works, what you need to get started, and how to interpret the results. Log Detective operates without manual intervention: when a build fails, Packit sends the logs to Log Detective, which extracts meaningful snippets using the Drain template mining algorithm and the BeeAI Framework (version 4.0). The result is a concise explanation of the failure and, where possible, a suggested fix—all presented on the Packit dashboard.

Automated Build Failure Analysis with Log Detective in Packit — Source: fedoramagazine.org

This guide is for developers and package maintainers who want to leverage automated failure analysis to speed up debugging. No additional setup beyond standard Packit usage is required.

Prerequisites

System Requirements

A Fedora or CentOS Stream environment capable of running Koji builds.
Packit service enabled for your repository (see Packit documentation).
Dist-git pull requests configured to trigger scratch Koji builds.

Account and Permissions

A Fedora Account System (FAS) account with packager privileges.
Your project must be registered with Packit and have webhook access to the dist-git repository.
No special Log Detective account or API key is needed—the service is invoked automatically by Packit.

Step-by-Step Instructions

How Build Failures Trigger Analysis

Initiate a Pull Request: Create a pull request in dist-git that triggers a scratch Koji build (e.g., by pushing changes to the spec file or sources).
Build Failure: If the Koji build fails, Packit’s existing failure detection logic identifies the failure.
Automatic Request: Packit sends a request to the Log Detective interface server, including all build logs and artifacts. No manual prompting or log selection is needed.
Analysis Processing: Log Detective, using the BeeAI Framework, parses the logs with the Drain algorithm to extract snippets—small, relevant portions of the log that reduce token usage and limit noise. The extracted snippets feed into a general-purpose language model that generates a human-readable failure explanation and, optionally, a suggested solution.
Result Delivery: The interface server posts the completed analysis onto the Fedora Messaging bus. Packit listens to this bus and retrieves the result, then links it to the corresponding pull request on the Packit dashboard.

Understanding the Analysis Workflow

The workflow involves three components:

Packit Service: Handles the build trigger, failure detection, and result display.
Log Detective Interface Server: A lightweight containerized service that receives analysis requests, communicates with the Log Detective agent, and publishes results to the Fedora Messaging bus.
Log Detective Agent: Built with BeeAI Framework v4.0, it processes logs using tools like the Drain template mining algorithm to identify patterns and extract snippets. These snippets are sent to a small, efficient language model to derive the failure reason and solution.

This architecture minimizes latency and cost by avoiding the transmission of entire log files. The snippet extraction ensures that the model context contains only the most informative lines.

Interpreting Results

Locate the Results: After a failed build, navigate to the Packit dashboard for your project. The pull request that triggered the build will have a new link or badge indicating Log Detective analysis is available.
Read the Analysis: The analysis consists of a clear statement of what went wrong (e.g., missing dependency, build time out, configuration error) and, when possible, a suggestion to fix it. The model currently uses only the build logs; it does not access external sources like upstream documentation or bug trackers.
Apply the Fix: Use the suggested solution as a starting point. Since the model is general-purpose, its recommendations may require adaptation to your specific package context.

Common Mistakes and Pitfalls

Assuming the model is infallible: Log Detective is a tool for assistance, not a replacement for experience. It works best for common failure patterns; rare or nuanced issues may receive incomplete or misleading advice.
Ignoring the extracted snippets: If you need to troubleshoot further, consider examining the raw log files directly. The snippets are a summary; the original logs contain the full context.
Expecting instantaneous analysis: There is a brief delay between the build failure and the appearance of results. The analysis can take a few minutes, especially under high load. Patience is required.
Duplicate analysis requests: Avoid manually requesting additional analyses via the Copr “Ask AI” button for the same build—Packit’s automatic request is already in progress. Duplicate requests waste resources.

Summary

Log Detective brings automated, zero-configuration analysis to Packit-triggered Koji build failures. By leveraging snippet extraction and a lightweight AI model, it provides quick, actionable insights directly on the Packit dashboard. No extra setup is needed, and the results are especially valuable for newcomers to Fedora packaging. The feature is currently limited to build logs, but future updates may expand its scope. Remember: treat Log Detective as a helpful assistant, not an oracle—your experience remains the ultimate debugging tool.