> ## Documentation Index
> Fetch the complete documentation index at: https://braintrust.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# What Loop can do

> Query logs, diagnose traces, write scorers, run experiments, and build dashboards through natural language, with example prompts for every task.

Loop has its own tools for driving playgrounds, custom views, dataset snapshots, and the workspace, plus the read and write tools that the [Braintrust MCP](/docs/integrations/developer-tools/mcp) server exposes to coding agents. It can chain several of these in a single turn: find a failure in your logs, write a scorer for it, and open that scorer in the workspace for you to review.

This page catalogs what Loop can do, grouped by what it's doing on your behalf: investigating your data, building objects from what it finds, running and automating the result, and helping you find your way around. Each area includes example prompts.

## Investigate

Ask questions about your data, from a single trace up to every project in your organization.

<h3 id="explore-your-data">
  <Icon icon="search" /> Explore your data
</h3>

Loop can query your logs, experiments, and datasets with SQL and inspect the schema of a data source. To dig through large or complex traces, it can also read, search, and list files in a sandbox, or run commands there. When it resolves a trace or generates a permalink, you can open that trace in the workspace and inspect it alongside the thread.

Loop can also:

* **Hand off to the SQL sandbox.** Loop can write a query into the [**<Icon icon="asterisk" /> SQL sandbox**](https://www.braintrust.dev/app/~/sql) and run it there, then [refine or fix it](/docs/reference/sql#loop) on request. The query stays in the sandbox so you can edit it by hand.
* **Start from the logs table.** Loop can [turn a question into a SQL filter](/docs/observe/filter#ask-questions-with-loop) without you writing the query, and [find traces similar](/docs/observe/filter#ask-questions-with-loop) to ones you've selected.
* **Resolve identities.** Loop resolves `span_attributes.created_by_user_id` to organization member names and emails, and `span_attributes.created_by_api_key_id` to API key and service token names, so you can ask about people and keys by name instead of by ID.

<Note>
  Fields you define in `metadata`, including ones named like `metadata.user_id`, describe your own application's users. Loop doesn't read them as Braintrust members.
</Note>

Example prompts:

<AccordionGroup>
  <Accordion title="Summarize logs and suggest next steps">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Look through my logs, explain what you find, and recommend the best next steps for my project.
    ```
  </Accordion>

  <Accordion title="Break down cost by model">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Show me the cost of traces named X over the last week, broken down by model. Include total, average, p50, p95, and p99 cost.
    ```
  </Accordion>

  <Accordion title="Graph caching and usage metrics">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Find the metrics related to prompt caching and LLM usage, then graph cache-hit rate and the number of LLM calls per trace.
    ```
  </Accordion>

  <Accordion title="Hand off a query to the SQL sandbox">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Write that query into the SQL sandbox and run it there so I can keep refining it.
    ```
  </Accordion>

  <Accordion title="Attribute traces to teammates">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Which teammates logged the most traces this week, and what did each one cost?
    ```
  </Accordion>
</AccordionGroup>

<h3 id="analyze-traces">
  <Icon icon="sparkles" /> Analyze traces
</h3>

Loop can work through a complex trace and explain what happened across many LLM and tool calls, either diagnosing a failure or summarizing the work performed. You start both from the trace view rather than from a Loop thread. See [Debug a trace](/docs/observe/debug-traces).

You can also ask about a trace from inside a Loop thread. Paste the trace's permalink or give Loop its ID, and Loop resolves it and pulls in the spans.

Example prompts:

<AccordionGroup>
  <Accordion title="Understand what happened">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Summarize this trace. What was the total latency, and where was the time spent?
    ```
  </Accordion>

  <Accordion title="Find related failures">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Are there other traces with similar errors? Show me what they have in common.
    ```
  </Accordion>
</AccordionGroup>

<h3 id="find-patterns">
  <Icon icon="scan-search" /> Find patterns
</h3>

Loop can search your logs and raw trace data for recurring problems and trends, regardless of whether they're already measured by a scorer, and record them as [patterns](/docs/observe/patterns).

Each pattern includes the evidence Loop found when creating it. When new evidence points at a pattern Loop already recorded, it updates that pattern rather than adding a second one.

Example prompts:

<AccordionGroup>
  <Accordion title="Record a failure you suspect">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    I think our tool calls are failing more often since last week. Dig into it, and if you find something, record it as a pattern.
    ```
  </Accordion>

  <Accordion title="Turn an investigation into a tracked pattern">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Look through this project's recent logs, find a recurring failure, and record it as a pattern with the supporting traces.
    ```
  </Accordion>

  <Accordion title="Ask how a pattern was found">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    How did you derive this pattern, and what would make it more accurate?
    ```
  </Accordion>
</AccordionGroup>

<Note>
  Loop can also find patterns on a schedule, without being asked. See [Enable Patterns](/docs/observe/patterns/enable).
</Note>

<h3 id="analyze-projects">
  <Icon icon="house" /> Analyze projects
</h3>

Loop can answer questions that span a whole project, rather than a single trace or experiment, and it can compare across every project in your organization.

To analyze a project in depth, open the <Icon icon="blend" /> **Loop agent** in the bottom right corner of a project's **<Icon icon="house" /> Overview** page, and provide a prompt. Loop has access to the project's logs, recent experiments, datasets, prompts, and the score progress chart on the overview page.

Example prompts:

<AccordionGroup>
  <Accordion title="Track a score over time">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    What's the trend in my accuracy score over the last 7 days? Which scorer has been improving the most?
    ```
  </Accordion>

  <Accordion title="Find the experiments that moved">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Which experiments had the biggest score changes?
    ```
  </Accordion>

  <Accordion title="Check recent activity">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Show me recent logs with errors, and tell me what datasets were created recently.
    ```
  </Accordion>

  <Accordion title="Watch token usage">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    How has my project's token usage changed over time?
    ```
  </Accordion>
</AccordionGroup>

To compare metrics across all your projects, select <Icon icon="blend" /> **Loop agent** from the organization-level [projects list](https://www.braintrust.dev/app). Loop can compare project metrics, identify trends in usage and performance, and show how different projects are being used.

Example prompts:

<AccordionGroup>
  <Accordion title="Rank projects by usage">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Which projects have the highest token usage, and which have the most experiments?
    ```
  </Accordion>

  <Accordion title="Compare performance across projects">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Compare latency metrics across my top 5 projects.
    ```
  </Accordion>

  <Accordion title="Spot trends in volume">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Show me trends in log volume across projects, and which projects have declining usage.
    ```
  </Accordion>

  <Accordion title="Audit recent projects">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    What projects were created in the last month?
    ```
  </Accordion>
</AccordionGroup>

## Build

Turn what you find into saved Braintrust objects. Each of these pauses for your approval before anything is created or changed.

<h3 id="author-prompts-and-evaluators">
  <Icon icon="triangle" /> Author prompts and evaluators
</h3>

Loop can suggest evaluators grounded in your application's real behavior, then create and save them rather than just describing them. An evaluator is either a [scorer](/docs/evaluate/write-scorers), which returns a number, or a classifier, which returns a label. Loop can also create and update prompts directly. New and updated evaluators open in the workspace for review.

In a playground, Loop can [turn your annotations into prompt revisions](/docs/evaluate/playgrounds#annotate-outputs), so the feedback you leave on individual outputs becomes a concrete edit.

Example prompts:

<AccordionGroup>
  <Accordion title="Suggest scorers for my project">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Suggest useful scorers for my project.
    ```
  </Accordion>

  <Accordion title="Write a scorer for known errors">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Write a scorer that detects the errors I just identified, then create and test it on a few traces.
    ```
  </Accordion>

  <Accordion title="Create a helpfulness judge">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Create an LLM-as-a-judge scorer for helpfulness based on these logs.
    ```
  </Accordion>

  <Accordion title="Set up online scoring">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Set up online scoring with the scorer I just created, and verify it runs automatically on new traces.
    ```
  </Accordion>
</AccordionGroup>

<h3 id="curate-datasets">
  <Icon icon="database" /> Curate datasets
</h3>

Loop can [generate datasets](/docs/annotate/datasets/create#generate-with-loop) from log patterns and edit rows in an existing dataset. When Loop edits dataset rows, the dataset opens as a read-only table in the workspace, so that you can review the changes. Select a row to inspect its trace inline, and the table refreshes as Loop makes further edits. To undo a round of edits, restore an earlier [snapshot](/docs/annotate/datasets/manage#save-snapshots), which Loop can also do for you.

Loop can also manage dataset versions and the environments that point at them:

* **Snapshots.** Loop can save a [snapshot](/docs/annotate/datasets/manage#save-snapshots) before it changes anything, list existing snapshots along with their environment tags and current head version, and restore the dataset to a snapshot. Restoring uses the same confirmation flow as the dataset UI, and can save the current state first.
* **Environments.** Loop can point an [environment](/docs/deploy/environments) such as `production` or `staging` at a specific dataset snapshot, or clear that mapping. It can list the organization's environments to check available slugs, and create one that doesn't exist yet.

<Warning>
  Restoring a dataset to a snapshot replaces its current rows, and retagging an environment changes which dataset state your applications resolve. Review what Loop proposes before you approve either action.
</Warning>

Example prompts:

<AccordionGroup>
  <Accordion title="Generate a dataset from errors">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Generate a dataset from logs with errors.
    ```
  </Accordion>

  <Accordion title="Add traces to a dataset">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Add these traces to my regression dataset and set the expected output.
    ```
  </Accordion>

  <Accordion title="Snapshot before editing">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Snapshot this dataset before you change anything, then remove the rows that no longer reflect our current behavior.
    ```
  </Accordion>

  <Accordion title="Review and restore a version">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    What snapshots exist for this dataset, and which one is production pointing at? Restore the one from before yesterday's edits.
    ```
  </Accordion>

  <Accordion title="Promote a dataset version">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Point the staging environment at the snapshot you just saved, and tell me which environments exist first.
    ```
  </Accordion>
</AccordionGroup>

<h3 id="build-views-and-dashboards">
  <Icon icon="chart-no-axes-column" /> Build views and dashboards
</h3>

Loop can create and edit [custom views](/docs/annotate/custom-views) for both trace and dataset row data, and it can [build charts](/docs/observe/dashboards/build-charts) for a dashboard. When Loop creates or updates a dashboard, you can open it in the workspace to review the result alongside the thread.

Example prompts:

<AccordionGroup>
  <Accordion title="Suggest trace visualizations">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    What are some useful trace visualizations you can make for me? List the options and I'll tell you which ones to create as custom trace views.
    ```
  </Accordion>

  <Accordion title="Build a custom dataset view">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Build a custom view for this dataset that renders the input and expected output side by side.
    ```
  </Accordion>

  <Accordion title="Build a daily cost dashboard">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Create a dashboard for daily cost analysis.
    ```
  </Accordion>

  <Accordion title="Investigate an error spike">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    This topic or error spike looks interesting. Show me which customers and interactions are affected, inspect representative traces, and explain the likely root cause.
    ```
  </Accordion>

  <Accordion title="Share a dashboard with your team">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Turn this Loop analysis into a shareable dashboard my team and executives can use to understand the result and monitor it over time.
    ```
  </Accordion>
</AccordionGroup>

<h3 id="configure-topics">
  <Icon icon="pentagon" /> Configure Topics
</h3>

Loop can configure the whole [Topics](/docs/observe/topics) pipeline, not just test individual facets. Given a goal, Loop works through the pipeline in stages:

* **Check the preprocessor.** Loop inspects your project's effective default [preprocessor](/docs/observe/topics/enable#write-a-custom-preprocessor) and runs it on real traces to confirm it returns usable conversation text. If the default returns nothing, Loop writes a custom preprocessor for your trace shape, tests it, and can set it as the project default. Setting the project default changes the preprocessor every function that takes one uses, including the built-in Topics facets, so Loop asks for approval first.
* **Confirm the scope.** Loop looks for a grouping key such as `metadata.conversation_id` and recommends whether to analyze a span, a trace, or a group of related traces.
* **Validate the facets.** Loop runs a facet on up to ten representative traces and shows you the results inline, so you can correct its reading before it commits to a prompt. Build up a set of traces worth checking, then have Loop re-run the facet against that same set as you refine it. For a larger set, Loop runs the facet as an eval instead.
* **Evaluate candidates.** Loop runs evals over candidate facet prompts to measure quality before anything is saved.
* **Enable the automation.** Loop creates the facets and turns on the Topics automation, then offers to rewind it over a historical window.

<Note>
  Rewinding a Topics automation processes historical traces and draws from your monthly [model credits](/docs/plans-and-limits#model-credits). Loop estimates the volume and asks for approval before it rewinds.
</Note>

Example prompts:

<AccordionGroup>
  <Accordion title="Set up Topics from scratch">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Set up Topics for my project.
    ```
  </Accordion>

  <Accordion title="Configure Topics with a custom preprocessor">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Help me configure Topics with a custom preprocessor. My traces don't store conversation text on LLM spans.
    ```
  </Accordion>

  <Accordion title="Discover error categories">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    I want to discover the kinds of errors my users hit. Build a facet for that, validate it on real traces, and enable it.
    ```
  </Accordion>
</AccordionGroup>

## Run and automate

Test a change with an evaluation, then turn the check that caught it into a scheduled run.

<h3 id="run-evaluations">
  <Icon icon="beaker" /> Run evaluations
</h3>

Loop runs evaluations two ways:

* **In a playground or experiment.** Loop can edit the task prompt or code, add or modify scorers, and re-run the evaluation to show the effect of a change.
* **As a new experiment.** Loop can run an experiment from a dataset, from rows it supplies inline, or from a prior experiment, using a saved or inline task and any scorers you choose. When a prior experiment supplies the data, its outputs become the expected values.

When Loop starts an evaluation, the runs open in an **Experiments** panel in the workspace listing each run's progress and scores, so you can compare results without leaving the chat.

Example prompts:

<AccordionGroup>
  <Accordion title="Build a dataset and run an experiment">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Build a dataset from the traces where my agent gave a wrong answer, then run an experiment on it with my current prompt and scorers, and show me which cases still fail.
    ```
  </Accordion>

  <Accordion title="Evaluate an agent decision">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Evaluate whether this agent made the right decision, but only use the relevant messages, reasoning, and tool calls from these very large traces.
    ```
  </Accordion>

  <Accordion title="Compare a cheaper model">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Evaluate whether we can switch this workflow to a cheaper candidate model. Compare quality, cost, and latency, and tell me whether it is safe to roll out.
    ```
  </Accordion>
</AccordionGroup>

<Note>
  Loop runs evaluations in Braintrust rather than by writing an `Eval()` script. To keep evaluations in your own repository and run them in CI, see [Run evaluations in code](/docs/evaluate/run-in-code).
</Note>

<h3 id="manage-automations-and-alerts">
  <Icon icon="radio" /> Manage automations and alerts
</h3>

Loop can set up recurring work and notifications from what it finds, so a problem you just investigated becomes something you get told about, or something Loop rechecks on a schedule.

* **Alerts.** Loop can create an [alert](/docs/observe/alerts) on individual matching logs, on an aggregate that crosses a threshold over a recent window, or on environment updates, generating the filter from your description.
* **Automations.** Loop can create a [scheduled Loop job](/docs/loop/automations) with its own instruction and schedule. It can also list the project's existing automations, including online scoring rules, alerts, exports, retention policies, and Topics automations, and return each one's full configuration.
* **Status.** Loop can pause an automation or set it back to active.
* **Updates.** Loop can change an alert or a Loop automation by rewriting it under its existing name. A rewrite replaces the whole configuration rather than patching a single field, so ask Loop to read the current configuration first, then check what it proposes before you approve. Scheduled exports and retention policies are inspect-only.

You can also edit any automation directly in **<Icon icon="settings-2" /> Settings** > [**<Icon icon="radio" /> Automations**](https://www.braintrust.dev/app/~/configuration/automations).

Example prompts:

<AccordionGroup>
  <Accordion title="Alert on the errors you just found">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Create an alert that notifies me when the error you just found shows up in my logs again.
    ```
  </Accordion>

  <Accordion title="Alert on a quality regression">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Alert me when the average score for this scorer drops below 0.8 over any six hour window.
    ```
  </Accordion>

  <Accordion title="Alert on cost growth">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Watch my daily spend and alert me when it goes more than 20% above last week's average.
    ```
  </Accordion>

  <Accordion title="Audit what's configured">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    What automations are configured in this project, and which ones are paused?
    ```
  </Accordion>

  <Accordion title="Work out why a scheduled run found nothing">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Show me the configuration for my pattern discovery automation, then explain why yesterday's run didn't record anything.
    ```
  </Accordion>
</AccordionGroup>

## Get help

Loop can help you find your way around Braintrust, and reach out to support when you can't resolve something.

<h3 id="search-docs-and-find-objects">
  <Icon icon="book-open" /> Search docs and find objects
</h3>

Loop can search the Braintrust documentation, resolve object names and URLs, list recent objects, summarize an experiment's metrics against a baseline, and generate shareable links to objects it creates.

Example prompts:

<AccordionGroup>
  <Accordion title="Look up how to set up human review">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    How do I set up human review?
    ```
  </Accordion>

  <Accordion title="Compare experiments">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Summarize my latest experiment compared to the previous one.
    ```
  </Accordion>

  <Accordion title="Get a shareable link">
    ```text wrap theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    Give me a shareable link to the dashboard you just made.
    ```
  </Accordion>
</AccordionGroup>

<h3 id="request-support">
  <Icon icon="life-buoy" /> Request support
</h3>

Loop can file a support ticket with the Braintrust team when you encounter something it can't resolve. Ask for help, or type `/` and choose the support command, and Loop guides you through creating the request. It includes relevant context from your current page and conversation so the support team can understand the issue.

<h2 id="next-steps">
  Next steps
</h2>

* [Run Loop on a schedule](/docs/loop/automations) to turn any of this into recurring work.
* [Enable Patterns](/docs/observe/patterns/enable) for automatic discovery of recurring problems.
* [Manage threads and limits](/docs/loop/manage) for sandbox behavior and cleanup.
* [What Loop can't do](/docs/loop#what-loop-cant-do) for the boundaries it works inside.