Back to the library

Optimization / Evaluation / Model upgrades

Use agent optimization to adapt an agent to a new model

Follow a competitive-intelligence agent through traces, a private benchmark, optimization candidates, and an experiment comparing versions.

In this demo

Jeff uses a competitive-intelligence agent to show why model upgrades can change behavior and why application-specific evaluation matters. The walkthrough examines execution traces, a private benchmark, optimizer candidates, and proposed repository changes. It finishes with an experiment comparing agent versions and discusses how to use evidence rather than assumptions when selecting an update.

Chapters

  1. 0:00Competitive-intelligence workflow
  2. 1:54The model-upgrade challenge
  3. 2:44Traces and private benchmarks
  4. 3:56Optimization candidates and proposed changes
  5. 8:02Compare versions with an experiment