SkillOpt: Executive Strategy for Self-Evolving Agent Skills

Source-first digest for monthly 2026_05 rank 5, rank_id p008.

• Routing status: success, route full_markdown
• PDF extraction: not used

Motivation / Background

Frontier models are now used as agents inside tool loops, file workspaces, and verifier-backed environments. In that setting, adaptation is often procedural rather than factual: the agent needs rules about how to inspect evidence, call tools, preserve file state, format answers, or stop looping. The paper's premise is that a skill document is a portable natural-language state for those procedures, and that this state should be trained rather than hand-written once.

The gap SkillOpt targets is that existing skill or prompt improvement methods are either manual, one-shot, or loosely self-revising. They can collect lessons from trajectories, but they usually lack the controls that make optimization reproducible: train/selection/test separation, bounded steps, rejected-update memory, and held-out acceptance. SkillOpt turns the skill into the only trainable object while the target model, harness, and evaluator stay fixed.

The high-level loop is shown in Figure 1: a frozen target model executes tasks with a current skill, a separate optimizer model proposes bounded edits from scored rollouts, and a held-out gate accepts only improving skill versions.

Claims And Evidence

Support scores are support-from-paper scores, not independent reproduction scores.

Core Technical Idea

SkillOpt defines a skill \(s\) as a natural-language policy inserted into the target agent context. For a frozen target model \(M\), harness \(h\), task \(x\), and skill \(s\), execution returns both a trajectory and a scalar score:

The loop separates training, selection, and test data. Training examples generate candidate skills, the selection split accepts or rejects candidates, and the test split is held out for final reporting:

The important design choice is that the optimizer does not change \(M\). It edits only the skill document, and only a validation-gated best_skill.md is deployed. The optimizer-side state can be richer during training: current skill, best skill, score cache, rejected-step buffer, and optional meta-skill guidance.

Method Details

Forward Pass: Rollout Evidence

At each optimization step, the target model runs a batch of training tasks with the current skill. The harness records task metadata, messages, tool calls, observations, command outputs, final answers, verifier feedback, and compact execution traces when the environment has tools. Small batches update quickly but noisily; larger batches expose recurring failures before the skill changes.

Backward Pass: Minibatch Reflection

The optimizer model splits rollout evidence into failed and successful trajectories, then partitions each group into reflection minibatches. Failure minibatches propose missing or corrective rules. Success minibatches preserve behaviors that already work. The local proposals are merged hierarchically: failures and successes are consolidated separately, then a failure-prioritized merge removes duplicates, conflicts, and example-specific edits.

Bounded Text Updates

The textual learning rate is the edit budget \(L_t\), the maximum number of skill edits applied at step \(t\). After reflection and merging, the optimizer ranks proposed add/delete/replace edits and clips them to the top \(L_t\). The default schedule uses \(L_t=4\) with cosine decay and a floor of \(L_t=2\). This turns skill improvement into local patching rather than arbitrary rewriting, so a useful rule is less likely to be erased by the next update.

Validation Gate And Rejected-Edit Buffer

Every candidate skill is evaluated on \(D_{\mathrm{sel}}\) with the same frozen model and harness. It is accepted only if its selection score is strictly greater than the current score; ties are rejected. If the candidate also beats the previous best, it becomes best_skill.md.

Rejected edits are not discarded completely. SkillOpt records the failure patterns, attempted edits, and selection-score drop in an epoch-local buffer. Later reflection calls receive this negative feedback so the optimizer can avoid repeating failed edits. The rejected buffer is a training-time mechanism only: it does not add deployment-time calls.

Epoch-Wise Slow/Meta Update

SkillOpt also has a slower update path at epoch boundaries. It samples the same training items under the previous and current epoch-end skills, groups them into improvements, regressions, persistent failures, and stable successes, and asks the optimizer to write a compact longitudinal guidance block into a protected skill region. That candidate is still validation-gated. A separate optimizer-side meta skill summarizes which edit patterns helped or failed, but this meta skill is not shipped to the target model.

Harness-Agnostic Deployment

The adapter interface is simple: construct training/evaluation batches, inject the current skill into the agent context, run the native harness, and return scored trajectories. Direct chat prepends the skill to the system/developer prompt. The Codex and Claude Code harnesses render the current skill into a per-task workspace and return compact execution traces. All modes consume the same final best_skill.md, which is why the paper can test cross-harness transfer.

Optimizer Prompt Contracts

The appendix gives concrete JSON contracts for failure analysis, success analysis, failure merge, success merge, final merge, ranking, slow update, and optimizer memory. The patch representation is restricted to append, insert_after, replace, and delete; merge stages carry support_count and source_type; the slow-update region is protected by SLOW_UPDATE_START / SLOW_UPDATE_END; and every candidate goes through the same held-out gate. These safeguards matter because otherwise a strong optimizer could produce plausible but harmful rewrites.

Experiments And Results

The experiments cover six benchmark families: SearchQA, SpreadsheetBench, OfficeQA, DocVQA, LiveMathematicianBench, and ALFWorld. They use native hard-score or exact-match evaluators on held-out test splits. The suite spans single-round QA, spreadsheet code/tool use, local-document and multimodal-document reasoning, math multiple choice, and embodied sequential decision making.

The default optimizer setup is four epochs, rollout batch size 40, reflection minibatch size 8, 16 analyst workers, merge batch size 8, \(L_t=4\) with cosine decay, strict held-out gating, slow update with 20 sampled tasks per epoch, patch edit mode, and an optional rejected-edit buffer. Benchmarks with small training pools adjust batch sizes while keeping the same gate and slow/meta machinery.

Main Results

Table 1. GPT-5.5 direct-chat held-out test scores. This table condenses the main result matrix around the frontier target model. SkillOpt is compared with the no-skill row and the strongest competing baseline in each benchmark cell.

The paper's headline aggregate is that SkillOpt is best or tied-best on 52 of 52 evaluated model/benchmark/harness cells. On GPT-5.5 direct chat, the average rises from 58.8 with no skill to 82.3 with SkillOpt, a +23.5 point gain. The authors also report a +5.4 point gap over an oracle that picks the best competing baseline per cell, whose six-benchmark average is 76.9.

Harness Results

Table 2. GPT-5.5 harness-backed scores. ALFWorld is omitted in these harness rows because it requires persistent embodied-environment interaction outside the standard Codex and Claude Code adapters.

The Codex harness average gain is reported as +24.8 over no skill and +14.0 over EvoSkill. The Claude Code harness gain is +19.1 over no skill and +3.2 over EvoSkill. The largest harness-side effect is SpreadsheetBench, where SkillOpt moves Codex from 27.5 to 85.0 and Claude Code from 22.1 to 80.4.

Ablations

Table 3. Hyperparameter and evidence ablation summary. Scores are reported as SearchQA / SpreadsheetBench / LiveMath when all three appear.

Table 4. Component ablations. The sharpest failure is removing both meta skill and slow update, which drops SpreadsheetBench from 77.5 to 55.0.

The ablations support the method story: exact batch sizes and schedules are not fragile, but bounded text learning, validation gating, rejected-edit feedback, and epoch-level consolidation are the mechanisms that make skill editing behave like a controlled training loop.

Transfer

Table 5. Transfer summary. Direct is in-domain SkillOpt on the target setting; Transferred applies a source skill without further optimization.

The strongest deployment signal is the SpreadsheetBench cross-harness transfer: a Codex-trained skill transfers to Claude Code with +59.7 over the Claude baseline, and a Claude-trained skill transfers to Codex with +43.6. The paper interprets this as evidence that the skill learns workbook-level procedures rather than only harness-specific commands.

Optimizer Strength

Table 6. Strong versus target-matched optimizer. The optimizer is a training-time choice; the deployed skill still calls only the target model.

The strong optimizer wins on all four cells, but the target-matched optimizer still recovers 56-74 percent of the strong-optimizer gain. This makes the method more than a teacher-student distillation story: the gated optimization loop itself contributes value.

Learned Skill Artifacts

Table 7. Cost and edit economy of GPT-5.5 / GPT-5.5 skill runs. These are final best_skill.md token counts and accepted bounded updates.

Final skills stay between 379 and 1,995 tokens and require only 1-4 accepted edits. The paper emphasizes that the optimizer may propose far more changes, but the validation gate keeps the deployed artifact small rather than turning it into a transcript of every reflection.

Table 8. Text-only learned-rule figure. The source figure has no graphic asset in figures.json; the paper represents it as a boxed text figure. I reproduce the rules as a table rather than copying a nonexistent image.

The learned rules are procedural, not instance-specific. They are also auditable: a practitioner can read the final skill, inspect accepted edits, and decide whether the rule is appropriate for deployment.

Qualitative Evolution

The paper closes with two qualitative case studies. In ALFWorld, the skill evolves from a generic search-transform-place plan into a more stateful policy with exact object-name matching, visited-location memory, destination memory, progress locks, and loop breakers. The reported held-out test score for this representative run improves from 49.3 to 74.6.

In SpreadsheetBench, the skill evolves from generic Python spreadsheet automation into workbook forensics: inspect the actual workbook, locate headers and target ranges across sheets, normalize keys and cell types, preserve formatting, write evaluated static values when the grader reads values, fill complete target ranges, keep helper computations in Python, and reopen the saved workbook to check boundary rows and blanks. The representative held-out test score improves from 40.4 to 78.9.

Limitations

SkillOpt needs reliable scored trajectories and a held-out selection split. The method is easiest to apply where exact-match metrics, executable checks, automatic verifiers, or stable task scorers exist. For subjective or multi-dimensional tasks, the validation gate would need stronger human or model-based evaluation. Training also costs extra rollout and optimizer-model calls, so the economics work best when a learned skill can be reused. Finally, a single skill may not cover highly heterogeneous domains, and transferred skills still need held-out evaluation because they can encode training-distribution heuristics.

Practical Takeaways

SkillOpt is most compelling when the domain has repeatable tasks, automatic scoring, and enough reuse to amortize offline optimization. The direct operational recipe is: maintain clean train/selection/test splits, collect trajectory evidence from the real deployment harness, make small patch-style skill edits, accept only strictly improving selection candidates, and keep rejected edits as negative feedback.

For agent engineering, the key idea is not "ask a stronger model to write a better prompt." It is to treat the skill as a versioned artifact with an optimization loop around it. The frozen target model remains fixed, the deployed artifact is just best_skill.md, and a stronger optimizer can be used offline without increasing inference-time cost.

The main caution is evaluation. The paper's strongest claims are backed by verifier-style benchmarks and held-out scores. For open-ended workflows, the validation gate is only as good as the evaluator, so a SkillOpt-style loop should not be trusted without a task-appropriate selection signal and post-transfer testing.

Reference Coverage

• Additional local references for validation and navigation: fig-epoch-curves, fig-skillopt-pipeline, table-component-ablation, table-gpt55-direct-main-results, table-harness-results, table-hyperparam-ablation.
• Additional local references for validation and navigation: table-learned-rules, table-optimizer-strength, table-skill-cost, table-transfer-summary.