[Training methods]

Training methods

Attempts matches the tuning method to your task, data, and eval harness—not the other way around. Every method runs against the same baseline-first workflow: measure the base model, approve a capped plan, train, then report the delta on held-out test data.

Available methods

DPO

Direct preference optimization

Learn which response is better

View method

RL

Reinforcement learning from rewards

Learn from measurable outcomes

View method

SFT

Supervised tuning

Learn from approved examples

Teach the model the outputs, formats, and behaviors your reviewers already accept. SFT is the first method we use when you have reliable input/output pairs and a task-specific eval harness.

Available in the Attempts evaluation workspace today

When to use

  • Structured extraction, classification, or formatting with labeled gold examples
  • Tasks where reviewers can agree on a single correct answer
  • First experimental step when generic models miss domain vocabulary or schema

Workflow with Attempts

  1. Ingest dataset with deterministic train/val/test splits by group
  2. Measure base model on the test split
  3. Approve capped SFT plan and budget
  4. Train, evaluate tuned model on the same test split, report delta
Data required
Input/output pairs (or chat transcripts) approved for training; group keys for leak-safe splits.
Evaluation needs
Held-out test groups, task metrics (exact match, F1, JSON validity, or custom checks).

DPO

Direct preference optimization

Learn which response is better

Improve style, safety, and ranking when humans can compare two candidate outputs but a single gold string is awkward. DPO uses preferred vs rejected pairs tied to the same prompt.

Scoped engagements; not in the self-serve workspace yet

When to use

  • Tone, brevity, or formatting preferences that are easier to rank than to rewrite
  • Reducing specific failure modes seen in baseline rollouts
  • Tasks with pairwise reviewer judgments instead of absolute labels

Workflow with Attempts

  1. Define preference criteria and audit a sample of pairs
  2. Baseline the current model on task metrics and failure buckets
  3. Scope DPO dataset size, base model, and budget cap
  4. Train, re-evaluate on held-out test prompts, compare to SFT baseline if both ran
Data required
Prompt-aligned preference pairs (chosen vs rejected), with consistent grading guidelines.
Evaluation needs
Preference accuracy or downstream task metrics on held-out prompts; guardrails against reward hacking.

RL

Reinforcement learning from rewards

Learn from measurable outcomes

Optimize toward rewards you can measure automatically—tool success, task completion, verifier scores, or business KPIs mapped to an eval harness. RL is the heaviest method and requires a trustworthy reward signal.

Scoped engagements with custom harness design

When to use

  • Agent or tool-use loops where success is programmatically scorable
  • Long-horizon tasks with rollout-level observability
  • Domains with simulators, verifiers, or human-in-the-loop graders at scale

Workflow with Attempts

  1. Design reward and rollout logging with Attempts engineers
  2. Baseline policy model with the same harness used in training
  3. Pilot RL budget on a narrow task slice before full spend
  4. Promote checkpoints only when test-harness metrics and cost targets hold
Data required
Task prompts, environment or tool interface, and a reliable reward or grader (not proxy vibes).
Evaluation needs
Rollout evaluators, regression suites, and cost/latency caps on exploration during training.

Compare methods

At a glance—final choice is always task-specific.

RequirementSFTDPORL
Labeled answersYes——
Preference pairs—Yes—
Reward / verifier signal—SometimesYes
Self-serve workspaceYes——
Typical first stepYes——
Browse models Request scoping call