Using your model

Deploy a trained model on a private GPU and call it from your application with your API key.

Deploy a trained model

  1. Open Training. A run that has finished training shows Deploy.
  2. The confirmation shows the hardware, the rate per active hour and the amount held at start. Tick the consent box and confirm.
  3. Your deployment appears under Deployments as starting, then running once it is ready. Starting usually takes a few minutes and is not charged.

Each trained model can have one active deployment at a time. A card on file is required, and any pending bill must be settled first.

Call it

Create a key under API keys, then send chat requests to your deployment's endpoint, shown on the Deployments page:

curl https://attempts.ai/api/inference/deployments/YOUR_DEPLOYMENT_ID/chat/completions \
  -H "Authorization: Bearer $ATTEMPTS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "Extract the total from: Invoice INV-9, total 42.00 USD"}]}'

The response follows the familiar chat completions shape:

{
  "object": "chat.completion",
  "model": "dep-…",
  "choices": [{"index": 0, "message": {"role": "assistant", "content": "{\"total\": 42.0}"}, "finish_reason": "stop"}]
}
  • Send messages as a list of {"role", "content"} objects, like your training data.
  • Requests can be up to 256 KB, and each request waits up to two minutes for an answer.
  • Keep your API key on your server. Never put it in browser or mobile code.

Idle shutdown and cold starts

After 15 minutes without requests, your deployment scales to zero and stops costing anything. The next request wakes it up: until it is ready you get 503 with “The model is starting up. Retry in a few minutes.” and a Retry-After header. Retry with a short delay. Requests made while it is starting are not queued.

What it costs

  • You pay the rate shown, per hour of active time: only while the deployment is running and able to answer. Starting up and idle time scaled to zero are free.
  • When it starts, one active hour plus 25% is held from your credits. The hold grows before each further active hour.
  • If your credits cannot cover the next active hour, the deployment stops automatically. Add credits and deploy again.
  • After it stops, the charge for its active time is settled from verified usage, rounded up to the cent, and the hold is released.

See Billing and credits for how charges, your card and pending bills work.

Evaluate your model

On a running deployment, choose Evaluate. Your model answers up to 100 examples from the run's held-out test split. Those examples were set aside when your dataset was split and never used for training.

Each answer is scored with the run's metrics, such as exact match or F1. The scores and the first 20 examples, with the expected and actual answers, appear under Evaluations.

The evaluation runs on your deployment, so it uses active GPU time at the deployment's rate. There is no other charge. If the deployment is waking up, the evaluation waits for it. If the deployment stops, the evaluation ends and can be started again. Only one evaluation runs on a deployment at a time.

Stop a deployment

Choose Stop deployment on the Deployments page. Requests stop working immediately. A deployment that receives no requests for several days may also be shut down automatically.