Control how a turn ends

A model can stop before the job is done: it writes a nice answer and forgets to call post_message. A provider can answer 503 in the middle of a turn. A user can send three messages while the agent works, and one of them must not join. The turn options of the harness give you a hook for each case.

One job can also need its own model or tools. Pass overrides to the prompt to give it its own settings. Two tools of one model call can need to run one after the other. Set toolExecution on the harness.

Send the model back to work#

The beforeFinish hook runs when the model stops calling tools. Check the transcript. If the job is not done, return a message. The model runs again in the same operation.

import { defineHarness } from '@tanstack/ai-harness'
import { openaiText } from '@tanstack/ai-openai'

const assistant = defineHarness({
  name: 'acme/assistant',
  adapter: openaiText('gpt-6.1-sol'),
  turn: {
    beforeFinish: ({ messages }) => {
      const posted = messages.some((message) =>
        message.toolCalls?.some((call) => call.function.name === 'post_message'),
      )
      if (posted) return undefined
      return {
        messages: [
          {
            role: 'user',
            content: 'You have not called post_message yet. Call it now.',
          },
        ],
      }
    },
  },
})

• The hook returns { messages?, records?, ephemeral? }. Return nothing when the work is done, and the turn ends.
• When the return changes the transcript, the model runs again. The hook runs again at the next stop.
• Always check messages. A hook that returns the same reminder every time never stops.
• maxFinishCycles (default 32) is the limit. A hook that still asks for more at the limit fails the turn.
• records are host records in the same append. See Append your own records.

Remind the model without saving the reminder#

A reminder in messages stays in the transcript, so every later model call sees it again. Return it as ephemeral, and only the next model call gets it:

const reminded = defineHarness({
  name: 'acme/reminded',
  adapter: openaiText('gpt-6.1-sol'),
  turn: {
    beforeFinish: ({ messages }) => {
      const done = messages.some((message) =>
        message.toolCalls?.some((call) => call.function.name === 'finish'),
      )
      if (done) return undefined
      return {
        ephemeral: [{ role: 'user', content: 'Call finish or give_up now.' }],
      }
    },
  },
})

• The model gets the ephemeral messages at the end of its context, for that one call.
• The transcript, the session log, and the message store never keep them.
• A return with only ephemeral messages also sends the model back to work, and it counts for maxFinishCycles.
• onJoin can return ephemeral messages too. They go to the model call of the join.
• A reminder that you send in your own loop, one turn at a time, does not count for maxFinishCycles. Pass it as ephemeral to send or continueWithoutMessage. Each model call of that turn gets it, and no store keeps it. See Give the model a note for one turn.

Retry model errors#

A model call can fail for a reason that passes in a few seconds: an overloaded provider, a rate limit, a dropped connection. Set onModelError to retryTransientErrors(). The harness runs the failed call again in the same operation.

import { retryTransientErrors } from '@tanstack/ai-harness'

const resilient = defineHarness({
  name: 'acme/resilient',
  adapter: openaiText('gpt-6.1-sol'),
  turn: { onModelError: retryTransientErrors() },
})

The defaults:

• maxRetries: 3 retries since the last finished tool phase.
• baseDelayMs: 2000. The delay doubles with each retry. Each delay is then multiplied by a random jitter from 0.75 to 1.0.
• isTransient: isTransientModelError. It is true for 429, 500, 502, 503, 504, overloaded, rate limit, network and connection errors, and timeouts.
• maxRetryAfterMs: 900000 (15 minutes). The longest retryAfterMs that the policy waits for.

Pass your own check when you know more about your provider:

import { isTransientModelError } from '@tanstack/ai-harness'

const picky = defineHarness({
  name: 'acme/picky',
  adapter: openaiText('gpt-6.1-sol'),
  turn: {
    onModelError: retryTransientErrors({
      maxRetries: 5,
      baseDelayMs: 500,
      isTransient: (error) =>
        error.code === 'upstream_busy' || isTransientModelError(error),
    }),
  },
})

• When the failed call streamed no text, the policy answers 'retry'. The model runs the call again from the start.
• When the failed call streamed some text, the policy answers 'continue'. The model continues that text. See Continue a partial answer.
• When the error has a retryAfterMs, the policy waits that long and skips the backoff. A cancel of the turn stops the wait.
• When retryAfterMs is more than maxRetryAfterMs, the policy does not retry, and the turn fails.

The OpenAI and Anthropic adapters set retryAfterMs from the retry-after-ms or retry-after header of the provider response. A hook of your own gets it as error.retryAfterMs.

Clients get a harness.turn.retry event for each retry. Its value has operationId, retries, error, and continued.

Continue a partial answer#

A stream can fail after the model wrote half of its answer. A retry drops that half, and the model writes the full answer again. Answer 'continue' from onModelError to keep it:

const continuing = defineHarness({
  name: 'acme/continuing',
  adapter: openaiText('gpt-6.1-sol'),
  turn: {
    onModelError: ({ error, retries, partial }) => {
      if (retries >= 3 || !isTransientModelError(error)) return undefined
      return partial ? 'continue' : 'retry'
    },
  },
})

When the hook answers 'continue':

• The partial text stays in the turn result and in the transcript.
• The next model call gets the partial assistant message, then a short user note. The note asks the model to continue from where it stopped, without repeating its text.
• The note stays in the transcript as a user message. It has metadata: { tanstack: { synthetic: true } }, so a UI can hide it.
• The harness.turn.retry event has continued: true.

partial is true when the failed call streamed text, reasoning, or a tool call before the error. Only text can be continued. When no text streamed, 'continue' acts as 'retry', and the failed call adds nothing to the turn result.

Retry on another model#

A provider can stay down for longer than your backoff. Return { retry: true, adapter } from onModelError to run the retry on another model:

import { anthropicText } from '@tanstack/ai-anthropic'

const backup = anthropicText('claude-sonnet-5-5')

const failover = defineHarness({
  name: 'acme/failover',
  adapter: openaiText('gpt-6.1-sol'),
  turn: {
    onModelError: ({ error, retries, adapter }) => {
      if (retries >= 3 || !isTransientModelError(error)) return undefined
      if (adapter === backup) return 'retry'
      return { retry: true, adapter: backup }
    },
  },
})

• adapter is the adapter of the failed call. model is its provider/model name, the same as TurnInfo.model.
• The retry and every later model call of this turn use the new adapter. The next turn starts on its own model again.
• TurnInfo.model and the harness.usage records name the new model.
• The retry counts in retries, the same as a 'retry'. Your limit on retries still stops the turn.
• A keyed adapter gets the key of the turn's user, as the turn's own adapter does.

The answer acts as 'retry': the text of the failed call is dropped. A recovery after a host stop runs the turn on its own model.

Compact after a context overflow#

A model call can fail because the conversation is longer than the context window of the model. Add the compaction middleware to the harness. It handles the overflow itself:

import { withCompaction } from '@tanstack/ai-compaction'

const compacting = defineHarness({
  name: 'acme/compacting',
  adapter: openaiText('gpt-6.1-sol'),
  middleware: [withCompaction({ maxTokens: 100_000, countTokens: 'usage' })],
  turn: { onModelError: retryTransientErrors() },
})

• A call that fails with the code CONTEXT_OVERFLOW_CODE ('context_overflow') goes to the onModelError of the middleware first, inside the chat() call of the turn. withCompaction compacts and calls the model again with ctx.retry(), once per model call.
• turn.onModelError sees the error only when that retry also fails. Then error.code is 'context_overflow'. isTransientModelError does not match this code.

To keep the compaction after a restart, see Compact a harness session.

Choose which messages join a turn#

A message with busy: 'steer' waits while a turn runs. Before each model call, the waiting messages join the turn as a prefix, in the order they arrived.

The join stops at the first message that:

• has an abort request (session.cancel({ operationId }) on it), or
• turn.canJoin refuses.

That message and the ones after it wait. canJoin runs again before each model call and at each final answer, so it can run more than once for one message. Keep it a quick check with no side effects. What still waits when the turn ends runs as its own turn, before other queued turns. A cancelled one settles aborted and does not run.

A session.runTool() turn takes no messages. It runs no model, so nothing could answer them. A message that arrives while it runs waits, and then runs as its own turn. See Run a tool without a model.

A turn that routing sends to root agents takes messages, the same as the main model. The root agents get them at each model call.

A joined message runs with the credentials of the sender of the turn. So without turn.canJoin, a waiting message joins the running turn only when the same person sent both. A message from another person waits and runs as its own turn after the running one, as its sender. Joins are a prefix, so it also holds back the messages that wait after it. Set canJoin to choose your own rule. Compare user with turnUser to keep one person's message out of another person's turn. See Keep other people out of a running turn.

onJoin runs before the model call that the joined messages reach. It can return messages, which come after the joined ones, and records, which land in the same append as the join.

• If onJoin throws, the turn fails, and those messages run as their own turns.
• If you cancel the turn while onJoin runs, the hook adds nothing to it, and those messages run as their own turns.

const selective = defineHarness({
  name: 'acme/selective',
  adapter: openaiText('gpt-6.1-sol'),
  turn: {
    canJoin: ({ message }) =>
      typeof message === 'string' && message.length <= 500,
    onJoin: ({ inputs }) => ({
      messages: [
        {
          role: 'user',
          content: `${inputs.length} new message(s) arrived. Answer them before you continue.`,
        },
      ],
      records: [{ type: 'app.join', count: inputs.length }],
    }),
  },
})

Tools or system prompts that a plugin or a middleware adds during a turn keep the prompt cache on models with a mid-conversation channel. See Mid-conversation changes.

Run tools one at a time#

The model can call several server tools in one model call. By default, they run at the same time. Some tools must not overlap, for example a tool that writes a file and a tool that runs the tests. Set toolExecution: 'sequential' on the harness:

import { toolDefinition } from '@tanstack/ai'
import { defineHarness } from '@tanstack/ai-harness'
import { openaiText } from '@tanstack/ai-openai'
import { z } from 'zod'

const writeFile = toolDefinition({
  name: 'write_file',
  description: 'Write a file',
  inputSchema: z.object({ path: z.string(), content: z.string() }),
}).server(async ({ path }) => ({ written: path }))

const runTests = toolDefinition({
  name: 'run_tests',
  description: 'Run the test suite',
  inputSchema: z.object({}),
}).server(async () => ({ passed: true }))

const coder = defineHarness({
  name: 'acme/coder',
  adapter: openaiText('gpt-6.1-sol'),
  tools: [writeFile, runTests],
  toolExecution: 'sequential',
})

With both values, the model gets the results in the order of its calls. Client tools do not change.

Use 'sequential' when:

• a tool must see the changes of an earlier tool in the same model call, for example run_tests after write_file.
• two identical calls must share one toolCacheMiddleware entry. When they run at the same time, both calls miss the cache.

Keep 'parallel' for slow tools that do not depend on each other. Then a model call with three web fetches takes as long as the slowest fetch.

The setting applies to every turn of the harness. overrides cannot change it for one prompt. To set it on chat() without a harness, see Run tools one at a time.

Give one prompt its own settings#

Most prompts are fine with the harness settings. One job can need more: a stronger model for a review, more reasoning, or a tool that only this job uses. Pass overrides to session.send:

import { toolDefinition } from '@tanstack/ai'
import { createHarnessHost } from '@tanstack/ai-harness'
import { memoryPersistence } from '@tanstack/ai-persistence'

const readDiff = toolDefinition({
  name: 'read_diff',
  description: 'Read the diff of the open pull request',
}).server(async () => 'diff --git a/src/cart.ts b/src/cart.ts')

const host = createHarnessHost({ persistence: memoryPersistence() })
const session = await host.open(resilient, { threadId: 'thread-1' })

const review = await session.send('Review the open pull request.', {
  overrides: {
    adapter: anthropicText('claude-opus-5-5'),
    reasoning: 'high',
    tools: [readDiff],
  },
})
console.log(review.text)

The overrides apply to this turn only. The next prompt without overrides uses the harness settings again. The type is TurnOverrides, from @tanstack/ai-harness.

Every model call of the turn uses the overrides:

• each step of the tool loop
• each onModelError retry
• each beforeFinish cycle
• the main model after the routed agents, with routing.afterAgentsAnswer: 'main-answers'
• the extra model call for steer messages that arrive at the final answer

The promptCache of the overrides also reaches the agents that the model calls as tools in this turn.

Tools for one turn#

• A tool name that the turn already has fails the turn, the same as two tools with one name.
• A durable tool gets step and append.
• A middleware onConfig that returns a new tool list does not drop these tools. The harness adds them back.

Where the overrides live#

The session log keeps the JSON overrides: model, modelOptions, reasoning, and promptCache. adapter and tools are objects, so they stay in memory only.

• A queued prompt keeps its overrides until its turn runs.
• A steer that joins a running turn uses the overrides of that turn. The session ignores the overrides of the steer.
• A send(text, { mode: 'steer', overrides }) that does not join runs as its own turn, with its own overrides.
• After a restart, recovery runs an unfinished turn with its JSON overrides. It runs without adapter and tools, with a warning. To give them again, return them from durability.recover. See Decide recovery yourself.
• A second prompt with the same inputId and message is a duplicate, also when its overrides are different.

To keep a model or instructions for every turn of a thread, also after a restart, store them with session.configure. See Store settings per thread.

What you have now#

• A turn that goes back to work until the job is done, with a limit.
• Model errors that retry with a backoff, a partial answer that the model continues, and a compaction after a context overflow.
• Waiting messages that join a turn only when you allow it.
• Tools that run one at a time when they must not overlap.
• A prompt that runs with its own model, reasoning, prompt cache, and tools.