Skip to main content

Structured output & tool calling

So you asked Gemini Nano for JSON and got back a paragraph, a code fence, and a small apology on top. Structured output ends that: you hand the model a JSON Schema as responseConstraint, and the runtime constrains the reply to that schema — or the call throws, instead of handing you prose to regex. This lesson forces valid JSON out of the Prompt API, then drives real JavaScript functions with the intent loop — the tool-calling pattern that actually works in the browser today.

What you'll build​

  • Constrain a session to a JSON Schema with responseConstraint and parse the result safely.
  • Read the four JSON Schema keywords that matter here: type, properties, enum, required.
  • Build tool calling out of responseConstraint: make the model emit a { toolName, args } intent, then run the function yourself.
  • Run that as an intent loop that dispatches to local functions turn by turn, capped so it can't spin forever.
Prerequisites

Desktop Chrome with built-in AI, and you've already met the session lifecycle in The Prompt API. If LanguageModel isn't there yet, start with Setup & the availability lifecycle; for which APIs are stable on which Chrome, see the compatibility matrix.

Force JSON with responseConstraint​

Here's the move everyone reaches for first: prompt the model in plain English, then parse whatever comes back with a regex you'll be maintaining forever. Don't. The reliable move has a name — constrain, don't parse. You pass a JSON Schema as responseConstraint on the prompt() call, and the runtime constrains the reply to that schema. The spec is blunt about the deal: the result is a string you can hand to JSON.parse(), and if the model can't produce something schema-compliant, the call throws a SyntaxError DOMException instead of returning junk. One catch: the reply is a string, so you parse it. The fence-stripping guard below is belt-and-suspenders — it earns its keep on any prompt() you didn't constrain, where an unconstrained model still likes to wrap JSON in a code fence.

A schema and a parse. That's the whole trick. The schema below leans on four JSON Schema keywords — type, properties, enum, required — broken down in Read a JSON Schema right after.

demo.js
// availability() → create() → use → destroy(): the arc from every lesson.
if ((await LanguageModel.availability()) !== 'available') return;

// The shape you want back, described as JSON Schema.
const EXTRACT_SCHEMA = {
  type: 'object',
  properties: {
    sentiment: { type: 'string', enum: ['positive', 'neutral', 'negative'] },
    topics: { type: 'array', items: { type: 'string' } },
    summary: { type: 'string' },
  },
  required: ['sentiment', 'topics', 'summary'],
};

const session = await LanguageModel.create({
  expectedInputs:  [{ type: 'text', languages: ['en'] }],
  expectedOutputs: [{ type: 'text', languages: ['en'] }],
  initialPrompts: [
    { role: 'system', content: 'You extract structured data from short text. Reply with JSON only.' },
  ],
});

// responseConstraint forces the reply to match EXTRACT_SCHEMA.
const raw = await session.prompt(`Extract from this review:\n\n${review}`, {
  responseConstraint: EXTRACT_SCHEMA,
});

const data = parseJson(raw); // the reply is a STRING — parse it (guard below)
console.log(data.sentiment, data.topics, data.summary);

session.destroy();

// The model sometimes wraps JSON in a fence — peel it off, then parse.
function parseJson(text) {
  const s = String(text).trim();
  const fenced = s.match(/^`{3}(?:json)?\s*\n?([\s\S]*?)\n?`{3}$/);
  return JSON.parse(fenced ? fenced[1].trim() : s);
}

Read a JSON Schema​

You don't need all of JSON Schema. To constrain Nano you need four keywords, and here's what each one buys you:

  • type — the kind of value. type: 'object' at the top means the reply is a JSON object; nest type: 'array' with items for lists, the way topics does above.
  • properties — the fields and their types. This is the shape the parse will hand you back.
  • enum — a closed set of allowed values. sentiment can only be positive, neutral, or negative; the model can't invent mildly-annoyed.
  • required — which fields must appear. Anything not listed is optional and may be dropped.

Add additionalProperties: false when you want to forbid stray keys — the intent schema two steps down uses exactly that.

Wire real functions with the intent loop​

Structured output is the model talking. Tools are the model doing. A JSON reply can't reach your database, hit a pricing endpoint, or read today's weather — a function can. The Prompt API explainer sketches a tools option on create() for exactly this, but Chrome hasn't wired it up on-device: pass it and the model either ignores your function or the on-device process falls over. So you don't wait on it. You build tool calling out of the one thing that's already rock solid — responseConstraint — and run the functions yourself.

The trick is to make the model emit its intent as JSON and run the tools yourself. One schema — { toolName, args, reply }, with toolName required. A system prompt that lists the tools and says: emit one JSON object per turn, toolName: "done" when you're finished, no code fences. Then you loop — prompt, parse, run the named function in plain JavaScript, feed the result back as the next prompt. Cap it at eight turns so a confused model can't spin forever.

demo.js
// One schema: which tool to call, its args, and the final reply.
const INTENT_SCHEMA = {
  type: 'object',
  required: ['toolName'],
  additionalProperties: false,
  properties: {
    toolName: { type: 'string', description: 'Tool to call next, or "done" to reply.' },
    args: { type: 'object', description: 'Arguments for the tool.' },
    reply: { type: 'string', description: 'Final answer — only when toolName is "done".' },
  },
};

// Local tools, keyed by name. Each takes args and returns a string.
const TOOLS = {
  getWeather: ({ city }) => JSON.stringify({ city, tempC: 21, sky: 'clear' }),
  calculate: ({ expression }) => JSON.stringify({ result: evalMath(expression) }),
};

const system = [
  'You answer by calling tools. Emit ONE JSON object per turn — no code fences.',
  'Tools: getWeather({ city }), calculate({ expression }).',
  'Call one tool per turn. When you can answer, emit { "toolName": "done", "reply": "..." }.',
].join('\n');

// Gate before create(): availability() is read-only and never downloads; create()
// is what pulls the model — and starting that download needs a user gesture, so run
// this from a click handler. Pass the SAME options to availability() and create().
const opts = {
  expectedInputs:  [{ type: 'text', languages: ['en'] }],
  expectedOutputs: [{ type: 'text', languages: ['en'] }],
};
if ((await LanguageModel.availability(opts)) === 'unavailable') return;

const session = await LanguageModel.create({
  ...opts,
  initialPrompts: [{ role: 'system', content: system }],
});

let next = question; // the user's question
for (let i = 0; i < 8; i++) {
  // responseConstraint rides on each prompt(), not create()
  const step = parseJson(await session.prompt(next, { responseConstraint: INTENT_SCHEMA })); // fence-strip guard from step 1
  if (!step || step.toolName === 'done') {
    render(step?.reply ?? 'Done.');
    break;
  }
  const tool = TOOLS[step.toolName];
  const result = tool ? tool(step.args ?? {}) : `Unknown tool "${step.toolName}".`;
  next = `Result of ${step.toolName}: ${result}. Call the next tool, or emit {"toolName":"done","reply":"..."}.`;
}

session.destroy();

Why the loop, and not native tools​

Same model, same JSON Schema muscle you already used for structured output — the loop just moves the dispatch into a for loop you can watch. And because it leans only on responseConstraint, which is shipped, it runs the same on a stranger's Chrome as it does on yours. The tools option in the explainer would be less code if it worked; until Chrome ships it on-device, the loop is the tool-calling pattern that actually runs today. You also get something native tools hide: every call, its args, and its result pass through your own JavaScript, so you can log each step and step in.

Try it

Run it locally: open 04-structured-output-and-tools/index.html from the chrome-ai-course repo in desktop Chrome. Or use the hosted demo: Tool calling demo.

Expected: the Extract panel turns a pasted review into parsed { sentiment, topics, summary } fields; then a multi-step question (a little percent math plus the weather in London) drives the assistant while a step log shows each getWeather or calculate call, its args, and its result before the final reply.

Requires: desktop Chrome with Gemini Nano available — see Setup & the availability lifecycle.

Gotchas & troubleshooting​

Your parse throws on valid-looking JSON

Symptom: JSON.parse() throws Unexpected token on output that looks like JSON. Cause: you parsed a reply from a prompt() call with no responseConstraint — an unconstrained model wraps JSON in prose or a code fence. A constrained call either returns schema-compliant JSON or throws SyntaxError, so it won't feed you a fence. Fix: constrain the call, and keep the fence-stripping parseJson from step 1 for any call you left unconstrained. Never call bare JSON.parse() on model output.

Passing tools to create() does nothing (or worse)

Symptom: you hand create() a tools array with execute handlers, and the model never calls them — or the on-device process falls over mid-prompt. Cause: the explainer defines native tool calling, but Chrome hasn't shipped it on-device yet. Fix: don't pass tools; drive your functions with the intent loop, which needs only responseConstraint.

Tool result comes back as [object Object]

Symptom: the model reasons over garbage. Cause: your tool returned an object and you fed it straight into the next prompt. Fix: JSON.stringify() the result to a string before you feed it back — the way every tool in the loop above does.

create() hangs, or GPU memory runs out

Symptom: the first create() never resolves, or after a few runs create() starts rejecting. Cause: the first call blocks on a multi-GB model download, and orphaned sessions hold GPU memory. Fix: wire a monitor and show progress (e.loaded is a 0..1 fraction — multiply by 100), and destroy() every session on teardown.

Recap​

  • responseConstraint — a JSON Schema you pass to prompt() — constrains the reply; the reply is a string, so JSON.parse() it (or the call throws SyntaxError on a schema it can't meet).
  • The four keywords that matter: type, properties, enum, required.
  • Chrome hasn't shipped the tools option on-device, so tool calling is a manual pattern you build yourself.
  • The intent loop — responseConstraint plus a { toolName, args, reply } schema plus a capped prompt/parse/run/feed cycle — is that pattern, and it runs everywhere responseConstraint does.

Structured output and the loop are the same instinct pointed two directions: stop hoping the model behaves, and take away its room to misbehave. The parser you never wrote is the one that never breaks.

Next steps​

Next: Multimodal: image input

Last updated on