Skip to main content

The Prompt API (LanguageModel)

Ok. The Prompt API — the bare LanguageModel global — is the one you actually came for: the raw chat surface onto Gemini Nano, running on your own GPU with no OpenAI key, no Anthropic invoice, and no network request at all. Conversational state, a system prompt, streaming, tuning, cloning, a token budget. By the end of this page you'll have a working on-device chat and know every knob worth turning.

What you'll build​

  • Create a LanguageModel session behind an availability() gate, declaring languages with expectedInputs/expectedOutputs.
  • Steer output with a system prompt and N-shot initialPrompts.
  • Stream replies with promptStreaming() and append the deltas (never replace).
  • Tune temperature and topK from params() (Extensions / Origin Trial only), and fork a session with clone().
  • Track contextUsage against contextWindow, survive context overflow, and destroy() on teardown.
Prerequisites

Desktop Chrome with built-in AI switched on. If LanguageModel isn't there yet, work through Setup & the availability lifecycle first, and check the compatibility matrix for what's stable where. This page assumes you know the availability states — it won't re-teach them.

Create a session​

The loop is always the same: ask whether the model is there, create a session, use it, tear it down. Feature-detect the global, then declare the languages you'll use with expectedInputs and expectedOutputs and pass that same declaration to both availability() and create() — availability() can only answer for the modalities and languages you actually intend to use, so the two calls must agree.

demo.js
if (typeof LanguageModel === 'undefined') {
  // No built-in AI on this browser — degrade gracefully (see Setup & availability).
}

// Declare the modalities/languages once, then pass the SAME object to
// availability() and create() — availability() only answers for what you'll use.
const expectations = {
  expectedInputs:  [{ type: 'text', languages: ['en'] }],
  expectedOutputs: [{ type: 'text', languages: ['en'] }],
};

const status = await LanguageModel.availability(expectations);
// "unavailable" | "downloadable" | "downloading" | "available"

// A user gesture is required to START the download, so kick create() off from a
// click handler. (Once the model is already downloaded, create() needs no fresh
// gesture — but gating on the click keeps the first run working too.)
button.addEventListener('click', async () => {
  if (status === 'unavailable') return;

  const session = await LanguageModel.create({
    ...expectations,
    monitor(m) {
      m.addEventListener('downloadprogress', (e) => {
        // e.loaded is a 0..1 fraction; e.total exists and is always 1.
        console.log(`Downloading model… ${Math.round(e.loaded * 100)}%`);
      });
    },
  });

  const reply = await session.prompt('Say hi in one word.');
  console.log(reply);
  session.destroy();
});

First create() on a fresh machine blocks on a multi-GB download, and the download only starts from a user gesture. That's why create() lives in a click handler, and why the monitor is wired from line one, not bolted on later.

Set a system prompt and initial (N-shot) prompts​

The system prompt is how you hand the model a job. It goes into initialPrompts as a {role:'system'} message — and it has to be first. Index 0, exactly once. Put it anywhere else, or pass two, and create() throws a TypeError before you ever hold a session. There is no top-level systemPrompt string; that shape is legacy, gone.

After the system message you can prime the model with example turns — user says this, assistant answers that — so it copies the pattern instead of guessing at it. That's N-shot prompting, and it's the cheapest steering you'll ever do.

demo.js
const session = await LanguageModel.create({
  expectedInputs:  [{ type: 'text', languages: ['en'] }],
  expectedOutputs: [{ type: 'text', languages: ['en'] }],
  initialPrompts: [
    { role: 'system', content: 'You are a concise, friendly assistant.' },
    { role: 'user', content: 'ping' },
    { role: 'assistant', content: 'pong' },
  ],
});

// The system message steers every turn and survives context overflow.
const reply = await session.prompt('marco');

Choose prompt() or promptStreaming()​

Two ways to get text out. prompt() hands you the whole reply once it's done — one await, one string. promptStreaming() hands you a ReadableStream and you loop over it with for await. Here's the part people get wrong: the chunks are deltas, not snapshots. Each chunk is the next slice of text, so you accumulate them. Render the raw chunk on its own and the previous words vanish.

Append. Don't replace.

demo.js
// Whole reply at once:
const reply = await session.prompt('Write a haiku about TypeScript.');

// Or stream incremental deltas and append them:
let text = '';
const stream = session.promptStreaming('Write a haiku about TypeScript.');
for await (const chunk of stream) {
  text += chunk; // deltas — append, never replace
  render(text);
}

Tune temperature and topK from params()​

Two decoding knobs. temperature is how much the model gambles — low is deterministic and repetitive, high is creative and occasionally unhinged. topK caps how many candidate tokens it samples from each step.

One catch, and it's a big one: these knobs only apply to the Prompt API for Chrome Extensions, or on the open web with the Origin Trial enabled. On a plain web page they're deprecated — pass them and the runtime logs a warning and ignores them, and LanguageModel.params() isn't even there. So feature-detect params(): if it's present you're in a context that honours the knobs and it hands you the defaults and ceilings; if it's absent, skip them entirely.

demo.js
const options = {
  expectedInputs:  [{ type: 'text', languages: ['en'] }],
  expectedOutputs: [{ type: 'text', languages: ['en'] }],
  initialPrompts: [{ role: 'system', content: 'You are a concise, friendly assistant.' }],
};

if (typeof LanguageModel.params === 'function') {
  // params() only exists for Extensions / the Origin Trial — the only places
  // temperature/topK actually do anything.
  const params = await LanguageModel.params();
  // { defaultTopK, maxTopK, defaultTemperature, maxTemperature }
  options.temperature = params.defaultTemperature;
  options.topK = params.defaultTopK;
}

const session = await LanguageModel.create(options);

Where they apply, these are locked in at create(). Want different settings mid-chat? New session.

Fork a conversation with clone()​

clone() branches a session. The copy inherits the system prompt, the initial prompts, and every turn up to the moment you cloned; after that the two diverge and never speak again. It's cheaper than rebuilding a primed session from scratch, and it's the honest way to run "give me two answers to the same question" or to keep an undo snapshot before a risky turn.

demo.js
const base = await LanguageModel.create({
  expectedInputs:  [{ type: 'text', languages: ['en'] }],
  expectedOutputs: [{ type: 'text', languages: ['en'] }],
  initialPrompts: [{ role: 'system', content: 'You are a concise, friendly assistant.' }],
});

const branch = await base.clone(); // shares history up to now, then diverges

const [a, b] = await Promise.all([
  base.prompt('Pitch me a startup name.'),
  branch.prompt('Pitch me a startup name.'),
]);

base.destroy();
branch.destroy();

Track tokens, quota, and context overflow​

A session has a budget — contextWindow input tokens, a fixed ceiling you read off the session rather than guess — and contextUsage climbs as the conversation grows. Watch the gap. When a new prompt would blow past the quota, the session quietly evicts your oldest turns to make room, one user/assistant pair at a time. The system prompt is never evicted; that's the whole reason it's special. And if even that can't free enough room, you get a QuotaExceededError and nothing is dropped.

demo.js
console.log(`${session.contextUsage} / ${session.contextWindow} input tokens`);

try {
  const reply = await session.prompt(veryLongText);
} catch (e) {
  if (e.name === 'QuotaExceededError') {
    // Couldn't evict enough to fit — start fresh (or summarize the history first).
    session.destroy();
    session = await LanguageModel.create({
      expectedInputs:  [{ type: 'text', languages: ['en'] }],
      expectedOutputs: [{ type: 'text', languages: ['en'] }],
      initialPrompts,
    });
  }
}

Two fields do the whole job: session.contextUsage is what you've spent, session.contextWindow is the ceiling. Watch the gap and you'll never be surprised by an overflow.

Destroy the session​

A session pins real GPU memory, and Nano isn't small. Leave one hanging and you starve every other feature on the page. So call destroy() when you're done and wire it to teardown, so a closed tab doesn't leak. One catch: a destroyed session is gone. Touch it again and the call rejects with an AbortError.

Recreate. Don't resurrect.

demo.js
session.destroy();

// Wire it to teardown so a closed tab doesn't leak GPU memory:
window.addEventListener('beforeunload', () => session?.destroy());

// Prompting after destroy() rejects with AbortError — create a new session instead.
Try it

Run it locally: open 03-prompt-api/index.html from the chrome-ai-course repo in desktop Chrome. Or use the hosted demo: on-device chat (with the API walkthrough).

Expected: you type a prompt, the reply streams in a few words at a time, and a token readout updates after each turn — tokens used out of the session's contextWindow.

Requires: desktop Chrome with Gemini Nano available — see Setup & the availability lifecycle.

Gotchas & troubleshooting​

Most of these are the same two mistakes wearing different hats: you didn't wait for the download, or you treated a streaming delta like a snapshot.

create() never resolves

Symptom: the first create() hangs forever with no error. Cause: two of them. Either the multi-GB model is downloading and you didn't wire a monitor, so there's nothing visible to wait on — or you called create() outside a user gesture on a downloadable model, so the download never even started. Fix: kick create() off from a click handler (a user gesture is required to start the download), always pass monitor(m), and read e.loaded (a 0..1 fraction, with e.total always 1) for progress. create() resolves when the download finishes.

The streamed reply shows only the last few characters

Symptom: the output flickers and ends on a fragment. Cause: you're replacing the output with each chunk, but the chunks are deltas, not the full text so far. Fix: append — text += chunk — then render the accumulated text.

create() throws a TypeError

Symptom: a TypeError before you ever get a session. Cause: your {role:'system'} message isn't at index 0 of initialPrompts, or there's more than one. Fix: exactly one system message, first in the array. No top-level systemPrompt.

QuotaExceededError on a long chat

Symptom: a prompt rejects with QuotaExceededError. Cause: even after evicting the oldest turns, the history plus the new prompt won't fit contextWindow. Fix: destroy and start a fresh session, or run the history through the Summarizer and prime a new session with the summary.

AbortError when prompting

Symptom: AbortError on prompt(). Cause: you already called destroy() on that session. Fix: sessions don't come back — create a new one.

create() suddenly throws NotSupportedError

Symptom: create() starts throwing NotSupportedError for calls that worked a minute ago — even option-less ones — and availability() flips to unavailable. Cause: the on-device model process crashed enough times to trip an undocumented cooldown. Fix: stop retrying in a tight loop. Check chrome://crashes, wait a few minutes — it clears itself.

Recap​

  • availability() gates create(); every create() declares languages with expectedInputs/expectedOutputs.
  • The system prompt lives at initialPrompts[0], exactly once; N-shot examples follow as user/assistant pairs.
  • promptStreaming() yields deltas — accumulate them, don't replace.
  • params() gives you the temperature/topK ranges — Extensions / Origin Trial only; feature-detect it, and on a plain page skip the knobs.
  • clone() forks a primed session; contextUsage/contextWindow track the budget; overflow evicts oldest turns or throws.
  • destroy() frees GPU memory, and a destroyed session is done for good.

Everyone else is still renting tokens.

Next steps​

Next: Structured output & tool calling.

Last updated on