World Labs

generateWorld() calls World Labs Marble. A prompt, image, or video returns a viewer URL, and often splat files, a mesh, and a panorama. The adapter has no chat.

1. Install#

npm i @tanstack/ai @tanstack/ai-worldlabs

pnpm add @tanstack/ai @tanstack/ai-worldlabs

yarn add @tanstack/ai @tanstack/ai-worldlabs

bun add @tanstack/ai @tanstack/ai-worldlabs

2. Set the API key#

Create a key in the World Labs platform. Put it in the environment of your server:

WORLDLABS_API_KEY=your-worldlabs-key

worldlabsWorld reads WORLDLABS_API_KEY. To pass the key in code, use createWorldLabsWorld:

import { createWorldLabsWorld } from '@tanstack/ai-worldlabs'

const adapter = createWorldLabsWorld('marble-1.1', process.env.WORLDLABS_API_KEY!)

3. Generate a world on the server#

Add a POST route at /api/world. The call waits until the world is ready, so give it a long timeout:

import { generateWorld } from '@tanstack/ai'
import { worldlabsWorld } from '@tanstack/ai-worldlabs'

export async function POST() {
  const world = await generateWorld({
    adapter: worldlabsWorld('marble-1.1'),
    prompt: 'A mystical forest with glowing mushrooms',
    threadId: 'thread-1',
    timeout: 10 * 60 * 1000,
  })

  return Response.json({
    url: world.url,
    thumbnailUrl: world.assets?.thumbnailUrl,
  })
}

World Labs bills in credits for each generation.

4. Show the world in the browser#

world.url is the Marble viewer (https://marble.worldlabs.ai/world/{id}). Open it in a new tab. Do not put it in an iframe:

const world = await fetch('/api/world', { method: 'POST' }).then((response) =>
  response.json(),
)

if (typeof world.url === 'string') {
  const link = document.createElement('a')
  link.href = world.url
  link.target = '_blank'
  link.textContent = 'Open in Marble'
  document.body.append(link)
}

if (typeof world.thumbnailUrl === 'string') {
  const image = document.createElement('img')
  image.src = world.thumbnailUrl
  image.alt = 'Preview of the generated world'
  document.body.append(image)
}

You now have a 3D world and a link to explore it.

A full app is in examples/ts-react-media. Open the World tab and pick a Marble model. The example loads one SPZ splat file in Spark (it prefers 500k) through /api/marble-splat. If there is no splat URL, it shows a thumbnail and an Open in Marble link.

Image, multi-image, and video#

prompt is the text. Put the media in modelOptions:

import { generateWorld } from '@tanstack/ai'
import { worldlabsWorld } from '@tanstack/ai-worldlabs'

const world = await generateWorld({
  adapter: worldlabsWorld('marble-1.1'),
  prompt: 'A cozy living room',
  threadId: 'thread-1',
  modelOptions: {
    image: { uri: 'https://example.com/room.jpg' },
    isPano: 'auto',
  },
})

Pass only one of image, images, or video. See Media options for each field.

Models and options#

Pass the id as a string literal, so TypeScript can narrow the options.

The package exports the list as WORLDLABS_WORLD_MODELS.

Media options#

Job options#

Optional metadata: displayName, seed, tags, disableRecaption, and permission.

Advanced#

What the result holds#

• world.url: the Marble viewer URL.
• world.assets: optional. It has splat, mesh, panorama, and thumbnail links only when the World Labs response includes them. These URLs are often signed CDN links.
• world.assets.splats.spzUrls: splat URLs by quality key, for example 100k, 500k, and full_res.

Jobs that do not wait#

On serverless, a job that runs for minutes can hit the limit of your platform. You have two choices:

• Pass a long timeout to generateWorld(), as in step 3.
• Set modelOptions: { wait: false } and poll the job yourself.

With wait: false, world.status is waiting and world.operationId is set. The adapter cannot resume that id. A new generateWorld() call with wait: true starts a new job. To get the result, poll GET /marble/v1/operations/{operationId} yourself.

Bring your own key#

worldlabsByok from @tanstack/ai-worldlabs/byok declares World Labs for the BYOK store, with WORLDLABS_API_KEY as its env var. See Bring Your Own Key.

API reference#

The config fields:

• apiKey: the World Labs API key.
• baseUrl: the API base URL. The default is https://api.worldlabs.ai.
• defaultHeaders: extra request headers.
• fetch: your own fetch function.