Grok Voice Agent API: How to Call It and Feed It Live Twitter (X) Data

On docs.x.ai the product is filed under Voice as "Speech to Speech" and the announcement on x.ai/news/grok-voice-agent-api calls it the Grok Voice Agent API. Both describe the same thing: a real-time, full-duplex speech model you talk to over a WebSocket, with no separate speech-to-text and text-to-speech hops in your code. The docs overview lists three voice products side by side: Speech to Speech (grok-voice-latest, the realtime WebSocket endpoint on api.x.ai), Speech to Text (grok-voice-transcribe-2.0, a REST endpoint) and Text to Speech (REST). This page covers the first.

Model names. grok-voice-latest is the alias to use; the models page resolves it to grok-voice-3.0 and marks grok-voice-2.0 deprecated. The announcement does not print a model id at all, which is why some search results guess; the ids above are from the docs model table.

Protocol. One WebSocket: wss://api.x.ai/v1/realtime?model=grok-voice-latest, authenticated with Authorization: Bearer <XAI_API_KEY> from a server, or with an ephemeral token when a browser or phone connects directly. The first message you send is session.update, which carries instructions (the system prompt), voice, turn_detection (server_vad for automatic end-of-speech, null for push-to-talk), input and output audio formats, and the tools array. The docs list the PCM rates 8000, 16000, 22050, 24000, 32000, 44100 and 48000 Hz, with 24000 as the default, plus audio/pcmu and audio/pcma for telephony and audio/opus.

Pricing, as printed on the Speech to Speech pricing page. Audio is $0.08 per minute or $4.80 per hour. Text input is $0.004 per conversation.item.create event, with the page's own exception that function_call_output items, the tool results the server asked for, are not billed. Sessions using server_vad are billed for the session duration; push-to-talk sessions are billed only for audio sent and received. The December 2025 announcement quoted a flat $0.05 per minute and compared it against a $0.10 per minute blended estimate for a competitor, so the price has moved once already; read the pricing page, not a blog, before you budget.

Limits. Concurrent sessions: 10 per team. Maximum session duration: 120 minutes. Region: us-east-1. The announcement claims an average time to first audio under one second; we did not measure it and quote it as their number.

Voices. The announcement names Ara, Eve and Leo; the docs use eve in examples and point to a voices roster endpoint under the text-to-speech API that also covers custom voice ids.

Built-in tools. The tools array accepts function (your own, with a JSON schema), web_search, x_search (described as searching X for posts and information, with handle and date filtering), file_search over uploaded collections, and mcp for remote Model Context Protocol servers. The existence of x_search matters for this page and we come back to it at the end: it is the model looking things up for itself, and it is not the same as your code receiving structured X data it can log, cache, filter or act on.

Transports. The docs overview and the announcement name LiveKit (an official plugin), and the overview also mentions Pipecat, Twilio and Voximplant integrations for phone and WebRTC audio. We did not find a "Voice Agent Builder" on the docs pages or the announcement we fetched, although some search results use that phrase; if you need a no-code builder, verify it on x.ai directly rather than from this page.

The event flow is the one thing to get exactly right, because a mistake here produces an agent that goes silent rather than one that errors. The docs describe it in four steps, and the names below are quoted from the Speech to Speech page.

1. Declare. Put each tool in session.update under tools with "type": "function", a name, a description the model reads to decide when to call it, and a parameters JSON schema. The message below declares the four X tools from this page; the instructions string also tells the model how to speak numbers and how many posts to read, which is where most of the "voice context" work happens.

2. Receive the call. When the model decides it needs data it emits response.function_call_arguments.done carrying name, call_id and arguments (a JSON string). If it needs several tools for one question it emits several of these before any audio.

3. Answer every call. For each call_id, send conversation.item.create with an item of type function_call_output, the same call_id, and output, which must be a string; serialize JSON into it. The docs are explicit that with parallel calls you resolve all of them and send every function_call_output before the next step.

4. Ask for the spoken answer. Send response.create. The model reads the tool outputs and streams response.output_audio.delta events back. The pricing page notes that the function_call_output items are not charged as text input, so returning four tool results costs nothing beyond the audio minutes of the turn.

Two documented extras are useful for a tool-heavy agent. conversation.item.create with an item of type force_message makes the agent say a given text verbatim; the docs give compliance disclosures and IVR prompts as the use case, and a short "let me check" before a slow fetch is the same mechanism. And reasoning.effort can be set to none, which matters when the tool output is already the answer and you want it spoken, not reasoned about.

Each tool is one HTTP GET against api.twitterapi.io with an X-API-Key header; no X developer account, app review or user login is involved for reads. The four cover what people actually ask a voice assistant about an account: who they are (user_info), what they posted (last_tweets), what people say to them (mentions), and whether they said something specific (search_from, an advanced_search with a from: operator). All four return a compact JSON string, because that is what function_call_output.output must be.

The response shapes differ per endpoint and the code reads them exactly: /twitter/user/info returns data{...} with followers, statusesCount, isBlueVerified and so on, and answers an unknown handle with HTTP 200, status: "error" and data: null, never a 404, so the tool checks status rather than the HTTP code. /twitter/user/last_tweets returns data.tweets[], 20 per page newest first, with has_next_page and next_cursor; /twitter/user/mentions and /twitter/tweet/advanced_search return a top-level tweets[]. Tweet objects carry likeCount, retweetCount, replyCount, viewCount and a nested author; there is no public_metrics.

The limit argument defaults to 5 and the schema tells the model to stay between 1 and 5. A page still returns 20 posts and is billed for 20 (15 credits each, $0.003 for a full page), so the limit is about what the model hears, not what you pay; the next section is about why five is already generous for speech.

A protected account is a real case: @xai returned a profile with protected: true and an empty timeline during our run. The tool turns that into a tiny object with a note the model can read aloud, instead of an empty list the model might narrate as "they have not posted". The empty search_from result for the same handle shows the other edge: an empty page is billed at the 15-credit minimum ($0.00015), not zero.

A raw tweet from /twitter/user/last_tweets has about 30 keys. The first GitHub post in our sample was 4,183 bytes on its own: the text, then an author object with the account's full profile repeated on every post, entities with URL expansions and mention offsets, extendedEntities.media, a link card, quoted_tweet and retweeted_tweet sub-objects when present, and two URL forms. Twenty of those make the 118,744-byte page we measured. None of it is wrong; almost none of it can be spoken.

The projection keeps seven tweet fields: text, createdAt, likeCount, replyCount, retweetCount, viewCount and isReply. For mentions it adds the author's userName, because "who said it" is the answer there. For profiles it keeps userName, name, followers, following, statusesCount, description, isBlueVerified and createdAt. Measured over the whole run, trimmed posts average 344 bytes each against 5,937 raw; trimmed profiles are 272 bytes against 1,046.

Dropped on purpose, and why. author on timeline posts (it is the same account twenty times; the agent already knows who it asked about). entities and extendedEntities (offsets and media URLs cannot be spoken; say "with a video" from extendedEntities.media[].type only if you want that, and we did not). url and twitterUrl (a voice agent cannot read a link; if you need it for a follow-up text message, keep id and rebuild the URL later). card, quoted_tweet, retweeted_tweet (useful for a screen, noise for an ear; keep retweeted_tweet.author.userName if "they reposted X" matters to you). bookmarkCount, quoteCount, lang, conversationId, inReplyToId and the rest of the threading fields. From profiles: profilePicture, coverPicture, pinnedTweetIds, entities, favouritesCount, mediaCount, canDm, isAutomated, location and url.

How much can be spoken anyway. The 400 timeline posts in the sample have a median text length of 27 words (181 characters), p90 about 47 words. At a conversational 150 words per minute the median post takes about 10.8 seconds to read aloud, so a 30-second answer fits roughly 2 full posts plus a sentence of framing. That is the argument for limit defaulting to 5 and the instructions saying "at most three": the model summarizes the rest, and anything past five posts is tokens the agent reads and nobody hears.

Numbers for the ear. Leave the integers exact in the JSON (the model rounds well when told to) and put the rounding rule in instructions: "2.7 million followers", not "two million seven hundred twenty-one thousand". The same goes for createdAt: keep the raw timestamp and let the model say "posted this morning"; converting to relative time in the tool bakes in a timezone the caller may not share.

We ran the four tools against GitHub, NASA, Vercel, Shopify, Stripe, Cloudflare, NVIDIA, OpenAI Developers, Docker, AWS and the protected @xai account: one profile call, up to two timeline pages, one mentions page and one from: search page each, 54 requests in total, 820 posts and 11 profiles returned. Every response was serialized with compact JSON separators before counting bytes, both raw and after the projection above, so the comparison is like for like. Tokens are approximated as bytes divided by four, the usual rule of thumb for English JSON; your tokenizer will differ, the ratio will not.

Three things stand out. First, the timeline page is the heaviest by a wide margin at about 29,686 tokens raw, because every post repeats the full author profile and organization accounts attach cards and media to most posts; a mentions page is lighter (60,144 bytes) because replies from individuals carry fewer attachments. Second, after trimming the three tweet tools converge to about 6,900 bytes for 20 posts, roughly 1,720 tokens, which is the size of a long paragraph; with limit=5 a tool result is a quarter of that. Third, the whole run shrank from 4.24 MB to 283 KB, 93.3% less, which is the difference between a tool layer that could be pasted into a voice turn and one that could not.

Why this matters for a voice model specifically: a chat model reading 69,622 tokens of tool output is slow and expensive but works; a speech model that has to start talking within a second is a different constraint, and the xAI docs' own advice on tool output is to avoid excessive verbosity to protect latency. The projection is how you follow that advice without losing the fields the answer depends on.

Wall-clock was measured around each call in the rate-limited client we use for page studies, after the limiter had granted the slot, so the number is the HTTP round trip plus serialization from one machine. The median over all 54 calls was 2.31 s, p90 2.77 s, slowest 3.18 s. Profile reads are the quick ones; 20-post pages take about a second more. Your tool layer will see different absolute numbers depending on where it runs, which is why the table shows the spread and not a single figure.

Cost. twitterapi.io bills 15 credits per tweet returned ($0.15 per 1,000 tweets, 15-credit minimum per request, $1 = 100,000 credits) and 18 credits per profile ($0.18 per 1,000). The official X API pay-per-usage rate card on docs.x.com/x-api/getting-started/pricing lists $0.005 per post read and $0.010 per user read, which is $5 per 1,000 posts against $0.15, about 33 times more per 1,000 tweets. A spoken question that fires all four tools therefore costs about $0.00918 here and $0.31 there; the whole 54-call study cost $0.1253 (12,528 credits) and would have been $4.21 at X's per-read prices. Note that trimming does not change what you pay: a page is 20 posts whether you keep 5 or 20, so set limit for the model's sake and pay per page.

Putting the voice minute next to it. At the docs' $0.08 per minute, a 2.53-second wait for four parallel tool calls is about $0.0034 of audio time with server_vad billing, which is in the same range as the $0.00918 the X reads cost. Sequential calls would make the wait 8.61 s, long enough that a caller assumes the line dropped, so dispatch tool calls concurrently (the loop below uses a thread per call) and consider a force_message acknowledgment before the fetch.

Rate limits on the tool side. A new twitterapi.io account is limited to 0.2 requests per second until the first payment; paid accounts get 20 requests per second, 50 on Scale and 500 on Business. Four parallel tool calls per turn, several callers at once, is a paid-tier pattern; at 0.2 requests per second the four calls serialize into 20 seconds and the agent is unusable. The sign-up credit ($1, 100,000 credits, no card, for new accounts that pass the automatic bot check, one per person or network, expiring after 31 days) covers development comfortably: the entire study here used an eighth of it.

The loop below is the smallest process that connects to grok-voice-latest, sends the session.update from earlier (saved as session.json), dispatches each function call to xtools.py in a worker thread, returns every function_call_output for the turn, and then sends response.create. Audio capture and playback are deliberately absent: the docs show those per transport, and LiveKit's plugin, Pipecat, Twilio and Voximplant each handle the microphone and speaker differently. Plug response.output_audio.delta into whichever player you use.

To repeat the caveat from the top: we have not run this against the Voice API, because we do not hold an xAI key. The event names are copied from the docs page fetched for this guide, and the tool side is tested; before shipping, diff the event names against the current docs, because real-time APIs rename events between versions more often than REST APIs rename fields.

Where the parallel-call rule bites. The model may emit two or three response.function_call_arguments.done events for a single question ("how big is Stripe and what did they post?" wants user_info and last_tweets). The loop collects them into pending, waits for the turn's response.done, resolves all outputs, sends every function_call_output, and only then one response.create. Sending response.create after the first output, which is the natural mistake, produces an answer that ignores the second tool.

Keep the output a string. json.dumps of a list or object is correct; passing the Python object is not, and the failure is silent in some clients. The tool functions already return strings for this reason.

Errors into speech, not into logs. A requests timeout or a 429 from the tool side should become a short JSON like {"error": "X data is slow right now"} returned as the output, so the agent says that instead of stalling. The sample raises on HTTP errors to keep it readable; wrap run_tool in a try/except in production.

The Voice API ships a built-in x_search tool, and the honest question is why you would write your own. Use x_search when the agent should look things up on X the way a person would, with no code on your side and no requirement to see what it found. Write function tools, like the four above, when any of these holds.

You need the numbers, not a paraphrase. A follower count, a view count or the exact text of the latest post comes back as structured JSON your process sees, logs and can compare with yesterday's. That is what makes "Stripe gained about 2,000 followers since last week" possible: the agent cannot remember, your database can.

You need to act on the result. Texting the caller a link to the post, opening a ticket when a mention is negative, or caching the timeline so the second caller in a minute costs nothing, all need the data in your hands. A built-in search result lives inside the model's turn.

You need predictable cost and scope. A function call is one endpoint, one page, one price. You decide whether a question is worth a 20-post page ($0.003) or a profile ($0.00018), and the instructions decide how often the model asks.

You need accounts, not topics. last_tweets and mentions are account-shaped; a search tool is query-shaped. For "what is NVIDIA saying about its new chip?" use search_from with a from: operator; for "what is everyone saying about the chip?" an advanced_search without from: is the right tool and its authors are private individuals, so keep that one for aggregates in a voice answer rather than reading strangers' posts aloud.

Both can coexist in one tools array: declare x_search for open-ended questions and the four functions for account questions, and tell the model in instructions which to prefer for which. If you do, log which the model chose for a week; in our experience with chat agents, the model's own search gets picked for questions you would rather have deterministic, and a sentence in the instructions fixes it.

If you are building the same agent on another stack, the tool layer transfers unchanged: it is plain functions returning strings, which is also what the OpenAI Agents SDK and the Vercel AI SDK want, and what an MCP server exposes to Claude.

Question. When a Grok voice agent calls X data tools during a spoken turn, how large is each tool result raw and trimmed, how long does each call take, and what does each call cost on twitterapi.io and on the official X API rate card?

Accounts and calls. Eleven organization accounts: GitHub, NASA, Vercel, Shopify, Stripe, Cloudflare, NVIDIA, OpenAI Developers, Docker, AWS and @xai, which turned out to be protected and stays in the sample as the edge case; AWS was added so that ten accounts had open timelines. Per account: one /twitter/user/info, up to two pages of /twitter/user/last_tweets, one page of /twitter/user/mentions and one page of /twitter/tweet/advanced_search with from:<account> and queryType=Latest. 54 requests, 820 posts and 11 profiles returned, 12,528 credits ($0.1253), collected in 2026.

Measurement. Every response body was re-serialized with compact JSON separators and counted in UTF-8 bytes, once raw and once after the projection described above; tokens are bytes divided by four. Wall-clock was taken around the client call after the rate limiter granted a slot, so it includes the HTTP round trip and not process start-up. Costs apply the published prices to the items each call returned; the X API figures use the pay-per-usage rate card ($0.005 per post read, $0.010 per user read) and are what the same reads would cost there, not something we ran.

What the sample cannot show. Latency from one machine in one network says nothing about yours; treat the spread as the finding, not the median. Eleven large organization accounts post with cards and media more than individuals do, so raw tweet sizes for a small account will be lower and the reduction percentage somewhat smaller. We did not run the Voice API, so nothing here measures Grok's time to first audio, its tokenization of tool output, or how it reads JSON aloud; the speakability numbers use a 150 words per minute reading rate, not a measurement of the model's voice.

Privacy. Timeline and search pages belong to organization accounts and are kept with the study. Mention pages are written by private individuals: they were measured for size, count and timing and then discarded; no mention text, author or id is stored, and no individual is named on this page.