Iframe Sandbox Security: Lessons From the ChatGPT Takeover Bug

Last week a security team showed how a single ChatGPT share link could take over the account of anyone who opened it. The bug is fixed and OpenAI paid a $5,000 bounty. But how it worked deserves ten minutes of any product owner's time, because it's a textbook failure of iframe sandbox security, and most teams shipping AI features are building the same kind of surface right now.

If your product shows AI-generated HTML, charts, previews or embedded widgets, you're running untrusted code right next to trusted code. Here's what went wrong and what to check in your own app.

TL;DR#

• Parameter Labs chained a third-party math library, a loose postMessage handler and a CSP allowlist gap to run code on chatgpt.com from inside a sandboxed HTML preview.
• The sandbox itself held. The hole was a trusted page that accepted messages from any window and turned them into live HTML.
• If you render AI output or user content in iframes, check message origins and sources, never pass raw HTML across frames, and treat share links as attacker input.

What happened in the ChatGPT account takeover#

ChatGPT's HTML Preview feature (the one that renders interactive pages the model writes) runs that code in a sandboxed iframe on a separate origin. In theory, code in that frame can't touch the main chatgpt.com page. Mukul Goyal at Parameter Labs found a way around it. Simplified, the chain looked like this:

1. A graph in the same conversation loaded MathLive, a JavaScript math-input library, which registers a message listener on the main page.
2. That listener didn't check which window sent a message.
3. The sandboxed preview sent it a "settings update" with a custom keyboard layout that was really attacker HTML.
4. MathLive wrote that HTML into the page with innerHTML.
5. The injected HTML included a srcdoc iframe with no sandbox attribute, so it inherited the chatgpt.com origin.
6. ChatGPT's Content Security Policy blocked inline scripts but allowed scripts from youtube.com, and a YouTube endpoint reflected attacker-controlled JavaScript.
7. That script read the user's access token and pulled recent conversations.
8. All of it was packed into an ordinary public share link. In one version, opening the link was enough.

Parameter Labs reported it on October 18. According to the researchers' timeline, OpenAI raised it to P1 severity, fixed two web front ends and closed it on October 26. Simon Willison called the $5,000 bounty "a bit cheap for a vulnerability this serious."

Why iframe sandbox security failed here#

Here's the interesting part: the sandbox did its job. Code in the preview frame never broke out on its own. It asked a trusted component to do something dangerous, and the component said yes.

A sandboxed iframe isolates a document, but it can still talk to the outside world through window.postMessage. Every listener on your parent page is a door, and one that skips sender checks makes the sandbox decorative.

The third-party library problem#

Nobody at OpenAI wrote the vulnerable listener on purpose. It arrived inside a library loaded for math rendering. Your attack surface includes every script your page loads. The MDN postMessage docs have told developers for years to verify the sender. Libraries don't always listen.

CSP is a seatbelt, not a wall#

The Content Security Policy blocked inline script and still lost, because one allowlisted domain had an endpoint that echoed JavaScript back. Allowlisting a big domain means trusting every endpoint it hosts.

Technical breakdown: where each control sat#

Here's how the layers stacked up in this incident:

Five layers, four gaps. Defense in depth fails when small gaps happen to line up.

A safer message handler is short:

const PREVIEW_ORIGIN = "https://preview.example.com";

window.addEventListener("message", (event) => {
  if (event.origin !== PREVIEW_ORIGIN) return;
  if (event.source !== previewFrame.contentWindow) return;
  if (event.data?.type !== "resize") return; // allowlist message types
  previewFrame.style.height = `${Number(event.data.height) || 0}px`;
});

Notice what it refuses: HTML, arbitrary settings, and messages from any other window. One more trap from MDN's iframe reference: combining allow-scripts and allow-same-origin on same-origin content lets the framed code remove its own sandbox. Serve untrusted content from a separate origin.

What this means for businesses shipping AI features#

You don't have to be OpenAI to have this problem. Each of these has the same shape:

• A support portal that renders AI-drafted replies with formatting
• A dashboard where an agent generates charts or HTML reports
• A product builder that previews customer-supplied templates
• Third-party widgets (chat, payments, scheduling) messaging your page

AI makes the pattern more common because model output is effectively untrusted input. A prompt injection hidden in an email, PDF or web page can steer what the model writes, so an attacker can shape the HTML your app renders without touching your servers. If that output lands in a frame that can message a privileged page, you've rebuilt the same chain.

The ChatGPT payload went after an access token and private conversations. In your product, that might be customer records, order history or an admin session, delivered through a link that looks normal.

A practical checklist for your team#

1. Inventory every message listener. Search your bundle, third-party code included, for addEventListener("message". Expect surprises.
2. Check origin and source. Every handler should verify event.origin and the exact event.source window before changing any state.
3. Allowlist message types. Accept a short list of known message shapes and drop the rest.
4. Never pass raw HTML between frames. Send data, then render it on your side with textContent, a sanitizer such as DOMPurify, or your framework's templating.
5. Serve untrusted content from a separate origin, and don't pair allow-scripts with allow-same-origin for it.
6. Keep generated frames sandboxed. Any iframe you create at runtime, especially with srcdoc, gets a sandbox attribute by default.
7. Tighten CSP allowlists. Prefer nonces or hashes over whole domains, and review any allowed domain with callback-style endpoints.
8. Treat share links and public pages as hostile. Test what happens when a signed-in user opens content someone else created.

For most apps this is days of review, not a rewrite.

How MagicMakers Lab approaches this#

When we build custom software with AI features, we treat model output like form input from a stranger: it's validated, rendered through safe paths and kept behind a boundary that only accepts specific, typed messages. That discipline matters most in customer-facing systems, like the automated order and support flows behind Framico or the consolidated platform we built for Business Hub, where one leaky component could expose real customer data. A day spent on message contracts early is far cheaper than an incident later.

Key takeaways#

• The ChatGPT bug didn't break the iframe sandbox. It went around it through a trusting message listener.
• Third-party libraries can add postMessage listeners your team never reviewed.
• AI-generated HTML is untrusted input, even when your own model produced it.
• A CSP domain allowlist is only as safe as the weakest endpoint on each domain.
• Most fixes are small: origin and source checks, typed messages, no raw HTML across frames.

FAQ#

Is an iframe sandbox enough to run untrusted code safely?#

No. A sandbox isolates the framed document, but that document can still send messages to the parent page. If any listener on the parent trusts those messages, attackers can get around the sandbox indirectly, as the ChatGPT HTML Preview bug showed. Pair the sandbox with a separate origin, strict postMessage checks, safe rendering and a tight Content Security Policy.

How do I secure window.postMessage?#

Compare event.origin against one exact expected origin and confirm event.source is the specific window you expect. Then accept only a short allowlist of message types with validated fields. Never treat message data as HTML or code, and avoid handlers that change settings or state based on anything an untrusted frame can send.

Can AI-generated HTML lead to account takeover?#

Yes, if your app renders it somewhere that can reach privileged code or data. Model output can be steered by prompt injection hidden in documents or web pages, so an attacker may control part of what gets rendered. Treat it as untrusted input: sandbox it, isolate its origin and strictly limit what it can ask the parent page to do.

What does allow-same-origin do in an iframe sandbox?#

It lets the framed document keep its real origin instead of being treated as a unique, opaque one. Combined with allow-scripts on content served from your own origin, the frame can reach the parent and remove its own sandbox. For untrusted content, serve it from a separate domain or leave allow-same-origin off entirely.

If your product renders AI output, customer content or third-party widgets and you aren't sure what's listening on the other side, we'll map where untrusted content enters your app and which gaps matter most. Book a free audit.

Sources#

• Do the Math: How We Took Over ChatGPT Accounts (Parameter Labs)
• How we took over ChatGPT accounts (Simon Willison)
• Window: postMessage() method (MDN)
• The iframe element: sandbox attribute (MDN)