I trained a GPT on corporate ipsum, and it runs in your browser

I'd never trained a GPT from scratch, so I built one as a joke. Corporate GPT is a 2.8-million-parameter transformer trained on corporate ipsum (which I generated with ChatGPT, so grain of salt there), and what it writes is barely passable corporate jargon: close enough to the real thing to be unsettling, which is the funny part. It runs entirely in your browser, in plain JavaScript. No server, no API calls.

A real sample, unedited:

By leveraging a proprietary framework that synthesizes cross-functional scalability vectors and with a multi-dimensional engagement architecture, organizations can optimize throughput gravity, thereby optimizing overall engagement velocity by an estimated 0.87% per quarter.

Throughput gravity. An estimated 0.87% per quarter. At 2.8 million parameters (GPT-2 Small is 44 times bigger) it's far too small to write coherent text, and it still got that close. Here's how it was put together.

The data

Corporate Shill already had a lot of corporate filler lying around: millions of words I'd generated with Gemini Flash and GPT-4.1 nano for the site's About page. The model trained on about 780,000 words of it, the articles and executive summaries.

The model

It's GPT-2, shrunk. Four layers, four attention heads, a 192-dimension embedding and a 256-token context window. The output layer shares its weights with the token embedding.

The vocabulary is 5,000 tokens, from a byte-level BPE tokenizer I trained on the same corpus. GPT-2's own vocabulary was out: its 50,257 tokens would have needed a 9.6-million-number embedding table on its own, more than three times the size of this whole model.

All of that comes to 2,788,992 parameters.

Training

PyTorch on a rented T4 GPU through Modal: 30 epochs, AdamW, a short warmup and then cosine decay. About five minutes, roughly five cents. Training loss went from 8.5 to 2.9, and validation loss from 7.3 to 3.2.

Running it in the browser

I exported the weights as raw 32-bit floats (11 MB, smaller than most hero images). Then Claude Code and I wrote the forward pass in plain JavaScript. No ONNX, no WebGPU, no libraries. Each new token runs the whole context through the model again, with no caching. Slow, and fine at this size.

PyTorch stores its matrices the other way around from how the JavaScript multiplies them, so the export transposes every 2D matrix except the embeddings.

You can't type a prompt; it always starts from scratch. What you can change is the temperature, from focused to creative, and the length. It samples from the 40 likeliest next tokens and streams the words in as it goes.

The page walks through every step, tokenizer to sampling, and you can download the model and the data.

Five cents of GPU and 11 MB of floats, and out comes throughput gravity. The Convergence Convergence Model was entirely its own idea.