Live demo
Every stage of this model ran on code I wrote: the tokeniser, the training framework and its GPU kernels, fine-tuning, evaluation, export, and the inference engine answering you below. It is small (14.6 million parameters) and trained only on conversation, so it is good at small talk and knows no facts.
It runs in your browser: ember.cpp is compiled to WebAssembly, the model (71 MB) downloads once, and nothing you type leaves your machine.
If the window stays blank, open it directly: huggingface.co/spaces/johnnyteutonic/ember-chat.
A Llama-style decoder: 6 layers, width 384, 6 heads, rotary position embeddings, SwiGLU and RMSNorm, 14.65M parameters and a 256-token context. The byte-level BPE tokeniser (8,192 tokens, with chat role markers) was trained with paperkiln's own BPE tool, and the trainer and the engine share one implementation of it.
12,000 steps on TinyChat dialogue (49M tokens) on a cloud L4 GPU; validation loss fell from 3.72 to 1.99. During the run I moved attention, rotary embeddings and the head plumbing onto the GPU, added a caching allocator and kept activations resident on the device, which cut a training step from 6.4 s to 1.8 s with the loss unchanged in every same-sitting comparison. Training resumed across those builds from checkpoints with optimiser state.
Supervised fine-tuning on 24,000 examples in the chat template, with the loss masked to the reply. It overfits after about 500 steps, so later runs stop there. A preference stage (DPO on self-play pairs) was tried at three learning rates and not kept: it raised pair accuracy while making replies drift off topic and repeat themselves. The next preference stage ranks samples with a judge model and has to pass a repetition gate first.
On 100 frozen TinyChat prompts the model ends every reply cleanly and never repeats itself; topicality and similarity to reference replies rose with every extra pretraining step. On a 50-question probe of facts, science and arithmetic it scores zero, as expected from its data: that gap is what the next model, pretrained on general educational text, is for.
The trained weights are exported to GGUF and loaded by ember.cpp, an inference engine written separately from the trainer. Greedy output is checked token for token against the trainer's own sampler. On a laptop GPU, decoding runs at 2.7 ms per token with each step replayed as a captured CUDA graph, and streamed replies show their first word in about 20 ms. Here the same engine runs as WebAssembly in your browser, and gives the same greedy replies as the native build.
A larger model pretrained on FineWeb-Edu, for questions as well as conversation; then training architectures that paperkiln extracts from arXiv papers, evidence for every value, and comparing them on the same chat benchmark.
Code: paperkiln (training) and ember.cpp (inference). Built with the help of AI coding assistants. Training data: TinyChat by starhopp3r, CC BY-NC 4.0, which the model inherits.