Talk to an AI that has never learned to speak

What Collidere is

Collidere is a 25.3 million parameter transformer that we wrote from scratch and initialized with random weights. It has never been pretrained on a web crawl. It has never been distilled from a larger model. It does not retrieve anything.

There are two of it. Model A is the control: random weights, no reading, nothing but live conversation. Model B had one phase of reading before launch, about three passes over 23.2 million words of recorded discussion, and after that it learns only from the conversations people have with it here. Both learn in batches, permanently altering their weights rather than holding anything in a context window. Both answer every message sent to them, side by side, and the difference between them is the experiment.

The model is small and it is not good at talking. That is the point. You are looking at the early part of a curve rather than a finished product. Most language models people meet have already read a meaningful fraction of the written internet by the time anyone talks to them, so whatever a conversation teaches them is a rounding error on top of everything they already know. Collidere inverts that ratio. The conversations are not a finishing layer here. For Model A they are the only thing it will ever learn from, and for Model B they are the only thing it learns from after one fixed corpus.

It is an experiment, not a product, and the measurements are the deliverable. After every update both models are scored on held-out text they never train on, asked the same fifty frozen questions, and checked for how their output shifts on every recorded disagreement. Nothing is tuned toward a benchmark or toward being right about the world. Facts do not matter to it. The mechanism does.

Collidere is not live yet. The two models, the moderation pipeline, the backups and the public interface are built and tested, but no live update has run and nothing has been trained on anything a visitor has written. This page describes what was built, what the first phase of training found, and what happens when it opens. Where a number appears it comes from the experiment's own record, and where something has not happened yet, the page says so.

Where the name comes from

Collidere is the Latin infinitive meaning to collide, or to strike together. The English word collision descends from it.

The name comes from an observation about how understanding gets made in a seminar room. Several people arrive holding different models of the world. They put those models under pressure against each other. Someone objects, someone reformulates, and by the end there is a claim on the table that nobody walked in with. The new thing was not carried into the room by any single participant. It came out of the contact between them.

Whether a program can do anything resembling that is an open question, and it is the question this project exists to measure. Not whether a model can absorb a transcript of a discussion, which is trivially true, but whether the live collision of incompatible positions moves a model differently than the same words would if they had merely been written down in advance.

The idea of a machine that learns the way a person does, rather than arriving finished, is older than the field itself. In the paper that proposed the imitation game, Turing suggested that instead of trying to write down an adult mind, one might build something closer to a child's and educate it^{10[10]Turing (1950) Computing Machinery and Intelligence. Mind 59(236), 433 to 460. doi:10.1093/mind/LIX.236.433.}. Collidere borrows the spirit of that suggestion and none of its ambition. It does not try to become a mind. It asks a much smaller question about where a model's changes come from when its education is a room full of people who disagree with each other.

The question

Language models are trained by predicting the next token. When training data contains two people asserting incompatible things about the same topic, gradient descent does not resolve the disagreement. It finds weights that assign probability to both positions, because that is the correct model of a world in which people say both things. Nothing anywhere in the objective function penalizes holding mutually inconsistent beliefs.

The prediction that follows is that contradiction in training data produces hedging rather than synthesis. A better calibrated fence sitter, not a new idea.

The closest measurement we know of approaches this from a different side. Krestnikov trains transformers from scratch on corpora in which the same problem appears with both correct and incorrect solutions, and finds that a preference for the correct answer emerges only when the errors are random. When the errors follow one coherent alternative rule, the preference vanishes^{2[2]Krestnikov (2026) Truth as a Compression Artifact in Language Model Training. arXiv:2603.11749.}. In controlled data, contradiction resolves by compressibility rather than by truth. Nobody has run the equivalent on live disagreement between people, where neither side is labeled correct and the contradictions are neither random nor synthetic.

That prediction is stated frequently and measured rarely. Collidere measures it. Specifically: when two people who disagree about the same topic both contribute to an update, does the depth of that disagreement change how much the model's output distribution moves? We tag contradiction pairs by whether anyone actually shifted position, and we track the distribution shift on each pair before and after every update.

The measurement is concrete. Before and after each update the model is shown a fixed prompt about the pair's topic and scored on both sides' text. Hedging has a signature: the model assigns more probability to both sides at once, its lean between them stays flat, and its own writing on the topic gets vaguer. Absorbing one side has a different one: the lean moves and the writing commits. Anything that deserved the word synthesis would have to be committed and new at the same time, and even then a person has to read it, because nothing here judges meaning automatically. A judge would be a pretrained model in the loop.

If depth never matters, the answer is hedging, and we will publish the curve that shows it. If depth does matter, that is a result nobody currently has.

A second, quieter question sits underneath the first. Human writing is lumpy. People reuse a favorite word, drop sentence fragments, drift off topic, and have a register they cannot escape. Large models smooth all of that toward a competent average, and the smoothness is much of what makes machine text recognizable. If a model learns only from a small number of specific people, does it inherit their unevenness instead of averaging it away? That is the burstiness measurement in the findings below, and so far the answer has gone the wrong way.

How it works

Collidere is small enough to describe completely. Text enters as bytes: every byte is a token, plus three symbols for the start of a turn, the end of a turn and padding. A byte tokenizer needs no corpus to build, so nothing pre-existing enters the system through it, at the cost of sequences about four times longer than a subword tokenizer would produce. Each turn is stored and trained as the speaker's number, a colon and the text, so the model always knows who is speaking.

The network is a decoder-only transformer written by hand: six layers, 512 wide, eight attention heads, a SwiGLU feed-forward block of 2,048, RMSNorm and rotary positions, with the embedding tied to the output head. That comes to 25,336,320 parameters. It trains on 512 byte windows and generates replies in a 448 byte window, for a reason Finding 2 explains. Weights are held in fp32 and computed in bf16, because the small learning rates of continual updates would otherwise round to no change at all.

Learning happens in batches, never per message. An update draws training windows from the conversation store, each one a 512 byte slice of a real conversation in reading order, and changes every weight in the network. Replay mixing resists catastrophic forgetting: three quarters of every update is drawn from turns the model has already trained on and one quarter from new ones, so it keeps rehearsing what it learned instead of overwriting it with whatever arrived last. Members of a contradiction pair are drawn four times as often as ordinary turns, because they are what the measurement is about. Every update is checkpointed, and the batches it drew are recorded.

That record makes attribution concrete. Every turn in the store carries its contributor's number, and every update records exactly which turns it trained on, so the whole history can be replayed from the same random initialization with any single contributor's influence removed, and the resulting weights compared against a full replay. The influence of one person on the model is a measurable distance rather than a guess, and if someone asks to be taken out, there is a procedure that does it.

The model's own output is stored and displayed but never trained on. Only human turns enter the training data. A model that trains on its own generations degrades quickly and predictably, a failure mode usually called model collapse, and the architecture forbids it by construction.

Nothing a contributor writes enters a training window automatically. A person reviews it, and it sits for 36 hours before it can be used. Contributors post under numbers rather than names, and the mapping between the two never leaves our infrastructure.

Review is deliberately human. At this volume a person can read every turn. A turn is removed for personal information, claims about a private person, harassment, spam, sexual content, an author who appears to be a minor, or text that is not live speech, such as a pasted article or another model's output. A position is never a reason to remove anything. Neither is profanity, rudeness toward the models, or the fragments, typos and repetitions that make people sound like themselves; those are the signal. The hold exists because a problem is often only visible across several turns, and a turn approved at noon should not already be in the weights by three in the morning.

A reply is produced on a single GPU. The public site runs on a small server that holds no model at all; a worker on the GPU machine pulls reply jobs from it, generates with both models, and posts the answers back. When a message arrives, Model A answers and then Model B, each from the room's last twelve turns and its own earlier replies, never the other model's. Replies are sampled at temperature 0.9 with a penalty on repeated byte sequences, because an undertrained byte model otherwise loops, and with an end-of-turn bias that keeps replies near the length people actually write. None of those settings reach the measurements, which always sample the plain distribution. If the worker has not checked in for a minute, the site says Collidere is asleep and refuses the message rather than queueing it.

The typical message takes about a second. Getting there was mostly a matter of the model learning to stop: before the re-seed described below, its replies ran long, and a message took two and a half seconds at the median, with more than a third over three seconds. A KV cache, the standard speedup for generation, turned out to buy nothing here. At batch size one a model this small is limited by the overhead of launching GPU kernels rather than by arithmetic, so each decoding step costs about ten milliseconds whether it computes one byte or 512.

The seed corpus

Model B read one corpus before launch: 3,051 discussions from the ChangeMyView community, collected by Tan and colleagues in 2016 and distributed through Cornell's ConvoKit^{1[1]Tan, Niculae, Danescu-Niculescu-Mizil, Lee (2016) Winning Arguments: Interaction Dynamics and Persuasion Strategies in Good-faith Online Discussions. WWW 2016. arXiv:1602.01103.}. After filtering and windowing, 274,762 turns and 23.2 million words.

ChangeMyView is a forum where someone posts a view they hold and invites others to change it. It is one of the most studied datasets in computational work on persuasion: beyond the original analysis, researchers have annotated the semantic types of its claims and premises and mined how its discussions are built^{11[11]Hidey, Musi, Hwang, Muresan, McKeown (2017) Analyzing the Semantic Types of Claims and Premises in an Online Persuasive Forum. Proceedings of the 4th Workshop on Argument Mining.,12[12]Chakrabarty, Hidey, Muresan, McKeown, Hwang (2019) AMPERSAND: Argument Mining for PERSuAsive oNline Discussions. EMNLP-IJCNLP 2019.}. That is exactly why nobody should be impressed that we used it. A well understood corpus with documented preprocessing is a feature of the experiment, not its contribution.

The corpus was chosen for a property that is rare in conversational data. On ChangeMyView, when a reply genuinely changes the original poster's view, the poster marks it with a delta. That mark is a human judgment about whether a position actually moved, recorded at the time by the person whose position it was. It gives us 4,234 exchanges where someone shifted and 3,861 where nobody did, labeled by the participants rather than inferred afterward.

Using the label took care. In the distribution, the delta flag does not mark a reply; it marks a whole reply path, including the original poster's own answers inside it. Pairing on the flag alone would have paired posters with themselves thousands of times, diluting exactly the comparison the labels exist for, without any total looking wrong. So a pair is the original post and a direct reply to it by somebody else, and nothing else counts. The known limitation runs the other way: only opening replies are labeled, and the concession recorded is the poster's alone, so a shallow pair may contain a reply that moved someone other than the poster. That noise pushes any measured difference between deep and shallow toward zero, not away from it.

Model A read none of it. That asymmetry is the comparison.

The corpus is acceptable where others were not because it is recorded collision rather than finished prose: people holding opposed positions, pushing on each other, and sometimes moving. The mechanism the project is about is intact in it; only the live property is missing. Coursework archives and a forum about competitive debate were considered and rejected, the first because it is copyrighted, pre-written prose with no collision in it at all, the second because its users are largely minors who could not consent.

Training

Model B was seeded twice. The first run, Collidere I, trained for 46 updates of 500 steps, 23,000 steps in all, which covered 3.04 passes over the corpus's training split in 3.72 hours. Held-out perplexity fell steeply in the first pass and slowly after it, the answers went from noise to English-shaped text, and the run produced the numbers the rest of the project was planned around.

Then, while measuring how long its replies were, we found that Collidere I had learned something that was not language at all: where its training window ended. That is Finding 2, below. Leaving it in place would have baked the artifact into every live update, so the windows were rebuilt and Model B was seeded again from the same random initialization. Collidere II used an identical configuration, the same 46 updates and the same learning rate schedule, so that the window rule is the only difference between the two runs. A slice of conversation carries slightly fewer loss-bearing bytes than the old windows did, so the same steps covered 2.88 passes rather than 3.04, and that is reported rather than topped up. The first run was archived, not overwritten, so every chart below can set the two side by side.

Everything the evaluator measured during seeding is in the figure below. Held-out perplexity is scored on ChangeMyView turns the model never trains on. Probe entropy and distinct-2 describe its answers to the fifty frozen questions. The burstiness ratio is Finding 1. The seed-pair KL shift is how far each update moved the model on the topics of the contradiction pairs, split by depth, which is the null result. Relative weight change is the size of each update against the weights it changed.

Held-out perplexity per byte fell from 3.31 after the first update to 2.51 at seed end in Collidere II, which is 1.33 bits per byte. The two runs track each other closely; Collidere II sits about a hundredth of a bit per byte behind at every update, consistent with slightly fewer passes and less context per window. Probe entropy fell and distinct-2 barely moved, which is what a model learning the statistics of a register without collapsing onto a handful of phrases looks like.

The decision to stop at three passes was made on Collidere I's curve, before any compute was spent on a fourth, and the reasoning mattered more than the budget. Model B exists to test whether prior exposure to the structure of disagreement changes how a model meets live disagreement. A model that has memorized its corpus can answer a live contradiction by recalling a thread it read, which confounds the comparison with Model A instead of informing it. More passes over the same 23 million words buy memorization before they buy anything else.

So the rule set in advance was to stop when the curves said so, never because hours were left over, and to read three signals before deciding: whether held-out loss was still falling in proportion to the work, whether the answers reproduced the corpus verbatim, and how well the model transferred to conversation it had never seen.

Each pass bought about half of what the one before it bought, or less: 0.230 nats per byte of held-out loss in the first, 0.039 in the second and 0.020 in the third. At the last update the training loss and the held-out loss met, 0.901 against 0.904, which is where further passes start buying the particular threads instead of their structure. There was no verbatim recall above the human baseline, and the transfer to live conversation was large. None of the three signals said the model needed more of the same corpus, so the rest of the compute budget was left unspent. The next thing that moves Model B is live conversation.

What we found

Three findings and one null result came out of seeding and the sandbox tests that followed it. None of them is the result the project exists for; that one needs live conversation. They are what building the apparatus turned up, and each one changed how the live phase will run.

Finding 1: corpus training smooths the model below human unevenness

Human writing is uneven. Some stretches are highly predictable and some are not, and the variance of that predictability is a measurable property. Machine generated text is unusually even by comparison, and that evenness is part of what makes it recognizable.

We measure it as the coefficient of variation of per byte loss in the model's own generations, divided by the same quantity measured on human text. A ratio of 1.00 means the model is as uneven as people are. Below 1 it is smoother than they are.

Across three passes over the corpus the ratio fell from 1.01 to 0.96. The model became smoother than the humans it was reading. The effect is small, but it survived a full audit against the positional artifact described below, which contributed between 0.002 and 0.009 of it against a total movement of about 0.05. Both halves of the ratio moved: the model came to find human text spikier, its coefficient of variation up 24 percent, faster than its own writing became spiky, up 17 percent.

It also stopped. Between the second and third pass the ratio's mean moved from 0.961 to 0.960, which is noise. The continued decline we saw in the first seeding run did not reproduce once the training windows were fixed; part of that run's later smoothing was the cue.

The open question is whether live conversation with specific people, who each have their own vocabulary and rhythm, pulls that ratio back up. Model B enters the live phase at 1.03, measured against held-out live conversation. That number is the baseline the live phase has to move.

Finding 2: the model learned where the window ends, not where a turn ends

In the first seeding run, 47.8 percent of end of turn targets sat at the final position of the training window. The model learned the wrong rule. It was not predicting that a turn had finished; it was predicting that the window was full.

The effect was invisible in every aggregate metric. It surfaced when display replies came back at six bytes on one generation path and four hundred on another. The probability the model gave to ending a turn in the middle of one was flat, around a thousandth or less, for every window up to 510 bytes, then jumped at the last two positions of a 512 byte window to 0.099 and 0.111 on live conversation text.

The cause was in how training examples were cut. A window was filled backward from a chosen turn until it held 512 bytes, so that turn's own end sat at the last position whenever it fit. Ninety-five percent of windows were full, and half of those ended on an end of turn. The display path slid a full window forward one byte at a time, so every byte of a reply sat at the one position where the model had learned that turns stop.

We rebuilt the windowing to take random slices of the conversation, re-seeded Model B from scratch, and re-scored both runs identically. End of turn targets at the final position went from 47.8 percent to 0.17 percent, and the probability spike went to 0.0020, level with every other position.

Two published numbers were casualties. Natural turn length was not 556 bytes; measured in a window that never reaches the cue, the median is 958. And the probability of the model ending a turn at a genuine turn boundary in the corpus was not 0.999. With position controlled it is 0.170, and at the end of a live conversation turn, 0.003. Both came from the first run, are corrected here, and are struck in our notes rather than quietly replaced.

The artifact also flattered validation. Scored the old way, the cue made held-out loss look about 1.3 percent better than it was on the corpus and 4.6 percent better on live conversation, where a short turn's end is a large share of its bytes. That is why validation is now scored with nearly full context but never at the last position, and why no transfer figure from the old scoring appears anywhere on this page.

The general lesson is recorded as a standing check: anywhere a pipeline caps, chunks, or windows text, test whether the model has learned position rather than content.

Finding 3: an update sized for a corpus destroys a model when the data is a day of conversation

Our update size was a fixed 200 steps, a number chosen early and sensible for a large corpus, where every step sees fresh text. Applied unchanged to a single day of live conversation, 200 steps over about 11 kilobytes of text works out to 248 passes over the same few sentences. Model B's entire seeding was fewer than three.

In a sandbox that took Model B from 1.92 to 3.44 bits per byte on held out live turns. It got worse at exactly the thing it was supposed to be learning. Retention of the seed corpus collapsed from 1.33 to 2.88 bits per byte, and the frozen probes began reciting contributors' sentences verbatim.

The parameter had not changed. The volume of data underneath it had fallen by four orders of magnitude. The job was tested end to end on a copy of the store before it was ever scheduled, which is the only reason this is a finding rather than the first night of the live record.

Updates are now sized in passes over whatever data is available rather than in fixed steps. A sweep found the usable range: at three passes the model improves on held out live turns from 1.92 to 1.71 bits per byte while retention stays near 1.35 and nothing is recited. At ten passes it still improves but retention degrades. At thirty and above the model gets worse on live turns, retention collapses, and it begins reproducing contributors' sentences word for word. Three is the size the sweep points to, and the nightly job stays switched off until it is set.

Model A behaves differently, which is informative in itself. Starting from nothing, it keeps improving up to about a hundred passes before it starts memorizing. One setting for both keeps them on identical updates, which is the design, so at three passes Model A moves slowly, from 8.43 to 5.59 bits per byte after one night. That slow curve is the cold-start result the experiment exists to publish.

Verbatim recall is a measurement problem and a privacy problem at the same time. A model that recites a contributor's sentences word for word is overfitting, and it is also repeating someone's words on a public page, pseudonymized and approved or not. So recall has to be checked on every live update rather than once. The sweep checks it; making it part of every live update's evaluation is on the list before the first one runs.

At seed end the check came back clean. Model B's answers share fewer six-word runs with its corpus than fresh human text of the same length does by chance, every one it shares is a stock phrase, and no ten-word run from its answers appears anywhere in the corpus. What the model took from 23 million words was register and structure, not text.

Null result: pair depth made no difference during seeding

Under uniform corpus training, deep and shallow contradiction pairs shifted the model's output distribution by 0.0448 and 0.0447 nats respectively. They are indistinguishable, which is what should happen when the training signal is one continuous pass over a corpus and nothing about the live collision is present.

This is the baseline. Any separation observed during the live phase has to beat it.

The null is also a check on the instrument. Deep and shallow pairs are scored identically and differ only in whether a poster conceded. If they had separated during uniform training, where both sides of every pair are ordinary text, the measurement would be picking up something other than depth, such as length or topic. They did not, in either run.

The last number from seeding is the one the live phase starts from. Seeding never saw any live conversation, so scoring the seed-end model on held-out live turns measures how much of what it learned carries over to a register it has never met. It carries a lot: random initialization costs 8.43 bits per byte on those turns, slightly worse than guessing uniformly among byte values, and Model B at seed end costs 1.92.

The gap between that 1.92 and the 1.33 the model reaches on held-out corpus text is the price of an unfamiliar register: short fragments, lowercase, game vocabulary. Closing it is the first thing live updates can visibly do, and the sandbox sweep suggests a single well-sized update would move it to about 1.71.

What the model sounds like

The same fifty prompts are put to the model after every update and the raw output is saved. Nothing is cleaned up or cherry picked. Below you can move through training and watch the same prompt answered at each stage.

At initialization the output is not bad text. It is not text at all: random bytes, most of them not valid UTF-8. Within one pass it is recognizably English-shaped, with correct letter frequencies and spaces in plausible places. By the third pass it produces grammatical fragments in the register of the corpus it read, which carry no stable meaning. That progression is the entire result of the seeding phase.

Reading the answers in order is the clearest way to see what a small model can and cannot take from a corpus. It learned how people in that forum type: the rhythm of their sentences, their hedges, the openings that announce a counterpoint, the habit of posting a link. It did not learn to hold a thread. Nothing at update 46 answers the question asked in any sense a person would accept, and nothing reads as a coherent claim. The live phase will be read the same way, prompt by prompt, and the change from here is what it will be judged on.

What happens next

Opening Collidere starts the live phase. Anyone with an arbastro account can talk with both models; approved turns train both identically in nightly batches; and after every update the full evaluation runs on both: held-out live turns, retention of the seed corpus for Model B, the fifty probes, burstiness against live text, and the distribution shift on every contradiction pair a reviewer tags as deep or shallow. No live update has run yet.

What would count as evidence is fixed in advance. For the central question, if deep pairs, where someone actually moved, shift the models more than shallow ones by more than the seeding baseline of no difference at all, depth matters and hedging is not the whole story. If they stay indistinguishable across many updates, the answer is hedging, and the curve gets published. For burstiness, if Model B's ratio rises from 1.03 and Model A ends at or above it on the same data, unevenness is being inherited from people; if both settle below 1, smoothing is what gradient descent does to any mixture of voices. For the comparison between the two models, the question is whether prior exposure to recorded disagreement changes how a model meets live disagreement, or only gets it to the same place faster.

This is not the kind of debate other work uses. Multiagent debate has pretrained models critique each other's answers at inference time to improve factual accuracy and reasoning^{5[5]Du, Li, Torralba, Tenenbaum, Mordatch (2023) Improving Factuality and Reasoning in Language Models through Multiagent Debate. arXiv:2305.14325.}, and debate as a safety proposal has two capable agents contend so that a human judge can check claims too hard to check directly^{6[6]Irving, Christiano, Amodei (2018) AI safety via debate. arXiv:1805.00899.}. In both, the parties disagreeing are models. Here they are people, there is no judge and no winner, and the only thing that changes is a small model's weights.

The hard constraint is volume. Conversation is an extraordinarily low bandwidth source of text. Compute-optimal training puts the budget for a model on the order of twenty training tokens per parameter^{7[7]Hoffmann et al. (2022) Training Compute-Optimal Large Language Models. NeurIPS 2022. arXiv:2203.15556.}, which for 25 million parameters and byte-sized tokens is hundreds of millions of bytes. Work on small models shows that fluent English is reachable at this scale, but from a synthetic corpus of simple stories written for the purpose in a child's vocabulary^{8[8]Eldan, Li (2023) TinyStories: How Small Can Language Models Be and Still Speak Coherent English?. arXiv:2305.07759.}, and the BabyLM challenge sets its budgets at 10 and 100 million words because that is roughly the language a child is exposed to while growing up^{9[9]Warstadt et al. (2023) Call for Papers, The BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus. arXiv:2301.11796.}. A room of people talking produces a small fraction of either.

Move the sliders below to see what that means for Model A, the model with nothing but conversation to learn from.

At twenty people each having one real conversation a day, weak but recognizable English is about four and a half years away, and the coherent band closer to eighteen. That is not a hardware or code problem, and it is the reason Model B exists. It is also the reason Model A is kept: a curve showing that conversation alone is too slow to bootstrap a model is a complete result, and it gets published with its curve if that is what happens.

Over the first months Model B should get better at live conversation and a little worse at its corpus, and Model A should crawl upward from noise. What we expect from the pair measurement is set out in its own section below, before the live phase begins.

When Collidere opens, the frame below shows both models' latest answer to the same frozen prompt, and the panel under it reports where the live phase stands. Until then, both say so.

What we expect to find

We expect hedging rather than synthesis, and the reason is structural. Nothing in next-token prediction penalizes holding inconsistent beliefs, so contradiction in training data produces a probability split rather than a resolution. We are stating that in advance, before the live phase begins, so that the result means something when it arrives.

Both outcomes are results. If the curve shows no separation between deep and shallow disagreement, we will have demonstrated something the field usually assumes without measuring, on data nobody has measured it on. If it shows separation, that is a result nobody currently has.

Two data points so far are consistent with the prediction. Deep and shallow pairs shifted the model identically during seeding, and burstiness moved below human unevenness rather than toward it. Neither is the live test.

Limitations

The model is too small to hold a thread. At 25 million parameters and a 448 byte generation window it can produce the texture of a reply, not follow a conversation across several turns, and nothing in this experiment will change that. Its answers will stay fragments for a long time.

Conversation delivers far too little data to reach general competence in any reasonable timeframe, as the calculator above shows. Model A may never become coherent. That is an accepted risk, and if it happens it is the result rather than a failure of the experiment.

The depth measurement has weaknesses of its own. Depth labels on live pairs come from reviewers rather than from a recorded concession, live pairs will be few for a long time, and a distribution shift is a coarse instrument for something as specific as a position moving.

Contradiction pairs form only within a single conversation. We capture disagreement between people who actually responded to one another, not between people who hold opposing views separately. This is deliberate, since matching topics across conversations is unreliable and false pairs would contaminate the measurement, but it means the sample is disagreement that was engaged rather than disagreement that merely exists.

Burstiness moved the wrong way under corpus training, and small movements in a ratio of two spreads are easy to over-read. Model B's live baseline of 1.03 rests on seventeen held-out turns from two conversations, and its second decimal does not carry until the live validation set grows.

Contributors are self-selected, not a representative sample of anyone. What the models learn is what a particular group of people said to them, and every finding is about that group.

The seed corpus is English, comes from one forum with its own norms, and predates 2016. Model B's prior is that forum's register, including its habit of posting links, which shows up in its answers as a placeholder.

Imported historical discussions are not published, because their authors never agreed to it. A small number of earlier conversations were imported, pseudonymized and reviewed to test the pipeline and to form the first live validation set; they stay out of every public view.

Our repository's smoke test numbers are plumbing checks and are never cited as evidence of anything. Synthetic contributors with fixed verbal tics are trivially predictable, so a perplexity that falls on them shows that the pipeline runs, not that conversational learning works.

Moderation cannot un-train a turn: if one is rejected after an update has consumed it, it is blocked from every later window, but removing its effect means rolling back and replaying the history without it, which today works per contributor and not yet per turn.

Related work

The machinery here is not new. Continual learning from deployment is an established pattern: CLaaS stores rollouts in a replay buffer and trains asynchronously behind a chat API^{3[3]Fallah, Naihin, Widawsky, Mao (2026) CLaaS: Continual Learning as a Service for Sample Efficient Online Learning. arXiv:2606.05559.}, and prior work has done persistent weight updates triggered by live interaction^{4[4](2024) Online Training of Large Language Models: Learn while chatting. arXiv:2403.04790.}. That work reports the same risks we accepted, that models struggle to absorb what contradicts what they already know and that repeated training on small datasets overfits and forgets; Finding 3 is our own instance of the second. Our replay design is standard practice and we claim none of it.

The closest neighbor to our central measurement is Krestnikov, who trains transformers from 3.5 million to 1 billion parameters from scratch on controlled corpora containing contradictory solutions to the same problem^{2[2]Krestnikov (2026) Truth as a Compression Artifact in Language Model Training. arXiv:2603.11749.}. When errors are random, a correctness preference emerges and scales with model size. When errors follow a single coherent alternative rule, that preference vanishes. That is our question, approached with our methodology, and on controlled data it is a cleaner experiment than ours.

What we can find no prior work on is the specific combination: live multi party human disagreement driving global weight updates over an extended continual regime, with contradiction pairs tagged by whether anyone actually moved and tracked longitudinally. Prior work on contradictory training data uses synthetic corpora in single training runs. Prior work on continual learning from interaction uses agent rollouts, single user feedback, or task completion signals. Neither covers this.

That is a narrow claim and it is the only one we are making. The comparison between a cold start and a seeded model fed identical live conversation is the one other part of the design we could not find elsewhere.

Changelog

1. Performance and infrastructure

   September 14, 2026

   The sampler is 1.92 times faster, with byte identical final weights before and after, checked under deterministic kernels. A KV cache is correct but buys nothing at batch size one, where each decoding step costs about 10 milliseconds of kernel launch overhead. Moderation and backups are built: every contributed turn waits for a human review and a 36 hour hold, and the conversation store is backed up off-machine daily, verified by a fresh restore.

2. Contradiction pairs scoped to a single conversation

   September 14, 2026

   A pair now forms only between people in the same conversation, so the sample is disagreement between people who actually responded to one another. Matching topics across conversations is unreliable, and false pairs would contaminate the measurement.

3. Update sizing changed from fixed steps to passes

   September 14, 2026

   A sweep on sandbox copies established the usable range. Three passes per update improved held-out live turns without degrading retention or producing verbatim recitation; thirty and above collapsed both. No live update has run yet.

4. Validation scoring moved to near-full context

   September 14, 2026

   Two earlier scoring schemes are superseded. The absolute numbers moved; every comparison on this page is like for like. The seeding phase is closed: no further change to scoring or windowing without a re-seed.

5. Collidere II re-seeded with random-slice windowing

   September 14, 2026

   Training windows are now random slices of the conversation, which took end of turn targets at the last window position from 47.8 percent to 0.17 percent. Model B was re-seeded from scratch with the same 46 updates, which covered 2.88 passes. Run 1 is archived rather than overwritten, so the two can be compared.

6. Positional end of turn artifact found

   September 14, 2026

   47.8 percent of end of turn targets sat at the final position of the training window: the model had learned where the window ends, not where a turn ends.

7. Collidere I seeded

   September 13, 2026

   The first three pass run over the ChangeMyView corpus: 46 updates, 23,000 steps, 3.04 passes, 3.72 hours.

References

1. 1.Tan, Niculae, Danescu-Niculescu-Mizil, Lee (2016) Winning Arguments: Interaction Dynamics and Persuasion Strategies in Good-faith Online Discussions. WWW 2016. arXiv:1602.01103.link
2. 2.Krestnikov (2026) Truth as a Compression Artifact in Language Model Training. arXiv:2603.11749.link
3. 3.Fallah, Naihin, Widawsky, Mao (2026) CLaaS: Continual Learning as a Service for Sample Efficient Online Learning. arXiv:2606.05559.link
4. 4.(2024) Online Training of Large Language Models: Learn while chatting. arXiv:2403.04790.link
5. 5.Du, Li, Torralba, Tenenbaum, Mordatch (2023) Improving Factuality and Reasoning in Language Models through Multiagent Debate. arXiv:2305.14325.link
6. 6.Irving, Christiano, Amodei (2018) AI safety via debate. arXiv:1805.00899.link
7. 7.Hoffmann et al. (2022) Training Compute-Optimal Large Language Models. NeurIPS 2022. arXiv:2203.15556.link
8. 8.Eldan, Li (2023) TinyStories: How Small Can Language Models Be and Still Speak Coherent English?. arXiv:2305.07759.link
9. 9.Warstadt et al. (2023) Call for Papers, The BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus. arXiv:2301.11796.link
10. 10.Turing (1950) Computing Machinery and Intelligence. Mind 59(236), 433 to 460. doi:10.1093/mind/LIX.236.433.link
11. 11.Hidey, Musi, Hwang, Muresan, McKeown (2017) Analyzing the Semantic Types of Claims and Premises in an Online Persuasive Forum. Proceedings of the 4th Workshop on Argument Mining.
12. 12.Chakrabarty, Hidey, Muresan, McKeown, Hwang (2019) AMPERSAND: Argument Mining for PERSuAsive oNline Discussions. EMNLP-IJCNLP 2019.