Mercury LLM Matching Claude and GPT is more Important than You Think [Breakdowns]

Why Diffusion Language Models Could Be the Biggest AI Shift Since Transformers

Feb 28, 2025

Hey, it’s Devansh 👋👋

In my series Breakdowns, I go through complicated literature on Machine Learning to extract the most valuable insights. Expect concise, jargon-free, but still useful analysis aimed at helping you understand the intricacies of Cutting-Edge AI Research and the applications of Deep Learning at the highest level.

I put a lot of effort into creating work that is informative, useful, and independent from undue influence. If you’d like to support my writing, please consider becoming a paid subscriber to this newsletter. Doing so helps me put more effort into writing/research, reach more people, and supports my crippling chocolate milk addiction. Help me democratize the most important ideas in AI Research and Engineering to over 100K readers weekly. Many companies have a learning budget that you can expense this newsletter to. You can use the following for an email template to request reimbursement for your subscription.

Help me buy chocolate milk

PS- We follow a “pay what you can” model, which allows you to support within your means, and support my mission of providing high-quality technical education to everyone for less than the price of a cup of coffee. Check out this post for more details and to find a plan that works for you.

Yesterday, Inception Labs released their demo of their new Diffusion-based Coding LLM, which has a set (a very niche) corner of the internet on fire. Their model seems to work up to 6x faster than standard LLMs, as shown in the video comparison below-

Even the quality is pretty strong, tying or outcompeting other small models in coding benchmarks, which is very strong showing for a new paradigm (we know less about Diffusion Text models than we do standard AR models)- “When evaluated on standard coding benchmarks, Mercury Coder achieves excellent quality across numerous benchmarks, often surpassing the performance of speed-optimized autoregressive models like GPT-4o Mini and Claude 3.5 Haiku while being up to 10x faster.”

I’ve been bullish on Diffusion Models for Language Models. Over the last year, I did 2 dedicated articles on Diffusion Models being better for generation: this one for across different AI domains, and this one on Diffusion Guided Language Modeling (which I called a GPT-2 level breakthrough, a prediction that aged well after this). DGLM, and the use of Diffusion Models for generation was also something that I put very highly in my “6 AI Trends that will Define 2025”- which means three of the ideas we earmarked in that article have already made big waves (Diffusion, Complex Reasoning based Inference (this is different from just thinking models, although they are a subclass), and AI in Quantum Computing).

Damnn, I’m really good at this. Let’s see if my most bullish pick goes anywhere. Anyhoo, back to Diffusion Large Language Models.

It is essential to monitor developments like this because they can be completely paradigm shifting. In the same way Transformers almost completely wiped out the use of RNNs in text, you never know when a new status-quo is around the corner. Early adopters tend to disproportionately benefit in experience, ROI, and enhancements. And these results are worth looking at-

In this article, I want to take a step back and ask why Diffusion Models might be the next it girl for LLMs, by looking at Mercury and other research topics. More specifically we will cover the following-

What is Diffusion?
What makes Diffusion Models worth looking into?
What are their drawbacks?
Why Diffusion Based Generations works Well (conceptually).
The future of Diffusion Models.

If this sounds interesting, keep reading.

PS- As a personal aside, my first technical breakdown after I moved to my current apartment in NY (which is also my first ever apartment in the City) was the “How Diffusion Models are Improving AI” article. Now, this piece on Diffusion Models will likely be my last technical article (almost certainly) in this apartment, and maybe in NYC (I haven’t decided on whether I will stay here yet). Funny how life comes full circle.

PPS- I’m in Tampa, Florida between the 13th and 15th and will be in San Francisco for A16Z Demo Day b/w 15th-21st March. Let’s meet if you’re around. Would appreciate any introductions you can make in either place as well.

Executive Highlights (tl;dr of the article)

What are Diffusion Models in Machine Learning: Diffusion Models are generative models (they generate data similar to what they were trained on). Diffusion Models follow 2 simple steps. First, we destroy training data by incrementally adding Gaussian noise. Training consists of recovering the data by reversing this noising process. A well-trained Diffusion Model can create whatever we want from a pool of random noise. Replace noise with a space of embeddings, and you can probably see why we’re cooking here.

The advantages of DMs: Diffusion models have 4 major advantages that make them worth future investing for text-based generation-

High-Quality Generation: Diffusion models generate outputs with exceptional quality and realism, surpassing previous generative models in many tasks.

Versatility: They are applicable to a wide range of data modalities, including images, audio, molecules, etc.

Controllability: Diffusion models offer a degree of control over the generation process, allowing users to guide the output based on specific requirements or conditions. They can also be stopped midway through generations, allowing more flexibility than traditional setups.
Better Feedback Signals: This is speculation from me, but this is my biggest reason for believing in Diffusion. The way I see it, it’s easier to score a bad output over an incomplete one (giving feedback to a bad piece of code with bugs/inefficient structures is easier than giving feedback to code that has half the logic and end logic missing). This should enable mid-generation “thinking” for diffusion LLMs as opposed to the pre (planning) or post (critique and iterate) generation thinking that’s prevalent in today’s outcomes.

*Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models*

The Drawback of Diffusion Models- As should be evident by their design, DMs are very expensive. There is research to mitigate their costs, but this is still a sore spot for DMs.

Why Diffusion Models Work Well (speculation by me)- By their very nature, Diffusion Models are trained to look at data points holistically at every inference step. In the way they do so, they combine the best of two elite generators.

Compared to generators like GANs- Diffusion has the luxury of creating outputs in multiple steps, giving us finer control (think about how hard it is to do anything complex in one go).
Compared to auto-regression (used by LLMs like ChatGPT)- DMs have greater flexibility (we can stop the generation at any point and have a somewhat usable result).

Lastly, the process of noising and denoising plays a similar role to strong data augmentation, wherein the model is forced to build deeper relationships for features and ensuring better security

Generative learning, recognized for its effective modeling of data distributions, offers inherent advantages in handling out-of-distribution instances, especially for enhancing robustness to adversarial attacks. Among these, diffusion classifiers, utilizing powerful diffusion models, have demonstrated superior empirical robustness... Experimental results show the superior certified robustness of these Noised Diffusion Classifiers (NDCs). Notably, we achieve over 80% and 70% certified robustness on CIFAR-10 under adversarial perturbations with \(\ell_2\) norms less than 0.25 and 0.5, respectively, using a single off-the-shelf diffusion model without any additional data.
-From the Paper- “Your Diffusion Model is Secretly a Certifiably Robust Classifier”. This paper was updated a few days ago, so the results are updated.

We can also chain Diffusion Models with other models very well, leading to very cool applications (and the vaunted Agentic Architectures that we love here)-

There’s also a somewhat obvious, but still important, fact that Diffusion Models can be parallelized better since they can run denoising chains in parallel as opposed to AR models, which have to wait for the last token to finish (although Spec Decoding will probably have something to say here). This leads to a huge speed boost for models like Mercury-

What sets dLLMs apart is their speed. While even speed-optimized autoregressive models run at most at 200 tokens per second, we can serve Mercury Coder on commodity NVIDIA H100s at speeds of over 1000 tokens per second, a 5x increase. Compared with some frontier models, which can run at less than 50 tokens per second, we offer a more than 20X speedup.
The throughput achieved by dLLMs was previously achievable only using specialized hardware, such as Groq, Cerebras, and SambaNova. Our algorithmic improvements are orthogonal to hardware acceleration and speedups would compound on faster chips.

For a side by side, check this-

Once we understand the background (and assuming you buy my basic premise that dLLMs are worth a deeper look), we must then think forward to think about how we want Diffusion Models to play. Here are some deep, deep, so deep thoughts that should probably be framed in a museum as a testament to the heights human genius can achieve-

Near-Term: Conquering Cost and Proving Value

Diffusion models are computationally expensive. The fewer iterations and higher throughput shown by Mercury as proof of efficiency aren’t apples to apples comparisons (since cost per inference of Diffusion is much higher, canceling the gains from lower iterations). We need some good new cost metrics to fairly compare dLLMs with AR models.
Develop better cost metrics (FLOPs, parallelizability, latency) and benchmark against AR models.
Optimize denoising with adaptive step reduction, learnable stopping criteria, and efficient schedules to reduce overhead.
Refine evaluation metrics for global coherence, constraint alignment, and reasoning quality to better assess model performance.

Mid-Term: Pushing Beyond Hard Tokens

As a latent space supremacist, this is the space I’m most excited for. I’ve been playing a lot with computations in the Latent Space (to enable things like MultiModality) and this seems to be extremely powerful. You can inject capabilities/processes here much more impactfully than you’d think. For dLLMs, I would imagine two particularly important trends.

Move towards continuous language spaces, eliminating tokenization constraints and enabling a more “robust” understanding of languages based on concepts for more natural text generation.
Enable mid-generation reasoning, allowing AI to simulate and refine outputs before finalizing them, improving logical consistency and coherence.

Long-Term: Lifelong Learning & Self-Evolving, Personalized LLMs

I think dLLMs could blur the line between training and inference, enabling real-time model adaptation so models can improve continuously. This is based on their ability to do mid-generation adaptation, which should also eventually allow us to train based on user feedback and behavior.
Personalize diffusion spaces for individual users, evolving AI behavior dynamically to cater to specific needs and preferences.

Personalization/Modification in the latent space is much cheaper than techniques like fine-tuning, which can enable a very high degree of personalization. This is something we beta-rolled out with IQIDIS (our legal AI startup), and every lawyer who’s used it has so far been very impressed with it. It’s still very early stages, but there is tremendous potential in the latent space if you can properly use it.

However, when it comes to this personalization/self learning- two major problems remain-

Unlearning old information. Given the inter-connected nature of the Latent Space, you can have a lot of knock off effects, so precision unlearning seems to be impossible based on what I know (but this could very well be my lack of imagination).
Knowledge Injection with new information (especially when the sources are sparse).

I don’t see a good way to do this for the model itself (you’d have to use external setups like RAG). And that might just be it (refine how model interacts and pull up context saved externally), but I would love to hear your thoughts on this. Any interesting ideas that y’allz have worked on/come across?

“Specifically, we design a geometric proteinmolecule interaction network (PMINet), and pretrain it with binding affinity signals to: (i) retrieve target-aware ligand molecules with high binding affinity to serve as references, and (ii) incorporate essential protein-ligand binding structures for steering molecular diffusion generation with two effective augmentation mechanisms, i.e., retrieval augmentation and self augmentation. Empirical studies on CrossDocked2020 dataset show IRDIFF can generate molecules with more realistic 3D structures and achieve stateof-the-art binding affinities towards the protein targets, while maintaining proper molecular properties”

The way I see it, some of these ideas will fail. But if we pull off even a fraction — continuous language spaces, self-improving reasoning, and lifelong AI learning — could redefine AI as we know it. This isn’t just an incremental upgrade to LLMs.

We are also excited to report that developers prefer Mercury’s code completions compared to existing code models. When benchmarked on Copilot Arena, Mercury Coder Mini is tied for second place, surpassing the performance of speed-optimized models like GPT-4o Mini and Gemini-1.5-Flash and even of larger models like GPT-4o. At the same time, it is the fastest model, about 4 times faster than GPT-4o Mini.
-From the Mercury Release

It’s the foundation of an entirely new way for AI to think. And so it’s worth thinking about. If you’re on a similar wavelength, let’s explore these ideas in more depth and see where they take us.

I put a lot of work into writing this newsletter. To do so, I rely on you for support. If a few more people choose to become paid subscribers, the Chocolate Milk Cult can continue to provide high-quality and accessible education and opportunities to anyone who needs it. If you think this mission is worth contributing to, please consider a premium subscription. You can do so for less than the cost of a Netflix Subscription (pay what you want here).

I provide various consulting and advisory services. If you‘d like to explore how we can work together, reach out to me through any of my socials over here or reply to this email.

Understanding Why Diffusion Models are Worth Looking Into

Skip this section if you understand Diffusion Models (or scroll down to the relevant subsections)

What are Diffusion Models

As we talked about, Diffusion Models are based on the noising and denoising input. While the details vary, we can boil down Diffusion-based Generation into two steps-

Forward Diffusion: We take a data sample, like an image, and iteratively add small amounts of Gaussian noise at each step. This slowly corrupts the image until it becomes unrecognizable noise. The model learns the pattern of noise added at each step. This is crucial for the reverse process.

Reverse Diffusion: We begin with the pure noise from step 1 as input. The model predicts the noise added at each step in the forward process and removes it. This progressively denoises the input, gradually transforming it into a meaningful data sample.

At the risk of oversimplification, we will move from here. Diffusion has a lot of important math details that are hidden in the details, but I think it’s more important to discuss how/why Diffusion is being used to solve various challenges. I also don’t have any insightful commentary to add to the math/derivation. If you’re interested in the Math, please refer to either Lillian Weng’s article on Diffusion or this piece by Assembly AI (where we got the last two images).

For now, we move on to the next subsection. Why use Diffusion Models when other techniques exist (esp given the costs)? There are two ways to answer this question. First, we will talk about the emperical benefits of DMs. Then, I will speculate on why the generation is better.

What makes Diffusion Models Useful

At its core, the entire Diffusion process gives us 4 advantages

High-Quality Generation: Diffusion models generate data with exceptional quality and realism, often surpassing previous generative models in many tasks. This stems from their ability to learn the underlying data distribution granularly through the iterative denoising process. The slow and steady refinement from pure noise to a coherent data sample results in highly realistic outputs. A recent paper showed a Diffusion LLM outperforming/matching AutoRegressive Benchmarks in a variety of benchmarks while being in the same FLOPs range-

“Autoregressive models (ARMs) are widely regarded as the cornerstone of large language models (LLMs). **We challenge this notion by introducing LLaDA, a diffusion model trained from scratch under the pre-training and supervised fine-tuning (SFT) paradigm**. LLaDA models distributions through a forward data masking process and a reverse process, parameterized by a vanilla Transformer to predict masked tokens. By optimizing a likelihood bound, it provides a principled generative approach for probabilistic inference. Across extensive benchmarks, LLaDA demonstrates strong scalability, outperforming our self-constructed ARM baselines. Remarkably, LLaDA 8B is competitive with strong LLMs like LLaMA3 8B in in-context learning and, after SFT, exhibits impressive instruction-following abilities in case studies such as multi-turn dialogue. Moreover, LLaDA addresses the reversal curse, surpassing GPT-4o in a reversal poem completion task. Our findings establish diffusion models as a viable and promising alternative to ARMs, challenging the assumption that key LLM capabilities discussed above are inherently tied to ARMs. s”

Versatility: Diffusion models are remarkably flexible and can be applied to a wide range of data modalities, including images, audio, molecules, and more. This versatility comes from the model’s core mechanism of manipulating noise, a concept that can be applied to any data type represented digitally. Whether it’s pixels in an image, amplitudes in a sound wave, or atoms in a molecule, diffusion models can learn to generate and manipulate them. Diffusion can also be molded into different use-cases. This makes them a natural fit for multimodal settings.
Step-by-Step Control: The step-by-step generation process in diffusion models allows users to exert greater control over the final output. Unlike traditional generative models that produce an output in one shot, diffusion models progressively refine the generated data from noise to a final sample. This gives us greater transparency and the ability to jump in the middle to take experiments in new directions. Inception Labs noted this in point 3 of their conclusion-

First two points I want to study more before agreeing with them. For 4 I need more insight into better cost metrics. But 3 is legit and the others aren’t atleast all hype (which is more than some places).

Chaining with other tools for Agents- The more complete step-by-step generations of Diffusion allows better analysis of the model's current state, enabling a better suite of thinking models and evaluators. This is likely the reason that Mercury matches the performance of traditional Heavyweights even though they most likely had fewer dollars to spend on talent and model training (and there’s less information on Diffusion-based text generators). I think a lot of people are sleeping on this. This is one of the reasons that DGLM outperforms every other technique on attribute control in Text Generations (even for multiple attributes at once)- since the attribute controller works on the fully created diffusion guide (as opposed to the incomplete AR decoded setup).

Compare this to Anthropic’s Constitutional Classifier, which is meant to accomplish a subset of this capability (defense against jailbreaks). It has to use classifiers on complete texts (on user input and the completed model output)-

Covered this here. To their credit, this approach is speedy, since classifiers can work very fast.

Which can significantly reduce the attribute control and nuance of your outputs.

I think the mid-generation evals and shifts possible with Diffusion Models is being slept on hard. 2015 Kante levels of being slept on.

Okay, but why do DMs work as well? Don’t think there’s too much out there formally, so let’s do some theory-crafting.

Why Diffusion Models are So Good

Take a complex generation task like writing this article. Traditional generators like GANs generate everything in one shot. As complexity scales up, this becomes exceedingly difficult. Think about how hard it would be to paint a detailed scene by throwing paint at a canvas once. This is essentially what a GAN does. If you’ve read JJK (you absolutely should), this is very similar to the whole “giving someone water w/o giving them the bottle” explanation for Sukuna’s Domain.

Did Gege love Sukuna a bit too much? Discuss-Img Source

This is why modern text generators are based on Autoregression. AR scales a lot better, allowing our models to handle more complex tasks. Since AR goes step by step, you also can stop generation mid-way through or take it in a new direction. Those are two advantages of Autoregression over traditional generators. However, AR models can get lost in their sauce (I’m sure we’ve all had experiences with this). Going back to our article example- it’s really hard to write good articles just using autocomplete without having a very clear picture of what you want to do. Pure AR also degrades quickly b/c we can’t go back and edit previously generated components (if I was writing an article word by word, then I could not go back to an earlier para for restructuring).

Diffusion has the same step-by-step advantage as AR but with a twist. Since we denoise/input the entire input at every time step, Diffusion allows us to be a lot more contextual. Unlike traditional LLMs, it won’t continue to build on the errors, since the denoising step is done at every iteration.

This is closest to how I write. I generally have a fuzzy idea of what I want to cover, which is refined as I write more. At any given step, I might go back and fix a much earlier mistake/”noise” that got embedded into the article. This makes the end result much more cohesive and polished, as opposed to something that’s written in only one direction.

Hopefully, this has convinced you about the utility of Diffusion Models. Would be awkward if that answer is a no. In that case, I will use my best social skill, to keep prattling on endlessly. I’ve used it to save many social situations. In fact, a recent date was so awed by my intelligence and charm that she ran away crying about how she “couldn’t take it anymore” (no doubt referring to how my blinding presence) and is still thinking about how to reply to me, a week later. Mariya, if you’re reading this, I’d be happy to send you my “How to interact with People like a Pro” book for a discount so that we you’re never lost for words again.

Now that we understand why Diffusion-based Large Language Models are investing in, the next logical question becomes, what do we invest in? Let’s get into this.

What is next for Diffusion LLMs?

Near-Term Battles: Cost, Efficiency, and Proving Real-World Value

Comparing diffusion language models (dLLMs) to autoregressive (AR) models isn’t as simple as it looks. Mercury’s release highlights higher throughput and fewer iterations as signs of efficiency, but that doesn’t tell the full story. The per-inference cost of diffusion models is still much higher, which cancels out many of the apparent speed gains. To move forward, we need better ways to measure performance — ones that account for both raw compute and real-world usability.

Computational efficiency isn’t just about FLOPs. While FLOPs give a basic idea of workload, they don’t tell us how well models use modern hardware. Diffusion models have massive parallelization potential — something AR models fundamentally lack (interestingly, this is why early LLMs beat RNNs, given that they could be trained in parallel while RNNs could not).

Since dLLMs operate on the entire sequence at once, they could theoretically take full advantage of tensor cores and high-memory bandwidth. If optimized properly, diffusion models could actually run more efficiently in large-scale deployments where batching and parallel inference are key. But that’s a big if. The research community needs real-world benchmarks to test whether this advantage translates into meaningful throughput improvements.

Latency is another area where dLLMs need better evaluation. Right now, benchmarks often compare time-to-first-token, which favors AR models since they output words sequentially. But that doesn’t capture the full picture. What if diffusion models generate complete, coherent drafts faster than AR models can produce their first few words? This could enable entirely new interaction patterns — where users get fully formed responses they can immediately evaluate, instead of watching AI type out answers in real time. Quality-vs-time curves would be a better way to measure this tradeoff, rather than relying on single-point latency comparisons.

The biggest near-term challenge is optimization. Adaptive denoising schedules could cut down unnecessary computation by adjusting the number of refinement steps based on content complexity. This could be done in two ways-

Simple outputs would need fewer iterations, while complex reasoning tasks would get extra processing.
Building a learnable stopping criteria, where the model stops generating as soon as it hits a high-confidence threshold instead of following a rigid step count.

These techniques could significantly reduce costs without sacrificing quality.

What do you think of these images. Are they worthwhile?

Beyond efficiency, we need better evaluation metrics. Right now, most benchmarks focus on token-level accuracy, but diffusion models introduce entirely new strengths that aren’t being measured properly.

Global coherence — how well a model maintains logical consistency across long passages — is one area where dLLMs might have a real edge. Unlike AR models, which commit to every token as they generate, diffusion models can refine earlier sections as they go, potentially producing more internally consistent outputs.
constraint alignment — following specific instructions, adhering to strict formatting rules, and maintaining factual accuracy. The iterative nature of diffusion could make it better at sticking to complex constraints than AR models, which often drift off track.

These near-term optimizations aren’t just about making diffusion models run faster. They set the stage for something much bigger. If we can accurately quantify the cost-benefit tradeoffs, dLLMs won’t just be an interesting alternative to AR models — we’d be able to better analyze when/how to apply one over another, or how to chain them together.

The breakthroughs made in this phase — whether in hardware utilization, efficiency strategies, or better evaluation metrics — will directly shape the next generation of AI models.

Once we have laid the grounds with our base, we can start getting ambitious.

Mid Term- Embracing the Continous Latent Space

This is where the dLLM revolution can really kick in.

Instead of using “hard tokens”, we can let our dLLMs operate in a fluid, continuous latent space. The benefits of this have been discussed extensively by Meta. Here is their Large Concepts Model paper (it’s not a coincidence that it relies on Diffusions here)-

LLMs have revolutionized the field of artificial intelligence and have emerged as the de-facto tool for many tasks. The current established technology of LLMs is to process input and generate output at the token level. This is in sharp contrast to humans who operate at multiple levels of abstraction, well beyond single words, to analyze information and to generate creative content. In this paper, we present an attempt at an architecture which operates on an explicit higher-level semantic representation, which we name a “concept”. Concepts are language- and modality-agnostic and represent a higher level idea or action in a flow. Hence, we build a“Large Concept Model”. In this study, as proof of feasibility, we assume that a concept corresponds to a sentence, and use an existing sentence embedding space, SONAR, which supports up to 200 languages in both text and speech modalities. The Large Concept Model is trained to perform autoregressive sentence prediction in an embedding space. We explore multiple approaches, namely MSE regression, variants of diffusion-based generation, and models operating in a quantized SONAR space. These explorations are performed using 1.6B parameter models and training data in the order of 1.3T tokens. We then scale one architecture to a model size of 7B parameters and training data of about 7.7T tokens. We perform an experimental evaluation on several generative tasks, namely summarization and a new task of summary expansion. Finally, we show that our model exhibits impressive zero-shot generalization performance to many languages, outperforming existing LLMs of the same size. The training code of our models is freely available.

This is more than a technical detail — it’s a paradigm shift. We’re moving from representing language as a sequence of discrete symbols to encoding it as a rich, nuanced landscape of meaning. Using softer decision boundaries allows for better stability and less sharpness- improving performance if your model can pull it off-

While the router in Sparse MoE layers (left) learns to assign individual input tokens to each of the available slots, in Soft MoE layers (right) each slot is the result of a (different) weighted average of all the input tokens. Learning to make discrete assignments introduces several optimization and implementation issues that Soft MoE sidesteps

In this continuous space, subtle variations represent nuances that tokens simply miss. This could make dLLMs incredibly robust — less susceptible to errors, better at understanding paraphrases, and more adaptable to diverse writing styles. This latent space could be the key to unlocking better conceptual understanding. Instead of processing words as isolated units, dLLMs could manipulate underlying concepts, grasping meaning at a deeper, more abstract level. And just like in image generation, this continuous space allows for seamless interpolation — blending ideas, styles, and concepts in ways that dog-walks token-based models.

But imo the real game-changer is mid-generation reasoning. We’ve touched on this a few times, so won’t repeat this too much here, but once we have good evals- we can start chaining multiple scorers, evaluators, and “explorers” that can look at a particular step in the generation step, and decide to switch paths/rewrite plans on the fly. This should unlock a new level of flexibility that doesn’t exist in current systems-

Once you start representing concepts well in the latent space, you can pull off insane levels of personalization that don’t exist in the current space (which is likely why Meta is pushing Latent Space based reasoning so heavily)-

Long Term- Evolving AI in the Latent Spaces

The real potential of diffusion language models goes beyond better text generation or reasoning. As they mature, they could fundamentally change how AI learns and adapts. The current divide between training and inference — a defining limit of today’s models — could disappear, leading to systems that evolve continuously through interaction.

Diffusion models are uniquely suited for this because they don’t just generate text step by step — they revise and refine it. Unlike autoregressive models, which commit to each token in sequence, diffusion models can rework earlier parts of their output, integrating new information as they go (I’m sorry, you’re probably sick of me repeating this, but this is important). This opens the door to AI that adapts in real time without needing expensive retraining.

High-Resolution Image Synthesis with Latent Diffusion Models- using the latent space for everything is better. Get on the hype train bois.

The most immediate impact of this adaptability is personalization. Right now, tailoring AI behavior requires prompt engineering (repetitive) or fine-tuning (computationally expensive and risks overfitting). Diffusion models offer a more efficient alternative — modifying latent space representations instead of retraining the entire model. This would allow AI to develop user-specific styles, preferences, and reasoning approaches with far less computational overhead.

This can be done with AR models (most simply by prompting, but the additional ties in Diffusion Models will allow a more rigirous and complete implementation of this).

While I haven’t tested the Diffusion and Concept Models for this, we are doing algebra in the latent space at IQIDIS for 3 properties-

MultiModality.
Long context retrieval and chaining ideas across very long and disconnected chunks (the latter is important in contract review and better case strategy).
User personalization- learning how users like to answer questions, which sources they prefer etc.

We’ve had varying degrees of success with this (2 is the hardest and requires a lot of setup, which we’re still trying to figure out how to scale so that it does it automatically, even if the user doesn’t do things the right way). I won’t say we’ve cracked this, but we likely will within the next 6–8 months (atleast a more basic, but still competent version). I’ll likely do a detailed breakdown at that stage (assuming my CEO doesn’t murder me for constantly sharing all our developments b/c of my love for open source).

But that’s not all folks (I say as I’m eating a carrot, true story).

This also unlocks continuous learning. Instead of being frozen at deployment, diffusion-based models could learn from ongoing interactions, gradually improving their internal representations. This would allow AI to refine everything from general concepts to specific user terminology, making it increasingly attuned to its environment over time.

This will address the staticness of today’s AI- dramatically reducing the need for constant retraining (which is making Nvidia very rich)-

Anyone want to donate a few Nvidia Stocks to me? Will be eternally grateful

That said, two major challenges stand in the way: unlearning outdated information and integrating new knowledge from limited sources. The interconnected nature of latent space makes precise unlearning difficult — changes in one area could ripple across the model in unintended ways. Similarly, incorporating new information without full retraining remains an open research problem.

Privacy is another key issue. If models are continuously adapting, how do we ensure user data stays protected? One promising approach is federated learning, where personalization happens on local devices instead of centralized servers.

Now this might look I’m reaching a lot. And to be fair, I’m reaching more than it seems. However, if one of these works out, we’re all heading to the Hall of Fame. And even if it doesn’t, isn’t it so much fun to be delusional about something grand?

Let me know if any of these ideas struck a chord. Or what an idiot I am if none of these did. Either way, hearing from you makes my heart skip a beat. Sometimes even two.

Thank you for being here and I hope you have a wonderful day,

AIs Don Quixote,

Dev <3

If you liked this article and wish to share it, please refer to the following guidelines.

That is it for this piece. I appreciate your time. As always, if you’re interested in working with me or checking out my other work, my links will be at the end of this email/post. And if you found value in this write-up, I would appreciate you sharing it with more people. It is word-of-mouth referrals like yours that help me grow. You can share your testimonials over here.