LLM Architecture Gallery Diff Tool
I added several updates to the LLM Architecture Gallery over the last two weeks.
The most useful one is the new architecture diff tool. It lets you select two models and compare their architecture stacks side by side.
The comparison view currently highlights:
- architecture figures
- attention block differences
- decoder type
- KV-cache footprint per token
- layer recipe
- model scale and context length
You can use the selectors in the diff tool directly, or click the Model A and Model B actions on individual gallery cards before jumping to the comparison view.
This is mainly meant for cases where two models look similar at a high level, but differ in one or two important details. DeepSeek V3 versus DeepSeek V3.2 is a good example because the stack is largely shared, but the attention block changes from MLA to MLA with DeepSeek Sparse Attention.
Source: lightly edited website version of my Substack note.
Read Next
Inkling: A New Open-Weight 975B MoE with a Few Surprises
Short note on Thinking Machines Lab's 975B Inkling open-weight model, its benchmark profile, sparse MoE design, short convolutions, embedding RMSNorm, and
200,000 Subscribers
Short note celebrating Ahead of AI reaching 200,000 subscribers.
GPT 5.6 Has 72 Possible Configurations. What's A Good Default?
Short note on how GPT 5.6 model and effort choices map onto training-time and inference-time scaling, producing 72 configurations.