DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression

Evan: Welcome to Daily Paper Cast.

Ashley: Today’s paper is from the Hugging Face daily paper list of September 18, 2026, and it has garnered 26 upvotes.

Evan: The title of the paper is DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression.

Ashley: The first two authors are Anyi Xu and Bing Zhang, with the corresponding author being DeepSeek-AI.

Evan: Alright, let’s dive into the Introduction section of this paper.

Ashley: Applications of long-horizon agents have rapidly expanded in recent years, making the processing of ultra-long contexts an increasingly crucial model workload.

As we move towards supporting these workloads efficiently, it’s essential to address the persistent storage, reuse, and transfer of large Key-Value, or KV, caches.

These requirements introduce significant computational, storage, and communication challenges.

Evan: Right, and while advances in sparse attention mechanisms have reduced computational costs, issues with persistent storage and data movement remain bottlenecks.

Ashley: Exactly.

The DeepSeek-V4 model, which leverages a global attention branch spanning the full context along with local sliding-window attention, shows significant improvements.

Evan: However, for sufficiently long sequences, the global KV ends up dominating the runtime KV footprint.

This is bounded by High Bandwidth Memory, or HBM, capacity.

Ashley: Correct.

In this context, certain KVs are persisted for prefix reuse, termed as persistent KV caches.

They are constrained by the SSD and host memory capacity, and further limited by I/O and interconnect bandwidth.

Evan: All these constraints hamper serving throughput, inflate deployment costs, and ultimately hinder the deployment of agents over longer task horizons and broader application scenarios.

Ashley: To address these issues, the paper introduces DeepSeek-V4.1-Flash.

This is a multimodal Mixture-of-Experts model, designed for more aggressive KV cache compression.

DeepSeek-V4.1-Flash has a massive 552 billion backbone parameters and supports multimodal inputs with contexts of up to one million tokens.

Evan: Alright, but how does this new version manage to handle such extensive data efficiently?

Ashley: The key innovation is the Causal Encoder-Decoder, or CED, architecture.

This architecture projects decoder global KV from the final encoder hidden states.

It activates 8 billion parameters per token during prefill and 16 billion during decode.

Evan: That sounds particularly beneficial for input-heavy scenarios.

Ashley: Indeed.

Despite its larger size, DeepSeek-V4.1-Flash requires only roughly one-fourth of the runtime KV cache storage and one-eighth of the persistent KV cache storage compared to its predecessor, DeepSeek-V4-Flash.

Evan: So, despite being more compact in terms of KV cache footprint, the model actually delivers better overall performance?

Ashley: Yes, and this level of KV cache compression comes from joint optimizations in the model architecture, cache precision, and deployment strategy.

Essentially, it's a local-processing backbone with compressed global context.

Their goal is to simplify the global attention mechanism while maintaining the local attention design.

Evan: That makes sense.

It sounds like the architecture has been streamlined for efficiency, but I'd love to know more about the specific techniques used for KV cache compression.

Ashley: Of course.

At the architectural level, they developed Compressed Sparse Attention 2, or CSA2.

This applies cross-layer reuse to global KV and indices, significantly reducing the required KV cache storage.

Through three modes—Full, Reindex, and Reuse—the model can optimize how KV cache and indexing work across layers.

Evan: And beyond architecture, were there any other optimizations introduced?

Ashley: On the precision level, they used FP4 for global KV caches during training, balancing compression with minimal performance degradation.

Deployment-wise, DeepSeek-V4.1-Flash introduces Sliding-Window Attention, or SWA, Bounded Replay, reducing the persistent KV cache footprint further by replaying only the most recent 𝑛win tokens.

Evan: Impressive.

This design not only contains the KV cache within manageable limits but also ensures high performance.

Ashley: Exactly.

Along with these architectural changes, the model halves the computational cost of extending context from 4K to 1 million tokens by requiring nearly constant single-token decode FLOPs.

Evan: So, it's about creating a balance between storage, computation, and overall efficiency?

Ashley: Exactly right.

This effectively mitigates the pressure on HBM and SSD capacities, reduces deployment costs, and facilitates large-scale deployment.

Evan: That wraps up the Introduction section.

Evan: Alright Ashley, let’s dive into the Method section to understand how DeepSeek-V4.1-Flash achieves these impressive results.

Can you walk us through the architecture?

Ashley: Sure thing.

DeepSeek-V4.1-Flash is a multimodal mixture-of-experts, or MoE, Transformer.

It takes both images and text as input and generates text autoregressively.

The language backbone consists of 40 causal Transformer layers, organized into a 20-layer causal encoder followed by a 20-layer decoder.

Evan: That's a massive structure!

How does it handle the different modalities?

Ashley: For that, it employs a vision encoder and an MLP projector to convert images into visual embeddings, which are processed jointly with text embeddings.

These multimodal data are included right from the start of language-model pretraining.

Evan: And what about the overall architecture specifics?

Ashley: Overall, DeepSeek-V4.1-Flash has 552 billion backbone parameters and 196 billion Engram parameters.

During prefill, it activates 8 billion parameters per token and 16 billion during decode.

Evan: You mentioned earlier about the Causal Encoder-Decoder architecture, the CED.

How does it contribute to efficiency?

Ashley: The CED architecture constructs the decoder’s global key-value cache using outputs from the causal encoder.

This design allows the system to bypass full decoder computation while retaining layer-local sliding-window attention, nearly halving the prefill computation.

Evan: That sounds like a significant efficiency boost, especially for long-context scenarios.

Ashley: Indeed.

It’s especially beneficial in agentic workloads where prefill requests are extensive.

CSA2, or Compressed Sparse Attention 2, also plays a crucial role.

It shares global key-value caches across layers and reuses sparse selections to optimize cache storage.

Evan: Can you elaborate on how CSA2 operates?

Ashley: CSA2 operates in three modes—Full, Reindex, and Reuse.

Full Mode involves generating global key-value entries and indexing.

Reindex Mode reuses the global key-value entries from a previous layer and produces new sparse selections.

Reuse Mode reuses the global key-value entries and selections from an earlier layer without recomputing them.

Evan: That sounds like it can significantly cut down on redundant computations.

Ashley: Exactly.

This structured reuse decreases both the KV cache storage and the indexing workload.

Furthermore, DeepSeek-V4.1-Flash uses a Hierarchical Sparse Indexer which restricts the indexers to a shared candidate pool selected by an earlier indexer, reducing the entries scored per query.

Evan: Beyond the architecture, what training mechanisms are in place to support this model?

Ashley: During training, DeepSeek-V4.1-Flash is pretrained on a multimodal corpus of 45 trillion tokens.

The sparse attention mechanism is trained from scratch at a sequence length of 64K tokens, and later extended to support contexts up to one million tokens.

Evan: Was there any special consideration for the training infrastructure?

Ashley: The training infrastructure is designed to handle such immense data.

It includes disaggregated vision-encoder execution, balanced image sharding for lengthy sequences, and cross-stage shared-state management for attention reuse.

This ensures efficient and scalable training for large-scale multimodal contexts.

Evan: And how is the inference system optimized?

Ashley: For inference, we see the implementation of Encoder and Decoder SWA Bounded Replay paths.

This includes communication-computation overlap, sharded Engram embedding tables, and inference kernel fusion, ensuring that each CSA2 Reuse Mode layer operates with only 15 kernels during prefill and 11 during decode.

Evan: Wow.

That's narrow kernel execution for such a large model, facilitating rapid processing.

Ashley: Indeed.

To further reduce the persistent KV cache footprint, SWA Bounded Replay is introduced.

It roughly halves the cache storage by replaying only the recent 𝑛win tokens, sidestepping the need for full forward pass reconstruction.

Evan: So, they utilize both architectural and precision-level optimizations to compress the KV cache footprint and maintain efficiency?

Ashley: Precisely.

They adopted quantization-aware training with FP4 for global key-value caches, preserving considerable storage without significant performance degradation.

The inference system is also designed to handle this format efficiently.

Evan: What about the dataset used for pretraining?

Ashley: The dataset construction is quite rigorous.

Text data is curated to ensure high informational value by filtering out model-generated content with limited value.

For multimodal data, they use image-text pairs, interleaved image-text data, and domain-specific data, focusing on cleaning and utilizing the data in its native form.

Evan: That must be an extensive data cleaning and preparation process.

Ashley: The pipeline involves multiple stages, including heuristic and statistical filtering, deduplication, and quality scoring using a specialized model, SmolVLM, to extract high-quality data.

Evan: So, they combine sophisticated data preprocessing with architectural innovations to build a highly efficient model?

Ashley: Exactly.

This combination ensures that DeepSeek-V4.1-Flash not only handles large contexts efficiently but also excels in diverse agentic and multimodal scenarios.

Evan: That wraps up the Method section of this paper.

Evan: Alright Ashley, now let’s delve into the Experiment and Results section.

What were the main experiments conducted in this paper?

Ashley: For sure, Evan.

The authors conducted comprehensive evaluations to validate the performance of DeepSeek-V4.1-Flash.

They benchmarked the model across several key dimensions: world knowledge, language understanding and reasoning, coding and mathematics, long context handling, and multimodal capabilities.

Evan: That's quite extensive.

So, how did DeepSeek-V4.1-Flash perform in these benchmarks?

Ashley: The results are impressive.

In the world knowledge dimension, benchmarks included AGIEval, MMLU-Pro, and C-Eval.

DeepSeek-V4.1-Flash showcased performance comparable to top open-source models.

For instance, on AGIEval, it scored 83.4 percent at 3-5 shots, just a bit lower than DeepSeek-V4-Pro at 84.4 percent.

Evan: Interesting.

And how did it fare in language understanding and reasoning tasks?

Ashley: DeepSeek-V4.1-Flash also showed strong results here.

For example, on the BigBench Hard, or BBH, benchmark, it achieved a score of 86.1 percent at 3-shot, which is very close to DeepSeek-V4-Pro’s 87.5 percent.

Evan: That's impressive consistency.

What about coding and mathematical benchmarks?

Ashley: In coding and mathematics, DeepSeek-V4.1-Flash excelled as well.

For instance, it achieved a pass rate of 79.4 percent on HumanEval, outperforming DeepSeek-V4-Pro, which scored 76.8 percent.

On the GSM8K benchmark for mathematics, it achieved 93.0 percent compared to DeepSeek-V4-Pro’s 92.6 percent at 8-shot.

Evan: Those are significant benchmarks that reflect real-world use cases.

How did the model handle long-context scenarios?

Ashley: For long-context tasks, DeepSeek-V4.1-Flash maintained robust performance.

On the LongBench-V2 benchmark, it scored 45.2 percent at 1-shot, similarly aligned with its predecessors.

The ability to handle one million tokens effectively is a standout feature.

Evan: What were the results in multimodal tasks?

Ashley: In multimodal benchmarks, DeepSeek-V4.1-Flash surpassed several baselines.

For example, it achieved a score of 56.5 percent on MMMU-Pro at 4-shot and 95.6 percent on DocVQA, outperforming top-tier open-source competitors in visual reasoning and interpreting professional charts.

Evan: Were there any additional evaluations conducted?

Ashley: Yes, they also carried out perplexity tests on internal datasets for enhanced realism.

This involved proprietary code repositories and academic materials focused on reasoning and problem-solving.

Evan: It's great to see such a thorough evaluation.

Were there any specific highlights in those tests?

Ashley: DeepSeek-V4.1-Flash achieved the lowest bits-per-byte, or BPB, on all tasks compared to its predecessors, demonstrating greater potential to serve as a strong base model for future applications.

Evan: What does all this mean for the practical deployment of DeepSeek-V4.1-Flash?

Ashley: The experiments solidify that DeepSeek-V4.1-Flash offers a favorable trade-off between capability and efficiency.

It excels across multiple dimensions, making it well-suited for deployment in real-world scenarios, including extensive text, multimodal, and agentic tasks.

Evan: That sounds promising.

Does the paper discuss any limitations or future directions?

Ashley: Yes, while the performance metrics are impressive, the authors acknowledge that there is still a performance gap in highly specialized, complex tasks compared to the leading closed-source systems.

Future work will focus on extending the stress-testing and evaluation frameworks and improving model robustness under extreme conditions.

Evan: It’s clear that DeepSeek-V4.1-Flash raises the bar but also leaves room for future improvements.

Ashley: Indeed.

They plan to continuously update the evaluation protocols and investigate coordinated scaling of data, model capacity, and reinforcement learning.

Evan: So, the journey doesn’t stop here.

There's ongoing work to push the boundaries of what these models can achieve.

Ashley: Exactly.

And with DeepSeek-V4.1-Flash as their new foundation, they’re well-positioned to explore the limits of AI model capabilities.

Evan: Well, that brings us to the end of the Experiment section.

Let’s transition to the next part of our discussion.

Evan: Alright Ashley, let’s shift our focus to the Related Work section.

How does DeepSeek-V4.1-Flash build upon and differ from previous work in key areas?

Ashley: Great question, Evan.

The Related Work section contextualizes the advancements made by DeepSeek-V4.1-Flash within the broader landscape.

It discusses prior research in sparse attention mechanisms and memory optimizations which are foundational to this model.

Evan: That makes sense.

Could you elaborate on the contributions from earlier versions of DeepSeek, particularly DeepSeek-V4?

Ashley: Certainly.

DeepSeek-V4 laid significant groundwork by integrating sparse attention mechanisms such as Compressed Sparse Attention, or CSA, and Heavily Compressed Attention, or HCA.

These techniques were pivotal in reducing the computational costs associated with long-sequence processing.

Evan: And were there other notable contributions in sparse attention mechanisms noted in the paper?

Ashley: Yes.

The paper also references work by Ainslie et al.

on Generalized Quantization for Attention, or GQA, which effectively reduced the number of key-value heads, and the MLA framework by DeepSeek-AI in 2024, which shared small latent variables across heads to further compress the representation.

Evan: I see.

So, these mechanisms collectively contributed to what CSA2 is building on today?

Ashley: Exactly.

CSA2, or Compressed Sparse Attention 2, extends these principles by reusing key-value cache entries and indices across layers in various modes—Full, Reindex, and Reuse.

This leverages previous approaches while enhancing storage and computational efficiency.

Evan: It seems like there have been numerous innovations in memory efficiency too.

How does that fit into the related work?

Ashley: DeepSeek-V4.1-Flash’s memory efficiency is significantly inspired by prior methods like IndexCache, introduced by Bai et al.

in 2026, which reused top-K indices across layers to minimize indexer computations.

Additionally, the YOCO framework by Sun et al., which simplifies encoding processes, has also been influential.

Evan: Interesting.

What about the hierarchical approaches?

Ashley: The Hierarchical Sparse Indexer in DeepSeek-V4.1-Flash is informed by Xu et al.'s work on hierarchical indexing for fine-grained sparse attention.

This approach scores token-level entries and prunes them based on block-level representations, significantly reducing computational burden for long contexts.

Evan: It sounds like a lot of collaborative and cumulative innovation brought us to this model.

How does this extend into the model’s training and precision optimizations?

Ashley: Previous efforts in quantization and training optimizations also feed into DeepSeek-V4.1-Flash.

For instance, the quantization-aware training techniques by Jacob et al.

in 2018 for FP4 format and the advancements in FP8 by Nvidia helped in the model’s FP4 cache precision and efficient storage.

Evan: How does this relate to DeepSeek-V4.1-Flash’s sliding-window attention mechanisms?

Ashley: The SWA Bounded Replay methodology in DeepSeek-V4.1-Flash extends the sliding-window attention concept, integrating insights from reconstruction methods and sparse attention optimizations developed over several iterations in sparse and sliding-window mechanisms.

Evan: Wow, that's a wealth of context.

Were there any other models or techniques referenced in the domain of multimodal training?

Ashley: In the domain of multimodal models, important groundwork includes the Vision Transformer, or ViT, framework by Dosovitskiy et al., which fundamentally influenced DeepSeek-ViT, the vision encoder used in DeepSeek-V4.1-Flash.

Techniques such as 2D-RoPE for positional embeddings and pixel-unshuffle operations for visual feature integration also trace back to earlier methodologies.

Evan: It sounds like DeepSeek-V4.1-Flash not only builds upon but significantly refines and integrates a multitude of previous innovations.

Ashley: This model exemplifies how integrating various advancements in sparse attention, memory efficiency, quantization, and multimodal processing can culminate in a highly efficient, powerful AI model.

Evan: Incredible.

That provides a comprehensive overview of how DeepSeek-V4.1-Flash fits within the landscape of related research.

It's clear that it’s a continuation of a deeply collaborative and iterative process.

Ashley: Exactly.

The Related Work section highlights the incremental nature of scientific progress and the collective efforts in advancing AI technologies.

Evan: Alright, that wraps up the Related Work section for today.

Evan: Alright Ashley, as we wrap up, could you summarize the key contributions and takeaways from the DeepSeek-V4.1-Flash paper?

Ashley: Of course, Evan.

DeepSeek-V4.1-Flash marks a significant advance in the field of AI, particularly regarding the management and efficiency of extensive data contexts.

The model's Causal Encoder-Decoder architecture is a key innovation, significantly reducing computational overhead during prefill while enhancing performance during decode.

Evan: Right, and we've also heard about the Compressed Sparse Attention 2, or CSA2, which optimizes key-value cache storage across layers, enhancing overall storage efficiency.

Ashley: Exactly.

CSA2 combined with the precision optimizations using FP4 significantly minimizes storage without major performance hits.

Moreover, the introduction of SWA Bounded Replay allows the model to maintain its performance while further reducing the persistent KV cache footprint.

Evan: All these elements together ensure that DeepSeek-V4.1-Flash can handle stunningly large contexts, up to one million tokens, making it highly suitable for diverse agentic and multimodal tasks.

Ashley: Yes, and the model's comprehensive evaluations in real-world scenarios—as highlighted by its performance in benchmarks for world knowledge, language understanding, coding, long-context handling, and multimodal capabilities—underscore its practical utility and effectiveness.

Evan: It's fascinating to see how these sophisticated optimizations and advances come together to make such powerful capabilities more efficient and accessible.

Ashley: Evan.

DeepSeek-V4.1-Flash is a testament to the ongoing progress in AI research, pushing boundaries while also addressing practical deployment challenges.

Evan: And that’s all for today’s episode.

Thanks for tuning in to our deep dive into DeepSeek-V4.1-Flash.

Ashley: We hope you found this discussion insightful.

Be sure to join us again for our next episode, where we'll explore more cutting-edge research from the AI community.

Evan: Don't forget to subscribe and leave us a review if you enjoyed the episode.

Ashley: Until next time, keep exploring the frontiers of AI and stay curious!

Evan: Goodbye for now.