Speculative decoding in large language models (LLMs) relies on a crucial hyperparameter, speculation length, which dictates the number of tokens proposed by a draft model for verification by a target model. Most systems utilize a fixed speculation length, typically set to 4, despite empirical evidence suggesting that adaptivity can yield better results. Researchers have introduced SpecKV, an adaptive speculative decoding approach that incorporates compression-aware gamma selection, allowing for more efficient inference. By dynamically adjusting the speculation length, SpecKV aims to optimize the trade-off between computational overhead and decoding accuracy. This adaptive approach has significant implications for the performance and efficiency of LLMs, particularly in applications where computational resources are limited1. So what matters to practitioners is that SpecKV's adaptive speculation length can lead to more efficient and accurate LLM inference, potentially enhancing the overall performance of these models in real-world applications.