Late Breaking Results: Quamba-SE: Soft-edge Quantizer for Activations in State Space Models
Late Breaking Results: Quamba-SE: Soft-edge Quantizer for Activations in State Space Models
We propose Quamba-SE, a soft-edge quantizer for State Space Model (SSM) activation quantization. Unlike existing methods, using standard INT8 operation, Quamba-SE employs three adaptive scales: high-precision for small values, standard scale for normal values, and low-precision for outliers. This preserves outlier information instead of hard clipping, while maintaining precision for other values. We evaluate on Mamba- 130M across 6 zero-shot benchmarks. Results show that Quamba- SE consistently outperforms Quamba, achieving up to +2.68% on individual benchmarks and up to +0.83% improvement in the average accuracy of 6 datasets.
Accepted to DATE Late Breaking Results 2026, Verona, Italy
Hardware Architecture, FOS: Computer and information sciences, State Space Models, Machine Learning (cs.LG), Machine Learning, Artificial Intelligence (cs.AI), Artificial Intelligence, Quantization, Quamba, Hardware Architecture (cs.AR), Inbäddad systemteknik, Embedded Systems
Hardware Architecture, FOS: Computer and information sciences, State Space Models, Machine Learning (cs.LG), Machine Learning, Artificial Intelligence (cs.AI), Artificial Intelligence, Quantization, Quamba, Hardware Architecture (cs.AR), Inbäddad systemteknik, Embedded Systems
selected citations These citations are derived from selected sources.This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).0 popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.Average influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).Average impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.Average
Citations provided by BIP!Popularity provided by BIP!