CommIT Group at NYU

Professor of ECE@ NYU

Neural Distributed Compression

Many applications from camera arrays to sensor networks require efficient compression and processing of correlated data, which in general is collected in a distributed fashion. While information-theoretic foundations of distributed compression are well investigated, the impact of theory in practice-oriented applications to this day has been somewhat limited. As the field of data compression is undergoing a transformation with the emergence of learning-based techniques, machine learning is becoming an important tool to reap the long-promised benefits of distributed compression. In this paper, we review the recent contributions in the broad area of learned distributed compression techniques for abstract sources and images. In particular, we discuss approaches that provide interpretable results operating close to information-theoretic bounds. We also highlight unresolved research challenges, aiming to inspire fresh interest and advancements in the field of learned distributed compression.

Visualization of our proposed learning-based compress-and-forward strategy (which leverages our prior work on neural Wyner-Ziv compressors) and demodulation decisions for the 4-PAM modulation. The horizontal lines denote the quantization boundaries, and the colors designate the transmitted index. The vertical lines denote the hard decision boundaries for the demodulator, and the markers represent the decisions. The transmitted symbols (denoted by cross, triangle, star, square) are also reported near the axis for reference.

 

Visualization of our proposed learning-based compress-and-forward strategy (which leverages our prior work on neural Wyner-Ziv compressors) and demodulation decisions for the 4-QAM modulation relay rate 𝑅 ≈1. Figure (a) shows the quantization boundaries; the colors designate the transmitted index. Figures (b) and (c) show the hard decision boundaries for the demodulator (on the complex plane), where different colors represent the different decisions. Figure (b) represents the decisions when the transmitted index corresponds to the blue one from Figure (a); Figure (c) represents the decisions when the transmitted index corresponds to the red one from Figure (a). The transmitted symbols (denoted by cross, triangle, star, square) are also reported for reference.

Related Publications:

E. Ozyilkan*, F. Carpi*, S. Garg, E. Erkip, “Learning-Based Compress-and-Forward Schemes for the Relay Channel“, to appear at IEEE Journal on Selected Areas in Communications, part of the special issue on new approaches to data compression.

E. Ozyilkan, J. BallĂ©, E. Erkip, “Neural Distributed Compressor Discovers Binning“, IEEE Journal on Selected Areas in Information Theory (JSAIT), part of the special issue on Toby Berger.

E. Ozyilkan, J. BallĂ©, S. Bhadane, A. B. Wagner, E. Erkip, “Breaking Smoothness: The Struggles of Neural Compressors with Discontinuous Mappings“, Compression and Machine Learning Workshop @ NeurIPS 2024, Vancouver, December 2024.

 

Please refer to Ezgi’s website for more information.