Information Theory

2025-04-25 | | Total: 17

#1 Coding for Computation: Efficient Compression of Neural Networks for Reconfigurable Hardware [PDF1] [Copy] [Kimi] [REL]

Authors: Hans Rosenberger, Rodrigo Fischer, Johanna S. Fröhlich, Ali Bereyhi, Ralf R. Müller

As state of the art neural networks (NNs) continue to grow in size, their resource-efficient implementation becomes ever more important. In this paper, we introduce a compression scheme that reduces the number of computations required for NN inference on reconfigurable hardware such as FPGAs. This is achieved by combining pruning via regularized training, weight sharing and linear computation coding (LCC). Contrary to common NN compression techniques, where the objective is to reduce the memory used for storing the weights of the NNs, our approach is optimized to reduce the number of additions required for inference in a hardware-friendly manner. The proposed scheme achieves competitive performance for simple multilayer perceptrons, as well as for large scale deep NNs such as ResNet-34.

Subjects: Machine Learning , Information Theory , Signal Processing

Publish: 2025-04-24 09:49:18 UTC


#2 Integrated Sensing and Communications for Unsourced Random Access: A Spectrum Sharing Compressive Sensing Approach [PDF1] [Copy] [Kimi] [REL]

Authors: Zhentian Zhang, Jian Dang, Kai-Kit Wong, Zaichen Zhang, Christos Masouros

This paper addresses the unsourced/uncoordinated random access problem in an integrated sensing and communications (ISAC) system, with a focus on uplink multiple access code design. Recent theoretical advancements highlight that an ISAC system will be overwhelmed by the increasing number of active devices, driven by the growth of massive machine-type communication (mMTC). To meet the demands of future mMTC network, fundamental solutions are required that ensure robust capacity while maintaining favorable energy and spectral efficiency. One promising approach to support emerging massive connectivity is the development of systems based on the unsourced ISAC (UNISAC) framework. This paper proposes a spectrum-sharing compressive sensing-based UNISAC (SSCS-UNISAC) and offers insights into the practical design of UNISAC multiple access codes. In this framework, both communication signals (data transmission) and sensing signals (e.g., radar echoes) overlap within finite channel uses and are transmitted via the proposed UNISAC protocol. The proposed decoder exhibits robust performance, providing 20-30 dB capacity gains compared to conventional protocols such as TDMA and ALOHA. Numerical results validate the promising performance of the proposed scheme.

Subject: Information Theory

Publish: 2025-04-24 14:53:28 UTC


#3 Path Integral Methods for Synthesizing and Preventing Stealthy Attacks in Nonlinear Cyber-Physical Systems [PDF] [Copy] [Kimi] [REL]

Authors: Apurva Patil, Kyle Morgenstein, Luis Sentis, Takashi Tanaka

This paper studies the synthesis and mitigation of stealthy attacks in nonlinear cyber-physical systems (CPS). To quantify stealthiness, we employ the Kullback-Leibler (KL) divergence, a measure rooted in hypothesis testing and detection theory, which captures the trade-off between an attacker's desire to remain stealthy and her goal of degrading system performance. First, we synthesize the worst-case stealthy attack in nonlinear CPS using the path integral approach. Second, we consider how a controller can mitigate the impact of such stealthy attacks by formulating a minimax KL control problem, yielding a zero-sum game between the attacker and the controller. Again, we leverage a path integral-based solution that computes saddle-point policies for both players through Monte Carlo simulations. We validate our approach using unicycle navigation and cruise control problems, demonstrating how an attacker can covertly drive the system into unsafe regions, and how the controller can adapt her policy to combat the worst-case attacks.

Subjects: Systems and Control , Information Theory

Publish: 2025-04-23 22:01:29 UTC


#4 P$_\ell$-Kyber: Packing $\ell$ Plaintexts and Lattice Coding for Kyber [PDF] [Copy] [Kimi] [REL]

Authors: Shuiyin Liu, Amin Sakzad

In this work, we propose a joint design of encoding and encryption processes for KEMs like Kyber, without assuming the independence of the decoding noise entries. Our design features two techniques: ciphertext packing and lattice packing. First, we extend the Peikert-Vaikuntanathan-Waters (PVW) method to the Kyber: $\ell$ plaintexts are packed into a single ciphertext. This scheme is referred to as P$_\ell$-Kyber. We prove that the P$_\ell$-Kyber is IND-CCA secure under the M-LWE hardness assumption. We show that the decryption decoding noise entries across the $\ell$ plaintexts (also known as layers) are mutually independent. Second, we propose a cross-layer lattice encoding scheme for the P$_\ell$-Kyber, where every $\ell$ cross-layer information symbols are encoded to a lattice point. This way we obtain a \emph{coded} P$_\ell$-Kyber, where the decoding noise entries for each lattice point are mutually independent. Therefore, the decryption failure rate (DFR) analysis does not require the assumption of independence among the decryption decoding noise entries. Both DFR and communication cost (CER) are greatly decreased thanks to ciphertext packing and lattice packing. Finally, we demonstrate that with $\ell=24$ and Leech lattice encoder, the proposed coded P$_\ell$-KYBER1024 achieves DFR $<2^{-281}$ and CER $ = 4.6$, i.e., a decrease of CER by $90\%$ compared to KYBER1024.

Subjects: Cryptography and Security , Information Theory

Publish: 2025-04-24 01:39:36 UTC


#5 Quantum-Enhanced Change Detection and Joint Communication-Detection [PDF] [Copy] [Kimi] [REL]

Authors: Zihao Gong, Saikat Guha

Quick detection of transmittance changes in optical channel is crucial for secure communication. We demonstrate that pre-shared entanglement using two-mode squeezed vacuum states significantly reduces detection latency compared to classical and entanglement-augmented coherent-state probes. The change detection latency is inversely proportional to the quantum relative entropy (QRE), which goes to infinity in the absence of thermal noise, suggesting idealized instantaneous detection. However, in realistic scenarios, we show that QRE scales logarithmically with the inverse of the thermal noise mean photon number. We propose a receiver that achieves this scaling and quantify its performance gains over existing methods. Additionally, we explore the fundamental trade-off between communication capacity and change detection latency, highlighting how pre-shared entanglement enhances both.

Subjects: Quantum Physics , Information Theory

Publish: 2025-04-24 04:17:55 UTC


#6 UNILoc: Unified Localization Combining Model-Based Geometry and Unsupervised Learning [PDF] [Copy] [Kimi] [REL]

Authors: Yuhao Zhang, Guangjin Pan, Musa Furkan Keskin, Ossi Kaltiokallio, Mikko Valkama, Henk Wymeersch

Accurate mobile device localization is critical for emerging 5G/6G applications such as autonomous vehicles and augmented reality. In this paper, we propose a unified localization method that integrates model-based and machine learning (ML)-based methods to reap their respective advantages by exploiting available map information. In order to avoid supervised learning, we generate training labels automatically via optimal transport (OT) by fusing geometric estimates with building layouts. Ray-tracing based simulations are carried out to demonstrate that the proposed method significantly improves positioning accuracy for both line-of-sight (LoS) users (compared to ML-based methods) and non-line-of-sight (NLoS) users (compared to model-based methods). Remarkably, the unified method is able to achieve competitive overall performance with the fully-supervised fingerprinting, while eliminating the need for cumbersome labeled data measurement and collection.

Subjects: Signal Processing , Information Theory

Publish: 2025-04-24 15:45:54 UTC


#7 Quantum Error Correction with Girth-16 Non-Binary LDPC Codes via Affine Permutation Construction [PDF] [Copy] [Kimi] [REL]

Author: Kenta Kasai

We propose a method for constructing quantum error-correcting codes based on non-binary low-density parity-check codes with girth 16. In conventional constructions using circulant permutation matrices, the girth is upper-bounded by 12, which limits the suppression of harmful short cycles. Our construction employs affine permutation matrices and a randomized sequential selection procedure designed to eliminate short cycles, which are known to limit decoding performance. Joint belief propagation decoding is applied over depolarizing channels. Numerical experiments confirm that the proposed codes reduce the number of low-weight codewords in $C_X \setminus C_Z^\perp$ and $C_Z \setminus C_X^\perp$, and thus have the potential to suppress error floors. In addition, we obtain a significantly improved upper bound on the minimum distance, which we conjecture to be tight.

Subjects: Quantum Physics , Information Theory

Publish: 2025-04-24 17:59:58 UTC


#8 A Coding-Enhanced Jamming Approach for Secure Semantic Communication over Wiretap Channels [PDF] [Copy] [Kimi] [REL]

Authors: Weixuan Chen, Qianqian Yang, Shuo Shao, Zhiguo Shi, Jiming Chen, Xuemin, Shen

As semantic communication (SemCom) gains increasing attention as a novel communication paradigm, ensuring the security of transmitted semantic information over open wireless channels becomes crucial. Existing secure SemCom solutions often lack explicit control over security. To address this, we propose a coding-enhanced jamming approach for secure SemCom over wiretap channels. This approach integrates deep joint source and channel coding (DeepJSCC) with neural network-based digital modulation, enabling controlled jamming through two-layer superposition coding. The outer constellation sequence encodes the source image, while the inner constellation sequence, derived from a secret image, acts as the jamming signal. By minimizing the mutual information between the outer and inner constellation sequences, the jamming effect is enhanced. The jamming signal is superposed on the outer constellation sequence, preventing the eavesdropper from recovering the source image. The power allocation coefficient (PAC) in the superposition coding can be adjusted to control system security. Experiments show that our approach matches existing methods in security while significantly improving reconstruction performance across varying channel signal-to-noise ratios (SNRs) and compression ratios.

Subjects: Information Theory , Image and Video Processing

Publish: 2025-04-23 05:45:36 UTC


#9 Relationship between Hölder Divergence and Functional Density Power Divergence: Intersection and Generalization [PDF] [Copy] [Kimi] [REL]

Author: Masahiro Kobayashi

In this study, we discuss the relationship between two families of density-power-based divergences with functional degrees of freedom -- the Hölder divergence and the functional density power divergence (FDPD) -- based on their intersection and generalization. These divergence families include the density power divergence and the $\gamma$-divergence as special cases. First, we prove that the intersection of the Hölder divergence and the FDPD is limited to a general divergence family introduced by Jones et al. (Biometrika, 2001). Subsequently, motivated by the fact that Hölder's inequality is used in the proofs of nonnegativity for both the Hölder divergence and the FDPD, we define a generalized divergence family, referred to as the $\xi$-Hölder divergence. The nonnegativity of the $\xi$-Hölder divergence is established through a combination of the inequalities used to prove the nonnegativity of the Hölder divergence and the FDPD. Furthermore, we derive an inequality between the composite scoring rules corresponding to different FDPDs based on the $\xi$-Hölder divergence. Finally, we prove that imposing the mathematical structure of the Hölder score on a composite scoring rule results in the $\xi$-Hölder divergence.

Subjects: Information Theory , Statistics Theory , Machine Learning

Publish: 2025-04-23 18:00:13 UTC


#10 Rate-Distortion-Perception Theory for the Quadratic Wasserstein Space [PDF] [Copy] [Kimi] [REL]

Authors: Xiqiang Qu, Jun Chen, Lei Yu, Xiangyu Xu

We establish a single-letter characterization of the fundamental distortion-rate-perception tradeoff with limited common randomness under the squared error distortion measure and the squared Wasserstein-2 perception measure. Moreover, it is shown that this single-letter characterization can be explicitly evaluated for the Gaussian source. Various notions of universal representation are also clarified.

Subjects: Information Theory , Machine Learning

Publish: 2025-04-24 04:13:56 UTC


#11 Service Rate Regions of MDS Codes & Fractional Matchings in Quasi-uniform Hypergraphs [PDF] [Copy] [Kimi] [REL]

Authors: Hoang Ly, Emina Soljanin

The service rate region (SRR) has emerged as a critical performance metric for distributed systems that store data redundantly. It measures the system's ability to serve multiple users concurrently. Mathematically, the SRR is a polytope in R^k where each dimension corresponds to the service request rate of one of the k data objects. This paper focuses on systems employing a class of Maximum Distance Separable (MDS) codes. For each code in the class, we characterize the k axes intercept points of its SRR, and the smallest standard simplex that includes the SRR. We use these results to show that the SRR grows with the increasing number of systematic columns in the generator matrices. We establish a graph-theoretic framework associating this SRR problem with fractional matchings in quasi-uniform hypergraphs. Identifying the SRR polytope is equivalent to determining a particular image of the fractional-matching polytope. We introduce a notion of Greedy Matching and show that it is sufficient to focus on these matchings to characterize the SRR rather than the entire matching polytope. With these tools, we determine the SRR of a large subset of the considered class of codes. Our results generalize previous characterizations of systematic and non-systematic MDS-coded systems, offering a unified framework for analyzing service rate regions of codes.

Subjects: Information Theory , Combinatorics

Publish: 2025-04-24 04:44:37 UTC


#12 Error Exponents for DNA Storage Codes with a Variable Number of Reads [PDF] [Copy] [Kimi] [REL]

Authors: Yan Hao Ling, Nir Weinberger, Jonathan Scarlett

In this paper, we study error exponents for a concatataned coding based class of DNA storage codes in which the number of reads performed can be variable. That is, the decoder can sequentially perform reads and choose whether to output the final decision or take more reads, and we are interested in minimizing the average number of reads performed rather than a fixed pre-specified value. We show that this flexibility leads to a considerable reduction in the error probability compared to a fixed number of reads, not only in terms of constants in the error exponent but also in the scaling laws. This is shown via an achievability result for a suitably-designed protocol, and in certain parameter regimes we additionally establish a matching converse that holds for all protocols within a broader concatenated coding based class.

Subject: Information Theory

Publish: 2025-04-24 07:56:21 UTC


#13 Subcode Ensemble Decoding of Polar Codes [PDF] [Copy] [Kimi] [REL]

Authors: Henning Lulei, Jonathan Mandelbaum, Marvin Rübenacke, Holger Jäkel, Stephan ten Brink, Laurent Schmalen

In the short block length regime, pre-transformed polar codes together with successive cancellation list (SCL) decoding possess excellent error correction capabilities. However, in practice, the list size is limited due to the suboptimal scaling of the required area in hardware implementations. Automorphism ensemble decoding (AED) can improve performance for a fixed list size by running multiple parallel SCL decodings on permuted received words, yielding a list of estimates from which the final estimate is selected. Yet, AED is limited to appropriately designed polar codes. Subcode ensemble decoding (ScED) was recently proposed for low-density parity-check codes and does not impose such design constraints. It uses multiple decodings in different subcodes, ensuring that the selected subcodes jointly cover the original code. We extend ScED to polar codes by expressing polar subcodes through suitable pre-transformations (PTs). To this end, we describe a framework classifying pre-transformations for pre-transformed polar codes based on their role in encoding and decoding. Within this framework, we propose a new type of PT enabling ScED for polar codes, analyze its properties, and discuss how to construct an efficient ensemble.

Subject: Information Theory

Publish: 2025-04-24 12:55:23 UTC


#14 Secure Network Function Computation for Linear Functions, Part II: Target-Function Security [PDF] [Copy] [Kimi] [REL]

Authors: Yang Bai, Xuan Guang, Raymond W. Yeung

In this Part II of a two-part paper, we put forward secure network function computation, where in a directed acyclic network, a sink node is required to compute a target function of which the inputs are generated as source messages at multiple source nodes, while a wiretapper, who can access any one but not more than one wiretap set in a given collection of wiretap sets, is not allowed to obtain any information about a security function of the source messages. In Part I of the two-part paper, we have investigated securely computing linear functions with the wiretapper who can eavesdrop any edge subset up to a certain size r, referred to as the security level, where the security function is the identity function. The notion of this security is called source security. In the current paper, we consider another interesting model which is the same as the above one except that the security function is identical to the target function, i.e., we need to protect the information on the target function from being leaked to the wiretapper. The notion of this security is called target-function security. We first prove a non-trivial upper bound on the secure computing capacity, which is applicable to arbitrary network topologies and arbitrary security levels. In particular, when the security level r is equal to 0, the upper bound reduces to the computing capacity without security consideration. Further, from an algebraic point of view, we prove two equivalent conditions for target-function security and source security for the existence of the corresponding linear function-computing secure network codes. With them, for any linear function over a given finite field, we develop a code construction of linear secure network codes for target-function security and thus obtain a lower bound on the secure computing capacity; and also generalize the code construction developed in Part I for source security.

Subject: Information Theory

Publish: 2025-04-24 12:57:20 UTC


#15 MacWilliams Theory over Zk and nu-functions over Lattices [PDF] [Copy] [Kimi] [REL]

Authors: Zhiyong Zheng, Fengxia Liu, Kun Tian

Continuing previous works on MacWilliams theory over codes and lattices, a generalization of the MacWilliams theory over $\mathbb{Z}_k$ for $m$ codes is established, and the complete weight enumerator MacWilliams identity also holds for codes over the finitely generated rings $\mathbb{Z}_k[\xi]$. In the context of lattices, the analogy of the MacWilliams identity associated with nu-function was conjectured by Solé in 1995, and we present a new formula for nu-function over the lattices associated with a ternary code, which is rather different from the original conjecture. Furthermore, we provide many counterexamples to show that the Solé conjecture never holds in the general case, except for the lattices associated with a binary code.

Subject: Information Theory

Publish: 2025-04-24 14:19:51 UTC


#16 Sparsity-Exploiting Channel Estimation For Unsourced Random Access With Fluid Antenna [PDF] [Copy] [Kimi] [REL]

Authors: Keru Zhou, Zhentian Zhang, Jian Dang, Qianqian Sun, Zaichen Zhang

This work explores the channel estimation (CE) problem in uplink transmission for unsourced random access (URA) with a fluid antenna receiver. The additional spatial diversity in a fluid antenna system (FAS) addresses the needs of URA design in multiple-input and multiple-output (MIMO) systems. We present two CE strategies based on the activation of different FAS ports, namely alternate ports and partial ports CE. Both strategies facilitate the estimation of channel coefficients and angles of arrival (AoAs). Additionally, we discuss how to refine channel estimation by leveraging the sparsity of finite scatterers. Specifically, the proposed partial ports CE strategy is implemented using a regularized estimator, and we optimize the estimator's parameter to achieve the desired AoA precision and refinement. Extensive numerical results demonstrate the feasibility of the proposed strategies, and a comparison with a conventional receiver using half-wavelength antennas highlights the promising future of integrating URA and FAS.

Subject: Information Theory

Publish: 2025-04-24 15:07:12 UTC


#17 DTECM: Digital Twin Enabled Channel Measurement and Modeling in Terahertz Urban Macrocell [PDF] [Copy] [Kimi] [REL]

Authors: Yuanbo Li, Ziming Yu, Chong Han

In this work, in the THz UMa, extensive channel measurements are conducted and an accurate channel model is developed by combining ray-tracing, computer vision (CV), and statistical methods. Specifically, substantial channel measurement campaigns with distances up to 410~m are conducted at 220~GHz, with nanosecond-level absolute time synchronization. Based on the measurement results, the propagation phenomena are analyzed in detail and the channel characteristics are calculated and statistically modeled. Furthermore, a digital twin enabled channel model (DTECM) is proposed, which generates THz channel responses in a hybrid manner. Specifically, the dominant paths are generated deterministically by using the ray-tracing technique and CV methods. Apart from the path gains determined by ray-tracing, the additional foliage loss is accurately modeled based on foliage information extracted from panoramic pictures. To maintain a low computational complexity for the DTECM, non-dominant paths are then generated statistically. Numeric results reveal that compared to the traditional statistical channel models, the DTECM reduces the path loss modeling error from 14~dB to 4~dB, showing its great superiority. Furthermore, a preliminary link performance evaluation using the DTECM indicates that THz UMa is feasible, though requiring high antenna gains and coverage extension techniques to achieve high spectral efficiencies and wide coverage.

Subject: Information Theory

Publish: 2025-04-24 15:40:57 UTC