Processing math: 3%

Information Theory

2025-07-04 | | Total: 10

#1 RIS-Aided Cooperative ISAC Networks for Structural Health Monitoring [PDF] [Copy] [Kimi] [REL]

Authors: Jie Yang, Chao-Kai Wen, Xiao Li, Shi Jin

Integrated sensing and communication (ISAC) is a key feature of future cellular systems, enabling applications such as intruder detection, monitoring, and tracking using the same infrastructure. However, its potential for structural health monitoring (SHM), which requires the detection of slow and subtle structural changes, remains largely unexplored due to challenges such as multipath interference and the need for ultra-high sensing precision. This study introduces a novel theoretical framework for SHM via ISAC by leveraging reconfigurable intelligent surfaces (RIS) as reference points in collaboration with base stations and users. By dynamically adjusting RIS phases to generate distinct radio signals that suppress background multipath interference, measurement accuracy at these reference points is enhanced. We theoretically analyze RIS-aided collaborative sensing in three-dimensional cellular networks using Fisher information theory, demonstrating how increasing observation time, incorporating additional receivers (even with self-positioning errors), optimizing RIS phases, and refining collaborative node selection can reduce the position error bound to meet SHM's stringent accuracy requirements. Furthermore, we develop a Bayesian inference model to identify structural states and validate damage detection probabilities. Both theoretical and numerical analyses confirm ISAC's capability for millimeter-level deformation detection, highlighting its potential for high-precision SHM applications.

Subjects: Information Theory , Signal Processing

Publish: 2025-07-03 15:43:06 UTC


#2 On the Convergence of Large Language Model Optimizer for Black-Box Network Management [PDF] [Copy] [Kimi] [REL]

Authors: Hoon Lee, Wentao Zhou, Merouane Debbah, Inkyu Lee

Future wireless networks are expected to incorporate diverse services that often lack general mathematical models. To address such black-box network management tasks, the large language model (LLM) optimizer framework, which leverages pretrained LLMs as optimization agents, has recently been promoted as a promising solution. This framework utilizes natural language prompts describing the given optimization problems along with past solutions generated by LLMs themselves. As a result, LLMs can obtain efficient solutions autonomously without knowing the mathematical models of the objective functions. Although the viability of the LLM optimizer (LLMO) framework has been studied in various black-box scenarios, it has so far been limited to numerical simulations. For the first time, this paper establishes a theoretical foundation for the LLMO framework. With careful investigations of LLM inference steps, we can interpret the LLMO procedure as a finite-state Markov chain, and prove the convergence of the framework. Our results are extended to a more advanced multiple LLM architecture, where the impact of multiple LLMs is rigorously verified in terms of the convergence rate. Comprehensive numerical simulations validate our theoretical results and provide a deeper understanding of the underlying mechanisms of the LLMO framework.

Subjects: Information Theory , Signal Processing

Publish: 2025-07-03 14:59:42 UTC


#3 Measurements and Modeling of Air-Ground Integrated Channel in Forest Environment Based on OFDM Signals [PDF] [Copy] [Kimi] [REL]

Authors: Zhe Xiao, Shu Sun, Na Liu, Lianming Xu, Li Wang

Forests are frequently impacted by climate conditions, vegetation density, and intricate terrain and geology, which contribute to natural disasters. Personnel engaged in or supporting rescue operations in such environments rely on robust communication systems to ensure their safety, highlighting the criticality of channel measurements in forest environments. However, according to current research, there is limited research on channel detection and modeling in forest areas in the existing literature. This paper describes the channel measurements campaign of air and ground in the Arxan National Forest Park of Inner Mongolia. It presents measurement results and propagation models for ground-to-ground (G2G) and air-to-ground (A2G) scenarios. The measurement campaign uses orthogonal frequency division multiplexing signals centered at 1.4 GHz for channel sounding. In the G2G measurement, in addition to using omnidirectional antennas to record data, we also use directional antennas to record the arrival angle information of the signal at the receiver. In the A2G measurement, we pre-plan the flight trajectory of the unmanned aerial vehicle so that it can fly at a fixed angle relative to the ground. We present path loss models suitable for G2G and A2G in forest environments based on the analysis of measurement results. The results indicate that the proposed model reduces error margins compared with other path loss models. Furthermore, we derive the multipath model expression specific to forest environments and conduct statistical analysis on key channel parameters e.g., shadow fading factor, root mean square delay spread, and Rician K factor. Our findings reveal that signal propagation obstruction due to tree crowns in A2G communication is more pronounced than tree trunk obstructions in G2G communication. Adjusting the elevation angle between air and ground can enhance communication quality.

Subject: Information Theory

Publish: 2025-07-03 04:15:13 UTC


#4 Resolution Limits of Non-Adaptive 20 Questions Estimation for Tracking Multiple Moving Targets [PDF] [Copy] [Kimi] [REL]

Authors: Chunsong Sun, Lin Zhou, Jingjing Wang, Weijie Yuan, Chunxiao Jiang, Alfred Hero

Motivated by the practical application of beam tracking of multiple devices in Multiple Input Multiple Output (MIMO) communication, we study the problem of non-adaptive twenty questions estimation for locating and tracking multiple moving targets under a query-dependent noisy channel. Specifically, we derive a non-asymptotic bound and a second-order asymptotic bound on resolution for optimal query procedures and provide numerical examples to illustrate our results. In particular, we demonstrate that the bound is achieved by a state estimator that thresholds the mutual information density over possible target locations. This single threshold decoding rule has reduced the computational complexity compared to the multiple threshold scheme proposed for locating multiple stationary targets (Zhou, Bai and Hero, TIT 2022). We discuss two special cases of our setting: the case with unknown initial location and known velocity, and the case with known initial location and unknown velocity. Both cases share the same theoretical benchmark {that applies to} stationary multiple target search in Zhou, Bai and Hero (TIT 2022) while the known initial location case is close to the theoretical benchmark for stationary target search when the maximal speed is inversely proportional to the number of queries. We also generalize our results to account for a piecewise constant velocity model introduced in Zhou and Hero (TIT 2023), where targets change velocity periodically. Finally, we illustrate our proposed algorithm for the application of beam tracking of multiple mobile transmitters in a 5G wireless network.

Subject: Information Theory

Publish: 2025-07-03 03:30:41 UTC


#5 Matrix Pencil-Based DoA Estimation for Hybrid Receivers in Snapshot-Limited Scenarios [PDF] [Copy] [Kimi] [REL]

Authors: Mona Mostafa, Ramy H. Gohary, Amr El-Keyi, Yahia A. Eldemerdash Ahmed

The goal of this paper is to estimate the directions of arrival (DoAs) for hybrid analog/digital (HAD) receivers when the number of snapshots is too small for statistical averaging to be reliable. This goal is achieved in fully-digital receivers by employing the matrix pencil method (MPM). Unfortunately, the MPM cannot be directly applied in HAD receivers because of the entanglement induced by the underlying analog combiners on the output signals. Furthermore, these analog combiners project the received signal onto a low-dimensional space, jeopardizing the reception of signals arriving from particular DoA ranges. To circumvent these difficulties, we propose two approaches to enable the MPM to extract the DoAs in HAD receivers. The two approaches avoid severe attenuation induced by low-dimensional projection by cycling over an exhaustive set of analog combiners, collectively spanning the entire space. The first approach can be applied to both fully-connected (FC) and partially-connected (PC) HADs and relies on the availability of periodic, potentially unknown, signals to disentangle the output of the HAD receiver. The second approach applies to PC-HADs only, and eliminates contingency on periodic signals by exploiting the underlying block diagonal structure. The superiority of the proposed approaches is demonstrated via numerical simulations and comparisons with the Cramér-Rao lower bound.

Subjects: Information Theory , Signal Processing

Publish: 2025-07-02 20:32:32 UTC


#6 MOTIF: Modular Thinking via Reinforcement Fine-tuning in LLMs [PDF16] [Copy] [Kimi9] [REL]

Authors: Purbesh Mitra, Sennur Ulukus

Recent advancements in the reasoning capabilities of large language models (LLMs) show that employing group relative policy optimization (GRPO) algorithm for reinforcement learning (RL) training allows the models to use more thinking/reasoning tokens for generating better responses. However, LLMs can generate only a finite amount of tokens while maintaining attention to the previously generated tokens. This limit, also known as the context size of an LLM, is a bottleneck in LLM reasoning with arbitrarily large number of tokens. To think beyond the limit of context size, an LLM must employ a modular thinking strategy to reason over multiple rounds. In this work, we propose MOTIF: Modular Thinking via Reinforcement Finetuning -- an RL training method for generating thinking tokens in multiple rounds, effectively allowing the model to think with additional context size. We trained the open-source model Qwen2.5-3B-Instruct on GSM8K dataset via parameter efficient fine-tuning and tested its accuracy on MATH500 and AIME2024 benchmarks. Our experiments show 3.8\% and 3.3\% improvements over vanilla GRPO based training in the respective benchmarks. Furthermore, this improvement was achieved with only 15\% of samples, thus demonstrating sample efficiency of MOTIF. Our code and models are available at https://github.com/purbeshmitra/MOTIF and https://huggingface.co/purbeshmitra/MOTIF, respectively.

Subjects: Computation and Language , Artificial Intelligence , Information Theory , Machine Learning , Systems and Control

Publish: 2025-07-03 17:55:43 UTC


#7 Designs from magic-augmented Clifford circuits [PDF] [Copy] [Kimi] [REL]

Authors: Yuzhen Zhang, Sagar Vijay, Yingfei Gu, Yimu Bao

We introduce magic-augmented Clifford circuits -- architectures in which Clifford circuits are preceded and/or followed by constant-depth circuits of non-Clifford (``magic") gates -- as a resource-efficient way to realize approximate k-designs, with reduced circuit depth and usage of magic. We prove that shallow Clifford circuits, when augmented with constant-depth circuits of magic gates, can generate approximate unitary and state k-designs with \epsilon relative error. The total circuit depth for these constructions on N qubits is O(\log (N/\epsilon)) +2^{O(k\log k)} in one dimension and O(\log\log(N/\epsilon))+2^{O(k\log k)} in all-to-all circuits using ancillas, which improves upon previous results for small k \geq 4. Furthermore, our construction of relative-error state k-designs only involves states with strictly local magic. The required number of magic gates is parametrically reduced when considering k-designs with bounded additive error. As an example, we show that shallow Clifford circuits followed by O(k^2) single-qubit magic gates, independent of system size, can generate an additive-error state k-design. We develop a classical statistical mechanics description of our random circuit architectures, which provides a quantitative understanding of the required depth and number of magic gates for additive-error state k-designs. We also prove no-go theorems for various architectures to generate designs with bounded relative error.

Subjects: Quantum Physics , Statistical Mechanics , Strongly Correlated Electrons , Information Theory , High Energy Physics - Theory

Publish: 2025-07-03 17:41:03 UTC


#8 Classification by Separating Hypersurfaces: An Entropic Approach [PDF] [Copy] [Kimi3] [REL]

Authors: Argimiro Arratia, Mahmoud El Daou, Henryk Gzyl

We consider the following classification problem: Given a population of individuals characterized by a set of attributes represented as a vector in {\mathbb R}^N, the goal is to find a hyperplane in {\mathbb R}^N that separates two sets of points corresponding to two distinct classes. This problem, with a history dating back to the perceptron model, remains central to machine learning. In this paper we propose a novel approach by searching for a vector of parameters in a bounded N-dimensional hypercube centered at the origin and a positive vector in {\mathbb R}^M, obtained through the minimization of an entropy-based function defined over the space of unknown variables. The method extends to polynomial surfaces, allowing the separation of data points by more complex decision boundaries. This provides a robust alternative to traditional linear or quadratic optimization techniques, such as support vector machines and gradient descent. Numerical experiments demonstrate the efficiency and versatility of the method in handling diverse classification tasks, including linear and non-linear separability.

Subjects: Machine Learning , Information Theory , Data Analysis, Statistics and Probability , Machine Learning

Publish: 2025-07-03 15:43:54 UTC


#9 Knowledge Graph-Based Explainable and Generalized Zero-Shot Semantic Communications [PDF] [Copy] [Kimi2] [REL]

Authors: Zhaoyu Zhang, Lingyi Wang, Wei Wu, Fuhui Zhou, Qihui Wu

Data-driven semantic communication is based on superficial statistical patterns, thereby lacking interpretability and generalization, especially for applications with the presence of unseen data. To address these challenges, we propose a novel knowledge graph-enhanced zero-shot semantic communication (KGZS-SC) network. Guided by the structured semantic information from a knowledge graph-based semantic knowledge base (KG-SKB), our scheme provides generalized semantic representations and enables reasoning for unseen cases. Specifically, the KG-SKB aligns the semantic features in a shared category semantics embedding space and enhances the generalization ability of the transmitter through aligned semantic features, thus reducing communication overhead by selectively transmitting compact visual semantics. At the receiver, zero-shot learning (ZSL) is leveraged to enable direct classification for unseen cases without the demand for retraining or additional computational overhead, thereby enhancing the adaptability and efficiency of the classification process in dynamic or resource-constrained environments. The simulation results conducted on the APY datasets show that the proposed KGZS-SC network exhibits robust generalization and significantly outperforms existing SC frameworks in classifying unseen categories across a range of SNR levels.

Subjects: Machine Learning , Artificial Intelligence , Information Theory

Publish: 2025-07-03 03:57:26 UTC


#10 Extended c-differential distinguishers of full 9 and reduced-round Kuznyechik cipher [PDF] [Copy] [Kimi] [REL]

Authors: Pantelimon Stanica, Ranit Dutta, Bimal Mandal

This paper introduces {\em truncated inner c-differential cryptanalysis}, a novel technique that for the first time enables the practical application of c-differential uniformity to block ciphers. While Ellingsen et al. (IEEE Trans. Inf. Theory, 2020) established the notion of c-differential uniformity using (F(x\oplus a), cF(x)), a key challenge remained: multiplication by c disrupts the structural properties essential for block cipher analysis, particularly key addition. We resolve this challenge by developing an \emph{inner} c-differential approach where multiplication by c affects the input: (F(cx\oplus a), F(x)). We prove that the inner c-differential uniformity of a function F equals the outer c-differential uniformity of F^{-1}, establishing a fundamental duality. This modification preserves cipher structure while enabling practical cryptanalytic applications. Our main contribution is a comprehensive multi-faceted statistical-computational framework, implementing truncated c-differential analysis against the full 9-round Kuznyechik cipher (the inner c-differentials are immune to the key whitening at the backend). Through extensive computational analysis involving millions of differential pairs, we demonstrate statistically significant non-randomness across all tested round counts. For the full 9-round cipher, we identify multiple configurations triggering critical security alerts, with bias ratios reaching 1.7\times and corrected p-values as low as 1.85 \times 10^{-3}, suggesting insufficient security margin against this new attack vector. This represents the first practical distinguisher against the full 9-round Kuznyechik.

Subjects: Cryptography and Security , Information Theory

Publish: 2025-07-02 22:27:33 UTC