2024-10-29 | | Total: 23
We propose a general transfer learning framework for clustering given a main dataset and an auxiliary one about the same subjects. The two datasets may reflect similar but different latent grouping structures of the subjects. We propose an adaptive transfer clustering (ATC) algorithm that automatically leverages the commonality in the presence of unknown discrepancy, by optimizing an estimated bias-variance decomposition. It applies to a broad class of statistical models including Gaussian mixture models, stochastic block models, and latent class models. A theoretical analysis proves the optimality of ATC under the Gaussian mixture model and explicitly quantifies the benefit of transfer. Extensive simulations and real data experiments confirm our method's effectiveness in various scenarios.
Marine Protected Areas (MPAs) have been established globally to conserve marine resources. Given their maintenance costs and impact on commercial fishing, it is critical to evaluate their effectiveness to support future conservation. In this paper, we use data collected from the Australian coast to estimate the effect of MPAs on biodiversity. Environmental studies such as these are often observational, and processes of interest exhibit spatial dependence, which presents challenges in estimating the causal effects. Spatial data can also be subject to preferential sampling, where the sampling locations are related to the response variable, further complicating inference and prediction. To address these challenges, we propose a spatial causal inference method that simultaneously accounts for unmeasured spatial confounders in both the sampling process and the treatment allocation. We prove the identifiability of key parameters in the model and the consistency of the posterior distributions of those parameters. We show via simulation studies that the causal effect of interest can be reliably estimated under the proposed model. The proposed method is applied to assess the effect of MPAs on fish biomass. We find evidence of preferential sampling and that properly accounting for this source of bias impacts the estimate of the causal effect.
Covariance matrix estimation is an important problem in multivariate data analysis, both from theoretical as well as applied points of view. Many simple and popular covariance matrix estimators are known to be severely affected by model misspecification and the presence of outliers in the data; on the other hand robust estimators with reasonably high efficiency are often computationally challenging for modern large and complex datasets. In this work, we propose a new, simple, robust and highly efficient method for estimation of the location vector and the scatter matrix for elliptically symmetric distributions. The proposed estimation procedure is designed in the spirit of the minimum density power divergence (DPD) estimation approach with appropriate modifications which makes our proposal (sequential minimum DPD estimation) computationally very economical and scalable to large as well as higher dimensional datasets. Consistency and asymptotic normality of the proposed sequential estimators of the multivariate location and scatter are established along with asymptotic positive definiteness of the estimated scatter matrix. Robustness of our estimators are studied by means of influence functions. All theoretical results are illustrated further under multivariate normality. A large-scale simulation study is presented to assess finite sample performances and scalability of our method in comparison to the usual maximum likelihood estimator (MLE), the ordinary minimum DPD estimator (MDPDE) and other popular non-parametric methods. The applicability of our method is further illustrated with a real dataset on credit card transactions.
Cascade models serve as effective tools for understanding the propagation of information and diseases within social networks. Nevertheless, their applicability becomes constrained when the states of the agents (nodes) are hidden and can only be inferred through indirect observations or symptoms. This study proposes a Mapper-based strategy to infer the status of agents within a hidden information cascade model using expert knowledge. To verify and demonstrate the method we identify agents who are likely to take advantage of information obtained from an inside information network. We do this using data on insider networks and stock market transactions. Recognizing the sensitive nature of allegations of insider trading, we design a conservative approach to minimize false positives, ensuring that innocent agents are not wrongfully implicated. The Mapper-based results systematically outperform other methods, such as clustering and unsupervised anomaly detection, on synthetic data. We also apply the method to empirical data and verify the results using a statistical validation method based on persistence homology. Our findings highlight that the proposed Mapper-based technique successfully identifies a subpopulation of opportunistic agents within the information cascades. The adaptability of this method to diverse data types and sizes is demonstrated, with potential for tailoring for specific applications.
When comparing multiple groups in clinical trials, we are not only interested in whether there is a difference between any groups but rather the location. Such research questions lead to testing multiple individual hypotheses. To control the familywise error rate (FWER), we must apply some corrections or introduce tests that control the FWER by design. In the case of time-to-event data, a Bonferroni-corrected log-rank test is commonly used. This approach has two significant drawbacks: (i) it loses power when the proportional hazards assumption is violated [1] and (ii) the correction generally leads to a lower power, especially when the test statistics are not independent [2]. We propose two new tests based on combined weighted log-rank tests. One as a simple multiple contrast test of weighted log-rank tests and one as an extension of the so-called CASANOVA test [3]. The latter was introduced for factorial designs. We propose a new multiple contrast test based on the CASANOVA approach. Our test promises to be more powerful under crossing hazards and eliminates the need for additional p-value correction. We assess the performance of our tests through extensive Monte Carlo simulation studies covering both proportional and non-proportional hazard scenarios. Finally, we apply the new and reference methods to a real-world data example. The new approaches control the FWER and show reasonable power in all scenarios. They outperform the adjusted approaches in some non-proportional settings in terms of power.
We introduce the almost goodness-of-fit test, a procedure to decide if a (parametric) model provides a good representation of the probability distribution generating the observed sample. We consider the approximate model determined by an M-estimator of the parameters as the best representative of the unknown distribution within the parametric class. The objective is the approximate validation of a distribution or an entire parametric family up to a pre-specified threshold value, the margin of error. The methodology also allows quantifying the percentage improvement of the proposed model compared to a non-informative (constant) one. The test statistic is the $\mathrm{L}^p$-distance between the empirical distribution function and the corresponding one of the estimated (parametric) model. The value of the parameter $p$ allows modulating the impact of the tails of the distribution in the validation of the model. By deriving the asymptotic distribution of the test statistic, as well as proving the consistency of its bootstrap approximation, we present an easy-to-implement and flexible method. The performance of the proposal is illustrated with a simulation study and the analysis of a real dataset.
In the literature on stochastic frontier models until the early 2000s, the joint consideration of spatial and temporal dimensions was often inadequately addressed, if not completely neglected. However, from an evolutionary economics perspective, the production process of the decision-making units constantly changes over both dimensions: it is not stable over time due to managerial enhancements and/or internal or external shocks, and is influenced by the nearest territorial neighbours. This paper proposes an extension of the Fusco and Vidoli [2013] SEM-like approach, which globally accounts for spatial and temporal effects in the term of inefficiency. In particular, coherently with the stochastic panel frontier literature, two different versions of the model are proposed: the time-invariant and the time-varying spatial stochastic frontier models. In order to evaluate the inferential properties of the proposed estimators, we first run Monte Carlo experiments and we then present the results of an application to a set of commonly referenced data, demonstrating robustness and stability of estimates across all scenarios.
The standard paradigm for confirmatory clinical trials is to compare experimental treatments with a control, for example the standard of care or a placebo. However, it is not always the case that a suitable control exists. Efficient statistical methodology is well studied in the setting of randomised controlled trials. This is not the case if one wishes to compare several experimental with no control arm. We propose hypothesis testing methods suitable for use in such a setting. These methods are efficient, ensuring the error rate is controlled at exactly the desired rate with no conservatism. This in turn yields an improvement in power when compared with standard methods one might otherwise consider using, such as a Bonferroni adjustment. The proposed testing procedure is also highly flexible. We show how it may be extended for use in multi-stage adaptive trials, covering the majority of scenarios in which one might consider the use of such procedures in the clinical trials setting. With such a highly flexible nature, these methods may also be applied more broadly outside of a clinical trials setting.
Large-scale datasets with count outcome variables are widely present in various applications, and the Poisson regression model is among the most popular models for handling count outcomes. This paper considers the high-dimensional sparse Poisson regression model and proposes bias-corrected estimators for both linear and quadratic transformations of high-dimensional regression vectors. We establish the asymptotic normality of the estimators, construct asymptotically valid confidence intervals, and conduct related hypothesis testing. We apply the devised methodology to high-dimensional mediation analysis with count outcome, with particular application of testing for the existence of interaction between the treatment variable and high-dimensional mediators. We demonstrate the proposed methods through extensive simulation studies and application to real-world epigenetic data.
In astronomical observations, the estimation of distances from parallaxes is a challenging task due to the inherent measurement errors and the non-linear relationship between the parallax and the distance. This study leverages ideas from robust Bayesian inference to tackle these challenges, investigating a broad class of prior densities for estimating distances with a reduced bias and variance. Through theoretical analysis, simulation experiments, and the application to data from the Gaia Data Release 1 (GDR1), we demonstrate that heavy-tailed priors provide more reliable distance estimates, particularly in the presence of large fractional parallax errors. Theoretical results highlight the "curse of a single observation," where the likelihood dominates the posterior, limiting the impact of the prior. Nevertheless, heavy-tailed priors can delay the explosion of posterior risk, offering a more robust framework for distance estimation. The findings suggest that reciprocal invariant priors, with polynomial decay in their tails, such as the Half-Cauchy and Product Half-Cauchy, are particularly well-suited for this task, providing a balance between bias reduction and variance control.
We present a general and flexible framework for detecting regime changes in complex, non-stationary data across multi-trial experiments. Traditional change point detection methods focus on identifying abrupt changes within a single time series (single trial), targeting shifts in statistical properties such as the mean, variance, and spectrum over time within that sole trial. In contrast, our approach considers changes occurring across trials, accommodating changes that may arise within individual trials due to experimental inconsistencies, such as varying delays or event duration. By leveraging diverse metrics to analyze time-frequency characteristics specifically topological changes in the spectrum and spectrograms, our approach offers a comprehensive framework for detecting such variations. Our approach can handle different scenarios with various statistical assumptions, including varying levels of stationarity within and across trials, making our framework highly adaptable. We validate our approach through simulations using time-varying autoregressive processes that exhibit different regime changes. Our results demonstrate the effectiveness of detecting changes across trials under diverse conditions. Furthermore, we illustrate the effectiveness of our method by applying it to predictive maintenance using the NASA bearing dataset. By analyzing the time-frequency characteristics of vibration signals recorded by accelerometers, our approach accurately identifies bearing failures, showcasing its strong potential for early fault detection in mechanical systems.
We study the high-dimensional partial linear model, where the linear part has a high-dimensional sparse regression coefficient and the nonparametric part includes a function whose derivatives are of bounded total variation. We expand upon the univariate trend filtering to develop partial linear trend filtering--a doubly penalized least square estimation approach based on $\ell_1$ penalty and total variation penalty. Analogous to the advantages of trend filtering in univariate nonparametric regression, partial linear trend filtering not only can be efficiently computed, but also achieves the optimal error rate for estimating the nonparametric function. This in turn leads to the oracle rate for the linear part as if the underlying nonparametric function were known. We compare the proposed approach with a standard smoothing spline based method, and show both empirically and theoretically that the former outperforms the latter when the underlying function possesses heterogeneous smoothness. We apply our approach to the IDATA study to investigate the relationship between metabolomic profiles and ultra-processed food (UPF) intake, efficiently identifying key metabolites associated with UPF consumption and demonstrating strong predictive performance.
In fields such as sociology, political science, public administration, and business management, particularly in the direction of international relations, Qualitative Comparative Analysis (QCA) has been widely adopted as a research method. This article addresses the limitations of the QCA method in its application, specifically in terms of low coverage, factor limitations, and value limitations. scpQCA enhances the coverage of results and expands the tolerance of the QCA method for multi-factor and multi-valued analyses by maintaining the consistency threshold. To validate these capabilities, we conducted experiments on both random data and specific case datasets, utilizing different approaches of CCM (Configurational Comparative Methods) such as scpQCA, CNA, and QCApro, and presented the different results. In addition, the robustness of scpQCA has been examined from the perspectives of internal and external across different case datasets, thereby demonstrating its extensive applicability and advantages over existing QCA algorithms.
The Frequentist, Assisted by Bayes (FAB) framework aims to construct confidence regions that leverage information about parameter values in the form of a prior distribution. FAB confidence regions (FAB-CRs) have smaller volume for values of the parameter that are likely under the prior, while maintaining exact frequentist coverage. This work introduces several methodological and theoretical contributions to the FAB framework. For Gaussian likelihoods, we show that the posterior mean of the parameter of interest is always contained in the FAB-CR. As such, the posterior mean constitutes a natural notion of FAB estimator to be reported alongside the FAB-CR. More generally, we show that for a likelihood in the natural exponential family, a transformation of the posterior mean of the natural parameter is always contained in the FAB-CR. For Gaussian likelihoods, we show that power law tails conditions on the marginal likelihood induce robust FAB-CRs, that are uniformly bounded and revert to the standard frequentist confidence intervals for extreme observations. We translate this result into practice by proposing a class of shrinkage priors for the FAB framework that satisfy this condition without sacrificing analytical tractability. The resulting FAB estimators are equal to prominent Bayesian shrinkage estimators, including the horseshoe estimator, thereby establishing insightful connections between robust FAB-CRs and Bayesian shrinkage methods.
Industrial applications often exhibit multiple in-control patterns due to varying operating conditions, which makes a single functional linear model (FLM) inadequate to capture the complexity of the true relationship between a functional quality characteristic and covariates, which gives rise to the multimode profile monitoring problem. This issue is clearly illustrated in the resistance spot welding (RSW) process in the automotive industry, where different operating conditions lead to multiple in-control states. In these states, factors such as electrode tip wear and dressing may influence the functional quality characteristic differently, resulting in distinct FLMs across subpopulations. To address this problem, this article introduces the functional mixture regression control chart (FMRCC) to monitor functional quality characteristics with multiple in-control patterns and covariate information, modeled using a mixture of FLMs. A monitoring strategy based on the likelihood ratio test is proposed to monitor any deviation from the estimated in-control heterogeneous population. An extensive Monte Carlo simulation study is performed to compare the FMRCC with competing monitoring schemes that have already appeared in the literature, and a case study in the monitoring of an RSW process in the automotive industry, which motivated this research, illustrates its practical applicability.
Inference for high-dimensional logistic regression models using penalized methods has been a challenging research problem. As an illustration, a major difficulty is the significant bias of the Lasso estimator, which limits its direct application in inference. Although various bias corrected Lasso estimators have been proposed, they often still exhibit substantial biases in finite samples, undermining their inference performance. These finite sample biases become particularly problematic in one-sided inference problems, such as one-sided hypothesis testing. This paper proposes a novel two-step procedure for accurate inference in high-dimensional logistic regression models. In the first step, we propose a Lasso-based variable selection method to select a suitable submodel of moderate size for subsequent inference. In the second step, we introduce a bias corrected estimator to fit the selected submodel. We demonstrate that the resulting estimator from this two-step procedure has a small bias order and enables accurate inference. Numerical studies and an analysis of alcohol consumption data are included, where our proposed method is compared to alternative approaches. Our results indicate that the proposed method exhibits significantly smaller biases than alternative methods in finite samples, thereby leading to improved inference performance.
The analysis of neural power spectra plays a crucial role in understanding brain function and dysfunction. While recent efforts have led to the development of methods for decomposing spectral data, challenges remain in performing statistical analysis and group-level comparisons. Here, we introduce Bayesian Spectral Decomposition (BSD), a Bayesian framework for analysing neural spectral power. BSD allows for the specification, inversion, comparison, and analysis of parametric models of neural spectra, addressing limitations of existing methods. We first establish the face validity of BSD on simulated data and show how it outperforms an established method (\fooof{}) for peak detection on artificial spectral data. We then demonstrate the efficacy of BSD on a group-level study of EEG spectra in 204 healthy subjects from the LEMON dataset. Our results not only highlight the effectiveness of BSD in model selection and parameter estimation, but also illustrate how BSD enables straightforward group-level regression of the effect of continuous covariates such as age. By using Bayesian inference techniques, BSD provides a robust framework for studying neural spectral data and their relationship to brain function and dysfunction.
We provide estimation and inference for the Generalised Dynamic Factor Model (GDFM) under the assumption that the dynamic common component can be expressed in terms of a finite number of lags of contemporaneously pervasive factors. The proposed estimator is simply an OLS regression of the observed variables on factors extracted via static principal components and therefore avoids frequency domain techniques entirely.
We present a new sampling-based approach for enabling efficient computation of low-rank Bayesian matrix completion and quantifying the associated uncertainty. Firstly, we design a new prior model based on the singular-value-decomposition (SVD) parametrization of low-rank matrices. Our prior is analogous to the seminal nuclear-norm regularization used in non-Bayesian setting and enforces orthogonality in the factor matrices by constraining them to Stiefel manifolds. Then, we design a geodesic Hamiltonian Monte Carlo (-within-Gibbs) algorithm for generating posterior samples of the SVD factor matrices. We demonstrate that our approach resolves the sampling difficulties encountered by standard Gibbs samplers for the common two-matrix factorization used in matrix completion. More importantly, the geodesic Hamiltonian sampler allows for sampling in cases with more general likelihoods than the typical Gaussian likelihood and Gaussian prior assumptions adopted in most of the existing Bayesian matrix completion literature. We demonstrate an applications of our approach to fit the categorical data of a mice protein dataset and the MovieLens recommendation problem. Numerical examples demonstrate superior sampling performance, including better mixing and faster convergence to a stationary distribution. Moreover, they demonstrate improved accuracy on the two real-world benchmark problems we considered.
Causal discovery is a fundamental problem with applications spanning various areas in science and engineering. It is well understood that solely using observational data, one can only orient the causal graph up to its Markov equivalence class, necessitating interventional data to learn the complete causal graph. Most works in the literature design causal discovery policies with perfect interventions, i.e., they have access to infinite interventional samples. This study considers a Bayesian approach for learning causal graphs with limited interventional samples, mirroring real-world scenarios where such samples are usually costly to obtain. By leveraging the recent result of Wienöbst et al. (2023) on uniform DAG sampling in polynomial time, we can efficiently enumerate all the cut configurations and their corresponding interventional distributions of a target set, and further track their posteriors. Given any number of interventional samples, our proposed algorithm randomly intervenes on a set of target vertices that cut all the edges in the graph and returns a causal graph according to the posterior of each target set. When the number of interventional samples is large enough, we show theoretically that our proposed algorithm will return the true causal graph with high probability. We compare our algorithm against various baseline methods on simulated datasets, demonstrating its superior accuracy measured by the structural Hamming distance between the learned DAG and the ground truth. Additionally, we present a case study showing how this algorithm could be modified to answer more general causal questions without learning the whole graph. As an example, we illustrate that our method can be used to estimate the causal effect of a variable that cannot be intervened.
Conditional independence and graphical models are crucial concepts for sparsity and statistical modeling in higher dimensions. For Lévy processes, a widely applied class of stochastic processes, these notions have not been studied. By the Lévy-Itô decomposition, a multivariate Lévy process can be decomposed into the sum of a Brownian motion part and an independent jump process. We show that conditional independence statements between the marginal processes can be studied separately for these two parts. While the Brownian part is well-understood, we derive a novel characterization of conditional independence between the sample paths of the jump process in terms of the Lévy measure. We define Lévy graphical models as Lévy processes that satisfy undirected or directed Markov properties. We prove that the graph structure is invariant under changes of the univariate marginal processes. Lévy graphical models allow the construction of flexible, sparse dependence models for Lévy processes in large dimensions, which are interpretable thanks to the underlying graph. For trees, we develop statistical methodology to learn the underlying structure from low- or high-frequency observations of the Lévy process and show consistent graph recovery. We apply our method to model stock returns from U.S. companies to illustrate the advantages of our approach.
Large Language Models (LLMs) have recently shown great promise in planning and reasoning applications. These tasks demand robust systems, which arguably require a causal understanding of the environment. While LLMs can acquire and reflect common sense causal knowledge from their pretraining data, this information is often incomplete, incorrect, or inapplicable to a specific environment. In contrast, causal representation learning (CRL) focuses on identifying the underlying causal structure within a given environment. We propose a framework that integrates CRLs with LLMs to enable causally-aware reasoning and planning. This framework learns a causal world model, with causal variables linked to natural language expressions. This mapping provides LLMs with a flexible interface to process and generate descriptions of actions and states in text form. Effectively, the causal world model acts as a simulator that the LLM can query and interact with. We evaluate the framework on causal inference and planning tasks across temporal scales and environmental complexities. Our experiments demonstrate the effectiveness of the approach, with the causally-aware method outperforming LLM-based reasoners, especially for longer planning horizons.
We propose a scalable kinetic Langevin dynamics algorithm for sampling parameter spaces of big data and AI applications. Our scheme combines a symmetric forward/backward sweep over minibatches with a symmetric discretization of Langevin dynamics. For a particular Langevin splitting method (UBU), we show that the resulting Symmetric Minibatch Splitting-UBU (SMS-UBU) integrator has bias $O(h^2 d^{1/2})$ in dimension $d>0$ with stepsize $h>0$, despite only using one minibatch per iteration, thus providing excellent control of the sampling bias as a function of the stepsize. We apply the algorithm to explore local modes of the posterior distribution of Bayesian neural networks (BNNs) and evaluate the calibration performance of the posterior predictive probabilities for neural networks with convolutional neural network architectures for classification problems on three different datasets (Fashion-MNIST, Celeb-A and chest X-ray). Our results indicate that BNNs sampled with SMS-UBU can offer significantly better calibration performance compared to standard methods of training and stochastic weight averaging.