2026-10-16 | | Total: 90
Tabular foundation models (TFMs) now outperform gradient-boosted trees in classification and regression. Yet in survival analysis, which predicts when an event occurs from censored data, often on small clinical cohorts, we find that no existing TFM beats the half-century-old Cox model. We address this issue by introducing SurvICL, a TFM that predicts survival curves in-context. SurvICL is pretrained on purely synthetic datasets generated from a novel flexible proportional-odds prior that outperforms other prior types in our ablation study. Unlike most other survival models, SurvICL is trained with a plain quantile regression loss, which is possible because its synthetic pre-training data generator can provide uncensored event times. Additionally, we introduce the largest survival benchmark to date spanning 103 datasets. On the 63 datasets held out from all design decisions, SurvICL outperforms 20 tree-based, deep, and foundation-model baselines without any tuning: it wins 82.9\% of pairwise comparisons in integrated Brier score, against 71.3\% for the runner-up, the Cox model, and is the first model to rank above Cox on small datasets.
The Oracle Approximating Shrinkage (OAS) estimator provides a closed-form, data-driven shrinkage intensity for regularizing sample covariance matrices toward a scaled identity target, but its derivation assumes Gaussian data. In practice, financial returns, neural signals, and many other high-dimensional datasets exhibit heavy tails and non-zero kurtosis, violating this assumption. We extend OAS to any distribution with finite fourth moments by incorporating a correction term $α$ built from the fourth-order joint cumulants of the data. The correction vanishes under Gaussianity, recovering the OAS formula as a special case; it admits a consistent $O(np)$ sample estimator and satisfies $ρ\in [0, 1]$ exactly for both the population cumulant and its sample analogue. Simulations and an equity-returns study confirm that the correction closes the gap between OAS and the Ledoit-Wolf estimator under non-Gaussian data, with OAS-Extended performing comparably across all tested settings. The contribution is a principled correction for OAS practitioners facing heavy-tailed data; structurally, it is distinct from prior work extending Ledoit-Wolf to elliptical distributions, which optimizes against a fixed oracle target and yields a different closed-form formula.
We introduce a class of marginal Structural Nested Mean Models (mSNMMs) that enable modeling time-varying effect modification by a subset of covariates required as adjustment variables. We focus on identifiability and estimation under conditional parallel trends assumptions, so mSNMMs are applicable in settings where time-varying Difference-in-Differences (DiD) methods are typically applied. Modeling effect modification by lower dimensional covariates reduces the risk of model misspecification and can improve interpretability relative to previously developed `full SNMMs'. mSNMMs enjoy two key advantages over alternative time-varying DiD methods. First, the parallel trends assumptions that they require are more plausible in the presence of time-varying confounding. Second, they are able to model effects of \textit{sustained} interventions even when treatment switches on and off in the data. We illustrate the methodology by applying mSNMMs to estimate how the effects of Medicaid expansion (a time-varying treatment) on local uninsurance rates vary with unemployment rate prior to expansion (a time-varying covariate), adjusting for additional time-varying demographic and economic variables.
Neural networks offer critical adaptability for controlling complex systems where nominal modeling proves intractable. However, when offline-trained neural networks are deployed within closed-loop control systems, their actions fundamentally alter the state distribution upon which they were trained. This paper introduces a unified taxonomy to categorize these sequential failure modes: the initial Closed-Loop Identification Error, the deployment-induced Action Error, and the continuous Tracking Error. Through discrete-time sensitivity analysis, this work formalizes how offline-learned parameters locally guarantee a trajectory divergence upon deployment, degrading performance due to local loss geometry. To mitigate this distribution shift, we propose actionable design policies centered on time-scale separated online adaptation. These mechanisms are validated through ablation studies on a nonlinear Duffing oscillator, demonstrating that bounded continuous learning successfully mitigates trajectory divergence whereas static deployments fail, offering concrete guidelines for practitioners deploying machine learning components in safety-critical feedback systems.
In the best-arm identification problem, an algorithm adaptively selects arms to find the arm of the largest mean reward. Successive Rejects (SR) is a simple fixed-budget best-arm identification algorithm that eliminates one candidate arm at each phase. At first glance, SR seems to have limited adaptivity, since it follows predetermined phase lengths. However, for Gaussian arms with common variance, SR's worst-instance error exponent normalized by the popular complexity measure $H_1(P)$ is optimal up to a constant factor, which partly explains why SR's performance is competitive compared with state-of-the-art adaptive sampling algorithms. In this paper, we extend the framework of SR by optimizing its predefined schedule of phase lengths. We formulate phase-schedule design as an optimization problem for the normalized error exponent. Then, we introduce a version of SR with the optimized schedule, which we call $H_1$-balanced Successive Rejects ($H_1$-SR). This schedule is optimal within the class of SR-type algorithms with deterministic equal-count schedules. For Gaussian arms with common variance, $H_1$-SR achieves at least $11/12$ of the optimal normalized rate over all adaptive algorithms for every $K\ge2$, and this approximation ratio converges to one as $1-O(1/\log K)$ as $K\to\infty$. This is a significant improvement over original SR, whose corresponding fraction of the optimal normalized rate converges to $1/4$. Finally, experiments on ten synthetic instances and two dataset-derived Gaussian benchmarks compare the optimized schedule with original SR at finite horizons. They show substantial gains on many instances while also exposing a regime in which more aggressive early elimination is costly.
Standard Bayesian experimental design (BED) methods assume the experimenter alone will act on the data, choosing the best action given their own beliefs and goals. In practice, data are often acted on by external parties whose procedures the experimenter cannot change and may not know, particularly when the data are released publicly. To extend BED to such settings, we introduce usage-aware experimental design, a framework in which an adaptive design policy is trained to minimise the usage risk: the experimenter's expected loss when one or more external rules act on the collected data. These rules may be non-Bayesian or black-box, and when several may apply, we either average over them or take the worst case. We show that, unlike in standard BED, more informative data can be harmful under such rules, so the usage risk cannot in general be expressed as an expected uncertainty reduction; our framework thus strictly generalises standard BED. We then use it to tackle a range of problems that are difficult to address with existing approaches.
Surface urban heat islands (SUHIs) are commonly studied using satellite-derived land surface temperature (LST), but many analyses rely on static summaries or pointwise urban-rural contrasts, providing limited inference on temporal change, spatial dependence, and uncertainty. This study models LST as a stochastic spatio-temporal process using a spatially varying coefficient formulation that decomposes LST into a baseline spatial field and a location-specific temporal trend. This structure allows persistent thermal differences and local warming rates to be estimated within a unified probabilistic framework. SUHI is derived from the posterior distribution of the baseline spatial field by contrasting the estimated LST at a given pixel with the minimum estimated LST among non-urban pixels. Posterior excursion probabilities are used to identify areas where temporal changes are supported under simultaneous probability control. The framework is applied to summer Landsat 8 LST observations for Bologna, Italy, from 2013 to 2025. Results show strong spatial heterogeneity in SUHI, with higher values over impervious surfaces and lower values over forests and water bodies. Summer LST increased overall, but local trends varied markedly across the municipality. Joint excursion sets identify areas with warming greater than the domain-wide average and smaller clusters with statistically supported cooling. These results reveal changing thermal patterns across Bologna and help distinguish areas experiencing increasing surface heat from locations where thermal conditions have improved over time. The framework therefore provides an uncertainty-aware basis for monitoring urban thermal change.
Tabular foundation models based on prior-data fitted networks (PFNs) predict on a new data set in a single forward pass, but they cannot report which inputs their predictions depend on. We propose SelectPFN, a PFN that performs Bayesian variable selection and prediction jointly for sparse nonlinear regression. In one forward pass it returns posterior inclusion probabilities for every variable, and its predictions are computed from sampled subsets only, so excluded variables provably cannot affect them; we show this is exactly what the Bayesian model prescribes. The network is pretrained once on synthetic tasks from a meta-prior built around a group spike-and-slab sparse-input neural network. Column-permutation equivariance combined with random interleaving of original and knockoff columns yields knockoff statistics with exact finite-sample false discovery rate control, even when the meta-prior is wrong. We develop theory for amortized selection: a Bayes-regret bound for the amortized median probability model, a characterization showing that the information needed to detect a variable grows like $(2/λ)\log(1/ω)$ in the prior weight $ω$ of structures that can use it, where $λ\le1$ is an efficiency, and a negative result showing that average-case training does not transfer to frequentist guarantees at a fixed truth. On out-of-prior nonlinear designs with p up to 500, SelectPFN improves the area under the precision-recall curve by 0.23-0.28 over the lasso, BART, DART, LassoNet and a group-penalized sparse neural network, at a false discovery proportion of about 0.07 and 0.04 seconds per data set. The advantage persists on real covariate matrices, and, as the theory predicts, detection of interaction-only variables rises monotonically with their prior weight.
Compound extreme events cause damage out of proportion to their frequency, and grouping random vectors by their joint extremal behaviour is a recurring task in climate science, hydrology, and finance. Classical clustering is driven by typical observations, while existing extremal procedures either summarize dependence by a single coefficient or discretize the simplex in an ad hoc way. We propose a dissimilarity that compares complete angular measures by optimal transport: each angular measure is estimated above a high radial threshold, mapped from the simplex into a Euclidean space of Aitchison coordinates, and compared by the 1-Wasserstein distance. The methodology is fully nonparametric and requires no fitted model. The dissimilarities feed any distance-based clustering algorithm. Under intermediate-threshold asymptotics and mild moment and separation conditions, hierarchical clustering and partitioning around medoids based on them recovers the latent partition with probability tending to one. In simulations, the proposed approach matches established competitors when clusters differ mainly in the strength of extremal dependence and performs substantially better when equal-strength clusters differ in the shape of their angular measures, especially when asymmetry drives the separation. Applications to rain--snowmelt--runoff extremes in Swiss catchments and to extremal co-movement between international equity markets and the US market reveal hydrologically and economically interpretable groups.
Statistical modeling of high-dimensional and large-sample data with complex correlation structures is increasingly important in biomedical research, particularly in large-scale omics applications such as genomic studies. Analyzing these large-scale datasets imposes substantial computational and memory demands, requiring efficient model training strategies that maintain predictive accuracy. We propose Prescreen, Then Subsample (PTS), a two-stage approach that combines variable selection with optimal-design-based informative subsampling to address both high dimensionality and large sample size. The first stage screens informative variables and can explicitly account for covariate correlation; the second stage selects informative observations in the prescreened variable space. In contrast to existing two-stage subdata selection approaches, PTS can incorporate multivariable fine-mapping to identify a correlation-aware covariate space for informative subsampling. In particular, we integrate the Sum of Single Effects (SuSiE) model for fine-mapping-based prescreening with information-based optimal subdata selection (IBOSS) for sample selection. Simulation studies based on correlated Gaussian covariates and biobank-scale synthetic genotype data show that PTS reduces model training time while achieving predictive performance comparable to full-data training. We find that SuSiE-based PTS provides improved predictive performance consistent with better stage-one screening precision compared with commonly used univariate prescreening methods. PTS therefore provides a flexible and scalable training framework for large-scale biomedical data with complex correlation structures, including genomic data with linkage disequilibrium.
We study the possibility of generalizing maximum-entropy reconstruction, which approximately recovers a probability distribution from finitely many (generalized) moments by minimizing the Kullback--Leibler (KL) divergence to a reference measure. The reconstruction lies in an exponential family and is also the best KL approximation of the unknown distribution within that family: moment matching and information projection coincide. Replacing KL by a smooth f-divergence, the reconstruction still exists in an associated family, but the best approximation property does not. Specifically, a family is called entropy-adapted if the projection of any distribution matches its moments. We show that, up to equivalence, KL is the only $C^2$ f-divergence that is entropy-adapted, whatever the choice of the generalized moments. We illustrate the consequences of this failure numerically.
Covariance-calibrated prediction-powered inference (CC-PPI) uses unlabeled prediction scores to calibrate the covariance weight in a GMM estimator. The raw target preserves the prediction-corrected channel and replaces the labeled prediction-covariance contribution in the auxiliary residual metric with its unlabeled counterpart. Spectral protection ensures positive definiteness, and the estimator uses an equal blend of the original covariance and the protected target. We establish first-order efficiency and derive exact finite-sample risk identities for linear regression, including a design-based improvement theorem with tolerance to prediction error. A quadratic decision model gives a midpoint minimax-regret interpretation of the half-strength default. Exact strength-path formulas and convergent risk approximations bound its distance from the best fixed strength on that path. Simulations and four real-data studies show substantial reductions in parameter MSE. Independent fixed-strength references yield only small further gains on the calibration paths studied.
Batting average factors exactly as BA = c [h + (1-h) b], where c is the rate of avoiding a strikeout, h is the home-run rate among non-strikeout at-bats, and b is batting average on balls in play (BABIP). The bracketed term is the rate f at which non-strikeout at-bats become hits, so the composition nests the two-factor identity BA = c f. Using the 256 major-league hitters with at least 300 at-bats in 2025, every season from 2004 to 2025, and 20,357 qualifying seasons from 1901 to 2019, we ask which rates behave like skills and how the corresponding abilities are related. Over 19 consecutive-season pairs the median year-to-year correlation is 0.86 for c, 0.74 for h and 0.41 for b; batting average (0.44) is no more repeatable than BABIP. A trivariate logistic-normal random-effects model gives single-season reliabilities of 0.91, 0.76 and 0.30, and a talent correlation between contact and power of -0.67 (95% profile interval -0.75 to -0.57), while BABIP talent is only weakly related to either. Historically, the negative covariance between c and f, which was near zero in the dead-ball era, explains why batting averages have become less dispersed, and so why no one has hit .400 since 1941. That covariance runs almost entirely through home runs: removing the contact--power channel from the 2010s covariance recovers 72% of the gap between modern and dead-ball dispersion in batting-average talent.
We develop a new class of priors for Bayesian small area models under strict external bench- marking constraints. The prior is fully supported on the constrained parameter set defined by the benchmarking constraints and thereby provides a proper constrained posterior that is fully supported on the restricted set. The posterior naturally constrains the small area estimates to conform to the benchmarking values at the aggregate level. In addition, it facilitates valid uncertainty quantification using credible sets that are fully contained within the constrained set. This novel framework has a wide range of applicability, ranging from the standard Gaus- sian Fay-Herriot model to more general Fay-Herriot models, such as robust Fay-Herriot models, where the area-level random effects are distributed as multivariate t, or Fay-Herriot models where there is a complex nonlinear relationship between the area means and the covariates. We illustrate the usefulness of the proposed methodology using simulation experiments as well as by estimating the state-level proportion of children aged 5-17 living under poverty based on the direct estimates obtained from the Small Area Income and Poverty Estimates (SAIPE) program of the U.S. Census Bureau and the national benchmark of the child poverty rate obtained from the Annual Social and Economic Supplement of the Current Population Survey (CPS ASEC) conducted by the U.S. Census Bureau. The numerical results reaffirm the advantages of the proposed benchmarking priors.
Robust covariance estimation under entry-wise contamination and missing data is challenging in various signal processing applications. Under the Student-$t$ model, observation-level reweighting induces a uniform attenuation across samples, which can lead to cumulative information loss even when most coordinates remain informative. The alternative Student-$t^*$ model enables coordinate-wise robustness, but its E-step is not available in closed form. We extend inference for the Student-$t^*$ model to incomplete observations through a stochastic approximation EM (SAEM) algorithm operating on a reduced latent space: stochastic updates of the coordinate-wise scales are combined with Rao-Blackwellized Gaussian conditional moments, which avoids explicit sampling of latent Gaussian variables while preserving a likelihood-based framework. Experiments on synthetic and real sensor data show a lower Fisher-Rao error than Gaussian, Student-$t$, imputation-based and detect-then-estimate estimators at moderate to high outlier rates under diffuse contamination.
Historical survival data can improve prediction in a cohort with few events, but prognostic differences can make borrowing harmful. To borrow while adapting, we propose calibrated transfer learning for the Cox model, in which a historical risk function enters the target analysis as a fixed offset with a prespecified correction for changing covariate effects and a separate baseline hazard. Calibration removes historical error in the correction space exactly, so only the remaining error governs the prediction rate, for any bounded learner. Accurate prediction does not, however, ensure valid treatment inference. In randomized trials, the remaining error biases the conditional treatment coefficient, because a treatment effect changes the prognostic composition of later risk sets. We characterize this bias and its sharp worst-case direction, and we introduce a cross-fitted projected score that removes it at first order and leaves only a second-order bias when a fixed error lies in its dictionary. A historical learning bound separates approximation, sampling, and optimization error. In simulations, calibration improves prediction when the historical risk is well learned, and the projected score restores interval coverage under drift outside the correction, also at a hazard ratio of 0.7. In a retrospective transfer between the POPLAR and OAK lung cancer trials, selected corrections reduce the integrated Brier score by 4.3\% relative to ridge-penalized clinical Cox with 100 training patients, without improving treatment precision.
The rapid rise of generative artificial intelligence (AI) following the public release of tools such as ChatGPT has begun to reshape how students engage with coding, statistical analysis, and written communication in higher education. Growing use of conversational AI systems creates opportunities for learning alongside challenges around reliability, transparency, privacy, and appropriate academic and professional practice, highlighting the need for structured education in their responsible use. This paper presents a framework for teaching the responsible and effective use of generative AI in statistics and data science. The framework is organised around four areas: Understanding, Development, Reporting, and Safety, covering conceptual foundations, practical interaction within analytical workflows, documentation and transparency, and issues including reliability, bias, privacy, and ethics. Its design draws on emerging work in AI literacy and established principles from statistics and data science education, and was further refined through workshop and conference activities and participant feedback. The framework has been implemented across multiple settings, including a massive open online course, short training sessions, and undergraduate and postgraduate skills-based teaching. We use these experiences to identify broader design considerations for educators, including the value of authentic analytical tasks, critical evaluation of generated outputs, integrated consideration of responsible practice, and flexibility as generative AI technologies continue to evolve.
Flow matching has emerged as a powerful generative family for simulation-based inference (SBI). However, sampling and density evaluation require numerical integration at inference time. This makes posterior sampling in neural posterior estimation costly. More importantly, it can make repeated likelihood evaluation prohibitively expensive and impractical for Markov chain Monte Carlo (MCMC) sampling, which may require thousands of repeated density evaluations. Here, we develop conditional mean flows for posterior and likelihood approximation, combining few-step sampling with tractable density evaluation. We show that the mean flow objective identifies conditional flow maps and that independent masking enables direct estimation of marginal likelihoods. We evaluate both posterior and likelihood estimation across several tasks and show that conditional mean flows retain the flexibility of flow-based density estimators while significantly reducing inference time. In particular, single-step conditional mean flow density evaluation yields estimation error comparable to 200-step flow matching while enabling about 200-fold faster downstream MCMC sampling.
Reliable uncertainty quantification is central to modern forecasting. Conformal prediction offers model-agnostic intervals with coverage guarantee under exchangeability, but existing software often targets general prediction problems or individual time series procedures rather than integrated workflows for multistep forecasting applications. We introduce conformalForecast, an open-source R package for constructing, updating, and evaluating conformal prediction intervals for multistep time series forecasts. The package separates point forecasting from calibration, accepts forecasts from diverse sources through a common interface, and preserves horizon-specific forecast errors and calibration states. It implements several conformal calibration schemes spanning empirical-quantile, feedback-based, and dependence-aware approaches, together with tools for assessing coverage, interval width, and interval scores over time and across horizons. Fitted objects can be extended as new observations arrive while retaining previously issued intervals and avoiding recomputation at earlier forecast origins. An application to short-term electricity price forecasting across five Australian regions illustrates the workflow. Aggregate coverage is generally close to the nominal level, while rolling coverage and interval efficiency vary across methods and horizons. In the reported experiment, incremental updating reproduces full-reconstruction intervals under identical forecasting and calibration settings while substantially reducing computation. conformalForecast provides a reproducible and extensible framework linking conformal methodology with sequential forecasting practice.
We develop a perturbation analysis of Feynman--Kac models, describing how local errors introduced at each step of the Feynman--Kac recursion propagate over time. Under general conditions on the potential functions and the Markov transitions, which are shown to hold for a large class of state-space models, we prove that the global error for any time horizon $T\geq 1$ is bounded by $C\,Δ_T\log(1+Δ_T^{-1})$, where $Δ_T$ is the largest local error; this bound holds both for the prediction and the updated models. These conditions require no Lyapunov function and no stability of the Markov transitions, and allow for non-compact state spaces: confinement is provided by the potential functions. Our perturbation bounds hold in total variation and in the Kolmogorov norm; for the latter, we extend the Hardy--Krause variation and the Koksma--Hlawka inequality to unbounded domains. Our perturbation theory is applied to derive time-uniform bounds for linearized Kalman filters, Euler schemes for partially observed diffusions and nudged state-space models, as well as high-probability error bounds, uniform over whole trajectories, for particle filters and, more generally, for mean field particle methods such as sequential Monte Carlo and diffusion Monte Carlo algorithms.
We unify several geometries of the probability simplex under one principle, their generation by a potential: convex potentials produce the Bregman divergences and dually flat structures of information geometry, and the exponentially concave potentials of Pal and Wong produce the logarithmic divergences and their projectively flat structures. Each structure carries a duality between maximum entropy and minimum divergence and, with it, a projection principle for estimation: the U-estimators of Eguchi, Komori and Ohara in the first case, the λ-deformed estimators of Wong and Zhang in the second, both governed by a Pythagorean relation and reducing to maximum likelihood in the undeformed limit. We complete the picture with estimators drawn from the third connection the divergence defines, the Levi-Civita one, by minimizing geodesic distance: existence, uniqueness and influence are then controlled by curvature and diameter rather than by flatness. Along the way the Aitchison geometry appears as the logarithmic-barrier instance of the construction at its critical weight, and the symmetrization of the Pal-Wong transport cost, a smoothing of the Hilbert projective metric, is shown to lose the dual pair and to support instead a Huber-type robust center with bounded influence in a geometry of infinite diameter.
Muon replaces a matrix gradient by an approximate polar factor and has shown strong empirical performance in neural-network training. The Isotropic Curvature Model (ICM) provides a curvature-based explanation for this update by asking when gradient orthogonalization minimizes a one-step local-loss surrogate. We investigate the stability of this explanation when the fixed input radius of the ICM is replaced by a general radial law. We introduce the Random-Radius Isotropic Curvature Model (RR-ICM) and the Polar-Factor Admissibility Set, which encodes exact scaled-polar optimality as a geometric condition on the positive polar factor of the gradient. We derive an exact characterization of this set for arbitrary radial laws and show that all nonscalar admissible responses arise from radial atoms aligned with curvature kinks. Consequently, every atomless radial law exhibits zero-capacity rigidity: exact scaled-polar optimality is possible only when the gradient is already a scaled column isometry. To connect the curvature-based framework with the established spectral geometry of Muon, we introduce the Minimax RR-ICM by replacing directional averaging with worst-case optimization over probability laws on the sphere. We prove that every nonzero global minimizer lies on the polar ray of the gradient, while its scale is determined by a one-dimensional convex problem. The corresponding admissibility geometry is a positive-semidefinite trace slab, in sharp contrast to the spherical-zonoid geometry of the RR-ICM. Together, these results expose the radial fragility of average-curvature explanations and provide a new minimax variational characterization of Muon's polar direction.
**Purpose:** Relative changes in event rates are a common primary endpoint in confirmatory asthma trials. Reliable assumptions about baseline event rates and extra-Poisson variability are essential for sample size planning. We synthesize placebo-group exacerbation data, evaluate calendar-time trends, and assess implications for trial design. **Methods:** We systematically reviewed randomized controlled trials reporting recurrent asthma exacerbations and extracted control-group outcomes classified by placebo type or asthma severity. Hierarchical negative binomial models accommodated heterogeneous reporting formats and jointly modeled event rates and overdispersion. We compared separate subgroup models with covariate-based models and used meta-regression to adjust for calendar-time trends. Trial design implications were illustrated using the NAVIGATOR trial. **Results:** Sixty-two trials met the inclusion criteria. The estimated relative change in annualized exacerbation rates was -1.8% per year (95% CrI: -7.5%, 4.3%), corresponding to a 16.5% decrease per decade (95% CrI: -54.2%, 52.2%). Substantial between-study heterogeneity in event rates and overdispersion was observed. Within-trial overdispersion was estimated as 1.53 (95% CrI: 0.60, 2.83). In an illustrative phase III design scenario, sample size requirements were sensitive to the target subgroup and uncertainty in the predicted event rate and overdispersion. **Conclusion:** Event rates in clinical trials exhibit considerable heterogeneity and potential temporal variation, and overdispersion is a critical design parameter. Comprehensive modelling of recurrent-event endpoints allows for evidence-based and robust sample size calculations and trial design.
Test-time scaling can improve the performance of large language models by generating multiple answers to a question and aggregating them. Weighted ensembles extend this idea by drawing answers from several models, with the weights determining how often each model is used. Learning these weights from a training dataset involves two sample sizes: $Q$ labeled questions and $N$ generations per model and question. We study their interplay through the clean risk, the population error of the ensemble's infinite-generation voting rule. Under a margin condition with exponent $β$, we show that empirical risk minimization (ERM) with a calibrated margin buffer achieves excess risk of order $Q^{-1/2}+N^{-β/2}$, up to logarithmic factors, for fixed numbers of models and candidate answers. To investigate the optimality of this dependence on $Q$ and $N$, we analyze a simplified submodel with two models and two possible answers per question. We characterize its minimax excess risk up to constants by a Hellinger modulus for a binomial-mixture experiment. Polynomial approximation and spectral analysis reveal three regimes. When questions are relatively scarce, the $Q^{-1/2}$ term governs learning and the general buffered ERM is optimal. In an intermediate regime, pooling generation counts across questions improves upon estimating each answer distribution separately, with only logarithmic gains from additional questions. With sufficiently many questions, the minimax excess risk reaches an $N^{-β}$ identification floor that persists even with infinitely many questions. The submodel's lower bounds also apply to general ensembles, matching the buffered-ERM upper bound up to logarithmic factors when questions are scarce and in parts of the intermediate regime. Questions remain about the transitions between regimes and the construction of matching pooled estimators for general ensembles.
Selecting covariates while controlling the false discovery rate (FDR) is critical for interpretable scientific discovery, especially with complex machine learning models. Traditional variable selection methods often rely on specific model assumptions, while Model-X knockoffs provide a general way that leverages knowledge of the covariate distribution to produce a variable selection algorithm. This study puts forward a complementary method to Model-X knockoffs, named Model-Agnostic Selection with agent-X importance (MAS), which utilizes the sample-splitting strategy and a novel variable importance measure to bypass distributional assumptions on covariates. The proposed model-agnostic agent-X importance is defined to evaluate the difference of oracle predictiveness on both the original distribution and the distribution with agent variable in population level, and can be efficiently estimated by constructing the agent variable as some constant and cross-fitting strategy. The MAS method integrates a resplit-per-covariate strategy to decouple estimator dependencies and a data aggregation technique to enhance power. Theoretical guarantees for asymptotic FDR control are established under mild conditions. Extensive simulations and real-data analyses demonstrate that MAS provides competitive performances with existing works in FDR control, computational efficiency and power.