arXiv subjects

Yanjun Han

Publications and source records attributed to Yanjun Han.

At least 37 records · Page 2Linked to original sources

arXiv2025

Besting Good--Turing: Optimality of Non-Parametric Maximum Likelihood for Distribution Estimation

When faced with a small sample from a large universe of possible outcomes, scientists often turn to the venerable Good--Turing estimator. Despite its pedigree, however, this estimator comes with considerable drawbacks, such as the need to hand-tune smoothing parameters and the lack of a precise optimality guarantee. We introduce a parameter-free estimator that bests Good--Turing in both theory and practice. Our method marries two classic ideas, namely Robbins's empirical Bayes and Kiefer--Wolfowitz non-parametric maximum likelihood estimation (NPMLE), to learn an implicit prior from data and then convert it into probability estimates. We prove that the resulting estimator attains the optimal instance-wise risk up to logarithmic factors in the competitive framework of Orlitsky and Suresh, and that the Good--Turing estimator is strictly suboptimal in the same framework. Our simulations on synthetic data and experiments with English corpora and U.S. Census data show that our estimator consistently outperforms both the Good--Turing estimator and explicit Bayes procedures.

math.ST↗

arXiv2025

Reconfigurable nonlinear optical computing device for retina-inspired computing

Optical neural networks are at the forefront of computational innovation, utilizing photons as the primary carriers of information and employing optical components for computation. However, the fundamental nonlinear optical device in the neural networks is barely satisfied because of its high energy threshold and poor reconfigurability. This paper proposes and demonstrates an optical sigmoid-type nonlinear computation mode of Vertical-Cavity Surface-Emitting Lasers (VCSELs) biased beneath the threshold. The device is programmable by simply adjusting the injection current. The device exhibits sigmoid-type nonlinear performance at a low input optical power ranging from merely 3-250 μW. The tuning sensitivity of the device to the programming current density can be as large as 15 μW*mm2/mA. Deep neural network architecture based on such device has been proposed and demonstrated by simulation on recognizing hand-writing digital dataset, and a 97.3% accuracy has been achieved. A step further, the nonlinear reconfigurability is found to be highly useful to enhance the adaptability of the networks, which is demonstrated by significantly improving the recognition accuracy by 41.76%, 19.2%, and 25.89% of low-contrast hand-writing digital images under high exposure, low exposure, and high random noise respectively.

physics.optics↗

arXiv2024

D-band MUTC Photodiode Module for Ultra-Wideband 160 Gbps Photonics-Assisted Fiber-THz Integrated Communication System

Current wireless communication systems are increasingly constrained by insufficient bandwidth and limited power output, impeding the achievement of ultra-high-speed data transmission. The terahertz (THz) range offers greater bandwidth, but it also imposes higher requirements on broadband and high-power devices. In this work, we present a modified uni-traveling-carrier photodiode (MUTC-PD) module with WR-6 waveguide output for photonics-assisted fiber-THz integrated wireless communications. Through the optimization of the epitaxial structure and high-impedance coplanar waveguide (CPW), the fabricated 6-um-diameter MUTC-PD achieves a high output power of -0.96 dBm at 150 GHz and ultra-flat frequency response at D-band. The MUTC-PD is subsequently packaged into a compact WR-6 module, incorporating planar-circuit-based RF-choke, DC-block and probe. The packaged PD module demonstrates high saturation power and flat frequency responses with minimal power roll-off of only 2 dB over 110-170 GHz. By incorporating the PD module into a fiber-THz integrated communication system, high data rates of up to 160 Gbps with 16 quadrature amplitude modulation (QAM) and a maximum symbol transmission rate of 60 Gbaud with QPSK modulation are successfully secured. The demonstration verifies the potential of the PD module for ultra-broadband and ultra-high-speed THz communications, setting a foundation for future research in high-speed data transmission.

physics.optics↗

arXiv2024

Assouad, Fano, and Le Cam with Interaction: A Unifying Lower Bound Framework and Characterization for Bandit Learnability

We develop a unifying framework for information-theoretic lower bound in statistical estimation and interactive decision making. Classical lower bound techniques -- such as Fano's method, Le Cam's method, and Assouad's lemma -- are central to the study of minimax risk in statistical estimation, yet are insufficient to provide tight lower bounds for \emph{interactive decision making} algorithms that collect data interactively (e.g., algorithms for bandits and reinforcement learning). Recent work of Foster et al. (2021, 2023) provides minimax lower bounds for interactive decision making using seemingly different analysis techniques from the classical methods. These results -- which are proven using a complexity measure known as the \emph{Decision-Estimation Coefficient} (DEC) -- capture difficulties unique to interactive learning, yet do not recover the tightest known lower bounds for passive estimation. We propose a unified view of these distinct methodologies through a new lower bound approach called \emph{interactive Fano method}. As an application, we introduce a novel complexity measure, the \emph{Fractional Covering Number}, which facilitates the new lower bounds for interactive decision making that extend the DEC methodology by incorporating the complexity of estimation. Using the fractional covering number, we (i) provide a unified characterization of learnability for \emph{any} stochastic bandit problem, (ii) close the remaining gap between the upper and lower bounds in Foster et al. (2021, 2023) (up to polynomial factors) for any interactive decision making problem in which the underlying model class is convex.

cs.LG↗

arXiv2024

Ultra-High-Efficiency Dual-Band Thin-Film Lithium Niobate Modulator Incorporating Low-k Underfill with 220 GHz Extrapolated Bandwidth for 390 Gbit/s PAM8 Transmission

High-performance electro-optic modulators play a critical role in modern telecommunication networks and intra-datacenter interconnects. Low driving voltage, large electro-optic bandwidth, compact device size, and multi-band operation ability are essential for various application scenarios, especially energy-efficient high-speed data transmission. However, it is challenging to meet all these requirements simultaneously. Here, we demonstrate a high-performance dual-band thin-film lithium niobate electro-optic modulator with low-k underfill to achieve overall performance improvement. The low-k material helps reduce the RF loss of the modulator and achieve perfect velocity matching with narrow electrode gap to overcome the voltage-bandwidth limitation, extending electro-optic bandwidth and enhancing modulation efficiency simultaneously. The fabricated 7-mm-long modulator exhibits a low half-wave voltage of 1.9 V at C-band and 1.54 V at O-band, featuring a low half-wave voltage-length product of 1.33 V*cm and 1.08 V*cm, respectively. Meanwhile, the novel design yields an ultra-wide extrapolated 3 dB bandwidth of 220 GHz (218 GHz) in the C-band (O-band). High-speed data transmission in both C- and O-bands using the same device has been demonstrated for the first time by PAM8 with data rates up to 390 Gbit/s, corresponding to a record-low energy consumption of 0.69 fJ/bit for next-generation cost-effective ultra-high-speed optical communications.

physics.optics↗

arXiv2024

Stochastic contextual bandits with graph feedback: from independence number to MAS number

We consider contextual bandits with graph feedback, a class of interactive learning problems with richer structures than vanilla contextual bandits, where taking an action reveals the rewards for all neighboring actions in the feedback graph under all contexts. Unlike the multi-armed bandits setting where a growing literature has painted a near-complete understanding of graph feedback, much remains unexplored in the contextual bandits counterpart. In this paper, we make inroads into this inquiry by establishing a regret lower bound $Ω(\sqrt{β_M(G) T})$, where $M$ is the number of contexts, $G$ is the feedback graph, and $β_M(G)$ is our proposed graph-theoretic quantity that characterizes the fundamental learning limit for this class of problems. Interestingly, $β_M(G)$ interpolates between $α(G)$ (the independence number of the graph) and $\mathsf{m}(G)$ (the maximum acyclic subgraph (MAS) number of the graph) as the number of contexts $M$ varies. We also provide algorithms that achieve near-optimal regret for important classes of context sequences and/or feedback graphs, such as transitively closed graphs that find applications in auctions and inventory control. In particular, with many contexts, our results show that the MAS number essentially characterizes the statistical complexity for contextual bandits, as opposed to the independence number in multi-armed bandits.

cs.LG↗

arXiv2024

On the Statistical Complexity of Sample Amplification

The ``sample amplification'' problem formalizes the following question: Given $n$ i.i.d. samples drawn from an unknown distribution $P$, when is it possible to produce a larger set of $n+m$ samples which cannot be distinguished from $n+m$ i.i.d. samples drawn from $P$? In this work, we provide a firm statistical foundation for this problem by deriving generally applicable amplification procedures, lower bound techniques and connections to existing statistical notions. Our techniques apply to a large class of distributions including the exponential family, and establish a rigorous connection between sample amplification and distribution learning.

math.ST↗

arXiv2024

Prediction from compression for models with infinite memory, with applications to hidden Markov and renewal processes

Consider the problem of predicting the next symbol given a sample path of length n, whose joint distribution belongs to a distribution class that may have long-term memory. The goal is to compete with the conditional predictor that knows the true model. For both hidden Markov models (HMMs) and renewal processes, we determine the optimal prediction risk in Kullback- Leibler divergence up to universal constant factors. Extending existing results in finite-order Markov models [HJW23] and drawing ideas from universal compression, the proposed estimator has a prediction risk bounded by redundancy of the distribution class and a memory term that accounts for the long-range dependency of the model. Notably, for HMMs with bounded state and observation spaces, a polynomial-time estimator based on dynamic programming is shown to achieve the optimal prediction risk Θ(log n/n); prior to this work, the only known result of this type is O(1/log n) obtained using Markov approximation [Sha+18]. Matching minimax lower bounds are obtained by making connections to redundancy and mutual information via a reduction argument.

math.ST↗

arXiv2024

Online Estimation via Offline Estimation: An Information-Theoretic Framework

$ $The classical theory of statistical estimation aims to estimate a parameter of interest under data generated from a fixed design ("offline estimation"), while the contemporary theory of online learning provides algorithms for estimation under adaptively chosen covariates ("online estimation"). Motivated by connections between estimation and interactive decision making, we ask: is it possible to convert offline estimation algorithms into online estimation algorithms in a black-box fashion? We investigate this question from an information-theoretic perspective by introducing a new framework, Oracle-Efficient Online Estimation (OEOE), where the learner can only interact with the data stream indirectly through a sequence of offline estimators produced by a black-box algorithm operating on the stream. Our main results settle the statistical and computational complexity of online estimation in this framework. $\bullet$ Statistical complexity. We show that information-theoretically, there exist algorithms that achieve near-optimal online estimation error via black-box offline estimation oracles, and give a nearly-tight characterization for minimax rates in the OEOE framework. $\bullet$ Computational complexity. We show that the guarantees above cannot be achieved in a computationally efficient fashion in general, but give a refined characterization for the special case of conditional density estimation: computationally efficient online estimation via black-box offline estimation is possible whenever it is possible via unrestricted algorithms. Finally, we apply our results to give offline oracle-efficient algorithms for interactive decision making.

stat.ML↗

arXiv2024

Optimal No-regret Learning in Repeated First-price Auctions

We study online learning in repeated first-price auctions where a bidder, only observing the winning bid at the end of each auction, learns to adaptively bid in order to maximize her cumulative payoff. To achieve this goal, the bidder faces censored feedback: if she wins the bid, then she is not able to observe the highest bid of the other bidders, which we assume is \textit{iid} drawn from an unknown distribution. In this paper, we develop the first learning algorithm that achieves a near-optimal $\widetilde{O}(\sqrt{T})$ regret bound, by exploiting two structural properties of first-price auctions, i.e. the specific feedback structure and payoff function. We first formulate the feedback structure in first-price auctions as partially ordered contextual bandits, a combination of the graph feedback across actions (bids), the cross learning across contexts (private values), and a partial order over the contexts. We establish both strengths and weaknesses of this framework, by showing a curious separation that a regret nearly independent of the action/context sizes is possible under stochastic contexts, but is impossible under adversarial contexts. In particular, this framework leads to an $O(\sqrt{T}\log^{2.5}T)$ regret for first-price auctions when the bidder's private values are \emph{iid}. Despite the limitation of the above framework, we further exploit the special payoff function of first-price auctions to develop a sample-efficient algorithm even in the presence of adversarially generated private values. We establish an $O(\sqrt{T}\log^3 T)$ regret bound for this algorithm, hence providing a complete characterization of optimal learning guarantees for first-price auctions.

cs.LG↗

arXiv2024

Learning and Collusion in Multi-unit Auctions

We consider repeated multi-unit auctions with uniform pricing, which are widely used in practice for allocating goods such as carbon licenses. In each round, $K$ identical units of a good are sold to a group of buyers that have valuations with diminishing marginal returns. The buyers submit bids for the units, and then a price $p$ is set per unit so that all the units are sold. We consider two variants of the auction, where the price is set to the $K$-th highest bid and $(K+1)$-st highest bid, respectively. We analyze the properties of this auction in both the offline and online settings. In the offline setting, we consider the problem that one player $i$ is facing: given access to a data set that contains the bids submitted by competitors in past auctions, find a bid vector that maximizes player $i$'s cumulative utility on the data set. We design a polynomial time algorithm for this problem, by showing it is equivalent to finding a maximum-weight path on a carefully constructed directed acyclic graph. In the online setting, the players run learning algorithms to update their bids as they participate in the auction over time. Based on our offline algorithm, we design efficient online learning algorithms for bidding. The algorithms have sublinear regret, under both full information and bandit feedback structures. We complement our online learning algorithms with regret lower bounds. Finally, we analyze the quality of the equilibria in the worst case through the lens of the core solution concept in the game among the bidders. We show that the $(K+1)$-st price format is susceptible to collusion among the bidders; meanwhile, the $K$-th price format does not have this issue.

cs.GT↗

arXiv2024

Statistical Complexity and Optimal Algorithms for Non-linear Ridge Bandits

We consider the sequential decision-making problem where the mean outcome is a non-linear function of the chosen action. Compared with the linear model, two curious phenomena arise in non-linear models: first, in addition to the "learning phase" with a standard parametric rate for estimation or regret, there is an "burn-in period" with a fixed cost determined by the non-linear function; second, achieving the smallest burn-in cost requires new exploration algorithms. For a special family of non-linear functions named ridge functions in the literature, we derive upper and lower bounds on the optimal burn-in cost, and in addition, on the entire learning trajectory during the burn-in period via differential equations. In particular, a two-stage algorithm that first finds a good initial action and then treats the problem as locally linear is statistically optimal. In contrast, several classical algorithms, such as UCB and algorithms relying on regression oracles, are provably suboptimal.

stat.ML↗

arXiv2023

Covariance alignment: from maximum likelihood estimation to Gromov-Wasserstein

Feature alignment methods are used in many scientific disciplines for data pooling, annotation, and comparison. As an instance of a permutation learning problem, feature alignment presents significant statistical and computational challenges. In this work, we propose the covariance alignment model to study and compare various alignment methods and establish a minimax lower bound for covariance alignment that has a non-standard dimension scaling because of the presence of a nuisance parameter. This lower bound is in fact minimax optimal and is achieved by a natural quasi MLE. However, this estimator involves a search over all permutations which is computationally infeasible even when the problem has moderate size. To overcome this limitation, we show that the celebrated Gromov-Wasserstein algorithm from optimal transport which is more amenable to fast implementation even on large-scale problems is also minimax optimal. These results give the first statistical justification for the deployment of the Gromov-Wasserstein algorithm in practice.

math.ST↗

arXiv2023

Suspended AlGaAs-Integrated Nonlinear Photonics

AlGaAs is a promising integrated nonlinear photonics material with enormous optical nonlinearity and high refractive index. Nevertheless, presently AlGaAs microring resonators exhibiting high quality factors and tight optical confinement rely predominantly on wafer bonding techniques, which entail an intricate fabrication process. Here, we present suspended AlGaAs waveguides and resonators as a viable platform for high efficiency integrated nonlinear photonics. The suspended microring resonator formed by combined plasma dry etching and chemical wet etching exhibits an intrinsic quality factor Q of 2.1*106, and Kerr comb generation with milliwatt level threshold is recorded. Our demonstration opens up a new prospect for employing AlGaAs devices in integrated nonlinear photonics.

physics.optics↗

arXiv2023

B-spline freeform surface tailoring for prescribed irradiance based on differentiable ray-tracing

A universal and flexible design method for freeform surface that can modulate the distribution of an zero-étendue source to an arbitrary irradiance distribution is a significant challenge in the field of non-imaging optics. Current design methods typically formulate the problem as a partial differential equation and solve it through sophisticated numerical methods, especially for off-axis situations. However, most of the current methods are unsuitable for directly solving multi-freeform surface or hybrid design problems that contains both freeform and spherical surfaces. To address these challenges, we propose the B-spline surface tailoring method, based on a differentiable ray-tracing algorithm. Our method features a computationally efficient B-spline model and a two-step optimization strategy based on optimal transport mapping. This allows for rapid, iterative adjustments to the surface shape based on deviations between the simulated and target distributions while ensuring a smooth resulting surface shape. In experiments, the proposed approach performs well in both paraxial and off-axis situations, and exhibits superior flexibility when applied to hybrid design case.

physics.optics↗

arXiv2023

Tight Guarantees for Interactive Decision Making with the Decision-Estimation Coefficient

A foundational problem in reinforcement learning and interactive decision making is to understand what modeling assumptions lead to sample-efficient learning guarantees, and what algorithm design principles achieve optimal sample complexity. Recently, Foster et al. (2021) introduced the Decision-Estimation Coefficient (DEC), a measure of statistical complexity which leads to upper and lower bounds on the optimal sample complexity for a general class of problems encompassing bandits and reinforcement learning with function approximation. In this paper, we introduce a new variant of the DEC, the Constrained Decision-Estimation Coefficient, and use it to derive new lower bounds that improve upon prior work on three fronts: - They hold in expectation, with no restrictions on the class of algorithms under consideration. - They hold globally, and do not rely on the notion of localization used by Foster et al. (2021). - Most interestingly, they allow the reference model with respect to which the DEC is defined to be improper, establishing that improper reference models play a fundamental role. We provide upper bounds on regret that scale with the same quantity, thereby closing all but one of the gaps between upper and lower bounds in Foster et al. (2021). Our results apply to both the regret framework and PAC framework, and make use of several new analysis and algorithm design techniques that we anticipate will find broader use.

cs.LG↗

arXiv2022

Oracle-Efficient Online Learning for Beyond Worst-Case Adversaries

In this paper, we study oracle-efficient algorithms for beyond worst-case analysis of online learning. We focus on two settings. First, the smoothed analysis setting of [RST11,HRS22] where an adversary is constrained to generating samples from distributions whose density is upper bounded by $1/σ$ times the uniform density. Second, the setting of $K$-hint transductive learning, where the learner is given access to $K$ hints per time step that are guaranteed to include the true instance. We give the first known oracle-efficient algorithms for both settings that depend only on the pseudo (or VC) dimension of the class and parameters $σ$ and $K$ that capture the power of the adversary. In particular, we achieve oracle-efficient regret bounds of $ \widetilde{O} ( \sqrt{T dσ^{-1}} ) $ and $ \widetilde{O} ( \sqrt{T dK} ) $ for learning real-valued functions and $ O ( \sqrt{T dσ^{-\frac{1}{2}} } )$ for learning binary-valued functions. For the smoothed analysis setting, our results give the first oracle-efficient algorithm for online learning with smoothed adversaries [HRS22]. This contrasts the computational separation between online learning with worst-case adversaries and offline learning established by [HK16]. Our algorithms also achieve improved bounds for worst-case setting with small domains. In particular, we give an oracle-efficient algorithm with regret of $O ( \sqrt{T(d |\mathcal{X}|)^{1/2} })$, which is a refinement of the earlier $O ( \sqrt{T|\mathcal{X}|})$ bound by [DS16].

cs.LG↗

arXiv2022

On the amortized complexity of approximate counting

Naively storing a counter up to value $n$ would require $Ω(\log n)$ bits of memory. Nelson and Yu [NY22], following work of [Morris78], showed that if the query answers need only be $(1+ε)$-approximate with probability at least $1 - δ$, then $O(\log\log n + \log\log(1/δ) + \log(1/ε))$ bits suffice, and in fact this bound is tight. Morris' original motivation for studying this problem though, as well as modern applications, require not only maintaining one counter, but rather $k$ counters for $k$ large. This motivates the following question: for $k$ large, can $k$ counters be simultaneously maintained using asymptotically less memory than $k$ times the cost of an individual counter? That is to say, does this problem benefit from an improved {\it amortized} space complexity bound? We answer this question in the negative. Specifically, we prove a lower bound for nearly the full range of parameters showing that, in terms of memory usage, there is no asymptotic benefit possible via amortization when storing multiple counters. Our main proof utilizes a certain notion of "information cost" recently introduced by Braverman, Garg and Woodruff in FOCS 2020 to prove lower bounds for streaming algorithms.

cs.DS↗