arXiv subjects

Hongjie Chen

Publications and source records attributed to Hongjie Chen.

51 records · Page 3Linked to original sources

arXiv2025

Efficient Model Selection for Time Series Forecasting via LLMs

Model selection is a critical step in time series forecasting, traditionally requiring extensive performance evaluations across various datasets. Meta-learning approaches aim to automate this process, but they typically depend on pre-constructed performance matrices, which are costly to build. In this work, we propose to leverage Large Language Models (LLMs) as a lightweight alternative for model selection. Our method eliminates the need for explicit performance matrices by utilizing the inherent knowledge and reasoning capabilities of LLMs. Through extensive experiments with LLaMA, GPT and Gemini, we demonstrate that our approach outperforms traditional meta-learning techniques and heuristic baselines, while significantly reducing computational overhead. These findings underscore the potential of LLMs in efficient model selection for time series forecasting.

cs.LG↗

arXiv2025

Improved Robust Estimation for Erdős-Rényi Graphs: The Sparse Regime and Optimal Breakdown Point

We study the problem of robustly estimating the edge density of Erdős-Rényi random graphs $G(n, d^\circ/n)$ when an adversary can arbitrarily add or remove edges incident to an $η$-fraction of the nodes. We develop the first polynomial-time algorithm for this problem that estimates $d^\circ$ up to an additive error $O([\sqrt{\log(n) / n} + η\sqrt{\log(1/η)} ] \cdot \sqrt{d^\circ} + η\log(1/η))$. Our error guarantee matches information-theoretic lower bounds up to factors of $\log(1/η)$. Moreover, our estimator works for all $d^\circ \geq Ω(1)$ and achieves optimal breakdown point $η= 1/2$. Previous algorithms [AJK+22, CDHS24], including inefficient ones, incur significantly suboptimal errors. Furthermore, even admitting suboptimal error guarantees, only inefficient algorithms achieve optimal breakdown point. Our algorithm is based on the sum-of-squares (SoS) hierarchy. A key ingredient is to construct constant-degree SoS certificates for concentration of the number of edges incident to small sets in $G(n, d^\circ/n)$. Crucially, we show that these certificates also exist in the sparse regime, when $d^\circ = o(\log n)$, a regime in which the performance of previous algorithms was significantly suboptimal.

cs.DS↗

arXiv2025

Local derivation on some class of subspace lattice algebras

Let $\mathcal{H}$ be a separable Hilbert space and $\mathcal{L}_{0}\subset B(\mathcal{H})$ a complete reflexive lattice. Let $\mathscr{K}$ be the direct sum of $n_0$ copies of $\mathcal{H}$ ($n_{0}\in\mathbb{N}$ and $n_0\geq 2$) or the direct sum of countably infinite many copies of $\mathcal{H}$ respectively. We construct two class of subspace lattices $\mathcal{L}$ on $\mathscr{K}$. Let $Alg\mathcal{L}$ be the corresponding subspace lattice algebra. We show that every local derivation from $Alg\mathcal{L} $ into $B(\mathscr{K})$ is a derivation.

math.OA↗

arXiv2024

Personalized Multimodal Large Language Models: A Survey

Multimodal Large Language Models (MLLMs) have become increasingly important due to their state-of-the-art performance and ability to integrate multiple data modalities, such as text, images, and audio, to perform complex tasks with high accuracy. This paper presents a comprehensive survey on personalized multimodal large language models, focusing on their architecture, training methods, and applications. We propose an intuitive taxonomy for categorizing the techniques used to personalize MLLMs to individual users, and discuss the techniques accordingly. Furthermore, we discuss how such techniques can be combined or adapted when appropriate, highlighting their advantages and underlying rationale. We also provide a succinct summary of personalization tasks investigated in existing research, along with the evaluation metrics commonly used. Additionally, we summarize the datasets that are useful for benchmarking personalized MLLMs. Finally, we outline critical open challenges. This survey aims to serve as a valuable resource for researchers and practitioners seeking to understand and advance the development of personalized multimodal large language models.

cs.CV↗

arXiv2024

Outlier-robust Mean Estimation near the Breakdown Point via Sum-of-Squares

We revisit the problem of estimating the mean of a high-dimensional distribution in the presence of an $\varepsilon$-fraction of adversarial outliers. When $\varepsilon$ is at most some sufficiently small constant, previous works can achieve optimal error rate efficiently \cite{diakonikolas2018robustly, kothari2018robust}. As $\varepsilon$ approaches the breakdown point $\frac{1}{2}$, all previous algorithms incur either sub-optimal error rates or exponential running time. In this paper we give a new analysis of the canonical sum-of-squares program introduced in \cite{kothari2018robust} and show that this program efficiently achieves optimal error rate for all $\varepsilon \in[0,\frac{1}{2})$. The key ingredient for our results is a new identifiability proof for robust mean estimation that focuses on the overlap between the distributions instead of their statistical distance as in previous works. We capture this proof within the sum-of-squares proof system, thus obtaining efficient algorithms using the sum-of-squares proofs to algorithms paradigm \cite{raghavendra2018high}.

cs.DS↗

arXiv2024

Private Edge Density Estimation for Random Graphs: Optimal, Efficient and Robust

We give the first polynomial-time, differentially node-private, and robust algorithm for estimating the edge density of Erdős-Rényi random graphs and their generalization, inhomogeneous random graphs. We further prove information-theoretical lower bounds, showing that the error rate of our algorithm is optimal up to logarithmic factors. Previous algorithms incur either exponential running time or suboptimal error rates. Two key ingredients of our algorithm are (1) a new sum-of-squares algorithm for robust edge density estimation, and (2) the reduction from privacy to robustness based on sum-of-squares exponential mechanisms due to Hopkins et al. (STOC 2023).

cs.DS↗

arXiv2024

Private graphon estimation via sum-of-squares

We develop the first pure node-differentially-private algorithms for learning stochastic block models and for graphon estimation with polynomial running time for any constant number of blocks. The statistical utility guarantees match those of the previous best information-theoretic (exponential-time) node-private mechanisms for these problems. The algorithm is based on an exponential mechanism for a score function defined in terms of a sum-of-squares relaxation whose level depends on the number of blocks. The key ingredients of our results are (1) a characterization of the distance between the block graphons in terms of a quadratic optimization over the polytope of doubly stochastic matrices, (2) a general sum-of-squares convergence result for polynomial optimization over arbitrary polytopes, and (3) a general approach to perform Lipschitz extensions of score functions as part of the sum-of-squares algorithmic paradigm.

cs.DS↗

arXiv2023

Private estimation algorithms for stochastic block models and mixture models

We introduce general tools for designing efficient private estimation algorithms, in the high-dimensional settings, whose statistical guarantees almost match those of the best known non-private algorithms. To illustrate our techniques, we consider two problems: recovery of stochastic block models and learning mixtures of spherical Gaussians. For the former, we present the first efficient $(ε, δ)$-differentially private algorithm for both weak recovery and exact recovery. Previously known algorithms achieving comparable guarantees required quasi-polynomial time. For the latter, we design an $(ε, δ)$-differentially private algorithm that recovers the centers of the $k$-mixture when the minimum separation is at least $ O(k^{1/t}\sqrt{t})$. For all choices of $t$, this algorithm requires sample complexity $n\geq k^{O(1)}d^{O(t)}$ and time complexity $(nd)^{O(t)}$. Prior work required minimum separation at least $O(\sqrt{k})$ as well as an explicit upper bound on the Euclidean norm of the centers.

cs.DS↗

arXiv2023

Reflected entropy and Markov gap in Lifshitz theories

We study the reflected entropy in $(1+1)$--dimensional Lifshitz field theory whose groundstate is described by a quantum mechanical model. Starting from tripartite Lifshitz groundstates, both critical and gapped, we derive explicit formulas for the Rényi reflected entropies reduced to two adjacent or disjoint intervals, directly in the continuum. We show that the reflected entropy in Lifshitz theory does not satisfy monotonicity, in contrast to what is observed for free relativistic fields. We analytically compute the full reflected entanglement spectrum for two disjoint intervals, finding a discrete set of eigenvalues which is that of a thermal density matrix. Furthermore, we investigate the Markov gap, defined as the difference between reflected entropy and mutual information, and find it to be universal and nonvanishing, signaling irreducible tripartite entanglement in Lifshitz groundstates. We also obtain analytical results for the reflected entropies and the Markov gap in $2 + 1$ dimensions. Finally, as a byproduct of our results on reflected entropy, we provide exact formulas for two other entanglement-related quantities, namely the computable cross-norm negativity and the operator entanglement entropy.

hep-th↗

arXiv2022

Supertranslation Goldstone and de Sitter Tachyons

Supertranslation Goldstone lies in certain "exceptional series" representations of $SL(2,\mathbb{C})$. Interestingly, $m^2=-3$ scalar tachyon in three dimensional de Sitter space also lies in the same representation. In this note, we analyze these theories, focusing on representation-theoretical aspects, and emphasize that "modulo certain polynomials", there is a unitary representation of the corresponding symmetry group.

hep-th↗

arXiv2022

On the well-spread property and its relation to linear regression

We consider the robust linear regression model $\boldsymbol{y} = Xβ^* + \boldsymbolη$, where an adversary oblivious to the design $X \in \mathbb{R}^{n \times d}$ may choose $\boldsymbolη$ to corrupt all but a (possibly vanishing) fraction of the observations $\boldsymbol{y}$ in an arbitrary way. Recent work [dLN+21, dNS21] has introduced efficient algorithms for consistent recovery of the parameter vector. These algorithms crucially rely on the design matrix being well-spread (a matrix is well-spread if its column span is far from any sparse vector). In this paper, we show that there exists a family of design matrices lacking well-spreadness such that consistent recovery of the parameter vector in the above robust linear regression model is information-theoretically impossible. We further investigate the average-case time complexity of certifying well-spreadness of random matrices. We show that it is possible to efficiently certify whether a given $n$-by-$d$ Gaussian matrix is well-spread if the number of observations is quadratic in the ambient dimension. We complement this result by showing rigorous evidence -- in the form of a lower bound against low-degree polynomials -- of the computational hardness of this same certification problem when the number of observations is $o(d^2)$.

cs.LG↗

arXiv2021

Topological reflected entropy in Chern-Simons theories

We study the reflected entropy between two spatial regions in $(2+1)$-dimensional Chern-Simons theories. Taking advantage of its replica trick formulation, the reflected entropy is computed using the edge theory approach and the surgery method. Both approaches yield identical results. In all cases considered in this paper, we find that the reflected entropy coincides with the mutual information, even though their Rényi versions differ in general. We also compute the odd entropy with the edge theory method. The reflected entropy and the odd entropy both possess a simple holographic dual interpretation in terms of entanglement wedge cross-section. We show that in $(2+1)$-dimensional Chern-Simons theories, both quantities are related in a similar manner as in two-dimensional holographic conformal field theories (CFTs), up to a classical Shannon piece.

hep-th↗

arXiv2020

Graph Deep Factors for Forecasting

Deep probabilistic forecasting techniques have recently been proposed for modeling large collections of time-series. However, these techniques explicitly assume either complete independence (local model) or complete dependence (global model) between time-series in the collection. This corresponds to the two extreme cases where every time-series is disconnected from every other time-series in the collection or likewise, that every time-series is related to every other time-series resulting in a completely connected graph. In this work, we propose a deep hybrid probabilistic graph-based forecasting framework called Graph Deep Factors (GraphDF) that goes beyond these two extremes by allowing nodes and their time-series to be connected to others in an arbitrary fashion. GraphDF is a hybrid forecasting framework that consists of a relational global and relational local model. In particular, we propose a relational global model that learns complex non-linear time-series patterns globally using the structure of the graph to improve both forecasting accuracy and computational efficiency. Similarly, instead of modeling every time-series independently, we learn a relational local model that not only considers its individual time-series but also the time-series of nodes that are connected in the graph. The experiments demonstrate the effectiveness of the proposed deep hybrid graph-based forecasting model compared to the state-of-the-art methods in terms of its forecasting accuracy, runtime, and scalability. Our case study reveals that GraphDF can successfully generate cloud usage forecasts and opportunistically schedule workloads to increase cloud cluster utilization by 47.5% on average.

cs.LG↗

arXiv2020

A Context Integrated Relational Spatio-Temporal Model for Demand and Supply Forecasting

Traditional methods for demand forecasting only focus on modeling the temporal dependency. However, forecasting on spatio-temporal data requires modeling of complex nonlinear relational and spatial dependencies. In addition, dynamic contextual information can have a significant impact on the demand values, and therefore needs to be captured. For example, in a bike-sharing system, bike usage can be impacted by weather. Existing methods assume the contextual impact is fixed. However, we note that the contextual impact evolves over time. We propose a novel context integrated relational model, Context Integrated Graph Neural Network (CIGNN), which leverages the temporal, relational, spatial, and dynamic contextual dependencies for multi-step ahead demand forecasting. Our approach considers the demand network over various geographical locations and represents the network as a graph. We define a demand graph, where nodes represent demand time-series, and context graphs (one for each type of context), where nodes represent contextual time-series. Assuming that various contexts evolve and have a dynamic impact on the fluctuation of demand, our proposed CIGNN model employs a fusion mechanism that jointly learns from all available types of contextual information. To the best of our knowledge, this is the first approach that integrates dynamic contexts with graph neural networks for spatio-temporal demand forecasting, thereby increasing prediction accuracy. We present empirical results on two real-world datasets, demonstrating that CIGNN consistently outperforms state-of-the-art baselines, in both periodic and irregular time-series networks.

cs.LG↗

arXiv2018

Learning Acoustic Word Embeddings with Temporal Context for Query-by-Example Speech Search

We propose to learn acoustic word embeddings with temporal context for query-by-example (QbE) speech search. The temporal context includes the leading and trailing word sequences of a word. We assume that there exist spoken word pairs in the training database. We pad the word pairs with their original temporal context to form fixed-length speech segment pairs. We obtain the acoustic word embeddings through a deep convolutional neural network (CNN) which is trained on the speech segment pairs with a triplet loss. Shifting a fixed-length analysis window through the search content, we obtain a running sequence of embeddings. In this way, searching for the spoken query is equivalent to the matching of acoustic word embeddings. The experiments show that our proposed acoustic word embeddings learned with temporal context are effective in QbE speech search. They outperform the state-of-the-art frame-level feature representations and reduce run-time computation since no dynamic time warping is required in QbE speech search. We also find that it is important to have sufficient speech segment pairs to train the deep CNN for effective acoustic word embeddings.

cs.CL↗