arXiv subjects

Tian Qin

Publications and source records attributed to Tian Qin.

At least 37 records · Page 2Linked to original sources

arXiv2025

Decomposing Elements of Problem Solving: What "Math" Does RL Teach?

Mathematical reasoning tasks have become prominent benchmarks for assessing the reasoning capabilities of LLMs, especially with reinforcement learning (RL) methods such as GRPO showing significant performance gains. However, accuracy metrics alone do not support fine-grained assessment of capabilities and fail to reveal which problem-solving skills have been internalized. To better understand these capabilities, we propose to decompose problem solving into fundamental capabilities: Plan (mapping questions to sequences of steps), Execute (correctly performing solution steps), and Verify (identifying the correctness of a solution). Empirically, we find that GRPO mainly enhances the execution skill-improving execution robustness on problems the model already knows how to solve-a phenomenon we call temperature distillation. More importantly, we show that RL-trained models struggle with fundamentally new problems, hitting a 'coverage wall' due to insufficient planning skills. To explore RL's impact more deeply, we construct a minimal, synthetic solution-tree navigation task as an analogy for mathematical problem-solving. This controlled setup replicates our empirical findings, confirming RL primarily boosts execution robustness. Importantly, in this setting, we identify conditions under which RL can potentially overcome the coverage wall through improved exploration and generalization to new solution paths. Our findings provide insights into the role of RL in enhancing LLM reasoning, expose key limitations, and suggest a path toward overcoming these barriers. Code is available at https://github.com/cfpark00/RL-Wall.

cs.AI↗

arXiv2025

On Kernel-based Variational Autoencoder

In this paper, we bridge Variational Autoencoders (VAEs) and kernel density estimations (KDEs) by approximating the posterior by KDEs and deriving an upper bound of the Kullback-Leibler (KL) divergence in the evidence lower bound (ELBO). The flexibility of KDEs makes the optimization of posteriors in VAEs possible, which not only addresses the limitations of Gaussian latent space in vanilla VAE but also provides a new perspective of estimating the KL-divergence in ELBO. Under appropriate conditions, we show that the Epanechnikov kernel is the optimal choice in minimizing the derived upper bound of KL-divergence asymptotically. Compared with Gaussian kernel, Epanechnikov kernel has compact support which should make the generated sample less noisy and blurry. The implementation of Epanechnikov kernel in ELBO is straightforward as it lies in the "location-scale" family of distributions where the reparametrization tricks can be directly employed. A series of experiments on benchmark datasets such as MNIST, Fashion-MNIST, CIFAR-10 and CelebA further demonstrate the superiority of Epanechnikov Variational Autoenocoder (EVAE) over vanilla VAE in the quality of reconstructed images, as measured by the FID score and Sharpness.

stat.ML↗

arXiv2025

Debiasing Kernel-Based Generative Models

We propose a novel two-stage framework of generative models named Debiasing Kernel-Based Generative Models (DKGM) with the insights from kernel density estimation (KDE) and stochastic approximation. In the first stage of DKGM, we employ KDE to bypass the obstacles in estimating the density of data without losing too much image quality. One characteristic of KDE is oversmoothing, which makes the generated image blurry. Therefore, in the second stage, we formulate the process of reducing the blurriness of images as a statistical debiasing problem and develop a novel iterative algorithm to improve image quality, which is inspired by the stochastic approximation. Extensive experiments illustrate that the image quality of DKGM on CIFAR10 is comparable to state-of-the-art models such as diffusion models and GAN models. The performance of DKGM on CelebA 128x128 and LSUN (Church) 128x128 is also competitive. We conduct extra experiments to exploit how the bandwidth in KDE affects the sample diversity and debiasing effect of DKGM. The connections between DKGM and score-based models are also discussed.

stat.ML↗

arXiv2025

SNAKE: A Sustainable and Multi-functional Traffic Analysis System utilizing Specialized Large-Scale Models with a Mixture of Experts Architecture

The rapid advancement of internet technology has led to a surge in data transmission, making network traffic classification crucial for security and management. However, there are significant deficiencies in its efficiency for handling multiattribute analysis and its ability to expand model knowledge, making it difficult to adapt to the ever-changing network environment and complex identification requirements. To address this issue, we proposed the SNAKE (Sustainable Network Analysis with Knowledge Exploration) system, which adopts a multi-gated mixture of experts architecture to construct a multi-functional traffic classification model. The system analyzes traffic attributes at different levels through multiple expert sub-models, providing predictions for these attributes via gating and a final Tower network. Additionally, through an intelligent gating configuration, the system enables extremely fast model integration and evolution across various knowledge expansion scenarios. Its excellent compatibility allows it to continuously evolve into a multi-functional largescale model in the field of traffic analysis. Our experimental results demonstrate that the SNAKE system exhibits remarkable scalability when faced with incremental challenges in diverse traffic classification tasks. Currently, we have integrated multiple models into the system, enabling it to classify a wide range of attributes, such as encapsulation usage, application types and numerous malicious behaviors. We believe that SNAKE can pioneeringly create a sustainable and multifunctional large-scale model in the field of network traffic analysis after continuous expansion.

cs.NI↗

arXiv2025

A Label is Worth a Thousand Images in Dataset Distillation

Data $\textit{quality}$ is a crucial factor in the performance of machine learning models, a principle that dataset distillation methods exploit by compressing training datasets into much smaller counterparts that maintain similar downstream performance. Understanding how and why data distillation methods work is vital not only for improving these methods but also for revealing fundamental characteristics of "good" training data. However, a major challenge in achieving this goal is the observation that distillation approaches, which rely on sophisticated but mostly disparate methods to generate synthetic data, have little in common with each other. In this work, we highlight a largely overlooked aspect common to most of these methods: the use of soft (probabilistic) labels. Through a series of ablation experiments, we study the role of soft labels in depth. Our results reveal that the main factor explaining the performance of state-of-the-art distillation methods is not the specific techniques used to generate synthetic data but rather the use of soft labels. Furthermore, we demonstrate that not all soft labels are created equal; they must contain $\textit{structured information}$ to be beneficial. We also provide empirical scaling laws that characterize the effectiveness of soft labels as a function of images-per-class in the distilled dataset and establish an empirical Pareto frontier for data-efficient learning. Combined, our findings challenge conventional wisdom in dataset distillation, underscore the importance of soft labels in learning, and suggest new directions for improving distillation methods. Code for all experiments is available at https://github.com/sunnytqin/no-distillation.

cs.LG↗

arXiv2025

Drift-oriented Self-evolving Encrypted Traffic Application Classification for Actual Network Environment

Encrypted traffic classification technology is a crucial decision-making information source for network management and security protection. It has the advantages of excellent response timeliness, large-scale data bearing, and cross-time-and-space analysis. The existing research on encrypted traffic classification has gradually transitioned from the closed world to the open world, and many classifier optimization and feature engineering schemes have been proposed. However, encrypted traffic classification has yet to be effectively applied to the actual network environment. The main reason is that applications on the Internet are constantly updated, including function adjustment and version change, which brings severe feature concept drift, resulting in rapid failure of the classifier. Hence, the entire model must be retrained only past very fast time, with unacceptable labeled sample constructing and model training cost. To solve this problem, we deeply study the characteristics of Internet application updates, associate them with feature concept drift, and then propose self-evolving encrypted traffic classification. We propose a feature concept drift determination method and a drift-oriented self-evolving fine-tuning method based on the Laida criterion to adapt to all applications that are likely to be updated. In the case of no exact label samples, the classifier evolves through fully fine-tuning continuously, and the time interval between two necessary retraining is greatly extended to be applied to the actual network environment. Experiments show that our approach significantly improves the classification performance of the original classifier on the following stage dataset of the following months (9\% improvement on F1-score) without any hard-to-acquire labeled sample. Under the current experimental environment, the life of the classifier is extended to more than eight months.

cs.NI↗

arXiv2024

Adaptive Offloading and Enhancement for Low-Light Video Analytics on Mobile Devices

In this paper, we explore adaptive offloading and enhancement strategies for video analytics tasks on computing-constrained mobile devices in low-light conditions. We observe that the accuracy of low-light video analytics varies from different enhancement algorithms. The root cause could be the disparities in the effectiveness of enhancement algorithms for feature extraction in analytic models. Specifically, the difference in class activation maps (CAMs) between enhanced and low-light frames demonstrates a positive correlation with video analytics accuracy. Motivated by such observations, a novel enhancement quality assessment method is proposed on CAMs to evaluate the effectiveness of different enhancement algorithms for low-light videos. Then, we design a multi-edge system, which adaptively offloads and enhances low-light video analytics tasks from mobile devices. To achieve the trade-off between the enhancement quality and the latency for all system-served mobile devices, we propose a genetic-based scheduling algorithm, which can find a near-optimal solution in a reasonable time to meet the latency requirement. Thereby, the offloading strategies and the enhancement algorithms are properly selected under the condition of limited end-edge bandwidth and edge computation resources. Simulation experiments demonstrate the superiority of the proposed system, improving accuracy up to 20.83\% compared to existing benchmarks.

cs.MM↗

arXiv2024

Riemann-Lebesgue Forest for Regression

We propose a novel ensemble method called Riemann-Lebesgue Forest (RLF) for regression. The core idea in RLF is to mimic the way how a measurable function can be approximated by partitioning its range into a few intervals. With this idea in mind, we develop a new tree learner named Riemann-Lebesgue Tree (RLT) which has a chance to perform Lebesgue type cutting,i.e splitting the node from response $Y$ at certain non-terminal nodes. We show that the optimal Lebesgue type cutting results in larger variance reduction in response $Y$ than ordinary CART \cite{Breiman1984ClassificationAR} cutting (an analogue of Riemann partition). Such property is beneficial to the ensemble part of RLF. We also generalize the asymptotic normality of RLF under different parameter settings. Two one-dimensional examples are provided to illustrate the flexibility of RLF. The competitive performance of RLF against original random forest \cite{Breiman2001RandomF} is demonstrated by experiments in simulation data and real world datasets.

stat.ML↗

arXiv2024

Multi-View Variational Autoencoder for Missing Value Imputation in Untargeted Metabolomics

Background: Missing data is a common challenge in mass spectrometry-based metabolomics, which can lead to biased and incomplete analyses. The integration of whole-genome sequencing (WGS) data with metabolomics data has emerged as a promising approach to enhance the accuracy of data imputation in metabolomics studies. Method: In this study, we propose a novel method that leverages the information from WGS data and reference metabolites to impute unknown metabolites. Our approach utilizes a multi-view variational autoencoder to jointly model the burden score, polygenetic risk score (PGS), and linkage disequilibrium (LD) pruned single nucleotide polymorphisms (SNPs) for feature extraction and missing metabolomics data imputation. By learning the latent representations of both omics data, our method can effectively impute missing metabolomics values based on genomic information. Results: We evaluate the performance of our method on empirical metabolomics datasets with missing values and demonstrate its superiority compared to conventional imputation techniques. Using 35 template metabolites derived burden scores, PGS and LD-pruned SNPs, the proposed methods achieved R^2-scores > 0.01 for 71.55% of metabolites. Conclusion: The integration of WGS data in metabolomics imputation not only improves data completeness but also enhances downstream analyses, paving the way for more comprehensive and accurate investigations of metabolic pathways and disease associations. Our findings offer valuable insights into the potential benefits of utilizing WGS data for metabolomics data imputation and underscore the importance of leveraging multi-modal data integration in precision medicine research.

q-bio.GN↗

arXiv2024

Distributional Dataset Distillation with Subtask Decomposition

What does a neural network learn when training from a task-specific dataset? Synthesizing this knowledge is the central idea behind Dataset Distillation, which recent work has shown can be used to compress large datasets into a small set of input-label pairs ($\textit{prototypes}$) that capture essential aspects of the original dataset. In this paper, we make the key observation that existing methods distilling into explicit prototypes are very often suboptimal, incurring in unexpected storage cost from distilled labels. In response, we propose $\textit{Distributional Dataset Distillation}$ (D3), which encodes the data using minimal sufficient per-class statistics and paired with a decoder, we distill dataset into a compact distributional representation that is more memory-efficient compared to prototype-based methods. To scale up the process of learning these representations, we propose $\textit{Federated distillation}$, which decomposes the dataset into subsets, distills them in parallel using sub-task experts and then re-aggregates them. We thoroughly evaluate our algorithm on a three-dimensional metric and show that our method achieves state-of-the-art results on TinyImageNet and ImageNet-1K. Specifically, we outperform the prior art by $6.9\%$ on ImageNet-1K under the storage budget of 2 images per class.

cs.LG↗

arXiv2024

Distinguishing the Knowable from the Unknowable with Language Models

We study the feasibility of identifying epistemic uncertainty (reflecting a lack of knowledge), as opposed to aleatoric uncertainty (reflecting entropy in the underlying distribution), in the outputs of large language models (LLMs) over free-form text. In the absence of ground-truth probabilities, we explore a setting where, in order to (approximately) disentangle a given LLM's uncertainty, a significantly larger model stands in as a proxy for the ground truth. We show that small linear probes trained on the embeddings of frozen, pretrained models accurately predict when larger models will be more confident at the token level and that probes trained on one text domain generalize to others. Going further, we propose a fully unsupervised method that achieves non-trivial accuracy on the same task. Taken together, we interpret these results as evidence that LLMs naturally contain internal representations of different types of uncertainty that could potentially be leveraged to devise more informative indicators of model confidence in diverse practical settings.

cs.LG↗

arXiv2023

On Subagging Boosted Probit Model Trees

With the insight of variance-bias decomposition, we design a new hybrid bagging-boosting algorithm named SBPMT for classification problems. For the boosting part of SBPMT, we propose a new tree model called Probit Model Tree (PMT) as base classifiers in AdaBoost procedure. For the bagging part, instead of subsampling from the dataset at each step of boosting, we perform boosted PMTs on each subagged dataset and combine them into a powerful "committee", which can be viewed an incomplete U-statistic. Our theoretical analysis shows that (1) SBPMT is consistent under certain assumptions, (2) Increase the subagging times can reduce the generalization error of SBPMT to some extent and (3) Large number of ProbitBoost iterations in PMT can benefit the performance of SBPMT with fewer steps in the AdaBoost part. Those three properties are verified by a famous simulation designed by Mease and Wyner (2008). The last two points also provide a useful guidance in model tuning. A comparison of performance with other state-of-the-art classification methods illustrates that the proposed SBPMT algorithm has competitive prediction power in general and performs significantly better in some cases.

stat.ML↗

arXiv2023

End-Edge Coordinated Joint Encoding and Neural Enhancement for Low-Light Video Analytics

In this paper, we investigate video analytics in low-light environments, and propose an end-edge coordinated system with joint video encoding and enhancement. It adaptively transmits low-light videos from cameras and performs enhancement and inference tasks at the edge. Firstly, according to our observations, both encoding and enhancement for low-light videos have a significant impact on inference accuracy, which directly influences bandwidth and computation overhead. Secondly, due to the limitation of built-in computation resources, cameras perform encoding and transmitting frames to the edge. The edge executes neural enhancement to process low contrast, detail loss, and color distortion on low-light videos before inference. Finally, an adaptive controller is designed at the edge to select quantization parameters and scales of neural enhancement networks, aiming to improve the inference accuracy and meet the latency requirements. Extensive real-world experiments demon-strate that, the proposed system can achieve a better trade-off between communication and computation resources and optimize the inference accuracy.

cs.MM↗

arXiv2023

Real-space Observation of Unidirectional Charge Density Wave and Complex Structural Modulation in the Pnictide Superconductor Ba$_{1-x}$Sr$_x$Ni$_2$As$_2$

Here we use low-temperature and variable-temperature scanning tunneling microscopy to study the pnictide superconductor, Ba$_{1-x}$Sr$_x$Ni$_2$As$_2$. In the low-temperature phase (triclinic phase) of BaNi$_2$As$_2$, we observe the unidirectional charge density wave (CDW) with $Q$ = 1/3 on both the Ba and NiAs surfaces. On the NiAs surface of the triclinic BaNi$_2$As$_2$, there are structural-modulation-induced chain-like superstructures with distinct periodicities. In the high-temperature phase (tetragonal phase) of BaNi$_2$As$_2$, the NiAs surface appears as the periodic 1 by 2 superstructure. Interestingly, in the triclinic phase of Ba$_{0.5}$Sr$_{0.5}$Ni$_2$As$_2$, the unidirectional CDW is suppressed on both the Ba/Sr and NiAs surfaces, and the Sr substitution stabilizes the periodic 1 by 2 superstructure on the NiAs surface, which enhance the superconductivity in Ba$_{0.5}$Sr$_{0.5}$Ni$_2$As$_2$. Our results provide important microscopic insights for the interplay among the unidirectional CDW, structural modulation, and superconductivity in this class of pnictide superconductors.

cond-mat.mes-hall↗

arXiv2023

Probing hidden Mott gap and incommensurate charge modulation on the polar surfaces of PdCrO$_2$

Here we report a combined study of low-temperature scanning tunneling microscopy (STM) and dynamical mean-field theory (DMFT) on PdCrO$_2$, a delafossite metal with an antiferromagnetic order below ~37.5 K. First, on the CrO$_2$-terminated polar surface we detect a gap-like feature both below and above the Néel temperature. The DMFT calculations indicate that this gap is opened due to the strong correlations of Cr-3d electrons, suggesting the hidden Mott nature of the gap. Then, we observe two kinds of Pd-terminated polar surfaces. One is a well-ordered Pd surface with the Fermi-surface-nesting-induced incommensurate charge modulation, while the other one is a reconstructed Pd surface with the individual nano-scale non-periodic domain structures. On the well-ordered Pd surface, the interference between the incommensurate charge modulation and the atomic lattice forms the periodic moiré pattern. Our results provide important microscopic information for fully understanding the correlated electronic properties of this class of materials.

cond-mat.str-el↗

arXiv2022

Meta-PDE: Learning to Solve PDEs Quickly Without a Mesh

Partial differential equations (PDEs) are often computationally challenging to solve, and in many settings many related PDEs must be be solved either at every timestep or for a variety of candidate boundary conditions, parameters, or geometric domains. We present a meta-learning based method which learns to rapidly solve problems from a distribution of related PDEs. We use meta-learning (MAML and LEAP) to identify initializations for a neural network representation of the PDE solution such that a residual of the PDE can be quickly minimized on a novel task. We apply our meta-solving approach to a nonlinear Poisson's equation, 1D Burgers' equation, and hyperelasticity equations with varying parameters, geometries, and boundary conditions. The resulting Meta-PDE method finds qualitatively accurate solutions to most problems within a few gradient steps; for the nonlinear Poisson and hyper-elasticity equation this results in an intermediate accuracy approximation up to an order of magnitude faster than a baseline finite element analysis (FEA) solver with equivalent accuracy. In comparison to other learned solvers and surrogate models, this meta-learning approach can be trained without supervision from expensive ground-truth data, does not require a mesh, and can even be used when the geometry and topology varies between tasks.

cs.LG↗

arXiv2022

Benefits of Permutation-Equivariance in Auction Mechanisms

Designing an incentive-compatible auction mechanism that maximizes the auctioneer's revenue while minimizes the bidders' ex-post regret is an important yet intricate problem in economics. Remarkable progress has been achieved through learning the optimal auction mechanism by neural networks. In this paper, we consider the popular additive valuation and symmetric valuation setting; i.e., the valuation for a set of items is defined as the sum of all items' valuations in the set, and the valuation distribution is invariant when the bidders and/or the items are permutated. We prove that permutation-equivariant neural networks have significant advantages: the permutation-equivariance decreases the expected ex-post regret, improves the model generalizability, while maintains the expected revenue invariant. This implies that the permutation-equivariance helps approach the theoretically optimal dominant strategy incentive compatible condition, and reduces the required sample complexity for desired generalization. Extensive experiments fully support our theory. To our best knowledge, this is the first work towards understanding the benefits of permutation-equivariance in auction mechanisms.

cs.GT↗

arXiv2022

Coexistence of Quasi-two-dimensional Superconductivity and Tunable Kondo Lattice in a van der Waals Superconductor

Realization of Kondo lattice in superconducting van der Waals materials not only provides a unique opportunity for tuning the Kondo lattice behavior by electrical gating or intercalation, but also is helpful for further understanding the heavy fermion superconductivity. Here we report a low-temperature and vector-magnetic-field scanning tunneling microscopy and spectroscopy study on a superconducting compound (4Hb-TaS2) with alternate stacking of 1T-TaS2 and 1H-TaS2 layers. We observe the quasi-two-dimensional superconductivity in the 1H-TaS2 layer with anisotropic response to the in-plane and out-of-plane magnetic fields. In the 1T-TaS2 layer, we detect the Kondo resonance peak that results from the Kondo screening of the unpaired electrons in the Star-of-David clusters. We also find the intensity of the Kondo resonance peak is sensitive to its relative position with the Fermi level, and it can be significantly enhanced when it's further shifted towards the Fermi level by evaporating Pb atoms onto the 1T-TaS2 surface. Our results are not only important for fully understanding the electronic properties of 4Hb-TaS2, but also pave the way for creating tunable Kondo lattice in the superconducting van der Waals materials.

cond-mat.supr-con↗