Searcharxiv⌕ Search

arXiv subjects

Chen Huang

Publications and source records attributed to Chen Huang.

At least 109 records · Page 6Linked to original sources
arXiv2024

Towards Analyzing and Understanding the Limitations of DPO: A Theoretical Perspective

Direct Preference Optimization (DPO), which derives reward signals directly from pairwise preference data, has shown its effectiveness on aligning Large Language Models (LLMs) with human preferences. Despite its widespread use across various tasks, DPO has been criticized for its sensitivity to the SFT's effectiveness and its hindrance to the learning capacity towards human-preferred responses, leading to less satisfactory performance. To overcome those limitations, the theoretical understanding of DPO are indispensable but still lacking. To this end, we take a step towards theoretically analyzing and understanding the limitations of DPO. Specifically, we provide an analytical framework using the field theory to analyze the optimization process of DPO. By analyzing the gradient vector field of the DPO loss function, we find that the DPO loss function decreases the probability of producing human dispreferred data at a faster rate than it increases the probability of producing preferred data. This provides theoretical insights for understanding the limitations of DPO discovered in the related research experiments, thereby setting the foundation for its improvement.

cs.CL↗
arXiv2024

Multi-Convergence-Angle Ptychography with Simultaneous Strong Contrast and High Resolution

Advances in bioimaging methods and hardware facilities have revolutionised the determination of numerous biological structures at atomic or near-atomic resolution. Among these developments, electron ptychography has recently attracted considerable attention because of its superior resolution, remarkable sensitivity to light elements, and high electron dose efficiency. Here, we introduce an innovative approach called multi-convergence-angle (MCA) ptychography, which can simultaneously enhance both contrast and resolution with continuous information transfer across a wide spectrum of spatial frequency. Our work provides feasibility of future applications of MCA-ptychography in providing high-quality two-dimensional images as input to three-dimensional reconstruction methods, thereby facilitating more accurate determination of biological structures.

physics.optics↗
arXiv2024

Search for GeV gamma-ray emission from the possible TeV-bright red dwarfs with Fermi-LAT

Red dwarfs have been suggested to be among the possible astrophysical species accelerating particles and emitting TeV $γ$-rays. As an effort to search for the GeV $γ$-ray counterparts of the suggested TeV emission from eight red dwarfs, we analyse the 0.2--500 GeV $γ$-ray emission of the regions covering them exploiting the $\sim$13.6 yr Pass 8 data of the Fermi Large Area Telescope. A GeV $γ$-ray emission excess with significance of 3.8$σ$ is detected in the direction of the red dwarf V962 Tau. This emission contains V962 Tau in 1$σ$ error radius and is independent of the catalog source. However, the stellar flare scenario can hardly explain the total energy and lightcurve derived from the $γ$-ray emission in view of the spectral analysis. We also analyse the lightcurves in the positions of the eight red dwarfs and no time bin with significance $>$5$σ$ is found. Therefore, no significant emission from the red dwarfs could be concluded to be detected by Fermi-LAT.

astro-ph.HE↗
arXiv2024

Calibrating coordinate system alignment in a scanning transmission electron microscope using a digital twin

In four-dimensional scanning transmission electron microscopy (4D STEM) a focused beam is scanned over a specimen and a diffraction pattern is recorded at each position using a pixelated detector. During the experiment, it must be ensured that the scan coordinate system of the beam is correctly calibrated relative to the detector coordinate system. Various simplified and approximate models are used implicitly and explicitly for understanding and analyzing the recorded data, requiring translation between the physical reality of the instrument and the abstractions used in data interpretation. Here, we introduce a calibration method where interactive live data processing in combination with a digital twin is used to match a set of models and their parameters with the action of a real-world instrument.

physics.ins-det↗
arXiv2024

Cross-Space Adaptive Filter: Integrating Graph Topology and Node Attributes for Alleviating the Over-smoothing Problem

The vanilla Graph Convolutional Network (GCN) uses a low-pass filter to extract low-frequency signals from graph topology, which may lead to the over-smoothing problem when GCN goes deep. To this end, various methods have been proposed to create an adaptive filter by incorporating an extra filter (e.g., a high-pass filter) extracted from the graph topology. However, these methods heavily rely on topological information and ignore the node attribute space, which severely sacrifices the expressive power of the deep GCNs, especially when dealing with disassortative graphs. In this paper, we propose a cross-space adaptive filter, called CSF, to produce the adaptive-frequency information extracted from both the topology and attribute spaces. Specifically, we first derive a tailored attribute-based high-pass filter that can be interpreted theoretically as a minimizer for semi-supervised kernel ridge regression. Then, we cast the topology-based low-pass filter as a Mercer's kernel within the context of GCNs. This serves as a foundation for combining it with the attribute-based filter to capture the adaptive-frequency information. Finally, we derive the cross-space filter via an effective multiple-kernel learning strategy, which unifies the attribute-based high-pass filter and the topology-based low-pass filter. This helps to address the over-smoothing problem while maintaining effectiveness. Extensive experiments demonstrate that CSF not only successfully alleviates the over-smoothing problem but also promotes the effectiveness of the node classification task.

cs.LG↗
arXiv2024

DREditor: An Time-efficient Approach for Building a Domain-specific Dense Retrieval Model

Deploying dense retrieval models efficiently is becoming increasingly important across various industries. This is especially true for enterprise search services, where customizing search engines to meet the time demands of different enterprises in different domains is crucial. Motivated by this, we develop a time-efficient approach called DREditor to edit the matching rule of an off-the-shelf dense retrieval model to suit a specific domain. This is achieved by directly calibrating the output embeddings of the model using an efficient and effective linear mapping. This mapping is powered by an edit operator that is obtained by solving a specially constructed least squares problem. Compared to implicit rule modification via long-time finetuning, our experimental results show that DREditor provides significant advantages on different domain-specific datasets, dataset sources, retrieval models, and computing devices. It consistently enhances time efficiency by 100-300 times while maintaining comparable or even superior retrieval performance. In a broader context, we take the first step to introduce a novel embedding calibration approach for the retrieval task, filling the technical blank in the current field of embedding calibration. This approach also paves the way for building domain-specific dense retrieval models efficiently and inexpensively.

cs.IR↗
arXiv2024

Higher-Point Gauge-Theory Couplings of Massive Spin-2 States in 4-Dimensional String Theories

We explicitly compute the (NS) sector conventional type-I superstring tree-level amplitudes at five points after compactifying to 4-D, express the QFT building block in the helicity basis, and give several attempts towards arbitrary $n$ points. More specifically, we consider the interaction of one first excited level and otherwise massless states of conventional type-I superstrings, where the four-dimensional states can, for instance, be realized via D$3$ branes. We construct the amplitude by using the Berends-Giele currents. From the recursion of Berends-Giele currents, we can generate the higher point amplitude. We also apply the BCFW recursion with massive external legs shifted and get the amplitude for arbitrary $n$ points.

hep-th↗
arXiv2023

Towards Equipping Transformer with the Ability of Systematic Compositionality

One of the key factors in language productivity and human cognition is the ability of systematic compositionality, which refers to understanding composed unseen examples of seen primitives. However, recent evidence reveals that the Transformers have difficulty generalizing the composed context based on the seen primitives. To this end, we take the first step to propose a compositionality-aware Transformer called CAT and two novel pre-training tasks to facilitate systematic compositionality. We tentatively provide a successful implementation of a multi-layer CAT on the basis of the especially popular BERT. The experimental results demonstrate that CAT outperforms baselines on compositionality-aware tasks with minimal impact on the effectiveness on standardized language understanding tasks.

cs.CL↗
arXiv2023

LiDAR: Sensing Linear Probing Performance in Joint Embedding SSL Architectures

Joint embedding (JE) architectures have emerged as a promising avenue for acquiring transferable data representations. A key obstacle to using JE methods, however, is the inherent challenge of evaluating learned representations without access to a downstream task, and an annotated dataset. Without efficient and reliable evaluation, it is difficult to iterate on architectural and training choices for JE methods. In this paper, we introduce LiDAR (Linear Discriminant Analysis Rank), a metric designed to measure the quality of representations within JE architectures. Our metric addresses several shortcomings of recent approaches based on feature covariance rank by discriminating between informative and uninformative features. In essence, LiDAR quantifies the rank of the Linear Discriminant Analysis (LDA) matrix associated with the surrogate SSL task -- a measure that intuitively captures the information content as it pertains to solving the SSL task. We empirically demonstrate that LiDAR significantly surpasses naive rank based approaches in its predictive power of optimal hyperparameters. Our proposed criterion presents a more robust and intuitive means of assessing the quality of representations within JE architectures, which we hope facilitates broader adoption of these powerful techniques in various domains.

cs.LG↗
arXiv2023

DUET: 2D Structured and Approximately Equivariant Representations

Multiview Self-Supervised Learning (MSSL) is based on learning invariances with respect to a set of input transformations. However, invariance partially or totally removes transformation-related information from the representations, which might harm performance for specific downstream tasks that require such information. We propose 2D strUctured and EquivarianT representations (coined DUET), which are 2d representations organized in a matrix structure, and equivariant with respect to transformations acting on the input data. DUET representations maintain information about an input transformation, while remaining semantically expressive. Compared to SimCLR (Chen et al., 2020) (unstructured and invariant) and ESSL (Dangovski et al., 2022) (unstructured and equivariant), the structured and equivariant nature of DUET representations enables controlled generation with lower reconstruction error, while controllability is not possible with SimCLR or ESSL. DUET also achieves higher accuracy for several discriminative tasks, and improves transfer learning.

cs.LG↗
arXiv2023

Adaptivity and Modularity for Efficient Generalization Over Task Complexity

Can transformers generalize efficiently on problems that require dealing with examples with different levels of difficulty? We introduce a new task tailored to assess generalization over different complexities and present results that indicate that standard transformers face challenges in solving these tasks. These tasks are variations of pointer value retrieval previously introduced by Zhang et al. (2021). We investigate how the use of a mechanism for adaptive and modular computation in transformers facilitates the learning of tasks that demand generalization over the number of sequential computation steps (i.e., the depth of the computation graph). Based on our observations, we propose a transformer-based architecture called Hyper-UT, which combines dynamic function generation from hyper networks with adaptive depth from Universal Transformers. This model demonstrates higher accuracy and a fairer allocation of computational resources when generalizing to higher numbers of computation steps. We conclude that mechanisms for adaptive depth and modularity complement each other in improving efficient generalization concerning example complexity. Additionally, to emphasize the broad applicability of our findings, we illustrate that in a standard image recognition task, Hyper- UT's performance matches that of a ViT model but with considerably reduced computational demands (achieving over 70\% average savings by effectively using fewer layers).

cs.LG↗
arXiv2023

Cryo-Electron Ptychography: Applications and Potential in Biological Characterisation

There is a clear need for developments in characterisation techniques that provide detailed information about structure-function relationships in biology. Using electron microscopy to achieve high resolution while maintaining a broad field of view remains a challenge, particularly for radiation sensitive specimens where the signal-to-noise ratio required to maintain structural integrity is limited by low electron fluence. In this review, we explore the potential of cryogenic electron ptychography as an alternative method for characterisation of biological systems under low fluence conditions. Using this method with increased information content from multiple sampled regions of interest, potentially allows 3D reconstruction with far fewer particles than required in conventional cryo-electron microscopy. This is important for achieving higher resolution for systems where distributions of homogeneous single particles are difficult to obtain. We discuss the progress, limitations and potential areas for future development of this approach for both single particle analysis and in applications to heterogeneous large objects.

q-bio.QM↗
arXiv2023

United v.s. Divided, Deconfinement of Social Tension as a Topological Phase Transition

The proverbs "the enemy of my enemy is my friend" and alike capture the essence of many body correlations in social relations, whose violation leads to social tension. We study how rule-breakers, who disrespect these norms, affect the structure and dynamics of signed social networks which tries to minimize social tension. We find two dynamic phases. A friendly society exhibits a "united phase" where insertion of a rule-breaker only leads to localized rearrangement. A hostile society exhibits a "divided phase", where insertion leads to macroscopic reorganization of social relations. In the divided phase, starting from the utopia state, where all relations are friendly, insertion of a {\em separatist}, a particular type of rule-breaker who makes friends with only half of its neighbors, leads to fragmentation, where the society breaks into many finite size, mutually antagonistic cliques. These phenomena are described by Ising lattice gauge theory, where social tension behave as $Z_2$ topological defects, which are confined in the united phase and deconfined in the divided phase. We further show that the connection between social dynamics and Ising lattice gauge theory is viable independently of connectivity structure of the social network.

cond-mat.stat-mech↗
arXiv2023

Quantile regression outcome-adaptive lasso: variable selection for causal quantile treatment effect estimation

Quantile treatment effects (QTEs) can characterize the potentially heterogeneous causal effect of a treatment on different points of the entire outcome distribution. Propensity score (PS) methods are commonly employed for estimating QTEs in non-randomized studies. Empirical and theoretical studies have shown that insufficient and unnecessary adjustment for covariates in PS models can lead to bias and efficiency loss in estimating treatment effects. Striking a balance between bias and efficiency through variable selection is a crucial concern in casual inference. It is essential to acknowledge that the covariates related treatment and outcome may vary across different quantiles of the outcome distribution. However, previous studies have overlooked to adjust for different covariates separately in the PS models when estimating different QTEs. In this article, we proposed the quantile regression outcome-adaptive lasso (QROAL) method to select covariates that can provide unbiased and efficient estimates of QTEs. A distinctive feature of our proposed method is the utilization of linear quantile regression models for constructing penalty weights, enabling covariate selection in PS models separately when estimating different QTEs. We conducted simulation studies to show the superiority of our proposed method over the outcome-adaptive lasso (OAL) method in variable selection. Moreover, the proposed method exhibited favorable performance compared to the OAL method in terms of root mean square error in a range of settings, including both homogeneous and heterogeneous scenarios. Additionally, we applied the QROAL method to datasets from the China Health and Retirement Longitudinal Study (CHARLS) to explore the impact of smoking status on the severity of depression symptoms.

stat.ME↗
arXiv2023

Semi-Supervised and Long-Tailed Object Detection with CascadeMatch

This paper focuses on long-tailed object detection in the semi-supervised learning setting, which poses realistic challenges, but has rarely been studied in the literature. We propose a novel pseudo-labeling-based detector called CascadeMatch. Our detector features a cascade network architecture, which has multi-stage detection heads with progressive confidence thresholds. To avoid manually tuning the thresholds, we design a new adaptive pseudo-label mining mechanism to automatically identify suitable values from data. To mitigate confirmation bias, where a model is negatively reinforced by incorrect pseudo-labels produced by itself, each detection head is trained by the ensemble pseudo-labels of all detection heads. Experiments on two long-tailed datasets, i.e., LVIS and COCO-LT, demonstrate that CascadeMatch surpasses existing state-of-the-art semi-supervised approaches -- across a wide range of detection architectures -- in handling long-tailed object detection. For instance, CascadeMatch outperforms Unbiased Teacher by 1.9 AP Fix on LVIS when using a ResNet50-based Cascade R-CNN structure, and by 1.7 AP Fix when using Sparse R-CNN with a Transformer encoder. We also show that CascadeMatch can even handle the challenging sparsely annotated object detection problem.

cs.CV↗
arXiv2023

Twist-phase-matching in two-dimensional materials

Optical phase-matching involves establishing a proper phase relationship between the fundamental and generated waves to enable efficient optical parametric processes. It is typically achieved through either birefringence or periodically assigned polarization. Here, we report that twist angle in two-dimensional (2D) materials can generate a nonlinear Berry optical phase to compensate the phase mismatch in the process of nonlinear optical frequency conversion, and the vertical assembly of 2D layers with a proper twist sequence will generate a nontrivial "twist-phase-matching" (twist-PM) regime. The twist-PM model offers superior flexibility in the design of optical crystals, which works for twisted layers with either periodic or random thickness distribution. The designed crystals from twisted rhombohedra boron nitride films give rise to a second-harmonic generation conversion efficiency of ~8% within a thickness of only 3.2 um, and a facile polarization controllability that is absent in conventional crystals (from linear to left-/right-handed circular/elliptical polarizations). Our methodology establishes a platform for the rational designing and atomic manufacturing of nonlinear optical crystals based on abundant 2D materials for various functionalities.

physics.optics↗
arXiv2023

MAST: Masked Augmentation Subspace Training for Generalizable Self-Supervised Priors

Recent Self-Supervised Learning (SSL) methods are able to learn feature representations that are invariant to different data augmentations, which can then be transferred to downstream tasks of interest. However, different downstream tasks require different invariances for their best performance, so the optimal choice of augmentations for SSL depends on the target task. In this paper, we aim to learn self-supervised features that generalize well across a variety of downstream tasks (e.g., object classification, detection and instance segmentation) without knowing any task information beforehand. We do so by Masked Augmentation Subspace Training (or MAST) to encode in the single feature space the priors from different data augmentations in a factorized way. Specifically, we disentangle the feature space into separate subspaces, each induced by a learnable mask that selects relevant feature dimensions to model invariance to a specific augmentation. We show the success of MAST in jointly capturing generalizable priors from different augmentations, using both unique and shared features across the subspaces. We further show that MAST benefits from uncertainty modeling to reweight ambiguous samples from strong augmentations that may cause similarity mismatch in each subspace. Experiments demonstrate that MAST consistently improves generalization on various downstream tasks, while being task-agnostic and efficient during SSL. We also provide interesting insights about how different augmentations are related and how uncertainty reflects learning difficulty.

cs.LG↗
arXiv2023

Effect of electron- and hole-doping on properties of kagome-lattice ferromagnet Fe3Sn2

We report a theoretical investigation of effects of Mn and Co substitution in the transition metal sites of the kagome-lattice ferromagnet, Fe3Sn2. Herein, hole- and electron-doping effects of Fe3Sn2 have been studied by density-functional theory calculations on the parent phase and on the substituted structural models of Fe3-xMxSn2 (M = Mn, Co; x = 0.5, 1.0). All optimized structures favor the ferromagnetic ground state. Analysis of the electronic density of states (DOS) and band structure plots reveals that the hole (electron) doping leads to a progressive decrease (increase) in the magnetic moment per Fe atom and per unit cell overall. The high DOS is retained nearby the Fermi level in the case of both Mn and Co substitutions. The electron doping with Co results in the loss of nodal band degeneracies, while in the case of hole doping with Mn emergent nodal band degeneracies and flatbands initially are suppressed in Fe2.5Mn0.5Sn2 but re-emerge in Fe2MnSn2. These results provide key insights into potential modifications of intriguing coupling between electronic and spin degrees of freedom observed in Fe3Sn2.

cond-mat.mtrl-sci↗