arXiv subjects

Ho-Kin Tang

Publications and source records attributed to Ho-Kin Tang.

At least 37 records · Page 2Linked to original sources

arXiv2024

Efficient sampling using Macrocanonical Monte Carlo and density of states mapping

In the context of Monte Carlo sampling for lattice models, the complexity of the energy landscape often leads to Markov chains being trapped in local optima, thereby increasing the correlation between samples and reducing sampling efficiency. This study proposes a Monte Carlo algorithm that effectively addresses the irregularities of the energy landscape through the introduction of the estimated density of states. This algorithm enhances the accuracy in the study of phase transitions and is not model-specific. Although our algorithm is primarily demonstrated on the two-dimensional square lattice model, the method is also applicable to a broader range of lattice and higher-dimensional models. Furthermore, the study develops a method for estimating the density of states of large systems based on that of smaller systems, enabling high-precision density of states estimation within specific energy intervals in large systems without sampling. For regions of lower precision, a re-weighting strategy is employed to adjust the density of states to enhance the precision further. This algorithm is not only significant within the field of lattice model sampling but may also inspire applications of the Monte Carlo method in other domains.

cond-mat.stat-mech↗

arXiv2024

Parameter Competition Balancing for Model Merging

While fine-tuning pretrained models has become common practice, these models often underperform outside their specific domains. Recently developed model merging techniques enable the direct integration of multiple models, each fine-tuned for distinct tasks, into a single model. This strategy promotes multitasking capabilities without requiring retraining on the original datasets. However, existing methods fall short in addressing potential conflicts and complex correlations between tasks, especially in parameter-level adjustments, posing a challenge in effectively balancing parameter competition across various tasks. This paper introduces an innovative technique named PCB-Merging (Parameter Competition Balancing), a lightweight and training-free technique that adjusts the coefficients of each parameter for effective model merging. PCB-Merging employs intra-balancing to gauge parameter significance within individual tasks and inter-balancing to assess parameter similarities across different tasks. Parameters with low importance scores are dropped, and the remaining ones are rescaled to form the final merged model. We assessed our approach in diverse merging scenarios, including cross-task, cross-domain, and cross-training configurations, as well as out-of-domain generalization. The experimental results reveal that our approach achieves substantial performance enhancements across multiple modalities, domains, model sizes, number of tasks, fine-tuning forms, and large language models, outperforming existing model merging methods. The code is publicly available at: \url{https://github.com/duguodong7/pcb-merging}.

cs.CV↗

arXiv2024

Variation of Electron-electron interaction in pyrochlore structures

We conduct a comprehensive \textit{ab initio} investigation of electron-electron interactions within the pyrochlore structures of R$_2$Ru$_2$O$_7$, R$_2$Ir$_2$O$_7$, Ca$_2$Ru$_2$O$_7$, and Cd$_2$Ru$_2$O$_7$, where R denotes a rare-earth element. Utilizing a multiorbital Hubbard model, we systematically explore the effects of various rare-earth elements and applied high pressure on the correlation strength in these compounds. Our calculations on the Coulomb interaction parameter $U$ and the bandwidth $W$ reveal that the chemical pressure for R$_2$Ru$_2$O$_7$ and R$_2$Ir$_2$O$_7$ leads to an unusual increase in $U/W$ ratio, hence, increase in correlation strength. Contrary to conventional understanding of bandwidth control, our study identifies that the Hubbard $U$ is more influential than the bandwidth $W$ behind the metal-insulator landscape of R$_2$Ru$_2$O$_7$ and R$_2$Ir$_2$O$_7$, leading to an interaction-controlled metal-insulator transition. We also find unexpected behavior in physical pressure. Whereas physical pressure leads to a decrease in the correlation strength $U/W$ as usual in R$_2$Ru$_2$O$_7$, the effect is notably small in Ca$_2$Ru$_2$O$_7$ and Cd$_2$Ru$_2$O$_7$, which provides an important clue to understanding unusual pressure-induced metal-insulator transition observed experimentally.

cond-mat.str-el↗

arXiv2024

Evolutionary Neural Architecture Search for 3D Point Cloud Analysis

Neural architecture search (NAS) automates neural network design by using optimization algorithms to navigate architecture spaces, reducing the burden of manual architecture design. While NAS has achieved success, applying it to emerging domains, such as analyzing unstructured 3D point clouds, remains underexplored due to the data lying in non-Euclidean spaces, unlike images. This paper presents Success-History-based Self-adaptive Differential Evolution with a Joint Point Interaction Dimension Search (SHSADE-PIDS), an evolutionary NAS framework that encodes discrete deep neural network architectures to continuous spaces and performs searches in the continuous spaces for efficient point cloud neural architectures. Comprehensive experiments on challenging 3D segmentation and classification benchmarks demonstrate SHSADE-PIDS's capabilities. It discovered highly efficient architectures with higher accuracy, significantly advancing prior NAS techniques. For segmentation on SemanticKITTI, SHSADE-PIDS attained 64.51% mean IoU using only 0.55M parameters and 4.5GMACs, reducing overhead by over 22-26X versus other top methods. For ModelNet40 classification, it achieved 93.4% accuracy with just 1.31M parameters, surpassing larger models. SHSADE-PIDS provided valuable insights into bridging evolutionary algorithms with neural architecture optimization, particularly for emerging frontiers like point cloud learning.

cs.CV↗

arXiv2024

Impacts of Darwinian Evolution on Pre-trained Deep Neural Networks

Darwinian evolution of the biological brain is documented through multiple lines of evidence, although the modes of evolutionary changes remain unclear. Drawing inspiration from the evolved neural systems (e.g., visual cortex), deep learning models have demonstrated superior performance in visual tasks, among others. While the success of training deep neural networks has been relying on back-propagation (BP) and its variants to learn representations from data, BP does not incorporate the evolutionary processes that govern biological neural systems. This work proposes a neural network optimization framework based on evolutionary theory. Specifically, BP-trained deep neural networks for visual recognition tasks obtained from the ending epochs are considered the primordial ancestors (initial population). Subsequently, the population evolved with differential evolution. Extensive experiments are carried out to examine the relationships between Darwinian evolution and neural network optimization, including the correspondence between datasets, environment, models, and living species. The empirical results show that the proposed framework has positive impacts on the network, with reduced over-fitting and an order of magnitude lower time complexity compared to BP. Moreover, the experiments show that the proposed framework performs well on deep neural networks and big datasets.

cs.NE↗

arXiv2024

Meta-heuristic Optimizer Inspired by the Philosophy of Yi Jing

Drawing inspiration from the philosophy of Yi Jing, the Yin-Yang pair optimization (YYPO) algorithm has been shown to achieve competitive performance in single objective optimizations, in addition to the advantage of low time complexity when compared to other population-based meta-heuristics. Building upon a reversal concept in Yi Jing, we propose the novel Yi optimization (YI) algorithm. Specifically, we enhance the Yin-Yang pair in YYPO with a proposed Yi-point, in which we use Cauchy flight to update the solution, by implementing both the harmony and reversal concept of Yi Jing. The proposed Yi-point balances both the effort of exploration and exploitation in the optimization process. To examine YI, we use the IEEE CEC 2017 benchmarks and compare YI against the dynamical YYPO, CV1.0 optimizer, and four classical optimizers, i.e., the differential evolution, the genetic algorithm, the particle swarm optimization, and the simulated annealing. According to the experimental results, YI shows highly competitive performance while keeping the low time complexity. The results of this work have implications for enhancing a meta-heuristic optimizer using the philosophy of Yi Jing. While this work implements only certain aspects of Yi Jing, we envisage enhanced performance by incorporating other aspects.

cs.NE↗

arXiv2024

Knowledge Fusion By Evolving Weights of Language Models

Fine-tuning pre-trained language models, particularly large language models, demands extensive computing resources and can result in varying performance outcomes across different domains and datasets. This paper examines the approach of integrating multiple models from diverse training scenarios into a unified model. This unified model excels across various data domains and exhibits the ability to generalize well on out-of-domain data. We propose a knowledge fusion method named Evolver, inspired by evolutionary algorithms, which does not need further training or additional training data. Specifically, our method involves aggregating the weights of different language models into a population and subsequently generating offspring models through mutation and crossover operations. These offspring models are then evaluated against their parents, allowing for the preservation of those models that show enhanced performance on development datasets. Importantly, our model evolving strategy can be seamlessly integrated with existing model merging frameworks, offering a versatile tool for model enhancement. Experimental results on mainstream language models (i.e., encoder-only, decoder-only, encoder-decoder) reveal that Evolver outperforms previous state-of-the-art models by large margins. The code is publicly available at {https://github.com/duguodong7/model-evolution}.

cs.CL↗

arXiv2024

dx2-y2-wave Bose Metal induced by the next-nearest-neighbor hopping t'

Superconductivity arises when electrons form Cooper pairs with phase coherence. In contrast, a lack of phase coherence in Cooper pairs can lead to an uncondensed metallic ground state known as the Bose metal state. In this study, we investigate an attractively interacting fermionic system with nearest-neighbor (NN) hopping (t) and next-nearest-neighbor (NNN) hopping (t') anisotropy between two species of spins in a two-dimensional (2D) lattice. Utilizing the constrained path quantum Monte Carlo (CPQMC) method, we demonstrate the existence of a dx2-y2-wave Cooper pair Bose metal (CPBM) phase with t'/t > 0.7. The CPBM phase exhibits a dome-like structure in the phase diagram of filling n~0.65, with the maximal region around an optimal t'/t ~ 0.2, suggesting that an appropriate value of t' facilitates the formation of the Bose metal. Furthermore, we find that a Bose metal formed by fermions with a closed Fermi surface confirms that the crucial condition for this exotic phenomenon is primarily the anisotropy of the Fermi surface, rather than its topology. Our finding of the dx2-y2-wave CPBM demonstrates the same pairing symmetry as the pseudogap behavior in cuprates, and its experimental realization in ultracold atom systems is also feasible.

cond-mat.str-el↗

arXiv2024

CADE: Cosine Annealing Differential Evolution for Spiking Neural Network

Spiking neural networks (SNNs) have gained prominence for their potential in neuromorphic computing and energy-efficient artificial intelligence, yet optimizing them remains a formidable challenge for gradient-based methods due to their discrete, spike-based computation. This paper attempts to tackle the challenges by introducing Cosine Annealing Differential Evolution (CADE), designed to modulate the mutation factor (F) and crossover rate (CR) of differential evolution (DE) for the SNN model, i.e., Spiking Element Wise (SEW) ResNet. Extensive empirical evaluations were conducted to analyze CADE. CADE showed a balance in exploring and exploiting the search space, resulting in accelerated convergence and improved accuracy compared to existing gradient-based and DE-based methods. Moreover, an initialization method based on a transfer learning setting was developed, pretraining on a source dataset (i.e., CIFAR-10) and fine-tuning the target dataset (i.e., CIFAR-100), to improve population diversity. It was found to further enhance CADE for SNN. Remarkably, CADE elevates the performance of the highest accuracy SEW model by an additional 0.52 percentage points, underscoring its effectiveness in fine-tuning and enhancing SNNs. These findings emphasize the pivotal role of a scheduler for F and CR adjustment, especially for DE-based SNN. Source Code on Github: https://github.com/Tank-Jiang/CADE4SNN.

cs.NE↗

arXiv2023

Spectral functions of the honeycomb lattice with both the Hubbard and long-range Coulomb Interactions

The absence of screening of the non-local Coulomb interaction in Dirac systems at charge neutrality leads to the breakdown of the Fermi liquid and divergence of the Fermi velocity. On the other hand, Mott Hubbard physics and the concomitant formation of local moments is dominated by the local effective Hubbard interaction. Using quantum Monte Carlo methods combined with stochastic analytical continuation, we compute the single particle spectral function of fermions on the honeycomb lattice for a realistic interaction that includes both the Hubbard interaction and long-ranged Coulomb repulsion. To a first approximation, we find that the generic high-energy features such as the formation of the upper Hubbard band are independent of the long-ranged Coulomb repulsion and determined mostly by the local effective Hubbard interaction. The sub-leading effects of adding the long-range interaction include an enhancement of the bandwidth and a decrease of the spin-polaron quasi-particle weight and lifetime.

cond-mat.str-el↗

arXiv2023

Ab Initio Studies on Interactions in K$_3$C$_{60}$ under High Pressure

Fullerene solids doped with alkali metals (A$_3$C$_{60}$, A = K, Rb, Cs) exhibit a superconducting transition temperature ($T_c$) as high as 40 K, and their unconventional superconducting properties have been a subject of debate. With application of high pressure on K$_3$C$_{60}$ and Rb$_3$C$_{60}$, the experiments demonstrate the decrease of $T_c$. In this paper, we focus on K$_3$C$_{60}$ and derive the structure of K$_3$C$_{60}$ under different pressures based on first-principles calculations, exploring the trends of Coulomb interactions at various pressures. By utilizing the Maximally Localized Wannier function approach, Constrained Density Functional Perturbation Theory (cDFPT), and Constrained Random Phase Approximation (cRPA), we construct a microscopic low-energy model near the Fermi level. Our results strongly indicate that, in the K$_3$C$_{60}$ system, as pressure increases, the effect of phonons is the key to intraorbital electron pairing. There is a dominance of the phonon-driven superconducting mechanism at high pressure.

cond-mat.supr-con↗

arXiv2022

Rapid detection of phase transitions from Monte Carlo samples before equilibrium

We found that Bidirectional LSTM and Transformer can classify different phases of condensed matter models and determine the phase transition points by learning features in the Monte Carlo raw data before equilibrium. Our method can significantly reduce the time and computational resources required for probing phase transitions as compared to the conventional Monte Carlo simulation. We also provide evidence that the method is robust and the performance of the deep learning model is insensitive to the type of input data (we tested spin configurations of classical models and green functions of a quantum model), and it also performs well in detecting Kosterlitz-Thouless phase transitions.

cond-mat.str-el↗

arXiv2021

Topological superfluids in two-dimensional Fermi gas with Rashba spin-orbit coupling

The realization of spin-orbit coupling (SOC) in ultracold atoms has triggered an intensive exploring of topological superfluids in the degenerate Fermi gases based on mean-field theory, which has not yet been reported in experiments. Here, we demonstrate the topological phase transitions in the system via the numerically exact quantum Monte Carlo method. Without prior assumptions, our unbiased real-space calculation shows that spin-orbit coupling can stabilize an unconventional pairing in the weak SOC regime, in which the Fulde-Ferrell-Larkin-Ovchinnikov pairing coexists with the Bardeen-Cooper-Schrieffer pairing. Furthermore, we use the jumps in the spin polarization at the time-reversal invariant momenta to qualify the topological phase transition, where we find the critical exponent deviated from the mean-field theory. Our results pave the way for the searching of unconventional pairing and topological superfluids with degenerate Fermi gases.

cond-mat.quant-gas↗

arXiv2021

Novel Approach to Unveil Quantum Phase Transitions Using Fidelity Map

Fidelity approach has been widely used to detect various types of quantum phase transitions, including some that are beyond the Landau symmetry breaking theory, in condensed matter models. However, challenges remain in locating the transition points with precision in several models with unconventional phases such as the quantum spin liquid phase in spin-1 Kitaev-Heisenberg model. In this work, we propose a novel approach, which we named the fidelity map, to detect quantum phase transitions with higher accuracy and sensitivity as compared to the conventional fidelity measures. Our scheme extends the fidelity concept from a single dimension quantity to a multi-dimensional quantity, and use a meta-heuristic algorithm to search for the critical points that globally maximized the fidelity within each phase. We test the scheme in three interacting condensed matter models, namely the spin-1 Kitaev Heisenberg model which consists of the quantum spin liquid phase and the topological Haldane phase, the spin-1/2 XXZ model which possesses a Berezinskii-Kosterlitz-Thouless transition, and the Su-Schrieffer-Heeger model that exhibits a topological quantum phase transition. The result shows that the fidelity map can capture a wide range of phase transitions accurately, thus providing a new tool to study phase transitions in unseen models without prior knowledge of the system's symmetry.

cond-mat.str-el↗

arXiv2021

A Novel Non-population-based Meta-heuristic Optimizer Inspired by the Philosophy of Yi Jing

Drawing inspiration from the philosophy of Yi Jing, Yin-Yang pair optimization (YYPO) has been shown to achieve competitive performance in single objective optimizations. Besides, it has the advantage of low time complexity when comparing to other population-based optimization. As a conceptual extension of YYPO, we proposed the novel Yi optimization (YI) algorithm as one of the best non-population-based optimizer. Incorporating both the harmony and reversal concept of Yi Jing, we replace the Yin-Yang pair with a Yi-point, in which we utilize the Levy flight to update the solution and balance both the effort of the exploration and the exploitation in the optimization process. As a conceptual prototype, we examine YI with IEEE CEC 2017 benchmark and compare its performance with a Levy flight-based optimizer CV1.0, the state-of-the-art dynamical Yin-Yang pair optimization in YYPO family and a few classical optimizers. According to the experimental results, YI shows highly competitive performance while keeping the low time complexity. Hence, the results of this work have implications for enhancing meta-heuristic optimizer using the philosophy of Yi Jing, which deserves research attention.

cs.NE↗

arXiv2020

Response to Comment on "The role of electron-electron interactions in two-dimensional Dirac fermions"

Hesselmann {\it et al}.~question one of our conclusions, namely, the suppression of Fermi velocity at the Gross-Neveu critical point for the specific case of vanishing long-range interactions and at zero energy. The possibility they raise could occur in any finite-size extrapolation of numerical data. While we cannot definitively rule out this possibility, we provide mathematical bounds on its likelihood.

cond-mat.str-el↗

arXiv2019

Universal Fermi-surface anisotropy renormalization for interacting Dirac fermions with long-range interactions

Recent experimental and numerical evidence suggest an intriguing universal relationship between the Fermi surface anisotropy of the non-interacting parent two-dimensional electron gas and the strongly correlated composite Fermi liquid formed in a strong magnetic field close to half-filing. Inspired by these observations, we explore more generally the question of anisotropy renormalization in interacting 2D Fermi systems. Using a recently developed non-perturbative and numerically-exact projective quantum Monte Carlo simulation as well as other numerical and analytic techniques, only for Dirac fermions with long-range Coulomb interactions do we find a universal square-root decrease of the Fermi-surface anisotropy. For the half-filled composite Fermi liquid, this result is surprising since a Dirac fermion ground state was only recently proposed as an alternative to the usual HLR state. The importance of the long-range interaction, expected for Dirac systems, is also consistent with recent transport measurements. Our proposed universality can be tested in several anisotropic Dirac materials including graphene, topological insulators organic conductors, and magic-angle twisted bilayer graphene.

cond-mat.str-el↗

arXiv2019

Electronic Ground State in Bilayer Graphene with Realistic Coulomb Interactions

Both insulating and conducting electronic behaviors have been experimentally seen in clean bilayer graphene samples at low temperature, and there is still no consensus on the nature of the interacting ground state at half-filling and in the absence of a magnetic field. Theoretically, several possibilities for the insulating ground states have been predicted for weak interaction strength. However, a recent renormalization-group calculation on a Hubbard model for charge-neutral bilayer graphene with short-range interactions suggests the emergence of low-energy Dirac fermions that would stabilize the metallic phase for weak interactions. Using a non-perturbative projective quantum Monte Carlo, we calculate the ground state for bilayer graphene using a realistic model for the Coulomb interaction that includes both short-range and long-range contributions. We find that a finite critical onsite interaction is needed to gap bilayer graphene, thereby confirming the Hubbard model expectations even in the presence of a long-range Coulomb potential, in agreement with our theoretical renormalization group analysis. In addition, we also find that the critical onsite interactions necessary to destabilize the metallic ground state decreases with increasing interlayer coupling.

cond-mat.str-el↗