arXiv subjects

Hao Ma

Publications and source records attributed to Hao Ma.

At least 73 records · Page 4Linked to original sources

arXiv2025

Step-KTO: Optimizing Mathematical Reasoning through Stepwise Binary Feedback

Large language models (LLMs) have recently demonstrated remarkable success in mathematical reasoning. Despite progress in methods like chain-of-thought prompting and self-consistency sampling, these advances often focus on final correctness without ensuring that the underlying reasoning process is coherent and reliable. This paper introduces Step-KTO, a training framework that combines process-level and outcome-level binary feedback to guide LLMs toward more trustworthy reasoning trajectories. By providing binary evaluations for both the intermediate reasoning steps and the final answer, Step-KTO encourages the model to adhere to logical progressions rather than relying on superficial shortcuts. Our experiments on challenging mathematical benchmarks show that Step-KTO significantly improves both final answer accuracy and the quality of intermediate reasoning steps. For example, on the MATH-500 dataset, Step-KTO achieves a notable improvement in Pass@1 accuracy over strong baselines. These results highlight the promise of integrating stepwise process feedback into LLM training, paving the way toward more interpretable and dependable reasoning capabilities.

cs.LG↗

arXiv2024

ECAFormer: Low-light Image Enhancement using Cross Attention

Low-light image enhancement (LLIE) is critical in computer vision. Existing LLIE methods often fail to discover the underlying relationships between different sub-components, causing the loss of complementary information between multiple modules and network layers, ultimately resulting in the loss of image details. To beat this shortage, we design a hierarchical mutual Enhancement via a Cross Attention transformer (ECAFormer), which introduces an architecture that enables concurrent propagation and interaction of multiple features. The model preserves detailed information by introducing a Dual Multi-head self-attention (DMSA), which leverages visual and semantic features across different scales, allowing them to guide and complement each other. Besides, a Cross-Scale DMSA block is introduced to capture the residual connection, integrating cross-layer information to further enhance image detail. Experimental results show that ECAFormer reaches competitive performance across multiple benchmarks, yielding nearly a 3% improvement in PSNR over the suboptimal method, demonstrating the effectiveness of information interaction in LLIE.

cs.CV↗

arXiv2024

Impact of dynamic Jahn-Teller effect on magnetic excitations, lattice vibration, and thermal conductivity in UxTh1-xO2 system

Vibrational and magnetic properties of single-crystal uranium-thorium dioxide (UxTh1-xO2) with a full range of 0<x<1 is investigated. Thorium dioxide is a diamagnet whose thermal properties are governed by lattice vibration. The addition of paramagnetic uranium ion leads to the emergence of magnetic effects that alter the thermophysical properties noticeably even at room temperature. The interaction of phonons with magnetic moments of uranium 5f electrons mediated by magnetoelastic coupling results in an anomalous low-temperature thermal conductivity profile. Analysis of the magnetic susceptibility measurements indicates a uranium-concentration-dependent reduction in effective magnetic moment previously associated with the dynamic Jahn-Teller (DJT) effect. The T2g Raman peak position follows a nonlinear trend as a function of uranium concentration and hints that these Raman active optical modes play a role in either DJT or mediating quadrupole-quadrupole interactions. A first principle-based thermal transport model is implemented to explain the low-temperature transport measurements, where the anomalous reduction is attributed to phonon-spin resonant scattering. The interplay between spins and phonons is also captured using high-resolution inelastic X-ray scattering (IXS) measurements of phonon linewidths. Our results provide new insights into the phonon interactions with the magnetic excitations governing DJT effect and impacting the low-temperature thermal transport processes in this material system. These findings have implications for understanding low-temperature thermal transport and magnetic properties in advanced materials for information processing and energy applications.

cond-mat.str-el↗

arXiv2024

Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Following

Large Language Models (LLMs) have demonstrated impressive capabilities in various tasks, including instruction following, which is crucial for aligning model outputs with user expectations. However, evaluating LLMs' ability to follow instructions remains challenging due to the complexity and subjectivity of human language. Current benchmarks primarily focus on single-turn, monolingual instructions, which do not adequately reflect the complexities of real-world applications that require handling multi-turn and multilingual interactions. To address this gap, we introduce Multi-IF, a new benchmark designed to assess LLMs' proficiency in following multi-turn and multilingual instructions. Multi-IF, which utilizes a hybrid framework combining LLM and human annotators, expands upon the IFEval by incorporating multi-turn sequences and translating the English prompts into another 7 languages, resulting in a dataset of 4,501 multilingual conversations, where each has three turns. Our evaluation of 14 state-of-the-art LLMs on Multi-IF reveals that it presents a significantly more challenging task than existing benchmarks. All the models tested showed a higher rate of failure in executing instructions correctly with each additional turn. For example, o1-preview drops from 0.877 at the first turn to 0.707 at the third turn in terms of average accuracy over all languages. Moreover, languages with non-Latin scripts (Hindi, Russian, and Chinese) generally exhibit higher error rates, suggesting potential limitations in the models' multilingual capabilities. We release Multi-IF prompts and the evaluation code base to encourage further research in this critical area.

cs.CL↗

arXiv2024

Constraining Dark Matter Models with a Light Mediator from CDEX-10 Experiment at China Jinping Underground Laboratory

We search for nuclear recoil signals of dark matter models with a light mediator using data taken from a p-type point-contact germanium detector of the CDEX-10 experiment at the China Jinping Underground Laboratory. The 90% confidence level upper limits on the DM-nucleon interaction cross section from 205.4 kg-day exposure data are derived, excluding new parameter space in 2~5 GeV DM mass when the mediator mass is comparable to or lighter than the typical momentum transfer. We further interpret our results to constrain a specific self-interacting dark matter model with a light mediator coupling to the photon through kinetic mixing, and set experimental limits on the model parameter region favored by astrophysical observations.

hep-ph↗

arXiv2024

DRLinSPH: An open-source platform using deep reinforcement learning and SPHinXsys for fluid-structure-interaction problems

Fluid-structure interaction (FSI) problems are characterized by strong nonlinearities arising from complex interactions between fluids and structures. These pose significant challenges for traditional control strategies in optimizing structural motion, often leading to suboptimal performance. In contrast, deep reinforcement learning (DRL), through agent interactions within numerical simulation environments and the approximation of control policies using deep neural networks (DNNs), has shown considerable promise in addressing high-dimensional FSI problems. Additionally, smoothed particle hydrodynamics (SPH) offers a flexible and efficient computational approach for modeling large deformations, fractures, and complex interface movements inherent in FSI, outperforming traditional grid-based methods. In this work, we present DRLinSPH, an open-source Python platform that integrates the SPH-based numerical environment provided by the open-source software SPHinXsys with the mature DRL platform Tianshou to enable parallel training for FSI problems. DRLinSPH has been successfully applied to four FSI scenarios: sloshing suppression using rigid and elastic baffles, optimization of wave energy capture through an oscillating wave surge converter (OWSC), and muscle-driven fish swimming in vortices. The results demonstrate the platform's accuracy, stability, and scalability, highlighting its potential to advance industrial solutions for complex FSI challenges.

cs.CE↗

arXiv2024

The Perfect Blend: Redefining RLHF with Mixture of Judges

Reinforcement learning from human feedback (RLHF) has become the leading approach for fine-tuning large language models (LLM). However, RLHF has limitations in multi-task learning (MTL) due to challenges of reward hacking and extreme multi-objective optimization (i.e., trade-off of multiple and/or sometimes conflicting objectives). Applying RLHF for MTL currently requires careful tuning of the weights for reward model and data combinations. This is often done via human intuition and does not generalize. In this work, we introduce a novel post-training paradigm which we called Constrained Generative Policy Optimization (CGPO). The core of CGPO is Mixture of Judges (MoJ) with cost-efficient constrained policy optimization with stratification, which can identify the perfect blend in RLHF in a principled manner. It shows strong empirical results with theoretical guarantees, does not require extensive hyper-parameter tuning, and is plug-and-play in common post-training pipelines. Together, this can detect and mitigate reward hacking behaviors while reaching a pareto-optimal point across an extremely large number of objectives. Our empirical evaluations demonstrate that CGPO significantly outperforms standard RLHF algorithms like PPO and DPO across various tasks including general chat, STEM questions, instruction following, and coding. Specifically, CGPO shows improvements of 7.4% in AlpacaEval-2 (general chat), 12.5% in Arena-Hard (STEM & reasoning), and consistent gains in other domains like math and coding. Notably, PPO, while commonly used, is prone to severe reward hacking in popular coding benchmarks, which CGPO successfully addresses. This breakthrough in RLHF not only tackles reward hacking and extreme multi-objective optimization challenges but also advances the state-of-the-art in aligning general-purpose LLMs for diverse applications.

cs.LG↗

arXiv2024

Quantum charged black holes

Within the framework of braneworld holography, we construct a quantum charged black hole localized on a three-dimensional anti-de Sitter (AdS) brane that intersects the asymptotic boundary of the four-dimensional AdS spacetime at the conformal defects and incorporates quantum backreaction effects from the conformal field theory (CFT) on the brane. This quantum charged black hole is an exact solution of the semiclassical gravitational equation corresponding to a theory with higher curvature gravity and nonminimally coupled nonlinear gauge field. In the framework of double holography, we investigate the thermodynamics of the quantum charged black hole from three perspectives: a pure bulk perspective, in which four-dimensional classical Einstein gravity couples to Maxwell electrodynamics and a codimension-one tensional brane; a brane perspective, where semiclassical higher curvature gravity is subject to quantum backreaction from the holographic CFT on the brane, yielding a quantum charged black hole; and a boundary perspective, where the defect CFT is coupled to a boundary CFT at the asymptotic boundary and the degrees of freedom for defect quantum conformal matter is considered. In so doing, we obtain doubly holographic formulations of both the first law of thermodynamics and the Smarr (energy) relations for the quantum charged black holes.

hep-th↗

arXiv2024

Safe & Accurate at Speed with Tendons: A Robot Arm for Exploring Dynamic Motion

Operating robots precisely and at high speeds has been a long-standing goal of robotics research. Balancing these competing demands is key to enabling the seamless collaboration of robots and humans and increasing task performance. However, traditional motor-driven systems often fall short in this balancing act. Due to their rigid and often heavy design exacerbated by positioning the motors into the joints, faster motions of such robots transfer high forces at impact. To enable precise and safe dynamic motions, we introduce a four degree-of-freedom~(DoF) tendon-driven robot arm. Tendons allow placing the actuation at the base to reduce the robot's inertia, which we show significantly reduces peak collision forces compared to conventional robots with motors placed near the joints. Pairing our robot with pneumatic muscles allows generating high forces and highly accelerated motions, while benefiting from impact resilience through passive compliance. Since tendons are subject to additional friction and hence prone to wear and tear, we validate the reliability of our robotic arm on various experiments, including long-term dynamic motions. We also demonstrate its ease of control by quantifying the nonlinearities of the system and the performance on a challenging dynamic table tennis task learned from scratch using reinforcement learning. We open-source the entire hardware design, which can be largely 3D printed, the control software, and a proprioceptive dataset of 25 days of diverse robot motions at webdav.tuebingen.mpg.de/pamy2.

cs.RO↗

arXiv2024

List-avoiding orientations

Given a graph $G$ with a set $F(v)$ of forbidden values at each $v \in V(G)$, an $F$-avoiding orientation of $G$ is an orientation in which $deg^+(v) \not \in F(v)$ for each vertex $v$. Akbari, Dalirrooyfard, Ehsani, Ozeki, and Sherkati conjectured that if $|F(v)| < \frac{1}{2} deg(v)$ for each $v \in V(G)$, then $G$ has an $F$-avoiding orientation, and they showed that this statement is true when $\frac{1}{2}$ is replaced by $\frac{1}{4}$. In this paper, we take a step toward this conjecture by proving that if $|F(v)| < \lfloor \frac{1}{3} deg(v) \rfloor$ for each vertex $v$, then $G$ has an $F$-avoiding orientation. Furthermore, we show that if the maximum degree of $G$ is subexponential in terms of the minimum degree, then this coefficient of $\frac{1}{3}$ can be increased to $\sqrt{2} - 1 - o(1) \approx 0.414$. Our main tool is a new sufficient condition for the existence of an $F$-avoiding orientation based on the Combinatorial Nullstellensatz of Alon and Tarsi.

math.CO↗

arXiv2024

PE-GPT: A Physics-Informed Interactive Large Language Model for Power Converter Modulation Design

This paper proposes PE-GPT, a custom-tailored large language model uniquely adapted for power converter modulation design. By harnessing in-context learning and specialized tiered physics-informed neural networks, PE-GPT guides users through text-based dialogues, recommending actionable modulation parameters. The effectiveness of PE-GPT is validated through a practical design case involving dual active bridge converters, supported by hardware experimentation. This research underscores the transformative potential of large language models in power converter modulation design, offering enhanced accessibility, explainability, and efficiency, thereby setting a new paradigm in the field.

eess.SY↗

arXiv2024

Phonon thermal transport in UO$_2$ via self-consistent perturbation theory

Computing thermal transport from first-principles in UO$_2$ is complicated due to the challenges associated with Mott physics. Here we use irreducible derivative approaches to compute the cubic and quartic phonon interactions in UO$_2$ from first-principles, and we perform enhanced thermal transport computations by evaluating the phonon Green's function via self-consistent diagrammatic perturbation theory. Our predicted phonon lifetimes at $T=600$ K agree well with our inelastic neutron scattering measurements across the entire Brillouin zone, and our thermal conductivity predictions agree well with previous measurements. Both the changes due to thermal expansion and self-consistent contributions are nontrivial at high temperatures, though the effects tend to cancel, and interband transitions yield a substantial contribution.

cond-mat.mtrl-sci↗

arXiv2024

Study of cosmogenic activation in $^{76}$Ge enriched germanium detectors during fabrication and transportation above ground

Rare event search experiments using germanium detectors are operated in underground laboratories to minimize the background induced by cosmic rays. However, the cosmogenic activation in germanium crystals on the ground during fabrication and transportation generates long half-life radionuclides and contributes a considerable background. We simulated the production rates of cosmogenic radionuclides in germanium and calculated the specifi c activities of cosmogenic radionuclides according to the scheduled fabrication and transportation processes of $^{76}$Ge enriched germanium detectors. The impact of cosmogenic background in germanium crystals for the next generation CDEX experiment was assessed with the scheduled exposure history above ground.

physics.ins-det↗

arXiv2024

Poisoning Attacks against Recommender Systems: A Survey

Modern recommender systems (RS) have seen substantial success, yet they remain vulnerable to malicious activities, notably poisoning attacks. These attacks involve injecting malicious data into the training datasets of RS, thereby compromising their integrity and manipulating recommendation outcomes for gaining illicit profits. This survey paper provides a systematic and up-to-date review of the research landscape on Poisoning Attacks against Recommendation (PAR). A novel and comprehensive taxonomy is proposed, categorizing existing PAR methodologies into three distinct categories: Component-Specific, Goal-Driven, and Capability Probing. For each category, we discuss its mechanism in detail, along with associated methods. Furthermore, this paper highlights potential future research avenues in this domain. Additionally, to facilitate and benchmark the empirical comparison of PAR, we introduce an open-source library, ARLib, which encompasses a comprehensive collection of PAR models and common datasets. The library is released at https://github.com/CoderWZW/ARLib.

cs.IR↗

arXiv2024

Extending Whisper with prompt tuning to target-speaker ASR

Target-speaker automatic speech recognition (ASR) aims to transcribe the desired speech of a target speaker from multi-talker overlapped utterances. Most of the existing target-speaker ASR (TS-ASR) methods involve either training from scratch or fully fine-tuning a pre-trained model, leading to significant training costs and becoming inapplicable to large foundation models. This work leverages prompt tuning, a parameter-efficient fine-tuning approach, to extend Whisper, a large-scale single-talker ASR model, to TS-ASR. Variants of prompt tuning approaches along with their configurations are explored and optimized for TS-ASR.Experimental results show that prompt tuning can achieve performance comparable to state-of-the-art full training approaches while only requiring about 1\% of task-specific model parameters. Notably, the original Whisper's features, such as inverse text normalization and timestamp tagging, are retained in target-speaker ASR, keeping the generated transcriptions natural and informative.

cs.CL↗

arXiv2023

Effective Long-Context Scaling of Foundation Models

We present a series of long-context LLMs that support effective context windows of up to 32,768 tokens. Our model series are built through continual pretraining from Llama 2 with longer training sequences and on a dataset where long texts are upsampled. We perform extensive evaluation on language modeling, synthetic context probing tasks, and a wide range of research benchmarks. On research benchmarks, our models achieve consistent improvements on most regular tasks and significant improvements on long-context tasks over Llama 2. Notably, with a cost-effective instruction tuning procedure that does not require human-annotated long instruction data, the 70B variant can already surpass gpt-3.5-turbo-16k's overall performance on a suite of long-context tasks. Alongside these results, we provide an in-depth analysis on the individual components of our method. We delve into Llama's position encodings and discuss its limitation in modeling long dependencies. We also examine the impact of various design choices in the pretraining process, including the data mix and the training curriculum of sequence lengths -- our ablation experiments suggest that having abundant long texts in the pretrain dataset is not the key to achieving strong performance, and we empirically verify that long context continual pretraining is more efficient and similarly effective compared to pretraining from scratch with long sequences.

cs.CL↗

arXiv2023

Adversarial Driving Behavior Generation Incorporating Human Risk Cognition for Autonomous Vehicle Evaluation

Autonomous vehicle (AV) evaluation has been the subject of increased interest in recent years both in industry and in academia. This paper focuses on the development of a novel framework for generating adversarial driving behavior of background vehicle interfering against the AV to expose effective and rational risky events. Specifically, the adversarial behavior is learned by a reinforcement learning (RL) approach incorporated with the cumulative prospect theory (CPT) which allows representation of human risk cognition. Then, the extended version of deep deterministic policy gradient (DDPG) technique is proposed for training the adversarial policy while ensuring training stability as the CPT action-value function is leveraged. A comparative case study regarding the cut-in scenario is conducted on a high fidelity Hardware-in-the-Loop (HiL) platform and the results demonstrate the adversarial effectiveness to infer the weakness of the tested AV.

cs.AI↗

arXiv2023

Data-Efficient Online Learning of Ball Placement in Robot Table Tennis

We present an implementation of an online optimization algorithm for hitting a predefined target when returning ping-pong balls with a table tennis robot. The online algorithm optimizes over so-called interception policies, which define the manner in which the robot arm intercepts the ball. In our case, these are composed of the state of the robot arm (position and velocity) at interception time. Gradient information is provided to the optimization algorithm via the mapping from the interception policy to the landing point of the ball on the table, which is approximated with a black-box and a grey-box approach. Our algorithm is applied to a robotic arm with four degrees of freedom that is driven by pneumatic artificial muscles. As a result, the robot arm is able to return the ball onto any predefined target on the table after about 2-5 iterations. We highlight the robustness of our approach by showing rapid convergence with both the black-box and the grey-box gradients. In addition, the small number of iterations required to reach close proximity to the target also underlines the sample efficiency. A demonstration video can be found here: https://youtu.be/VC3KJoCss0k.

cs.RO↗