arXiv subjects

Hao Hao

Publications and source records attributed to Hao Hao.

At least 37 records · Page 2Linked to original sources

arXiv2025

SID: Benchmarking Guided Instruction Capabilities in STEM Education with a Socratic Interdisciplinary Dialogues Dataset

Fostering students' abilities for knowledge integration and transfer in complex problem-solving scenarios is a core objective of modern education, and interdisciplinary STEM is a key pathway to achieve this, yet it requires expert guidance that is difficult to scale. While LLMs offer potential in this regard, their true capability for guided instruction remains unclear due to the lack of an effective evaluation benchmark. To address this, we introduce SID, the first benchmark designed to systematically evaluate the higher-order guidance capabilities of LLMs in multi-turn, interdisciplinary Socratic dialogues. Our contributions include a large-scale dataset of 10,000 dialogue turns across 48 complex STEM projects, a novel annotation schema for capturing deep pedagogical features, and a new suite of evaluation metrics (e.g., X-SRG). Baseline experiments confirm that even state-of-the-art LLMs struggle to execute effective guided dialogues that lead students to achieve knowledge integration and transfer. This highlights the critical value of our benchmark in driving the development of more pedagogically-aware LLMs.

cs.AI↗

arXiv2025

ELMES: An Automated Framework for Evaluating Large Language Models in Educational Scenarios

The emergence of Large Language Models (LLMs) presents transformative opportunities for education, generating numerous novel application scenarios. However, significant challenges remain: evaluation metrics vary substantially across different educational scenarios, while many emerging scenarios lack appropriate assessment metrics. Current benchmarks predominantly measure general intelligence rather than pedagogical capabilities. To address this gap, we introduce ELMES, an open-source automated evaluation framework specifically designed for assessing LLMs in educational settings. ELMES features a modular architecture that enables researchers to create dynamic, multi-agent dialogues through simple configuration files, facilitating flexible scenario design without requiring extensive programming expertise. The framework incorporates a hybrid evaluation engine that objectively quantifies traditionally subjective pedagogical metrics using an LLM-as-a-Judge methodology. We conduct systematic benchmarking of state-of-the-art LLMs across four critical educational scenarios: Knowledge Point Explanation, Guided Problem-Solving Teaching, Interdisciplinary Lesson Plan Generation, and Contextualized Question Generation, employing fine-grained metrics developed in collaboration with education specialists. Our results demonstrate distinct capability distributions among models, revealing context-specific strengths and limitations. ELMES provides educators and researchers with an accessible evaluation framework that significantly reduces adaptation barriers for diverse educational applications while advancing the practical implementation of LLMs in pedagogy. The framework is publicly available at \emph{https://github.com/sii-research/elmes.git}.

cs.CY↗

arXiv2025

A Large Language Model for Chemistry and Retrosynthesis Predictions

Large language models (LLM) have achieved impressive progress across a broad range of general-purpose tasks, but their effectiveness in chemistry remains limited due to scarce domain-specific datasets and the demand for precise symbolic and structural reasoning. Here we introduce ECNU-ChemGPT(name after East China Normal University), a chemistry-specialized LLM engineered for deep chemical knowledge understanding and accurate retrosynthetic route planning. Our approach is distinguished by four key strategies: structured prompt-based knowledge distillation from authoritative chemistry textbooks to construct a high-quality question-answering dataset; domain-specific prompt engineering using curated chemical keywords, combined with LLMs APIs for data derivation and knowledge distillation; large-scale fine-tuning on a meticulously cleaned and enriched Pistachio reaction dataset to enhance retrosynthesis prediction accuracy; and integration of BrainGPT, a dynamic multi-model scheduling framework that enables task-specific invocation of multiple specialized models trained for diverse chemistry-related tasks. ECNU-ChemGPT exhibits superior performance on chemistry question-answering and retrosynthetic planning benchmarks, outperforming leading general-purpose models-including Deepseek-R1, Qwen-2.5, and GPT-4o. In retrosynthesis, it achieves a Top-1 accuracy of 68.3% on the USPTO_50K dataset and successfully reconstructed 13 complete experimental pathways for real-world drug molecules from medicinal chemistry journals. These results underscore the effectiveness of domain-adapted fine-tuning combined with dynamic multi-model task scheduling, providing a scalable and robust solution for chemical knowledge question answering and retrosynthetic planning.

physics.chem-ph↗

arXiv2024

Un-evaluated Solutions May Be Valuable in Expensive Optimization

Expensive optimization problems (EOPs) are prevalent in real-world applications, where the evaluation of a single solution requires a significant amount of resources. In our study of surrogate-assisted evolutionary algorithms (SAEAs) in EOPs, we discovered an intriguing phenomenon. Because only a limited number of solutions are evaluated in each iteration, relying solely on these evaluated solutions for evolution can lead to reduced disparity in successive populations. This, in turn, hampers the reproduction operators' ability to generate superior solutions, thereby reducing the algorithm's convergence speed. To address this issue, we propose a strategic approach that incorporates high-quality, un-evaluated solutions predicted by surrogate models during the selection phase. This approach aims to improve the distribution of evaluated solutions, thereby generating a superior next generation of solutions. This work details specific implementations of this concept across various reproduction operators and validates its effectiveness using multiple surrogate models. Experimental results demonstrate that the proposed strategy significantly enhances the performance of surrogate-assisted evolutionary algorithms. Compared to mainstream SAEAs and Bayesian optimization algorithms, our approach incorporating the un-evaluated solution strategy shows a marked improvement.

cs.NE↗

arXiv2024

It's Morphing Time: Unleashing the Potential of Multiple LLMs via Multi-objective Optimization

In this paper, we introduce a novel approach for addressing the multi-objective optimization problem in large language model merging via black-box multi-objective optimization algorithms. The goal of model merging is to combine multiple models, each excelling in different tasks, into a single model that outperforms any of the individual source models. However, model merging faces two significant challenges: First, existing methods rely heavily on human knowledge or intuition. Second, it's difficult to obtain the great model merging configuration in limited evaluations. To address these challenges, we formalize model merging as a multi-objective optimization problem and propose an automated optimization approach named MM-MO. This method leverages multi-objective optimization algorithms to autonomously search for optimal merging configurations across various tasks, alleviating the need for human intervention. In MM-MO, a weak-to-strong method is employed to enhance the acquisition function, allowing previously evaluated superior configurations to guide the search for new ones. Meanwhile, Fisher information is applied to screen these configurations, increasing the possibility of identifying high-quality merging configuration. Additionally, we designed a sparsity metric as an additional optimization objective to enhance the model's generalization performance across different tasks. We conducted comprehensive experiments with other mainstream model merging methods, demonstrating that the proposed MM-MO algorithm is competitive and effective in achieving high-quality model merging.

cs.CL↗

arXiv2024

Approximating Multiple Robust Optimization Solutions in One Pass via Proximal Point Methods

Robust optimization provides a principled and unified framework to model many problems in modern operations research and computer science applications, such as risk measures minimization and adversarially robust machine learning. To use a robust solution (e.g., to implement an investment portfolio or perform robust machine learning inference), the user has to a priori decide the trade-off between efficiency (nominal performance) and robustness (worst-case performance) of the solution by choosing the uncertainty level hyperparameters. In many applications, this amounts to solving the problem many times and comparing them, each from a different hyperparameter setting. This makes robust optimization practically cumbersome or even intractable. We present a novel procedure based on the proximal point method (PPM) to efficiently approximate many Pareto efficient robust solutions at once. This effectively reduces the total compute requirement from $N \times T$ to $2 \times T$, where $N$ is the number of robust solutions to be generated, and $T$ is the time to obtain one robust solution. We prove this procedure can produce exact Pareto efficient robust solutions for a class of robust linear optimization problems. For more general problems, we prove that with high probability, our procedure gives a good approximation of the efficiency-robustness trade-off in random robust linear optimization instances. We conduct numerical experiments to demonstrate.

math.OC↗

arXiv2024

Evolutionary Retrosynthetic Route Planning

Molecular retrosynthesis is a significant and complex problem in the field of chemistry, however, traditional manual synthesis methods not only need well-trained experts but also are time-consuming. With the development of big data and machine learning, artificial intelligence (AI) based retrosynthesis is attracting more attention and has become a valuable tool for molecular retrosynthesis. At present, Monte Carlo tree search is a mainstream search framework employed to address this problem. Nevertheless, its search efficiency is compromised by its large search space. Therefore, this paper proposes a novel approach for retrosynthetic route planning based on evolutionary optimization, marking the first use of Evolutionary Algorithm (EA) in the field of multi-step retrosynthesis. The proposed method involves modeling the retrosynthetic problem into an optimization problem, defining the search space and operators. Additionally, to improve the search efficiency, a parallel strategy is implemented. The new approach is applied to four case products and compared with Monte Carlo tree search. The experimental results show that, in comparison to the Monte Carlo tree search algorithm, EA significantly reduces the number of calling single-step model by an average of 53.9%. The time required to search three solutions decreases by an average of 83.9%, and the number of feasible search routes increases by 1.38 times. The source code is available at https://github.com/ilog-ecnu/EvoRRP.

cs.AI↗

arXiv2024

Large Language Models as Surrogate Models in Evolutionary Algorithms: A Preliminary Study

Large Language Models (LLMs) have achieved significant progress across various fields and have exhibited strong potential in evolutionary computation, such as generating new solutions and automating algorithm design. Surrogate-assisted selection is a core step in evolutionary algorithms to solve expensive optimization problems by reducing the number of real evaluations. Traditionally, this has relied on conventional machine learning methods, leveraging historical evaluated evaluations to predict the performance of new solutions. In this work, we propose a novel surrogate model based purely on LLM inference capabilities, eliminating the need for training. Specifically, we formulate model-assisted selection as a classification and regression problem, utilizing LLMs to directly evaluate the quality of new solutions based on historical data. This involves predicting whether a solution is good or bad, or approximating its value. This approach is then integrated into evolutionary algorithms, termed LLM-assisted EA (LAEA). Detailed experiments compared the visualization results of 2D data from 9 mainstream LLMs, as well as their performance on optimization problems. The experimental results demonstrate that LLMs have significant potential as surrogate models in evolutionary computation, achieving performance comparable to traditional surrogate models only using inference. This work offers new insights into the application of LLMs in evolutionary computation. Code is available at: https://github.com/hhyqhh/LAEA.git

cs.NE↗

arXiv2024

A First Look at Kolmogorov-Arnold Networks in Surrogate-assisted Evolutionary Algorithms

Surrogate-assisted Evolutionary Algorithm (SAEA) is an essential method for solving expensive expensive problems. Utilizing surrogate models to substitute the optimization function can significantly reduce reliance on the function evaluations during the search process, thereby lowering the optimization costs. The construction of surrogate models is a critical component in SAEAs, with numerous machine learning algorithms playing a pivotal role in the model-building phase. This paper introduces Kolmogorov-Arnold Networks (KANs) as surrogate models within SAEAs, examining their application and effectiveness. We employ KANs for regression and classification tasks, focusing on the selection of promising solutions during the search process, which consequently reduces the number of expensive function evaluations. Experimental results indicate that KANs demonstrate commendable performance within SAEAs, effectively decreasing the number of function calls and enhancing the optimization efficiency. The relevant code is publicly accessible and can be found in the GitHub repository.

cs.NE↗

arXiv2024

Model Uncertainty in Evolutionary Optimization and Bayesian Optimization: A Comparative Analysis

Black-box optimization problems, which are common in many real-world applications, require optimization through input-output interactions without access to internal workings. This often leads to significant computational resources being consumed for simulations. Bayesian Optimization (BO) and Surrogate-Assisted Evolutionary Algorithm (SAEA) are two widely used gradient-free optimization techniques employed to address such challenges. Both approaches follow a similar iterative procedure that relies on surrogate models to guide the search process. This paper aims to elucidate the similarities and differences in the utilization of model uncertainty between these two methods, as well as the impact of model inaccuracies on algorithmic performance. A novel model-assisted strategy is introduced, which utilizes unevaluated solutions to generate offspring, leveraging the population-based search capabilities of evolutionary algorithm to enhance the effectiveness of model-assisted optimization. Experimental results demonstrate that the proposed approach outperforms mainstream Bayesian optimization algorithms in terms of accuracy and efficiency.

cs.NE↗

arXiv2023

Enhancing SAEAs with Unevaluated Solutions: A Case Study of Relation Model for Expensive Optimization

Surrogate-assisted evolutionary algorithms (SAEAs) hold significant importance in resolving expensive optimization problems~(EOPs). Extensive efforts have been devoted to improving the efficacy of SAEAs through the development of proficient model-assisted selection methods. However, generating high-quality solutions is a prerequisite for selection. The fundamental paradigm of evaluating a limited number of solutions in each generation within SAEAs reduces the variance of adjacent populations, thus impacting the quality of offspring solutions. This is a frequently encountered issue, yet it has not gained widespread attention. This paper presents a framework using unevaluated solutions to enhance the efficiency of SAEAs. The surrogate model is employed to identify high-quality solutions for direct generation of new solutions without evaluation. To ensure dependable selection, we have introduced two tailored relation models for the selection of the optimal solution and the unevaluated population. A comprehensive experimental analysis is performed on two test suites, which showcases the superiority of the relation model over regression and classification models in the selection phase. Furthermore, the surrogate-selected unevaluated solutions with high potential have been shown to significantly enhance the efficiency of the algorithm.

cs.NE↗

arXiv2023

Self-supervised Learning and Graph Classification under Heterophily

Self-supervised learning has shown its promising capability in graph representation learning in recent work. Most existing pre-training strategies usually choose the popular Graph neural networks (GNNs), which can be seen as a special form of low-pass filter, fail to effectively capture heterophily. In this paper, we first present an experimental investigation exploring the performance of low-pass and high-pass filters in heterophily graph classification, where the results clearly show that high-frequency signal is important for learning heterophily graph representation. On the other hand, it is still unclear how to effectively capture the structural pattern of graphs and how to measure the capability of the self-supervised pre-training strategy in capturing graph structure. To address the problem, we first design a quantitative metric to Measure Graph Structure (MGS), which analyzes correlation between structural similarity and embedding similarity of graph pairs. Then, to enhance the graph structural information captured by self-supervised learning, we propose a novel self-supervised strategy for Pre-training GNNs based on the Metric (PGM). Extensive experiments validate our pre-training strategy achieves state-of-the-art performance for molecular property prediction and protein function prediction. In addition, we find choosing the suitable filter sometimes may be better than designing good pre-training strategies for heterophily graph classification.

cs.LG↗

arXiv2022

A superconducting nanowire photon number resolving four-quadrant detector-based Gigabit deep-space laser communication receiver prototype

Deep space explorations require transferring huge amounts of data quickly from very distant targets. Laser communication is a promising technology that can offer a data rate of magnitude faster than conventional microwave communication due to the fundamentally narrow divergence of light. This study demonstrated a photon-sensitive receiver prototype with over Gigabit data rate, immunity to strong background photon noise, and simultaneous tracking ability. The advantages are inherited from a joint-optimized superconducting nanowire single-photon detector (SNSPD) array, designed into a four-quadrant structure with each quadrant capable of resolving six photons. Installed in a free-space coupled and low-vibration cryostat, the system detection efficiency reached 72.7%, the detector efficiency was 97.5%, and the total photon counting rate was 1.6 Gcps. Additionally, communication performance was tested for pulse position modulation (PPM) format. A series of signal processing methods were introduced to maximize the performance of the forward error correction (FEC) code. Consequently, the receiver exhibits a faster data rate and better sensitivity by about twofold (1.76 photons/bit at 800 Mbps and 3.40 photons/bit at 1.2 Gbps) compared to previously reported results (3.18 photon/bit at 622 Mbps for the Lunar Laser Communication Demonstration). Furthermore, communications in strong background noise and with simultaneous tracking ability were demonstrated aimed at the challenges of daylight operation and accurate tracking of dim beacon light in deep space scenarios.

physics.ins-det↗

arXiv2022

The Machine Learning for Combinatorial Optimization Competition (ML4CO): Results and Insights

Combinatorial optimization is a well-established area in operations research and computer science. Until recently, its methods have focused on solving problem instances in isolation, ignoring that they often stem from related data distributions in practice. However, recent years have seen a surge of interest in using machine learning as a new approach for solving combinatorial problems, either directly as solvers or by enhancing exact solvers. Based on this context, the ML4CO aims at improving state-of-the-art combinatorial optimization solvers by replacing key heuristic components. The competition featured three challenging tasks: finding the best feasible solution, producing the tightest optimality certificate, and giving an appropriate solver configuration. Three realistic datasets were considered: balanced item placement, workload apportionment, and maritime inventory routing. This last dataset was kept anonymous for the contestants.

cs.LG↗

arXiv2021

Improve photon number discrimination for a superconducting series nanowire detector by applying a digital matched filter

Photon number resolving (PNR) is an important capacity for detectors working in quantum and classical applications. Although a conventional superconducting nanowire single-photon detector (SNSPD) is not a PNR detector, by arranging nanowires in a series array and multiplexing photons over space, such series PNR-SNSPD can gain quasi-PNR capacity. However, the accuracy and maximum resolved photon number are both limited by the signal-to-noise (SNR) ratio of the output pulses. Here, we introduce a matched filter, which is an optimal filter in terms of SNR for SNSPD pulses. Experimentally, compared to conventional readout using a room-temperature amplifier, the normalized spacing between pulse amplitudes from adjacent photon number detections increased by a maximum factor of 2.1 after the matched filter. Combining with a cryogenic amplifier to increase SNR further, such spacing increased by a maximum factor of 5.3. In contrast to a low pass filter, the matched filter gave better SNRs while maintaining good timing jitters. Minimum timing jitter of 55 ps was obtained experimentally. Our results suggest that the matched filter is a useful tool for improving the performance of the series PNR-SNSPD and the maximum resolved photon number can be expected to reach 65 or even large.

quant-ph↗

arXiv2021

An optics-free computational spectrometer using a broadband and tunable dynamic detector

Optical spectrometers are the central instruments for exploring the interaction between light and matter. The current pursuit of the field is to design a spectrometer without the need for wavelength multiplexing optics to effectively reduce the complexity and physical size of the hardware. Based on computational spectroscopic results and combining a broadband-responsive dynamic detector, we successfully demonstrate an optics-free single-detector spectrometer that maps the tunable quantum efficiency of a superconducting nanowire into an ill-conditioned matrix to build a solvable inverse mathematical equation. Such a spectrometer can realize a broadband spectral responsivity ranging from 660 to 1900 nm. The spectral resolution at the telecom is 6 nm, exceeding the energy resolving capacity of existing infrared single-photon detectors. Meanwhile, benefiting from the optics-free setup, precise time-of-flight measurements can be simultaneously achieved. We have demonstrated a spectral LiDAR with 8 spectral channels. This work provides a concise method for building multifunctional spectrometers and paves the way for applying superconducting nanowire detectors in spectroscopy.

physics.optics↗

arXiv2020

Microfluidic device coupled with total internal reflection microscopy for in situ observation of precipitation

In situ observation of precipitation or phase separation induced by solvent addition is important in studying its dynamics. Combined with optical and fluorescence microscopy, microfluidic devices have been leveraged in studying the phase separation in various materials including biominerals, nanoparticles, and inorganic crystals. However, strong scattering from the subphases in the mixture is problematic for in situ study of phase separation with high temporal and spatial resolution. In this work, we present a quasi-2D microfluidic device combined with total internal reflection microscopy as an approach for in situ observation of phase separation. The quasi-2D microfluidic device comprises of a shallow main channel and a deep side channel. Mixing between a solution in the main channel (solution A) and another solution (solution B) in the side channel is predominantly driven by diffusion due to high fluid resistance from the shallow height of the main channel, which is confirmed using fluorescence microscopy. Moreover, relying on diffusive mixing, we can control the composition of the mixture in the main channel by tuning the composition of solution B. We demonstrate the application of our method for in situ observation of asphaltene precipitation and beta-alanine crystallization.

cond-mat.soft↗

arXiv2020

Oiling-out Crystallization of Beta-Alanine onSolid Surfaces Controlled by Solvent Exchange

Droplet formation in oiling-out crystallization has important implication for separation and purification of pharmaceutical active ingredients by using an antisolvent. In this work, we report the crystallization processes of oiling-out droplets on surfaces during solvent exchange. Our model ternary solution is beta-alanine dissolved in isopropanol and water mixture. As the antisolvent isopropanol displaced the alanine solution pre-filled in a microchamber, liquid-liquid phase separation occurred at the mixing front. The alanine-rich subphase formed surface microdroplets that subsequently crystallized with progression of solvent exchange. We find that the flow rates have significant influence on the droplet size, crystallization process, and growth rate, and final morphology of the crystals. At fast flow rates the droplets solidified rapidly and formed spherical-cap structures resembling the shape of droplets, in contrast to crystal microdomains or thin films formed at slow flow rates. On a highly hydrophilic surface, the crystals formed thin film without droplets formed on the surface. We further demonstrated that by the solvent exchange crystals can be formed by using a stock solution with a very low concentration of the precursor, and the as-prepared crystals can be used as seeds to trigger crystallization in bulk solution. Our results suggest that the solvent exchange has the potential to be an effective approach for controlling oiling-out crystallization, which can be applied in wide areas, such as separation and purification of many food, medical, and therapeutic ingredients.

cond-mat.soft↗