arXiv subjects

Hong Qian

Publications and source records attributed to Hong Qian.

At least 37 records · Page 2Linked to original sources

arXiv2025

LLMOPT: Learning to Define and Solve General Optimization Problems from Scratch

Optimization problems are prevalent across various scenarios. Formulating and then solving optimization problems described by natural language often requires highly specialized human expertise, which could block the widespread application of optimization-based decision making. To automate problem formulation and solving, leveraging large language models (LLMs) has emerged as a potential way. However, this kind of approach suffers from the issue of optimization generalization. Namely, the accuracy of most current LLM-based methods and the generality of optimization problem types that they can model are still limited. In this paper, we propose a unified learning-based framework called LLMOPT to boost optimization generalization. Starting from the natural language descriptions of optimization problems and a pre-trained LLM, LLMOPT constructs the introduced five-element formulation as a universal model for learning to define diverse optimization problem types. Then, LLMOPT employs the multi-instruction tuning to enhance both problem formalization and solver code generation accuracy and generality. After that, to prevent hallucinations in LLMs, such as sacrificing solving accuracy to avoid execution errors, the model alignment and self-correction mechanism are adopted in LLMOPT. We evaluate the optimization generalization ability of LLMOPT and compared methods across six real-world datasets covering roughly 20 fields such as health, environment, energy and manufacturing, etc. Extensive experiment results show that LLMOPT is able to model various optimization problem types such as linear/nonlinear programming, mixed integer programming, and combinatorial optimization, and achieves a notable 11.08% average solving accuracy improvement compared with the state-of-the-art methods. The code is available at https://github.com/caigaojiang/LLMOPT.

cs.AI↗

arXiv2025

Differentiating Student Feedbacks for Knowledge Tracing

Knowledge tracing (KT) is a crucial task in computer-aided education and intelligent tutoring systems, predicting students' performance on new questions from their responses to prior ones. An accurate KT model can capture a student's mastery level of different knowledge topics, as reflected in their predicted performance on different questions. This helps improve the learning efficiency by suggesting appropriate new questions that complement students' knowledge states. However, current KT models have significant drawbacks that they neglect the imbalanced discrimination of historical responses. A significant proportion of question responses provide limited information for discerning students' knowledge mastery, such as those that demonstrate uniform performance across different students. Optimizing the prediction of these cases may increase overall KT accuracy, but also negatively impact the model's ability to trace personalized knowledge states, especially causing a deceptive surge of performance. Towards this end, we propose a framework to reweight the contribution of different responses based on their discrimination in training. Additionally, we introduce an adaptive predictive score fusion technique to maintain accuracy on less discriminative responses, achieving proper balance between student knowledge mastery and question difficulty. Experimental results demonstrate that our framework enhances the performance of three mainstream KT methods on three widely-used datasets.

cs.CY↗

arXiv2025

Emergence of Newtonian Deterministic Causality from Stochastic Motions in Continuous Space and Time

Since Newton's time, deterministic causality has been considered a crucial prerequisite in any fundamental theory in physics. In contrast, the present work investigates stochastic dynamical models for motion in one spatial dimension, in which Newtonian mechanics becomes an emergent property: We present a coherent theory in which a Hamilton-Jacobi equation (HJE) emerges in a description of the evolution of entropy $-ϕ(x,t)=ε\log$(Probability) of a system under observation and in the limit of large information extent $ε^{-1}$ in homogeneous space and time. The variable $ϕ$ represents a non-random high-order statistical concept that is distinct from probability itself as $ε=0$; the HJE embodies an emergent law of deterministic causality in continuous space and time with an Imaginary Scale symmetry $(t,x,ϕ)\leftrightarrow (it,ix,-iϕ)$. $ϕ(x,t)$ exhibits a nonlinear wave phenomenon with a mathematical singularity in finite time, overcoming which we introduce viscosity $ε(\partial^2ϕ/\partial x^2)$ and wave $iε(\partial^2 ϕ/\partial x^2)$ perturbations, articulating dissipation and conservation, which break the Imaginary Scale symmetry: They lead to the Brownian motion and Schrödinger's equation of motion, respectively. Last but not least, Lagrange's action in classical mechanics acquires an entropic interpretation and Hamilton's principle is established.

cond-mat.stat-mech↗

arXiv2025

Context-aware Diversity Enhancement for Neural Multi-Objective Combinatorial Optimization

Multi-objective combinatorial optimization (MOCO) problems are prevalent in various real-world applications. Most existing neural MOCO methods rely on problem decomposition to transform an MOCO problem into a series of singe-objective combinatorial optimization (SOCO) problems and train attention models based on a single-step and deterministic greedy rollout. However, inappropriate decomposition and undesirable short-sighted behaviors of previous methods tend to induce a decline in diversity. To address the above limitation, we design a Context-aware Diversity Enhancement algorithm named CDE, which casts the neural MOCO problems as conditional sequence modeling via autoregression (node-level context awareness) and establishes a direct relationship between the mapping of preferences and diversity indicator of reward based on hypervolume expectation maximization (solution-level context awareness). Based on the solution-level context awareness, we further propose a hypervolume residual update strategy to enable the Pareto attention model to capture both local and non-local information of the Pareto set/front. The proposed CDE can effectively and efficiently grasp the context information, resulting in diversity enhancement. Experimental results on three classic MOCO problems demonstrate that our CDE outperforms several state-of-the-art baselines.

cs.LG↗

arXiv2025

Language Representation Favored Zero-Shot Cross-Domain Cognitive Diagnosis

Cognitive diagnosis aims to infer students' mastery levels based on their historical response logs. However, existing cognitive diagnosis models (CDMs), which rely on ID embeddings, often have to train specific models on specific domains. This limitation may hinder their directly practical application in various target domains, such as different subjects (e.g., Math, English and Physics) or different education platforms (e.g., ASSISTments, Junyi Academy and Khan Academy). To address this issue, this paper proposes the language representation favored zero-shot cross-domain cognitive diagnosis (LRCD). Specifically, LRCD first analyzes the behavior patterns of students, exercises and concepts in different domains, and then describes the profiles of students, exercises and concepts using textual descriptions. Via recent advanced text-embedding modules, these profiles can be transformed to vectors in the unified language space. Moreover, to address the discrepancy between the language space and the cognitive diagnosis space, we propose language-cognitive mappers in LRCD to learn the mapping from the former to the latter. Then, these profiles can be easily and efficiently integrated and trained with existing CDMs. Extensive experiments show that training LRCD on real-world datasets can achieve commendable zero-shot performance across different target domains, and in some cases, it can even achieve competitive performance with some classic CDMs trained on the full response data on target domains. Notably, we surprisingly find that LRCD can also provide interesting insights into the differences between various subjects (such as humanities and sciences) and sources (such as primary and secondary education).

cs.CL↗

arXiv2024

It's Morphing Time: Unleashing the Potential of Multiple LLMs via Multi-objective Optimization

In this paper, we introduce a novel approach for addressing the multi-objective optimization problem in large language model merging via black-box multi-objective optimization algorithms. The goal of model merging is to combine multiple models, each excelling in different tasks, into a single model that outperforms any of the individual source models. However, model merging faces two significant challenges: First, existing methods rely heavily on human knowledge or intuition. Second, it's difficult to obtain the great model merging configuration in limited evaluations. To address these challenges, we formalize model merging as a multi-objective optimization problem and propose an automated optimization approach named MM-MO. This method leverages multi-objective optimization algorithms to autonomously search for optimal merging configurations across various tasks, alleviating the need for human intervention. In MM-MO, a weak-to-strong method is employed to enhance the acquisition function, allowing previously evaluated superior configurations to guide the search for new ones. Meanwhile, Fisher information is applied to screen these configurations, increasing the possibility of identifying high-quality merging configuration. Additionally, we designed a sparsity metric as an additional optimization objective to enhance the model's generalization performance across different tasks. We conducted comprehensive experiments with other mainstream model merging methods, demonstrating that the proposed MM-MO algorithm is competitive and effective in achieving high-quality model merging.

cs.CL↗

arXiv2024

Estimate exponential memory decay in Hidden Markov Model and its applications

Inference in hidden Markov model has been challenging in terms of scalability due to dependencies in the observation data. In this paper, we utilize the inherent memory decay in hidden Markov models, such that the forward and backward probabilities can be carried out with subsequences, enabling efficient inference over long sequences of observations. We formulate this forward filtering process in the setting of the random dynamical system and there exist Lyapunov exponents in the i.i.d random matrices production. And the rate of the memory decay is known as $λ_2-λ_1$, the gap of the top two Lyapunov exponents almost surely. An efficient and accurate algorithm is proposed to numerically estimate the gap after the soft-max parametrization. The length of subsequences $B$ given the controlled error $ε$ is $B=\log(ε)/(λ_2-λ_1)$. We theoretically prove the validity of the algorithm and demonstrate the effectiveness with numerical examples. The method developed here can be applied to widely used algorithms, such as mini-batch stochastic gradient method. Moreover, the continuity of Lyapunov spectrum ensures the estimated $B$ could be reused for the nearby parameter during the inference.

stat.ML↗

arXiv2024

A Dual-Fusion Cognitive Diagnosis Framework for Open Student Learning Environments

Cognitive diagnosis model (CDM) is a fundamental and upstream component in intelligent education. It aims to infer students' mastery levels based on historical response logs. However, existing CDMs usually follow the ID-based embedding paradigm, which could often diminish the effectiveness of CDMs in open student learning environments. This is mainly because they can hardly directly infer new students' mastery levels or utilize new exercises or knowledge without retraining. Textual semantic information, due to its unified feature space and easy accessibility, can help alleviate this issue. Unfortunately, directly incorporating semantic information may not benefit CDMs, since it does not capture response-relevant features and thus discards the individual characteristics of each student. To this end, this paper proposes a dual-fusion cognitive diagnosis framework (DFCD) to address the challenge of aligning two different modalities, i.e., textual semantic features and response-relevant features. Specifically, in DFCD, we first propose the exercise-refiner and concept-refiner to make the exercises and knowledge concepts more coherent and reasonable via large language models. Then, DFCD encodes the refined features using text embedding models to obtain the semantic information. For response-related features, we propose a novel response matrix to fully incorporate the information within the response logs. Finally, DFCD designs a dual-fusion module to merge the two modal features. The ultimate representations possess the capability of inference in open student learning environments and can be also plugged in existing CDMs. Extensive experiments across real-world datasets show that DFCD achieves superior performance by integrating different modalities and strong adaptability in open student learning environments.

cs.AI↗

arXiv2024

ORCDF: An Oversmoothing-Resistant Cognitive Diagnosis Framework for Student Learning in Online Education Systems

Cognitive diagnosis models (CDMs) are designed to learn students' mastery levels using their response logs. CDMs play a fundamental role in online education systems since they significantly influence downstream applications such as teachers' guidance and computerized adaptive testing. Despite the success achieved by existing CDMs, we find that they suffer from a thorny issue that the learned students' mastery levels are too similar. This issue, which we refer to as oversmoothing, could diminish the CDMs' effectiveness in downstream tasks. CDMs comprise two core parts: learning students' mastery levels and assessing mastery levels by fitting the response logs. This paper contends that the oversmoothing issue arises from that existing CDMs seldom utilize response signals on exercises in the learning part but only use them as labels in the assessing part. To this end, this paper proposes an oversmoothing-resistant cognitive diagnosis framework (ORCDF) to enhance existing CDMs by utilizing response signals in the learning part. Specifically, ORCDF introduces a novel response graph to inherently incorporate response signals as types of edges. Then, ORCDF designs a tailored response-aware graph convolution network (RGC) that effectively captures the crucial response signals within the response graph. Via ORCDF, existing CDMs are enhanced by replacing the input embeddings with the outcome of RGC, allowing for the consideration of response signals on exercises in the learning part. Extensive experiments on real-world datasets show that ORCDF not only helps existing CDMs alleviate the oversmoothing issue but also significantly enhances the models' prediction and interpretability performance. Moreover, the effectiveness of ORCDF is validated in the downstream task of computerized adaptive testing.

cs.CY↗

arXiv2024

Leveraging Pedagogical Theories to Understand Student Learning Process with Graph-based Reasonable Knowledge Tracing

Knowledge tracing (KT) is a crucial task in intelligent education, focusing on predicting students' performance on given questions to trace their evolving knowledge. The advancement of deep learning in this field has led to deep-learning knowledge tracing (DLKT) models that prioritize high predictive accuracy. However, many existing DLKT methods overlook the fundamental goal of tracking students' dynamical knowledge mastery. These models do not explicitly model knowledge mastery tracing processes or yield unreasonable results that educators find difficulty to comprehend and apply in real teaching scenarios. In response, our research conducts a preliminary analysis of mainstream KT approaches to highlight and explain such unreasonableness. We introduce GRKT, a graph-based reasonable knowledge tracing method to address these issues. By leveraging graph neural networks, our approach delves into the mutual influences of knowledge concepts, offering a more accurate representation of how the knowledge mastery evolves throughout the learning process. Additionally, we propose a fine-grained and psychological three-stage modeling process as knowledge retrieval, memory strengthening, and knowledge learning/forgetting, to conduct a more reasonable knowledge tracing process. Comprehensive experiments demonstrate that GRKT outperforms eleven baselines across three datasets, not only enhancing predictive accuracy but also generating more reasonable knowledge tracing results. This makes our model a promising advancement for practical implementation in educational settings. The source code is available at https://github.com/JJCui96/GRKT.

cs.AI↗

arXiv2024

Expensive Multi-Objective Bayesian Optimization Based on Diffusion Models

Multi-objective Bayesian optimization (MOBO) has shown promising performance on various expensive multi-objective optimization problems (EMOPs). However, effectively modeling complex distributions of the Pareto optimal solutions is difficult with limited function evaluations. Existing Pareto set learning algorithms may exhibit considerable instability in such expensive scenarios, leading to significant deviations between the obtained solution set and the Pareto set (PS). In this paper, we propose a novel Composite Diffusion Model based Pareto Set Learning algorithm, namely CDM-PSL, for expensive MOBO. CDM-PSL includes both unconditional and conditional diffusion model for generating high-quality samples. Besides, we introduce an information entropy based weighting method to balance different objectives of EMOPs. This method is integrated with the guiding strategy, ensuring that all the objectives are appropriately balanced and given due consideration during the optimization process; Extensive experimental results on both synthetic benchmarks and real-world problems demonstrates that our proposed algorithm attains superior performance compared with various state-of-the-art MOBO algorithms.

cs.LG↗

arXiv2024

Nonequilibrium and nonlinear kinetics as key determinants for bistability in fission yeast G2-M transition

A living cell is an open, nonequilibrium biochemical system where ATP hydrolysis serves as the energy source for a wide range of intracellular processes, possibly including the assurance for decision-making. In the fission yeast cell cycle, the transition from G2 to M phase is driven by the activation of Cdc13/Cdc2 and Cdc25 and the deactivation of Wee1 through phosphorylation-dephosphorylation cycles with feedback loops. Here, we present a kinetic description of the G2-M circuit which reveals that both cellular ATP level and ATP hydrolysis free energy critically control Cdc2 activation. Using fission yeast nucleoplasmic extract (YNPE), we experimentally verify that increased ATP level drives the activation of Cdc2 which exhibits bistability and hysteresis in response to changes in cellular ATP level and ATP hydrolysis energy. These findings suggest that cellular ATP level and ATP hydrolysis energy are determinants of the bistability and robustness of Cdc2 activation during G2-M transition.

q-bio.MN↗

arXiv2024

Inductive Cognitive Diagnosis for Fast Student Learning in Web-Based Online Intelligent Education Systems

Cognitive diagnosis aims to gauge students' mastery levels based on their response logs. Serving as a pivotal module in web-based online intelligent education systems (WOIESs), it plays an upstream and fundamental role in downstream tasks like learning item recommendation and computerized adaptive testing. WOIESs are open learning environment where numerous new students constantly register and complete exercises. In WOIESs, efficient cognitive diagnosis is crucial to fast feedback and accelerating student learning. However, the existing cognitive diagnosis methods always employ intrinsically transductive student-specific embeddings, which become slow and costly due to retraining when dealing with new students who are unseen during training. To this end, this paper proposes an inductive cognitive diagnosis model (ICDM) for fast new students' mastery levels inference in WOIESs. Specifically, in ICDM, we propose a novel student-centered graph (SCG). Rather than inferring mastery levels through updating student-specific embedding, we derive the inductive mastery levels as the aggregated outcomes of students' neighbors in SCG. Namely, SCG enables to shift the task from finding the most suitable student-specific embedding that fits the response logs to finding the most suitable representations for different node types in SCG, and the latter is more efficient since it no longer requires retraining. To obtain this representation, ICDM consists of a construction-aggregation-generation-transformation process to learn the final representation of students, exercises and concepts. Extensive experiments across real-world datasets show that, compared with the existing cognitive diagnosis methods that are always transductive, ICDM is much more faster while maintains the competitive inference performance for new students.

cs.AI↗

arXiv2024

Exact power spectrum in a minimal hybrid model of stochastic gene expression oscillations

Stochastic oscillations in individual cells are usually characterized by a non-monotonic power spectrum with an oscillatory autocorrelation function. Here we develop an analytical approach of stochastic oscillations in a minimal hybrid model of stochastic gene expression including promoter state switching, protein synthesis and degradation, as well as a genetic feedback loop. The oscillations observed in our model are noise-induced since the deterministic theory predicts stable fixed points. The autocorrelated function, power spectrum, and steady-state distribution of protein concentration fluctuations are computed in closed form without making any approximations. Using the exactly solvable model, we illustrate sustained oscillations as a circular motion along a stochastic hysteresis loop induced by gene state switching. A triphasic stochastic bifurcation upon the increasing strength of negative feedback is observed, which reveals how stochastic bursts evolve into stochastic oscillations. In our model, oscillations tend to occur when the protein is relatively stable and when gene switching is relatively slow. Translational bursting is found to enhance the robustness and broaden the region of stochastic oscillations. These results provide deeper insights into R. Thomas' two conjectures for single-cell gene expression kinetics.

q-bio.MN↗

arXiv2023

Symbolic Cognitive Diagnosis via Hybrid Optimization for Intelligent Education Systems

Cognitive diagnosis assessment is a fundamental and crucial task for student learning. It models the student-exercise interaction, and discovers the students' proficiency levels on each knowledge attribute. In real-world intelligent education systems, generalization and interpretability of cognitive diagnosis methods are of equal importance. However, most existing methods can hardly make the best of both worlds due to the complicated student-exercise interaction. To this end, this paper proposes a symbolic cognitive diagnosis~(SCD) framework to simultaneously enhance generalization and interpretability. The SCD framework incorporates the symbolic tree to explicably represent the complicated student-exercise interaction function, and utilizes gradient-based optimization methods to effectively learn the student and exercise parameters. Meanwhile, the accompanying challenge is that we need to tunnel the discrete symbolic representation and continuous parameter optimization. To address this challenge, we propose to hybridly optimize the representation and parameters in an alternating manner. To fulfill SCD, it alternately learns the symbolic tree by derivative-free genetic programming and learns the student and exercise parameters via gradient-based Adam. The extensive experimental results on various real-world datasets show the superiority of SCD on both generalization and interpretability. The ablation study verifies the efficacy of each ingredient in SCD, and the case study explicitly showcases how the interpretable ability of SCD works.

cs.CY↗

arXiv2023

Statistical Uncertainty Principle in Stochastic Dynamics

Maximum entropy principle identifies forces conjugated to observables and the thermodynamic relations between them, independent upon their underlying mechanistic details. For data about state distributions or transition statistics, the principle can be derived from limit theorems of infinite data sampling. This derivation reveals its empirical origin and clarify the meaning of applying it to large but finite data. We derive an uncertainty principle for the statistical variations of the observables and the inferred forces. We use a toy model for molecular motor as an example.

cond-mat.stat-mech↗

arXiv2023

On Thermodynamic Information

Information based thermodynamic logic is revisited. It consists of two parts: Part A applies the modern theory of probability in which an arbitrary convex function ϕis employed as an analytic "device" to express information as statistical dependency contained in the topological sub-σ-algebra structure. Via thermo-doubling, Fenchel-Young equality (FYE) that consists of ϕ(x) and its conjugate ψ(y) establishes the notion of equilibrium between x and y through duality symmetry and the principle of maximum entropy/minimum free energy. Part B deals with a given set of repetitive measurements, where an inherent convex function emerges via the mathematics of large deviations. Logarithm-based Shannon entropy with ϕ(x)=-\log x figures prominently for i.i.d. sample statistics. Information can be a measure of the agreement between a statistical observation and its theoretical models. Maximum likelihood principle arises here and FYE provides a thermodynamic energetic narrative of recurrent data.

cond-mat.stat-mech↗

arXiv2023

On the Opportunities of Green Computing: A Survey

Artificial Intelligence (AI) has achieved significant advancements in technology and research with the development over several decades, and is widely used in many areas including computing vision, natural language processing, time-series analysis, speech synthesis, etc. During the age of deep learning, especially with the arise of Large Language Models, a large majority of researchers' attention is paid on pursuing new state-of-the-art (SOTA) results, resulting in ever increasing of model size and computational complexity. The needs for high computing power brings higher carbon emission and undermines research fairness by preventing small or medium-sized research institutions and companies with limited funding in participating in research. To tackle the challenges of computing resources and environmental impact of AI, Green Computing has become a hot research topic. In this survey, we give a systematic overview of the technologies used in Green Computing. We propose the framework of Green Computing and devide it into four key components: (1) Measures of Greenness, (2) Energy-Efficient AI, (3) Energy-Efficient Computing Systems and (4) AI Use Cases for Sustainability. For each components, we discuss the research progress made and the commonly used techniques to optimize the AI efficiency. We conclude that this new research direction has the potential to address the conflicts between resource constraints and AI development. We encourage more researchers to put attention on this direction and make AI more environmental friendly.

cs.AI↗