arXiv subjects

Minghui Sun

Publications and source records attributed to Minghui Sun.

14 recordsLinked to original sources

arXiv2026

Multimodal Training to Unimodal Deployment: Leveraging Unstructured Data During Training to Optimize Structured Data Only Deployment

Unstructured Electronic Health Record (EHR) data, such as clinical notes, contain clinical contextual observations that are not directly reflected in structured data fields. This additional information can substantially improve model learning. However, due to their unstructured nature, these data are often unavailable or impractical to use when deploying a model. We introduce a multimodal learning framework that leverages unstructured EHR data during training while producing a model that can be deployed using only structured EHR data. Using a cohort of 3,466 children evaluated for late talking, we generated note embeddings with BioClinicalBERT and encoded structured embeddings from demographics and medical codes. A note-based teacher model and a structured-only student model were jointly trained using contrastive learning and contrastive knowledge distillation loss, producing a strong classifier (AUROC = 0.985). Our proposed model reached an AUROC of 0.705, outperforming the structured-only baseline of 0.656. These results demonstrate that incorporating unstructured data during training enhances the model's capacity to identify task-relevant information within structured EHR data, enabling a deployable structured-only phenotype model.

cs.LG↗

arXiv2026

NEST: Nested Event Stream Transformer for Sequences of Multisets

Event stream data often exhibit hierarchical structure in which multiple events co-occur, resulting in a sequence of multisets (i.e., bags of events). In electronic health records (EHRs), for example, medical events are grouped into a sequence of clinical encounters with well-defined temporal structure, but the order and timing of events within each encounter may be unknown or unreliable. Most existing foundation models (FMs) for event stream data flatten this hierarchy into a one-dimensional sequence, leading to (i) computational inefficiency associated with dense attention and learning spurious within-set relationships, and (ii) lower-quality set-level representations from heuristic post-training pooling for downstream tasks. Here, we show that preserving the original hierarchy in the FM architecture provides a useful inductive bias that improves both computational efficiency and representation quality. We then introduce Nested Event Stream Transformer (NEST), a FM for event streams comprised of sequences of multisets. Building on this architecture, we formulate Masked Set Modeling (MSM), an efficient paradigm that promotes improved set-level representation learning. Experiments on real-world multiset sequence data show that NEST captures real-world dynamics while improving both pretraining efficiency and downstream performance.

cs.LG↗

arXiv2026

DeepH-pack: A general-purpose neural network package for deep-learning electronic structure calculations

In computational physics and materials science, first-principles methods, particularly density functional theory, have become central tools for electronic structure prediction and materials design. Recently, rapid advances in artificial intelligence (AI) have begun to reshape the research landscape, giving rise to the emerging field of deep-learning electronic structure calculations. Despite numerous pioneering studies, the field remains in its early stages; existing software implementations are often fragmented, lacking unified frameworks and standardized interfaces required for broad community adoption. Here we present DeepH-pack, a comprehensive and unified software package that integrates first-principles calculations with deep learning. By incorporating fundamental physical principles into neural-network design, such as the nearsightedness principle and the equivariance principle, DeepH-pack achieves robust cross-scale and cross-material generalizability. This allows models trained on small-scale structures to generalize to large-scale and previously unseen materials. The toolkit preserves first-principles accuracy while accelerating electronic structure calculations by several orders of magnitude, establishing an efficient and intelligent computational paradigm for large-scale materials simulation, high-throughput materials database construction, and AI-driven materials discovery.

cond-mat.mtrl-sci↗

arXiv2025

Operator-Consistent Physics-Informed Learning for Wafer Thermal Reconstruction in Lithography

Thermal field reconstruction in post-exposure bake (PEB) is critical for advanced lithography, yet current physics-informed neural networks (PINNs) suffer from inconsistent accuracy due to a misalignment between geometric coordinates, physical fields, and differential operators. To resolve this, we introduce a novel architecture that unifies these elements on a single computation graph by integrating LSTM-gated mechanisms within a Liquid Neural Network (LNN) backbone. This specific combination of gated liquid layers is necessary to dynamically regulate the network's spectral behavior and enforce operator-level consistency, which ensures stable training and high-fidelity predictions. Applied to a 2D PEB scenario with internal heat generation and convective boundaries, our model formulates residuals via differential forms and a composite loss functional. The results demonstrate rapid convergence, uniformly low errors, strong agreement with FEM benchmarks, and stable training without late-stage oscillations, outperforming existing baselines in accuracy and robustness. Our framework thus establishes a reliable foundation for high-fidelity thermal modeling and offers a transferable strategy for operator-consistent neural surrogates in other physical domains.

math-ph↗

arXiv2025

Borrowing From the Future: Enhancing Early Risk Assessment through Contrastive Learning

Risk assessments for a pediatric population are often conducted across multiple stages. For example, clinicians may evaluate risks prenatally, at birth, and during Well-Child visits. Although predictions made at later stages typically achieve higher precision, it is clinically desirable to make reliable risk assessments as early as possible. Therefore, this study focuses on improving prediction performance in early-stage risk assessments. Our solution, \textbf{Borrowing From the Future (BFF)}, is a contrastive multi-modal framework that treats each time window as a distinct modality. In BFF, a model is trained on all available data throughout the time while performing a risk assessment using up-to-date information. This contrastive framework allows the model to ``borrow'' informative signals from later stages (e.g., Well-Child visits) to implicitly supervise the learning at earlier stages (e.g., prenatal/birth stages). We validate BFF on two real-world pediatric outcome prediction tasks, demonstrating consistent improvements in early risk assessments. The code is available at https://github.com/scotsun/bff.

cs.LG↗

arXiv2025

CLEAR: Unlearning Spurious Style-Content Associations with Contrastive LEarning with Anti-contrastive Regularization

Learning representations unaffected by superficial characteristics is important to ensure that shifts in these characteristics at test time do not compromise downstream prediction performance. For instance, in healthcare applications, we might like to learn features that contain information about pathology yet are unaffected by race, sex, and other sources of physiologic variability, thereby ensuring predictions are equitable and generalizable across all demographics. Here we propose Contrastive LEarning with Anti-contrastive Regularization (CLEAR), an intuitive and easy-to-implement framework that effectively separates essential (i.e., task-relevant) characteristics from superficial (i.e., task-irrelevant) characteristics during training, leading to better performance when superficial characteristics shift at test time. We begin by supposing that data representations can be semantically separated into task-relevant content features, which contain information relevant to downstream tasks, and task-irrelevant style features, which encompass superficial attributes that are irrelevant to these tasks, yet may degrade performance due to associations with content present in training data that do not generalize. We then prove that our anti-contrastive penalty, which we call Pair-Switching (PS), minimizes the Mutual Information between the style attributes and content labels. Finally, we instantiate CLEAR in the latent space of a Variational Auto-Encoder (VAE), then perform experiments to quantitatively and qualitatively evaluate the resulting CLEAR-VAE over several image datasets. Our results show that CLEAR-VAE allows us to: (a) swap and interpolate content and style between any pair of samples, and (b) improve downstream classification performance in the presence of previously unseen combinations of content and style. Our code will be made publicly available.

cs.LG↗

arXiv2024

Deep Band Crossings Enhanced Nonlinear Optical Effects

Nonlinear optical (NLO) effects in materials with band crossings have attracted significant research interests due to the divergent band geometric quantities around these crossings. Most current research has focused on band crossings between the valence and conduction bands. However, such crossings are absent in insulators, which are more relevant for NLO applications. In this work, we demonstrate that NLO effects can be significantly enhanced by band crossings within the valence or conduction bands, which we designate as "deep band crossings" (DBCs). As an example, in two dimensions, we show that shift conductivity can be substantially enhanced or even divergent due to a mirror-protected "deep Dirac nodal point". In three dimensions, we propose GeTe as an ideal material where shift conductivity is enhanced by "deep Dirac nodal lines". The ubiquity of this enhancement is further confirmed by high-throughput calculations. Other types of DBCs and NLO effects are also discussed. By manipulating band crossings between arbitrary bands, our work offers a simple, practical, and universal way to greatly enhance NLO effects.

cond-mat.mes-hall↗

arXiv2024

Deep learning density functional theory Hamiltonian in real space

Deep learning electronic structures from ab initio calculations holds great potential to revolutionize computational materials studies. While existing methods proved success in deep-learning density functional theory (DFT) Hamiltonian matrices, they are limited to DFT programs using localized atomic-like bases and heavily depend on the form of the bases. Here, we propose the DeepH-r method for deep-learning DFT Hamiltonians in real space, facilitating the prediction of DFT Hamiltonian in a basis-independent manner. An equivariant neural network architecture for modeling the real-space DFT potential is developed, targeting a more fundamental quantity in DFT. The real-space potential exhibits simplified principles of equivariance and enhanced nearsightedness, further boosting the performance of deep learning. When applied to evaluate the Hamiltonian matrix, this method significantly improved in accuracy, as exemplified in multiple case studies. Given the abundance of data in the real-space potential, this work may pave a novel pathway for establishing a ``large materials model" with increased accuracy.

physics.comp-ph↗

arXiv2024

Improving density matrix electronic structure method by deep learning

The combination of deep learning and ab initio materials calculations is emerging as a trending frontier of materials science research, with deep-learning density functional theory (DFT) electronic structure being particularly promising. In this work, we introduce a neural-network method for modeling the DFT density matrix, a fundamental yet previously unexplored quantity in deep-learning electronic structure. Utilizing an advanced neural network framework that leverages the nearsightedness and equivariance properties of the density matrix, the method demonstrates high accuracy and excellent generalizability in multiple example studies, as well as capability to precisely predict charge density and reproduce other electronic structure properties. Given the pivotal role of the density matrix in DFT as well as other computational methods, the current research introduces a novel approach to the deep-learning study of electronic structure properties, opening up new opportunities for deep-learning enhanced computational materials study.

physics.comp-ph↗

arXiv2024

Neural-network Density Functional Theory Based on Variational Energy Minimization

Deep-learning density functional theory (DFT) shows great promise to significantly accelerate material discovery and potentially revolutionize materials research. However, current research in this field primarily relies on data-driven supervised learning, making the developments of neural networks and DFT isolated from each other. In this work, we present a theoretical framework of neural-network DFT, which unifies the optimization of neural networks with the variational computation of DFT, enabling physics-informed unsupervised learning. Moreover, we develop a differential DFT code incorporated with deep-learning DFT Hamiltonian, and introduce algorithms of automatic differentiation and backpropagation into DFT, demonstrating the capability of neural-network DFT. The physics-informed neural-network architecture not only surpasses conventional approaches in accuracy and efficiency, but also offers a new paradigm for developing deep-learning DFT methods.

physics.comp-ph↗

arXiv2023

NoiseTrans: Point Cloud Denoising with Transformers

Point clouds obtained from capture devices or 3D reconstruction techniques are often noisy and interfere with downstream tasks. The paper aims to recover the underlying surface of noisy point clouds. We design a novel model, NoiseTrans, which uses transformer encoder architecture for point cloud denoising. Specifically, we obtain structural similarity of point-based point clouds with the assistance of the transformer's core self-attention mechanism. By expressing the noisy point cloud as a set of unordered vectors, we convert point clouds into point embeddings and employ Transformer to generate clean point clouds. To make the Transformer preserve details when sensing the point cloud, we design the Local Point Attention to prevent the point cloud from being over-smooth. In addition, we also propose sparse encoding, which enables the Transformer to better perceive the structural relationships of the point cloud and improve the denoising performance. Experiments show that our model outperforms state-of-the-art methods in various datasets and noise environments.

cs.CV↗

arXiv2022

Functional requirements decomposition in set-based design

Designing systems is typically uncertain and ambiguous at early stages. Set-based design supports alternative exploration and gradual uncertainty reduction during the early lifecycle, making it practical for complex systems design. In parallel, the functional requirements decomposition helps to advance the design incrementally. However, current literature on set-based design lacks formal guidance in how to decompose functional requirements. To bridge this gap, we introduce a four-step method to decompose functional requirements for set-based design hierarchically. We systematically define, reason, and narrow the sets, breaking down the functional requirements into formal sub-requirements. This method allows parallel abstraction, ensuring the resulting system satisfies the top-level functional requirements.

eess.SY↗

arXiv2022

A new safety-guided design methodology to complement model-based safety analysis for safety assurance

With the rapid advancement of Formal Methods, Model-based Safety Analysis (MBSA) has been gaining tremendous attention for its ability to rigorously verify whether the safety-critical scenarios are adequately addressed by the design solution of a cyber-physical human system. However, there is a gap. If specific safety-critical scenarios are not included in the given design solution (i.e., the model) in the first place, the results of MBSA cannot be trusted for safety assurance. To tackle this problem, we propose a new safety-guided design methodology (called STPA+) to complement MBSA. Inspired by STPA, STPA+ treats a system as a control structure, which is particularly fit for systems with complex interactions between human, machine, and automation. Three methods are developed in STPA+ to tackle the possible omissions of safety-critical scenarios caused by incorrectly defined safety constraints, improperly constrained process model, and inadequately designed controller. In this way, STPA+ directly derives an adequately defined design solution as the input to an MBSA verification program and bridges the gap between current MBSA approaches and safety assurance.

eess.SY↗

arXiv2019

Correct-by-construction requirement decomposition

In systems engineering, accurately decomposing requirements is crucial for creating well-defined and manageable system components, particularly in safety-critical domains. Despite the critical need, rigorous, top-down methodologies for effectively breaking down complex requirements into precise, actionable sub-requirements are scarce, especially compared to the wealth of bottom-up verification techniques. Addressing this gap, we introduce a formal decomposition for contract-based design that guarantees the correctness of decomposed requirements if specific conditions are met. Our (semi-)automated methodology augments contract-based design with reachability analysis and constraint programming to systematically identify, verify, and validate sub-requirements representable by continuous bounded sets -- continuous relations between real-valued inputs and outputs. We demonstrate the efficacy and practicality of a correct-by-construction approach through a comprehensive case study on a cruise control system, highlighting how our methodology improves the interpretability, tractability, and verifiability of system requirements.

eess.SY↗