arXiv subjects

Yuxuan Liu

Publications and source records attributed to Yuxuan Liu.

At least 163 records · Page 9Linked to original sources

arXiv2024

Disentangle Estimation of Causal Effects from Cross-Silo Data

Estimating causal effects among different events is of great importance to critical fields such as drug development. Nevertheless, the data features associated with events may be distributed across various silos and remain private within respective parties, impeding direct information exchange between them. This, in turn, can result in biased estimations of local causal effects, which rely on the characteristics of only a subset of the covariates. To tackle this challenge, we introduce an innovative disentangle architecture designed to facilitate the seamless cross-silo transmission of model parameters, enriched with causal mechanisms, through a combination of shared and private branches. Besides, we introduce global constraints into the equation to effectively mitigate bias within the various missing domains, thereby elevating the accuracy of our causal effect estimation. Extensive experiments conducted on new semi-synthetic datasets show that our method outperforms state-of-the-art baselines.

cs.LG↗

arXiv2023

Signature of Topological Semimetal in Harmonic-honeycomb ReO3

Transition-metal honeycomb compounds are capturing scientific attention due to their distinctive electronic configurations, underscored by the triangular-lattice spin-orbit coupling and competition between multiple interactions, paving the way for potential manifestations of phenomena such as Dirac semimetal, superconductivity, and quantum spin liquid states. These compounds can undergo discernible pressure-induced alterations in their crystallographic and electronic paradigms, as exemplified by our high-pressure (HP) synthesis and exploration of the honeycomb polymorph of ReO3 (P6322). This HP-P6322 polymorph bears a phase transition from P6322 to P63/mmc upon cooling around Tp = 250 K, as evidenced by the evolution of temperature-dependent magnetization (M-T curves), cell dimension, and conductivity initiated by an inherent bifurcation of the oxygen position in the ab plane. Insightful analysis of its band structure positions suggests this HP-P6322 polymorph being a plausible candidate for Dirac semimetal properties. This phase transition evokes anomalies in the temperature-dependent variation of paramagnetism (non-linearity) and a crossover from semiconductor to temperature-independent metal, showing a temperature independent conductivity behavior below ~200 K. Under increasing external pressure, both the Tp and resistance of this HP-polymorph is slightly magnetic-field dependent and undergo a "V"-style evolution (decreasing and then increasing) before becoming pressure independent up to 20.2 GPa. Theoretical calculations pinpoint this anionic disorder as a probable catalyst for the decrement in the conductive efficiency and muted temperature-dependent conductivity response.

cond-mat.mtrl-sci↗

arXiv2023

Entanglement of defect subregions in double holography

In the framework of double holography, we investigate the entanglement behavior of a brane subregion in AdS spacetime coupled to a bath on its boundary and also extract the contribution from the quantum matter within this subregion. From the boundary perspective, the brane subregion serves as the entanglement wedge of a subsystem on the conformal defect. In the ground state, we find the subsystem undergoes an entanglement phase transition induced by the degrees of freedom on the brane. With subcritical parameters, the wedge and entanglement entropy sharply decrease to zero. In contrast, in the supercritical regime, both the wedge and entropy stabilize, enabling analysis of both entanglement and reflected entropy. In this phase, we derive formulas for entanglement measures based on defect and bath central charges in the semi-classical limit. For entanglement entropy, the classical geometry only contributes a subleading term with logarithmic divergence, but the brane-bath CFT entanglement exhibits a dominant linear divergence, even in the semi-classical limit. Regarding reflected entropy within the defect subsystem, classical geometry contributes a leading term with logarithmic divergence, while the quantum matter within the entanglement wedge only contributes a finite term.

hep-th↗

arXiv2023

Democratizing Reasoning Ability: Tailored Learning from Large Language Model

Large language models (LLMs) exhibit impressive emergent abilities in natural language processing, but their democratization is hindered due to huge computation requirements and closed-source nature. Recent research on advancing open-source smaller LMs by distilling knowledge from black-box LLMs has obtained promising results in the instruction-following ability. However, the reasoning ability which is more challenging to foster, is relatively rarely explored. In this paper, we propose a tailored learning approach to distill such reasoning ability to smaller LMs to facilitate the democratization of the exclusive reasoning ability. In contrast to merely employing LLM as a data annotator, we exploit the potential of LLM as a reasoning teacher by building an interactive multi-round learning paradigm. This paradigm enables the student to expose its deficiencies to the black-box teacher who then can provide customized training data in return. Further, to exploit the reasoning potential of the smaller LM, we propose self-reflection learning to motivate the student to learn from self-made mistakes. The learning from self-reflection and LLM are all tailored to the student's learning status, thanks to the seamless integration with the multi-round learning paradigm. Comprehensive experiments and analysis on mathematical and commonsense reasoning tasks demonstrate the effectiveness of our method. The code will be available at https://github.com/Raibows/Learn-to-Reason.

cs.CL↗

arXiv2023

PROSE: Predicting Operators and Symbolic Expressions using Multimodal Transformers

Approximating nonlinear differential equations using a neural network provides a robust and efficient tool for various scientific computing tasks, including real-time predictions, inverse problems, optimal controls, and surrogate modeling. Previous works have focused on embedding dynamical systems into networks through two approaches: learning a single solution operator (i.e., the mapping from input parametrized functions to solutions) or learning the governing system of equations (i.e., the constitutive model relative to the state variables). Both of these approaches yield different representations for the same underlying data or function. Additionally, observing that families of differential equations often share key characteristics, we seek one network representation across a wide range of equations. Our method, called Predicting Operators and Symbolic Expressions (PROSE), learns maps from multimodal inputs to multimodal outputs, capable of generating both numerical predictions and mathematical equations. By using a transformer structure and a feature fusion approach, our network can simultaneously embed sets of solution operators for various parametric differential equations using a single trained network. Detailed experiments demonstrate that the network benefits from its multimodal nature, resulting in improved prediction accuracy and better generalization. The network is shown to be able to handle noise in the data and errors in the symbolic representation, including noisy numerical values, model misspecification, and erroneous addition or deletion of terms. PROSE provides a new neural network framework for differential equations which allows for more flexibility and generality in learning operators and governing equations from data.

cs.LG↗

arXiv2023

Calibrating LLM-Based Evaluator

Recent advancements in large language models (LLMs) on language modeling and emergent capabilities make them a promising reference-free evaluator of natural language generation quality, and a competent alternative to human evaluation. However, hindered by the closed-source or high computational demand to host and tune, there is a lack of practice to further calibrate an off-the-shelf LLM-based evaluator towards better human alignment. In this work, we propose AutoCalibrate, a multi-stage, gradient-free approach to automatically calibrate and align an LLM-based evaluator toward human preference. Instead of explicitly modeling human preferences, we first implicitly encompass them within a set of human labels. Then, an initial set of scoring criteria is drafted by the language model itself, leveraging in-context learning on different few-shot examples. To further calibrate this set of criteria, we select the best performers and re-draft them with self-refinement. Our experiments on multiple text quality evaluation datasets illustrate a significant improvement in correlation with expert evaluation through calibration. Our comprehensive qualitative analysis conveys insightful intuitions and observations on the essence of effective scoring criteria.

cs.CL↗

arXiv2023

FSNet: Redesign Self-Supervised MonoDepth for Full-Scale Depth Prediction for Autonomous Driving

Predicting accurate depth with monocular images is important for low-cost robotic applications and autonomous driving. This study proposes a comprehensive self-supervised framework for accurate scale-aware depth prediction on autonomous driving scenes utilizing inter-frame poses obtained from inertial measurements. In particular, we introduce a Full-Scale depth prediction network named FSNet. FSNet contains four important improvements over existing self-supervised models: (1) a multichannel output representation for stable training of depth prediction in driving scenarios, (2) an optical-flow-based mask designed for dynamic object removal, (3) a self-distillation training strategy to augment the training process, and (4) an optimization-based post-processing algorithm in test time, fusing the results from visual odometry. With this framework, robots and vehicles with only one well-calibrated camera can collect sequences of training image frames and camera poses, and infer accurate 3D depths of the environment without extra labeling work or 3D data. Extensive experiments on the KITTI dataset, KITTI-360 dataset and the nuScenes dataset demonstrate the potential of FSNet. More visualizations are presented in \url{https://sites.google.com/view/fsnet/home}

cs.CV↗

arXiv2023

RNGDet++: Road Network Graph Detection by Transformer with Instance Segmentation and Multi-scale Features Enhancement

The road network graph is a critical component for downstream tasks in autonomous driving, such as global route planning and navigation. In the past years, road network graphs are usually annotated by human experts manually, which is time-consuming and labor-intensive. To annotate road network graphs effectively and efficiently, automatic algorithms for road network graph detection are demanded. Most existing methods either adopt a post-processing step on semantic segmentation maps to produce road network graphs, or propose graph-based algorithms to directly predict the graphs. However, these works suffer from hard-coded algorithms and inferior performance. To enhance the previous state-of-the-art (SOTA) method RNGDet, we add an instance segmentation head to better supervise the training, and enable the network to leverage multi-scale features of the backbone. Since the new proposed approach is improved from RNGDet, we name it RNGDet++. Experimental results show that our RNGDet++ outperforms baseline methods in terms of almost all evaluation metrics on two large-scale public datasets. Our code and supplementary materials are available at \url{https://tonyxuqaq.github.io/projects/RNGDetPlusPlus/}.

cs.CV↗

arXiv2023

CenterLineDet: CenterLine Graph Detection for Road Lanes with Vehicle-mounted Sensors by Transformer for HD Map Generation

With the fast development of autonomous driving technologies, there is an increasing demand for high-definition (HD) maps, which provide reliable and robust prior information about the static part of the traffic environments. As one of the important elements in HD maps, road lane centerline is critical for downstream tasks, such as prediction and planning. Manually annotating centerlines for road lanes in HD maps is labor-intensive, expensive and inefficient, severely restricting the wide applications of autonomous driving systems. Previous work seldom explores the lane centerline detection problem due to the complicated topology and severe overlapping issues of lane centerlines. In this paper, we propose a novel method named CenterLineDet to detect lane centerlines for automatic HD map generation. Our CenterLineDet is trained by imitation learning and can effectively detect the graph of centerlines with vehicle-mounted sensors (i.e., six cameras and one LiDAR) through iterations. Due to the use of the DETR-like transformer network, CenterLineDet can handle complicated graph topology, such as lane intersections. The proposed approach is evaluated on the large-scale public dataset NuScenes. The superiority of our CenterLineDet is demonstrated by the comparative results. Our code, supplementary materials, and video demonstrations are available at \href{https://tonyxuqaq.github.io/projects/CenterLineDet/}{https://tonyxuqaq.github.io/projects/CenterLineDet/}.

cs.CV↗

arXiv2023

Nonlinear Intensity, Scale and Rotation Invariant Matching for Multimodal Images

We present an effective method for the matching of multimodal images. Accurate image matching is the basis of various applications, such as image registration and structure from motion. Conventional matching methods fail when handling noisy multimodal image pairs with severe scale change, rotation, and nonlinear intensity distortion (NID). Toward this need, we introduce an image pyramid strategy to tackle scale change. We put forward an accurate primary orientation estimation approach to reduce the effect of image rotation at any angle. We utilize multi-scale and multi-orientation image filtering results and a feature-to-template matching scheme to ensure effective and accurate matching under large NID. Integrating these improvements significantly increases noise, scale, rotation, and NID invariant capability. Our experimental results confirm the excellent ability to achieve high-quality matches across various multimodal images. The proposed method outperforms the mainstream multimodal image matching methods in qualitative and quantitative evaluations. Our implementation is available at https://github.com/Zhongli-Fan/NISR.

cs.CV↗

arXiv2022

Random Feature Models for Learning Interacting Dynamical Systems

Particle dynamics and multi-agent systems provide accurate dynamical models for studying and forecasting the behavior of complex interacting systems. They often take the form of a high-dimensional system of differential equations parameterized by an interaction kernel that models the underlying attractive or repulsive forces between agents. We consider the problem of constructing a data-based approximation of the interacting forces directly from noisy observations of the paths of the agents in time. The learned interaction kernels are then used to predict the agents behavior over a longer time interval. The approximation developed in this work uses a randomized feature algorithm and a sparse randomized feature approach. Sparsity-promoting regression provides a mechanism for pruning the randomly generated features which was observed to be beneficial when one has limited data, in particular, leading to less overfitting than other approaches. In addition, imposing sparsity reduces the kernel evaluation cost which significantly lowers the simulation cost for forecasting the multi-agent systems. Our method is applied to various examples, including first-order systems with homogeneous and heterogeneous interactions, second order homogeneous systems, and a new sheep swarming system.

cs.LG↗

arXiv2022

Learning Instrumental Variable from Data Fusion for Treatment Effect Estimation

The advent of the big data era brought new opportunities and challenges to draw treatment effect in data fusion, that is, a mixed dataset collected from multiple sources (each source with an independent treatment assignment mechanism). Due to possibly omitted source labels and unmeasured confounders, traditional methods cannot estimate individual treatment assignment probability and infer treatment effect effectively. Therefore, we propose to reconstruct the source label and model it as a Group Instrumental Variable (GIV) to implement IV-based Regression for treatment effect estimation. In this paper, we conceptualize this line of thought and develop a unified framework (Meta-EM) to (1) map the raw data into a representation space to construct Linear Mixed Models for the assigned treatment variable; (2) estimate the distribution differences and model the GIV for the different treatment assignment mechanisms; and (3) adopt an alternating training strategy to iteratively optimize the representations and the joint distribution to model GIV for IV regression. Empirical results demonstrate the advantages of our Meta-EM compared with state-of-the-art methods.

cs.AI↗

arXiv2022

Black holes Entangled by Radiation

We construct three models to describe the scenario where two eternal black holes are separated by a flat space, and can eventually be entangled by exchanging radiations. In the doubly holographic setup, we compute the entanglement entropy and the mutual information among the subsystems and obtain the dynamic phase structure of the entanglement. The formation of entanglement between the two black holes is delayed by the space where the radiations must travel through. Finally, if the two black holes exchange sufficient Hawking modes, the final state is characterized by a connected entanglement wedge; otherwise, the final entanglement wedge contains two separated islands. In the former case, the entanglement wedge of the two black holes forms at the time scale of the size of the flat space between them. While in both cases, unitarity of the evolution is preserved. When the sizes of two black holes are not equal, we observe a loss of entanglement between the smaller black hole and the radiation at late times. In the field theory side, we consider two Sachdev-Ye-Kitaev (SYK) clusters coupled to a Majorana chain, which resemble two black holes connected by a radiation region. We numerically compute the same entanglement measures, and obtain similar phase structures as the bulk results. In general, a time delay of the entanglement between the two SYK clusters is found in cases with a long Majorana chain. In particular, when the two SYK clusters are different in size, similar entanglement loss between the smaller SYK cluster and the Majorana chain is observed. Finally, we investigate a chain model composed of EPR clusters with particle exchanges between neighboring clusters, and reproduce the features of entanglement observed in the other models.

hep-th↗

arXiv2022

Holographic Einstein Ring of a Charged AdS Black Hole

Taking into account that the real quantum materials are engineered generically at a finite chemical potential, we investigate the Einstein ring structure for the lensed response of the complex scalar field as a probe wave on the charged AdS black hole in the context of AdS/CFT. On the one hand, we find that the resulting Einstein ring radius has no variation with the chemical potential, which is similar to the behavior for the weakly interacting quantum system. On the other hand, not only can such a ring exist well within the screen, but also the temperature dependence of its radius exhibits a distinct feature in the sense that it displays an appreciable increase at low temperatures while the ring keeps unchanged right at the edge of the screen for the weakly interacting system. Note that such a Einstein ring emerges in the large frequencies and can be well captured by the photon sphere away from the black hole horizon in the geometric optics approximation, thus such a distinct feature may be regarded as a universal behavior associated with the high energy modes of the strongly coupled system which has a gravity dual.

hep-th↗

arXiv2022

Critical Dynamics in Holographic First-Order Phase Transition

We study the critical phenomena of the dynamical transition from a metastable state to a stable state in the model of first-order phase transition via two different triggering mechanisms. Three universal stages during the fully nonlinear evolution are extracted. On the one side, by perturbing the scalar source, an isolated seed nucleus is injected into an initial homogeneous state in the supercooled region. For critical parameters of the seed nucleus, the real-time dynamics reveal that the system will converge to a critically unstable state. For supercritical parameters, the system exhibits a phase separation, while for subcritical parameters falls back to homogeneous. The shape independence of the seed nucleus is also investigated, which implies that the critical phenomena are universal. On the other side, we propose a novel mechanism to render the critical phenomena via a collision of two gravitational shock waves on the dual geometries. Specifying a collision velocity, the initial system can be also attracted to a critically unstable state. Aside from these dynamical constructions, we also quantitatively analyze the critical nucleus preventing the system from reaching the final phase separation. We find the depth of the critical nucleus increases almost linearly with the temperature, which implies that the hotter the supercooled state is, the harder for it to trigger phase separation.

hep-th↗

arXiv2022

Tackling Long-Tailed Category Distribution Under Domain Shifts

Machine learning models fail to perform well on real-world applications when 1) the category distribution P(Y) of the training dataset suffers from long-tailed distribution and 2) the test data is drawn from different conditional distributions P(X|Y). Existing approaches cannot handle the scenario where both issues exist, which however is common for real-world applications. In this study, we took a step forward and looked into the problem of long-tailed classification under domain shifts. We designed three novel core functional blocks including Distribution Calibrated Classification Loss, Visual-Semantic Mapping and Semantic-Similarity Guided Augmentation. Furthermore, we adopted a meta-learning framework which integrates these three blocks to improve domain generalization on unseen target domains. Two new datasets were proposed for this problem, named AWA2-LTS and ImageNet-LTS. We evaluated our method on the two datasets and extensive experimental results demonstrate that our proposed method can achieve superior performance over state-of-the-art long-tailed/domain generalization approaches and the combinations. Source codes and datasets can be found at our project page https://xiaogu.site/LTDS.

cs.CV↗

arXiv2022

RNGDet: Road Network Graph Detection by Transformer in Aerial Images

Road network graphs provide critical information for autonomous-vehicle applications, such as drivable areas that can be used for motion planning algorithms. To find road network graphs, manually annotation is usually inefficient and labor-intensive. Automatically detecting road network graphs could alleviate this issue, but existing works still have some limitations. For example, segmentation-based approaches could not ensure satisfactory topology correctness, and graph-based approaches could not present precise enough detection results. To provide a solution to these problems, we propose a novel approach based on transformer and imitation learning in this paper. In view of that high-resolution aerial images could be easily accessed all over the world nowadays, we make use of aerial images in our approach. Taken as input an aerial image, our approach iteratively generates road network graphs vertex-by-vertex. Our approach can handle complicated intersection points with various numbers of incident road segments. We evaluate our approach on a publicly available dataset. The superiority of our approach is demonstrated through the comparative experiments. Our work is accompanied with a demonstration video which is available at \url{https://tonyxuqaq.github.io/projects/RNGDet/}.

cs.CV↗

arXiv2022

Integrating Dependency Tree Into Self-attention for Sentence Representation

Recent progress on parse tree encoder for sentence representation learning is notable. However, these works mainly encode tree structures recursively, which is not conducive to parallelization. On the other hand, these works rarely take into account the labels of arcs in dependency trees. To address both issues, we propose Dependency-Transformer, which applies a relation-attention mechanism that works in concert with the self-attention mechanism. This mechanism aims to encode the dependency and the spatial positional relations between nodes in the dependency tree of sentences. By a score-based method, we successfully inject the syntax information without affecting Transformer's parallelizability. Our model outperforms or is comparable to the state-of-the-art methods on four tasks for sentence representation and has obvious advantages in computational efficiency.

cs.CL↗