arXiv subjects

Yin Wang

Publications and source records attributed to Yin Wang.

At least 37 records · Page 2Linked to original sources

arXiv2025

Sage Deer: A Super-Aligned Driving Generalist Is Your Copilot

The intelligent driving cockpit, an important part of intelligent driving, needs to match different users' comfort, interaction, and safety needs. This paper aims to build a Super-Aligned and GEneralist DRiving agent, SAGE DeeR. Sage Deer achieves three highlights: (1) Super alignment: It achieves different reactions according to different people's preferences and biases. (2) Generalist: It can understand the multi-view and multi-mode inputs to reason the user's physiological indicators, facial emotions, hand movements, body movements, driving scenarios, and behavioral decisions. (3) Self-Eliciting: It can elicit implicit thought chains in the language space to further increase generalist and super-aligned abilities. Besides, we collected multiple data sets and built a large-scale benchmark. This benchmark measures the deer's perceptual decision-making ability and the super alignment's accuracy.

cs.CV↗

arXiv2025

STFM: A Spatio-Temporal Information Fusion Model Based on Phase Space Reconstruction for Sea Surface Temperature Prediction

The sea surface temperature (SST), a key environmental parameter, is crucial to optimizing production planning, making its accurate prediction a vital research topic. However, the inherent nonlinearity of the marine dynamic system presents significant challenges. Current forecasting methods mainly include physics-based numerical simulations and data-driven machine learning approaches. The former, while describing SST evolution through differential equations, suffers from high computational complexity and limited applicability, whereas the latter, despite its computational benefits, requires large datasets and faces interpretability challenges. This study presents a prediction framework based solely on data-driven techniques. Using phase space reconstruction, we construct initial-delay attractor pairs with a mathematical homeomorphism and design a Spatio-Temporal Fusion Mapping (STFM) to uncover their intrinsic connections. Unlike conventional models, our method captures SST dynamics efficiently through phase space reconstruction and achieves high prediction accuracy with minimal training data in comparative tests

cs.LG↗

arXiv2025

Fg-T2M++: LLMs-Augmented Fine-Grained Text Driven Human Motion Generation

We address the challenging problem of fine-grained text-driven human motion generation. Existing works generate imprecise motions that fail to accurately capture relationships specified in text due to: (1) lack of effective text parsing for detailed semantic cues regarding body parts, (2) not fully modeling linguistic structures between words to comprehend text comprehensively. To tackle these limitations, we propose a novel fine-grained framework Fg-T2M++ that consists of: (1) an LLMs semantic parsing module to extract body part descriptions and semantics from text, (2) a hyperbolic text representation module to encode relational information between text units by embedding the syntactic dependency graph into hyperbolic space, and (3) a multi-modal fusion module to hierarchically fuse text and motion features. Extensive experiments on HumanML3D and KIT-ML datasets demonstrate that Fg-T2M++ outperforms SOTA methods, validating its ability to accurately generate motions adhering to comprehensive text semantics.

cs.CV↗

arXiv2025

SeMi: When Imbalanced Semi-Supervised Learning Meets Mining Hard Examples

Semi-Supervised Learning (SSL) can leverage abundant unlabeled data to boost model performance. However, the class-imbalanced data distribution in real-world scenarios poses great challenges to SSL, resulting in performance degradation. Existing class-imbalanced semi-supervised learning (CISSL) methods mainly focus on rebalancing datasets but ignore the potential of using hard examples to enhance performance, making it difficult to fully harness the power of unlabeled data even with sophisticated algorithms. To address this issue, we propose a method that enhances the performance of Imbalanced Semi-Supervised Learning by Mining Hard Examples (SeMi). This method distinguishes the entropy differences among logits of hard and easy examples, thereby identifying hard examples and increasing the utility of unlabeled data, better addressing the imbalance problem in CISSL. In addition, we maintain a class-balanced memory bank with confidence decay for storing high-confidence embeddings to enhance the pseudo-labels' reliability. Although our method is simple, it is effective and seamlessly integrates with existing approaches. We perform comprehensive experiments on standard CISSL benchmarks and experimentally demonstrate that our proposed SeMi outperforms existing state-of-the-art methods on multiple benchmarks, especially in reversed scenarios, where our best result shows approximately a 54.8\% improvement over the baseline methods.

cs.CV↗

arXiv2024

Achieving low contact resistance through copper-intercalated bilayer MoS$_2$

The high contact resistance between MoS$_2$ and metals hinders its potential as an ideal solution for overcoming the short channel effect in silicon-based FETs at sub-3nm scales. We theoretically designed a MoS$_2$-based transistor, featuring bilayer MoS$_2$ connected to Cu-intercalated bilayer MoS$_2$ electrodes. At 0.6 V, contact resistance is 16.7 $Ω\cdot μ$m (zigzag) and 30.0 $Ω\cdot μ$m (armchair), nearing or even surpassing the 30 $Ω\cdot μ$m quantum limit for single-layer materials. This low resistance is attributed to the elimination of the tunneling barrier and the creation of ohmic contacts. Additionally, the small contact potential difference enables lower operating voltages. Our intercalation design offers a novel approach to achieving low contact resistance in 2D electronic devices.

cond-mat.mes-hall↗

arXiv2024

Observation of nonaxisymmetric standard magnetorotational instability induced by a free-shear layer

The standard magnetorotational instability (SMRI) is widely believed to be responsible for the observed accretion rates in astronomical disks. It is a linear instability triggered in the differentially rotating ionized disk flow by a magnetic field component parallel to the rotation axis. Most studies focus on axisymmetric SMRI in conventional base flows with a Keplerian profile for accretion disks or an ideal Couette profile for Taylor-Couette flows, since excitation of nonaxisymmetric SMRI in such flows requires a magnetic Reynolds number Rm more than an order of magnitude larger. Here, we report that in a magnetized Taylor-Couette flow, nonaxisymmetric SMRI can be destabilized in a free-shear layer in the base flow at Rm $\gtrsim$ 1, the same threshold as for axisymmetric SMRI. Global linear analysis reveals that the free-shear layer reduces the required Rm, possibly by introducing an extremum in the vorticity of the base flow. Nonlinear simulations validate the results from linear analysis and confirm that a novel instability recently discovered experimentally (Nat. Commun. 13, 4679 (2022)) is the nonaxisymmetric SMRI. Our finding has astronomical implications since free-shear layers are ubiquitous in celestial systems.

astro-ph.HE↗

arXiv2024

Flatbands from Bound States in the Continuum for Orbital Angular Momentum Localization

A flatband material is a system characterized by energy bands with zero dispersion, allowing for the compact localization of wavefunctions in real space. This compact localization significantly enhances inter-particle correlations and light-matter interactions, leading to notable advancements such as fractional Chern insulators in condensed matter systems and flat-band lasers in photonics. Previous flatband platforms, including twisted bilayer graphene and artificial kagome/Lieb lattices, typically focused on nondegenerate flatbands, lacking access to the high degeneracy that can facilitate the localization of orbital angular momentum (OAM). Here, we propose a general framework to construct highly degenerate flatbands from bound states in the continuum (BICs)--a concept originating from quantum theory but significantly developed in photonics and acoustics in recent years. The degeneracy of flatbands is determined by the number of BICs within each unit cell in a lattice. We experimentally validate this approach in two-dimensional (2D) and three-dimensional (3D) acoustic crystals, demonstrating flatbands with 4-fold and 12-fold degeneracies, respectively. The high degeneracy provides sufficient internal degrees of freedom, enabling the selective excitation of localized OAM at any position in any direction. Our results pave the way for exploring BIC-constructed flatbands and their localization properties.

physics.optics↗

arXiv2024

TASL-Net: Tri-Attention Selective Learning Network for Intelligent Diagnosis of Bimodal Ultrasound Video

In the intelligent diagnosis of bimodal (gray-scale and contrast-enhanced) ultrasound videos, medical domain knowledge such as the way sonographers browse videos, the particular areas they emphasize, and the features they pay special attention to, plays a decisive role in facilitating precise diagnosis. Embedding medical knowledge into the deep learning network can not only enhance performance but also boost clinical confidence and reliability of the network. However, it is an intractable challenge to automatically focus on these person- and disease-specific features in videos and to enable networks to encode bimodal information comprehensively and efficiently. This paper proposes a novel Tri-Attention Selective Learning Network (TASL-Net) to tackle this challenge and automatically embed three types of diagnostic attention of sonographers into a mutual transformer framework for intelligent diagnosis of bimodal ultrasound videos. Firstly, a time-intensity-curve-based video selector is designed to mimic the temporal attention of sonographers, thus removing a large amount of redundant information while improving computational efficiency of TASL-Net. Then, to introduce the spatial attention of the sonographers for contrast-enhanced video analysis, we propose the earliest-enhanced position detector based on structural similarity variation, on which the TASL-Net is made to focus on the differences of perfusion variation inside and outside the lesion. Finally, by proposing a mutual encoding strategy that combines convolution and transformer, TASL-Net possesses bimodal attention to structure features on gray-scale videos and to perfusion variations on contrast-enhanced videos. These modules work collaboratively and contribute to superior performance. We conduct a detailed experimental validation of TASL-Net's performance on three datasets, including lung, breast, and liver.

cs.CV↗

arXiv2024

QUBIQ: Uncertainty Quantification for Biomedical Image Segmentation Challenge

Uncertainty in medical image segmentation tasks, especially inter-rater variability, arising from differences in interpretations and annotations by various experts, presents a significant challenge in achieving consistent and reliable image segmentation. This variability not only reflects the inherent complexity and subjective nature of medical image interpretation but also directly impacts the development and evaluation of automated segmentation algorithms. Accurately modeling and quantifying this variability is essential for enhancing the robustness and clinical applicability of these algorithms. We report the set-up and summarize the benchmark results of the Quantification of Uncertainties in Biomedical Image Quantification Challenge (QUBIQ), which was organized in conjunction with International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2020 and 2021. The challenge focuses on the uncertainty quantification of medical image segmentation which considers the omnipresence of inter-rater variability in imaging datasets. The large collection of images with multi-rater annotations features various modalities such as MRI and CT; various organs such as the brain, prostate, kidney, and pancreas; and different image dimensions 2D-vs-3D. A total of 24 teams submitted different solutions to the problem, combining various baseline models, Bayesian neural networks, and ensemble model techniques. The obtained results indicate the importance of the ensemble models, as well as the need for further research to develop efficient 3D methods for uncertainty quantification methods in 3D segmentation tasks.

eess.IV↗

arXiv2024

GPT as Psychologist? Preliminary Evaluations for GPT-4V on Visual Affective Computing

Multimodal large language models (MLLMs) are designed to process and integrate information from multiple sources, such as text, speech, images, and videos. Despite its success in language understanding, it is critical to evaluate the performance of downstream tasks for better human-centric applications. This paper assesses the application of MLLMs with 5 crucial abilities for affective computing, spanning from visual affective tasks and reasoning tasks. The results show that \gpt has high accuracy in facial action unit recognition and micro-expression detection while its general facial expression recognition performance is not accurate. We also highlight the challenges of achieving fine-grained micro-expression recognition and the potential for further study and demonstrate the versatility and potential of \gpt for handling advanced tasks in emotion recognition and related fields by integrating with task-related agents for more complex tasks, such as heart rate estimation through signal processing. In conclusion, this paper provides valuable insights into the potential applications and challenges of MLLMs in human-centric computing. Our interesting examples are at https://github.com/EnVision-Research/GPT4Affectivity.

cs.CV↗

arXiv2023

Attentive Mask CLIP

Image token removal is an efficient augmentation strategy for reducing the cost of computing image features. However, this efficient augmentation strategy has been found to adversely affect the accuracy of CLIP-based training. We hypothesize that removing a large portion of image tokens may improperly discard the semantic content associated with a given text description, thus constituting an incorrect pairing target in CLIP training. To address this issue, we propose an attentive token removal approach for CLIP training, which retains tokens with a high semantic correlation to the text description. The correlation scores are computed in an online fashion using the EMA version of the visual encoder. Our experiments show that the proposed attentive masking approach performs better than the previous method of random token removal for CLIP training. The approach also makes it efficient to apply multiple augmentation views to the image, as well as introducing instance contrastive learning tasks between these views into the CLIP framework. Compared to other CLIP improvements that combine different pre-training targets such as SLIP and MaskCLIP, our method is not only more effective, but also much more efficient. Specifically, using ViT-B and YFCC-15M dataset, our approach achieves $43.9\%$ top-1 accuracy on ImageNet-1K zero-shot classification, as well as $62.7/42.1$ and $38.0/23.2$ I2T/T2I retrieval accuracy on Flickr30K and MS COCO, which are $+1.1\%$, $+5.5/+0.9$, and $+4.4/+1.3$ higher than the SLIP method, while being $2.30\times$ faster. An efficient version of our approach running $1.16\times$ faster than the plain CLIP model achieves significant gains of $+5.3\%$, $+11.3/+8.0$, and $+9.5/+4.9$ on these benchmarks.

cs.CV↗

arXiv2023

Large-scale Weakly Supervised Learning for Road Extraction from Satellite Imagery

Automatic road extraction from satellite imagery using deep learning is a viable alternative to traditional manual mapping. Therefore it has received considerable attention recently. However, most of the existing methods are supervised and require pixel-level labeling, which is tedious and error-prone. To make matters worse, the earth has a diverse range of terrain, vegetation, and man-made objects. It is well known that models trained in one area generalize poorly to other areas. Various shooting conditions such as light and angel, as well as different image processing techniques further complicate the issue. It is impractical to develop training data to cover all image styles. This paper proposes to leverage OpenStreetMap road data as weak labels and large scale satellite imagery to pre-train semantic segmentation models. Our extensive experimental results show that the prediction accuracy increases with the amount of the weakly labeled data, as well as the road density in the areas chosen for training. Using as much as 100 times more data than the widely used DeepGlobe road dataset, our model with the D-LinkNet architecture and the ResNet-50 backbone exceeds the top performer of the current DeepGlobe leaderboard. Furthermore, due to large-scale pre-training, our model generalizes much better than those trained with only the curated datasets, implying great application potential.

cs.CV↗

arXiv2023

Fg-T2M: Fine-Grained Text-Driven Human Motion Generation via Diffusion Model

Text-driven human motion generation in computer vision is both significant and challenging. However, current methods are limited to producing either deterministic or imprecise motion sequences, failing to effectively control the temporal and spatial relationships required to conform to a given text description. In this work, we propose a fine-grained method for generating high-quality, conditional human motion sequences supporting precise text description. Our approach consists of two key components: 1) a linguistics-structure assisted module that constructs accurate and complete language feature to fully utilize text information; and 2) a context-aware progressive reasoning module that learns neighborhood and overall semantic linguistics features from shallow and deep graph neural networks to achieve a multi-step inference. Experiments show that our approach outperforms text-driven motion generation methods on HumanML3D and KIT test sets and generates better visually confirmed motion to the text conditions.

cs.CV↗

arXiv2023

Dynamic Hyperbolic Attention Network for Fine Hand-object Reconstruction

Reconstructing both objects and hands in 3D from a single RGB image is complex. Existing methods rely on manually defined hand-object constraints in Euclidean space, leading to suboptimal feature learning. Compared with Euclidean space, hyperbolic space better preserves the geometric properties of meshes thanks to its exponentially-growing space distance, which amplifies the differences between the features based on similarity. In this work, we propose the first precise hand-object reconstruction method in hyperbolic space, namely Dynamic Hyperbolic Attention Network (DHANet), which leverages intrinsic properties of hyperbolic space to learn representative features. Our method that projects mesh and image features into a unified hyperbolic space includes two modules, ie. dynamic hyperbolic graph convolution and image-attention hyperbolic graph convolution. With these two modules, our method learns mesh features with rich geometry-image multi-modal information and models better hand-object interaction. Our method provides a promising alternative for fine hand-object reconstruction in hyperbolic space. Extensive experiments on three public datasets demonstrate that our method outperforms most state-of-the-art methods.

cs.CV↗

arXiv2023

Emergency Resource Layout with Multiple Objectives under Complex Disaster Scenarios

Effective placement of emergency rescue resources, particularly with joint suppliers in complex disaster scenarios, is crucial for ensuring the reliability, efficiency, and quality of emergency rescue activities. However, limited research has considered the interaction between different disasters and material classification, which are highly vital to the emergency rescue. This study provides a novel and practical framework for reliable strategies of emergency rescue under complex disaster scenarios. The study employs a scenario-based approach to represent complex disasters, such as earthquakes, mudslides, floods, and their interactions. In optimizing the placement of emergency resources, the study considers government-owned suppliers, framework agreement suppliers, and existing suppliers collectively supporting emergency rescue materials. To determine the selection of joint suppliers and their corresponding optimal material quantities under complex disaster scenarios, the research proposes a multi-objective model that integrates cost, fairness, emergency efficiency, and uncertainty into a facility location problem. Finally, the study develops an NSGA-II-XGB algorithm to solve a disaster-prone province example and verify the feasibility and effectiveness of the proposed multi-objective model and solution methods. The results show that the methodology proposed in this paper can greatly reduce emergency costs, rescue time, and the difference between demand and suppliers while maximizing the coverage of rescue resources. More importantly, it can optimize the scale of resources by determining the location and number of materials provided by joint suppliers for various kinds of disasters simultaneously. This research represents a promising step towards making informed configuration decisions in emergency rescue work.

stat.AP↗

arXiv2023

Do as You Say: Consistency Detection of Data Practice in Program Code and Privacy Policy in Mini-App

Mini-app is an emerging form of mobile application that combines web technology with native capabilities. Its features, e.g., no need to download and no installation, have made it popular rapidly. However, privacy issues that violate the laws or regulations are breeding in the swiftly expanding mini-app ecosystem. The consistency between what the mini-app does about the data in the program code and what it declares in its privacy policy description is important. But no work has systematically investigated the privacy problem of the mini-app before. In this paper, to our best knowledge, we are the first to conduct the compliance detection of data practice and policy description in mini-apps. In this paper, we first customize a taint analysis method based on data entity dependency network to adapt to the characteristics of the JavaScript language in the mini-apps. Then, we transform data types and data operations to data practices in program codes and privacy policies, so as to finish a fine-grained consistency matching model.We crawl 100,000 mini-apps on WeChat client in the wild and extract 2,998 with a privacy policy. Among them, only 318 meet the consistency requirements, 2,680 are inconsistent, and the proportion of inconsistencies is as high as 89.4%. The inconsistency in the mini-app is very serious. Based on 6 real-world cases analyzed, in order to reduce this potential data leakage risk, we suggest that the developer should reduce the collection of irrelevant information and the straightforward use of templates, and the platform should provide data flow detection tools and privacy policy writing support.

cs.CR↗

arXiv2023

Adaptive Discriminative Regularization for Visual Classification

How to improve discriminative feature learning is central in classification. Existing works address this problem by explicitly increasing inter-class separability and intra-class similarity, whether by constructing positive and negative pairs for contrastive learning or posing tighter class separating margins. These methods do not exploit the similarity between different classes as they adhere to i.i.d. assumption in data. In this paper, we embrace the real-world data distribution setting that some classes share semantic overlaps due to their similar appearances or concepts. Regarding this hypothesis, we propose a novel regularization to improve discriminative learning. We first calibrate the estimated highest likelihood of one sample based on its semantically neighboring classes, then encourage the overall likelihood predictions to be deterministic by imposing an adaptive exponential penalty. As the gradient of the proposed method is roughly proportional to the uncertainty of the predicted likelihoods, we name it adaptive discriminative regularization (ADR), trained along with a standard cross entropy loss in classification. Extensive experiments demonstrate that it can yield consistent and non-trivial performance improvements in a variety of visual classification tasks (over 10 benchmarks). Furthermore, we find it is robust to long-tailed and noisy label data distribution. Its flexible design enables its compatibility with mainstream classification architectures and losses.

cs.LG↗

arXiv2022

Observation of axisymmetric standard magnetorotational instability in the laboratory

We report the first direct evidence for the axisymmetric standard magnetorotational instability (SMRI) from a combined experimental and numerical study of a magnetized liquid-metal shear flow in a Taylor-Couette cell with independently rotating and electrically conducting end caps. When a uniform vertical magnetic field $B_i$ is applied along the rotation axis, the measured radial magnetic field $B_r$ on the inner cylinder increases linearly with a small magnetic Reynolds number $Rm$ due to the magnetization of the residue Ekman circulation. Onset of the axisymmetric SMRI is identified from the nonlinear increase of $B_r$ beyond a critical $Rm$ in both experiments and nonlinear numerical simulations. The axisymmetric SMRI exists only at sufficiently large $Rm$ and intermediate $B_i$, a feature consistent with theoretical predictions. Our simulations further show that the axisymmetric SMRI causes the velocity and magnetic fields to contribute an outward flux of axial angular momentum in the bulk region, just as it should in accretion disks.

astro-ph.IM↗