Searcharxiv⌕ Search

arXiv subjects

Shan Li

Publications and source records attributed to Shan Li.

At least 37 records · Page 2Linked to original sources
arXiv2024

Deep Learning Technique for Human Parsing: A Survey and Outlook

Human parsing aims to partition humans in image or video into multiple pixel-level semantic parts. In the last decade, it has gained significantly increased interest in the computer vision community and has been utilized in a broad range of practical applications, from security monitoring, to social media, to visual special effects, just to name a few. Although deep learning-based human parsing solutions have made remarkable achievements, many important concepts, existing challenges, and potential research directions are still confusing. In this survey, we comprehensively review three core sub-tasks: single human parsing, multiple human parsing, and video human parsing, by introducing their respective task settings, background concepts, relevant problems and applications, representative literature, and datasets. We also present quantitative performance comparisons of the reviewed methods on benchmark datasets. Additionally, to promote sustainable development of the community, we put forward a transformer-based human parsing framework, providing a high-performance baseline for follow-up research through universal, concise, and extensible solutions. Finally, we point out a set of under-investigated open issues in this field and suggest new directions for future study. We also provide a regularly updated project page, to continuously track recent developments in this fast-advancing field: https://github.com/soeaver/awesome-human-parsing.

cs.CV↗
arXiv2024

FairSample: Training Fair and Accurate Graph Convolutional Neural Networks Efficiently

Fairness in Graph Convolutional Neural Networks (GCNs) becomes a more and more important concern as GCNs are adopted in many crucial applications. Societal biases against sensitive groups may exist in many real world graphs. GCNs trained on those graphs may be vulnerable to being affected by such biases. In this paper, we adopt the well-known fairness notion of demographic parity and tackle the challenge of training fair and accurate GCNs efficiently. We present an in-depth analysis on how graph structure bias, node attribute bias, and model parameters may affect the demographic parity of GCNs. Our insights lead to FairSample, a framework that jointly mitigates the three types of biases. We employ two intuitive strategies to rectify graph structures. First, we inject edges across nodes that are in different sensitive groups but similar in node features. Second, to enhance model fairness and retain model quality, we develop a learnable neighbor sampling policy using reinforcement learning. To address the bias in node features and model parameters, FairSample is complemented by a regularization objective to optimize fairness.

cs.LG↗
arXiv2023

Edge Inference with Fully Differentiable Quantized Mixed Precision Neural Networks

The large computing and memory cost of deep neural networks (DNNs) often precludes their use in resource-constrained devices. Quantizing the parameters and operations to lower bit-precision offers substantial memory and energy savings for neural network inference, facilitating the use of DNNs on edge computing platforms. Recent efforts at quantizing DNNs have employed a range of techniques encompassing progressive quantization, step-size adaptation, and gradient scaling. This paper proposes a new quantization approach for mixed precision convolutional neural networks (CNNs) targeting edge-computing. Our method establishes a new pareto frontier in model accuracy and memory footprint demonstrating a range of quantized models, delivering best-in-class accuracy below 4.3 MB of weights (wgts.) and activations (acts.). Our main contributions are: (i) hardware-aware heterogeneous differentiable quantization with tensor-sliced learned precision, (ii) targeted gradient modification for wgts. and acts. to mitigate quantization errors, and (iii) a multi-phase learning schedule to address instability in learning arising from updates to the learned quantizer and model parameters. We demonstrate the effectiveness of our techniques on the ImageNet dataset across a range of models including EfficientNet-Lite0 (e.g., 4.14MB of wgts. and acts. at 67.66% accuracy) and MobileNetV2 (e.g., 3.51MB wgts. and acts. at 65.39% accuracy).

cs.LG↗
arXiv2023

Optimal Vehicle Trajectory Planning for Static Obstacle Avoidance using Nonlinear Optimization

Vehicle trajectory planning is a key component for an autonomous driving system. A practical system not only requires the component to compute a feasible trajectory, but also a comfortable one given certain comfort metrics. Nevertheless, computation efficiency is critical for the system to be deployed as a commercial product. In this paper, we present a novel trajectory planning algorithm based on nonlinear optimization. The algorithm computes a kinematically feasible and comfort-optimal trajectory that achieves collision avoidance with static obstacles. Furthermore, the algorithm is time efficient. It generates an 6-second trajectory within 10 milliseconds on an Intel i7 machine or 20 milliseconds on an Nvidia Drive Orin platform.

cs.RO↗
arXiv2023

Ultrasonic backscattering model for Rayleigh waves in polycrystals with Born and independent scattering approximations

This paper presents theoretical and numerical models for the backscattering of 2D Rayleigh waves in single-phase, untextured polycrystalline materials with statistically equiaxed grains. The theoretical model, based on our prior inclusion-induced Rayleigh wave scattering model and the independent scattering approximation, considers single scattering of Rayleigh-to-Rayleigh (R-R) waves. The numerical finite element model is established to accurately simulate the scattering problem and evaluate the theoretical model. Good quantitative agreement is observed between the theoretical model and the finite element results, especially for weakly scattering materials. The agreement decreases with the increase of the anisotropy index, owing to the reduced applicability of the Born approximation. However, the agreement remains generally good when weak multiple scattering is involved. In addition, the R-R backscattering behaviour of 2D Rayleigh waves is similar to the longitudinal-to-longitudinal and transverse-to-transverse backscattering of bulk waves, with the former exhibiting stronger scattering. These findings establish a foundation for using Rayleigh waves in quantitative characterisation of polycrystalline materials.

physics.app-ph↗
arXiv2023

Interactive Trajectory Planner for Mandatory Lane Changing in Dense Non-Cooperative Traffic

When the traffic stream is extremely congested and surrounding vehicles are not cooperative, the mandatory lane changing can be significantly difficult. In this work, we propose an interactive trajectory planner, which will firstly attempt to change lanes as long as safety is ensured. Based on receding horizon planning, the ego vehicle can abort or continue changing lanes according to surrounding vehicles' reactions. We demonstrate the performance of our planner in extensive simulations with eight surrounding vehicles, initial velocity ranging from 0.5 to 5 meters per second, and bumper to bumper gap ranging from 4 to 10 meters. The ego vehicle with our planner can change lanes safely and smoothly. The computation time of the planner at every step is within 10 milliseconds in most cases on a laptop with 1.8GHz Intel Core i7-10610U.

cs.RO↗
arXiv2022

Theoretical and numerical modeling of Rayleigh wave scattering by an elastic inclusion

This work presents theoretical and numerical models for the backscattering of two-dimensional Rayleigh waves by an elastic inclusion, with the host material being isotropic and the inclusion having arbitrary shape and crystallographic symmetry. The theoretical model is developed based on the reciprocity theorem using the far-field Green's function and the Born approximation, assuming a small acoustic impedance difference between the host and inclusion materials. The numerical finite element (FE) model is established to deliver relatively accurate simulation of the scattering problem and to evaluate the approximations of the theoretical model. Quantitative agreement is observed between the theoretical model and the FE results for arbitrarily-shaped surface/subsurface inclusions with isotropic/anisotropic properties. The agreement is excellent when the wavelength of the Rayleigh wave is larger than, or comparable to, the size of the inclusion, but it deteriorates as the wavelength gets smaller. Also, the agreement decreases with the anisotropy index for inclusions of anisotropic symmetry. The results lay the foundation for using Rayleigh waves for quantitative characterization of surface/subsurface inclusions, while also demonstrating its limitations.

physics.class-ph↗
arXiv2022

Characterizations of (Jordan) derivation on Banach algebra with local actions

Let $\mathcal{A}$ be a unital Banach $*$-algebra and $\mathcal{M}$ be a unital $*$-$\mathcal{A}$-bimodule. If $W$ is a left separating point of $\mathcal{M}$, we show that every $*$-derivable mapping at $W$ is a Jordan derivation, and every $*$-left derivable mapping at $W$ is a Jordan left derivation under the condition $W \mathcal{A}=\mathcal{A}W$. Moreover we give a complete description of linear mappings $δ$ and $τ$ from $\mathcal{A}$ into $\mathcal{M}$ satisfying $δ(A)B^*+Aτ(B)^*=0$ for any $A, B\in \mathcal{A}$ with $AB^*=0$ or $δ(A)\circ B^*+A\circτ(B)^*=0$ for any $A, B\in \mathcal{A}$ with $A\circ B^*=0$, where $A\circ B=AB+BA$ is the Jordan product.

math.RA↗
arXiv2022

Local Lie $n$-derivations on certain algebras

We prove that each local Lie $n$-derivation is a Lie $n$-derivation under mild assumptions on the unital algebras with a nontrivial idempotent. As applications, we obtain descriptions of local Lie $n$-derivations on generalized matrix algebras, triangular algebras, nest algebras, von Neumann algebras, and the algebras of locally measurable operators affiliated with a von Neumann algebra.

math.OA↗
arXiv2021

Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors

Although the quest for more accurate solutions is pushing deep learning research towards larger and more complex algorithms, edge devices demand efficient inference and therefore reduction in model size, latency and energy consumption. One technique to limit model size is quantization, which implies using fewer bits to represent weights and biases. Such an approach usually results in a decline in performance. Here, we introduce a method for designing optimally heterogeneously quantized versions of deep neural network models for minimum-energy, high-accuracy, nanosecond inference and fully automated deployment on chip. With a per-layer, per-parameter type automatic quantization procedure, sampling from a wide range of quantizers, model energy consumption and size are minimized while high accuracy is maintained. This is crucial for the event selection procedure in proton-proton collisions at the CERN Large Hadron Collider, where resources are strictly limited and a latency of ${\mathcal O}(1)~μ$s is required. Nanosecond inference and a resource consumption reduced by a factor of 50 when implemented on field-programmable gate array hardware are achieved.

physics.ins-det↗
arXiv2020

RAF-AU Database: In-the-Wild Facial Expressions with Subjective Emotion Judgement and Objective AU Annotations

Much of the work on automatic facial expression recognition relies on databases containing a certain number of emotion classes and their exaggerated facial configurations (generally six prototypical facial expressions), based on Ekman's Basic Emotion Theory. However, recent studies have revealed that facial expressions in our human life can be blended with multiple basic emotions. And the emotion labels for these in-the-wild facial expressions cannot easily be annotated solely on pre-defined AU patterns. How to analyze the action units for such complex expressions is still an open question. To address this issue, we develop a RAF-AU database that employs a sign-based (i.e., AUs) and judgement-based (i.e., perceived emotion) approach to annotating blended facial expressions in the wild. We first reviewed the annotation methods in existing databases and identified crowdsourcing as a promising strategy for labeling in-the-wild facial expressions. Then, RAF-AU was finely annotated by experienced coders, on which we also conducted a preliminary investigation of which key AUs contribute most to a perceived emotion, and the relationship between AUs and facial expressions. Finally, we provided a baseline for AU recognition in RAF-AU using popular features and multi-label learning methods.

cs.CV↗
arXiv2020

Half-Heusler thermoelectric materials: NMR studies

We report $^{59}$Co, $^{93}$Nb, and $^{121}$Sb nuclear magnetic resonance (NMR) measurements combined with density functional theory (DFT) calculations on a series of half-Heusler semiconductors, including NbCoSn, ZrCoSb, TaFeSb and NbFeSb, to better understand their electronic properties and general composition-dependent trends. These materials are of interest as potentially high efficiency thermoelectric materials. Compared to the other materials, we find that ZrCoSb tends to have a relatively large amount of local disorder, apparently antisite defects. This contributes to a small excitation gap corresponding to an impurity band near the band edge. In NbCoSn and TaFeSb, Curie-Weiss-type behavior is revealed, which indicates a small density of interacting paramagnetic defects. Very large paramagnetic chemical shifts are observed associated with a Van Vleck mechanism due to closely spaced $d$ bands splitting between the conduction and valence bands. Meanwhile, DFT methods were generally successful in reproducing the chemical shift trend for these half-Heusler materials, and we identify an enhancement of the larger-magnitude shifts, which we connect to electron interaction effects. The general trend is connected to changes in $d$-electron hybridization across the series.

cond-mat.mtrl-sci↗
arXiv2020

Deep Job Understanding at LinkedIn

As the world's largest professional network, LinkedIn wants to create economic opportunity for everyone in the global workforce. One of its most critical missions is matching jobs with processionals. Improving job targeting accuracy and hire efficiency align with LinkedIn's Member First Motto. To achieve those goals, we need to understand unstructured job postings with noisy information. We applied deep transfer learning to create domain-specific job understanding models. After this, jobs are represented by professional entities, including titles, skills, companies, and assessment questions. To continuously improve LinkedIn's job understanding ability, we designed an expert feedback loop where we integrated job understanding models into LinkedIn's products to collect job posters' feedback. In this demonstration, we present LinkedIn's job posting flow and demonstrate how the integrated deep job understanding work improves job posters' satisfaction and provides significant metric lifts in LinkedIn's job recommendation system.

cs.IR↗
arXiv2020

Learning to Ask Screening Questions for Job Postings

At LinkedIn, we want to create economic opportunity for everyone in the global workforce. A critical aspect of this goal is matching jobs with qualified applicants. To improve hiring efficiency and reduce the need to manually screening each applicant, we develop a new product where recruiters can ask screening questions online so that they can filter qualified candidates easily. To add screening questions to all $20$M active jobs at LinkedIn, we propose a new task that aims to automatically generate screening questions for a given job posting. To solve the task of generating screening questions, we develop a two-stage deep learning model called Job2Questions, where we apply a deep learning model to detect intent from the text description, and then rank the detected intents by their importance based on other contextual features. Since this is a new product with no historical data, we employ deep transfer learning to train complex models with limited training data. We launched the screening question product and our AI models to LinkedIn users and observed significant impact in the job marketplace. During our online A/B test, we observed $+53.10\%$ screening question suggestion acceptance rate, $+22.17\%$ job coverage, $+190\%$ recruiter-applicant interaction, and $+11$ Net Promoter Score. In sum, the deployed Job2Questions model helps recruiters to find qualified applicants and job seekers to find jobs they are qualified for.

cs.IR↗
arXiv2019

A Deeper Look at Facial Expression Dataset Bias

Datasets play an important role in the progress of facial expression recognition algorithms, but they may suffer from obvious biases caused by different cultures and collection conditions. To look deeper into this bias, we first conduct comprehensive experiments on dataset recognition and crossdataset generalization tasks, and for the first time explore the intrinsic causes of the dataset discrepancy. The results quantitatively verify that current datasets have a strong buildin bias and corresponding analyses indicate that the conditional probability distributions between source and target datasets are different. However, previous researches are mainly based on shallow features with limited discriminative ability under the assumption that the conditional distribution remains unchanged across domains. To address these issues, we further propose a novel deep Emotion-Conditional Adaption Network (ECAN) to learn domain-invariant and discriminative feature representations, which can match both the marginal and the conditional distributions across domains simultaneously. In addition, the largely ignored expression class distribution bias is also addressed by a learnable re-weighting parameter, so that the training and testing domains can share similar class distribution. Extensive cross-database experiments on both lab-controlled datasets (CK+, JAFFE, MMI and Oulu-CASIA) and real-world databases (AffectNet, FER2013, RAF-DB 2.0 and SFEW 2.0) demonstrate that our ECAN can yield competitive performances across various facial expression transfer tasks and outperform the state-of-theart methods.

cs.CV↗
arXiv2019

Knowledge-aided Two-dimensional Autofocus for Spotlight SAR Filtered Backprojection Imagery

Filtered backprojection (FBP) algorithm is a popular choice for complicated trajectory SAR image formation processing due to its inherent nonlinear motion compensation capability. However, how to efficiently autofocus the defocused FBP imagery when the motion measurement is not accurate enough is still a challenging problem. In this paper, a new interpretation of the FBP derivation is presented from the Fourier transform point of view. Based on this new viewpoint, the property of the residual 2-D phase error in FBP imagery is analyzed in detail. Then, by incorporating the derived a priori knowledge on the 2-D phase error, an accurate and efficient 2-D autofocus approach is proposed. The new approach performs the parameter estimation in a dimension-reduced parameter subspace by exploiting the a priori analytical structure of the 2-D phase error, therefore possesses much higher accuracy and efficiency than conventional blind methods. Finally, experimental results clearly demonstrate the effectiveness and robustness of the proposed method.

eess.SP↗