arXiv subjects

Jing Zhang

Publications and source records attributed to Jing Zhang.

At least 703 records · Page 39Linked to original sources

arXiv2022

RU-Net: Regularized Unrolling Network for Scene Graph Generation

Scene graph generation (SGG) aims to detect objects and predict the relationships between each pair of objects. Existing SGG methods usually suffer from several issues, including 1) ambiguous object representations, as graph neural network-based message passing (GMP) modules are typically sensitive to spurious inter-node correlations, and 2) low diversity in relationship predictions due to severe class imbalance and a large number of missing annotations. To address both problems, in this paper, we propose a regularized unrolling network (RU-Net). We first study the relation between GMP and graph Laplacian denoising (GLD) from the perspective of the unrolling technique, determining that GMP can be formulated as a solver for GLD. Based on this observation, we propose an unrolled message passing module and introduce an $\ell_p$-based graph regularization to suppress spurious connections between nodes. Second, we propose a group diversity enhancement module that promotes the prediction diversity of relationships via rank maximization. Systematic experiments demonstrate that RU-Net is effective under a variety of settings and metrics. Furthermore, RU-Net achieves new state-of-the-arts on three popular databases: VG, VRD, and OI. Code is available at https://github.com/siml3/RU-Net.

cs.CV↗

arXiv2022

Center-of-Mass-based Robust Grasp Pose Adaptation Using RGBD Camera and Force/Torque Sensing

Object dropping may occur when the robotic arm grasps objects with uneven mass distribution due to additional moments generated by objects' gravity. To solve this problem, we present a novel work that does not require extra wrist and tactile sensors and large amounts of experiments for learning. First, we obtain the center-of-mass position of the rod object using the widely fixed joint torque sensors on the robot arm and RGBD camera. Further, we give the strategy of grasping to improve grasp stability. Simulation experiments are performed in "Mujoco". Results demonstrate that our work is effective in enhancing grasping robustness.

cs.RO↗

arXiv2022

DearKD: Data-Efficient Early Knowledge Distillation for Vision Transformers

Transformers are successfully applied to computer vision due to their powerful modeling capacity with self-attention. However, the excellent performance of transformers heavily depends on enormous training images. Thus, a data-efficient transformer solution is urgently needed. In this work, we propose an early knowledge distillation framework, which is termed as DearKD, to improve the data efficiency required by transformers. Our DearKD is a two-stage framework that first distills the inductive biases from the early intermediate layers of a CNN and then gives the transformer full play by training without distillation. Further, our DearKD can be readily applied to the extreme data-free case where no real images are available. In this case, we propose a boundary-preserving intra-divergence loss based on DeepInversion to further close the performance gap against the full-data counterpart. Extensive experiments on ImageNet, partial ImageNet, data-free setting and other downstream tasks prove the superiority of DearKD over its baselines and state-of-the-art methods.

cs.CV↗

arXiv2022

MetaCVR: Conversion Rate Prediction via Meta Learning in Small-Scale Recommendation Scenarios

Different from large-scale platforms such as Taobao and Amazon, CVR modeling in small-scale recommendation scenarios is more challenging due to the severe Data Distribution Fluctuation (DDF) issue. DDF prevents existing CVR models from being effective since 1) several months of data are needed to train CVR models sufficiently in small scenarios, leading to considerable distribution discrepancy between training and online serving; and 2) e-commerce promotions have significant impacts on small scenarios, leading to distribution uncertainty of the upcoming time period. In this work, we propose a novel CVR method named MetaCVR from a perspective of meta learning to address the DDF issue. Firstly, a base CVR model which consists of a Feature Representation Network (FRN) and output layers is designed and trained sufficiently with samples across months. Then we treat time periods with different data distributions as different occasions and obtain positive and negative prototypes for each occasion using the corresponding samples and the pre-trained FRN. Subsequently, a Distance Metric Network (DMN) is devised to calculate the distance metrics between each sample and all prototypes to facilitate mitigating the distribution uncertainty. At last, we develop an Ensemble Prediction Network (EPN) which incorporates the output of FRN and DMN to make the final CVR prediction. In this stage, we freeze the FRN and train the DMN and EPN with samples from recent time period, therefore effectively easing the distribution discrepancy. To the best of our knowledge, this is the first study of CVR prediction targeting the DDF issue in small-scale recommendation scenarios. Experimental results on real-world datasets validate the superiority of our MetaCVR and online A/B test also shows our model achieves impressive gains of 11.92% on PCVR and 8.64% on GMV.

cs.LG↗

arXiv2022

Observation of Dimension-Crossover of a Tunable 1D Dirac Fermion in Topological Semimetal NbSi$_x$Te$_2$

Condensed matter systems in low dimensions exhibit emergent physics that does not exist in three dimensions. When electrons are confined to one dimension (1D), some significant electronic states appear, such as charge density wave, spin-charge separations and Su-Schrieffer-Heeger (SSH) topological state. However, a clear understanding of how the 1D electronic properties connects with topology is currently lacking. Here we systematically investigated the characteristic 1D Dirac fermion electronic structure originated from the metallic NbTe$_2$ chains on the surface of the composition-tunable layered compound NbSi$_x$Te$_2$ ($x$ = 0.40 and 0.43) using angle-resolved photoemission spectroscopy. We found the Dirac fermion forms a Dirac nodal line structure protected by the combined $\widetilde{\mathcal{M}}{\rm_y}$ and time-reversal symmetry T and proves the NbSi$_x$Te$_2$ system as a topological semimetal, in consistent with the ab-initio calculations. As $x$ decreases, the interaction between adjacent NbTe2 chains increases and Dirac fermion goes through a dimension-crossover from 1D to 2D, as evidenced by the variation of its Fermi surface and Fermi velocity across the Brillouin zone in consistence with a Dirac SSH model. Our findings demonstrate a tunable 1D Dirac electron system, which offers a versatile platform for the exploration of intriguing 1D physics and device applications.

cond-mat.mtrl-sci↗

arXiv2022

Direct Visualization and Manipulation of Tunable Quantum Well State in Semiconducting Nb2SiTe4

Quantum well states (QWSs) can form at the surface or interfaces of materials with confinement potential. They have broad applications in electronic and optical devices such as high mobility electron transistor, photodetector and quantum well laser. The properties of the QWSs are usually the key factors for the performance of the devices. However, direct visualization and manipulation of such states are in general challenging. In this work, by using angle-resolved photoemission spectroscopy (ARPES) and scanning tunneling microscopy/spectroscopy (STM/STS), we directly probe the QWSs generated on the vacuum interface of a narrow band gap semiconductor Nb2SiTe4. Interestingly, the position and splitting of QWSs could be easily manipulated via potassium (K) dosage onto the sample surface. Our results suggest Nb2SiTe4 to be an intriguing semiconductor system to study and engineer the QWSs, which has great potential in device applications.

cond-mat.mtrl-sci↗

arXiv2022

I3CL:Intra- and Inter-Instance Collaborative Learning for Arbitrary-shaped Scene Text Detection

Existing methods for arbitrary-shaped text detection in natural scenes face two critical issues, i.e., 1) fracture detections at the gaps in a text instance; and 2) inaccurate detections of arbitrary-shaped text instances with diverse background context. To address these issues, we propose a novel method named Intra- and Inter-Instance Collaborative Learning (I3CL). Specifically, to address the first issue, we design an effective convolutional module with multiple receptive fields, which is able to collaboratively learn better character and gap feature representations at local and long ranges inside a text instance. To address the second issue, we devise an instance-based transformer module to exploit the dependencies between different text instances and a global context module to exploit the semantic context from the shared background, which are able to collaboratively learn more discriminative text feature representation. In this way, I3CL can effectively exploit the intra- and inter-instance dependencies together in a unified end-to-end trainable framework. Besides, to make full use of the unlabeled data, we design an effective semi-supervised learning method to leverage the pseudo labels via an ensemble strategy. Without bells and whistles, experimental results show that the proposed I3CL sets new state-of-the-art results on three challenging public benchmarks, i.e., an F-measure of 77.5% on ICDAR2019-ArT, 86.9% on Total-Text, and 86.4% on CTW-1500. Notably, our I3CL with the ResNeSt-101 backbone ranked 1st place on the ICDAR2019-ArT leaderboard. The source code will be available at https://github.com/ViTAE-Transformer/ViTAE-Transformer-Scene-Text-Detection.

cs.CV↗

arXiv2022

A Roadmap for Big Model

With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm. Researchers have achieved various outcomes in the construction of BMs and the BM application in many fields. At present, there is a lack of research work that sorts out the overall progress of BMs and guides the follow-up research. In this paper, we cover not only the BM technologies themselves but also the prerequisites for BM training and applications with BMs, dividing the BM review into four parts: Resource, Models, Key Technologies and Application. We introduce 16 specific BM-related topics in those four parts, they are Data, Knowledge, Computing System, Parallel Training System, Language Model, Vision Model, Multi-modal Model, Theory&Interpretability, Commonsense Reasoning, Reliability&Security, Governance, Evaluation, Machine Translation, Text Generation, Dialogue and Protein Research. In each topic, we summarize clearly the current studies and propose some future research directions. At the end of this paper, we conclude the further development of BMs in a more general view.

cs.LG↗

arXiv2022

Looking Outside the Window: Wide-Context Transformer for the Semantic Segmentation of High-Resolution Remote Sensing Images

Long-range contextual information is crucial for the semantic segmentation of High-Resolution (HR) Remote Sensing Images (RSIs). However, image cropping operations, commonly used for training neural networks, limit the perception of long-range contexts in large RSIs. To overcome this limitation, we propose a Wide-Context Network (WiCoNet) for the semantic segmentation of HR RSIs. Apart from extracting local features with a conventional CNN, the WiCoNet has an extra context branch to aggregate information from a larger image area. Moreover, we introduce a Context Transformer to embed contextual information from the context branch and selectively project it onto the local features. The Context Transformer extends the Vision Transformer, an emerging kind of neural network, to model the dual-branch semantic correlations. It overcomes the locality limitation of CNNs and enables the WiCoNet to see the bigger picture before segmenting the land-cover/land-use (LCLU) classes. Ablation studies and comparative experiments conducted on several benchmark datasets demonstrate the effectiveness of the proposed method. In addition, we present a new Beijing Land-Use (BLU) dataset. This is a large-scale HR satellite dataset with high-quality and fine-grained reference labels, which can facilitate future studies in this field.

cs.CV↗

arXiv2022

Salient Objects in Clutter

This paper identifies and addresses a serious design bias of existing salient object detection (SOD) datasets, which unrealistically assume that each image should contain at least one clear and uncluttered salient object. This design bias has led to a saturation in performance for state-of-the-art SOD models when evaluated on existing datasets. However, these models are still far from satisfactory when applied to real-world scenes. Based on our analyses, we propose a new high-quality dataset and update the previous saliency benchmark. Specifically, our dataset, called Salient Objects in Clutter~\textbf{(SOC)}, includes images with both salient and non-salient objects from several common object categories. In addition to object category annotations, each salient image is accompanied by attributes that reflect common challenges in common scenes, which can help provide deeper insight into the SOD problem. Further, with a given saliency encoder, e.g., the backbone network, existing saliency models are designed to achieve mapping from the training image set to the training ground-truth set. We, therefore, argue that improving the dataset can yield higher performance gains than focusing only on the decoder design. With this in mind, we investigate several dataset-enhancement strategies, including label smoothing to implicitly emphasize salient boundaries, random image augmentation to adapt saliency models to various scenarios, and self-supervised learning as a regularization strategy to learn from small datasets. Our extensive results demonstrate the effectiveness of these tricks. We also provide a comprehensive benchmark for SOD, which can be found in our repository: https://github.com/DengPingFan/SODBenchmark.

cs.CV↗

arXiv2022

FIBA: Frequency-Injection based Backdoor Attack in Medical Image Analysis

In recent years, the security of AI systems has drawn increasing research attention, especially in the medical imaging realm. To develop a secure medical image analysis (MIA) system, it is a must to study possible backdoor attacks (BAs), which can embed hidden malicious behaviors into the system. However, designing a unified BA method that can be applied to various MIA systems is challenging due to the diversity of imaging modalities (e.g., X-Ray, CT, and MRI) and analysis tasks (e.g., classification, detection, and segmentation). Most existing BA methods are designed to attack natural image classification models, which apply spatial triggers to training images and inevitably corrupt the semantics of poisoned pixels, leading to the failures of attacking dense prediction models. To address this issue, we propose a novel Frequency-Injection based Backdoor Attack method (FIBA) that is capable of delivering attacks in various MIA tasks. Specifically, FIBA leverages a trigger function in the frequency domain that can inject the low-frequency information of a trigger image into the poisoned image by linearly combining the spectral amplitude of both images. Since it preserves the semantics of the poisoned image pixels, FIBA can perform attacks on both classification and dense prediction models. Experiments on three benchmarks in MIA (i.e., ISIC-2019 for skin lesion classification, KiTS-19 for kidney tumor segmentation, and EAD-2019 for endoscopic artifact detection), validate the effectiveness of FIBA and its superiority over state-of-the-art methods in attacking MIA models as well as bypassing backdoor defense. Source code will be available at https://github.com/HazardFY/FIBA.

cs.CV↗

arXiv2022

Recurrent Glimpse-based Decoder for Detection with Transformer

Although detection with Transformer (DETR) is increasingly popular, its global attention modeling requires an extremely long training period to optimize and achieve promising detection performance. Alternative to existing studies that mainly develop advanced feature or embedding designs to tackle the training issue, we point out that the Region-of-Interest (RoI) based detection refinement can easily help mitigate the difficulty of training for DETR methods. Based on this, we introduce a novel REcurrent Glimpse-based decOder (REGO) in this paper. In particular, the REGO employs a multi-stage recurrent processing structure to help the attention of DETR gradually focus on foreground objects more accurately. In each processing stage, visual features are extracted as glimpse features from RoIs with enlarged bounding box areas of detection results from the previous stage. Then, a glimpse-based decoder is introduced to provide refined detection results based on both the glimpse features and the attention modeling outputs of the previous stage. In practice, REGO can be easily embedded in representative DETR variants while maintaining their fully end-to-end training and inference pipelines. In particular, REGO helps Deformable DETR achieve 44.8 AP on the MSCOCO dataset with only 36 training epochs, compared with the first DETR and the Deformable DETR that require 500 and 50 epochs to achieve comparable performance, respectively. Experiments also show that REGO consistently boosts the performance of different DETR detectors by up to 7% relative gain at the same setting of 50 training epochs. Code is available via https://github.com/zhechen/Deformable-DETR-REGO.

cs.CV↗

arXiv2022

Durable and Recoverable Hydrophilicity of Polyethylene Terephthalate Fabric Prepared with Plasma Selective Etching

Durable delustered PET (PET-TiO2) fabrics super hydrophilic surface has been obtained by plasma selecting etching. The aging effect of their hydrophilicity after plasma treatment has been investigated with storage time. After Ar/O2 radio frequency (RF) plasma treatment for only 7 min, PET-TiO2 fabric showed water contact angle of 0o. After 10 month storage time, it keeps its water contact angle below 75.7o. Further more, with Xenon light irradiation for 10 min, it is firstly found that it has well-recovered water contact angle to 5°. While the contact angle of PET fabric for 7 min returns to 123.0° and its hydrophilicity disappeared almost completely and showed no response to Xenon light irradiation. The water absorption rate of 7 min plasma treated PET-TiO2 fabric increased by 57.54%. By field emission scanning electron microscopy (FE-SEM), X-ray photoelectron spectroscopy (XPS) and X-ray diffraction analysis(XRD) measurement, waviness structure of humps and ridges with irregular particles or pits were found on the plasma treated PET-TiO2 fabric surface and increased Ti atomic percentage was observed. It is verified that TiO2 particles inside PET-TiO2 fiber have been exposed to its surface by plasma selective etching of its organic component. It suppresses the aging effect and is characterized with durable and recoverable hydrophilicity. This one step, quick, green and cost-resonable manufacture method has a pratical application for durable superhydrophilic surfaces.

physics.plasm-ph↗

arXiv2022

SerialTrack: ScalE and Rotation Invariant Augmented Lagrangian Particle Tracking

We present a new particle tracking algorithm to accurately resolve large deformation and rotational motion fields, which takes advantage of both local and global particle tracking algorithms. We call this method the ScalE and Rotation Invariant Augmented Lagrangian Particle Tracking (SerialTrack). This method builds an iterative scale and rotation invariant topology-based feature for each particle within a multi-scale tracking algorithm. The global kinematic compatibility condition is applied as a global augmented Lagrangian constraint to enhance the tracking accuracy. An open source software package implementing this numerical approach to track both 2D and 3D, incremental and cumulative deformation fields is provided.

cs.RO↗

arXiv2022

Halogenation induced transition of superconductor-to-semiconductor in MXene-like MOene with direct band gap and long carrier lifetime

Traditional MXenes with intriguing mechanical and electronic properties, together with the fertilities of elemental compositions and chemical decorations have aroused much attentions. However, the semiconducting traits with direc band gap are extremetely rare among reported MXenes. Thus, broadening the family of MXene beyond carbides and nitrides with unique behaviors is still an extraordinary and fascinating field.

cond-mat.mtrl-sci↗

arXiv2022

Local Partial Zero-Forcing Combining for Cell-Free Massive MIMO Systems

Cell-free massive multiple-input multiple-output (MIMO) provides more uniform spectral efficiency (SE) for users (UEs) than cellular technology. The main challenge to achieve the benefits of cell-free massive MIMO is to realize signal processing in a scalable way. In this paper, we consider scalable fullpilot zero-forcing (FZF), partial FZF (PFZF), protective weak PFZF (PWPFZF), and local regularized ZF (LRZF) combining by exploiting channel statistics. We derive closed-form expressions of the uplink SE for FZF, PFZF, and PWPFZF combining with large-scale fading decoding over independent Rayleigh fading channels, taking channel estimation errors and pilot contamination into account. Moreover, we investigate the impact of the number of pilot sequences, antennas per AP, and APs on the performance. Numerical results show that LRZF provides the highest SE. However, PWPFZF is preferable when the number of pilot sequences is large and the number of antennas per AP is small. The reason is that PWPFZF has lower computational complexity and the SE expression can be computed in closed-form. Furthermore, we investigate the performance of PWPFZF combining with fractional power control and the numerical results show that it improves the performance of weak UEs and realizes uniformly good service for all UEs in a scalable fashion.

cs.IT↗

arXiv2022

Learning Affordance Grounding from Exocentric Images

Affordance grounding, a task to ground (i.e., localize) action possibility region in objects, which faces the challenge of establishing an explicit link with object parts due to the diversity of interactive affordance. Human has the ability that transform the various exocentric interactions to invariant egocentric affordance so as to counter the impact of interactive diversity. To empower an agent with such ability, this paper proposes a task of affordance grounding from exocentric view, i.e., given exocentric human-object interaction and egocentric object images, learning the affordance knowledge of the object and transferring it to the egocentric image using only the affordance label as supervision. To this end, we devise a cross-view knowledge transfer framework that extracts affordance-specific features from exocentric interactions and enhances the perception of affordance regions by preserving affordance correlation. Specifically, an Affordance Invariance Mining module is devised to extract specific clues by minimizing the intra-class differences originated from interaction habits in exocentric images. Besides, an Affordance Co-relation Preserving strategy is presented to perceive and localize affordance by aligning the co-relation matrix of predicted results between the two views. Particularly, an affordance grounding dataset named AGD20K is constructed by collecting and labeling over 20K images from 36 affordance categories. Experimental results demonstrate that our method outperforms the representative models in terms of objective metrics and visual quality. Code: github.com/lhc1224/Cross-View-AG.

cs.CV↗

arXiv2022

AlignTransformer: Hierarchical Alignment of Visual Regions and Disease Tags for Medical Report Generation

Recently, medical report generation, which aims to automatically generate a long and coherent descriptive paragraph of a given medical image, has received growing research interests. Different from the general image captioning tasks, medical report generation is more challenging for data-driven neural models. This is mainly due to 1) the serious data bias: the normal visual regions dominate the dataset over the abnormal visual regions, and 2) the very long sequence. To alleviate above two problems, we propose an AlignTransformer framework, which includes the Align Hierarchical Attention (AHA) and the Multi-Grained Transformer (MGT) modules: 1) AHA module first predicts the disease tags from the input image and then learns the multi-grained visual features by hierarchically aligning the visual regions and disease tags. The acquired disease-grounded visual features can better represent the abnormal regions of the input image, which could alleviate data bias problem; 2) MGT module effectively uses the multi-grained features and Transformer framework to generate the long medical report. The experiments on the public IU-Xray and MIMIC-CXR datasets show that the AlignTransformer can achieve results competitive with state-of-the-art methods on the two datasets. Moreover, the human evaluation conducted by professional radiologists further proves the effectiveness of our approach.

eess.IV↗