arXiv subjects

Zhipeng Zhang

Publications and source records attributed to Zhipeng Zhang.

122 records · Page 7Linked to original sources

arXiv2022

Attract me to Buy: Advertisement Copywriting Generation with Multimodal Multi-structured Information

Recently, online shopping has gradually become a common way of shopping for people all over the world. Wonderful merchandise advertisements often attract more people to buy. These advertisements properly integrate multimodal multi-structured information of commodities, such as visual spatial information and fine-grained structure information. However, traditional multimodal text generation focuses on the conventional description of what existed and happened, which does not match the requirement of advertisement copywriting in the real world. Because advertisement copywriting has a vivid language style and higher requirements of faithfulness. Unfortunately, there is a lack of reusable evaluation frameworks and a scarcity of datasets. Therefore, we present a dataset, E-MMAD (e-commercial multimodal multi-structured advertisement copywriting), which requires, and supports much more detailed information in text generation. Noticeably, it is one of the largest video captioning datasets in this field. Accordingly, we propose a baseline method and faithfulness evaluation metric on the strength of structured information reasoning to solve the demand in reality on this dataset. It surpasses the previous methods by a large margin on all metrics. The dataset and method are coming soon on \url{https://e-mmad.github.io/e-mmad.net/index.html}.

cs.CL↗

arXiv2022

Learning Target-aware Representation for Visual Tracking via Informative Interactions

We introduce a novel backbone architecture to improve target-perception ability of feature representation for tracking. Specifically, having observed that de facto frameworks perform feature matching simply using the outputs from backbone for target localization, there is no direct feedback from the matching module to the backbone network, especially the shallow layers. More concretely, only the matching module can directly access the target information (in the reference frame), while the representation learning of candidate frame is blind to the reference target. As a consequence, the accumulation effect of target-irrelevant interference in the shallow stages may degrade the feature quality of deeper layers. In this paper, we approach the problem from a different angle by conducting multiple branch-wise interactions inside the Siamese-like backbone networks (InBN). At the core of InBN is a general interaction modeler (GIM) that injects the prior knowledge of reference image to different stages of the backbone network, leading to better target-perception and robust distractor-resistance of candidate feature representation with negligible computation cost. The proposed GIM module and InBN mechanism are general and applicable to different backbone types including CNN and Transformer for improvements, as evidenced by our extensive experiments on multiple benchmarks. In particular, the CNN version (based on SiamCAR) improves the baseline with 3.2/6.9 absolute gains of SUC on LaSOT/TNL2K, respectively. The Transformer version obtains SUC scores of 65.7/52.0 on LaSOT/TNL2K, which are on par with recent state of the arts. Code and models will be released.

cs.CV↗

arXiv2021

One More Check: Making "Fake Background" Be Tracked Again

The one-shot multi-object tracking, which integrates object detection and ID embedding extraction into a unified network, has achieved groundbreaking results in recent years. However, current one-shot trackers solely rely on single-frame detections to predict candidate bounding boxes, which may be unreliable when facing disastrous visual degradation, e.g., motion blur, occlusions. Once a target bounding box is mistakenly classified as background by the detector, the temporal consistency of its corresponding tracklet will be no longer maintained. In this paper, we set out to restore the bounding boxes misclassified as ``fake background'' by proposing a re-check network. The re-check network innovatively expands the role of ID embedding from data association to motion forecasting by effectively propagating previous tracklets to the current frame with a small overhead. Note that the propagation results are yielded by an independent and efficient embedding search, preventing the model from over-relying on detection results. Eventually, it helps to reload the ``fake background'' and repair the broken tracklets. Building on a strong baseline CSTrack, we construct a new one-shot tracker and achieve favorable gains by 70.7 $\rightarrow$ 76.4, 70.6 $\rightarrow$ 76.3 MOTA on MOT16 and MOT17, respectively. It also reaches a new state-of-the-art MOTA and IDF1 performance. Code is released at https://github.com/JudasDie/SOTS.

cs.CV↗

arXiv2021

Learn to Match: Automatic Matching Network Design for Visual Tracking

Siamese tracking has achieved groundbreaking performance in recent years, where the essence is the efficient matching operator cross-correlation and its variants. Besides the remarkable success, it is important to note that the heuristic matching network design relies heavily on expert experience. Moreover, we experimentally find that one sole matching operator is difficult to guarantee stable tracking in all challenging environments. Thus, in this work, we introduce six novel matching operators from the perspective of feature fusion instead of explicit similarity learning, namely Concatenation, Pointwise-Addition, Pairwise-Relation, FiLM, Simple-Transformer and Transductive-Guidance, to explore more feasibility on matching operator selection. The analyses reveal these operators' selective adaptability on different environment degradation types, which inspires us to combine them to explore complementary features. To this end, we propose binary channel manipulation (BCM) to search for the optimal combination of these operators. BCM determines to retrain or discard one operator by learning its contribution to other tracking steps. By inserting the learned matching networks to a strong baseline tracker Ocean, our model achieves favorable gains by $67.2 \rightarrow 71.4$, $52.6 \rightarrow 58.3$, $70.3 \rightarrow 76.0$ success on OTB100, LaSOT, and TrackingNet, respectively. Notably, Our tracker, dubbed AutoMatch, uses less than half of training data/time than the baseline tracker, and runs at 50 FPS using PyTorch. Code and model will be released at https://github.com/JudasDie/SOTS.

cs.CV↗

arXiv2021

Towards More Flexible and Accurate Object Tracking with Natural Language: Algorithms and Benchmark

Tracking by natural language specification is a new rising research topic that aims at locating the target object in the video sequence based on its language description. Compared with traditional bounding box (BBox) based tracking, this setting guides object tracking with high-level semantic information, addresses the ambiguity of BBox, and links local and global search organically together. Those benefits may bring more flexible, robust and accurate tracking performance in practical scenarios. However, existing natural language initialized trackers are developed and compared on benchmark datasets proposed for tracking-by-BBox, which can't reflect the true power of tracking-by-language. In this work, we propose a new benchmark specifically dedicated to the tracking-by-language, including a large scale dataset, strong and diverse baseline methods. Specifically, we collect 2k video sequences (contains a total of 1,244,340 frames, 663 words) and split 1300/700 for the train/testing respectively. We densely annotate one sentence in English and corresponding bounding boxes of the target object for each video. We also introduce two new challenges into TNL2K for the object tracking task, i.e., adversarial samples and modality switch. A strong baseline method based on an adaptive local-global-search scheme is proposed for future works to compare. We believe this benchmark will greatly boost related researches on natural language guided tracking.

cs.CV↗

arXiv2020

Towards Accurate Pixel-wise Object Tracking by Attention Retrieval

The encoding of the target in object tracking moves from the coarse bounding-box to fine-grained segmentation map recently. Revisiting de facto real-time approaches that are capable of predicting mask during tracking, we observed that they usually fork a light branch from the backbone network for segmentation. Although efficient, directly fusing backbone features without considering the negative influence of background clutter tends to introduce false-negative predictions, lagging the segmentation accuracy. To mitigate this problem, we propose an attention retrieval network (ARN) to perform soft spatial constraints on backbone features. We first build a look-up-table (LUT) with the ground-truth mask in the starting frame, and then retrieves the LUT to obtain an attention map for spatial constraints. Moreover, we introduce a multi-resolution multi-stage segmentation network (MMS) to further weaken the influence of background clutter by reusing the predicted mask to filter backbone features. Our approach set a new state-of-the-art on recent pixel-wise object tracking benchmark VOT2020 while running at 40 fps. Notably, the proposed model surpasses SiamMask by 11.7/4.2/5.5 points on VOT2020, DAVIS2016, and DAVIS2017, respectively. We will release our code at https://github.com/researchmm/TracKit.

cs.CV↗

arXiv2020

Ocean: Object-aware Anchor-free Tracking

Anchor-based Siamese trackers have achieved remarkable advancements in accuracy, yet the further improvement is restricted by the lagged tracking robustness. We find the underlying reason is that the regression network in anchor-based methods is only trained on the positive anchor boxes (i.e., $IoU \geq0.6$). This mechanism makes it difficult to refine the anchors whose overlap with the target objects are small. In this paper, we propose a novel object-aware anchor-free network to address this issue. First, instead of refining the reference anchor boxes, we directly predict the position and scale of target objects in an anchor-free fashion. Since each pixel in groundtruth boxes is well trained, the tracker is capable of rectifying inexact predictions of target objects during inference. Second, we introduce a feature alignment module to learn an object-aware feature from predicted bounding boxes. The object-aware feature can further contribute to the classification of target objects and background. Moreover, we present a novel tracking framework based on the anchor-free model. The experiments show that our anchor-free tracker achieves state-of-the-art performance on five benchmarks, including VOT-2018, VOT-2019, OTB-100, GOT-10k and LaSOT. The source code is available at https://github.com/researchmm/TracKit.

cs.CV↗

arXiv2019

Record-Breaking Magnetoresistance at the Edge of a Microflake of Natural Graphite

Placing several electrodes at the edge of a micrometer-size Sri Lankan natural graphite sample at distances comparable to the size of the internal crystalline regions, we found record values for the change of the resistance with magnetic field. At low temperatures and at $B \sim 21$T the magnetoresistance (MR) reaches $\sim 10^7$%. The MR values exceed by far all earlier reported ones for graphite and they are comparable or even larger (at $T > 50$K) than the largest reported in solids including the Weyl semimetals. The origin of this large MR lies in the existence of highly conducting 2D interfaces aligned parallel to the graphene planes.

cond-mat.str-el↗

arXiv2019

Deeper and Wider Siamese Networks for Real-Time Visual Tracking

Siamese networks have drawn great attention in visual tracking because of their balanced accuracy and speed. However, the backbone networks used in Siamese trackers are relatively shallow, such as AlexNet [18], which does not fully take advantage of the capability of modern deep neural networks. In this paper, we investigate how to leverage deeper and wider convolutional neural networks to enhance tracking robustness and accuracy. We observe that direct replacement of backbones with existing powerful architectures, such as ResNet [14] and Inception [33], does not bring improvements. The main reasons are that 1)large increases in the receptive field of neurons lead to reduced feature discriminability and localization precision; and 2) the network padding for convolutions induces a positional bias in learning. To address these issues, we propose new residual modules to eliminate the negative impact of padding, and further design new architectures using these modules with controlled receptive field size and network stride. The designed architectures are lightweight and guarantee real-time tracking speed when applied to SiamFC [2] and SiamRPN [20]. Experiments show that solely due to the proposed network architectures, our SiamFC+ and SiamRPN+ obtain up to 9.8%/5.7% (AUC), 23.3%/8.8% (EAO) and 24.4%/25.0% (EAO) relative improvements over the original versions [2, 20] on the OTB-15, VOT-16 and VOT-17 datasets, respectively.

cs.CV↗

arXiv2017

Detection of incoherent broadband terahertz light using antenna-coupled high-electron-mobility field-effect transistors

The sensitivity of direct terahertz detectors based on self-mixing of terahertz electromagnetic wave in field-effect transistors is being improved with noise-equivalent power close to that of Schottky-barrier-diode detectors. Here we report such detectors based on AlGaN/GaN two-dimensional electron gas at 77~K are able to sense broadband and incoherent terahertz radiation. The measured photocurrent as a function of the gate voltage agrees well with the self-mixing model and the spectral response is mainly determined by the antenna. A Fourier-transform spectrometer equipped with detectors designed for 340, 650 and 900~GHz bands allows for terahertz spectroscopy in a frequency range from 0.1 to 2.0~THz. The 900~GHz detector at 77~K offers an optical sensitivity about $1~\mathrm{pW/\sqrt{Hz}}$ being comparable to a commercial silicon bolometer at 4.2~K. By further improving the sensitivity, room-temperature detectors would find applications in active/passive terahertz imaging and terahertz spectroscopy.

cond-mat.mes-hall↗

arXiv2017

High-efficiency spontaneous terahertz emission from electrically-excited plasmons in AlGaN/GaN two-dimensional electron gas

The advance of terahertz science and technology yet lays wait for the breakthrough in high-efficiency and high-power solid-state terahertz sources applicable at room temperature. Plasmon in two-dimensional electron gas (2DEG) has long been pursued as a type of promising active medium for terahertz emitters. However, a high wall-plug efficiency sufficient for high-power operation has not been achieved. Here we report spontaneous terahertz emission tunable in a frequency range from 0.3 to 2.5 THz through electrical excitation of plasmon or plasmon-polariton modes in a grating-gate coupled AlGaN/GaN 2DEG at 7 K. A wall-plug efficiency ranging from $10^{-6}$ to $10^{-1}$ is found by tuning the energy (4.5 eV to 0.5 eV) of tunneling electrons between the grating gates and the 2DEG. The total emission power, albeit below 15 nW, is already detectable by using a direct terahertz field-effect transistor detector based on the same type of 2DEG. The findings indicate that both high-efficiency and high-power terahertz plasmon-polariton emission could be obtained by a proper injection of tunneling electrons with low energy and high current density.

cond-mat.mes-hall↗

arXiv2016

Uniform Regularity and Vanishing Viscosity limit for the chemotaxis-Navier-Stokes system in a 3D bounded domain

We investigate the uniform regularity and vanishing viscosity limit for the incompressible chemotaxis-Navier-Stokes system in a smooth bounded domain $Ω\subset\mathbb{R}^3$. It is shown that there exists a unique strong solution of the incompressible chemotaxis-Navier-Stokes system in a finite time interval which is independent of the viscosity coefficient. Moreover, the solution is uniformly bounded in a conormal Sobolev space, which allows us to take the vanishing viscosity limit to obtain the incompressible inviscid chemotaxis-Navier-Stokes system.

math.AP↗

arXiv2016

Zero kinematic viscosity-magnetic diffusion limit of the incompressible viscous magnetohydrodynamic equations with Navier boundary conditions

We investigate the zero kinematic viscosity-magnetic diffusion limit of the incompressible viscous magnetohydrodynamic equations with Navier boundary conditions in a smooth bounded domain $Ω\subset\mathbb{R}^3$. We obtain the uniform regularity of solutions with respect to the kinematic viscosity coefficient and the magnetic diffusivity coefficient. These solutions are uniformly bounded in a conormal Sobolev space and $W^{1,\infty}(Ω)$ which allow us to take the zero kinematic viscosity-magnetic diffusion limit. Moreover, we also get the rates of convergence.

math.AP↗

arXiv2013

Mott Variable Range Hopping and Weak Antilocalization Effect in Heteroepitaxial Na2IrO3 Thin Films

Iridate thin films are a prerequisite for any application utilizing their cooperative effects resulting from the interplay of stron spin-orbit coupling and electronic correlations. Here, heteroepitaxial Na2IrO3 thin films with excellent (001) out-of-plane crystalline orientation and well defined in-plane epitaxial relationship are presented on various oxide substrates. Resistivity is dominated by a three-dimensional variable range hopping mechanism in a large temperature range between 300 K and 40 K. Optical experiments show the onset of a small optical gap of about 200 meV and a splitting of the Ir 5d-t2g manifold. Positive magnetoresistance below 3 T and 25 K shows signatures of a weak antilocalization effect. This effect can be associated with surface states in a topological insulator and hence supports proposals for a topological insulator phase present in Na2IrO3.

cond-mat.str-el↗