arXiv subjects

Shan Zhang

Publications and source records attributed to Shan Zhang.

At least 55 records · Page 3Linked to original sources

arXiv2024

LW-DETR: A Transformer Replacement to YOLO for Real-Time Detection

In this paper, we present a light-weight detection transformer, LW-DETR, which outperforms YOLOs for real-time object detection. The architecture is a simple stack of a ViT encoder, a projector, and a shallow DETR decoder. Our approach leverages recent advanced techniques, such as training-effective techniques, e.g., improved loss and pretraining, and interleaved window and global attentions for reducing the ViT encoder complexity. We improve the ViT encoder by aggregating multi-level feature maps, and the intermediate and final feature maps in the ViT encoder, forming richer feature maps, and introduce window-major feature map organization for improving the efficiency of interleaved attention computation. Experimental results demonstrate that the proposed approach is superior over existing real-time detectors, e.g., YOLO and its variants, on COCO and other benchmark datasets. Code and models are available at (https://github.com/Atten4Vis/LW-DETR).

cs.CV↗

arXiv2024

SQUID oscillations in PbTe nanowire networks

Network structures by semiconductor nanowires hold great promise for advanced quantum devices, especially for applications in topological quantum computing. In this study, we created networks of PbTe nanowires arranged in loop configurations. Using shadow-wall epitaxy, we defined superconducting quantum interference devices (SQUIDs) using the superconductor Pb. These SQUIDs exhibit oscillations in supercurrent upon the scanning of a magnetic field. Most of the oscillations can be fitted assuming a sinusoidal current-phase relation for each Josephson junction. Under certain conditions, the oscillations are found to be skewed, suggesting possible deviation from a sinusoidal behavior. Our results highlight the potential of PbTe nanowires for building complex quantum devices in the form of networks.

cond-mat.mes-hall↗

arXiv2024

Gate-tunable subband degeneracy in semiconductor nanowires

Degeneracy and symmetry have a profound relation in quantum systems. Here, we report gate-tunable subband degeneracy in PbTe nanowires with a nearly symmetric cross-sectional shape. The degeneracy is revealed in electron transport by the absence of a quantized plateau. Utilizing a dual gate design, we can apply an electric field to lift the degeneracy, reflected as emergence of the plateau. This degeneracy and its tunable lifting were challenging to observe in previous nanowire experiments, possibly due to disorder. Numerical simulations can qualitatively capture our observation, shedding light on device parameters for future applications.

cond-mat.mes-hall↗

arXiv2024

Gate-Compatible Circuit Quantum Electrodynamics in a Three-Dimensional Cavity Architecture

Semiconductor-based superconducting qubits offer a versatile platform for studying hybrid quantum devices in circuit quantum electrodynamics (cQED) architecture. Most of these cQED experiments utilize coplanar waveguides, where the incorporation of DC gate lines is straightforward. Here, we present a technique for probing gate-tunable hybrid devices using a three-dimensional (3D) microwave cavity. A recess is machined inside the cavity wall for the placement of devices and gate lines. We validate this design using a hybrid device based on an InAs-Al nanowire Josephson junction. The coupling between the device and the cavity is facilitated by a long superconducting strip, the antenna. The Josephson junction and the antenna together form a gatemon qubit. We further demonstrate the gate-tunable cavity shift and two-tone qubit spectroscopy. This technique could be used to probe various quantum devices and materials in a 3D cQED architecture that requires DC gate voltages.

quant-ph↗

arXiv2024

On the optimal control problem for the k-FORQ/MCH equation with viscosity

In this paper, we investigate the optimal control problem for the k-FORQ/MCH equation with strong viscosity.We prove the existence and uniqueness of this equation under the initial and boundary conditions by Galerkin method. From these results and the Lion's theory we deduce the existence of an optimal solution to the control problem governed by the viscous k-FORQ/MCH equation. Using augmented Lagrangian method, we deduce the first-order necessary optimality condition and two second-order sufficient optimality conditions.

math.OC↗

arXiv2023

Semantic-Aware Autoregressive Image Modeling for Visual Representation Learning

The development of autoregressive modeling (AM) in computer vision lags behind natural language processing (NLP) in self-supervised pre-training. This is mainly caused by the challenge that images are not sequential signals and lack a natural order when applying autoregressive modeling. In this study, inspired by human beings' way of grasping an image, i.e., focusing on the main object first, we present a semantic-aware autoregressive image modeling (SemAIM) method to tackle this challenge. The key insight of SemAIM is to autoregressive model images from the semantic patches to the less semantic patches. To this end, we first calculate a semantic-aware permutation of patches according to their feature similarities and then perform the autoregression procedure based on the permutation. In addition, considering that the raw pixels of patches are low-level signals and are not ideal prediction targets for learning high-level semantic representation, we also explore utilizing the patch features as the prediction targets. Extensive experiments are conducted on a broad range of downstream tasks, including image classification, object detection, and instance/semantic segmentation, to evaluate the performance of SemAIM. The results demonstrate SemAIM achieves state-of-the-art performance compared with other self-supervised methods. Specifically, with ViT-B, SemAIM achieves 84.1% top-1 accuracy for fine-tuning on ImageNet, 51.3% AP and 45.4% AP for object detection and instance segmentation on COCO, which outperforms the vanilla MAE by 0.5%, 1.0%, and 0.5%, respectively.

cs.CV↗

arXiv2023

In situ tuning of dynamical Coulomb blockade on Andreev bound states in hybrid nanowire devices

Electron interactions in quantum devices can exhibit intriguing phenomena. One example is assembling an electronic device in series with an on-chip resistor. The quantum laws of electricity of the device is modified at low energies and temperatures by dissipative interactions induced by the resistor, a phenomenon known as dynamical Coulomb blockade (DCB). The DCB strength is usually non-adjustable in a fixed environment defined by the resistor. Here, we design an on-chip circuit for InAs-Al hybrid nanowires where the DCB strength can be gate-tuned in situ. InAs-Al nanowires could host Andreev or Majorana zero-energy states. This technique enables tracking the evolution of the same state while tuning the DCB strength from weak to strong. We observe the transition from a zero-bias conductance peak to split peaks for Andreev zero-energy states. Our technique opens the door to in situ tuning interaction strength on zero-energy states.

cond-mat.mes-hall↗

arXiv2023

Probing the Galactic halo with RR Lyrae stars -- IV. On the Oosterhoff dichotomy of RR Lyrae stars

We use 3653 (2661 RRab, 992 RRc) RR Lyrae stars (RRLs) with 7D (3D position, 3D velocity, and metallicity) information selected from SDSS, LAMOST, and Gaia EDR3, and divide the sample into two Oosterhoff groups (Oo I and Oo II) according to their amplitude-period behaviour in the Bailey Diagram. We present a comparative study of these two groups based on chemistry, kinematics, and dynamics. We find that Oo I RRLs are relatively more metal rich, with predominately radially dominated orbits and large eccentricities, while Oo II RRLs are relatively more metal poor, and have mildly radially dominated orbits. The Oosterhoff dichotomy of the Milky Way's halo is more apparent for the inner-halo region than for the outer-halo region. Additionally, we also search for this phenomenon in the halos of the two largest satellite galaxies, the Large and Small Magellanic clouds (LMC, SMC), and compare over different bins in metallicity. We find that the Oosterhoff dichotomy is not immutable, and varies based on position in the Galaxy and from galaxy-to-galaxy. We conclude that the Oosterhoff dichotomy is the result of a combination of stellar and galactic evolution, and that it is much more complex than the dichotomy originally identified in Galactic globular clusters.

astro-ph.GA↗

arXiv2023

Group DETR: Fast DETR Training with Group-Wise One-to-Many Assignment

Detection transformer (DETR) relies on one-to-one assignment, assigning one ground-truth object to one prediction, for end-to-end detection without NMS post-processing. It is known that one-to-many assignment, assigning one ground-truth object to multiple predictions, succeeds in detection methods such as Faster R-CNN and FCOS. While the naive one-to-many assignment does not work for DETR, and it remains challenging to apply one-to-many assignment for DETR training. In this paper, we introduce Group DETR, a simple yet efficient DETR training approach that introduces a group-wise way for one-to-many assignment. This approach involves using multiple groups of object queries, conducting one-to-one assignment within each group, and performing decoder self-attention separately. It resembles data augmentation with automatically-learned object query augmentation. It is also equivalent to simultaneously training parameter-sharing networks of the same architecture, introducing more supervision and thus improving DETR training. The inference process is the same as DETR trained normally and only needs one group of queries without any architecture modification. Group DETR is versatile and is applicable to various DETR variants. The experiments show that Group DETR significantly speeds up the training convergence and improves the performance of various DETR-based models. Code will be available at \url{https://github.com/Atten4Vis/GroupDETR}.

cs.CV↗

arXiv2023

Multi-Mode Online Knowledge Distillation for Self-Supervised Visual Representation Learning

Self-supervised learning (SSL) has made remarkable progress in visual representation learning. Some studies combine SSL with knowledge distillation (SSL-KD) to boost the representation learning performance of small models. In this study, we propose a Multi-mode Online Knowledge Distillation method (MOKD) to boost self-supervised visual representation learning. Different from existing SSL-KD methods that transfer knowledge from a static pre-trained teacher to a student, in MOKD, two different models learn collaboratively in a self-supervised manner. Specifically, MOKD consists of two distillation modes: self-distillation and cross-distillation modes. Among them, self-distillation performs self-supervised learning for each model independently, while cross-distillation realizes knowledge interaction between different models. In cross-distillation, a cross-attention feature search strategy is proposed to enhance the semantic feature alignment between different models. As a result, the two models can absorb knowledge from each other to boost their representation learning performance. Extensive experimental results on different backbones and datasets demonstrate that two heterogeneous models can benefit from MOKD and outperform their independently trained baseline. In addition, MOKD also outperforms existing SSL-KD methods for both the student and teacher models.

cs.CV↗

arXiv2023

Conductance Quantization in PbTe Nanowires

PbTe nanowires coupled to a superconductor have recently been proposed as a potential Majorana platform. The hallmark of the one-dimensional nature of ballistic nanowires is their quantized conductance. Here, we report the observation of conductance plateaus at multiples of the quantized value $2e^2/h$ in PbTe nanowires at finite magnetic fields. The quantized plateaus, as a function of source-drain bias and magnetic field, allow for the extraction of the Landé $g$-factor, sub-band spacing and effective mass. The coefficient of 2 in the plateau conductance indicates the presence of valley degeneracy arising from the crystal orientation of the nanowires, which are grown on a (001) substrate. Occasionally, this degeneracy can be lifted by a gate voltage that breaks the mirror symmetry. Our results demonstrate the one-dimensionality of PbTe nanowires and fulfill one of the necessary conditions for the realization of Majorana zero modes.

cond-mat.mes-hall↗

arXiv2023

Teacher-Student Knowledge Distillation for Radar Perception on Embedded Accelerators

Many radar signal processing methodologies are being developed for critical road safety perception tasks. Unfortunately, these signal processing algorithms are often poorly suited to run on embedded hardware accelerators used in automobiles. Conversely, end-to-end machine learning (ML) approaches better exploit the performance gains brought by specialized accelerators. In this paper, we propose a teacher-student knowledge distillation approach for low-level radar perception tasks. We utilize a hybrid model for stationary object detection as a teacher to train an end-to-end ML student model. The student can efficiently harness embedded compute for real-time deployment. We demonstrate that the proposed student model runs at speeds 100x faster than the teacher model.

cs.AI↗

arXiv2023

Gatemon qubit based on a thin InAs-Al hybrid nanowire

We study a gate-tunable superconducting qubit (gatemon) based on a thin InAs-Al hybrid nanowire. Using a gate voltage to control its Josephson energy, the gatemon can reach the strong coupling regime to a microwave cavity. In the dispersive regime, we extract the energy relaxation time $T_1\sim$0.56 $μ$s and the dephasing time $T_2^* \sim$0.38 $μ$s. Since thin InAs-Al nanowires can have fewer or single sub-band occupation and recent transport experiment shows the existence of nearly quantized zero-bias conductance peaks, our result holds relevancy for detecting Majorana zero modes in thin InAs-Al nanowires using circuit quantum electrodynamics.

cond-mat.mes-hall↗

arXiv2022

Semantics-Consistent Feature Search for Self-Supervised Visual Representation Learning

In contrastive self-supervised learning, the common way to learn discriminative representation is to pull different augmented "views" of the same image closer while pushing all other images further apart, which has been proven to be effective. However, it is unavoidable to construct undesirable views containing different semantic concepts during the augmentation procedure. It would damage the semantic consistency of representation to pull these augmentations closer in the feature space indiscriminately. In this study, we introduce feature-level augmentation and propose a novel semantics-consistent feature search (SCFS) method to mitigate this negative effect. The main idea of SCFS is to adaptively search semantics-consistent features to enhance the contrast between semantics-consistent regions in different augmentations. Thus, the trained model can learn to focus on meaningful object regions, improving the semantic representation ability. Extensive experiments conducted on different datasets and tasks demonstrate that SCFS effectively improves the performance of self-supervised learning and achieves state-of-the-art performance on different downstream tasks.

cs.CV↗

arXiv2022

Group DETR v2: Strong Object Detector with Encoder-Decoder Pretraining

We present a strong object detector with encoder-decoder pretraining and finetuning. Our method, called Group DETR v2, is built upon a vision transformer encoder ViT-Huge~\cite{dosovitskiy2020image}, a DETR variant DINO~\cite{zhang2022dino}, and an efficient DETR training method Group DETR~\cite{chen2022group}. The training process consists of self-supervised pretraining and finetuning a ViT-Huge encoder on ImageNet-1K, pretraining the detector on Object365, and finally finetuning it on COCO. Group DETR v2 achieves $\textbf{64.5}$ mAP on COCO test-dev, and establishes a new SoTA on the COCO leaderboard https://paperswithcode.com/sota/object-detection-on-coco

cs.CV↗

arXiv2022

Time-rEversed diffusioN tEnsor Transformer: A new TENET of Few-Shot Object Detection

In this paper, we tackle the challenging problem of Few-shot Object Detection. Existing FSOD pipelines (i) use average-pooled representations that result in information loss; and/or (ii) discard position information that can help detect object instances. Consequently, such pipelines are sensitive to large intra-class appearance and geometric variations between support and query images. To address these drawbacks, we propose a Time-rEversed diffusioN tEnsor Transformer (TENET), which i) forms high-order tensor representations that capture multi-way feature occurrences that are highly discriminative, and ii) uses a transformer that dynamically extracts correlations between the query image and the entire support set, instead of a single average-pooled support embedding. We also propose a Transformer Relation Head (TRH), equipped with higher-order representations, which encodes correlations between query regions and the entire support set, while being sensitive to the positional variability of object instances. Our model achieves state-of-the-art results on PASCAL VOC, FSOD, and COCO.

cs.CV↗

arXiv2022

Atomic Origin of Annealing Embrittlement in Metallic Glasses

An atomistic understanding of annealing embrittlement is a longstanding issue for metallic glasses, which is still lacking due to the insurmountable gap between the thermal history of atomic models and laboratory-made samples. Here, based on a thermal-cycling annealing method that can vary the effective quenching rate over ten orders of magnitude, we perform an atomistic study of the ductile-brittle transition in a ternary model metallic glass, which can be keyed to the annealing embrittlement in bulk metallic glasses. We reveal that thermal annealing can effectively obliterate thermally active-able "defects", which are abundant in the hyper-quenched and ductile glass but gives rise to strain-created shear events in the well-annealed and brittle glass. While the activation of the strain-created events eventually causes single shear banding, other local structural disruptions can be "healed" by the same type of events upon stress reversal, thereby hindering shear band broadening or multiplication, and resulting in annealing embrittlement.

cond-mat.mtrl-sci↗

arXiv2022

A Neural Network Based Method with Transfer Learning for Genetic Data Analysis

Transfer learning has emerged as a powerful technique in many application problems, such as computer vision and natural language processing. However, this technique is largely ignored in application to genetic data analysis. In this paper, we combine transfer learning technique with a neural network based method(expectile neural networks). With transfer learning, instead of starting the learning process from scratch, we start from one task that have been learned when solving a different task. We leverage previous learnings and avoid starting from scratch to improve the model performance by passing information gained in different but related task. To demonstrate the performance, we run two real data sets. By using transfer learning algorithm, the performance of expectile neural networks is improved compared to expectile neural network without using transfer learning technique.

stat.AP↗