arXiv subjects

Son Tran

Publications and source records attributed to Son Tran.

28 records · Page 2Linked to original sources

arXiv2022

Vision-Language Pre-Training with Triple Contrastive Learning

Vision-language representation learning largely benefits from image-text alignment through contrastive losses (e.g., InfoNCE loss). The success of this alignment strategy is attributed to its capability in maximizing the mutual information (MI) between an image and its matched text. However, simply performing cross-modal alignment (CMA) ignores data potential within each modality, which may result in degraded representations. For instance, although CMA-based models are able to map image-text pairs close together in the embedding space, they fail to ensure that similar inputs from the same modality stay close by. This problem can get even worse when the pre-training data is noisy. In this paper, we propose triple contrastive learning (TCL) for vision-language pre-training by leveraging both cross-modal and intra-modal self-supervision. Besides CMA, TCL introduces an intra-modal contrastive objective to provide complementary benefits in representation learning. To take advantage of localized and structural information from image and text input, TCL further maximizes the average MI between local regions of image/text and their global summary. To the best of our knowledge, ours is the first work that takes into account local structure information for multi-modality representation learning. Experimental evaluations show that our approach is competitive and achieves the new state of the art on various common down-stream vision-language tasks such as image-text retrieval and visual question answering.

cs.CV↗

arXiv2022

Multi-modal Alignment using Representation Codebook

Aligning signals from different modalities is an important step in vision-language representation learning as it affects the performance of later stages such as cross-modality fusion. Since image and text typically reside in different regions of the feature space, directly aligning them at instance level is challenging especially when features are still evolving during training. In this paper, we propose to align at a higher and more stable level using cluster representation. Specifically, we treat image and text as two "views" of the same entity, and encode them into a joint vision-language coding space spanned by a dictionary of cluster centers (codebook). We contrast positive and negative samples via their cluster assignments while simultaneously optimizing the cluster centers. To further smooth out the learning process, we adopt a teacher-student distillation paradigm, where the momentum teacher of one view guides the student learning of the other. We evaluated our approach on common vision language benchmarks and obtain new SoTA on zero-shot cross modality retrieval while being competitive on various other transfer tasks.

cs.CV↗

arXiv2021

Parallel Multi-Scale Networks with Deep Supervision for Hand Keypoint Detection

Keypoint detection plays an important role in a wide range of applications. However, predicting keypoints of small objects such as human hands is a challenging problem. Recent works fuse feature maps of deep Convolutional Neural Networks (CNNs), either via multi-level feature integration or multi-resolution aggregation. Despite achieving some success, the feature fusion approaches increase the complexity and the opacity of CNNs. To address this issue, we propose a novel CNN model named Multi-Scale Deep Supervision Network (P-MSDSNet) that learns feature maps at different scales with deep supervisions to produce attention maps for adaptive feature propagation from layers to layers. P-MSDSNet has a multi-stage architecture which makes it scalable while its deep supervision with spatial attention improves transparency to the feature learning at each stage. We show that P-MSDSNet outperforms the state-of-the-art approaches on benchmark datasets while requiring fewer number of parameters. We also show the application of P-MSDSNet to quantify finger tapping hand movements in a neuroscience study.

cs.CV↗

arXiv2021

CogniFNN: A Fuzzy Neural Network Framework for Cognitive Word Embedding Evaluation

Word embeddings can reflect the semantic representations, and the embedding qualities can be comprehensively evaluated with human natural reading-related cognitive data sources. In this paper, we proposed the CogniFNN framework, which is the first attempt at using fuzzy neural networks to extract non-linear and non-stationary characteristics for evaluations of English word embeddings against the corresponding cognitive datasets. In our experiment, we used 15 human cognitive datasets across three modalities: EEG, fMRI, and eye-tracking, and selected the mean square error and multiple hypotheses testing as metrics to evaluate our proposed CogniFNN framework. Compared to the recent pioneer framework, our proposed CogniFNN showed smaller prediction errors of both context-independent (GloVe) and context-sensitive (BERT) word embeddings, and achieved higher significant ratios with randomly generated word embeddings. Our findings suggested that the CogniFNN framework could provide a more accurate and comprehensive evaluation of cognitive word embeddings. It will potentially be beneficial to the further word embeddings evaluation on extrinsic natural language processing tasks.

cs.AI↗

arXiv2021

SLADE: A Self-Training Framework For Distance Metric Learning

Most existing distance metric learning approaches use fully labeled data to learn the sample similarities in an embedding space. We present a self-training framework, SLADE, to improve retrieval performance by leveraging additional unlabeled data. We first train a teacher model on the labeled data and use it to generate pseudo labels for the unlabeled data. We then train a student model on both labels and pseudo labels to generate final feature embeddings. We use self-supervised representation learning to initialize the teacher model. To better deal with noisy pseudo labels generated by the teacher network, we design a new feature basis learning component for the student network, which learns basis functions of feature representations for unlabeled data. The learned basis vectors better measure the pairwise similarity and are used to select high-confident samples for training the student network. We evaluate our method on standard retrieval benchmarks: CUB-200, Cars-196 and In-shop. Experimental results demonstrate that our approach significantly improves the performance over the state-of-the-art methods.

cs.CV↗

arXiv2020

Fashion Outfit Complementary Item Retrieval

Complementary fashion item recommendation is critical for fashion outfit completion. Existing methods mainly focus on outfit compatibility prediction but not in a retrieval setting. We propose a new framework for outfit complementary item retrieval. Specifically, a category-based subspace attention network is presented, which is a scalable approach for learning the subspace attentions. In addition, we introduce an outfit ranking loss that better models the item relationships of an entire outfit. We evaluate our method on the outfit compatibility, FITB and new retrieval tasks. Experimental results demonstrate that our approach outperforms state-of-the-art methods in both compatibility prediction and complementary item retrieval

cs.CV↗

arXiv2019

Correlated Insulating and Superconducting States in Twisted Bilayer Graphene Below the Magic Angle

The emergence of flat bands and correlated behaviors in 'magic angle' twisted bilayer graphene (tBLG) has sparked tremendous interest, though many aspects of the system are under intense debate. Here we report observation of both superconductivity and the Mott-like insulating state in a tBLG device with a twist angle of approximately 0.93, which is smaller than the magic angle by 15%. At an electron concentration of +/-5 electrons per moire unit cell, we observe a narrow resistance peak with an activation energy gap of approximately 0.1 meV, indicating the existence of an additional correlated insulating state. This is consistent with theory predicting the presence of a high-energy band with an energetically flat dispersion. At a doping of +/-12 electrons per moire unit cell we observe a resistance peak due to the presence of Dirac points in the spectrum. Our results reveal that the magic range of tBLG is in fact larger than what is previously expected, and provide a wealth of new information to help decipher the strongly correlated phenomena observed in tBLG.

cond-mat.mes-hall↗

arXiv2018

Integer and Fractional Quantum Hall effect in Ultra-high Quality Few-layer Black Phosphorus Transistors

As a high mobility two-dimensional semiconductor with strong structural and electronic anisotropy, atomically thin black phosphorus (BP) provides a new playground for investigating the quantum Hall (QH) effect, including outstanding questions such as the functional dependence of Landau level (LL) gaps on magnetic field B, and possible anisotropic fractional QH states. Using encapsulating few-layer BP transistors with mobility up to 55,000 cm2/Vs, we extract LL gaps over an exceptionally wide range of B for QH states at filling factors ν=-1 to -4, which are determined to be linear in B, thus resolving a controversy raised by its anisotropy. Furthermore, a fractional QH state at ν~ -4/3 and an additional feature at -0.56+/- 0.1 are observed, underscoring BP as a tunable 2D platform for exploring electron interactions.

cond-mat.mes-hall↗

arXiv2017

Surface Transport and Quantum Hall Effect in Ambipolar Black Phosphorus Double Quantum Wells

Quantum wells constitute one of the most important classes of devices in the study of 2D systems. In a double layer QW, the additional "which-layer" degree of freedom gives rise to celebrated phenomena such as Coulomb drag, Hall drag and exciton condensation. Here we demonstrate facile formation of wide QWs in few-layer black phosphorus devices that host double layers of charge carriers. In contrast to tradition QWs, each 2D layer is ambipolar, and can be tuned into n-doped, p-doped or intrinsic regimes. Fully spin-polarized quantum Hall states are observed on each layer, with enhanced Lande g-factor that is attributed to exchange interactions. Our work opens the door for using 2D semiconductors as ambipolar single, double or wide QWs with unusual properties such as high anisotropy.

cond-mat.mes-hall↗

arXiv2016

Weak Localization and Electron-electron Interactions in Few Layer Black Phosphorus Devices

Few layer phosphorene(FLP) devices are extensively studied due to its unique electronic properties and potential applications on nano-electronics . Here we present magnetotransport studies which reveal electron-electron interactions as the dominant scattering mechanism in hexagonal boron nitride-encapsulated FLP devices. From weak localization measurements, we estimate the electron dephasing length to be 30 to 100 nm at low temperatures, which exhibits a strong dependence on carrier density n and a power-law dependence on temperature (~T-0.4). These results establish that the dominant scattering mechanism in FLP is electron-electron interactions.

cond-mat.mes-hall↗