arXiv subjects

Feng Li

Publications and source records attributed to Feng Li.

At least 163 records · Page 9Linked to original sources

arXiv2024

T-Rex2: Towards Generic Object Detection via Text-Visual Prompt Synergy

We present T-Rex2, a highly practical model for open-set object detection. Previous open-set object detection methods relying on text prompts effectively encapsulate the abstract concept of common objects, but struggle with rare or complex object representation due to data scarcity and descriptive limitations. Conversely, visual prompts excel in depicting novel objects through concrete visual examples, but fall short in conveying the abstract concept of objects as effectively as text prompts. Recognizing the complementary strengths and weaknesses of both text and visual prompts, we introduce T-Rex2 that synergizes both prompts within a single model through contrastive learning. T-Rex2 accepts inputs in diverse formats, including text prompts, visual prompts, and the combination of both, so that it can handle different scenarios by switching between the two prompt modalities. Comprehensive experiments demonstrate that T-Rex2 exhibits remarkable zero-shot object detection capabilities across a wide spectrum of scenarios. We show that text prompts and visual prompts can benefit from each other within the synergy, which is essential to cover massive and complicated real-world scenarios and pave the way towards generic object detection. Model API is now available at \url{https://github.com/IDEA-Research/T-Rex}.

cs.CV↗

arXiv2024

TAPTR: Tracking Any Point with Transformers as Detection

In this paper, we propose a simple and strong framework for Tracking Any Point with TRansformers (TAPTR). Based on the observation that point tracking bears a great resemblance to object detection and tracking, we borrow designs from DETR-like algorithms to address the task of TAP. In the proposed framework, in each video frame, each tracking point is represented as a point query, which consists of a positional part and a content part. As in DETR, each query (its position and content feature) is naturally updated layer by layer. Its visibility is predicted by its updated content feature. Queries belonging to the same tracking point can exchange information through self-attention along the temporal dimension. As all such operations are well-designed in DETR-like algorithms, the model is conceptually very simple. We also adopt some useful designs such as cost volume from optical flow models and develop simple designs to provide long temporal information while mitigating the feature drifting issue. Our framework demonstrates strong performance with state-of-the-art performance on various TAP datasets with faster inference speed.

cs.CV↗

arXiv2024

BlindDiff: Empowering Degradation Modelling in Diffusion Models for Blind Image Super-Resolution

Diffusion models (DM) have achieved remarkable promise in image super-resolution (SR). However, most of them are tailored to solving non-blind inverse problems with fixed known degradation settings, limiting their adaptability to real-world applications that involve complex unknown degradations. In this work, we propose BlindDiff, a DM-based blind SR method to tackle the blind degradation settings in SISR. BlindDiff seamlessly integrates the MAP-based optimization into DMs, which constructs a joint distribution of the low-resolution (LR) observation, high-resolution (HR) data, and degradation kernels for the data and kernel priors, and solves the blind SR problem by unfolding MAP approach along with the reverse process. Unlike most DMs, BlindDiff firstly presents a modulated conditional transformer (MCFormer) that is pre-trained with noise and kernel constraints, further serving as a posterior sampler to provide both priors simultaneously. Then, we plug a simple yet effective kernel-aware gradient term between adjacent sampling iterations that guides the diffusion model to learn degradation consistency knowledge. This also enables to joint refine the degradation model as well as HR images by observing the previous denoised sample. With the MAP-based reverse diffusion process, we show that BlindDiff advocates alternate optimization for blur kernel estimation and HR image restoration in a mutual reinforcing manner. Experiments on both synthetic and real-world datasets show that BlindDiff achieves the state-of-the-art performance with significant model complexity reduction compared to recent DM-based methods. Code will be available at \url{https://github.com/lifengcs/BlindDiff}

cs.CV↗

arXiv2024

Geometric and electronic properties of two kinds of CrO2 magnetic monolayers: D3d and D2h phases

Due to the high magnetic coupling strength between the Cr elements, the bulk phase CrO2 is one of several ferromagnetic oxides known to have the highest Curie temperature. When the dimensionality of the material is reduced from 3D to 2D, the 2D CrO2 system material is expected to maintain a high Curie temperature. In this work, we predict two new phases of CrO2 monolayer (D3d and D2h) by using first-principles calculations. We have found that the Curie temperature of 2D CrO2 is much lower than that of its bulk phase, but still remains as high as 191K, which is comparable to that of Fe2Cr2Ge6. In addition, 1L D3d-CrO2 is in the ferromagnetic state, while 1L D2h-CrO2 is in the antiferromagnetic state. Also, the different geometric structure affects its electrical properties: the 1L D3d-CrO2 is a half-metal while 1L D2h-CrO2 is a semiconductor. Our studies have shown that there is a wealth of electrical and magnetic properties in CrO2.

cond-mat.mtrl-sci↗

arXiv2024

Dual orthogonally-polarized lasing assisted by imaginary Fermi arcs in organic microcavities

The polarization control of micro/nano lasers is an important topic in nanophotonics. Up to now, the simultaneous generation of two distinguishable orthogonally-polarized lasing modes from a single organic microlaser remains a critical challenge. Here, we demonstrate simultaneously orthogonally-polarized dual lasing from a microcavity filled with an organic single crystal exhibiting selective strong coupling. We show that the non-Hermiticity due to polarization-dependent losses leads to the formation of real and imaginary Fermi arcs with exceptional points. Simultaneous orthogonally-polarized lasing becomes possible thanks to the eigenstate mixing by the photonic spin-orbit coupling at the imaginary Fermi arcs. Our work provides a novel way to develop linearly-polarized lasers and paves the way for the future fundamental research in topological photonics, non-Hermitian optics, and other fields.

cond-mat.mes-hall↗

arXiv2024

Bi-center conditions and bifurcation of limit cycles in a class of $Z_2$-equivariant cubic switching systems with two nilpotent points

In this paper, we generalize the Poincaré-Lyapunov method for systems with linear type centers to study nilpotent centers in switching polynomial systems and use it to investigate the bi-center problem of planar $Z_2$-equivariant cubic switching systems associated with two symmetric nilpotent singular points. With a properly designed perturbation, 6 explicit bi-center conditions for such polynomial systems are derived. Then, based on the $6$ center conditions, by using Bogdanov-Takens bifurcation theory with general perturbations, we prove that there exist at least $20$ small-amplitude limit cycles around the nilpotent bi-center for a class of $Z_2$-equivariant cubic switching systems. This is a new lower bound of cyclicity for such cubic polynomial systems, increased from $12$ to $20$.

math.DS↗

arXiv2024

Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks

We introduce Grounded SAM, which uses Grounding DINO as an open-set object detector to combine with the segment anything model (SAM). This integration enables the detection and segmentation of any regions based on arbitrary text inputs and opens a door to connecting various vision models. As shown in Fig.1, a wide range of vision tasks can be achieved by using the versatile Grounded SAM pipeline. For example, an automatic annotation pipeline based solely on input images can be realized by incorporating models such as BLIP and Recognize Anything. Additionally, incorporating Stable-Diffusion allows for controllable image editing, while the integration of OSX facilitates promptable 3D human motion analysis. Grounded SAM also shows superior performance on open-vocabulary benchmarks, achieving 48.7 mean AP on SegInW (Segmentation in the wild) zero-shot benchmark with the combination of Grounding DINO-Base and SAM-Huge models.

cs.CV↗

arXiv2024

Partonic Critical Opalescence and Its Impact on the Jet Quenching Parameter $\hat{q}$

Jet quenching parameter $\hat{q}$ is essential for characterizing the interaction strength between jet partons and nuclear matter. Based on the quark-meson (QM) model, we develop a new framework for calculating $\hat{q}$ at finite chemical potentials, in which $\hat{q}$ is related to the spectral function of the chiral order parameter. A perturbative calculation up to the one-loop order indicates that the momentum broadening of jets is enhanced at both the high temperature and high chemical potential, and approximately proportional to the parton number density in the partonic phase. We further investigate the behavior of $\hat{q}$ in the vicinity of the critical endpoint (CEP) by coupling our calculation with a recently developed equation of state that includes a CEP in the universality class of the Ising model, from which we discover the partonic critical opalescence (PCO) -- a prominent enhancement of the momentum broadening of jets near CEP, contributed by the scatterings via the $σ$ exchange process. Hence, for the first time, jet quenching is connected with the search of CEP.

nucl-th↗

arXiv2024

Tailoring chaotic motion of microcavity photons in ray and wave dynamics by tuning the curvature of space

Microcavity photon dynamics in curved space is an emerging interesting area at the crossing point of nanophotonics, chaotic science and non-Euclidean geometry. We report the sharp difference between the regular and chaotic motions of cavity photons subjected to the varying space curvature. While the island modes of regular motion rise in the phase diagram in the curved space, the chaotic modes show special mechanisms to adapt to the space curvature, including the fast diffusion of ray dynamics, and the localization and hybridization of the Husimi wavepackets among different periodic orbits. These obser-vations are unique effects enabled by the combination of the chaotic trajectory, the wave nature of light and the non-Euclidean orbital motion, and therefore make the system a versatile optical simulator for chaotic science under quan-tum mechanics in curved space-time.

physics.optics↗

arXiv2024

Incentivizing Massive Unknown Workers for Budget-Limited Crowdsensing: From Off-Line and On-Line Perspectives

How to incentivize strategic workers using limited budget is a very fundamental problem for crowdsensing systems; nevertheless, since the sensing abilities of the workers may not always be known as prior knowledge due to the diversities of their sensor devices and behaviors, it is difficult to properly select and pay the unknown workers. Although the uncertainties of the workers can be addressed by the standard Combinatorial Multi-Armed Bandit (CMAB) framework in existing proposals through a trade-off between exploration and exploitation, we may not have sufficient budget to enable the trade-off among the individual workers, especially when the number of the workers is huge while the budget is limited. Moreover, the standard CMAB usually assumes the workers always stay in the system, whereas the workers may join in or depart from the system over time, such that what we have learnt for an individual worker cannot be applied after the worker leaves. To address the above challenging issues, in this paper, we first propose an off-line Context-Aware CMAB-based Incentive (CACI) mechanism. We innovate in leveraging the exploration-exploitation trade-off in an elaborately partitioned context space instead of the individual workers, to effectively incentivize the massive unknown workers with a very limited budget. We also extend the above basic idea to the on-line setting where unknown workers may join in or depart from the systems dynamically, and propose an on-line version of the CACI mechanism. We perform rigorous theoretical analysis to reveal the upper bounds on the regrets of our CACI mechanisms and to prove their truthfulness and individual rationality, respectively. Extensive experiments on both synthetic and real datasets are also conducted to verify the efficacy of our mechanisms.

cs.AI↗

arXiv2023

Spin control with triplet and doublet excitons in organic semiconductors

Spin triplet exciton formation sets limits on technologies using organic semiconductors that are confined to singlet-triplet photophysics. In contrast, excitations in the spin doublet manifold in organic radical semiconductors can show efficient luminescence. Here we explore the dynamics of the spin allowed process of intermolecular energy transfer from triplet to doublet excitons. We employ a carbene-metal-amide (CMA-CF3) as a model triplet donor host, since following photoexcitation it undergoes extremely fast intersystem crossing to set up a population of triplet excitons within 4 ps. This enables a foundational study for tracking energy transfer from triplets to a model radical semiconductor, TTM-3PCz. Over 90% of all radical luminescence originates from the triplet channel in this system under photoexcitation. We find that intermolecular triplet-to-doublet energy transfer can occur directly and rapidly, with 12% of triplet excitons transferring already on sub-ns timescales. This enhanced triplet harvesting mechanism is utilised in efficient near-infrared organic light-emitting diodes, which can be extended to other opto-electronic and -spintronic technologies by radical-based spin control in molecular semiconductors.

physics.chem-ph↗

arXiv2023

APG: Adaptive Parameter Generation Network for Click-Through Rate Prediction

In many web applications, deep learning-based CTR prediction models (deep CTR models for short) are widely adopted. Traditional deep CTR models learn patterns in a static manner, i.e., the network parameters are the same across all the instances. However, such a manner can hardly characterize each of the instances which may have different underlying distributions. It actually limits the representation power of deep CTR models, leading to sub-optimal results. In this paper, we propose an efficient, effective, and universal module, named as Adaptive Parameter Generation network (APG), which can dynamically generate parameters for deep CTR models on-the-fly based on different instances. Extensive experimental evaluation results show that APG can be applied to a variety of deep CTR models and significantly improve their performance. Meanwhile, APG can reduce the time cost by 38.7\% and memory usage by 96.6\% compared to a regular deep CTR model. We have deployed APG in the industrial sponsored search system and achieved 3\% CTR gain and 1\% RPM gain respectively.

cs.IR↗

arXiv2023

LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models

With the recent significant advancements in large multi-modal models (LMMs), the importance of their grounding capability in visual chat is increasingly recognized. Despite recent efforts to enable LMMs to support grounding, their capabilities for grounding and chat are usually separate, and their chat performance drops dramatically when asked to ground. The problem is the lack of a dataset for grounded visual chat (GVC). Existing grounding datasets only contain short captions. To address this issue, we have created GVC data that allows for the combination of grounding and chat capabilities. To better evaluate the GVC capabilities, we have introduced a benchmark called Grounding-Bench. Additionally, we have proposed a model design that can support GVC and various types of visual prompts by connecting segmentation models with language models. Experimental results demonstrate that our model outperforms other LMMs on Grounding-Bench. Furthermore, our model achieves competitive performance on classic grounding benchmarks like RefCOCO/+/g and Flickr30K Entities. Our code will be released at https://github.com/UX-Decoder/LLaVA-Grounding .

cs.CV↗

arXiv2023

Direct Manipulation of quantum entanglement from the non-Hermitian nature of light-matter interaction

Biphoton process is an essential benchmark for quantum information science and technologies, while great efforts have been made to improve the coherence of the system for better quantum correlations. Nevertheless, we find that the non-Hermitian features induced by the atomic quantum interference could be well employed for the direct control of entanglement. We report the demonstration of exceptional point (EP) in biphotons by measuring the light-atom interaction as a natural non-Hermitian system, in which the electromagnetically induced transparency regime provides a powerful mechanism to precisely tune the non-Hermitian coupling strength. Such biphoton correlation is tuned within an unprecedented large range from Rabi oscillation to antibunching-exponential-decay, also indicating high-dimensional entanglement within the strong and weak light-matter coupling regimes. The EP at the transition point between the two regimes is clearly observed with the biphoton quantum correlation measurements, exhibiting a single exponential decay and manifesting the coalesced single eigenstate. Our results provide a unique method to realize the controllability of natural non-Hermitian processes without the assistance of artificial photonic structures, and paves the way for quantum control by manipulating the non-Hermitian features of the light-matter interaction.

quant-ph↗

arXiv2023

Convolutional Neural Networks for Segmentation of Malignant Pleural Mesothelioma: Analysis of Probability Map Thresholds (CALGB 30901, Alliance)

Malignant pleural mesothelioma (MPM) is the most common form of mesothelioma. To assess response to treatment, tumor measurements are acquired and evaluated based on a patient's longitudinal computed tomography (CT) scans. Tumor volume, however, is the more accurate metric for assessing tumor burden and response. Automated segmentation methods using deep learning can be employed to acquire volume, which otherwise is a tedious task performed manually. The deep learning-based tumor volume and contours can then be compared with a standard reference to assess the robustness of the automated segmentations. The purpose of this study was to evaluate the impact of probability map threshold on MPM tumor delineations generated using a convolutional neural network (CNN). Eighty-eight CT scans from 21 MPM patients were segmented by a VGG16/U-Net CNN. A radiologist modified the contours generated at a 0.5 probability threshold. Percent difference of tumor volume and overlap using the Dice Similarity Coefficient (DSC) were compared between the standard reference provided by the radiologist and CNN outputs for thresholds ranging from 0.001 to 0.9. CNN annotations consistently yielded smaller tumor volumes than radiologist contours. Reducing the probability threshold from 0.5 to 0.1 decreased the absolute percent volume difference, on average, from 43.96% to 24.18%. Median and mean DSC ranged from 0.58 to 0.60, with a peak at a threshold of 0.5; no distinct threshold was found for percent volume difference. No single output threshold in the CNN probability maps was optimal for both tumor volume and DSC. This work underscores the need to assess tumor volume and spatial overlap when evaluating CNN performance. While automated segmentations may yield comparable tumor volumes to that of the reference standard, the spatial region delineated by the CNN at a specific threshold is equally important.

eess.IV↗

arXiv2023

Infinite forecast combinations based on Dirichlet process

Forecast combination integrates information from various sources by consolidating multiple forecast results from the target time series. Instead of the need to select a single optimal forecasting model, this paper introduces a deep learning ensemble forecasting model based on the Dirichlet process. Initially, the learning rate is sampled with three basis distributions as hyperparameters to convert the infinite mixture into a finite one. All checkpoints are collected to establish a deep learning sub-model pool, and weight adjustment and diversity strategies are developed during the combination process. The main advantage of this method is its ability to generate the required base learners through a single training process, utilizing the decaying strategy to tackle the challenge posed by the stochastic nature of gradient descent in determining the optimal learning rate. To ensure the method's generalizability and competitiveness, this paper conducts an empirical analysis using the weekly dataset from the M4 competition and explores sensitivity to the number of models to be combined. The results demonstrate that the ensemble model proposed offers substantial improvements in prediction accuracy and stability compared to a single benchmark model.

cs.LG↗

arXiv2023

T-Rex: Counting by Visual Prompting

We introduce T-Rex, an interactive object counting model designed to first detect and then count any objects. We formulate object counting as an open-set object detection task with the integration of visual prompts. Users can specify the objects of interest by marking points or boxes on a reference image, and T-Rex then detects all objects with a similar pattern. Guided by the visual feedback from T-Rex, users can also interactively refine the counting results by prompting on missing or falsely-detected objects. T-Rex has achieved state-of-the-art performance on several class-agnostic counting benchmarks. To further exploit its potential, we established a new counting benchmark encompassing diverse scenarios and challenges. Both quantitative and qualitative results show that T-Rex possesses exceptional zero-shot counting capabilities. We also present various practical application scenarios for T-Rex, illustrating its potential in the realm of visual prompting.

cs.CV↗

arXiv2023

Visual In-Context Prompting

In-context prompting in large language models (LLMs) has become a prevalent approach to improve zero-shot capabilities, but this idea is less explored in the vision domain. Existing visual prompting methods focus on referring segmentation to segment the most relevant object, falling short of addressing many generic vision tasks like open-set segmentation and detection. In this paper, we introduce a universal visual in-context prompting framework for both tasks. In particular, we build on top of an encoder-decoder architecture, and develop a versatile prompt encoder to support a variety of prompts like strokes, boxes, and points. We further enhance it to take an arbitrary number of reference image segments as the context. Our extensive explorations show that the proposed visual in-context prompting elicits extraordinary referring and generic segmentation capabilities to refer and detect, yielding competitive performance to close-set in-domain datasets and showing promising results on many open-set segmentation datasets. By joint training on COCO and SA-1B, our model achieves $57.7$ PQ on COCO and $23.2$ PQ on ADE20K. Code will be available at https://github.com/UX-Decoder/DINOv.

cs.CV↗