arXiv subjects

Jun Chen

Publications and source records attributed to Jun Chen.

At least 217 records · Page 12Linked to original sources

arXiv2024

ShadowRefiner: Towards Mask-free Shadow Removal via Fast Fourier Transformer

Shadow-affected images often exhibit pronounced spatial discrepancies in color and illumination, consequently degrading various vision applications including object detection and segmentation systems. To effectively eliminate shadows in real-world images while preserving intricate details and producing visually compelling outcomes, we introduce a mask-free Shadow Removal and Refinement network (ShadowRefiner) via Fast Fourier Transformer. Specifically, the Shadow Removal module in our method aims to establish effective mappings between shadow-affected and shadow-free images via spatial and frequency representation learning. To mitigate the pixel misalignment and further improve the image quality, we propose a novel Fast-Fourier Attention based Transformer (FFAT) architecture, where an innovative attention mechanism is designed for meticulous refinement. Our method wins the championship in the Perceptual Track and achieves the second best performance in the Fidelity Track of NTIRE 2024 Image Shadow Removal Challenge. Besides, comprehensive experiment result also demonstrate the compelling effectiveness of our proposed method. The code is publicly available: https://github.com/movingforward100/Shadow_R.

cs.CV↗

arXiv2024

Meerkat: Audio-Visual Large Language Model for Grounding in Space and Time

Leveraging Large Language Models' remarkable proficiency in text-based tasks, recent works on Multi-modal LLMs (MLLMs) extend them to other modalities like vision and audio. However, the progress in these directions has been mostly focused on tasks that only require a coarse-grained understanding of the audio-visual semantics. We present Meerkat, an audio-visual LLM equipped with a fine-grained understanding of image and audio both spatially and temporally. With a new modality alignment module based on optimal transport and a cross-attention module that enforces audio-visual consistency, Meerkat can tackle challenging tasks such as audio referred image grounding, image guided audio temporal localization, and audio-visual fact-checking. Moreover, we carefully curate a large dataset AVFIT that comprises 3M instruction tuning samples collected from open-source datasets, and introduce MeerkatBench that unifies five challenging audio-visual tasks. We achieve state-of-the-art performance on all these downstream tasks with a relative improvement of up to 37.12%.

cs.CV↗

arXiv2024

Association between a Failed Prominence Eruption and the Drainage of Mass from Another Prominence

Sympathetic eruptions of solar prominences have been studied for decades, however, it is usually difficult to identify their causal links. Here we present two failed prominence eruptions on 26 October 2022 and explore their connections. Using stereoscopic observations, the south prominence (PRO-S) erupts with untwisting motions, flare ribbons occur underneath, and new connections are formed during the eruption. The north prominence (PRO-N) rises up along with PRO-S, and its upper part disappears due to catastrophic mass draining along an elongated structure after PRO-S failed eruption. We suggest that the eruption of PRO-S initiates due to a kink instability, further rises up, and fails to erupt due to reconnection with surrounding fields. The elongated structure connecting PRO-N overlies PRO-S, which causes the rising up of PRO-N along with PRO-S and mass drainage after PRO-S eruption. This study suggests that a prominence may end its life through mass drainage forced by an eruption underneath.

astro-ph.SR↗

arXiv2024

The association of domain-specific physical activity and sedentary activity with stroke: A prospective cohort study

Background The incidence of stroke places a heavy burden on both society and individuals. Activity is closely related to cardiovascular health. This study aimed to investigate the relationship between the varying domains of PA, like occupation-related Physical Activity (OPA), transportation-related Physical Activity (TPA), leisure-time Physical Activity (LTPA), and Sedentary Activity (SA) with stroke. Methods Our analysis included 30,400 participants aged 20+ years from 2007 to 2018 National Health and Nutrition Examination Survey (NHANES). Stroke was identified based on the participant's self-reported diagnoses from previous medical consultations, and PA and SA were self-reported. Multivariable logistic and restricted cubic spline models were used to assess the associations. Results Participants achieving PA guidelines (performing PA more than 150 min/week) were 35.7% less likely to have a stroke based on both the total PA (odds ratio [OR] 0.643, 95% confidence interval [CI] 0.523-0.790) and LTPA (OR 0.643, 95% CI 0.514-0.805), while OPA or TPA did not demonstrate lower stroke risk. Furthermore, participants with less than 7.5 h/day SA levels were 21.6% (OR 0.784, 95% CI 0.665-0.925) less likely to have a stroke. The intensities of total PA and LTPA exhibited nonlinear U-shaped associations with stroke risk. In contrast, those of OPA and TPA showed negative linear associations, while SA intensities were positively linearly correlated with stroke risk. Conclusions LTPA, but not OPA or TPA, was associated with a lower risk of stroke at any amount, suggesting that significant cardiovascular health would benefit from increased PA. Additionally, the positive association between SA and stroke indicated that prolonged sitting was detrimental to cardiovascular health. Overall, increased PA within a reasonable range reduces the risk of stroke, while increased SA elevates it.

physics.med-ph↗

arXiv2024

Information Compression in the AI Era: Recent Advances and Future Challenges

This survey articles focuses on emerging connections between the fields of machine learning and data compression. While fundamental limits of classical (lossy) data compression are established using rate-distortion theory, the connections to machine learning have resulted in new theoretical analysis and application areas. We survey recent works on task-based and goal-oriented compression, the rate-distortion-perception theory and compression for estimation and inference. Deep learning based approaches also provide natural data-driven algorithmic approaches to compression. We survey recent works on applying deep learning techniques to task-based or goal-oriented compression, as well as image and video compression. We also discuss the potential use of large language models for text compression. We finally provide some directions for future research in this promising field.

cs.IT↗

arXiv2024

Electric field controlled valley-polarized photocurrent switch based on the circular bulk photovoltaic effect

Efficient electric manipulation of valley degrees of freedom is critical and challenging for the advancement of valley-based information science and technology. We put forth an electrical scheme, based on a two-band Dirac model, that can switch the fully valley-polarized photocurrent between K and K' valleys using the circular bulk electro-photovoltaic effect. This is accomplished by applying an out-of-plane electric field to the two-dimensional valley materials, which enables continuous tuning of the Berry curvature and its sign flip. We found that the switch of the fully valley-polarized photocurrent is directly tied to the sign change of Berry curvature, which accompanies a topological phase transition, for instance, the quantum spin Hall effect and the quantum valley Hall effect. This scheme has been confirmed in monolayer BiAsI2 and germanene through first-principles calculations. Our paper offers a promising strategy for the development of a volatile valley-addressable memory device and could inspire further research in this area.

cond-mat.mes-hall↗

arXiv2024

Topological photonic alloy

We present the new concept of photonic alloy as a non-periodic topological material. By mixing non-magnetized and magnetized rods in a non-periodic 2D photonic crystal configuration, we realized photonic alloys in the microwave regime. Our experimental findings reveal that the photonic alloy sustains non-reciprocal chiral edge states (CESs) even at very low concentration of magnetized rods. The non-trivial topology and the associated edge states of these non-periodic systems can be characterized by the winding of the reflection phase. Our results indicate that the threshold concentrations for the investigated system within the first non-trivial band gap to exhibit topological behavior approach zero in the thermodynamic limit for substitutional alloys, while the threshold remains non-zero for interstitial alloys. At low concentration, the system exhibits an inhomogeneous structure characterized by isolated patches of non-percolating magnetic domains that are spaced far apart within a topologically trivial photonic crystal. Surprisingly, the system manifests CESs despite a local breakdown of time-reversal symmetry rather than a global one. Photonic alloys represent a new category of disordered topological materials, offering exciting opportunities for exploring topological materials with adjustable gaps.

cond-mat.mes-hall↗

arXiv2024

An Introduction to Vision-Language Modeling

Following the recent popularity of Large Language Models (LLMs), several attempts have been made to extend them to the visual domain. From having a visual assistant that could guide us through unfamiliar environments to generative models that produce images using only a high-level text description, the vision-language model (VLM) applications will significantly impact our relationship with technology. However, there are many challenges that need to be addressed to improve the reliability of those models. While language is discrete, vision evolves in a much higher dimensional space in which concepts cannot always be easily discretized. To better understand the mechanics behind mapping vision to language, we present this introduction to VLMs which we hope will help anyone who would like to enter the field. First, we introduce what VLMs are, how they work, and how to train them. Then, we present and discuss approaches to evaluate VLMs. Although this work primarily focuses on mapping images to language, we also discuss extending VLMs to videos.

cs.LG↗

arXiv2024

DehazeDCT: Towards Effective Non-Homogeneous Dehazing via Deformable Convolutional Transformer

Image dehazing, a pivotal task in low-level vision, aims to restore the visibility and detail from hazy images. Many deep learning methods with powerful representation learning capability demonstrate advanced performance on non-homogeneous dehazing, however, these methods usually struggle with processing high-resolution images (e.g., $4000 \times 6000$) due to their heavy computational demands. To address these challenges, we introduce an innovative non-homogeneous Dehazing method via Deformable Convolutional Transformer-like architecture (DehazeDCT). Specifically, we first design a transformer-like network based on deformable convolution v4, which offers long-range dependency and adaptive spatial aggregation capabilities and demonstrates faster convergence and forward speed. Furthermore, we leverage a lightweight Retinex-inspired transformer to achieve color correction and structure refinement. Extensive experiment results and highly competitive performance of our method in NTIRE 2024 Dense and Non-Homogeneous Dehazing Challenge, ranking second among all 16 submissions, demonstrate the superior capability of our proposed method. The code is available: https://github.com/movingforward100/Dehazing_R.

cs.CV↗

arXiv2024

SPAR: Personalized Content-Based Recommendation via Long Engagement Attention

Leveraging users' long engagement histories is essential for personalized content recommendations. The success of pretrained language models (PLMs) in NLP has led to their use in encoding user histories and candidate items, framing content recommendations as textual semantic matching tasks. However, existing works still struggle with processing very long user historical text and insufficient user-item interaction. In this paper, we introduce a content-based recommendation framework, SPAR, which effectively tackles the challenges of holistic user interest extraction from the long user engagement history. It achieves so by leveraging PLM, poly-attention layers and attention sparsity mechanisms to encode user's history in a session-based manner. The user and item side features are sufficiently fused for engagement prediction while maintaining standalone representations for both sides, which is efficient for practical model deployment. Moreover, we enhance user profiling by exploiting large language model (LLM) to extract global interests from user engagement history. Extensive experiments on two benchmark datasets demonstrate that our framework outperforms existing state-of-the-art (SoTA) methods.

cs.IR↗

arXiv2024

An experimental and theoretical investigation of spray characteristics of impinging jets in impact wave regime

The current study focuses on experimentally and theoretically improving the characterization of the drop size and drop velocity for like-on-like doublet impinging jets. The experimental measurements were made using phase Doppler anemometry (PDA) at jet Weber numbers We j corresponding to the impact wave regime of impinging jet atomization. A more suitable dynamic range was used for PDA measurements compared to the literature, resulting in more accurate experimental measurements for drop diameters and velocities. There is some disagreement in the literature regarding the ability of linear stability analysis to accurately predict drop diameters in the impact wave regime. This work seeks to provide some clarity. It was discovered that the assumed uniform jet velocity profile was a contributing factor for deviation between diameter predictions based on models in the literature and experimental measurements. Analytical expressions that depend on parameters based on the assumed jet velocity profile are presented in this work. Predictions based on the parabolic and 1/7th power law turbulent profiles were considered and show better agreement with the experimental measurements compared to predictions based on the previous models. Experimental mean drop velocity measurements were compared with predictions from a force balance analysis, and it was observed that the assumed jet velocity profile also influences the predicted velocities, with the turbulent profile agreeing best with the experimental mean velocity. It is concluded that the assumed jet velocity profile has a predominant effect on drop diameter and velocity predictions.

physics.flu-dyn↗

arXiv2024

Research on the Tender Leaf Identification and Mechanically Perceptible Plucking Finger for High-quality Green Tea

BACKGROUND: Intelligent identification and precise plucking are the keys to intelligent tea harvesting robots, which are of increasing significance nowadays. Aiming at plucking tender leaves for high-quality green tea producing, in this paper, a tender leaf identification algorithm and a mechanically perceptible plucking finger have been proposed. RESULTS: Based on segmentation algorithm and color features, the tender leaf identification algorithm shows an average identification accuracy of over 92.8%. The mechanically perceptible plucking finger plucks tender leaves in a way that a human hand does so as to remain high quality of tea products. Though finite element analysis, we determine the ideal size of grippers and the location of strain gauge attachment on a gripper to enable the employment of feedback control of desired gripping force. Revealed from our experiments, the success rate of tender leaf plucking reaches 92.5%, demonstrating the effectiveness of our design. CONCLUSION: The results show that the tender leaf identification algorithm and the mechanically perceptible plucking finger are effective for tender leaves identification and plucking, providing a foundation for the development of an intelligent tender leaf plucking robot.

cs.RO↗

arXiv2024

Deep RAW Image Super-Resolution. A NTIRE 2024 Challenge Survey

This paper reviews the NTIRE 2024 RAW Image Super-Resolution Challenge, highlighting the proposed solutions and results. New methods for RAW Super-Resolution could be essential in modern Image Signal Processing (ISP) pipelines, however, this problem is not as explored as in the RGB domain. Th goal of this challenge is to upscale RAW Bayer images by 2x, considering unknown degradations such as noise and blur. In the challenge, a total of 230 participants registered, and 45 submitted results during thee challenge period. The performance of the top-5 submissions is reviewed and provided here as a gauge for the current state-of-the-art in RAW Image Super-Resolution.

cs.CV↗

arXiv2024

Semantic-Aware Multi-Terminal Coding for Gaussian Mixture Sources

A novel distributed source coding model which named semantic-aware multi-terminal (MT) source coding is proposed and investigated in the paper, where multiple agents independently encode an imperceptible semantic source, while both semantic and observations are reconstructed within their respective fidelity criteria. We start from a generalized single-letter characterization of sum rate-distortion region of this problem. Furthermore, we propose a mixed MSE-Log loss framework for this model and specifically depict the rate-distortion bounds when sources are Gaussian mixture distributed. For this case, we first present a relative tight outer bound and explore the activeness of semantic and observation distortion constraints, in which we find that good observation reconstruction will not incur too much semantic errors, but not vice versa. Moreover, we provide a practical coding scheme functioning as an achievable regime of inner bound with the performance analysis and simulation results, which verifies the feasibility of the idea "detect and compress" for Gaussian mixture sources. Our results provide theoretical instructions on the fundamental limits and can be used to guide the practical semantic-aware coding designs for multi-user scenarios.

cs.IT↗

arXiv2024

GoodDrag: Towards Good Practices for Drag Editing with Diffusion Models

In this paper, we introduce GoodDrag, a novel approach to improve the stability and image quality of drag editing. Unlike existing methods that struggle with accumulated perturbations and often result in distortions, GoodDrag introduces an AlDD framework that alternates between drag and denoising operations within the diffusion process, effectively improving the fidelity of the result. We also propose an information-preserving motion supervision operation that maintains the original features of the starting point for precise manipulation and artifact reduction. In addition, we contribute to the benchmarking of drag editing by introducing a new dataset, Drag100, and developing dedicated quality assessment metrics, Dragging Accuracy Index and Gemini Score, utilizing Large Multimodal Models. Extensive experiments demonstrate that the proposed GoodDrag compares favorably against the state-of-the-art approaches both qualitatively and quantitatively. The project page is https://gooddrag.github.io.

cs.CV↗

arXiv2024

Output-Constrained Lossy Source Coding With Application to Rate-Distortion-Perception Theory

The distortion-rate function of output-constrained lossy source coding with limited common randomness is analyzed for the special case of squared error distortion measure. An explicit expression is obtained when both source and reconstruction distributions are Gaussian. This further leads to a partial characterization of the information-theoretic limit of quadratic Gaussian rate-distortion-perception coding with the perception measure given by Kullback-Leibler divergence or squared quadratic Wasserstein distance.

cs.IT↗

arXiv2024

Electrically controlled nonvolatile switching of single-atom magnetism in a Dy@C84 single-molecule transistor

Single-atom magnetism switching is a key technique towards the ultimate data storage density of computer hard disks and has been conceptually realized by leveraging the spin bistability of a magnetic atom under a scanning tunnelling microscope. However, it has rarely been applied to solid-state transistors, an advancement that would be highly desirable for enabling various applications. Here, we demonstrate realization of the electrically controlled Zeeman effect in Dy@C84 single-molecule transistors, thus revealing a transition in the magnetic moment from 3.8 μB to 5.1 μB for the ground-state GN at an electric field strength of 3-10 MV/cm. The consequent magnetoresistance significantly increases from 600% to 1100% at the resonant tunneling point. Density functional theory calculations further corroborate our realization of nonvolatile switching of single-atom magnetism, and the switching stability emanates from an energy barrier of 92 meV for atomic relaxation. These results highlight the potential of using endohedral metallofullerenes for high-temperature, high-stability, high-speed, and compact single-atom magnetic data storage.

cond-mat.mtrl-sci↗

arXiv2024

AutoDFP: Automatic Data-Free Pruning via Channel Similarity Reconstruction

Structured pruning methods are developed to bridge the gap between the massive scale of neural networks and the limited hardware resources. Most current structured pruning methods rely on training datasets to fine-tune the compressed model, resulting in high computational burdens and being inapplicable for scenarios with stringent requirements on privacy and security. As an alternative, some data-free methods have been proposed, however, these methods often require handcraft parameter tuning and can only achieve inflexible reconstruction. In this paper, we propose the Automatic Data-Free Pruning (AutoDFP) method that achieves automatic pruning and reconstruction without fine-tuning. Our approach is based on the assumption that the loss of information can be partially compensated by retaining focused information from similar channels. Specifically, We formulate data-free pruning as an optimization problem, which can be effectively addressed through reinforcement learning. AutoDFP assesses the similarity of channels for each layer and provides this information to the reinforcement learning agent, guiding the pruning and reconstruction process of the network. We evaluate AutoDFP with multiple networks on multiple datasets, achieving impressive compression results. For instance, on the CIFAR-10 dataset, AutoDFP demonstrates a 2.87\% reduction in accuracy loss compared to the recently proposed data-free pruning method DFPC with fewer FLOPs on VGG-16. Furthermore, on the ImageNet dataset, AutoDFP achieves 43.17\% higher accuracy than the SOTA method with the same 80\% preserved ratio on MobileNet-V1.

cs.LG↗