arXiv subjects

Junjie Zhang

Publications and source records attributed to Junjie Zhang.

At least 73 records · Page 4Linked to original sources

arXiv2024

Paleoinspired Vision: From Exploring Colour Vision Evolution to Inspiring Camera Design

The evolution of colour vision is captivating, as it reveals the adaptive strategies of extinct species while simultaneously inspiring innovations in modern imaging technology. In this study, we present a simplified model of visual transduction in the retina, introducing a novel opsin layer. We quantify evolutionary pressures by measuring machine vision recognition accuracy on colour images shaped by specific opsins. Building on this, we develop an evolutionary conservation optimisation algorithm to reconstruct the spectral sensitivity of opsins, enabling mutation-driven adaptations to to more effectively spot fruits or predators. This model condenses millions of years of evolution within seconds on GPU, providing an experimental framework to test long-standing hypotheses in evolutionary biology , such as vision of early mammals, primate trichromacy from gene duplication, retention of colour blindness, blue-shift of fish rod and multiple rod opsins with bioluminescence. Moreover, the model enables speculative explorations of hypothetical species, such as organisms with eyes adapted to the conditions on Mars. Our findings suggest a minimalist yet effective approach to task-specific camera filter design, optimising the spectral response function to meet application-driven demands. The code will be made publicly available upon acceptance.

cs.CV↗

arXiv2024

Enhancing Graph Contrastive Learning with Reliable and Informative Augmentation for Recommendation

Graph neural network(GNN) has been a powerful approach in collaborative filtering(CF) due to its ability to model high-order user-item relationships. Recently, to alleviate the data sparsity and enhance representation learning, many efforts have been conducted to integrate contrastive learning(CL) with GNNs. Despite the promising improvements, the contrastive view generation based on structure and representation perturbations in existing methods potentially disrupts the collaborative information in contrastive views, resulting in limited effectiveness of positive alignment. To overcome this issue, we propose CoGCL, a novel framework that aims to enhance graph contrastive learning by constructing contrastive views with stronger collaborative information via discrete codes. The core idea is to map users and items into discrete codes rich in collaborative information for reliable and informative contrastive view generation. To this end, we initially introduce a multi-level vector quantizer in an end-to-end manner to quantize user and item representations into discrete codes. Based on these discrete codes, we enhance the collaborative information of contrastive views by considering neighborhood structure and semantic relevance respectively. For neighborhood structure, we propose virtual neighbor augmentation by treating discrete codes as virtual neighbors, which expands an observed user-item interaction into multiple edges involving discrete codes. Regarding semantic relevance, we identify similar users/items based on shared discrete codes and interaction targets to generate the semantically relevant view. Through these strategies, we construct contrastive views with stronger collaborative information and develop a triple-view graph contrastive learning approach. Extensive experiments on four public datasets demonstrate the effectiveness of our proposed approach.

cs.IR↗

arXiv2024

Bulk Crystal Growth and Single-Crystal-to-Single-Crystal Phase Transitions in the Averievite CsClCu5V2O10

Quasi-two-dimensional averievites with triangle-kagome-triangle trilayers are of interest due to their rich structural and magnetic transitions and strong spin frustration that are expected to host quantum spin liquid ground state with suitable substitution or doping. Herein, we report growth of bulk single crystals of averievite CsClCu5V2O10 with dimensions of several millimeters on edge in order to (1) address the open question whether the room temperature crystal structure is P-3m1, P-3, P21/c or else, (2) to elucidate the nature of phase transitions, and (3) to study direction-dependent physical properties. Single-crystal-to-single-crystal structural transitions at ~305 K and ~127 K were observed in the averievite CsClCu5V2O10 single crystals. The nature of the transition at ~305 K, which was reported as P-3m1-P21/c transition, was found to be a structural transition from high temperature P-3m1 to low temperature P-3 by combining variable temperature synchrotron X-ray single crystal and high-resolution powder diffraction. In-plane and out-of-plane magnetic susceptibility and heat capacity measurements confirm a first-order transition at 305 K, a structural transition at 127 K and an antiferromagnetic transition at 24 K. These averievites are thus ideal model systems for a deeper understanding of structural transitions and magnetism.

cond-mat.str-el↗

arXiv2024

Curriculum-scheduled Knowledge Distillation from Multiple Pre-trained Teachers for Multi-domain Sequential Recommendation

Pre-trained recommendation models (PRMs) have received increasing interest recently. However, their intrinsically heterogeneous model structure, huge model size and computation cost hinder their adoptions in practical recommender systems. Hence, it is highly essential to explore how to use different pre-trained recommendation models efficiently in real-world systems. In this paper, we propose a novel curriculum-scheduled knowledge distillation from multiple pre-trained teachers for multi-domain sequential recommendation, called CKD-MDSR, which takes full advantages of different PRMs as multiple teacher models to boost a small student recommendation model, integrating the knowledge across multiple domains from PRMs. Specifically, CKD-MDSR first adopts curriculum-scheduled user behavior sequence sampling and distills informative knowledge jointly from the representative PRMs such as UniSRec and Recformer. Then, the knowledge from the above PRMs are selectively integrated into the student model in consideration of their confidence and consistency. Finally, we verify the proposed method on multi-domain sequential recommendation and further demonstrate its universality with multiple types of student models, including feature interaction and graph based recommendation models. Extensive experiments on five real-world datasets demonstrate the effectiveness and efficiency of CKD-MDSR, which can be viewed as an efficient shortcut using PRMs in real-world systems.

cs.IR↗

arXiv2024

Cascade of phase transitions and large magnetic anisotropy in a triangle-kagome-triangle trilayer antiferromagnet

Spins in strongly frustrated systems are of intense interest due to the emergence of intriguing quantum states including superconductivity and quantum spin liquid. Herein we report the discovery of cascade of phase transitions and large magnetic anisotropy in the averievite CsClCu5P2O10 single crystals. Under zero field, CsClCu5P2O10 undergoes a first-order structural transition at around 225 K from high temperature centrosymmetric P-3m1 to low temperature noncentrosymmetric P321, followed by an AFM transition at 13.6 K, another structural transition centering at ~3 K, and another AFM transition at ~2.18 K. Based upon magnetic susceptibility and magnetization data with magnetic fields perpendicular to the ab plane, a phase diagram, consisting of a paramagnetic state, two AFM states and four field-induced states including two magnetization plateaus, has been constructed. Our findings demonstrate that the quasi-2D CsClCu5P2O10 exhibits rich structural and metamagnetic transitions and the averievite family is a fertile platform for exploring novel quantum states.

cond-mat.str-el↗

arXiv2024

Privacy-Preserving Student Learning with Differentially Private Data-Free Distillation

Deep learning models can achieve high inference accuracy by extracting rich knowledge from massive well-annotated data, but may pose the risk of data privacy leakage in practical deployment. In this paper, we present an effective teacher-student learning approach to train privacy-preserving deep learning models via differentially private data-free distillation. The main idea is generating synthetic data to learn a student that can mimic the ability of a teacher well-trained on private data. In the approach, a generator is first pretrained in a data-free manner by incorporating the teacher as a fixed discriminator. With the generator, massive synthetic data can be generated for model training without exposing data privacy. Then, the synthetic data is fed into the teacher to generate private labels. Towards this end, we propose a label differential privacy algorithm termed selective randomized response to protect the label information. Finally, a student is trained on the synthetic data with the supervision of private labels. In this way, both data privacy and label privacy are well protected in a unified framework, leading to privacy-preserving models. Extensive experiments and analysis clearly demonstrate the effectiveness of our approach.

cs.LG↗

arXiv2024

Pseudo-Prompt Generating in Pre-trained Vision-Language Models for Multi-Label Medical Image Classification

The task of medical image recognition is notably complicated by the presence of varied and multiple pathological indications, presenting a unique challenge in multi-label classification with unseen labels. This complexity underlines the need for computer-aided diagnosis methods employing multi-label zero-shot learning. Recent advancements in pre-trained vision-language models (VLMs) have showcased notable zero-shot classification abilities on medical images. However, these methods have limitations on leveraging extensive pre-trained knowledge from broader image datasets, and often depend on manual prompt construction by expert radiologists. By automating the process of prompt tuning, prompt learning techniques have emerged as an efficient way to adapt VLMs to downstream tasks. Yet, existing CoOp-based strategies fall short in performing class-specific prompts on unseen categories, limiting generalizability in fine-grained scenarios. To overcome these constraints, we introduce a novel prompt generation approach inspirited by text generation in natural language processing (NLP). Our method, named Pseudo-Prompt Generating (PsPG), capitalizes on the priori knowledge of multi-modal features. Featuring a RNN-based decoder, PsPG autoregressively generates class-tailored embedding vectors, i.e., pseudo-prompts. Comparative evaluations on various multi-label chest radiograph datasets affirm the superiority of our approach against leading medical vision-language and multi-label prompt learning methods. The source code is available at https://github.com/fallingnight/PsPG

cs.CV↗

arXiv2024

Large, Complex, and Realistic Safety Clothing and Helmet Detection: Dataset and Method

Detecting safety clothing and helmets is paramount for ensuring the safety of construction workers. However, the development of deep learning models in this domain has been impeded by the scarcity of high-quality datasets. In this study, we construct a large, complex, and realistic safety clothing and helmet detection (SFCHD) dataset. SFCHD is derived from two authentic chemical plants, comprising 12,373 images, 7 categories, and 50,552 annotations. We partition the SFCHD dataset into training and testing sets with a ratio of 4:1 and validate its utility by applying several classic object detection algorithms. Furthermore, drawing inspiration from spatial and channel attention mechanisms, we design a spatial and channel attention-based low-light enhancement (SCALE) module. SCALE is a plug-and-play component with a high degree of flexibility. Extensive evaluations of the SCALE module on both the ExDark and SFCHD datasets have empirically demonstrated its efficacy in enhancing the performance of detectors under low-light conditions. The dataset and code are publicly available at https://github.com/lijfrank-open/SFCHD-SCALE.

cs.CV↗

arXiv2024

Fine-Grained Urban Flow Inference with Multi-scale Representation Learning

Fine-grained urban flow inference (FUFI) is a crucial transportation service aimed at improving traffic efficiency and safety. FUFI can infer fine-grained urban traffic flows based solely on observed coarse-grained data. However, most of existing methods focus on the influence of single-scale static geographic information on FUFI, neglecting the interactions and dynamic information between different-scale regions within the city. Different-scale geographical features can capture redundant information from the same spatial areas. In order to effectively learn multi-scale information across time and space, we propose an effective fine-grained urban flow inference model called UrbanMSR, which uses self-supervised contrastive learning to obtain dynamic multi-scale representations of neighborhood-level and city-level geographic information, and fuses multi-scale representations to improve fine-grained accuracy. The fusion of multi-scale representations enhances fine-grained. We validate the performance through extensive experiments on three real-world datasets. The resutls compared with state-of-the-art methods demonstrate the superiority of the proposed model.

cs.CV↗

arXiv2024

Wide-bandgap semiconductor of three-dimensional unconventional stoichiometric NaCl2 crystal

The expanding applications call for novel new-generation wide-bandgap semiconductors. Here, we show that a compound only composed of the ordinary elements Na and Cl, namely three-dimensional NaCl2 crystal, is a wide-bandgap semiconductor. This finding benefits from the breaking of conventional stoichiometry frameworks in the theoretical design, leading to the discovery of three-dimensional XY2 (X = Na, Li, K; Y = Cl, F, Br, I) crystals, with covalent bonds of Y pairs inducing the wide bandgap from 2.24 to 4.45 eV. Crucially, such an unexpected NaCl2 crystal was successfully synthesized under ambient conditions. The unconventional stoichiometric strategy with other chemical elements potentially yields more wide-bandgap semiconductors, offering the capability for bandgap tuning. These unconventional stoichiometric materials may also exhibit superconductivity, transparent inorganic electrides, high-energy-density, and beyond.

cond-mat.mtrl-sci↗

arXiv2024

SAM-E: Leveraging Visual Foundation Model with Sequence Imitation for Embodied Manipulation

Acquiring a multi-task imitation policy in 3D manipulation poses challenges in terms of scene understanding and action prediction. Current methods employ both 3D representation and multi-view 2D representation to predict the poses of the robot's end-effector. However, they still require a considerable amount of high-quality robot trajectories, and suffer from limited generalization in unseen tasks and inefficient execution in long-horizon reasoning. In this paper, we propose SAM-E, a novel architecture for robot manipulation by leveraging a vision-foundation model for generalizable scene understanding and sequence imitation for long-term action reasoning. Specifically, we adopt Segment Anything (SAM) pre-trained on a huge number of images and promptable masks as the foundation model for extracting task-relevant features, and employ parameter-efficient fine-tuning on robot data for a better understanding of embodied scenarios. To address long-horizon reasoning, we develop a novel multi-channel heatmap that enables the prediction of the action sequence in a single pass, notably enhancing execution efficiency. Experimental results from various instruction-following tasks demonstrate that SAM-E achieves superior performance with higher execution efficiency compared to the baselines, and also significantly improves generalization in few-shot adaptation to new tasks.

cs.CV↗

arXiv2024

3DBench: A Scalable 3D Benchmark and Instruction-Tuning Dataset

Evaluating the performance of Multi-modal Large Language Models (MLLMs), integrating both point cloud and language, presents significant challenges. The lack of a comprehensive assessment hampers determining whether these models truly represent advancements, thereby impeding further progress in the field. Current evaluations heavily rely on classification and caption tasks, falling short in providing a thorough assessment of MLLMs. A pressing need exists for a more sophisticated evaluation method capable of thoroughly analyzing the spatial understanding and expressive capabilities of these models. To address these issues, we introduce a scalable 3D benchmark, accompanied by a large-scale instruction-tuning dataset known as 3DBench, providing an extensible platform for a comprehensive evaluation of MLLMs. Specifically, we establish the benchmark that spans a wide range of spatial and semantic scales, from object-level to scene-level, addressing both perception and planning tasks. Furthermore, we present a rigorous pipeline for automatically constructing scalable 3D instruction-tuning datasets, covering 10 diverse multi-modal tasks with more than 0.23 million QA pairs generated in total. Thorough experiments evaluating trending MLLMs, comparisons against existing datasets, and variations of training protocols demonstrate the superiority of 3DBench, offering valuable insights into current limitations and potential research directions.

cs.CV↗

arXiv2024

Multimodal Physical Fitness Monitoring (PFM) Framework Based on TimeMAE-PFM in Wearable Scenarios

Physical function monitoring (PFM) plays a crucial role in healthcare especially for the elderly. Traditional assessment methods such as the Short Physical Performance Battery (SPPB) have failed to capture the full dynamic characteristics of physical function. Wearable sensors such as smart wristbands offer a promising solution to this issue. However, challenges exist, such as the computational complexity of machine learning methods and inadequate information capture. This paper proposes a multi-modal PFM framework based on an improved TimeMAE, which compresses time-series data into a low-dimensional latent space and integrates a self-enhanced attention module. This framework achieves effective monitoring of physical health, providing a solution for real-time and personalized assessment. The method is validated using the NHATS dataset, and the results demonstrate an accuracy of 70.6% and an AUC of 82.20%, surpassing other state-of-the-art time-series classification models.

eess.SP↗

arXiv2024

Voxel or Pillar: Exploring Efficient Point Cloud Representation for 3D Object Detection

Efficient representation of point clouds is fundamental for LiDAR-based 3D object detection. While recent grid-based detectors often encode point clouds into either voxels or pillars, the distinctions between these approaches remain underexplored. In this paper, we quantify the differences between the current encoding paradigms and highlight the limited vertical learning within. To tackle these limitations, we introduce a hybrid Voxel-Pillar Fusion network (VPF), which synergistically combines the unique strengths of both voxels and pillars. Specifically, we first develop a sparse voxel-pillar encoder that encodes point clouds into voxel and pillar features through 3D and 2D sparse convolutions respectively, and then introduce the Sparse Fusion Layer (SFL), facilitating bidirectional interaction between sparse voxel and pillar features. Our efficient, fully sparse method can be seamlessly integrated into both dense and sparse detectors. Leveraging this powerful yet straightforward framework, VPF delivers competitive performance, achieving real-time inference speeds on the nuScenes and Waymo Open Dataset. The code will be available.

cs.CV↗

arXiv2024

Low-temperature aqueous solution growth of the acousto-optic TeO2 single crystals

$α$-TeO2 is widely used in acousto-optic devices due to its excellent physical properties. Conventionally, $α$-TeO2 single crystals were grown using melt methods. Here, we report for the first time the growth of $α$-TeO2 single crystals using the aqueous solution method below 100 °C. Solubility curve of $α$-TeO2 was measured, and then single crystals with dimensions of 3.5x3.5x2.5 mm3 were successfully grown using seed crystals that were synthesized from spontaneous nucleation. The as-grown single crystals belong to the P41212 space group, evidenced by single crystal X-ray diffraction and Rietveld refinement on powder diffraction. Rocking curve measurements show that the as-grown crystals exhibit high crystallinity with a full-width at half maxima (FWHM) of 57.2''. Ultraviolet-Visible absorption spectroscopy indicates the absorption edge is 350 nm and the band gap is estimated to be 3.58 eV. The density and Vickers hardness of as-grown single crystals are measured to be 6.042 g/cm3 and 404 kg/mm2, repectively. Our findings provide an easy-to-access and energy-saving method for growing single crystals of inorganic compounds.

cond-mat.mtrl-sci↗

arXiv2024

Sequence-level Semantic Representation Fusion for Recommender Systems

With the rapid development of recommender systems, there is increasing side information that can be employed to improve the recommendation performance. Specially, we focus on the utilization of the associated \emph{textual data} of items (eg product title) and study how text features can be effectively fused with ID features in sequential recommendation. However, there exists distinct data characteristics for the two kinds of item features, making a direct fusion method (eg adding text and ID embeddings as item representation) become less effective. To address this issue, we propose a novel {\ul \emph{Te}}xt-I{\ul \emph{D}} semantic fusion approach for sequential {\ul \emph{Rec}}ommendation, namely \textbf{\our}. The core idea of our approach is to conduct a sequence-level semantic fusion approach by better integrating global contexts. The key strategy lies in that we transform the text embeddings and ID embeddings by Fourier Transform from \emph{time domain} to \emph{frequency domain}. In the frequency domain, the global sequential characteristics of the original sequences are inherently aggregated into the transformed representations, so that we can employ simple multiplicative operations to effectively fuse the two kinds of item features. Our fusion approach can be proved to have the same effects of contextual convolution, so as to achieving sequence-level semantic fusion. In order to further improve the fusion performance, we propose to enhance the discriminability of the text embeddings from the text encoder, by adaptively injecting positional information via a mixture-of-experts~(MoE) modulation method. Our implementation is available at this repository: \textcolor{magenta}{\url{https://github.com/RUCAIBox/TedRec}}.

cs.IR↗

arXiv2024

Large Language Models are Zero-Shot Rankers for Recommender Systems

Recently, large language models (LLMs) (e.g., GPT-4) have demonstrated impressive general-purpose task-solving abilities, including the potential to approach recommendation tasks. Along this line of research, this work aims to investigate the capacity of LLMs that act as the ranking model for recommender systems. We first formalize the recommendation problem as a conditional ranking task, considering sequential interaction histories as conditions and the items retrieved by other candidate generation models as candidates. To solve the ranking task by LLMs, we carefully design the prompting template and conduct extensive experiments on two widely-used datasets. We show that LLMs have promising zero-shot ranking abilities but (1) struggle to perceive the order of historical interactions, and (2) can be biased by popularity or item positions in the prompts. We demonstrate that these issues can be alleviated using specially designed prompting and bootstrapping strategies. Equipped with these insights, zero-shot LLMs can even challenge conventional recommendation models when ranking candidates are retrieved by multiple candidate generators. The code and processed datasets are available at https://github.com/RUCAIBox/LLMRank.

cs.IR↗

arXiv2024

Active Flow Control for Bluff Body Drag Reduction Using Reinforcement Learning with Partial Measurements

Active flow control for drag reduction with reinforcement learning (RL) is performed in the wake of a 2D square bluff body at laminar regimes with vortex shedding. Controllers parameterised by neural networks are trained to drive two blowing and suction jets that manipulate the unsteady flow. RL with full observability (sensors in the wake) successfully discovers a control policy which reduces the drag by suppressing the vortex shedding in the wake. However, a non-negligible performance degradation (~50% less drag reduction) is observed when the controller is trained with partial measurements (sensors on the body). To mitigate this effect, we propose an energy-efficient, dynamic, maximum entropy RL control scheme. First, an energy-efficiency-based reward function is proposed to optimise the energy consumption of the controller while maximising drag reduction. Second, the controller is trained with an augmented state consisting of both current and past measurements and actions, which can be formulated as a nonlinear autoregressive exogenous model, to alleviate the partial observability problem. Third, maximum entropy RL algorithms (Soft Actor Critic and Truncated Quantile Critics) which promote exploration and exploitation in a sample efficient way are used and discover near-optimal policies in the challenging case of partial measurements. Stabilisation of the vortex shedding is achieved in the near wake using only surface pressure measurements on the rear of the body, resulting in similar drag reduction as in the case with wake sensors. The proposed approach opens new avenues for dynamic flow control using partial measurements for realistic configurations.

physics.flu-dyn↗