arXiv subjects

Feng Zhang

Publications and source records attributed to Feng Zhang.

At least 145 records · Page 8Linked to original sources

arXiv2024

Energy Consumption Optimization, Response Time Differences and Indicators in Cortical Working Memory Revealed by Nonequilibrium

The neocortex, a complex system driving multi-region interactions, remains a core puzzle in neuroscience. Despite quantitative insights across brain scales, understanding the mechanisms underlying neural activities is challenging. Advances from Hopfield networks to large-scale cortical models have deepened neural network theory, yet these models often fall short of capturing global brain functions. In large-scale cortical networks, an intriguing hierarchy of timescales reflects diverse information processing speeds across spatial regions. As a non-equilibrium system, the brain incurs significant energy costs, with long-distance connectivity suggesting an evolutionary spatial organization. To explore these complexities, we introduce a nonequilibrium landscape flux approach to analyze cortical networks. This allows us to quantify potential landscapes and principal transition paths, uncovering dynamical characteristics across timescales. We examine whether temporal hierarchies correlate with stimuli distribution and how hierarchical networks exhibit differential responses. Furthermore, our analysis quantifies the thermodynamic cost of sustaining cognition, highlighting a link to network connectivity. These findings provide insights into energy consumption during cognitive processes and emphasize the spatial benefits for working memory tasks. Experimental validation is challenging due to evolutionary variability, making our theoretical approach valuable for quantifying complex dynamics. By assessing time irreversibility and critical slowdown, we gain predictive insights into network bifurcations and state transitions, offering practical tools for identifying cortical state changes. These results advance our understanding of cortical dynamics.

q-bio.NC↗

arXiv2024

DaYu: Data-Driven Model for Geostationary Satellite Observed Cloud Images Forecasting

In the past few years, Artificial Intelligence (AI)-based weather forecasting methods have widely demonstrated strong competitiveness among the weather forecasting systems. However, these methods are insufficient for high-spatial-resolution short-term nowcasting within 6 hours, which is crucial for warning short-duration, mesoscale and small-scale weather events. Geostationary satellite remote sensing provides detailed, high spatio-temporal and all-day observations, which can address the above limitations of existing methods. Therefore, this paper proposed an advanced data-driven thermal infrared cloud images forecasting model, "DaYu." Unlike existing data-driven weather forecasting models, DaYu is specifically designed for geostationary satellite observations, with a temporal resolution of 0.5 hours and a spatial resolution of ${0.05}^\circ$ $\times$ ${0.05}^\circ$. DaYu is based on a large-scale transformer architecture, which enables it to capture fine-grained cloud structures and learn fast-changing spatio-temporal evolution features effectively. Moreover, its attention mechanism design achieves a balance in computational complexity, making it practical for applications. DaYu not only achieves accurate forecasts up to 3 hours with a correlation coefficient higher than 0.9, 6 hours higher than 0.8, and 12 hours higher than 0.7, but also detects short-duration, mesoscale, and small-scale weather events with enhanced detail, effectively addressing the shortcomings of existing methods in providing detailed short-term nowcasting within 6 hours. Furthermore, DaYu has significant potential in short-term climate disaster prevention and mitigation.

physics.ao-ph↗

arXiv2024

Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent

In this paper, we introduce Hunyuan-Large, which is currently the largest open-source Transformer-based mixture of experts model, with a total of 389 billion parameters and 52 billion activation parameters, capable of handling up to 256K tokens. We conduct a thorough evaluation of Hunyuan-Large's superior performance across various benchmarks including language understanding and generation, logical reasoning, mathematical problem-solving, coding, long-context, and aggregated tasks, where it outperforms LLama3.1-70B and exhibits comparable performance when compared to the significantly larger LLama3.1-405B model. Key practice of Hunyuan-Large include large-scale synthetic data that is orders larger than in previous literature, a mixed expert routing strategy, a key-value cache compression technique, and an expert-specific learning rate strategy. Additionally, we also investigate the scaling laws and learning rate schedule of mixture of experts models, providing valuable insights and guidances for future model development and optimization. The code and checkpoints of Hunyuan-Large are released to facilitate future innovations and applications. Codes: https://github.com/Tencent/Hunyuan-Large Models: https://huggingface.co/tencent/Tencent-Hunyuan-Large

cs.CL↗

arXiv2024

First Proof of Principle Experiment for Muon Production with Ultrashort High Intensity Laser

Muons, which play a crucial role in both fundamental and applied physics, have traditionally been generated through proton accelerators or from cosmic rays. With the advent of ultra-short high-intensity lasers capable of accelerating electrons to GeV levels, it has become possible to generate muons in laser laboratories. In this work, we show the first proof of principle experiment for novel muon production with an ultra-short, high-intensity laser device through GeV electron beam bombardment on a lead converter target. The muon physical signal is confirmed by measuring its lifetime which is the first clear demonstration of laser-produced muons. Geant4 simulations were employed to investigate the photo-production, electro-production, and Bethe-Heitler processes response for muon generation and their subsequent detection. The results show that the dominant contributions of muons are attributed to the photo-production/electro-production and a significant yield of muons up to 0.01 $μ$/$e^-$ out of the converter target could be achieved. This laser muon source features compact, ultra-short pulse and high flux. Moreover, its implementation in a small laser laboratory is relatively straightforward, significantly reducing the barriers to entry for research in areas such as muonic X-ray elemental analysis, muon spin spectroscopy and so on.

physics.acc-ph↗

arXiv2024

Development of the cadmium zinc TElluride Radiation Imager (TERI)

The cadmium zinc TElluride Radiation Imager, or TERI, is an instrument to space qualify large-volume $4 \times 4 \times 1.5 \ \mathrm{cm}^3$ pixelated CdZnTe (CZT) detector technology. The CZT's anode is composed of a $22 \times 22$ array of pixels while the cathode is planar. TERI will contain four of those crystals with each pixel having an energy range of $40 \ \mathrm{keV}$ up to $3 \ \mathrm{MeV}$ with a resolution of $1.3 \%$ full-width-at-half maximum at $662 \ \mathrm{keV}$ all while operating in room temperature. As the detectors are 3D position sensitive, TERI can Compton image events. TERI is fitted with a coded-aperture mask which permits imaging low energy photons in the photoelectric regime. TERI's primary mission is to space-qualify large-volume CZT and measure its degradation due to radiation damage in a space environment. Its secondary mission includes detecting and localizing astrophysical gamma-ray transients. TERI is manifested on DoD's STP-H10 mission for launch to the International Space Station in early 2025.

astro-ph.IM↗

arXiv2024

Study of longitudinal development of air showers in the knee energy range

Ground-based cosmic ray experiments detect cosmic ray mainly by measuring the longitudinal and lateral distribution of secondary particles produced in the extensive air shower (EAS). The EAS of cosmic ray in the knee energy region is simulated via CORSIKA software. Several simulation samples with different energy, composition and zenith angles were carried out to understand the longitudinal development of electron, muon and Cherenkov light in EAS. All the results presented were obtained assuming an observation plane at an altitude of 4400 m a.s.l. The differences of longitudinal development between electron and Cherenkov light were studied, and the reconstruction uncertainty of shower maximum for electron from Cherenkov light was estimated to be 10-15g/cm$^2$ for nuclei above 1 PeV. The performances of energy measurement and the composition discrimination ability based on longitudinal development were studied and compared with that from lateral distribution. It was found that number of electron per depth at its shower maximum has the smallest shower-to-shower fluctuations, but the shower-to-shower fluctuations of electron density measured at observation level was very close to it when the appropriate zenith angle was employed. The shower-to-shower fluctuations of shower maximum for electron is 50-55 g/cm$^2$ for proton, and 20-25 g/cm$^2$ for iron, but the composition discrimination ability between nuclei from muon density measured at observation level is much better than the shower maximum variable from longitudinal development. The hadronic model dependencies of the longitudinal development and lateral distribution were also discussed.

astro-ph.HE↗

arXiv2024

From Sim-to-Real: Toward General Event-based Low-light Frame Interpolation with Per-scene Optimization

Video Frame Interpolation (VFI) is important for video enhancement, frame rate up-conversion, and slow-motion generation. The introduction of event cameras, which capture per-pixel brightness changes asynchronously, has significantly enhanced VFI capabilities, particularly for high-speed, nonlinear motions. However, these event-based methods encounter challenges in low-light conditions, notably trailing artifacts and signal latency, which hinder their direct applicability and generalization. Addressing these issues, we propose a novel per-scene optimization strategy tailored for low-light conditions. This approach utilizes the internal statistics of a sequence to handle degraded event data under low-light conditions, improving the generalizability to different lighting and camera settings. To evaluate its robustness in low-light condition, we further introduce EVFI-LL, a unique RGB+Event dataset captured under low-light conditions. Our results demonstrate state-of-the-art performance in low-light environments. Project page: https://naturezhanghn.github.io/sim2real.

cs.CV↗

arXiv2024

StructEval: Deepen and Broaden Large Language Model Assessment via Structured Evaluation

Evaluation is the baton for the development of large language models. Current evaluations typically employ a single-item assessment paradigm for each atomic test objective, which struggles to discern whether a model genuinely possesses the required capabilities or merely memorizes/guesses the answers to specific questions. To this end, we propose a novel evaluation framework referred to as StructEval. Starting from an atomic test objective, StructEval deepens and broadens the evaluation by conducting a structured assessment across multiple cognitive levels and critical concepts, and therefore offers a comprehensive, robust and consistent evaluation for LLMs. Experiments on three widely-used benchmarks demonstrate that StructEval serves as a reliable tool for resisting the risk of data contamination and reducing the interference of potential biases, thereby providing more reliable and consistent conclusions regarding model capabilities. Our framework also sheds light on the design of future principled and trustworthy LLM evaluation protocols.

cs.CL↗

arXiv2024

Event-Based Motion Magnification

Detecting and magnifying imperceptible high-frequency motions in real-world scenarios has substantial implications for industrial and medical applications. These motions are characterized by small amplitudes and high frequencies. Traditional motion magnification methods rely on costly high-speed cameras or active light sources, which limit the scope of their applications. In this work, we propose a dual-camera system consisting of an event camera and a conventional RGB camera for video motion magnification, providing temporally-dense information from the event stream and spatially-dense data from the RGB images. This innovative combination enables a broad and cost-effective amplification of high-frequency motions. By revisiting the physical camera model, we observe that estimating motion direction and magnitude necessitates the integration of event streams with additional image features. On this basis, we propose a novel deep network tailored for event-based motion magnification. Our approach utilizes the Second-order Recurrent Propagation module to proficiently interpolate multiple frames while addressing artifacts and distortions induced by magnified motions. Additionally, we employ a temporal filter to distinguish between noise and useful signals, thus minimizing the impact of noise. We also introduced the first event-based motion magnification dataset, which includes a synthetic subset and a real-captured subset for training and benchmarking. Through extensive experiments in magnifying small-amplitude, high-frequency motions, we demonstrate the effectiveness and accuracy of our dual-camera system and network, offering a cost-effective and flexible solution for motion detection and magnification.

cs.CV↗

arXiv2024

UFineBench: Towards Text-based Person Retrieval with Ultra-fine Granularity

Existing text-based person retrieval datasets often have relatively coarse-grained text annotations. This hinders the model to comprehend the fine-grained semantics of query texts in real scenarios. To address this problem, we contribute a new benchmark named \textbf{UFineBench} for text-based person retrieval with ultra-fine granularity. Firstly, we construct a new \textbf{dataset} named UFine6926. We collect a large number of person images and manually annotate each image with two detailed textual descriptions, averaging 80.8 words each. The average word count is three to four times that of the previous datasets. In addition of standard in-domain evaluation, we also propose a special \textbf{evaluation paradigm} more representative of real scenarios. It contains a new evaluation set with cross domains, cross textual granularity and cross textual styles, named UFine3C, and a new evaluation metric for accurately measuring retrieval ability, named mean Similarity Distribution (mSD). Moreover, we propose CFAM, a more efficient \textbf{algorithm} especially designed for text-based person retrieval with ultra fine-grained texts. It achieves fine granularity mining by adopting a shared cross-modal granularity decoder and hard negative match mechanism. With standard in-domain evaluation, CFAM establishes competitive performance across various datasets, especially on our ultra fine-grained UFine6926. Furthermore, by evaluating on UFine3C, we demonstrate that training on our UFine6926 significantly improves generalization to real scenarios compared with other coarse-grained datasets. The dataset and code will be made publicly available at \url{https://github.com/Zplusdragon/UFineBench}.

cs.CV↗

arXiv2024

PLIP: Language-Image Pre-training for Person Representation Learning

Language-image pre-training is an effective technique for learning powerful representations in general domains. However, when directly turning to person representation learning, these general pre-training methods suffer from unsatisfactory performance. The reason is that they neglect critical person-related characteristics, i.e., fine-grained attributes and identities. To address this issue, we propose a novel language-image pre-training framework for person representation learning, termed PLIP. Specifically, we elaborately design three pretext tasks: 1) Text-guided Image Colorization, aims to establish the correspondence between the person-related image regions and the fine-grained color-part textual phrases. 2) Image-guided Attributes Prediction, aims to mine fine-grained attribute information of the person body in the image; and 3) Identity-based Vision-Language Contrast, aims to correlate the cross-modal representations at the identity level rather than the instance level. Moreover, to implement our pre-train framework, we construct a large-scale person dataset with image-text pairs named SYNTH-PEDES by automatically generating textual annotations. We pre-train PLIP on SYNTH-PEDES and evaluate our models by spanning downstream person-centric tasks. PLIP not only significantly improves existing methods on all these tasks, but also shows great ability in the zero-shot and domain generalization settings. The code, dataset and weights will be released at~\url{https://github.com/Zplusdragon/PLIP}

cs.CV↗

arXiv2024

Square-Root Inverse Filter-based GNSS-Visual-Inertial Navigation

While Global Navigation Satellite System (GNSS) is often used to provide global positioning if available, its intermittency and/or inaccuracy calls for fusion with other sensors. In this paper, we develop a novel GNSS-Visual-Inertial Navigation System (GVINS) that fuses visual, inertial, and raw GNSS measurements within the square-root inverse sliding window filtering (SRI-SWF) framework in a tightly coupled fashion, which thus is termed SRI-GVINS. In particular, for the first time, we deeply fuse the GNSS pseudorange, Doppler shift, single-differenced pseudorange, and double-differenced carrier phase measurements, along with the visual-inertial measurements. Inherited from the SRI-SWF, the proposed SRI-GVINS gains significant numerical stability and computational efficiency over the start-of-the-art methods. Additionally, we propose to use a filter to sequentially initialize the reference frame transformation till converges, rather than collecting measurements for batch optimization. We also perform online calibration of GNSS-IMU extrinsic parameters to mitigate the possible extrinsic parameter degradation. The proposed SRI-GVINS is extensively evaluated on our own collected UAV datasets and the results demonstrate that the proposed method is able to suppress VIO drift in real-time and also show the effectiveness of online GNSS-IMU extrinsic calibration. The experimental validation on the public datasets further reveals that the proposed SRI-GVINS outperforms the state-of-the-art methods in terms of both accuracy and efficiency.

cs.RO↗

arXiv2024

Liberating Seen Classes: Boosting Few-Shot and Zero-Shot Text Classification via Anchor Generation and Classification Reframing

Few-shot and zero-shot text classification aim to recognize samples from novel classes with limited labeled samples or no labeled samples at all. While prevailing methods have shown promising performance via transferring knowledge from seen classes to unseen classes, they are still limited by (1) Inherent dissimilarities among classes make the transformation of features learned from seen classes to unseen classes both difficult and inefficient. (2) Rare labeled novel samples usually cannot provide enough supervision signals to enable the model to adjust from the source distribution to the target distribution, especially for complicated scenarios. To alleviate the above issues, we propose a simple and effective strategy for few-shot and zero-shot text classification. We aim to liberate the model from the confines of seen classes, thereby enabling it to predict unseen categories without the necessity of training on seen classes. Specifically, for mining more related unseen category knowledge, we utilize a large pre-trained language model to generate pseudo novel samples, and select the most representative ones as category anchors. After that, we convert the multi-class classification task into a binary classification task and use the similarities of query-anchor pairs for prediction to fully leverage the limited supervision signals. Extensive experiments on six widely used public datasets show that our proposed method can outperform other strong baselines significantly in few-shot and zero-shot tasks, even without using any seen class samples.

cs.CV↗

arXiv2024

Interest Clock: Time Perception in Real-Time Streaming Recommendation System

User preferences follow a dynamic pattern over a day, e.g., at 8 am, a user might prefer to read news, while at 8 pm, they might prefer to watch movies. Time modeling aims to enable recommendation systems to perceive time changes to capture users' dynamic preferences over time, which is an important and challenging problem in recommendation systems. Especially, streaming recommendation systems in the industry, with only available samples of the current moment, present greater challenges for time modeling. There is still a lack of effective time modeling methods for streaming recommendation systems. In this paper, we propose an effective and universal method Interest Clock to perceive time information in recommendation systems. Interest Clock first encodes users' time-aware preferences into a clock (hour-level personalized features) and then uses Gaussian distribution to smooth and aggregate them into the final interest clock embedding according to the current time for the final prediction. By arming base models with Interest Clock, we conduct online A/B tests, obtaining +0.509% and +0.758% improvements on user active days and app duration respectively. Besides, the extended offline experiments show improvements as well. Interest Clock has been deployed on Douyin Music App.

cs.IR↗

arXiv2024

A note on generalized spherical maximal means

The goal of this note is to provide an alternative proof of Theorem 1.1 (i) in [4], that is, if $n\geq 2$ and $M^α$ is bounded on $L^{p}(\mathbb{R}^{n})$ for some $α\in \mathbb{C}$ and $p\geq 2$, then we have \begin{align*} \operatorname{Re} α\geq \max\left\{\frac{1-n}{2}+\frac{1}{p},\frac{1-n}{p}\right\}. \end{align*}

math.CA↗

arXiv2024

Polarized Adding Method of Discrete Ordinate Approximation for Ultraviolet-Visible and Near-Infrared Radiative Transfer

The polarization characteristics of atmospheric scattering are important and should not be ignored in radiative transfer simulations. In this study, a new vector radiative transfer model called the polarized adding method of discrete ordinate approximation (POLDDA) is proposed for use in remote sensing applications for ultraviolet-visible and near-infrared spectra. The single-layer radiative transfer process and inhomogeneous multi-layer connection are solved using the discrete ordinate method (DOM) and adding methods, respectively. By combining the advantages of DOM and the adding method, the Stokes vector (including the I-, Q-, U-, and V-components) calculated using the new method conforms to the results of PolRadtran/RT3, whether in a Rayleigh scattering atmosphere or the water cloud case. Moreover, the relative root-mean-square error (RMSE) values of the Stokes vector for the test cases between MYSTIC and the new method or RT3 prove the accuracy of the proposed method. Meanwhile, the new method has a higher computational efficiency than RT3, particularly for an atmosphere with a large scattering optical depth. Unlike RT3, the computation time of the proposed method does not increase with the optical depth of each layer.

physics.ao-ph↗

arXiv2024

Variational inequalities for generalized spherical means

In this paper, we establish the $L^{p}(\mathbb{R}^{d})$-boundedness of the variation operator and the $δ$-jump operator for generalized spherical means, and we also show the necessary conditions for the $L^{p}(\mathbb{R}^{d})$-boundedness of these operators. These results are almost optimal when $d=2$.

math.CA↗

arXiv2024

Diffusion Deepfake

Recent progress in generative AI, primarily through diffusion models, presents significant challenges for real-world deepfake detection. The increased realism in image details, diverse content, and widespread accessibility to the general public complicates the identification of these sophisticated deepfakes. Acknowledging the urgency to address the vulnerability of current deepfake detectors to this evolving threat, our paper introduces two extensive deepfake datasets generated by state-of-the-art diffusion models as other datasets are less diverse and low in quality. Our extensive experiments also showed that our dataset is more challenging compared to the other face deepfake datasets. Our strategic dataset creation not only challenge the deepfake detectors but also sets a new benchmark for more evaluation. Our comprehensive evaluation reveals the struggle of existing detection methods, often optimized for specific image domains and manipulations, to effectively adapt to the intricate nature of diffusion deepfakes, limiting their practical utility. To address this critical issue, we investigate the impact of enhancing training data diversity on representative detection methods. This involves expanding the diversity of both manipulation techniques and image domains. Our findings underscore that increasing training data diversity results in improved generalizability. Moreover, we propose a novel momentum difficulty boosting strategy to tackle the additional challenge posed by training data heterogeneity. This strategy dynamically assigns appropriate sample weights based on learning difficulty, enhancing the model's adaptability to both easy and challenging samples. Extensive experiments on both existing and newly proposed benchmarks demonstrate that our model optimization approach surpasses prior alternatives significantly.

cs.CV↗