arXiv subjects

Jiajia Liu

Publications and source records attributed to Jiajia Liu.

At least 55 records · Page 3Linked to original sources

arXiv2025

Ming-Omni: A Unified Multimodal Model for Perception and Generation

We propose Ming-Omni, a unified multimodal model capable of processing images, text, audio, and video, while demonstrating strong proficiency in both speech and image generation. Ming-Omni employs dedicated encoders to extract tokens from different modalities, which are then processed by Ling, an MoE architecture equipped with newly proposed modality-specific routers. This design enables a single model to efficiently process and fuse multimodal inputs within a unified framework, thereby facilitating diverse tasks without requiring separate models, task-specific fine-tuning, or structural redesign. Importantly, Ming-Omni extends beyond conventional multimodal models by supporting audio and image generation. This is achieved through the integration of an advanced audio decoder for natural-sounding speech and Ming-Lite-Uni for high-quality image generation, which also allow the model to engage in context-aware chatting, perform text-to-speech conversion, and conduct versatile image editing. Our experimental results showcase Ming-Omni offers a powerful solution for unified perception and generation across all modalities. Notably, our proposed Ming-Omni is the first open-source model we are aware of to match GPT-4o in modality support, and we release all code and model weights to encourage further research and development in the community.

cs.AI↗

arXiv2025

Improving the Γ-functions Method for Vortex Identification

Vortices have been observed at various heights within the solar atmosphere and are suggested to potentially play great roles in heating the solar upper atmosphere. Multiple automated vortex detection methods have been developed and applied to detect vortices. We aim to improve the $Γ$-functions method for vortex identification by optimizing the value of $Γ_{1min}$ and the approach to calculate $Γ_1$ and $Γ_2$ used to determine vortex center and edge. In this way, we can detect vortices more accurately and enable more statistical studies that can improve our knowledge of the generation and evolution of vortices in the solar atmosphere. We apply the automated swirl detection algorithm (ASDA, one representative of $Γ$-functions method) with different parameters to various synthetic data, with each containing 1000 Lamb-Oseen vortices, and search for the optimal $Γ_{1min}$ and kernel size when calculating $Γ_1$ and $Γ_2$. We also compare another detection method using simulation and observational data to validate the results obtained from the synthetic data. The best performance is found with the Optimized ASDA, which combines different kernel sizes (5, 7, 9, and 11) to calculate $Γ_1$ and $Γ_2$ with a fixed $Γ_{1min}$ = 0.63 to detect vortex center. We find that more vortices can be detected by the Optimized ASDA with higher location, radius, and rotation speed accuracies. The above results are further confirmed by comparing vortices detected by the Optimized ASDA and the SWIRL method on CO5BOLD numerical simulation data and SST observational data.

astro-ph.SR↗

arXiv2025

Advancing bioinformatics with large language models: components, applications and perspectives

Large language models (LLMs) are a class of artificial intelligence models based on deep learning, which have great performance in various tasks, especially in natural language processing (NLP). Large language models typically consist of artificial neural networks with numerous parameters, trained on large amounts of unlabeled input using self-supervised or semi-supervised learning. However, their potential for solving bioinformatics problems may even exceed their proficiency in modeling human language. In this review, we will provide a comprehensive overview of the essential components of large language models (LLMs) in bioinformatics, spanning genomics, transcriptomics, proteomics, drug discovery, and single-cell analysis. Key aspects covered include tokenization methods for diverse data types, the architecture of transformer models, the core attention mechanism, and the pre-training processes underlying these models. Additionally, we will introduce currently available foundation models and highlight their downstream applications across various bioinformatics domains. Finally, drawing from our experience, we will offer practical guidance for both LLM users and developers, emphasizing strategies to optimize their use and foster further innovation in the field.

q-bio.QM↗

arXiv2024

Photospheric Swirls in a Quiet-Sun Region

Swirl-shaped flow structures have been observed throughout the solar atmosphere, in both emission and absorption, at different altitudes and locations, and are believed to be associated with magnetic structures. However, the distribution patterns of such swirls, especially their spatial positions, remain unclear. Using the Automated Swirl Detection Algorithm (ASDA), we identified swirls from the high-resolution photospheric observations, centered on Fe I 630.25 nm, of a quiet region near the Sun's central meridian by the Swedish 1-m Solar Telescope. Through a detailed study of the locations of the detected small-scale swirls with an average radius of $\sim$300 km, we found that most of them are located in lanes between mesogranules (which have an average diameter of $\sim$5.4 Mm) instead of the commonly believed intergranular lanes. The squared rotation, expansion/contraction, vector speeds, and proxy kinetic energy are all found to follow Gaussian distributions. Their rotation speed, expansion/contraction speed, and circulation are positively correlated with their radius. These results suggest that photospheric swirls at different scales and locations across the observational 56.5" $\times$ 57.5" field-of-view (FOV) could share the same triggering mechanism at preferred spatial and energy scales. A comparison with previous work suggests that the number of photospheric swirls is positively correlated with the number of local magnetic concentrations, stressing the close relation between swirls and local magnetic concentrations:the number of swirls should positively correlate with the number and strength of local magnetic concentrations.

astro-ph.SR↗

arXiv2024

A Spatiotemporal Stealthy Backdoor Attack against Cooperative Multi-Agent Deep Reinforcement Learning

Recent studies have shown that cooperative multi-agent deep reinforcement learning (c-MADRL) is under the threat of backdoor attacks. Once a backdoor trigger is observed, it will perform abnormal actions leading to failures or malicious goals. However, existing proposed backdoors suffer from several issues, e.g., fixed visual trigger patterns lack stealthiness, the backdoor is trained or activated by an additional network, or all agents are backdoored. To this end, in this paper, we propose a novel backdoor attack against c-MADRL, which attacks the entire multi-agent team by embedding the backdoor only in a single agent. Firstly, we introduce adversary spatiotemporal behavior patterns as the backdoor trigger rather than manual-injected fixed visual patterns or instant status and control the attack duration. This method can guarantee the stealthiness and practicality of injected backdoors. Secondly, we hack the original reward function of the backdoored agent via reward reverse and unilateral guidance during training to ensure its adverse influence on the entire team. We evaluate our backdoor attacks on two classic c-MADRL algorithms VDN and QMIX, in a popular c-MADRL environment SMAC. The experimental results demonstrate that our backdoor attacks are able to reach a high attack success rate (91.6\%) while maintaining a low clean performance variance rate (3.7\%).

cs.AI↗

arXiv2024

Power Optimization and Deep Learning for Channel Estimation of Active IRS-Aided IoT

In this paper, channel estimation of an active intelligent reflecting surface (IRS) aided uplink Internet of Things (IoT) network is investigated. Firstly, the least square (LS) estimators for the direct channel and the cascaded channel are presented, respectively. The corresponding mean square errors (MSE) of channel estimators are derived. Subsequently, in order to evaluate the influence of adjusting the transmit power at the IoT devices or the reflected power at the active IRS on Sum-MSE performance, two situations are considered. In the first case, under the total power sum constraint of the IoT devices and active IRS, the closed-form expression of the optimal power allocation factor is derived. In the second case, when the transmit power at the IoT devices is fixed, there exists an optimal reflective power at active IRS. To further improve the estimation performance, the convolutional neural network (CNN)-based direct channel estimation (CDCE) algorithm and the CNN-based cascaded channel estimation (CCCE) algorithm are designed. Finally, simulation results demonstrate the existence of an optimal power allocation strategy that minimizes the Sum-MSE, and further validate the superiority of the proposed CDCE / CCCE algorithms over their respective traditional LS and minimum mean square error (MMSE) baselines.

eess.SP↗

arXiv2024

TAPFixer: Automatic Detection and Repair of Home Automation Vulnerabilities based on Negated-property Reasoning

Trigger-Action Programming (TAP) is a popular end-user programming framework in the home automation (HA) system, which eases users to customize home automation and control devices as expected. However, its simplified syntax also introduces new safety threats to HA systems through vulnerable rule interactions. Accurately fixing these vulnerabilities by logically and physically eliminating their root causes is essential before rules are deployed. However, it has not been well studied. In this paper, we present TAPFixer, a novel framework to automatically detect and repair rule interaction vulnerabilities in HA systems. It extracts TAP rules from HA profiles, translates them into an automaton model with physical and latency features, and performs model checking with various correctness properties. It then uses a novel negated-property reasoning algorithm to automatically infer a patch via model abstraction and refinement and model checking based on negated-properties. We evaluate TAPFixer on market HA apps (1177 TAP rules and 53 properties) and find that it can achieve an 86.65% success rate in repairing rule interaction vulnerabilities. We additionally recruit 23 HA users to conduct a user study that demonstrates the usefulness of TAPFixer for vulnerability repair in practical HA scenarios.

cs.CR↗

arXiv2024

Improving the Automated Coronal Jet Identification with U-NET

Coronal jets are one of the most common eruptive activities in the solar atmosphere. They are related to rich physics processes, including but not limited to magnetic reconnection, flaring, instabilities, and plasma heating. Automated identification of off-limb coronal jets has been difficult due to their abundant nature, complex appearance, and relatively small size compared to other features in the corona. In this paper, we present an automated coronal jet identification algorithm (AJIA) that utilizes true and fake jets previously detected by a laborious semi-automated jet detection algorithm (SAJIA, Liu et al. 2023) as the input of an image segmentation neural network U-NET. It is found that AJIA could achieve a much higher (0.81) detecting precision than SAJIA (0.34), meanwhile giving the possibility of whether each pixel in an input image belongs to a jet. We demonstrate that with the aid of artificial neural networks, AJIA could enable fast, accurate, and real-time coronal jet identification from SDO/AIA 304 Åobservations, which are essential in studying the collective and long-term behavior of coronal jets and their relation with the solar activity cycles.

astro-ph.SR↗

arXiv2024

Large Multimodal Model Compression via Efficient Pruning and Distillation at AntGroup

The deployment of Large Multimodal Models (LMMs) within AntGroup has significantly advanced multimodal tasks in payment, security, and advertising, notably enhancing advertisement audition tasks in Alipay. However, the deployment of such sizable models introduces challenges, particularly in increased latency and carbon emissions, which are antithetical to the ideals of Green AI. This paper introduces a novel multi-stage compression strategy for our proprietary LLM, AntGMM. Our methodology pivots on three main aspects: employing small training sample sizes, addressing multi-level redundancy through multi-stage pruning, and introducing an advanced distillation loss design. In our research, we constructed a dataset, the Multimodal Advertisement Audition Dataset (MAAD), from real-world scenarios within Alipay, and conducted experiments to validate the reliability of our proposed strategy. Furthermore, the effectiveness of our strategy is evident in its operational success in Alipay's real-world multimodal advertisement audition for three months from September 2023. Notably, our approach achieved a substantial reduction in latency, decreasing it from 700ms to 90ms, while maintaining online performance with only a slight performance decrease. Moreover, our compressed model is estimated to reduce electricity consumption by approximately 75 million kWh annually compared to the direct deployment of AntGMM, demonstrating our commitment to green AI initiatives. We will publicly release our code and the MAAD dataset after some reviews\footnote{https://github.com/MorinW/AntGMM$\_$Pruning}.

cs.AI↗

arXiv2023

Learning Implicit Entity-object Relations by Bidirectional Generative Alignment for Multimodal NER

The challenge posed by multimodal named entity recognition (MNER) is mainly two-fold: (1) bridging the semantic gap between text and image and (2) matching the entity with its associated object in image. Existing methods fail to capture the implicit entity-object relations, due to the lack of corresponding annotation. In this paper, we propose a bidirectional generative alignment method named BGA-MNER to tackle these issues. Our BGA-MNER consists of \texttt{image2text} and \texttt{text2image} generation with respect to entity-salient content in two modalities. It jointly optimizes the bidirectional reconstruction objectives, leading to aligning the implicit entity-object relations under such direct and powerful constraints. Furthermore, image-text pairs usually contain unmatched components which are noisy for generation. A stage-refined context sampler is proposed to extract the matched cross-modal content for generation. Extensive experiments on two benchmarks demonstrate that our method achieves state-of-the-art performance without image input during inference.

cs.LG↗

arXiv2023

On the 5-Minute Oscillations of Photospheric and Chromospheric Swirls

Swirls are ubiquitous in the solar atmosphere. They are believed to be related to the excitation of different modes of magnetohydrodynamic waves and pulses, as well as spicules. However, statistical studies of their collective behaviour are rare. In this paper, we aim to study the collective, as well as the behaviour of individual photospheric and chromospheric swirls detected by the automated swirl detection algorithm (ASDA) from observations obtained by the Swedish 1-m Solar Telescope and the Hinode satellite. Detailed analysis of six different parameters of photospheric and chromospheric swirls is performed employing the wavelet analysis. Two clusters of periods with significant wavelet power, one from $3-8$ minutes and the other from $10-14$ minutes, have been found. The former coincides with the dominant period of the global $p$-mode spectrum. Wavelet and Fast Fourier Transform (FFT) analysis of example swirls also reveals similar periods. These results suggest that global $p$-modes might be important for triggering photospheric and thus chromospheric swirls. A novel scenario of global $p$-modes providing energy and mass fluxes to the upper solar atmosphere via generating swirls, Alfvén pulses and spicules is then proposed.

astro-ph.SR↗

arXiv2023

Power-law Distribution of Solar-Cycle Modulated Coronal Jets

Power-law distributions have been studied as a significant characteristic of non-linear dissipative systems. Since discovering the power-law distribution of solar flares that was later extended to nano-flares and stellar flares, it has been widely accepted that different scales of flares share the same physical process. Here, we present the newly developed Semi-Automated Jet Identification Algorithm (SAJIA) and its application for detecting more than 1200 off-limb solar jets during Solar Cycle 24. Power-law distributions have been revealed between the intensity/energy and frequency of these events, with indices found to be analogous to those for flares and coronal mass ejections (CMEs). These jets are also found to be spatially and temporally modulated by the solar cycle forming a butterfly diagram in their latitudinal-temporal evolution, experiencing quasi-annual oscillations in their analysed properties, and very likely gathering in certain active longitudinal belts. Our results show that coronal jets display the same nonlinear behaviour as that observed in flares and CMEs, in solar and stellar atmospheres, strongly suggesting that they result from the same nonlinear statistics of scale-free processes as their counterparts in different scales of eruptive events. Although these jets, like flares and other large-scale dynamic phenomena, are found to be significantly modulated by the solar cycle, their corresponding power-law indices still remain similar.

astro-ph.SR↗

arXiv2022

Don't Watch Me: A Spatio-Temporal Trojan Attack on Deep-Reinforcement-Learning-Augment Autonomous Driving

Deep reinforcement learning (DRL) is one of the most popular algorithms to realize an autonomous driving (AD) system. The key success factor of DRL is that it embraces the perception capability of deep neural networks which, however, have been proven vulnerable to Trojan attacks. Trojan attacks have been widely explored in supervised learning (SL) tasks (e.g., image classification), but rarely in sequential decision-making tasks solved by DRL. Hence, in this paper, we explore Trojan attacks on DRL for AD tasks. First, we propose a spatio-temporal DRL algorithm based on the recurrent neural network and attention mechanism to prove that capturing spatio-temporal traffic features is the key factor to the effectiveness and safety of a DRL-augment AD system. We then design a spatial-temporal Trojan attack on DRL policies, where the trigger is hidden in a sequence of spatial and temporal traffic features, rather than a single instant state used in existing Trojan on SL and DRL tasks. With our Trojan, the adversary acts as a surrounding normal vehicle and can trigger attacks via specific spatial-temporal driving behaviors, rather than physical or wireless access. Through extensive experiments, we show that while capturing spatio-temporal traffic features can improve the performance of DRL for different AD tasks, they suffer from Trojan attacks since our designed Trojan shows high stealthy (various spatio-temporal trigger patterns), effective (less than 3.1\% performance variance rate and more than 98.5\% attack success rate), and sustainable to existing advanced defenses.

cs.CR↗

arXiv2022

Polymeric jets throw light on the origin and nature of the forest of solar spicules

Spicules are plasma jets, observed in the dynamic interface region between the visible solar surface and the hot corona. At any given time, it is estimated that about 3 million spicules are present on the Sun. We find an intriguing parallel between the simulated spicular forest in a solar-like atmosphere and the numerous jets of polymeric fluids when both are subjected to harmonic forcing. In a radiative magnetohydrodynamic numerical simulation with sub-surface convection, solar global surface oscillations are excited similarly to those harmonic vibrations. The jets thus produced match remarkably well with the forests of spicules detected in observations of the Sun. Taken together, the numerical simulations of the Sun and the laboratory fluid dynamics experiments provide insights into the mechanism underlying the ubiquity of jets: the nonlinear focusing of quasi-periodic waves in anisotropic media of magnetized plasma as well as polymeric fluids under gravity is sufficient to generate a forest of spicules on the Sun.

astro-ph.SR↗

arXiv2022

Solar Ring Mission: Building a Panorama of the Sun and Inner-heliosphere

Solar Ring (SOR) is a proposed space science mission to monitor and study the Sun and inner heliosphere from a full 360° perspective in the ecliptic plane. It will deploy three 120°-separated spacecraft on the 1-AU orbit. The first spacecraft, S1, locates 30° upstream of the Earth, the second, S2, 90° downstream, and the third, S3, completes the configuration. This design with necessary science instruments, e.g., the Doppler-velocity and vector magnetic field imager, wide-angle coronagraph, and in-situ instruments, will allow us to establish many unprecedented capabilities: (1) provide simultaneous Doppler-velocity observations of the whole solar surface to understand the deep interior, (2) provide vector magnetograms of the whole photosphere - the inner boundary of the solar atmosphere and heliosphere, (3) provide the information of the whole lifetime evolution of solar featured structures, and (4) provide the whole view of solar transients and space weather in the inner heliosphere. With these capabilities, Solar Ring mission aims to address outstanding questions about the origin of solar cycle, the origin of solar eruptions and the origin of extreme space weather events. The successful accomplishment of the mission will construct a panorama of the Sun and inner-heliosphere, and therefore advance our understanding of the star and the space environment that holds our life.

astro-ph.SR↗

arXiv2022

TAPInspector: Safety and Liveness Verification of Concurrent Trigger-Action IoT Systems

Trigger-action programming (TAP) is a popular end-user programming framework that can simplify the Internet of Things (IoT) automation with simple trigger-action rules. However, it also introduces new security and safety threats. A lot of advanced techniques have been proposed to address this problem. Rigorously reasoning about the security of a TAP-based IoT system requires a well-defined model and verification method both against rule semantics and physical-world features, e.g., concurrency, rule latency, extended action, tardy attributes, and connection-based rule interactions, which has been missing until now. By analyzing these features, we find 9 new types of rule interaction vulnerabilities and validate them on two commercial IoT platforms. We then present TAPInspector, a novel system to detect these interaction vulnerabilities in concurrent TAP-based IoT systems. It automatically extracts TAP rules from IoT apps, translates them into a hybrid model by model slicing and state compression, and performs semantic analysis and model checking with various safety and liveness properties. Our experiments corroborate that TAPInspector is practical: it identifies 533 violations related to rule interaction from 1108 real-world market IoT apps and is at least 60000 times faster than the baseline without optimization.

cs.CR↗

arXiv2022

A Temporal-Pattern Backdoor Attack to Deep Reinforcement Learning

Deep reinforcement learning (DRL) has made significant achievements in many real-world applications. But these real-world applications typically can only provide partial observations for making decisions due to occlusions and noisy sensors. However, partial state observability can be used to hide malicious behaviors for backdoors. In this paper, we explore the sequential nature of DRL and propose a novel temporal-pattern backdoor attack to DRL, whose trigger is a set of temporal constraints on a sequence of observations rather than a single observation, and effect can be kept in a controllable duration rather than in the instant. We validate our proposed backdoor attack to a typical job scheduling task in cloud computing. Numerous experimental results show that our backdoor can achieve excellent effectiveness, stealthiness, and sustainability. Our backdoor's average clean data accuracy and attack success rate can reach 97.8% and 97.5%, respectively.

cs.LG↗

arXiv2021

Developing Dipole-scheme Heterojunction Photocatalysts

The high recombination rate of photogenerated carriers is the bottleneck of photocatalysis, severely limiting the photocatalytic efficiency. Here, we develop a dipole-scheme (D-scheme for short) photocatalytic model and materials realization. The D-scheme heterojunction not only can effectively separate electrons and holes by a large polarization field, but also boosts photocatalytic redox reactions with large driving photovoltages and without any carrier loss. By means of first-principles and GW calculations, we propose a D-scheme heterojunction prototype with two real polar materials, PtSeTe/LiGaS2. This D-scheme photocatalyst exhibits a high capability of the photogenerated carrier separation and near-infrared light absorption. Moreover, our calculations of the Gibbs free energy imply a high ability of the hydrogen and oxygen evolution reaction by a large driving force. The proposed D-scheme photocatalytic model is generalized and paves a valuable route of significantly improving the photocatalytic efficiency.

cond-mat.mtrl-sci↗