arXiv subjects

Ke Chen

Publications and source records attributed to Ke Chen.

At least 199 records · Page 11Linked to original sources

arXiv2024

CARAT: Contrastive Feature Reconstruction and Aggregation for Multi-Modal Multi-Label Emotion Recognition

Multi-modal multi-label emotion recognition (MMER) aims to identify relevant emotions from multiple modalities. The challenge of MMER is how to effectively capture discriminative features for multiple labels from heterogeneous data. Recent studies are mainly devoted to exploring various fusion strategies to integrate multi-modal information into a unified representation for all labels. However, such a learning scheme not only overlooks the specificity of each modality but also fails to capture individual discriminative features for different labels. Moreover, dependencies of labels and modalities cannot be effectively modeled. To address these issues, this paper presents ContrAstive feature Reconstruction and AggregaTion (CARAT) for the MMER task. Specifically, we devise a reconstruction-based fusion mechanism to better model fine-grained modality-to-label dependencies by contrastively learning modal-separated and label-specific features. To further exploit the modality complementarity, we introduce a shuffle-based aggregation strategy to enrich co-occurrence collaboration among labels. Experiments on two benchmark datasets CMU-MOSEI and M3ED demonstrate the effectiveness of CARAT over state-of-the-art methods. Code is available at https://github.com/chengzju/CARAT.

cs.MM↗

arXiv2023

Biomimetic Synchronization in biciliated robots

Direct mechanical coupling is known to be critical for establishing synchronization among cilia. However, the actual role of the connections is still elusive - partly because controlled experiments in live samples are challenging. Here, we employ an artificial ciliary system to address this issue. Two cilia are formed by chains of self-propelling robots and anchored to a shared base so that they are purely mechanically-coupled. The system mimics biological ciliary beating but allows fine control over the beating dynamics. We find that the artificial cilia exhibit rich motion behaviors, depending on the mechanical coupling scheme. Particularly, their synchronous beating display two distinct modes - analogous to those observed in C. reinhardtii, the biciliated model organism for studying synchronization. Close examination suggests that the system evolves towards the most dissipative mode. Using this guideline in both simulations and experiments, we are able to direct the system into a desired state by altering the modes' respective dissipation. Our results have significant implications in understanding the synchronization of cilia.

cond-mat.soft↗

arXiv2023

Goal Exploration Augmentation via Pre-trained Skills for Sparse-Reward Long-Horizon Goal-Conditioned Reinforcement Learning

Reinforcement learning (RL) often struggles to accomplish a sparse-reward long-horizon task in a complex environment. Goal-conditioned reinforcement learning (GCRL) has been employed to tackle this difficult problem via a curriculum of easy-to-reach sub-goals. In GCRL, exploring novel sub-goals is essential for the agent to ultimately find the pathway to the desired goal. How to explore novel sub-goals efficiently is one of the most challenging issues in GCRL. Several goal exploration methods have been proposed to address this issue but still struggle to find the desired goals efficiently. In this paper, we propose a novel learning objective by optimizing the entropy of both achieved and new goals to be explored for more efficient goal exploration in sub-goal selection based GCRL. To optimize this objective, we first explore and exploit the frequently occurring goal-transition patterns mined in the environments similar to the current task to compose skills via skill learning. Then, the pretrained skills are applied in goal exploration. Evaluation on a variety of spare-reward long-horizon benchmark tasks suggests that incorporating our method into several state-of-the-art GCRL baselines significantly boosts their exploration efficiency while improving or maintaining their performance. The source code is available at: https://github.com/GEAPS/GEAPS.

cs.LG↗

arXiv2023

Bias Resilient Multi-Step Off-Policy Goal-Conditioned Reinforcement Learning

In goal-conditioned reinforcement learning (GCRL), sparse rewards present significant challenges, often obstructing efficient learning. Although multi-step GCRL can boost this efficiency, it can also lead to off-policy biases in target values. This paper dives deep into these biases, categorizing them into two distinct categories: "shooting" and "shifting". Recognizing that certain behavior policies can hasten policy refinement, we present solutions designed to capitalize on the positive aspects of these biases while minimizing their drawbacks, enabling the use of larger step sizes to speed up GCRL. An empirical study demonstrates that our approach ensures a resilient and robust improvement, even in ten-step learning scenarios, leading to superior learning efficiency and performance that generally surpass the baseline and several state-of-the-art multi-step GCRL benchmarks.

cs.LG↗

arXiv2023

The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluation

We introduce the Song Describer dataset (SDD), a new crowdsourced corpus of high-quality audio-caption pairs, designed for the evaluation of music-and-language models. The dataset consists of 1.1k human-written natural language descriptions of 706 music recordings, all publicly accessible and released under Creative Common licenses. To showcase the use of our dataset, we benchmark popular models on three key music-and-language tasks (music captioning, text-to-music generation and music-language retrieval). Our experiments highlight the importance of cross-dataset evaluation and offer insights into how researchers can use SDD to gain a broader understanding of model performance.

cs.SD↗

arXiv2023

Manifold-Aware Self-Training for Unsupervised Domain Adaptation on Regressing 6D Object Pose

Domain gap between synthetic and real data in visual regression (e.g. 6D pose estimation) is bridged in this paper via global feature alignment and local refinement on the coarse classification of discretized anchor classes in target space, which imposes a piece-wise target manifold regularization into domain-invariant representation learning. Specifically, our method incorporates an explicit self-supervised manifold regularization, revealing consistent cumulative target dependency across domains, to a self-training scheme (e.g. the popular Self-Paced Self-Training) to encourage more discriminative transferable representations of regression tasks. Moreover, learning unified implicit neural functions to estimate relative direction and distance of targets to their nearest class bins aims to refine target classification predictions, which can gain robust performance against inconsistent feature scaling sensitive to UDA regressors. Experiment results on three public benchmarks of the challenging 6D pose estimation task can verify the effectiveness of our method, consistently achieving superior performance to the state-of-the-art for UDA on 6D pose estimation.

cs.CV↗

arXiv2023

Thermal conductivity of MgO in giant planetary interior conditions predicted by deep potential

Thermal conductivity $κ$ of MgO plays a fundamental role in understanding the thermal evolution and mantle convection in the interior of terrestrial planets. However, previous theoretical calculations deviate from each other and the $κ$ of high-pressure B2 phase remains undetermined. Here, by combining molecular dynamics and deep potential trained with first-principles data, we systematically investigate the $κ$ of MgO from ambient state to the core-mantle boundary (CMB) of super-Earth with $5M_{\oplus}$. We point out the significance of 4-phonon scatterings and modify the conventional thermal conductivity model of MgO by considering the density-dependent proportion of 3-phonon and 4-phonon scatterings. The $κ$ profiles of MgO in Earth and super-Earth are further estimated. For super-Earth, we predict a significant reduction of $κ$ at the B1-B2 phase transition area near the CMB. This work provides new insights into thermal transport under extreme conditions and an improved thermal model for terrestrial planets.

astro-ph.EP↗

arXiv2023

Pseudo-differential integral autoencoder network for inverse PDE operators

Partial differential equations (PDEs) play a foundational role in modeling physical phenomena. This study addresses the challenging task of determining variable coefficients within PDEs from measurement data. We introduce a novel neural network, "pseudo-differential IAEnet" (pd-IAEnet), which draws inspiration from pseudo-differential operators. pd-IAEnet achieves significantly enhanced computational speed and accuracy with fewer parameters compared to conventional models. Extensive benchmark evaluations are conducted across a range of inverse problems, including Electrical Impedance Tomography (EIT), optical tomography, and seismic imaging, consistently demonstrating pd-IAEnet's superior accuracy. Notably, pd-IAEnet exhibits robustness in the presence of measurement noise, a critical characteristic for real-world applications. An exceptional feature is its discretization invariance, enabling effective training on data from diverse discretization schemes while maintaining accuracy on different meshes. In summary, pd-IAEnet offers a potent and efficient solution for addressing inverse PDE problems, contributing to improved computational efficiency, robustness, and adaptability to a wide array of data sources.

math.NA↗

arXiv2023

Nest-DGIL: Nesterov-optimized Deep Geometric Incremental Learning for CS Image Reconstruction

Proximal gradient-based optimization is one of the most common strategies to solve inverse problem of images, and it is easy to implement. However, these techniques often generate heavy artifacts in image reconstruction. One of the most popular refinement methods is to fine-tune the regularization parameter to alleviate such artifacts, but it may not always be sufficient or applicable due to increased computational costs. In this work, we propose a deep geometric incremental learning framework based on the second Nesterov proximal gradient optimization. The proposed end-to-end network not only has the powerful learning ability for high-/low-frequency image features, but also can theoretically guarantee that geometric texture details will be reconstructed from preliminary linear reconstruction. Furthermore, it can avoid the risk of intermediate reconstruction results falling outside the geometric decomposition domains and achieve fast convergence. Our reconstruction framework is decomposed into four modules including general linear reconstruction, cascade geometric incremental restoration, Nesterov acceleration, and post-processing. In the image restoration step, a cascade geometric incremental learning module is designed to compensate for missing texture information from different geometric spectral decomposition domains. Inspired by the overlap-tile strategy, we also develop a post-processing module to remove the block effect in patch-wise-based natural image reconstruction. All parameters in the proposed model are learnable, an adaptive initialization technique of physical parameters is also employed to make model flexibility and ensure converging smoothly. We compare the reconstruction performance of the proposed method with existing state-of-the-art methods to demonstrate its superiority. Our source codes are available at https://github.com/fanxiaohong/Nest-DGIL.

eess.IV↗

arXiv2023

Deep Operator Learning Lessens the Curse of Dimensionality for PDEs

Deep neural networks (DNNs) have achieved remarkable success in numerous domains, and their application to PDE-related problems has been rapidly advancing. This paper provides an estimate for the generalization error of learning Lipschitz operators over Banach spaces using DNNs with applications to various PDE solution operators. The goal is to specify DNN width, depth, and the number of training samples needed to guarantee a certain testing error. Under mild assumptions on data distributions or operator structures, our analysis shows that deep operator learning can have a relaxed dependence on the discretization resolution of PDEs and, hence, lessen the curse of dimensionality in many PDE-related problems including elliptic equations, parabolic equations, and Burgers equations. Our results are also applied to give insights about discretization-invariance in operator learning.

cs.LG↗

arXiv2023

Atomic Insights into the Oxidative Degradation Mechanisms of Sulfide Solid Electrolytes

Electrochemical degradation of solid electrolytes is a major roadblock in the development of solid-state batteries, and the formed solid-solid interphase (SSI) plays a key role in the performance of solid-state batteries. In this study, by combining experimental X-ray absorption spectroscopy (XAS) measurements, first-principles simulations, and unsupervised machine learning, we have unraveled the atomic-scale oxidative degradation mechanisms of sulfide electrolytes at the interface using the baseline Li3PS4 (LPS) electrolyte as a model system. The degradation begins with a decrease of Li neighbor affinity to S atoms upon initial delithiation, followed by the formation of S-S bonds as the PS4 tetrahedron deforms. After the first delithiation cycle, the PS4 motifs become strongly distorted and PS3 motifs start to form. Spectral fingerprints of the local structural evolution are identified, which correspond to the main peak broadening and the peak shifting to a higher energy by about 2.5 eV in P K-edge XAS and a new peak emerging at 2473 eV in S K-edge XAS during delithiation. The spectral fingerprints serve as a proxy for the electrochemical stability of phosphorus sulfide solid electrolytes beyond LPS, as demonstrated in argyrodite Li6PS5Cl. We observed that the strong distortion and destruction of PS4 tetrahedra and the formation of S-S bonds are correlated with an increased interfacial impedance. To the best of our knowledge, this study showcases the first atomic-scale insights into the oxidative degradation mechanism of the LPS electrolyte, which can provide guidance for controlling macroscopic reactions through microstructural engineering and, more generally, can advance the rational design of sulfide electrolytes.

cond-mat.mtrl-sci↗

arXiv2023

AudioSR: Versatile Audio Super-resolution at Scale

Audio super-resolution is a fundamental task that predicts high-frequency components for low-resolution audio, enhancing audio quality in digital applications. Previous methods have limitations such as the limited scope of audio types (e.g., music, speech) and specific bandwidth settings they can handle (e.g., 4kHz to 8kHz). In this paper, we introduce a diffusion-based generative model, AudioSR, that is capable of performing robust audio super-resolution on versatile audio types, including sound effects, music, and speech. Specifically, AudioSR can upsample any input audio signal within the bandwidth range of 2kHz to 16kHz to a high-resolution audio signal at 24kHz bandwidth with a sampling rate of 48kHz. Extensive objective evaluation on various audio super-resolution benchmarks demonstrates the strong result achieved by the proposed model. In addition, our subjective evaluation shows that AudioSR can acts as a plug-and-play module to enhance the generation quality of a wide range of audio generative models, including AudioLDM, Fastspeech2, and MusicGen. Our code and demo are available at https://audioldm.github.io/audiosr.

cs.SD↗

arXiv2023

Adopting Dynamic VAR Compensators to Mitigate PV Impacts on Unbalanced Distribution Systems

The growing integration of distributed energy resources into distribution systems poses challenges for voltage regulation. Dynamic VAR Compensators (DVCs) are a new generation of power electronics-based Volt/VAR compensation devices designed to address voltage issues in distribution systems with a high penetration of renewable generation resources. Currently, the IEEE Std. 1547-based Volt/VAR Curve (VV-C) is widely used as the local control scheme for controlling a DVC. However, the effectiveness of this scheme is not well documented, and there is limited literature on alternative control and placement schemes that can maximize the effective use of a DVC. In this paper, we propose an optimal dispatch and control mechanism to enhance the conventional VV-C based localized DVC control. First, we establish a multi-objective optimization framework to identify the optimal dispatch strategy and suitable placement for the DVC. Next, we introduce two supervisory control strategies to determine the appropriate instances for adjusting the VV-C when the operating condition changes. The outlined scheme comprises two primary stages: time segmentation and VV-C fitting. Within this framework, each time segment aims to produce optimized Q-V trajectories. The proposed method is tested on a modified IEEE 123-bus test system using OpenDSS for a wide range of operating scenarios, including sunny and cloudy days. Simulation results demonstrate that the proposed scheme effectively reduces voltage variations compared to the standard VV-C specified in IEEE Std. 1547.

eess.SY↗

arXiv2023

Super-Resolution Surface Reconstruction from Few Low-Resolution Slices

In many imaging applications where segmented features (e.g. blood vessels) are further used for other numerical simulations (e.g. finite element analysis), the obtained surfaces do not have fine resolutions suitable for the task. Increasing the resolution of such surfaces becomes crucial. This paper proposes a new variational model for solving this problem, based on an Euler-Elastica-based regulariser. Further, we propose and implement two numerical algorithms for solving the model, a projected gradient descent method and the alternating direction method of multipliers. Numerical experiments using real-life examples (including two from outputs of another variational model) have been illustrated for effectiveness. The advantages of the new model are shown through quantitative comparisons by the standard deviation of Gaussian curvatures and mean curvatures from the viewpoint of discrete geometry.

math.AP↗

arXiv2023

Towards Improving Harmonic Sensitivity and Prediction Stability for Singing Melody Extraction

In deep learning research, many melody extraction models rely on redesigning neural network architectures to improve performance. In this paper, we propose an input feature modification and a training objective modification based on two assumptions. First, harmonics in the spectrograms of audio data decay rapidly along the frequency axis. To enhance the model's sensitivity on the trailing harmonics, we modify the Combined Frequency and Periodicity (CFP) representation using discrete z-transform. Second, the vocal and non-vocal segments with extremely short duration are uncommon. To ensure a more stable melody contour, we design a differentiable loss function that prevents the model from predicting such segments. We apply these modifications to several models, including MSNet, FTANet, and a newly introduced model, PianoNet, modified from a piano transcription network. Our experimental results demonstrate that the proposed modifications are empirically effective for singing melody extraction.

cs.SD↗

arXiv2023

MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategies

Diffusion models have shown promising results in cross-modal generation tasks, including text-to-image and text-to-audio generation. However, generating music, as a special type of audio, presents unique challenges due to limited availability of music data and sensitive issues related to copyright and plagiarism. In this paper, to tackle these challenges, we first construct a state-of-the-art text-to-music model, MusicLDM, that adapts Stable Diffusion and AudioLDM architectures to the music domain. We achieve this by retraining the contrastive language-audio pretraining model (CLAP) and the Hifi-GAN vocoder, as components of MusicLDM, on a collection of music data samples. Then, to address the limitations of training data and to avoid plagiarism, we leverage a beat tracking model and propose two different mixup strategies for data augmentation: beat-synchronous audio mixup and beat-synchronous latent mixup, which recombine training audio directly or via a latent embeddings space, respectively. Such mixup strategies encourage the model to interpolate between musical training samples and generate new music within the convex hull of the training data, making the generated music more diverse while still staying faithful to the corresponding style. In addition to popular evaluation metrics, we design several new evaluation metrics based on CLAP score to demonstrate that our proposed MusicLDM and beat-synchronous mixup strategies improve both the quality and novelty of generated music, as well as the correspondence between input text and generated music.

cs.SD↗

arXiv2023

ShuffleMix: Improving Representations via Channel-Wise Shuffle of Interpolated Hidden States

Mixup style data augmentation algorithms have been widely adopted in various tasks as implicit network regularization on representation learning to improve model generalization, which can be achieved by a linear interpolation of labeled samples in input or feature space as well as target space. Inspired by good robustness of alternative dropout strategies against over-fitting on limited patterns of training samples, this paper introduces a novel concept of ShuffleMix -- Shuffle of Mixed hidden features, which can be interpreted as a kind of dropout operation in feature space. Specifically, our ShuffleMix method favors a simple linear shuffle of randomly selected feature channels for feature mixup in-between training samples to leverage semantic interpolated supervision signals, which can be extended to a generalized shuffle operation via additionally combining linear interpolations of intra-channel features. Compared to its direct competitor of feature augmentation -- the Manifold Mixup, the proposed ShuffleMix can gain superior generalization, owing to imposing more flexible and smooth constraints on generating samples and achieving regularization effects of channel-wise feature dropout. Experimental results on several public benchmarking datasets of single-label and multi-label visual classification tasks can confirm the effectiveness of our method on consistently improving representations over the state-of-the-art mixup augmentation.

cs.CV↗

arXiv2023

Multitrack Music Transformer

Existing approaches for generating multitrack music with transformer models have been limited in terms of the number of instruments, the length of the music segments and slow inference. This is partly due to the memory requirements of the lengthy input sequences necessitated by existing representations. In this work, we propose a new multitrack music representation that allows a diverse set of instruments while keeping a short sequence length. Our proposed Multitrack Music Transformer (MMT) achieves comparable performance with state-of-the-art systems, landing in between two recently proposed models in a subjective listening test, while achieving substantial speedups and memory reductions over both, making the method attractive for real time improvisation or near real time creative applications. Further, we propose a new measure for analyzing musical self-attention and show that the trained model attends more to notes that form a consonant interval with the current note and to notes that are 4N beats away from the current step.

cs.SD↗