arXiv subjects

Yi Ma

Publications and source records attributed to Yi Ma.

At least 145 records · Page 8Linked to original sources

arXiv2024

Gen4Gen: Generative Data Pipeline for Generative Multi-Concept Composition

Recent text-to-image diffusion models are able to learn and synthesize images containing novel, personalized concepts (e.g., their own pets or specific items) with just a few examples for training. This paper tackles two interconnected issues within this realm of personalizing text-to-image diffusion models. First, current personalization techniques fail to reliably extend to multiple concepts -- we hypothesize this to be due to the mismatch between complex scenes and simple text descriptions in the pre-training dataset (e.g., LAION). Second, given an image containing multiple personalized concepts, there lacks a holistic metric that evaluates performance on not just the degree of resemblance of personalized concepts, but also whether all concepts are present in the image and whether the image accurately reflects the overall text description. To address these issues, we introduce Gen4Gen, a semi-automated dataset creation pipeline utilizing generative models to combine personalized concepts into complex compositions along with text-descriptions. Using this, we create a dataset called MyCanvas, that can be used to benchmark the task of multi-concept personalization. In addition, we design a comprehensive metric comprising two scores (CP-CLIP and TI-CLIP) for better quantifying the performance of multi-concept, personalized text-to-image diffusion methods. We provide a simple baseline built on top of Custom Diffusion with empirical prompting strategies for future researchers to evaluate on MyCanvas. We show that by improving data quality and prompting strategies, we can significantly increase multi-concept personalized image generation quality, without requiring any modifications to model architecture or training algorithms.

cs.CV↗

arXiv2024

Near-Field Fading Channel Modeling for ELAAs: From Communication to ISAC

Extremely large aperture array (ELAA) is anticipated to serve as a pivotal feature of future multiple-input multiple-output (MIMO) systems in 6G. Near-field (NF) fading channel models are essential for reliable link-level simulation and ELAA system design. In this article, we propose a framework designed to generate NF fading channels for both communication and integrated sensing and communication (ISAC) applications. The framework allows a mixed of line of sight (LoS) and non-LoS (NLoS) links. It also considers spherical wave model and spatially non-stationary shadow fading. Based on this framework, we propose a three-dimensional (3D) fading channel model for ELAA systems deployed with a uniform rectangular array (URA). It can capture the impact of sensing object for ISAC applications. Moreover, all parameters involved in the framework are based on specifications or measurements from the 3rd Generation Partnership Project (3GPP) documents. Therefore, the proposed framework and channel model have the potential to contribute to the standard in various aspects, including ISAC, extra-large (XL-) MIMO, and reconfigurable intelligent surface (RIS) aided MIMO systems. Finally, future directions for ELAA are presented, including not only NF channel modeling but also the design of next-generation transceivers.

cs.IT↗

arXiv2024

Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning

A compelling use case of offline reinforcement learning (RL) is to obtain a policy initialization from existing datasets followed by fast online fine-tuning with limited interaction. However, existing offline RL methods tend to behave poorly during fine-tuning. In this paper, we devise an approach for learning an effective initialization from offline data that also enables fast online fine-tuning capabilities. Our approach, calibrated Q-learning (Cal-QL), accomplishes this by learning a conservative value function initialization that underestimates the value of the learned policy from offline data, while also being calibrated, in the sense that the learned Q-values are at a reasonable scale. We refer to this property as calibration, and define it formally as providing a lower bound on the true value function of the learned policy and an upper bound on the value of some other (suboptimal) reference policy, which may simply be the behavior policy. We show that offline RL algorithms that learn such calibrated value functions lead to effective online fine-tuning, enabling us to take the benefits of offline initializations in online fine-tuning. In practice, Cal-QL can be implemented on top of the conservative Q learning (CQL) for offline RL within a one-line code change. Empirically, Cal-QL outperforms state-of-the-art methods on 9/11 fine-tuning benchmark tasks that we study in this paper. Code and video are available at https://nakamotoo.github.io/Cal-QL

cs.LG↗

arXiv2024

Gradient weighting for speaker verification in extremely low Signal-to-Noise Ratio

Speaker verification is hampered by background noise, particularly at extremely low Signal-to-Noise Ratio (SNR) under 0 dB. It is difficult to suppress noise without introducing unwanted artifacts, which adversely affects speaker verification. We proposed the mechanism called Gradient Weighting (Grad-W), which dynamically identifies and reduces artifact noise during prediction. The mechanism is based on the property that the gradient indicates which parts of the input the model is paying attention to. Specifically, when the speaker network focuses on a region in the denoised utterance but not on the clean counterpart, we consider it artifact noise and assign higher weights for this region during optimization of enhancement. We validate it by training an enhancement model and testing the enhanced utterance on speaker verification. The experimental results show that our approach effectively reduces artifact noise, improving speaker verification across various SNR levels.

cs.SD↗

arXiv2023

On Chernoff Lower-Bound of Outage Threshold for Non-Central $χ^2$-Distributed Beamforming Gain in URLLC Systems

The cumulative distribution function (CDF) of a non-central $χ^2$-distributed random variable (RV) is often used when measuring the outage probability of communication systems. For ultra-reliable low-latency communication (URLLC), it is important but mathematically challenging to determine the outage threshold for an extremely small outage target. This motivates us to investigate lower bounds of the outage threshold, and it is found that the one derived from the Chernoff inequality (named Cher-LB) is the most effective lower bound. This finding is associated with three rigorously established properties of the Cher-LB with respect to the mean, variance, reliability requirement, and degrees of freedom of the non-central $χ^2$-distributed RV. The Cher-LB is then employed to predict the beamforming gain in URLLC for both conventional multi-antenna systems (i.e., MIMO) under first-order Markov time-varying channel and reconfigurable intellgent surface (RIS) systems. It is exhibited that, with the proposed Cher-LB, the pessimistic prediction of the beamforming gain is made sufficiently accurate for guaranteed reliability as well as the transmit-energy efficiency.

eess.SP↗

arXiv2023

EventAid: Benchmarking Event-aided Image/Video Enhancement Algorithms with Real-captured Hybrid Dataset

Event cameras are emerging imaging technology that offers advantages over conventional frame-based imaging sensors in dynamic range and sensing speed. Complementing the rich texture and color perception of traditional image frames, the hybrid camera system of event and frame-based cameras enables high-performance imaging. With the assistance of event cameras, high-quality image/video enhancement methods make it possible to break the limits of traditional frame-based cameras, especially exposure time, resolution, dynamic range, and frame rate limits. This paper focuses on five event-aided image and video enhancement tasks (i.e., event-based video reconstruction, event-aided high frame rate video reconstruction, image deblurring, image super-resolution, and high dynamic range image reconstruction), provides an analysis of the effects of different event properties, a real-captured and ground truth labeled benchmark dataset, a unified benchmarking of state-of-the-art methods, and an evaluation for two mainstream event simulators. In detail, this paper collects a real-captured evaluation dataset EventAid for five event-aided image/video enhancement tasks, by using "Event-RGB" multi-camera hybrid system, taking into account scene diversity and spatiotemporal synchronization. We further perform quantitative and visual comparisons for state-of-the-art algorithms, provide a controlled experiment to analyze the performance limit of event-aided image deblurring methods, and discuss open problems to inspire future research.

cs.CV↗

arXiv2023

Investigating the Catastrophic Forgetting in Multimodal Large Language Models

Following the success of GPT4, there has been a surge in interest in multimodal large language model (MLLM) research. This line of research focuses on developing general-purpose LLMs through fine-tuning pre-trained LLMs and vision models. However, catastrophic forgetting, a notorious phenomenon where the fine-tuned model fails to retain similar performance compared to the pre-trained model, still remains an inherent problem in multimodal LLMs (MLLM). In this paper, we introduce EMT: Evaluating MulTimodality for evaluating the catastrophic forgetting in MLLMs, by treating each MLLM as an image classifier. We first apply EMT to evaluate several open-source fine-tuned MLLMs and we discover that almost all evaluated MLLMs fail to retain the same performance levels as their vision encoders on standard image classification tasks. Moreover, we continue fine-tuning LLaVA, an MLLM and utilize EMT to assess performance throughout the fine-tuning. Interestingly, our results suggest that early-stage fine-tuning on an image dataset improves performance across other image datasets, by enhancing the alignment of text and visual features. However, as fine-tuning proceeds, the MLLMs begin to hallucinate, resulting in a significant loss of generalizability, even when the image encoder remains frozen. Our results suggest that MLLMs have yet to demonstrate performance on par with their vision models on standard image classification tasks and the current MLLM fine-tuning procedure still has room for improvement.

cs.CL↗

arXiv2023

Role of Uncertainty in Anticipatory Trajectory Prediction for a Ping-Pong Playing Robot

Robotic interaction in fast-paced environments presents a substantial challenge, particularly in tasks requiring the prediction of dynamic, non-stationary objects for timely and accurate responses. An example of such a task is ping-pong, where the physical limitations of a robot may prevent it from reaching its goal in the time it takes the ball to cross the table. The scene of a ping-pong match contains rich visual information of a player's movement that can allow future game state prediction, with varying degrees of uncertainty. To this aim, we present a visual modeling, prediction, and control system to inform a ping-pong playing robot utilizing visual model uncertainty to allow earlier motion of the robot throughout the game. We present demonstrations and metrics in simulation to show the benefit of incorporating model uncertainty, the limitations of current standard model uncertainty estimators, and the need for more verifiable model uncertainty estimation. Our code is publicly available.

cs.RO↗

arXiv2023

Leveraging User-Wise SVD for Accelerated Convergence in Iterative ELAA-MIMO Detections

Numerous low-complexity iterative algorithms have been proposed to offer the performance of linear multiple-input multiple-output (MIMO) detectors bypassing the channel matrix inverse. These algorithms exhibit fast convergence in well-conditioned MIMO channels. However, in the emerging MIMO paradigm utilizing extremely large aperture arrays (ELAA), the wireless channel may become ill-conditioned because of spatial non-stationarity, which results in a considerably slower convergence rate for these algorithms. In this paper, we propose a novel ELAA-MIMO detection scheme that leverages user-wise singular value decomposition (UW-SVD) to accelerate the convergence of these iterative algorithms. By applying UW-SVD, the MIMO signal model can be converted into an equivalent form featuring a better-conditioned transfer function. Then, existing iterative algorithms can be utilized to recover the transmitted signal from the converted signal model with accelerated convergence towards zero-forcing performance. Our simulation results indicate that proposed UW-SVD scheme can significantly accelerate the convergence of the iterative algorithms in spatially non-stationary ELAA channels. Moreover, the computational complexity of the UW-SVD is comparatively minor in relation to the inherent complexity of the iterative algorithms.

cs.IT↗

arXiv2023

Rethinking Decision Transformer via Hierarchical Reinforcement Learning

Decision Transformer (DT) is an innovative algorithm leveraging recent advances of the transformer architecture in reinforcement learning (RL). However, a notable limitation of DT is its reliance on recalling trajectories from datasets, losing the capability to seamlessly stitch sub-optimal trajectories together. In this work we introduce a general sequence modeling framework for studying sequential decision making through the lens of Hierarchical RL. At the time of making decisions, a high-level policy first proposes an ideal prompt for the current state, a low-level policy subsequently generates an action conditioned on the given prompt. We show DT emerges as a special case of this framework with certain choices of high-level and low-level policies, and discuss the potential failure of these choices. Inspired by these observations, we study how to jointly optimize the high-level and low-level policies to enable the stitching ability, which further leads to the development of new offline RL algorithms. Our empirical results clearly show that the proposed algorithms significantly surpass DT on several control and navigation benchmarks. We hope our contributions can inspire the integration of transformer architectures within the field of RL.

cs.LG↗

arXiv2023

Iteratively Refined Behavior Regularization for Offline Reinforcement Learning

One of the fundamental challenges for offline reinforcement learning (RL) is ensuring robustness to data distribution. Whether the data originates from a near-optimal policy or not, we anticipate that an algorithm should demonstrate its ability to learn an effective control policy that seamlessly aligns with the inherent distribution of offline data. Unfortunately, behavior regularization, a simple yet effective offline RL algorithm, tends to struggle in this regard. In this paper, we propose a new algorithm that substantially enhances behavior-regularization based on conservative policy iteration. Our key observation is that by iteratively refining the reference policy used for behavior regularization, conservative policy update guarantees gradually improvement, while also implicitly avoiding querying out-of-sample actions to prevent catastrophic learning failures. We prove that in the tabular setting this algorithm is capable of learning the optimal policy covered by the offline dataset, commonly referred to as the in-sample optimal policy. We then explore several implementation details of the algorithm when function approximations are applied. The resulting algorithm is easy to implement, requiring only a few lines of code modification to existing methods. Experimental results on the D4RL benchmark indicate that our method outperforms previous state-of-the-art baselines in most tasks, clearly demonstrate its superiority over behavior regularization.

cs.LG↗

arXiv2023

General In-Hand Object Rotation with Vision and Touch

We introduce RotateIt, a system that enables fingertip-based object rotation along multiple axes by leveraging multimodal sensory inputs. Our system is trained in simulation, where it has access to ground-truth object shapes and physical properties. Then we distill it to operate on realistic yet noisy simulated visuotactile and proprioceptive sensory inputs. These multimodal inputs are fused via a visuotactile transformer, enabling online inference of object shapes and physical properties during deployment. We show significant performance improvements over prior methods and the importance of visual and tactile sensing.

cs.RO↗

arXiv2023

Representation Learning via Manifold Flattening and Reconstruction

This work proposes an algorithm for explicitly constructing a pair of neural networks that linearize and reconstruct an embedded submanifold, from finite samples of this manifold. Our such-generated neural networks, called Flattening Networks (FlatNet), are theoretically interpretable, computationally feasible at scale, and generalize well to test data, a balance not typically found in manifold-based learning methods. We present empirical results and comparisons to other models on synthetic high-dimensional manifold data and 2D image data. Our code is publicly available.

cs.LG↗

arXiv2023

Emergence of Segmentation with Minimalistic White-Box Transformers

Transformer-like models for vision tasks have recently proven effective for a wide range of downstream applications such as segmentation and detection. Previous works have shown that segmentation properties emerge in vision transformers (ViTs) trained using self-supervised methods such as DINO, but not in those trained on supervised classification tasks. In this study, we probe whether segmentation emerges in transformer-based models solely as a result of intricate self-supervised learning mechanisms, or if the same emergence can be achieved under much broader conditions through proper design of the model architecture. Through extensive experimental results, we demonstrate that when employing a white-box transformer-like architecture known as CRATE, whose design explicitly models and pursues low-dimensional structures in the data distribution, segmentation properties, at both the whole and parts levels, already emerge with a minimalistic supervised training recipe. Layer-wise finer-grained analysis reveals that the emergent properties strongly corroborate the designed mathematical functions of the white-box network. Our results suggest a path to design white-box foundation models that are simultaneously highly performant and mathematically fully interpretable. Code is at \url{https://github.com/Ma-Lab-Berkeley/CRATE}.

cs.CV↗

arXiv2023

A Spatially Non-stationary Fading Channel Model for Simulation and (Semi-) Analytical Study of ELAA-MIMO

In this paper, a novel spatially non-stationary fading channel model is proposed for multiple-input multiple-output (MIMO) system with extremely-large aperture service-array (ELAA). The proposed model incorporates three key factors which cause the channel spatial non-stationarity: 1) link-wise path-loss; 2) shadowing effect; 3) line-of-sight (LoS)/non-LoS state. With appropriate parameter configurations, the proposed model can be used to generate computer-simulated channel data that matches the published measurement data from practical ELAA-MIMO channels. Given such appealing results, the proposed fading channel model is employed to study the cumulative distribution function (CDF) of ELAA-MIMO channel capacity. For all of our studied scenarios, it is unveiled that the ELAA-MIMO channel capacity obeys the skew-normal distribution. Moreover, the channel capacity is also found close to the Gaussian or Weibull distribution, depending on users' geo-location and distribution. More specifically, for single-user equivalent scenarios or multiuser scenarios with short user-to-ELAA distances (e.g., 1 m), the channel capacity is close to the Gaussian distribution; and for others, it is close to the Weibull distribution. Finally, the proposed channel model is also employed to study the impact of channel spatial non-stationarity on linear MIMO receivers through computer simulations. The proposed fading channel model is available at https://github.com/ELAA-MIMO/non-stationary-fading-channel-model.

cs.IT↗

arXiv2023

Unsupervised Manifold Linearizing and Clustering

We consider the problem of simultaneously clustering and learning a linear representation of data lying close to a union of low-dimensional manifolds, a fundamental task in machine learning and computer vision. When the manifolds are assumed to be linear subspaces, this reduces to the classical problem of subspace clustering, which has been studied extensively over the past two decades. Unfortunately, many real-world datasets such as natural images can not be well approximated by linear subspaces. On the other hand, numerous works have attempted to learn an appropriate transformation of the data, such that data is mapped from a union of general non-linear manifolds to a union of linear subspaces (with points from the same manifold being mapped to the same subspace). However, many existing works have limitations such as assuming knowledge of the membership of samples to clusters, requiring high sampling density, or being shown theoretically to learn trivial representations. In this paper, we propose to optimize the Maximal Coding Rate Reduction metric with respect to both the data representation and a novel doubly stochastic cluster membership, inspired by state-of-the-art subspace clustering results. We give a parameterization of such a representation and membership, allowing efficient mini-batching and one-shot initialization. Experiments on CIFAR-10, -20, -100, and TinyImageNet-200 datasets show that the proposed method is much more accurate and scalable than state-of-the-art deep clustering methods, and further learns a latent linear representation of the data.

cs.LG↗

arXiv2023

On Chernoff Lower-Bound of Outage Threshold for Non-Central $χ^2$-Distributed MIMO Beamforming Gain

The cumulative distribution function (CDF) of a non-central $χ^2$-distributed random variable (RV) is often used when measuring the outage probability of communication systems. For adaptive transmitters, it is important but mathematically challenging to determine the outage threshold for an extreme target outage probability (e.g., $10^{-5}$ or less). This motivates us to investigate lower bounds of the outage threshold, and it is found that the one derived from the Chernoff inequality (named Cher-LB) is the most {effective} lower bound. The Cher-LB is then employed to predict the multi-antenna transmitter beamforming-gain in ultra-reliable and low-latency communication, concerning the first-order Markov time-varying channel. It is exhibited that, with the proposed Cher-LB, pessimistic prediction of the beamforming gain is made sufficiently accurate for guaranteed reliability as well as the transmit-energy efficiency.

eess.SP↗

arXiv2023

Alternative Normalized-Preconditioning for Scalable Iterative Large-MIMO Detection

Signal detection in large multiple-input multiple-output (large-MIMO) systems presents greater challenges compared to conventional massive-MIMO for two primary reasons. First, large-MIMO systems lack favorable propagation conditions as they do not require a substantially greater number of service antennas relative to user antennas. Second, the wireless channel may exhibit spatial non-stationarity when an extremely large aperture array (ELAA) is deployed in a large-MIMO system. In this paper, we propose a scalable iterative large-MIMO detector named ANPID, which simultaneously delivers 1) close to maximum-likelihood detection performance, 2) low computational-complexity (i.e., square-order of transmit antennas), 3) fast convergence, and 4) robustness to the spatial non-stationarity in ELAA channels. ANPID incorporates a damping demodulation step into stationary iterative (SI) methods and alternates between two distinct demodulated SI methods. Simulation results demonstrate that ANPID fulfills all the four features concurrently and outperforms existing low-complexity MIMO detectors, especially in highly-loaded large MIMO systems.

cs.IT↗