arXiv subjects

Zhao Wang

Publications and source records attributed to Zhao Wang.

At least 109 records · Page 6Linked to original sources

arXiv2024

Point Cloud-Assisted Neural Image Compression

High-efficient image compression is a critical requirement. In several scenarios where multiple modalities of data are captured by different sensors, the auxiliary information from other modalities are not fully leveraged by existing image-only codecs, leading to suboptimal compression efficiency. In this paper, we increase image compression performance with the assistance of point cloud, which is widely adopted in the area of autonomous driving. We first unify the data representation for both modalities to facilitate data processing. Then, we propose the point cloud-assisted neural image codec (PCA-NIC) to enhance the preservation of image texture and structure by utilizing the high-dimensional point cloud information. We further introduce a multi-modal feature fusion transform module (MMFFT) to capture more representative image features, remove redundant information between channels and modalities that are not relevant to the image content. Our work is the first to improve image compression performance using point cloud and achieves state-of-the-art performance.

eess.IV↗

arXiv2024

MICAS: Multi-grained In-Context Adaptive Sampling for 3D Point Cloud Processing

Point cloud processing (PCP) encompasses tasks like reconstruction, denoising, registration, and segmentation, each often requiring specialized models to address unique task characteristics. While in-context learning (ICL) has shown promise across tasks by using a single model with task-specific demonstration prompts, its application to PCP reveals significant limitations. We identify inter-task and intra-task sensitivity issues in current ICL methods for PCP, which we attribute to inflexible sampling strategies lacking context adaptation at the point and prompt levels. To address these challenges, we propose MICAS, an advanced ICL framework featuring a multi-grained adaptive sampling mechanism tailored for PCP. MICAS introduces two core components: task-adaptive point sampling, which leverages inter-task cues for point-level sampling, and query-specific prompt sampling, which selects optimal prompts per query to mitigate intra-task sensitivity. To our knowledge, this is the first approach to introduce adaptive sampling tailored to the unique requirements of point clouds within an ICL framework. Extensive experiments show that MICAS not only efficiently handles various PCP tasks but also significantly outperforms existing methods. Notably, it achieves a remarkable $4.1\%$ improvement in the part segmentation task and delivers consistent gains across various PCP applications.

cs.CV↗

arXiv2024

Cooperative engineering the multiple radio-frequency fields to reduce the X-junction barrier for ion trap chips

With the increasing number of ion qubits and improving performance of sophisticated quantum algorithms, more and more scalable complex ion trap electrodes have been developed and integrated. Nonlinear ion shuttling operations at the junction are more frequently used, such as in the areas of separation, merging, and exchanging. Several studies have been conducted to optimize the geometries of the radio-frequency (RF) electrodes to generate ideal trapping electric fields with a lower junction barrier and an even ion height of the RF saddle points. However, this iteration is time-consuming and commonly accompanied by complicated and sharp electrode geometry. Therefore, high-accuracy fabrication process and high electric breakdown voltage are essential. In the current work, an effective method was proposed to reduce the junction's pseudo-potential barrier and ion height variation by setting several individual RF electrodes and adjusting each RF voltage amplitude without changing the geometry of the electrode structure. The simulation results show that this method shows the same effect on engineering the trapping potential and reducing the potential barrier, but requires fewer parameters and optimization time. By combining this method with the geometrical shape-optimizing, the pseudo-potential barrier and the ion height variation near the junction can be further reduced. In addition, the geometry of the electrodes can be simplified to relax the fabrication precision and keep the ability to engineer the trapping electric field in real-time even after the fabrication of the electrodes, which provides a potential all-electric degree of freedom for the design and control of the two-dimensional ion crystals and investigation of their phase transition.

quant-ph↗

arXiv2024

Theseus: Exploring Efficient Wafer-Scale Chip Design for Large Language Models

The emergence of the large language model~(LLM) poses an exponential growth of demand for computation throughput, memory capacity, and communication bandwidth. Such a demand growth has significantly surpassed the improvement of corresponding chip designs. With the advancement of fabrication and integration technologies, designers have been developing Wafer-Scale Chips~(WSCs) to scale up and exploit the limits of computation density, memory capacity, and communication bandwidth at the level of a single chip. Existing solutions have demonstrated the significant advantages of WSCs over traditional designs, showing potential to effectively support LLM workloads. Despite the benefits, exploring the early-stage design space of WSCs for LLMs is a crucial yet challenging task due to the enormous and complicated design space, time-consuming evaluation methods, and inefficient exploration strategies. To address these challenges, we propose Theseus, an efficient WSC design space exploration framework for LLMs. We construct the design space of WSCs with various constraints considering the unique characteristics of WSCs. We propose efficient evaluation methodologies for large-scale NoC-based WSCs and introduce multi-fidelity Bayesian optimization to efficiently explore the design space. Evaluation results demonstrate the efficiency of Theseus that the searched Pareto optimal results outperform GPU cluster and existing WSC designs by up to 62.8\%/73.7\% in performance and 38.6\%/42.4\% in power consumption for LLM training, while improving up to 23.2$\times$ and 15.7$\times$ for the performance and power of inference tasks. Furthermore, we conduct case studies to address the design tradeoffs in WSCs and provide insights to facilitate WSC designs for LLMs.

cs.AR↗

arXiv2024

A Digital and Compact High-Precision Locking System for Pulse Laser Repetition Frequency

This paper proposes a novel approach that employs error amplification and ADC-based dual-mixer time-difference (ADC-based-DMTD) technique for high-precision locking of laser repetition frequency with compact size. This electronic system consists of two main components: a digitized error amplification module (EAM) and an FPGA-based digital frequency locking module (DFLM). The EAM mainly integrates a configurable frequency generator (CFG), a configurable frequency multiplier (CFM) and a mixer to process the laser pulses and a high-stability reference source (e.g., an atomic clock), enabling high-precision locking of pulse lasers operating at different repetition frequencies. By employing frequency multiplication and mixing, the EAM amplifies the laser's frequency error and performs frequency down-conversion, enhancing measurement sensitivity and reducing the hardware requirements of the back-end. The DFLM receives the EAM outputs by using an ADC-based-DMTD method to precisely measure frequency errors, then the digital proportional-integral-derivative (PID) controller provides feedback to achieve accurate frequency locking. Initial testing with a voltage-controlled oscillator (VCO) demonstrated excellent locking performance, achieving an Allan deviation of $9.58 \times 10^{-14}$ at 10 seconds and a standard deviation (STD) of 7.7 \textmu Hz root mean square (RMS) after locking, marking a five-order-of-magnitude stability enhancement. In laboratory experiments with a custom-built femtosecond fiber laser, the system achieved robust locking of the repetition frequency, with a stability improvement from $1.51 \times 10^{-7}$ to $1.12 \times 10^{-12}$ at a 10-second gate time and an STD of 0.43 mHz RMS after locking.

physics.optics↗

arXiv2024

OKG: On-the-Fly Keyword Generation in Sponsored Search Advertising

Current keyword decision-making in sponsored search advertising relies on large, static datasets, limiting the ability to automatically set up keywords and adapt to real-time KPI metrics and product updates that are essential for effective advertising. In this paper, we propose On-the-fly Keyword Generation (OKG), an LLM agent-based method that dynamically monitors KPI changes and adapts keyword generation in real time, aligning with strategies recommended by advertising platforms. Additionally, we introduce the first publicly accessible dataset containing real keyword data along with its KPIs across diverse domains, providing a valuable resource for future research. Experimental results show that OKG significantly improves keyword adaptability and responsiveness compared to traditional methods. The code for OKG and the dataset are available at https://github.com/sony/okg.

cs.IR↗

arXiv2024

Mean-Field Control for Diffusion Aggregation system with Coulomb Interaction

The mean-field control problem for a multi-dimensional diffusion-aggregation system with Coulomb interaction (the so called parabolic elliptic Keller-Segel system) is considered. The existence of optimal control is proved through the $Γ$-convergence of the corresponding control problem of the interacting particle system. There are three building blocks in the whole argument. Firstly, for the optimal control problem on the particle level, instead of using classical method for stochastic system, we study directly the control problem of high-dimensional parabolic equation, i.e. the Liouville equation of it. Secondly, we obtain a strong propagation of chaos result for the interacting particle system by combining the convergence in probability and relative entropy method. Due to this strong mean field limit result, we avoid giving compact support requirement for control functions, which has been often used in the literature. Thirdly, because of strong aggregation effect, additional difficulties arise from control function in obtaining the well-posedness theory of the diffusion-aggregation equation, so that the known method cannot be directly applied. Instead, we use a combination of local existence result and bootstrap argument to obtain the global solution in the sub-critical regime.

math.OC↗

arXiv2024

3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation

Despite that the segment anything model (SAM) achieved impressive results on general-purpose semantic segmentation with strong generalization ability on daily images, its demonstrated performance on medical image segmentation is less precise and not stable, especially when dealing with tumor segmentation tasks that involve objects of small sizes, irregular shapes, and low contrast. Notably, the original SAM architecture is designed for 2D natural images, therefore would not be able to extract the 3D spatial information from volumetric medical data effectively. In this paper, we propose a novel adaptation method for transferring SAM from 2D to 3D for promptable medical image segmentation. Through a holistically designed scheme for architecture modification, we transfer the SAM to support volumetric inputs while retaining the majority of its pre-trained parameters for reuse. The fine-tuning process is conducted in a parameter-efficient manner, wherein most of the pre-trained parameters remain frozen, and only a few lightweight spatial adapters are introduced and tuned. Regardless of the domain gap between natural and medical data and the disparity in the spatial arrangement between 2D and 3D, the transformer trained on natural images can effectively capture the spatial patterns present in volumetric medical images with only lightweight adaptations. We conduct experiments on four open-source tumor segmentation datasets, and with a single click prompt, our model can outperform domain state-of-the-art medical image segmentation models on 3 out of 4 tasks, specifically by 8.25%, 29.87%, and 10.11% for kidney tumor, pancreas tumor, colon cancer segmentation, and achieve similar performance for liver tumor segmentation. We also compare our adaptation method with existing popular adapters, and observed significant performance improvement on most datasets.

cs.CV↗

arXiv2024

Fuzzing Microservices: A Series of User Studies in Industry on Industrial Systems with EvoMaster

With several microservice architectures comprising of thousands of web services, used to serve 630 million customers, companies like Meituan face several challenges in the verification and validation of their software. This paper reports on our experience of integrating EvoMaster (a search-based white-box fuzzer) in the testing processes at Meituan over almost 2 years. Two user studies were carried out in 2021 and in 2023 to evaluate two versions of EvoMaster, respectively, in tackling the test generation for industrial web services which are parts of a large e-commerce microservice system. The two user studies involve in total 321,131 lines of code from five APIs and 27 industrial participants at Meituan. Questionnaires and interviews were carried out in both user studies with employees at Meituan. The two user studies demonstrate clear advantages of EvoMaster (i.e., code coverage and fault detection) and the urgent need to have such a fuzzer in industrial microservices testing. To study how these results could generalize, a follow up user study was done in 2024 with five engineers in the five different companies. Our results show that, besides their clear usefulness, there are still many critical challenges that the research community needs to investigate to improve performance further.

cs.SE↗

arXiv2024

The Electrical Design of a Membrane Antenna for Lunar-based Low-frequency Radio Telescope

Detecting primordial fluctuations from the cosmic dark ages requires extremely large low-frequency radio telescope arrays deployed on the far side of the Moon. The antenna of such an array must be lightweight, easily storable and transportable, deployable on a large scale, durable, and capable of good electrical performance. A membrane antenna is an excellent candidate to meet these criteria. We study the design of a low-frequency membrane antenna for a lunar-based low-frequency (<30 MHz) radio telescope constructed from polyimide film widely used in aerospace applications, owing to its excellent dielectric properties and high stability as a substrate material. We first design and optimize an antenna in free space through dipole deformation and coupling principles, then simulate an antenna on the lunar surface with a simple lunar soil model, yielding an efficiency greater than 90% in the range of 12-19 MHz and greater than 10% in the range of 5-35 MHz. The antenna inherits the omni-directional radiation pattern of a simple dipole antenna in the 5-30 MHz frequency band, giving a large field of view and allowing detection of the 21 cm global signal when used alone. A demonstration prototype is constructed, and its measured electrical property is found to be consistent with simulated results using |S11| measurements. This membrane antenna can potentially fulfill the requirements of a lunar low-frequency array, establishing a solid technical foundation for future large-scale arrays for exploring the cosmic dark ages.

astro-ph.IM↗

arXiv2024

ViMo: Generating Motions from Casual Videos

Although humans have the innate ability to imagine multiple possible actions from videos, it remains an extraordinary challenge for computers due to the intricate camera movements and montages. Most existing motion generation methods predominantly rely on manually collected motion datasets, usually tediously sourced from motion capture (Mocap) systems or Multi-View cameras, unavoidably resulting in a limited size that severely undermines their generalizability. Inspired by recent advance of diffusion models, we probe a simple and effective way to capture motions from videos and propose a novel Video-to-Motion-Generation framework (ViMo) which could leverage the immense trove of untapped video content to produce abundant and diverse 3D human motions. Distinct from prior work, our videos could be more causal, including complicated camera movements and occlusions. Striking experimental results demonstrate the proposed model could generate natural motions even for videos where rapid movements, varying perspectives, or frequent occlusions might exist. We also show this work could enable three important downstream applications, such as generating dancing motions according to arbitrary music and source video style. Extensive experimental results prove that our model offers an effective and scalable way to generate diversity and realistic motions. Code and demos will be public soon.

cs.CV↗

arXiv2024

EndoGS: Deformable Endoscopic Tissues Reconstruction with Gaussian Splatting

Surgical 3D reconstruction is a critical area of research in robotic surgery, with recent works adopting variants of dynamic radiance fields to achieve success in 3D reconstruction of deformable tissues from single-viewpoint videos. However, these methods often suffer from time-consuming optimization or inferior quality, limiting their adoption in downstream tasks. Inspired by 3D Gaussian Splatting, a recent trending 3D representation, we present EndoGS, applying Gaussian Splatting for deformable endoscopic tissue reconstruction. Specifically, our approach incorporates deformation fields to handle dynamic scenes, depth-guided supervision with spatial-temporal weight masks to optimize 3D targets with tool occlusion from a single viewpoint, and surface-aligned regularization terms to capture the much better geometry. As a result, EndoGS reconstructs and renders high-quality deformable endoscopic tissues from a single-viewpoint video, estimated depth maps, and labeled tool masks. Experiments on DaVinci robotic surgery videos demonstrate that EndoGS achieves superior rendering quality. Code is available at https://github.com/HKU-MedAI/EndoGS.

cs.CV↗

arXiv2024

Design and Implementation of a Scalable Correlator Based on ROACH2+GPU Cluster for Tianlai 96-Dual-Polarization Antenna Array

The digital correlator is one of the most crucial data processing components of a radio telescope array. With the scale of radio interferometeric array growing, many efforts have been devoted to developing a cost-effective and scalable correlator in the field of radio astronomy. In this paper, a 192-input digital correlator with six CASPER ROACH2 boards and seven GPU servers has been deployed as the digital signal processing system for Tianlai cylinder pathfinder located in Hongliuxia observatory. The correlator consists of 192 input signals (96 dual-polarization), 125-MHz bandwidth, and full-Stokes output. The correlator inherits the advantages of the CASPER system, for example, low cost, high performance, modular scalability, and a heterogeneous computing architecture. With a rapidly deployable ROACH2 digital sampling system, a commercially expandable 10 Gigabit switching network system, and a flexible upgradable GPU computing system, the correlator forms a low-cost and easily-upgradable system, poised to support scalable large-scale interferometeric array in the future.

astro-ph.IM↗

arXiv2024

Holistic-Motion2D: Scalable Whole-body Human Motion Generation in 2D Space

In this paper, we introduce a novel path to $\textit{general}$ human motion generation by focusing on 2D space. Traditional methods have primarily generated human motions in 3D, which, while detailed and realistic, are often limited by the scope of available 3D motion data in terms of both the size and the diversity. To address these limitations, we exploit extensive availability of 2D motion data. We present $\textbf{Holistic-Motion2D}$, the first comprehensive and large-scale benchmark for 2D whole-body motion generation, which includes over 1M in-the-wild motion sequences, each paired with high-quality whole-body/partial pose annotations and textual descriptions. Notably, Holistic-Motion2D is ten times larger than the previously largest 3D motion dataset. We also introduce a baseline method, featuring innovative $\textit{whole-body part-aware attention}$ and $\textit{confidence-aware modeling}$ techniques, tailored for 2D $\underline{\text T}$ext-driv$\underline{\text{EN}}$ whole-bo$\underline{\text D}$y motion gen$\underline{\text{ER}}$ation, namely $\textbf{Tender}$. Extensive experiments demonstrate the effectiveness of $\textbf{Holistic-Motion2D}$ and $\textbf{Tender}$ in generating expressive, diverse, and realistic human motions. We also highlight the utility of 2D motion for various downstream applications and its potential for lifting to 3D motion. The page link is: https://holistic-motion2d.github.io.

cs.CV↗

arXiv2024

Integrating Domain Knowledge for handling Limited Data in Offline RL

With the ability to learn from static datasets, Offline Reinforcement Learning (RL) emerges as a compelling avenue for real-world applications. However, state-of-the-art offline RL algorithms perform sub-optimally when confronted with limited data confined to specific regions within the state space. The performance degradation is attributed to the inability of offline RL algorithms to learn appropriate actions for rare or unseen observations. This paper proposes a novel domain knowledge-based regularization technique and adaptively refines the initial domain knowledge to considerably boost performance in limited data with partially omitted states. The key insight is that the regularization term mitigates erroneous actions for sparse samples and unobserved states covered by domain knowledge. Empirical evaluations on standard discrete environment datasets demonstrate a substantial average performance increase of at least 27% compared to existing offline RL algorithms operating on limited data.

cs.LG↗

arXiv2024

Augmenting Offline RL with Unlabeled Data

Recent advancements in offline Reinforcement Learning (Offline RL) have led to an increased focus on methods based on conservative policy updates to address the Out-of-Distribution (OOD) issue. These methods typically involve adding behavior regularization or modifying the critic learning objective, focusing primarily on states or actions with substantial dataset support. However, we challenge this prevailing notion by asserting that the absence of an action or state from a dataset does not necessarily imply its suboptimality. In this paper, we propose a novel approach to tackle the OOD problem. We introduce an offline RL teacher-student framework, complemented by a policy similarity measure. This framework enables the student policy to gain insights not only from the offline RL dataset but also from the knowledge transferred by a teacher policy. The teacher policy is trained using another dataset consisting of state-action pairs, which can be viewed as practical domain knowledge acquired without direct interaction with the environment. We believe this additional knowledge is key to effectively solving the OOD issue. This research represents a significant advancement in integrating a teacher-student network into the actor-critic framework, opening new avenues for studies on knowledge transfer in offline RL and effectively addressing the OOD challenge.

cs.AI↗

arXiv2024

$\text{Di}^2\text{Pose}$: Discrete Diffusion Model for Occluded 3D Human Pose Estimation

Continuous diffusion models have demonstrated their effectiveness in addressing the inherent uncertainty and indeterminacy in monocular 3D human pose estimation (HPE). Despite their strengths, the need for large search spaces and the corresponding demand for substantial training data make these models prone to generating biomechanically unrealistic poses. This challenge is particularly noticeable in occlusion scenarios, where the complexity of inferring 3D structures from 2D images intensifies. In response to these limitations, we introduce the Discrete Diffusion Pose ($\text{Di}^2\text{Pose}$), a novel framework designed for occluded 3D HPE that capitalizes on the benefits of a discrete diffusion model. Specifically, $\text{Di}^2\text{Pose}$ employs a two-stage process: it first converts 3D poses into a discrete representation through a \emph{pose quantization step}, which is subsequently modeled in latent space through a \emph{discrete diffusion process}. This methodological innovation restrictively confines the search space towards physically viable configurations and enhances the model's capability to comprehend how occlusions affect human pose within the latent space. Extensive evaluations conducted on various benchmarks (e.g., Human3.6M, 3DPW, and 3DPW-Occ) have demonstrated its effectiveness.

cs.CV↗

arXiv2024

Towards enhancing quantum expectation estimation of matrices through partial Pauli decomposition techniques and post-processing

We introduce an approach for estimating the expectation values of arbitrary $n$-qubit matrices $M \in \mathbb{C}^{2^n\times 2^n}$ on a quantum computer. In contrast to conventional methods like the Pauli decomposition that utilize $4^n$ distinct quantum circuits for this task, our technique employs at most $2^n$ unique circuits, with even fewer required for matrices with limited bandwidth. Termed the \textit{partial Pauli decomposition}, our method involves observables formed as the Kronecker product of a single-qubit Pauli operator and orthogonal projections onto the computational basis. By measuring each such observable, one can simultaneously glean information about $2^n$ distinct entries of $M$ through post-processing of the measurement counts. This reduction in quantum resources is especially crucial in the current noisy intermediate-scale quantum era, offering the potential to accelerate quantum algorithms that rely heavily on expectation estimation, such as the variational quantum eigensolver and the quantum approximate optimization algorithm.

quant-ph↗