arXiv subjects

Junyi Zhu

Publications and source records attributed to Junyi Zhu.

At least 37 records · Page 2Linked to original sources

arXiv2025

WiReSens Toolkit: An Open-source Platform towards Accessible Wireless Tactile Sensing

Past research has widely explored the design and fabrication of resistive matrix-based tactile sensors as a means of creating touch-sensitive devices. However, developing portable, adaptive, and long-lasting tactile sensing systems that incorporate these sensors remains challenging for individuals having limited prior experience with them. To address this, we developed the WiReSens Toolkit, an open-source platform for accessible wireless tactile sensing. Central to our approach is adaptive hardware for interfacing with resistive sensors and a web-based GUI that mediates access to complex functionalities for developing scalable tactile sensing systems, including 1) multi-device programming and wireless visualization across three distinct communication protocols 2) autocalibration methods for adaptive sensitivity and 3) intermittent data transmission for low-power operation. We validated the toolkit's usability through a user study with 11 novice participants, who, on average, successfully configured a tactile sensor with over 95\% accuracy in under five minutes, calibrated sensors 10x faster than baseline methods, and demonstrated enhanced tactile data sense-making.

cs.HC↗

arXiv2025

Linear Combination of Saved Checkpoints Makes Consistency and Diffusion Models Better

Diffusion Models (DM) and Consistency Models (CM) are two types of popular generative models with good generation quality on various tasks. When training DM and CM, intermediate weight checkpoints are not fully utilized and only the last converged checkpoint is used. In this work, we find that high-quality model weights often lie in a basin which cannot be reached by SGD but can be obtained by proper checkpoint averaging. Based on these observations, we propose LCSC, a simple but effective and efficient method to enhance the performance of DM and CM, by combining checkpoints along the training trajectory with coefficients deduced from evolutionary search. We demonstrate the value of LCSC through two use cases: $\textbf{(a) Reducing training cost.}$ With LCSC, we only need to train DM/CM with fewer number of iterations and/or lower batch sizes to obtain comparable sample quality with the fully trained model. For example, LCSC achieves considerable training speedups for CM (23$\times$ on CIFAR-10 and 15$\times$ on ImageNet-64). $\textbf{(b) Enhancing pre-trained models.}$ Assuming full training is already done, LCSC can further improve the generation quality or speed of the final converged models. For example, LCSC achieves better performance using 1 number of function evaluation (NFE) than the base model with 2 NFE on consistency distillation, and decreases the NFE of DM from 15 to 9 while maintaining the generation quality on CIFAR-10. Our code is available at https://github.com/imagination-research/LCSC.

cs.CV↗

arXiv2025

Existence of global solutions to the massive Thirring model in the non-laboratory coordinates

The massive Thirring model in the non-laboratory coordinates is considered by the Riemann-Hilbert approach. Existence of global solutions is shown for the cases of the associated Riemann-Hilbert problem without eigenvalues or resonances. The Lipschitz continuity of the map from the potential $v_0(x)\in H^2(\mathbb{R})\cap H^{1,1}(\mathbb{R})$ to the scattering data is given in the direct scattering transform. Two transform matrices are introduced to curb the convergence of the Volterra integral equations and the relevant estimates of the modified Jost functions. For small potential, the solvability of the Riemann-Hilbert problems without eigenvalues or resonances is discussed. The Lipschitz continuity of the map from the scattering data to the potential $v(x)$ is shown. The reconstructions for potential $u(x,t)$ and $v(x,t)$ are finished by considering the time dependence of the scattering data and by constructing the conservation laws obtain via the dressing method.

math-ph↗

arXiv2024

Implicit Neural Representations for Robust Joint Sparse-View CT Reconstruction

Computed Tomography (CT) is pivotal in industrial quality control and medical diagnostics. Sparse-view CT, offering reduced ionizing radiation, faces challenges due to its under-sampled nature, leading to ill-posed reconstruction problems. Recent advancements in Implicit Neural Representations (INRs) have shown promise in addressing sparse-view CT reconstruction. Recognizing that CT often involves scanning similar subjects, we propose a novel approach to improve reconstruction quality through joint reconstruction of multiple objects using INRs. This approach can potentially utilize the advantages of INRs and the common patterns observed across different objects. While current INR joint reconstruction techniques primarily focus on speeding up the learning process, they are not specifically tailored to enhance the final reconstruction quality. To address this gap, we introduce a novel INR-based Bayesian framework integrating latent variables to capture the common patterns across multiple objects under joint reconstruction. The common patterns then assist in the reconstruction of each object via latent variables, thereby improving the individual reconstruction. Extensive experiments demonstrate that our method achieves higher reconstruction quality with sparse views and remains robust to noise in the measurements as indicated by common numerical metrics. The obtained latent variables can also serve as network initialization for the new object and speed up the learning process.

cs.CV↗

arXiv2024

FastMem: Fast Memorization of Prompt Improves Context Awareness of Large Language Models

Large language models (LLMs) excel in generating coherent text, but they often struggle with context awareness, leading to inaccuracies in tasks requiring faithful adherence to provided information. We introduce FastMem, a novel method designed to enhance instruction fine-tuned LLMs' context awareness through fast memorization of the prompt. FastMem maximizes the likelihood of the prompt before inference by updating only the last Feed-Forward Network (FFN) module. This targeted approach ensures efficient optimization without overfitting, significantly improving the model's ability to comprehend and accurately follow the context. Our experiments demonstrate substantial gains in reading comprehension, text summarization and adherence to output structures. For instance, FastMem improves the accuracy of Llama 3-8B-Inst on the NQ-SWAP dataset from 59.1% to 71.6%, and reduces the output structure failure rate of Qwen 1.5-4B-Chat from 34.9% to 25.5%. Extensive experimental results highlight FastMem's potential to offer a robust solution to enhance the reliability and accuracy of LLMs in various applications. Our code is available at: https://github.com/IAAR-Shanghai/FastMem

cs.CL↗

arXiv2024

Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costs

The rapid advancement of large language models (LLMs) has led to architectures with billions to trillions of parameters, posing significant deployment challenges due to their substantial demands on memory, processing power, and energy consumption. Sparse Mixture-of-Experts (SMoE) architectures have emerged as a solution, activating only a subset of parameters per token, thereby achieving faster inference while maintaining performance. However, SMoE models still face limitations in broader deployment due to their large parameter counts and significant GPU memory requirements. In this work, we introduce a gradient-free evolutionary strategy named EEP (Efficient Expert P}runing) to enhance the pruning of experts in SMoE models. EEP relies solely on model inference (i.e., no gradient computation) and achieves greater sparsity while maintaining or even improving performance on downstream tasks. EEP can be used to reduce both the total number of experts (thus saving GPU memory) and the number of active experts (thus accelerating inference). For example, we demonstrate that pruning up to 75% of experts in Mixtral $8\times7$B-Instruct results in a substantial reduction in parameters with minimal performance loss. Remarkably, we observe improved performance on certain tasks, such as a significant increase in accuracy on the SQuAD dataset (from 53.4% to 75.4%), when pruning half of the experts. With these results, EEP not only lowers the barrier to deploying SMoE models,but also challenges the conventional understanding of model pruning by showing that fewer experts can lead to better task-specific performance without any fine-tuning. Code is available at https://github.com/imagination-research/EEP.

cs.LG↗

arXiv2024

Emergent Moiré fringes in direct-grown quasicrystal

Quasicrystals represent a category of rarely structured solids that challenge traditional periodicity in crystal materials. Recent advancements in the synthesis of two-dimensional (2D) van der Waals materials have paved the way for exploring the unique physical properties of these systems. Here, we report on the synthesis of 2D quasicrystals featuring 30° alternating twist angles between multiple graphene layers, using chemical vapor deposition (CVD). Strikingly, we observed periodic Moiré patterns in the quasicrystal, a finding that has not been previously reported in traditional alloy-based quasicrystals. The Moiré periodicity, varying with the parity of the constituent layers, aligns with the theoretical predictions that suggest a stress cancellation mechanism in force. The emergence of Moiré fringes is attributed to the spontaneous mismatched lattice constant in the oriented graphene layers, proving the existence of atomic relaxation. This phenomenon, which has been largely understudied in graphene systems with large twist angles, has now been validated through our use of scanning transmission electron microscopy (STEM). Our CVD-grown Moiré quasicrystal provides an ideal platform for exploring the unusual physical properties that arise from Moiré periodicity within quasicrystals.

cond-mat.mtrl-sci↗

arXiv2023

MVHumanNet: A Large-scale Dataset of Multi-view Daily Dressing Human Captures

In this era, the success of large language models and text-to-image models can be attributed to the driving force of large-scale datasets. However, in the realm of 3D vision, while remarkable progress has been made with models trained on large-scale synthetic and real-captured object data like Objaverse and MVImgNet, a similar level of progress has not been observed in the domain of human-centric tasks partially due to the lack of a large-scale human dataset. Existing datasets of high-fidelity 3D human capture continue to be mid-sized due to the significant challenges in acquiring large-scale high-quality 3D human data. To bridge this gap, we present MVHumanNet, a dataset that comprises multi-view human action sequences of 4,500 human identities. The primary focus of our work is on collecting human data that features a large number of diverse identities and everyday clothing using a multi-view human capture system, which facilitates easily scalable data collection. Our dataset contains 9,000 daily outfits, 60,000 motion sequences and 645 million frames with extensive annotations, including human masks, camera parameters, 2D and 3D keypoints, SMPL/SMPLX parameters, and corresponding textual descriptions. To explore the potential of MVHumanNet in various 2D and 3D visual tasks, we conducted pilot studies on view-consistent action recognition, human NeRF reconstruction, text-driven view-unconstrained human image generation, as well as 2D view-unconstrained human image and 3D avatar generation. Extensive experiments demonstrate the performance improvements and effective applications enabled by the scale provided by MVHumanNet. As the current largest-scale 3D human dataset, we hope that the release of MVHumanNet data with annotations will foster further innovations in the domain of 3D human-centric tasks at scale.

cs.CV↗

arXiv2023

Improving Differentially Private SGD via Randomly Sparsified Gradients

Differentially private stochastic gradient descent (DP-SGD) has been widely adopted in deep learning to provide rigorously defined privacy, which requires gradient clipping to bound the maximum norm of individual gradients and additive isotropic Gaussian noise. With analysis of the convergence rate of DP-SGD in a non-convex setting, we identify that randomly sparsifying gradients before clipping and noisification adjusts a trade-off between internal components of the convergence bound and leads to a smaller upper bound when the noise is dominant. Additionally, our theoretical analysis and empirical evaluations show that the trade-off is not trivial but possibly a unique property of DP-SGD, as either canceling noisification or gradient clipping eliminates the trade-off in the bound. This observation is indicative, as it implies DP-SGD has special inherent room for (even simply random) gradient compression. To verify the observation and utilize it, we propose an efficient and lightweight extension using random sparsification (RS) to strengthen DP-SGD. Experiments with various DP-SGD frameworks show that RS can improve performance. Additionally, the produced sparse gradients of RS exhibit advantages in reducing communication cost and strengthening privacy against reconstruction attacks, which are also key problems in private machine learning.

cs.LG↗

arXiv2023

Surrogate Model Extension (SME): A Fast and Accurate Weight Update Attack on Federated Learning

In Federated Learning (FL) and many other distributed training frameworks, collaborators can hold their private data locally and only share the network weights trained with the local data after multiple iterations. Gradient inversion is a family of privacy attacks that recovers data from its generated gradients. Seemingly, FL can provide a degree of protection against gradient inversion attacks on weight updates, since the gradient of a single step is concealed by the accumulation of gradients over multiple local iterations. In this work, we propose a principled way to extend gradient inversion attacks to weight updates in FL, thereby better exposing weaknesses in the presumed privacy protection inherent in FL. In particular, we propose a surrogate model method based on the characteristic of two-dimensional gradient flow and low-rank property of local updates. Our method largely boosts the ability of gradient inversion attacks on weight updates containing many iterations and achieves state-of-the-art (SOTA) performance. Additionally, our method runs up to $100\times$ faster than the SOTA baseline in the common FL scenario. Our work re-evaluates and highlights the privacy risk of sharing network weights. Our code is available at https://github.com/JunyiZhu-AI/surrogate_model_extension.

cs.LG↗

arXiv2023

Confidence-aware Personalized Federated Learning via Variational Expectation Maximization

Federated Learning (FL) is a distributed learning scheme to train a shared model across clients. One common and fundamental challenge in FL is that the sets of data across clients could be non-identically distributed and have different sizes. Personalized Federated Learning (PFL) attempts to solve this challenge via locally adapted models. In this work, we present a novel framework for PFL based on hierarchical Bayesian modeling and variational inference. A global model is introduced as a latent variable to augment the joint distribution of clients' parameters and capture the common trends of different clients, optimization is derived based on the principle of maximizing the marginal likelihood and conducted using variational expectation maximization. Our algorithm gives rise to a closed-form estimation of a confidence value which comprises the uncertainty of clients' parameters and local model deviations from the global model. The confidence value is used to weigh clients' parameters in the aggregation stage and adjust the regularization effect of the global model. We evaluate our method through extensive empirical studies on multiple datasets. Experimental results show that our approach obtains competitive results under mild heterogeneous circumstances while significantly outperforming state-of-the-art PFL frameworks in highly heterogeneous settings. Our code is available at https://github.com/JunyiZhu-AI/confidence_aware_PFL.

cs.LG↗

arXiv2021

Dbar dressing method to nonlinear Schrödinger equation with nonzero boundary conditions

The Dbar dressing method is extended to study the focusing/defocusing nonlinear Schrödinger (NLS) equation with nonzero boundary condition. A special type of complex function is considered. The function is meromorphic outside an annulus with center 0 and satisfies a local Dbar problem inside the annulus. The theory of such function is extended to construct the Lax pair of the NLS equation with nonzero boundary condition. In this procedure, the relation between the NLS potential and the solution of the Dbar problem is established. A certain distribution for the Dbar problem is introduced to obtain the focusing/defocusing NLS equation and the conservation laws. The explicit solutions of the focusing/defocusing NLS equation with nonzero boundary condition are given from special distributions.

nlin.SI↗

arXiv2021

Dbar-approach to coupled nonlocal NLS equation and general nonlocal Reduction

The coupled nonlocal NLS equation is studied by virtue of the $2\times2$ Dbar-problem. Two spectral transform matrices are introduced to define two associated Dbar-problems. The relations between the coupled nonlocal NLS potential and the solution of the Dbar-problem are constructed. The spatial transform method is extended to obtain the coupled nonlocal NLS equation and its conservation laws. The general nonlocal reduction of the coupled nonlocal NLS equation to the nonlocal NLS equation is discussed in detail. The explicit solutions are derived.

nlin.SI↗

arXiv2021

R-GAP: Recursive Gradient Attack on Privacy

Federated learning frameworks have been regarded as a promising approach to break the dilemma between demands on privacy and the promise of learning from large collections of distributed data. Many such frameworks only ask collaborators to share their local update of a common model, i.e. gradients with respect to locally stored data, instead of exposing their raw data to other collaborators. However, recent optimization-based gradient attacks show that raw data can often be accurately recovered from gradients. It has been shown that minimizing the Euclidean distance between true gradients and those calculated from estimated data is often effective in fully recovering private data. However, there is a fundamental lack of theoretical understanding of how and when gradients can lead to unique recovery of original data. Our research fills this gap by providing a closed-form recursive procedure to recover data from gradients in deep neural networks. We name it Recursive Gradient Attack on Privacy (R-GAP). Experimental results demonstrate that R-GAP works as well as or even better than optimization-based approaches at a fraction of the computation under certain conditions. Additionally, we propose a Rank Analysis method, which can be used to estimate the risk of gradient attacks inherent in certain network architectures, regardless of whether an optimization-based or closed-form-recursive attack is used. Experimental results demonstrate the utility of the rank analysis towards improving the network's security. Source code is available for download from https://github.com/JunyiZhu-AI/R-GAP.

cs.LG↗

arXiv2020

Inverse scattering transform for the Tzitzéica equation

The inverse scattering transform is extended to investigate the Tzitzéica equation. A set of sectionally analytic eigenfunctions and auxiliary eigenfunctions are introduced. We note that in this procedure, the auxiliary eigenfunctions play an important role. Besides, the symmetries of the analytic eigenfunctions and scattering data are discussed. The asymptotic behaviors of the Jost eigenfunctions are derived systematically. A Riemann-Hilbert problem is constructed to study the inverse scattering problem. Lastly, some novel exact solutions are obtained for reflectionless potentials.

nlin.SI↗

arXiv2020

Kinetic Processes and surfactant design of Group I elements on CZTS (1-1-2-) surface

Cu2ZnSnS4 (CZTS) is a promising thin-film solar-cell material consisted of earth abundant and nontoxic elements. Yet, there exists a fundamental bottle neck that hinders the performance of the device due to complexed intrinsic defects properties and detrimental secondary phases. Recently, it was proven experimentally that Na and K in co-evaporation growth of CZTS can enlarge the grain size and suppress formation of ZnS secondary phase near surface, but the reasons are not well understood. We used first principle calculations to investigate the kinetic processes on CZTS (1-1-2-) surface involving Group I elements, including Na, K, and Cs, to demonstrate their surfactant effects. Both the structure of the reconstructed surfaces involving Group I elements and various diffusion paths of a Zn ad-atom in these reconstructed surfaces were explored. The advantages and concerns of the surfactant effects of Na, K, Cs, were systematically compared and discussed. Although Group I elements protect Cu sites on the subsurface layer, a disordered metastable configuration with a diffusion barrier of about 400meV was found. Therefore, a precise control of growth condition is essential to avoid the metastable phase. In addition, our studies provide a systematical design principle for surfactant effects during the growth.

cond-mat.mtrl-sci↗

arXiv2019

Carrier free long-range magnetism in Mo doped one quintuple layer Bi2Te3 and Sb2Te3

One of the keys to the realization of Quantum Anomalous Hall effect (QAHE) is long range ferromagnetism, which is only experimentally realized in Cr or V doped (Bi,Sb)2Te3 system. Both elements are 3d transition metals and 4d transition metals are found to be ineffective to produce long range ferromagnetism in Bi2Se3. Still, whether long range ferromagnetism can be realized by magnetic doping of 4d elements is an open question. Based on density functional theory calculations, we predict that long range ferromagnetism can be realized in Mo doped Bi2Te3 and Sb2Te3, which are semiconducting. The coupling strength is comparable with that of Cr doped Bi2Te3 and Sb2Te3. Therefore, Mo doped Bi2Te3 and Sb2Te3 or their alloys can be new systems to realize diluted magnetic semiconductors and QAHE.

cond-mat.mtrl-sci↗

arXiv2019

Localization in Aerial Imagery with Grid Maps using LocGAN

In this work, we present LocGAN, our localization approach based on a geo-referenced aerial imagery and LiDAR grid maps. Currently, most self-localization approaches relate the current sensor observations to a map generated from previously acquired data. Unfortunately, this data is not always available and the generated maps are usually sensor setup specific. Global Navigation Satellite Systems (GNSS) can overcome this problem. However, they are not always reliable especially in urban areas due to multi-path and shadowing effects. Since aerial imagery is usually available, we can use it as prior information. To match aerial images with grid maps, we use conditional Generative Adversarial Networks (cGANs) which transform aerial images to the grid map domain. The transformation between the predicted and measured grid map is estimated using a localization network (LocNet). Given the geo-referenced aerial image transformation the vehicle pose can be estimated. Evaluations performed on the data recorded in region Karlsruhe, Germany show that our LocGAN approach provides reliable global localization results.

cs.CV↗