arXiv subjects

Tao Li

Publications and source records attributed to Tao Li.

At least 307 records · Page 17Linked to original sources

arXiv2024

MM-TTS: Multi-modal Prompt based Style Transfer for Expressive Text-to-Speech Synthesis

The style transfer task in Text-to-Speech refers to the process of transferring style information into text content to generate corresponding speech with a specific style. However, most existing style transfer approaches are either based on fixed emotional labels or reference speech clips, which cannot achieve flexible style transfer. Recently, some methods have adopted text descriptions to guide style transfer. In this paper, we propose a more flexible multi-modal and style controllable TTS framework named MM-TTS. It can utilize any modality as the prompt in unified multi-modal prompt space, including reference speech, emotional facial images, and text descriptions, to control the style of the generated speech in a system. The challenges of modeling such a multi-modal style controllable TTS mainly lie in two aspects:1)aligning the multi-modal information into a unified style space to enable the input of arbitrary modality as the style prompt in a single system, and 2)efficiently transferring the unified style representation into the given text content, thereby empowering the ability to generate prompt style-related voice. To address these problems, we propose an aligned multi-modal prompt encoder that embeds different modalities into a unified style space, supporting style transfer for different modalities. Additionally, we present a new adaptive style transfer method named Style Adaptive Convolutions to achieve a better style representation. Furthermore, we design a Rectified Flow based Refiner to solve the problem of over-smoothing Mel-spectrogram and generate audio of higher fidelity. Since there is no public dataset for multi-modal TTS, we construct a dataset named MEAD-TTS, which is related to the field of expressive talking head. Our experiments on the MEAD-TTS dataset and out-of-domain datasets demonstrate that MM-TTS can achieve satisfactory results based on multi-modal prompts.

eess.AS↗

arXiv2024

Towards Robust Neural Networks via Orthogonal Diversity

Deep Neural Networks (DNNs) are vulnerable to invisible perturbations on the images generated by adversarial attacks, which raises researches on the adversarial robustness of DNNs. A series of methods represented by the adversarial training and its variants have proven as one of the most effective techniques in enhancing the DNN robustness. Generally, adversarial training focuses on enriching the training data by involving perturbed data. Such data augmentation effect of the involved perturbed data in adversarial training does not contribute to the robustness of DNN itself and usually suffers from clean accuracy drop. Towards the robustness of DNN itself, we in this paper propose a novel defense that aims at augmenting the model in order to learn features that are adaptive to diverse inputs, including adversarial examples. More specifically, to augment the model, multiple paths are embedded into the network, and an orthogonality constraint is imposed on these paths to guarantee the diversity among them. A margin-maximization loss is then designed to further boost such DIversity via Orthogonality (DIO). In this way, the proposed DIO augments the model and enhances the robustness of DNN itself as the learned features can be corrected by these mutually-orthogonal paths. Extensive empirical results on various data sets, structures and attacks verify the stronger adversarial robustness of the proposed DIO utilizing model augmentation. Besides, DIO can also be flexibly combined with different data augmentation techniques (e.g., TRADES and DDPM), further promoting robustness gains.

cs.CV↗

arXiv2024

Single-Shot Readout of a Nuclear Spin in Silicon Carbide

Solid-state qubits with a photonic interface is very promising for quantum networks. Color centers in silicon carbide have shown excellent optical and spin coherence, even when integrated with membranes and nano-structures. Additionally, nuclear spins coupled with electron spins can serve as long-lived quantum memories. Pioneering work in previous has realized the initialization of a single nuclear spin and demonstrated its entanglement with an electron spin. In this paper, we report the first realization of single-shot readout for a nuclear spin in SiC. We obtain a deterministic readout fidelity of 98.2% with a measurement duration of 1.13 ms. With a dual-step readout scheme, we obtain a readout fidelity as high as 99.5% with a success efficiency of 89.8%. Our work complements the experimental toolbox of harnessing both electron and nuclear spins in SiC for future quantum networks.

quant-ph↗

arXiv2023

The optical conductivity of the 2D $t-J$ model and the origin of electron incoherence in the high-T$_{c}$ cuprate superconductors: a variational study

Understanding the origin of electron incoherence is the first step toward a theoretical description of the non-Fermi liquid behavior of the high-T$_{c}$ cuprate superconductors. Such electron incoherence manifests itself most evidently in the non-Drude behavior of the optical response of the system and the anomalous density fluctuation behavior in the long wave length limit. The spectral weight transfer related to such dissipative response, which is absent in conventional Fermi liquid metal, has direct consequence on the dc transport property of the system in the normal state and the superfluid stiffness in the superconducting state. It is found that such electron incoherence remains significant even in the clean limit and at low temperature and thus must be attributed to the strong electron correlation effect in the cuprate superconductors. Here we study such an intrinsic effect in the 2D $t-J$ model through the variational calculation of its optical conductivity $σ(ω)$. We assume a resonating valence bond ground state as our starting point and find that a significant portion of the total optical spectral weight remains incoherent throughout the phase diagram. The optical absorption is found to extend all the way to an energy of the order of the bare band width. We find that both the total optical weight $\bar{K}$ and the integrated incoherent optical weight $I$ increase monotonically with doping, with their ratio $R_{incoh}=I/\bar{K}$ decreasing monotonically with doping. Our results indicate that the majority part of electron incoherence in the 2D $t-J$ model can be attributed to the electron fractionalization mechanism assumed in such a treatment. We also find that the Drude weight deduced from $D=\bar{K}-I$ scales linearly with hole doping, without any sign of a non-monotonic behavior in the overdoped regime.

cond-mat.str-el↗

arXiv2023

Towards Generalized Multi-stage Clustering: Multi-view Self-distillation

Existing multi-stage clustering methods independently learn the salient features from multiple views and then perform the clustering task. Particularly, multi-view clustering (MVC) has attracted a lot of attention in multi-view or multi-modal scenarios. MVC aims at exploring common semantics and pseudo-labels from multiple views and clustering in a self-supervised manner. However, limited by noisy data and inadequate feature learning, such a clustering paradigm generates overconfident pseudo-labels that mis-guide the model to produce inaccurate predictions. Therefore, it is desirable to have a method that can correct this pseudo-label mistraction in multi-stage clustering to avoid the bias accumulation. To alleviate the effect of overconfident pseudo-labels and improve the generalization ability of the model, this paper proposes a novel multi-stage deep MVC framework where multi-view self-distillation (DistilMVC) is introduced to distill dark knowledge of label distribution. Specifically, in the feature subspace at different hierarchies, we explore the common semantics of multiple views through contrastive learning and obtain pseudo-labels by maximizing the mutual information between views. Additionally, a teacher network is responsible for distilling pseudo-labels into dark knowledge, supervising the student network and improving its predictive capabilities to enhance the robustness. Extensive experiments on real-world multi-view datasets show that our method has better clustering performance than state-of-the-art methods.

cs.CV↗

arXiv2023

Towards Inductive Robustness: Distilling and Fostering Wave-induced Resonance in Transductive GCNs Against Graph Adversarial Attacks

Graph neural networks (GNNs) have recently been shown to be vulnerable to adversarial attacks, where slight perturbations in the graph structure can lead to erroneous predictions. However, current robust models for defending against such attacks inherit the transductive limitations of graph convolutional networks (GCNs). As a result, they are constrained by fixed structures and do not naturally generalize to unseen nodes. Here, we discover that transductive GCNs inherently possess a distillable robustness, achieved through a wave-induced resonance process. Based on this, we foster this resonance to facilitate inductive and robust learning. Specifically, we first prove that the signal formed by GCN-driven message passing (MP) is equivalent to the edge-based Laplacian wave, where, within a wave system, resonance can naturally emerge between the signal and its transmitting medium. This resonance provides inherent resistance to malicious perturbations inflicted on the signal system. We then prove that merely three MP iterations within GCNs can induce signal resonance between nodes and edges, manifesting as a coupling between nodes and their distillable surrounding local subgraph. Consequently, we present Graph Resonance-fostering Network (GRN) to foster this resonance via learning node representations from their distilled resonating subgraphs. By capturing the edge-transmitted signals within this subgraph and integrating them with the node signal, GRN embeds these combined signals into the central node's representation. This node-wise embedding approach allows for generalization to unseen nodes. We validate our theoretical findings with experiments, and demonstrate that GRN generalizes robustness to unseen nodes, whilst maintaining state-of-the-art classification accuracy on perturbed graphs.

cs.LG↗

arXiv2023

On recognition of the direct squares of the simple groups with abelian Sylow 2-subgroups

The spectrum of a group is the set of orders of its elements. Finite groups with the same spectra as the direct squares of the finite simple groups with abelian Sylow 2-subgroups are considered. It is proved that the direct square $J_1\times J_1$ of the sporadic Janko group $J_1$ and the direct squares ${^2}G_2(q)\times{^2}G_2(q)$ of the simple small Ree groups ${^2}G_2(q)$ are uniquely characterized by their spectra in the class of finite groups, while for the direct square $PSL_2(q)\times PSL_2(q)$ of a 2-dimensional simple linear group $PSL_2(q)$, there are always infinitely many groups (even solvable groups) with the same spectra.

math.GR↗

arXiv2023

On the Price of Transparency: A Comparison between Overt Persuasion and Covert Signaling

Transparency of information disclosure has always been considered an instrumental component of effective governance, accountability, and ethical behavior in any organization or system. However, a natural question follows: \emph{what is the cost or benefit of being transparent}, as one may suspect that transparency imposes additional constraints on the information structure, decreasing the maneuverability of the information provider. This work proposes and quantitatively investigates the \emph{price of transparency} (PoT) in strategic information disclosure by comparing the perfect Bayesian equilibrium payoffs under two representative information structures: overt persuasion and covert signaling models. PoT is defined as the ratio between the payoff outcomes in covert and overt interactions. As the main contribution, this work develops a bilevel-bilinear programming approach, called $Z$-programming, to solve for non-degenerate perfect Bayesian equilibria of dynamic incomplete information games with finite states and actions. Using $Z$-programming, we show that it is always in the information provider's interest to choose the transparent information structure, as $0\leq \textrm{PoT}\leq 1$. The upper bound is attainable for any strictly Bayesian-posterior competitive games, of which zero-sum games are a particular case. For continuous games, the PoT, still upper-bounded by $1$, can be arbitrarily close to $0$, indicating the tightness of the lower bound. This tight lower bound suggests that the lack of transparency can result in significant loss for the provider. We corroborate our findings using quadratic games and numerical examples.

cs.GT↗

arXiv2023

Experimental Generation of Spin-Photon Entanglement in Silicon Carbide

A solid-state approach for quantum networks is advantages, as it allows the integration of nanophotonics to enhance the photon emission and the utilization of weakly coupled nuclear spins for long-lived storage. Silicon carbide, specifically point defects within it, shows great promise in this regard due to the easy of availability and well-established nanofabrication techniques. Despite of remarkable progresses made, achieving spin-photon entanglement remains a crucial aspect to be realized. In this paper, we experimentally generate entanglement between a silicon vacancy defect in silicon carbide and a scattered single photon in the zero-phonon line. The spin state is measured by detecting photons scattered in the phonon sideband. The photonic qubit is encoded in the time-bin degree-of-freedom and measured using an unbalanced Mach-Zehnder interferometer. Photonic correlations not only reveal the quality of the entanglement but also verify the deterministic nature of the entanglement creation process. By harnessing two pairs of such spin-photon entanglement, it becomes straightforward to entangle remote quantum nodes at long distance.

quant-ph↗

arXiv2023

Instability of the $U(1)$ spin liquid with a large spinon Fermi surface in the Heisenberg-ring exchange model on the triangular lattice

It is widely believed that the $U(1)$ spin liquid with a large spinon Fermi surface(SFS state) can be realized in the spin-$\frac{1}{2}$ $J_{1}-J_{4}$ model on the triangular lattice, when the ring exchange coupling $J_{4}$ is sufficiently strong to suppress the 120 degree magnetic ordered state. This belief is supported by many variational studies on this model and seems to be consistent with the observations on the organic spin liquid materials such as $κ$-(BEDT-TTF)$_{2}$Cu$_{2}$(CN)$_{3}$ and EtMe$_{3}$Sb[Pd(dmit)$_{2}$]$_{2}$, which are systems close to their Mott transition and thus have large $J_{4}$. Here we show through systematic variational search that such a state is never favored in the $J_{1}-J_{4}$ model on the triangular lattice. Instead, a state with broken spatial symmetry is favored in the most probable parameter regime for the SFS state and has an energy much lower than that of the SFS state and other proposed variational states. More specifically, we find that for $J_{4}\ge 0.09J_{1}$, the model favors a valence bond solid state with a $4\times6$ period in its local spin correlation pattern and has a variational energy that is about $5\%$ lower than that of the SFS state. This state is separated from the $π$-flux state for $J_{4}\le 0.045J_{1}$ by an intermediate symmetry breaking phase with a zigzag pattern in its local spin correlation. We find that the variational phase diagram we got is in qualitative agreement with that obtained from exact diagonalization on a $6\times6$ cluster.

cond-mat.str-el↗

arXiv2023

Can Hubbard model resist electric current?

It is claimed by a recent quantum Monte Carlo simulation that the linear-in-temperature DC resistivity observed in the high-$T_{c}$ cuprate superconductors can be reproduced in the pure two dimensional Hubbard model\cite{Huang}. Here we show perturbatively that such a translational invariant electronic model can not support a steady state current in the presence of a uniform electric field at any finite temperature. Instead, the Hubbard model is perfectly conducting in the linear response regime and will undergo Bloch oscillation at finite electric field for any finite temperature. Nevertheless, the quantum Monte Carlo simulation can provide us the key information on the temperature dependence of the Drude weight, a quantity of central importance in the holographic description of the transport properties of the strange metal phase.

cond-mat.str-el↗

arXiv2023

Gigahertz-rate-switchable wavefront shaping through integration of metasurfaces with photonic integrated circuit

Achieving spatiotemporal control of light at high-speeds presents immense possibilities for various applications in communication, computation, metrology, and sensing. The integration of subwavelength metasurfaces and optical waveguides offers a promising approach to manipulate light across multiple degrees of freedom at high-speed in compact photonic integrated circuit (PICs) devices. Here, we demonstrate a gigahertz-rate-switchable wavefront shaping by integrating metasurface, lithium niobite on insulator (LNOI) photonic waveguide and electrodes within a PIC device. As proofs of concept, we showcase the generation of a focus beam with reconfigurable arbitrary polarizations, switchable focusing with lateral focal positions and focal length, orbital angular momentum light beams (OAMs) as well as Bessel beams. Our measurements indicate modulation speeds of up to gigahertz rate. This integrated platform offers a versatile and efficient means of controlling light field at high-speed within a compact system, paving the way for potential applications in optical communication, computation, sensing, and imaging.

physics.optics↗

arXiv2023

PLV-IEKF: Consistent Visual-Inertial Odometry using Points, Lines, and Vanishing Points

In this paper, we propose an Invariant Extended Kalman Filter (IEKF) based Visual-Inertial Odometry (VIO) using multiple features in man-made environments. Conventional EKF-based VIO usually suffers from system inconsistency and angular drift that naturally occurs in feature-based methods. However, in man-made environments, notable structural regularities, such as lines and vanishing points, offer valuable cues for localization. To exploit these structural features effectively and maintain system consistency, we design a right invariant filter-based VIO scheme incorporating point, line, and vanishing point features. We demonstrate that the conventional additive error definition for point features can also preserve system consistency like the invariant error definition by proving a mathematically equivalent measurement model. And a similar conclusion is established for line features. Additionally, we conduct an invariant filter-based observability analysis proving that vanishing point measurement maintains unobservable directions naturally. Both simulation and real-world tests are conducted to validate our methods' pose accuracy and consistency. The experimental results validate the competitive performance of our method, highlighting its ability to deliver accurate and consistent pose estimation in man-made environments.

cs.RO↗

arXiv2023

Non-linear optics at twist interfaces in h-BN/SiC heterostructures

Understanding the emergent electronic structure in twisted atomically thin layers has led to the exciting field of twistronics. However, practical applications of such systems are challenging since the specific angular correlations between the layers must be precisely controlled and the layers have to be single crystalline with uniform atomic ordering. Here, we suggest an alternative, simple and scalable approach where nanocrystalline two-dimensional (2D) film on three-dimensional (3D) substrates yield twisted-interface-dependent properties. Ultrawide-bandgap hexagonal boron nitride (h-BN) thin films are directly grown on high in-plane lattice mismatched wide-bandgap silicon carbide (4H-SiC) substrates to explore the twist-dependent structure-property correlations. Concurrently, nanocrystalline h-BN thin film shows strong non-linear second-harmonic generation and ultra-low cross-plane thermal conductivity at room temperature, which are attributed to the twisted domain edges between van der Waals stacked nanocrystals with random in-plane orientations. First-principles calculations based on time-dependent density functional theory manifest strong even-order optical nonlinearity in twisted h-BN layers. Our work unveils that directly deposited 2D nanocrystalline thin film on 3D substrates could provide easily accessible twist-interfaces, therefore enabling a simple and scalable approach to utilize the 2D-twistronics integrated in 3D material devices for next-generation nanotechnology.

cond-mat.mtrl-sci↗

arXiv2023

Lifting the Veil: Unlocking the Power of Depth in Q-learning

With the help of massive data and rich computational resources, deep Q-learning has been widely used in operations research and management science and has contributed to great success in numerous applications, including recommender systems, supply chains, games, and robotic manipulation. However, the success of deep Q-learning lacks solid theoretical verification and interpretability. The aim of this paper is to theoretically verify the power of depth in deep Q-learning. Within the framework of statistical learning theory, we rigorously prove that deep Q-learning outperforms its traditional version by demonstrating its good generalization error bound. Our results reveal that the main reason for the success of deep Q-learning is the excellent performance of deep neural networks (deep nets) in capturing the special properties of rewards namely, spatial sparseness and piecewise constancy, rather than their large capacities. In this paper, we make fundamental contributions to the field of reinforcement learning by answering to the following three questions: Why does deep Q-learning perform so well? When does deep Q-learning perform better than traditional Q-learning? How many samples are required to achieve a specific prediction accuracy for deep Q-learning? Our theoretical assertions are verified by applying deep Q-learning in the well-known beer game in supply chain management and a simulated recommender system.

cs.LG↗

arXiv2023

Optimization of convolutional neural networks for background suppression in the PandaX-III experiment

The tracks recorded by a gaseous detector provide a possibility for charged particle identification. For searching the neutrinoless double beta decay events of 136Xe in the PandaX-III experiment, we optimized the convolutional neural network based on the Monte Carlo simulation data to improve the signal-background discrimination power. EfficientNet is chosen as the baseline model and the optimization is performed by tuning the hyperparameters. In particular, the maximum discrimination power is achieved by optimizing the channel number of the top convolutional layer. In comparison with our previous work, the significance of discrimination has been improved by about 70%.

physics.ins-det↗

arXiv2023

Bayesian Domain Invariant Learning via Posterior Generalization of Parameter Distributions

Domain invariant learning aims to learn models that extract invariant features over various training domains, resulting in better generalization to unseen target domains. Recently, Bayesian Neural Networks have achieved promising results in domain invariant learning, but most works concentrate on aligning features distributions rather than parameter distributions. Inspired by the principle of Bayesian Neural Network, we attempt to directly learn the domain invariant posterior distribution of network parameters. We first propose a theorem to show that the invariant posterior of parameters can be implicitly inferred by aggregating posteriors on different training domains. Our assumption is more relaxed and allows us to extract more domain invariant information. We also propose a simple yet effective method, named PosTerior Generalization (PTG), that can be used to estimate the invariant parameter distribution. PTG fully exploits variational inference to approximate parameter distributions, including the invariant posterior and the posteriors on training domains. Furthermore, we develop a lite version of PTG for widespread applications. PTG shows competitive performance on various domain generalization benchmarks on DomainBed. Additionally, PTG can use any existing domain generalization methods as its prior, and combined with previous state-of-the-art method the performance can be further improved. Code will be made public.

cs.LG↗

arXiv2023

A Zero-Shot Language Agent for Computer Control with Structured Reflection

Large language models (LLMs) have shown increasing capacity at planning and executing a high-level goal in a live computer environment (e.g. MiniWoB++). To perform a task, recent works often require a model to learn from trace examples of the task via either supervised learning or few/many-shot prompting. Without these trace examples, it remains a challenge how an agent can autonomously learn and improve its control on a computer, which limits the ability of an agent to perform a new task. We approach this problem with a zero-shot agent that requires no given expert traces. Our agent plans for executable actions on a partially observed environment, and iteratively progresses a task by identifying and learning from its mistakes via self-reflection and structured thought management. On the easy tasks of MiniWoB++, we show that our zero-shot agent often outperforms recent SoTAs, with more efficient reasoning. For tasks with more complexity, our reflective agent performs on par with prior best models, even though previous works had the advantages of accessing expert traces or additional screen information.

cs.CL↗