Searcharxiv⌕ Search

arXiv subjects

Qian Yang

Publications and source records attributed to Qian Yang.

At least 91 records · Page 5Linked to original sources
arXiv2023

Dict-TTS: Learning to Pronounce with Prior Dictionary Knowledge for Text-to-Speech

Polyphone disambiguation aims to capture accurate pronunciation knowledge from natural text sequences for reliable Text-to-speech (TTS) systems. However, previous approaches require substantial annotated training data and additional efforts from language experts, making it difficult to extend high-quality neural TTS systems to out-of-domain daily conversations and countless languages worldwide. This paper tackles the polyphone disambiguation problem from a concise and novel perspective: we propose Dict-TTS, a semantic-aware generative text-to-speech model with an online website dictionary (the existing prior information in the natural language). Specifically, we design a semantics-to-pronunciation attention (S2PA) module to match the semantic patterns between the input text sequence and the prior semantics in the dictionary and obtain the corresponding pronunciations; The S2PA module can be easily trained with the end-to-end TTS model without any annotated phoneme labels. Experimental results in three languages show that our model outperforms several strong baseline models in terms of pronunciation accuracy and improves the prosody modeling of TTS systems. Further extensive analyses demonstrate that each design in Dict-TTS is effective. The code is available at \url{https://github.com/Zain-Jiang/Dict-TTS}.

eess.AS↗
arXiv2023

The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice

Despite the growing consensus that stakeholders affected by AI systems should participate in their design, enormous variation and implicit disagreements exist among current approaches. For researchers and practitioners who are interested in taking a participatory approach to AI design and development, it remains challenging to assess the extent to which any participatory approach grants substantive agency to stakeholders. This article thus aims to ground what we dub the "participatory turn" in AI design by synthesizing existing theoretical literature on participation and through empirical investigation and critique of its current practices. Specifically, we derive a conceptual framework through synthesis of literature across technology design, political theory, and the social sciences that researchers and practitioners can leverage to evaluate approaches to participation in AI design. Additionally, we articulate empirical findings concerning the current state of participatory practice in AI design based on an analysis of recently published research and semi-structured interviews with 12 AI researchers and practitioners. We use these empirical findings to understand the current state of participatory practice and subsequently provide guidance to better align participatory goals and methods in a way that accounts for practical constraints.

cs.HC↗
arXiv2023

Synthesis and characterisation of LK-99

Recently, two arXiv preprints (arXiv:2307.12008, arXiv:2307.12037) reported signatures of superconductivity above room temperature (SART) and at ambient pressure, striking worldwide experimental research efforts in replicating the results3-6, as well as theoretical attempts to explain the purported superconductivity7-11. The material of interest has chemical formula Pb$_{10-x}$Cu$_x$(PO$_4$)$_6$O, where $x \approx 1$, and was named by the authors as LK-99. It belongs to lead apatite family, and was synthesised from two precursors, lanarkite (PbSO$_4\cdot$PbO) and copper phosphide (Cu$_3$P). Here we performed a systematic study on LK-99, starting from solid-state synthesis, followed by characterisation and transport measurements. We did not observe any signatures of superconductivity in our samples of LK-99.

cond-mat.supr-con↗
arXiv2023

Enhancing Multi-modal and Multi-hop Question Answering via Structured Knowledge and Unified Retrieval-Generation

Multi-modal multi-hop question answering involves answering a question by reasoning over multiple input sources from different modalities. Existing methods often retrieve evidences separately and then use a language model to generate an answer based on the retrieved evidences, and thus do not adequately connect candidates and are unable to model the interdependent relations during retrieval. Moreover, the pipelined approaches of retrieval and generation might result in poor generation performance when retrieval performance is low. To address these issues, we propose a Structured Knowledge and Unified Retrieval-Generation (SKURG) approach. SKURG employs an Entity-centered Fusion Encoder to align sources from different modalities using shared entities. It then uses a unified Retrieval-Generation Decoder to integrate intermediate retrieval results for answer generation and also adaptively determine the number of retrieval steps. Extensive experiments on two representative multi-modal multi-hop QA datasets MultimodalQA and WebQA demonstrate that SKURG outperforms the state-of-the-art models in both source retrieval and answer generation performance with fewer parameters. Our code is available at https://github.com/HITsz-TMG/SKURG.

cs.CL↗
arXiv2023

Semi-Siamese Network for Robust Change Detection Across Different Domains with Applications to 3D Printing

Automatic defect detection for 3D printing processes, which shares many characteristics with change detection problems, is a vital step for quality control of 3D printed products. However, there are some critical challenges in the current state of practice. First, existing methods for computer vision-based process monitoring typically work well only under specific camera viewpoints and lighting situations, requiring expensive pre-processing, alignment, and camera setups. Second, many defect detection techniques are specific to pre-defined defect patterns and/or print schematics. In this work, we approach the defect detection problem using a novel Semi-Siamese deep learning model that directly compares a reference schematic of the desired print and a camera image of the achieved print. The model then solves an image segmentation problem, precisely identifying the locations of defects of different types with respect to the reference schematic. Our model is designed to enable comparison of heterogeneous images from different domains while being robust against perturbations in the imaging setup such as different camera angles and illumination. Crucially, we show that our simple architecture, which is easy to pre-train for enhanced performance on new datasets, outperforms more complex state-of-the-art approaches based on generative adversarial networks and transformers. Using our model, defect localization predictions can be made in less than half a second per layer using a standard MacBook Pro while achieving an F1-score of more than 0.9, demonstrating the efficacy of using our method for in-situ defect detection in 3D printing.

cs.CV↗
arXiv2023

Probing the Origin of Changing-look Quasar Transitions with Chandra

Extremely variable quasars can also show strong changes in broad-line emission strength and are known as changing-look quasars (CLQs). To study the CLQ transition mechanism, we present a pilot sample of CLQs with X-ray observations in both the bright and faint states. From a sample of quasars with bright-state archival SDSS spectra and (Chandra or XMM-Newton) X-ray data, we identified five new CLQs via optical spectroscopic follow-up, and then obtained new target-of-opportunity X-ray observations with Chandra. No strong absorption is detected in either the bright- or the faint-state X-ray spectra. The intrinsic X-ray flux generally changes along with the optical variability, and the X-ray power-law slope becomes harder in the faint state. Large amplitude mid-infrared variability is detected in all five CLQs, and the MIR variability echoes the variability in the optical with a time lag expected from the light-crossing time of the dusty torus for CLQs with robust lag measurements. The changing-obscuration model is not consistent with the observed X-ray spectra and spectral energy distribution changes seen in these CLQs. It is highly likely that the observed changes are due to the changing accretion rate of the supermassive black hole, so the multiwavelength emission varies accordingly, with promising analogies to the accretion states of X-ray binaries.

astro-ph.GA↗
arXiv2023

Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias

Scaling text-to-speech to a large and wild dataset has been proven to be highly effective in achieving timbre and speech style generalization, particularly in zero-shot TTS. However, previous works usually encode speech into latent using audio codec and use autoregressive language models or diffusion models to generate it, which ignores the intrinsic nature of speech and may lead to inferior or uncontrollable results. We argue that speech can be decomposed into several attributes (e.g., content, timbre, prosody, and phase) and each of them should be modeled using a module with appropriate inductive biases. From this perspective, we carefully design a novel and large zero-shot TTS system called Mega-TTS, which is trained with large-scale wild data and models different attributes in different ways: 1) Instead of using latent encoded by audio codec as the intermediate feature, we still choose spectrogram as it separates the phase and other attributes very well. Phase can be appropriately constructed by the GAN-based vocoder and does not need to be modeled by the language model. 2) We model the timbre using global vectors since timbre is a global attribute that changes slowly over time. 3) We further use a VQGAN-based acoustic model to generate the spectrogram and a latent code language model to fit the distribution of prosody, since prosody changes quickly over time in a sentence, and language models can capture both local and long-range dependencies. We scale Mega-TTS to multi-domain datasets with 20K hours of speech and evaluate its performance on unseen speakers. Experimental results demonstrate that Mega-TTS surpasses state-of-the-art TTS systems on zero-shot TTS, speech editing, and cross-lingual TTS tasks, with superior naturalness, robustness, and speaker similarity due to the proper inductive bias of each module. Audio samples are available at https://mega-tts.github.io/demo-page.

eess.AS↗
arXiv2023

FluentSpeech: Stutter-Oriented Automatic Speech Editing with Context-Aware Diffusion Models

Stutter removal is an essential scenario in the field of speech editing. However, when the speech recording contains stutters, the existing text-based speech editing approaches still suffer from: 1) the over-smoothing problem in the edited speech; 2) lack of robustness due to the noise introduced by stutter; 3) to remove the stutters, users are required to determine the edited region manually. To tackle the challenges in stutter removal, we propose FluentSpeech, a stutter-oriented automatic speech editing model. Specifically, 1) we propose a context-aware diffusion model that iteratively refines the modified mel-spectrogram with the guidance of context features; 2) we introduce a stutter predictor module to inject the stutter information into the hidden sequence; 3) we also propose a stutter-oriented automatic speech editing (SASE) dataset that contains spontaneous speech recordings with time-aligned stutter labels to train the automatic stutter localization model. Experimental results on VCTK and LibriTTS datasets demonstrate that our model achieves state-of-the-art performance on speech editing. Further experiments on our SASE dataset show that FluentSpeech can effectively improve the fluency of stuttering speech in terms of objective and subjective metrics. Code and audio samples can be found at https://github.com/Zain-Jiang/Speech-Editing-Toolkit.

cs.SD↗
arXiv2023

Towards Prototyping Driverless Vehicle Behaviors, City Design, and Policies Simultaneously

Autonomous Vehicles (AVs) can potentially improve urban living by reducing accidents, increasing transportation accessibility and equity, and decreasing emissions. Realizing these promises requires the innovations of AV driving behaviors, city plans and infrastructure, and traffic and transportation policies to join forces. However, the complex interdependencies among AV, city, and policy design issues can hinder their innovation. We argue the path towards better AV cities is not a process of matching city designs and policies with AVs' technological innovations, but a process of iterative prototyping of all three simultaneously: Innovations can happen step-wise as the knot of AV, city, and policy design loosens and tightens, unwinds and reties. In this paper, we ask: How can innovators innovate AVs, city environments, and policies simultaneously and productively toward better AV cities? The paper has two parts. First, we map out the interconnections among the many AV, city, and policy design decisions, based on a literature review spanning HCI/HRI, transportation science, urban studies, law and policy, operations research, economy, and philosophy. This map can help innovators identify design constraints and opportunities across the traditional AV/city/policy design disciplinary bounds. Second, we review the respective methods for AV, city, and policy design, and identify key barriers in combining them: (1) Organizational barriers to AV-city-policy design collaboration, (2) computational barriers to multi-granularity AV-city-policy simulation, and (3) different assumptions and goals in joint AV-city-policy optimization. We discuss two broad approaches that can potentially address these challenges, namely, "low-fidelity integrative City-AV-Policy Simulation (iCAPS)" and "participatory design optimization".

cs.HC↗
arXiv2023

Disentangling $1/f$ noise from confined ion dynamics

Ion transport through biological and solid-state nanochannels is known to be a highly noisy process. The power spectrum of current fluctuations is empirically known to scale like the inverse of frequency, following the long-standing yet poorly understood Hooge's law. Here, we report measurements of current fluctuations across nanometer-scale two-dimensional channels with different surface properties. The structure of fluctuations is found to depend on channel's material. While in pristine channels current fluctuations scale like $1/f^{1+a}$ with $a = 0 - 0.5$, the noise power spectrum of activated graphite channels displays different regimes depending on frequency. Based on these observations, we develop a theoretical formalism directly linking ion dynamics and current fluctuations. We predict that the noise power spectrum take the form $1/f \times S_\text{channel}(f)$, where $1/f$ fluctuations emerge in fluidic reservoirs on both sides of the channel and $S_\text{channel}$ describes fluctuations inside it. Deviations to Hooge's law thus allow direct access to the ion transport dynamics of the channel -- explaining the entire phenomenology observed in experiments on 2D nanochannels. Our results demonstrate how current fluctuations can be used to characterize nanoscale ion dynamics.

cond-mat.soft↗
arXiv2023

Med-EASi: Finely Annotated Dataset and Models for Controllable Simplification of Medical Texts

Automatic medical text simplification can assist providers with patient-friendly communication and make medical texts more accessible, thereby improving health literacy. But curating a quality corpus for this task requires the supervision of medical experts. In this work, we present $\textbf{Med-EASi}$ ($\underline{\textbf{Med}}$ical dataset for $\underline{\textbf{E}}$laborative and $\underline{\textbf{A}}$bstractive $\underline{\textbf{Si}}$mplification), a uniquely crowdsourced and finely annotated dataset for supervised simplification of short medical texts. Its $\textit{expert-layman-AI collaborative}$ annotations facilitate $\textit{controllability}$ over text simplification by marking four kinds of textual transformations: elaboration, replacement, deletion, and insertion. To learn medical text simplification, we fine-tune T5-large with four different styles of input-output combinations, leading to two control-free and two controllable versions of the model. We add two types of $\textit{controllability}$ into text simplification, by using a multi-angle training approach: $\textit{position-aware}$, which uses in-place annotated inputs and outputs, and $\textit{position-agnostic}$, where the model only knows the contents to be edited, but not their positions. Our results show that our fine-grained annotations improve learning compared to the unannotated baseline. Furthermore, $\textit{position-aware}$ control generates better simplification than the $\textit{position-agnostic}$ one. The data and code are available at https://github.com/Chandrayee/CTRL-SIMP.

cs.CL↗
arXiv2023

Conversation Regression Testing: A Design Technique for Prototyping Generalizable Prompt Strategies for Pre-trained Language Models

Pre-trained language models (LLMs) such as GPT-3 can carry fluent, multi-turn conversations out-of-the-box, making them attractive materials for chatbot design. Further, designers can improve LLM chatbot utterances by prepending textual prompts -- instructions and examples of desired interactions -- to its inputs. However, prompt-based improvements can be brittle; designers face challenges systematically understanding how a prompt strategy might impact the unfolding of subsequent conversations across users. To address this challenge, we introduce the concept of Conversation Regression Testing. Based on sample conversations with a baseline chatbot, Conversation Regression Testing tracks how conversational errors persist or are resolved by applying different prompt strategies. We embody this technique in an interactive design tool, BotDesigner, that lets designers identify archetypal errors across multiple conversations; shows common threads of conversation using a graph visualization; and highlights the effects of prompt changes across bot design iterations. A pilot evaluation demonstrates the usefulness of both the concept of regression testing and the functionalities of BotDesigner for chatbot designers.

cs.HC↗
arXiv2023

The SDSS-V Black Hole Mapper Reverberation Mapping Project: Unusual Broad-Line Variability in a Luminous Quasar

We present a high-cadence multi-epoch analysis of dramatic variability of three broad emission lines (MgII, H$β$, and H$α$) in the spectra of the luminous quasar ($λL_λ$(5100Å) = $4.7 \times 10^{44}$ erg s$^{-1}$) SDSS J141041.25+531849.0 at $z = 0.359$ with 127 spectroscopic epochs over 9 years of monitoring (2013-2022). We observe anti-correlations between the broad emission-line widths and flux in all three emission lines, indicating that all three broad emission lines "breathe" in response to stochastic continuum variations. We also observe dramatic radial velocity shifts in all three broad emission lines, ranging from $Δ{v}$ $\sim$400 km s$^{-1}$ to $\sim$800 km s$^{-1}$, that vary over the course of the monitoring period. Our preferred explanation for the broad-line variability is complex kinematics in the broad-line region gas. We suggest a model for the broad-line variability that includes a combination of gas inflow with a radial gradient, an azimuthal asymmetry (e.g., a hot spot), superimposed on the stochastic flux-driven changes to the optimal emission region ("line breathing"). Similar instances of line-profile variability due to complex gas kinematics around quasars are likely to represent an important source of false positives in radial velocity searches for binary black holes, which typically lack the kind of high-cadence data we analyze here. The long-duration, wide-field, and many-epoch spectroscopic monitoring of SDSS-V BHM-RM provides an excellent opportunity for identifying and characterizing broad emission-line variability, and the inferred nature of the inner gas environment, of luminous quasars.

astro-ph.GA↗
arXiv2022

Machine learning approach to genome of two-dimensional materials with flat electronic bands

Many-body physics of electron-electron correlations plays a central role in condensed mater physics, it governs a wide range of phenomena, stretching from superconductivity to magnetism, and is behind numerous technological applications. To explore this rich interaction-driven physics, two-dimensional (2D) materials with flat electronic bands provide a natural playground thanks to their highly localised electrons. Currently, thousands of 2D materials with computed electronic bands are available in open science databases, awaiting such exploration. Here we used a new machine learning algorithm combining both supervised and unsupervised machine intelligence to automate the otherwise daunting task of materials search and classification, to build a genome of 2D materials hosting flat electronic bands. To this end, a feedforward artificial neural network was employed to identify 2D flat band materials, which were then classified by a bilayer unsupervised learning algorithm. Such a hybrid approach of exploring materials databases allowed us to reveal completely new material classes outside the known flat band paradigms, offering new systems for in-depth study on their electronic interactions.

cond-mat.mes-hall↗
arXiv2022

Chunk-aware Alignment and Lexical Constraint for Visual Entailment with Natural Language Explanations

Visual Entailment with natural language explanations aims to infer the relationship between a text-image pair and generate a sentence to explain the decision-making process. Previous methods rely mainly on a pre-trained vision-language model to perform the relation inference and a language model to generate the corresponding explanation. However, the pre-trained vision-language models mainly build token-level alignment between text and image yet ignore the high-level semantic alignment between the phrases (chunks) and visual contents, which is critical for vision-language reasoning. Moreover, the explanation generator based only on the encoded joint representation does not explicitly consider the critical decision-making points of relation inference. Thus the generated explanations are less faithful to visual-language reasoning. To mitigate these problems, we propose a unified Chunk-aware Alignment and Lexical Constraint based method, dubbed as CALeC. It contains a Chunk-aware Semantic Interactor (arr. CSI), a relation inferrer, and a Lexical Constraint-aware Generator (arr. LeCG). Specifically, CSI exploits the sentence structure inherent in language and various image regions to build chunk-aware semantic alignment. Relation inferrer uses an attention-based reasoning network to incorporate the token-level and chunk-level vision-language representations. LeCG utilizes lexical constraints to expressly incorporate the words or chunks focused by the relation inferrer into explanation generation, improving the faithfulness and informativeness of the explanations. We conduct extensive experiments on three datasets, and experimental results indicate that CALeC significantly outperforms other competitor models on inference accuracy and quality of generated explanations.

cs.CL↗
arXiv2022

Mixing of surface and bulk electronic states at a graphite-hexagonal boron nitride interface

Van der Waals assembly enables exquisite design of electronic states in two-dimensional (2D) materials, often by superimposing a long-wavelength periodic potential on a crystal lattice using moiré superlattices. Here we show that electronic states in three-dimensional (3D) crystals such as graphite can also be tuned by the superlattice potential arising at the interface with another crystal, namely, crystallographically aligned hexagonal boron nitride. Such alignment is found to result in a multitude of Lifshitz transitions and Brown-Zak oscillations for near-surface 2D states whereas, in high magnetic fields, fractal states of Hofstadter's butterfly extend deep into graphite's bulk. Our work shows a venue to control 3D spectra by using the approach of 2D twistronics.

cond-mat.mes-hall↗
arXiv2022

A Transient "Changing-look'' Active Galactic Nucleus Resolved on Month Timescales from First-year Sloan Digital Sky Survey V Data

We report the discovery of a new ``changing-look'' active galactic nucleus (CLAGN) event, in the quasar SDSS J162829.17+432948.5 at z=0.2603, identified through repeat spectroscopy from the fifth Sloan Digital Sky Survey (SDSS-V). Optical photometry taken during 2020--2021 shows a dramatic dimming of $Δ$g${\approx}$1 mag, followed by a rapid recovery on a timescale of several months, with the ${\lesssim}$2 month period of rebrightening captured in new SDSS-V and Las Cumbres Observatory spectroscopy. This is one of the fastest CLAGN transitions observed to date. Archival observations suggest that the object experienced a much more gradual dimming over the period of 2011--2013. Our spectroscopy shows that the photometric changes were accompanied by dramatic variations in the quasar-like continuum and broad-line emission. The excellent agreement between the pre- and postdip photometric and spectroscopic appearances of the source, as well as the fact that the dimmest spectra can be reproduced by applying a single extinction law to the brighter spectral states, favor a variable line-of-sight obscuration as the driver of the observed transitions. Such an interpretation faces several theoretical challenges, and thus an alternative accretion-driven scenario cannot be excluded. The recent events observed in this quasar highlight the importance of spectroscopic monitoring of large active galactic nucleus samples on weeks-to-months timescales, which the SDSS-V is designed to achieve.

astro-ph.GA↗
arXiv2022

A Southern Photometric Quasar Catalog from the Dark Energy Survey Data Release 2

We present a catalog of 1.4 million photometrically-selected quasar candidates in the southern hemisphere over the $\sim 5000\,{\rm deg^2}$ Dark Energy Survey (DES) wide survey area. We combine optical photometry from the DES second data release (DR2) with available near-infrared (NIR) and the all-sky unWISE mid-infrared photometry in the selection. We build models of quasars, galaxies, and stars with multivariate Skew-t distributions in the multi-dimensional space of relative fluxes as functions of redshift (or color for stars) and magnitude. Our selection algorithm assigns probabilities for quasars, galaxies, and stars, and simultaneously calculates photometric redshifts (photo-$z$) for quasar and galaxy candidates. Benchmarking on spectroscopically confirmed objects, we successfully classify (with photometry) 94.7% of quasars, 99.3% of galaxies, and 96.3% of stars when all IR bands (NIR $YJHK$ and WISE $W1W2$) are available. The classification and photo-$z$ regression success rates decrease when fewer bands are available. Our quasar (galaxy) photo-$z$ quality, defined as the fraction of objects with the difference between the photo-$z$ $z_p$ and the spectroscopic redshift $z_s$, $|Δz| = |z_s - z_p|/(1 + z_s)\le 0.1$, is 92.2% (98.1%) when all IR bands are available, decreasing to 72.2% (90.0%) using optical DES data only. Our photometric quasar catalog achieves estimated completeness of 89% and purity of 79% at $r<21.5$ (0.68 million quasar candidates), with reduced completeness and purity at $21.5<r\lesssim 24$. Among the 1.4 million quasar candidates, 87,857 have existing spectra and 84,978 (96.7%) of them are spectroscopically confirmed quasars. Finally, we provide quasar, galaxy, and star probabilities for all (0.69 billion) photometric sources in the DES DR2 coadded photometric catalog.

astro-ph.GA↗