arXiv subjects

Zichen Tang

Publications and source records attributed to Zichen Tang.

27 records · Page 2Linked to original sources

arXiv2024

ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models

Knowledge Base Question Answering (KBQA) aims to answer natural language questions over large-scale knowledge bases (KBs), which can be summarized into two crucial steps: knowledge retrieval and semantic parsing. However, three core challenges remain: inefficient knowledge retrieval, mistakes of retrieval adversely impacting semantic parsing, and the complexity of previous KBQA methods. To tackle these challenges, we introduce ChatKBQA, a novel and simple generate-then-retrieve KBQA framework, which proposes first generating the logical form with fine-tuned LLMs, then retrieving and replacing entities and relations with an unsupervised retrieval method, to improve both generation and retrieval more directly. Experimental results show that ChatKBQA achieves new state-of-the-art performance on standard KBQA datasets, WebQSP, and CWQ. This work can also be regarded as a new paradigm for combining LLMs with knowledge graphs (KGs) for interpretable and knowledge-required question answering. Our code is publicly available.

cs.CL↗

arXiv2024

Bandwidth-Aware and Overlap-Weighted Compression for Communication-Efficient Federated Learning

Current data compression methods, such as sparsification in Federated Averaging (FedAvg), effectively enhance the communication efficiency of Federated Learning (FL). However, these methods encounter challenges such as the straggler problem and diminished model performance due to heterogeneous bandwidth and non-IID (Independently and Identically Distributed) data. To address these issues, we introduce a bandwidth-aware compression framework for FL, aimed at improving communication efficiency while mitigating the problems associated with non-IID data. First, our strategy dynamically adjusts compression ratios according to bandwidth, enabling clients to upload their models at a close pace, thus exploiting the otherwise wasted time to transmit more data. Second, we identify the non-overlapped pattern of retained parameters after compression, which results in diminished client update signals due to uniformly averaged weights. Based on this finding, we propose a parameter mask to adjust the client-averaging coefficients at the parameter level, thereby more closely approximating the original updates, and improving the training convergence under heterogeneous environments. Our evaluations reveal that our method significantly boosts model accuracy, with a maximum improvement of 13% over the uncompressed FedAvg. Moreover, it achieves a $3.37\times$ speedup in reaching the target accuracy compared to FedAvg with a Top-K compressor, demonstrating its effectiveness in accelerating convergence with compression. The integration of common compression techniques into our framework further establishes its potential as a versatile foundation for future cross-device, communication-efficient FL research, addressing critical challenges in FL and advancing the field of distributed machine learning.

cs.DC↗

arXiv2024

CMOS-Compatible, AlScN-Based Integrated Electro-Optic Modulator

Commercial production of integrated photonic devices is limited by scalability of desirable material platforms. We explore a relatively new photonic material, AlScN, for its use in electro-optic modulation. Its CMOS-compatibility could facilitate large-scale production of integrated photonic modulators, and it exhibits an enhanced second-order optical nonlinearity compared to intrinsic AlN, indicating the possibility for efficient modulation. Here, we measure the electro-optic effect in AlScN-based modulators, demonstrating $V_πL$ around 750 V$\cdot$cm. Since the electro-optic response is smaller than expected, we discuss potential causes for the reduced response and future outlook for AlScN-based photonics.

physics.optics↗

arXiv2024

ArtNeRF: A Stylized Neural Field for 3D-Aware Cartoonized Face Synthesis

Recent advances in generative visual models and neural radiance fields have greatly boosted 3D-aware image synthesis and stylization tasks. However, previous NeRF-based work is limited to single scene stylization, training a model to generate 3D-aware cartoon faces with arbitrary styles remains unsolved. We propose ArtNeRF, a novel face stylization framework derived from 3D-aware GAN to tackle this problem. In this framework, we utilize an expressive generator to synthesize stylized faces and a triple-branch discriminator module to improve the visual quality and style consistency of the generated faces. Specifically, a style encoder based on contrastive learning is leveraged to extract robust low-dimensional embeddings of style images, empowering the generator with the knowledge of various styles. To smooth the training process of cross-domain transfer learning, we propose an adaptive style blending module which helps inject style information and allows users to freely tune the level of stylization. We further introduce a neural rendering module to achieve efficient real-time rendering of images with higher resolutions. Extensive experiments demonstrate that ArtNeRF is versatile in generating high-quality 3D-aware cartoon faces with arbitrary styles.

cs.CV↗

arXiv2024

Wideband, Efficient AlScN-Si Acousto-Optic Modulator in a Commercially Available Silicon Photonics Process

Acousto-optic integration offers numerous applications including low-loss microwave signal processing, nonreciprocal light propagation, frequency comb generation, and broadband acousto-optic modulation. State-of-the-art acousto-optic systems are mainly implemented entirely using in-house fabrication processes, which despite excellent performance typically suffer from low yield and are not compatible with mass production through foundry processes. Here, we demonstrate a highly efficient wideband acousto-optic modulator (AOM) implemented on a silicon photonics foundry process enabling high-yield low-cost mass production of AOMs with other photonic and electronic devices on the same substrate. In the reported structure, a 150 $μ$m long AlScN-based acoustic transducer launches surface acoustic waves (SAW), which modulate the light passing through a silicon optical waveguide. A modulation efficiency of -18.3 dB over a bandwidth of 112 MHz is achieved, which to our knowledge is the highest reported efficiency and bandwidth combination among silicon based AOMs, resulting in about an order of magnitude $BW(V_πL)^{-1}$ figure-of-merit improvement compared to the state-of-the-art CMOS compatible AOMs. The monolithically integrated acousto-optic platform developed in this work will pave the way for low-cost, miniature microwave filters, true time delays, frequency combs, and other signal processors with the advanced functionality offered by foundry-integrated photonic circuits.

physics.optics↗

arXiv2023

Near 6 GHz Sezawa Mode Surface Acoustic Wave Resonators using AlScN on SiC

Surface Acoustic Wave (SAW) devices featuring Aluminum Scandium Nitride (AlScN) on a 4H-Silicon Carbide (SiC) substrate, offer a unique blend of high sound velocity, low thermal resistance, substantial piezoelectric response, simplified fabrication, as well as suitability for high-temperature and harsh environment operation. This study presents high-frequency SAW resonators employing AlScN thin films on SiC substrates, utilizing the second SAW mode (referred to as the Sezawa mode). The resonators achieve remarkable performance, boasting a K2 value of 5.5% and a maximum Q-factor (Qmax) of 1048 at 4.7 GHz, outperforming previous benchmarks. Additionally, a SAW resonator with a 960 nm wavelength attains 5.9 GHz frequency with record K2 (4.0%) and Qmax (887). Our study underscores the potential of the AlScN on SiC platform for advanced radio-frequency applications.

physics.app-ph↗

arXiv2023

HAHE: Hierarchical Attention for Hyper-Relational Knowledge Graphs in Global and Local Level

Link Prediction on Hyper-relational Knowledge Graphs (HKG) is a worthwhile endeavor. HKG consists of hyper-relational facts (H-Facts), composed of a main triple and several auxiliary attribute-value qualifiers, which can effectively represent factually comprehensive information. The internal structure of HKG can be represented as a hypergraph-based representation globally and a semantic sequence-based representation locally. However, existing research seldom simultaneously models the graphical and sequential structure of HKGs, limiting HKGs' representation. To overcome this limitation, we propose a novel Hierarchical Attention model for HKG Embedding (HAHE), including global-level and local-level attention. The global-level attention can model the graphical structure of HKG using hypergraph dual-attention layers, while the local-level attention can learn the sequential structure inside H-Facts via heterogeneous self-attention layers. Experiment results indicate that HAHE achieves state-of-the-art performance in link prediction tasks on HKG standard datasets. In addition, HAHE addresses the issue of HKG multi-position prediction for the first time, increasing the applicability of the HKG link prediction task. Our code is publicly available.

cs.AI↗

arXiv2023

NQE: N-ary Query Embedding for Complex Query Answering over Hyper-Relational Knowledge Graphs

Complex query answering (CQA) is an essential task for multi-hop and logical reasoning on knowledge graphs (KGs). Currently, most approaches are limited to queries among binary relational facts and pay less attention to n-ary facts (n>=2) containing more than two entities, which are more prevalent in the real world. Moreover, previous CQA methods can only make predictions for a few given types of queries and cannot be flexibly extended to more complex logical queries, which significantly limits their applications. To overcome these challenges, in this work, we propose a novel N-ary Query Embedding (NQE) model for CQA over hyper-relational knowledge graphs (HKGs), which include massive n-ary facts. The NQE utilizes a dual-heterogeneous Transformer encoder and fuzzy logic theory to satisfy all n-ary FOL queries, including existential quantifiers, conjunction, disjunction, and negation. We also propose a parallel processing algorithm that can train or predict arbitrary n-ary FOL queries in a single batch, regardless of the kind of each query, with good flexibility and extensibility. In addition, we generate a new CQA dataset WD50K-NFOL, including diverse n-ary FOL queries over WD50K. Experimental results on WD50K-NFOL and other standard CQA datasets show that NQE is the state-of-the-art CQA method over HKGs with good generalization capability. Our code and dataset are publicly available.

cs.AI↗

arXiv2022

Scalable CMOS-BEOL compatible AlScN/2D Channel FE-FETs

Intimate integration of memory devices with logic transistors is a frontier challenge in computer hardware. This integration is essential for augmenting computational power concurrently with enhanced energy efficiency in big-data applications such as artificial intelligence. Despite decades of efforts, reliable, compact, energy efficient and scalable memory devices are elusive. Ferroelectric Field Effect Transistors (FE-FETs) are a promising candidate but their scalability and performance in a back-end-of-line (BEOL) process remain unattained. Here, we present scalable BEOL compatible FE-FETs using two-dimensional (2D) MoS2 channel and AlScN ferroelectric dielectric. We have fabricated a large array of FE-FETs with memory windows larger than 7.8 V, ON/OFF ratios of greater than 10^7, and ON current density greater than 250 uA/um, all at ~80 nm channel lengths. Our devices show stable retention up to 20000 secs and endurance up to 20000 cycles in addition to 4-bit pulse programmable memory features thereby opening a path towards scalable 3D hetero-integration of 2D semiconductor memory with Si CMOS logic.

physics.app-ph↗