arXiv subjects

Wenjing Fang

Publications and source records attributed to Wenjing Fang.

15 recordsLinked to original sources

arXiv2024

Ditto: Quantization-aware Secure Inference of Transformers upon MPC

Due to the rising privacy concerns on sensitive client data and trained models like Transformers, secure multi-party computation (MPC) techniques are employed to enable secure inference despite attendant overhead. Existing works attempt to reduce the overhead using more MPC-friendly non-linear function approximations. However, the integration of quantization widely used in plaintext inference into the MPC domain remains unclear. To bridge this gap, we propose the framework named Ditto to enable more efficient quantization-aware secure Transformer inference. Concretely, we first incorporate an MPC-friendly quantization into Transformer inference and employ a quantization-aware distillation procedure to maintain the model utility. Then, we propose novel MPC primitives to support the type conversions that are essential in quantization and implement the quantization-aware MPC execution of secure quantized inference. This approach significantly decreases both computation and communication overhead, leading to improvements in overall efficiency. We conduct extensive experiments on Bert and GPT2 models to evaluate the performance of Ditto. The results demonstrate that Ditto is about $3.14\sim 4.40\times$ faster than MPCFormer (ICLR 2023) and $1.44\sim 2.35\times$ faster than the state-of-the-art work PUMA with negligible utility degradation.

cs.CR↗

arXiv2024

Entropy Engineered Middle-In Synthesis of Dual Single-Atom Compounds for Nitrate Reduction Reaction

Despite the immense potential of Dual Single-Atom Compounds (DSACs), the challenges in their synthesis process, including complexity, stability, purity, and scalability, remain primary concerns in current research. Here, we present a general strategy, termed "Entropy-Engineered Middle-In Synthesis of Dual Single-Atom Compounds" (EEMIS-DSAC), which is meticulously crafted to produce a diverse range of DSACs, effectively addressing the aforementioned issues. Our strategy integrates the advantages of both bottom-up and top-down paradigms, proposing a new insight to optimize the catalyst structure. The as-fabricated DSACs exhibited excellent activity and stability in the nitrate reduction reaction (NO3RR). In a significant advancement, our prototypical CuNi DSACs demonstrated outstanding performance under conditions reminiscent of industrial wastewater. Specifically, under a NO3- concentration of 2000 ppm, it yielded a Faradaic efficiency (FE) for NH3 of 96.97 %, coupled with a mass productivity of 131.47 mg h-1 mg-1 and an area productivity of 10.06 mg h-1 cm-2. Impressively, even under a heightened NO3- concentration of 0.5 M, the FE for NH3 peaked at 90.61 %, with mass productivity reaching 1024.50 mg h-1 mg-1 and an area productivity of 78.41 mg h-1 cm-2. This work underpins the potential of the EEMIS-DSAC approach, signaling a promising frontier for high-performing DSACs.

physics.app-ph↗

arXiv2023

LMSanitator: Defending Prompt-Tuning Against Task-Agnostic Backdoors

Prompt-tuning has emerged as an attractive paradigm for deploying large-scale language models due to its strong downstream task performance and efficient multitask serving ability. Despite its wide adoption, we empirically show that prompt-tuning is vulnerable to downstream task-agnostic backdoors, which reside in the pretrained models and can affect arbitrary downstream tasks. The state-of-the-art backdoor detection approaches cannot defend against task-agnostic backdoors since they hardly converge in reversing the backdoor triggers. To address this issue, we propose LMSanitator, a novel approach for detecting and removing task-agnostic backdoors on Transformer models. Instead of directly inverting the triggers, LMSanitator aims to invert the predefined attack vectors (pretrained models' output when the input is embedded with triggers) of the task-agnostic backdoors, which achieves much better convergence performance and backdoor detection accuracy. LMSanitator further leverages prompt-tuning's property of freezing the pretrained model to perform accurate and fast output monitoring and input purging during the inference phase. Extensive experiments on multiple language models and NLP tasks illustrate the effectiveness of LMSanitator. For instance, LMSanitator achieves 92.8% backdoor detection accuracy on 960 models and decreases the attack success rate to less than 1% in most scenarios.

cs.CL↗

arXiv2022

Private, Efficient, and Accurate: Protecting Models Trained by Multi-party Learning with Differential Privacy

Secure multi-party computation-based machine learning, referred to as MPL, has become an important technology to utilize data from multiple parties with privacy preservation. While MPL provides rigorous security guarantees for the computation process, the models trained by MPL are still vulnerable to attacks that solely depend on access to the models. Differential privacy could help to defend against such attacks. However, the accuracy loss brought by differential privacy and the huge communication overhead of secure multi-party computation protocols make it highly challenging to balance the 3-way trade-off between privacy, efficiency, and accuracy. In this paper, we are motivated to resolve the above issue by proposing a solution, referred to as PEA (Private, Efficient, Accurate), which consists of a secure DPSGD protocol and two optimization methods. First, we propose a secure DPSGD protocol to enforce DPSGD in secret sharing-based MPL frameworks. Second, to reduce the accuracy loss led by differential privacy noise and the huge communication overhead of MPL, we propose two optimization methods for the training process of MPL: (1) the data-independent feature extraction method, which aims to simplify the trained model structure; (2) the local data-based global model initialization method, which aims to speed up the convergence of the model training. We implement PEA in two open-source MPL frameworks: TF-Encrypted and Queqiao. The experimental results on various datasets demonstrate the efficiency and effectiveness of PEA. E.g. when $ε$ = 2, we can train a differentially private classification model with an accuracy of 88% for CIFAR-10 within 7 minutes under the LAN setting. This result significantly outperforms the one from CryptGPU, one SOTA MPL framework: it costs more than 16 hours to train a non-private deep neural network model on CIFAR-10 with the same accuracy.

cs.CR↗

arXiv2021

Large-Scale Secure XGB for Vertical Federated Learning

Privacy-preserving machine learning has drawn increasingly attention recently, especially with kinds of privacy regulations come into force. Under such situation, Federated Learning (FL) appears to facilitate privacy-preserving joint modeling among multiple parties. Although many federated algorithms have been extensively studied, there is still a lack of secure and practical gradient tree boosting models (e.g., XGB) in literature. In this paper, we aim to build large-scale secure XGB under vertically federated learning setting. We guarantee data privacy from three aspects. Specifically, (i) we employ secure multi-party computation techniques to avoid leaking intermediate information during training, (ii) we store the output model in a distributed manner in order to minimize information release, and (iii) we provide a novel algorithm for secure XGB predict with the distributed model. Furthermore, by proposing secure permutation protocols, we can improve the training efficiency and make the framework scale to large dataset. We conduct extensive experiments on both public datasets and real-world datasets, and the results demonstrate that our proposed XGB models provide not only competitive accuracy but also practical performance.

cs.LG↗

arXiv2021

When Homomorphic Encryption Marries Secret Sharing: Secure Large-Scale Sparse Logistic Regression and Applications in Risk Control

Logistic Regression (LR) is the most widely used machine learning model in industry for its efficiency, robustness, and interpretability. Due to the problem of data isolation and the requirement of high model performance, many applications in industry call for building a secure and efficient LR model for multiple parties. Most existing work uses either Homomorphic Encryption (HE) or Secret Sharing (SS) to build secure LR. HE based methods can deal with high-dimensional sparse features, but they incur potential security risks. SS based methods have provable security, but they have efficiency issue under high-dimensional sparse features. In this paper, we first present CAESAR, which combines HE and SS to build secure large-scale sparse logistic regression model and achieves both efficiency and security. We then present the distributed implementation of CAESAR for scalability requirement. We have deployed CAESAR in a risk control task and conducted comprehensive experiments. Our experimental results show that CAESAR improves the state-of-the-art model by around 130 times.

cs.CR↗

arXiv2020

Secret Sharing based Secure Regressions with Applications

Nowadays, the utilization of the ever expanding amount of data has made a huge impact on web technologies while also causing various types of security concerns. On one hand, potential gains are highly anticipated if different organizations could somehow collaboratively share their data for technological improvements. On the other hand, data security concerns may arise for both data holders and data providers due to commercial or sociological concerns. To make a balance between technical improvements and security limitations, we implement secure and scalable protocols for multiple data holders to train linear regression and logistic regression models. We build our protocols based on the secret sharing scheme, which is scalable and efficient in applications. Moreover, our proposed paradigm can be generalized to any secure multiparty training scenarios where only matrix summation and matrix multiplications are used. We demonstrate our approach by experiments which shows the scalability and efficiency of our proposed protocols, and finally present its real-world applications.

cs.LG↗

arXiv2020

Adapted tree boosting for Transfer Learning

Secure online transaction is an essential task for e-commerce platforms. Alipay, one of the world's leading cashless payment platform, provides the payment service to both merchants and individual customers. The fraud detection models are built to protect the customers, but stronger demands are raised by the new scenes, which are lacking in training data and labels. The proposed model makes a difference by utilizing the data under similar old scenes and the data under a new scene is treated as the target domain to be promoted. Inspired by this real case in Alipay, we view the problem as a transfer learning problem and design a set of revise strategies to transfer the source domain models to the target domain under the framework of gradient boosting tree models. This work provides an option for the cold-starting and data-sharing problems.

cs.LG↗

arXiv2020

Unpack Local Model Interpretation for GBDT

A gradient boosting decision tree (GBDT), which aggregates a collection of single weak learners (i.e. decision trees), is widely used for data mining tasks. Because GBDT inherits the good performance from its ensemble essence, much attention has been drawn to the optimization of this model. With its popularization, an increasing need for model interpretation arises. Besides the commonly used feature importance as a global interpretation, feature contribution is a local measure that reveals the relationship between a specific instance and the related output. This work focuses on the local interpretation and proposes an unified computation mechanism to get the instance-level feature contributions for GBDT in any version. Practicality of this mechanism is validated by the listed experiments as well as applications in real industry scenarios.

cs.LG↗

arXiv2019

Uni-traveling-carrier photodetector with high-contrast grating focusing-reflection mirrors

A novel uni-traveling-carrier photodetector (UTC-PD) structure with an integrated focusing-reflection (FR) mirror realized by a non-periodic concentric circular high-contrast grating (NP-CC-HCG), referred to as FR-UTC-PD, is proposed to enhance responsivity in conventional UTC-PDs. The FR-UTC-PD allows improving the responsivity by 36.5% at a 1.55-um wavelength as compared to a UTC-PD without integrated an FR mirror with 84.59% reflectivity. For 40-um-diameter PDs, the obtained 3-dB bandwidths are unaltered with values of 18 GHz at -3.0 V bias voltage. The radio-frequency (RF) output power and photocurrent are -1.77 dBm and 17.56 mA, respectively, at 10 GHz and the -6.0 V bias voltage.

physics.ins-det↗

arXiv2018

Giant intrinsic photoresponse in pristine graphene

When the Fermi level matches the Dirac point in graphene, the reduced charge screening can dramatically enhance electron-electron (e-e) scattering to produce a strongly interacting Dirac liquid. While the dominance of e-e scattering already leads to novel behaviors, such as electron hydrodynamic flow, further exotic phenomena have been predicted to arise specifically from the unique kinematics of e-e scattering in massless Dirac systems. Here, we use optoelectronic probes, which are highly sensitive to the kinematics of electron scattering, to uncover a giant intrinsic photocurrent response in pristine graphene. This photocurrent emerges exclusively at the charge neutrality point and vanishes abruptly at non-zero charge densities. Moreover, it is observed at places with broken reflection symmetry, and it is selectively enhanced at free graphene edges with sharp bends. Our findings reveal that the photocurrent relaxation is strongly suppressed by a drastic change of fast photocarrier kinematics in graphene when its Fermi level matches the Dirac point. The emergence of robust photocurrents in neutral Dirac materials promises new energy-harvesting functionalities and highlights intriguing electron dynamics in the optoelectronic response of Dirac fluids.

cond-mat.mes-hall↗

arXiv2018

Electrical homogeneity of large-area chemical vapor deposited multilayer hexagonal boron nitride sheets

Hexagonal boron nitride (h-BN) is a two dimensional (2D) layered insulator with superior dielectric performance that offers excellent interaction with other 2D materials (e.g. graphene, MoS2). Large-area h-BN can be readily grown on metallic substrates via chemical vapor deposition (CVD), but the impact of local inhomogeneities on the electrical properties of the h-BN and their effect in electronic devices is unknown. Here it is shown that the electrical properties of h-BN stacks grown on polycrystalline Pt vary a lot depending on the crystalline orientation of the Pt grain, but within the same grain the electrical properties of the h-BN are very homogeneous. The reason is that the thickness of the CVD-grown h-BN stack is different on each Pt grain. Conductive atomic force microscopy (CAFM) maps show that the tunneling current across the h-BN stack fluctuates up to 3 orders of magnitude from one Pt grain to another. However, probe station experiments reveal that the variability of electronic devices fabricated within the same Pt grain is surprisingly small. As cutting-edge electronic devices are ultra-scaled, and as the size of the metallic substrate grains can easily exceed 100 μm (in diameter), CVD-grown h-BN stacks may be useful to fabricate electronic devices with low variability.

cond-mat.mtrl-sci↗

arXiv2016

Tuning ultrafast electron thermalization pathways in a van der Waals heterostructure

Ultrafast electron thermalization - the process leading to Auger recombination, carrier multiplication via impact ionization and hot carrier luminescence - occurs when optically excited electrons in a material undergo rapid electron-electron scattering to redistribute excess energy and reach electronic thermal equilibrium. Due to extremely short time and length scales, the measurement and manipulation of electron thermalization in nanoscale devices remains challenging even with the most advanced ultrafast laser techniques. Here, we overcome this challenge by leveraging the atomic thinness of two-dimensional van der Waals (vdW) materials in order to introduce a highly tunable electron transfer pathway that directly competes with electron thermalization. We realize this scheme in a graphene-boron nitride-graphene (G-BN-G) vdW heterostructure, through which optically excited carriers are transported from one graphene layer to the other. By applying an interlayer bias voltage or varying the excitation photon energy, interlayer carrier transport can be controlled to occur faster or slower than the intralayer scattering events, thus effectively tuning the electron thermalization pathways in graphene. Our findings, which demonstrate a novel means to probe and directly modulate electron energy transport in nanoscale materials, represent an important step toward designing and implementing novel optoelectronic and energy-harvesting devices with tailored microscopic properties.

cond-mat.mes-hall↗

arXiv2015

Parallel Stitching of Two-Dimensional Materials

Diverse parallel stitched two-dimensional heterostructures are synthesized, including metal-semiconductor (graphene-MoS2), semiconductor-semiconductor (WS2-MoS2), and insulator-semiconductor (hBN-MoS2), directly through selective sowing of aromatic molecules as the seeds in chemical vapor deposition (CVD) method. Our methodology enables the large-scale fabrication of lateral heterostructures with arbitrary patterns, and clean and precisely aligned interfaces, which offers tremendous potential for its application in integrated circuits.

cond-mat.mtrl-sci↗

arXiv2013

Large-scale 2D Electronics based on Single-layer MoS2 Grown by Chemical Vapor Deposition

2D nanoelectronics based on single-layer MoS2 offers great advantages for both conventional and ubiquitous applications. This paper discusses the large-scale CVD growth of single-layer MoS2 and fabrication of devices and circuits for the first time. Both digital and analog circuits are fabricated to demonstrate its capability for mixed-signal applications.

cond-mat.mtrl-sci↗