arXiv subjects

Yanggang Wang

Publications and source records attributed to Yanggang Wang.

3 recordsLinked to original sources

arXiv2022

ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization

We propose a multitask pretraining approach ZeroPrompt for zero-shot generalization, focusing on task scaling and zero-shot prompting. While previous models are trained on only a few dozen tasks, we scale to 1,000 tasks for the first time using real-world data. This leads to a crucial discovery that task scaling can be an efficient alternative to model scaling; i.e., the model size has little impact on performance with an extremely large number of tasks. Our results show that task scaling can substantially improve training efficiency by 30 times in FLOPs. Moreover, we present a prompting method that incorporates a genetic algorithm to automatically search for the best prompt for unseen tasks, along with a few other improvements. Empirically, ZeroPrompt substantially improves both the efficiency and the performance of zero-shot learning across a variety of academic and production datasets.

cs.LG↗

arXiv2022

GPS: Genetic Prompt Search for Efficient Few-shot Learning

Prompt-based techniques have demostrated great potential for improving the few-shot generalization of pretrained language models. However, their performance heavily relies on the manual design of prompts and thus requires a lot of human efforts. In this paper, we introduce Genetic Prompt Search (GPS) to improve few-shot learning with prompts, which utilizes a genetic algorithm to automatically search for high-performing prompts. GPS is gradient-free and requires no update of model parameters but only a small validation set. Experiments on diverse datasets proved the effectiveness of GPS, which outperforms manual prompts by a large margin of 2.6 points. Our method is also better than other parameter-efficient tuning methods such as prompt tuning.

cs.CL↗

arXiv2022

Artifcial-intelligence-driven discovery of catalyst \textit{genes} with application to CO2 activation on semiconductor oxides

Catalytic-materials design requires predictive modeling of the interaction between catalyst and reactants. This is challenging due to the complexity and diversity of structure-property relationships across the chemical space. Here, we report a strategy for a rational design of catalytic materials using the artifcial intelligence approach (AI) subgroup discovery. We identify catalyst \textit{genes} (features) that correlate with mechanisms that trigger, facilitate, or hinder the activation of carbon dioxide (CO$_2$) towards a chemical conversion. The AI model is trained on frst-principles data for a broad family of oxides. We demonstrate that surfaces of experimentally identifed good catalysts consistently exhibit combinations of \textit{genes} resulting in a strong elongation of a C-O bond. The same combinations of \textit{genes} also minimize the OCO-angle, the previously proposed indicator of activation, albeit under the constraint that the Sabatier principle is satisfed. Based on these fndings, we propose a set of new promising catalyst materials for CO$_2$ conversion.

cond-mat.mtrl-sci↗