Yuan-1.0

Shawn-IEITSystems/Yuan-1.0 is ranked 16,485 among these 8 peers; microsoft/CSWin-Transformer is ranked 16,476.

0: Large-Scale Pre-trained Language Model in Zero-Shot and Few-Shot Learning Introduction Recent work like GPT-3 has demonstrated excellent performance of Zero-Shot and Few-Shot learning on many natural language processing (NLP) tasks by scaling up model size, data size and the amount of compute. While training a model like GPT-3 requires huge amount of computing power that is a challenge to the researchers. In this work, we propose a method that incorporates large

-scale distributed training performance into model architecture design. 0, the current largest singleton language model with 246B parameters, which achieved excellent performance on thousands GPUs, and state-of-the-art results on different natural language processing tasks. We propose a data processing method that can efficiently filter massive amount of data from Internet. A new dataset with 5TB high-quality text, the current largest Chinese text corpus, is built based on this method. We propose a method based on calibration

and label expansion to improve the Zero-Shot and Few-Shot performance, and steady performance improvements were observed. 0 generated are difficult to distinguish from articles written by humans. Please find details in the paper of Yuan-1. 0 We will open the corpus (1TB) and API of the Yuan model, as well as the codes for fine-tune, few-shot and zero-shot learning. Please visit official website ( for details to get access of the corpus and APIs of

Yuan model. Requirements The inference code is provided on python3. Before start using Yuan API to build your application, several python libs are required. You can simply install them via pip tools. How to use Yuan-API After submit application on official website, it will take several days (normally less than one week) for us to check your application. Please keep your registered account and phone number properly, which will be used to generate an unique key to get access

the API. For more details, please check the example code, , and follow the API document. Applications Here we summarize some simple application example configuration methods for users' reference. The parameters not mentioned therein have adopted default values. |not recommended| Please look forward to more applications.