气候变化领域集成服务门户(CCPortal): Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences

CCPortal

DOI	10.1073/pnas.2016239118
	Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences
	Rives A.; Meier J.; Sercu T.; Goyal S.; Lin Z.; Liu J.; Guo D.; Ott M.; Zitnick C.L.; Ma J.; Fergus R.
发表日期	2021
ISSN	00278424
卷号	118 期号:15
英文摘要	In the field of artificial intelligence, a combination of scale in data and model capacity enabled by unsupervised learning has led to major advances in representation learning and statistical generation. In the life sciences, the anticipated growth of sequencing promises unprecedented data on natural sequence diversity. Protein language modeling at the scale of evolution is a logical step toward predictive and generative artificial intelligence for biology. To this end, we use unsupervised learning to train a deep contextual language model on 86 billion amino acids across 250 million protein sequences spanning evolutionary diversity. The resulting model contains information about biological properties in its representations. The representations are learned from sequence data alone. The learned representation space has a multiscale organization reflecting structure from the level of biochemical properties of amino acids to remote homology of proteins. Information about secondary and tertiary structure is encoded in the representations and can be identified by linear projections. Representation learning produces features that generalize across a range of applications, enabling state-of-the-art supervised prediction of mutational effect and secondary structure and improving state-of-the-art features for long-range contact prediction. © 2021 National Academy of Sciences. All rights reserved.
英文关键词	Deep learning; Generative biology; Protein language model; Representation learning; Synthetic biology
语种	英语
来源期刊	Proceedings of the National Academy of Sciences of the United States of America
文献类型	期刊论文
条目标识符	http://gcip.llas.ac.cn/handle/2XKMVOVA/179861
作者单位	Facebook AI Research, New York, NY 10003, United States; Department of Computer Science, New York University, New York, NY 10012, United States; Harvard University, Cambridge, MA 02138, United States; Booth School of Business, University of Chicago, Chicago, IL 60637, United States; Yale Law School, New Haven, CT 06511, United States
推荐引用方式 GB/T 7714	Rives A.,Meier J.,Sercu T.,et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences[J],2021,118(15).
APA	Rives A..,Meier J..,Sercu T..,Goyal S..,Lin Z..,...&Fergus R..(2021).Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences.Proceedings of the National Academy of Sciences of the United States of America,118(15).
MLA	Rives A.,et al."Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences".Proceedings of the National Academy of Sciences of the United States of America 118.15(2021).