Generative Pre-trained Transformer 3 (GPT-3)¶
Autoregressive language model released in 2020 that uses deep learning to produce human-like text
| Publisher | License | Version | Release |
|---|---|---|---|
| OpenAI | Microsoft has exclusive licensing of GPT-3 for Microsoft's products and services | GPT-3.5 | May 28, 2020 |
Model Summary¶
GPT-3 is a deep learning language model released in 2020 that uses generative pre-training to produce human-like text. It is the third-generation language prediction model in the GPT series, successor to GPT-2 created by OpenAI. GPT-3 has a capacity of 175 billion parameters and is capable of performing zero-shot, few-shot and one-shot learning. It can generate text that is difficult to distinguish from human-written text, and has potential applications in both beneficial and harmful applications. Microsoft has exclusive licensing of GPT-3, while OpenAI provides a public-facing API. There are concerns about the environmental impact of training and storing the model, as well as potential issues with academic integrity.
Model Resources¶
📄 Research Paper | 🐱 GitHub | 🕸️ Wikipedia
Info
This model card was generated using PromptxAI API querying recent web content sources with large language model generations. As of Feb 2023 it is not possible to query models like GPT-3 (via applications like ChatGPT) on the latest web content. This is because the model is trained on a static dataset and is not updated with new web content. PromptxAI API solves this problem by chaining recent web content sources with large language model outputs. This allows you to query models like GPT-3 on latest web content.
Model Details¶
Size: 175 billion parameters, requiring 800GB to store
Use Cases: Text generation, summarizing texts, answering questions, coding in CSS, JSX, and Python
Training corpus: Common Crawl, WebText2, Books1, Books2, Wikipedia
Training method: Generative pre-training
Evaluation method: Human evaluators
Compute: 800GB
Features: Zero-shot, few-shot and one-shot learning, edit and insert capabilities
Limitations: Potential to perpetuate fake news, environmental impact of training and storing the models, potential for misuse
Strengths: Eerily good at writing amazingly coherent text, improved language understanding performances in natural language processing (NLP)