Most AI systems today are classifiers, meaning they can be trained to distinguish between images of dogs and cats. Generative AI systems can be trained to generate an image of a dog or a cat that doesn't exist in the real world. The ability for technology to be creative is a game changer.
Generative AI enables systems to create high-value artifacts, such as video, narrative, training data and even designs and schematics.
Generative Pre-trained Transformer (GPT), for example, is a large-scale natural language technology that uses deep learning to produce human-like text. The third generation (GPT-3), which predicts the most likely next word in a sentence based on its absorbed accumulated training, can write stories, songs and poetry, and even computer code — and enables ChatGPT to do your teenager’s homework in seconds.
Beyond text, digital-image generators, such as DALL·E 2, Stable Diffusion and Midjourney, can generate images from text.
There are a number of AI techniques employed for generative AI, but most recently, foundation models have taken the spotlight.
Foundation models are pretrained on general data sources in a self-supervised manner, which can then be adapted to solve new problems. Foundation models are based mainly on transformer architectures, which embody a type of deep neural network architecture that computes a numerical representation of training data.
Transformer architectures learn context and, thus, meaning by tracking relationships in sequential data. Transformer models apply an evolving set of mathematical techniques, called attention or self-attention, to detect subtle ways even distant data elements in a series influence and depend on each other.