Foundation models are a new type of neural network trained on an enormous volume of raw, unlabeled data. A foundation model is generally trained through a process known as unsupervised learning — it's given access to either a data set or an ongoing stream of data with no instructions on what to do with that data. The model must then analyze the data of its own accord, which it does through a variety of techniques, including:
Traditional machine learning models result in artificial intelligence created for a specific purpose, also known as narrow AI. This type of artificial intelligence rarely expands beyond the original task for which it was created. Foundation models are AI systems with a completely new coat of paint.
Instead of being developed for a specific problem, role, or task, a foundation model is far more general and expansive in scope. They can, as the name suggests, be used as a foundation for many different AI use cases. In some cases, they've even managed to create what at first glance appears to be completely original artwork — though given that most foundation models are trained on data scraped from the Internet, whether or not their creations can be called original is up for debate.
Compared to traditional machine learning (ML), foundation models are considerably more efficient and sophisticated.
To understand foundation models, one must also understand the two concepts that exist at their core.
A neural network is a unique type of machine learning model inspired by the human brain. They take the form of a series of interconnected algorithms, each of which acts as a sort of artificial 'neuron' in the network. These algorithms work together to collect, analyze, and classify data on a colossal scale.
Because of the way they're architected, neural networks are uniquely capable of adapting and adjusting to changing input. Whereas a traditional machine learning algorithm might need to be completely retrained in order to adapt to a new environment, a neural network can process and analyze that environment based on the information it already possesses.
Deep learning is a type of machine learning that layers multiple neural networks atop one another, enabling a significantly greater level of sophistication.
Currently, the most popular type of foundation model is represented by ChatGPT. OpenAI's chat tool is a generative large language model built atop the transformer architecture. This means that it's:
Computer vision models are another common type of foundation model, trained to recognize and identify images and other visual elements.
There is currently some debate over the future of artificial intelligence — namely, the question of whether proprietary or open-source will come to dominate the market in the immediate future. Beyond the fact that foundation models represent some of the most advanced programming the world has seen in quite some time, anyone familiar with the debate between proprietary and open-source systems is already familiar with the arguments being made here.
Proprietary foundation models offer more reliable update schedules, improved security and governance, and expert support. Open-source foundation models, on the other hand, provide a great deal more freedom and flexibility, but are frequently a great deal more challenging to configure.