Databricks Unveils Dolly, Open-source Alternative to ChatGPT
The race to develop AI models with instruction-following capabilities has intensified as researchers and technology giants around the world race to incorporate AI capabilities into their products since the release of ChatGPT in November last year.
Despite the heightened activity, it is worth noting that most state-of-the-art AI models like ChatGPT remain closed to most but a select group with either the resources or access. Crucially, the vast amount of processing power required to train these models puts them out of the reach of many independent researchers or organizations seeking to study them.
Introducing Dolly
Databricks says it wants to change this landscape with the introduction of Dolly as an open-source model, which it bills as a “cheap-to-build” large language model or LLM. And despite using just 6 billion parameters compared to 175 billion for GPT-3, Databricks says Dolly exhibits a surprising degree of the instruction following capabilities exhibited by ChatGPT.
Databricks achieved this by building on the open-source EleutherAI model from two years ago – which sports 6 billion parameters. This was then modified slightly using data from Stanford’s Alpaca model to elicit instruction following capabilities such as brainstorming and text generation not present in the original model.
What follows was a vast improvement in instruction-following capability and behavior, highlighting how a small, high-quality dataset can make a world of difference. And yes, Dolly got its name from Dolly the sheep, the first cloned mammal – given how it is an open-source clone of an Alpaca (Alpaca is itself based on Meta’s LLaMA).
Databricks says this development has the potential to transform LLMs from a luxury available to a few companies to a widely accessible commodity, enabling every company to own and customize AI models to enhance their products.
“We believe models like Dolly will help democratize LLMs, transforming them from something very few companies can afford into a commodity every company can own and customize to improve their products,” wrote Databricks in a blog post.
“We’re in the earliest days of the democratization of AI for the enterprise, and much work remains to be done, but we believe the technology underlying Dolly represents an exciting new opportunity for companies that want to cheaply build their own instruction-following models.”
Image credit: iStockphoto/simoningate
