Confluent Unveils Data Streaming for AI Initiative
Data streaming pioneer Confluent today announced Data Streaming for AI, an initiative to help organisations accelerate their development of real-time AI applications.
Confluent offers managed Kafka, an open-source distributed event store and stream-processing platform originally developed at LinkedIn. Today, Apache Kafka is used by a growing list of firms in the technology space, the FSI sector, and enterprises.
The move will see Confluent expanding its ecosystem to include AI technologies and committing to a roadmap of product enhancements to help businesses build AI apps faster.
Supporting AI with data
According to Confluent, the resurgent interest in AI can be credited to breakthroughs across reusable large language models (LLM), more accessible machine learning models, and more powerful GPU capabilities.
However, a fundamental challenge in modern AI is a lack of access to the relevant, real-time data that AI applications need in a timely, secure, and scalable way. Real-time AI demands more than fast algorithms but requires trustworthy, relevant data served in near real-time.
The idea behind the initiative is to develop a shared source of real-time truth for all operational and analytical data to overcome today’s fractured data infrastructure and deliver trusted, real-time data.
To enable this, Confluent is expanding partnerships with leading companies in the AI and vector database space, including MongoDB, Pinecone, Rockset, Weaviate, and Zilliz. This will serve to provide real-time contextual data through vector search.
It is also developing strategic partnership agreements with Google Cloud and Microsoft Azure to develop integrations and go-to-market efforts around AI. Finally, it is launching proof-of-concept architectures with Allata and iLink to offer tailored solutions for vertical use cases.
Specifically, Allata developed a data mesh accelerator framework to build an AI-enabled tool and equip its sales team with real-time information. For its part, iLink built an intelligent chat application to help airlines improve customer service and streamline communication.
“Data streaming is a foundational technology for the future of AI,” said Jay Kreps, chief executive officer and co-founder of Confluent. Kreps created Kafka with fellow co-founders Neha Narkhede and Jun Rao.
“Continuously enriched, trustworthy data streams are key to building next-gen AI applications that are accurate and have the rich, real-time context modern use cases demand. We want to make it easier for every company to build powerful AI applications and are leveraging our expansive ecosystem of partners and data streaming expertise to help achieve that,” said Kreps.
Paul Mah is the editor of DSAITrends. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose. You can reach him at [email protected].
Image credit: iStockphoto/jo youngju
