Through the launch of QDS Community Edition, Enterprise Edition and Cloud Agents, Qubole is building an autonomous data platform with intelligent automation to address the difficulty enterprises have in scaling their data teams and initiatives.
The new QDS platform self-manages, self-optimizes and learns by watching platform usage. The architecture analyzes metadata (queries, clusters, users and data) generated by platform usage and applies machine learning and artificial intelligence to create alerts, insights, recommendations and autonomous agents which perform actions automatically. The company claims that QDS platform is the only solution that intelligently automates and analyzes usage to learn how to make data teams more effective.
“Despite all the promise and technological advancements, operationalizing big data efforts is still impossible for most organizations. Data teams simply can’t scale to meet the growing demand for data across organizations. To address this hurdle, we are on a mission to remove the manual effort that comes with maintaining a big data infrastructure and empower data teams to focus on high-value, strategic work,” said Ashish Thusoo, Co-Founder and CEO, Qubole.
“With the release of Qubole’s autonomous data platform, data teams can optimize cost, performance and reliability by intelligently automating tedious manual data management tasks – helping organizations gain the full value of their data.”
QDS Enterprise
QDS Enterprise Edition is stated to replaces the existing QDS solution and will offer alerts, isights and recommendations that provide intelligent, actionable information to data professionals. It’s new machine learning infrastructure can be rule-based, workload aware and/or predictive and increasingly accurate by learning from user behavior over time.
- Alerts provide proactive notifications to DataOps and data engineers to identify real-time situations that require immediate attention.
- Insights provide analytics to data professionals that help highlight data and platform.
- Recommendations provide prescriptive actions that data professionals can perform.
QDS Cloud Agents
QDS Cloud Agents claims to be the first product that can autonomously execute a range of data management tasks that are typically time-consuming, manual tasks. The QDS Cloud Agents include a range of cloud-based software agents that offload tasks from human data professionals, based on a set policy, that can reduce cost, offload manual effort and improve reliability.
QDS Cloud Agents is an optional add-on to the QDS Enterprise edition. The individual agents are available for AWS, Microsoft and Oracle BMC clouds, with the exception of the Spot Agent.
The initial release of QDS Cloud Agents includes the following three agents:
- Workload Aware Auto-Scaling Agent: Optimizes cluster size precisely to workload requirements, reducing the over-provisioning typically seen with alternative solutions. The agent is workload-aware and manages scaling up and down dynamically based on actual processing load to reduce overall compute costs by up to 50%
- Spot Shopper Agent (available for AWS Only): Intelligently shops across AWS cloud to assemble the compute instances in the optimal combination of performance and cost. This can include AWS Spot Instances and Spot Blocks, mixed instance types, different availability zones and provisioned dynamically at run time. According to the company the Spot Shopper can reduce compute resource costs by up to 80% vs on-demand instances, and up to 50% more than commodity spot instance solutions.
- Data Caching Agent: This agent optimizes the locality of data for fast, interactive access speeds. Data Transporter takes into consideration which data sets are frequently accessed and intelligently moves data in the background for the best performance.
QDS Community
QDS Community is a free product, providing the same functionality as QDS Enterprise. Community is free forever for up to four nodes and five clusters. This solution, according to the company, is ideal for developers interested in building and testing applications before moving to production.
The company states that all the three agents will be available from July 2017.
 has also introduced three new products; QDS Community Edition, QDS Enterprise Edition and QDS Cloud Agents){kind=link}