Big data has been a buzzword for so long you’d think most companies would have it mastered by now. Yet the promise of big data has mostly fallen out of reach for modern businesses due to the technological complexity behind collecting and analyzing the massive, varied sets of data created on a daily basis.
Decision Moments, Mindtree’s breakthrough decision science platform, can begin to bridge that gap by helping businesses generate meaningful and compelling insights from large pools of data. The Mindtree press release paints a broad overview of Decision Moments and its capabilities, but the tech specialists among you may still be asking, “So what’s under the hood?” This article intends to answer that question.
The platform brings in the right mix of Infrastructure-as-a-Service (IaaS) and Platform-as-a-Service (PaaS) components to strike the ideal balance between agility and speed to market. Leveraging proven designs and frameworks from the big data and data science worlds, the Decision Moments platform is built on the following core architectural principles:
- Multi-tenant platform—to ensure isolation across data stores and processing layers across tenants
- Reactive paradigm—a resilient platform enabling highly engaging apps, with the ability to linearly scale out
- API-first approach—data stores, machine-learning algorithms, business specific services—everything is API-based
- Cost conscious—by leveraging appropriate stores for hot, warm and cold data
- Data security—at rest and in transit, coupled with data encryption and anonymization strategies
- Opex model—leveraging Microsoft Azure PaaS services like HDInsight, Power BI, Event Hub, IoT Hub, SQL DW and Blob storage
- Unified operations tier—a big data ecosystem with an abundance of tools and technologies provides a unified interface for system management and monitoring
Constituent parts of the decision science engine
Beneath the unified operations tier lie several constituent parts that are independent layers responsible for major functions.
The data ingestion layer enables batch, micro-batch and real-time data ingestion. Components of this layer use centralized metadata management to enable flexibility in data transformation and validation processes. This layer also promotes a self-service model to allow changes in ETL processes without the involvement of an engineer team.
The business data lake allows for storage of structured, semi-structured and unstructured data in one place. Customers can choose from different data storage techniques, while various noSQL engines—including document databases, column family-oriented stores, key-value paid databases, and graph-based stores—are seamlessly supported to enable real-time high throughput data read and write requirements. An abstraction layer on top of data stores hides the complexity and enables users to easily switch underlying storage engines.
The data insights tier leverages SparkR and Microsoft Azure capabilities to generate insights through 20 industry-specific algorithms, via both supervised and unsupervised learning techniques. Logistic regression, clustering, collaborative filtering, random decision forest and decision tree, neural network techniques—all are used to power machine-learning algorithms. This tier powers business apps across the value chain, and the models are easily tweaked or extended based on customer requirements and underlying data stores.
The processing tier leverages various established big data architecture patterns like Lambda, Kappa and Zeta to process and mine big data workloads.
Reusable big data components accelerate the development and automate the deployment of big data applications. These include OLAP, hierarchy builders, pre-built data visualization templates, data ingestion and insight delivery services, data lineage tracker, and automated data archival and backup.
And the API layer leverages micro services-based technologies like Node.js or Spring Boot to power highly scalable, loosely coupled, resilient API architecture. These APIs power omnichannel apps and application-administration interfaces.
That’s a lot to fit into one platform—and maybe that’s why it’s taken so long. We’ve been waiting more than a decade for big data to be at our fingertips because analyzing and making use of it is an incredibly complex task. Now that the technology can be properly harnessed, it’s time to get the most out of it.
Reach out to us to start a conversation on how to seamlessly implement Decision Moments with your existing analytics investments.