Analytics Genie, a marketplace for Analytics/ Data Science products platforms, is focused on emerging capabilities and highlighting vendors who offer specialized products/ platforms which deliver those capabilities. This article looks at Anomaly Detection platforms, which are rapidly becoming popular across industries and solving complex business problems.
Reports rich and insights poor is a common phenomena.
In today’s data-rich environment businesses are investing a lot in data capture, aggregation, and analysis. With lots of data comes a lot of complexity too, both from data storage/ retrieval and the ability to make sense of it. In the data value chain, the “making sense” phase is the most valuable and drives the ROI for the entire data infrastructure investment. Reporting/ Visual Analytics players have been working hard to make this phase more sophisticated, but the fact that they are generally unidimensional (one variable at a time) is always a challenge.
Any real-life phenomenon is generally multi-dimensional with individual and interaction effects between various factors. Hence use of data science modeling, especially the non-linear machine learning models and deep learning models like neural networks, is essential to discover the patterns behind any phenomenon.
Future Re-imagined:
As a business process owner, imagine you have a system in place that is continuously monitoring your business performance, looking at hundreds of data points collectively and notifying you, if and only if, there is something unusual. The system sends you an alert pro-actively, instead of you having to scan multiple disjointed reports every day.
The good news is that such systems exist today, and they are getting more sophisticated by the day.
Most such systems use Anomaly Detection techniques, which are not new themselves but are being used to solve complex problems across different industries.
Anomaly detection explained:
Anomaly detection explained:
Simply put, Anomalies are outliers and Anomaly detection is generally understood to be the identification of rare items, events, or observations that deviate significantly from most of the data and do not conform to a well-defined notion of normal behavior.
Interestingly, outliers themselves are often indicative of a process issue/bottleneck, that business owners would love to know about proactively.
Applications of Anomaly detection capability:
The cyber security industry has been the leader in using Anomaly detection methods with Intrusion Detection being the prime use case, but the use of Anomaly detection methods is expanding to other application areas.
The cyber security industry has been the leader in using Anomaly detection methods with Intrusion Detection being the prime use case, but the use of Anomaly detection methods is expanding to other application areas.
As the industry matures, Analytics Genie predicts a verticalization of solutions with Fintech, Healthcare, and Manufacturing being the focus industries. The use of Anomaly detection using images and videos will also expand dramatically and bring new solutions to age-old problems.
For wider business applications, detecting an Anomaly isn’t enough:
For an Anomaly detection platform to function effectively, detecting an anomaly is not enough. False positives can kill interest in the platform very swiftly. There is hence a need for a whole business workflow that detects, broadcasts, tracks, and summarizes business actions taken to resolve anomalies, with a feedback loop from both false positives and true negatives.
A solid anomaly detection platform needs to have the following components:
- Seamless access to data.
- Rich library of algorithms to detect an Anomaly
- A reporting/ alerting layer
- A workflow where business users can investigate/ record and report the investigation of anomalies.
- A learning system that identifies false positives and true negatives.
- Integration into the overall IT ticketing system
Building an Anomaly Detection Platform:
With advances in cloud technology and open-source data science libraries, it is possible to build a custom Anomaly detection platform. AWS, AZURE, GCP, and other major cloud players already provide Anomaly detection capabilities on their own, or through their partner networks. However, building a comprehensive platform is not trivial and needs a large investment commitment for multiple years.
While for some, the custom build may make sense, most organizations are better off buying a platform off the shelf and then customizing it for their needs. However, as is the case with many use cases in AI today, as verticalization of use cases emerges, so will the conflict on IP.
Anomaly Detection Platforms: The differentiation
While the algorithms for detecting outliers are the core of any platform, the science behind that is commoditized, with most ML frameworks providing an assembly of models readily. As the verticalization of the Anomaly detection platforms takes place, pre-trained models, transfer learning, industry benchmarks, data classification, etc. will be the primary differentiators.
Currently, however, the anomaly detection to action “workflow” is where a dedicated anomaly detection platform differentiates from a generic data science platform.
Analytics Genie has identified a few players, who we believe have good Anomaly detection platforms and can be used for broader (non-IT) business applications.
Key Players in the Anomaly Detection capability:
ANODOT: Founded in 2014 and headquartered in the US, Anodot has built out a comprehensive platform with patented detection methods, a wide range of APIs, and integration with business workflows. Anodot seems to be verticalizing its platform, which signals maturity and experience in solving real business problems. Check out Anodot’s profile at Analytics Genie
CHOOCH: Founded in 2015, this Silicon Valley company focuses on Computer vision and focuses on defect detection using images and videos. Chooch has served a wide spectrum of industries and use cases. Check out Chooch’s profile at Analytics Genie
TIBCO: TIBCO has brought three of its products, TIBCO Spotfire, TIBCO Streaming, and TIBCO Data Science to provide real-time monitoring and diagnosis of anomalies. They have also expanded to multiple use cases. Check out TIBCO’s profile at Analytics Genie
EVENTUS: Eventus has focused solely on trade surveillance and risk monitoring solutions and has combined domain expertise and technology to create a niche product. They have built all the major components of an Anomaly detection system that includes generation and action on alerts using RPA. Check out Eventus’s profile at Analytics Genie
ANOMALO: Founded in 2018, Anomalo, is focusing on data quality/ integrity checks in large data environments. Their recent partnership with Databricks gives them a wider audience. Check out Anomalo’s profile at Analytics Genie
Disclaimer: This is not an exhaustive list, and the research is based on publicly available information on the organizations below. Analytics Genie is not responsible for any misrepresentation of facts in the publicly available information on the vendors above.
Leave a Reply