Tag: datasets

Feed: Featured Blog Posts - Data Science Central. Author: Robert R. Tucci. Recently, someone (not me) suggested in a tweet to Judea Pearl, let's have a Kaggle** competition on the topic of Causal AI. I love the idea, so I am trying to promote it with this blog post. Big Pharma could fund such a competition for much less than what they pay for a TV commercial, and the technical insights they might gain from this exercise might prove to be invaluable to the pharmaceutical industry. (Plus it would be good publicity for them, much better than a lame TV ... Read More

Feed: Recent Announcements. For retailers specifically, not all SKUs are treated equally. Usually 80% of revenue is driven by 20% of SKUs, and retailers look to optimize forecasting accuracy for those top 20% SKUs. Although you can create a separate forecasting model for the top 20% SKUs, the model’s ability to learn from relevant items outside of the top 20% is limited and accuracy may suffer. Further, retailers may prefer to over-stock certain SKUs or under-stock others, and would prefer to train one model but assess forecasting accuracy for different SKUs at different stocking levels.  To evaluate forecasting accuracy at ... Read More

Feed: AWS Architecture Blog. As businesses race to digitally transform, the challenge is to cope with the amount of data, and the value of that data diminishes over time. The challenge is to analyze, learn, and infer from real-time data to predict future states, as well as to detect anomalies and get accurate results. In this blog post, we’ll explain the architecture for a solution that can achieve real-time inference on streaming data. We’ll also cover the integration of Amazon Kinesis Data Analytics (KDA) with Apache Flink to asynchronously invoke any underlying services (or databases). Managed real-time in-stream data inference ... Read More

Feed: R-bloggers. Author: Dario Radečić. Business Intelligence (BI) is used to transform data into actionable insights that provide value to an organization and help achieve its business goals. Reports and dashboards are go-to approaches for modern-day business intelligence tools. At Appsilon, we are big advocates of R and R Shiny, but we also have significant experience with business intelligence tools. Besides, coding custom solutions from scratch (or coding in general) isn’t always the right solution, especially for simple tasks. The custom-built BI solutions have their place, but more on that towards the end of the article. Are you an Excel ... Read More

Feed: CronJ. Author: Nupur Pal; People travel a lot these days. They are often running to visit a place or meet a client. Did you know? The UNWTO states that near to 1.45 billion people have traveled in 2019! And the numbers will reach 2 billion by the end of this year! This pressurizes airports and other transportation modes greatly. But ever wondered how they provide such advanced security despite such a rush? Well they have a helping hand of course! And it’s none other than AI solutions! Airports are preferred to other modes due to their efficiency. They use ... Read More

Feed: AWS Big Data Blog. In May 2020, we announced the general availability of real-time anomaly detection for Elasticsearch. With that release we leveraged the Random Cut Forest (RCF) algorithm to identify anomalous behaviors in the multi-dimensional data streams generated by Elasticsearch queries. We focused on aggregation first, to enable our users to quickly and accurately detect anomalies in their data streams. However, consider the example data in the following table. timestamp avg. latency region 12:00 pm 3.1 Seattle 12:00 pm 4.1 New York 12:00 pm 5.9 Berlin 12:01 pm 2.6 Seattle 12:01 pm 5.3 New York 12:01 pm 5.8 ... Read More

Feed: Recent Announcements. Supporting high cardinality data streams is challenging due to its computational and scale demands to handle the complexity of the data. This problem is further exacerbated when operators need to pinpoint anomalous entities from massive datasets in near real-time. Amazon Elasticsearch Service uses a distributed architecture to address the challenges with performance and scalability by disseminating the load across the instances in your domain. Anomaly detection for high cardinality datasets is powered by the Random Cut Forest (RCF) algorithm, a light weight unsupervised machine learning algorithm, proven in production to handle multi-dimensional data streams at scale. The ... Read More

Feed: AWS Big Data Blog. AWS Glue provides a serverless environment to prepare (extract and transform) and load large amounts of datasets from a variety of sources for analytics and data processing with Apache Spark ETL jobs. This posts discusses a new AWS Glue Spark runtime optimization that helps developers of Apache Spark applications and ETL jobs, big data architects, data engineers, and business analysts scale their data processing and batch jobs running on AWS Glue automatically. Customers use Spark for a wide variety of ETL and analytics workloads on datasets with diverse characteristics. They want to ensure fast and ... Read More

Feed: R-bloggers. Author: Nathaniel Schmucker. Introduction This is part 1 of a 4 part series on how to build maps using R. How to load geospatial data into your workspace and prepare it for visualization How to make static maps using ggplot2 How to make interactive maps (pan, zoom, click) using leaflet How to add interactive maps to a Shiny dashboard One of the benefits of R is its large ecosystem of packages developed and maintained by individuals and teams, and available to the general public via CRAN. The downside of this, however, is that it is sometimes difficult for ... Read More

Feed: Featured Blog Posts - Data Science Central. Author: Avinash Udaykumar. Consistent asset health across many levels from cell sites to regions is critical to ensure uninterrupted operations for telecom operators. However, proactively identifying anomalous patterns like equipment malfunction remains a major challenge. Drawing on Subex’s deep implementation experience, this paper describes the chief steps necessary to deploy successful pattern detection solutions. It also examines some high impact use cases for pattern detection that have systematically delivered value for telecom carriers. High operational efficiency is an important differentiator for telecom operators establishing their brand reputation on consistency and service availability ... Read More