Tag: workflow

Feed: R-bloggers. Author: Nicholas Tierney. Take a look at the data This is a phrase that comes up when you first get a dataset. It is also ambiguous. Does it mean to do some exploratory modelling? Or make some histograms, scatterplots, and boxplots? Is it both? Starting down either path, you often encounter the non-trivial growing pains of working with a new dataset. The mix ups of data types – height in cm coded as a factor, categories are numerics with decimals, strings are datetimes, and somehow datetime is one long number. And let's not forget everyone's favourite: missing data ... Read More

Feed: AWS Partner Network (APN) Blog. Author: Karthik Thirugnanasambandam. AWS Service Catalog allows organizations to centrally manage commonly deployed IT services, achieve consistent governance, and help meet compliance requirements. AWS Service Catalog provides a standardized landscape for product provisioning. Users browse listings of products (services or applications) that they have access to, locate the product that they want to use, and launch it on their own as a provisioned product. The AWS Service Catalog API also provides programmatic control over all user actions.Let’s say you need to build an approval workflow for a launch request from a user. Many solutions ... Read More

Feed: What's New. Previously, Systems Manager State Manager could not trigger Amazon CloudWatch Events. Now, you can use CloudWatch Events to receive notifications when the status of a Systems Manager State Manager association or instance changes. For example, you can be notified if an association is in a failed state and use an AWS Lambda function to stop that instance and start a new one. You can also trigger a Systems Manager Automation workflow, helping you to quickly respond to system events. Automation workflows can be triggered on a fixed schedule or by a change in your AWS environment. Changes ... Read More

Feed: Featured Blog Posts - Data Science Central. Author: Jesus Ramos. Data Denialism A common scenario that data analysts in general encounter is what I like to describe as "data denialism". Often, and especially while consulting, an analyst will find that the data tells a different story than what the customer holds to be true. It is also often the case that, when presenting this finding, the customer will outright deny the evidence, asserting that either the data or the analysis must be wrong. For example, it may be that a retailer focused on the low-end market is getting most ... Read More

Feed: CDH – Cloudera Engineering Blog. Author: Justin Kestelyn. CDH 5.7 includes a lot of changes (more than 1,500) to Hue, the Web UI that makes Apache Hadoop easier to use. In this new release, the emphasis on performance and security carries over from 5.5. The overall improvement in the SQL user experience is also considerable. In this post, we’ll cover some highlights. New Hive Metastore Interface This app is now on a single page, offering more speed and easier access to statistics and data previews. Read about more of the improvements below; or, watch a demo of all of them in action ... Read More

Feed: R-bloggers. Author: msuzen. Preamble There is a lot of confusion among practitioners regarding the concept of overfitting. It seems like, a kind of an urban legend or a meme, a folklore is circulating in data science or allied fields with the following statement:Applying cross-validation prevents overfitting and a good out-of-sample performance, low generalisation error in unseen data, indicates not an overfit. This statement is of course not true: cross-validation does not prevent your model to overfit and good out-of-sample performance does not guarantee not-overfitted model. What actually people refer to in one aspect of this statement is called overtraining. Unfortunately, this meme is not only ... Read More

Feed: Microsoft Azure Blog. Author: Corey Sanders. Most modern applications are built using events – whether it is reacting to changes coming from IoT devices, responding to user clicks on mobile apps, or initiating business processes from customer requests. With the growth of event-based programming, there is an increased focus on serverless platforms, like Azure Functions, a serverless compute engine, and Azure Logic Apps, a serverless workflow orchestration engine. Both services enable you to focus on your application without worrying about any infrastructure, provisioning, or scaling. Today, I am excited to announce that we are making event-based and serverless applications ... Read More

Feed: CDH – Cloudera Engineering Blog. Author: Justin Kestelyn. Thanks to Karthik Vadla, Abhi Basu, and Monica Martinez-Canales of Intel Corp. for the following guest post about using CDH for cost-effective processing/indexing of DICOM (medical) images. Medical imaging has rapidly become the best non-invasive method to evaluate a patient and determine whether a medical condition exists. Imaging is used to assist in the diagnosis of a condition and, in most cases, is the first step of the journey through the modern medical system. Advances in imaging technology has enabled us to gather more detailed, higher-resolution 2D, 3D, 4D, and microscopic ... Read More

Feed: AWS Compute Blog. Author: Christie Gifrin. Sam Dengler, Amazon Web Services Solutions Architect Serverless architectures allow solution builders to focus on solving challenges particular to their business, without assuming the overhead of managing infrastructure in AWS. AWS Lambda is a service that lets you run code without provisioning or managing servers.When using Lambda in a serverless architecture, the goal should be to design tightly focused functions that do one thing and do it well. When these functions are composed to accomplish larger goals in microservice architectures, the complexity shifts from the internal components to the external communication between components ... Read More

Feed: R-bloggers. Author: David Smith. Timo Grossenbacher, data journalist with Swiss Radio and TV in Zurich, had a bit of a surprise when he attempted to recreate the results of one of the R Markdown scripts published by SRF Data to accompany their data journalism story about vested interests of Swiss members of parliament. Upon re-running the analysis in R last week, Timo was surprised when the results differed from those published in August 2015. There was no change to the R scripts or data in the intervening two-year period, so what caused the results to be different? Image credit: Timo ... Read More