Tag: Data Lake

Feed: AWS Big Data Blog. AWS Glue is an increasingly popular way to develop serverless ETL (extract, transform, and load) applications for big data and data lake workloads. Organizations that transform their ETL applications to cloud-based, serverless ETL architectures need a seamless, end-to-end continuous integration and continuous delivery (CI/CD) pipeline: from source code, to build, to deployment, to product delivery. Having a good CI/CD pipeline can help your organization discover bugs before they reach production and deliver updates more frequently. It can also help developers write quality code and automate the ETL job release management process, mitigate risk, and more.AWS ... Read More

Feed: Microsoft Azure Blog. Author: Arpan Shah. Today we’re at RSA, and we are delighted to sponsor and participate in this industry event centered in security. I thought I’d take the opportunity to share our perspective on cloud security with Azure. As we all know, companies worldwide are challenged by the ongoing volume of evolving security threats and with retaining qualified security talent to respond to these threats. In fact, the average large organization gets 17,000 security alerts each week, which results in an of average 99 days to discover security breaches. That contrasts with the less than 48 hours ... Read More

Feed: Hortonworks Blog – Hortonworks. Author: Matt Spillar. Early last year we announced the Hortonworks Data Heroes initiative. It’s our way of recognizing the Data Visionaries, Data Scientists, and Data Architects transforming their businesses and organizations through Big Data. Hortonworks has over 1,300 total customers spanning every industry, each with a unique Big Data journey. Our Data Heroes program lets us highlight a few customers who are helping create radical change in their industries. WHO ARE THE DATA HEROES? A Data Visionary:  who recognizes the potential of Big Data to transform their organization, and leads the business to turn that vision into reality. Winners will show ... Read More

Feed: Hortonworks Blog – Hortonworks. Author: Matt Spillar. We’re less than two weeks away from DataWorks Summit Berlin (April 16-19)! We have a number of impressive keynote and breakout speakers lined up. These speakers include Ian Pillay and Bradley Smith from Standard Bank, and Jeroen Wolffensperger and Martijn Groen from Rabobank. Standard Bank Standard Bank South Africa is one of South Africa’s largest financial services groups, operating in 20 countries across Africa and other key markets around the world. It is Africa’s biggest lender by assets, offering a range of banking and related financial services. Ian Pillay and Bradley Smith will be speaking ... Read More

Feed: Hortonworks Blog – Hortonworks. Author: Saumitra Buragohain. Source: Nvidia Blog Into the Woods: This Drone Goes Where No GPS Can Use Cases When we are in the outdoors, many of us often feel the need for a camera- that is intelligent enough to follow us, adjust to the terrain heights and visually navigate through the obstacles, while capturing panoramic videos.  Here, I am talking about autonomous self-flying drones, very similar to cars on auto pilot. The difference is that we are starting to see proliferation of artificial intelligence into affordable, everyday use cases, compared to relatively expensive cars. These ... Read More

Feed: Hortonworks Blog – Hortonworks. Author: Michael Lin. For things that are dearest, most important and valuable to us, we come up with ways to protect them. Insurance policies, laws, to even safe deposit boxes and the hiring of security guards are all means to safeguard whatever deemed precious to us. The introduction of General Data Protection Regulation (GDPR) protects valuable personal information of all individuals within the European Union, as the power of data is beyond imagination in present time. A timely case in point is the recent turmoil at Facebook. The data of more than 50 million Facebook users ... Read More

Feed: Microsoft Azure Blog. Author: Anu Kohli. Data science and machine learning can be applied to solve many common business scenarios, yet there are many barriers preventing organizations from adopting them. Collaboration between data scientists, data engineers, and business analysts and curating data, structured and unstructured, from disparate sources are two examples of such barriers - and we haven’t even gotten to the complexity involved when trying to do these things with large volumes of data.   Recommendation engines, churn analysis, and intrusion detection are common scenarios that many organizations are solving across multiple industries. They require machine learning, streaming analytics, ... Read More

Feed: AWS Big Data Blog. Trying something new can often be a daunting task. Where do you start? What resources are available to help guide you through unfamiliar territory? Where can you go if you need additional help? Whether you’ve just signed up for your first AWS account or you’ve been with us for some time, there’s always something new to learn as our services evolve to meet the ever-changing needs of our customers. To help ensure you’re set up for success as you build with AWS, we put together this quick reference guide for Big Data training and resources ... Read More

Feed: Planet PostgreSQL. Presented by: Dr. Kristin Tufte When: 6-8pm Thursday April 19, 2018   This talk will describe work in Smart Cities in the Portland region. We’ll begin with the framework and motivation for the Smart Cities work and the question What is a Smart City? We’ll discuss Portland’s approach to Smart Cities, provide some historical context and then give an overview of ongoing Smart Cities projects including work on AV policy, the Portland Urban Data Lake, new sensors and earthquake resilience. The goal of the talk is to give the audience an overview of the work being done ... Read More

Feed: R-bloggers. Author: quintuitive. (This article was first published on R – Quintuitive, and kindly contributed to R-bloggers) Recently, while working on the Azure Data Lake R extension, I had to figure out a good way to create a zip file containing a package together with all its dependencies. This came down to understanding where does R store and search for packages. Despite the documentation, it did require additional reading and experimentation. First, the accompanying video: Before getting into package search paths, let’s first figure out how does an R package look in the file system: An R package is ... Read More