Tag: Spark

Feed: MemSQL Blog. Author: Floyd Smith. Bill Vorhies of Data Science Central and Rick Negrin of MemSQL recently delivered a webinar on data lakes and the use of Hadoop in the age of operational machine learning (ML) and artificial intelligence (AI). In this blog post, we summarize some of the key points from the webinar, as a preview for the webinar. You can also view the recorded webinar at any point. Businesses need flexible access to their data to make the best decisions. Unfortunately, the systems in use today impose complexity and structural delays. MemSQL, a fast, fully scalable relational ... Read More

Feed: Hortonworks Blog – Hortonworks. Author: Paul Codding. Building on the vision and concepts outlined previously in Arun and Saumitra’s blogs, we wanted to show the Open Hybrid Architecture Initiative (OHAI) concepts in action, and see how they could be used to solve a data-driven business problem. In this example, we’ll use look at how our Open Hybrid Architecture Initiative (OHAI) applies to an enterprise. Hortonworks Air Freight has a business problem to solve. They are a global logistics provider with hubs in San Francisco and Dallas, but they need to plan out their next hub on the east coast and ... Read More

Feed: Microsoft Azure Blog. Author: Rob Caron. PASS Summit v.20 PASS Summit is the largest conference for technical professionals who leverage the Microsoft Data Platform. The 20th summit, which took place in Seattle last week, played host to a number of Azure-related announcements. Harness the future with the ultimate hybrid platform for data and AI Rohan Kumar, Corporate Vice President, Azure Data, gave the Day 1 keynote at PASS Summit v.20. In this post, he provides an overview of his keynote, including how hybrid connects all your data and enables comprehensive AI and analytics. Microsoft’s consistent data platform across on-premises ... Read More

Feed: Yellowbrick Data. Author: Gary Orenstein; Welcome to the Yellowbrick blog! We took some time to sit down with our CEO Neil Carson to get his perspective on the company, our product, and the overall data market. Q: What does Yellowbrick do? We help businesses derive value from their data by giving them the flexibility to deploy analytics anywhere. Specifically we do this by building the Yellowbrick Data Warehouse. Q: What does that look like? It looks like a standard data warehouse and talks the same protocols that interoperate with other popular data warehouse protocols from products like IBM/Netezza, Teradata, ... Read More

Feed: Microsoft Azure Blog. Author: Ashish Thapliyal. Today, we are excited to share our tips and tricks series on how to migrate on-premises Hadoop infrastructure to Microsoft Azure HDInsight. Every day, thousands of customers run their mission-critical big data workloads on Azure HDInsight. Many of our customers migrate workloads to HDInsight from on-premises due to its enterprise-grade capabilities and support for open source workloads such as Hive, Spark, Hadoop,  Kafka, HBase, Phoenix, Storm, and R. This six-part guide takes you through the migration process and shows you not only how to move your Hadoop workloads to Azure HDInsight, but also ... Read More

Feed: CDH – Cloudera Engineering Blog. Author: Tom Wheeler. In mid-2017, we were working with one of the world’s largest healthcare companies to put a new data application into production. The customer had grown through acquisition and in order to maintain compliance with the FDA, they needed to aggregate data in real-time from dozens of different divisions of the company. The consumers of this application, of course, did not care how we built the data pipeline. However, they cared greatly that if it broke, and the end-users did not get their data, they would be out of compliance and could ... Read More

Feed: Planet big data. Author: Benjamin Ensor. In the decade since the global financial crisis, much of the financial services industry has made a strong recovery. While many governments are still nursing large debts, most banks, investment management firms, and insurance companies have long since returned to good health. Growing economies, rising interest rates, and increased trading activity have delivered strong earnings for many financial firms. But there are dark clouds on the horizon. Climate change, volatile politics, trade wars, and relentless cyberattacks all threaten the economic stability on which most financial firms thrive. As they prepare to weather the ... Read More

Feed: AWS Big Data Blog. Many organizations use a setup that includes multiple VPCs based on the Amazon VPC service, with databases isolated in separate VPCs for security, auditing, and compliance purposes. This blog post shows how you can use AWS Glue to perform extract, transform, load (ETL) and crawler operations for databases located in multiple VPCs.The solution presented here uses a dedicated AWS Glue VPC and subnet to perform the following operations on databases located in different VPCs: Scenario 1: Ingest data from an Amazon RDS for MySQL database, transform it in AWS Glue, and output the results to ... Read More

Feed: SAS Blogs. Author: Phil Simon. Read an article: Key questions to kick off your data analytics projects. I tell many of my analytics students that they ought to consider careers in data science. After all, the supply and demand is wildly out of kilter. What's more, this trend shows no signs of abating. It would be silly to contend, though, that data scientists are magicians. Even those who can magically obtain data, analyze it and make recommendations cannot overcome significant organizational impediments. Put differently, many organizations fail to use their data scientists properly or at least set them up ... Read More

Feed: All - O'Reilly Media. Author: O'Reilly Media. Computer and paper on a desk (source: Pexels via Pixabay) Learn new topics and refine your skills with 140 live online training courses we opened up for November, December, and January on our learning platform. Artificial intelligence and machine learning Artificial Intelligence for Big Data, November 28-29 Essential Machine Learning and Exploratory Data Analysis with Python and Jupyter Notebook , December 3 Deep Learning for Machine Vision, December 4 Beginning Machine Learning with Scikit-Learn, December 5 Managed Machine Learning Systems and Internet of Things, December 5-6 Natural Language Processing (NLP) from Scratch, ... Read More