Tag: In-Memory

Feed: CDH – Cloudera Engineering Blog. Author: Cy Jervis. Analytical and operational access patterns are very different and until now the Hadoop ecosystem has not had a single storage engine that could support both. As a result, engineers have been forced to implement complex architectures that stitch multiple systems together in order to provide these capabilities. On one hand immutable data on HDFS offers superior analytic performance, while mutable data in Apache HBase is best for operational workloads. Apache Kudu bridges this gap.   Kudu’s architecture is shaped towards the ability to provide very good analytical performance, while at the ... Read More

Feed: Featured Blog Posts - Data Science Central. Author: Venkatesan M. 1) Compare Hadoop & Spark                       Criteria                                           Hadoop                                                   Spark Dedicated storage                           HDFS                 ... Read More

Feed: AWS Big Data Blog. This is a guest post by Jeremy Winters and Ritu Mishra, Solution Architects at Full 360. In their own words, “Full 360 is a cloud first, cloud native integrator, and true believers in the cloud since inception in 2007, our focus has been on helping customers with their journey into the cloud. Our practice areas – Big Data and Warehousing, Application Modernization, and Cloud Ops/Strategy – represent deep, but focused expertise.”AWS Glue is a fully managed ETL (extract, transform, and load) service that makes it simple and cost-effective to categorize your data, clean it, enrich ... Read More

Feed: erwin Expert Blog – erwin, Inc.. Author: Kevin Schofield. There’s no escaping data’s role in the cloud, and so it’s crucial that we analyze the cloud’s impact on data modeling.  The fourth post in this series concluded with the observation that more and more managed data lakes are being implemented in the cloud. The second post in this series focused on the Internet of Things (IoT), the constellation of web-connected devices, vehicles, buildings and related sensors and software that are greatly facilitated by the cloud. But what exactly is “the cloud,” and what challenges and opportunities does it present ... Read More

Feed: SAS Blogs. Author: Stephen Foerster. The Cloud Analytic Server (CAS for short) is SAS’ latest high-performance, scalable, in-memory analytic data server.  In this post, I’d like to discuss the CAS physical data model, i.e.what features CAS offers for data storage, and how to use them to maximize performance in CAS (and consequently SAS Visual Analytics 8.1 too). So, specifically let’s answer the question: What CAS physical table storage features can we use to get better performance in CAS and SAS Visual Analytics/CAS? CAS Physical Table Storage Features The following data storage features affect how CAS tables are physically structured: ... Read More

Feed: R-bloggers. Author: Simon Jackson. (This article was first published on blogR, and kindly contributed to R-bloggers) @drsimonj here to share my code for using Welch’s t-test to compare group means using summary statistics.  Motivation I’ve just started working with A/B tests that use big data. Where once I’d whimsically run t.test(), now my data won’t fit into memory! I’m sharing my solution here in the hope that it might help others.  In-memory data As a baseline, let’s start with an in-memory case by comparing whether automatic and manual cars have different Miles Per Gallon ratings on average (using the ... Read More

Feed: Microsoft Azure Blog. Author: Josh Caplan. In April we announced the general availability of Azure Analysis Services, which evolved from the proven analytics engine in Microsoft SQL Server Analysis Services. The success of any modern data-driven organization requires that information is available at the fingertips of every business user, not just IT professionals and data scientists, to guide their day-to-day decisions. Self-service BI tools have made huge strides in making data accessible to business users. However, most business users don’t have the expertise or desire to do the heavy lifting that is typically required, including finding the right sources ... Read More

Feed: MemSQL Blog. Author: Mike Mohler. Data today moves in extremely large volumes at light speed all while requiring high levels of protection on the content. Traditional database and data warehouse technologies are unable to handle these workloads without the need for expensive hardware appliances. The NoSQL and Hadoop-based systems are very complex and unable to support real-time analysis. A new way to process fast-moving, structured content is needed to achieve this level of scale and security. Why not a highly secure and extreme scalable SQL-based approach? Imagine the volume of telemetry data that is required for processing a swarm ... Read More

Feed: Latest imported feed items on Analytics Matters. More than ever, “WOW!” business outcomes are driven by data scientists, and there are many types of data scientist. Everyone can benefit from understanding the ways data scientists explore, analyze and execute strategic business decisions and plans. The 2017 Teradata PARTNERS conference offers many sessions focused on Data Science – so get an instant education on Data Science. Take a look at the options: State of the Art Data Science for the Data Driven Enterprise – Look at Data Science in practice at Verizon Wireless! Learn how to build a data driven ... Read More

Feed: AWS Big Data Blog. Apache Hive is one of the most popular tools for analyzing large datasets stored in a Hadoop cluster using SQL. Data analysts and scientists use Hive to query, summarize, explore, and analyze big data.With the introduction of Hive LLAP (Low Latency Analytical Processing), the notion of Hive being just a batch processing tool has changed. LLAP uses long-lived daemons with intelligent in-memory caching to circumvent batch-oriented latency and provide sub-second query response times. This post provides an overview of Hive LLAP, including its architecture and common use cases for boosting query performance. You will learn ... Read More