Tag: In-Memory

Feed: Cloudera Engineering Blog » Hadoop. Author: Lars George. This post is a follow up to an earlier one called Apache Hadoop Log Files: Where to find them in CDH, and what info they contain. It lists nicely the various places Hadoop uses to store log and other info files while it is running. Over time we have seen changes though in different Hadoop and CDH releases that affect where files are stored or how long they are retained. Below strives to document the status quo (no, not the band) of all the log and info files found while running ... Read More

Feed: Planet MySQL. Author: Mark Callaghan. I used Linux perf to get more details on system performance while running the point-query test with modern sysbench. This is for an in-memory workload and part of my series on low-concurrency CPU regressions for bug 86215.tl;drI have more explaining to do The increase in instructions/query explains the decrease in queries/second from MySQL 5.6 to 5.7 to 8.0. ConfigurationI tested MySQL using upstream 5.6.35, 5.7.17 and 8.0.1. For 8.0.1 I used the latin1 charset and latin1_swedish_ci collation. My servers are described here and are named i3 NUC and i5 NUC.My usage of sysbench, including the tests run, ... Read More

Feed: What's New. Amazon DynamoDB Accelerator (DAX) is a fully managed, highly available, in-memory cache for DynamoDB that delivers up to a 10x performance improvement – from milliseconds to microseconds – even at millions of requests per second. DAX does all the heavy lifting required to add in-memory acceleration to your DynamoDB tables, without requiring you to manage cache invalidation, data population, or cluster management. Now you can simply focus on building great applications for your customers without worrying about performance at scale ... Read More

Feed: Database Trends and Applications : All Articles. VIEW FROM THE TOP BY David Flower, CEOVoltDB is the only operational in-memory transactional database designed for modern applications that require an unprecedented combination of data scale, volume, and accuracy. Unlike other databases, including OLTP, Big Data, and NoSQL, VoltDB does not force users to compromise on data consistency, speed, or flexibility.VoltDB supports the three fundamental modern application data requirements. VoltDB processes a relentless volume of data points—millions and billions—every second from users and data sources. VoltDB ingests, analyzes, and acts on data in milliseconds—while ensuring data is always accurate, all the time, ... Read More

Feed: Cloudera Engineering Blog » CDH. Author: Justin Kestelyn. Cloudera has announced support for Spark SQL/DataFrame API and MLlib. This post explains their benefits for app developers, data analysts, data engineers, and data scientists. In July 2015, Cloudera re-affirmed its position since 2013: that Apache Spark is on course to replace MapReduce as the default general-purpose data processing engine for Apache Hadoop. Thanks to initiatives like the One Platform Initiative, Spark continues its march toward comprehensive enterprise readiness, and its adoption is rapidly spreading across industries and diverse use cases. More important, there is continuous and sustained innovation in the ... Read More

Feed: Cloudera Engineering Blog » CDH. Author: Justin Kestelyn. Thanks to Michal Malohlava, Amy Wang, and Avni Wadhwa of H20.ai for providing the following guest post about building ML apps using Sparkling Water and Apache Spark on CDH. The Sparkling Water project is nearing its one-year anniversary, which means Michal Malohlava, our main contributor, has been very busy for the better part of this past year. The Sparkling Water project combines H2O machine-learning algorithms with the execution power of Apache Spark. This means that the project is heavily dependent on two of the fastest growing machine-learning open source projects out ... Read More

Feed: Planet MySQL. Author: Mark Callaghan. As I document the low-concurrency CPU regressions in MySQL 5.7 and 8 one of the questions is whether the performance schema is part of the problem. I don't think it is as the worst-case overhead I measure is about 7% and the typical overhead is less than 5% for a single-threaded & in-memory workload.  Eventually I will repeat this test for workloads with more concurrency because there are overheads that won't be apparent at low-concurrency. The overhead is larger for simple queries, like sysbench point-query, and larger for complex queries. The overhead is also ... Read More

Feed: Hazelcast Blog. Author: Christoph Engelbert. We had the chance to speak with Ben Evans, Java Champion, author for O’Reilly and InfoQ, as well as co-founder of JClarity, a startup which delivers performance tools and services to help development and ops teams. Together with his recent whitepaper on Hazelcast in combination with Spark, he also delivered a small application, BetLeopard, to demonstrate the integration between the two technologies. Chris Engelbert: You’re around in the Java world for quite some time, when did you start with Java and why? Ben Evans: Well, one of the things about me is that I ... Read More

Feed: Hazelcast Blog. Author: Hazelcast Community. DataMountaineer recently published a blog post entitled, “Hazelcast – Low Latency Datastores for IOT“. In the post, DataMountaineer presents a strategy for building low latency datastores for Internet of Things using Kafka and Hazelcast. The Internet of Things is on the rise, it was certainly a buzzword of 2016. Gartner thinks so, they say there will be 20 billion devices online by 2020 with all of them transmitting (streaming) data. These devices are not limited to new devices, more and more we are seeing our clients want to connect into manufacturing control systems such ... Read More

Feed: Cloudera Engineering Blog » CDH. Author: Justin Kestelyn. Our thanks to Databricks, the company behind Apache Spark (incubating), for providing the guest post below. Cloudera and Databricks recently announced that Cloudera will distribute and support Spark in CDH. Look for more posts describing Spark internals and Spark + CDH use cases in the near future. Apache Hadoop has revolutionized big data processing, enabling users to store and process huge amounts of data at very low costs. MapReduce has proven to be an ideal platform to implement complex batch applications as diverse as sifting through system logs, running ETL, computing ... Read More