Tag: Cache

Feed: Cloudera Engineering Blog » Hadoop. Author: Tom White. Apache Hadoop 0.21.0 was released on August 23, 2010. The last major release was 0.20.0 in April last year, so it’s not surprising that there are so many changes in this release, given the amount of activity in the Hadoop development community. In fact, there were over 1300 issues fixed in JIRA (Common, HDFS, MapReduce), the issue tracker used for Apache Hadoop development. Bear in mind that the 0.21.0 release, like all dot zero releases, isn’t suitable for production use. With such a large delta from the last release, it is ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: John Kreisa. This post was contributed by Jonathan Seidman from Orbitz. Jonathan is a Lead Engineer on the Intelligent Marketplace/Machine Learning team at Orbitz Worldwide . You can hear more from Jonathan at Hadoop World October 12th in NYC. Orbitz Worldwide (NYSE:OWW) is composed of a global portfolio of online consumer travel brands including Orbitz, Cheaptickets, The Away Network, ebookers and HotelClub, Additionally, the company operates business-to-business service: Orbitz Worldwide Distribution provides third parties such as Amtrak, Delta, LAN, KLM, Air France and a number of other leading airlines hotel booking capabilities, and Orbitz for ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Chad Metcalf. One of the more common requests we receive from the community is to package Apache HBase with Cloudera’s Distribution for Apache Hadoop. Lately, I’ve been doing a lot of work on making Cloudera’s packages easy to use, and recently, the HBase team has pitched in to help us deliver compatible HBase packages. We’re pretty excited about this, and we’re looking forward to your feedback. A big thanks to Andrew Purtell, a Senior Architect at TrendMicro and HBase Contributor, for leading this packaging project and providing this guest blog post. -Chad Metcalf ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Tom White. This piece is based on the talk “Practical MapReduce” that I gave at Hadoop User Group UK on April 14. 1. Use an appropriate MapReduce language There are many languages and frameworks that sit on top of MapReduce, so it’s worth thinking up-front which one to use for a particular problem. There is no one-size-fits-all language; each has different strengths and weaknesses. Java: Good for: speed; control; binary data; working with existing Java or MapReduce libraries. Pipes: Good for: working with existing C++ libraries. Streaming: Good for: writing MapReduce programs in ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Christophe Bisciglia. Welcome to the first guest post on the Cloudera blog. The other day, we saw Toby from Swingly tweeting about using Apache Hadoop to process millions of other tweeters’ tweets. We were curious, and Toby put together a great writeup about how they use Hadoop to crunch data. We have a few other guest posts in the pipeline, but if you are doing something really fun with Hadoop and want to share, we’d love to hear from you. Get in touch! -Christophe How can you run hundreds of memory intensive annotation tasks ... Read More

Feed: Planet MySQL. Author: Wagner Bianchi. Most of the time when you start a database design you don’t imagine how your applications need to scale. Sometimes, you need to shard your databases among some different hosts and then, on each shard you want to split reads and writes between master and slaves. This blog is about MariaDB MaxScale being able to handle different databases across shards, and splitting up reads and writes into each of the shards to achieve the maximum level of horizontal scalability with MariaDB Server. After reading this blog you will be able to: Know how MariaDB ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Aaron Kimball. Configuring a Hadoop cluster is something akin to voodoo. There are a large number of variables in hadoop-default.xml that you can override in hadoop-site.xml. Some specify file paths on your system, but others adjust levers and knobs deep inside Hadoop’s guts. Unfortuately, there’s little or no documentation on how to set them well. Is there a single optimal configuration? Are there some settings that can just be “set to 11?” At Cloudera, we’re working hard to make Hadoop easier to use and to make configuration less painful. Our Hadoop Configuration Tool ... Read More

Feed: Microsoft Azure Blog. Author: John Deutscher. This month we are announcing the release of support for Azure Active Directory (AAD) authentication in Azure Media Services. Customers of our REST API and .NET client libraries can now use AAD authentication to authorize requests. In addition, we are releasing a new management blade in the Azure Portal to simplify the usage of User and Service Principal authentication with AAD. With the release of this update to our REST API, we are now able to provide the same role-based access management (RBAC) as provided by the Azure Resource Management (ARM) service. By ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Jeff Hammerbacher. It is common for a MapReduce program to require one or more files to be read by each map or reduce task before execution. For example, you may have a lookup table that needs to be parsed before processing a set of records. To address this scenario, Hadoop’s MapReduce implementation includes a distributed file cache that will manage copying your file(s) out to the task execution nodes. The DistributedCache was introduced in Hadoop 0.7.0; see HADOOP-288 for more detail on its origins. There is a great deal of existing documentation for ... Read More

Feed: What's New. Amazon DynamoDB Accelerator (DAX) is a fully managed, highly available, in-memory cache for DynamoDB that delivers up to a 10x performance improvement – from milliseconds to microseconds – even at millions of requests per second. DAX does all the heavy lifting required to add in-memory acceleration to your DynamoDB tables, without requiring you to manage cache invalidation, data population, or cluster management. Now you can simply focus on building great applications for your customers without worrying about performance at scale ... Read More