Category: Cloud Offerings

Feed: Cloudera Engineering Blog » Hadoop. Author: Chad Metcalf. One of the more common requests we receive from the community is to package Apache HBase with Cloudera’s Distribution for Apache Hadoop. Lately, I’ve been doing a lot of work on making Cloudera’s packages easy to use, and recently, the HBase team has pitched in to help us deliver compatible HBase packages. We’re pretty excited about this, and we’re looking forward to your feedback. A big thanks to Andrew Purtell, a Senior Architect at TrendMicro and HBase Contributor, for leading this packaging project and providing this guest blog post. -Chad Metcalf ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Amr Awadallah. (guest blog post by Pete Skomoroch) In a previous post, I outlined how to build a basic trend tracking site called trendingtopics.org with Cloudera’s Distribution for Hadoop and Hive.  TrendingTopics uses Hadoop to identify the top articles trending on Wikipedia and displays related news stories and charts.  The data powering the site was pulled from an Amazon EBS Wikipedia Public Dataset containing 8 months of hourly pageview logfiles.  In addition to the pageview logs, the EBS data volume also includes the full text content and link graph for all articles.  This ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Alex Loddengaard. Apache Hadoop’s jobtracker, namenode, secondary namenode, datanode, and tasktracker all generate logs. That includes logs from each of the daemons under normal operation, as well as configuration logs, statistics, standard error, standard out, and internal diagnostic information. Many  users aren’t entirely sure what the differences are among these logs, how to analyze them, or even how to handle simple administrative tasks like log rotation.  This blog post describes each category of log, and then details where they can be found for each Hadoop component. The log categories are: Hadoop Daemon Logs ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Matt Massie. In March of this year, we released our distribution for Apache Hadoop.  Our initial focus was on stability and making Hadoop easy to install. This original distribution, now named CDH1, was based on the most stable version of Apache Hadoop at the time:0.18.3. We packaged up Apache Hadoop, Pig and Hive into RPMs and Debian packages to make managing Hadoop installations easier.  For the first time ever, Hadoop cluster managers were able to bring up a deployment by running one of the following commands depending on your Linux distribution: # yum ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Christophe Bisciglia. It’s been a crazy few weeks here at Cloudera, and while there is no sign of things letting up before Hadoop World: NYC 2009 on October 2nd, we wanted to take a minute to share the latest details about the speakers, and to say thanks to our sponsors who have recently come on board. We’re absolutely thrilled to have such a wide variety of organizations sharing their experiences with Apache Hadoop. In addition to a deeply technical track headlined by Cloudera, Yahoo! and Facebook, those new to Hadoop will appreciate an ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Amr Awadallah. At Cloudera, we frequently work with leading Hadoop developers to produce guest blog posts of general interest to the community. We started a project with Pete Skomoroch a while back, and we were so impressed with his work, we’ve decided to bring Pete on as a regular guest blogger. Pete can show you how to do some pretty amazing things with Hadoop, Pig and Hive and has a particular bias towards Amazon EC2. With that, I’m happy to welcome Pete to the blog, and hope you enjoy his first post as much ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Christophe Bisciglia. On June 10th, more than 750 people from around the world descended on the Santa Clara Marriott to share their love for a little stuffed elephant named Hadoop. It was a good week to be part of this exploding community, and I want to extend Cloudera’s heartfelt thanks to everyone who made it possible, especially our friends at Yahoo! who organized this Summit. Most importantly, I want to thank all of you who were able to participate. I know many of you couldn’t make it to California this time, so I ... Read More

Feed: AWS Big Data Blog. Amazon Redshift Spectrum enables you to run Amazon Redshift SQL queries against data that is stored in Amazon S3. With Amazon Redshift Spectrum, you can extend the analytic power of Amazon Redshift beyond the data that is stored on local disks in your data warehouse. You can query vast amounts of data in your Amazon S3 “data lake” without having to go through a tedious and time-consuming extract, transfer, and load (ETL) process. Amazon Redshift Spectrum applies sophisticated query optimization and scales processing across thousands of nodes to deliver fast performance.In this blog post, we have ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Tom White. This piece is based on the talk “Practical MapReduce” that I gave at Hadoop User Group UK on April 14. 1. Use an appropriate MapReduce language There are many languages and frameworks that sit on top of MapReduce, so it’s worth thinking up-front which one to use for a particular problem. There is no one-size-fits-all language; each has different strengths and weaknesses. Java: Good for: speed; control; binary data; working with existing Java or MapReduce libraries. Pipes: Good for: working with existing C++ libraries. Streaming: Good for: writing MapReduce programs in ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Christophe Bisciglia. There’s been a lot of buzz about Apache Hadoop lately. Just the other day, some of our friends at Yahoo! reclaimed the terasort record from Google using Hadoop, and the folks at Facebook let on that they ingest 15 terabytes a day into their 2.5 petabyte Hadoop-powered data warehouse. But many people still find themselves wondering just how all this works, and what it means to them. We get a lot of common questions while working with customers, speaking at conferences, and teaching new users about Hadoop. If you, or folks ... Read More