Category: Cloudera

Feed: Cloud – Cloudera Engineering Blog. Author: Shelby Khan. Fig 1 – Architecture Introduction Many of Cloudera’s customers set up Cloudera Manager to collect their clusters’ diagnostic data on a regular schedule and automatically send that data to Cloudera. Cloudera analyzes this data, identifies potential problems in the customer’s environment, and alerts customers, requiring fewer back-and-forths with our customers when they file a support case and provides Cloudera with critical information to improve future versions of all of Cloudera’s software. If Cloudera discovers a serious issue, Cloudera searches this diagnostic data and proactively notifies Cloudera customers who might encounter problems ... Read More

Feed: Cloudera Engineering Blog. Author: Gabor Roczei. Background Authentication is a basic security requirement for any computing environment. In simple terms, users and services must prove their identity (authenticate) to the system before they can use system features. Kerberos provides strong authentication which is used in the exchange between requesting user or process and service during authentication. When a user authenticates to a particular Hadoop component, the user’s Kerberos principal is presented. The principal is presented in the form user@REALM. The Kerberos principal is mapped [1] to a short name after authentication. For example: user@EXAMPLE.COM --> user user@EXAMPLE.COM --> user ... Read More

Feed: Cloudera Engineering Blog. Author: Bethann Noble. Cloudera Fast Forward Labs is an applied machine learning research and consulting services group within Cloudera, which helps enterprises accelerate data value creation through the adoption of emerging ML techniques, cutting-edge technical architectures and industry leading ML best practices. We focus on expert knowledge transfer and skills development, empowering organizations to continually evolve, differentiate themselves and ultimately own the future of their business by leveraging open technologies and data. Enabling ethical and responsible ML outcomes to our customers, at scale, is our highest priority.  We like to think of ourselves as your data ... Read More

Feed: Cloudera Engineering Blog. Author: Jesus Camacho Rodriguez. The annual ACM SIGMOD/PODS Conference is a leading international forum for database researchers, practitioners, developers, and users to explore cutting-edge ideas and results, and to exchange techniques, tools, and experiences. This year ACM SIGMOD/PODS will be held in Amsterdam, The Netherlands on June 30th – July 5th, 2019, and Cloudera will be present in the conference, contributing to and learning from the broader research community. Last year, Apache Hive was recognized with the SIGMOD Software Systems Award “for developing seminal software systems that served to bring relational-style declarative programming to the Hadoop ... Read More

Feed: Cloudera Engineering Blog. Author: Jeff Fletcher. Once the data science is done (and you know where your data comes from, what it looks like, and what it can predict) comes the next big step: you now have to put your model into production and make it useful for the rest of the business. This is the start of the model operations life cycle. The key focus areas (detailed in the diagram below) are usually managed by machine learning engineers after the data scientists have done their work. ML Engineering includes (but isn’t necessarily limited to): the data pipeline (the ... Read More

Feed: CDH – Cloudera Engineering Blog. Author: Shelby Khan. HDFS erasure coding (EC), a major feature delivered in Apache Hadoop 3.0, is also available in CDH 6.1 for use in certain applications like Spark, Hive, and MapReduce. The development of EC has been a long collaborative effort across the wider Hadoop community. Including EC with CDH 6.1 helps customers adopt this new feature by adding Cloudera’s first-class enterprise support. While previous versions of HDFS achieved fault tolerance by replicating multiple copies of data (similar to RAID1 on traditional storage arrays), EC in HDFS significantly reduces storage overhead while achieving similar ... Read More

Feed: CDH – Cloudera Engineering Blog. Author: Michael Gregory. Disclaimer: the scenario below is hypothetical.   Any similarity to any specific telecommunications company is purely coincidental.   Although we use the example of a telecommunications company the following applies to every organization with customers or voluntary stakeholders.   Introduction Imagine that you are a Chief Data Officer at a major telecommunications provider and the CEO has asked you to overhaul the existing customer churn analytics.  The current process relies on manual export of data from dozens of data sources including ERP, CRM, and Call Detail Record (CDR) databases onto a user’s ... Read More

Feed: CDH – Cloudera Engineering Blog. Author: Krishna Maheshwari. Cloudera recently launched CDH 6.2 which includes two new key features in Apache HBase: Serial replication Bucket cache now supports Intel’s Optane memory Serial replication HBase has a sophisticated asynchronous replication mechanism that supports complex topologies today that include global round-robin, two way, span-in and span-out topologies. This replication capability, to date, provides eventual consistency — meaning that the order in which updates are replicated is not necessarily the same as the order in which they were applied to the database.  While this worked for many customers, order of updates on ... Read More

Feed: CDH – Cloudera Engineering Blog. Author: Tristan Stevens. Cloudera Search is a highly scalable and flexible search solution based on Apache Solr which enables exploration, discovery and analytics over massive, unstructured and semi-structured datasets (for example logs, emails, dna-strings, claims forms, jpegs, xls sheets, etc). It has been adopted by a large number of Cloudera customers across a wide range of industries for high ROI and SLA-bound workloads, with many of those having strict requirements around security and compliance. In CDH6.2 we introduce two new features to Cloudera Search relating to document-level security. The first extends this role-based model ... Read More

Feed: CDH – Cloudera Engineering Blog. Author: Tristan Stevens. Cloudera Search is a highly scalable and flexible search solution based on Apache Solr which enables exploration, discovery and analytics over massive, unstructured and semi-structured datasets (for example logs, emails, dna-strings, claims forms, jpegs, xls sheets, etc). It has been adopted by a large number of Cloudera customers across a wide range of industries for high ROI and SLA-bound workloads, with many of those having strict requirements around security and compliance. In CDH 5.1.0 we introduced document-level security for Cloudera Search using Apache Sentry. This model provided a mechanism where user ... Read More