Tag: BigInsights

Feed: Hadoop Dev. Author: Shay Roe. IntroductionAutomatic synchronization of the Hive metastore and Big SQL catalog was introduced in Big SQL 4.2 and is a significant enhancement to how Big SQL manages its catalog tables. With this feature enabled, Big SQL will automatically synchronize Hive metastore changes into the Big SQL catalog, so that, any Hive DDL operations (CREATE, ALTER, DROP), will be automatically reflected in the Big SQL catalog. If a new table is created in Hive, for example, that table will automatically be available in Big SQL. This blog is the first in a three-part series that will ... Read More

Feed: Hadoop Dev. Author: Vinayak Agrawal. Today (Aug 11, 2017) we are announcing the GA of IBM Big Replicate 2.1.1 ... Read More

Feed: Hadoop Dev. Author: Linda.Liu. Introduction In a previous post, it demonstrated how to configure LDAP integration with IBM Open Platform on a BigInsights Cluster. Here is the link.In this post, it concentrates on the missing content from the previous post for the Microsoft Active Directory (AD) integration. Objective This technical document is intended to show viewers step by step instructions on how to setup AD on RedHat and/or CentOS Operating System and the integration on a Hadoop Cluster. Version Tested RedHat v7.x, CentOS v7.x Ambari v2.4.x Lessons Learned Don’t include spaces in a bind id (ex. “Service, EnterpriseLdap”) AD ... Read More

Feed: Hadoop Dev. Author: DONGYINGJIAO. Apache Oozie is a workflow scheduler that is used to manage Apache Hadoop jobs. Oozie combines multiple jobs sequentially into one logical unit of work as a directed acyclic graph (DAG) of actions. Oozie is reliable, scalable, extensible, and well integrated with the Hadoop stack, with YARN as its architectural center. It provides several types of Hadoop jobs out of the box, such as Java map-reduce, Pig, Hive, Sqoop, SSH, and DistCp, as well as system-specific jobs, such as Java programs and shell scripts. Apache Spark is a fast general purpose cluster computing system. It ... Read More

Feed: Hadoop Dev. Author: HuiHugoCao. The following directions detail the manual installation of software into IBM Open Platform for Apache Hadoop. These directions, and any binaries that may be provided as part of this article (either hosted by IBM or otherwise), are provided for convenience and make no guarantees as to stability, performance, or functionality of the software being installed. Product support for this software will not be provided (including upgrade support for either IOP or the software described). Questions or issues encountered should be discussed on the BigInsights StackOverflow forum or the appropriate Apache Software Foundation mailing list for ... Read More

Feed: Hadoop Dev. Author: Nailah Bissoon. Adding logical Big SQL workers should be done in non-peak hours when there are no Big SQL applications running on the system. Big SQL maybe restarted one or more times so that the underlying memory can be shuffled correctly within each compute node during logical worker operations. An introduction to this feature is presented in Logical Big SQL Workers Boost Performance. Important: Add and reconfigure logical Big SQL workers when all compute nodes (or Big SQL Workers from Ambari) are active. In ‘Add Services Wizard’->‘Customize Services’-> ‘Advanced bigsql-env’ drop down menu there are 2 ... Read More

Feed: Hortonworks Blog – Hortonworks. Author: Frank Mong. Big Data was challenged from the day it was coined…it’s a problem looking for an answer. Likewise, data science presents an amazing opportunity to answer our biggest questions, but its fate has been tied to the breadth, quantity, freshness and speed of the data it relies upon. For years, these competing issues have been left for organizations to solve on their own. No more. Hortonworks executed a major coup in the battle for big data dominance today with IBM selecting the Hortonworks Data Platform (HDP) distribution for their existing and future customers ... Read More

Feed: Featured Blog Posts - Data Science Central. Author: Raj Dalal. Among the marquee brands that a data science professional would like to join, Uber is likely to be top of many lists. But do you really have what it takes to become a Uberite? BigInsights Principal Raj Dalal met up with Uber’s Chief Data Architect M C Srivas on a recent visit to San Francisco, where, in the course of the hour-long conversation, Srivas spoke of what data analytics meant for Uber, and how data innovation was being used to further, what is now popularly known around the world ... Read More

Feed: Hadoop Dev. Author: QinLiu. This blog provides you with steps to add metric widgets for a custom service in Ambari. It assumes that you have a running cluster with the Metrics Collector started and have the working custom service defined in the stack but not installed yet. It will cover the steps to add metric widgets to the service dashboard, collect metrics, push the metrics to the Metrics Collector, and deploy the service having metric widgets on the service dashboard. In this example, VSFTPD in IOP 4.2 will be used as the custom service and temp_disk_usage as the metric ... Read More

Feed: Hadoop Dev. Author: George_Mathew. IBM® Open Platform (IOP) 4.3 with Apache Hadoop based on Ambari, includes a new pack of serviceability tools to assist on the deployment of the product, maintenance and diagnosis. These serviceability features will help customers to facilitate the installation of the product, as well as technical support personnel to identify operations causing downtime or disruption and restore the product into service. Requirements The script should be run from the Ambari server node. Passwordless ssh has to be setup from Ambari server node to the client nodes as detailed in Knowledge Center Documentation (Procedure → Step ... Read More