Tag: Container

Feed: Hadoop – Cloudera Engineering Blog. Author: Sandy Ryza. Cloudera Manager lets you add a YARN service in the same way you would add any other Cloudera Manager-managed service. In Apache Hadoop 2, YARN and MapReduce 2 (MR2) are long-needed upgrades for scheduling, resource management, and execution in Hadoop. At their core, the improvements separate cluster resource management capabilities from MapReduce-specific logic. They enable Hadoop to share resources dynamically between MapReduce and other parallel processing frameworks, such as Cloudera Impala; allow more sensible and finer-grained resource configuration for better cluster utilization; and permit Hadoop to scale to accommodate more and ... Read More

Feed: IBM Big Data & Analytics Hub - All Content; Author: cindy-russell; It seems that we’re reaching the point where the Internet of Things (IoT) is moving from the domain of enthusiastic early-adopters to the more challenging, more profitable territory of mainstream enterprise technology.In particular, companies in industries that depend on the attention and loyalty of large numbers of consumers – retail and air travel, for example –have already woken up to the possibilities. If you can recognize an individual customer’s mobile device as they move through an airport or walk past your store, you can potentially take action to improve that ... Read More

Feed: Planet MySQL. Author: VividCortex. For virtually all development teams, testing code is a given: It's one of the most important parts of software development. Whether your organization includes a separate team devoted to QA, or your developers are testing their own code, QA is the primary way your team ensures that your application's logic is working correctly, and it's the best way for you to identify issues as early as possible.As a result, QA is critical for engineering velocity, and it helps shape your users' overall experience when engaging with your product. Nobody likes finding a broken app or ... Read More

Feed: All Oracle Press Releases. Empowering organizations to move workloads to the cloud while keeping their data on their own premises, Oracle today announced significant expansion of the breadth of services available through Oracle Cloud at Customer. The portfolio now spans all of the major Oracle PaaS categories and for the first time, also features Oracle SaaS services. Since its introduction just over a year ago, Oracle Cloud at Customer has experienced unprecedented growth with leading global organizations across six continents and more than 30 countries adopting the solution, including AT&T and Bank of America. Oracle Cloud at Customer is ... Read More

Feed: Hadoop – Cloudera Engineering Blog. Author: Justin Kestelyn. Apache Hive was one of the first projects to bring higher-level languages to Apache Hadoop. Specifically, Hive enables the legions of trained SQL users to use industry-standard SQL to process their Hadoop data. However, as you probably have gathered from all the recent community activity in the SQL-over-Hadoop area, Hive has a few limitations for users in the enterprise space. Until recently, two in particular – concurrency and security – were largely unaddressed. To address these gaps, for Hive release 0.11, Cloudera engineers built and contributed new infrastructure for meeting these needs ... Read More

Feed: Planet MySQL. Author: Vlad Mihalcea. Follow @vlad_mihalcea As previously explained, CDC (Change Data Capture) is one of the best ways to interconnect an OLTP database system with other systems like Data Warehouse, Caches, Spark or Hadoop. Debezium is an open source project developed by Red Hat which aims to simplify this process by allowing you to extract changes from various database systems (e.g. MySQL, PostgreSQL, MongoDB) and push them to Apache Kafka. In this article, we are going to see how you can extract events from MySQL binary logs using Debezium. First, you need a database-specific Debezium connector to ... Read More

Feed: Hadoop – Cloudera Engineering Blog. Author: Justin Kestelyn. This post is the first in a series of blog posts about Cloudera Morphlines, a new command-based framework that simplifies data preparation for Apache Hadoop workloads. To check it out or help contribute, you can find the code here. Cloudera Morphlines is a new open source framework that reduces the time and effort necessary to integrate, build, and change Hadoop processing applications that extract, transform, and load data into Apache Solr, Apache HBase, HDFS, enterprise data warehouses, or analytic online dashboards. If you want to integrate, build, or facilitate transformation pipelines without programming ... Read More

Feed: SQL Server Blog. Author: SQL Server Team. This post was authored by Tony Petrossian, Partner Group Program Manager, Database Systems Group SQL Server 2017 will bring with it support for the Linux OS and containers running on Windows, Linux, and macOS. Our goal is to enable SQL Server to run in modern IT infrastructure in any public or private cloud. With support for containers, SQL Server can now be used in many popular DevOps scenarios.  Developers working with Continuous Integration/Continuous Deployment (CI/CD) pipelines can now include SQL Server 2017 containers as a component of their applications for an integrated ... Read More

Feed: Hadoop – Cloudera Engineering Blog. Author: Sandy Ryza. Starting in CDH 4.2, YARN/MapReduce 2 (MR2) includes an even more powerful Fair Scheduler. In addition to doing nearly all that it could do in MapReduce 1 (MR1), the YARN Fair Scheduler can schedule non-MapReduce jobs, schedule based on fine-grained memory instead of slots, and support hierarchical queues. In this post, you’ll learn what the Fair Scheduler’s role is and how it fulfills it, what it means to be a YARN “scheduler,” and dive into its new features and how to get them running on your cluster. YARN/MR2 vs. MR1 YARN uses ... Read More

Feed: Hadoop – Cloudera Engineering Blog. Author: Justin Kestelyn. Below you’ll find the official announcement from Cloudera and Twitter about Parquet, an efficient general-purpose columnar file format for Apache Hadoop. Parquet is designed to bring efficient columnar storage to Hadoop. Compared to, and learning from, the initial work done toward this goal in Trevni, Parquet includes the following enhancements: Efficiently encode nested structures and sparsely populated data based on the Google Dremel definition/repetition levels Provide extensible support for per-column encodings (e.g. delta, run length, etc) Provide extensibility of storing multiple types of data in column data (e.g. indexes, bloom filters, statistics) Offer ... Read More