Tag: Kafka

Feed: Planet MySQL; Author: MC Brown; Continuent are pleased to announce the release of Tungsten Replicator and Tungsten Clustering 5.2.0This release is one of our most exciting new releases for a while, as it contains some significant new features and lays the groundwork for some additional new functionality in the upcoming 5.3.0 and 6.0 releases due later this year. In particular, this release includes the following new features: New replicator filtering environment to make filtering quicker and easier to use, and more flexible New filter configuration standard for new filters New filter to make replication out of a cluster easier ... Read More

Feed: Cloud – Cloudera Engineering Blog. Author: Cy Jervis. Cloudera is pleased to announce that Cloudera Enterprise 5.12 is now generally available (GA). The release includes enhancements for running in cloud environments (with broader ADLS support and improved AWS Spot Instance support), usability and productivity improvements for both data science and analytic workloads, as well as performance gains and self-service performance management across a range of workloads. As usual, there are also a number of quality enhancements, bug fixes, and other improvements across the stack. Here is a partial list of what’s included (see the Release Notes for a full ... Read More

Feed: IBM Big Data & Analytics Hub - All Content; Author: cindy-russell; It seems that we’re reaching the point where the Internet of Things (IoT) is moving from the domain of enthusiastic early-adopters to the more challenging, more profitable territory of mainstream enterprise technology.In particular, companies in industries that depend on the attention and loyalty of large numbers of consumers – retail and air travel, for example –have already woken up to the possibilities. If you can recognize an individual customer’s mobile device as they move through an airport or walk past your store, you can potentially take action to improve that ... Read More

Feed: Planet MySQL. Author: Vlad Mihalcea. Follow @vlad_mihalcea As previously explained, CDC (Change Data Capture) is one of the best ways to interconnect an OLTP database system with other systems like Data Warehouse, Caches, Spark or Hadoop. Debezium is an open source project developed by Red Hat which aims to simplify this process by allowing you to extract changes from various database systems (e.g. MySQL, PostgreSQL, MongoDB) and push them to Apache Kafka. In this article, we are going to see how you can extract events from MySQL binary logs using Debezium. First, you need a database-specific Debezium connector to ... Read More

Feed: Planet MySQL. Author: MC Brown. Continuent Tungsten 5.2 is just around the corner. This is one of our most exciting Tungsten product releases for some time! In this webinar we’re going to have a look at a host of new features in the new release, includingThree new Replication Applier Targets (Kafka, Cassandra and Elasticsearch)New improvements to our core command-line tools trepctl and thlNew foundations for our filtering services, andImprovements to the compatibility between replication and clustering This webinar is going be a packed session and we’ll show all the exciting stuff with more in-depth follow-up sessions in the coming weeks ... Read More

Feed: Hazelcast Blog. Author: Vladimir Shreiner. High performance has always been a major design goal of Hazelcast Jet. In order to verify various performance aspects of Jet, we’ve introduced the following benchmarks: Word count, to measure and compare the raw computational power during batch processing. Trade monitor, which is based on a streaming aggregation and involves windowing and event-time based processing. 5,000 jobs benchmark which was motivated by use cases where thousands of various data processing feeds run on the same cluster. Parallelism is achieved by sharing the resources among sequential jobs rather than parallelizing one job. Let’s have a ... Read More

Feed: Hazelcast Blog. Author: Greg Luck. I am very excited to announce that after months of engineering work and assistance from Pivotal, Hazelcast Jet is now available as a Tile on Pivotal Cloud Foundry (PCF) – the first general purpose data processing platform available on PCF. As a member of the Cloud Foundry community we are determined to enhance the services available on the platform by equipping developers with an open distributed processing engine for big data streams driven by a mature and stable tribe of contributors. Deploying Hazelcast in the cloud has never been easier. Hazelcast Jet for Pivotal ... Read More

Feed: Google Cloud Big Data and Machine Learning Blog. Author: Google Cloud Big Data and Machine Learning Blog Team. Innovation in data processing and machine learning technology Thursday, July 6, 2017 By Reuven Lax, Software Engineer The series concludes with a description of how exactly-once processing in Cloud Dataflow is supported by sources and sinks. In the previous post in this series, we expanded on how Cloud Dataflow’s streaming shuffle supports exactly-once deliveries. We explained how a variety of clever optimizations allow Cloud Dataflow to efficiently filter duplicates out of shuffle deliveries. In this final installment, we’ll explore the remaining ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Rob Weltman. 2011 was a breakthrough year for Apache Hadoop as many more mainstream organizations large and small turned to Hadoop to manage and process Big Data, while enterprise software and hardware vendors have also made Hadoop a prominent part of their offerings. Big Data and Hadoop became synonymous in much of the enterprise discourse, and Big Data interest is not restricted to Big Companies. Apache Hadoop Releases Hadoop had three major releases in 2011: 1.0 (AKA 0.20.205.x), 0.22, and 0.23. 1.0.0 adds HDFS support for HBase, Webhdfs, and HDFS performance improvements 0.23 ... Read More

Feed: Planet MySQL. Author: Roberto V. Zicari. “Spark and Ignite can complement each other very well. Ignite can provide shared storage for Spark so state can be passed from one Spark application or job to another. Ignite can also be used to provide distributed SQL with indexing that accelerates Spark SQL by up to 1,000x.”–Nikita Ivanov. I have interviewed Nikita Ivanov,CTO of GridGain.Main topics of the interview are Apache Ignite, Apache Spark and MySQL, and how well they perform on big data analytics. RVZ Q1. What are the main technical challenges of SaaS development projects? Nikita Ivanov: SaaS requires that ... Read More