Tag: Kafka

Feed: Microsoft Azure Blog. Author: Rob Caron. Now in preview Announcing the public preview of Windows Container Support in Azure App Service - Windows Server Containers in Web App are now available in public preview. Azure App Service provides pre-defined application stacks on Windows like ASP.NET or Node.js, running on IIS. The preconfigured Windows environment locks down the operating system from administrative access, software installations, changes to the global assembly cache, and so on. If your application requires more access than the preconfigured environment allows, you can deploy a custom Windows container instead. Azure App Service caches several parent images ... Read More

Feed: Featured Blog Posts - Data Science Central. Author: ajit jaokar. In this post, I discuss the development of the Enterprise AI business case through a framework of four quadrants.  According to Gartner: “The mindset shift required for AI can lead to “cultural anxiety” because it calls for a deep change in behaviors and ways of thinking”. Deployment of AI in an Enterprise is complex and multi-disciplinary. Hence, this framework is evolutionary.  The vendors and initiatives listed are included to illustrate the framework. In progressive orders of complexity (and opportunity) the four quadrants for the Enterprise AI business case are: ... Read More

Feed: AWS Architecture Blog. This post is a contribution from AWS customer, Veritone. It was originally published on the company’s Website.  Here at Veritone, we deal with a lot of data. Our product uses the power of cognitive computing to analyze and interpret the contents of structured and unstructured data, particularly audio and video. We use cognitive computing to provide valuable insights to our customers. Our platform is designed to ingest audio, video and other types of data via a series of batch processes (called “engines”) that process the media and attach some sort of output to it, such as ... Read More

Feed: Microsoft Azure Blog. Author: Olivier Bloch. Whether you’re looking for in-the-moment insights or long-term trends, the value of information is only as good as its usefulness for your business. Some information must be acted upon immediately, but it can also be stored for long-term, big-picture analysis of trends over time. For each case there are different tools to use, and Azure IoT has you covered. Near real-time data analysis Azure Stream Analytics is a pay-as-you-go service that ingests data from a variety of sources. In addition to real-time data from IoT Hub, it can also take in historical data ... Read More

Feed: Avro – Cloudera Engineering Blog. Author: Cy Jervis. Part 3: Configuring Clients Earlier, we introduced Kafka Serializers and Deserializers that are capable of writing and reading Kafka records in Avro format. In this part we will going to see how to configure producers and consumers to use them. Setting up a Kafka Topic for use as a Schema Store KafkaTopicSchemaProvider works with a Kafka topic as its persistent store. This topic will contain at most thousands of records: the schemas. It does not need multiple partitions, but it needs to be available even when one of the brokers is ... Read More

Feed: Hortonworks Blog – Hortonworks. Author: Ram Venkatesh. This post was authored by Ram Venkatesh, Hortonworks VP, Engineering; and James Malone, Cloud Product Manager, Google.  If you’re looking for a fully-managed cloud service for running data and analytics clusters, thanks in large part to the Apache Hadoop and Spark communities, you might very well look to Cloud Dataproc, which offers both long-running and job-scoped clusters. Job-scoped clusters let you tailor your clusters to a specific job, instead of spending development time building individual job configurations that compete for resources on a multi-purpose cluster. Dataproc even has features such as workflow ... Read More

Feed: Planet MySQL; Author: Uber Engineering; From driver and rider locations and destinations, to restaurant orders and payment transactions, every interaction on Uber’s transportation platform is driven by data. Data powers Uber’s global marketplace, enabling more reliable and seamless user experiences across our products for riders, drivers, and eaters worldwide, as well as empowering our own employees to more efficiently do their jobs. Uber takes data-driven to the next level with the complexity of its systems and breadth of data, processing trillions of Kafka messages per day, storing hundreds of petabytes of data in HDFS across multiple data centers, and ... Read More

Feed: Featured Blog Posts - Data Science Central. Author: Igor Bobriakov. Spark is a powerful tool which can be applied to solve many interesting problems. Some of them have been discussed in our previous posts. Today we will consider another important application, namely streaming. Streaming data is the data which continuously comes as small records from different sources. There are many use cases for streaming technology such as sensor monitoring in industrial or scientific devices, server logs checking, financial markets monitoring, etc. In this post, we will examine the case with sensors temperature monitoring. For example, we have several sensors (1,2,3,4,...) ... Read More

Feed: Featured Blog Posts - Data Science Central. Author: Kai Waehner. I built a scenario for a hybrid machine learning infrastructure leveraging Apache Kafka as scalable central nervous system. The public cloud is used for training analytic models at extreme scale (e.g. using TensorFlow and TPUs on Google Cloud Platform (GCP) via Google ML Engine. The predictions (i.e. model inference) are executed on premise at the edge in a local Kafka infrastructure (e.g. leveraging Kafka Streams or KSQL for streaming analytics). This post focuses on the on premise deployment. I created a Github project with a KSQL UDF for sensor ... Read More

Feed: All - O'Reilly Media. Author: O'Reilly Media. Typing at a desktop computer (source: Free-photos via Pixabay) Learn new topics and refine your skills with more than 120 live online training courses we opened up for August and September on our learning platform. Space is limited and these courses often fill up. Artificial intelligence and machine learning Machine Learning in Python and Jupyter for Beginners, August 3 Artificial Intelligence: AI for Business, August 3 High Performance TensorFlow in Production: Hands on with GPUs and Kubernetes, August 14-15 Artificial Intelligence: Real-world Applications, August 30 Getting Started with Machine Learning, September 5 ... Read More