Tag: Apache

Feed: Recent Announcements. The new kafka.t3.small broker type for Amazon MSK is ideal for customers getting started with Apache Kafka or Amazon MSK, looking to test Apache Kafka applications in a development environment, or have low throughput production workloads. All Amazon MSK features apply to clusters with kafka.t3.small brokers including two and three availability zone deployments, encryption in transit and at rest, and each cluster is paired with a highly available, fully managed Apache ZooKeeper cluster at no additional cost. The kafka.t3.small is an Amazon EC2 T3 instance in standard mode which provides a baseline level of CPU performance with ... Read More

Feed: AWS Public Sector Blog. Author: Sanjay Padhi, Ph.D. The rapidly changing and dynamic global health situation has impacted the lives of many people including researchers at universities and institutions worldwide. Many academic institutions are migrating to remote operations. Researchers are processing data, collaborating online, and trying to maintain labs remotely. Amazon and Amazon Web Services (AWS) are responding to these events in support of our communities and deploying resources and technology to enable remote learning and home working. Researchers can access the following to help them through this unprecedented situation: High Performance Computing (HPC) and container-based resources AWS ParallelCluster is a fully supported and ... Read More

Feed: AWS Big Data Blog. Apache Spark is a powerful data processing engine that gives data analyst and engineering teams easy to use APIs and tools to analyze their data, but it can be challenging for teams to manage their Python and R library dependencies. Installing every dependency that a job may need before it runs and dealing with library version conflicts is time-consuming and complicated. Amazon EMR 6.0.0 simplifies this by allowing you to use Docker images from Docker Hub and Amazon ECR to package your dependencies. This allows you to package and manage your dependencies for individual Spark ... Read More

Feed: Big Data Made Simple. Author: Philip Piletic. Perhaps the single biggest problem with data lakes is the fact that it seems like nobody can clearly define what they’re best used for. Computer scientists will say that a data lake is nothing more than a repository of data that’s stored in a natural raw format, but that definition sounds awfully close to a file system. There’s no reason that server managers will want to explore some exotic solution when a Btrfs volume or an Apache Flafka implementation would do the job.That being said, there are a number of reasons you’d ... Read More

Feed: AWS Partner Network (APN) Blog. Author: Scott Kellish. By Scott Kellish, Partner Solutions Architects at AWSBy Basil Faruqui, Principal Solutions Marketing Manager at BMC Digital Business AutomationBy Joe Goldberg, Innovation Evangelist at BMCPredictive maintenance, which analyzes sensor data to predict equipment failures, has emerged as one of the most common business use cases of machine learning (ML) and the Internet of Things (IoT). To build and train ML models, you need data science expertise. If you’re going to run those ML models in production at scale, you need data engineering expertise to build a pipeline for data ingestion, storage, ... Read More

Feed: R-bloggers. Author: Mirai Solutions. [This article was first published on Mirai Solutions, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here) Want to share your content on R-bloggers? click here if you have a blog, or here if you don't. XLConnect 1.0.1 now available from We are pleased to announce the release of XLConnect 1.0.1. This release is mainly a maintenance release and comes with the following changes: Upgrade to Apache POI 4.1 Adapt Java implementation to make use of Java 8 and remove some dependencies Remove dependency on XLConnectJars, download required ... Read More

Feed: AWS Compute Blog. Author: David LaBissoniere. In my last post I discussed AWS Elastic Beanstalk’s new public roadmap on GitHub. Today I want to talk about our new generation of Elastic Beanstalk platforms built on top of Amazon Linux 2 (AL2).Late last year we launched a public beta of a new Elastic Beanstalk platform for Amazon Corretto — Amazon’s no-cost, production-ready distribution of the Open Java Development Kit (OpenJDK). This is also our first platform based on AL2. This year we have launched two more beta AL2 platforms: Docker and Python. More beta platforms are arriving soon, followed by ... Read More

Feed: AWS Partner Network (APN) Blog. Author: Sanjay Nighojkar. By Suraj Nathani, Sr. Solutions Architect at Mactores CognitionBy Anil Patel, Data Engineer at Mactores CognitionBy Sanjay Nighojkar, Manager, Partner Solutions Architect at AWSSeagate Technology is a United States-based data storage company with worldwide manufacturing facilities that generate huge amounts of manufacturing and testing data. With petabytes of data accumulated over 20 years, and more being generated each day, it was imperative for Seagate to have systems in place to ensure the cost of collecting, storing, and processing data did not exceed their return on investment (ROI). Seagate’s on-premises big data ... Read More

Feed: R-bloggers. Author: R/exams. [This article was first published on R/exams, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here) Want to share your content on R-bloggers? click here if you have a blog, or here if you don't. Webinar hosted by Why R? Foundation: Slides, video, and e-learning resources. Why R? Webinar 001 R/exams was presented in the first Why R? webinar by Achim Zeileis. Marcin Kosiński was the host and did a great job setting up the Zoom session that was streamed live on YouTube where more than 100 participants joined ... Read More

Feed: Recent Announcements. A few key features included in 2.4.1 are: KIP-392, Allow Consumers to Fetch from Closest Replica, this allows consumers to fetch data from the nearest replica partition rather than from the leader partition, which reduces latency and potential cross Availability Zone data transfer.  KIP-480, Sticky Partitioner, this provides performance improvements to the default Apache Kafka partitioner with up to 50% reduction in latency.  KIP-455, Create an Administrative API for Replica Reassignment Apache, this provides a new Apache Kafka Admin API for partition reassignment and addresses limitations of the Apache ZooKeeper implementation. Amazon MSK is a fully managed ... Read More