Tag: S3

Feed: AWS Public Sector Blog. Author: Subash D'Souza. Modern data engineering covers several key components of building a modern data lake. Most databases and data warehouses, to an extent, do not lend themselves well to a DevOps model. DataOps grew out of frustrations trying to build a scalable, reusable data pipeline in an automated fashion. DataOps was founded on applying DevOps principles on top of data lakes to help build automated solutions in a more agile manner. With DataOps, users apply principles of data processing on the data lake to curate and collect the transformed data for downstream processing. One ... Read More

Feed: Recent Announcements. You can access the Apache Flink dashboard via a browser by clicking on the ‘Apache Flink Dashboard’ button on the Kinesis Data Analytics console or by calling the service’s new CreateApplicationPresignedURL API. The dashboard is enabled by default for all Kinesis Data Analytics for Apache Flink applications and includes data on Apache Flink job performance and critical application health such as watermarks, checkpoints, and backpressure.  Amazon Kinesis Data Analytics is the easiest way to transform and analyze streaming data in real time with Apache Flink. Apache Flink is an open source framework and engine for processing data ... Read More

Feed: Recent Announcements. With this launch, Kinesis Data Analytics supports JDK 11 and Scala 2.12, with support for Python coming soon. Amazon Kinesis Data Analytics is the easiest way to transform and analyze streaming data in real time with Apache Flink. Apache Flink is an open source framework and engine for processing data streams. Amazon Kinesis Data Analytics reduces the complexity of building and managing Apache Flink applications. Amazon Kinesis Data Analytics for Apache Flink integrates with Amazon Managed Streaming for Apache Kafka (Amazon MSK), Amazon Kinesis Data Streams, Amazon Elasticsearch Service, Amazon DynamoDB streams Amazon S3, custom integrations, and ... Read More

Feed: R-bloggers. Author: Ian. [This article was first published on Fells Stats, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here) Want to share your content on R-bloggers? click here if you have a blog, or here if you don't. Okay, last one I swear. We just got updated results from the Pfizer-Biontech vaccine candidate. With 8 out of 170 cases belonging to the treatment group, the point estimate is remarkably similar to the Moderna preliminary results (~95% effectiveness). I will note that I was right in saying that the Pfizer-Biontech vaccine is ... Read More

Feed: Cloudwards. Author: Ben Stockton; Hot Storage vs Cold Storage vs Warm Storage Hot storage refers to data that is frequently used and accessed, like documents on your hard drive, which means it requires faster (and more expensive) hardware. This might include documents that are locally stored and regularly accessed, or documents that are quickly accessible online through a service like Google Drive (see our Google Drive review). Cold storage is the opposite; this is data that you want to keep (probably off site and on slower equipment), but you rarely need to access, meaning cold data is less expensive ... Read More

Feed: Recent Announcements. Amazon Textract is a machine learning service that makes it easy to extract printed text, handwriting, and data from virtually any document. Today, we are pleased to announce that Amazon Textract supports encryption of its asynchronous API output stored in your Amazon S3 buckets using your own AWS Key Management Service (KMS) Customer Master Keys (CMKs). With this feature, you have the flexibility to manage which encryption keys are used to protect your data and text extracted by Amazon Textract. For more information on how to accomplish this, please read our newest blog post.   ... Read More

Feed: AWS Compute Blog. Author: Ben Peven. This post is contributed by Nicola Venuti, Sr. HPC Solutions Architect. AWS ParallelCluster is an AWS-supported, open source cluster management tool that makes it easy to deploy and manage High Performance Computing (HPC) clusters on AWS. While AWS ParallelCluster includes many benefits for its users, it has not provided straightforward support for monitoring your workloads. In this post, I walk you through an add-on extension that you can use to help monitor your cloud resources with AWS ParallelCluster. Product overview AWS ParallelCluster offers many benefits that hide the complexity of the underlying platform ... Read More

Feed: AWS Big Data Blog. The Amazon Redshift Data API makes it easy for any application written in Python, Go, Java, Node.JS, PHP, Ruby, and C++ to interact with Amazon Redshift. Traditionally, these applications use JDBC connectors to connect, send a query to run, and retrieve results from the Amazon Redshift cluster. This requires extra steps like managing the cluster credentials and configuring the VPC subnet and security group. In some use cases, you don’t want to manage connections or pass credentials on the wire. The Data API simplifies these steps so you can focus on data consumption instead of ... Read More

Feed: AWS Architecture Blog. Enterprise AWS customers are often managing many accounts under a payer account, and sometimes accounts are closed before Reserved Instances (RI) or Savings Plans (SP) are fully used. Manually tracking account closures and requesting RI and SP migration from the closed accounts can become complex and error prone. This blog post describes a solution for automating the process of requesting migration of unused RI and SP from closed accounts that are linked to a payer account. The solution helps reduce manual work and loss of unused RI and SP due to human error. The solution automatically detects newly ... Read More

Feed: AWS Big Data Blog. This is a guest blog post by AWS partner Aspire Ventures As we enter winter, with fewer options to be outdoors, our personal choices can impact our risk of contracting the COVID-19 virus even more. The New England Journal of Medicine publication showed real-world examples of the effectiveness of masks and social distancing in mitigating severity of COVID-19 infection and keeping people asymptomatic. CNN reported on a study that showed people who contracted COVID-19 were twice as likely to have visited a restaurant in the prior two weeks. What if we had actionable crowding, mask ... Read More