- Home
- Tag: ACID
Posts tagged ACID
ACID – The acronym standing for the properties maintained by standard database management systems, standing for Atomicity, Consistency, Isolation, and Durability.
Tag: ACID
Announcing general availability of Amazon Athena ACID transactions, powered by Apache Iceberg
Feed: Recent Announcements. We are excited to announce the general availability of Amazon Athena ACID transactions, a new capability that adds insert, update, delete, and time travel operations to Athena's SQL data manipulation language (DML). Athena ACID transactions enable multiple concurrent users to make reliable, row-level modifications to their Amazon S3 data from Athena's console, API, and ODBC and JDBC drivers. Built on the Apache Iceberg table format, Athena ACID transactions are optimized for Amazon S3 storage, support seamless schema evolution, and ensure atomic operations across other services and engines that support the Iceberg table format such as Amazon EMR, ... Read More
Amazon EMR now supports Apache Iceberg, a highly performant, concurrent, ACID-compliant table format for data lakes
Feed: Recent Announcements. We are excited to announce that Amazon EMR 6.5.0 now includes Apache Iceberg version 0.12. Apache Iceberg is an open table format for large data sets in Amazon S3 and provides fast query performance over large tables, atomic commits, concurrent writes, and SQL-compatible table evolution. With the current release, you can use Apache Spark 3.1.2 on EMR clusters with the Iceberg table format. Apache Iceberg offers an open source table format for data stored in data lakes that helps data engineers manage complex challenges such as managing continuously evolving data sets while maintaining query performance. Iceberg allows ... Read More
Announcing Amazon Athena ACID transactions, powered by Apache Iceberg (Preview)
Feed: Recent Announcements. We are excited to announce the public preview of Amazon Athena ACID transactions, a new capability that adds write, delete, update, and time travel operations to Athena's SQL data manipulation language (DML). Athena ACID transactions enables multiple concurrent users to make reliable, row-level modifications to their Amazon S3 data from Athena's console, API, and ODBC and JDBC drivers. Built on the Apache Iceberg table format, Athena ACID transactions are compatible with other services and engines such as Amazon EMR and Apache Spark that support the Iceberg table format. Using Athena ACID transactions, you can now make business- ... Read More
Effective data lakes using AWS Lake Formation, Part 3: Using ACID transactions on governed tables

Feed: AWS Big Data Blog. Data lakes on Amazon Simple Storage Service (Amazon S3) have become the default repository for all enterprise data and serve as common choice for a large number of users querying from a variety of analytics and ML tools. Often times you want to ingest data continuously into the data lake from multiple sources and query against the data lake from many analytics tools concurrently. Previously, getting consistent results meant that you had to build custom pipelines to keep data consistent and resulted in delays before the data became available to the consuming tools. We announced AWS Lake Formation transactions, row-level ... Read More
IT Resilience Within AWS Cloud, Part II: Architecture and Patterns

Feed: AWS Architecture Blog. In Part I of this two-part blog, we outlined best practices to consider when building resilient applications in hybrid on-premises/cloud environments. We also showed you how to adapt mindsets and organizational culture. In Part II, we’ll provide technical considerations related to architecture and patterns for resilience in AWS Cloud. Considerations on architecture and patterns The AWS Well-Architected Framework defines resilience as having “the capability to recover when stressed by load (more requests for service), attacks (either accidental through a bug, or deliberate through intention), and failure of any component in the workload’s components.” Resilience is an ... Read More
Avinash Vallarapu: Setting up Streaming Replication in PostgreSQL 13 and Streaming Replication Internals

Feed: Planet PostgreSQL. While migrating to PostgreSQL, some of the users may be curious about the type of replication offered by PostgreSQL. Some might want to know if they have to pay any additional enterprise license in order to achieve this requirement. This is because, a few commercial databases require a customer to take an enteprise license to setup replication. But, PostgreSQL, like advanced Open Source databases, automatically offers several types of replication methods built-in. This means, we never have to consider taking any additional license or an enterprise solution and stay away from vendor lock-in while using PostgreSQL. In ... Read More
Better Together: Matillion ETL for Delta Lake on Databricks Now Available

Feed: Matillion. Author: Rob Callaghan; Unlock your lakehouse potential with Matillion’s cloud-native data integration platform With the arrival of Matillion ETL for Delta Lake on Databricks, data professionals across the organization can now leverage visually designed data transformations to build their lakehouse architecture and unify their data ecosystem.These days, companies need a place to store structured data within the cloud, and a cloud data warehouse fits that bill. But often in a modern data ecosystem, companies also have to deal with semi-structured data coming from a variety of data sources that doesn’t fit into that tabular data paradigm. Nor does it ... Read More
What is the Difference Between a Data Analytics Hub and a Lakehouse?
Feed: Actian. Author: Lewis Carr. In the opening installment of this blog series—Data Lakes, Data Warehouses and Data Hubs: Do we need another choice? I explore why simply migrating these on-prem data integration, management, and analytics platforms to the Cloud does not fully address modern data analytics needs. In comparing these three platforms, it becomes clear that all of them meet certain critical needs, but none of them meet the needs of business end-users without significant support from IT. In the second blog in this series — What is a Data Analytics Hub? — I introduce the term Data Analytics ... Read More
2 Advanced Analytics Trends to Watch for in 2021

Feed: Matillion. Author: Julie Polito; 2020 was a year of rapid and tumultuous change for nearly every business – especially in the cloud. Our need to work 100 percent remotely, and our need to have data accessible to us no matter where we worked, accelerated the move to the cloud for many organizations. We’re likely to see that trend continuing into 2021 as many organizations decide that the remote way of work and sharing information via virtual channels is part of the new normal. But along with organizations speeding up a move to the cloud, we’re seeing another long awaited macro ... Read More
Effective data lakes using AWS Lake Formation, Part 2: Creating a governed table for streaming data sources

Feed: AWS Big Data Blog. We announced the preview of AWS Lake Formation transactions, row-level security, and acceleration at AWS re:Invent 2020. In Part 1 of this series, we explained how to set up a governed table and add objects to it. In this post, we expand on this example, and demonstrate how to ingest streaming data into governed tables using Lake Formation transactions. In typical streaming use cases, new data is continuously ingested into a data lake. This creates many small files that may impact query performance. In addition, a common requirement is to isolate queries from updates without requiring downtime or having ... Read More
Recent Comments