Tag: row

Feed: Planet PostgreSQL. Real time, fast update of materialized views  In  this blog entry I have introduced the ongoing project: Incremental View Maintenance (IVM), which allows to real time, fast update of materialized views. Since the bolg, IVM now can handle outer joins and self joins. The out come of the project is in public in a form of set of patches against git master branch of PostgreSQL. Also a GitHub repository for this project is in public as well.Docker image for IVMSo skilled PostgreSQL developers can test IVM today. However it's not that easy for people who do not ... Read More

Feed: Planet PostgreSQL. Big data has an unfortunate tendency to get messy. A few years in, a growing database that use to be small, lean, and well-designed, has better odds than not of becoming something large, bloated, and with best practices tossed aside and now considered unsalvageable.There’s a few common reasons that this happens, some better than others: Technological limitation: The underlying tech doesn’t support the scale. Say transactions or referential integrity across partitions in a sharded system. Stability: Certain operations come to be considered too risky. e.g. Batch update operations that have unpredictable performance properties. Cost/effort: Doing things the ... Read More

Feed: Planet MySQL; Author: Kristian Köhntopp; Kristian Köhntopp  | December 1, 2020 |  Posted In: MySQL, Open Source Databases, Tools The tables in PERFORMANCE_SCHEMA (P_S) are not actually tables. You should not think of them as tables, even if your SQL works on them. You should not JOIN them, and you should not GROUP or ORDER BY them.Unlocked memory buffers without indexesThe stuff in P_S has been created with “keep the impact on production small” in mind. That is, from a users point of view, you can think of them as unlocked memory buffers – the values in there change as you ... Read More

Feed: AWS Big Data Blog. As 2020 draws to a close, we’ve put together this post to walk you through all that’s changed in Amazon QuickSight this year. For your reading convenience, this post is broken up into the following sections: Embedded Analytics at scale Embedded analytics has been a key focus area for both independent software vendors (ISVs) and enterprises in the past year. Thousands of customers have since chosen to launch embedded dashboards in QuickSight for both their internal and external end-users, including the NFL, Blackboard, Comcast, Panasonic Avionics, EHE Health, and more, with the ability to scale ... Read More

Feed: Planet PostgreSQL. In one of my previous blog posts, Why PostgreSQL WAL Archival is Slow, I tried to explain three of the major design limitations of PostgreSQL’s WAL archiver which is not so great for a database with high WAL generation. In this post, I want to discuss how pgBackRest is addressing one of the problems (cause number two in the previous post) using its Asynchronous WAL archiving feature.Let me explain the problem in a bit more detail here.The PostgreSQL archiver process has an internal function pgarch_archiveXlog()  which calls the system()  system call which executes the archive_command  which will ... Read More

Feed: Planet MySQL; Author: Vitess; Django is a popular framework for Python application developers. It includes packages which make tasks like authorization and content administration easier. Django supports a number of databases including MySQL which makes it possible to run a Django application over Vitess without having to change the application code. Let’s take a look at how to combine the strengths of these two open source frameworks.We built this example using Vitess operator. You can see the details of the implementation in the blog post Vitess Operator for Kubernetes.Prerequisites Python environment locally (3.X)Kubernetes access (minikube, GKE)Support for Django ORM ... Read More

Feed: AWS Big Data Blog. Amazon Athena is an interactive query service that makes it easy to analyze data directly in Amazon Simple Storage Service (Amazon S3) using standard SQL. It’s a serverless platform with no need to set up or manage infrastructure. Athena scales automatically—running queries in parallel—so results are fast, even with large datasets and complex queries. You pay only for the queries that you run and the charges are based on the amount of data scanned by each query. You’re not charged for data definition language (DDL) statements like CREATE, ALTER, or DROP TABLE, statements for managing ... Read More

Feed: Planet MySQL; Author: Marco Tusa; How native ProxySQL stands in failover support (both v2.0.15 and v2.1.0)In recent times I have been designing several solutions focused on High Availability and Disaster Recovery. Some of them using Percona Server for MySQL with group replication, some using Percona XtraDB Cluster (PXC). What many of them had in common was the use of ProxySQL for the connection layer. This is because I consider the use of a layer 7 Proxy preferable, given the possible advantages provided in ReadWrite split and SQL filtering. The other positive aspect provided by ProxySQL, at least for Group Replication, ... Read More

Feed: Planet PostgreSQL. By Ryan Lambert -- Published November 29, 2020A few months ago I started experimenting with a few project ideas involving data over space and time. Naturally, I want to use Postgres and PostGIS as the main workhorse for these projects, the challenge was working out exactly how to pull it all together. After a couple false starts I had to put those projects aside for other priorities. In my free time I have continued working through some related reading on the topic. I found why you should be using PostGIS trajectories by Anita Graser and recommend reading ... Read More

Feed: R-bloggers. Author: Greski, Len. [This article was first published on Data Science Depot, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here) Want to share your content on R-bloggers? click here if you have a blog, or here if you don't. Background A user on Stackoverflow recently asked a question about a program to generate Monte Carlo simulations on Bernoulli trials to calculate coverage percentages using Wald confidence intervals. One of the problems in the code is that probability value calculations are executed on individual observations rather than sums of successes and ... Read More