Tag: PostgreSQL

Feed: Planet PostgreSQL. This diary-of-an-engineer post from our Citus open source engineering team was originally published on the Azure Postgres Blog. When working on the internals of Citus, an open source extension to Postgres that transforms Postgres into a distributed database, we often get to talk with customers that have interesting challenges you won’t find everywhere. Just a few months back, I encountered an analytics workload that was a really good fit for Citus. But we had one problem: the percentile calculations on their data (over 300 TB of data) could not meet their SLA of 30 seconds. To make ... Read More

Feed: Planet PostgreSQL. PL/Python can be a convenient and powerful way to take advantage of your PostgreSQL database. In an earlier post, I talked a little bit about how you can use modules included in the Python standard library with PL/Python functions. In this post, we'll try running NumPy in a simple user-defined function which also takes advantage of PL/Python database access functions. The function will show a working example of how to easily convert a data table in Postgres to a NumPy array ... Read More

Feed: AWS Big Data Blog. General Data Protection Regulation (GDPR) is an important aspect of today’s technology world, and processing data in compliance with GDPR is a necessity for those who implement solutions within the AWS public cloud. One article of GDPR is the “right to erasure” or “right to be forgotten” which may require you to implement a solution to delete specific users’ personal data.In the context of the AWS big data and analytics ecosystem, every architecture, regardless of the problem it targets, uses Amazon Simple Storage Service (Amazon S3) as the core storage service. Despite its versatility and ... Read More

Feed: Planet PostgreSQL. This post by the technical lead of our Citus open source team was originally published on the Azure Postgres Blog. I recently gave a talk about the Citus extension to Postgres at the Warsaw PostgreSQL Users Group. Unfortunately, I did not get to go in person to beautiful Warsaw, but it was still a nice way to interact with the global Postgres community and talk about what Citus is, how it works, and what it can do for you. If you are already familiar with Postgres then this talk should be a good introduction to all the ... Read More

Feed: Planet PostgreSQL. This blog is about my work on the Postgres open source database, and is published on Planet PostgreSQL. PgLife allows monitoring of all Postgres community activity. Why Not to Choose Postgres Wednesday, September 16, 2020 Postgres has made great strides in adding features to match proprietary databases, and it has many complex features that other databases don't have. However, that doesn't mean it is the best fit for every organization. There are still reasons not to use Postgres: Skills in another relational database, and no desire or value to learn Postgres Custom applications written for another database ... Read More

Feed: Planet MySQL; Author: Joshua Otwell; I recently watched a fantastic Python Pandas library tutorial series on YouTube. Without a doubt, Pandas is great for all sorts of data stuff. On the same token, MySQL Shell in Python mode is quite powerful in the sense that Python and the MySQL Shell (version >= 8.0) are somewhat united in the same environment. Although Pandas is in a league all its own when it comes to data analysis, between the power of MySQL and Python, we can also perform some basic analysis easily in MySQL Shell Python mode. In this blog post, ... Read More

Feed: Planet PostgreSQL. One of the new features in PostgreSQL 13 is the SQL-standard WITH TIES clause to use with LIMIT — or, as the standard calls that, FETCH FIRST n ROWS. Thanks are due to Surafel Temesgen as initial patch author; Tomas Vondra and yours truly for some additional code fixes; and reviewers Andrew Gierth and Erik Rijkers. You can peruse the commit message. A caucus race. An illustration by Maggie Taylor.Ties are very frequently when ranking things; for instance, in a caucus race you could have many ties, and for sure you don’t want to deprive participants of ... Read More

Feed: Planet PostgreSQL. © Laurenz Albe 2020HOT updates are not a new feature. They were introduced by commit 282d2a03dd in 2007 and first appeared in PostgreSQL 8.3.But since HOT is not covered by the PostgreSQL documentation (although there is a README.HOT in the source tree), it is not as widely known as it should be: Hence this article that explains the concept, shows HOT in action and gives tuning advice.What is HOT?HOT is an acronym for “Heap Only Tuple” (and that made a better acronym than Overflow Update CHaining). It is a feature that overcomes some of the inefficiencies of ... Read More

Feed: Planet PostgreSQL. Sometimes, PostgreSQL databases need to import large quantities of data in a single or a minimal number of steps. This is commonly known as bulk data import where the data source is typically one or more large files. This process can be sometimes unacceptably slow. There are many reasons for such poor performance: indexes, triggers, foreign keys, GUID primary keys, or even the Write Ahead Log (WAL) can all cause delays. In this article, we will cover some best practice tips for bulk importing data into PostgreSQL databases. However, there may be situations where none of these ... Read More

Feed: Planet PostgreSQL. PostgreSQL 13 is a lucky release! There are some very impactful new features and the release is coming out at its regularly scheduled time in the fall.One of my favorite parts around the PostgreSQL release process is when I can take a step back, review and discuss the features going into a new release, and learn how they can make a positive impact on our users. For example, in a past hallway track at PGCon, I learned how PostgreSQL 12 introduced the ability to do a form of multi-factor authentication! Each release has a lot of these ... Read More