Tag: INSERT

Feed: AWS Big Data Blog. General Data Protection Regulation (GDPR) is an important aspect of today’s technology world, and processing data in compliance with GDPR is a necessity for those who implement solutions within the AWS public cloud. One article of GDPR is the “right to erasure” or “right to be forgotten” which may require you to implement a solution to delete specific users’ personal data.In the context of the AWS big data and analytics ecosystem, every architecture, regardless of the problem it targets, uses Amazon Simple Storage Service (Amazon S3) as the core storage service. Despite its versatility and ... Read More

Feed: Planet MySQL; Author: Kristian Köhntopp; Iterating schemas over time is not an uncommon thing. Often requirements emerge only after you have data, and then directed action is possible. Consequently, working on existing data, and structuring and cleaning it up is a common task.In todays example we work with a log table that logged state transitions of things in freeform VARCHAR fields. After some time the log table grew quite sizeable, and the log strings are repeated rather often, contributing to the overall size of the table considerably. We are starting with this table: CREATE TABLE `log` ( `id` int ... Read More

Feed: AWS Big Data Blog. Stream processing is very useful in use cases where we need to detect a problem quickly and improve the outcome based on data, for example production line monitoring or supply chain optimizations.This blog post walks you through process of streaming existing data files and ongoing changes from Amazon Simple Storage Service (Amazon S3) to Amazon Kinesis. You achieve this by using AWS Database Migration Service (AWS DMS). AWS DMS enables you to seamlessly migrate data from supported sources to relational databases, data warehouses, streaming platforms, and other data stores in AWS cloud. Many SaaS, third-party ... Read More

Feed: Redis. Author: Tugdual Grall. RediSearch 2.0 is now out in public preview! Most of the features in this major new release have been driven by your feedback, with a focus on improving the developer experience and enhanced scalability. But this blog post concentrates on helping you get started using RediSearch 2.0’s new data indexing capabilities and better ways to create an index.  Having a rich query and aggregation engine inside your Redis database opens the door to many new applications that go well beyond caching. You can use Redis as your primary database even when you need to access ... Read More

Feed: Planet MySQL; Author: Mike Frank; The Challenge Often with sensitive information, you need to have an audit log. Not just that a table had a select run, but that specific cells within the table were accessed.  Frequently data such as this will contain a classification level as part of the row, defining policies for how it is handled, audited, etc. Sensitive Data such as that might be labeled as –  Highly Sensitive  Top Secret  Classified  Restricted  Clearance Required  Highly Confidential  Protected Data that is categorized or labeled in some manner is typically covered by regulations requiring your compliance. Compliance ... Read More

Feed: Planet PostgreSQL. © Laurenz Albe 2020HOT updates are not a new feature. They were introduced by commit 282d2a03dd in 2007 and first appeared in PostgreSQL 8.3.But since HOT is not covered by the PostgreSQL documentation (although there is a README.HOT in the source tree), it is not as widely known as it should be: Hence this article that explains the concept, shows HOT in action and gives tuning advice.What is HOT?HOT is an acronym for “Heap Only Tuple” (and that made a better acronym than Overflow Update CHaining). It is a feature that overcomes some of the inefficiencies of ... Read More

Feed: Planet PostgreSQL. Sometimes, PostgreSQL databases need to import large quantities of data in a single or a minimal number of steps. This is commonly known as bulk data import where the data source is typically one or more large files. This process can be sometimes unacceptably slow. There are many reasons for such poor performance: indexes, triggers, foreign keys, GUID primary keys, or even the Write Ahead Log (WAL) can all cause delays. In this article, we will cover some best practice tips for bulk importing data into PostgreSQL databases. However, there may be situations where none of these ... Read More

Feed: Featured Blog Posts - Data Science Central. Author: Niranjan. Data science is a collective pool of various algorithms, tools, machine learning principles that work in unison to extract hidden patterns from raw data. It requires a diverse set of skills and demands knowledge from aspects of mathematics, science, communication, and business. Honing a diverse skill set, data scientists gain the ability to analyze numbers and influence decisions. The core objective of data scientists lay in bridging the gap between numbers and actions by using information to affect real world decisions. This demands excellent communication skills along with understanding the ... Read More

Feed: Featured Blog Posts - Data Science Central. Author: Kate Shao. As its name suggests, searching for videos by image is the process of retrieving from the repository videos containing similar frames to the input image. One of the key steps is to turn videos into embeddings, which is to say, extract the key frames and convert their features to vectors. Now, some curious readers might wonder what the difference is between searching for video by image and searching for an image by image? In fact, searching for the key frames in videos is equivalent to searching for an image by ... Read More

Feed: Planet PostgreSQL. Indexes are one of the core features of all the database management systems (DBMS). Indexes have a very long history in PostgreSQL, which has quite a rich set of index features. PostgreSQL has B-Tree, Hash,  GIN, GIST, and BRIN indexes. And because the development around indexes is still going on, PostgreSQL 13 provides some enhancements.We can divide the overall improvements into two categories:Changes which are transparent to the user. There won’t be any visible changes, but they will get the benefit automatically after the upgrade, probably after a rebuild of the index.  No application change required.The second ... Read More