Tag: column

Feed: Planet PostgreSQL. This diary-of-an-engineer post from our Citus open source engineering team was originally published on the Azure Postgres Blog. When working on the internals of Citus, an open source extension to Postgres that transforms Postgres into a distributed database, we often get to talk with customers that have interesting challenges you won’t find everywhere. Just a few months back, I encountered an analytics workload that was a really good fit for Citus. But we had one problem: the percentile calculations on their data (over 300 TB of data) could not meet their SLA of 30 seconds. To make ... Read More

Feed: Planet MySQL; Author: Kristian Köhntopp; Iterating schemas over time is not an uncommon thing. Often requirements emerge only after you have data, and then directed action is possible. Consequently, working on existing data, and structuring and cleaning it up is a common task.In todays example we work with a log table that logged state transitions of things in freeform VARCHAR fields. After some time the log table grew quite sizeable, and the log strings are repeated rather often, contributing to the overall size of the table considerably. We are starting with this table: CREATE TABLE `log` ( `id` int ... Read More

Feed: AWS Big Data Blog. Stream processing is very useful in use cases where we need to detect a problem quickly and improve the outcome based on data, for example production line monitoring or supply chain optimizations.This blog post walks you through process of streaming existing data files and ongoing changes from Amazon Simple Storage Service (Amazon S3) to Amazon Kinesis. You achieve this by using AWS Database Migration Service (AWS DMS). AWS DMS enables you to seamlessly migrate data from supported sources to relational databases, data warehouses, streaming platforms, and other data stores in AWS cloud. Many SaaS, third-party ... Read More

Feed: Planet MySQL; Author: Johannes Schlüter; Recently there was a question about which clients were connected to a server being asked in the MySQL Community Slack. The relevant information is available from performance schema, as most connectors will send information about themselves when connecting: select * from performance_schema.session_connect_attrs; +----------------+-----------------+------------------------+------------------+ | PROCESSLIST_ID | ATTR_NAME | ATTR_VALUE | ORDINAL_POSITION | +----------------+-----------------+------------------------+------------------+ | 130 | _pid | 17412 | 0 | | 130 | _platform | x86_64 | 1 | | 130 | _os | Linux-5.4.0 | 2 | | 130 | _source_host | maniacmansion | 3 | | 130 | _client_name ... Read More

Feed: erwin Expert Blog – erwin, Inc.. Author: Michael Adjei. Replace manual and recurring tasks for fast, reliable data lineage and overall data governanceIt’s paramount that organizations understand the benefits of automating end-to-end data lineage. Critically, it makes it easier to get a clear view of how information is created and flows into, across and outside an enterprise.The importance of end-to-end data lineage is widely understood and ignoring it is risky business. But it’s also important to understand why and how automation plays a critical role.Benjamin Franklin said, “Lost time is never found again.” According to erwin’s “2020 State of ... Read More

Feed: Cloudwards. Author: Fergus O'Sullivan; Trello Basics However, we’ll stick to the basics for this piece so as not to make it into a multi-hour slog. The best place to begin is, as always, the beginning, so let’s start by taking a look at the fundamentals of Trello: boards, lists and cards. How to Create Boards Now, a kanban board is nothing without the actual board, so let’s start with that first. Assuming you’ve signed up to Trello — which is a simple matter of going to the website and hitting the appropriate button — all you need to do ... Read More

Feed: The Information Lab. Author: Ben Moss. 17 September, 2020 | Ben Moss Writing your output, to your input, in Alteryx We recently wrote a blog outlining a method that allows you to take your input source, and write it to the same file as your output. In this post we are going to show you the opposite, how we can write our output data to the same file as our input data, and it’s quite simple! First things first, as Excel is the only output type that supports ‘multiple tables’ in a single file, then this will only work ... Read More

Feed: Planet MySQL; Author: Mike Frank; The Challenge Often with sensitive information, you need to have an audit log. Not just that a table had a select run, but that specific cells within the table were accessed.  Frequently data such as this will contain a classification level as part of the row, defining policies for how it is handled, audited, etc. Sensitive Data such as that might be labeled as –  Highly Sensitive  Top Secret  Classified  Restricted  Clearance Required  Highly Confidential  Protected Data that is categorized or labeled in some manner is typically covered by regulations requiring your compliance. Compliance ... Read More

Feed: AWS Big Data Blog. Amazon Redshift is a fast, scalable, secure, and fully managed cloud data warehouse that makes it simple and cost-effective to analyze all your data using standard SQL, and your existing ETL, Business Intelligence (BI), and reporting tools. Tens of thousands of customers use Amazon Redshift to process exabytes of data per day and power analytics workloads such as BI, predictive analytics, and real-time streaming analytics. As a data engineer or application developer, for some use cases, you want to interact with Amazon Redshift to load or query data with a simple API endpoint without having ... Read More

Feed: James Serra's Blog. Author: James Serra. The public preview version of Azure Synapse Analytics has three compute options and four types of storage that it can access (mentioned in my blog at SQL on-demand in Azure Synapse Analytics). This gives twelve possible combinations of querying data. Not all of these combinations currently are supported and some have a few quirks of which I list below. (NOTE: I’ll demo these features at my sessions at European Digital Week on 9/25 (session info), SQL Bits on 10/3 (session info), PASS Summit on 11/10 (session info), and Big Data Conference Europe on ... Read More