Tag: JSON

Feed: AWS Big Data Blog. Data is being produced, streamed, and consumed at an immense rate, and that rate is projected to grow exponentially in the future. In particular, JSON is the most widely used data format across streaming technologies and workloads. As applications, websites, and machines increasingly adopt data streaming technologies such as Apache Kafka and Amazon Kinesis Data Streams, which serve as a highly available transport layer that decouples the data producers from data consumers, it can become progressively more challenging for teams to coordinate and evolve JSON Schemas. Adding or removing a field or changing the data ... Read More

Feed: Recent Announcements. Amazon Quantum Ledger Database (Amazon QLDB) now supports JSON output format for QLDB Exports, in addition to existing Amazon Ion output format. Customers can select JSON when requesting an export and will receive journal data in newline-delimited JSON format. JSON is an open standard data interchange format that is widely used by Amazon customers. QLDB export with JSON format enables direct integration with common software libraries, tools, and popular Amazon services such as AWS Glue and Amazon Athena. Amazon QLDB is a fully managed ledger database that provides a transparent, immutable, and cryptographically verifiable log. Customers can use ... Read More

Feed: Planet NoSQL. Author: Keshav Murthy.  Use N1QL when you're in a JSON pickle. - ConfuciusFor the JSON data model, the general advice is to think of collections as tables, JSON documents as denormalized rows, and field names as columns — roughly. All this holds true in databases like Couchbase and MongoDB when recommendations are strictly followed. There are many reasons why users don't simply follow this key-value pair model all the time. Here are the main reasons:You want to convert a map/hashmap data structure where the keys are dynamic.Timeseries data when the field names are usually encoded timestamps.Dictionary based encoding.Existing document ... Read More

Feed: Planet NoSQL. Author: Keshav Murthy.  AbstractCouchbase is a distributed JSON database. It provides distributed data processing, high-performance indexing, and declarative query language N1QL along with search index, eventing, and analytics.N1QL (Non First Normal Query Language) is SQL for JSON. N1QL is to JSON what SQL is to relations. Developers say what needs to be done and the N1QL engine will figure out the "how." Current N1QL optimizer is a rule-based optimizer. Cost-Based Optimizer (CBO) for SQL was invented at IBM some 40 years ago and has been critical to the success of RDBMS and the developer/DBA productivity. NoSQL databases were invented some ... Read More

Feed: Planet NoSQL. Author: Keshav Murthy. Enterprise applications require both exact search (equality, range predicate) and pattern search (LIKE or text search). The B-Tree based indexes in databases help perform an exact search in milliseconds. Pattern search is a whole different problem. Predicates of patterns (name LIKE "%turin%") will scan the whole index — unsuitable for application use. Recognizing the need for speed, we've introduced a TOKENS() functions in Couchbase 4.6 which retuns an array of tokens from each field or document. We then index the array using the array indexing in Couchbase N1QL. This transforms the pattern search problem to a simple equality look up, ... Read More

Feed: Planet NoSQL. Author: Keshav Murthy.   Array is THE difference between the relational model and the JSON model.   — Gerald SangudiAbstractJSON array gives you flexibility in the type of elements,  number of elements, size of the elements, and the depth of the elements.  This adds to the flexibility of operational JSON databases like Couchbase and MongoDB. The performance of queries with an array predicate in operational databases depends on the array indexes. However, the array indexes in these databases come with significant limitations. E.g. only one array key is allowed per index. The array indexes, even when created, can only process AND predicates efficiently ... Read More

Feed: Planet PostgreSQL. Time for a renovation on an old post that dealt with Postgres and hstore. This post will move from the hstore data type to json(b). At the time the original post was written introduction of the json type was about three months away and jsonb two and half years out. It also took several years for the operator and function choices to be fleshed out. Now that the support has been mature for some time it seems a good time to make the change. To recap, the original post dealt with using hstore to capture deleted records ... Read More

Feed: Redis. Author: eduardo.carneiro. JSON is the de facto format for data exchange, a common output of modern APIs, and modeling application data as JSON is intuitive for developers. Being able to store and query JSON data in its natural format, and without translation, enables developers to iterate faster and be more productive. Document databases offer a flexible data model without needing to pre-define anything about their structure or values. This makes them ideal for scenarios where the data model is constantly changing or not fully known at the start of the project. However, the lack of predefined schema often ... Read More

Feed: Planet PostgreSQL. As previously mentioned here on the blog, PostgreSQL 14 was just released. Need a primer? Umair Shahid recently put together a great overview, PostgreSQL 14 – Performance, Security, Usability, and Observability. I recently did a deep-dive into using JSON from within PostgreSQL in a two blog post series titled Storing and using JSON within PostgreSQL, While all the information from those two blogs still is relevant with the release of PostgreSQL 14 (PG14), we need to provide a quick update to show off some improvements. Keeping this brief:  Prior to PG14, pulling fields out of JSON looked ... Read More

Feed: Planet PostgreSQL. In the last installment of the pipeline blog series, we explored writing streaming JSON data into PostgreSQL using Kafka Connect. In this blog, it’s time to find out if our fears of encountering mutant monsters in the gloom of an abandoned uranium mine were warranted or not.1. What Is Apache Superset?What is Apache Superset? Well, it could be a set of (possibly mutant) superheroes (supposedly, many superheroes got their powers from mutations, and biological mutations are actually normal and not a byproduct of radiation)?(Source: Shutterstock)Or perhaps it could be related to a mathematical superset? B is a ... Read More