Tag: INSERT

Feed: Google Cloud Big Data and Machine Learning Blog. Author: Google Cloud Big Data and Machine Learning Blog Team. Innovation in data processing and machine learning technology Monday, June 26, 2017 By Jibran Saithi, Lead Architect, Qubit At Qubit, we enable over 400 million personalized experiences per month, and to power those experiences we process data from websites, mobile apps and other customer touchpoints. Managing and processing this data in near-real time is a significant challenge; various stages of the pipeline process events at over 500K messages per second, all of which need to be validated, enriched and routed with ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Aaron Kimball. Cloudera customers usually have two major sources of data: log files, which can be imported to Hadoop via Flume, and relational databases. Throughout the previous releases of CDH2 and CDH3, Cloudera has included a package we’ve developed called Sqoop. Sqoop can perform batch imports and exports between relational databases and Hadoop, storing data in HDFS and creating Hive tables to hold results. We described its motivation and some use cases in a previous blog post a while ago. In CDH3b2, we’ve included a greatly-expanded version of Sqoop which has had a ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Chad Metcalf. One of the more common requests we receive from the community is to package Apache HBase with Cloudera’s Distribution for Apache Hadoop. Lately, I’ve been doing a lot of work on making Cloudera’s packages easy to use, and recently, the HBase team has pitched in to help us deliver compatible HBase packages. We’re pretty excited about this, and we’re looking forward to your feedback. A big thanks to Andrew Purtell, a Senior Architect at TrendMicro and HBase Contributor, for leading this packaging project and providing this guest blog post. -Chad Metcalf ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Amr Awadallah. (guest blog post by Pete Skomoroch) In a previous post, I outlined how to build a basic trend tracking site called trendingtopics.org with Cloudera’s Distribution for Hadoop and Hive.  TrendingTopics uses Hadoop to identify the top articles trending on Wikipedia and displays related news stories and charts.  The data powering the site was pulled from an Amazon EBS Wikipedia Public Dataset containing 8 months of hourly pageview logfiles.  In addition to the pageview logs, the EBS data volume also includes the full text content and link graph for all articles.  This ... Read More

Feed: Planet MySQL. Author: Valeriy Kravchuk. This is a "Howto" kind of post, and some speculations and historical references aside, it will show you how to build MariaDB ColumnStore (current version 1.0.9) from GitHub source, how to install and configure it for basic usage, as well as how to resolve some minor problems you may get in the process. Keep reading and eventually you'll get the real Howto below :) * * * I try not to care about any software issues besides good old MySQL, InnoDB storage engine internals, query optimization and some MyRocks or a little bit of ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Amr Awadallah. At Cloudera, we frequently work with leading Hadoop developers to produce guest blog posts of general interest to the community. We started a project with Pete Skomoroch a while back, and we were so impressed with his work, we’ve decided to bring Pete on as a regular guest blogger. Pete can show you how to do some pretty amazing things with Hadoop, Pig and Hive and has a particular bias towards Amazon EC2. With that, I’m happy to welcome Pete to the blog, and hope you enjoy his first post as much ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Christophe Bisciglia. On June 10th, more than 750 people from around the world descended on the Santa Clara Marriott to share their love for a little stuffed elephant named Hadoop. It was a good week to be part of this exploding community, and I want to extend Cloudera’s heartfelt thanks to everyone who made it possible, especially our friends at Yahoo! who organized this Summit. Most importantly, I want to thank all of you who were able to participate. I know many of you couldn’t make it to California this time, so I ... Read More

Feed: Planet MySQL. Author: MySQL Server Dev Team. MySQL 8.0.1では、utf8mb4の大文字小文字およびアクセント記号付きの文字を判別するas_cs collationに加えて、日本語用のCollation(文字照合順)を追加しました。 utf8mb4_ja_0900_as_csについて 日本語に関する文字照合およびソートのルールは複雑です。日本語ではひらがな、カタカナ、漢字、アルファベット(ラテン文字)を混在させて利用しています。さらに、全角と半角が存在する文字もあります。では、‘あ’, ‘ア’, ‘a’, ‘ｱ’はどのようにソートされるのでしょうか？ Unicode照合アルゴリズム(UCA / Unicode Collation Algorithm)の中で規定されているデフォルトUnicode照合基本テーブル(DUCET / Default Unicode Collation Element Table)にてUnicode文字列の比較アルゴリズムを定義しています。このDUCETは言語ごとにカスタマイズされた情報をXML形式で提供する共通ロケールデータリポジトリ(CLDR, Common Locale Data Repository)にて、日本語に関するソート順を定義した部分[reorder Latn Kana Hani]での日本語に関するソート順のルールによると、ラテン文字は全てのひらがなやカタカナよりも前に位置づけられているため‘a’が最初となります。では残りの‘あ’, ‘ア’, ‘ｱ’はどうなるでしょうか？ここでは以下のルールが適用されます: ‘&あ なぜ‘あ’, ‘ア’, ‘ｱ’は等価なのでしょうか？UCAではマルチレベルの比較アルゴリズムを使用して照合を個なっています。この比較アルゴリズムの4次レベルでは‘あ’が‘ア’よりも前に来るとされているにも関わらずです。ポイントはCLDRでは日本語の文字照合順の強度は3次レベルであると定義されています。従って4次レベルでの差は無視されることとなります。これはユーザーの期待した挙動ではないかもしれません。そこで我々は日本語を扱うMySQLユーザーの要望をもとにしたCollationをMySQL 8.0に追加することを検討しています。 日本工業規格の一つJIS X 0208 (http://www.jisc.go.jp/app/pager?id=94516)では、日本語の漢字6,355文字とそのソート順について定義しています。utf8mb4_ja_0900_as_csはこのJIS X 0208に準じた漢字のソートを行います。しかしこのJIS X 0208では含まれない漢字も多数存在します。これらの漢字についてutf8mb4_ja_0900_as_csはUCA(Unicode Collation Algorithm)で定義されたそれぞれの文字の重み(implicit weight) (http://www.unicode.org/reports/tr10/#Implicit_Weights)を利用します。 以下の文字を例にソート順を確認してみます: ‘王’, ‘人’, ‘兵’, ‘﨎’, ‘㐀’最初の3文字はJIS X 0208で定義されていますが、残りの2文字は定義されていません。 mysql> create table jpn(a varchar(10)) collate utf8mb4_ja_0900_as_cs; Query OK, 0 rows affected (0.90 sec) mysql> insert into jpn values ('王'), ('﨎'), ('人'), ('㐀'), ('兵'); Query OK, 5 rows affected (0.05 sec) Records: 5 Duplicates: 0 Warnings: 0 mysql> select ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Aaron Kimball. Editor’s note (added Nov. 9. 2013): Valuable data in an organization is often stored in relational database systems. To access that data, you could use external APIs as detailed in this blog post below, or you could use Apache Sqoop, an open source tool (packaged inside CDH) that allows users to import data from a relational database into Apache Hadoop for further processing. Sqoop can also export those results back to the database for consumption by other clients. Apache Hadoop’s strength is that it enables ad-hoc analysis of unstructured or semi-structured data. Relational databases, by contrast, ... Read More

Feed: AllegroGraph - THE Semantic Graph Database. Author: cnorvell. Franz Extends its Semantic Graph Database AllegroGraph 5.0 Unveils New Multi-dimensional Indexing for Complex Location Aware Applications OAKLAND, Calif. — February 9, 2015 — Franz, Inc., the leading supplier of Semantic Graph Database technology, today announcedAllegroGraph 5.0 with unique multi-dimensional indexing capabilities. Data Scientists can think of these new multi-dimensional indices as similar to OLAP data cubes or HyperCubes but with the powerful dynamic data linking of a Graph Database. AllegroGraph was the first Graph Database to offer advanced temporal and geospatial libraries. With this latest release AllegroGraph again leads the industry with multi-dimensional event based data processing for solutions ... Read More