Tag: Spark

Feed: Big Data Made Simple - One source. Many perspectives.. Author: Baiju NT. Big Data can be intimidating! If you are new to Big Data, please read ‘What is Big Data’, ‘Who coined Big Data’ to get you started. With the basic concepts under your belt, let’s focus on some key terms to impress your date or boss or family. By the way, I am putting together a much more exhaustive list of Big Data terms (almost 100) and if you are interested in that, please leave a comment below with a note ‘I want more of Big data’ or ... Read More

Feed: Hortonworks Blog – Hortonworks. Author: Wei Wang. I had the honor and pleasure to host an executive panel at “Women in Big Data Luncheon” on June 13, during DataWorks Summit in San Jose. Our distinctive panelists included: Anu Jain – Senior Director, Big Data Platforms, Tools and Products, TargetTendu Yogurtcu – CTO, SyncsortSuja Viswesan – Big Data Platform, LinkedInAlison Biggan – Global Head of Product/Industry Marketing, SAPRadhika Rangarajan – Founder of Women in Big Data, Director of Big Data, Intel I want to thank the panelists for such an inspiring and fun conversation. Second, thanks to the 200+ men ... Read More

Feed: Alation. Author: Stephanie McReynolds. We are living in a new era of data defined by two massively disruptive trends – one architectural and the other organizational. Architecturally the introduction of Hadoop, a file system designed to store massive amounts of data, radically affected the cost model of data. Organizationally the innovation of self-service analytics, pioneered by Tableau and Qlik, fundamentally transformed the user model for data analysis. A “big data” revolution has ensued.Disruptive Trend #1: Hadoop More than any other advancement in analytic systems over the last 10 years, Hadoop has disrupted data ecosystems. By dramatically lowering the cost ... Read More

Feed: IBM Big Data & Analytics Hub - All Content. Author: cindy-russell. It seems that we’re reaching the point where the Internet of Things (IoT) is moving from the domain of enthusiastic early-adopters to the more challenging, more profitable territory of mainstream enterprise technology.In particular, companies in industries that depend on the attention and loyalty of large numbers of consumers – retail and air travel, for example –have already woken up to the possibilities. If you can recognize an individual customer’s mobile device as they move through an airport or walk past your store, you can potentially take action to improve that ... Read More

Feed: The Information Lab. Author: Ravi Mistry. 23 June, 2017 | Ravi Mistry The Data School; 2 years on. On the 23rd of June 2015, there was a spark in Central London. And two years later, this spark is now a burning flame, which has been a roaring success. I am, of course, talking about the Data School. An idea started by Tom Brown, Craig Bloodworth and (perhaps the most important piece of the jigsaw) Andy Kriebel ended up in eight complete strangers taking a leap of faith and joining an immersion programme like no other. It promised four months ... Read More

Feed: AllegroGraph - THE Semantic Graph Database. Author: cnorvell. Webcast from June 15, 2016 Graphs and Artificial Intelligence have long been a focus for Franz Inc. and currently we are collaborating with Montefiore Health System, Intel, Cloudera, and Cisco to improve a patient's ability to understand the probabilities of their future health status. By combining artificial intelligence, semantic technologies, big data, graph databases and dynamic visualizations we are deploying a Cognitive Probability Graph concept as a means to help predict future medical events. The power of Cognitive Probability Graphs stems from the capability to combine the probability space (statistical patient ... Read More

Feed: Big Data Made Simple - One source. Many perspectives.. Author: Baiju NT. In the world of Big Data, information has evolved from being a merely passive construct into an active and dynamic abstraction. The language of this information is the computer code, but before diving into cluster programming, there are several things that you must first understand. Amdahl’s Law and the Limits of Parallel Computation Despite the power and promise of parallel computation, there are limits to what parallel computers can accomplish. In 1967, computer scientist Gene Amdahl presented a paper at that year’s AFIPS Spring Joint Computer Conference ... Read More

Feed: Hadoop Dev. Author: Nailah Bissoon. When YARN is enabled for Big SQL, YARN controls how much memory and CPU resources are assigned to Big SQL at any given point in time. Other applications such as Spark or Hive can continue to use YARN resources when YARN is enabled for Big SQL. Read more about Big SQL 4.2.5 and YARN integration. There are several YARN and Big SQL configuration parameters that are at play with YARN is enabled with Big SQL. These configuration parameters and best practice to setting these parameter are discussed in this blog. YARN Container Memory Settings ... Read More

Feed: Hadoop Dev. Author: Nailah Bissoon. YARN is a central resource manager component which manages and allocates resources to applications running on Hadoop. Multiple applications can run on Hadoop and they can share a common resource manager via YARN. Apache Slider allows for non-YARN services to be managed by YARN. Big SQL is integrated with YARN through Apache Slider. When a component requests resources from YARN, it is put on a YARN priority queue and YARN schedules the allocation based on priority. When the request is serviced, YARN will determine the number of containers that should be allocated to that ... Read More

Feed: Percona Database Performance Blog. Author: Vadim Tkachenko. Vadim Tkachenko  | June 22, 2017 |  Posted In: Benchmarks, MySQL, Performance Schema PREVIOUS POST In this blog post, we’ll look at how ClickHouse performs in a general analytical workload using the star schema benchmark test. We have mentioned ClickHouse in some recent posts (ClickHouse: New Open Source Columnar Database, Column Store Database Benchmarks: MariaDB ColumnStore vs. Clickhouse vs. Apache Spark), where it showed excellent results. ClickHouse by itself seems to be event-oriented RDBMS, as its name suggests (clicks). Its primary purpose, using Yandex Metrica (the system similar to Google Analytics), also points to ... Read More