Tag: Updates

Feed: CDH – Cloudera Engineering Blog. Author: Cy Jervis. Analytical and operational access patterns are very different and until now the Hadoop ecosystem has not had a single storage engine that could support both. As a result, engineers have been forced to implement complex architectures that stitch multiple systems together in order to provide these capabilities. On one hand immutable data on HDFS offers superior analytic performance, while mutable data in Apache HBase is best for operational workloads. Apache Kudu bridges this gap.   Kudu’s architecture is shaped towards the ability to provide very good analytical performance, while at the ... Read More

Feed: R-bloggers. Author: Thinking inside the box. Welcome to the tenth post in the rarely ranting R recommendations series, or R4 for short. A few days ago we showed how to tell the linker to strip shared libraries. As discussed in the post, there are two options. One can either set up ~/.R/Makevars by passing the strip-debug option to the linker. Alternatively, one can adjust src/Makevars in the package itself with a bit a Makefile magic. Of course, there is a third way: just run strip --strip-debug over all the shared libraries after the build. As the path is standardized, ... Read More

Feed: All - O'Reilly Media. Author: Elsa Sotiriadis. A 1925 artist's conception of what Nikola Tesla's wireless power transmission system might look like in the future (source: Wikimedia Commons) Sign up for BioCoder news and updates to say on top of the biological revolution. Biology is becoming the new digital. As the engineering of biology starts delivering solutions at industrial scale while becoming more data-driven and automated, investors are getting excited to fund breakthrough biotechnologies that unlock this potential for a whole range of industries. Europe has been steadily rising as a global hub for innovation and is attracting big ... Read More

Feed: R-bloggers. Author: Florian Teschner. The last months, I have worked on brand logo detection in R with Keras. Starting with a model from scratch adding more data and using a pretrained model. The goal is to build a (deep) neural net that is able to identify brand logos in images. Just to recall, the dataset is a combination of the Flickr27-dataset, with 270 images of 27 classes and self-scraped images from google image search. In case you want to reproduce the analysis, you can download the set here. In the last post, I used the VGG-16 pretrained model and ... Read More

Feed: R-bloggers. Author: statcompute. The dropout approach developed by Hinton has been widely employed in deep learnings to prevent the deep neural network from overfitting, as shown in https://statcompute.wordpress.com/2017/01/02/dropout-regularization-in-deep-neural-networks. In the paper http://proceedings.mlr.press/v38/korlakaivinayak15.pdf, the dropout can also be used to address the overfitting in boosting tree ensembles, e.g. MART, caused by the so-called “over-specialization”. In particular, while first few trees added at the beginning of ensembles would dominate the model performance, the rest added later can only improve the prediction for a small subset, which increases the risk of overfitting. The idea of DART is to build an ensemble by ... Read More

Feed: R-bloggers. Author: Thinking inside the box. On the heels of the very recent bi-monthly RcppArmadillo release comes a quick bug-fix release 0.7.960.1.1 which just got onto CRAN (and I will ship a build to Debian in a moment). There were three distinct issues I addressed in three quick pull requests: The excellent Google Summer of Code work by Binxiang Ni had only encountered direct use of sparse matrices as produced by the Matrix. However, while we waited for 0.7.960.1.0 to make it onto CRAN, the quanteda package switched to derived classes—which we now account for via the is() method ... Read More

Feed: R-bloggers. Author: sindri. Probability is at the heart of data science. Simulation is also commonly used in algorithms such as the bootstrap. After completing this exercise, you will have a slightly stronger intuition for probability and for writing your own simulation algorithms. Most of the problems in this set have an exact analytical solution, which is not the case for all probability problems, but they are great for practice since we can check against the exact correct answer. To get the most out of the exercises, it pays off to read the instructions carefully and think about what the solution ... Read More

Feed: R-bloggers. Author: R on Guangchuang Yu. I am very glad to find that someone figure out how to use ggjoy with ggtree. I really love ggjoy and believe it can be a good tool to visualize gene set enrichment (GSEA) result. DOSE/clusterProfiler support several visualization methods. running score: enrichment map: category-gene-network: dotplot: These visualization methods are designed for visualizing/summarizing enrichment results and none of them can incoporate expression values. In DOSE v>=3.3.2, I defined a joyplot function, which can visualize GSEA result with expression distribution of the enriched categories. Here is an example: require(clusterProfiler) data(geneList) x Related To leave ... Read More

Feed: R-bloggers. Author: Peter's stats stuff - R. I recently put up a simple web app, built with R Shiny, to let users explore the relationship between party vote in the 2014 New Zealand general election and a range of demographic and attitudinal questions in the New Zealand Election Study. The image below is a link to the web app: The original motivation was to answer a question on Twitter for a breakdown of National party vote by gender. I was surprised how interesting I found the resulting tool though. Without a fancy graphic, just a table of numbers, there’s ... Read More

Feed: R-bloggers. Author: statcompute. The dropout approach developed by Hinton has been widely employed in the context of deep learnings to prevent the deep neural network from over-fitting, as shown in https://statcompute.wordpress.com/2017/01/02/dropout-regularization-in-deep-neural-networks. In the paper http://proceedings.mlr.press/v38/korlakaivinayak15.pdf, the dropout is also proposed to address the over-fitting in tree boosting ensembles, e.g. MART, caused by the so-called “over-specialization”. In particular, while first few trees added at the beginning of ensembles would dominate the model performance, the rest trees added later can only contribute to learning from a small subset, which increases the risk of over-fittings. The idea of DART is to build ... Read More