Tag: Big Data

Feed: Big Data Made Simple - One source. Many perspectives.. Author: Baiju NT. Big data has been advertised as a panacea for corporate problems. Everything from marketing to HR or logistics to the sales funnel can be solved by AI algorithms which feed on Big Data. Yet, only a third of the respondents from a recent Forbes survey had seen revenue boosts of over 3% after implementing Big Data, showing there is still room to grow and the processes are still not entirely understood.  Why are the numbers so small, what makes projects fail and  what causes CEOs to lose ... Read More

Feed: R-bloggers. Author: David Smith. What do women do in films? If you analyze the stage directions in film scripts — as Julia Silge, Russell Goldenberg and Amber Thomas have done for this visual essay for ThePudding — it seems that women (but not men) are written to snuggle, giggle and squeal, while men (but not women) shoot, gallop and strap things to other things.   This is all based on an analysis of almost 2,000 film scripts mostly from 1990 and after. The words come from pairs of words beginning with "he" and "she" in the stage directions (but not ... Read More

Feed: R-bloggers. Author: hrbrmstr. I’ve blathered about my crawl_delay project before and am just waiting for a rainy weekend to be able to crank out a follow-up post on it. Working on that project involved sifting through thousands of Web Archive (WARC) files. While I have a nascent package on github to work with WARC files it’s a tad fragile and improving it would mean reinventing many wheels (i.e. there are longstanding solid implementations of WARC libraries in many other languages that could be tapped vs writing a C++-backed implementation). One of those implementations is JWAT, a library written in ... Read More

Feed: R-bloggers. Author: Nicholas Tierney. Take a look at the data This is a phrase that comes up when you first get a dataset. It is also ambiguous. Does it mean to do some exploratory modelling? Or make some histograms, scatterplots, and boxplots? Is it both? Starting down either path, you often encounter the non-trivial growing pains of working with a new dataset. The mix ups of data types – height in cm coded as a factor, categories are numerics with decimals, strings are datetimes, and somehow datetime is one long number. And let's not forget everyone's favourite: missing data ... Read More

Feed: R-bloggers. Author: Rstats on Julia Silge. I have a new visual essay up at The Pudding today, using text mining to explore how women are portrayed in film. The R code behind this analysis in publicly available on GitHub. I was so glad to work with the talented Russell Goldenberg and Amber Thomas on this project, and many thanks to Matt Daniels for inviting me to contribute to The Pudding. I’ve been a big fan of their work for a long time! Related To leave a comment for the author, please follow the link and comment on their blog: ... Read More

Feed: R-bloggers. Author: Data Science Riot!. The recent release of the blscrapeR package brings the “tidyverse” into the fold. Inspired by my recent collaboration with Kyle Walker on his excellent tidycensus package, blscrapeR has been optimized for use within the tidyverse as of the current version 3.0.0. New things you’ll notice right away include: All data now returned as tibbles. dplyr and purrr are now imported packages, along with magrittr and ggplot, which were imported from the start. No need to call any packages other than tidyverse and blscrapeR. Major internal changes install.packages("blscrapeR") The BLS: More than Unemployment The American ... Read More

Feed: Planet big data. Author: 9Clouds. Using a virtual machine while working in the Deep Web and Dark Web is key to obtaining information safely and securely. At BrightPlanet, we’ve communicated the benefits of using a virtual machine while working in the traditionally “unsafe” Deep Web and Dark Web.Before business leaders make important decisions, they should have access to all relevant information leading up to the decision. In addition to helping navigate Deep Web and Dark Web search, virtual machines are also helpful in harvesting open source data from public websites, allowing you to obtain all the relevant information you ... Read More

Feed: R-bloggers. Author: R Views. This example groups stocks together in a network that highlights associations within and between the groups using only historical price data. The result is far from ground-breaking; you can already guess the output. For the most part, the stocks get grouped together into pretty obvious business sectors. Despite the obvious result, the process of teasing out latent groupings from historic price data is interesting. That’s the focus of this example. A central idea of the approach taken here comes from the great paper of Ledoit and Wolf, “Honey, I Shrunk the Sample Covariance Matrix” (http://www.ledoit.net/honey.pdf) ... Read More

Feed: Featured Blog Posts - Data Science Central. Author: William Vorhies. Summary:  Reinforcement Learning (RL) is likely to be the next big push in artificial intelligence.  It’s the core technique for robotics, smart IoT, game play, and many other emerging areas.  But the concept of modeling in RL is very different from our statistical techniques and deep learning.  In this two part series we’ll take a look at the basics of RL models, how they’re built and used.  In the next part, we’ll address some of the complexities that make development a challenge. Now that we have pretty much conquered ... Read More

Feed: R-bloggers. Author: John Mount. The R package seplyr supplies a few neat new coding notations. An Abacus, which gives us the term “calculus.” The first notation is an operator called the “named map builder”. This is a cute notation that essentially does the job of stats::setNames(). It allows for code such as the following: library("seplyr") names a b #> "x" "y" This can be very useful when programming in R, as it allows indirection or abstraction on the left-hand side of inline name assignments (unlike c(a = 'x', b = 'y'), where all left-hand-sides are concrete values even if ... Read More