Author: R-Bloggers

Feed: R-bloggers. Author: . In my last Statistics Sunday post, I briefly mentioned the concept of regular expressions, also known as regex (though note that in some contexts, these refer to different things – see here). A regular expression is a text string, which you ask your program to match. You can use this to look for files with a particular name or extension, or search a corpus of text for a specific word or word(s) that match a certain pattern. This concept is used in many programming languages, and R is no exception. In order to use a regular ... Read More

Feed: R-bloggers. Author: . First Statistics Sunday in far too long! It’s going to be a short one, but it describes a great trick I learned recently while completing a time study for our exams at work. To give a bit of background, this time study involves analzying time examinees spent on their exam and whether they were able to complete all items. We’ve done time studies in the past to select time allowed for each exam, but we revisit on a cycle to make certain the time allowed is still ample. All of our exams are computer-administered, and we ... Read More

Feed: R-bloggers. Author: . Two statistical indices crossed my inbox in the last week, both of which use fast food restaurants to measure a concept indirectly. First up, in the wake of recent hurricanes, is the Waffle House Index. As The Economist explains: Waffle House, a breakfast chain from the American South, is better known for reliability than quality. All its restaurants stay open every hour of every day. After extreme weather, like floods, tornados and hurricanes, Waffle Houses are quick to reopen, even if they can only serve a limited menu. That makes them a remarkably reliable if informal ... Read More

Feed: R-bloggers. Author: . Statistics Sunday: Some Psychometrics Tricks in R It’s been a long time since I’ve posted a Statistics Sunday post! Now that I’m moved out of my apartment and into my house, I have a bit more time on my hands, but work has been quite busy. Today, I’m preparing for 2 upcoming standard-setting studies by drawing a sample of items from 2 of our exams. So I thought I’d share what I’m up to in order to pass on some of these new psychometric tricks I’ve learned to help me with this project. Because I can’t ... Read More

Feed: R-bloggers. Author: . When You Need a New Book to ReadI log all of my books on Goodreads. On top of that, whenever I hear about a new book I have to read, I add it on Goodreads, so I remember it. Of course, this means my Goodreads bookshelves are a little out of control. Fortunately, I can use R to dig through my Goodreads to-read shelf and figure out the next book to read and/or buy. If you’re on Goodreads, you can easily download your entire bookshelf, including your to-read books, by going to “My Books” then clicking ... Read More

Feed: R-bloggers. Author: . Statistics Sunday: Visualizing RegressionI had some much needed downtime this weekend, after an exhausting week, along with some self-care – Saturday I had a one-hour deep tissue massage, which left me a little bruised but much more relaxed, and Sunday I spent a few hours in the salon chair having my color touched up, which left me much blonder. Which is why I’m a little late with my Statistics Sunday post, but today, I’m introducing another recently discovered r package: rpart. Short for “recursive partitioning,” this package creates decision trees for classification, regression, and survival analyses ... Read More

Feed: R-bloggers. Author: . Using Text Analysis to Become a Better WriterWe all have words we love to use, and that we perhaps use too much. As an example: I have a tendency to use the same transitional statements, to the point that, before I submit a manuscript, I do a find all to see how many times I’ve used some of my favorites, e.g., additionally, though, and so on. I’m sure we all have our own words we use way too often. Text analysis can also be used to discover patterns in writing, and for a writer, may be ... Read More

Feed: R-bloggers. Author: . Topics and Categories in the Russian Troll Tweets I decided to return to the analysis I conducted for the IRA tweets dataset. (You can read up on that analysis and R code here.) Specifically, I returned to the LDA results, which looked like they lined up pretty well with the account categories identified by Darren Linvill and Patrick Warren. But with slightly altered code, we can confirm that or see if there’s more to the topics data than meets the eye. (Spoiler alert: There is more than meets the eye.) I reran much of the original ... Read More

Feed: R-bloggers. Author: . IRA Tweet Data You may have heard that two researchers at Clemson University analyzed almost 3 millions tweets from the Internet Research Agency (IRA) – a “Russian troll factory”. In partnership with FiveThirtyEight, they made all of their data available on GitHub. So of course, I had to read the files into R, which I was able to do with this code: files c("IRAhandle_tweets_1.csv","IRAhandle_tweets_2.csv","IRAhandle_tweets_3.csv","IRAhandle_tweets_4.csv","IRAhandle_tweets_5.csv","IRAhandle_tweets_6.csv","IRAhandle_tweets_7.csv","IRAhandle_tweets_8.csv","IRAhandle_tweets_9.csv")my_files paste0("~/Downloads/russian-troll-tweets-master/",files)each_file function(file) {tweet read_csv(file) }library(tidyverse) tweet_data NULLfor (file in my_files) {temp each_file(file)temp$id sub(".csv", "", file)tweet_data rbind(tweet_data, temp)} Note that this is a large file, with 2,973,371 observations of 16 variables. Let’s do ... Read More

Feed: R-bloggers. Author: . Highlighting Specific Cases in ggplot2Here’s my belated Statistics Sunday post, using a cool technique I just learned about: gghighlight. This R package works with ggplot2 to highlight a subset of data. To demonstrate, I’ll use a dataset I analyzed for a previous post about my 2017 reading habits. [Side note: My reading goal for this year is 60 books, and I’m already at 43! I may have to increase my goal at some point.] setwd("~/R")library(tidyverse) booksread_csv("2017_books.csv", col_names = TRUE) ## Warning: Duplicated column names deduplicated: 'Author' => 'Author_1' [13] ## Parsed with column specification:## cols(## .default ... Read More