Tag: GitHub

Feed: R-bloggers. Author: John Johnson. So the goal here is to focus back on Greenville County and have even more granularity. I look at median house prices near Greenville and then overlay the park data downloaded earlier. This time, for the Census data, I use the tidycensus package that came out recently. Furthermore, instead of using ggplot2 to create a static map, I use the leaflet package to create an interactive map, and, furthermore integrate data from disparate sources in a convenient way.The local parks file can be found here courtesy of a small group of dedicated volunteers and an ... Read More

Feed: IBM Big Data & Analytics Hub - All Content; Author: karan-sachdeva; The latest executive report published by IBM Institute for Business Value puts the estimated cost of cyber crime to the global economy in a range of USD 375–575 billion per year. Reputational damage, which is hard to calculate, comes on top of all this. No industry and geography has remained untouched with recent spurt of cyber attacks. Most companies have cyber security systems in place with some Fortune 500 and government agencies having highly advanced sophisticated systems in place. The difficulty is, not many have constructed living, dynamic and vibrant ... Read More

Feed: R-bloggers. Author: nsaunders. “‘Demographic tsunami’ will keep Sydney, Melbourne property prices high” screams the headline. While the census showed Australia overall is aging, there’s been a noticeable lift in the number of people aged between 25 to 32.As the accompanying graph shows… Whoa, that is one ugly chart. First thought: let’s not be too hard on Fairfax Media, they’ve sacked most of their real journalists and they took the chart from someone else. Second thought: if you want to visualise change over time, time as an axis rather than a coloured bar is generally a good idea. Can we ... Read More

Feed: Hadoop – Cloudera Engineering Blog. Author: Justin Kestelyn. History teaches us that ecosystem growth is fueled by enthusiasm, tools (including frameworks and APIs), and knowledge in roughly equal measures. To this point, the Apache Hadoop ecosystem has been blessed with the first two ingredients – thanks to the magic of open source – but in the third category, there is still plenty of work to be done. For Cloudera, our Academic Partnership program is a major part of that effort. Through that program, accredited nonprofit universities around the world get access to Cloudera’s own Hadoop curriculum for their computer ... Read More

Feed: Hadoop – Cloudera Engineering Blog. Author: Justin Kestelyn. In this installment of “Meet the Project Founder,” we speak with Josh Wills (@josh_wills), Cloudera’s Senior Director of Data Science and founder of Apache Crunch and Cloudera ML. What led you to your project idea(s)?When I first started at Cloudera in 2011, I had a fairly vague job description, no real responsibilities, and wasn’t all that familiar with the Apache Hadoop stack, so I started working on various pet projects in order to learn more about the tools and the use cases in domains like healthcare and energy. My first project, ... Read More

Feed: What's New. AWS Mobile Hub released enhancements to the project import feature today, allowing a developer to import Cloud Logic functions through the one-click import from GitHub feature.  With the Cloud Logic import feature, developers can also specify environment variables and CORS settings, allowing for the same code to be used for multiple functions with different settings (for example, test vs. production workloads or CRUD API for different tables within a DynamoDB database). The Cloud Logic import feature will also provide appropriate authorization for a linked Amazon DynamoDB database and Amazon API gateway, taking care of all resource deployments, ... Read More

Feed: R-bloggers. Author: John Johnson. US Census Data The US Census collects a number of demographic measures and publishes aggregate data through its website. There are several ways to use Census data in R, from the Census API to the USCensus2010 package. If you are interested in geopolitical data in the US, I recommend exploring both these options – the Census API requires a key for each person who uses it, and the package requires downloading a very large dataset. The setups for both require some effort, but once that effort is done you don’t have to do it again ... Read More

Feed: R-bloggers. Author: That’s so Random. Virtually every dataset has them; id variables that link a record to a subject and/or time point. Often one column, or a combination of columns, forms the unique id of a record. For instance, the combination of patient_id and visit_id, or ip_adress and visit_time. The first step in most of my analyses is almost always checking the uniqueness of a variable, or a combination of variables. If it is not unique, may assumptions about the data may be wrong, or there are data quality issues. Since I do this so often, I decided to ... Read More

Feed: Hadoop – Cloudera Engineering Blog. Author: Colin McCabe. One of the key principles behind Apache Hadoop is the idea that moving computation is cheaper than moving data — we prefer to move the computation to the data whenever possible, rather than the other way around. Because of this, the Hadoop Distributed File System (HDFS) typically handles many “local reads” reads where the reader is on the same node as the data: Initially, local reads in HDFS were handled the same way as remote reads: the client connected to the DataNode via a TCP socket and transferred the data via ... Read More

Feed: Planet MySQL. Author: Preetam Jinka. Back in May I wrote about a web service I’m working on that uses local storage and replication with two-phase commit. I pulled out the core of it and created a package that I’m calling the Rig.It’s up on GitHub already: https://github.com/Preetam/rig The goal of the Rig is to take some web service and add a log and replication on top. A service is simply something that accepts operations. type Service interface { Validate(Operation) error Apply(uint64, Operation) error LockResources(Operation) bool UnlockResources(Operation) } Each operation is associated with an entry in a log. type LogPayload ... Read More