Tag: python

Feed: Cloudera Engineering Blog » Hadoop. Author: Tom White. This piece is based on the talk “Practical MapReduce” that I gave at Hadoop User Group UK on April 14. 1. Use an appropriate MapReduce language There are many languages and frameworks that sit on top of MapReduce, so it’s worth thinking up-front which one to use for a particular problem. There is no one-size-fits-all language; each has different strengths and weaknesses. Java: Good for: speed; control; binary data; working with existing Java or MapReduce libraries. Pipes: Good for: working with existing C++ libraries. Streaming: Good for: writing MapReduce programs in ... Read More

Feed: Planet big data. Author: Praveen Sripati. I came across Algorithmia a few months back and didn't get a change to try it out. Again it came into focus with a Series A funding of $10.5M. More about the funding here.Algorithmia  is a place where algorithms, models or functions can be discovered and be used for credits which we can buy. We get 5,000 credits every month for free. For example if a model costs 20 credits, then it can be called 250 times a month.Create a free account here and get the API key from the profile. Now we ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Tom White. It’s a new year, the time when we take a moment to look back at the previous one, and forward to what might be coming next. In the world of Hadoop a lot happened in 2008. Organization At the beginning of the year, Hadoop was a sub-project of Lucene. In January, Hadoop became a Top Level Project at Apache, in recognition of its success and diversity of community. This allowed sub-projects to be added, the first of which was HBase, previously a contrib project. ZooKeeper, a service for coordinating distributed systems, ... Read More

Feed: All - O'Reilly Media. Author: Mike Lee Williams. Dice roll. (source: Pixabay). This article contains highlights from a series of three Orioles (interactive online tutorial notebooks) on probabilistic programming from scratch. In the Orioles, Mike Lee Williams explains both the mathematical background and the Python code more deeply, and delves into a variety of real-world statistical problems. If you enjoy teaching with, learning from, or sharing analyses using Jupyter notebooks, come to JupyterCon in New York, August 23-25, 2017. Real-world data is almost always incomplete or innaccurate in some way. This means that the uncertain conclusions we draw from ... Read More

Feed: Microsoft Azure Blog. Author: John Deutscher. This month we are announcing the release of support for Azure Active Directory (AAD) authentication in Azure Media Services. Customers of our REST API and .NET client libraries can now use AAD authentication to authorize requests. In addition, we are releasing a new management blade in the Azure Portal to simplify the usage of User and Service Principal authentication with AAD. With the release of this update to our REST API, we are now able to provide the same role-based access management (RBAC) as provided by the Azure Resource Management (ARM) service. By ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Tom White. The first release (0.19.0) from the 0.19 branch of Apache Hadoop Core was made on November 24. Many changes go into a release like this, and it can be difficult to get a feel for the more significant ones, even with the detailed Jira log, change log, and release notes. (There’s also JDiff documentation, which is a great way to see how the public API changed, via a JavaDoc-like interface.) This post gives a high-level feel for what’s new. Core Hadoop Core now requires Java 6 to run (HADOOP-2325). Most installations are ... Read More

Feed: Cloudera Engineering Blog » Hadoop. Author: Jeff Hammerbacher. It is common for a MapReduce program to require one or more files to be read by each map or reduce task before execution. For example, you may have a lookup table that needs to be parsed before processing a set of records. To address this scenario, Hadoop’s MapReduce implementation includes a distributed file cache that will manage copying your file(s) out to the task execution nodes. The DistributedCache was introduced in Hadoop 0.7.0; see HADOOP-288 for more detail on its origins. There is a great deal of existing documentation for ... Read More

Feed: Percona Database Performance Blog. Author: Tim Vaillancourt. Tim Vaillancourt  | June 21, 2017 |  Posted In: Insight for DBAs, Insight for Developers, MongoDB PREVIOUS POST In this short blog post, we will discuss a helpful feature for tracing MongoDB queries: Cursor Comments. Cursor Comments Much like other database systems, MongoDB supports the ability for application developers to set comment strings on their database queries using the Cursor Comment feature. This feature is very useful for both DBAs and developers for quickly and efficiently tying a MongoDB query found on the database server to a line of code in the application source. Once ... Read More

Feed: Latest imported feed items on Analytics Matters. Making intelligent and accurate predictions is the core objective of machine learning and artificial intelligence applications. To achieve that objective, the machine learning or artificial intelligence application needs clean and well-organized information in a robust ecosystem architecture. Machine Learning (ML) is the process of a computer system making a prediction based on samples of past observations. There are various types of ML methods. One of the approaches is where the ML algorithm is trained using a labeled or unlabeled training data set to produce a model. New input data is introduced to the ... Read More

Feed: Featured Blog Posts - Data Science Central. Author: Ramesh Dontha. One of the best ways to learn about any topic is start with very fundamental questions like What, Why etc? Good old Socratic method. In this series of articles on data mining, I plan to approach this topic in a similar fashion.  What is Data Mining? Simply put, Data mining is the process of sifting through large data sets to identify and describe patterns, discover and establish relationships with an intent to predict future trends based on those patterns and relationships. Why is data mining relevant now? Haven’t we been ‘mining’ data from time immemorial? Yes and No ... Read More