Tag: Pip

Feed: R-bloggers. Author: hrbrmstr. I’ve blathered about trust before 12, but said blatherings were in a “what if” context. Unfortunately, the if has turned into a when, which begged for further blathering on a recent FOSS ecosystem cybersecurity incident. The gg_spiffy @thomasp85 linked to a post by the SK-CSIRT detailing the discovery and take-down of a series of malicious Python packages. Here’s their high-level incident summary: SK-CSIRT identified malicious software libraries in the official Python packagerepository, PyPI, posing as well known libraries. A prominent example is a fakepackage urllib-1.21.1.tar.gz, based upon a well known packageurllib3-1.21.1.tar.gz. Such packages may have been ... Read More

Feed: AWS Big Data Blog. Amazon EMR lets you have complete control over your cluster, giving you the flexibility to customize a cluster and install additional applications easily. EMR customers often use bootstrap actions to install and configure custom software in a cluster. However, bootstrap actions only run during the cluster or node startup. This makes it difficult for you to make configuration changes after a cluster is already running.EMR clusters can also use a custom Amazon Machine Image (AMI). With the new support for launching clusters with custom Amazon Linux AMIs, customizing an EMR cluster is now even easier ... Read More

Feed: Cloud – Cloudera Engineering Blog. Author: Cy Jervis. This article shows how to build and publish a customized Docker image for usage as an engine in Cloudera Data Science Workbench. Such an image or engine customization gives you the benefit of being able to work with your favorite tool chain inside the web based application. Motivation: Cloudera Data Science Workbench (CDSW) enables data scientists to use their favorite tools such as R, Python, or Scala based libraries out of the box in an isolated secure sandbox environment. In a previous blog post I have shown an Apache Maven based ... Read More

Feed: All - O'Reilly Media. Author: Phillip Kuznetsov, Noah Meyer Golmant. Montage (source: Mary-Lynn on Flickr) Check out the accompanying GitHub repo with the Python code and Jupyter Notebook for the article. Pixel Recurrent Neural Networks (PixelRNNs) combine a number of techniques to generate natural-looking images using neural networks. PixelRNNs model the distribution of image data sets using several new techniques, including a novel spatial LSTM cell, and sequentially infer the pixels in an image to (a) generate novel images or (b) predict unseen pixels to complete an occluded image. Figure 1. Images produced by a PixelRNN model trained on ... Read More

Feed: Featured Blog Posts - Data Science Central. Author: Favio Vázquez. Outdated, inaccurate, or duplicated data won’t drive optimal data driven solutions. When data is inaccurate, leads are harder to track and nurture, and insights may be flawed. The data on which you base your big data strategy must be accurate, up-to-date, as complete as possible, and should not contain duplicate entries. Clean data results in better decisions. Cleaning data is the most time-consuming and least enjoyable data science task (until Optimus), but one of the most important ones. No one can start a data science, machine learning or data driven solution without being sure that the ... Read More

Feed: CDH – Cloudera Engineering Blog. Author: Cy Jervis. Cloudera Data Science Workbench provides freedom for data scientists. It gives them the flexibility to work with their favorite libraries using isolated environments with a container for each project. In JVM world such as Java or Scala, using your favorite packages on a Spark cluster is easy. Each application manages preferred packages using fat JARs, and it brings independent environments with the Spark cluster. Many data scientists prefer Python to Scala for data science, but it is not straightforward to use a Python library on a PySpark cluster without modification. To ... Read More

Feed: All - O'Reilly Media. Author: Manu Jeevan, Suresh Chelladurai. Road signs (source: Pixabay) The full Jupyter Notebook for this post can be accessed here. Although there are many deep learning frameworks, including TensorFlow, Keras, Torch, and Caffe, Apache MXNet in particular is gaining popularity due to its scalability across multiple GPUs. In this blog post, we'll tackle a computer vision problem: classifying German traffic signs using a convolutional neural network. The network takes a color photo containing a traffic sign image as input, and tries to identify the type of sign. The full notebook can be found here In ... Read More

Feed: Planet big data. Author: Kiril Schewzow. In this post we will use modern Natural Language Processing techniques to find similar posts in a Facebook group.You have probably been in a situation where you want to post something in a Facebook group but you are not sure whether almost the same post already exists and is maybe just hiding on the next page.You could click through the pages and read all the posts or you could use the search function, but both ways have their disadvantages. While the first one is obviously very tedious, for the second one you will ... Read More

Feed: Microsoft Azure Blog. Author: Alice Jin. To help you leverage your telemetry data and better monitor the behavior of your Azure applications, we are happy to provide a Jupyter Notebook template that extends the power of Application Insights. Instead of making ad hoc queries in the Application Insights portal when an issue arises, you can now write a Jupyter Notebook that routinely queries for telemetry data, performs advanced analytics, and sends the derived data back to Application Insights for monitoring and alerting. You can execute the Jupyter Notebook using Azure WebJob either on a schedule or via webhook. Through ... Read More

Feed: Google Cloud Big Data and Machine Learning Blog. Author: Google Cloud Big Data and Machine Learning Blog Team. Innovation in data processing and machine learning technology Tuesday, July 11, 2017 By Tahir Fayyaz, Customer Engineer Users of Google BigQuery, the cloud-native data warehouse service from Google Cloud Platform (GCP), have access to an ever-expanding range of public datasets for exploration. Some of these datasets are quite large, however, so connecting to them directly from a dashboard created in Google Data Studio or other tools can cause undesirable latency (or costs if you have exceeded your free trial or tier) ... Read More