Tag: Hive

Feed: AWS Big Data Blog. Whatever your industry, data is central to how organizations function. When we discuss data strategy with customers, it’s often in the context of how do they ingest, store, process, analyze, distribute, and ultimately secure data.Amazon EMR is a managed Hadoop framework that you use to process vast amounts of data. One of the reasons that customers choose Amazon EMR is its security features. For example, customers like FINRA in regulated industries such as financial services, and in healthcare, choose Amazon EMR as part of their data strategy. They do so to adhere to strict regulatory ... Read More

Feed: What's New. TIBCO Data Science is a big data analytics platform for enterprises that provides an array of tools (from visual workflows to Jupyter Python Notebooks) and connects natively to most data sources, including Apache Hadoop, Spark, Hive, and relational databases. The collaborative user interface enables data scientists, data engineers, and business users to work together and build machine learning workflows in an intuitive web interface with a minimum of code. The advanced analytics platform provides security and governance. It also enables the analytics team to share and deploy predictive analytics and machine learning insights with the rest of ... Read More

Feed: CDH – Cloudera Engineering Blog. Author: Tom Wheeler. For a user-facing system like Apache Impala, bad performance and downtime can have serious negative impacts on your business. Given the complexity of the system and all the moving parts, troubleshooting can be time-consuming and overwhelming. In this blog post series, we are going to show how the charts and metrics on Cloudera Manager (CM) can help troubleshoot Impala performance issues. They can also help to monitor the system to predict and prevent future outages. CM provides a comprehensive suite of time-series and pre-aggregated metrics and charts at varying levels of ... Read More

Feed: Hortonworks Blog – Hortonworks. Author: Bikas Saha. This is the third in a series of data engineering blogs that we plan to publish. The first blog outlined the data science and data engineering capabilities of Hortonworks Data Platform. Motivation Apache Spark is becoming the de-facto processing framework for all kinds of complex processing including ETL, LOB business data processing and machine learning. This is because of the simple and intuitive APIs for common data processing operations and deep integration with popular languages like Java, Scala, Python and R. Increasingly, it’s also being adopted for continuous processing scenarios using the ... Read More

Feed: Hortonworks Blog – Hortonworks. Author: Dinesh Chandrasekhar. We are excited to announce the General Availability of Hortonworks DataFlow (HDF) 3.3. Throughout 2018, the HDF releases have been focusing on making the platform more robust for taking on advanced streaming architectures. We brought about several innovations and key enhancements on the operational side as well as on the development side of the enterprise. We have made HDF a highly reliable platform for Kafka in keeping up with the latest release as well as making sure that it is tightly integrated with the rest of our platform for comprehensive security and ... Read More

Feed: SAS Blogs. Author: Alyssa Grube. Sometimes, during my day-to-day job, I get to experience some really fun and unique things. I’m thankful that these experiences, while work-related, sometimes spark deeper reflection about the world around me. Case in point: a few weeks ago, I zipped into a bee suit to tour and learn about SAS’ beehives and left the experience with a renewed perspective on sustainability and teamwork. Wait. Hold up. Beehives? At a software company? Yup, you read that right! We’ve got honey bees! About 125,000 of them, actually. Since 2017, SAS has partnered with Bee Downtown to ... Read More

Feed: Microsoft Azure Blog. Author: Rob Caron. Microsoft Connect(); 2018 On Tuesday, December 4th, Microsoft Connect(); 2018 provided a full day of developer-focused content—including updates on Azure and Visual Studio, keynotes, demos, and real-time coding with experts. Scott Guthrie’s keynote provided all the Azure and Visual Studio news on how you can stay productive and focus on what matters to you. Guthrie’s keynote was followed by a demo-packed session hosted by Scott Hanselman and friends that provided a full tour on how you can build your apps on Azure using best-in-class developer tools. The day continued with coding sessions hosted ... Read More

Feed: Microsoft Azure Blog. Author: Ashish Thapliyal. Today we are sharing an update to the Azure HDInsight integration with Azure Data Lake Storage Gen 2. This integration will enable HDInsight customers to drive analytics from the data stored in Azure Data Lake Storage Gen 2 using popular open source frameworks such as Apache Spark, Hive, MapReduce, Kafka, Storm, and HBase in a secure manner. Azure Data Lake Storage Gen2 Azure Data Lake Storage Gen2 is the only data lake designed specifically for enterprises to run large scale analytics workloads in the cloud. It unifies the core capabilities from the first ... Read More

Feed: Microsoft Azure Blog. Author: Dhruv Goel. Azure HDInsight is an easy, cost-effective, enterprise-grade service for open source analytics. With HDInsight, you get managed clusters for various Apache big data technologies, such as Spark, MapReduce, Kafka, Hive, HBase, Storm and ML Services backed by a 99.9% SLA. In addition, you can take advantage of HDInsight’s rich ISV application ecosystem to tailor the solution for your specific scenario. HDInsight covers a wide variety of big data technologies, and we have received many requests for a detailed guide. Whether you want to just get started with HDInsight, or become a Big Data ... Read More

Feed: What's New. Parquet is a columnar storage file format, similar to ORC (optimized row-columnar) and is available to any project in the Hadoop ecosystem regardless of the choice of data processing framework, data model, or programming language. The columnar format lets the you read, decompress, and process only the columns that are required for the current query. For querying S3 Inventory with AWS services such as Amazon Athena or Amazon Redshift Spectrum, or tools such as Apache Hive, Spark, HBase or Presto, we recommend configuring your S3 Inventory report in either Parquet or ORC for faster query performance and ... Read More