Mastering Machine Learning with Spark 2.x


Book Description

Unlock the complexities of machine learning algorithms in Spark to generate useful data insights through this data analysis tutorial About This Book Process and analyze big data in a distributed and scalable way Write sophisticated Spark pipelines that incorporate elaborate extraction Build and use regression models to predict flight delays Who This Book Is For Are you a developer with a background in machine learning and statistics who is feeling limited by the current slow and “small data” machine learning tools? Then this is the book for you! In this book, you will create scalable machine learning applications to power a modern data-driven business using Spark. We assume that you already know the machine learning concepts and algorithms and have Spark up and running (whether on a cluster or locally) and have a basic knowledge of the various libraries contained in Spark. What You Will Learn Use Spark streams to cluster tweets online Run the PageRank algorithm to compute user influence Perform complex manipulation of DataFrames using Spark Define Spark pipelines to compose individual data transformations Utilize generated models for off-line/on-line prediction Transfer the learning from an ensemble to a simpler Neural Network Understand basic graph properties and important graph operations Use GraphFrames, an extension of DataFrames to graphs, to study graphs using an elegant query language Use K-means algorithm to cluster movie reviews dataset In Detail The purpose of machine learning is to build systems that learn from data. Being able to understand trends and patterns in complex data is critical to success; it is one of the key strategies to unlock growth in the challenging contemporary marketplace today. With the meteoric rise of machine learning, developers are now keen on finding out how can they make their Spark applications smarter. This book gives you access to transform data into actionable knowledge. The book commences by defining machine learning primitives by the MLlib and H2O libraries. You will learn how to use Binary classification to detect the Higgs Boson particle in the huge amount of data produced by CERN particle collider and classify daily health activities using ensemble Methods for Multi-Class Classification. Next, you will solve a typical regression problem involving flight delay predictions and write sophisticated Spark pipelines. You will analyze Twitter data with help of the doc2vec algorithm and K-means clustering. Finally, you will build different pattern mining models using MLlib, perform complex manipulation of DataFrames using Spark and Spark SQL, and deploy your app in a Spark streaming environment. Style and approach This book takes a practical approach to help you get to grips with using Spark for analytics and to implement machine learning algorithms. We'll teach you about advanced applications of machine learning through illustrative examples. These examples will equip you to harness the potential of machine learning, through Spark, in a variety of enterprise-grade systems.




Implementing MLOps in the Enterprise


Book Description

With demand for scaling, real-time access, and other capabilities, businesses need to consider building operational machine learning pipelines. This practical guide helps your company bring data science to life for different real-world MLOps scenarios. Senior data scientists, MLOps engineers, and machine learning engineers will learn how to tackle challenges that prevent many businesses from moving ML models to production. Authors Yaron Haviv and Noah Gift take a production-first approach. Rather than beginning with the ML model, you'll learn how to design a continuous operational pipeline, while making sure that various components and practices can map into it. By automating as many components as possible, and making the process fast and repeatable, your pipeline can scale to match your organization's needs. You'll learn how to provide rapid business value while answering dynamic MLOps requirements. This book will help you: Learn the MLOps process, including its technological and business value Build and structure effective MLOps pipelines Efficiently scale MLOps across your organization Explore common MLOps use cases Build MLOps pipelines for hybrid deployments, real-time predictions, and composite AI Learn how to prepare for and adapt to the future of MLOps Effectively use pre-trained models like HuggingFace and OpenAI to complement your MLOps strategy




Engineering MLOps


Book Description

Get up and running with machine learning life cycle management and implement MLOps in your organization Key FeaturesBecome well-versed with MLOps techniques to monitor the quality of machine learning models in productionExplore a monitoring framework for ML models in production and learn about end-to-end traceability for deployed modelsPerform CI/CD to automate new implementations in ML pipelinesBook Description Engineering MLps presents comprehensive insights into MLOps coupled with real-world examples in Azure to help you to write programs, train robust and scalable ML models, and build ML pipelines to train and deploy models securely in production. The book begins by familiarizing you with the MLOps workflow so you can start writing programs to train ML models. Then you'll then move on to explore options for serializing and packaging ML models post-training to deploy them to facilitate machine learning inference, model interoperability, and end-to-end model traceability. You'll learn how to build ML pipelines, continuous integration and continuous delivery (CI/CD) pipelines, and monitor pipelines to systematically build, deploy, monitor, and govern ML solutions for businesses and industries. Finally, you'll apply the knowledge you've gained to build real-world projects. By the end of this ML book, you'll have a 360-degree view of MLOps and be ready to implement MLOps in your organization. What you will learnFormulate data governance strategies and pipelines for ML training and deploymentGet to grips with implementing ML pipelines, CI/CD pipelines, and ML monitoring pipelinesDesign a robust and scalable microservice and API for test and production environmentsCurate your custom CD processes for related use cases and organizationsMonitor ML models, including monitoring data drift, model drift, and application performanceBuild and maintain automated ML systemsWho this book is for This MLOps book is for data scientists, software engineers, DevOps engineers, machine learning engineers, and business and technology leaders who want to build, deploy, and maintain ML systems in production using MLOps principles and techniques. Basic knowledge of machine learning is necessary to get started with this book.




Learning Ray


Book Description

Get started with Ray, the open source distributed computing framework that simplifies the process of scaling compute-intensive Python workloads. With this practical book, Python programmers, data engineers, and data scientists will learn how to leverage Ray locally and spin up compute clusters. You'll be able to use Ray to structure and run machine learning programs at scale. Authors Max Pumperla, Edward Oakes, and Richard Liaw show you how to build machine learning applications with Ray. You'll understand how Ray fits into the current landscape of machine learning tools and discover how Ray continues to integrate ever more tightly with these tools. Distributed computation is hard, but by using Ray you'll find it easy to get started. Learn how to build your first distributed applications with Ray Core Conduct hyperparameter optimization with Ray Tune Use the Ray RLlib library for reinforcement learning Manage distributed training with the Ray Train library Use Ray to perform data processing with Ray Datasets Learn how work with Ray Clusters and serve models with Ray Serve Build end-to-end machine learning applications with Ray AIR




Introducing MLOps


Book Description

More than half of the analytics and machine learning (ML) models created by organizations today never make it into production. Some of the challenges and barriers to operationalization are technical, but others are organizational. Either way, the bottom line is that models not in production can't provide business impact. This book introduces the key concepts of MLOps to help data scientists and application engineers not only operationalize ML models to drive real business change but also maintain and improve those models over time. Through lessons based on numerous MLOps applications around the world, nine experts in machine learning provide insights into the five steps of the model life cycle--Build, Preproduction, Deployment, Monitoring, and Governance--uncovering how robust MLOps processes can be infused throughout. This book helps you: Fulfill data science value by reducing friction throughout ML pipelines and workflows Refine ML models through retraining, periodic tuning, and complete remodeling to ensure long-term accuracy Design the MLOps life cycle to minimize organizational risks with models that are unbiased, fair, and explainable Operationalize ML models for pipeline deployment and for external business systems that are more complex and less standardized




Machine Learning Design Patterns


Book Description

The design patterns in this book capture best practices and solutions to recurring problems in machine learning. The authors, three Google engineers, catalog proven methods to help data scientists tackle common problems throughout the ML process. These design patterns codify the experience of hundreds of experts into straightforward, approachable advice. In this book, you will find detailed explanations of 30 patterns for data and problem representation, operationalization, repeatability, reproducibility, flexibility, explainability, and fairness. Each pattern includes a description of the problem, a variety of potential solutions, and recommendations for choosing the best technique for your situation. You'll learn how to: Identify and mitigate common challenges when training, evaluating, and deploying ML models Represent data for different ML model types, including embeddings, feature crosses, and more Choose the right model type for specific problems Build a robust training loop that uses checkpoints, distribution strategy, and hyperparameter tuning Deploy scalable ML systems that you can retrain and update to reflect new data Interpret model predictions for stakeholders and ensure models are treating users fairly




Deep Learning for Coders with fastai and PyTorch


Book Description

Deep learning is often viewed as the exclusive domain of math PhDs and big tech companies. But as this hands-on guide demonstrates, programmers comfortable with Python can achieve impressive results in deep learning with little math background, small amounts of data, and minimal code. How? With fastai, the first library to provide a consistent interface to the most frequently used deep learning applications. Authors Jeremy Howard and Sylvain Gugger, the creators of fastai, show you how to train a model on a wide range of tasks using fastai and PyTorch. You’ll also dive progressively further into deep learning theory to gain a complete understanding of the algorithms behind the scenes. Train models in computer vision, natural language processing, tabular data, and collaborative filtering Learn the latest deep learning techniques that matter most in practice Improve accuracy, speed, and reliability by understanding how deep learning models work Discover how to turn your models into web applications Implement deep learning algorithms from scratch Consider the ethical implications of your work Gain insight from the foreword by PyTorch cofounder, Soumith Chintala




Machine Learning Engineering with Python


Book Description

Transform your machine learning projects into successful deployments with this practical guide on how to build and scale solutions that solve real-world problems Includes a new chapter on generative AI and large language models (LLMs) and building a pipeline that leverages LLMs using LangChain Key Features This second edition delves deeper into key machine learning topics, CI/CD, and system design Explore core MLOps practices, such as model management and performance monitoring Build end-to-end examples of deployable ML microservices and pipelines using AWS and open-source tools Book DescriptionThe Second Edition of Machine Learning Engineering with Python is the practical guide that MLOps and ML engineers need to build solutions to real-world problems. It will provide you with the skills you need to stay ahead in this rapidly evolving field. The book takes an examples-based approach to help you develop your skills and covers the technical concepts, implementation patterns, and development methodologies you need. You'll explore the key steps of the ML development lifecycle and create your own standardized "model factory" for training and retraining of models. You'll learn to employ concepts like CI/CD and how to detect different types of drift. Get hands-on with the latest in deployment architectures and discover methods for scaling up your solutions. This edition goes deeper in all aspects of ML engineering and MLOps, with emphasis on the latest open-source and cloud-based technologies. This includes a completely revamped approach to advanced pipelining and orchestration techniques. With a new chapter on deep learning, generative AI, and LLMOps, you will learn to use tools like LangChain, PyTorch, and Hugging Face to leverage LLMs for supercharged analysis. You will explore AI assistants like GitHub Copilot to become more productive, then dive deep into the engineering considerations of working with deep learning.What you will learn Plan and manage end-to-end ML development projects Explore deep learning, LLMs, and LLMOps to leverage generative AI Use Python to package your ML tools and scale up your solutions Get to grips with Apache Spark, Kubernetes, and Ray Build and run ML pipelines with Apache Airflow, ZenML, and Kubeflow Detect drift and build retraining mechanisms into your solutions Improve error handling with control flows and vulnerability scanning Host and build ML microservices and batch processes running on AWS Who this book is for This book is designed for MLOps and ML engineers, data scientists, and software developers who want to build robust solutions that use machine learning to solve real-world problems. If you’re not a developer but want to manage or understand the product lifecycle of these systems, you’ll also find this book useful. It assumes a basic knowledge of machine learning concepts and intermediate programming experience in Python. With its focus on practical skills and real-world examples, this book is an essential resource for anyone looking to advance their machine learning engineering career.




Ultimate MLOps for Machine Learning Models


Book Description

TAGLINE The only MLOps guide you'll ever need KEY FEATURES ● Acquire a comprehensive understanding of the entire MLOps lifecycle, from model development to monitoring and governance. ● Gain expertise in building efficient MLOps pipelines with the help of practical guidance with real-world examples and case studies. ● Develop advanced skills to implement scalable solutions by understanding the latest trends/tools and best practices. DESCRIPTION This book is an essential resource for professionals aiming to streamline and optimize their machine learning operations. This comprehensive guide provides a thorough understanding of the MLOps life cycle, from model development and training to deployment and monitoring. By delving into the intricacies of each phase, the book equips readers with the knowledge and tools needed to create robust, scalable, and efficient machine learning workflows. Key chapters include a deep dive into essential MLOps tools and technologies, effective data pipeline management, and advanced model optimization techniques. The book also addresses critical aspects such as scalability challenges, data and model governance, and security in machine learning operations. Each topic is presented with practical insights and real-world case studies, enabling readers to apply best practices in their job roles. Whether you are a data scientist, ML engineer, or IT professional, this book empowers you to take your machine learning projects from concept to production with confidence. It equips you with the practical skills to ensure your models are reliable, secure, and compliant with regulations. By the end, you will be well-positioned to navigate the ever-evolving landscape of MLOps and unlock the true potential of your machine learning initiatives. WHAT WILL YOU LEARN ● Implement and manage end-to-end machine learning lifecycles. ● Utilize essential tools and technologies for MLOps effectively. ● Design and optimize data pipelines for efficient model training. ● Develop and train machine learning models with best practices. ● Deploy, monitor, and maintain models in production environments. ● Address scalability challenges and solutions in MLOps. ● Implement robust security practices to protect your ML systems. ● Ensure data governance, model compliance, and security in ML operations. ● Understand emerging trends in MLOps and stay ahead of the curve. WHO IS THIS BOOK FOR? This book is for data scientists, machine learning engineers, and data engineers aiming to master MLOps for effective model management in production. It’s also ideal for researchers and stakeholders seeking insights into how MLOps drives business strategy and scalability, as well as anyone with a basic grasp of Python and machine learning looking to enter the field of data science in production. TABLE OF CONTENTS 1. Introduction to MLOps 2. Understanding Machine Learning Lifecycle 3. Essential Tools and Technologies in MLOps 4. Data Pipelines and Management in MLOps 5. Model Development and Training 6. Model Optimization Techniques for Performance 7. Efficient Model Deployment and Monitoring Strategies 8. Scalability Challenges and Solutions in MLOps 9. Data, Model Governance, and Compliance in Production Environments 10. Security in Machine Learning Operations 11. Case Studies and Future Trends in MLOps Index




Practical MLOps


Book Description

Getting your models into production is the fundamental challenge of machine learning. MLOps offers a set of proven principles aimed at solving this problem in a reliable and automated way. This insightful guide takes you through what MLOps is (and how it differs from DevOps) and shows you how to put it into practice to operationalize your machine learning models. Current and aspiring machine learning engineers--or anyone familiar with data science and Python--will build a foundation in MLOps tools and methods (along with AutoML and monitoring and logging), then learn how to implement them in AWS, Microsoft Azure, and Google Cloud. The faster you deliver a machine learning system that works, the faster you can focus on the business problems you're trying to crack. This book gives you a head start. You'll discover how to: Apply DevOps best practices to machine learning Build production machine learning systems and maintain them Monitor, instrument, load-test, and operationalize machine learning systems Choose the correct MLOps tools for a given machine learning task Run machine learning models on a variety of platforms and devices, including mobile phones and specialized hardware