Modern Data Engineering With Apache Spark A Handson Guide For Building Missioncritical Streaming Applications Scott Haines by Scott Haines 9781484274514, 1484274512 instant download after payment.

This book guides you through the role of the modern Data Engineer in an easy to follow approach that involves building a real project using the power of Apache Spark. The book shows how to get up and running locally with Spark while providing details and additional tips and tricks for moving from your local machine to the cloud over the course of your learning journey. Along the way, you will learn key skills that will help you in your path from novice to seasoned Data Engineer. During your journey through this book you’ll unlock key Spark fundamentals while building a reference catalog of reusable code and design patterns. This book teaches the essentials of effective data modeling and how to work with many different file types and formats from across many popular data sources including Kafka, HDFS, and Redis. Reading this book empowers you to take advantage of SparkSQL as a device to optimize your data pipelines while teaching you to craft modular and testable Spark applications. Learn to create and deploy mission critical streaming spark applications in a low stress environment that paves the way for your own path to production.​What You Will Learn Conceptualize and Architect distributed data systems Model Data for Continuous 24x7x365 delivery of events and metrics Speed up experimentation through the use of Apache Zeppelin and Docker Write ETL systems that run in both batch and streaming modes Harness effective strategies for optimizing data in your data lakes Simplify data transformation with Spark Pipelines Bridge data engineering with machine learning Who This Book Is For Data Engineering with Apache Spark is for the professional Software Engineer looking to take their current skills and apply them to new and exciting challenges within the vast Big Data ecosystem. It is for the practicing Data Engineers who are looking for a guiding light while traversing the many challenges of moving from batch to streaming modes. Data Engineering with Apache Spark is the right choice for Data Architects looking to provide clear and concise direction for how best to harness and use Apache Spark within their organization.