A Practitioner's Guide to Building Trustworthy Data Pipelines
Do your product dashboards look funky? Are your quarterly reports stale? Is the dataset you're using broken or just plain wrong? These problems affect almost every team, yet they're usually addressed on an ad hoc basis and in a reactive manner. If you answered yes to any of the questions above, this book is for you.
Do your product dashboards look funky? Are your quarterly reports stale? Is the dataset you're using broken or just plain wrong? These problems affect almost every team, yet they're usually addressed on an ad hoc basis and in a reactive manner. If you answered yes to any of the questions above, this book is for you.Many data engineering teams today face the "good pipelines, bad data" problem. It doesn't matter how advanced your data infrastructure is if the data you're piping is bad. In this book, Barr Moses, Lior Gavish, and Molly Vorwerck from the data reliability company Monte Carlo explain how to tackle data quality and trust at scale by leveraging best practices and technologies used by some of the world's most innovative companies.Build more trustworthy and reliable data pipelinesWrite scripts to make data checks and identify broken pipelines with data observabilityProgram your own data quality monitors from scratchDevelop and lead data quality initiatives at your companyGenerate a dashboard to highlight your company's key data assetsAutomate data lineage graphs across your data ecosystemBuild anomaly detectors for your critical data assets