现代机器学习手册：简化机器学习生产路径的最佳实践

to production ML TABLE OF CONTENTS Introduction3Primary Use Cases for Machine Learning4Traditional Challenges of Machine Learning6Why Migrating to Snowflake ML Accelerates Production7Common Architectural Patterns for ML in Snowflake13 THE MODERN ML PLAYBOOK INTRODUCTION While flashy new LLM and generative AI applications may grab headlines, machinelearning (ML) remains one of the most dominant and critical technologies for Machine learning has proven so effective at analyzing data that ML models arenow used to generate predictions in nearly every sector of society. However,despite the best efforts of many ML teams, many models still never make itto production, due to fragmented tool sets, inefficient data pipelines and the adopting a single unified platform for data and ML models. PRIMARY USE CASES FORMACHINE LEARNING Today, ML models are the cornerstone of an incredibly broad range of use cases. Here are some of themost common applications: identifying anomalous patterns in transaction data, banksuse ML systems to block fraudulent credit card chargeswithin milliseconds. Models analyze hundreds of features to e-commerce, recommendation systems are the backboneof the modern digital experience. Using collaborativefiltering, matrix factorization and neural networks trained risk, determining creditworthiness based on financial history,income, employment and other factors. These models needto make consistent decisions across millions of applications use time-series ML models to predict future demand,optimizing inventory levels and reducing waste. Thesemodels run continuously, making predictions for thousands groups of customers based on their behavior, demographicsand purchase history. This enables targeted marketing,personalized pricing and churn prediction. to predict click-through rates and conversions, determiningwhich ads to show and how much to bid in real-time auctions can detect cancers, identify diseases and assist radiologists.These specialized models often outperform general-purposeAI because they’re optimized for specific diagnostic tasks happen. By analyzing time-series data generated by sensorsembedded inside equipment and facilities, the models are CUSTOMER SPOTLIGHT IGS’s ML model accurately detects which panels areunderperforming, avoiding service outages and reducing thenumber of false positives requiring a costly and unnecessary How IGS Energy uses machine learning tocreate a greener future forms of energy, including renewable electricity, carbon-neutralgas and solar. A key element of the Ohio-based power provider’ssuccess is accurately forecasting energy consumption, so itcan lock in lower wholesale electricity prices and pass those ML model in Snowflake saves IGS resources while helpingits customers derive more value from their investments ingreen technology. ML consumption model and apply it to allof its household and business accountswhile still maintaining high levels ofaccuracy. Thisslashed the time andcost needed to generate predictions by details to generate demand predictions required IGS to build aunique ML model for each customer — a time- and resource-hungry process. The company needed a more flexible andscalable ML solution, and IGS found it in Snowflake. installed at customer locations. Instead of asking employeesto comb through spreadsheets searching for data anomalies, TRADITIONAL CHALLENGESOF MACHINE LEARNING Despite decades of advancement, successfully deploying ML models remains highly challenging due to adisparate array of tools and the difficulty of managing the underlying infrastructure. together multiple tools across environments that can be difficult to govern and costly to maintain. They oftenend up shuffling data between where it lives and where it’s processed, leading to unnecessary complexity, machine learning teams include: The best way to keep your sensitive, private data safe and secureis to keep it on a single platform without data egress. Training large models requires expensive GPU resources, andthese costs compound at scale when compute is not properly configured, which often happens when infrastructure needs tobe manually set up and maintained.. When GPUs sit idle duringpre-processing phases, organizations may end up paying for4. Distrust in prediction results over timeAfter models make it to production, it is common for Data scientists can spend up to 80% of their time wranglingdata, debugging pipelines and waiting for training jobs rather than developing actual models. These headaches areoften caused by using multiple ML tools and platforms thatrequire manual integration and configuration, which canlead to duplicated tooling and inefficient compute usage. model behaviors to change over time due to incomplete understanding of the world in training data, input data driftand data quality issues. Changes in data or the environmentcan have a tremendous impact on the model quality. Difficulty source data, ML development and M