徐振中_想要一个简单易用的实时ml平台？你必须知道的一些复杂细节

Considerations for Abstracting Complexities ofa Real-time ML Platform Zhenzhong XuCofounder & CTO @ claypot.aiJuly, 2023 The discovery ofsomething invisible The Invisible Interface Ubiquitous Easy and responsive Just works! The endeavor to make things useful Real-time Decisionsthat powers your business Customer supportDynamic pricing/discountingRisk AssessmentAccount Take OverAdsSentiment analysisObjectdetection… The world is moving towards real-time ●Instacart: The Journey to Real-Time Machine Learning(2022)○Directly reducesmillions of fraud-related costsannually.●LinkedIn’s Real-time Anti-abuse(2022)○LinkedIn moved from an offline pipeline (hours) to real-time pipeline (minutes), and saw30% increase in bad actors caught onlineand21% improvement in fake account detection.●How WhatsApp catches and fights abuse(2022 |slides)○A few 100ms delay can increase the spam by 20-30%.●How Pinterest Leverages Realtime User Actions in Recommendation to Boost Engagement(2022)○According to Pinterest, this“has been one of our most impactful innovations recently,increasing Home feed engagement by 11% while reducing Pinner hide volume by 10%●Airbnb: Real-time Personalization using Embeddings for Search Ranking(2018)○Moving from offline scoring to online scoring grows bookings by+5.1% The hard things towardsreal-time decisions ●Data silo and staleness●Collaborationoverhead●Tech complexity Challenge 1 : FromExperimentation to Production ●Slow prototyping●Local vs. remote execution●Divergent language & runtime Local Experimentation with Traditional Models Local Experimentation with LLMs Need aninvisibleinterface to plug into compute ecosystems Declare features with familiar APIs @transformationdefaverage_transaction_amount_by_merchant(tx: Transactions,wspec: WindowSpec): returntx.groupby(["cc_num","merchant"])["amt"].window(wspec).mean() Data Science Friendly:Python <> SQL Same code can run on different computation engines Compile into a relational expression (RE), whichis SQL equivalent Compile & optimize RE into the computationengine(e.g., Panda, DuckDb, Flink, Spark) best suited forthe job Spin up and manage computation jobs Solution1 : RelationalExpression based Compilation ●Unified yet familiar API●Pluggable to many compute engines●Minimize human error●Prototype in minutes Challenge 2: Streaming andBatch Divided ●Evolving architecture●Difficult to backfill●Train-predict inconsistencies Kappa (Streaming) Architecture Batch and streaming source unified to simplify backfill Need an invisible interface to plug into storage ecosystems Data Fabric for a Streaming Pipeline Training dataset backfill requirespoint-in-time correctness Point-in-time joins to generate training data Given a spine (entity keys + timestamp + label), join features to generate training data train_df = pitc_join_features(spine_df,features=["tx_max_1h","user_unique_ip_30d",],) Solution 2: Abstract streamingand batch data storage ●Unified streaming & batch source●Unified online & offline feature stores●Pluggable to most storage technologies Challenge 3: It should just work! ●Cost, latency, correctnesssurprises!●Lack optimizations knobs Optimization @transformation deftransaction_count(tx: Transactions, wspec: WindowSpec):returntx[tx.status =="failed"].groupby("account_id").window(wspec).count() Various intelligent optimization can be done tomake appropriate tradeoff across storage andcompute systems. Solution 3: Optimization knobs ●Abstract optimization complexity●User controls with high level knobs●Trust, nosurprises! Make invisible interfacepossible! ●Ubiquitous●Easy and responsive●Just works! https://zhenzhongxu.com/zhenzhong@claypot.ai the invisible interface