2023年自然语言处理算法鲁棒性研究思考报告

张奇 复旦大学 0各类自然语言处理算法快速发展，在很多任务上甚至超越人类 0各类自然语言处理算法快速发展，在很多任务上甚至超越人类 Post-credit Scenes自然语言处理真的被解决了吗？ 0算法在实际应用中的效果却不尽如人意 搜索引擎线上，精度95%条件下召回率小于20% 能够回答的部分绝大多数都是原文匹配类型 0算法在实际应用中的效果却不尽如人意 0不经过鲁棒性评估会面临巨大风险 Post-credit Scenes自然语言处理仍然面临很多问题 0模型对测试数据的微小变化非常敏感 !"#$%&'()(*+,-./0123456789:;<=>!"? !"@$ABCDEF7GHIJKHL&'? !"M$NOPQRSTUVWXYZ[\]^'_` !"#$%&'()(*+,-./0123456789:;<=>!"? !"@$ABCDEF7GHIJKHL&'? !"M$NOPQRSTUVWXYZ[\]^'_` AAAI2020BestPaperWINOGRANDE: An AdversarialWinograd Schema Challenge at Scale Winograd Schema Challenge (WSC)Commonsensereasoning The trophy doesn’t fit into the brown suitcase becauseit’s toolarge.trophy/ suitcaseThe trophy doesn’t fit into the brown suitcase becauseit’s toosmall.trophy /suitcase RoBERTalargeachieves91.3%accuracyon a variant of WSC dataset Haveneurallanguagemodelssuccessfullyacquiredcommonsenseorareweoverestimatingthetruecapabilitiesofmachinecommonsense? Dataset-specific Biases Sakaguchietal.,WINOGRANDE: An AdversarialWinograd Schema Challenge at Scale, AAAI2020. Insteadofmanuallyidentifiedlexicalfeatures,theyadoptadenserepresentationofinstancesusingtheirprecomputedneuralnetworkembeddings. MainSteps: 1.RoBERTafine-tuned on a small subset of thedataset.2.An ensemble of linear classifiers (logistic regressions)3.Trained on random subsets of the data4.Determinewhether the representation is stronglyindicative of the correct answer option5.Discard thecorresponding instances 数据集采样对模型训练和测试重要影响–ContrastSets (a) A two-dimensional dataset that requires a complex decisionboundary to achieve high accuracy. (b) If the same data distribution is instead sampled withsystematic gaps (e.g., due to annotator bias), a simple decisionboundary can perform well oni.i.d. test data (shown outlinedin pink). (c) Since filling in all gaps in the distribution is infeasible, acontrast set instead fills in a local ball around a test instance toevaluate the model’s decision boundary 数据集采样对模型训练和测试重要影响–ContrastSets !"#$%&'()*+,-./01234 Thedatasetauthorsmanuallyperturbthetestinstancesinsmallbutmeaningfulwaysthat(typically)changethegoldlabel,creatingcontrastsets. 数据集采样对模型训练和测试重要影响–ContrastSets 1 Aspect-I: Intrinsic natureword length (wLen); sentence length (sLen)OOV density (oDen);Aspect-II: Familiarityword frequency (wFre);character frequency (cFre)Aspect-III: Label consistencylabel consistency of word (wCon);label consistency of character (cCon) Self-diagnosis：aims to locate the bucket on which theinputmodel has obtained the worst performance withrespecttoagivenattribute. Aided-diagnosis(A,B):aimstocomparetheperformanceofdifferentmodelsondifferentbucket. Fu et al. ,RethinkCWS: Is Chinese Word Segmentation a Solved Task?，EMNLP2020 细粒度评测-Rethinking Generalization of Neural Models EntityCoverage Ratio(ECR)The measureentitycoverageratioisusedtodescribethedegreetowhichentitiesinthetestsethavebeenseeninthetrainingsetwiththesamecategory. 细粒度评测-EXPLAINABOARD: An Explainable Leaderboard for NLP Standard splits: Training:sections 00–18Development:sections 19-21Testing:sections22-24 Blue balls–TrainingOrangeballs--Test 问题1：为什么基于基准测试集合和常用评价指标的模式不能反映上述问题？ !"#$%&'()*+,-./01 !"#$%&'()*+,"-'./01234567 2"3456789:;<=>?@ABCDE !"89./01:;<=>?@ !"#$%&'()(*+,-./0123456789:;<=>!"? !"@$ABCDEF7GHIJKHL&'? !"M$NOPQRSTUVWXYZ[\]^'_` Visualizing and Understanding Recurrent Networks Several examples of cells with interpretable activations discovered inLSTMtrainedwithLinux KernelandWar and Peace. Karpathyet al. ,Visualizing and Understanding Recurrent Networks,2016 2ContextualWord Embeddings Theypresentedadetailedempiricalstudyofhowthechoiceofneuralarchitecture(e.g.LSTM,CNN,or self attention)influences both end task accuracy and qualitative properties of therepresentationsthatarelearned. 2IntegratedGradients归因方法 IntegratedGradients(IG)(Sundararajan et al.,2017)to isolate question words that a deeplearningsystemusestoproduceananswer. Red--high attributionBlue--negative attributionGray--near-zero attribution For image networks, the baselineinputx'couldbe the black image, while for text models it couldbe the zero embedding vector. Mudrakartaetal.Did the Model Understand the Question?ACL2018Sundararajanetal.,Axiomatic attribution for deep networks.2017 2IntegratedGradients归因方法 基于Bert的用户检索词---文章语义匹配模型 用户查询:硫酸沙丁胺醇吸入气雾剂用法 Attentionheads exhibiting patterns The best performing attentions heads ofBERT on WSJ dependency parsing BERT’sattention heads exhibit patterns such as attending to delimiter tokens,specificpositionaloffsets,orbroadlyattendingoverthewholesentence,withheadsinthesamelayeroftenexhibitingsimilarbehaviors Certainattentionheadscorrespondwelltolinguisticnotionsofsyntaxandcoreference. Attention-basedprobingclassifierdemonstratedthatsubstantialsyntacticinformationcouldbecapturedinBERT’sattention. Clark et al. ,What Does BERT Look At?An Analysis of BERT’s Attention,ACL2019 2Attention是否可以解释？ Attentionlayersexplicitlyweightinputcomponents’representations,itisalsooftenassumedtha