语用变量和语用口语测试项目特征的作用

Date:5 December2025Authorname andORCID number:Soo Jung Youn(0000-0001-6918-9399) Contents 3.2 Test instruments.......................................................................................................15 3.2.1 Aptis speaking test...............................................................................................153.2.2 Pragmatic speaking test......................................................................................16 3.4.1 Test administration...............................................................................................233.4.2 Rater training and scoring...................................................................................23 3.5 Data analysis...........................................................................................................23 Results................................................................................................................................25 4.1.1 Examinees.............................................................................................................274.1.2 Raters...................................................................................................................274.1.3 Items....................................................................................................................284.1.4 Rating criteria.......................................................................................................29 4.3.3 Section 3.............................................................................................................354.3.4 Section 4.............................................................................................................36Discussion...........................................................................................................................38Conclusion and implications...............................................................................................42 List of tables Table 1:Retired Aptis speaking test tasks....................................................................................16Table 2:Overview of pragmatic speaking test............................................................................17Table 3:Pragmatic variables and definitions..............................................................................20Table 4:Example pragmatic test items and coded pragmatic variables.........................................21Table 5:Summary statistics for MFRM analysis (Aptis test)...........................................................26Table 6:Summary statistics for MFRM analysis (Pragmatic test).....................................................27Table 7:Item characteristics for each section of the pragmatic test.............................................28Table 8:Correlations between pragmatic variables and item characteristics..................................30Table 9:Example responses at different pragmatic performance levels for Item 5..........................31Table 10:Example responses at different pragmatic performance levels for Item 10.........................33 Abstract This study reports the development of pragmatic speaking test items for adult learners ofEnglish, targeting proficiency levels ranging from lower B1 to upper C2. In order to assess andoperationalize the multidimensional nature of L2 pragmatic competence,a range ofpragmatic variables derived from L2 pragmatic theories were adopted in designing thepragmatic speaking test items. The pragmatic speaking test, consisting of four distinct sectionswith a total of 24 items, was developed. The roles of pragmatic variables in explaining itemdifficulty and discrimination were further examined. Fifty examinees completed both theretired Aptis speaking test and the pragmatic speaking test, with their performances scored byfive trained raters. The data was analyzedboth quantitatively and qualitatively. Specifically, Author Soo Jung Youn is an associate professor of English education, Daegu National University ofEducation, Korea. Her research interests include language testing and assessment, secondlanguage pragmatics, interactional competence, task-based language teaching,and mixed- Introduction Second language (L2) pragmatic competence encompasses a range of elements, such as theknowledge of linguistic forms, functions, contexts, norms of interaction, culturalappropriateness, and social relationships among speakers (Celce-Murcia, Dörnyei, & Thurrell,1995; Taguchi & Roever, 2017; Roever, 2022). Diverse research issues on assessing themultifaceted dimensions of L2 pragmatic competence have been investigated, evidenced bythe extensive line of research and discussions. These range from assessingsociocultural andlinguistic knowledge related to speech acts using multiple assessment methods (e.g. Hudson,Detmer, & Brown, 1992, 1995) to broadening the construct of pragmatic competence toencompass spoken interaction (e