Implementation of a Bayesian System for Decision Analysis in a Program of Individually Prescribed Instruction

ACT RESEARCH REPORTNo. 6060September 1973IMPLEMENTATION OF A BAYESIAN SYSTEM FOR DECISION ANALYSIS IN A PROGRAM OF INDIVIDUALLY PRESCRIBED INSTRUCTION5CLoUJ THE AMERICAN COLLEGE TESTING PROGRAMQCP. 0. BOX 168, IOWA CITY, IOWA 52240 ABSTRACTThe decision process required fo r Individually Prescribed Instruction ( IPI), an adaptive instructional program developed at the University of Pittsburgh, is described. In I P I, short tests are used to determine the level o f proficiency o f each student in precisely defined learning objectives. The output of these tests is used to guide instructional planning for individual students.The nature and effect of errors in pro ficien cy decisions are described and a procedure for reducing the p ro ba b ility of such errors is proposed. The plan calls fo r a Bayesian procedure which would incorporate p rior inform ation on the instructional program, for example the distribution of the percentage of items answered co rre ctly by students. Such a procedure would perm it inferences about the true level of fun ctio ning of each student.The final section of the paper proposes two methods for im plem enting these procedures in an ongoing IPI program: one approach calls for the integration of the procedure as a part of a com puter-based instructional management system, whereas the second approach describes how the procedure can be made tractable in a typical, non-automated IPI classroom. Prepared by the Research and Development DivisionThe Am erican College Testing ProgramFor additional copies write:Publications DivisionThe American College Testing ProgramP.O. Box 168, Iowa City, Iowa 52240(Check o r money order must accom pany request.) Price: $1.00 IMPLEMENTATION OF A BAYESIAN SYSTEM FOR DECISION ANALYSISIN A PROGRAM OF INDIVIDUALLY PRESCRIBED INSTRUCTIONThe research reported herein was performed pursuantto Grant No. OEG-0-72-0711 with the Office of Education, U.S. Department of Health, Education, and Welfare, Melvin R. Novick, Principal Investigator. Contractors undertaking such projects under Government sponsorship are encouraged to express freely their professional judgment in the conduct of the project. Points of view or opinions stated do not, therefore, necessarily represent official Office of Education position or policy. IMPLEMENTATION OF A BAYESIAN SYSTEM FOR DECISION ANALYSISIN A PROGRAM OF INDIVIDUALLY PRESCRIBED INSTRUCTIONRichard L. Ferguson Melvin R. NovickINTRODUCTIONThe feasibility of instructional programs designed to adapt to the individual needs of learners has been adequately demonstrated by educational systems like Individually Prescribed Instruction {Glaser, 1968) and A Program for Learning in Accordance with Needs (Flanagan, 1967). Although these programs accomplish individualization in somewhat different ways, each includes components which can be described by the following sequence of operations:1. Specification of the learning objectives in terms of observable student behavior.2. Assessment of the student’s entering compe­tencies.3. Assignment or election of educational materials and/or experiences fitted to the student’s individual needs.4. Continuous assessment and monitoring of the student’s performance and progress.Since programs like IPI and PLAN callforadaptation of the learning environment to meet individual requirements, they necessarily rely heavily on the systematic assessment of student progress. Glaser (1968) has observed that, in I PI, test data serve as the primary source of information enabling teachers to make differential decisions regarding studentinstruction. Thus, steps (2) and (4) play a prominent role in the successful implementation of IPI. A review of current decision-making procedures for four selected individualized instructional programs has been given by Hambleton (1973).The fundamental purpose for testing in in­dividualized instructional programs like IPI and PLAN is to ascertain whether or not the student has attained some prescribed level of proficiency in a specified learning objective. Hambleton and Novick (1973) have observed that, “Questions of precise achievement levels and comparisons among indi­viduals on these levels seem to be largely irrel­evant.” Because test data are used initially to place a student at the appropriate point within an instructional program or sequence, and thus to identify appropriate learning materials or ex­periences given his needs, the test models which have emerged to serve this function are very dif­ferent from those used for standard instructional models. Because these tests relate a student's per­formance on