计算机化语言测试效度研究 基于证据的作文自动评分效度验证
作者：高怀勇著
出版时间：2018年版
内容简介
本书分别从应试者的应试心理活动与测试表现，测试评分的准确度与其他效标之间的关联性，测试的区分度和聚合度以及测试的社会影响等视角探索了以风靡全国的句酷在线作文自动评分系统(JAESS)为代表的计算机化作文自动评分的效度，并提出相关对策与建议。
目录
Chapter One Introduction
1.1 Study Background
1.2 Purpose of This Study
1.3 Research Questions
1.4 Definitions of Key Terms
1.5 Contents of the Book
1.6 Summary
Chapter Two Literature Review
2.1 Introduction
2.2 Automated Essay Scoring Abroad
2.2.1 Project Essay Grader (PEG)
2.2.2 Intelligent Essay Assessor (IEA)
2.2.3 E-rater
2.2.4 IntelliMetric
2.2.5 Bayesian Essay Test Scoring sYstem (BETSY)
2.2.6 Intelligent Essay Marking System (IEMS) and Automark
2.2.7 Conceptual Rater (C-Rater)
2.2.8 Schema Extract Analyse and Report (SEAR)
2.2.9 Paperless School Free-text Marking Engine (PS-ME)
2.2.10 Summary of Automated Essay Scoring Abroad
2.3 Automated Essay Scoring in China
2.3.1 New Horizon College English Online Learning and Automated Scoring System
2.3.2 Bingo Intelligent English Automated Essay Scoring System (BIEAESS)
2.3.3 Juku Automated Essay Scoring System (JAESS)
2.4 General Review of Validation Studies of AES
2.4.1 Relationship between Automated Scores and Human Scores
2.4.2 Relationship between Automated Scores and Non-test Indicators
2.4.3 Validation Studies on Scoring Process and Mental Model
2.4.4 Evidence-based Validation Studies
2.4.5 Argument-based Validation Studies
2.4.6 AES Studies in China and Deficiencies of Previous Studies
2.5 Summary
Chapter Three Theoretical Framework
3.1 Introduction
3.2 Theory of Validity and Validation
3.2.1 Theory of Validity
3.2.2 Validation
3.3 Validity Evidence of Writing Assessment
3.3.1 Theoretical Construct of Writing Ability
3.3.2 Convergent and Divergent Validity Evidence of Writing Assessment
3.3.3 Criterion-related Evidence of Writing Assessment
3.3.4 Consequential Evidence of Writing Assessment
3.4 Validation Procedures in Action
3.5 Summary
Chapter Four Methodology
4.1 Introduction
4.2 Participants
4.2.1 Student Participants
4.2.2 Instructors
4.2.3 Raters
4.3 Research Design
4.4 Instruments and Materials
4.4.1 The Writing Task
4.4.2 Teachers' Evaluation of Students' Overall Writing Ability
4.4.3 Questionnaire about Writing Process
4.4.4 Questionnaire about Non-writing Ability
4.4.5 Questionnaire about Theory-based Evidence
4.4.6 Questionnaire about Consequential Evidence of JAESS for Students
4.4.7 Questionnaire about Consequential Evidence of JAESS for Teachers
4.4.8 Rating Criteria
4.4.9 The Interview
4.5 Pilot Studies of Various Questionnaires
4.5.1 Pilot Study of QWP
4.5.2 Pilot Study of QNA
4.5.3 Pilot Study of QTE
4.5.4 Pilot Study of QCES
4.5.5 Pilot Study of QCET
4.6 Data Collection Procedure
4.6.1 Collection of Student Data
4.6.2 Collection of Teacher Data
4.7 Data Preparation
4.8 Data Analysis
4.9 Summary
Chapter Five Results and Discussion
5.1 Introduction
5.2 Results and Discussion of Theoretical Construct
5.2.1 Discussion of the Results Concerned with Theory-based Evidence
5.2.2 Discussion of the Results Concerned with Construct Evidence
5.2.3 Summary about Theoretical Construct Evidence
5.3 Discussion of the Results Concerned with Relationships between AES Scores and Non-test Indicators
5.3.1 Discussion of the Results Concerned with Convergent Validity Evidence
5.3.2 Discussion of the Results Concerned with Divergent Validity Evidence
5.3.3 Summary about the Relationships between AES Scores and Non-test Indicators
5.4 Discussion of the Results Concerned with Criterion-related Validity Evidence
5.5 Discussion of the Results Concerned with Consequential Evidence
5.5.1 Discussion of the Results from Questionnaire QCES
5.5.2 Discussion of the Results from Questionnaire QCET
5.5.3 Discussion of the Results from Interview about Consequential Evidence
5.5.4 Summary about Consequential Evidence
5.6 Summary
Chapter Six Conclusion and Implications
6.1 Introduction
6.2 Research Questions Revisited
6.3 Conclusion of the Study
6.3.1 Theoretical Construct
6.3.2 Relationships between AES Scores and Non-test Indicators
6.3.3 Relationships between AES Scores and Human Scores
6.3.4 Consequential Evidence
6.4 Limitations of the Study
6.5 Implications of the Study
6.5.1 Theoretical Implication
6.5.2 Implications for Test Developers in General
6.5.3 Implications for Test Users
6.6 Future Research Directions
6.7 Summary
List of Tables
List of Figures
List of Abbreviations
Bibliography
