Scoring academic writing from subjectivity to objectivity: A scale for evaluating students' written product

Abstract

Due to the drawbacks of adopting any of the common scoring scales independently and in order to make the scoring process less subjective and more objective, the researcher thought of developing a rubric for evaluating students' written product. Some of the well-known scoring scales are time consuming, others mainly depend on the scorer's impression which is not always accurate. The researcher here made a hybrid of both the holistic and analytic scoring scales and developed it to suit academic writing. The rubric was refereed by a group of specialists who assured its validity. Content validity and internal consistency were also calculated. To test the suitability of this scoring rubric for the purpose it was designed for, a stratified random sample of 30 essays was selected from a corpus of N=120 essays written by Palestinian tertiary level students majoring in English in the academic year 2009/ 2010. Pearson correlation and Alpha scale were used in order to assure the reliability of the scale. The Scoring scale follows a taxonomy in which errors experienced in the subjects' writings are divided into three major categories: errors related to conventions, content development and style. Each category includes a number of subcategories; for instance, conventions include writing mechanics and grammar and vocabulary common errors; each of which is scored out of 25. In addition, content development includes cohesion and coherence, which are scored out of 20 marks and 15 marks respectively and finally style and cogency are marked out of 15 marks only.