Which assessment practices were employed by teaching staff during the COVID-19 pandemic?
Assessment purposes
The participants relayed that assessment purposes during COVID-19 were focused on summation of students’ levels of learning and assigning grades. Some of the participants reported making less use of formative assessment, as the focus was on finding alternative ways to assess the students and assign grades. The data acquired as a result of surveying teaching staff in Saudi HE—which is beyond the scope of this paper—showed that formative assessment practices were almost demolished during the pandemic, given that the participants had to focus on finalising graded assessment tasks to release total coursework grades, as required by the MoE (Almossa and Alzahrani (2022).
Six out of the seven participants mentioned changing their assessment practices during the pandemic, to align with the MoE guidelines and the shift to online mode. Two out of the seven participants said that they used formative assessment before the pandemic, while the other five mentioned using mid-term and final examinations, assignments, projects, quizzes, and presentations. During the pandemic, the respondents followed the rules and guidelines specified by the MoE. The MoE had raised the percentage of coursework to 80% of the total assessment and gave the teaching staff a month to design, implement, accord scores, and share the results with students. This left three weeks for withdrawal before the final assessment, which was worth 20% of the student’s final grade. The assessment methods participants used during the COVID-19 pandemic included assignments, presentations, portfolio, quizzes, open-book examination, projects, oral discussions, and mid-term and final examinations.
Assessment processes
The participants reported focusing on assessment design, scoring, and communication. They wanted to create alternative assessment tasks that were reliable, but found this challenging due to the limited time available for preparing and conducting the assessment tasks, number of students, and their lack of preparedness for new assessment methods.
The participants mentioned that, in the pre-COVID-19 era, they had used various assessment methods as per their department’s regulations and compliance with the MoE (60% for the final examination, and at least 40% for the mid-term examination and coursework). These included projects, research papers, assignments, and open-book examinations. Further, they stated that their assessment practices had undergone variation due to the pandemic’s impact on their teaching, learning, and assessment experiences. All the participants mentioned that online assessment facilitated more freedom in using various assessment types. It also facilitated greater flexibility in introducing new ideas through different types of questions in examinations (including both multiple-choice and true–false questions), mark distribution (which was fixed earlier), and deciding on assessment tasks. The second participant, NOOR, who was a teaching assistant in a theology college, spoke about how the pandemic broke the department’s restrictive traditional assessment rules: ‘We now have the freedom to use various assessment strategies, that were not even considered as options before the pandemic. We have the freedom to give research projects, group tasks, open-book exams, and more.’ Yet, the freedom to redesign assessment came with challenges and restrictions, as the participants reported.
The freedom in assessment design, implementation, and scoring was an advantage that participants appreciated, regardless of the hardships. Another advantage of online assessment was how quickly and accurately Blackboard corrected examinations and produced instant statistical post-examination reports. This was not as easy before the pandemic, when exams were paper-based. Clearly, the use of technology enhanced the process of administration, correction, and reporting, by enabling the instant release of grades and feedback. NASSER, an assistant professor in a computer sciences college, explained the convenience of correcting students’ assignments on his phone: ‘I downloaded a Blackboard instructor app on my phone, which made it easier to monitor my students’ submissions. When I receive notifications of submissions, I correct the assignment on my phone.’
Two out of the seven participants reported no change in their assessment practices because they taught graduate students, and used the same assessment methods that they had followed before the pandemic. RAID, an associate professor in an education college, expressed this view, as she did not experience fundamental changes in her practices. Instead, she only experienced a lack of face-to-face interaction to assess students:
‘As a postgraduate professor, I do not feel any fundamental changes have occurred. Except for the fact that I could not see my students face-to-face and changed some presentation assessment criteria, I did not experience any other changes. I gave them an open-book exam and 24 hours to finish it to assess their high-order skills. I assessed how they analysed the information, linked them, gave their points of view, and showed their creativity.’
Assessment-related communication
All the participants reported that online learning offered more channels for online assessment-related communication with their students, such as discussions on the requirements for and expectations from assessments. While some students resisted changes in assessment methods or types and many felt overloaded by assessment demands, several staff members remarked on the positive changes in their students’ behaviour; for example, they noticed that some students had become more confident and better communicators (more active in discussions and small talk), showed greater commitment to their studies, and improved their performance in assessments after attending online classes. Further, some of the participants mentioned that their students’ understanding of the subject and performance in assessments had improved, thereby improving the overall course learning outcomes. However, five participants stated that they missed maintaining personal contact with students during examinations; for example, being in the lecture or examination hall to interact with students and invigilate examinations, seeing their facial expressions, and having small discussions with them. Students were not required to keep their cameras open during lectures or examinations, which also increased the likelihood of cheating.
The students found themselves in a situation wherein they had to learn how to work on assessment tasks they had not been previously introduced to or trained in, such as writing research papers. AHMED—a lecturer in a business college—noted the following transition from traditional to alternative assessments, and how difficult it was to introduce and communicate the new tasks to students: ‘They found the methods vague, and had lots of questions. If I ask them to do a simple research project, not a real one, I just ask them to collect information about a topic. Students ask for written guidelines (step-by-step documents) to do the tasks. They were scared of this new experience at first, but after discussions and completion of the tasks, they gained confidence.’
The participants also noted that communication from their departments could be improved for better conveyance and clarification of information on the expectations of their students and for the freedom to act upon them without restrictions. MONA believed that faculty could have benefited from a clearer departmental plan.
Assessment fairness
The participants shared their views on fairness and concerns of academic integrity. NASSER mentioned that cheating was easy—not only in examinations, but also in assignments and projects—so he approached it by discussing the results with his students in order to determine who had actually understood the subject, and who had cheated. This process, as he described it, had a negative side: ‘Students who gained marks that they didn’t deserve would learn a lesson from the open discussion in front of their classmates, but this is exhausting for us [faculty].’ In response to the question on how he achieved fairness in examinations, the answer was, ‘I choose one day to conduct exams for all the groups, to achieve fairness, so they don’t copy the questions. But with e-learning, they can capture the screen. Even though the questions are randomised, parts of them can be revealed.’ This meant that, regardless of having a randomised question bank, some questions would be similar and possibly leaked between students.
NOOR mentioned several practices that were followed to ensure that assessment practices were standardised, differentiated, and equal. She articulated her approach as follows:
‘I differentiate between assessment methods. First, I don’t put the biggest weight on examinations, as some students are not good at taking exams. Second, I like to ask my colleagues about possible assessment strategies, and test these before implementation. If I notice any issues in students’ performances, I make changes with the second group. Third, students’ results give me an indication of whether the assessment tool is fair. If it is not fair, I assign other methods for students to get grades. I also use their feedback to improve the exam questions.’
NASSER and NOOR stressed the role of collaboration with colleagues and students in designing and delivering fair assessments, and evaluating assessment tools for future development. The awareness and commitment that they showed prove that the exercise of care in their practices had developed with accumulated teaching experiences and constant consultation. Meanwhile, SAEED relied on assignments as he found them the best way to achieve fairness during online assessment, and MONA believed that following clear assessment standards were an important part of ensuring fairness.
Measurement theory
The participants listed the use of several methods to achieve reliability and validity in their assessments, such as reviewing questions and collaborating with colleagues. NOOR described her approach:
‘I use specifications to make sure I write balanced questions, in terms of numbers and percentages from all the lectures. I compare my students’ coursework grades with their final exam grade. I also compare their grades with those of my colleagues’ students, because it gives me an indication of their level knowledge and any possible problems. Also, I give my colleague a random sample to correct, and ask her for a report.’
In the same vein, SAEED described his process of writing multiple choice questions—reviewing the questions and answer options—to ensure that the questions and options were clear, fair, and suitable. Both he and NOOR exchanged examination samples with their colleagues to check and review them for suitability.
What were the main challenges faced by the participants in securing their online assessment to ensure academic integrity was maintained during the pandemic?
The participants were asked about the challenges they faced in redesigning and administering assessment tasks during the pandemic to be able to secure these tasks. The participants reported an increase in the workload, as a result of the efforts to secure assessment. In this process, there were several concerns of academic integrity, assessment security measures, technical issues, inadequate training, and vague guidelines.
Increased workload to secure assessment
The increase in academic workload, due to the demand of securing assessment, was a noticeable factor that affected the participants’ experiences. Securing the assessment process included redesigning tasks to fit the new mode and putting certain measures in place to minimise cheating. AHMED’s opinion was seconded by SAEED, an assistant professor in a social sciences college, who found that online assessments had increased the teaching staff’s workload since they had to introduce students to new assessment methods and tasks, encourage them to examine various options, attend to their questions and concerns, and work with colleagues to complete the tasks. He added that group work between students was also an extra workload, saying, ‘We replaced the final exam with projects, and that added to our load as students could not interact physically, hindering their team building. I found that in group tasks, only one person does all the work.’ Further, participants of the study who were very interested in using innovative assessment methods also felt the increase in workload, but insisted on using accurate and varied methods despite facing several limitations in online learning and teaching during the pandemic.
Securing assessment
Securing online assessment by conquering online cheating was one of the foremost challenges and disadvantages of online assessment, as reported by participants who were concerned about academic integrity. An important point to note is that all the participants conducted assessment and examinations without live proctoring (i.e., neither cameras were opened nor online invigilation was implemented, thereby increasing instances of cheating). NOOR shared her views as follows:
‘There were negative sides to online examinations. I could not judge the credibility of the marks. It took me time and effort to check if cheating had occurred in an exam. In some cases, I knew the level a student was at and hence was surprised by her mark in the exam. I could tell that some of them had cheated by comparing their answers. Comparing answers in the online mode is difficult than comparing on papers. In some cases, I had to call the student to check if cheating had happened or not.’
When the participants were asked about how they managed to secure assessment and minimise cheating, MONA, an assistant professor in an Islamic studies college, answered that there was a need to build a huge question bank to reduce the risk of cheating attempts in addition to other measures, such as giving students different versions of the questions. NASSER mentioned that he had learned from the first phase, and thus created a question bank during the summer: ‘I overcame negative outcomes during the summer course by making a question bank: true and false, multiple choice, essay questions, et cetera. The questions are varied, and show students’ individual differences. I created 200 questions; 20 questions are randomly chosen for each student.’
One of the challenges was the issue of students creating groups for cheating. Some students made use of not just the available access to resources, but also to each other for answering exam questions. NASSER noted that one of the reasons behind easy cheating was the lack of or poor training of the teaching staff, so they chose the easiest options in the absence of guidelines to inform their decisions. Online learning and assessment are powerful tools, but the proper use of this technology is yet to be learnt. Therefore, participants stressed the importance of policymakers issuing adequate training and guidelines. In fact, the major limitation mentioned by the participants was the gap in online learning due to lack of clear guidelines and sufficient supporting documents. NASSER shared this view as follows: ‘They tell us to have mercy on students; we also want to measure their learning and improve it. But the skills and technical support are poor. I suggest they add a Q and A section, and advanced-level documents.’
Technical issues
While some of the teaching staff considered correction and scoring in online assessment correction a convenience, the use of phones to take online examinations was reported as being troublesome by many students who did not have other options, such as a laptop or tablet. SAEED mentioned that some of the technical issues faced by his class were caused by his students’ complete reliance on using their phones. According to SAEED, ‘Students were supposed to have a laptop and speedy Internet connection, but, unfortunately, they pay 4000 Saudi Riyals for a phone but not 1000SR for a laptop. We faced problems during exams because they used their phones, even though the guidelines said they had to use a computer.’
AMEERA also mentioned that the differences in internet infrastructure among students made the entire online teaching/learning process difficult. To overcome this issue of unfairness, due to privileges or lack thereof, she suggested facilitating online assessments on campus in the future, stating that, ‘exams can be conducted online but in the university labs, where an excellent network is in place and there are technicians to support and solve problems instantly.’