Proctoring in online exams: how does it work?
Stepik is currently the only Russian-language educational online platform for which anyone can create an exam with proctoring (online support and student monitoring). Moreover, such an exam may have a specified duration, a limited time to pass, the exact date of commencement.
In this article we will tell you about why proctoring is needed, and describe in detail the integration of the ProctorEdu online proctoring system with the Stepik platform.
')
If you know what online proctoring is and what benefits it has for the implementation of the online exam process, skip the following narrative and go straight to the implementation questions in the What is the Stepik exam?
Almost any areas of knowledge - from IT to HR - are suitable for translating educational materials and test items into an online format. For example, you can conduct online training for new employees in the company, or perform automatic code verification, or teach how to work with databases - all this directly on the educational platform without the need for teachers or students to deploy the infrastructure on local computers. This solves the problem of checking the knowledge gained, since You can conduct a final exam in the course in the same online format in which the student took an online course. You do not need to force a person who wants to get a certificate to go to the Testing Center during working hours, be nervous because of a new unfamiliar environment, when you can provide the opportunity to take an online exam in the same familiar interface at a convenient time while sitting in a comfortable chair.
Consider the IT industry: for example, the Microsoft Certificate Authority allows you to pass certification exams remotely, Hortonworks organizes obtaining certificates for administering its own platform, providing access to a remote virtual machine and a list of tasks to be done.
And yet, despite the inconvenience for the examinees, the cost of certificates in the IT industry is quite high, and the equipment of physical premises for examining purposes, for which the person who passes the test must be checked and strict control over the process of passing can be carried out. But if you look at a situation where a fast-growing company, which operates in several regions, needs to be massively trained and tested the knowledge of employees, and then on the basis of this information, making financial and HR decisions, the process of organizing the physical presence of employees during exams can become very expensive and organizational. difficult to scale.
Let's imagine that a new competitor to Uber is Taxibus, which calls politeness and professionalism of its drivers as its key advantage. To do this, the driver, working with Taxibus, must quarterly pass the theory of traffic rules and the exam on the rules of effective communication. In the case of a successful exam, the driver receives a reduced rate, which allows him to give the transport company not 7, but 5% of the commission of the cost of each trip. About 7 thousand drivers in 15 cities of the country have already joined the work with Taxibus, and in order not to spend money on renting premises and paying salaries to the staff of proctors who control the exam process, the company has developed online examinations that are available to drivers at any time personal computers or mobile devices. However, some drivers do not want to prepare for exams, but they are attracted by the opportunity to receive an additional bonus from the company. Therefore, they ask their friends, who have already passed the exam, to pass the exam instead.
In order to unequivocally confirm the identity of the examiner and to be sure that the exam was passed honestly, without the use of outside help, the training organizers can connect the online proctoring system to the exams in their learning platform. The main task of such systems is to confirm compliance with the exam regulations in automatic or semi-automatic modes (which we will say below), as access to the physical proctor with the physical presence of the examiner, through access to the webcam, microphone and user’s desktop. Read more about how such systems work here .
In the case of passing the online examinations in IT disciplines, the same system can be used. Proctoring systems are quite complex in themselves, since they require working with several video streams, organizing video and audio information storage systems, face and speech recognition technologies, developing proctor interfaces for managing the exam and examiner process. Therefore, Stepik did not develop his online proctoring technology, but uses existing solutions. In this post, we want to tell some technical details about how we integrated ProctorEdu online proctoring system to work with our platform, and how our course teachers can set up proctoring in exams in their Stepik courses for more reliable verification of exam sessions.
In Stepik, lessons are combined into modules or training weeks. Any module can be turned into an exam by selecting the “Exam” module type in the settings and setting the exam duration. Students will be able to proceed to the exam in the time period set by the course instructor by clicking the “Start exam” button. While the exam lasts, the student sees a countdown timer. After completing the exam, the student will no longer be able to score points in the exam.
When a user starts an exam, an exam-session record is created for him in the database, in which the date of the exam start by the user is recorded. Knowing the start date and exam duration exam_duration_minutes, we in the interface display the remaining time to the user. The user will receive only those points for the correct solutions, which he managed to get before the end of the exam.
The implementation of the ability to take exams with online proctoring imposed additional restrictions on the ability to create an exam session - the user should be able to sign up for an available proctoring session and proceed to the exam only when he received permission from the proctor. In this case, the proctor should be able to early close access to the exam in case of violations, as well as assess the presence and absence of violations during the exam, which may affect the final grade. To this end, we added another proctor-session model in Stepik, which is associated with the exam module and allows you to create an exam-session only if there is a running proctor-session (with the start_date installed).
The proctoring system will now be able to control the proctor-session through the API using the following commands:
The proctoring provider can also get a list of user exams using the fetch method to display them in the interface when recording for a proctoring session.
As we mentioned above, for a student, the difference between an exam with proctoring and a regular exam is that it is necessary to register for proctoring in advance when the proctor who controls the exam has free time slots. In the Stepik interface, this is solved on the exam page - the user is prompted to sign up for a proctoring session, the user clicks a link to the ProctorEdu (the proctor-session is created for this user at this point), is registered there with his Stepik account via the OAuth2 protocol and selects available for passing the exam.
The student will be able to start the exam (ie, create an exam-session) only if there is a running proctor-session (for which the proctor has executed the start command). When a student exam is available for passing, he sees the active button "Start exam" and can proceed to pass the exam.
To manage proctor-sessions of users, the proctoring provider (to which we assign the special role “proctor”) registers our server application using the OAuth2 protocol. This application will be able to manage student proctoring sessions on the Stepik platform, using the commands we described above.
For the teacher or organization that decided to conduct proctoring exams, do the following:
To use the services of proctoring, what we have already said is enough. for teachers, all the settings end there. Next, we will describe in more detail what is happening inside the ProctorEdu system when interacting with Stepik, since there are also many interesting technical issues here.
ProctorEdu Automated Proctoring System is a cloud-based SaaS solution (software as a service) that allows you to organize and conduct proctoring exams. Sessions are recorded and analyzed for violations, after which reports are generated.
ProctorEdu provides interfaces for organizing the working space of proctors on the part of an educational organization, but can also provide its proctors.
For a more complete interaction with e-learning systems (LMS) and for the purpose of automating some functions, ProctorEdu system provides integration mechanisms with other systems, including integration at the API and UI level. ProctorEdu system supports two integration options:
Connecting the LMS to the ProctorEdu system involves implementing a number of API functions on the LMS side and configuring the entry point to the system through an external authorization provider using the OAuth2 protocol. In this case, the student passes the exam through the proctoring system, and already the proctoring system gives access to the LMS interfaces for passing tests or other types of tasks under the supervision of the proctor.
Another option allows you to add proctoring functions to the LMS itself through seamless integration at the interface level. A Supervisor SDK development toolkit is available for this, which allows you to use proctoring functions directly from the LMS exam page, just plug in a special JavaScript library in the LMS and set up a call to the corresponding library functions. It is quite simple and comparable in complexity to placing the code of the online consultant on the site page.
Stepik connection scheme to the ProctorEdu system
The first option described above is currently used to integrate with the Stepik platform: a special API has been developed that is designed so that you can flexibly configure the communication protocol between the LMS and the ProctorEdu system in JSON format.
The interaction scenario with the ProctorEdu system is shown in the following figure and can be described as follows:
(the picture is clickable)
ProctorEdu System Interaction Scenario
Before each exam (session), the computer is automatically checked for communication and computer parameters for compliance with the minimum technical requirements: working webcam and microphone, network bandwidth of at least 1 Mbps, network response time of no more than 1000 ms, WebRTC traffic is allowed, set browser extension (optional).
The exam will not begin until these requirements are met. During the exam itself, the webcam, the desktop, and the student’s additional camera are broadcast and recorded, and violations are automatically tracked. At the beginning of each session, a photograph of the student’s face is automatically taken, this photograph passes the procedure of identification, and the percentage of a person’s similarity in a session with a profile photo is determined.
Throughout the exam, continuous verification of the person’s identity at the computer, which is based on the methods of automatic detection and recognition of faces and the method of analyzing keyboard handwriting, is carried out. The proctor can monitor the students assigned to him, see their broadcasts, see notifications of recorded violations. If necessary, the proctor can communicate with the student via video link or via chat. The proctor can also interrupt the exam at any time and indicate the reason in the session commentary.
After the exam is completed, an assessment of the degree of confidence in the exam results in percentages is formed, an interactive protocol with video recording of all broadcasts, a history of correspondence and minute-by-minute detailing of the recorded violations is formed.
Student interface during the passage of the Stepik exam, conducted through the ProctorEdu system
The interaction with Stepik is carried out by a student from the interface of the ProctorEdu system, in this case the exam page is loaded via the IFRAME at the address specified in the session. Examinations are imported from Stepik when a student enters the ProctorEdu system.
To interact with the Stepik platform, the ProctorEdu system API includes the following functions:
At the end of the article we want to bring the pros and cons of exams with online proctoring.
Pros:
Minuses:
If you are interested in doing your exam with Proctoring at Stepik or a full-fledged online course, drop us an email .
Do you think that proctoring results are reliable enough? If you were faced with the task of conducting an online exam, would you use the proctoring function?
In this article we will tell you about why proctoring is needed, and describe in detail the integration of the ProctorEdu online proctoring system with the Stepik platform.
')
If you know what online proctoring is and what benefits it has for the implementation of the online exam process, skip the following narrative and go straight to the implementation questions in the What is the Stepik exam?
Introduction
Almost any areas of knowledge - from IT to HR - are suitable for translating educational materials and test items into an online format. For example, you can conduct online training for new employees in the company, or perform automatic code verification, or teach how to work with databases - all this directly on the educational platform without the need for teachers or students to deploy the infrastructure on local computers. This solves the problem of checking the knowledge gained, since You can conduct a final exam in the course in the same online format in which the student took an online course. You do not need to force a person who wants to get a certificate to go to the Testing Center during working hours, be nervous because of a new unfamiliar environment, when you can provide the opportunity to take an online exam in the same familiar interface at a convenient time while sitting in a comfortable chair.
Consider the IT industry: for example, the Microsoft Certificate Authority allows you to pass certification exams remotely, Hortonworks organizes obtaining certificates for administering its own platform, providing access to a remote virtual machine and a list of tasks to be done.
And yet, despite the inconvenience for the examinees, the cost of certificates in the IT industry is quite high, and the equipment of physical premises for examining purposes, for which the person who passes the test must be checked and strict control over the process of passing can be carried out. But if you look at a situation where a fast-growing company, which operates in several regions, needs to be massively trained and tested the knowledge of employees, and then on the basis of this information, making financial and HR decisions, the process of organizing the physical presence of employees during exams can become very expensive and organizational. difficult to scale.
How can proctoring save everyone?
Let's imagine that a new competitor to Uber is Taxibus, which calls politeness and professionalism of its drivers as its key advantage. To do this, the driver, working with Taxibus, must quarterly pass the theory of traffic rules and the exam on the rules of effective communication. In the case of a successful exam, the driver receives a reduced rate, which allows him to give the transport company not 7, but 5% of the commission of the cost of each trip. About 7 thousand drivers in 15 cities of the country have already joined the work with Taxibus, and in order not to spend money on renting premises and paying salaries to the staff of proctors who control the exam process, the company has developed online examinations that are available to drivers at any time personal computers or mobile devices. However, some drivers do not want to prepare for exams, but they are attracted by the opportunity to receive an additional bonus from the company. Therefore, they ask their friends, who have already passed the exam, to pass the exam instead.
In order to unequivocally confirm the identity of the examiner and to be sure that the exam was passed honestly, without the use of outside help, the training organizers can connect the online proctoring system to the exams in their learning platform. The main task of such systems is to confirm compliance with the exam regulations in automatic or semi-automatic modes (which we will say below), as access to the physical proctor with the physical presence of the examiner, through access to the webcam, microphone and user’s desktop. Read more about how such systems work here .
In the case of passing the online examinations in IT disciplines, the same system can be used. Proctoring systems are quite complex in themselves, since they require working with several video streams, organizing video and audio information storage systems, face and speech recognition technologies, developing proctor interfaces for managing the exam and examiner process. Therefore, Stepik did not develop his online proctoring technology, but uses existing solutions. In this post, we want to tell some technical details about how we integrated ProctorEdu online proctoring system to work with our platform, and how our course teachers can set up proctoring in exams in their Stepik courses for more reliable verification of exam sessions.
What is the Stepik Exam? ?
In Stepik, lessons are combined into modules or training weeks. Any module can be turned into an exam by selecting the “Exam” module type in the settings and setting the exam duration. Students will be able to proceed to the exam in the time period set by the course instructor by clicking the “Start exam” button. While the exam lasts, the student sees a countdown timer. After completing the exam, the student will no longer be able to score points in the exam.
When a user starts an exam, an exam-session record is created for him in the database, in which the date of the exam start by the user is recorded. Knowing the start date and exam duration exam_duration_minutes, we in the interface display the remaining time to the user. The user will receive only those points for the correct solutions, which he managed to get before the end of the exam.
How is the integration with the proctoring provider technically arranged?
The implementation of the ability to take exams with online proctoring imposed additional restrictions on the ability to create an exam session - the user should be able to sign up for an available proctoring session and proceed to the exam only when he received permission from the proctor. In this case, the proctor should be able to early close access to the exam in case of violations, as well as assess the presence and absence of violations during the exam, which may affect the final grade. To this end, we added another proctor-session model in Stepik, which is associated with the exam module and allows you to create an exam-session only if there is a running proctor-session (with the start_date installed).
The proctoring system will now be able to control the proctor-session through the API using the following commands:
- Start - start the session (setting start_date);
- Stop - the user can complete the exam ahead of time through the ProctorEdu interface;
- Submit - the end of the session by the proctor and the removal of the conclusion (the score field, in which the number from 0 to 1 is stored (the scale with “not approved” / “approved” boundaries)
The proctoring provider can also get a list of user exams using the fetch method to display them in the interface when recording for a proctoring session.
As we mentioned above, for a student, the difference between an exam with proctoring and a regular exam is that it is necessary to register for proctoring in advance when the proctor who controls the exam has free time slots. In the Stepik interface, this is solved on the exam page - the user is prompted to sign up for a proctoring session, the user clicks a link to the ProctorEdu (the proctor-session is created for this user at this point), is registered there with his Stepik account via the OAuth2 protocol and selects available for passing the exam.
The student will be able to start the exam (ie, create an exam-session) only if there is a running proctor-session (for which the proctor has executed the start command). When a student exam is available for passing, he sees the active button "Start exam" and can proceed to pass the exam.
To manage proctor-sessions of users, the proctoring provider (to which we assign the special role “proctor”) registers our server application using the OAuth2 protocol. This application will be able to manage student proctoring sessions on the Stepik platform, using the commands we described above.
How to create exam with proctoring on Stepik?
For the teacher or organization that decided to conduct proctoring exams, do the following:
- Create a course;
- Make it a module exam, and fill it with lessons with exam tasks;
- Set the start and end dates of the exam, the duration of the exam session;
- Enable proctoring mode for the course and set a link to the proctoring provider domain, which allows you to create a branded domain for different proctoring customers (currently not available in the interface and made by the platform administrators upon request ).
To use the services of proctoring, what we have already said is enough. for teachers, all the settings end there. Next, we will describe in more detail what is happening inside the ProctorEdu system when interacting with Stepik, since there are also many interesting technical issues here.
Integration of ProctorEdu and Stepik: a view from the proctoring system
ProctorEdu Automated Proctoring System is a cloud-based SaaS solution (software as a service) that allows you to organize and conduct proctoring exams. Sessions are recorded and analyzed for violations, after which reports are generated.
ProctorEdu provides interfaces for organizing the working space of proctors on the part of an educational organization, but can also provide its proctors.
How is the integration of ProctorEdu and LMS in general?
For a more complete interaction with e-learning systems (LMS) and for the purpose of automating some functions, ProctorEdu system provides integration mechanisms with other systems, including integration at the API and UI level. ProctorEdu system supports two integration options:
- Connecting the LMS to the ProctorEdu system;
- Adding proctoring features to LMS.
Connecting the LMS to the ProctorEdu system involves implementing a number of API functions on the LMS side and configuring the entry point to the system through an external authorization provider using the OAuth2 protocol. In this case, the student passes the exam through the proctoring system, and already the proctoring system gives access to the LMS interfaces for passing tests or other types of tasks under the supervision of the proctor.
Another option allows you to add proctoring functions to the LMS itself through seamless integration at the interface level. A Supervisor SDK development toolkit is available for this, which allows you to use proctoring functions directly from the LMS exam page, just plug in a special JavaScript library in the LMS and set up a call to the corresponding library functions. It is quite simple and comparable in complexity to placing the code of the online consultant on the site page.
Integration of ProctorEdu and Stepik
Stepik connection scheme to the ProctorEdu system
The first option described above is currently used to integrate with the Stepik platform: a special API has been developed that is designed so that you can flexibly configure the communication protocol between the LMS and the ProctorEdu system in JSON format.
The interaction scenario with the ProctorEdu system is shown in the following figure and can be described as follows:
- The student is authorized in the ProctorEdu system via the Stepik platform using the OAuth2 protocol. On the login page there is a button to enter via Stepik. Clicking this button will request access to the current user profile on the Stepik platform. After allowing access, the system automatically enters the system without the need to enter a login and password.
- The student enters the ProctorEdu interface and fills in the profile. Filling out a profile is not an obligatory step depending on the requirements of a particular educational organization. A profile photo of the person is taken, text fields are filled in and additional documents are loaded if necessary. The photo from the user profile is further used to identify the student’s identity at each exam. If a profile picture is not taken, it will be taken and saved at the beginning of the first exam and will be used later to identify the student’s identity.
- After that, the student can either immediately proceed to the exam, or it will be necessary to select the time of the exam and proceed to the exam at the chosen time. Exam planning is an optional feature and depends on the availability of proctors online. If the exam scenario assumes an online observer, then it makes sense to use the schedule. In this case, the proctors draw up a schedule of their work, and students plan their exams based on this schedule. It should be noted that there is a limit on the number of simultaneous exams per proctor, usually it does not exceed nine.
(the picture is clickable)
ProctorEdu System Interaction Scenario
Before each exam (session), the computer is automatically checked for communication and computer parameters for compliance with the minimum technical requirements: working webcam and microphone, network bandwidth of at least 1 Mbps, network response time of no more than 1000 ms, WebRTC traffic is allowed, set browser extension (optional).
The exam will not begin until these requirements are met. During the exam itself, the webcam, the desktop, and the student’s additional camera are broadcast and recorded, and violations are automatically tracked. At the beginning of each session, a photograph of the student’s face is automatically taken, this photograph passes the procedure of identification, and the percentage of a person’s similarity in a session with a profile photo is determined.
Throughout the exam, continuous verification of the person’s identity at the computer, which is based on the methods of automatic detection and recognition of faces and the method of analyzing keyboard handwriting, is carried out. The proctor can monitor the students assigned to him, see their broadcasts, see notifications of recorded violations. If necessary, the proctor can communicate with the student via video link or via chat. The proctor can also interrupt the exam at any time and indicate the reason in the session commentary.
After the exam is completed, an assessment of the degree of confidence in the exam results in percentages is formed, an interactive protocol with video recording of all broadcasts, a history of correspondence and minute-by-minute detailing of the recorded violations is formed.
Student interface during the passage of the Stepik exam, conducted through the ProctorEdu system
The interaction with Stepik is carried out by a student from the interface of the ProctorEdu system, in this case the exam page is loaded via the IFRAME at the address specified in the session. Examinations are imported from Stepik when a student enters the ProctorEdu system.
To interact with the Stepik platform, the ProctorEdu system API includes the following functions:
- Getting a list of exams from Stepik.
- As soon as the user is authorized in the ProctorEdu system, a list of available for examinations on the Stepik platform is displayed to him. If we take these exams, they can already be scheduled for a specific time set in the course settings and the Stepik module, or the student can sign up for a convenient time, taking into account the work schedule of the proctors;
- Alert about the beginning of the session proctoring.
- When the exam time comes, the student connects to the appropriate session. At this point, the API sends a request for Stepik and access to this exam opens. Thus, it is impossible to start the exam without a proctoring system.
- Alert about the end of the session proctoring.
- After passing the exam, the student completes the proctoring session. At this point, the API sends a request to Stepik to complete the exam and close access to the exam.
- Sending the conclusion of the session on Stepik.
- After checking the session protocol, the conclusion of the session is automatically or manually determined by the proctor, which can be positive or negative. This result is transmitted to Stepik and displayed next to the exam. If the conclusion is negative, the exam results are not counted.
Conclusion
At the end of the article we want to bring the pros and cons of exams with online proctoring.
Pros:
- Saving money and time for organizing computer classes for exams, searching and training people who need to follow the exam process - automatic proctoring is a proven, scalable and easy-to-configure tool. Simply create a course with tasks and enable proctoring for it. The number of examinees can be one thousand, and ten thousand people.
- Ease of passing - the examinee can choose the time at which he is comfortable to pass the exam - proctoring is available not only during working hours.
- You can take the exam from home in a comfortable environment that promotes high exam results.
- Minimum expenses for equipment: a computer, a webcam and a microphone are at all or are very cheap.
- No problems with the interest of local organizers in the successful results of the examinees. On the EGE, teachers often help their students because teachers are punished or rewarded for the results of their students. There are cases when even in the classrooms in which video cameras are installed, the teacher arranges with the students to ask the teacher out loud. Then, in the absence of sound, this kind of cheating is difficult to detect.
- Fast results: asynchronous procoring allows you to filter only suspicious sessions that can already be viewed manually and confirm that the exam was fair.
Minuses:
- The presence of a fairly fast and stable Internet channel (1 Mbps network connection speed).
- The problem of leakage of tasks - we send a “first mover” who clicks and remembers the answers and then sends them to friends (solved by the examiner by creating variations of steps, by an intensive flow of questions, the answers to which are difficult to remember).
If you are interested in doing your exam with Proctoring at Stepik or a full-fledged online course, drop us an email .
Do you think that proctoring results are reliable enough? If you were faced with the task of conducting an online exam, would you use the proctoring function?
Source: https://habr.com/ru/post/329420/
All Articles