Mumbai University: Online mess, offline delay

On July 31, 2017 Kalyani Shinde, a third year BSc student at a reputed college affiliated to University of Mumbai, was eagerly awaiting her results. But, to her disappointment when she checked the university website, it showed that her result was ‘held in reserve’. When she enquired about her reserved result, Kalyani did not get any answer from the university officials except for a promise that she would get her result ‘soon’.

In January 2017, MU Vice-Chancellor Sanjay Deshmukh announced that all the evaluations would be done by the university through an on-screen process from April. The initiative was expected to reduce the malpractice of tampering with answersheets. What looked like the biggest revolution in the examination system fell apart due to the delays and glitches in the entire process. The days, after the announcement was made, were marred by a series of delays — from the tendering process to setting up the evaluation system which led to chaos like never before.

With results of over 4 lakh students yet to be announced, DNA’s investigation revealed several shocking anomalies at various stages in the process, which have not only led to delay but have also raised concerns about the transparency of the process.

The series of errors began at the time of examinations. Hundreds of papers under various courses could not be processed due to errors in marking their seat numbers in the OMR sheets — leaving a circle unshaded, making additional circles, marking two figures in the same row etc. These errors went unnoticed by the supervisors causing a big hassle for officials at the university while scanning the papers.

While teachers at each centre packed the answersheets into separate packets and sub packets as per the subject and the student criteria — freshers, repeaters and old students — each have a separate marking scheme, when these papers entered the university’s scanning section at the Kalina campus, the sub packets were ‘mixed’ leading to further confusion. At this point, as the answersheets were wrongly mixed, the two agencies that have been assigned the task of scanning made errors in bar-coding the papers before they were torn. Bar codes of some papers were interchanged due to which the first page of each answersheet could not be rightly linked to the rest of the paper. Even before ensuring whether the bar coding has been done correctly, the agencies gave a ‘Spine Cut’ to the paper, which means that each page of the paper was separated from each other for scanning. As a result of these errors, several supplements were shown as individual papers while teachers had no idea about what to do with such papers. Mixing of the packets also meant papers having separate Question Paper (QP) codes being uploaded in the same lot leading to teachers getting papers of the ‘wrong’ subject.

Finally, when the papers entered the results section, all the errors in the previous stages added up, and made it difficult to locate for certain results. This led to a long and tiring process of ‘reconciliation’  which is the process of cleaning errors and matching the different parts of the paper together to be able to generate the results.

While the entire process of onscreen evaluation is supposed to be smooth due to the complete digitisation involved in the various stages, poor planning of the whole process, ignoring the role of stakeholders and lack of monitoring at various levels has led to errors which have led to a complete collapse of the system. With this mess, the ordeal of thousands of students like Kalyani is not likely to get over anytime soon.

HOW ON- SCREEN MARKING (OSM) WORKS

1 EXAMINATION AND PRE-EVALUATION PROCESS

Students write their papers in the examination hall at their respective centres. Before students begin writing the answers, the supervisor asks the students to shade the OMR sheet containing circles wherein they are supposed to shade the bubbles for their seat number and question paper code (QP). Separate codes are assigned for each of the subject combinations. Also, there are separate codes for freshers, repeaters and old students within these subject codes. After students shade the circles as per their details, the supervisor’s sign the paper after ensuring that all the details are properly filled. The teacher then takes attendance on a sheet. After time is up, the supervisor collects all the answersheets from students. These collected answersheets are then sorted as per subjects and then as per the three categories of students —freshers, repeaters, and old students. The first page is separated from the rest of the paper and is packed separately. Answersheets of one subject are kept under one big packet with several sub packets. This packet is then submitted to the dispatch section in the centre.

Illustration: Gajanan Nirphale

Likely Errors: If there are mistakes in the shading of bubbles, the system fails to recognise the papers.  

2 SCANNING AND PRE-ASSESSMENT PROCESS

From the dispatch centre, the paper travels to the Image section, set up at the university’s Kalina campus. The papers are scanned here. At this stage, two assessment agencies — Meritrac and SSK — have different tasks.

The top half of the first page of the paper is separated from the bottom half. The top half of the page has the seat number and all the other details like QP code, while the bottom half has all the other details barring seat number. Two separate companies scan these two parts of the first page. Before scanning, a bar code identical to the upper half of the paper is attached to the lower half and the remaining pages of the paper. Papers are then given a ‘Spine Cut’, which means the stitch is removed and every single page becomes separate for scanning. These papers then arrive at the Centralised Assessment Process centre. Here, SSK, scans the first part and sends it to the system. Then, Meritrac, scans the second part and sends it to the system.

Illustration: Gajanan Nirphale

Likely Errors: If there are errors in bar coding, the data with the top half and the bottom half would not tally. This would cause issues later at the time of declaration of results as locating answersheets coinciding with the seat numbers would be difficult.

3 ASSESMENT

Each teacher is allotted papers of certain QP codes. They can open the scanned paper in a PDF format and asses them. The teacher logs into the system using the username and password sent on email. Before logging in, every teacher gets a One Time Password (OTP) on their mobile phone which they have to enter into the system to be able to access the papers. The system records the time of log in and log out. Only when the system of evaluation is ‘not active’, which means that not a single teacher is evaluating papers can a fresh teacher be ‘tagged’ into the system after which he can evaluate papers.

Teachers might get papers of ‘other’ subjects due to errors in QP codes. In such cases, they can ‘slide’ the paper (put it aside, and go to the next paper). Each set of papers have different login id and password. A teacher who assesses papers for multiple courses has to log into multiple dashboards. All questions need to be assessed before the paper can be submitted. Ten per cent papers in each subject are moderated, which is the last step in the assessment.

Illustration: Gajanan Nirphale

Likely Errors: Sometimes, due to sorting errors, the supplement and the main paper get scanned separately, as a result of which teachers might only get one part of the paper. If the teacher slides more than 10 papers in one session, the system does not generate more papers. At such a point, the teacher calls the help desk and gives details of the errors after which new papers are re-generated. Teachers often finish papers from a single dashboard but later realise that a lot of papers are pending in other dashboards.

4 RESULT DECLARATION

After the papers are evaluated and the marks entered, the task of Result Unit begins.

Here, a university official checks the attendance sheet for each class and centre with the coinciding entries of answer sheets and their codes in the system. He also checks the supervisor’s report for each bunch.

Illustration: Gajanan Nirphale

Likely Errors: Results of students whose papers have errors caused due to wrong shading of bubbles or wrong bar coding then go to the reconciliation unit. Here, a senior official tallies the QP code, the corresponding seat number, and all the sheets of the paper and then feeds the final result into the system. This process is called as ‘Reconciliation’. The results of students whose papers are still being reconciled are often reserved by the university.

CHRONOLOGY OF THE DELAY IN DECLARING RESULTS

• JANUARY 24, 2017: MU VC Dr Sanjay Deshmukh announces that the university would evaluate close to 19 lakh answersheets through the On Screen Evaluation mode.
   
• APRIL 5, 2017: MU invites tenders for the OSM process on its official website. Tender deadline extended twice as MU struggles to find minimum number of bidders.
   
• APRIL 28, 2017: MU appoints Meritrac, a Bengaluru-based agency for the task and begins setting up the evaluation process
   
• MAY 2017: Evaluation work finally kicks off, almost a month after majority exams at the university get over. Teachers continue to experience glitches, server failures during evaluation.
   
• JULY 4, 2017: Governor of Maharashtra and the Chancellor of MU, Vidyasagar Rao raps MU V-C for the delay in declaring the results. Orders all the results to be declared by July 31.
   
• JULY 21, 2017: MU ropes in Nagpur University seeking help for evaluating papers for the Commerce stream. This is followed by setting up centres at Shivaji University, Kolhapur and Aurangabad university.
   
• JULY 28, 2017: MU V-C admits in a high level meeting that results cannot be declared within the deadline after several concerns regarding the issue are raised in the state assembly.
   
• JULY 31, 2017: MU officials say that they would try to declare most results by August 5.
   
• AUGUST 1, 2017: MU V-C while speaking to protesting students from the Yuva Sena admits that Commerce results cannot be declared until August 15.
   
• AUGUST 2, 2017: With over 2 lakh answersheets yet to be evaluated, MU opens Help Desk to help students in getting confidential results for admissions to other universities.

FACTS AND FIGURES

Total students
4,77,719

Total exams
477

Total answersheets

Arts
256808
Commerce
834439
Law
102483
Management
119187
Science
186863
Technology
276919
Total
17,76,699

Pending for evaluation

Arts
9660
Commerce
230493
Law
24993
Management
2221
Science
2364
Technology
1163

Figures as on August 2, 2017