Indian edtech sector took off in 2020 as schools are shut amid pandemic. It gave unicorns such as Byju’s both revenue and new users. For the first time, edtech solutions turned from merely being after school remedial supplements to the main fare for more than 285 million students (1).
The impact is quite telling since the Indian mainstream edtech sector boosted more than a whopping 2.2 billion USD. Moreover, almost every edtech firm added millions of free users on their platforms after lowering their paywalls.
Byju’s, the pack’s acknowledged leader, now has more than 70 million users for its interactive video content (2). At the same time, its competitors also grew in bounds and leaps. Unacademy, the second-largest edtech unicorn in India, quadrupled its paid user base between February and September to 350,000. Vedantu, a rival to both Unacademy and Byju’s, claimed a 10x growth from its 50,000 user bases in January.
Unacademy, which is on the sprint of making acquisitions this year, added a new company on its list, NeoStencil (3). On the last day of 2020, it acquired the government test-prep platform. Notably, Neostencil is Unacademy’s sixth acquisition in the last 12 months. There is no doubt that the move flanked a breakout year for the Indian edtech sector.
Regardless of billion-dollar valuations and impressive growth metrics, the question of whether online learning works or not has become most relevant in 2020. Notably, edtech firms have not put their financial tonnage on creating a metric to measure learning outcomes to date. And if they have, there is no available public data.
According to Vikas Verma, an investment director of Avaana Capital (4), a VC firm familiar with the Indian edtech sector, the first phase of edtech growth, they have focused on customer acquisition. Stakeholders like parents and VC depend on engagement and user experience only as proxy indicators; hence there is no demand for deeper metrics.
Doing More With Less
It is worth highlighting that the billion-dollar valuation for these firms is based on the immense growth headrooms they continue to have. For the most part, their smartphone-friendly and English-firm lessons are mostly accessible to the metro and elite audiences. These are merely 10 to 20% of the student population.
India’s most impacted student group is the 113 million government school students, making up about 50% of the total student population. They typically students with low-resource settings and limited access to online content and smartphones. For them, the shutting of schools means a breakneck fall in learning levels (5).
In response to the current scenario, an open-source ground up and public edtech superstructure was set up almost overnight. It is powered by a diversified group of non-profit organizations such as Avanti Learning Centres, Central Square Foundation, state education departments, and government teachers.
Together, the team aims to send quality content sourced from non-profits, curated via state education departments or external consultants such as CSF and Samagra via familiar platforms such as WhatsApp. They are hoping that it would help students keep up with their more privileged equals.
These free, low-tech, and public edtech has already swept up viewership numbers to their counterparts VC-backed picks such as Byju’s. For example, TicTacLearn (6), a 10,000 video-strong content created by CSF, Google, a partner in the initiative, has witnessed massive traction on YouTube.
It is a Hindi Youtube channel that hosts short, animated explainers. Their videos are available in five Indian languages. TicTacLearn started in 2018 after getting a 3 million USD grant from Google. While the grant amount is a fraction of the Indian education budget, it is pulling the content gap in more than six Indian states. It surged during the pandemic, and between May and November, it even gained a slim margin over Byju’s in terms of viewers with an average completion rate of 40%, as per CSF (7).
However, unlike its private peers, the public edtech platform can’t afford to coast on only vanity metrics, especially when it has, even if temporary, replaced schools as the main learning source.
According to Akshay Saxena, co-founder of Avanti, a non-profit edtech platform (8), any intervention that does not measure its impact would remain a black box and is impossible to adopt at scale. Avanti specializes in maths and science content and offers these state programs for 9 to 12 grades.
He added that it is the reason why Avanti’s interventions in Haryana are now forming the basis for a randomized and controlled trial study by the J-Pal, Jameel Action Poverty Lab (9), an international research organization. It includes tracking access and engagement with the online lessons sent to students. While the same size is small, with only 1,563 students he admitted, the interim data is encouraging. He added that among surveyed students, 60% of them engaged with WhatsApp lessons in the 1 to 2 days before the survey.
However, the data is not yet sufficient enough to make the leap from engagement to real learning. However, mapping learning outcomes under the frugally-funded public edtech platform set up a strong contradiction to what popular edtech companies controlled by millions of dollars in VC funds spend their money on. It could help obtain the learning results from groups of edtech users.
The J-Pal Survey
The J-Pal survey, held between August and October, randomly selected students from two groups. One group was unity where Avanti conducted offline programs even before the pandemic, and another one where Avanti had no physical presence. The study’s goal was to test whether a physical workforce could increase enrollment in online government classes via channels such as WhatsApp groups. Notably, the difference the study found was negligible.
According to Saxena, it is a clear indication that public edtech infrastructure could get a huge lever if it has government-mandated usage. Previously, state governments were not interested in edtech solutions and lay their bets on hardware solutions such as tablets and smart classes. However, after the pandemic, they are now corralled to look for cheaper solutions online.
Now the public edtech sector initiatives have different names in different states, such as ‘Har Ghar Paathshaala’ in Himachal Pradesh, ‘Ghar se Padhao’ in Haryana, and Uttar Pradesh has ‘Mission Prerna.’ State governments are also looking to boost this content on government platforms such as Diksha and satellite channels such as Edusat (10).
According to Saxena, the government would listen if one can figure out the content and a reliable dissemination strategy. It seems like the bureaucracy is moving swiftly in Indian states, unlike their stereotypical image. With partners’ help, state governments have created parent-teacher WhatsApp groups, curated content, and conducted assessments.
Another crucial finding of the J-Pal interim study is the near-monopoly of WhatsApp to communicate with students. The survey indicates that more than 80% of students access most of their content via WhatsApp groups. WhatsApp groups between the state and teachers also made it easier to send lessons during the pandemic hit.
The mainstream edtech firms managed to conjure up a perception of hectic usage during the pandemic via several press releases. For instance, Byju’s claimed an increase of 71 to 90 minutes daily on a user’s time spent on every session.
Unacademy and Vendantu also claimed that their users had watched a total of billion minutes of content on each of their platforms by June. But, these numbers do not indicate any engagement or learning outcomes.
In a way, conventional edtech solutions have failed to emerge as vital contributors to the public edtech foundation.
The study further indicates that Byju’s use as a solution held at an appalling 2%. At the same time, Vedantu also didn’t do much better at 4%.
There are arguments that the mainstream content, which is primarily in English and needs high-end smartphones, is out of reach for this population segment. Alternatively, the public edtech infrastructure is bite-sized, vernacular, and mapped to the grade-wise syllabus, making it easier to navigate by students and their parents.
One could also say that the government machinery, WhatsApp groups, and content have observed better results than mainstream counterparts. However, if it improves learning outcomes or not is a keen-witted inquisition.
The Lack of Real Metrics
Good metrics are invaluable. Conventionally, though, the Indian education system, both offline and online, has merely paid lip-service to metrics. Apart from a few competitive exams nation-wide and the dreaded highschool board exams, there are virtually no agreed benchmarks to test students learning.
Such lack of good metrics in the edtech is only a reflection of what has happened in offline education. According to Vamsi Krishna, the co-founder of Vedantu, it is baffling that more edtech platforms don’t talk about their results openly. However, even these metrics are not ideal for determining the learning outcome.
It is why a public edtech system is compelled to measure outcomes and publish its finding, an exciting break for the conventionals. Several experts also believe that these traditional platforms have not exactly improved students’ learning outcomes, except for being on focus for the last six to seven months since the pandemic’s rage. Many also believe that they have merely prevented students from forgetting what they have learned so far.
Aditya Chopra, a part of a governance consulting firm Samagra (11), helps the Haryana state government for the same reason. He shifted the goalposts and started curating and disseminating the content. He believes that if there are more than 20 competencies to master in a year, we have to narrow it down to four to five primary ones. And even if children are receiving the WhatsApp messages, we are not sure about compliance.
J-Pal Avanti survey also highlighted the same fact. During the survey, only 18% of the students reported having studies after a WhatsApp text, and only 11% had watched YouTube videos.
Chopra and his team didn’t want to rely on engagement metrics such as Youtube views alone. They realized that the metric is skewed since students could get content from other sources, too, like directly from their teachers.
The insight level evolved within a year since, apart from video lessons, teachers also started to send weekly quizzes via Whatsapp to students in grade five to twelve. It proved to be an interesting dipstick. Samagra also witnesses the engagement rate shoot through the roof with these quizzes. About 65% of Haryana’s student population across all grades were attempting the quiz every week.
It is impressive, according to Chopra, that the average test scores are about 70%. He also admitted that these quizzes are not nuanced tests but that the program is trying to maintain the status quo in learning through them.
The quiz is a handy metric used by convention edtech company usually at the end of the season or a video. While they give each child and parents a detailed report of their performance, it has never been shared publicly if it has even been collected.
Notably, even third-party researchers such as J-Pal did not get access to scrutinize it. There is only one exception when in the past year, J-Pal conducted an RCT on Mindspark, an in-classroom, an online adaptive tool built by Educational Initiatives, Bengaluru (12).
After almost ten years of its inception, Vedantu has now started experimenting with a metric developed in-house to assess if its programs are productive. Krishna has called it a height chart for learning since it checks how much a user has grown within a year of enrolling into their classes.
It checks each concept, breaks them into competencies, and compares a student’s grades with an average score. Notably, the metrics are only a year old, and Krishan believes it would evolve even further. He says that if the user’s learning levels have improved more than the average, then we can say that Vedantu is effective.
The Next Wave
As thorough it may be, Vendantu’s metric is still an internal stage that is not yet open to scrutiny by a third party. However, there is no denying that the purpose of metrics these conventional edtech players have developed would likely strengthen their terrains.
According to Verma, edtechs would win the next edtech wave by publishing actual results and not only announcing fundings (13). He added that previously, there was no choice since it was only Byju’s. But now, people can start comparing techs based on their metrics. Learning outcomes can be a strategic play for a young edtech firm that may not have fundings like Byju’s but can offer stronger learning outcomes.
Verma also believes that the challenge here is to build a common benchmark and the Indian edtech industry is still very young to make such a move.
However, the public edtech foundation depends on it, especially when the schools are planning to reopen in 2021 partially. Chopra explained that schools are unlikely to go back on full attendance (14). For instance, schools did open partially for senior grades in November, but the attendance only hovered around 20 to 30%.
While engagement levels are a definite win, the public edtech system requires better metrics than a weekly WhatsApp quiz, which can be easily gamed to make at-home learning a fixture of the academic year and not a fallback.
Both Avanti and CSF are planning to invest in baseline and end-line surveys for their part to know how notifications, gamification, nudges, and personalization tactics used liberally by conventional edtech platforms can help improve learning levels in students. There are possibilities for the data to have some implications for the entire public edtech system.
It is also harsh to judge mainstream edtech platforms based on state and donor-funded efforts since both works towards different ends. However, the pandemic has blurred some of the lines, and for the first time, both systems are almost eye-to-eye. These conventional edtech firms need to consider what a lean, public edtech system, with the government backing, can potentially achieve (15) for more than half of the Indian student population.
