Application of A/B Testing Experimentation on Government Digital Products to Enhance Teachers' Skills and Capabilities in Indonesia

Authors

  • Bagoes Rahmat Widiarso Metra-NET TELKOM
  • Bhaskoro Abdillah Muthohar Metra-NET TELKOM
  • Septi Rito Tombe Metra-NET TELKOM
  • Putri Wikie Novianti Metra-NET TELKOM

DOI:

https://doi.org/10.26740/jpsi.v7n2.p67-75

Keywords:

Microlearning, self-determined learning, Bayesian

Abstract

The Ministry of Education, Culture and Research and Technology (MoECRT) in Indonesia has developed a digital product named “Microlearning” within the Merdeka Mengajar digital Platform. “Microlearning” enables teachers to enhance their skills and capabilities by doing self-determined learning that is followed-up by the submission of Aksi-Nyata reports. We improved users’ journey flow and provided an alternative of user-interface designs in the platform, in which we hypothesized that these improvements could boost the number of teachers participating in the Aksi-Nyata report submissions - the main expected outcome of the “Microlearning” digital product. We conducted a randomized-control-trials experimentation in the digital platform to scientifically quantify its effect on the product, by including 1.6 million users who logged-in to the platform during a two weeks period. This experimentation (known as an A/B Testing) allowed us to simultaneously compare two groups within a specific period and within a controlled-environment. The bayesian-based experimentation revealed that the new developments increased the number of users who started to learn a topic, as well as increased the Aksi-Nyata report submission in the “Microlearning” product, as compared to the legacy-designs. A follow-up study showed even more convincing results, which shows the number of submitted Aksi-Nyata reports increased by 590% up to two months after implementing the analytics-based initiatives for all logged-in users in the platform.

References

Bryan, J., Wickham, H., Posit, P. B. C., Kalicinski, M., Valery, K., Leitienne, C., Colbert, B., Hoerl, D., Miller, E. 2022. Computer Software. [Online]. Available: https://cran.r-project.org/web/packages/readxl/index.html.

Harvard Business Review. 2017. A Refresher on A/B Testing. Cambridge. Last accessed on April 4, 2023 on page https://hbr.org/2017/06/a-refresher-on-ab-testing

Han, J., Yin, H., Boylan, M. 2016. Teacher Motivation: Definition, Research Development and Implications for Teachers. Cogent Education Vol. 3 Issue 1.

Ingarianti, T. M. 2017. Faktor-faktor yang Mempengaruhi Komitmen Karir. Jurnal Ilmiah Psikologi Terapan Universitas Gadjah Mada 5(2): 202-209.

Kemendikbud Ristek RI. 2022. Implementasi Kurikulum Merdeka. Jakarta. Last accessed on December 22, 2022 on page https://kurikulum.gtk.kemdikbud.go.id/.

Kominfo RI. 2020. Hasil Survei PBB, e-Government Indonesia Naik Peringkat. Jakarta. Last accessed on December 22, 2022 on page https://www.kominfo.go.id/content/detail/30024/hasil-survei-pbb-e-government-indonesia-naik-peringkat/0/artikel.

Pedersen, T. L., Wickham, H., Chang, W., Henry, L., Takahashi, K., Wilke, C., Woo, K., Yutani, H., Dunnington, D., Posit, P. B. C. 2021. Computer Software. [Online]. Available: https://cran.r-project.org/web/packages/ggplot2/index.html.

Portman, F. 2021. Computer Software. [Online]. Available: https://cran.r-project.org/web/packages/bayesAB/index.html.

Puza, Borek. 2015. Bayesian Methods for Statistical Analysis. Canberra: The Australian National University eView.

Robert, C. P. 2007. The Bayesian Choice: From Decision-Theoretic Foundations to Computational Implementation Second Edition. Paris: Springer Science & Business Media.

Samaniego, F. J. 2010. A Comparison of the Bayesian and Frequentist Approaches to Estimation. Paris: Springer Science & Business Media LLC.

The Signal. 2022. Users: New vs Returning. San Francisco. Last accessed on April 8, 2023 on page https://mixpanel.com/blog/new-vs-returning-users/.

UN Department of Economic and Social Affair. 2022. UN E-Government Survey 2022 The Future of Digital Government. Jakarta. Last accessed on December 22, 2022 on page https://desapublications.un.org/publications/un-e-government-survey-2022.

Vaughan, D., Wickham, H., Francois, R., Muller, K. 2022. Computer Software. [Online]. Available: https://cran.r-project.org/web/packages/dplyr/index.html.

Wei, William W. S. 2006. Time Series Analysis: Univariate and Multivariate Methods Second Edition. Philadelphia: Pearson Education, Inc.

Westland, J. C. 2022. A Comparative Study of Frequentist vs Bayesian A/B Testing in The Detection of E-commerce Fraud. Journal of Electric Business & Digital Economics Vol. 1 No. 1/2, pp. 3-23.

Wickham, H. 2022. Computer Software. [Online]. Available: https://cran.r-project.org/web/packages/tidyverse/index.html.

Xue, X., Fu, Y., Xia, S., Li, X. 2022. A/B Test Research on the Selection of Push Scheme in Network Precision Marketing. ISCTT 2022; 7th International Conference on Information Science, Computer Technology and Transportation, Xishuangbanna, China, 2022, pp. 1-6.

Yen, H. S., Lim, H. E., Campbell, J. K. 2015. Age and Productivity of Academics: A Case Study of a Public University in Malaysia. Malaysian Journal of Economic Studies 52(1): 97-116.

Additional Files

Published

2023-05-31

How to Cite

Widiarso, B. R., Muthohar, B. A., Tombe, S. R., & Novianti, P. W. (2023). Application of A/B Testing Experimentation on Government Digital Products to Enhance Teachers’ Skills and Capabilities in Indonesia. JPSI (Journal of Public Sector Innovations), 7(2), 67–75. https://doi.org/10.26740/jpsi.v7n2.p67-75

Issue

Section

Articles
Abstract views: 413 , PDF Downloads: 0