Application of A/B Testing Experimentation on Government Digital Products to Enhance Teachers' Skills and Capabilities in Indonesia


  • Bagoes Rahmat Widiarso Metra-NET TELKOM
  • Bhaskoro Abdillah Muthohar Metra-NET TELKOM
  • Septi Rito Tombe Metra-NET TELKOM
  • Putri Wikie Novianti Metra-NET TELKOM



Microlearning, self-determined learning, Bayesian


The Ministry of Education, Culture and Research and Technology (MoECRT) in Indonesia has developed a digital product named “Microlearning” within the Merdeka Mengajar digital Platform. “Microlearning” enables teachers to enhance their skills and capabilities by doing self-determined learning that is followed-up by the submission of Aksi-Nyata reports. We improved users’ journey flow and provided an alternative of user-interface designs in the platform, in which we hypothesized that these improvements could boost the number of teachers participating in the Aksi-Nyata report submissions - the main expected outcome of the “Microlearning” digital product. We conducted a randomized-control-trials experimentation in the digital platform to scientifically quantify its effect on the product, by including 1.6 million users who logged-in to the platform during a two weeks period. This experimentation (known as an A/B Testing) allowed us to simultaneously compare two groups within a specific period and within a controlled-environment. The bayesian-based experimentation revealed that the new developments increased the number of users who started to learn a topic, as well as increased the Aksi-Nyata report submission in the “Microlearning” product, as compared to the legacy-designs. A follow-up study showed even more convincing results, which shows the number of submitted Aksi-Nyata reports increased by 590% up to two months after implementing the analytics-based initiatives for all logged-in users in the platform.


Bryan, J., Wickham, H., Posit, P. B. C., Kalicinski, M., Valery, K., Leitienne, C., Colbert, B., Hoerl, D., Miller, E. 2022. Computer Software. [Online]. Available:

Harvard Business Review. 2017. A Refresher on A/B Testing. Cambridge. Last accessed on April 4, 2023 on page

Han, J., Yin, H., Boylan, M. 2016. Teacher Motivation: Definition, Research Development and Implications for Teachers. Cogent Education Vol. 3 Issue 1.

Ingarianti, T. M. 2017. Faktor-faktor yang Mempengaruhi Komitmen Karir. Jurnal Ilmiah Psikologi Terapan Universitas Gadjah Mada 5(2): 202-209.

Kemendikbud Ristek RI. 2022. Implementasi Kurikulum Merdeka. Jakarta. Last accessed on December 22, 2022 on page

Kominfo RI. 2020. Hasil Survei PBB, e-Government Indonesia Naik Peringkat. Jakarta. Last accessed on December 22, 2022 on page

Pedersen, T. L., Wickham, H., Chang, W., Henry, L., Takahashi, K., Wilke, C., Woo, K., Yutani, H., Dunnington, D., Posit, P. B. C. 2021. Computer Software. [Online]. Available:

Portman, F. 2021. Computer Software. [Online]. Available:

Puza, Borek. 2015. Bayesian Methods for Statistical Analysis. Canberra: The Australian National University eView.

Robert, C. P. 2007. The Bayesian Choice: From Decision-Theoretic Foundations to Computational Implementation Second Edition. Paris: Springer Science & Business Media.

Samaniego, F. J. 2010. A Comparison of the Bayesian and Frequentist Approaches to Estimation. Paris: Springer Science & Business Media LLC.

The Signal. 2022. Users: New vs Returning. San Francisco. Last accessed on April 8, 2023 on page

UN Department of Economic and Social Affair. 2022. UN E-Government Survey 2022 The Future of Digital Government. Jakarta. Last accessed on December 22, 2022 on page

Vaughan, D., Wickham, H., Francois, R., Muller, K. 2022. Computer Software. [Online]. Available:

Wei, William W. S. 2006. Time Series Analysis: Univariate and Multivariate Methods Second Edition. Philadelphia: Pearson Education, Inc.

Westland, J. C. 2022. A Comparative Study of Frequentist vs Bayesian A/B Testing in The Detection of E-commerce Fraud. Journal of Electric Business & Digital Economics Vol. 1 No. 1/2, pp. 3-23.

Wickham, H. 2022. Computer Software. [Online]. Available:

Xue, X., Fu, Y., Xia, S., Li, X. 2022. A/B Test Research on the Selection of Push Scheme in Network Precision Marketing. ISCTT 2022; 7th International Conference on Information Science, Computer Technology and Transportation, Xishuangbanna, China, 2022, pp. 1-6.

Yen, H. S., Lim, H. E., Campbell, J. K. 2015. Age and Productivity of Academics: A Case Study of a Public University in Malaysia. Malaysian Journal of Economic Studies 52(1): 97-116.

Additional Files





Abstract views: 259 , PDF Downloads: 0