Leveraging Techniques of Epistemic Network Analysis to Discover Behaviors of Student Learning Reflections in Online Learning Environments
Received: 16 March 2024 | Revised: 3 April 2024 | Accepted: 6 April 2024 | Online: 1 June 2024
Corresponding author: Sahar Alqahtani
Abstract
In the domain of learning analytics, reflective writing has introduced trends to enhance the learning and teaching experience. Epistemic Network Analysis (ENA), is a recent development in the techniques of learning analytics regarding handling huge amounts of text and visualizing learners’ interactions in the form of network graphs. In this context, 43 students participated in 10 tasks over a 16-week semester on a blended course. The current article aims to explore their reflective behaviors through this new learning methodology and establish via the ENA technique whether there is any relationship between such behaviors and course performance. The findings show the effectiveness of ENA in investigating students’ overall learning reflection patterns and revealing the frequencies of each reflection type for both high- and low-performing students. The group of high performers demonstrated a stronger connection with positive feelings regarding the learning experience, whereas the low performers exhibited a negative attitude toward the learning process. The obtained results provide insights into students' impressions of specific teaching or learning methods. Linking the reflection behavior to the level of student performance enables teachers to improve course design and provide appropriate interventions, which may be reflected in enhanced student performance.
Keywords:
Epistemic Network Analysis (ENA), model graph-based analysis, learning reflection, reflective practice, reflective writingDownloads
References
D. A. Schon, The Reflective Practitioner: How Professionals Think In Action, 1st edition. New York, NY, USA: Basic Books, 1983.
D. Boud, R. Keogh, and D. Walker, "Promoting reflection in learning a model," in Boundaries of Adult Learning, Routledge, 2013, pp. 32–56.
K. R. Murphy, "The effect of reflective practice on high school science students’ critical and reflective thinking," Ph.D. dissertation, Western Connecticut State University, Danbury, CT, USA, 2014.
R. G. Bringle, P. H. Clayton, and J. A. Hatcher, "Research on Service Learning: An Introduction," in Research on Service Learning, 1st Edition., London, UK: Routledge, 2012, pp. 3–25.
J. Eyler and D. E. Giles, Where’s the Learning in Service-Learning? Jossey-Bass, Inc, 1999.
K. D. Tanner, "Promoting Student Metacognition," CBE—Life Sciences Education, vol. 11, no. 2, pp. 113–120, Jun. 2012.
T. A. Mesbah, R. A. R. Abed, A. S. Al-Sagheer, and M. S. Ghaly, "Relationship between Metacognitive Awareness and Reflective Learning of Medical Students at the Faculty of Medicine, Suez Canal University," Journal of Education, Society and Behavioural Science, vol. 32, no. 4, 2019, Art. no. JESBS.53452.
B. Eilam and S. Reiter, "Long-Term Self-Regulation of Biology Learning Using Standard Junior High School Science Curriculum," Science Education, vol. 98, no. 4, pp. 705–737, 2014.
G. Schraw and R. S. Dennison, "Assessing Metacognitive Awareness," Contemporary Educational Psychology, vol. 19, no. 4, pp. 460–475, Oct. 1994.
C. Lang, G. Siemens, A. F. Wise, D. Gašević, and A. Merceron, Eds., Handbook of Learning Analytics, 2nd ed. Society for Learning Analytics and Research, 2022.
A. B. Rashid, R. R. R. Ikram, Y. Thamilarasan, L. Salahuddin, N. F. A. Yusof, and Z. B. Rashid, "A Student Learning Style Auto-Detection Model in a Learning Management System," Engineering, Technology & Applied Science Research, vol. 13, no. 3, pp. 11000–11005, Jun. 2023.
A. B. Altamimi, "Big Data in Education: Students at Risk as a Case Study," Engineering, Technology & Applied Science Research, vol. 13, no. 5, pp. 11705–11714, Oct. 2023.
M. Alsuwaiket, A. H. Blasi, and R. A. Al-Msie’deen, "Formulating Module Assessment for Improved Academic Performance Predictability in Higher Education," Engineering, Technology & Applied Science Research, vol. 9, no. 3, pp. 4287–4291, Jun. 2019.
D. W. Shaffer, W. Collier, and A. R. Ruis, "A Tutorial on Epistemic Network Analysis: Analyzing the Structure of Connections in Cognitive, Social, and Interaction Data," Journal of Learning Analytics, vol. 3, no. 3, pp. 9–45, Dec. 2016.
D. W. Shaffer et al., "Epistemic Network Analysis: A Prototype for 21st-Century Assessment of Learning," International Journal of Learning and Media, vol. 1, no. 2, pp. 33–53, May 2009.
D. W. Shaffer, Quantitative Ethnography. Madison, WI, USA: Cathcart Press, 2017.
D. W. Shaffer, "Epistemic frames for epistemic games," Computers & Education, vol. 46, no. 3, pp. 223–234, Apr. 2006.
D. W. Shaffer and A. R. Ruis, "Chapter 15 - Epistemic Network Analysis: A Worked Example of Theory-Based Learning Analytics," in Handbook of Learning Analytics, 2nd ed., C. Lang, G. Siemens, A. F. Wise, D. Gašević, and A. Merceron, Eds. Society for Learning Analytics and Research, 2022.
D. W. Shaffer, "Formatting Data For Epistemic Network Analysis," GAPS, Technical Report 2014-1, 2014.
V. Rolim, R. Ferreira, R. D. Lins, and D. Gasevic, "A network-based analytic approach to uncovering the relationship between social and cognitive presences in communities of inquiry," The Internet and Higher Education, vol. 42, pp. 53–65, Jul. 2019.
R. Ferreira, V. Kovanovic, D. Gasevic, and V. Rolim, "Towards Combined Network and Text Analytics of Student Discourse in Online Discussions," in International Conference on Artificial Intelligence in Education, London, United Kingdom, Jun. 2018, pp. 111–126.
V. Rolim, R. Ferreira Leite de Mello, V. Kovanovic, and D. Gasevic, "Analysing Social Presence in Online Discussions Through Network and Text Analytics," in 19th International Conference on Advanced Learning Technologies, Maceio, Brazil, Jul. 2019, pp. 163–167.
Z. Cai, B. Eagan, and N. M. Dowell, "Epistemic Network Analysis and Topic Modeling for Chat Data from Collaborative Learning Environment," in 10th International Conference on Educational Data Mining, Wuhan, China, Jun. 2017, pp. 104–111.
J. Oshima, R. Oshima, and S. Saruwatari, "Analysis of students’ ideas and conceptual artifacts in knowledge-building discourse," British Journal of Educational Technology, vol. 51, no. 4, pp. 1308–1321, 2020.
M. H. Khan, "Epistemic Network Analysis in Problem-Based Learning," M.S. thesis, University of Eastern Finland, Joensuu, Finland, 2020.
V. Nachtigall and H. Sung, "Students’ Collaboration Patterns in a Productive Failure Setting: An Epistemic Network Analysis of Contrasting Cases," in International Conference on Quantitative Ethnography, Madison, WI, USA, Oct. 2019, pp. 165–176.
P. Levine, B. Eagan, and D. W. Shaffer, "Deliberation as an Epistemic Network: A Method for Analyzing Discussion," in International Conference on Quantitative Ethnography, Copenhagen, Denmark, Oct. 2022, pp. 17–32.
D. M. Bressler, A. M. Bodzin, B. Eagan, and S. Tabatabai, "Using Epistemic Network Analysis to Examine Discourse and Scientific Practice During a Collaborative Game," Journal of Science Education and Technology, vol. 28, no. 5, pp. 553–566, Oct. 2019.
A. Csanadi, B. Eagan, I. Kollar, D. W. Shaffer, and F. Fischer, "When coding-and-counting is not enough: using epistemic network analysis (ENA) to analyze verbal data in CSCL research," International Journal of Computer-Supported Collaborative Learning, vol. 13, no. 4, pp. 419–438, Dec. 2018.
A. L. Siebert-Evenstone, G. A. Irgens, W. Collier, Z. Swiecki, A. R. Ruis, and D. W. Shaffer, "In Search of Conversational Grain Size: Modeling Semantic Structure using Moving Stanza Windows," Journal of Learning Analytics, vol. 4, no. 3, pp. 123–139, Dec. 2017.
S. Karumbaiah, R. S. Baker, A. Barany, and V. Shute, "Using Epistemic Networks with Automated Codes to Understand Why Players Quit Levels in a Learning Game," in International Conference on Quantitative Ethnography, Madison, WI, USA, Oct. 2019, pp. 106–116.
N. Mohammadhassan and A. Mitrovic, "Discovering Differences in Learning Behaviours During Active Video Watching Using Epistemic Network Analysis," in International Conference on Quantitative Ethnography, Copenhagen, Denmark, Oct. 2022, pp. 362–377.
S. S. Fougt, A. Siebert-Evenstone, B. Eagan, S. Tabatabai, and M. Misfeldt, "Epistemic network analysis of students’ longer written assignments as formative/summative evaluation," in 8th International Conference on Learning Analytics and Knowledge, Sydney, NSW, Australia, Mar. 2018, pp. 126–130.
H. Wang, C. Wang, and F. Wu, "How Multimedia Influence Group Interaction in STEM Education An Epistemic Network Analysis for Online Synchronous Collaborative Learning," in International Symposium on Educational Technology, Bangkok, Thailand, Aug. 2020, pp. 303–306.
S. Bruckner, J. Schneider, O. Zlatkin-Troitschanskaia, and H. Drachsler, "Epistemic Network Analyses of Economics Students’ Graph Understanding: An Eye-Tracking Study," Sensors, vol. 20, no. 23, Jan. 2020, Art. no. 6908.
J. Vandenberg et al., "Prompting collaborative and exploratory discourse: An epistemic network analysis study," International Journal of Computer-Supported Collaborative Learning, vol. 16, no. 3, pp. 339–366, Sep. 2021.
S. Iqbal et al., "Uncovering Associations Between Cognitive Presence and Speech Acts: A Network-Based Approach," in 12th International Learning Analytics and Knowledge Conference, Mar. 2022, pp. 315–325.
E. Farrow, J. Moore, and D. Gasevic, "A network analytic approach to integrating multiple quality measures for asynchronous online discussions," in 11th International Learning Analytics and Knowledge Conference, Irvine, CA, USA, Apr. 2021, pp. 248–258.
M. T. H. Chi and R. Wylie, "The ICAP Framework: Linking Cognitive Engagement to Active Learning Outcomes," Educational Psychologist, vol. 49, no. 4, pp. 219–243, Oct. 2014.
L. Wu, Q. Liu, G. Mao, and S. Zhang, "Using epistemic network analysis and self-reported reflections to explore students’ metacognition differences in collaborative learning," Learning and Individual Differences, vol. 82, Aug. 2020, Art. no. 101913.
Y. Yi, X. Lu, and J. Leng, "Exploring the Development of Reflection Among Pre-service Teachers in Online Collaborative Writing: An Epistemic Network Analysis," in International Conference on Quantitative Ethnography, Madison, WI, USA, Oct. 2019, pp. 257–266.
T. D. Ullmann, "Reflective writing analytics: empirically determined keywords of written reflection," in Seventh International Learning Analytics & Knowledge Conference, Vancouver, BC, Canada, Mar. 2017, pp. 163–167.
Y. Cui, A. F. Wise, and K. L. Allen, "Developing reflection analytics for health professions education: A multi-dimensional framework to align critical concepts with data features," Computers in Human Behavior, vol. 100, pp. 305–324, Nov. 2019.
W. Luo and D. Litman, "Determining the Quality of a Student Reflective Response," in Twenty-Ninth International Florida Artificial Intelligence Research Society Conference, Key Largo, FL, USA, Dec. 2016, pp. 226–231.
A. Gibson, K. Kitto, and P. Bruza, "Towards the Discovery of Learner Metacognition From Reflective Writing," Journal of Learning Analytics, vol. 3, no. 2, pp. 22–36, Sep. 2016.
C. Tsingos-Lucas, S. Bosnic-Anticevich, C. R. Schneider, and L. Smith, "Using Reflective Writing as a Predictor of Academic Success in Different Assessment Formats," American Journal of Pharmaceutical Education, vol. 81, no. 1, Feb. 2017, Art. no. 8.
W. Suraworachet, Q. Zhou, and M. Cukurova, "Impact of combining human and analytics feedback on students’ engagement with, and performance in, reflective writing tasks," International Journal of Educational Technology in Higher Education, vol. 20, no. 1, Jan. 2023, Art. no. 1.
O. Ezezika and N. Johnston, "Development and Implementation of a Reflective Writing Assignment for Undergraduate Students in a Large Public Health Biology Course," Pedagogy in Health Promotion, vol. 9, no. 2, pp. 101–115, Jun. 2023.
H. Alrashidi, N. Almujally, M. Kadhum, T. Daniel Ullmann, and M. Joy, "Evaluating an Automated Analysis Using Machine Learning and Natural Language Processing Approaches to Classify Computer Science Students’ Reflective Writing," in Pervasive Computing and Social Networking, G. Ranganathan, R. Bestak, and X. Fernando, Eds. New York, NY, USA: Springer, 2023, pp. 463–477.
K. Q. Fisher, L. Hirshfield, A. Siebert-Evenstone, G. A. Irgens, and M. Koretsky, "Network Analysis of Interactions between Students and an Instructor during Design Meetings," in ASEE Annual Conference & Exposition, New Orleans, LA, USA, Jun. 2016, pp. 1–13.
S. Zhang, Q. Liu, and Z. Cai, "Exploring primary school teachers’ technological pedagogical content knowledge (TPACK) in online collaborative discourse: An epistemic network analysis," British Journal of Educational Technology, vol. 50, no. 6, pp. 3437–3455, 2019.
L. Paquette, T. Grant, Y. Zhang, G. Biswas, and R. Baker, "Using Epistemic Networks to Analyze Self-regulated Learning in an Open-Ended Problem-Solving Environment," in International Conference on Quantitative Ethnography, Nov. 2021, pp. 185–201.
N. Melzner, M. Greisel, M. Dresel, and I. Kollar, "Using Process Mining (PM) and Epistemic Network Analysis (ENA) for Comparing Processes of Collaborative Problem Regulation," in International Conference on Quantitative Ethnography, Madison, WI, USA, Oct. 2019, pp. 154–164.
J. Saint, D. Gasevic, W. Matcha, N. A. Uzir, and A. Pardo, "Combining analytic methods to unlock sequential and temporal patterns of self-regulated learning," in 10th International Conference on Learning Analytics and Knowledge, New York, NY, USA, Mar. 2020, pp. 402–411.
Y. Fan, W. Matcha, N. A. Uzir, Q. Wang, and D. Gasevic, "Learning Analytics to Reveal Links Between Learning Design and Self-Regulated Learning," International Journal of Artificial Intelligence in Education, vol. 31, no. 4, pp. 980–1021, Dec. 2021.
S. Zhang, J. Chen, Y. Wen, H. Chen, Q. Gao, and Q. Wang, "Capturing regulatory patterns in online collaborative learning: A network analytic approach," International Journal of Computer-Supported Collaborative Learning, vol. 16, no. 1, pp. 37–66, Mar. 2021.
M. Ferreira, R. F. Mello, R. D. Lins, and D. Gasevic, "Analytics of Emerging and Scripted Roles in Online Discussions: An Epistemic Network Analysis Approach," in International Conference on Artificial Intelligence in Education, Utrecht, The Netherlands, Jun. 2021, pp. 156–161.
M. A. D. Ferreira, R. Ferreira Mello, V. Kovanovic, A. Nascimento, R. Lins, and D. Gasevic, "NASC: Network analytics to uncover socio-cognitive discourse of student roles," in 12th International Learning Analytics and Knowledge Conference, Mar. 2022, pp. 415–425.
W. Matcha et al., "Detection of Learning Strategies: A Comparison of Process, Sequence and Network Analytic Approaches," in European Conference on Technology Enhanced Learning, Delft, The Netherlands, Sep. 2019, pp. 525–540.
N. A. Uzir, D. Gasevic, J. Jovanovic, W. Matcha, L.-A. Lim, and A. Fudge, "Analytics of time management and learning strategies for effective online learning in blended environments," in 10th International Conference on Learning Analytics and Knowledge, Frankfurt, Germany, Mar. 2020, pp. 392–401.
H. YuekMing and L. A. Manaf, "Assessing Learning Outcomes through Students’ Reflective Thinking," Procedia - Social and Behavioral Sciences, vol. 152, pp. 973–977, Oct. 2014.
Downloads
How to Cite
License
Copyright (c) 2024 Sahar Alqahtani
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
