publications

2022

  1. Empirical Evaluation of Deep Learning Models for Knowledge Tracing: Of Hyperparameters and Metrics on Performance and Replicability
    Sami Sarsa, Juho Leinonen, and Arto Hellas
    arXiv preprint arXiv:2112.15072 2022

  2. CodeProcess Charts: Visualizing the Process of Writing Code
    Raj Shrestha, Juho Leinonen, Arto Hellas, Petri Ihantola, and John Edwards
    In Australasian Computing Education Conference 2022

  3. Methodological Considerations for Predicting At-risk Students
    Charles Koutcheme, Sami Sarsa, Arto Hellas, Lassi Haaranen, and Juho Leinonen
    In Australasian Computing Education Conference 2022

  4. :trophy:
    Time-on-Task Metrics for Predicting Performance
    Juho Leinonen, Francisco Enrique Vicente Castro, and Arto Hellas
    In Proceedings of the 53rd ACM Technical Symposium on Computer Science Education V. 1 2022
    Best paper award

  5. A Comparison of Immediate and Scheduled Feedback in Introductory Programming Projects
    Juho Leinonen, Paul Denny, and Jacqueline Whalley
    In Proceedings of the 53rd ACM Technical Symposium on Computer Science Education V. 1 2022

  6. Digital Education for All: Multi-University Study of Increasing Competent Student Admissions at Scale
    Nea Pirttinen, Juho Leinonen, Annemari Auvinen, Vesa Lappalainen, Katja Tynkkynen, Henrik Hedberg, Mikko-Jussi Laakso, and Kjell Lemström
    In [email protected]’22: Proceedings of the Ninth ACM Conference on Learning @ Scale 2022

  7. Digitalisaation vaikutus oppimiseen, oppimistilanteisiin ja oppimistuloksiin: DigiVOO-hankkeen väliraportti 2022
    Mari-Pauliina Vainikainen, Sanna Oinas, Satu Koivuhovi, Kukka-Maaria Polso, Juho Leinonen, Faruk Nazeri, Laura Nyman, Cristiana Mergianian, Natalija Gustavson, Esko Lindgren, Mikko Asikainen, Petri Ihantola, and Risto Hotulainen
    2022

  8. Seeking Flow from Fine-Grained Log Data
    Benjamin Ultan Cowley, Arto Hellas, Petri Ihantola, Juho Leinonen, and Michiel Spape
    In 2022 IEEE/ACM 44th International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET) 2022

  9. Pausing While Programming: Insights From Keystroke Analysis
    Raj Shrestha, Juho Leinonen, Albina Zavgorodniaia, Arto Hellas, and John Edwards
    In 2022 IEEE/ACM 44th International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET) 2022

  10. Time-on-Task Metrics for Predicting Performance
    Juho Leinonen, Francisco Enrique Vicente Castro, and Arto Hellas
    ACM Inroads 2022

  11. Planning a Multi-institutional and Multi-national Study of the Effectiveness of Parsons Problems
    Barbara J Ericson, Paul Denny, James Prather, Rodrigo Duran, Arto Hellas, Juho Leinonen, Craig S Miller, Briana Morrison, Janice L Pearce, and Susan H Rodger
    In Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 2 2022

  12. Can Students Review Their Peers? Comparison of Peer and Instructor Reviews
    Nea Pirttinen, and Juho Leinonen
    In Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 1 2022

  13. Who Continues in a Series of Lifelong Learning Courses?
    Sami Sarsa, Arto Hellas, and Juho Leinonen
    In Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 1 2022

  14. :trophy:
    Automatic Generation of Programming Exercises and Code Explanations Using Large Language Models
    Sami Sarsa, Paul Denny, Arto Hellas, and Juho Leinonen
    In Proceedings of the 2022 ACM Conference on International Computing Education Research V. 1 2022
    Best paper award

2021

  1. :trophy:
    Exploring the Effects of Contextualized Problem Descriptions on Problem Solving
    Juho Leinonen, Paul Denny, and Jacqueline Whalley
    In Australasian Computing Education Conference 2021
    Best paper award

  2. Promoting Early Engagement with Programming Assignments Using Scheduled Automated Feedback
    Paul Denny, Jacqueline Whalley, and Juho Leinonen
    In Australasian Computing Education Conference 2021

  3. Exploring Personalization of Gamification in an Introductory Programming Course
    Mitchell Rogers, Wendy Yao, Andrew Luxton-Reilly, Juho Leinonen, Danielle Lottridge, and Paul Denny
    In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education 2021

  4. Exploring the Complexity of Crowdsourced Programming Assignments
    Nea Pirttinen, and Juho Leinonen
    In Seventh SPLICE Workshop at SIGCSE 2021 “CS Education Infrastructure for All III: From Ideas to Practice” 2021

  5. Analyzing Fine-Grained Material Usage Behavior
    Charles Koutcheme, Juho Leinonen, Juha Sorva, and Arto Hellas
    In CS Education Infrastructure for All III: From Ideas to Practice: Seventh SPLICE Project Workshop 2021

  6. Morning or Evening? An Examination of Circadian Rhythms of CS1 Students
    Albina Zavgorodniaia, Raj Shrestha, Juho Leinonen, Arto Hellas, and John Edwards
    In 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET) 2021

  7. Digital Education For All: Better Students Through Open Doors?
    Nea Pirttinen, Juho Leinonen, and Kjell Lemström
    In Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1 2021

  8. Does the Early Bird Catch the Worm? Earliness of Students’ Work and its Relationship with Course Outcomes
    Juho Leinonen, Francisco Enrique Vicente Castro, and Arto Hellas
    In Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1 2021

  9. Fine-Grained Versus Coarse-Grained Data for Estimating Time-on-Task in Learning Programming
    Juho Leinonen, Francisco Enrique Vicente Castro, and Arto Hellas
    In Educational Data Mining 2021 2021

  10. Persistence of Time Management Behavior of Students and Its Relationship with Performance in Software Projects
    Joonas Häkkinen, Petri Ihantola, Matti Luukkainen, Antti Leinonen, and Juho Leinonen
    In Proceedings of the 17th ACM Conference on International Computing Education Research 2021

  11. Visual recipes for slicing and dicing data: teaching data wrangling using subgoal graphics
    Lovisa Sundin, Nourhan Sakr, Juho Leinonen, Sherif Aly, and Quintin Cutts
    In 21st Koli Calling International Conference on Computing Education Research 2021

  12. Facilitating API lookup for novices learning data wrangling using thumbnail graphics
    Lovisa Sundin, Nourhan Sakr, Juho Leinonen, and Quintin Cutts
    Foundations of Data Science 2021

2020

  1. A study of keystroke data in two contexts: Written language and programming language influence predictability of learning outcomes
    John Edwards, Juho Leinonen, and Arto Hellas
    In Proceedings of the 51st ACM Technical Symposium on Computer Science Education 2020

  2. Selection of Code Segments for Exclusion from Code Similarity Detection
    Simon, Oscar Karnalim, Judy Sheard, Ilir Dema, Amey Karkare, Juho Leinonen, Michael Liut, and Renee McCauley
    In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education 2020

  3. Crowdsourcing Content Creation for SQL Practice
    Juho Leinonen, Nea Pirttinen, and Arto Hellas
    In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education 2020

  4. Programming Versus Natural Language: On the Effect of Context on Typing in CS1
    John Edwards, Juho Leinonen, Chetan Birthare, Albina Zavgorodniaia, and Arto Hellas
    In Proceedings of the 2020 ACM Conference on International Computing Education Research 2020

  5. Students’ Preferences Between Traditional and Video Lectures: Profiles and Study Success
    Petri Ihantola, Juho Leinonen, and Matti Rintala
    In Koli Calling’20: Proceedings of the 20th Koli Calling International Conference on Computing Education Research 2020

  6. Choosing code segments to exclude from code similarity detection
    Simon, Oscar Karnalim, Judy Sheard, Ilir Dema, Amey Karkare, Juho Leinonen, Michael Liut, and Renée McCauley
    2020

2019

  1. Exploring the applicability of simple syntax writing practice for learning programming
    Antti Leinonen, Henrik Nygren, Nea Pirttinen, Arto Hellas, and Juho Leinonen
    In Proceedings of the 50th ACM Technical Symposium on Computer Science Education 2019

  2. Does creating programming assignments with tests lead to improved performance in writing unit tests?
    Vilma Kangas, Nea Pirttinen, Henrik Nygren, Juho Leinonen, and Arto Hellas
    In Proceedings of the ACM Conference on Global Computing Education 2019

  3. Non-restricted Access to Model Solutions: A Good Idea?
    Henrik Nygren, Juho Leinonen, and Arto Hellas
    In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education 2019

  4. Pass rates in STEM disciplines including computing
    Simon, Andrew Luxton-Reilly, Vangel Ajanovski, Eric Fouh, Chris Gonsalvez, Juho Leinonen, Jack Parkinson, Matthew Poole, and Neena Thota
    In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education 2019

  5. Admitting Students through an open online course in programming: a multi-year analysis of study success
    Juho Leinonen, Petri Ihantola, Antti Leinonen, Henrik Nygren, Jaakko Kurhila, Matti Luukkainen, and Arto Hellas
    In Proceedings of the 2019 ACM Conference on International Computing Education Research 2019

  6. Integrating CrowdSorcerer: Lessons Learned
    Nea Pirttinen, and Juho Leinonen
    In Proceedings of SPLICE 2019 workshop Computing Science Education Infrastructure From Tools to Data at 15th ACM International Computing Education Research Conference 2019

  7. Experimenting with model solutions as a support mechanism
    Henrik Nygren, Juho Leinonen, Nea Pirttinen, Antti Leinonen, and Arto Hellas
    In Proceedings of the 1st UK & Ireland Computing Education Research Conference 2019

  8. Keystroke data in programming courses
    Juho Leinonen
    2019

  9. Pass rates in introductory programming and in other stem disciplines
    Simon, Andrew Luxton-Reilly, Vangel V Ajanovski, Eric Fouh, Christabel Gonsalvez, Juho Leinonen, Jack Parkinson, Matthew Poole, and Neena Thota
    2019

2018

  1. A study of pair programming enjoyment and attendance using study motivation and strategy metrics
    Onni Aarne, Petrus Peltola, Juho Leinonen, and Arto Hellas
    In Proceedings of the 49th ACM Technical Symposium on Computer Science Education 2018

  2. Supporting self-regulated learning with visualizations in online learning environments
    Kalle Ilves, Juho Leinonen, and Arto Hellas
    In Proceedings of the 49th ACM Technical Symposium on Computer Science Education 2018

  3. Taxonomizing features and methods for identifying at-risk students in computing courses
    Arto Hellas, Petri Ihantola, Andrew Petersen, Vangel V Ajanovski, Mirela Gutica, Timo Hynninen, Antti Knutas, Juho Leinonen, Chris Messom, and Soohyun Nam Liao
    In Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education 2018

  4. Crowdsourcing programming assignments with CrowdSorcerer
    Nea Pirttinen, Vilma Kangas, Irene Nikkarinen, Henrik Nygren, Juho Leinonen, and Arto Hellas
    In Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education 2018

  5. Analysis of Students’ Peer Reviews to Crowdsourced Programming Assignments
    Nea Pirttinen, Vilma Kangas, Henrik Nygren, Juho Leinonen, and Arto Hellas
    In Proceedings of the 18th Koli Calling International Conference on Computing Education Research 2018

  6. Predicting academic performance: a systematic literature review
    Arto Hellas, Petri Ihantola, Andrew Petersen, Vangel V Ajanovski, Mirela Gutica, Timo Hynninen, Antti Knutas, Juho Leinonen, Chris Messom, and Soohyun Nam Liao
    In Proceedings companion of the 23rd annual ACM conference on innovation and technology in computer science education 2018

2017

  1. Privacy versus Information in Keystroke Latency Data
    Juho Leinonen
    2017

  2. :trophy:
    Performance and consistency in learning to program
    Alireza Ahadi, Raymond Lister, Shahil Lal, Juho Leinonen, and Arto Hellas
    In Proceedings of the Nineteenth Australasian Computing Education Conference 2017
    Best student paper award

  3. Preventing keystroke based identification in open data sets
    Juho Leinonen, Petri Ihantola, and Arto Hellas
    In Proceedings of the Fourth (2017) ACM Conference on Learning @ Scale 2017

  4. Using and collecting fine-grained usage data to improve online learning materials
    Leo Leppanen, Juho Leinonen, Petri Ihantola, and Arto Hellas
    In 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering Education and Training Track (ICSE-SEET) 2017

  5. Plagiarism in take-home exams: help-seeking, collaboration, and systematic cheating
    Arto Hellas, Juho Leinonen, and Petri Ihantola
    In Proceedings of the 2017 ACM conference on innovation and technology in computer science education 2017

  6. Comparison of time metrics in programming
    Juho Leinonen, Leo Leppänen, Petri Ihantola, and Arto Hellas
    In Proceedings of the 2017 acm conference on international computing education research 2017

  7. Student Modeling Based on Fine-Grained Programming Process Snapshots
    Juho Leinonen
    In Proceedings of the 2017 ACM Conference on International Computing Education Research 2017

  8. Predicting academic success based on learning material usage
    Leo Leppänen, Juho Leinonen, Petri Ihantola, and Arto Hellas
    In Proceedings of the 18th Annual Conference on Information Technology Education 2017

  9. Tracking Students’ Internet Browsing in a Machine Exam
    Henrik Nygren, Juho Leinonen, and Arto Hellas
    In Proceedings of the 6th Computer Science Education Research Conference 2017

  10. Adolescent and Adult Student Attitudes Towards Progress Visualizations
    Onni Aarne, Petrus Peltola, Antti Leinonen, Juho Leinonen, and Arto Hellas
    In International Conference on Informatics in Schools: Situation, Evolution, and Perspectives 2017

  11. :trophy:
    Thought crimes and profanities whilst programming
    Juho Leinonen, and Arto Hellas
    In Proceedings of the 17th Koli Calling International Conference on Computing Education Research 2017
    Best presentation award

  12. Identification based on typing patterns between programming and free text
    Petrus Peltola, Vilma Kangas, Nea Pirttinen, Henrik Nygren, and Juho Leinonen
    In Proceedings of the 17th Koli Calling International Conference on Computing Education Research 2017

2016

  1. Automatic inference of programming performance and experience from typing patterns
    Juho Leinonen, Krista Longi, Arto Klami, and Arto Vihavainen
    In Proceedings of the 47th ACM Technical Symposium on Computing Science Education 2016

  2. Typing patterns and authentication in practical programming exams
    Juho Leinonen, Krista Longi, Arto Klami, Alireza Ahadi, and Arto Vihavainen
    In Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education 2016

  3. Short pauses while studying considered harmful
    Leo Leppänen, Juho Leinonen, and Arto Vihavainen
    In EDULEARN 2016 2016

  4. Pauses and spacing in learning to program
    Leo Leppänen, Juho Leinonen, and Arto Hellas
    In Proceedings of the 16th Koli Calling International Conference on Computing Education Research 2016

2015

  1. :trophy:
    Identification of programmers from typing patterns
    Krista Longi, Juho Leinonen, Henrik Nygren, Joni Salmi, Arto Klami, and Arto Vihavainen
    In Proceedings of the 15th Koli Calling conference on computing education research 2015
    Best paper award