Game Analytics Research Articles

Editorial on EAIT 2015, Issue 1

This first issue of Education and Information Technologies for 2015 again illustrates the truly international nature of this journal with articles from the USA, Sweden, the Netherlands, Ireland, India, South Africa, France and Greece. It also illustrates the wide range of topics carried by the journal. In this issue these range from an article on serious game playing in university education to one investigating the attitudes of Greek kindergarten teachers towards computers. Beginning this new volume of EAIT for 2015 is an article by Christian Sebastian Loh and Yanyan Sheng from Southern Illinois University, USA. Titled:Measuring the (dis-)similarity between expert and novice behaviors as serious games analytics, it examines players’ actions and behaviours within an online gaming environment as user-generated data for performance assessment. The research described used string similarity to differentiate likely-experts from a group of unknown performers based on how similar their ingame actions were to those of experts. Their findings indicated that string similarity is viable as an empirical assessment method to differentiate likely-experts from novices and potentially useful as the first performance metric for serious games analytics. The second article: Finding potential problems in the thesis process in higher education: Analysis of e-mails to develop a support system was contributed by Naghmeh Aghaee from Stockholm University, Sweden. The article begins by noting that some students have difficulty in managing to complete their final year Bachelor’s and Master’s level project or thesis and goes on to describe an information and communication platform, called SciPro developed in the Department of Computer and Systems Sciences (DSV) at Stockholm University to support students and supervisors during the thesis process. The reported study explored problems learners faced during their final project courses and found six exhaustive and mutually exclusive categories of problems which it categorised as: thesis initiation, info-mail, technical issues, exemption, supervision and final seminar. Billy R. Brocato from Texas A&M University, and Alessandro Bonanno and Stacy Ulbig from Sam Houston State University, USA next investigate: Student perceptions and instructional evaluations: A multivariate analysis of online and face-to-face classroom settings. Their reported study examined students’ evaluations of faculty Educ Inf Technol (2015) 20:1–3 DOI 10.1007/s10639-015-9380-x

Maximum Similarity Index (MSI): A metric to differentiate the performance of novices vs. multiple-experts in serious games

In learning environments, appropriate objectives are needed to create the conditions for learning and consequently the performance to occur. It follows that appropriate metrics would also be necessary to properly measure what actually constitute performance in situ (within that environment), and to measure if learning has indeed occurred. Serious games environments can be problematic for performance measurement because publishers often posit the game would automatically facilitate learning by their design. Stakeholders, on the other hand, require empirical proofs to quantify performance improvement and calculate Returns of Investment.Serious games environment (an open-ended scenario) with ‘more-than-one correct solutions’ can be difficult for data analysis. In a previous study, we demonstrated the possible use of String Similarity Index to differentiate novices from experts based on how (dis-)similar their performances are within a ‘single-solution’ serious game environment. This study extends the previous study by differentiating a group of novices from the experts based on how (dis)similar their performances are within a ‘multiple-solution’ serious game environment. To facilitate the calculation of performance, we create a new metric for this purpose called, Maximum Similarity Index, to take into consideration the existence of multiple expert solutions. Our findings indicated that Maximum Similarity Index can be a useful metric for serious games analytics when such scenarios present themselves, both for the differentiation of novices from experts, and for the ranking of the player cohort. In a secondary analysis, we compared Maximum Similarity Index to other commonly available game metrics (such as time of completion) and found it to be more appropriate than other game metrics for the measurement of performance in serious games.

Serious Gaming Analytics: What Students´ Log Files Tell Us about Gaming and Learning

In this paper we explore existing log files of the VIBOA environmental policy game. Our aim is to identify relevant player behaviours and performance patterns. The VIBOA game is a 50 hours master level serious game that supports inquiry-based learning: students adopt the role of an environmental consultant in the (fictitious) consultancy agency VIBOA, and have to deal with complex, multi-faceted environmental problems in an academic and methodologically sound way. A sample of 118 master students played the game. We used learning analytics to extract relevant data from the logging and find meaningful patterns and relationships. We observed substantial behavioural variability across students. Correlation analysis suggest a behavioural trade that reflects the rate of “switching” between different game objects or activities. We were able to establish a model that uses switching indicators as predictors for the efficiency of learning. Also we found slight evidence that students who display increased switching behaviours need more time to complete the games. We conclude the paper by critically evaluating our findings, making explicit the limitations of our study and making suggestions for future research that links together learning analytics and serious gaming.

Automated spatio-temporal analysis techniques for game environment

This paper introduces spatio-temporal analysis techniques for games. Game analytics is emerging field in Business Insight (BI) area. The benefits of adopting game analytics technique in commercial game development can help decision making in game design and quality assurance which are not quantified yet. For last 10 years, researchers in game user research area have proposed the frontier of techniques in game analytic field. Among them, spatio-temporal analysis field is most important for understanding of users in game environments intuitively. In this paper, we summarize four key areas of spatio-temporal analysis: visualization techniques, trajectory analysis, in-house telemetry system, and web-based middlewares in detail.

Measuring the (dis-)similarity between expert and novice behaviors as serious games analytics

The behavioral differences between expert and novice performance is a well-studied area in training literature. Advances in technology have made it possible to trace players' actions and behaviors within an online gaming environment as user-generated data for performance assessment. In this study, we introduce the use of string similarity to differentiate likely-experts from a group of unknown performers (mixture of novices and experts) based on how similar their in-game actions are to that of experts. Our findings indicate that string similarity is viable as an empirical assessment method to differentiate likely-experts from novices and potentially useful as the first performance metric for Serious Games Analytics (SEGA).

Game analytics for game user research, part 1: a workshop review and case study.

The emerging field of game user research (GUR) investigates interaction between players and games and the surrounding context of play. Game user researchers have explored methods from, for example, human-computer interaction, psychology, interaction design, media studies, and the social sciences. They've extended and modified these methods for different types of digital games, such as social games, casual games, and serious games. This article focuses on quantitative analytics of in-game behavioral user data and its emergent use by the GUR community. The article outlines open problems emerging from several GUR workshops. In addition, a case study of a current collaboration between researchers and a game company demonstrates game analytics' use and benefits.

Generations of Game Analytics, Achievements and High Scores

This paper poses the question, how has game recording evolved over the generations and how will it affect future generations of players and game developers who have access to the digital past? High scores have a history behind them and as generations of games have moved forward they begin to record much more than just scores. The present player generation can record complete replays of their entire gameplay performances and even play against other recorded player ghosts. As the future generations of gamers and developers take over they will have unprecedented access to the digital history archive as it becomes easier and easier to record and store the past. Deciding what to do with that past will be the next generation’s task as games move into the future.