Analysis, Design, Development, Implementation, and Evaluation of a Serious Game Designed to Inform Users on Environmental Issues

Purpose – This study presented the results of each stage of the analysis, design, development, implementation, and evaluation (ADDIE) of a game designed to inform users on environmental issues. The game was named “Juan’s Kart” that aimed to teach players about waste segregation. Method – The analysis phase involved the conceptualization of a game and the determination of its design dimension. The design phase entailed development of game design guidelines, identification of hardware and software requirements specifications, and actual development of the game. The developed game was uploaded in Google Play. Four hundred respondents evaluated the game. Results – The game received favorable ratings from the respondents. Contributions – Thus, the game was successfully developed. It may be utilized as a training tool and its effectiveness may be investigated. Research Implications – Recommendations to improve the game were also presented.


INTRODUCTION
Games were once developed for the sole purpose of entertainment.It also once had a bad reputation and different studies reported negative impacts on its users (e.g., Allison, Von Wahlde, Schockley, & Gabbard, 2006;Griffiths, Davies, & Chappel, 2004;Kraut et al., 1998;Sun et al., 2008;Wood, Griffiths, & Parke, 2007;De Guzman & Fabian, 2009).On the other hand, other studies (e.g., Cole & Griffiths 2007;Utz, 2000;Yee, 2006;Bringula et al., 2013) reported that games have benefits on users such as opportunity to create strong friendships and emotional relationships, meet new friends, play with real friends and family members, express their feelings, and have a new form of entertainment, leisure, and relaxation.Recently, positive results of games on academic performance (Batson and Feinberg, 2006;Tuzun et al., 2009;Chow, Woodford, & Maes, 2011;Salter et al., 2012), motivation to learn, retention (Chow et al., 2011), creativity (Hutton & Sundar, 2010), adolescent development (Durkin & Barber, 2002) were also documented when gaming was used as an educational tool.Hence, the conflicting viewpoints towards games can be attributed on the purpose or nature of the game itself.
The growing interest of academia, research community, and game industry to integrate game concepts in an educational setting prospered to a new game type.This type of game that engages gamers to play and at the same learn is called serious game (Muratet, Torguet, Jessel, & Viallet, 2011).It is considered as an attractive approach for training and education (Wang & Hu, 2011).There were serious games developed for different courses such language learning and teaching (Sorensen & Meyer, 2007), music (Barate, Bergoni, & Ludovido, 2013), intercultural business communication (Nieto & Carbonell, 2012), computer programming (Muratet et al., 2011), and environment (Wang & Tseng, 2014;Bringula et al., 2014).
Following these threads of investigations, this study presented the results of each phase of the ADDIE (Analysis, Design, Develop, Implement, and Evaluate) design model on the development of a serious game designed to inform users on environmental issues.Specifically, it attempted to report the results of the a) design of a valid, reliable, and good fit serious game guidelines, b) developed a serious game based on the developed guidelines, and c) implementation and evaluation of the developed game.

LITERATURE REVIEW
A game is a system in which players engage in an artificial conflict, defined by rules, which result into quantifiable outcome (Salen & Zimmerman, 2004).It is also a system in which players engage in an abstract challenge, defined by rules, interactivity, and feedback, that results into a quantifiable outcome often eliciting an emotional reaction (Koster, 2005).It means that effective game is embodied with systematic elements that can have a good relationship with the users.Like consumable goods, games are purchased, used, and eventually cast away (Hunicke, LeBlanc, & Zubek, 2004).In order for the game to be successful, it has to capture the attention of gamers.
Different studies were conducted on game design concepts in an attempt to attract possible game users.In the study of Hunicke et al. (2004), they proposed a Mechanism-Dynamic-Aesthetics (MDA) framework in developing computer games.Through the MDA framework, it attempted to bridge the gap between game design and development, game criticism, and technical game research.The researchers further argued that it would strengthen the iterative process of game development.Amory (2007) proposed a different model.Amory proposed the game object model (GOM).According to this model, games should be relevant, explorative, emotive, engaging, challenging, and gender-inclusive.These key elements would materialize if challenges, puzzles, or quests were included in games.The researcher further argued that inclusions of these game design features would facilitate access to explicit knowledge, conversations, and reflection that eventually lead to tacit knowledge construction.
Similarly, Gunter, Kenny, and Vick (2006) suggested that there were three generally accepted game design principles in the context of education.These are the Keller's ARCS Motivational Model, Gagne's Events of Instruction, and Bloom's Taxonomy.Gunter et al. explained that the Keller's ARCS Motivational Model consisted of four factorsattention, relevance, confidence, and satisfaction.On the other hand, the second principle (i.e., Gagne's theory) suggested that the nine events that all instructional materials to follow would include gain the learners' attention, inform the learners of the objectives, stimulate recall of prior learning, present stimulus or lesson, provide learning guidance and instruction, elicit performance, provide feedback, assess performance, and enhance retention and transfer.
Other researchers attempted to identify the criteria of quality games.Martinez, Adi, and Prima (2012) specifically identified that colors are one of the key features of aesthetics and one of the important aspects in designing a popular game.Distinctive colors and color situations were important tools in game development that could stimulate excitement and could make a game more realistic.Meanwhile, Solarski (2003) encouraged that dynamic composition should be the topmost consideration of video game developers.Dynamic composition was further subdivided into character shape, character animations, environmental shapes, and pathways.
Objectives or goals were also identified as one criterion of quality games.According to Schreiber (2011), developers must know the objective of the game.Once the objective of the game was set, many of the other formal elements would be clearer.Richard (2010) defines objective as what the player must achieve in order to win the game.Richard also stated that it was also critically important to know the objective of the game, whereas a game with unclear objective will not be consider as a good game.
Alongside with the game objectives or goals, the storyline of the game is also considered.Schreiber (2011) said that storyline should be considered by game designers since it was the narrative thread of the game.It should also be clearly stated at the beginning to the end of the game (Martinez et al., 2012).Another integral part of a game was reward system.Based on the article of Jakobsson and Sotamaa (2011), reward system is a complex and multicategorical game design component.It was shown that reward systems were closely linked to the motivation of the gamers to continue playing games (Begy & Consalvo, 2011).Mohd and Daud (2013) proposed that there are five crucial features of a game that should be focus.These are game design, controls, social features, assets, and menu screens/navigational features.These five features explain the technical structure of a mobile game in order to make the game successful (e.g., Angry Birds).Petridis et al. (2012) stated that the key elements in developing a serious game were fidelity, consistency and support for tools for creating immersion and flow.There were two types of fidelityvisual and functional.Consistency depicted facial features of humans.Lastly, corresponding features for tools form immersion and flow which includes game scripting tools.Krall and Menzies (2012) highlighted that replayability is one important consideration in developing a game.Replayability is the quantifiable scale to the enjoyment factor of the game.The underlying principles of replayability involved mastery, impact, completion, experience, socialization (i.e., able to make new friends or even enemies), and challenge.Playing for challenge means individuals see game as a challenge in life and overcoming these obstacles will build a euphoria and later it will turn into bragging rights.Each game was genuinely unique which might provide long lasting and memorable experiences.People also play games because mastering a game produce an inherent joy, aside for being competitive on it and the yearning of an individual to be the best on it.Further, gamers were engaged in playing games because they experienced some level of control over circumstances in the game.Lastly, they played games for completion because there certain amount of enjoyment was felt whenever they completed something.Srisuriyasavad and Prompoon (2013) proposed the Game Usability Framework.The framework is based on ISO 9126 were physical and digital prototypes are the components of the game prototype.The study focused on the former.Physical prototype is a non-digital prototype that can be simply constructed with paper or any handicraft materials.Its dimensions are foundation, structure, formal details, and refinement.These dimensions have usability requirements which are derived from Playtest.Playtest is the process to test a game with five criteria namely Functional, Internally Complete, Balanced, Fun, and accessible.Functional criteria intended to test a game prototype in terms of rules and playability is simply done by asking the tester if the game was enjoyable.Internally complete is aimed to test the game prototype details completeness, to find conflicting rules and optimum points, and, to examine if the tester spends suitable time for a game and can progress to the next level or not.Balanced is the actual test of the game while Accessible is the examination of ease of use of the game.Fun examined the enjoyable aspects of the game considering the game's dramatic elements.Dramatic elements consisted of competition, fantasy, social interaction, exploration, self, story, construction/destruction, collection, goal, and stimulation.These are the similarities in the studies of Hanuckie et al. (2004) and Amory (2007).Srisuriyasavad and Prompoon (2013) further argued that Foundation, Structure, Formal Details, and Refinement require the following usability quality requirements, respectively: Fun, Fun and Functional, Functional, Internal Complete, Balanced, and Fun and Accessible.
The study of Hamalainen (2008) implemented an educational game in a vocational education.A game environment was developed, named Mustakarhu, which simulated the work context of a vocational design process.The study also investigated how effective the game environment is in the context of vocational learning where gamers (i.e., students) were asked to design four different customized hotel rooms.Game design was guided by principles such as usage of illustrative environment, authentic work-problems, emphasis on collaboration and coordination between players, and inclusion of individual puzzles which were designed to elicit effort and commitment of all players for successful task completion.The study revealed that the game enrich learning and the pedagogical use of technology.
According to Crawford (2003), competition is where opponents are "constrained from impeding each other and instead devote the entirety of their attention to optimizing their own performance."This occurs in racing games.Two players race to the finish line and each tries to go as quickly as possible but they don't interfere with one another.In this case the meaning of the play is to achieve the best possible accomplishment against the environment, obstacles, and the opponent.Winning is accomplished by being faster, cleverer, or more skilled than the opponents.This statement defines competition which is critical in the playability of the game.Schelle (2008) stated that in game design parlance, the term often used for effective, exciting, and engaging feedback is "juicy."When creating feedback, designers strive for juiciness.Jesse Schelle, a professor at Carnegie Mellon University's Entertainment Technology Center, describes "juicy" simply as "a ripe peach, just a little bit of interaction with it gives you a continuous flow of delicious reward.""Juiciness" of a game is critical in terms of its popularity.Juan's Kart, therefore should have a "juicy" part in the game.Deci (1975) also stated that the reflection of the feedback on oneself increases players' perception of competence, which in turn increases their intrinsic motivation, a term used to describe a task one finds inherently rewarding.This means that humans specifically gamers, value the importance of their game achievements.And as they playing the game the worth of their character increases (as they receive rewards and gain experiences).Fryer ( 2010) also stated that the other studies reported similar increased accomplishment, but only when rewards were tied to inputs rather than outputs.This means that the gamer should also be rewarded in terms of his quality of play (i.e. on how the gamers achieve high scores) rather than the quantity of play (i.e.number of plays).
According to Kapp (2012), player self-efficacy (which refers to an individual's perception about his or her own ability to produce a desired result for a specific task) is another important factor that game designers must consider.Increasing player self-efficacy is important because it has been linked to increased goal commitment, increased strategy creation and use, and a more positive response to negative feedback.Baumeister (1999) stated that there are four factor that designers can address in order to affect a player's self-efficacy.The first is level of expertise on the subject matter.This is another important reason to make sure that there are achievements available for players at all skill levels.Seeing people around you succeed, or vicarious experience, is the second factor that influences self-efficacy.This affect is likely to be particularly powerful if the person being observed appears to be at the same ability level of the observer.Examples of utilizing this in games are leaderboards for online games or the "brags" in system like Onlive.Social persuasion (giving someone a verbal boost) is the third method of influencing self-efficacy.This can be as simple as telling someone "good job" after a performance or the "50 NOTE STREAK!" messages that appear in Guitar Hero.How a person feels is the fourth factor, which includes stress level, emotional condition, and perceived physical state.Kapp and O'Driscoll (2010) stated that the possibility of creating episodic memory is very strong.Many 3D immersive games have the visual and temporal-spatial relations to provide a strong, rich association between what you are doing in the event and your long-term memory.This means that 3D assets can help users have better articulation of the storyline or the lesson within the game than that presented in 2D.This may help Juan's Kart users to develop a better understanding of waste management.Also, Vorderer and Byant (2006) stated that the gamers simultaneously expose the player to modeling, rehearsal, and reinforcement of the social behavior that is involved in the game's theme.Smith and Ragan (1999) stated that research has also shown that if a person can be encouraged to perform an important act that is counter to the person's own attitudes, a change in the person's attitude can occur.
Based on Wang and Sun (2011), rewards can be used to attract attention, build social connections with other players, to establish status, or to collaborate when searching for hidden items.According to them, reward systems make the game more enjoyable.It can also arouse curiosity and sense of fun associated with taking chances (when getting the rewards).
Lastly, according to the Game Flow model of Sweetser and Wyeth (2005), there were eight elements to consider assessing player enjoyment in games.They defined concentration as the ability of the player to concentrate in games.The authors argued that the difficulty of the challenge should be the same to the skill level of players.Control is the feeling of players that they have the control over the game.Clear goals should provide the player clear objectives at suitable times.Immersion is a combination of deep experience and effortless involvement of players in the game.Lastly, social interaction allows gamers to support and create opportunities for communication with other gamers.

SYNTHESIS
The studies presented previously provided guiding principles in game development.These guidelines have to be empirically tested for its validity and reliability.Further, while it is important to report the impact of games on learners, readers will only left guessing how the game was developed.Hence, readers would only be informed on the impact of the game on the users and how it was developed.It can be argued that readers must also learn other aspects of game design such as game conceptualization, design, development, and implementation.
This article would attempt to address this gap.A serious game designed to inform users on environmental issues was developed to achieve this goal.This type of game was chosen due to the increasing concerns on climate change.The game, named Juan's Kart, will serve as technology-relevant information campaign for the youth.The study would also be deemed important since other researchers may replicate or improve this study, or even develop their own game as guided by the ADDIE design model.

Analysis, Design, and Development of "Juan's Kart"
The recent super typhoon that hit the Philippines was a wake-up call to take actions towards climate change.Simple ways like proper waste segregation may have significant impact on taking care of our environment.In order to make the youth socially aware of this situation, a serious game dedicated to informing gamers about environmental issues was developed.The focus of the game is to teach elementary students (Grade 1) proper waste segregation.
With that concept and motivation, the design criteria in developing the game were determined.The analysis phase began with the review of related studies on game design guidelines.Based from these studies, items that were deemed relevant to serious games were chosen.An initial 50-item instrument was constructed as shown in Table 1.The instrument was distributed to four cyber cafés in Manila that offer gaming services.Cyber cafés were chosen as the research locale of the study since these were public venues that offer gaming services.All respondents who identified themselves as gamers; hence, they qualify to answer the initial instrument.
It was suggested that the number of cases (or respondents) be at least five times the number of items in the questionnaire in order to employ a valid factor analysis on the data (Dancey & Reidy, 2002;Hair, Black, Babin, & Anderson, 2010).Three hundred forms were distributed and only 269 forms were retrieved.Exploratory factor analysis (EFA) was used to determine the dimensions of the questionnaire while Cronbach's alpha analysis was employed to determine the reliability of the criteria.A valid and reliable item should have at least a factor loading of 0.50 and a Cronbach's alpha of at least 0.70, respectively.The results of the EFA reduced the number of items to 32.The 32 items were then again subjected to discriminant factor analysis.This statistical tool determined if the model was of good fit, that is, if the items do not overlap and uniquely contributes to its assigned criteria.In a statistical sense, a model was of good fit if it satisfied the following: Comparative-Fit Index >= 0.90, Adjusted goodness-of-fit index (AGFI) >= 0.90, and Root mean square error of approximation (RMSEA) < 0.07 (Hair et al., 2010;Feldt, 2013).The final instrument is shown in Table 2.The detailed discussion of the final construction of the instrument can be found in Bringula et al. (2014).
With guidelines on hand, the design and development of the game commence.The developers attempted to put all the guidelines while developing Juan's Kart.First, the game concept was enhanced.Figure 1 shows the game concept.Afterwards, game design and mechanics were formulated.These were shown in Figure 2.

Storyline of the Game
Juan is an environment-conscious Filipino teen-ager.He collects and segregates waste correctly.He discovered that correct waste segregation could earn him money.He is not only making waste useful and profitable; he also helps the environment, as well.He can keep on collecting trash as long as he does not collide with the posts and boulders.To gather waste efficiently, he uses a cart.Juan can magnetize the cart and waste would automatically put into to his cart while passing through them.He can also put a shield to his cart to protect it from boulders and posts.

Game Mechanics and Rewards System
Juan's Kart is a 3D endless-running arcade game.The character of the game is named Juan.Juan will collect as many recyclable bottles as he can and he tries to segregate different kinds on waste.The game starts with the main menu with three options: Play Game Menu, Shop Menu, and Exit Menu.The play game menu is where you can play the game.By clicking the Play button you will enter the game itself.Juan will collect as many recyclable bottles as he can and he will segregate different kinds of waste.The bottles and sorted waste will be converted into cash, specifically Php 0.25 per bottle or waste.The money that the player accumulated will be recorded and can be used in upgrading power-ups.There are three (3) different power-ups in the game: the Booster, in which Juan will have a burst of speed for 10 seconds.He will also have the ability to run over the obstacles without getting killed.The Bottle Magnet will give him the ability to magnetize nearby bottles.Lastly, the Times 2 (x2) multiplier will multiply the number of bottles he collected by 2.
The power-ups were spawned randomly, in which each of them have 10 seconds duration.The power-up upgrades can be found on the second menuthe Shop.There are three (3) option upgrades that can be found at the Shopnamely, Booster, for Magnet, and Times 2 (x2) Multiplier.The player can upgrade each power-up up to four times.Booster adds 2 seconds in time duration per game level.The maximum duration that can be upgraded is up to 18 seconds.On the other hand, the upgrade for Bottle Magnet increases the time duration of the power-up for 5 seconds in each level, and the maximum time upgrade duration is up to 30 seconds.Finally, the Times 2 (x2) Multiplier adds +1 to the multiplier.The maximum upgrade for the multiplier is 5 times.
There is a mini-game in Juan's Kart where-in the user will try to sort or segregate the non-biodegradable waste to biodegradable waste by means of putting the respected types of garbage into to their assigned trash bin.The user has only 30 seconds to accomplish this task.Once the user put the waste into improper trash bin, the time left will be deducted by 2.5 seconds, which serves as a penalty.The different waste will prompt its object name (i.e.banana peel, orange peel, carrot peel, light bulb, plastic bag, plastic spoon and fork, tire, rubber, and Styrofoam) aside from the garbage visual representation in order for the user to know what the object is.There are obstacles in the game (e.g., rolling boulders, trains, cars, water tanks, barricades, and railroad crossing post) that makes the game more challenging.

Figure 2. Game Mechanics and Rewards System
The concept of the game was put into realization through the use of different software development tools (SDT).The first SDT used was Unity 3D (free version).It is a game engine that allows game developers to build games for personal computers and mobile devices.It has built-in functions and optimization tools that allow development of 2D or 3D games.From this engine, games can be developed using any of these programming languages (e.g., C#, Javascript/UnityScript, and Boo programming languages).Javascript and UnityScript were the programming languages used in the game development.3D Studio Max was utilized for creating assets and texturing workflow while Adobe Photoshop was used for modifying in-game art textures and assets.

Implementation and Evaluation
The developed game was designed to be played on any android devices with android operating systems of at least Android 3.0.The game could be downloaded from Google Play (See Figure 3.).Figure 3 shows the main game stage of Juan's Kart  a 3D, endless running mobile game.The game stage has two elements.The first element is the game view while the second element is the game display toolbar.The former is the game itself while the latter is a toolbar that prompts the number of bottles collected, the total distance run, and the pause button.

Figure 3. Juan's Kart in Google Play
Figure 4 shows the overall game play.In Figure 4-a, it depicts the bottle collection of Juan.Only bottles that he had passed through would be collected.These bottles were converted to money.If he passed through three piles of garbage, this means that Juan has to segregate first before proceeding collecting bottles.This is the part where gamers learn waste segregation.Gamers would select the garbage to be discarded as shown in Figure 4-c.Gamers were given 30 seconds to segregate the trash.Once the gamer selected an item (i.e., garbage), the game would inform the gamer what garbage was selected.This would clarify what was the item.He could then discard it either as biodegradable or non-biodegradable item.Labeled and color-coded garbage bins were provided.Once an item was discarded correctly, Juan would receive additional bonus bottles.On the hand, as shown in Figure 4-d, an error message "You discarded it wrong" would be flashed on the screen and a 2.5-second penalty would be deducted.
Gamers were forced to collide on the pile of trash.This also served as the bonus round.The pile of trash was automatically presented as the game starts.In this manner, gamer was force to learn the proper way of segregating garbage.The succeeding waste segregations would appear randomly.Waste segregation was called also mini-game.The second power-up is Bottler Magnet.It allows Juan to "magnetize" bottles while he is passing by The duration of the Bottle Magnet is also 10 seconds and can also be increased through upgrades.Five seconds are added to the duration of the Bottle Magnet for every upgrade.There are 4 level upgrades which are equivalent to 30 seconds increase in time duration.The x2 Multiplier was the last power-up.As the name suggests, it multiplies by 2 all bottles collected.In effect, it multiplies the money that Juan can collect.This power-up has also 10 seconds duration.It can also be upgraded up to the fifth multiplier.
The menu in Figure 7 shows the money collected and money accumulated at the end of the game.This would be prompted after Juan collided on an obstacle.The number of bottles accumulated and the converted value of the bottles are shown in this menu.It also shows the length of distance travel by the gamer as well as an icon if it is the new record high score or not.There is also a Rate button that will directly prompt in the online survey form.The money accumulated is shown in Figure 7-b.Bottles that were converted into money (Philippine Peso) is shown here.It gives the gamer the idea of how much he had accumulated and the price of power-up upgrades.A Rate button, as shown in Figure 7-a, was utilized by the developers to the demographics (e.g., age, gender, civil status, highest educational attainment, monthly family income, and length of years playing games) of the gamers and their perceptions towards the game.It would show-up once the player is already dead in the game.The player has the freedom whether he will rate the game or not.Whenever the user clicked the Rate button, it will prompt the linked of the online survey which is from Google Docs.The survey questionnaire has 19 questions, which was based from Table 2, which could be answered by Strongly Disagree (1) to Strongly Agree (5).The survey aimed to determine if the four elements of the serious game were able to capture the game design considerations such as aesthetics, game objective, reward system, and storyline.One-way Chi-square was utilized to determine if the responses significantly differ from one another.
There were 400 gamers that answered the survey form.Table 3 shows that the average age of the gamers was 19 years old, most were male (f = 256, 64%), college students (f = 350, 88%), belonged to a family with a monthly income of at least Php 28,000 (f = 266, 67%), and been playing games for at least 3.5 years (f = 293, 73%).Aesthetics were measured in terms of the appeal of game objects, proper game graphics, appropriate game objects, non-presence of broken graphics, pleasant colors, and overall appeal.About a third of the respondents agreed (f = 140) that the developers used visually appealing objects for the game.It was also shown that respondents had a strong agreement (f = 225) that the objects of the games were visually appealing and that the objects of the game were appropriate (f = 188).It was also evident that the graphics ("Agree", f = 201) of the game and color selection ("Agree", f = 199) were all fine.Overall, respondents "agreed" (f = 202) that the game had an overall pleasing design.
In terms of Game Objectives, most of the respondents agreed that the goal of the game was clear (f = 190) and complex (f = 186).There was a strong agreement that the goal of the game was unique (f = 196) and interesting (f = 183).They further perceived that the reward system of the game as useful (f = 185), satisfying (f = 183), and unique (f = 187).There was a strong agreement that the game provided interesting rewards (f = 186).Lastly, three out of five items of Storyline was rated "agree".These were the entertaining (f = 188), uniqueness (f = 189), and fascinating (f = 191) components of Storyline.On the other hand, respondents rated "strongly agree" on the different level of difficulties (f = 185) and exciting feature the game.

Figure 6 .
Game power-ups There were three power-ups in the game as shown in Figure6.Booster was the first power-up.It makes Juan and his kart faster and can pass through all the barriers for 10 seconds.The time duration of Booster will increase through upgrades.One level upgrade adds two seconds to the Booster.It has 4 level upgrades in which the maximum booster time duration is 18 seconds.
Figure 7. (a) End Game and (b) Money accumulated