Button Blender : Remixing Input To Improve Video Game .

Work-in-Progress: AccessibilityCHI 2013: Changing Perspectives, Paris, FranceButton Blender: Remixing Input toImprove Video Game AccessibilityKa rim Sa idAbstractUMBC Information SystemsOver time, advances in video game system hardwarehave facilitated the evolution of video game mechanicsfrom simple to complex. Game input devices havefollowed this trend, evolving from simple joysticks tomulti-button, sensor-enabled controllers. Unfortunately,the complexity of modern game controllers presentssignificant challenges to some players, includingnovices and gamers with accessibility needs. ButtonBlender reduces such challenges by recording,remixing, and replaying game controller input, allowingnovice players to play like experts.1000 Hilltop Circle197719852005Baltimore, MD USA 21250ksaid1@umbc.eduFigure 1 – Over time, video gamecontrollers have trended toward anincreased number of buttons.Representative examples: Atari2600 (1977): one button; NintendoEntertainment System (1985): fourbuttons; Microsoft Xbox 360(2005): nine buttons.Sha un K. Ka neUMBC Information Systems1000 Hilltop CircleBaltimore, MD USA 21250skane@umbc.eduAuthor KeywordsGaming; Accessibility; Input; RemixingACM Classification User Interfaces - Input Devices and StrategiesIntroductionCopyright is held by the author/owner(s).CHI’13 Extended Abstracts, April 27 – May 2, 2013, Paris, France.ACM 978-1-4503-1952-2/13/04.Video games are an increasingly important part ofpastime and popular culture. Video gaming offersopportunities for relaxation, competition, andcamaraderie building. Moreover, the proliferation ofwidely varied gaming platforms and the burgeoningcasual gamer market have encouraged the growth ofgaming across broad segments of the population.43

Work-in-Progress: AccessibilityCHI 2013: Changing Perspectives, Paris, FranceRepeatRecordButton Blender begins with anexpert play-through of a game or aprerecorded (CSV) script of one.RemixPlay-through scripts are tweakedwith editing of controller inputevents, press timings, etc.ReplayManipulated scripts are playedback to the emulator along withreal-time player controls.Figure 2 – Button Blender’s workflow. Button Blender allows players to record and tweak a scripted play-through of a game and thenplay the script back to an emulator simultaneously with real-time controls.Video game hardware has evolved alongsidemainstream computing hardware, with increases incomputing power, graphical complexity, and even newinput modes. These advances have enabled gamedesigners to create richer, more immersiveexperiences. To accommodate such complex games,game input devices and control schemes have becomeincreasingly complex (Figure 1). Contemporary gamecontrollers demand a high degree of manual dexterityand coordination, which can present a barrier toinexperienced players, as well as to players who lackmanual dexterity, such as children, older adults, andgamers with motor impairments.Towards the mitigation of these challenges, gamercommunities and philanthropic organizations haveemerged (e.g., The AbleGamers Foundation1). Thesecommunities are primarily concerned with educatinggamers with disabilities and game producers, andsupporting the advancement of hardware, software,and research towards game accessibility.In turn, research has focused on building frameworksfor the evaluation of video game usability (e.g., [2, 4,5, 6, 8]). Additionally, towards solving gameaccessibility problems, research has investigated thedevelopment of specialty hardware such as Hernandezet al’s pedal system for children with cerebral palsy [3].Researchers have also productively coopted video gamedevices for alternative usages, including text input witha modified joystick [7]. However, these studies allrecommend extensive modification or augmentation ofvideo game controllers or controller-like hardware, orshift their explicit focus away from games and gamers.The present research investigates the gaps presentedby this previous work. Specifically, we introduce oursoftware framework, Button Blender, as well as anumber of potential use cases, which empower playersto improve video game accessibility and enhance noviceplay using only standard devices and widely availablegaming platforms.Button Blender1http://www.ablegamers.comOur framework, Button Blender (Figure 2), aims toimprove accessibility by recording game controller input,44

Work-in-Progress: AccessibilityCHI 2013: Changing Perspectives, Paris, Franceremixing it to varying effect, and replaying it to a runninggame process. Button Blender can make novice play morelike expert play, enhancing game accessibility for bothnovice players and players with reduced dexterity. Thecurrent prototype operates on game images runninginside a game system emulator2, which enables ButtonBlender to easily capture game input, remix it, and feed itto the game process. Button Blender operates onstandard, unmodified games and using standard gamecontrollers. This section describes the components of theButton Blender system and its primary features.System ComponentsButton Blender uses the Snes9x3 emulator for runninggame software and managing game state. Snes9xenables play of game images designed for Nintendo’sSuper Nintendo Entertainment System (SNES), as wellas fan-made “homebrew” titles designed for that gamesystem. We chose Snes9x due to its large existingsoftware library and its scriptability. We developed theButton Blender prototype on the Windows 7 platform.Figure 3 – Button Blender producing a play-through scriptduring a game of Super Mario World.Blender’s additional features could be activated using thecontroller without affecting the underlying game controls.While we believe that Button Blender’s approach will beeffective for improving the usability and accessibility formany types of games, our present exploration of thesystem has focused on simple, two-dimensional“platformer” games such as Nintendo’s Super Mario World(Figure 3). This game genre features relatively simple andrepetitive interactions that may be more straightforwardto modify than complex 3D games. Furthermore, thisgenre of games was quite popular in the SNES era, andremains popular today, as seen in Team Meat’s SuperMeat Boy4 and Number None, Inc.’s Braid5.Button Blender currently uses a Microsoft Xbox 360controller (Figure 1, right) for input. This controllerfeatures nine buttons, four triggers, and three directionalpads. In contrast, the standard SNES system supports sixface buttons, two triggers, and one directional pad. Wedeliberately chose a hardware controller with morebuttons than the games supported so that Button23While running video game emulation raises potential legalissues, emulators provide an easily instrumented environment,making it possible to capture and manipulate both gamecontroller input and machine state. Future versions of ButtonBlender may use more robust virtual machine platforms,enabling Button Blender’s use on a wide variety of PC m5http://braid-game.com45

Work-in-Progress: AccessibilityCHI 2013: Changing Perspectives, Paris, FranceManipulating Game InputButton Blender’s primary function is to record, remix,and replay game input.a user to press two keys in sequence, rather than holdone key while pressing another. Button Blender canautomatically add sticky-key-like functionality toexisting games by “locking” a button. For example,holding down the “walk forward” key for severalseconds will lock this key, causing the player’scharacter to continue to walk forward even if the buttonis not being held down, and until the button is pressedagain. This feature allows a player who has difficultypressing multiple buttons simultaneously to free herhand and more easily press other buttons.Button Blender captures game inputs via a customPython script that runs in the background duringgameplay. This script captures input event data (e.g.,controller ID, buttons pressed, joystick position) andstores them in a time-stamped, comma-separated logfile. This file is referred to as the play-through file.Recording of the play-through file can be started andstopped via one of the unused controller buttons.Make Any Game a One-Button GameA one-button game is a game that requires the user toonly press one button to play the entire game. Forexample, in Halfbrick Studios’ Jetpack Joyride6 theplayer’s character automatically walks forward at afixed pace, while the player uses a single button tomake the character jump over obstacles. These onebutton games are often considered to be accessible, asthey do not require complex button combinations to bepressed. However, relatively few one-button gamesexist. Button Blender can turn a traditional game into aone-button game by recording an expert’s play-throughfile, filtering out all instances of a certain button press(e.g., pressing the “jump” button), and combiningreplay of the script with the novice’s input. In this way,the game proceeds as if the expert player were movingthe character, but with jump events controlled by thenovice player.These play-through files can be loaded back into ButtonBlender and used to manipulate data in a subsequentgame. Button Blender uses the Microsoft Windows APIto capture, override, and emulate controller input.Thus, Button Blender can capture a button press madeby the player, replace it with an alternative buttonpress from the play-through script, and pass thereplaced button event to the emulator. The followingsection illustrates how this core functionality can beused to provide gamers with various types of playassistance.Usage ScenariosThe Record-Remix-Replay architecture provided byButton Blender can be applied in various ways toenhance video game usability and accessibility. Thissection describes several “scripts” that are supportedby the Button Blender prototype.Sticky ButtonsPeople with motor impairments often have difficultypressing multiple keys simultaneously. “Sticky keys”are a popular keyboard interface feature [1] that allowTag Team Play and Expert AssistanceSometimes, a game player who experiences difficultywith a certain segment of a game may seek help from a6http://halfbrick.com/our-games/jetpack-joyride46