Origami Simulator: A Multi-Touch Experience

Origami Simulator: a Multi-TouchExperienceSamuel Hsiao-Heng ChangAbstractDept of Software EngineeringWe present a 3D origami simulator with multi-touchinteraction. This is a preliminary exploration ofmanipulating 3D models with multi-touch. Following auser centered approach, we analyzed how people makepaper origami models and mapped the common actionsinto two-touch gestures. The user study suggested thatpeople enjoyed the simulator and think the techniquescan be applied to other 3D modeling environments.University of AucklandPrivate Bag 92119Auckland, New Zealandhcha155@ec.auckland.ac.nzLachlan StuartFigure 1. The two touch origami.Dept of Software EngineeringUniversity of AucklandPrivate Bag 92119KeywordsAuckland, New ZealandMulti-touch interaction, 3D manipulation.lstu013@ec.auckland.ac.nzBeryl PlimmerDept of Computer ScienceUniversity of AucklandACM Classification KeywordsH.5.2 User Interfaces: Input devices and strategies,Interaction stylesPrivate Bag 92119IntroductionAuckland, New Zealand3D modelling tools are for experts: this is the commonperception among people. Traditionally the mapping ofthe 3D interaction in a virtual world to the 2Dinteraction on a monitor is done via mouse andkeyboard input. Since these devices limit the degrees offreedom users have [5] 3D tools typically include manymodes, views and menu functions to achieve modelmanipulation. The complex interaction decreases theusability and confuses users.beryl@cs.auckland.ac.nzCopyright is held by the author/owner(s).CHI 2009, April 4 – 9, 2009, Boston, MA, USAACM 978-1-60558-246-7/09/04.

However, 3D modelling is becoming more widely used.It has long been an essential part of architecture andengineering. New games leverage improved graphicscapabilities to include complex 3D spaces. More peopleinteracting with 3D spaces is driving research into morenatural interaction methods. Alternative devices havebeen developed, however many are not intuitive to use.Figure 2. The study of how peoplefold.Figure 3. Target origami shapes.Figure 4. One of the foldinginstructions.Ideally, interacting with a virtual 3D object should beintuitive, mapping closely to the actions for the physicalequivalent, for example shaping clay or folding paper.We investigate the use of a multi-touch screen toachieve this goal. This technology has penetrated thehardware market [2, 4], and it has proved to beeffective for 2D interactions. Its advantages of directinteraction with the display and bimanual input attractsresearch to improve its intuitiveness [1, 7]. AlthoughSteinicke et al. [8] discussed the limitations andpossible solutions to multi-touching 3D data, andHancock et al. [3] explored the application of rotatingand translating 3D objects with multi-touch, nomanipulation method is yet explored.The objective of this project is to use multi-touch toincrease the efficacy of 3D model interactions. Weselected origami as the context because it is relativelysimple to model, yet provides many interactionopportunities. Furthermore because most people havesome paper-folding skills evaluations are not limited toexperts. To focus on the interaction rather than thesoftware modelling, our first step was a simulator withthe functionality for users to build a paper plane.Interaction Requirements DiscoveryIntuitive interaction often comes from mimicking howpeople perform actions in the real world. Multi-touchscreens provide opportunities for bimanual input suchthat natural interaction becomes possible, as peopleoften use two hands to manipulate real-world objects.From origami books we found that there are manystandard folding methods (and some rarer artisticfolds). Hovever, most methods can be categorized as a“fold”, to create a fold line, or a “tuck”, where a part ofthe paper is pressed between two other sides.The tutorials revealed the common techniques, but nothow people perform them. We collected performanceinformation through a three part observation study often people. First participants were asked to fold threesimple origami shapes (Figure 3) that require variousdifferent folds and tucks. Folding instructions weredrawn on paper (Figure 4), without describing howexactly folding should be done. Second, participantswere asked to fold a paper plane, as this is what usersshould be able to create with our simulator. Finally,they were asked to create the most complex origamiobject they knew. This allowed us to collect richinformation on paper folding behaviour. Participantsworked on a tabletop and were videoed.Fold was universally achieved by dragging a corner toanother corner, or dragging one edge to another edge.We refer to these two types of folding as point-to-pointand line-to-line. A line can be an edge or a crease line,while a point can be a corner or an intersection point ofmultiple lines (Figure 5). Most participants did not knowhow to tuck and the rest performed tucks in a widevariety of ways. Many other actions were observed inthe study. The main ones were flipping the paper overand rotating the paper.

Prototype SystemA NextWindow [6] two-point touch screen was used. Itis a 40” wide overlay which is placed on top of a largescreen; its API returns the position and size of bothtouches. To utilize it, we first built a 3D environmentcontaining a paper model that could be manipulated.Building the 3D environmentThe paper model was built upon vertices, which definepolygon meshes. The number of vertices starts withfour, and increases with additional folds, as shown inFigure 5. Lighting and 2D texture adds to the papermetaphor of the model. Points (vertices) and lines(edges or fold-lines) can be selected to simulate pointto-point or line-to-line folds.Figure 5. Paper modelLeft column yellow parts are points,right column green parts are lines.A paper initially consists of fourpoints and four lines, when twolines cross each other, a point willbe dynamically generated at theintersection.Figure 6. Finger-tip and knuckleA physics engine was used to further simulate the realworld behavior. It calculates the movement of eachvertex, which not only made the reaction of the papermore plausible, but also reduced the amount ofprogramming and user work required, as they enabledsome real world manipulation strategies [9].Functionality opportunitiesWe define functionality opportunities as actions towhich functions can be directly assigned, without anabstract gesture. “On paper” (point or line selected)and “not on paper” is a binary choice: combining thiswith two touches gives four functionality opportunities,including on paper with one finger and not-on paperwith two fingers. To achieve more basic touch functionstwo different touch sizes, small and large, are used,which increased the functionality opportunities to eight.Two obstacles were encountered when detecting thesize of a touch. First, due to the hardware limitations,the size of the second touch point is often incorrectlydetected. Second, friction can make it hard to move awhole hand on the glass surface. The first problemcaused us to abandon using size as a variable for twotouch actions, and after many experiments, we foundusing knuckles, as shown in Figure 10, can greatlyreduce the friction problem.Interaction with the simulatorMany techniques exist in origami, but a maximum of sixfunctional opportunities existed. We chose to supportdrag, fold, tuck and look around because they are mostcommon. Fold and tuck are the essential actions oforigami. Drag is the action of holding parts of the paperand pulling them together or apart. Look aroundinvolves examining the paper from a different angle orzooming, which can result in better view which, in turn,can reduce the difficulty of the folding process.We attempted to simulate the real world actions;however, this goal could not be achieved for someactions since we were limited by the 2D screen and thelimited functional opportunities. For example, point-topoint fold in the real world is normally done by pressingone finger on a corner to stabilize the paper, pinchinganother corner of the paper with the thumb and firstfinger of the other hand and moving it to the firstcorner, holding them together with the first finger andcreating a crease line with the second hand. Such foldsinvolve many complex actions and movements in 3Dspace. Although we attempted firstly to directlysimulate these actions, because finger on 2D screen isdifferent from manipulating real paper, they were notintuitive; furthermore, the complex interaction methodsprevented us from implementing them with ourhardware.

Figure 7. Drag to rotate.Figure 8. Folding a triangle.Figure 9. Tucking example.Figure 10. Turning the paperover and zoom in.Taking a user centered approach we first analyzed thefrequency of actions. Drag and look around were themost common ones; they are given the simplestgesture – finger-tip on screen. They are distinguishedby whether the finger is touching on paper (drag) ornot (look around). Certain drags are mapped to twotouches, such as opening the folded paper.Furthermore standard interaction techniques wereadopted: for look around, we use the metaphor ofrotating the camera angle; zoom uses two finger lookaround.Fold is considered to be dragging paper points together,however since the model does not support bending, thefull range of real world actions could not be supported.Our observations suggest that only point-to-point andline-to-line fold occur in simple origami. We simplifiedthe fold action, so it can be done with only one knuckle,by selecting one point/line and dragging the knuckle toanother point/line; the folding takes effect when thetouch is released. Although different from its real worldcounterpart, it is intuitive to learn and the best choicewithin the hardware environment. We specifically builttuck but found that with the combination of 3D modeland physics engine, tucking can be achieved by simplydragging the paper around fold lines.Finger-tipKnuckleOn paperOne TwoDragFold --Off paperOneOrientationFunctional gestureTwoZoom--Table 1. The mapping of direct manipulationsIn summary there are eight direct manipulationpossibilities, however double knuckle was not usedbecause of size detection errors when two touches arepresented. The functions mapping are shown in Table 1.The implemented common functions allow people tointeract with the paper model and build origami. Toexpand the functionality, button and functional gesturesare provided. Functions such as close application, newpaper, reorient are implemented as buttons. Tominimise screen clutter more advanced functions aresupported by gestures. The gestures are triggered bycreating a path on screen; a path can be as simple as amovement towards left, or a combination of severalsimple paths such as left-up-down sequence. Functionssuch as redo/undo and beautify the paper model areimplemented as functional gestures.To evaluate the effect of using size to differentiateactions, we introduced a special button as analternative. By tapping the button users can switchbetween “finger-tip” and “knuckle” interactions, withoutthe need to worry about their finger status. In thestudy described below we called this „button mode‟.Initial user evaluationA user evaluation was conducted with nine studentparticipants who had diverse origami skills and digital3D modeling experience. After a pre-training session,each participant was asked to complete three tasks:fold a basic shape and unfold; fold and unfold the sameshape with button mode; and fold a paper plane.Instructions and real origami products were available tothem for reference.The first task was designed to test if the interactionsystem feels natural. The second task was designed totest if participants prefer to distinguish the touch mode