Dal Alert!

Receive alerts from Dalhousie by text message.


MCSc Thesis Defence - Combining Tangible and Gestural Interaction during Collaborative Gameplay with Multiple Displays

Who: Anniruddha Waje

Title: Combining Tangible and Gestural Interaction during Collaborative Gameplay with Multiple Displays

Examining Committee:

Derek Reilly - Faculty of Computer Science (Supervisor)
Kirstie Hawkey - Faculty of Computer Science (Reader)
Bonnie MacKay - Faculty of Computer Science (Reader)


Chair: Raghav Sampangi - Faculty of Computer Science



We explore the use of combined gestural and tangible interfaces during collaborative gameplay by youth in small groups. In our setup, gestural interaction is used to manipulate reference content projected on a wall-mounted display, to support a main task involving tangible interaction on a tabletop display. We designed two games for this setup: Block Rock involves placing tangibles (blocks) in a target configuration shown on the wall display, and Furniture Finder requires toy furniture to be placed in a target configuration, but the specific furniture to place must first be “uncovered” on the wall display. We conducted a within-subjects user study with 92 children between the ages of 8 and 15, and considered the impact of several factors on collaboration: game type (Block Rock vs. Furniture Finder), orientation control (discrete vs. continuous), group size (dyads vs. triads), and age range (concrete vs. formal operational stage of cognitive development). Using structured coding we derived five basic gameplay strategies. The most commonly used strategy was for one player to perform gestural interaction while other players place tangibles, while the frequency of other strategies varied by game type, orientation control and group size.   Furniture Finder encouraged sequential role-based coordination, as one player uncovered furniture on the 3D model while others placed that furniture on the table, while Block Rock permitted players to work independently, often leading to resource contention. Gestural control occurred more often in Block Rock when discrete gestures were used to rotate the 3D model in canonical orientations than when a grab gesture was available to control rotation continuously. Continuous control also led to faster game completion time than discrete control. In general, infringement occurred more often over tangible control than gestural control, but discrete gestures led to more contention over getting the “right” view on the 3D model in Block Rock.  Triads observed the work of their partners more frequently than dyads (the measure is normalized by group size), but this had no impact on either game completion time or score. Interestingly, we find no pronounced differences in collaboration behaviour between the two age groups. We discuss the implications of our findings on the design of collaborative games that employ similar multimodal interface configurations.



Room 430, Goldberg Computer Science Building