MET:Interactive Multitouch Displays
Authored by Tobias Blaskovits for ETEC 510 65C (2010)
Multitouch displays are the latest technological evolution of touch-sensitive screens. While single point of contact touch screens are commonplace, multitouch is the latest attempt to improve the Human-Computer Interaction (HCI). Instead of using peripheral devices, such a mouse, keyboard, or stylus, users can interface with their computer with the touch of their fingers, allowing for multiple inputs at the same time. The movement of the fingers across the screen create various gestures, sending commands to the device. Such sensing devices are inherently also able to accommodate multiple users simultaneously, which is especially useful for larger interaction scenarios such as interactive walls and tabletops.
How Interactive Multitouch Displays Work
To allow the use of touch commands that require multiple fingers, numerous designs have been utilized. One such design, FTIR ( frustrated total internal reflection ) has been utilized in small-scale touch screen devices as well as large-scale touch walls. These devices operate by projecting an image through acrylic or glass, and then backlight the image with LED's. When the surface is touched by an object or a finger, the light scatters and the reflection is picked up with sensors or cameras that send the data to software [7]. The type of reflection measured dictates the response to the touch. This process has been refined beyond the "poking" actions you get with a typical touch screen, or the gross gesturing found in video-based interactive interfaces. FTIR is still currently being experimented with and looking forward for advances in efficiency, usability, and intuitiveness. With the addition of a pressure-sensitive coating, touch surfaces can be made pressure-sensitive. Input response varys depending on how firmly the screen is pressed [7].
- Frustrated Total Internal Reflection
- Touch 3.jpg
- Touch 4.jpg
On a smaller scale, many handheld technologies use a panel that carries an electrical charge. In order to achieve this, the construction of the screen includes a layer of capacitive material. Touching the screen with a finger, disrupts the panel’s electrical field. The disruption is registered and sent to the software, which then initiates a response to the gesture [8]. This process is similar to that which is utilized by devices such as iPhones.
- iPhone Capacitive Multitouch Screen Construction
- Iphone-8.jpg
- Iphone-9.jpg
Link to Video on Multitouch Screens
Benefits of Interactive Multitouch Displays
From handhelds to desktops, the dominant paradigm for computing has been one user per device. Interactive tabletops offer the potential to break this restrictive trend. Already, much of the research on interactive tabletops examines how they can support collaborative activities. Multi-touch screens have been utilized as playful learning interfaces for kids, shareable domestic applications and creative work tools. Learning interfaces for kids should meet the children's needs for being playful, creative, and active [3]. Kids are curious about new objects, love to move around, and use their whole bodies. In accordance with constructivist learning theories interfaces for children should support exploring, active engagement, and the usage of form and material [1].
Future interfaces should be designed to meet the needs of these user groups. Good interfaces should not only offer multimedia functions, communication and cooperation mechanisms but also foster creativity and cooperative learning in the circle of families and friends, e.g. within games. Interactive surfaces like walls and tables provide useful platforms for shareable interfaces in domestic contexts. They should be integrated sensitively into the room interior, without having a computer-like appearance, for example as an interactive coffee table. Embedded into a table, household appliances can stay in the background, supporting the group when needed but not enforcing their attention permanently [1].
Being creative is essential for human beings. Traditional computing tools do not meet the needs within creative work processes very well, especially when it comes to group work situations. Furthermore, people love to use expansive surfaces and real material like paper and physical objects, when working on creative tasks. The design of multitouch interfaces concentrates and supports creativity by including real items to allow for natural, free, and unstructured ways of interaction [1].
Interactive Multitouch Displays in the Classroom
With the integration of computers into the classroom environment, is an expectation that computers will dramatically increase interactivity and collaboration among peers. Although this happens to some degree, the issue with such technology is that only one child can control the keyboard and mouse at the same time. A multitouch screen potentially solves this problem because it supports multi-user interaction. Teaching children to physically work together is an affordance of using a multi-touch screen that is not possible with previously developed classroom technologies.
In addition, most technology that we currently use in education is designed to be placed in the hands of the teacher. Even interactive white boards (IWBs), designed to increase interactivity, do not accomplish this goal. Multi-touch screens truly create interactivity and collaboration in the regular face to face classroom environment [2]. Technology such as interactive walls and tabletop surfaces meet the users needs in group interaction situations much better than previous technologies. Multitouch in the classroom offers large surfaces, involves body movements, includes real objects and is smoothly integrated into the learning environment [1]. In learning contexts, many of these affordances can lead to much better tools, supporting kids' activity, fostering creativity, and enabling group communication.
Interactive Multitouch Displays and Designs for Learning
Touch is a compelling input modality for interactive devices. Research looking at different types of interfaces, single touch versus multi-touch, have shown very promising support for the use of multi-touch screens in learning situations. Results showed that touch condition did not affect the frequency or equity of interactions, but did influence the nature of children’s discussion [1]. In the multiple-touch condition, children talked more about the task as opposed to the single touch condition, where they talked more about taking turns [2].
Technological support for collaborative activity in schools has traditionally been limited to the shared use of single computers. However, an emerging generation of shareable interfaces are being promoted as the new technology to support collaborative learning. Shareable interfaces, allow several people in the same place to interact on the same task using their own input device. For example, multi-touch tabletops are horizontal surfaces that allow multiple people to interact simultaneously through touch input. These technologies offer the potential for new ways to support and structure co-located collaborative learning activities. In addition, touch input may be a more appealing and natural means of input as users manipulate objects directly and easily with their fingers. However, there are few studies that directly examine their effect on children’s interactions and we therefore know very little about their influence on behaviour.
Current Multi-Touch Applications
StoryTable
- Enforces a co-operative task structure such that children can simultaneously work on individual parts of the task but are then forced to perform crucial operations together in order to progress [2].
SIDES
- Encourages co-operation as adolescents have to work together to build a path by combining individually owned pieces [2].
OurSpace
- A desk positioning and seating allocation application designed for three concurrent users. A bird’s eye virtual floor plan of the participants’ classroom is placed in the center of the tabletop so that all participants, irrespective of their position, have good access to it. Participants use their fingers to drag icons of students and desks onto the classroom plan. When a student is dragged over an available desk seat, the seat is highlighted and the student is oriented toward that seat position; when dropped, the student icon snaps to that seat [3].
DigiTile or DigiQuilt
- A construction kit for learning about math and art by designing colorful mosaic tiles. In addition to being aesthetically pleasing, these tiles lend themselves to mathematical analysis. The designs embody fraction concepts and are often symmetric. The application provides feedback on the mathematical concepts; for instance, the fraction of the entire tile that is a certain color is displayed on the button for selecting that color. DigiTile is intended to be used by two concurrent users arranged side-by-side. The learners are given increasingly difficult challenges to accomplish, such as creating a design that is half red or creating a design that is horizontally symmetric [3]. It is based on the instructional theory of constructionism, which holds that people learn particularly well when designing personally-meaningfully public artifacts. In constructionism, learners are motivated to construct the artifact, because it is meaningful to themselves and because it can be shared with others [4].
WordCat
- A word categorization game for two users positioned side-by. It is based on a model of interaction, where learners work on the same task individually but must agree on a solution before they can proceed. The challenge is to sort twelve words into the four central bins (three words per bin), so that each of the columns and each of the rows form categories related to the meaning or the shape of the word [3].
Successful use of Interactive Multitouch Displays
The successful use of multitouch screens have been documented for use in business applications, however many people are excited about the possibilities for the classroom. Current research has focused on multi-touch screens for use with various types of visualization software. Most visualization software is run on a PC, as a result the input is traditionally provided by a mouse and keyboard. This means that usually one person has control of the software, while others are left to watch. Research has focused on a more interactive and comfortable way of examining molecular visualizations designed to bring people together in person to collaborate using an interactive digital environment [5]. With their large horizontal surface, multi-touch tables are extremely suited for people to stand or sit around. This makes it possible for users to interact with the table and with the people around it at the same time. The multi-touch surface provides a way for everyone to interact with the system, not just the user controlling the mouse and keyboard.
References
[1] Doring, T., Holleis, P. and Schmidt, A. (2009). Playful, Shareable and Creative - Three Examples for New Directions in User Interface Design. Retrieved from http://shareitproject.org/workshop/workshop_shareable_interfaces_tanja_doering.pdf
[2] Harris, A., Jochen, R., Bonnett, V.,Yuill, N., Fleck, R., Marshall, P. and Rogers, Y. (2009). Around the Table: Are Multiple-Touch Surfaces Better Than Single-Touch for Children’s Collaborative Interactions? In CSCL 2009 Proceedings (pp. 335-344). Retrieved from http://portal.acm.org/citation.cfm?id=1600104&dl=GUIDE&coll=GUIDE&CFID=80913262&CFTOKEN=90234796
[3] Jochen, R. (2009). Towards a Classroom Ecology of Devices: Interfaces for Collaborative Scripts. In SFC Workshop – CSCL 2009, Pre-Workshop Proceedings 8 (pp. 8-12). Retrieved from http://mlab.csd.auth.gr/cscl2009/SFC-files/SFC-2009-Rick.pdf
[4] Jochen, R. and Rogers, Y. (2008). From DigiQuilt to DigiTile: Adapting Educational Technology to a Multi-Touch Table. Retrieved from http://mcs.open.ac.uk/pervasive/pdfs/rickTabletop08.pdf
[5] Logtenberg, J. (2009). Multi-user interaction with molecular visualizations on a multi-touch table (Masters Thesis). Retreived from http://essay.utwente.nl/view/programme/3015.html
[6] Van Veen, M., De Vries, A., De Jong, A. and Isenberg, T. (2009). Assisting Gesture Interaction on Multi-Touch Screens. Retrieved from http://www.cs.rug.nl/~isenberg/personal/papers/Cleveringa_2009_AGI.pdf
[7] Multi-Touch. (n.d.) Wikipedia: The Free Encyclopedia. Retrieved on February 28, 2010, from http://en.wikipedia.org/wiki/Multi-touch
[8] Multi-Touch Systems. (n.d.) Retrieved on February 28, 2010, from http://www.howstuffworks.com/iphone2.htm