Building Infrastructure for Digital Surfaces
- Nicholas Graham, Queen’s University
- Carl Gutwin, University of Saskatchewan
As an emerging interface technology, surfaces do not enjoy the mature development environments of traditional desktop computing. In this theme, we are exploring how to simplify the development of surface-based applications, particularly those involving multiple surfaces, multiple users and multiple surface types. This theme is organized around a novel reference architecture for surface-based systems. The architecture has five layers, of which the middle three represent the core work areas of this theme.
- Basic Services: Includes basic abstractions for low-level surface hardware, such as projector tiling and input processing.
- Middleware: Network support for multi-surface applications, which involves issues such as abstracting distributed systems, allowing programmers to treat MSAa as single surface, and dynamic recruitment and use of available surfaces.
- User Interface Services: User interface toolkits at the surface level, which is built on the results of Theme 1 (Humanizing the Digital Interface: Departure From Desktop Computing). For example, this layer involves multi-user widget sets, abstractions for rotation and orientation, and generalized pointing and selection techniques for surfaces.
More than thirty SurfNet projects are currently investigating infrastructure for surface-based applications. Some of the current research is using application development to elicit requirements for services, and to help with the exploratory design of APIs; other projects are producing toolkits and services that fit into the reference architecture.
Several projects primarily resident in other themes list the development of infrastructural requirements as part of their goals – this approach will ensure that the infrastructure developed in the project matches the needs of application developers.
Research is underway on each of the three main service layers. Concrete tools have been developed at the basic services layer, such as the EquisFTIR toolkit for fast and accurate input handling for optical surfaces, and the Haptic Puck Toolkit for enabling touch sensation on surfaces. At the middleware layer, we are exploring the design space of tools and APIs enabling multi-surface development. Our approach relies on creation of both special-purpose infrastructure (e.g., the iOSRemoteConnector for making remote calls between PCs and iOS Devices), as well as more generic networking techniques such as the .Networking GT toolkit for networked collaborative applications. At the user-interface service layer, we are exploring the next generation of user interface widgets applicable to multi-surface applications. Current projects are as diverse as the use of synthesized audio to improve workspace awareness, the proximity toolkit for detecting peoples’ positions around surfaces, and the libjtouch toolkit for touch-based input.
To provide an overarching framework for this work, we are continuing to develop the reference architecture. Our goal here is to better understand how the many levels and types of infrastructural toolkits and APIs fit together. We are particularly interested in the degree to which toolkits can be platform-independent and interoperative (e.g., the TouchToolkit, while recognizing that in practice we cannot hope to build a global platform-independent toolkit for all aspects of surface development.