Brigade: Multi-surface Technology for Retail Environments
PROJECT DESCRIPTION
As digital retail environments become more and more pervasive and hardware becomes increasingly economical, opportunities are arising to personalize the individual retail experience. Our research looks at one of these opportunities: to link customer profiles with purchase history, preferences and a product databases to make recommendations in store. The Brigade project is a prototype of a future retail environment, that combines unique identifiers for each customer, creating a shopping experience unparalleled by any other.
To explore the retail environment and build a multisurface environment, we collaborated with an industry design partner in the retail domain, The Brigade Creative Corp., located in Calgary, Canada. From our yearlong collaboration, we created a prototype multisurface solution for the retail environment, with the goal of allowing different retail stakeholders (e.g. customers and employees) to interact with information across multiple devices. The prototype was designed to be scalable and applied to many different retail environments, from smaller shops to larger chain stores.
DEVICES IN THE RETAIL SPACE
The illustration below is a design scenario and layout of a futuristic retail store. This store would come equipped with the following devices: Table Top (Area 2 in the illustration), High Resolution Wall Display (Area 3) and a 4K UHDTV (Area 4). To enhance the shopping experience all devices listed above would be working synchronously using a communication framework (Multi Surface Environment API) which also keeps track of the positioning of all devices and customer in the retail space using Kinect cameras. In addition to tracking individuals, the Kinect cameras also allow customers to interact with the Wall Display, explained further down.
SCENARIO
The customer holding a cell phone (Area 1) enters the store (bottom right of the illustration). By doing so their smartphone loads the customer’s wish list which they will be able to view later. The middle of the room contains an interactive touch table (Area 2). Here the user can browse store inventory or view their wish list stored on their smartphone. At any time the user can view more details, about selected goods on the wall display (Area 3). The user must be in close proximity to the wall display in order to interact with it. Interactions are supported by hand gestures, which are processed by a Kinect camera mounted on top of the wall display. To rotate the product, the user has to lift their arm and with an open palm move in the direction they’d like to move the product. Furthermore the wave gesture triggers the “find in store” features which show where the specified item is located in the retail space. In the mean time employees can post promotional items, coupons to the 4K TV (Area 4). Below you can find a detailed description of the outlined steps.
STEP 1: A CUSTOMER ENTERS THE STORE A customer enters the store with the intention of purchasing a ski boot. As they enter the store, their mobile phone (through a mobile application) notifies them of a type of boot that they might be interested in and directs them to a digital tabletop for more information.
STEP 2: A CUSTOMER USES A TABLETOP The customer approaches the tabletop and places their mobile phone on the tabletop and personalized content appears on the tabletop. The customer then views product information, recommendations, ratings and store location, before eventually finding a pair of ski boots. The customer would like a video of the boot being used, and using the tabletop, a “flick” gesture is performed on the tabletop towards a large highresolution wall display, where the video of the boot, as well as other information will be displayed and interacted with.
STEP 3: INTERACTING WITH A WALL DISPLAY The customer can interact with a 3D image of the boot using gestures in front of the display. Sweeping either arm (left or right) to virtually rotate the boot provides a more complete aesthetic view digitally. The customer then selects the “find in store” option on the wall display by performing a closed fist gesture, causing a map to appear on their smart phone showing the store location of the boot.
STEP 4: MARKETING CONTENT After locating the boot, the customer heads towards a checkout counter and passes by an advertising display showing new and promotional products. At the same time, a store employee is using a tablet to add a new ad to the displays carousel. The employee “flicks” an ad towards the advertising display, sending it to the display. The customer sees that the ad is a coupon for the ski boots and approaches the display in order to receive it. The customer performs a “flick down” gesture on their smartphone and the ad is now saved to their personal account for checkout.
PUBLICATIONS
Sydney Pratte, Teddy Seyed, Frank Maurer: Exploring Multi-Surface Interactions in Retail Environments. In Proceedings of the ACM Interactive Tabletops and Surfaces 2014 (ITS 2014), Dresden, Germany, 2014.
