The Future Face of the In-Store Customer Experience: Face Recognition Technology

By: Julia Ricci, VP, Information Technology - Retail Shared Services & Solutions, Total Wine & More

Julia Ricci, VP, Information Technology - Retail Shared Services & Solutions, Total Wine & More

“The road you are on, John Anderton, is the one less traveled”.

That’s how Mr. Anderton, a character played by Tom Cruise in Spielberg’s 2002 “Minority Report’’ movie, is greeted by a Lexus ad video when walking into a mall. In the movie, the character is acknowledged by name and offered personal ads by digital billboards, based on his past purchase history and loyalty records.

The movie depicts a futuristic retail world, where people are individually recognized through iris scanning technology and offered a very personalized shopping experience. We won’t have to wait until 2054, however, for that experience to become reality. Some of the biometric technologies depicted in the movie are already here (e.g. finger printing, iris scanning, face recognition). Big technology players, like Apple and Facebook for example, have been experimenting with face recognition technologies for quite some time now.

"The ongoing development of machine/deep learning in AI may lead to the possibility of an increasingly wider application of face recognition technologies, through enhanced affordability and ease of use"

Facial recognition is a biometric software application capable of uniquely identifying or verifying a person by comparing and analyzing patterns based on the person's facial contours, distance between points in the face, nose width etc.

Apple has been offering a face unlock feature since the launch of the iPhone X in 2017. Apple uses its complex TrueDepth camera system to create a 3D scan of a user’s facial pattern comprising of 30,000 invisible points. The pattern is then sent to a neural engine to check whether the registered face is same as the scanned one. The technology is considered secure and convenient, and it is also available to authenticate purchases through Apple Pay. It works in the dark and can adapt to the face change through a dynamic AI learning system.

In Facebook, if the feature is enabled, the system will automatically tag the user in pictures by comparing the actual picture with the data it has stored from the user profile (pictures, videos etc.). The technology is based on machine and deep learning, a very exciting and evolving area of AI. Under research for decades, deep learning is now achieving state-of-the-art results across various problem domains and applications.

The ongoing development of machine/deep learning in AI may lead to the possibility of an increasingly wider application of face recognition technologies, through enhanced affordability and ease of use.

Retail software development shops of any size or budget may be able to incorporate face recognition functionality in their loyalty/marketing apps through API services available in the marketplace. A good example is Amazon Rekognition. While built on the proven, deep learning technology developed by Amazon’s computer vision scientists, the API service requires no machine learning expertise to use. Simple integration, continuous learning, low cost and accuracy of face detection and analysis would potentially make face recognition a commodity service for retail software applications, such as user verification, in-store personalized experience, people counting and obviously public safety.

It is worth noting that a personalized, targeted customer experience has been commonplace in the online shopping world for some time now. Customized offers, marketing messages and advertising based on the customer’s purchase and viewing history is a widely available feature on most e-commerce web sites. Now, AI (and face recognition in particular) could bring the same convenient and personalized experience available online to the brick and mortar world.

Typical retail use cases may include:

• Personalized in-store customer experience:

• Personalized, targeted real-time marketing messages (e.g. displays ads).

• Alert to store associates that a VIP customer has walked into the store. The store associate would have immediate access to the customer’s Loyalty data (e.g. past purchase history, rewards, preferences etc.), for a personalized interaction and selling experience, without the customer having to provide a loyalty card or phone number, or log in to an app on their mobile phone. This scenario obviously assumes that the customer has opted in by providing their face picture, in exchange, for example, for some special discounts/rewards programs.

• Observation of the customers’ reaction in front of a product (emotions analysis) based on facial expressions.

• In-Store Analytics/Traffic Patterns:

• Face Recognition technologies can identify some traits of the customer through face analysis linked to demographics (e.g. gender, age range). In this use case, the retailer could link product purchases and other in-store behaviors (such as dwelling time, path, and facial expressions) to specific demographic groups, which they would then target though personalized advertisement across channels. Analysis of facial emotions could potentially even be leveraged to optimize products selection, display, and location.

• Age Verification for in store age-restricted purchases (e.g. alcohol)

Beside marketing, shopping applications, face recognition would provide obvious support for security-related use cases, through real-time detection and alerting of known threats (e.g. known criminals walking into a store or other public place). A good example is the platform provided by FaceFirst, a California-based company. FaceFirst offers API and SKD for integration into various systems/applications to enable surveillance, access control, customer engagement and other applications for law enforcement agencies, transportation centers, retailers and others.

The future face of the shopping experience, however, does not come without challenges, risks and limitations. Customers will be faced with a dilemma around convenience versus privacy.

While theft/crime prevention is potentially a very positive outcome, it should be noted that any technology that can identify individuals in public spaces is potentially a privacy invasion and could even lead to crime increases (e.g. stalking, identity theft etc.).

In absence of comprehensive regulation laws, there is the risk that customers may be getting scanned without their knowledge. Recent research indicates that up to 30 percent of retailers in the UK are experimenting with face recognition technology.

While the challenges surrounding the adoption of face recognition may limit and possibly slow down the wider adoption of the technology in retail, it is evident that the potential opportunities offered by the new platform cannot be ignored. Whether it’s to prevent or detect fraud and loss in stores, or to provide a personal and convenient customer experience, face recognition and AI technologies are here now to enable important technology-driven transformations.

The technology will bring the brick and mortar customer experience closer to the online/digital one. At the same time, it has the potential to take us back to a sort of ‘small village’ experience, where everybody is recognized and greeted, as opposed to today’s large mall, anonymous kind of interaction. Past and future may eventually converge in the future ‘face of retail’.