Virtual Assistants Take on COVID-19 Screening, Docent Duties

Image credit: CloudConstable

The next time you head to the museum, you might be greeted by a robot. Museums are using robots with artificial intelligence to engage with visitors, freeing staff to handle more complex interactions and activities. They take different forms—some use images on screens and others have full-body robots that can move throughout a building or space, but all rely on artificial intelligence, cameras, and special voice and gesture interfaces to enable human-like interactions with visitors.

The animated Master Corporal Lana is now stationed at the Ontario Regiment Royal Canadian Armoured Corps (RCAC) Museum. The museum, which claims to house the largest collection of working military vehicles in North America, recently opened after the COVID-19 shutdown. As part of new pandemic-related operational changes, visitors will be welcomed by Master Corporal Lana, a virtual assistant that uses artificial intelligence to interact with people.

Through a kiosk with a large screen, Lana welcomes people, asks COVID-related screening questions, and takes temperatures before allowing visitors to enter the museum. She can answer questions, count visitors, check tickets, and provide crowd control. Interactions flow easily because the technology behind her is doing the heavy work.

The Virtual Assistant Is In

Based on the Animated Virtual Agent (AVA) designed by CloudConstable, based in Ontario, Canada, the technology includes an Intel® NUC 9 Pro Mini PC, which enables Lana to operate as a smart kiosk, and Intel® vPro™ technology for secure remote management. Lana incorporates natural language processing technology, which allows her to converse with, ask, and answer questions of visitors. The Intel distribution of the OpenVINO™ toolkit supports AI inference acceleration, and the Intel® RealSense™ depth camera D415 allows Lana to recognize verbal and non-verbal communications. The ability to identify gestures, such as a nod or headshake, allow interactions to be touch-free.

The camera incorporates facial recognition, which can identify staff members or registered volunteers who have access to restricted museum areas. Those people have to opt-in to be recognized by name and face. The Intel® RealSense™ depth camera is set up so Lana will only interact with visitors within a certain range—about 6 feet at the Ontario Regiment Museum. That capability protects the privacy of people in the background, while allowing Lana to interact with people who approach her.

In addition to assisting with gesture control, the camera works in tandem with a custom sensor array that includes a long wavelength infrared (LWIR) thermal scanner. As part of Lana’s COVID-screening duties, she takes and records the temperatures of museum-goers. The sensors can detect elevated body temperatures, but distance can affect accuracy. Lana is able to calibrate the thermal scanner, and the depth camera ensures accurate readings.

A Visitor Favorite: Smithsonian’s Robotic Docents

The Smithsonian Institute is also using robots to engage with visitors. Six Pepper robots, developed by SoftBank Robotics, act as docents in some of the Smithsonian Museums. The robots stand about 4 feet tall and move about the museum space answering questions, telling stories, translating information, and taking selfies with visitors.

two images of museum halls, one with text "what absence is made of" and the other showing a robot

Image credit: Smithsonian/Hirshhorn Museum

Pepper uses perception modules to recognize people and interact with them. She uses touch sensors, LEDs, and microphones, which allow her to engage with tourists in different ways, including on her touch screen. Infrared sensors, an inertial unit, 2D and 3D cameras, and sonar allow Pepper to move around the museum, leading visitors to specific areas while not crashing into other people. She is also able to show visitors how to interact with some exhibits, answer questions they might be unwilling to ask a human, and move guests through the museum to manage lines at specific exhibits.

Because of her uniqueness, Pepper is an attraction of her own. Attendance doubled at a Rosa Parks exhibit at the National Museum of African American History and Culture when Pepper was there, and overall visitors find her engaging, according to museum staffers.

AI-enabled robots can provide customized interactions and new ways to engage visitors. They also can track queries to help streamline museum flow and improve exhibits using data directly from user interactions.

Now Hiring: Automated Assistants

Intel is committed to respecting human rights and avoiding complicity in human rights abuses. See Intel’s Global Human Rights Principles. Intel’s products and software are intended only to be used in applications that do not cause or contribute to a violation of an internationally recognized human right.