Navigation Turing Test – Evaluating Human-Like Navigation

Many algorithms pass benchmarks, like navigation from a given location to a goal location in 3D games.

But passing benchmarks doesn't guarantee human-like navigation behavior nor cognitively or neurally plausible human-like algorithms/representations. This matters whether...2/n pic.twitter.com/0PM2yhvLWW

— Ida Momennejad (@criticalneuro) May 23, 2021

We used a modified version of a 4X4 player game by Ninja Theory, Bleeding Edge, for solo navigation.

The 3rd person perspective (camera following agent) allows us to observe how different algorithms learn to navigate to 16 goals & judge the human-likeness of their navigation
4/n pic.twitter.com/q3VNfHd5uw

— Ida Momennejad (@criticalneuro) May 23, 2021

Navigation Turing Test (NTT)
Once the agents were sufficiently trained, we designed 2 NTT studies in which we showed pairs of videos of agent vs. human navigation or pairs of 2 agents navigating the game.

Participants judged which of each pair's videos was played by a human
6/n pic.twitter.com/3ZxdPfFxcx

— Ida Momennejad (@criticalneuro) May 23, 2021

Eventually, we'd want algorithms that can judge human-likeness of artificial behavior rather than asking human participants, & one day meta-cognitively optimize behavior w these judgments.
So we compared a number of different architectures & inputs for artificial NTT judges
8/n pic.twitter.com/fyzJjUz8O0

— Ida Momennejad (@criticalneuro) May 23, 2021

Lessons:
- just passing navigation benchmarks doesn't lead to human-like nav
nor human-like algorithms or representations
- automated assessment of human-likeness of human vs agent easier than agent vs agent
- future work can study key behavioral features of human-like nav
10/n pic.twitter.com/GjLpyJgRwe

— Ida Momennejad (@criticalneuro) May 23, 2021

Toward these goals & inspired by meta-cognition in humans a future direction is to add in-built artificial judges of human-likeness that can flexibly modify and adapt parameters of the agent.
Building machines that learn to evaluate human-likeness can advance ML & cog neuro
12/n pic.twitter.com/59muMH5NX6

— Ida Momennejad (@criticalneuro) May 23, 2021

This assumes that such metacognition has enabled us humans to observe our own behavior, compare it with others' behavior, & flexibly adapt/adjust common human behaviors.
Speculation: could lead to RL/ML approaches to metacognition & social/cultural cog, & advance all 3.
n/n pic.twitter.com/uVzmTEJZn5

— Ida Momennejad (@criticalneuro) May 23, 2021