Automated Detection of Mutual Gaze and Joint Attention in Dual-Camera Settings via Dual-Stream Transformers

Abstract

Detecting mutual gaze and joint attention in infant–caregiver interactions typically requires labor-intensive manual coding. This preprint introduces a dual-stream Transformer that processes synchronized video recordings from two cameras, integrates gaze-related visual information, and models relationships between interaction partners. Tests on infant–caregiver recordings indicate performance above the comparison methods reported by the authors. The proposed approach aims to make video-based research on gaze coordination more scalable.

Publication
arXiv (preprint), arXiv:2604.27105

Automated measurement of mutual gaze and joint attention

How can researchers detect mutual gaze and joint attention when an infant and caregiver are recorded by different cameras? This preprint proposes a dual-stream Transformer that processes two synchronized video views and combines gaze-related information to identify coordinated looking behavior.

The model was evaluated using recordings of infant–caregiver interactions and, in the authors’ comparisons, outperformed a convolutional baseline and a multimodal language model. The work offers a promising route toward reducing time-consuming manual coding while preserving the advantages of studying interaction in more naturalistic settings.

This work is a preprint, not a peer-reviewed journal article.

Read the preprint · Download the PDF