Anomaly Detection for People with Visual Impairments Using an Egocentric 360-Degree Cameras

WACV 2025

Inpyo Song1, Sanghyeon Lee2, Minjun Joo1, Jangwon Lee1,
Sungkyunkwan University1, Korea Aerospace University2
Paper Code Data (Coming Soon) Lab

FDPN detects abnormal activities in 360o egocentric videos.

Abstract

Recent advancements in computer vision have led to a renewed interest in developing assistive technologies for individuals with visual impairments. Although extensive research has been conducted in the field of computer vision-based assistive technologies, most of the focus has been on understanding contexts in images, rather than addressing their physical safety and security concerns. To address this challenge, we propose the first step towards detecting anomalous situations for visually impaired people by observing their entire surroundings using an egocentric 360-degree camera. We first introduce a novel egocentric 360-degree video dataset called VIEW360 (Visually Impaired Equipped with Wearable 360-degree camera), which contains abnormal activities that visually impaired individuals may encounter, such as shoulder surfing and pickpocketing. Furthermore, we propose a new architecture called the Frame and Direction Prediction Network (FDPN), which facilitates frame-level prediction of abnormal events and identifying of their directions. Finally, we evaluate our approach on the VIEW360 and publicly available UCF-Crime datasets, and demonstrate that our proposed approach achieves state-of-the-art performance on both datasets.

Dataset Collection

Using wearable 360o camera, we collected a dataset of 360-degree egocentric videos of anomaly activities that visually impaired people commonly encounter in their daily lives.

Dataset Distribution

We collected our dataset in various locations, including Automated Teller Machines (ATMs), Cafes, Bus Stations, and other public places. The dataset contains 276 normal and 299 abnormal videos.

Dataset Comparison

Sample images of the three datasets, our VIEW360 dataset, UCF-Crime, and XD-Violence. This figure illustrates notable differences between our VIEW360 dataset and other widely used weakly-supervised anomaly detection datasets.

Proposed Approach

To overcome the limitations of existing weakly-supervised methods, which typically assign uniform anomaly scores to video snippets (a brief segment extracted from a video), leading to broad predictions, we developed the Frame and Direction Prediction Network (FDPN). FDPN enhances detection by analyzing anomalies at the frame level with a coarse-to-fine learning strategy, using snippet-level predictions for pseudo-supervision. This method is particularly effective for spotting sudden anomalies, like quick, unexpected events. Additionally, FDPN incorporates saliency-driven image masking to improve accuracy by focusing on visually prominent areas likely to contain anomalies. It also introduces direction classification for anomalies from an egocentric viewpoint, using a saliency heatmap to provide navigational assistance for visually impaired users.

More Qualitative Results

The anomaly detection results of our FDPN on the VIEW360 dataset is illustrated in the graph, with ground truth abnormal frames are marked by blue box.

BibTeX

@InProceedings{Song_2025_WACV,
  author    = {Song, Inpyo and Lee, Sanghyeon and Joo, Minjun and Lee, Jangwon},
  title     = {Anomaly Detection for People with Visual Impairments using an Egocentric 360-Degree Camera},
  booktitle = {Proceedings of the Winter Conference on Applications of Computer Vision (WACV)},
  month     = {February},
  year      = {2025},
  pages     = {2828-2837}
}