28 December 2024 Enhanced remote photoplethysmography using hybrid RGB and near-infrared cameras
Seunghyun Kim, Kunyoung Lee, Jeongwon Hwang, Hakjin Lee, Eui Chul Lee
Author Affiliations +
Abstract

This study investigates the enhancement of remote photoplethysmography (rPPG) using a hybrid system that integrates RGB and near-infrared (NIR) cameras. rPPG, a non-contact method for heart rate (HR) measurement, relies on analyzing subtle color changes in facial videos. We compare traditional signal processing methods, such as independent component analysis and principal component analysis, with advanced deep learning models such as DeepPhys and PhysFormer. Traditional methods often struggle with lighting variations and motion artifacts, whereas deep learning approaches demonstrate improved robustness and accuracy. Our hybrid approach leverages the strengths of both RGB and NIR cameras, addressing their individual limitations. We propose two methods, the first is a four-channel method, which overlays RGB and NIR frames to create a four-channel image, and the second is an ensemble method, which applies weights to the tokens extracted from each video and then sums them. These hybrid approaches significantly enhance performance, achieving higher percentage of time with error within 6 BPM (PTE6) scores across various scenarios, including still conditions and motion environments. PTE6 represents the proportion of the entire timeline where the predicted HR is within six beats per minute (BPM) of the actual HR and is a commonly used metric in HR prediction studies. For example, the four-channel method achieved a PTE6 score of 72.89% in still conditions and 56.73% in motion scenarios, outperforming the single-camera setups. Specifically, the RGB camera achieved 70.97% and the NIR camera 67.89% in still conditions, whereas in motion scenarios, the RGB camera scored 53.24% and the NIR camera 37.99%. These findings highlight the potential of combining RGB and NIR cameras to improve the accuracy and reliability of rPPG. This approach opens new avenues for advanced telemedicine applications, providing a robust solution for continuous health monitoring in diverse environments.

© 2024 SPIE and IS&T

Funding Statement

Seunghyun Kim, Kunyoung Lee, Jeongwon Hwang, Hakjin Lee, and Eui Chul Lee "Enhanced remote photoplethysmography using hybrid RGB and near-infrared cameras," Journal of Electronic Imaging 33(6), 063058 (28 December 2024). https://doi.org/10.1117/1.JEI.33.6.063058
Received: 21 August 2024; Accepted: 16 December 2024; Published: 28 December 2024
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
RGB color model

Near infrared

Cameras

Video

Data modeling

Light sources and illumination

Education and training

Back to Top