In recent times, several studies have presented single-modality systems for non-contact biosignal monitoring. While these systems often yield estimations correlating with clinical-grade devices, their practicality is limited due to constraints in real-time processing, scalability, and interoperability. Moreover, these studies have seldom explored the combined use of multiple modalities or the integration of various sensors. Addressing these gaps, we introduce a distributed computing architecture designed to remotely acquire biosignals from both radars and cameras. This architecture is supported by conceptual blocks that distribute tasks across sensing, computing, data management, analysis, communication, and visualization. Emphasizing interoperability, our system leverages RESTful APIs, efficient video streaming, and standardized health-data protocols. Our framework facilitates the integration of additional sensors and improves signal analysis efficiency. While the architecture is conceptual, its feasibility has been evaluated through simulations targeting specific challenges in networked remote photoplethysmography (rPPG) systems. Additionally, we implemented a prototype to demonstrate the architectural principles in action, with modules and blocks operating in independent threads. This prototype specifically involves the analysis of biosignals using mmWave radars and RGB cameras, illustrating the potential for the architecture to be adapted into a fully distributed system for real-time biosignal processing.
1 Introduction
The development of digital health platforms has significantly altered the healthcare industry. These platforms leverage advanced technologies to improve patient care, make medical workflows more efficient, and boost diagnostic accuracy [38]. The rise of mobile health (mHealth) applications, powered by sophisticated algorithms, represents a major advancement in the domain of real-time health monitoring. Integrated with wearable technologies, these applications are effective in collecting, processing, and transmitting biosignal data continuously, initiating a new phase in patient monitoring [18]. Additionally, telemedicine platforms have begun to incorporate biosignal data, offering healthcare professionals augmented data sources that enhance the accuracy of remote consultations and mitigate the challenges posed by geographical distances [37, 52]. The advent of wearable devices with advanced sensors has broadened the scope of mHealth applications, enabling comprehensive monitoring of vital signs, such as ECGs [25]. The application of machine learning algorithms has further improved the early detection of conditions, for example, atrial fibrillation [57]. The introduction of Remote Patient Monitoring (RPM) systems has additionally benefited patient care by providing healthcare professionals with real-time data, facilitating prompt clinical responses [52]. Despite these advancements, the integration of multimodal biosignal data into a coherent, scalable, and interoperable system remains a significant challenge.
We introduce a distributed computing architecture designed to bridge the gap in integrating multimodal biosignal data, crucial for advancing healthcare technologies. This solution is crafted to efficiently manage the complexities and variances of biosignal data, ensuring scalability and interoperability, key aspects for the evolution of healthcare systems. At the heart of modern healthcare is the need for effective medical data exchange. Health Information Exchanges (HIEs), leveraging advanced protocols, are instrumental in enabling the smooth flow of information across various healthcare environments, promoting accessibility and consistent interpretation of data [3, 58]. Protocols such as HL7 and FHIR are critical in enhancing interoperability, ensuring that healthcare systems can communicate seamlessly and without data discrepancies [47]. The increasing volume and complexity of patient data require robust healthcare systems capable of secure, fast processing, and storage. Cloud-based solutions offer scalable storage and rapid data processing capabilities, ensuring data security and accessibility [49]. For organizations prioritizing data privacy, on-site storage presents a viable alternative [8]. Electronic Health Records (EHRs) facilitate patient health history management, supported by strong security measures such as encryption [51, 63]. Anonymization algorithms enable the safe utilization of patient data in large-scale studies, preserving individual privacy [24]. Our architecture aims to refine user experience across the healthcare spectrum, making systems more intuitive for everyone from patients to professionals [20]. By proposing this architecture, we not only align with current technological advancements in healthcare but also seek to enhance the functionality of mHealth and RPM systems, potentially transforming a wide range of healthcare services. Our contributions are detailed as follows:
We introduce a distributed computing architecture for remote acquisition of multimodal biosignals, utilizing cameras and radars.
We propose an interoperable system with RESTful APIs, efficient video streaming, and adherence to standardized health-data protocols.
We validate our architecture’s effectiveness in remote health monitoring through a remote photoplethysmography (rPPG) subsystem evaluation, demonstrating its scalability and adaptability for improving mHealth and RPM systems in networked healthcare environments.
Anzeige
In the following sections, we detail this architecture, emphasizing its role in advancing remote health monitoring and addressing the current limitations in data integration and interoperability.
2 Related Work
Recent advancements in biosignal monitoring and analysis systems have significantly influenced the creation of healthcare software and hardware architectures. A comprehensive survey by [43] provides an overview of wearable sensor-based systems for health monitoring. Key research has focused on developing innovative solutions for efficient and real-time monitoring of physiological signals, utilizing technologies such as FPGA, wireless DSP architectures, cloud computing, IoT, and wearable sensors. FPGA-based systems, like those presented by [32], emphasize highly integrated hardware designs but may lack in modularity and scalability. Wireless DSP architectures, as discussed by [45], focus on biosignal recording and monitoring using ARM-based Bluetooth wireless systems. While effective in biosignal recording, they fall short in offering comprehensive, multimodal integration. The BioStream system by [9] represents a leap in real-time physiological signal monitoring and emphasizes multipatient monitoring capabilities, an area where our proposed architecture innovates by providing a more unified and interoperable platform.
On the other hand, cloud computing applications in biosignal analysis have been explored by [53], proposing architectures for seamless integration with health information systems, yet these solutions often do not address the challenge of real-time, multimodal data processing. Smart sensor architectures, such as those researched by [44], highlight the importance of in-sensor processing to enhance usability and reduce power consumption, but do not integrate smart sensing capabilities within the broader cloud-connected framework. Affordable and open-source platforms for biosignal measurements, like the Biosignal PI developed by [2], demonstrate the potential for compact and medically safe systems, but they are not integrated into distributed healthcare monitoring systems. Wireless-enabled processor modules, as shown in studies by [19, 55], focus on real-time signal acquisition and transmission, without focusing in scalability and interoperability across different systems. IoT-based wearable systems, as developed by [26, 61], have shown improved performance in patient monitoring. Specialized biosignal extraction devices, such as those by [4, 23], highlight the need for accurate signal processing. Multimodal wearable systems for emergency applications, like the one developed by [33], integrate various monitoring technologies for potential use in critical care settings. Advances in wearable electronics, as discussed by [17], highlight the role of advanced materials and low-power consumption in biosignal monitoring.
While these studies collectively indicate significant progress in biosignal monitoring and analysis systems, underscoring the importance of innovative technological integration for enhanced patient care and monitoring, our architecture built upon these advances offers a novel, scalable, and interoperable solution that addresses the complexities of modern healthcare monitoring.
Anzeige
3 Key Elements in Physiological Signal Devices
The block diagram in Fig. 1 depicts the essential components of medical devices used for physiological signal monitoring. Biosensors, the core components, convert physiological activity into electrical signals. These are then digitized by an Analog-to-Digital Converter (ADC), making them suitable for digital processing.
Fig. 1.
Simplified block diagram illustrating the core components of medical devices for physiological signal monitoring: sensors, ADC, processor, and controller, with functionalities extending to display, storage, and network integration, and possibly therapeutic intervention based on the signal analysis.
×
Digitization is governed by the Nyquist-Shannon sampling theorem to prevent aliasing, ensuring the sampling rate is at least twice the maximum signal frequency. The bit resolution of the ADC affects the precision of the signal’s digital representation. Post-digitization, the processor amplifies and filters the signal, preparing it for analysis, display, recording, and potential therapeutic actions. The integration of these devices with digital healthcare systems enhances patient monitoring by allowing for real-time interventions, long-term data storage, and telehealth capabilities, facilitating better care coordination. Device controllers maintain operational efficiency, while displays provide user interfaces. Effective device design prioritizes security, power efficiency, and user accessibility, all of which are key for the successful adoption of technology in enhancing patient care and health monitoring.
4 Key Challenges of rPPG Acquisition Systems
Physiological signal acquisition systems pose significant challenges in terms of accuracy and reliability of the extracted signals [5, 27]. For rPPGs, most of the studies in the literature address issues such as motion artifacts, skin tone variations, noise, occlusions, and illumination variations, which can degrade accuracy [48]. These studies included stabilizing the region of interest with optical flow and tracking [12, 29], tracking and aligning the face to remove head and face movements [15], using bandpass, adaptive, detrending, or LSTM-based filters to normalize the PPG signals and remove noise and motion artifacts [12, 14, 35], signal separation methods such as PCA, ICA or OMIT [15], signal separation methods based on skin reflection models such as CHROM [16] or POS [60], and correlating signals using a normalized reference waveform or a noise reference signal [31, 59]. A few studies have also focused on tackling illumination variations [31, 42], and skin tone variations [30].
However, challenges related to network and computing constraints in remote and streaming PPG systems remain underexplored, potentially impacting their performance. These challenges encompass limited bandwidth, packet loss, latency, video compression algorithms, resolution, and computing resources [40, 48]. Hardware constraints involve computational limitations and other factors affecting signal quality, such as sensor capabilities or algorithmic complexity impacting system performance. Therefore, careful design is a critical aspect to consider, ensuring that the hardware can adequately support the algorithmic demands for accurate and efficient signal processing [27]. Previous research has focused on the impact of video compression on the quality of the recovered BVP signal [7, 21, 36, 50]. Researchers have found that different compression schemes and codecs can lead to small quality losses in the extracted signal, which can significantly affect the signal’s features and morphology. Some studies have proposed methods to address the issue of video compression artifacts, such as using image filtering or end-to-end deep learning-based methods [64] to improve video quality and reduce file sizes or to use singular spectrum analysis to reconstruct and select signal components [66]. Other studies have examined the effect of reduced frame rate and image resolution on heart rate estimation. However, they have generally found no significant differences in mean absolute error or error distributions resulting from reduced frame rates if they are kept in typical values (15–20fps) [10, 56]. Several strategies have also been studied for the efficient codification of PPGs, but they require a particular learning strategy and architecture [65]. Álvarez et al. [7] investigate the impact of network and hardware constraints on the performance of the rPPG systems. Their approach included simulations and experimentation to develop mitigation strategies. These strategies specifically addressed challenges related to frame dropping, frame resolution, and frame rate in the rPPG system.
5 Proposed Architecture for rPPG and rBSG Acquisition
Many non-contact health monitoring systems align well with clinical devices in controlled studies but face challenges in real-world application, often neglecting real-time processing or distributed computation, scalability, and interoperability. To address these limitations, we propose a distributed computing architecture designed to overcome these challenges by supporting remote biosignal acquisition through both radar and camera technologies. It features a modular framework that organizes tasks related to sensing, computation, data management, and analysis. The system’s interoperability, facilitated by RESTful APIs, efficient video streaming, and standard health-data protocols, enables the seamless addition of new sensors and signal analysis tasks. This feature allows the system to adapt to a range of applications and technologies, enhancing its versatility.
Fig. 2.
Modular Architecture for Multimodal Biosignal Acquisition and Analysis
×
Conceptual demonstration of the system is provided via the implementation of a real-time and networked biosignal acquisition and analysis system using RGB cameras, and its possible extensibility to other sensors. This not only showcases the system’s robust capabilities in handling different sensor types, but also its potential in harmonizing these diverse data streams for comprehensive biosignal analysis. With this, our proposed architecture results a in scalable, interoperable, and multi-modal remote biosignal monitoring system.
5.1 Overview of the Proposed Architecture
The proposed architecture, depicted in Fig. 2, consists of five main categories of blocks: sensing, processing, communications, data storage, and user interaction. These blocks communicate over a network, which can be a Local Area Network (LAN) or an Internet of Things (IoT) network. The design aims to seamlessly integrate various sensors, supporting diverse applications, and incorporates signal analysis and feature extraction methods within the processing blocks. This architecture follows the principles of a standard microservices architecture, where each processing procedure is designated as a “service,” representing a software component. These components can run on a cluster of computers or a microcomputer with sufficient resources. They can be developed using established microservices frameworks like Docker Compose.
Figure 2 shows the proposed distributed computing architecture’s block diagram, where each component is network-linked, facilitating data exchange. Key interconnected blocks, such as sensing, signal analysis, storage, and visualization, enable two-way communication, while others are designed for one-way data flow. Sensing blocks capture various data types, like video and radio frequency (RF) signals, and apply signal processing to yield processed data. This data is then standardized by an aggregation module into formats like Fast Healthcare Interoperability Resources (FHIR) [1], ready for storage in databases or cloud services. Analysis modules access this data, allowing any data analysis to work with the standardized inputs. Interface modules serve end-users by retrieving processed data for visualization. The architecture’s modularity ensures components can be updated or replaced without disrupting the system’s overall functionality.
5.2 Interoperability
Our architecture ensures interoperability, allowing various software components and systems to communicate and collaborate effectively. This facilitates the exchange of data between elements, enhancing system efficiency and flexibility. This is especially critical in digital health, where various devices and software must interact without creating data silos, promoting comprehensive patient care and more effective healthcare delivery.
The architecture integrates a wide range of hardware and software libraries critical for functions spanning from data acquisition to advanced analysis and visualization. These libraries enable the use of various sensors and the execution of complex data processing. For example, hardware libraries might include those necessary for interfacing with cameras and radars, while software libraries could encompass those for data processing, machine learning, and visualization.
Containerization, a lightweight alternative to full machine virtualization, is a core strategy in our architecture, instrumental in achieving software scalability and portability. By encapsulating a software component with its dependencies into a single, self-contained unit (container), it ensures consistent execution across varying computing environments, from a developer’s local machine to testing environments and production servers. This approach minimizes the infamous problem of “it works on my machine” and significantly simplifies software deployment and scaling. A potential drawback could be added complexity in managing multiple containers, but this is generally mitigated by using container orchestration tools. The architecture adopts RESTful APIs (Representational State Transfer APIs) to facilitate smooth and efficient data exchange between blocks. RESTful APIs are widely recognized for their simplicity and scalability, which contribute to enhancing system reliability. Supporting CRUD operations (Create, Read, Update, Delete), they facilitate straightforward communication and interoperability. However, for real-time communication, protocols like WebSocket may be preferable due to REST’s request/response nature. Our architecture integrates video and signal streaming for real-time data transmission, vital in remote patient monitoring for immediate processing and feedback. While advantageous for real-time applications, it may pose challenges in network bandwidth management and necessitate efficient compression algorithms to handle large data volumes. Extensibility, other fundamental aspect of our architecture, enables the integration of new features and functionalities over time. Its modular design facilitates the incorporation of additional sensors, processing methods, or subsystems, ensuring adaptability to evolving needs and technologies. While offering future-proofing and customization benefits, this approach may involve initial design complexity to ensure smooth integration of future enhancements.
While security and compliance are not the core focus of this article, they are nevertheless essential considerations in the design of our digital health platform. Our architecture could be implemented to operate under the regulatory framework of the European Union (EU), which lays a foundation upon which robust security measures can be implemented.
5.3 Sensing and On-Device Computing
In our architecture, sensing and on-device computing are crucial components. They are responsible for capturing biosignals and conducting initial data processing, which forms the basis for further analysis. While the main focus is on standard cameras for sensing, the modular design enables the integration of additional sensors, like radars, to enhance the versatility of the system.
Camera-Based Subsystems rely on standard RGB webcams, as illustrated in Fig. 3, and optionally, other devices like thermal cameras. RGB webcams, known for their widespread availability and user-friendliness, offer a cost-effective means to capture high-quality visual data for various applications, from tracking body movements to facial analysis. Additionally, integrating a thermal camera enables infrared radiation detection, facilitating non-contact body temperature measurement, crucial in many health contexts [34]. These subsystems prioritize versatility and resilience, functioning effectively across different lighting conditions and accommodating multiple subjects in the frame. By converting visual data into standardized digital formats and supporting video transmission via MJPEG streams [54], they seamlessly integrate with the system, efficiently communicating collected data through RESTful APIs. Thus, the integration of RGB cameras is central to our architecture’s monitoring capabilities, enabling comprehensive remote health monitoring across diverse conditions and scenarios.
Fig. 3.
Real-time Camera based subsystem to compute biosignals and vital signs
×
Alternatively to the camera subsystems, our platform allows the use of other non-contact sensors such as radars [41]. As an example platform, we have integrated a Texas Instruments IWR1443 mmWave FMCW radar system operating in the 76–81 GHz frequency range, including four receivers and three transmitters. This mmWave radar can monitor vital signs up to 1.3 m away, exploiting the Doppler effects to measure bodily movements induced by respiratory and cardiovascular activities, a technique known as remote ballistography (rBSC) [5]. Similarly to the camera, the signal processing involves a 16-second running window, updated every second. Raw signals and computed values of breath rate, heart rate, and distance are transmitted via network packets for real-time monitoring.
Real-time computation is crucial in our architecture for applications requiring immediate feedback. On-device computation offers benefits such as reducing data transmission, saving bandwidth, and decreasing communication costs. It also enhances system robustness by reducing reliance on continuous connectivity, conserves energy, and safeguards privacy by minimizing raw data exposure. In the proposed architecture, camera-based or radar-based subsystems collect raw data in our architecture, followed by real-time pre-processing at the device level. This step refines biosignals by filtering them within the relevant frequency band, typically associated with heart or respiration dynamics. Pre-processing involves techniques like noise filtering, normalization, and signal enhancement for high-quality biosignals. On-device computation also calculates key physiological indicators such as Heart Rate, Respiration Rate, and Heart Rate Variability (HRV) parameters, using advanced digital signal processing techniques. This preliminary computation significantly enhances system efficiency, reducing network data load and feedback latency to healthcare professionals.
5.4 Aggregation, Storage, and Standardization
In our proposed architecture, the Aggregation, Storage, and Standardization phase serves as the central processing hub, providing integration between the data collection devices and the end-users. This stage encompasses the receipt of raw and processed data from various end devices, its subsequent storage, and the generation of standardized health data messages as depicted in Fig. 4.
Fig. 4.
Aggregation and data storage modules in the proposed platform. The system can be extended to other complementary sensors, such as radars (purple). (Color figure online)
×
The Aggregator, potentially hosted in a Docker container, serves as a central hub in our biosignal acquisition architecture. It receives heterogeneous data from sensor devices and converts it into standardized formats like Fast Healthcare Interoperability Resources (FHIR). FHIR, introduced by HL7 in 2014, facilitates electronic healthcare information exchange via web-based data exchange methodologies. FHIR’s “resources” provide fundamental units of healthcare information adaptable to diverse applications, enabling data exchange in XML and JSON formats through RESTful APIs. Its strengths include extensive data definitions, adaptable exchange protocols, and widespread open-source tool support, ensuring consistent and secure healthcare information exchange [39]. Once converted to the FHIR standard, data is securely stored for further processing. InfluxDB, a time-series database, is recommended for our architecture due to its suitability for real-time biosignal and vital sign data handling. Alternatively, other solutions may be considered, especially if raw video storage is required. InfluxDB ensures rapid read/write operations, enabling processing to match real-time data capture. Operating within a Docker container, it offers scalability and easy deployment across various computing environments. This setup allows efficient access to biosignals and vital signs for subsequent processing stages, including AI model training.
5.5 Data Analysis and AI Computation
The data analysis and AI component of the proposed biosignal processing system is critical for interpreting biosignals and executing advanced computations. It includes sub-modules for cloud-based analysis, machine learning algorithms, and multimodal data integration. Immediate metrics like heart and respiration rates are processed on-device. Complex analyses are performed in the cloud, where features from the biosignals are extracted to train machine learning models. These models predict health indicators such as stress, depression levels, SpO2, and respiratory conditions with high accuracy, thanks to cloud computing’s capacity to handle extensive computations.
Model training uses a comprehensive dataset of patient information, including medical histories and diagnoses, in a supervised learning framework. This ensures high-quality, accurately labeled data. The diversity of patient data increases the models’ reliability and applicability. Transfer learning tailors these models to individual patients’ needs, improving personalization. Multimodal data processing is achieved by fusing features from various sensors, enhancing the system’s ability to provide a thorough analysis of the biosignals. This not only improves data richness but also the precision of the health assessments generated by the AI models.
5.6 Interactive User Interface
The user interface (UI) acts as a crucial link between users and underlying technology. Our proposed interactive UI manage biosignal measurements being designed for collaborative users. It enables measurement initiation by user positioning, omitting conventional buttons for a more intuitive interaction paradigm [13]. Figure 5 displays an example.
When users position themselves, the UI immediately starts measurements, recording and transmitting video and biosignals simultaneously for real-time sync. Stable network connection is crucial for video transmission, while device-level computation handles real-time vital sign calculations. The UI adds quality measurements to the data stream, confirming signal stability and assessing movement intensity to address facial motion artifacts. It detects when users leave the camera’s view, signaling the end of the session. These metrics are sent via RESTful API for data interpretation and system optimization. To ensure real-time operation, the UI minimizes latency and integrates into a networked environment. Leveraging our unsupervised biosignal acquisition methods [15], the UI enables accurate and swift vital sign computation, supporting integration with medical monitoring processes.
Fig. 5.
A figure depicting an example interactive user interface. When the user moves to the center of the camera’s FoV, the biosignal measurement is initiated.
×
6 Evaluation of the Camera-Based rPPG Component
We evaluate our architecture under two operational setups, highlighting the networked design. The first setup is cloud-based, using deep learning for facial analysis and algorithms to extract BVP signals from RGB data, ideal for environments with robust network support that enables high-quality video transmission and advanced processing. The second setup is designed for real-time operations on embedded systems, improving processing speed and focusing on efficient biosignal processing directly on the device, through algorithmic and implementation optimizations. This approach also addresses privacy concerns by potentially eliminating the need for video transmission. The real-time model demonstrates how components can work efficiently within a networked system.
We assessed the performance of both configurations in terms of speed and accuracy to determine the architecture’s effectiveness. Both systems use a design that splits tasks across multiple threads, enhancing efficiency and facilitating their fit into a broader distributed framework. This strategy enables seamless operation and data sharing among components, applying mitigation strategies to the effects of network or hardware limitations [7]. These solutions are critical for ensuring the reliability of biosignal extraction and processing, regardless of the operational framework.
6.1 Experimental Setup and Configurations
The experimental setup for the evaluation of our camera-based rPPG component is based on two configurations, both based on the Face2PPG pipeline [15]. Face2PPG-RT is designed for real-time processing on embedded systems, while Face2PPG-Server is optimized for higher accuracy but with increased computational demands. Referenced in Table 1, these setups facilitates their comparison across different configurations.
The Face2PPG-RT configuration uses a RESTful API with the Restbed library for handling HTTP requests. It initiates an HTTP server in the MainWindow constructor to manage requests and responses on a specified port. Data is transmitted to the HTTP server every 33 ms as an MJPEG stream via a dedicated function, ensuring continuous data flow. Multithreading supports parallel operation of the camera, HTTP server, RESTful API, and GUI, improving system responsiveness and stability for real-time monitoring of patient bio-signals. The prototype incorporates optimization libraries like Lapack, BLAS, OpenMP, and FFTW3. The test hardware includes an Intel® Core i7-6700HQ CPU with HD Graphics 530, and 8GB of RAM, running Linux Ubuntu 18.04.6. Face detection uses YuNet CNN [62], and face alignment and skin segmentation is based on ERT-GTX models [6]. The segmentation method [15], focuses on the cheeks and forehead, and converts to CIE Lab color space.
The Face2PPG-Server prototype adopts a Python-based implementation using a deep learning-based face detection method with a Single Shot Multibox Detection (SSD) network [15]. Faces are aligned using the Deep Alignment Network (DAN) [28], These 85 landmarks construct a mesh of 131 triangles and selects the optimal facial regions for extracting raw RGB signals dynamically [15]. Testing was conducted on a high-performance setup featuring an AMD® Ryzen(TM) 3700X 8-core processor at 3.6GHz, with 64 Gigabytes of RAM, 4 terabyte SSD and two NVIDIA GeForce® RTX(TM) 2080.
6.2 Benchmark Datasets, Protocol and Metrics
We conducted an extensive evaluation of the rPPG systems performance across four publicly available datasets, each comprising videos and reference PPG signals obtained using medically-graded oximeters, alongside videos captured with user-grade webcams, three with uncompressed videos (1.5 GB per video) and one with highly compressed videos ( 2 MB per video). The datasets are: COHFACE [22], comprised of 160 highly compressed videos of 40 subjects, recorded at 20 Hz and a resolution of 640\(\,\times \,\)480 pixels. LGI-PPGI-Face-Video-Database, contains 24 videos from 6 users across 4 scenarios [46], recorded at 640\(\,\times \,\)480 pixels at 25 fps. UBFC-RPPG, consists of two subsets: UBFC1 or simple and UBFC2 or realistic [11]. UBFC1 contains 8 videos where participants remained seated in an office room under natural light conditions, while UBFC2 includes 42 videos recorded under constrained conditions. The videos are captured at 640\(\,\times \,\)480 pixels at 30 fps.
To evaluate the accuracy of our rPPG measurements, we followed the assessment protocol described in [15] that compares the estimated heart rates extracted from the video streams to the reference (ground-truth) contact-based PPG signals. We employ three well established metrics: Mean Absolute Error (MAE), Root-Mean-Square Error (RMSE), and Pearson Correlation Coefficient (PCC) of the heart-rate envelope. In addition, we measure the processing time required to estimate heart rate per window (12-second), and we calculate the time per frame expended for each module within each pipeline. We also compute the total frame rate of each configuration in frames per second (FPS).
6.3 Speed Performance
Table 1 illustrates the computational performance of the different modules within our Face2PPG pipelines across the two proposed configurations: Embedded (targeted for real-time operation) and Cloud (for unconstrained conditions, typically found in server environments).
Table 1.
Configuration Setups and performance speed for individual modules in both rPPG systems, as well as the overall speed per face and frame.
Module
Face2PPG-RT
Face2PPG-Server
Embedded
Configuration
Speed
ms/frame
Cloud
Configuration
Speed
ms/frame
Face Detection
OpenCV YuNet
1.27
OpenCV SSD
20.35
Face Alignment
Dlib ERT GTX
3.74
DAN
82.52
Face Normalization
Mesh
4.37
Mesh
12.48
Skin Segmentation
Fix Patches
0.35
Best Regions
105.41
RBG to BVP
Lab
0.46
POS
0.079
Filtering
BP FIR
0.01
BP FIR
0.01
Spectral Analysis
FFT
0.21
Welch
1.02
Language
C++
–
Python
–
Total Time
–
10.41
–
221.87
Total FPS
–
96.1
–
4.5
The performance evaluation of Face2PPG-RT indicates that it processes each frame in about 10.41 milliseconds (ms), with ‘Face Normalization’ being the most time-consuming module at 4.37 ms per frame. Despite the computational demands, it maintains a high frame rate of 96 FPS, suitable for real-time operation. On the other hand, Face2PPG-Server shows a longer processing time of 221.87 ms per frame, (4.51 FPS). This increase is primarily due to the ‘Face Alignment’ and ‘Skin Segmentation’ highlighting the added complexity of multi-region analysis. Table 1 details the speed performance, demonstrating the versatility of the proposed distributed framework in handling both real-time and in-depth analytical tasks.
6.4 Accuracy Performance in Vital Signs Measurement
To evaluate the accuracy of the rPPG configurations, where we measure the heart rate accuracy using several databases. The results, summarized in Table 2, shows that embedded setup is just slightly less effective than the ones from server setup. These findings show that the impact of the performance of the real-time configuration is relatively minimal.
Table 2.
Error comparison between the two configurations (Embedded and Cloud) of the proposed rPPG system in four different databases.
Pipeline
LGI-PPGI
COHFACE
UBFC1
UBFC2
MAE ± SD
PCC
MAE ± SD
PCC
MAE ± SD
PCC
MAE ± SD
PCC
Face2PPG-Server
4.5 ± 3.3
0.57
8.0 ± 4.4
0.06
0.9 ± 0.4
0.96
0.9 ± 0.9
0.98
Face2PPG-RT
5.9 ± 8.0
0.49
11.3 ± 7.3
−0.01
1.5 ± 1.2
0.83
6.7 ± 6.1
0.54
7 Conclusion
Advancements in wireless communications, particularly with 5G and the upcoming 6G, are significantly enhancing remote patient monitoring in healthcare by enabling faster data exchange and lower latency, crucial for real-time health monitoring. Our study introduces a distributed computing architecture that utilizes radar and camera technologies for efficient, real-time biosignal processing and analysis. This architecture integrates various sensors and ensures interoperability, marking a significant step forward in remote health data acquisition. The evaluation of the camera-based rPPG component within our framework confirms its utility in healthcare by showcasing its ability to manage biosignals in real-time within a networked environment, while emphasizing data security. These findings highlight the architecture’s potential to enhance remote health monitoring and patient care. In essence, as demand for real-time, remote health monitoring grows, our research offers a robust, adaptable framework using the latest in communication technology. Future work will focus on expanding this framework and assessing its performance in clinical settings, aiming to further support remote healthcare delivery.
Acknowledgements
This research was supported by the Research Council of Finland (former Academy of Finland) 6G Flagship Programme (Grant Number: 346208) and PROFI5 HiDyn (Grant Number: 326291).
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
