As virtual reality (VR) applications become popular, the desire to enable high-quality, lightweight, and mobile VR can potentially be achieved by performing the VR rendering and encoding computations at the edge and streaming the rendered video to the VR glasses. However, if the rendering has to be performed after the edge gets to know of the user’s new head and body position, the ultra-low latency requirements of VR will not be met by the roundtrip delay. In this article, we introduce edge intelligence, wherein the edge can predict, pre-render and cache the VR video in advance, to be streamed to the user VR glasses as soon as needed. The edge-based predictive pre-rendering approach can address the challenging six Degrees of Freedom (6DoF) VR content. Compared to 360-degree videos and 3DoF (head motion only) VR, 6DoF VR supports both head and body motion, thus not only viewing direction but also viewing position can change. Hence, our proposed VR edge intelligence comprises of predicting both the head and body motions of a user accurately using past head and body motion traces. In this article, we develop a multi-task long short-term memory (LSTM) model for body motion prediction and a multi-layer perceptron (MLP) model for head motion prediction. We implement the deep learning-based motion prediction models and validate their accuracy and effectiveness using a dataset of over 840,000 samples for head and body motion.
Renewable energy (RE) is a promising solution to save grid power in mobile edge computing (MEC) systems and thus reducing the carbon footprints. However, to effectively operate the RE-based MEC system, a method for predicting the state of energy (SoE) in the battery is essential, not only to prevent the battery from over-charging or over-discharging, but also allowing the MEC applications to adjust their loads in advance based on the energy availability. In this work, we consider RE-powered MEC systems at the Road-side Unit (RSU) and focus on predicting its battery’s SoE by using machine learning technique. We developed a real-world RE-powered RSU testbed consisting of edge computing devices, small cell base station, and solar as well as wind power generators. By operating RE-powered RSU for serving real-world computation task offloading demands, we collect the corresponding data sequences of battery’s SoE and other observable parameters of the MEC systems that impact the SoE. Using a variant of Long Short-term Memory (LSTM) model with additional convolutional layers, we form a CNN-LSTM model which can predict the SoE accurately with very low prediction error. Our results show that CNN-LSTM outperforms other Recurrent Neural Networks (RNN) based models for predicting intra-hour and hour-ahead SoE.
To function optimally and efficiently, connected autonomous vehicles must be equipped with both adequate computational resources and computing architectures. In this paper, the researchers determine the critical performance metrics required for connected vehicles, and also empirically demonstrate which factors and metrics satisfy the static and dynamic computing needs for optimal performance. The researchers also examine the tradeoffs related to different offloading strategies and how different edge computing architectures for vehicular computing might be made feasible for lightweight, high-performance and low-power computing paradigms.
A 400 MHz narrowband MedRadio transmitter for short-range communication is presented. A new technique for PVT-robust, calibration- and regulation-free synthesis of the RF carrier is reported based on generating poly-phasors at 50 MHz with no power overhead. This is accomplished using a passive polyphase filter directly integrated within a crystal oscillator followed by an 8× edge combiner to synthesize the RF carrier with -109 dBc/Hz phase noise at 100 kHz offset. A dual supply, inverse class-E power amplifier is implemented for high efficiency at low output power (-17.5 dBm). Open-loop operation permits aggressive duty-cycling (< 40 ns start-up time). State-of-the-art ultra-low power is reported from a prototype BPSK transmitter fabricated in 22 nm CMOS FDX when operated from a 0.4/0.2 V supply consuming 67 μW with 27% global efficiency.
This article presents a miniaturized wake-up receiver (WuRX) that is temperature-compensated yet still consumes only 22.3 nW (7.3 nW excluding the temperature compensation blocks) while operating at 9 GHz (X-band). By moving the carrier frequency to 9 GHz and designing a high impedance, passive envelope detector (ED), the transformer size is reduced to 0.02 cm2 while still achieving 13.5 dB of passive RF voltage gain. To further reduce the area, a global common mode feedback (CMFB) technique is utilized across the ED and baseband (BB) amplifier that eliminates the need for off-chip ac-coupling components. Multiple temperature-compensation techniques are proposed to maintain constant bandwidth of the signal path and constant clock frequency. The WuRX was implemented in 65- (RF) and 180-nm (BB) CMOS and achieves −69.5- and −64-dBm sensitivity with and without an antenna, respectively. Importantly, the sensitivity is demonstrated to vary by only 3 dB from −10 to 40 ◦C. This article demonstrates state of-the-art performance for WuRXs operating at >1 GHz while achieving the smallest area and best temperature insensitivity. Index Terms— Near-zero power, temperature compensation, wake-up receivers (WuRXs), X-band.
As part of their treatment, physical therapy patients routinely undergo balance evaluation, which is traditionally performed by a physical therapist in a clinic and can be subjective, inconvenient and time-consuming. This study combines sensors and deep learning to provide an automated balance evaluation system for use in both the clinic and the home. To achieve this, the researchers use a deep learning-based model and a depth camera to estimate the user’s Center of Mass (CoM) position -- a method that outperforms other CoM estimation methods in terms of accuracy and ease-of-use. They then make use of a balance evaluation system to evaluate the subject’s dynamic balance in a Gait Initiation (GI) task. To do this, the subject’s CoM position is estimated by the study’s CoM estimation model, and a Wii balance board is used to measure the Center of Pressure (CoP). The researchers then use CoP-CoM trajectory during the GI task to assess and quantify the patient’s dynamic balance control. The proposed model is able to quantify balance level for both healthy subjects and those with Parkinson’s Disease in a way that is consistent with the human PT’s assessments in traditional balance evaluation tests, making it a portable and low-cost tool for on-demand balance evaluation.
Delivering the kind of truly immersive 360-degree virtual reality experiences that both consumers and enterprise users expect requires ultra-high bandwidth and ultra-low latency networks. Developing these networks for mobile use is an even greater challenge. To optimize mobile VR performance, developers commonly stream only the field of view (FOV) to reduce bandwidth. However, when a user changes head position, the FOV must also change in real time to avoid excessive latency. This paper proposes a predictive adaptive streaming approach, where the predicted view is adaptively encoded at a relatively high quality, given bandwidth conditions, and is transmitted in advance. This leads to a simultaneous reduction in bandwidth and latency. This method is based on a deep-learning-based viewpoint prediction model developed by the researchers, which uses past head motions to predict where a user will be looking in the 360-degree view. The method for high-quality, low latency visualization is validated by way of a very large dataset consisting of head motion traces from more than 36,000 viewers for nineteen 360-degree/VR videos.
As 360-degree videos and virtual reality (VR) applications become popular for consumer and enterprise use cases, the desire to enable truly mobile experiences also increases. Delivering 360-degree videos and cloud/edge-based VR applications require ultra-high bandwidth and ultra-low latency , challenging to achieve with mobile networks. A common approach to reduce bandwidth is streaming only the field of view (FOV). However, extracting and transmitting the FOV in response to user head motion can add high latency, adversely affecting user experience. In this paper, we propose a predictive adaptive streaming approach, where the predicted view with high predictive probability is adaptively encoded in relatively high quality according to bandwidth conditions and transmitted in advance, leading to a simultaneous reduction in bandwidth and latency. The predictive adaptive streaming method is based on a deep-learning-based viewpoint prediction model we develop, which uses past head motions to predict where a user will be looking in the 360-degree view. Using a very large dataset consisting of head motion traces from over 36,000 viewers for nineteen 360-degree/VR videos, we validate the ability of our predictive adaptive streaming method to offer high-quality view while simultaneously significantly reducing bandwidth.
An ultra-low power electrocardiogram (ECG) recording front-end intended for implantable sensors is presented. The noise-limited, high power first stage of a two stage amplifier utilizes stacking of operational transconductance amplifiers (OTAs) for noise and power efficiency improvements. The proposed technique involves upmodulated/chopped signals being applied to ac-coupled, stacked inverter-based OTAs that inherently sum the individual transconductances while reusing the same current, thereby enhancing the noise efficiency. Two prototype designs were fabricated in a 180-nm CMOS process. The three-stack version consumes 13.2 nW and occupies 0.18 mm2, whereas the five-stack implementation consumes 18.7 nW and occupies 0.24 mm2. State-of-the-art NEF and PEF metrics of less than unity, 0.86 and 0.99, respectively, are reported for the five-stack version. These correspond to ∼3× improvement in terms of energy efficiency compared to prior ultra-low power, sub-100-nW amplifiers.
As virtual reality (VR) and augmented reality (AR) applications grow in popularity across economic sectors, so does the need for high-quality, lightweight and mobile VR. Yet to enable a truly mobile VR experience, mobile head-mounted displays (HMDs) must make use of bandwidth-constrained mobile networks while also satisfying ultra-low latency requirements. To accomplish this, the researchers propose that the heavy computational tasks of video rendering take place in advance in edge/cloud-based Six Degrees of Freedom (6DoF) servers. Both head and body motions and viewing position changes are supported by 6DoF, which allows the system (by way of a deep learning-based model) to accurately and efficiently predict viewing direction and position using past head and body motion traces. This predictive model could feasibly lead to a reduction in latency as the video is streamed to users via lightweight VR glasses.
Although blood pressure (BP) is one of the chief indicators of human health and is highly correlated to various health behaviors such as sleep and exercise, little is known about how each of these health behaviors may affect any given individual’s BP. For this study, the researchers hypothesized they could predict an individual’s BP using health behavior and historical BP, and then identify the most important factors in BP prediction for that individual. This information could then be used to provide personal health behavior recommendations to improve and manage BP. Using data collected from off-the-shelf wearable devices and wireless home BP monitors, the researchers investigated the relationship between BP and health behaviors using a personalized BP model based on a random forest model with Feature Selection (RFFS). To enhance RFFS and account for problematic concept drifts and anomaly points, the RFFS model was paired with an Online Weighted Resampling technique. The experimental results demonstrate that the RFFS/OWR prediction model achieves a lower prediction error when compared with existing machine-learning methods.
Photographers typically use a high ISO mode on their cameras to capture fast-moving objects, to record details in low-light environments and to avoid blurry images when taking photos without tripods. However, a high ISO also introduces much realistic “noise” to the image, which is difficult to remove using traditional denoising methods. This study proposes a novel denoising method for high ISO JPEG images by fusing collaborative and convolutional filtering. Collaborative filtering — which is effective for recovering repeatable structures —- is performed by first estimating the noise variance according to the Bayer pattern of noise variance maps. Convolutional filtering — which is effective for recovering irregular patterns —- is achieved by using a Convolutional Neural Network (CNN) to remove the noise. The two results are then combined to generate the final denoising via a proposed deep CNN. Experimental results show that this method outperforms state-of-the-art realistic noise removal methods. The study also developed a dataset with noisy and clean image pairs for high ISO JPEG images to promote further research on this topic.
Evolving in tandem with the computing and communication needs of vehicles is the need to technologically support these vehicles in a way that is both sustainable and also minimizes Quality of Service (QoS) loss. This study explores the use of Solar-powered Roadside Units (SRSUs), which consist of a small cell base station (SBS) and a road-edge computing (REC) node to which vehicles can offload various application tasks at high throughput and low latency. To mitigate any QoS loss inherent to using a low-cost solar system, REC or limited bandwidth resources, the researchers propose a dynamic offloading approach whereby the different subtasks of a vehicle application are optimally processed -- either locally using the vehicle’s own computer resources, or remotely using the REC resource (taking into consideration the energy, computing or bandwidth constraints of the SRSU network). This is achieved using a heuristic algorithm that jointly makes the optimal user association, offloads subtasks and allocates SRSU network resources. The results of a simulation framework used to test the algorithm demonstrate that the proposed approach can significantly reduce QoS compared with other best effort strategies.
This study proposes a machine-learning based on-demand virtual physical therapy (PT) system to enable patients with Parkinson’s Disease to improve balance and mobility. As patients engage in three PT tasks with varying levels of difficulty, their movements are captured by a Kinect sensor and are automatically evaluated using criteria carefully designed by a PT co-author. The patient’s motion data is then used to propose a two-phase human action understanding algorithm TPHAU to understand the patient’s movements, as well as an error identification model to identify the patient’s movement errors. To emulate a real physical therapist’s guidance, automated, personalized and timely task recommendations are then made using a machine-learning-based model trained from real patient and PT data. Experiments show that the proposed methods are highly accurate in terms of helping patients understand PT actions, identifying errors and recommending tasks.
Diagnosing ophthalmologic diseases in patients by way of image analysis typically requires capturing, collecting and aligning retinal images from multiple modalities, each of which conveys complementary information. This task is complicated, however, by two major algorithmic challenges: 1) inconsistent features from each modality make it difficult for algorithms to find mutual information between two modalities and 2) most data lacks labels necessary for training learning-based models. This study proposes a combined vessel segmentation and deformable registration model based on a Convolutional Neural Network (CNN) to achieve this task. The vessel segmentation network is trained without ground truth using a learning scheme based on style transfer to extract mutual patterns and transform images of different modalities into consistent representations. This network is paired with a deformable registration model, which uses content loss to help find dense correspondences between multi-modal images based on their consistent representations, while also improving segmentation results. This model is demonstrably better than other comparable models at both deformable and rigid transformation tasks.
The most important indicator of potential cardiovascular disease is blood pressure (BP), yet traditional cuff-based methods for measuring BP can lead to inaccurate measurements. These methods are, moreover, not practical for the continuous monitoring required for a personalized approach to detecting abnormal BP fluctuations. This study proposes a novel machine-learning model for estimating BP using a subject’s previous BP measurements combined with wavelet decomposition to extract features from the photoplethysmogram (PPG) signal, a simple and popular tool for non-invasive diagnosis. To process the arterial BP time series, the researchers use an exponentially weighted moving average (EMWA) and a peak direction technique to derive systolic blood pressure (SBP), diastolic blood pressure (DBP) and their corresponding trends. They then construct a predictive model based on these features using a random forest model, as well as the MIMIC dataset to analyze and compare the results with other BP estimation methods. The proposed approach demonstrates a smaller estimation error than the cuff-based standard and all other methods studied, with mean average errors for SBP and DBP equal to 3.43 and 1.73 respectively.
The Center of Mass (CoM) calculation is an important tool for evaluating a person’s balance and predicting fall risk (particularly as it applies to the effectiveness of physical rehabilitation programs). Traditional techniques for measuring the CoM position include laboratory-grade devices such as a force plate, which are expensive and inconvenient for home use. Inspired by the rapid development of vision-based techniques, this paper proposes a deep learning-based framework that uses a single depth camera to measure the CoM position. The framework estimates the horizontal CoM position of a subject using body parameters obtained from depth image data collected from multiple subjects in various postures. The proposed approach has proven to be highly accurate in estimating the CoM of existing subjects or a new subject and does not require the complicated calibration or subject identification characteristic of existing CoM techniques. It thereby creates a portable and low-cost alternative for enabling automated balance evaluation at home.
The continuing proliferation of wireless electronic devices, coupled with the promise of fifth generation mobile networks (5G) and Internet-of-Things (IoT) scale connectivity, will demand innovative design techniques and solutions on all network and device layers for both wireless and optical systems. Broadband and software-defined connectivity is at the forefront of research efforts to address these new challenges. The work presented in this paper explores the limits of current CMOS technology with the goal of achieving a true DC-100 GHz software-defined transmitter front-end, and with the maximum achievable instantaneous bandwidth. This paper presents a DC-60 GHz I/Q modulator/transmitter chip in 45 nm SOI CMOS, that can serve as a critical building block for next generation multi-standard and high-capacity wireless backhaul links. This new transmitter will address new applications, such as short-range device-to-device communications, server-to-server connectivity in data centers, and fifth generation mm-wave software-defined transceivers, while still supporting traditional mobile links and connectivity below 6 GHz.
Blood pressure (BP) is one of the most important indicator of human health. In this paper, we investigate the relationship between BP and health behavior (e.g. sleep and exercise). Using the data collected from off-the-shelf wearable devices and wireless home BP monitors, we propose a data driven personalized model to predict daily BP level and provide actionable insight into health behavior and daily BP. In the proposed machine learning mode l using Random Forest (RF), trend and periodicity features of BP time-series are extracted to improve prediction. To further enhance the performance of the prediction model, we propose RF with Feature Selection (RFFS), which performs RF-based feature selection to filter out unnecessary features. Our experimental results demonstrate that the proposed approach is robust to different individuals and has smaller prediction error than existing methods. We also validate the effectiveness of personalized recommendation of health behavior generated by RFFS model.
As 360-degree videos and virtual reality (VR) applications become popular for consumer and enterprise use cases, the desire to enable truly mobile experiences also increases. Delivering 360-degree videos and cloud/edge-based VR applications require ultra-high bandwidth and ultra-low latency , challenging to achieve with mobile networks. A common approach to reduce bandwidth is streaming only the field of view (FOV). However, extracting and transmitting the FOV in response to user head motion can add high latency, adversely affecting user experience. In this paper, we propose a predictive view generation approach, where only the predicted view is extracted (for 360-degree video) or rendered (in case of VR) and transmitted in advance, leading to a simultaneous reduction in bandwidth and latency. The view generation method is based on a deep-learning-based viewpoint prediction model we develop, which uses past head motions to predict where a user will be looking in the 360-degree view. Using a very large dataset consisting of head motion traces from over 36,000 viewers for nineteen 360-degree/VR videos, we validate the ability of our viewpoint prediction model and predictive view generation method to offer very high accuracy while simultaneously significantly reducing bandwidth.
We explore the viability of Solar-powered Road Side Units (SRSU), consisting of small cell base stations and Mobile Edge Computing (MEC) servers, and powered solely by solar panels with battery, to provide connected vehicles with a low- latency, easy-to-deploy and energy-efficient communication and edge computing infrastructure. However, SRSU may entail a high risk of power deficiency, leading to severe Quality of Service (QoS) loss due to spatial and temporal fluctuation of solar power generation. Meanwhile, the data traffic demand also varies with space and time. The mismatch between solar power generation and SRSU power consumption makes optimal use of solar power challenging. In this paper, we model the above problem with three sub-problems, the SRSU power consumption minimization problem, the temporal energy balancing problem and spatial energy balancing problem. Three algorithms are proposed to solve the above sub-problems, and they together provide a complete joint battery charging and user association control algorithm to minimize the QoS loss under delay constraint of the computing tasks. Results with a simulated urban environment using actual solar irradiance and vehicular traffic data demonstrates that the proposed solution reduces the QoS loss significantly compared to greedy approaches.
Traditional physical therapy treatment for patients with Parkinson’s disease (PD) requires regular visits with the physical therapist (PT), which may be expensive and inconvenient. In this paper, we propose a learning-based personalized treatment system to enable home-based training for PD patients. It uses the Kinect sensor to monitor the patient’s movements at home. Three physical therapy tasks with multiple difficulty levels are selected by our PT co-author to help PD patients improve balance and mobility. Criteria for each task are carefully designed such that patient’s performance can be automatically evaluated by the proposed system. Given the patient’s motion data, we propose a two-phase human action understanding algorithm TPHAU to understand the patient’s movements. To evaluate patient performance, we use Support Vector Machine to identify the patient’s error in performing the task. Therefore, the patient’s error can be reported to the PT, who can remotely supervise the patient’s performance and conformance on the training tasks. Moreover, the PT can update the tasks that the patient should perform through the cloud-based platform in a timely manner, which enables personalized treatment for the patient. To validate the proposed approach, we have collected data from PD patients in the clinic. Experiments on real patient data show that the proposed methods can accurately understand patient’s actions and identify patient’s movement error in performing the task.