Telehealth: Opportunitiesfor (Communications)Research?

UrbashiMitra Daphney Zois,GautamThatte&MarcoLevorato UniversityofSouthernCalifornia

Telehealth: Opportunitiesfor (Communications)Research?

UrbashiMitra Daphney Zois,GautamThatte&MarcoLevorato UniversityofSouthernCalifornia

whereisthecommunications?

Telehealth: Opportunitiesfor (Communications)Research?

UrbashiMitra Daphney Zois,GautamThatte&MarcoLevorato UniversityofSouthernCalifornia

whereisthecommunications? itsattheend

Objectives
TheKNOWMENetwork
Highlyinterdisciplinaryadventure Computerengineering,wirelesscommunications, machinelearning,robotics/humanmotion modeling,preventivehealth Employofftheshelfsensors Originalapplication:pediatricobesity Optimizealllayersofthesystem
Systemarchitecture Multimodalphysicalactivityrecognition Energyefficientsensorselection (human)Energyexpenditureestimation KNOWMEonrealpeople/freeliving
U.S. 2003-2006, Ages 12-19

Deployawirelessbodyareasensingnetwork

KNOWME
WhatIlearned

ActivityDetectionforHealth
biometric waveform OXI feature

GPS ACC decision

ECG ACC

Q:whotolistentoandforhowlong?

Energyconstraintsaredifferent!
NokiaN95cellphonefusioncenter optimizeenergy resourcesatsensor
AliveTechnologiesACCandECG(chest strap)andmobileinternalAccelerometer| Pulseoximeter notused CostofBluetoothmeasurements>>Costof NokiaACC BluetoothimplementationimposesTDMA!

optimizeenergy resourcesatN95 sensorsare heterogeneous traditional sensornetwork ourbody sensingnetwork

Fusioncentercapabilities
Cellphonesarenotbiometricsignalprocessing devices(yet)

FromSensorstoFeatures
Accelerometer(ACC)
sample sample sample

Electrocardiograph(ECG)

ECGperiod (=sample)

sample =averagevariance oftriaxial accelerometerfor windoweddata

Featuresextractedfrom data
Usedtodevelophypotheses intraining Comparedtomodelsin testingforactivitydetection

HeterogeneousSensorPerformance
Certainsensorscanbetterdiscriminatebetweenspecific setsofactivities
bothACCandECG OnlyACC OnlyECG

HeterogeneousSensorPerformance
Certainsensorscanbetterdiscriminatebetweenspecific setsofactivities
bothACCandECG OnlyACC OnlyECG

SignalModel
Measurementscollectedatfusioncenter
Severalfeaturesextractedfromeachsensor ModeltemporalcorrelationviaAR(1)process
measurement meanofGaussianmodeledfeature

AR(1)parameter

measurementnoise

#ofmeasurements fromkth sensor

Featuresonlyweaklycorrelated;assumeindependence
Somesensorsdiscriminatebetterincertaincases Useoffeatureselectionresultsinlowfeaturefeaturecorrelation

Afewchallengesinherenttooptimizingthesystem

Multihypothesis Testing
Discriminatebetweenmultiple hypotheses
e.g. Liedown,sit,sit&fidget,stand,stand&fidget,walk,run

featureofBluetoothimplementation:TDMAaccess CanonicalgeneralGaussianproblem
Eachactivity~multivariateGaussian
sensorsindependent

N 1 x 1

N 1 x N 1

blockdiagonal

NonlinearDecisionRegions

Decisionregionsfor bivariateGaussiansfor sixactivities

Distinct meansandcovariancematricesforeach subject |personalizedtraining

EffectofIntegerAllocations
Asallocationchanges,structureofcovariancematrix changes,e.g.for N=6

Linearalgebraicgymnastics:combinatorialinteger optimization Realvaluedvectoroptimizationwith continuous N Realtime implementationonNokiaN95

Thatte,Li,Lee,Emken,Annavaram,Narayanan,SpruijtMetz,andM, IEEETrans. OnSignalProcessing,April2011

SystemState
System State: activity performed at time k,

examplestatespace andMarkovchain 1:Sit 2:Stand 3:Run 4:Walk possible activities supported by the system probability transition matrix

POMDPSystemExecution

state switches

sensors sensing

state detector

decisionprocess
sensorselection beliefstate update

Different fromtypicalPOMDPs:dependenceonprevious state,alsocontroldoesnotaffectstate,butyourability toaccuratelyobservethestate

PerformanceMeasures
Worstcase error probability: sensors discrimination capabilities

approximatedusingpreviousmethods

Energy Cost: transmission power consumption

: communication cost from sensor to cell phone (different for each sensor)

Total Cost:

Stateestimation&Optimization
Imperfect state information requires use of belief state
Sufficient statistics, new update rule needed ML state estimate

A general partially observable, stochastic control problem Solve using DP methods/computationally expensive

ApproximationSchemes
Canshowthatcosttogofunctionis
Positivelyhomogeneous,concave,piecewiselinear, superlinear

Suggestsanapproximationscheme
MinimumIntegratedCostTimeSharingSensorSelection (MICT3S)algorithm UseapproximateDPtodeterminesolutions,,atcorner pointsofsimplex Finalsolutionviatimesharingusingbeliefestimate

Complexity:

 3 sensors: ACC Mean, ACC Variance, ECG Period 4 activities: Sit, Stand, Run, Walk
ACCmean
sit stand run walk

Simulations

ECGperiod

Statistics from real data Costs from real devices Energy efficiency # ACC Mean $ ECG Period Detection Accuracy
# ACC mean & ACC var $ ECG period

ACCvar

G.Thatteetal.,IEEETrans.onSignalProc.,April2011

Expect limited use of ECG period

NearOptimalityofMICT3S
AverageTotalCost

N=12samples L=5horizonlength

EA: every sensor same number of samples

Weightfactor

DP and MICT3S attempt to optimize average total cost Approximate algorithm (MICT3S) performance coincides with optimal (DP)

TradeoffCurves
AverageDetectionError

Energygains achievedby proposedschemes fordetection accuracyequaltoEA

AverageEnergyCost

Contributions
Derived implementable performance approximations for ML activity detection in WBANs Proposed a modified POMDP model to optimize detection in heterogeneous WBANs
Different discrimination capabilities & communication costs of sensors Limited energy budget of fusion center Imperfect state information

Derived: optimal DP algorithm + three approximate schemes Energy gains as high as 68% (versus equal allocation) with 99% accuracy and very few samples!!!
Zois,Levorato,&M,insubmission January2012

Futurechallenge
States: Physicalactivity states Contextualstates Emotionalstates ACTUATIONstates Howtohandle stateexplosion? Implicationsfor otherlargescale (PO)MDPs

theWHOLEsystem

KNOWMEArchitecture
combinehealthsensorswithmobilephone
metabolic:ACC,OXI,ECG emotional:GSR location:GPS userinitiated:SMS,Tweets, video/image/voicetags
End-to-end encryption of sensitive data 3G GSM Wi-Fi

Web Server

Internet
Check right to use system Filter noisy updates Web enable data access

Doctor

Database Server

threetierKNOWMEarchitecture

MobileClientDesign

ChallengesofSystemArchitecture
KNOWMEexecutionprioritylessthanothermobile applications(incoming/outgoingcalls) Devicemanager
maintainsidentitiesofdifferentsensors sourcesbetweendifferentsensors

Datacollector
Receives/synchronizessensordatafromdevicemanager Healthrecordscollected,buffered,senttolocalstorage, transmitterandanalyzer

KMCore elementsdeterminedbyimplemented Bluetooth/TDMAmultipleaccess

LessonsLearned
Energyconsumptionissuesare significant
N95performsallcoordination, processing,computation Employdatabuffering,adaptive sensorthrottling,dynamicsensor selection,datatransmission choices

Statedetectionisthemost expensiveactivity
Statedetectionatphoneand transmit? Compressdataanddostatedetectionatbackendserver?

Applicationstabilityanissue
Designsdoneonemulator/debugging

WiFI:compression moreexpensive

Floatingpoint:needsmartimplementationofsignalprocessingalgorithms

Multimodalsignalprocessing
Mean Amplitude
1 0.5 0 -0.5 -1 -2 0 2

Mean Amplitude

Lying

Walking
1 0.5 0 -0.5 -1 -2 0 2

Machinelearning techniques
Collected ECG Signal Normalized ECG beats Peak Detection Amplitude Scaling Phase Alignment DC bias Removal

Standard Deviation

0.4 0.3 0.2 0.1 0 -2 0 2

Standard Deviation

phase [-:] Lying

phase [-:] Walking

0.4 0.3 0.2 0.1 0 -2 0 2

phase [-:]

phase [-:]

( c ) Normalized ECG heartbeats ECG Temporal Feature Extraction Accelerometer Temporal Feature Extraction ECG Cepstral Feature Extraction Accelerometer Cepstral Feature Extraction

(b) ECG pre-processing SVM Modeling with GLDS Kernel SVM Modeling with GLDS kernel GMM Modeling GMM Modeling with HLDA Detected Activities Score Level Fusion Weighted Sum Fusion

Supportvector machines Featurefusion methods

Novelfeature design
Cepstral Timedomain

Feature Extraction Activity Modeling (a) System overview

Nonlinear methods

Li,Rozgic,Thatte,Lee,Emken,Annavaram,M,SpruijtMetz,Narayanan , IEEETransonNeuralSystemsandRehab.Eng., August2010.

Lessonslearned
Highwithinindividualvariability
Sensorplacement Location,useremotion,fitness ECGsignalsappeartobemoresensitivetorobustness issues

Withinasingleactivity,significantsignalvariation
Howtomodel?

Robustnessincreasedviamultisessiontraining

Whatgoesaround,comesaround

Whatgoesaround,comesaround
CDMA?

Whatgoesaround,comesaround
CDMA?
Significanteffortdevotedtoworkingaround Bluetooth/TDMA(implementedonphone/on sensors) Systemarchitectureissues:devicemanagement, threaddistinction,maintainingbuffersfordifferent sensors

Pros:
Fewsensors shortspreadingsequences Multiuserdetectioncomplexitycomparable tomachinelearningmethods Nodevicemanagement,threaddistinction, buffermaintenance alldoneinsignal processing
E.g.noneedtokeeptrackofnonfunctioning sensors,etc.

theBIGConclusion
TheScientificMethod
Shorttermresearchtodrivelongtermimplementation

Therearesomethingsthat areunknowable unlessyou

Buildasystem Workwithdomainexperts Deploythesystem

Significantroomfortheory, algorithmdesign, (comm theory,notyet?)

http://www.nle2ndgrade.com

theBIGConclusion
TheScientificMethod
Shorttermresearchtodrivelongtermimplementation

Therearesomethingsthat areunknowable unlessyou

Buildasystem Workwithdomainexperts Deploythesystem

Significantroomfortheory, algorithmdesign (comm theory,notyet?) itslikehavingadoctorinyourpocket

http://www.nle2ndgrade.com