Vous êtes sur la page 1sur 62


Also in this issue:
> Cybersecurity and the Future
> Mutual Dependence Demands
Mutual Sharing

JULY 2017 www.computer.org


The IEEE Computer Society is launching INTERFACE, a new communication

tool to help members engage, collaborate and stay current on CS activities.
Use INTERFACE to learn about member accomplishments and find out how
your peers are changing the world with technology.

We’re putting our professional section and student branch chapters in

the spotlight, sharing their recent activities and giving leaders a window
into how chapters around the globe meet member expectations. Plus,
INTERFACE will keep you informed on CS activities so you never miss a
meeting, career development opportunity or important industry update.

Launching this spring. Watch your email for its debut.

IEEE COMPUTER SOCIETY http://computer.org • +1 714 821 8380

Editor Manager, Editorial Content
Lee Garber Carrie Clark

Contributing Staff Senior Manager, Editorial Services

Christine Anthony, Brian Brannon, Lori Cameron, Cathy Martin, Robin Baldwin
Chris Nelson, Meghan O’Dell, Dennis Taylor, Rebecca Torres,
Bonnie Wylie Director, Products and Services
Evan Butterfield
Production & Design
Carmen Flores-Garvey, Monette Velasco, Jennie Zhu-Mai, Senior Advertising Coordinator
Mark Bartosik Debbie Sims

Circulation: ComputingEdge (ISSN 2469-7087) is published monthly by the IEEE Computer Society. IEEE Headquarters, Three Park Avenue, 17th Floor, New
York, NY 10016-5997; IEEE Computer Society Publications Office, 10662 Los Vaqueros Circle, Los Alamitos, CA 90720; voice +1 714 821 8380; fax +1 714 821 4010;
IEEE Computer Society Headquarters, 2001 L Street NW, Suite 700, Washington, DC 20036.
Postmaster: Send address changes to ComputingEdge-IEEE Membership Processing Dept., 445 Hoes Lane, Piscataway, NJ 08855. Periodicals Postage Paid at
New York, New York, and at additional mailing offices. Printed in USA.
Editorial: Unless otherwise stated, bylined articles, as well as product and service descriptions, reflect the author’s or firm’s opinion. Inclusion in ComputingEdge
does not necessarily constitute endorsement by the IEEE or the Computer Society. All submissions are subject to editing for style, clarity, and space.
Reuse Rights and Reprint Permissions: Educational or personal use of this material is permitted without fee, provided such use: 1) is not made for profit;
2) includes this notice and a full citation to the original work on the first page of the copy; and 3) does not imply IEEE endorsement of any third-party products
or services. Authors and their companies are permitted to post the accepted version of IEEE-copyrighted material on their own Web servers without permission,
provided that the IEEE copyright notice and a full citation to the original work appear on the first scree n of the posted copy. An accepted manuscript is a version
which has been revised by the author to incorporate review suggestions, but not the published version with copy-editing, proofreading, and formatting added by
IEEE. For more information, please go to: http://www.ieee .org/publications_standards/publications/rights/paperversionpolicy.html. Permission to reprint/republish
this material for commercial, advertising, or promotional purposes or for creating new collective works for resale or redistribution must be obtained from IEEE by
writing to the IEEE Intellectual Property Rights Office, 445 Hoes Lane, Piscataway, NJ 08854-4141 or pubs-permissions@ieee .org. Copyright © 2017 IEEE.
All rights reserved.
Abstracting and Library Use: Abstracting is permitted with credit to the source. Libraries are permitted to photocopy for private use of patrons, provided the per-
copy fee indicated in the code at the bottom of the first page is paid through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923.
Unsubscribe: If you no longer wish to receive this ComputingEdge mailing, please email IEEE Computer Society Customer Service at help@computer.org and
type “unsubscribe ComputingEdge” in your subject line.
IEEE prohibits discrimination, harassment, and bullying. For more information, visit www.ieee.org/web/aboutus/whatis/policies/p9-26.html.

IEEE Computer Society Magazine Editors in Chief

Computer IEEE Micro IEEE Intelligent Systems
Sumi Helal, University of Florida Lieven Eeckhout, Ghent V.S. Subrahmanian, University
University of Maryland
IEEE Software IEEE MultiMedia
Diomidis Spinellis, Athens IEEE Computer Graphics Yong Rui, Lenovo Research
University of Economics and and Applications and Technology
Business L. Miguel Encarnação, ACT, Inc.

IEEE Annals of the History

IEEE Internet Computing IEEE Pervasive Computing of Computing
M. Brian Blake, University of Maria Ebling, IBM T.J. Watson Nathan Ensmenger, Indiana
Miami Research Center University Bloomington

IT Professional Computing in Science IEEE Cloud Computing

San Murugesan, BRITE & Engineering Mazin Yousif, T-Systems
Professional Services Jim X. Chen, George Mason International
IEEE Security & Privacy
Ahmad-Reza Sadeghi, Technical
University of Darmstadt

www.computer.org/computingedge 1


IoT Quality
Visual IoT: Architectural
Control for Data Authentication and Challenges and
and Application Authorization for Opportunities; Toward
Needs the Internet a Self-Learning and
of Things Energy-Neutral IoT
6 Spotlight on Transactions:
A Haptic Compass for Navigation

7 Editor’s Note: Keeping Up with the Internet of Things

8 Software-Engineering the Internet of Things

11 IoT Blame Game


16 IoT Quality Control for Data and Application Needs


22 Continuous Authentication and Authorization for the

Internet of Things

27 Visual IoT: Architectural Challenges and

Opportunities; Toward a Self-Learning and
Energy-Neutral IoT

32 Osmotic Flow: Osmotic Computing + IoT Workflow


40 Internet of Things for Smart Cities: Interoperability

and Open Data

46 IoT: From Sports to Fashion and Everything


49 Cybersecurity and the Future


52 The Need to Help Journalists with Data and

Information Visualization


Mutual Dependence Demands Mutual Sharing

Internet of 4 Magazine Roundup
57 Computing Careers: Careers Related to the Internet of Things
Things for
Smart Cities:
and Open Data Subscribe to ComputingEdge for free at

but also generate new ques-
tions. This is the focus of “On
the Simulation of Everything,”
from IEEE Internet Computing’s
May/June 2017 issue, by Inter-
net pioneer Vinton G. Cerf.

Computing in Science &

he IEEE Computer IEEE Software
Society’s lineup of 13 The US Department of Energy’s
peer-reviewed technical Today’s abundance of process- (DOE) Exascale Computing
magazines covers cutting-edge ing power is affecting software Project is a partnership between
topics ranging from software engineering, according to “How the DOE Office of Science and
design and computer graphics Abundance Changes Soft- the US National Nuclear Security
to Internet computing and secu- ware Engineering,” from IEEE Administration. The project’s mis-
rity, from scientific applications Software’s May/June 2017 issue. sion is to transform today’s high-
and machine intelligence to By reducing the cost of failure, performance computing (HPC)
cloud migration and microchip abundance changes how devel- ecosystem via a multifaceted
design. Here are highlights from opers use computing technolo- plan. The plan’s elements include
recent issues. gies. Also, abundance changes developing mission-critical appli-
developers’ roles by moving cations of unprecedented com-
Computer their focus from the technology plexity, supporting US national
to its management. security initiatives, partnering
Competitions offer a compelling with the US HPC industry to
platform for engaging students IEEE Internet Computing develop exascale computer archi-
and professionals in devel- tectures, and collaborating with
oping new technologies and Affordable computing and mem- US software vendors to develop
skills. Computer’s July 2017 ory have enabled innovative an exascale-capable software
special issue on challenge- work in computer visualization stack suitable for industrial- and
based learning explores vari- and simulation. As these tech- academic-scale systems. This is
ous challenge-based approaches nologies model high-resolution the focus of “The Exascale Com-
to education and technical complexities, they offer new and puting Project,” from CiSE’s
innovation. sometimes unexpected answers May/June 2017 issue.

4 July 2017 Published by the IEEE Computer Society 2469-7087/17/$33.00 © 2017 IEEE
IEEE Security & Privacy IEEE Intelligent Systems transportation access will play a
growing role in this process. The
Election security isn’t just a mat- IEEE Intelligent Systems’ May/June authors of “What Can We Learn
ter of a secure system defending 2017 special issue addresses com- from Smart Urban Mobility Tech-
against attacks from an external putational advertising. It includes nologies?,” from IEEE Pervasive
adversary. It must also provide articles on topics such as whether Computing’s April–June 2017
sound evidence of an accurate out- computational advertising is a par- issue, note that these approaches
come. IEEE S&P’s May/June 2017 adigm shift for marketing, the use not only affect mobility practices
special issue on electronic vot- of verbal intent in semantic con- and user behavior but can also
ing contains articles on end-to-end textual advertising, and a three- improve urban transportation
verifiability, which can confirm phase approach for exploiting planning.
that an election has handled votes opinion mining in computational
correctly from casting to tallying. advertising. IT Professional

IEEE Cloud Computing IEEE MultiMedia IT Pro’s May/June 2017 special issue
addresses the latest developments
One of the latest developments in The author of “Multimedia Re- in mobile data analytics. This is
cloud computing is called server- search: What Is the Right an increasingly popular field that
less computing, even though Approach?,” from IEEE MultiMedia’s deals with analytics—especially
servers still handle the necessary April–June 2017 issue, asks whether big-data analytics—on resource-
processing. Serverless computing sufficient work is being done to constrained mobile devices.
can save money for simple work- figure out how to determine the
flows. But first, users must model best approaches to solving various IEEE Micro
the economic impact of their archi- types of research problems.
tecture and operation choices, Designers who are making deep-
explains the author of “Be Wary of IEEE Annals of the History of learning computing more efficient
the Economics of ‘Serverless’ Cloud Computing can’t rely solely on hardware. Incor-
Computing,” from IEEE Cloud Com- porating software-optimization tech-
puting’s March/April 2017 issue. Micro Computer Machines, estab- niques such as model compression
lished in Canada in 1971, was leads to significant power savings
IEEE Computer Graphics and among the first companies to work and performance improvement. This
Applications on a personal microcomputer. The and related matters are discussed
evolution of the company’s views in “Software-Hardware Code-
3D spatial user interface tech- on software was representative of sign for Efficient Neural Net-
nologies could make games more personal computing’s many trans- work Acceleration,” from IEEE
immersive and engaging. Although formations in its early years, accord- Micro’s March/April 2017 issue.
technologies such as stereoscopic ing to “MCM on Personal Software,”
3D displays are now available for from IEEE Annals’ January–March Computing Now
games, it’s still unclear how their 2017 issue.
use affects play and performance. The Computing Now website
The authors of “Enhancing the IEEE Pervasive Computing (computingnow.computer.org)
Gaming Experience Using 3D Spa- features up-to-the-minute com-
tial User Interface Technologies,” Smart mobility technologies puting news and blogs, along with
from CG&A’s May/June 2017 issue, help people access and exploit articles ranging from peer-reviewed
discuss how these approaches urban transportation options. New research to opinion pieces by indus-
affect gameplay. apps on the forefront of digitized try leaders.

www.computer.org/computingedge 5

A Haptic Compass drive motor that generates open-loop

torques around an axis orthogonal to
the palm of the hand by accelerating

for Navigation the internal masses (the motor’s rotor

and flywheel) in one direction or the
other. A haptic-signal feedback gen-
erator translates the user’s location
Lynette A. Jones, MIT
and orientation relative to an environ-
mental target to determine the deliv-
This installment highlighting the work published ered torque’s direction and amplitude.
For example, if the user experiences
in IEEE Computer Society journals comes from a torque to the left, then the target is
on the left; torque amplitude provides
IEEE Transactions on Haptics. information regarding the distance to
the target.
Experimental validation of the hap-
tic compass indicated that with torque
amplitudes greater than 10 mNm, us-
ers could accurately identify the initial
direction in at least 90 percent of the
test trials. Users responded faster to
the torque signal as torque amplitude
increased, averaging 8 seconds for
torques greater than 50 mNm. Signals
in the 5–15 Hz frequency range were
perceived most clearly; the orientation
of signals at lower frequencies took
longer to be perceived. Users also per-
formed best when the haptic feedback
was proportional to the angular error
between the user and the target.

Figure 1. Handheld haptic compass for use as a navigation aid. n a more realistic evaluation of the
haptic compass, the authors tasked

blindfolded subjects with finding 15
ver the past decade, in- location on the user’s body at which indoor waypoints. Not only did all the
terest has grown in using the vibration occurs conveys informa- users find the waypoints successfully,
haptic cues to aid naviga- tion to the user about the intended di- but they also considered the device
tion, both for people with rection of movement. intuitive to use with readily perceiv-
visual impairments and for people In “Development and Experimental able signals.
with normal vision walking in unfa- Validation of a Haptic Compass based
miliar environments. Many devices on Asymmetric Torque Stimuli,” Jean-
offer GPS-based audio instructions, Philippe Choinière and Clément Goss- LYNETTE A. JONES is a senior re-
but haptic cues are appealing because elin describe the design, control, and search scientist in MIT’s Department
they’re private and more practical in experimental validation of a hand- of Mechanical Engineering, and
noisy environments. Numerous wear- held haptic compass for use as a nav- editor in chief of IEEE Transactions
able devices based on vibrating motors igation aid (IEEE Trans. Haptics, vol. on Haptics. Contact her at ljones@
have been developed and evaluated as 10, no. 1, 2017, pp. 29–39). The device, mit.edu.
navigation aids. In these systems, the shown in Figure 1, comprises a direct

6 July 2017 Published by the IEEE Computer Society 2469-7087/17/$33.00 © 2017 IEEE
6 COM PUTE R PUBLISHED BY THE IEEE COMPUTER SOCIET Y 0 0 1 8 - 9 1 6 2 / 1 7/ $ 3 3 .0 0 © 2 0 1 7 I E E E

Keeping Up with the

Internet of Things

T he Internet of Things (IoT) has enabled

the design of new smart devices and has
created new capabilities in areas such as
information gathering; data analytics; automation;
and the remote access of home, office, and indus-
on a new way to efficiently execute IoT services
and applications at the network edge, to help cope
with the volume and variety of big data that IoT
devices produce.
The authors of IEEE Internet Computing’s
trial systems. July’s ComputingEdge issue exam- “Internet of Things for Smart Cities: Interoperabil-
ines many of the IoT’s exciting possibilities, as ity and Open Data,” provide a case study of Swe-
well as the challenges it faces. den’s GreenIoT platform.
Transforming IoT infrastructures into a general- “IoT: From Sports to Fashion and Everything
purpose computing fabric could change how mod- In-Between,” from IEEE Pervasive Computing,
ern computation interfaces with our environment, looks at how pervasive computing and the IoT
according to IEEE Software’s “Software-Engineering now affect many diverse fields.
the Internet of Things.” This issue also includes articles on topics
Early identification, mitigation, and preven- other than the IoT:
tion of problems with IoT applications could limit
liability issues for system designers, notes Com- • Computer’s “Cybersecurity and the Future” dis-
puter’s “The IoT Blame Game.” cusses the challenges security will have to meet
The authors of IEEE Intelligent Systems’s “IoT as technology changes in the coming years.
Quality Control for Data and Application Needs” • As news migrates to mobile phones, media
discuss some of the challenges of and solutions to companies are turning to data visualization
evaluating the quality of data in IoT systems. to whet readers’ appetites for stories they can
“Continuous Authentication and Authoriza- read at length on their home or work comput-
tion for the Internet of Things,” from IEEE Internet ers, according to “The Need to Help Journalists
Computing, addresses the development of these with Data and Information Visualization,” from
security techniques for the IoT. IEEE Computer Graphics and Applications.
IEEE Micro’s “Visual IoT: Architectural Chal- • To benefit fully from today’s technology, we
lenges and Opportunities; Toward a Self-Learning must complement it with an inexpensive sys-
and Energy-Neutral IoT,” includes invited position tem that protects confidentiality and reports on
papers about these two potentially important IoT the volume, pattern, and character of incom-
approaches. ing digital attacks, contend the authors of
“Osmotic Flow: Osmotic Computing + IoT IEEE Security & Privacy’s “Mutual Dependence
Workflow,” from IEEE Cloud Computing, focuses Demands Mutual Sharing.”

2469-7087/17/$33.00 © 2017 IEEE Published by the IEEE Computer Society July 2017 7
FROM THE EDITOR Editor in Chief: Diomidis Spinellis
Athens University of Economics
and Business, dds@computer.org

the Internet of Things
Diomidis Spinellis

ENIAC WAS BUILT during the Sec- its units in the manner required for ing fabric can similarly change how
ond World War, from 1943 to 1945. solving a particular problem, such modern computation interfaces with
Many consider it the fi rst electronic, as generating sine and cosine tables our environment.
general-purpose, large-scale digital or computing artillery trajectories
computer. Picture it as a room en- and shock wave reflections. As you A Maze of Problems …
compassing 36 racks, three printer can imagine, such programming was and Ways Out
panels, a card reader, and a card time-consuming and error-prone. No doubt, a paradigm shift from
punch. Each rack used hundreds of Then, in 1948 a remarkable thing balkanized IoT applications to an
vacuum tubes to perform a specific happened. Inspired by the design of integrated infrastructure in which
function. Many racks acted as accu- EDVAC (Electronic Discrete Vari- individual IoT nodes are fi rst-class
mulators: they received pulses corre- able Automatic Computer), dis- citizens raises formidable chal-
sponding to the digits of a decimal cussed over summer school lectures lenges. Start with requirements. In
number and increased accordingly at the University of Pennsylvania’s a standalone IoT application, it can
the number stored in them. Oth- Moore School, ENIAC’s designers be easy to satisfy a major functional
ers were more specialized; a 1944 realized they could wire it in a revo- requirement—say, home security—
floor plan has racks labeled multi- lutionary way. The wiring wouldn’t by controlling the balance of di-
plier, partial product, square rooter, solve a particular numeric program. verse nonfunctional requirements,
denominator, multiplicand, and so Instead, the designers would repur- such as performance, reliability,
on. Three function table racks, ini- pose some of ENIAC’s accumula- and usability. However, when mul-
tialized through switches, could be tors so that it would read instruc- tiple IoT nodes and applications get
carted around on wheels. tions prescribing what actions to integrated, diverse requirements
Today we build, connect, and con- perform from its numeric function will interfere with each other (what
figure most Internet of Things (IoT) tables. Think of this as building a happens when a burglar triggers a
systems by linking together their sen- command interpreter by assembling fi re alarm?), requiring difficult pri-
sor, actuator, and computing nodes together already existing discrete oritizations and multicriteria deci-
through cloud infrastructures, mo- electronic components. Thus, the sion making. Given the fluid nature
bile apps, and the sharing of security new wiring transformed ENIAC of IoT systems, many decisions we
credentials. Similarly, ENIAC was into a versatile stored-program make today during requirements’
programmed by setting up function computer. Rewiring IoT infrastruc- elicitation might need to be obtained
tables and switches and connecting tures into a general-purpose comput- dynamically as the systems operate.

IEEE Software To be the best source of reliable, useful, peer-reviewed information for leading software practitioners—
Mission Statement the developers and managers who want to keep up with rapid technology change.

84 I E E E S O July
F T W2017
ARE | P U B L I S H E D B Y T H E I E E E CPublished
C I E Computer
TY Society 2469-7087/17/$33.00
0740-7459 / 1 7 / $ 3 3 . 0 0 © 2 0 1 7©I E2017
PURPOSE: The IEEE Computer Society is
the world’s largest association of computing
professionals and is the leading provider of
technical information in the field. Visit our

website at www.computer.org.
OMBUDSMAN: Email ombudsman@

Next Board Meeting: 12–13 November 2017,
Designing and constructing in- Then, consider configuration Phoenix, AZ, USA

tegrated IoT systems will also be management, which will entail ac- Lead Editor:COMMITTEE
EXECUTIVE Brian Brannon,
tough. Standardization in the IoT commodating diverse stakeholders bbrannon@computer.org
President: Jean-Luc Gaudiot
President-Elect: Hironori Kasahara; Past
Content Editor: Dennis Taylor
area is still in its infancy, requiring and dynamically changing systems. President: Roger U. Fujii; Secretary: Forrest
Shull; Editors: Lee Garber,
VP, Treasurer: Meghan
David Lomet;O’Dell,
complex adaptation layers. Require- In DevOps settings, we’re already
VP, Rebecca Torres
Publications: Gregory T. Byrd; VP, Member
ments associated with control, pri- facing tricky problems when a ven- & Geographic Activities: Cecilia Metra; VP,
Publications Coordinator:
vacy, and reliability might dictate dor’s configuration clashes with lo- Professional & Educational Activities: Andy T.
Chen; VP, Standards Activities: Jon Rosdahl;
design decisions that will be at odds cally implemented changes. This Lead
VP, Designer:
Technical Jennie Zhu-Mai
& Conference Activities: Hausi
with IoT nodes’ processing capacity, problem will only multiply when A. Müller; 2017–2018 IEEE Director & Delegate
Production Editor: Monette Velasco
Division VIII: Dejan S. Milojičić; 2016–2017 IEEE
power budget, bandwidth, and ubiq- systems with multiple vendors evolve Webmaster:
Director Brandi
& Delegate OrtegaV: Harold Javid; 2017
uity. The rewired ENIAC’s 60 “or- over time with configurations be- IEEE Director-Elect
Multimedia & Delegate
Editor: Division V-Elect:
Erica Hardison
John W. Walz
der codes,” which we today would ing set up by unspecialized and un- Illustrators: Annie Jiu, Robert Stack,
call its instruction set, required two trained users. Unless the state of the BOARD
and AlexOFTorres
Term Expiring 2017: Alfredo Benso, Sy-Yen Kuo,
decades of innovations in program- art improves dramatically, we might Cover
Ming Artist:
C. Lin, PeterLombardi,
Fabrizio Bollinger Hausi A. Müller,
ming languages, operating systems, face the choice between unrealized Director,
Dimitrios Products
Serpanos, & Services:
Forrest J. Shull
EvanExpiring 2018: Ann DeMarle, Fred Douglis,
and databases to provide us with the IoT promises and a mess of conflict- Vladimir Getov, Bruce M. McMillin, Cecilia Metra,
tools and abstractions we now rou- ing, incompatible configurations that Senior
Kunio Manager,
Uchiyama, Editorial
Stefano ZaneroServices:
Term Baldwin2019: Saurabh Bagchi, Leila De
tinely use to build computing appli- will make the 1990s DLL (dynamic Floriani, David S. Ebert, Jill I. Gostin, William
Acting Editorial Content Manager:
cations. Progress of a similar ambi- linked library) hell appear like heav- Gropp, Sumi Helal, Avi Mendelson
Carrie Clark
tion and scale might well be required enly peace. On the other hand, the EXECUTIVE STAFF
Senior Business Development Manager:
to truly harness the IoT. capabilities of modern decentralized Executive Director: Angela R. Burgess; Director,
Sandra Brown
Governance & Associate Executive Director: Anne
IoT nodes’ long-term mainte- configuration management systems, Senior
Marie Advertising
Kelly; Coordinators:
Director, Finance & Accounting:
nance will be another nightmare. container technologies, and package Marian
Sunny Anderson,
Hwang; manderson@computer.org
Director, Information Technology
& Services: Sumit Kacker; Director, Membership
Many devices will be embedded managers might offer us the building Debbie Sims, dsims@computer.org
Development: Eric Berkowitz; Director, Products
into buildings, streetlights, bridges, blocks of a possible solution. & Services: Evan M. Butterfield; Director, Sales &
C S P UChris
Marketing: B L I Jensen
cars, appliances, and other places Nailing down the quality of IoT-
for Publications), Alfredo Benso,
with lifetimes at least an order lon- based systems will also be an ar- COMPUTER OFFICES
Irena Bojanova,
Washington, D.C.:Evan
2001Butterfield, Robert
L St., Ste. 700,
ger than the typical PC, smartphone, duous, long-term task. Start with Dupuis, David
Washington, D.C.Ebert, Davide Falessi, Vladimir
or server. The IoT nodes will re- security. By definition, IoT sys- Getov,+1
Phone: Jose
371 0101 •Forrest
Fax: +1Shull,
202 George
728 9614
Email: hq.ofc@computer.org
quire regular corrective and per- tems will be interconnected and of- Los Alamitos: 10662 Los Vaqueros Circle, Los
fective maintenance, even as their fer access to the physical world—a Alamitos,
C S MCA AG90720
A Z I•NPhone:
E O +1 P E714 821 I8380
Email: help@computer.org
vendors inevitably switch priorities dream come true for cyberwarriors, COMMITTEE
Membership & Publication Orders
or go out of business over time. We spooks, and cyberterrorists. Today, George
Phone: +1Thiruvathukal
800 272 6657(Chair),
• Fax: +1Gul714
821 4641 •
must therefore come up with ways to we’re witnessing the edifice’s cracks M. Brian
Email: Blake, Jim X. Chen, Maria Ebling,
Asia/Pacific: Watanabe
Lieven Eeckhout, Building,
Miguel 1-4-2 Minami-
Encarnação, Nathan
smooth the handover of IoT devices as rogues take over IoT devices to Aoyama, Minato-ku, Tokyo 107-0062, Japan •
Ensmenger, Sumi Helal, San Murugesan, Yong
between vendors. launch distributed denial-of-service Phone: +81 3 3408 3118 • Fax: +81 3 3408 3553 •
Rui, Ahmad-Reza Sadeghi, Diomidis Spinellis,
Email: tokyo.ofc@computer.org
Also, maintaining a system made attacks; tomorrow’s attacks might VS Subrahmanian, Mazin Yousif
up of hundreds (or millions) of diverse be deadly. President
Editorial: &AllCEO:
Karen are subject toPresident-
Bartleson; editing for
embedded IoT nodes is a completely Complaints from people unable clarity,
Elect: style, Jefferies;
James and space.Past
Unless otherwise Barry
President: stated,L.
articles and departments, as well as
Shoop; Secretary: William Walsh; Treasurer: product and service
different ballgame than looking after to control their IoT-enabled fridges descriptions, reflect the author’s or firm’s opinion.
John W. Walz; Director & President, IEEE-USA:
hint that usability will be another Inclusion in IEEE Software does not necessarily constitute
a monolithic cloud application and its Karen Pedersen;
endorsement Director
by IEEE or the &IEEE
systems software. By now, we have potent source of ridicule and prob- Association: Forrest Don Wright; Director & VP,
To Submit: Access the IEEE Computer Society’s
Educational Activities: S.K. Ramesh; Director &
some limited experience in such tasks lems. Furthermore, the physical-world Web-based system, ScholarOne, at http://
VP, Membership and Geographic Activities: Mary
mc.manuscriptcentral.com/sw-cs. Be sure to select the
through component-based ecosys- interfaces of IoT nodes mean that Ellen
manuscript Director
type when & VP, Publication
submitting. ArticlesServices
must be
and Products:
original and notSamir 4,700 words Director
exceedEl-Ghazaly; & VP,
including figures
tems, such as Node.js and microser- reliability failures, of which current and tables,Activities:
Technical which countMarina
for 200 Ruggieri;
words each.Director
vice architectures. With IoT mainte- software has too many, can have & Delegate Division V: Harold Javid; Director &
IEEE prohibits discrimination, harassment and bullying:
For moreDivision VIII:
visitDejan S. Milojičić
nance, we must apply and extend this much more serious repercussions information, www.ieee.org
knowledge to devices, longer times- than the inconvenience of an odd
cales, and a much larger scope. lost file or application crash.
revised 31 May 2017

0740-7459/17/$33.00 © 2017 IEEE JANUARY/FEBRUARY 2017 | IEEE SOFTWARE 59

For detailed information on submitting After four years of dedicated service, Ipek Ozkaya handed the baton of the
articles, write for our editorial guidelines IEEE Software advisory board chair to Rafael Prikladnicki, whom our readers

(software@computer.org) or access
already know as the editor of the Voice of Evidence column. Ipek will continue
to serve as associate editor of the magazine’s departments, taking over from
LETTERS TO THE E D I T O R Richard Selby, who staffed this post over the past two years. Also, after deliver-
Send letters to ing two years of phenomenal growth in IEEE Software’s social media presence,
Editor, IEEE Software Alexander Serebrenik passed on the corresponding management responsibility
10662 Los Vaqueros Circle to his team member Damian Andrew Tamburri.

Los Alamitos, CA 90720 IEEE Software advertises new volunteer openings through social media and
the SEWORLD mailing list (www.sigsoft.org/resources/seworld.html). Keep an
Please provide an email address eye out if you want to join our team!
or daytime phone number with your letter.


www.computer.org/software/subscribe Finally, cooperating to achieve with multiple stakeholders, such as
the best overall performance of an cities, private residences, large office
IoT system, when each device might buildings, and mobility applications.
be selfishly trying to maximize its Instead, market-based mechanisms
Please specify IEEE Software. own, will also be tricky. We’ve suc- should probably be introduced as a
cessfully solved a similar problem in way to reduce the friction between
MEMBERSHIP the case of Internet bandwidth al- technology, social interactions, and
location, through the design of the the physical world.
member.services@ieee.org. TCP protocol. Similarly ingenious

MI SSING approaches might well be required
OR DA MAGED C OP I E S for the IoT. lighted by budget and
help@computer.org. Inevitably, the problems I out- schedule overruns followed
lined will feed into an engineering- by unreliable operation,
Find OF
R EP RINTS outAmore
RT I CL E S management challenge: coordinating ENIAC’s birth was anything but
For price information or to order reprints, multiple stakeholders with confl ict- auspicious. However, thanks to the
and get involved:
email software@computer.org
ing interests. One option might in- perseverance, ingenuity, and open-
or fax +1 714 821 4010.
cybersecurity.ieee.org volve restricting the setup of IoT ness of the people behind it, it be-
R EP RINT PERMISS I O N systems to walled gardens with came a defi ning milestone for mod-
To obtain permission to reprint an article, strictly defi ned standards, processes, ern computing. With hard work and
contact the Intellectual Property Rights and compliance testing. This ap- some luck, the IoT can usher in simi-
Office at copyrights@ieee.org. proach might work in a simple con- lar changes to how we interact with
trolled environment, such as a small- the physical world.
business building or plant. However,
it will rob us of the innovation that
open environments can spur and will
likely severely limit IoT applications
and their impact. Read your subscriptions
Promoting Sustainable Forestry
So, unless a winner-takes-all sce- This article originally appeared
through the myCS in
IEEE Software, publications portal
vol. 34, no. at
1, 2017.
nario materializes, a closed system
will be unsuitable for environments

6 I E E E S O F Computing
T W A R E | Edge
W W W. C O M P U T E R . O R G / S O F T W A R E | @ I E E E S O F T WA R E July 2017
THE IOT CONNECTION Google; roywant@google.com

The IoT Blame Game

Jeffrey Voas, IEEE Fellow
Phillip A. Laplante, Pennsylvania State University

Applications in the Internet of Things offer many

benefits and risks. Poor design, unplanned systems, and most forms of automa-
interconnections, and human adversaries raise tion. In short, the IoT will impact ev-
eryone in many ways.
the stakes and liabilities for system designers. With such great societal impact,
it’s imperative that IoT-based sys-
Early identification, mitigation, and prevention tems are trustworthy. Here, “trust-
worthy” means that a system should
of these threats can help limit liability issues. be secure and reliable, and it should
possess many other attributes asso-
ciated with quality.1,2 In addition,
privacy in such systems is of particular importance be-
FROM THE EDITOR cause they’ll likely generate large amounts of data due
This edition of the IoT Connection considers the legal ram- to their sensing and monitoring capabilities.3 Therefore,
ifications of faulty Internet of Things (IoT) devices, their as- techniques, tools, and methods to mitigate diverse trust
sociated infrastructure, and the heterogeneous patchwork challenges are needed before such systems can safely
of systems that will become the global IoT. As we begin manage daily life.
to rely on this ubiquitous network for critical tasks, a fault IoT systems built from “things” are increasing in diver-
might result in serious consequences. The authors discuss sity, scale, and number. But as they move from laboratory
why the IoT will challenge our legal system to do the right
proofs of concept to fielded applications, new and unfore-
thing when lawsuits arise. —Roy Want
seen risks are likely to emerge. Possibly the most serious

of these issues surrounds cyber-physical systems that con-
he Internet of Things (IoT) is expected to drive a trol life-critical applications.
technological revolution because the easy con- Liability matters quickly manifest because of the po-
nection of and communication among “things” tential for unplanned interactions between critical and
will allow engineers to quickly build large num- noncritical things. Further, the heterogeneity and lack
bers of intelligent networks and associated infrastructure. of ownership and control of many of the things in a spe-
The IoT is expected to impact everything from food pro- cifically purposed network of things (NoT)3 can exacer-
duction and healthcare to transportation, communication bate the problem. Ultimately, parties involved in a failed

2469-7087/17/$33.00 © 2017 IEEE Published by the IEEE Computer Society July 2017 11
C O M P U T E R 0 0 1 8 - 9 1 6 2 / 1 7/ $ 3 3 .0 0 © 2 0 1 7 I E E E PUBLISHED BY THE IEEE COMPUTER SOCIET Y JUNE 2017 69

system will start to assign blame, in- transmitted over wired and wireless vulnerabilities in certain smart power
formally, then move on to public plead- communications to an aggregation meters, and in older super visory control
ings, and, ultimately, the matter will point, where processors apply various and data acquisition (SCADA) systems
be dealt with in legal proceedings—we algorithms to create output signals, that control many of the critical infra-
call this the “blame game.” The blame which are then sent to other receptors structure systems around the world.
game and liability go hand in hand. that facilitate decisions or actuators. SCADA’s advantage—that it can re-
Because conducting a root-cause Each one of these interactions pres- motely control large operations from
great distances, such as an offshore
oil facility—is also its downside: the
communications that enable remote
control and monitoring can be com-
It can be very difficult to properly attribute promised. More recently, certain
blame when a system is in contact with hundreds traffic-control sensors and network-
of leased third-party products and services connected smart LED lightbulbs were
and their interactions. shown to be vulnerable to hacking.
Such vulnerabilities can be found in
any of the ecosystems in the aforemen-
analysis of failure in these kinds of ents a failure point that could result tioned container transport example:
systems is complex, we explore some in catastrophe. automotive systems, railway trans-
of the more interesting challenges For example, consider an applica- portation systems, and even in-flight
and how they relate to the matter of tion that provides seamless tracking systems. In fact, vulnerabilities can be
liability. We also dismiss the common of a container of critical medical sup- found and exploited anywhere, includ-
belief that standards, for any aspect of plies (or even transplant organs) from ing in-home appliances.4
a system composed of things, can take a large hospital to a rural hospital far
a one-size-fits-all approach, because away. The container leaves the large IDENTIFYING THE SUSPECTS
“all” can’t be well defined or bounded. hospital and is transferred to an am- To mitigate this problem, IoT app de-
bulance, which travels the roadways to velopers must first answer the follow-
SYSTEMS AND an airport. The container is then trans- ing question: Will we own and control
VULNERABILITIES ferred onto an airplane, and then, upon all of the assets (things, clouds, data,
We consider critical systems to be landing, to a train, then to another sources of data, communication chan-
those involving infrastructure that ambulance, and it finally arrives at the nels, software, and so on) that com-
touches or supports human life in a rural hospital. During its travels, the prise our NoT? If not, we’re in a com-
way that could cause harm in the event container has to cross through many promised position from the outset. If
of malfunction. A short list of critical different IoT ecosystems, complying we answered yes, we’re in better shape
systems includes those in with different standards (highway, than most.
airport, railway, hospital) and per- Let’s look at a short list of the risks
› telecommunications, forming handoffs through a variety of we face by answering no.
› the water supply, communications links (including in
› electrical power generation and the air, where communication is usu- › Leased data. This is data that
distribution, ally forbidden). Imagine the engineer- comes from suppliers at the
› road transportation, ing problems in building this applica- time of their choosing and with
› railway transportation, tion such that patients’ lives aren’t put the integrity of their choosing.
› air transportation, at risk, and also consider the number We can try to SLA (service-level
› public safety services, and of things that affect the application agreement) our way into a better
› healthcare.4 that might be difficult to trust. agreement here, but good luck
There’s another problem with these enforcing it. This is an imme-
In any of these applications, certain complex, multi-domain systems. Con- diate supply-chain concern. It’s
types of failures could lead to serious nectivity through handheld devices, also a business concern—what if
injury or fatality. smart homes, smart cars, and wireless- our competitors lease the same
Applications that rely on net- enabled devices increases the attack data from the same supplier?
works of connected things will re- surface for hackers to exploit. In fact, Are all being treated fairly?
ceive inputs from sensors, cameras, vulnerabilities in critical infrastruc- › Faulty interfaces. Interfaces are
various other devices, and even so- ture systems have already been re- at the endpoints of the “veins”
cial media feeds. These inputs can be ported. For example, there are known and “arteries” of an NoT. We’ll

12 Computing Edge July 2017

likely rely on a wireless ser- when something goes wrong. Leased involve “fault injection dynamic
vice provider—is that provider data from a third-party supplier methods that simulate the effects
trustworthy? might instantly enter into a network that real faults will most likely have
› Defective things. This aspect in- in near-zero time, and then the next as opposed to simulating the faults
cludes every third-party thing. If set of data enters similarly. Who’s themselves as a means of quantify-
we didn’t create that thing, we’re keeping track of these near-instanta- ing the risks created by the software
subjected to whatever the third neous data transactions? If no one, no component of a system.”6 But until
party’s limited warranty says. attribution of blame is possible. The we can accurately and scientifically
And it might not be in our favor point here is that there’s a certain dis- measure these risks, we likely won’t
or even useful. posability attribute to the things and have a means for probabilistically
› Faulty or subpar architecture. Did services that might be used, tempo- and mathematically bounding and
we architect your network, or rarily, in an NoT, making attributions quantifying liability.
did we contract that task out? of liability difficult and not timely
Were the best things used in enough to pursue. VETTING AND
implementing the architecture? Liability in an NoT is characterized CERTIFICATION
Were security and privacy even by two questions: In a previous work, Jeffrey Voas de-
considered in that architecture? scribed three generic areas of reducing
Did we have enough time to do › Who’s liable when a thing fails? the risk of failures (and hence liabil-
an economic tradeoff analysis › What’s the probability that a ity): process, personnel, and product
for when to select more expen- problem event will occur? In certification.8
sive things for higher integrity other words, what is the risk
and when to use lower-quality associated with using a thing?6 People. Although never fail-proof,
components? We can easily the risk of failure for any kind of en-
over-engineer a solution here. For systems of things, we can gineered product can be significantly
› Data tampering and integrity. Data amend these questions to reduced by having the best people
is the “blood” that flows through work on it. High-quality engineers can
a network. How secure is our › Who’s liable when the system be characterized by their educational
data from malicious tampering, fails? background and experience. Some
delay, or theft? › What’s the possibility of system have also argued that for safety-critical
› Expected operational usage. Do we failure? systems in the IoT era, professional
have a good idea of the environ- licensure is an important consider-
ment that our NoT will operate Let’s start with the first question. ation.4 Other ways of ensuring excel-
in? Did we design our network For a homegrown, non-interconnected lence in engineering personnel include
for that or was this task hired system, the answer has to be the de- certifications, ongoing training, and
out? Getting this wrong will veloper. But for systems that are con- observance of professional codes of
almost certainly cause problems nected to other systems locally and ethics. IEEE and ACM are updating
from the outset of deployment. through the Internet, the answer be- their codes of ethics so that they’re
comes more difficult. According to a appropriate for the complexities of IoT
This short list gives a sense of how lawyer specializing in the IoT and In- systems. The problem here is that we
difficult it can be to properly attribute ternet liability: “In case of (planned) don’t have a consistent understanding
blame when a system is in contact with interconnected technologies, when of what makes a computer scientist or
hundreds of leased third-party prod- there is a ‘malfunctioning thing’ it is other engineering graduate an “IoT en-
ucts and services and their interactions. difficult to determine the perimeter gineer.” Therefore, certifying IoT engi-
of the liability of each supplier. The is- neers is currently impossible.
NEED A LAWYER? sue is even more complex for artificial
Liability frameworks for the afore- intelligence systems involving a mas- Process. Reducing the risk of failure
mentioned issues (and others) should sive amount of collected data so that it requires a comprehensive approach
be jointly created by governments, might be quite hard to determine the involving selecting the correct engi-
industry, risk analysts, and insur- reason why the system made a specific neering lifecycle model and associated
ers.5 But that approach is based on a decision at a specific time.”7 processes, embracing process discipline
gross assumption: that these things The answer to the second question and standards compliance, understand-
and the networks they populate will is very difficult to determine. A pow- ing complex interactions, and select-
have lifespans long enough to sort out erful technique for determining the ing the right tools for use throughout
the legal challenges that might arise risks of a system-level failure would the system’s lifecycle. These activities

www.computer.org/computingedge 13
JUNE 2017 71

describe those used by the highest-level increased threat vectors, new vulnera- of auditing and logging is that these
software professionals. In fact, we bilities, and risks. processes offer the ability to increase
should insist on appropriate certifica- Consider, for example, this sce- reliability and resilience without
tion and licensure for electrical, com- nario: hacked refrigerator software requiring major changes to the ar-
puter, control systems, and software interacts with an app on a woman’s chitectures of specifically purposed
engineers who build critical infrastruc- smartphone, installing a security ex- NoTs. Ultimately, through increased
ture systems. Depending on the domain ploit that can be propagated to other auditing, IoT systems will continue to
(power, nuclear, or civil), other licensed applications with which her phone improve in terms of both security and
engineers will also need to be involved. interacts. The woman enters her reliability. End users will benefit from
However, presuming that a great pro- automobile and her phone interacts improved operational transparency,
cess results in a great product is overly with the vehicle’s operator interface empowering them to identify compo-
simplistic and very risky. software, which downloads the new nents that can be used together, thus
software, including the defect. Unfor- improving IoT systems’ utility.
Product. The third way to reduce the tunately, the software defect causes Of course, we don’t know if the av-
risk of failure is through product cer- an interaction problem (for example, erage consumer can understand the
tification. This approach assesses the a deadlock) that leads to a failure in results from such an auditing and
things within the IoT to determine the software-controlled safety system logging process. Perhaps the goal of
their quality. Here, rigorous, repeat- during a crash, leading to injury. The this activity would be more akin to
able, and reproducible assessment potential for this chain of events to creating a dashboard with the most
technologies are needed to establish play out demonstrates why interoper- relevant and mitigatable information
a thing’s trustworthiness. If inferior ability in IoT technologies is so chal- from a variety of tools. Although con-
assessment technologies are em- lenging regarding identifying and sumers aren’t all computer scientists
ployed, we might quickly return to mitigating risk, and assigning blame or electrical engineers, they do un-
“the quagmire created by process and when something goes wrong. derstand the importance of antivirus
personnel certification: not knowing Another way in which interactions software. They buy and install it, and
how good or bad the behavior of the in the IoT network introduces murky they’re comfortable doing so. Ideally,
software will be.”9 Product certifica- legal issues is when different compo- the same could happen with IoT audit-
tion has long been the holy grail of nents in the application reside in loca- ing and logging.
certification—and it remains elusive, tions with different jurisdictional law
except in cases where time-to-market with respect to privacy and data own-

and costs aren’t as important (for ex- ership. This heterogeneous, grand- he owner of an NoT will have to
ample, safety-critical systems under scale, distributed nature of IoT-based take some level of responsibility
government regulation). systems will require a corresponding of the risks of the things and ser-
Current certification approaches legal framework that can adequately vices in the system. Fortunately, there
to things within the IoT are, at best, take into account scalability, vertical- are a few commercial products avail-
mired in quicksand. We can expend a ity, ubiquity, and interoperability.9 able that attempt to help consumers
lot of time and effort on certification limit liability for their NoTs, including
with questionable benefits. This situa- KEEPING WATCH Dojo Labs (www.dojo-labs.com), Bit-
tion is no different than the difficulties Most consumers will hope that the IoT defender BOX (www.bitdefender.com
encountered in vetting mobile apps or offers reasonable levels of security, /box), CUJO (www.getcujo.com), Nor-
any other component that is a minis- reliability, and privacy. But how can ton Core (us.norton.com/core), and
cule part of total system functionality. these assurances be given without pre- Internet of Things Armor (www.iot
These three Ps outline the basics of the scriptive architectures and certified or armor.net). We hope this marketplace
blame game: we can accuse the people, verified things? continues to grow.
products, or process, or some combina- Possibly the best approach for con-
tion of the three. sumers would be to securely audit and
log their internal and external opera- REFERENCES
CHAIN OF CUSTODY tions and data interactions. The pres- 1. J. Voas and G. Hurlburt, “Third Party
Interactions (both planned and un- ence of an auditing system that can Software’s Trust Quagmire,” Com-
planned) between critical and non- operate independent of any IoT vendor puter, vol. 48, no. 12, 2015, pp. 80–87.
critical systems create significant of third-party things will foster in- 2. C. Kolias et al., “Learning Internet of
problems of risk and liability. These creased vendor interoperability as well Things Security ‘Hands-On,’” IEEE
interacting, dynamic, cross-domain as an acceptance of standards among Security & Privacy, vol. 14, no. 1, 2016,
ecosystems create the potential for IoT technologies. Another advantage pp. 37–46.

14 Computing Edge July 2017

3. J. Voas, Networks of ‘Things,’ NIST
Special Publication 800-183, July
2016; nvlpubs.nist.gov/nistpubs
-183.pdf. Recognizing Excellence in High-Performance Computing
4. P.A. Laplante, “IEEE Insight, Critical
Infrastructure and the IoT: A Licens- Nominations are Solicited for the
ing Perspective,” IEEE-USA InSight,
18 Sept. 2015; insight.ieeeusa.org SEYMOUR CRAY
5. M. Chui, M. Löffler, and R. Roberts,
“The Internet of Things,” McKinsey & KEN KENNEDY AWARDS
Q., March 2010; www.mckinsey
6. J. Voas et al., “A ‘Crystal Ball’ for
Software Liability,” Computer, vol. 30,
no. 6, 1997, pp. 29–36
7. G. Coraggio, “The Internet of Things
Deadline: 1 July 2017
and its Legal Dilemmas,” VC Experts All nomination details available at awards.computer.org
Blog, 15 Dec. 2016; blog.vcexperts
8. J.M. Voas, “Limited Software War- The Seymour Cray Award is awarded to recognize innova-
ranties,” Proc. 7th IEEE Int’l Conf. tive contributions to high-performance computing systems
Workshop Eng. Computer-Based Sys- that best exemplify the creative spirit demonstrated by Sey-
mour Cray. The award consists of a crystal memento and
tems (ECBS 00), 2000; doi: 10.1109 honorarium of US$10,000.
9. Z. Yan, P.Z. Zheng, and A.V. Athana-
sios, “A Survey on Trust Management Are Enemy Hackers Slipping
for Internet of Things,” J. Network and through Your Team’s Defenses?
Computer Applications, vol. 42, 2014, The award, which consists of a certificate and a US$2,000
pp. 120–134. honorarium, is presented annually to an individual for
Protect Your Organization from Hackers
“an outstanding contribution in the application of high-
by Thinking Like Them
performance computers using innovative approaches.”

Take Our E-Learning Courses

in the Art of Hacking
Contact him at j.voas@ieee.org. ACM/IEEE-CS KEN KENNEDY AWARD
A certificate and US$5,000 honorarium are awarded jointly
byYou andand
the ACM your
the staff can take
IEEE Computer theseforcourses
Society outstand- where you are
PHILLIP A. LAPLANTE is a professor andcontributions
ing at your own pace, getting
to programmability or hands-on,
productivity inreal-world training
of software and systems engineer- high-performance
you cantogether
put towith significant
work immediately.
community service or mentoring contributions.
ing at Pennsylvania State University
and an IEEE Fellow. Contact him at

Read your subscriptions

This article originally appeared
through the myCS in
Computer, vol. 50, no. 6,portal
publications 2017. at

www.computer.org/computingedge 15
JUNE 2017 73
Editor: Amit Sheth, Kno.e.sis at Wright State University, amit@knoesis.org

IoT Quality Control for Data

and Application Needs
Tanvi Banerjee and Amit Sheth, Kno.e.sis Center at Wright State University

ith the rapid growth of sensors and devices in the IoT systems determine the choice of imple-
that communicate—that is, the Internet of mentation design, as well as analysis algorithms to
achieve the optimal quality of service.2 Specifically,
Things (IoT)—smart devices have permeated ev-
the lower OSI layers have been extensively investi-
ery facet of modern life. These IoT devices are gated in several studies to extract and transmit the
within our bodies, on our bodies, in the environ- raw sensor data from IoT devices through proto-
ment both inside and outside our homes, observ- cols such as MAC or IEEE 802.3.2 However, the
ing our behavior patterns on a day-to-day basis, higher layers often get overlooked, especially from
and assisting in production systems and surveil- the perspective of the target domain. Using two
lance. Figure 1 highlights some of the more popu- scenarios, we will show how data quality can be
lar IoT applications in the world. evaluated contextually and how the different OSI
However, with these sensors’ ubiquity and per- layers are affected by the specific user needs of the
vasiveness comes vast amounts of data that need system.
to be processed and analyzed to extract meaning- Before we get into the use cases, we need to un-
ful or actionable information from the data for rec- derstand data quality, “the degree to which a set of
ommending appropriate changes in the real world. inherent characteristics fulfills the requirements.”3
This requires using not only semantic approaches,1 Within quality, we have two categories: specifica-
but also data streamlining to ensure that the de- tion and conformance quality.4 Specification qual-
cisions made are not erroneous. Moreover, due to ity refers to how well a device matches with other
the sheer volume of the data from these IoT de- similar devices in that domain. Conformance qual-
vices, any errors from user entry, data corruption, ity looks at the “correctness” or the veracity of the
data accumulation, data integration, or data pro- readings from the device. We add one more quality
cessing can snowball, causing massive errors that control that needs to be examined in this setting:
can detrimentally affect the decision-making pro- the device’s semantic quality. Clearly, the interpre-
cess. Consequently, there needs to be a clear un- tation of the data from the sensor has a key role
derstanding of the challenges associated with data to play in an application targeted toward a specific
quality and a way to evaluate and ensure that data healthcare requirement, as our two use cases show.
quality is maintained for different applications.
Use Case 1: Lung Function
Mapping the OSI Framework for the Elderly
to IoT Quality With the rapid growth of wearable technologies in
There are several implementation concepts behind the mobile industry, the healthcare industry is push-
determining the IoT architectures that are applica- ing the boundaries of continuous activity monitor-
tion-driven and face different challenges, whether ing using wearables.5 In this scenario, we consider
at the hardware level, software level, or integra- the use of a popular wearable vest called Hexoskin
tion. Specifically from the Open Systems Intercon- for measuring physiological changes in older adults.6
nection (OSI) perspective, three quality areas need Specifically, let’s look at one of the physiological
to be examined: the device level (data link layer), sensors, the lung volume measures that compute
the network level (network layer), and the appli- the tidal volume of the lungs using the last inspira-
cation level (presentation and application layers). tion (80 mL to 10 L) at a frequency of 1 Hz, and the
The choice of IoT devices and design protocols frequency of the inspiration and expiration events.

68 1541-1672/17/$33.00 © 2017 IEEE IEEE INTELLIGENT SYSTEMS

16 July 2017 Published by the IEEE Computer
Published Society
by the IEEE Computer Society 2469-7087/17/$33.00 © 2017 IEEE
and home
Now let us look at some of the chal-
Miscellaneous Smart
lenges in the OSI layers that need to be infrastructure
considered for this use case.

Applying Data Semantics in the

Presentation Layer IoT applications
Industrial Surveillance
In terms of preprocessing the sensor
data on the Hexoskin, the vendor’s
website (www.hexoskin.com) men-
tions baseline change and noise detec-
tion but does not provide details into Transportation Healthcare
what this entails. In particular, one
aspect of these wearables that is over-
looked when it comes to its use in the
nonactive population is that the base- Figure 1. Key application areas using the Internet of Things (IoT). This article focuses
line or even the noise measurement on the healthcare applications of the IoT.
can differ significantly in older popu-
lations. In fact, articles7 and studies8
describe the lung change in the older this index is greater than 0.7. This is Incorporating Knowledge
population as similar to emphysema. where the knowledge indicates that at the Application Layer
This can lead to a low correlation in there may be concern for obstructive One way to overcome this manifesta-
lung function and activity, and must pulmonary disease such as emphy- tion of aging in the sensor readings is
be considered when vital sign func- sema.10 However, this could also be to use the raw sensor information and
tions are interpreted.9 Furthermore, a manifestation of aging, 8 so we need change the baselines for the older pop-
this can affect the analysis when us- to dig one step further in the pro- ulation or for populations with cer-
ing the vest on the elderly population. cess and ask contextual questions, tain health conditions like emphysema.
This constraint is not surprising given such as whether Bob has a history of Here, the semantic mapping between
the fact that the sensor was primarily smoking, or conduct additional tests, the sensor values and knowledge of the
built for fitness measurement. Figure such as a flow volume loop, to diag- population using the system needs to
2 shows how understanding the chal- nose whether Bob has a lung condi- be taken into account to ensure that the
lenges at the upper layers of the OSI tion. If in fact the additional tests in- results are accurate and useful. To po-
model can improve the performance dicate that Bob has emphysema, the tentially incorporate this information,
of wearable systems in healthcare wisdom (relevant actionable medical we use ontologies or knowledge repre-
applications. science) comes in its treatment and sentations to annotate the data specifi-
Consider an older man, Bob, who handling the day-to-day variabilities cally for this application. An example
is wearing the Hexoskin vest for ac- in the symptoms that allow Bob an of such an ontology is the semantic sen-
tivity measurement. Suppose the lung improved quality of life. In this sce- sor network ontology.11 Creating such
function readings (data) for him are nario, the data and information part shared semantic definitions helps inte-
FEV1 (forced exhaled volume in 1 of the pyramid map to the presenta- grate new data into historical, tempo-
second) = 2.04 L and FVC (forced vi- tion layer, which checks the context ral, and spatial contexts. Definitions of
tal capacity) = 3 L. Although the val- and veracity of the data readings, sensors and their capabilities are also
ues themselves have no significance wherein the data are interpreted to useful for quality reasoning. For exam-
for chronic obstructive pulmonary enable semantic quality control of ple, if the accuracy of a sensor depends
disease (COPD), the FEV1/FVC ra- the IoT device readings for this appli- on phenomena other than that which
tio (also called the Tiffeneau-Pinelli cation. Similarly, as Figure 2 shows, it measures, then a specification of this
index) represents the information as the knowledge and wisdom portions can be used as a guide to search for
the proportion of a person’s vital ca- of the pyramid map to the applica- spatially and temporally related mea-
pacity that he or she can expire in the tion layer to incorporate additional surements of the phenomena on which
first second of forced expiration. We data sources for more meaningful the accuracy depends, which then de-
see from the FEV1/FVC ratio that data analysis. fines the application’s quality metrics.

MARCH/APRIL 2017 www.computer.org/intelligent 69

www.computer.org/computingedge 17
Knowledge base


FEV1 = 2.04 FVC = 3


• Service time HTTP, SIP, SSI, DNS, FTP,

Application • Service accuracy HTTP, NFS, NTP, SMPP,
layer • Information accuracy

Presentation • Data type conversions MIME, XDR

• Applying semantics to data

• Reliability IP(v4, v6), ICMP, Ipsec,

Network • Sampling parameters
IGMP, IPX, Apple talk, x.25
layer • Data fragmentation


Data link • Error checking
• Network access/permission
layer • Frame synchronization
IEEE 802.3, x.25 LAPB

Figure 2. The bottom of the figure shows four of the OSI layers: data link, network, presentation, and application. The top
shows the DIKW (data-information-knowledge-wisdom) pyramid that maps wearable sensor data to meaningful information.
The current use case is in the area of lung function analysis in the elderly population.

Specifically, in our example from Fig- devices to measure the physical and from home14 and movement within the
ure 2, for the older adult population, cognitive health conditions of older living spaces.13,14 Figure 3 shows such a
we can restrict the operating range adults living independently in their system in the home of an older couple;
(property ssn#MeasurementRange) homes. Such a system can be used to the IoT devices are all routed through
to accommodate the differences in the detect falls and to study the residents’ a common channel via the Internet
breathing measurements for the target continuous behavior pattern and gen- and stored in a secure database. Cor-
population, which can help improve erate alerts when the residents devi- responding behavior analysis is shared
the accuracy of the readings through ate from their normal behavior. Apart with the clinician as well as the couple’s
identification of more errors or discrep- from the wearable sensors discussed family members. In this use case, we
ancies in the data values. Many of the earlier that can measure physiological will discuss two challenges in terms of
other parameters or entities described changes in the residents, environmen- the network and application layers that
in the ontology are specifically left un- tal sensors are also placed in this smart relate to a more complex multimodal
defined to fit user requirements and en- environment. These include a depth- data fusion IoT-focused application.
able reusability, which can be leveraged sensor-based system that is in place for
for specific target populations to im- continuous anonymized fall monitoring Incorporating Data Integration
prove user-centric applications, espe- and activity analysis, including seden- at the Network Layer
cially in healthcare. tary behavior.12,13 Moreover, wireless The fall-detection system we described
motion sensors are in place in the envi- earlier requires a common framework
Use Case 2: Smart Home ronment to study the interactions of the that can integrate wearable sensors
System people in the environment, as well as with environmental sensors such as the
In this use case, consider a smart home- to further examine their behavior pat- wireless body/personal area network
monitoring system that uses several IoT terns, such as measuring the time away (WBAN).15 WBAN allows long-term,

70 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS

18 Computing Edge July 2017

Smart home system

• Service time
Application • Service accuracy
layer • Information accuracy SMTP, SNMP, DHCP

Presentation • Data type conversions MIME, XDR

• Applying semantics to data

• Reliability IP(v4, v6), ICMP, Ipsec,

Network • Sampling parameters
IGMP, IPX, Apple talk, x.25
layer • Data fragmentation


• Error checking
Data link • Network access/permission
layer • Frame synchronization IEEE 802.3, x.25 LAPB

Figure 3. A smart home system in which an older couple is monitored using wearables and environmental sensors for fall
prevention and activity monitoring. The data are further transmitted via the Internet to enable real-time clinical decision

unobtrusive, ambulatory health moni- Effect of Application-Driven camera) is 2 frames per second, that
toring with instantaneous feedback to Quality of Service at the OSI will be the resolution of the overall sys-
the user about the current health sta- Application Layer tem, conservatively speaking. Although
tus. The devices are connected wire- A crucial and often overlooked chal- this frame rate might be sufficient for
lessly via low-powered networking pro- lenge in terms of quality of service for activity-monitoring systems, the reso-
tocols such as Zigbee (motion sensors), the smart home system is that quality lution could be too low if we want to
Zwave, or Bluetooth (activity track- is dominated by its weakest link—that detect falls occurring inside the home.
ers), as well as through high-powered is, the lowest-quality sensor device. Moreover, the depth sensor’s low frame
wired connections (Kinect). Device in- This could include a failed sensor, de- rate can seriously affect the fall-recog-
teroperability is crucial to ensure that vice-specific network connectivity is- nition system if it is a combination of
all the data are recorded simultane- sues, or even database malfunctions. multimodal sensors such as depth and
ously, continually, and accurately. As an example, consider the fall- audio, which relies on the sensor fusion
However, systems such as the semantic detection system in the smart home for detecting the fall occurrence. To ad-
gateway bypass the network interop- setting. This system comprises hetero- dress this, we can use two factors for
erability that acts as a bridge between geneous sensors, such as depth and au- activity recognition: the individual sen-
the IoT devices and the Internet to al- dio sensors, to detect falls inside the sors’ data quality and knowledge of the
low part of the data processing to oc- home and alert the clinicians. How- activity itself.
cur in the gateway, enabling faster ever, if the lowest frame rate among An important point to note here is
decision support.16 the sensing devices (say, the depth that both of the factors discussed in

MARCH/APRIL 2017 www.computer.org/intelligent 71

www.computer.org/computingedge 19
Table 1. Factors affecting quality of service in smart home fall-detection systems.
Factor Information (example) Solution
Activity understanding A fall event corresponds to a “sudden change in height, Incorporating the semantic data quality check to look for the
with a sudden increased downward movement, as well as a temporal sequence information of the activities.
corresponding trigger to the person being on the ground.”13
Disparity in sensor The depth sensor is more prone to generate false alarms Weighted aggregation of the sensor devices depending on the
data quality in detecting falls as compared to the motion sensors in the location of the fall event. Fuzzy aggregation methods such as
living area. Sugeno and Choquet integrals17 also allow uncertainty to be
incorporated, which can take into account sensor data noise.

Table 1 require prior knowledge of In a report from Cisco on IoE inno- 4. ISO 9000:2005, Quality Management
the activities, as well as evaluation vations,18 a key insight was on the in- Systems—Fundamentals and Vocabu-
of the sensor data quality that can creased usage of mobile applications lary, ISO, 2005.
be leveraged to learn the aggregation for interacting with IoE processes. 5. A. Sheth, P. Anantharam, and K.
measures for multimodal data fusion. For a fall-detection system, using a Thirunarayan, “kHealth: Proactive
fall-detection mobile application will Personalized Actionable Information for
Internet of Everything or further allow clinicians and family Better Healthcare,” Workshop Personal
Indispensable Role of Humans in members to access real-time feedback Data Analytics in the Internet of Things,
Quality Control on their patient or loved one’s sta- 2014; http://knoesis.org/node/2237.
The two factors discussed in the qual- tus, thereby transforming the current 6. T. Banerjee et al., “Evaluating a Poten-
ity of service of the smart home sys- clinical decision support system. tial Commercial Tool for Healthcare
tem in Table 1 are essential for the ef- Application for People with Dementia,”
fectiveness of the smart home system.
Proc. Int’l Conf. Health Informatics and
However, despite the incorporation of verall, we see the effect of the Medical Systems, 2015; http://corescholar
activity knowledge as well as the IoT IoT, and even the IoE, on two use .libraries.wright.edu/cgi/viewcontent.cgi?
device quality, the performance of the cases in daily life. Through improved article=2442&context=knoesis.
state-of-the-art fall-detection system is IoT data quality, the IoT can have a 7. L.J. Martin, “Aging Changes in Vital
still low. One way to improve the sys- staggering impact on different facets Signs,” Medline Plus, 27 Oct. 2014; http://
tem’s performance is to incorporate of our existence, from entertainment, medlineplus.gov/ency/article/004019.htm.
human knowledge into the IoT system surveillance, transportation, and 8. R. Knudson et al., “Changes in the Nor-
architecture. In fact, the Internet of daily activities to industry applica- mal Maximal Expiratory Flow-Volume
Everything (IoE) is a concept that ex- tions and healthcare. Curve with Growth and Aging,” Am.
tends the IoT framework on machine- Rev. Respiratory Disease, vol. 127, no.
to-machine (M2M) communications 6, 1983, pp. 725–734.
to encompass people and processes for Acknowledgments 9. A. de Pablo et al., “Pathophysiological
a much larger scale of data analytics. We thank John Hughes from Wright State Consequences of Lung Volume Reduc-
For our smar t home-monitoring Internal Medicine and Mary Catherine Schafer tion Surgery in Patients with Emphy-
system, by incorporating a human- from Jefferson City Medical Group for their sema,” Archivos de Bronconeumología,
valuable suggestions on the clinical aspects of
in-the-loop, the fall-detection system the use cases described in this article. vol. 39, no. 10, 2003, pp. 464–468.
can achieve a much lower false-alarm 10. R. Pellegrino et al., “Interpretive Strate-
rate that will alert the clinician and gies for Lung Function,” European
family members of a fall only after a References Respiratory J., vol. 26, no. 5, 2005,
technical nursing staff dedicated for 1. A. Sheth, “Internet of Things to Smart pp. 948–968.
this purpose has confirmed its occur- IoT Through Semantic, Cognitive, and 11. M. Compton et al., “The SSN Ontology
rence. This can not only reduce clini- Perceptual Computing,” IEEE Intelligent of the W3C Semantic Sensor Network
cian fatigue but also prevent undue Systems, vol. 31, no. 2, 2016, pp. 108–112. Incubator Group,” Web Semantics: Sci-
panic through a more mediated IoE 2. J. Gubbi et al., “Internet of Things ence, Services and Agents on the World
approach. Moreover, through an ac- (IoT): A Vision, Architectural Elements, Wide Web, vol. 17, 2012, pp. 25–32.
tive learning process, the existing and Future Directions,” Future Genera- 12. T. Banerjee et al., “Recognizing Complex
fall-detection system can update the tion Computer Systems, vol. 29, 2013, Instrumental Activities of Daily Living
algorithm to reduce the number of pp. 1645–1660. Using Scene Information and Fuzzy
instances where the human-in-the- 3. “ASQ Quality GlossaryQ,” 2017; http:// Logic,” Computer Vision and Image Un-
loop is required using the IoE design. asq.org/glossary/q.html. derstanding, vol. 140, 2015, pp. 68–82.

72 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS

20 Computing Edge July 2017
13. T. Banerjee et al., “Monitoring Hospital tecture for IOT Interoperability,” Proc. specifically in the domain of eldercare tech-
Rooms for Safety Using Depth Images,” IEEE Int’l Conf. Mobile Services, 2015, nologies. Contact her at tanvi@knoesis.org.
Proc. AAAI Fall Symp. Series AI for pp. 313–319.
Gerontechnology, 2012; www.eldertech 17. L.M. Campos and M. Jorge, “Charac- Amit Sheth is the LexisNexis Ohio Emi-
.missouri.edu/wp-content/uploads/2016/07 terization and Comparison of Sugeno nent Scholar, executive director of the Ohio
/Monitoring-Hospital-Rooms-for-Safety and Choquet Integrals,” Fuzzy Sets and Center of Excellence in Knowledge-Enabled
-Using-Depth-Images.pdf. Systems, vol. 52, no. 1, 1992, pp. 61–67. Computing (Kno.e.sis) at Wright State Uni-
14. G. Alexander et al., “Density Map Visu- 18. J. Bradley et al., Internet of Everything versity, and an IEEE Fellow. Contact him at
alization as a Tool to Monitor Activity in the Public Sector: Generating Value amit@knoesis.org; http://knoesis.org.amit.
Levels of Older Adults,” Gerontechnol- in an Era of Change, Cisco, 2014.
ogy, vol. 9, no. 2, 2010, p. 186.
15. I. Taimiya Sylla, “Wireless Body Area Net- Tanvi Banerjee is an assistant professor of
works: What Engineers Need to Know,” computer science and engineering at Wright
EE Times, 26 Sept. 2011; www.eetimes State University and Kno.e.sis. Her research Read your subscriptions
This article originally
through theappeared
myCS pub- in
.com/document.asp?doc_id=1279106. interests include sensor validation and tying IEEE Intelligent Systems, vol. 32,
lications portal at http://
16. P. Desai, A. Sheth, and P. Anantharam, machine learning with sensor data for action- no. 2, 2017. mycs.computer.org.
“Semantic Gateway as a Service Archi- able information in healthcare applications,

Take the
CS Library
IEEE-CS Charles Babbage Award
Deadline 15 October 2017

you go!
Established in memory of Charles Babbage in recognition of significant
contributions in the field of parallel computation. The candidate would
have made an outstanding, innovative contribution or contributions to
parallel computation. It is hoped, but not required, that the winner will
have also contributed to the parallel computation community through
teaching, mentoring, or community service.

CRITERIA IEEE Computer Society magazines and Transactions are now

This award covers all aspects of parallelavailable
computing to subscribers in
including the portable ePub format.
computational aspects, novel applications, parallel algorithms, theory of
parallel computation, parallel computing technologies, among others.


Just download the articles from the IEEE Computer Society Digital
A certificate and a $1,000 honorarium and you
presented to a can
singleread them on any device that supports ePub.
The winner will be invited to present a paper and/or presentation at
For more information, including NOMINATION
a list of SITE visit
compatible devices,
the annual IEEE-CS International Parallel and Distributed Processing awards.computer.org
Symposium (IPDPS 2017).
NOMINATION SUBMISSION www.computer.org/epub
Open to all. Nominations are being accepted electronically at www.
computer.org/web/awards/charles-babbage. Three endorsements are
required. The award shall be presented to a single recipient.

MARCH/APRIL 2017 www.computer.org/intelligent 73

www.computer.org/computingedge 21
Natural Web Interfaces
Editor: Munindar P. Singh • m.singh@ieee.org

Continuous Authentication
and Authorization for the
Internet of Things
Muhammad Shahzad and Munindar P. Singh • North Carolina State University

How can users be authenticated and authorized continuously for the Inter-
net of Things, when most small smart devices lack the conventional interfaces
used for authentication (such as keyboards, mice, and touchscreens)? Here, the
authors explore potential solutions along with a related case study.

e’re venturing into the era of the Inter- we authenticate and authorize users for the IoT,
net of Things (IoT). As computing devices where we lack conventional user interfaces? For
become smaller, smarter, and ubiquitous, example, one of the latest features of the Apple
computing has begun to embed into our environ- Watch is that if a user owns a (sufficiently new
ments by attaching to physical objects or things. version of) Macbook Pro, an iPhone, and an
IoT is bringing computing both onto our bodies Apple Watch, the user can set up the Macbook
and into our daily surroundings. Examples of on- Pro to automatically unlock without entering a
body computing devices include human activ- password. More specifically, as soon as the user
ity trackers, smart watches, and semi-permanent opens the lid of her Macbook Pro, the laptop
insulin pumps. Examples of in-environment com- automatically unlocks if the following four con-
puting devices include intelligent thermostats, ditions are satisfied:
smart appliances, remotely controllable house-
hold equipment, and weather-based automated • The user is wearing the Apple Watch.
lawn irrigation systems. • The Apple Watch is connected to the user’s
Although IoT devices often have compute power iPhone via Bluetooth.
close to those of conventional computing devices • The watch is in close proximity to the Mac-
from a few years ago, one of the ways in which typ- book Pro.
ical IoT devices differ is that they lack conventional • Either the iPhone or the watch has been unlocked
user interfaces in the form of keyboards, mice, and at least once since the user last put on the watch.
touchscreens. Examples of such computing devices
include the Fitbit activity tracker, sewable comput- This convenient feature carries a security threat,
ing devices such as the Arduino Lilypad, and smart however. Suppose an attacker, possibly posing as a
fabrics. A motivation for eliminating such user friend, gets hold of the user’s watch and has physical
interfaces isn’t so much to reduce the cost as that access to her computer. Such a scenario might occur
the conventional interfaces often aren’t appropriate if the two are in a lunch meeting and the user steps
for the intended applications. For example, a smart away from the table to pick up something from the
fabric can have embedded antennas and communi- buffet, but leaves behind her watch and computer. If
cate information about the person wearing the fab- the attacker wears that watch and the user happens
ric to devices such as smartphones, but it wouldn’t to unlock her phone while away from the table —
quite make sense to attach a touchscreen to a shirt but within the Bluetooth range of the watch — the
or a keypad to a Fitbit. watch unlocks as well. Then the attacker can use
This lack of a user interface gives rise to a the watch to unlock the user’s Macbook Pro without
fundamentally challenging question: How do having to guess her password.

86 Published by the IEEE Computer Society 1089-7801/17/$33.00 © 2017 IEEE IEEE INTERNET COMPUTING
22 July 2017 Published by the IEEE Computer Society 2469-7087/17/$33.00 © 2017 IEEE
Continuous Authentication and Authorization for the Internet of Things

Although this technology employs (often two or three) LEDs and a few the PPG signal could contain enough
an Apple Watch, which does have a (again, often two or three) light sen- information to enable user authenti-
conventional user interface in the form sors. Both IMUs and PPG sensors can cation. Using the PPG signal is partic-
of a touchscreen, in principle, it can enable user authentication. ularly challenging, however, because
be extended to any wearable device Specifically, using the IMU, we this signal is sensitive to the motion
with a Bluetooth interface, such as a can develop authentication tech- of a person’s limbs: that is, the PPG
Fitbit. These kinds of examples high- niques that are based on the principle measurement depends on the person’s
light the present challenge: How can that users frequently move their limbs speed of movement. Fortunately, most
we continuously authenticate a person in unique patterns throughout the devices these days that come with a
using a device without a conventional time they use the device. An exam- PPG sensor also come with an IMU.
interface? Here, we consider various ple of a well-known trait that differs Therefore, we can potentially use
solutions, including a case study for across users is gait. If we can extract information from the IMU sensors to
a Wi-Fi-based human authentication the patterns in the output of an IMU measure the amount of motion of the
system (Wi-Fi uses radio frequencies sensor due to the user’s unique gait, limb and combine the two signals — or
near 2.4 and 5 GHz). we can use simple machine learning correct the measurement of the PPG
techniques to learn these patterns and sensor — to enable authentication.
Prospective Solutions apply them to continuously authenti-
Because of the diversity of devices cate the user based on gait. If such a Authentication on Devices That
and applications, a universal solution technique was developed and put into Don’t Maintain Permanent
to the problem of continuous authen- practice, a device of the first category Physical Contact
tication of users on devices without (worn by the user) could monitor the A more challenging problem is to
conventional interfaces might not user continuously and frequently design an authentication scheme that
exist. However, we can make progress authenticate the user’s legitimacy can identify users for devices that
by dividing IoT devices with which before allowing the user to perform don’t maintain permanent contact
humans interact into two categories appropriate operations. For example, with users. Such devices include those
and studying solution directions for the watch in the Macbook Pro setting embedded into our environment. For
these categories separately. The first wouldn’t authenticate the attacker, example, consider an application
category consists of devices that which would prevent the Macbook that integrates a user’s calendar with
maintain permanent physical contact Pro from being spuriously unlocked. the user’s home lighting and is con-
with the user during usage, such as Several behavioral biometrics solu- trolled with speech. Whenever a cal-
activity trackers, smart watches, and tions have been proposed that employ endar generates a notification for the
insulin pumps. The second category the IMU to authenticate users.1,2 user, the user’s location is automati-
consists of devices that don’t main- Similarly, the PPG sensor pro- cally determined through proximity
tain permanent physical contact with vides an opportunity to study the or movement sensors, and lights of
humans, such as intelligent thermo- PPG signal for unique patterns in the appropriate room are flashed to
stats, occupancy sensors, and smart blood flow rhythm. Researchers have alert the user. Suppose this applica-
household appliances. shown that due to slight variations tion is enabled or disabled through
in every human’s heartbeat rhythm, voice commands from a specific user.
Authentication on Devices That echocardiogram (ECG) signals con- Then, an attacker could replay previ-
Maintain Continuous Physical tain small information — but this is ously recorded voice commands of
Contact enough to indicate what’s unique to the original user. That is, voice-based
Devices that maintain contact with an individual user.3,4 Consequently, authentication is insufficient. We need
the user can support new forms of just by using the ECG signal, we can effective methods to continuously and
biometric authentication. Most of design user schemes to authenticate unintrusively authenticate users with-
these devices fall into two categories. users. Although several ECG-based out, for example, requiring the user to
Devices of the first category either user authentication systems have wear sensors such as IMUs.
contain an inertial measurement been proposed, this technology has A potential approach for devel-
unit (IMU), which is comprised of yet to achieve sufficient effective- oping such an authentication system
an accelerometer and a gyroscope, ness to see widespread deployment. is to employ pervasive modalities
or can have an IMU embedded quite Because the PPG signal is generated such as radio frequency (RF) signals,
easily. Devices of the second category based on the amount of blood flow ambient light, and sound, which are
contain a photoplethysmogram (PPG) in the user’s veins, which depends on present all around us. The intuition
sensor, which is comprised of a few how the user’s heart pumps blood, behind RF-based authentication is

www.computer.org/computingedge 23
Natural Web Interfaces

that the wireless channel metrics — meaningless. The point of authentica- eldercare setting, if the environmen-
such as channel state information tion is to provide a basis for making tal sensors (whether Wi-Fi or light-
(CSI) and received signal strength a decision — about which resources based) indicate a lack of movement
(RSS) — change based on a user’s to provide access to which person for a prolonged period, the elder-
presence and movement. The patterns for what purpose and when. In broad care application might disclose data
of change in these metrics depend on terms, we aren’t so much interested from wearable devices that capture
the way the user moves. Because dif- in seeing who specifically is around the resident elder’s health condition.
ferent users have different gaits, they but what information or device to Conversely, the data from a wearable
produce different patterns of change share with that person under what sensor being anomalous might lead
in wireless channel metrics. An RF- circumstances. the application to verify whether a
based user authentication system can Consider a situation where a user qualified caregiver was currently in
apply machine learning techniques is wearing various health-monitoring the same room as the resident.
to associate each user with his or her devices, including an ECG reader.
patterns of change and identify the Ordinarily, the data gathered by such Case Study
user at runtime based on the learned devices would be confidential. Now, To validate the effectiveness of such
associations. suppose the user is having a seri- an approach to leverage variations in
Specifically, with a human walk- ous medical problem, such as a heart pervasive modalities, working with
ing around, because a human is attack. In such a case, it might be colleagues, we developed a Wi-Fi-
mostly made of water, the Wi-Fi acceptable behavior for the relevant based human authentication system,
signal reflected by the human body software application to reveal the data called WifiU, which recognizes users
generates unique, although small, from his ECG reader as well as data based on their gait.6 We developed
variations in CSI measurements on about recent physical activity to any- WifiU entirely using COTS Wi-Fi
the receiver due to the well-known one who is present nearby and might devices to capture fine-grained gait
multipath effect of wireless signals. be willing and able to help. But if patterns. WiFiU consists of two Wi-Fi
These variations in CSI enable signal the user is already in a hospital, then devices: one for continuously send-
processing techniques to obtain gait perhaps the application doesn’t need ing signals, which can be a router,
information such as walking speed, to be quite so forthcoming in reveal- and one for continuously receiving
gait cycle time, footstep length, and ing its user’s information to strang- signals, which can be a laptop. In
movement speeds of legs and torso. ers. That is, here the decision changes WifiU, the receiver measures chan-
Because each human has a unique from a focus on authentication of the nel state information (CSI) of each
gait, the gait patterns that the Wi-Fi counterparty to determining some received Wi-Fi frame. Fundamentally,
receiver obtains can be used to recog- attributes of the information resource WifiU recognizes humans based on
nize a walking human subject. and of the current context. Indeed, who they are, because WifiU extracts
Similarly, the intuition behind modern approaches5 to authorization unique biometrics information from
light-based authentication is that as a express policies in terms of attributes Wi-Fi signals and uses it to perform
user moves in an indoor environment, of principals, resources, and contexts human authentication.
the amount of light he or she reflects instead of specific identifiers or roles. Compared with traditional gait-
and blocks depends on his or her pat- The IoT can readily accommodate recognition systems, which use cam-
terns of movement. As different users such approaches by accumulating a eras, floor sensors, or wearable sensors
have different gaits, the patterns of rich variety of attribute values from to capture gait information, WifiU is
change in intensity of light, as mea- the available devices. easier to deploy and has better cov-
sured by light sensors deployed on In this example, the decision about erage. From the deployment perspec-
the floor, are also different. A light- whether to share some data is based tive, WifiU doesn’t require any special
based user authentication system can upon the data values themselves hardware (such as floor sensors) and
learn these patterns and apply them (for example, sharing with anyone doesn’t require the human subject to
to identify users at runtime. A similar if the ECG indicates distress), but in wear any hardware (such as an IMU).
intuition holds for audio-based user general the decision might be based Wi-Fi devices are ubiquitous and most
authentication. on the totality of available informa- homes and offices are covered by
tion. In particular, we can have situ- Wi-Fi signals. The hardware that we
Authorization in the IoT ations where an application grants experimented with — namely a Net-
So far we’ve talked of authentication access to wearable devices based on Gear JR6100 Wi-Fi router and Think-
because it provides concrete use cases. environmental devices or the other Pad X200 laptop (with an Intel 5300
But authentication by itself is usually way around. For example, in a home WiFi NIC) — required no modifications.

88 www.computer.org/internet/ IEEE INTERNET COMPUTING

24 Computing Edge July 2017
Continuous Authentication and Authorization for the Internet of Things

Furthermore, unlike cameras, WifiU

doesn’t require lighting and works in Sender
the dark just as well as in bright light. Walking route (5.5 m)
In designing WifiU, we faced

1.6 m
many technical challenges. For exam-

6.5 m
ple, it’s nontrivial to profile gait pat-
terns using CSI dynamics. Extracting Receiver
gait information from CSI signals is
difficult, because the signal reflec-
tions of different body parts are
mixed together in the CSI waveform. Table
As different human body parts move
at different speeds while walking, the 7.7 m
radio signal reflections from different
body parts have different frequencies.
Figure 1. Data collection environment. Walking in an area of 50 square
To separate the radio signal reflec-
meters, we gathered more than 2,800 gait instances from 50 human subjects.
tions from the different body parts,
we convert CSI waveforms (of two
dimensions: time and amplitude) into this accuracy might be acceptable technology. Therefore, the prospects
spectrograms in the time-frequency for many personalized services such of unintrusive authentication and
domain (of three dimensions: time, as adjusting room temperature and authorization leading to context-sen-
frequency, and amplitude). We apply background music in smart buildings, sitive policies are encouraging.
spectrogram enhancement techniques it might not be high enough for set- However, these unintrusive authen-
to reduce signal noise. The resulting tings that require high accuracy, such tication technologies create potential
spectrograms yield detailed human as accessing your email. Third, the privacy threats through the infrastruc-
gait information similar to those gen- number of walking human subjects ture in that an attacker who can obtain
erated by Doppler radars. is limited to one. In practice, the tar- access to the infrastructure might apply
We conducted experiments on gets should walk along a given path these techniques without the user being
WiFiU using our gait database that con- one by one to ensure good recogni- aware of having been identified. For
tains more than 2,800 gait instances tion performance, as is the case for an example, an attacker can potentially
collected from 50 human subjects airport security check. To address this read Wi-Fi signals to identify victims
walking in a typical laboratory with an limitation in future work, we plan to without being detected. Consider a sce-
area of 50 square meters (see Figure 1). use multiple Wi-Fi receivers to sepa- nario where a burglar attempts to figure
We anonymized all collected data to rate signals of multiple humans using out who is at home by eavesdropping
protect participants’ privacy. Over the the differences in received signals at on the Wi-Fi signal emitted by the
50 subjects, WifiU achieves recog- multiple receivers. Wi-Fi router in the victim’s house. As
nition accuracies of 79.3, 89.5, and Wi-Fi signals can penetrate through
93.0 percent for the Top-1, Top-2, and obstacles such as furniture, wooden
Top-3 candidates, respectively. (Here,
Top-N means that one of the selected
N persons is the person who truly gen-
T he IoT is in a nascent stage, but the
arc of technology and the poten-
tial benefits it offers suggest that the
doors, and walls, the burglar needs to
only passively measure the CSI of the
signal outside the house without need-
erated that gait observation.) IoT’s presence will only increase. By ing to decode the Wi-Fi packets’ con-
With the current implementation placing people in information-rich tent. Therefore, it would be difficult for
using a single wireless link, WifiU environments, especially those that the victim to prevent certain breaches
has three limitations. First, the dis- are natural and feel natural, the IoT of privacy. Although avoiding privacy
tance between the human subject exposes users to new security and breach isn’t the focus of this article, we
and the Wi-Fi devices is limited to privacy threats. It simultaneously hope this work highlights this privacy
six meters. To address this limitation demands stronger (that is, continuous) risk to the research community and
in future work, we can deploy mul- authentication and authorization and encourages future work. A previous
tiple Wi-Fi sender-receiver pairs in takes away conventional informa- column7 addresses the privacy risks
the target area. Second, the recogni- tion modalities. Fortunately, the IoT in intelligent user interfaces, some of
tion accuracy is limited to 92.3 per- provides new ways to address these which might be exacerbated in combi-
cent for Top-1 candidates. Whereas challenges through innovative uses of nation with the IoT.

www.computer.org/computingedge 25
Natural Web Interfaces

Acknowledgment 4. Z. Zhang et al., “ECG-Cryptography and of computer networks. Shahzad has a PhD in
Munindar Singh thanks the US Department Authentication in Body Area Networks,” computer science from Michigan State Uni-
of Defense for support through the Science of IEEE Trans. Information Technology in Bio- versity. Contact him at mshahza@ncsu.edu.
Security Lablet. medicine, vol. 16, no. 6, 2012, pp. 1070–1078.
5. D.R. Kuhn, E.J. Coyne, and T.R. Weil, “Add- Munindar P. Singh is a computer science professor
References ing Attributes to Role-Based Access Control,” at North Carolina State University. His research
1. R. Mayrhofer and H. Gellersen, “Shake Computer, vol. 43, no. 6, 2010, pp. 79–81. interests include the conception, engineering, and
Well before Use: Authentication Based on 6. W. Wang, A.X. Liu, and M. Shahzad, “Gait governance of sociotechnical systems as a way
Accelerometer Data,” Proc. Int’l Conf. Per- Recognition Using WiFi Signals,” Proc. to tackle concerns such as security and privacy.
vasive Computing, 2007, pp. 144–161. Int’l Conf. Pervasive and Ubiquitous Com- Singh is a Fellow of IEEE and the American Asso-
2. T.T. Ngo et al., “The Largest Inertial Sen- puting, 2016, pp. 363–373. ciation for Artificial Intelligence (AAAI), a for-
sor-Based Gait Database and Performance 7. C.J. Hazard and M.P. Singh, “Privacy Risks mer Editor in Chief of IEEE Internet Computing,
Evaluation of Gait-Based Personal Authen- in Intelligent User Interfaces,” IEEE Internet and the current Editor in Chief of ACM
tication,” Pattern Recognition, vol. 47, no. Computing, vol. 20, no.6, 2016, pp. 57–61. Transactions on Internet Technology. Contact
1, 2014, pp. 228–237. him at singh@ncsu.edu.
3. S.I. Safie, J.J. Soraghan, and L. Petropou- Muhammad Shahzad is an assistant professor
lakis, “Electrocardiogram (ECG) Bio- in the Department of Computer Science at
Read your subscriptions
metric Authentication Using Pulse Active North Carolina State University. His research
Ratio (PAR),” IEEE Trans. Information interests include the measurement, network-
This article originally appeared
through the myCS pub-in
IEEE Internetlications
Computing, at http://
portal vol. 21,
Forensics and Security, vol. 6, no. 4, 2011, ing, and user interface aspects of the IoT as
pp. 1315–1322.
IEEE_half_horizontal_Q6:Layout well as measurements,
1 4/21/11 4:21 PM design,
Page and1modeling
no. 2, 2017. mycs.computer.org.

Experimenting with your Now there’s

hiring process?
Read all your IEEE magazines
even more to
Finding the best computing job or hire shouldn’t be left to chance.
and journals your WAY on
IEEE Computer Society Jobs is your ideal recruitment resource, targeting
over 85,000 expert researchers and qualified about
top-level managers inyour
engineering, robotics, programming, artificial intelligence, networking and
communications, consulting, modeling, data structures, and other computer
science-related fields worldwide. Whether you’re looking to hire or be hired,
IEEE Computer Society Jobs provides real results by matching hundreds of
relevant jobs with this hard-to-reach audience each month, in Computer
Introducing myCS, the digital magazineand/or online-only!
portal from IEEE Computer Society.
Go beyond static, hard-to-read PDFs
with an easily accessible, customizable,
and adaptive experience. http://www.computer.org/jobs
There’s No Additional Cost!
The IEEE Computer Society is a partner in the AIP Career Network, a collection of online job sites for scientists, engineers, and
computing professionals. Other partners include Physics Today, the American Association of Physicists in Medicine (AAPM), American
Association of Physics Teachers (AAPT), American Physical Society (APS), AVS Science and Technology, and the Society of Physics
Students (SPS) and Sigma Pi Sigma.

►LEARN MORE AT: mycs.computer.org

90 www.computer.org/internet/ IEEE INTERNET COMPUTING
26 Computing
IEEE myCS half Page Space Edge
Ad 2016_4-26-16.indd 1 4/28/16 July
3:03 2017
Expert Opinion

Visual IoT: Architectural

Challenges and Opportunities


...... The emergence of ultra-low- (recording of public and personal events, computation), location (where to store
power sensing devices along with con- such as sports and music), and, more the data), and location (where to enable
nectivity to gateways and cloud services recently, interactive environments (aug- interfaces and tools for analytics). Let’s
has led to an end-to-end Internet of mented, virtual, and merged reality; see start by examining the computing loca-
Things (IoT) architecture for many real- www.intel.com/content/www/us/en/ tion challenge and then move to memory
world usages. Visual IoT is one such architecture-and-technology/virtual-reality- and interfaces.
class of IoT that poses significant end-to- overview.html) and robotics and drones1
end challenges due to the need for sens- (navigation, delivery, interaction, and Computing in Visual IoT: Partitioning and
ing and processing of visual data. The assistance). With the emergence of Heterogeneity
richness of visual data provides many depth-sensing cameras, such as Intel Partitioning the work in an end-to-end
opportunities for analytics, while at the RealSense (www.intel.com/content/www/ visual IoT architecture is a challenge,
same time requiring high computational us/en/architecture-and-technology/real- because it requires the balancing of mul-
capabilities and therefore potentially sense-overview.html), analysis of the tiple important dimensions:
high-bandwidth data transfer to a more captured visual scene becomes even
 the sensing node’s battery life,
powerful node in the end-to-end architec- more attractive for many of these
 the latency of the interaction,
ture. Memory and storage needs are scenarios.
 throughput benefits on the
also more pronounced in visual IoT solu- In many of these scenarios, three
server versus bandwidth costs of
tions, requiring careful thought to devel- types of platforms compose the end-to-
the transfer, and
oping an intelligent memory hierarchy for end IoT architecture (see Figure 1):
 security and privacy implications
visual storage and retrieval. In this article,
 visual sensing nodes that capture of the data.
I examine the computing, memory, and
the data and potentially do some
interface implications for end-to-end vis- The partitioning problem is essentially
local processing;
ual IoT architectures and discuss poten- across heterogeneous platforms as well
 gateways, phones, or on-
tial solutions and tradeoffs in each of as within heterogeneous processing ele-
premise platforms that can stage
these areas. Before we start, let’s go ments (cores versus GPUs versus accel-
the data and provide higher com-
over a brief overview of visual IoT usage erators) within a platform. Also, the
puting capability; and
domains. application can dictate the partitioning
 cloud servers that provide serv-
strategy in some cases, in which a sub-
ices for search, analytics, or sim-
Visual IoT Overview ply storage.
set of operations on the sensor node can
Beyond photography, cameras have deliver some minimal useful experience,
been used widely in multiple domains, Much like real estate, the key to effi- while the processing at the server is
ranging from security (for example, sur- ciently architecting a visual IoT architec- used for heavier computations.
veillance and monitoring), entertainment ture is location (where to perform the Let’s take an example scenario of a
visual agent monitoring a home environ-
ment. The key aspects of a home agent
Editors' note: We invited two industry experts to discuss the opportunities and
challenges surrounding the Internet of Things. The following are their views on
the topic. —Vijay Janapa Reddi and Hyesoon Kim 1. anomaly detection,
2. saliency and summarization,


0272-1732/16/$33.00 c 2016 IEEE
2469-7087/17/$33.00 © 2017 IEEE
Published by the IEEE Computer Society
Published by the IEEE Computer Society July 2017
45 27

Heterogeneous computation: Where to run? Dynamic partitioning

Sensor node
CPU CPU Accel Remote platform Cloud storage
Sensor node

Heterogeneous memory: Where to store? What to store?

Visual Gateway
Sensor node
SRAM DRAM NVM Remote platform Cloud storage

Interfaces and tools: What interfaces to expose? What type of tools?

Learning capabilities: Offline or online? Continuous?

Figure 1. Visual IoT. Example implications on architecture research include how to dynamically partition the computation
across the heterogeneous architectures, manage the memory across the end-to-end system, and integrate offline/online
learning capabilities and tools.

3. detection of patterns of behav- customized for different homes and simi- were identified. In a recent study,3 the
ior, and lar environments. As a result, solutions authors demonstrated the ability to sum-
4. recognition and interaction with such as remote offloading are becoming marize a video by optimizing for similar-
a person in the home from a more important from a flexibility and cus- ity and coverage. Analyzing such
Q&A standpoint. tomization point of view.2 Research and algorithms and capabilities and convert-
The scenario becomes even more development in heterogeneous architec- ing them into appropriate computing and
complicated if the visual agent is mobile tures with partitioning capabilities that memory implementations4 is a critical
(like a robot) versus a static visual agent retain flexibility while maximizing effi- area of research for future visual sensor
in which the backgrounds can be prede- ciency and customizability will continue nodes, as well as the gateways and
termined. For aspects 1 and 4, the to be predominant for visual IoT, as well servers that maintain the data.
response time is critical, so local proc- as other rich environments. Beyond saliency and summarization
essing is desirable, whereas for aspects of frames, it is also critical to identify key
2 and 3, batch processing is more useful Memory in Visual IoT: Saliency, Storage, entities and activities in visual data to
because of the large amount of data and Hierarchy enable fast search and indexing. The
needed before processing. Another major challenge in the end-to- question becomes what metadata needs
Such scenarios are common and end visual IoT architecture is manage- to be extracted and where such meta-
require careful examination of whether ment of the visual data. Although the data should be stored (on the sensor
the processing can be performed on the richness of visual data is attractive, it is node, in the gateway, or on the cloud
computing core of the sensor node itself, also true that much of the visual data server). In addition, there is a question of
offloaded to a local accelerator, sent to a captured can be potentially discarded, what type of memory is most suitable
gateway within the home, or offloaded and only a summary typically needs to for the metadata in question. This calls
to a private cloud where the analysis can be retained. The key is to figure out for an end-to-end heterogeneous mem-
be accomplished. A static solution would what visual data summary must be ory architecture consisting of different
end up determining how to employ the retained by potentially extracting the sali- memory types, ranging from cache to
most efficient engine (fixed function or ent segments of the visual stream. The DRAM to nonvolatile memory to storage.
configurable accelerator) at each node in basis for saliency depends entirely on Identifying the right balance of such het-
the end-to-end architecture. Instead of the usage in question. For example, in erogeneous memory across each of the
statically determining this heterogene- the home scenario, the salient aspects nodes in the end-to-end architecture is
ous architecture and partitioning balance, might be the key activities that hap- critical as visual IoT usages explode and
a dynamic partitioning solution is even pened throughout the day and the cause bandwidth challenges for retrieval
more suitable if the solution has to be anomalies and novel occurrences that of data.

Computing Edge July 2017
Interfaces and Tools for Visual IoT: Learning en-us/intel-knowledge-builder-toolkit]) are Trusted Mobile Cloud Computing,”
and Development critical for the rapid deployment of IoT Proc. Int’l Conf. Parallel and Distrib-
Finally, it is important to consider appro- solutions. uted Systems, 2013, pp. 240–248.
priate interfaces for visual IoT platforms. 3. S. Chakraborty, O. Tickoo, and R. Iyer,
For example, as machine learning techni-
ques get adopted to analyze sensor data, V isual IoT is a rapidly growing class
of usages with the proliferation of
smart cameras with increasing capabil-
“Adaptive Keyframe Selection for
Video Summarization,” Proc. IEEE
it becomes important to understand how Winter Conf. Applications of Com-
to take advantage of both offline and ities. Future areas of research include puter Vision, 2015, pp. 702–709.
online learning techniques. As an exam- developing heterogeneous architectures
4. T. Lee et al., “Low-Complexity HOG
ple, if the visual agent wants to under- and dynamic partitioning capabilities
for Efficient Video Saliency,” IEEE
stand gestures made by the people in a across end-to-end visual IoT, examining
Int’l Conf. Image Processing, 2015;
home, it is extremely useful to enable heterogeneous memory stores for visual
interfaces and tools that allow the agent data management and retrieval, and
5. “Intel Curie Module & Intel IQ SW
to train on and download these capabil- tools and interfaces for fast deployment
Fact Sheet,” Intel, Aug. 2015; www.
ities. By making such capabilities broadly of analyzing visual and other IoT
available, developers will be able to pro- solutions. MICRO
vide many analytics capabilities employ-
ing rich sensor data and potentially ............................................................
crowdsourced training data. Especially References
1. K. Kaplan, “The Future of Drones:
as sensor nodes become more capable
Market Prepares for Takeoff,” Intel, Ravi Iyer is a senior principal engineer,
(such as the Intel Curie Module5 with
Sept. 2016; http://iq.intel.com/drone- CTO, and director in New Business Ini-
pattern matching capability), new tools
economy-prepares-takeoff. tiatives at Intel. He is an IEEE Fellow.
that enable developers to use such capa-
2. H. Eom et al., “OpenCL-Based Contact him at ravishankar.iyer@intel.
bilities (such as the Intel Knowledge
Remote Offloading Framework for com.
Builder toolkit [http://software.intel.com/


Toward a Self-Learning and

Energy-Neutral IoT

...... A typical Internet of Things Such a simple IoT device is no longer must be detected or recognized in situ
(IoT) device comprises five components: adequate because emerging applications and reported immediately, because
sensor, microcontroller, memory, bat- (such as medical, structural/environmen- transmitting the sensor data via radio to
tery/energy harvester, and radio. It is a tal monitoring, and e-textiles) demand its host to do this will be costly in terms
device that collects, preprocesses, ambient intelligence or cognition and of energy and latency.1 For this reason,
stores, and transfers data received from real-time response from IoT devices. A the cognitive action must take place in
a sensor to a host (for example, a reader new class of IoT devices called self- the device, not in the host. For example,
via RFID, a smartphone via Bluetooth, or learning IoT devices will emerge to pro- an implantable chip must detect an
the cloud through a gateway) wherein vide cognitive services, such as situa- abnormal condition in the organ and
data processing is performed. The micro- tional awareness, anomaly detection, must take an action in real time. It cannot
controller is mainly responsible for con- activity, and pattern and emotion recog- afford to wait for a critical decision to be
trol and simple data preprocessing, and nition, which are essentially machine made by the host.
the radio is used to transmit short data learning algorithms. Real-time response A self-learning IoT device will accom-
packets. Hence, the battery can last for from self-learning IoT devices is needed modate multiple sensors and a more
years before it is recharged or replaced. because an anomaly or critical activity powerful computation engine to perform


computationally intensive sensor fusion data directly to the computation engine, signs of hospital patients.6 The flexible
and to run machine learning algorithms. such that it may have an analog prepro- armband contains a solar panel, piezo-
It will need a good size on-device mem- cessing front end tightly coupled to the electric speaker, temperature sensor,
ory (SRAM and nonvolatile memory) to sensors, and the machine learning algo- and power supply circuit, all of which
store the code and buffer the streaming rithms will run on the engine’s digital are organic components in a wearable
sensor data before and after data fusion. back end engine. Today, state-of-the-art form factor. It is self-powered by the
Integrating multiple sensors, a relatively microcontrollers have up to 4 Mbytes of solar panel, and the speaker sounds an
higher-performance computation engine, flash memory and much less static alarm when the temperature sensor
and more on-chip storage in a self- RAM, and future IoT devices will not measures a temperature between 36.5
learning IoT device will consume more have orders of magnitude larger on-chip to 38.5 degrees Celsius. These early
energy than a simple IoT device, and will storage because of the cost issues. demonstrators are the precursors of
put incredible pressure on the battery. Machine learning algorithms—in particu- future self-learning and energy-neutral
Self-learning IoT devices will be deployed lar, deep neural networks—take up a sig- printed IoT devices. The main advantage
in such environments in which recharg- nificant storage space, so it will be a of printed electronics is that they allow
ing or replacing the battery is not possi- challenge to store a large number of net- low-cost customization thanks to the
ble—for example, an IoT device work parameters on chip. Hardware and low-cost flexible substrate and materi-
implanted in a human body, integrated software compression techniques will als, and do not require costly clean
into a building’s foundation, or be used to deal with the large parameter rooms, unlike silicon. This offers a
embedded in the textile fabric. Hence, space in deep neural networks. Also, unique opportunity, in particular, to cus-
the battery in the device must operate alternative machine learning algorithms tomize the computation engine to the
for a long time (for example, more than that are more adaptive, resource effi- needs of the cognitive application that
10 years) and be charged by multiple cient, and energy efficient (such as non- will be running on the device. For exam-
energy harvesters that are integrated parametric Bayesian methods4) can be ple, an energy-efficient support vector
into the device to harvest ambient developed for self-learning and energy- machine (SVM) can be designed as a
energy (such as thermal, vibration, solar, neutral IoT devices. The management of custom computation engine (rather
pressure) in order to charge the battery. the harvested energy is a critical process, than using a less energy-efficient gen-
This is a concept called energy neutrality,2 and the data must be sensed, fused, eral-purpose computation engine) and
in which the battery will always be stored, and processed, and the response printed for a single-use smart packaging
charged by energy harvesters in the given, before the harvested energy in the product, because it will run only the
device3 and should never be recharged battery depletes. The harvested energy SVM. This will not be possible in silicon,
by human intervention. management must be performed by a because customization (that is, ASIC) is
The next phase in the IoT’s evolution combination of innovative software and extremely costly. Thus, printed elec-
is self-learning and energy-neutral devi- hardware techniques, such as the predic- tronics will pave the way to low-cost
ces having the properties of cognition, tion of the harvested energy before task customization of efficient computation
real-time response, and perpetual execution, or new instructions to control engines for future printed self-learning
energy. The main challenge is to run the energy harvesting process. and energy-neutral IoT devices.
computation and memory-intensive sen- Self-learning and energy-neutral IoT
sor fusion and machine learning algo-
rithms in a device powered only by the
harvested energy. This opens up oppor-
devices will also emerge in the printed
electronics world.5 Printed electronics
offers cost-effective fabrication of elec-
F uture IoT devices will become more
intelligent and aware of their envi-
ronment, and will integrate more capable
tunities to design novel computation tronics with low-cost substrates and computation engines to perform cogni-
engines, memory subsystems, and materials (such as plastic and paper), tive activities. However, these devices
energy management units, considering simpler processing and patterning will still be constrained by energy effi-
not only energy efficiency but also steps, and disposability. It has found ciency and limited energy capacity, as in
energy neutrality. The computation applications in sensors, RFIDs, solar today’s dumb IoT devices. They will be
engine in the device must be equipped cells, batteries, and displays in the fields so deeply embedded that they will not
with single-instruction, multiple data/ of medical, wearable, textile, automo- be accessible to replace or recharge their
digital signal processing capabilities and tive, and packaging applications. Smart batteries, and will have to depend on
be coupled with one or more machine printed devices have already been dem- energy harvesters to become self-
learning hardware accelerators (such as onstrated as smart tags, labels, pack- sustained or energy-neutral. This will be
a deep neural network). Alternatively, the ages, e-textiles, and wearables. For even more prominent for printed elec-
computation engine can be tightly example, T.E. Halterman built a printed tronic devices that will be manufactured
coupled with sensors that will stream alarm armband that monitors the vital for a single use. The main engineering

Computing Edge July 2017
challenge is to design an energy-neutral Communications Surveys & Tutorials, and Electronics over Large Flexible
device that will be deployed in critical vol. 16, no. 1, 2014, pp. 181–194. Substrates: A Review,” IEEE Sensors
missions and stay operational for a long 2. M. Magno et al., “Infinitime: A Multi- J., vol. 15, 2015, pp. 3164–3181.
time, but at the same time run computa- sensor Energy Neutral Wearable 6. T.E. Halterman, “Flexible, 3D Printed,
tionally complex machine learning algo- Bracelet,” International Green Com- Solar Powered Thermal Alarm for
rithms. Nevertheless, this challenge puting Conference (IGCC), 2014. Patient Monitoring,” 3D Print, 26 Feb.
brings up unique opportunities for sys- 3. A.S. Weddell et al., “A Survey of Multi- 2015; http://3dprint.com/47116/3d-
tem architects, designers, and software source Energy Harvesting Systems,” printed-fever-alarm.
developers to come up with holistic solu- Proc. Conf. Design, Automation and
tions not only for the self-learning and Test in Europe, 2013, pp. 905–908. Emre Ozer is a principal research engineer
energy-neutral IoT devices in silicon, but 4. Y. Raykov et al., “Predicting Room at ARM Research in Cambridge. Contact
also in emerging printed electronics. MICRO Occupancy with a Single Passive him at emre.ozer@arm.com.
Infrared (PIR) Sensor through Behav-
............................................................ ior Extraction,” Proc. ACM Int’l Jt.
References Conf. Pervasive and Ubiquitous Com-
1. R.C. Carrano et al., “Survey and Taxon- puting, 2016, pp. 1016–1027. This articleRead
originally appeared
your subscriptions in
the myCS publications portal at
omy of Duty Cycling Mechanisms in 5. S. Khan, L. Lorenzelli, and R. Dahiya, IEEE Micro, vol. 36, no. 6, 2016.
Wireless Sensor Networks,” IEEE “Technologies for Printing Sensors

D. Mills
D. Mills

Call for Software Engineering Award Nominations

Call for Software Engineering Award Nominations
Established in Harlan D. Mills’ name to recognize researchers and practitioners
Established in Harlan
who D. Mills’ name long-standing,
have demonstrated to recognize researchers
who have demonstrated
contributions to the theorylong-standing,
and practice ofsustained, and meaningful
the information sciences,
focusing to the theory
on contributions and practice
to the practice of the information
of software engineeringsciences,
thefocusing on contributions
application of sound theory.to the
Thepractice of software
award consists of a engineering through
$3,000 honorarium,
the application of sound
plaque, and atheory.
a $3,000 honorarium,
week of the annual
plaque, andon
International Conference a possible
Softwareinvited talk during
Engineering the
(ICSE), week of the annual
co-sponsored by the
Deadline for 2018 Nominations: International Conference
IEEE Computeron Software
Society Engineering
Technical (ICSE),
Council co-sponsored
on Software by the
15 for2017
October 2018 Nominations: IEEE Computer Society Technical Council on Software Engineering.
15 October 2017 The award nomination requires at least 3 endorsements.
Nomination site: The award nomination requires at leastare
Self-nominations 3 endorsements.
not accepted.
Nomination site:
awards.computer.org Self-nominations are not
Nominees/nominators doaccepted.
not need
awards.computer.org to be IEEE orNominees/nominators
IEEE Computer Society domembers.
not need
to be IEEE or IEEE Computer Society members.



Osmotic Flow: Osmotic

Computing + IoT Workflow
he rapid evolution of Internet of Things (IoT) devices
(e.g., sensors and gateways) and the almost ubiqui-
Matteo Nardelli tous connectivity (e.g., 4G, Wi-Fi, RFID/NFC, Blue-
University of Rome Tor Vergata
tooth, IEEE 802.15.4) are forcing us to radically
Stefan Nastic rethink how to effectively deal with massive volume, velocity,
and Schahram Dustdar and variety of big data produced by such IoT devices. There
TU Wien are currently 6.4 billion IoT devices in use around the world
and their number, capabilities, as well as the scope of their
Massimo Villari
University of Messina use, keeps growing rapidly. According to Gartner (http://www
.gartner.com/newsroom/id/3165317) the number of IoT devic-
Rajiv Ranjan es will reach 20.8 billion by 2020, and, by then, IoT service
Newcastle University
spending will reach $1,534 billion and hardware spending
$1,477 billion.

As IoT expands into various application domains in real-time (in the form of data ingestion and data
such as healthcare, utility grids, cities, agriculture, analysis).
transportation, industry 4.0, and disaster manage- A possible solution to augment the scalability of
ment, need for investigating on-the-fly computa- CDCs lies in taking advantage of the ever-increasing
tion over the IoT data streams is ever more pressing. computational and storage capabilities available at
Indeed, most IoT applications are modeled as data the network edge.2,3,4 We note in the previous instal-
transformation workflows that consists of: i) mul- ment of “Blue Skies” sensing and networking devices
tiple interdependent, heterogeneous data analysis available at the network edge constitute a new type
computational and programming models that re- of computing infrastructure, the Edge Datacentre
alise various data transformation tasks from data (EDC).5 An EDC may vary in scope and capability,
ingestion to analysis, ii) virtualised/non-virtualised including gateways (Raspberry Pi 3, UDOO board,
computational and network infrastructure, iii) esp8266, Meshlium Xtreme, Arduino), software de-
communication media of various kinds (including fined network solutions (e.g. Cisco IOx), or smart
wireless). Currently, powerful Cloud Datacentres phones equipped with sensors. To facilitate highly
(CDCs, e.g. AWS1) provide computation and data distributed and federated computing environments,
storage resources for IoT workflows, but they suffer we proposed Osmotic Computing paradigm5 that en-
from limited bandwidth and network latency, and ables the automatic deployment of microservices over
support neither latency-sensitive applications nor inter-connected EDC and CDC. The benefits of in-
applications that rely heavily on the data streaming tegrating EDC and CDC has already been recognised
from IoT data sources for computing intelligence by several companies and open source initiatives, in-

32 July 2017 Published by the IEEE Computer Society 2469-7087/17/$33.00 © 2017 IEEE
68 I EEE CLO U D CO M P U T I N G P U B L I S H ED BY T H E I EEE CO M P U T ER S O CI E T Y 2 3 2 5 - 6 0 9 5 / 1 7/ $ 3 3 . 0 0 © 2 0 1 7 I E E E
cluding CISCO, AWS1, and Google3, and the Open- A data transformation task encapsulates a microser-
Fog Consortium.4 For example, AWS has enriched its vice (e.g., Docker, Unikernel), a computational mod-
CDC offerings with near-edge computing and storage el (e.g. statistics, clustering, classification, anomaly
capabilities (i.e., Snowball Edge, Greengrass). detection, accumulation), and a data analysis pro-
Nevertheless, the usage of an Osmotic Com- gramming model (e.g., stream processing, batch pro-
puting infrastructure (CDC+EDC) poses new chal- cessing, SQL, NoSQL, data ingestion).
lenges for IoT workflow application developers and
operations managers as they need the awareness of Motivation
resource/device (CDC server vs. IoT gateway) het- Let us consider a contemporary Smart City, where
erogeneity, virtualisation software heterogeneity a plethora of IoT sensing devices with Internet con-
(e.g., hypervisor vs. container), data analytic pro- nectivity are disseminated all over the urban en-
gramming model heterogeneity (stream processing vironment. Buses, trains, and taxis continuously
vs. batch processing), geographic distribution, and communicate their position; vehicles notify congested
network performance uncertainties. routes; citizens often geo-locate their position in mes-
Existing streaming data analysis platforms includ- sages, photos, videos, or accessing specific services.
ing (e.g., Spark6, Heron7, Google Dataflow8, AWS, All these IoT data sources continuously produce
Kinesis1, StreamCloud, Apache Storm), are CDC- ever-increasing streams of data that can be collect-
centric, hence they do not meet the resource manage- ed and processed to get the so-called “pulse of the
ment and scheduling requirements for IoT workflows city”, thus fostering awareness and capacity of tak-
that require coordinated mapping for data analysis ing informed decisions. Intelligent services aimed
activities to both CDC and EDC. Many workflow ap- at improving the citizens’ quality of life can be built
plication management platforms such as Pegasus, Tri- by merging, filtering, correlating, and transforming
ana, Taverna, Galaxy, e-Science Central, and Kepler these diverse data streams.
support the development, deployment and execution For example, a smart traffic light IoT application
of scientific workflow applications on CDC without (see Figure 1) can identify traffic congestions and
considering newly evolved EDC capabilities. Apache proactively change traffic light priorities and speed
Oozie and Linkedin Azkaban support a Hadoop work- limits, so to reduce ripple effects and relieve the en-
flow, but in a rather rigid manner that works well for vironmental impact. Let us focus on a single road di-
only batch processing activities. Data analytics plat- vided in multiple segments and managed by traffic
forms such as YARN, Mesos Amazon IoT and Google light. The traffic light is instrumented with appropri-
Cloud Dataflow can support manual provisioning of ate IoT sensor (e.g., light state sensor, CCTV) and ac-
multiple data transformation tasks on CDC resources, tuator. The traffic light sensor produce data streams
but only in a performance-agnostic way. about their current state (i.e., color of light turned
on and light change timings) while the CCTV (traf-
The Osmotic Flow Model fic congestion indicator) produces visual evidence of
We propose Osmotic Flow, a new model for holisti- congestion. The smart traffic light IoT application as-
cally programming, mapping and executing IoT data sumes that traffic congestion is directly proportional
transformation workflow applications on a distrib- to the number of cars queued at each intersection
uted infrastructure combining both EDC and CDC and inversely proportional to the average speed per
resources. In the Osmotic Flow model, an IoT work- road segment, each road segment is equipped with
flow application is modelled as a directed (potential- above IoT sensor and actuator. The First analytic
ly cyclic) graph with data transformation tasks as its task merges the data streams from the light sensors
nodes, and dataflow dependencies (or control flow and CCTV sensors to develop awareness on traffic
dependencies for computational synchronization, congestion across the dependent road segments. The
if/as needed) between data transformation tasks as second analytic task combines output from first ana-
its vertices. Osmotic Flow model permits data trans- lytic tasks with appropriate traffic flow computation-
formation tasks to be distributed, managed, and ex- al model to develop aggregated knowledge of traffic
ecuted across any available CDC and EDC provider. flow and congestion across the segments. The traffic

www.computer.org/computingedge 33
M A R CH/A P R I L 201 7 I EEE CLO U D CO M P U T I N G 69

Smart traffc light IoT application

Traffic lights sta Traffic lights management

te information

Traffic cicator

Cloud computing

Edge computing

FIGURE 1. A high-level description of the smart traffic light IoT workflow application

flow computational model dynamically emits the Scalability and Elasticity. Due to the huge amount
traffic light control commands to the actuators about of data that will be processed in a real-time fash-
the switching off/on of the traffic lights across the ion, scalability represent a key design requirement
segments such as that it leads to optimal traffic flow. for IoT workflow applications. For example, a recent
The classic approach for realising this kind of analysis of a (single) healthcare-related IoT workflow
IoT workflow application (see Figure 1) relies ex- application (with 30 million of users) showed data
clusively on CDC resources, which could be distant flows up to 25,000 tuples per second.10 The Osmotic
from data sources, hence leading to excessive event Flow model should consider scalability and elastic-
detection (e.g., traffic congestion) delay. For exam- ity by design, so that applications can automatically
ple, the first analytic step of the traffic light sensor grow and shrink based on data volume and velocity.
data aggregation should be mapped to nearby EDC
resource, while resource-intensive second analytic Focus on data transformation. The IoT application
task should be mapped to CDC resource, as it needs providers, who wants to realise a data transformation
to execute complex traffic flow computational model. workflow, desires to focus only on data transforma-
tion, without spending too much time on manage-
Design Goals of the Osmotic Flow model ment or configuration operations. Hence, the Osmotic
We identify the key requirements that drive the de- Flow model should enable an easy deployment in-
sign of Osmotic Flow programming model. terface where data transformations should be easily

34 Computing Edge July 2017

Output endpoint
IoT workflow

Input endpoint
defined and, at the same time, each transformation
should be seamlessly mapped to either EDC or CDC Transformation
based on performance needs. As a consequence, the function
deployment process is transparently performed by the
underlying run-time engine. However, the application
providers can customize the framework behavior so Simple Complex
to better address his/her specific performance needs. Transformation transformation

Efficient composition of data transformations. The Contract

Osmotic Flow model should support the composi- Placement Governance QoS
tion of cross-workflow data transformations and
linking, so to easily realise complex workflows. To
this end, Osmotic Flow should consider by design FIGURE 2. Depiction of IoT Transformation Functions in the Osmotic
the possibility of composing data streams coming Flow model
from multiple, public IoT devices and applications,
thus promoting the principle of sharing and reus-
ability. Our Osmotic Flow model should allow the • a transformation function or task: it encapsulates
application provider to easily define new streams, the user-defined analytics logic which transforms
which extract high value information from raw data, (e.g., combines, filters, splits) incoming data
without worrying about low level concerns related to streams and passes the results to next transforma-
their runtime execution, such as resource allocation, tion functions or final sinks of the workflow.
streams deployment, elasticity, and governance. • a contract: it is a high-level configuration and
performance requirement descriptor of the
Network Awareness. The emerging IoT environment transformation functions or tasks.
calls for strong network awareness. The Osmotic
Flow model should not neglect the presence of com-
munication delays while performing the deployment Types of IoT Data Streams
of data transformation tasks to CDC and/or EDC. An IoT data stream can be ephemeral or public. An
ephemeral stream is a special kind of stream that
Main Entities in the Osmotic Flow model exists only if a sink is (directly or indirectly) inter-
In Osmotic Flow, as depicted in Figure 2, includes ested to incoming flow. An indirect interest is mani-
one or more input endpoint that receives data from fest when one or more streams lie in between the
an external data sources and one or more output stream and the final destination is the workflow
endpoints, which emit processed or transformed endpoint (i.e., sink). Being ephemeral, the existence
data towards sinks or other streams. A workflow is of the stream depends on the presence of (direct or
characterised by the following elements: indirect) interested sinks and its scope is restricted
within the same application, i.e., it can be used only
• one or more data sources: a data source is an entity, by user-defined transformations running within the
potentially external to the system, that continu- same application that contains the stream. A public
ously generates events or data. For example, a data stream is a globally available stream and, as such,
source can be an IoT device emitting temperature can be part of more than one IoT workflow applica-
measurements or traffic congestion conditions. tion (for example traffic pattern stream data can be
• one or more sinks: a sink is a final endpoint in used for smart traffic management applications as
the data transformation flow (or pipelne). In- well as air pollution monitoring applications).
terested parties can subscribe to the sink for
receiving notification information (e.g., traffic Types of Transformation Tasks
congestion information requested by drivers in Transformation functions are the only piece of code
Figure 1). that has to be defined by the application providers. A

www.computer.org/computingedge 35
M A R CH/A P R I L 201 7 I EEE CLO U D CO M P U T I N G 71

transformation function encapsulates the data analytics • Placement constraints: these constraints guide
logic. For an efficient execution, the stream model re- the placement of transformation tasks over the
quires transformation functions to be as scalable as pos- distributed CDC+EDC infrastructure. If no re-
sible, therefore the latter are either stateless or provide strictions are provided, the run-time execution
an explicit definition of their state. We distinguish be- engine can deploy a transformation task either
tween two kind of transformations: simple and batch. on CDCs or on EDCs. Placement constraints
A simple transformation is applied to every in- be included to, e.g., maximize the utilization
coming data in parallel and produces zero, one, or of nearby EDC resources or exploit centralized
more outgoing data. For example, a simple transfor- Cloud resources.
mation can be realised using only one type of data • Governance constraints: together with the previ-
analysis programming model. An example of simple ous one, the governance rules enable to specify
transformation (in context of Figure 1) could include further restrictions regarding the transforma-
tracking vehicles that exceed speed limit. tion task deployment and adaptation. These re-
A complex transformation fuses one or more strictions are often related to security, privacy,
data streams before applying a transformation us- or law concerns. For example, a governance rule
ing one or more data analysis programming models can exclude every edge resource belonging to a
(e.g., stream processing, batch processing, NoSQL). specific geographical region or can require to
Complex transformation can produce zero, one, or encrypt the exchanged data, so to meet strin-
more outgoing data streams. The group of incom- gent law restrictions.
ing data streams fully determine the function state, • QoS or performance constraints: these ones ex-
which can then be manipulated by the transfor- press non-functional properties that should be
mation. A flow of data can be determined accord- met during the stream execution, so to obtain
ing to two modes: window and window-and-key. A a desired quality level. For example, constrains
window-based transformation creates a time-based can bound the maximum stream latency or min-
or count-based window of events that have to be imum stream throughput or the event detection
combined before running the transformation. For accuracy.
example, a window-based transformation can com-
putes statistics on traffic patterns (see Figure 1) on
a road segment in the last 30 seconds. On the other Osmotic Flow Scheduling Architecture and
hand, a window-and-key transformation is a special Research Issues
case of windowed transformation that has a finer Figure 3 provides a system-level description of the
granularity in selecting the data streams for fusion. Osmotic Flow model. Whenever an IoT application
A classic example for a window-and-key transforma- provider wants to execute a workflow of data trans-
tion is the implementation of the vehicle counter formations, he/she submits the application code to
task, which computes statistics on how many times the nearest Osmotic Resource Manager using a sub-
a particular vehicle has travelled across a road-seg- mission client. Then, the Osmotic Resource Man-
ment in last hour/week/month. Hence, the trans- ager allocates a new Node Manager, which, in turn,
formation across both historical and real-time data first determines the application placement, gover-
is dependent on the same key (i.e., vehicle registra- nance, and QoS constraints, and then distributes
tion number). the data transformation tasks to appropriate EDC
Simple transformations are stateless function, and/or CDC resources. The main software compo-
whereas complex transformations provide an explicit nents include
definition of their current state (real-time data) as
well as it depends on the past state (historical data). Osmotic Resource Manager (ORM). An Osmotic
Resource Manager coordinates Edge and Cloud
Contract resources and supports the Node Manager in de-
The contract provides a high-level description of the termining the placement of Osmotic Flow transfor-
IoT workflow application’s configuration including: mation functions. In the proposed model, multiple

36 Computing Edge July 2017

ORM can coexist and cooperate, where each one
coordinates a pool of nearby CDC/EDC resources.
The federation of multiple ORM enables the deploy-
ment of applications on the combined (EDC+CDC) USR
infrastructure. ORM
To ensure scalable communication and coordi- * NM W W
nation between ORMs, future research should focus
on developing self-healing load coordination proto- Cloud computing
cols that can cope with changes in the infrastruc- * Edge computing
tures and IoT device state, and that can dynamically
adapt to failures, connections, and detachments of
ORMs and EDC/CDC resources. Another research
thread could be to develop cooperative and oppor- NM W NM W W
ORM Osmotic Resource
tunistic workload coordination protocol such that Manager
ORMs are able to balance their workload with each NM W W NM Node Manager
other in order to make sure that no CDC/EDC re- W Worker
USR Universal Stream
source are wasted due to redundant data streams. Repository
Several workload coordination solutions already
exist for CDC environments (e.g., Quincy, Omega,
Sparrow, Mercury),11 nevertheless the features of FIGURE 3. Scheduling Architecture of Osmotic Flow Model
this new environment (CDC+EDC), as well as the
characteristics of the Osmotic Flow model, foster
the development of new load coordination policies scribe concepts as IoT sensor characteristics, or data
and protocols, tailored for the specific setting where formats; nevertheless, they are not suitable to cap-
a significant heterogeneity of resources as well ture characteristics relevant to CDC/EDC resources.
as multiple of data transformation tasks has to be
management. Node Manager. The Node Manager is a per-ma-
chine agent that supports the ORM in control-
Universal Stream Repository (USR). To enable the ling the available resources of the EDC/CDC
sharing and reuse of high-value streams, Osmotic infrastructure. Besides launching and terminat-
Flow includes a Universal Stream Repository (USR), ing the execution of workers, the Node Manager
which collects and provides the descriptor of every monitors and reports statistical information about
public stream available in the Osmotic Flow eco- resource utilization (i.e., CPU, memory, network)
system. An IoT application provider relies on the to the ORM. Moreover, the Node Manager pro-
descriptor to discovery existing streams and reuse vides information to the ORM for determining the
them within his/her application. communication delays of the node with the other
Therefore the future research should focus on components of the infrastructure. Observe that
developing holistic data model for expressing EDC/ communication delays can be obtained by means
CDC resources and data stream characteristics us- of either active/passive measurements (e.g., with
ing an ontology-based representation, which enables a network coordinate system12), or with some net-
encoding both dynamic (e.g., performance, status of work support (e.g., SDN).
stream) and static (e.g., functionality provided, types Though Simple Network Management Protocol
of events) QoS parameters. The ontology will thus (SNMP), using the Management Information Base
guide decisions made on the types of data transfor- (MIB), has been highly adopted for monitoring re-
mation tasks that are deemed most suitable for the sources in CDC, it lacks the ability to monitor EDC
deployment in the CDC or at the network Edge. An resources (as identified by the Internet Engineering
open access USR should be built. Existing ontologies Task Force13) due to huge computational overhead
such as Semanrtic Sensor Network (SSN) can de- and the constrained nature of Edge resources. Hence,

www.computer.org/computingedge 37
M A R CH/A P R I L 201 7 I EEE CLO U D CO M P U T I N G 73

future research will need to investigate modeling of a ill data, several data analytic programming
novel Edge resource monitoring agent that harness- models and frameworks have been proposed.
es lightweight IoT protocols such as Constrained Ap- Nevertheless, most of them are designed to run in
plication Protocol and a new Edge resource-specific CDC, thus neglecting the presence of EDC resourc-
MIB interface provided by the Internet Engineering es. Osmotic Flow builds on the strengths of existing
Task Force.13 solutions and creates a novel approach for execut-
ing data analytics in a Cloud-supported Edge envi-
Worker. The Worker is in charge of executing one ronment. Similar to Google Cloud Dataflow8 and
or more transformation functions. To this end, a Apache Spark, Osmotic Flow defines a very simple
worker collects data from the stream data source and scalable programming model that enables to
(i.e., another worker or an external data source), automatically deploy transformations with a high
runs the user-defined transformation function, and degree of parallelism. However, differently from ex-
emits outgoing streams. In other words, the worker isting approaches, Osmotic Flow focuses mainly on
takes care of the distributed execution of the user processing continuous and unbounded streams of
code, that defines only how to manipulate input data data on decentralized CDC+EDC resources. More-
to obtain output data. Since streams are executed by over, since the Osmotic Flow includes resource
workers, they directly communicate to transfer data management capabilities, it can optimize how Edge
up to the final consumers. Being stateless or with a and Cloud nodes are allocated among multiple and
window-based state, multiple transformation func- concurrent data transformation functions.
tions can run concurrently in a worker, and multiple
workers can run concurrently on the infrastructure. References
Moreover, since a transformation function is defined 1. Amazon Web Services, https://aws.amazon.com
with a fine granularity (i.e., per event or per win- 2. F. Bonomi, R. Milito, J. Zhu, and S. Addepalli.
dow), it can be transparently scaled as the number Fog computing and its role in the Internet of
of incoming events increases or decreases, up to— Things. In Proc. of MCC ’12, pages 13–16. ACM,
theoretically—creating an instance per each event. 2012.
As Osmotic Flow model thrives to support mul- 3. Google Edge Network, https://peering.google.com.
tiple type and mix of data transformation tasks on 4. OpenFog Consortium, https://www.openfog
shared EDC+CDC infrastructures, the Worker consortium.org.
need to be equipped with scheduling intelligence 5. M. Villari, M. Fazio, S. Dustdar, O. Rana and R.
to automatically discover and resolve contention be- Ranjan, “Osmotic Computing: A New Paradigm
tween co-deployed data transformation tasks. Dur- for Edge/Cloud Integration,” in IEEE Cloud
ing deployment of data transformation tasks, the Computing, vol. 3, no. 6, pp. 76-83, Nov.-Dec.
Worker must consider which data transformation 2016. doi: 10.1109/MCC.2016.124.
tasks should be combined on an EDC and/or CDC 6. M. Zaharia, M. Chowdhury, T. Das, A. Dave, J.
resource, to minimize resource contention due to Ma, M. McCauley, M. J. Franklin, S. Shenker,
workload interference. Workload resource consump- and I. Stoica. “Resilient distributed datasets: A
tion and QoS are not additive, so understanding fault-tolerant abstraction for in-memory cluster
the nature of their composition is critical to decid- computing”. In Proc. of USENIX NSDI ’12, 2012.
ing which transformation tasks can be deployed 7. S. Kulkarni, N. Bhagat, M. Fu, V. Kedigehalli,
together. Existing content detection approaches C. Kellogg, S. Mittal, J. M. Patel, K. Ramasamy,
such as Paragon14 that applies collaborative filter- and S. Taneja. “Twitter Heron: Stream Process-
ing techniques for resolving contention between co- ing at Scale”. In Proc. of SIGMOD ‘15, 2015.
deployed, hypervisor-based application workloads on 8. Google Cloud Dataflow, https://cloud.google.com/
CDC are agnostic to the new hardware (e.g. Rasp- dataflow/
berry, Pi 3, UDOO board, Cisco IOx) and virtualisa- 9. V. Gulisano, R. Jiménez-Peris, M. Patiño-
tion features (e.g., Containers, Unikernels) of EDC Martínez, C. Soriente and P. Valduriez, “Stream-
resources. Cloud: An Elastic and Scalable Data Streaming

38 Computing Edge July 2017

System,” in IEEE Transactions on Parallel and Computing Lab (PC3L). Contact him at snastic
Distributed Systems, vol. 23, no. 12, pp. 2351- @infosys.tuwien.ac.at.
2365, Dec. 2012.
10. R. Cortés, X. Bonnaire, O. Marin, and P. Sens, SCHAHRAM DUSTDAR is a full professor of com-
“Stream processing of healthcare sensor data: puter science heading the Distributed Systems Group
studying user traces to identify challenges from at TU Wien, Austria. His work focuses on Internet
a big data perspective”, Procedia Computer Sci- technologies. He is an IEEE Fellow, a member of the
ence, vol. 52, 2015, pp. 1004-1009, 2015. Academy Europeana, and an ACM Distinguished
11. P. Pietzuch, J. Ledlie, J. Shneidman, M. Rous- Scientist. Contact him at dustdar@dsg.tuwien.ac.at
sopoulos, M. Welsh, and M. Seltzer “Network- or dsg.tuwien.ac.at.
aware operator placement for stream-processing
systems”. In Proc. of IEEE ICDE ’06, 2006. MASSIMO VILLARI is an associate professor of
12. F. Dabek, R. Cox, F. Kaashoek, and R. Morris. computer science at the University of Messina. His re-
“Vivaldi: A decentralized network coordinate search interests include cloud computing, Internet of
system”. SIGCOMM Comput. Commun. Rev., Things, big data analytics, and security systems. Vil-
34(4), 2004. lari has a PhD in computer engineering from the Uni-
13. P. V. D. Stok et al.,”CoAP Management Inter- versity of Messina. He’s a member of IEEE and IARIA
faces,” October 2016, IETF Internet-Draft Work- boards. Contact him at mvillari@unime.it
in-Progress, https://datatracker.ietf.org/doc/draft
-vanderstok-core-comi/. RAJIV RANJAN is a reader in the School of Com-
14. C. Delimitrou and C. Kozyrakis. 2013. Paragon: puting Science at Newcastle University, UK; chair
QoS-aware scheduling for heterogeneous data- professor in the School of Camputer, Chinese Uni-
centers. SIGPLAN Not. 48, 4 (March 2013), 77-88. versity of Geoscience, Wuhan, China; and a visiting
DOI=http://dx.doi.org/10.1145/2499368.2451125. scientist at Data61, CSIRO, Australia. His research
15. D. Puthal, S. Nepal, R. Ranjan and J. Chen, interests include grid computing, peer-to-peer net-
“Threats to Networking Cloud and Edge Data- works, cloud computing, Internet of Things, and big
centers in the Internet of Things”, IEEE Cloud data analytics. Ranjan has a PhD in computer science
Computing, vol. 3, no. 3, pp. 64–71, 2016. and software engineering from the University of Mel-
bourne (2009). Contact him at raj.ranjan@ncl.ac.uk
or http://rajivranjan.net.
MATTEO NARDELLI is a PhD student of computer
science at the University of Rome Tor Vergata, Italy.
His research interests are in the field of distributed
computer systems, with a focus on the execution of
data stream applications in geographically distrib-
uted environments. Contact him at nardelli@ing

STEFAN NASTIC is a Postdoctoral Research Assis-

tant at the Distributed Systems Group at TU Wien,
Austria. His research interests include: Internet of
This article originally appeared in
Things and Edge Computing; Cloud Computing;
IEEE Cloud Computing, vol. 4, no. 2, 2017.
Big Data Analytics; and Smart Cities. He received
a PhD in software engineering and Internet tech-
nologies from TU Wien. Nastic has been involved
in several EU-funded research projects such as Read your subscriptions through
the myCS publications portal at
SMART-FI, U-Test and SM4ALL, as well as, large http://mycs.computer.org.
industrial projects such as Pacific Controls Cloud

www.computer.org/computingedge 39
M A R CH/A P R I L 201 7 I EEE CLO U D CO M P U T I N G 75
Editor: Yong Cui • cuiyong@tsinghua.edu.cn

Internet of Things for Smart

Cities: Interoperability and
Open Data
Bengt Ahlgren • SICS Swedish ICT

Markus Hidell • KTH Royal Institute of Technology, Sweden

Edith C.-H. Ngai • Uppsala University, Sweden

The Internet of Things (IoT) for smart cities needs accessible open data and
open systems, so that industries and citizens can develop new services and
applications. As an example, the authors provide a case study of the GreenIoT
platform in Uppsala, Sweden.

oday’s cities face a variety of challenges, infographics/IoT). The IoT market size is forecast to
including job creation, economic growth, grow from US$157 billion in 2016 to $661 billion
environmental sustainability, and social resil- by 2021.2 The adoption of cloud platforms, devel-
ience. Emissions from motor vehicles have become opment of cheaper and smarter sensors, and evolu-
a major source of air pollution in the world’s large tion of high-speed networks are expected to drive
and medium-sized cities. Many large cities expe- the growth of the IoT market.
rience serious air pollution and greenhouse gas Many cities, such as London and New York,
emission (GHG), which is made worse by increas- see the increasing need and interest of the pub-
ing traffic congestion. With these challenges in lic sectors to explore IoT technologies to improve
mind, the European Union and many other coun- traffic flow, reduce pollution and energy con-
tries are investing in information and communica- sumption, and collect data for policing. Smart cit-
tion technology (ICT) research and innovation, and ies are an urban development vision to integrate
developing policies to improve the quality of life of multiple ICT solutions to manage a city’s assets
citizens and sustainability of cities. Given the trend to create a sustainable environment, improve the
of ICT for smart sustainable cities, understanding quality of life, and enhance efficiency and eco-
where we are in the evolution of the Internet is nomical value. The number of new IoT products
critical to future city-planning processes. and applications has grown exponentially in
The Internet of Things (IoT) has been viewed recent years. Various communication standards
as a promising technology with great potential for and protocols have been suggested in the com-
addressing many societal challenges. Cisco believes munity, and some have been adopted in differ-
that many organizations are currently experienc- ent IoT devices. However, there are also quite
ing the IoT, the networked connection of physi- a few proprietary protocols and cloud services
cal objects and the cyberspace.1 According to the in the IoT, which make the interoperability and
International Data Corporation (IDC)’s Worldwide sharing of data across different devices and plat-
Internet of Things Forecast, 2015–2020, 30 bil- forms quite challenging. Open data in smart cities
lion connected (autonomous) things are predicted means not only global data collected and opened
to be part of the IoT by 2020 (see www.idc.com/ by the government, but also includes the sharing

52 Published by the IEEE Computer Society 1089-7801/16/$33.00 © 2016 IEEE IEEE INTERNET COMPUTING
40 July 2017 Published by the IEEE Computer Society 2469-7087/17/$33.00 © 2017 IEEE
Internet of Things for Smart Cities: Interoperability and Open Data

Table 1. Standardized IP-based communication protocols for Internet of Things (IoT) devices.
Layer Protocol
Application IETF Constrained Application Message Queuing Telemetry Transport (MQTT)
Protocol (CoAP)/REST engine
Transport UDP TCP
Network IPv6, RPL
Adaptation IPv6 over low-power wireless personal area networks (6LoWPAN)
Media access control (MAC) Carrier sense multiple access (CSMA)
Physical IEEE 802.15.4

of data among individual citizens and and costs.5 Similarly, in the US, Cisco instance, IEEE 802.15.4 has been widely
industries with the government and and a wide range of public and private adopted in many smart devices as the
general public. In this article, we’ll stakeholders in Chicago have been MAC and Physical layer protocol. Sev-
discuss the advantages of open data driving smart community initiatives to eral network layer and application layer
and standards within the IoT, current improve neighboring services and the protocols have also been proposed for
limitations, and future trends. quality of life.6 IoT solutions are more constrained devices. Standard protocols
effective when they facilitate open are important to guarantee interoper-
IoT for Smart Cities data and encourage public engage- ability of different IoT devices.
The IoT provides individuals, society, ment, to achieve the goals of increas- However, using open standards
and the business world new opportu- ing productivity, decreasing costs, and doesn’t automatically result in open
nities to access volumes of data and to improving citizens’ quality of life. systems. In our context, an open sys-
develop new applications and services tem means an integrated open IoT
for creating a cleaner environment and Interoperability and Open infrastructure solution for smart cit-
more intelligent society.3 The informa- Standard Development ies, providing access to open data and
tion society is rapidly becoming a cen- With the popularity of IoT devices, APIs for cloud services. In many cit-
tral pillar for urban planners, architects, many IoT protocols and standards ies, that infrastructure will be paid for,
developers, and transportation provid- have been developed. In contrast to at least in part, by the city authorities
ers, as well as in public service provision. ordinary computers, IoT devices are using public funding. To motivate this
One good example is using smartphones normally constrained when it comes to investment, and get the most benefit
and smart meters to regulate energy memory space and processing capac- for society, we argue that any smart city
consumption in the Hyllie smart net- ity. In addition, IoT devices might be IoT infrastructure needs to be a truly
works of Malmö, Sweden.4 The system deployed where there’s limited or no open system, where equipment from
enables people to measure, monitor, access to continuous power supply, many vendors can be used, and where
control, and influence their own energy which means that they need to operate the generated data can be more or less
consumption, and be able to indepen- under power supplied from batteries or freely used by anyone to develop new
dently produce renewable energy (for small solar panels. As a consequence, services, based on low-level as well as
example, by using solar panels). One power-efficient communication pro- processed sensor and IoT data. This
way to optimize the use of renewable tocols with small memory footprints kind of system will maximize innova-
energy and reduce costs is to decide and limited demands on processing tion in the IoT domain, much as the
how and when you want to charge your have been developed to support IoT Internet has done for information and
electric car. Consumers are informed of devices. Traditional TCP/IP protocols communication services.
the supply of renewable energy in the haven’t been designed with these Many current IoT systems — for
system and how much electricity costs requirements in mind. Over the past example, for air quality monitoring or
via smartphones or tablets. years, however, IoT protocols have the smart home — are either incom-
From a public sector leadership per- been standardized on virtually all lay- plete systems with limited function-
spective, cities can be viewed as micro- ers of the protocol stack. These proto- alities (that is, in terms of sensing,
cosms of the interconnected networks cols typically have low complexity as storage, and analytics), or are closed,
for building a clean, energy-efficient, an important design goal and are opti- proprietary systems dedicated for a
and sustainable society. In Amster- mized for constrained environments. particular task. The latter are verti-
dam, a network-enabled LED street- Table 1 shows a few examples of cally integrated systems, sometimes
lighting system has been developed to IP-based open protocol standards com- called stove pipes or vertical silos,
reduce the city’s energy consumption monly used for IoT communication. For which can’t be combined or extended

www.computer.org/computingedge 41

applications, such as environmental

Sensor on city
infrastructure monitoring, transportation, factory
process optimization, and home secu-
Air pollution rity, and enables third-party innova-
monitoring tion in new IoT-based services. Driven
by Uppsala Municipality, we imple-
Street lighting ment and demonstrate GreenIoT as
City planning control a testbed in the city of Uppsala (the
fourth largest city in Sweden) to sup-
port air pollution monitoring and traf-
fic planning. Because the particulate
on vehicles Sensors on Wearable sensors level of Uppsala occasionally exceeds
the EU standard, in particular during
the winter and early spring, one objec-
Traffic control Smart parking tive is to reduce air pollution through
active monitoring, traffic manage-
ment, and better city planning.
Existing IoT technologies have
Figure 1. An IoT system that includes heterogeneous sensors to collect data for largely contributed to hardware,
smart city development. (Photo provided courtesy of Bengt Ahlgren.) software and protocol design. How-
ever, a major challenge of the IoT lies
in how to extract valuable informa-
easily with third-party components well-defined interfaces and data for- tion from vast volumes of data gen-
or services. The result is that once mats that can unleash the potential of erated from the smart devices (also
invested in a particular system, you’re open data, and that enable third parties known as the “Big Data” problem). Our
locked into that vendor’s system. Ver- to independently develop new applica- GreenIoT solution leverages cloud
tically integrated systems are particu- tions and services, possibly combining computing to support intelligent data
larly problematic for the public sector, several data sources. Providing open management, and integrate with
because this prevents fair competition data has huge potential for innovation green networking and sensing tech-
in public procurement and is less suit- in digital applications and services, niques to support energy-efficient and
able for large-scale data sharing. resulting in very large economic values. sustainable operations. The GreenIoT
Patrik Fältström7 argues similarly These interfaces (APIs) through which platform in Uppsala will be based
that market forces work against open the IoT data can be accessed at mul- on open standards, open to the pub-
interoperability, especially in the IoT tiple levels of refinement — from raw lic and supporting industries to test
domain where, for example, a smart data directly from sensors, to highly their new sensing products. It pro-
lighting system from one vendor only processed data — also need standard- vides open data and open APIs for
works with light bulbs from the same ization. The challenge is to provide an third parties to access the sensor
vendor. Systems are designed as end- open system that lets users access the data and make use of the cloud ser-
to-cloud-to-end, where the cloud part open data and cloud services without vices. The open data generated by
is vendor-controlled with limited pos- being locked by a particular platform. the smart devices and platform will
sibilities for third parties, and where The open system should also allow drive the development of innovative
the IoT devices often speak proprie- third-parties to innovate based on the applications and services.
tary protocols to the cloud. Fältström open data and open APIs. One major goal of the project is
argues that this lack of interoperabil- an integrated solution for an envi-
ity severely limits the market growth Case Study: GreenIoT Project ronmental sensing system, which
(for example, with smart light bulbs). in Sweden enables experimentation with appli-
Also, the dependence on a cloud ser- We developed a GreenIoT solution that cations and services using open
vice might render the device non- incorporates smart sensing and cloud environmental data, particularly for
functional, should that cloud service computing technologies to encompass sustainable urban and transportation
for any reason, temporarily or perma- a more interactive and responsive city planning (see Figure 1). The GreenIoT
nently, disappear. administration with private and public architecture is manifested in terms
Instead of these stove pipes, we need parties. The proposed open GreenIoT of a testbed in Uppsala. The sens-
horizontally designed systems with platform supports a wide range of ing system and application platform

54 www.computer.org/internet/ IEEE INTERNET COMPUTING

42 Computing Edge July 2017
Internet of Things for Smart Cities: Interoperability and Open Data

are built from unique technology

that provides open interfaces at
several levels, energy and resource API
efficiency, and application inde- Fetch data,
pendence. We use a unique tool for Processing
preprocessing, and so on
visualization in four dimensions,
which supports smart city simula-
tions and is fully integrated with the
sensor data for real-time feedback.
Data format, encoding,
The testbed, including the open data Sensor metadata, and so on
and open APIs, allow third parties to Gateway
Sensor network
develop and experiment new sensing
products and services that could be
exported to international markets. Figure 2. The GreenIoT architecture. Our focus is on open access and
To fulfill user requirements — from interoperability, to fulfill user requirements – from advanced tools for city
advanced tools for city planning as planning as well as from novel applications making sensor data useful to citizens.
well as from novel applications mak-
ing sensor data useful to citizens — processing that can be executed by of open IoT infrastructure – including
we devised the GreenIoT architecture the sensor data cloud services and open APIs, common data formats, and
(see Figure 2). then retrieve refined data according how to avoid vendor lock-in. Open
Data produced by sensor net- to their demands. The open APIs and systems enabling innovation in new
works are delivered through sensor open data format will facilitate the services are particularly important for
gateways for storage and processing sharing of open data and guarantee publicly funded IoT infrastructures, to
managed by cloud services for sen- the accessibility of cloud services maximize the benefits for society.
sor data. The sensors use a publish/ without relying on a single device
subscribe protocol, Message Queu- manufacturer or service provider. Acknowledgments
ing Telemetry Transport (MQTT), to This work is supported in part by the GreenIoT
communicate data in an open format project grant (2015-00347) from VINNOVA,
through a broker for further storage
and processing in the cloud, or for
direct use by applications and ser-
T he vision of the “smart city,” mak-
ing use of the IoT to provide ser-
vices for the good of citizens and
Sweden’s innovation agency, and in part by EIT
Digital in the ACTIVE project.

vices. We’re also experimenting with public authorities, promises solutions References
information-centric networking8 for to some of today’s societal challenges 1. S. Mitchell et al., The Internet of Everything
direct access to sensor data. such as air quality, transportation, for Cities, Cisco, 2015; www.cisco.com/web/
Sensor data can be retrieved by and energy efficiency. These IoT sys- strategy/docs/gov/everything-for-cities.pdf.
tools and applications through well- tems must be based on open data and 2. Research and Markets, Internet of Things
defined APIs. The sensor data cloud open standards, including protocols (IoT) Market by Software Solution (Real-
services support both requests for raw and interfaces, so that the systems Time Streaming Analytics, Security, Data
sensor data and for pre-processed enable third-party innovation in new Management, Remote Monitoring, & Net-
sensor data. Pre-processed data can services, and to avoid vendor lock-in. work Bandwidth Management), Platform,
be described as a grid of estimated Standardized protocols might not be Service, Application Domain, and Region –
values for a geographical region, enough to achieve these goals — sys- Global Forecast to 2021, tech. report,
where the values are calculated from tems must be designed with openness Apr. 2016; www.researchandmarkets.com/
the actual data produced by sensors in mind at all levels. Based on this research/gsjxb5/internet_of.
in that region. A set of pre-process- concept, we designed and developed a 3. C. Zhu et al., “Green Internet of Things for
ing types has been defined, such as GreenIoT platform in Sweden to dem- Smart World,” IEEE Access, vol. 3, Nov.
interpolated data, hourly average, onstrate the benefits of open data and 2015, pp. 2151–2162.
daily average, and weekly average. open platforms for smart city devel- 4. Malmö Stad, Climate-Smart Hyllie — Test-
These types should be seen as a start- opment. Over the next year, we will ing the Sustainable Solutions of the Future,
ing point, and more types are likely develop applications and carry out Swedish Energy Agency, 2013; http://malmo.
to be defined in the future. In the experiments using the Uppsala City se/download/18.760b3241144f4d60d3b6
long run, it even should be possible IoT testbed, and formulate guidelines 9cd/1397120343885/Hyllie+klimatkontrakt_
for tools and applications to define for public bodies for the procurement broschyr_EN_2013.pdf.

www.computer.org/computingedge 43

5. Philips, “Connected Lighting System,” press 8. B. Ahlgren et al., “A Survey of Informa- (IoT). Hidell has a PhD in telecommunication
release, 2014; www.newscenter.philips.com/ tion-Centric Networking,” IEEE Comm., from the KTH Royal Institute of Technology.
main/standard/news/press/2014/20140327- vol. 50, no. 7, 2012, pp. 1024–1049. Contact him at mahidell@kth.se.
of-office-lighting-with-groundbreaking- Bengt Ahlgren is a senior researcher in the Deci- Edith C.-H. Ngai is an associate professor in
connected-lighting-system.wpd#. sions, Networking, and Analytics (DNA) lab the Department of Information Technology,
VL46kS5rNow. at SICS Swedish ICT. His current research Uppsala University, Sweden, and a visit-
6. City of Chicago, “Digital Roadmap to focus is on designing networks based on ing researcher at Ericsson Research. Her
Improve Quality of Life,” press release, an information-centric paradigm, where research interests include the IoT, mobile
Apr. 2015; www.cityofchicago.org/city/en/ storage for caching is integrated in the net- cloud computing, information-centric net-
depts/mayor/press_room/press_releases/ work infrastructure. Ahlgren has a PhD in working, smart cities, and urban comput-
2013/september_2013/mayor_emanuel_ computer systems from Uppsala University, ing. Ngai has a PhD in computer science
releasescityofchicagosfirstevertechnology Sweden. Contact him at bengta@sics.se. and engineering from the Chinese Univer-
plan.html. sity of Hong Kong. She’s a senior member
7. P. Fältström, “Market-Driven Challenges to Markus Hidell is an associate professor in com- of IEEE and a member of the ACM. Contact
Open Internet Standards,” Global Commis- munication systems and at the Network Sys- her at edith.ngai@it.uu.se.
sion on Internet Governance Paper Series, tems Laboratory (NSLab) at the KTH Royal
Centre for International Governance Inno- Institute of Technology, Sweden. His current
Read your subscriptions
vation (CIGI), paper series no. 33, May research interests are in the area of com-
This article originally appeared
through the myCS pub-in
2016; www.cigionline.org/publications/ munication protocols and network archi-
IEEE Internetlications
portal vol. 20,
at http://
market-driven-challenges-open-internet- tectures, including network virtualization,
no. 6, 2016. mycs.computer.org.
standards. energy efficiency, and the Internet of Things

Advertising Personnel Southwest,STANDARDS
California: AWARD
Mike Hughes
Marian Anderson: Sr. Advertising Coordinator Email: mikehughes@computer.org
A plaque and $2,000 honorarium is presented in recognition of
Email: manderson@computer.org Phone: +1 805 529 6790
Phone: +1 714 816 2139 | Fax: +1 714 821 4010 outstanding skills and dedication to diplomacy, team facilitation, and
joint achievement in the development or promotion of standards in the
Sandy Brown: Sr. Business Development Mgr. Heather Buonadies
computer industry where individual aspirations, corporate competition,
Email sbrown@computer.org Email:
and h.buonadies@computer.org
organizational rivalry could otherwise be counter to the benefit
Phone: +1 714 816 2144 | Fax: +1 714 821 4010 Phone: +1 973 304 4123
of society.
Fax: +1 973 585 7071
Advertising Sales Representatives (display)
Advertising A COLLEAGUE
Representatives (Classified Line)
Central, Northwest, Far East: DUE: 15 OCTOBER 2017
Eric Kincaid Heather Buonadies
Email: e.kincaid@computer.org Email: h.buonadies@computer.org
Phone: +1 214 673 3742 • Requires 3 endorsements.
Phone: +1 973 304 4123
Fax: +1 888 886 8599 • Self-nominations are
Fax: +1 not585
973 accepted.

Northeast, Midwest, Europe, Middle East: • Do not need IEEE or IEEE Computer Society membership to apply.
Advertising Sales Representatives (Jobs Board)
Ann & David Schissler
Email: a.schissler@computer.org, d.schissler@computer.org
Phone: +1 508 394 4026 Submit your nominationHeather Buonadies
electronically: awards.computer.org | Questions: awards@computer.org
Fax: +1 508 394 1707 Email: h.buonadies@computer.org
Phone: +1 973 304 4123
Fax: +1 973 585 7071

56 www.computer.org/internet/ IEEE INTERNET COMPUTING

44 Computing Edge July 2017
Move Your Career Forward
IEEE Computer Society Membership

Explore These Internet of Things Resources

IEEE Pervasive Computing
Designed for researchers, practitioners, and educators, IEEE Pervasive
Computing explores the role of computing in the physical world as
characterized by the Internet of Things, ubiquitous computing, and
the like.

IEEE Internet Computing

With insightful columns and peer-reviewed feature articles on all
aspects of Internet computing, from programming and standards to
security and networking, IEEE Internet Computing is the top choice for
industry and academic readers.

IEEE Computer Society Technical Committee on

Very Large Scale Integration
The TCVLSI addresses the interactions among the various aspects of VLSI design
including semiconductor processes and system-level, logic-level, and circuit-level
design. It also covers computer-aided design techniques that facilitate the VLSI
design process.


The Community for Technology Leaders
From the Editor in Chief Editor in Chief: Maria R. Ebling n IBM T.J. Watson Research Center n ebling@us.ibm.com

IoT: From Sports to Fashion

and Everything In-Between
Maria R. Ebling, IBM T.J. Watson Research Center

P ervasive computing and the Inter-

net of Things have widespread
impact across fields as seemingly
is a regulation size 5, and, amazingly,
the battery life lasts for approximately
2,000 kicks over the course of a week
html) and Evrythng (https://evrythng.
diverse as sports and fashion. (www.adidas.com/us/micoach-smart- products). In addition, consumers can
ball/G83963.html). also use these tags to find lost items and
IOT IN SPORTS IoT is also influencing golf. GolfTEC perform other functions, such as
It’s easy to find IoT in sports, whether and K-VEST use sensors and display
you’re interested in wearables or technology to provide feedback to golf- • ensure we don’t accidentally throw
objects. I’m sure the vast majority of ers. The sensors in GolfTEC measure a hand-wash item into the washing
Pervasive Computing readers own—or launch angles, spin rates, club speed, and machine or a line-dry item into the
at least have seen—fitness bracelets such the like (www.golftec.com/about-golftec/ dryer,
as the FitBit. Less common wearables technology). Video allows the golfer to • search for a duplicate or a new ver-
include clothing such as the Hexoskin observe his or her swing from multiple sion of the same product, and
smart shirt, which measures your heart angles. The K-VEST measures the golfer’s • learn fashion tips for how to wear the
rate, heart-rate variability, breathing hip, shoulder, and hands to provide feed- item or pair it with other clothes and
rate, and breathing volume in addition back on body position (www.kandicom- accessories that we already own or
to your steps and pace (www.hexoskin. ergolf.com/technology/k-vest). could purchase.
com). It can also track your sleep, mea- We’re seeing more and more IoT
suring your heart rate, breathing, and technology influencing sports. Right I can even envision manufacturers
sleep positions throughout the night. now, the focus is on data collection, but and retailers someday using the tags
This type of technology extends I anticipate such data will eventually to learn about the product’s “end of
beyond sports to monitor people working be used to teach us how to play a new life” in my closet, discovering why I’m
in extreme and high-risk environments, sport and optimize our performance donating the item or throwing it out.
such as firefighters and military person- and fitness levels. Is it because the item has simply been
nel. Smart garments can send an alarm, worn so many times that it has reached
including a location, if a firefighter or sol- IOT IN FASHION its end of life, or has my fashion sense
dier is hurt in the line of duty. Perhaps more surprising are the inroads matured? Or did the item shrink even
Beyond wearables, Adidas now that IoT is making into the fashion though I followed the laundering
makes a smart soccer ball, called the industry. In the coming years, we can instructions? Such data might help com-
miCoach smart ball, which has sensors expect our clothing and accessories panies make manufacturing decisions
embedded within the ball and retails to come with RFID tags that retailers to improve the quality and durability of
for US $200. The sensors can detect the can use to manage their supply chain their clothing, and retailers could note
speed, spin, strike, and flight path and and reduce both theft and counterfeit- my modified fashion preferences.
can send the data back to your smart- ing—examples include Avery Den- Another application of pervasive
phone via the miCoach app. The ball nison (http://rfid.averydennison.com/ technologies in the fashion industry is

MISSION STATEMENT: IEEE Pervasive Computing is a catalyst for advancing research and practice in mobile and ubiquitous computing.
It is the premier publishing forum for peer-reviewed articles, industry news, surveys, and tutorials for a broad, multidisciplinary community.

2 46 July 2017
PER VA SI V E computing Published by the IEEE Computer Society
Published by the IEEE CS n 2469-7087/17/$33.00
© 2017
2016 IEEE
Cisco’s StyleMe Virtual Fashion Mir- less active and for whom exercising at IEEE Computer Society
ror.1 This display technology lets con- home is an appealing option compared Publications Office
sumers virtually try on clothing, create to traveling to a gym. The authors look 10662 Los Vaqueros Circle
Los Alamitos, CA 90720
outfits, and view different colors, all at four design dimensions: interaction
without entering a dressing room. The mechanisms, monitoring and sensing
display overlaps the customer’s reflec- capabilities, coaching and tailoring
tion with pictures of clothing so that features, and persuasion and motiva- STAFF
consumers can see what the outfit tion strategies. Lead Editor
might look like on their body. Simi- In our second feature article, Suining Brian Brannon
larly, Panasonic’s makeup mirror lets He, Bo Ji, and S.-H. Gary Chan from bbrannon@computer.org
consumers virtually try on different The Hong Kong University of Science Content Editor
Shani Murray
types of makeup, from false eyelashes and Technology present “Chameleon:
Assoc. Peer Review Manager
to eyeshadow and blush (www.youtube. Survey-Free Updating of a Fingerprint
Hilda Carman
com/watch?v=JtwVVhvEwU8). Beyond Database for Indoor Localization.”
Publications Coordinator
the ability to see how you’d look with He, Ji, and Chan strive to make it easy pervasive@computer.org
various beauty products, the mirror can to keep fingerprint databases up-to- Contributors

also teach an inexperienced consumer date in spite of updated access points Keri Schreiner,
how and where to apply the various (APs). They make the observation that Joan Taylor, and Teri Sullivan
products. unaltered APs cluster a user’s location

Director, Products & Services

The use of IoT in the fashion industry whereas altered APs tend to disperse Evan Butterfield
is in the early days, but I foresee some the user’s location. With this observa-
for Senior Manager, Editorial Services
Robin Baldwin

fascinating applications in the years tion, they can use the unaltered APs to
to come—much beyond the LED-
enhanced outfits that we occasionally
localize the user and use the location to
update the fingerprint database of the
for Senior Business Development Manager
Sandra Brown

Software Development Manager

see stars wearing to gala events.2 altered APs. They study this approach seeks practical, readable
Cecelia Huffman
both on their university campus and in articles that will appeal to experts and
Senior Advertising Coordinator
IN THIS ISSUE a major airport. If their results can be IEEE Software
nonexperts alike.
Marian seeks
The practical,
Anderson aimsreadable
One area that will be important for mov- replicated, this approach could have articles
to delivermanderson@computer.org
that will appeal
reliable, to leading-edge
useful, experts and
ing IoT and pervasive computing tech- major implications for the maintenance nonexperts
informationalike. The magazine
to software developers, aimsengineers,
IEEE Software seeks practical, readable
nology forward is the theme of this issue. of indoor fingerprint databases. to
anddeliver reliable,
managers useful,
to help them leading-edge
stay on top of
articles that will appeal to experts and
Energy harvesting is critically important Our third feature article, “Emerg- information
IEEE toComputing
technology software
change. developers,
(ISSN engineers,
1536-1268) is
published alike.
quarterly The
by the magazine
Computer Society.
for certain types of IoT applications in ing Trust Implications of Data-Rich and managers design,
requirements, to help construction,
them stay on top tools,of
Headquarters, Three Park
to deliver reliable, Ave., 17th
useful, Floor, New York,
both the sports and fashion domains. Systems,” by Bran Knowles, identifies NY 10016-5997;
project IEEE Computer
management, change. Society
processTopics Publications
Without it, we risk forcing users to the challenges in building users’ per- information
Office, to Vaqueros
10662 Los softwareCircle,
developers, engineers,
PO Box 3014, Los
Alamitos, design,
CA 90720-1314, construction,
testing, phone
education 821tools,
+1 714and training,
recharge the technology too often for ception of trust in pervasive systems. In and managers to help them stay on top of
IEEE Computer
quality, Societyand
standards, Headquarters,
process 2001 L St.,
more. improvement,
usability, reducing long-term adoption. the article, Knowles uses a hypothetical Ste.
rapid700, Washington,change.
technology DC 20036. Subscribe
Topics to IEEE
Pervasive testing,
Computing education
by visiting and training,
In addition to our theme articles on fitness application, called the Fangle- requirements,
design, construction, tools,
quality, guidelines:
standards, and more.
energy harvesting, we also have several Bangle, to illustrate the concepts. The project management, process improvement,
feature articles. As I mentioned ear- six challenges identified include how maintenance,
Author guidelines:
testing, education and training,
details: software@computer.org
lier, IoT technology might eventually such systems let users verify the data quality,
Postmaster:standards, and more.
Send undelivered copies and address
changes to IEEE Pervasive Computing, Membership
be used to optimize fitness levels, and collected, how the data chains of these
Further details:
Processing Dept., software@computer.org
IEEE Service Center, 445 Hoes
along those lines, our first feature article systems are mapped and presented Author
Lane, guidelines:
Piscataway, NJ 08854-4141. Periodicals
postage paid at New York, NY, and at additional mailing
is “Fitness Applications for Home-Based to users, and how users can be given www.computer.org/software/author
offices. Canadian GST #125634188. Canada Post
Training,” by Iman Khaghani-Far, insight into the underlying algorithms Further details:
Publications Mail software@computer.org
Agreement Number 40013885.
Svetlana Nikitina, Marcos Báez, Ekat- without overwhelming them with tech- Return undeliverable Canadian addresses to PO Box
122, Niagara Falls, ON L2E 6S8. Printed in the USA.
erina A. Taran, and Fabio Casati. In nology. Understanding and addressing IEEE

this article, the authors survey fitness these components to trust is critical to
applications across a variety of plat- the long-term acceptance and, conse-
prohibits discrimination, harassment and
forms, including smartphone platforms quently, the ultimate success of perva- bullying: For more information, visit www.ieee.org/
(Android and iOS), desktop platforms sive computing technologies. web/aboutus/whatis/policies/p9-26.html.

(Windows and Mac), and console plat- On a related note, in our Perva- IEEE

forms (Nintendo and Xbox), looking sive Health department, Kelly Caine
explicitly at support for older adults discusses the many ways we inadver-
who are frequently more isolated and tently give away private health data in

PER VA SI V E computing 3
EDITORIAL BOARD CHANGES Finally, our Notes from the Commu-
nity department covers everything from
First, I’m pleased to announce that board member Robin Kravets will be taking on interactive light to Pokémon Go. Of
the role of Associate Editor in Chief. I look forward to her contributions, especially particular interest was the discussion
given her expertise in the areas of mobile computing and communications.
I’d also like to introduce two new board members, Florian Michahelles and Daqing of Project Jacquard, an effort between
Zhang. Google and Levi’s to create a jacket for
Florian Michahelles heads the Web of Things research group at urban cyclists who want to interact
Siemens Corporate Technology. The Web of Things team borrows with their phones while cycling. Mul-
methods from the Semantic Web and investigates how machines, titouch sensors woven into the cuff of
devices, and sensors can communicate and share data based on the jacket let the cyclists answer or place
ontologies and semantics without requiring prior defined com-
munication standards. The research group is being formed in calls and use navigation apps.
Berkeley by international researchers who collaborate with leading
US universities in industry-funded and publicly funded projects.

Michahelles received his PhD from the Swiss Federal Institute of he breadth of our field is amaz-
Technology (ETH) Zurich. Contact him at florian.michahelles@siemens.com.
ing. The fact that pervasive and
Daqing Zhang is a Chair Professor at the School of EECS, Peking IoT technologies can be applied in such
University, China and Vice Chair of the Pervasive Computing
diverse areas as fashion and sports
Federation of China. His research interests include context-aware
computing, mobile computing, big data analytics, and pervasive helps illustrate that fact. Someone once
elderly care. Zhang obtained his PhD from the University of Rome accused me of thinking that fashion and
“La Sapienza.” He is the associate editor for ACM Transactions sports are the same field. In fact, I might
on Intelligent Systems and Technology and IEEE Transactions on Big
be guilty of that—at least when you con-
Data. Contact him at dqzhang@sei.pku.edu.cn.
sider the use of IoT technology within
these fields. One could argue that the
cycling jacket proves my point.
her article “Privacy Is Healthy.” She temporarily one-handed due to a bro-
discusses the limitations of the Health ken shoulder, the idea is very appeal-
Insurance Portability and Account- ing. The biggest issue I had, though, REFERENCES
ability Act, especially as it relates to was in aiming and snapping photos—a
1. L. Fretwell, “Cisco StyleMe Virtual
non-clinical data collected by fitness- use case that was, unfortunately, not Fashion Mirror,” Cisco, Dec. 2011;
tracking apps and health-monitoring included in their user study. www.cisco.com/c/dam/en_us/about/
systems. She gives recommendations In this issue’s Conferences depart- ac79/docs/retail/StyleMeEngagementOv
for actions that users and developers ment, Mateusz Mikusz, Sarah Clinch,
can take to improve their own and and Sougata Sen provide an overview 2. P. Chi, “Met Gala 2016: Claire Danes’s
their users’ privacy. This call to action of MobiSys, which was held this past Glow-in-the-Dark Gown Upstaged a
Red-Carpet Robot Army,” Vanity Fair,
is critically important for developers: June in Singapore, and ASSET, the 3 May 2016; www.vanityfair.com/
if users misunderstand the privacy first Asian Students Symposium on style/2016/05/met-gala-2016-red-carpet.
implications of the applications they Emerging Technologies. MobiSys
use and discover that the health infor- included 31 papers and two impressive
mation they thought was private has keynotes and covered topics ranging
been released, our industry will face a from smart environments, to sensing, Maria R. Ebling is a director at the IBM T.J.
major crisis in terms of consumer trust, to location awareness, to security and Watson Research Center. She manages a team
and adoption of our applications will privacy. This year’s best paper award building systems capable of supporting a
greatly suffer. went to Endri Bregu and his colleagues Smarter Planet while not forgetting about the
In our Smartphones department, for their paper, entitled “Reactive Con- people who use such systems. Ebling received
Yu-Chih Tung and Kang G. Shin trol of Autonomous Drones,” about a her PhD in computer science from Carnegie Mel-
present “ForcePhone: Software Lets new way of controlling drones, such lon University. She’s a member of the IBM Acad-
Smartphones Sense Touch Force.” This that sensor readings only trigger con- emy of Technology, a distinguished member of
work describes a mechanism by which trolling components if the sensor val- the ACM, and a senior member of IEEE. Contact
smartphones that don’t have the force- ues have changed. If you were unable her at ebling@us.ibm.com.
sensing hardware (that is, all Android to attend the conference this year, I
phones and lower-end Apple phones) encourage you to read through the
can use software-based force-sensing. summary. It will make you aware of This article originally appeared in
The idea behind this technology is to the many papers relevant to this com- IEEESelected
Pervasive Computing, vol. 15,
CS articles and columns
make it easier to use your phone one- munity and will help you identify those no. 4,are2016.
also available for free at
handed. As someone who has been of most interest to you. http://ComputingNow.computer.org.

4 48 VA SI V E computing
PER Computing Edge July 2017

Cybersecurity vulnerable or not. We’re using new Inter-

net architectures and next-generation
networks and protocols, potentially us-

and the Future This article

appeared in
ing software-defined components, and
looking to ameliorate underlying flaws
by incorporating lessons learned from
Computer, the past. We’ll likely fix a few mistakes
Sven Dietrich, City University of New York
vol. 50, no. 4, but also introduce new attack vectors
We need our computing systems to perform as 2017. that we can’t imagine yet.
Harnessing vulnerable devices into
intended, unaffected by adversaries big or small. maliciously behaving networks such

as botnets or as a source of exfiltrating
s the world becomes in- legacy problems with devices that can’t private or sensitive information (such
creasingly interconnected, easily be changed and form an integral as email or health and financial data)
with more devices from part of a critical infrastructure. We’re will continue as long as devices remain
our daily lives becoming also creating secure ways of verifying, connected and always on, vulnerable to
part of the Internet of Things (IoT), patching, and updating our devices to flaws, improperly configured due to hu-
one issue will certainly prevail: cyber- maintain their functionality, integ- man error, or intentionally sabotaged
security. We care about our security rity, and health at both the software by insiders. Just as bots migrated from
and privacy, and this undoubtedly and hardware level, as well as ways to government and university lab systems
won’t change over the next 50 years. let devices communicate securely. to home computers and mobile devices,
There will be shades of gray—based We’ll face new challenges when and distributed denial-of-service at-
on cultural perceptions, needs, and quantum computers become com- tacks turned IoT devices (such as cam-
trends—but privacy is part of human monly available. As current compu- eras, DVRs, home automation, and
nature and affi rmed as such in the Uni- tational, algorithmic, and distributed broadband routers) into botnets, we can
versal Declaration of Human Rights. techniques allow us to bypass, crack, only imagine what the next target will
We need our computing systems to and compromise security protocols be, from smartwatches to connected
perform as instructed or intended, un- and cryptographic primitives using toothbrushes and pacemakers.
affected by adversaries big or small. conventional computing (for example,
Computing devices exist in our the recent Google-led effort to find col-

businesses, homes, pockets, bodies (in lisions in the cryptographic hash func- s we add more devices to our
the form of medical devices), hospitals tion SHA-1), we’ll face even tougher lives and connect them to the
and medical offices, transportation obstacles when quantum computing somewhat fragile Internet (by
systems, industrial production and en- techniques threaten to undermine our cable, Wi-Fi, or some method yet to be
ergy generation systems, fi nancial and security and privacy safeguards (for discovered), it’s important to remem-
payment systems, voting systems, and instance, allowing for the factoring of ber that information can flow both
even in the dams that protect our land RSA moduli or breaking other public- ways, maliciously or not.
from rising sea levels. These devices key cryptography mechanisms). Quan-
all have one thing in common: they tum computing in the hands of a few
execute code on one or more silicon powerful actors would create an even SVEN DIETRICH is an associate
chips. Code can be flawed or vulnera- greater imbalance. The security chal- professor in the Mathematics and
ble to exploits, which can compromise lenges that could arise from a quantum Computer Science Department at
the aforementioned massively prolif- computing environment itself are yet the City University of New York’s
erated systems with various degrees of to be seen, compared to the conven- (CUNY’s) John Jay College of Criminal
impact on our world and lives. tional containment challenges that Justice, is on the doctoral faculty of
While we continue to promote se- exist now and in some cases can be the Computer Science department
cure coding practices for existing and mitigated (by secure enclaves in recent at the CUNY Graduate Center, and is
upcoming programming languages Intel chips, for example). on the IEEE Cybersecurity Initiative
and microchips and to educate stake- Interconnections pose a contin- Steering Committee. Contact him at
holders about better cybersecurity ued risk. We’re developing novel ways spock@ieee.org.
practices, we need to think about of connecting old and new devices,

2469-7087/17/$33.00 © 2017 IEEE Published by the IEEE Computer Society July 2017 49
C O M P U T E R 0 0 1 8 - 9 1 6 2 / 1 7/ $ 3 3 .0 0 © 2 0 1 7 I E E E PUBLISHED BY THE IEEE COMPUTER SOCIET Y APRIL 2017 7

New Membership
Options for a Better Fit
And a better match for
your career goals. Now
IEEE Computer Society
lets you choose your
membership — and the
benefits it provides — to fit
your specific career needs.
With four professional
membership categories and
one student package, you
can select the precise
industry resources, offered
exclusively through the
Computer Society, that will
help you achieve your goals.

Learn more at www.computer.org/membership.

Achieve your career goals
with the fit that’s right for you.
Explore your options below.

Training &
Preferred Plus Research Basic Student
Select your $60 $126 $55 $115 $55 $115 $40 $99 $8
membership IEEE
Does not include
IEEE membership

Computer magazine (12 digital issues)*

ComputingEdge magazine (12 issues)

Members-only discounts
on conferences and events

Members-only webinars

Unlimited access to Computing Now,

computer.org, and the new
mobile-ready myCS

Local chapter membership

Skillsoft’s Skillchoice™ Complete

with 67,000+ books, videos, courses,
practice exams and mentorship
Books24x7 on-demand access to 15,000
technical and business resources

Two complimentary Computer

Society magazine subscriptions

myComputer mobile app 30 tokens 30 tokens 30 tokens

Computer Society Digital Library Member pricing Member pricing Included
downloads downloads

Training webinars Member pricing Member pricing Member pricing
webinars webinars

Priority registration to Computer

Society events

Right to vote and hold office

One-time 20% Computer Society

online store discount

* Print publications are available for an additional fee. See catalog for details.

Graphically Speaking Editor: André Stork

The Need to Help Journalists with Data

and Information Visualization
Susan Reilly
Florida Atlantic University

A s of 2015, 89 percent of mobile users (144

million users) accessed news via their mo-
bile devices, and it is estimated that by
2020 mobile devices will account for two-thirds
of all online activity.1 In turn, smaller screen sizes
cycle, which requires the presentation of fresh sto-
ries around the clock in order to stay competitive.
As people become increasingly computer literate, it
is hoped that online news readers will be drawn in
by interesting data visualizations on their mobile
reduce reading time and ease of reading, and mo- devices and then later read the stories that were
bile access reduces news seeking as well as interac- the motivation for those data visualizations on
tion with graphical elements.2 At the same time, their home computers.
the quantity and availability of news seems to be In a news environment where high story turn-
resulting in people reading shorter stories more over is necessary, the strength of data visualization
frequently throughout the day. lies in the viewer’s ability to process visual infor-
As news migrates to mobile phones, media mation more rapidly than verbal information.
companies are turning to data visualization to What seems to work is the clear presentation of
whet readers’ appetites for stories they can read relationships, the accurate representation of quan-
at length later on their home or work computers. tities, the easy comparison of qualities, and the
However, journalists are trained to write stories, not clear ranked order of values.4
in statistics or coding. The big news organizations Yet, journalists are trained to work with words,
have the funds to hire computer graphics experts, using them to construct stories that explain
but local news organizations need help. events. Now they are being asked to add a visual
component to those stories as well. Quite frankly,
Why Do Journalists Need Data Visualization? many feel overwhelmed. A professional journalist
Data visualization has been around since antiquity described her beat this way:
with star charts and navigational maps. During the
19th century, with the growth of industrialization, Now my job isn’t just to report, but also to
large bodies of data were collected by businesses find an audience for the story by doing social
and governments to aid in product development media promotion—Twitter and Facebook. Then
and social planning. With the growth of data col- I do format and production work. We have a
lections, ever-more sophisticated mathematical content management system for print. I add a
techniques have been used to analyze them. Gov- cutline and attach a photo. We have a free site
ernments and corporations continue to collect and and a paid site, so I produce a story for the free
store huge amounts of data. The introduction of site that has a taste and a summary and then I
human-computer interaction in the 20th century produce a story for print that is more substan-
aided in the statistical analyses of these vast data- tial. Now I have to think of [mobile] because
bases and the visual presentation of patterns, rela- six out of 10 readers are reading on their cell
tionships, and hierarchies.3 phones. We learned how to do all these fancy
The latest field to utilize data visualization is interactive graphics for the computer, but we
journalism. News organizations’ enthusiasm is found out that they don’t work on phones. You
being driven by the relentless 24/7 Internet news need really simple graphics on phones.

852 March/April
July 2017 2017 Published by
Published by the
the IEEE
Society 0272-1716/17/$33.00©©2017
2469-7087/17/$33.00 2017 IEEE
What Are Journalists Trying to Do? that is so simple it works on mobile devices. But
Alberto Cairo, the Knight Chair in Visual Jour- to read the full story, you have to use a desktop
nalism at the University of Miami, has suggested computer.
that there are two ways for data visualization to The first screen in the data visualization asks,
be effective: one is an infographic that provides “Why is Pinellas County the worst place in Florida
“spontaneous insight” (the aha moment) and the to be black and go to public school?” Subsequent
other is an infographic that provides “knowledge screens illustrate how after a decision by the Pinel-
building insight” gained through the exploration las County School Board in 2007 to stop integrat-
of a complex system. The former can be under- ing schools, the percentage of black students began
stood immediately, while the latter takes time to to rise in five elementary schools in black commu-
explore.5 The aha moment might work best on mo- nities. At the same time, funding for those schools
bile devices that readers look at for short periods, fell and teacher turnover increased. Ultimately,
multiple times during the day; the “knowledge 95 percent of students at those five elementary
building insight” might work better on home or schools failed the state’s reading and math tests,
work computers where the increased screen size the greatest concentration of academic failure in
facilitates longer periods of interactivity. Florida. The final screen in the data visualization
An example of a data visualization that cre- asks, “Who is responsible?” The investigative re-
ates an aha moment is Bill Bunge’s iconic map, port resulted in the US Department of Education
“Where Commuters Run Over Black Children on opening a civil rights investigation into whether
the Pointes-Downtown Track” (see civic.mit.edu/ the school district systematically discriminates
blog/kanarinka/the-detroit-geographic-expedition against black children.
Bunge’s map was originally created by the Detroit
Geographic Expedition, which addressed racial in- Many journalists are trying to create
equality in Detroit’s inner core in the 1970s. The
data was collected from police reports. With a data visualizations with little training and
shockingly simple legend, the map shows how com-
muter traffic from affluent white suburbs speeding
with programs that were developed for
through a black neighborhood repeatedly resulted other professions.
in injuries to young children crossing the streets.6
The simple visual representation of black children
hit by white commuters’ cars became an allegory Large national news organizations, like the
for racial conflict in 1970s Detroit. Bunge’s map is New York Times (with a circulation of more than
simple enough to be viewed easily on a mobile de- 4 million, print and digital), have full-time em-
vice and it provides, in Alberto Cairo’s terms, spon- ployees trained in statistical methods, data min-
taneous insight. This map is one of many Bunge ing, coding, and computer graphic applications
constructed to visualize the findings of the Detroit who are tremendously helpful to the reporters
Geographic Expedition. Yet, to get the full story, investigating complex problems. By cleverly ma-
you still have to read the report. nipulating large numbers of records, a good stat-
An example of a data visualization provid- istician can locate the place where the story lies,
ing Cairo’s knowledge building insight can be and a good computer graphics expert can use a
found in the 2016 Pulitzer Prize winning inves- program like the D3.js JavaScript library to make
tigative report “Failure Factories” by the Tampa it visible.
Bay Times (www.tampabay.com/projects/2015/ Many small local newspapers and television
investigations/pinellas-failure-factories/chart stations can’t afford statisticians or computer
-failing-black-students/). A team of journalists graphics professionals, however. These organiza-
combined data visualization and reporting into a tions are making do by redefining the jobs of the
powerful series on the abandonment of school in- existing journalists who are the most computer
tegration in Pinellas County, Florida. The report- savvy and hoping for the best. Often a data vi-
ers retrieved data from test scores, poverty rates, sualization that works on a home computer is
racial demographics, and school board meeting hard to read on a small screen or doesn’t work in
transcripts that showed the gradual transition of newsprint where the legends are in shades of gray.
average schools into the worst in the state.7 The Badly designed data visualizations can be baffling
journalists wrote the story, and the data direc- and, worse, a waste of both the journalist’s and
tor and data reporter created a data visualization the reader’s time.

www.computer.org/computingedge IEEE Computer Graphics and Applications 539

Graphically Speaking

What Do Journalists Need? information that citizens need in order to make

There are roughly 83,000 professional journal- good decisions in a democratic society.
ists working in newspaper and television news-
rooms across the United States.8 Many are trying
to create data visualizations with little training References
and with programs that were developed for other 1. Knight Foundation, “Mobile-First News: How
professions. They spend too much time trying to People Use Smartphones to Access Information,”
adapt these programs to their needs. For example, 11 May 2016; kf-site-production.s3.amazonaws
the designated journalists/data analysts at two local .com/publications/pdfs/000/000/187/original/
newspapers in South Florida (with circulations of Topos_KF_Mobile-Report_Final_052616.pdf.
roughly 100,000) use Excel, Access, MySQL, R, 2. J. Dunaway, “Mobile vs. Computer: Implications for
and QGIS. They have to look at the numbers several News Audiences and Outlets,” Shorenstein Center
ways before deciding the best way to tell the story on Media, Politics, and Public Policy, 30 Aug.
in graphics. Since coding is problematic, they often 2016; shorensteincenter.org/mobile-vs-computer
rely on D3.js alternatives, like DataWrapper, Google -news-audiences-and-outlets.
Chart Tools, and StoryMap JS. 3. M. Friendly, “A Brief History of Data Visualization,”
Local journalists need tools that make it easier Handbook of Data Visualization, C. Chen, W. Hardle,
to take raw data and convert them into interest- and A. Unwin, eds., Springer, 2008, pp. 15–56.
ing graphics without requiring advanced coding 4. S. Few, “Data Visualization for Human Perception,”
skills. It would be helpful if computer graphics The Encyclopedia of Human-Computer Interaction, 2nd
researchers and designers would create a tool ed., M. Soegaard and D. Rikke Friis, eds., Interaction
that featured a menu of options with recom- Design Foundation, 2014.
mendations to both organize and present visual 5. G. McGhee, “The ‘Rules’ of Data Visualization Get
information in ways that news consumers can and Update,” National Geographic, 16 Oct. 2015;
understand quickly. This might include different news.nationalgeographic.com/2015/10/151016
ways to analyze statistics as well as suggestions -data-points-alberto-cairo-interview.
for data-driven documents. Perhaps a good exist- 6. C. D’Ignazio, “The Detroit Geographic Expedition
ing tool could be adapted, but it should be menu- and Institute: A Case Study in Civic Mapping,”
driven rather than code-driven, and the menu 2013; civic.mit.edu/blog/kanarinka/the-detroit
should be built by journalists. The more intuitive -geographic-expedition-and-institute-a-case-study
the tool, the more journalists will be able to take -in-civic-mapping.
advantage of data visualization. 7. S. Nesmith, “Tampa Bay Times’ Investigation Is a
Another key requirement in this field is respon- Model for How to Report on School Resegregation,”
sive design for different screen sizes. News orga- Columbia Journalism Rev., 19 Aug. 2015; www.cjr
nizations need a way to customize headlines as .org/united_states_project/tampa_bay_times_school
readers move between devices. Some graphics don’t _resegregation.php.
scale down well, so journalists need a way to easily 8. L. Willnat and D. Weaver, “The American Journalist
adapt static graphics to different screen sizes. A in the Digital Age: Key Findings,” School of
tool that allowed readers to tag small screen sto- Journalism, Indiana Univ., 2014; news.indiana.edu/
ries to read at length on their PCs would be ideal. releases/iu/2014/05/2013-american-journalist-key

W ithout a strong business model to replace the

revenue lost to Internet advertising, many
in the journalism profession fear the continued
Susan Reilly is a full professor of multimedia studies at Flor-
ida Atlantic University. She is a writer and producer of public
disappearance of local news organizations. television documentaries and the author of several books on
Despite the difficulty of finding ways to make it critical media theory. Contact her at sreilly@fau.edu.
cost effective, local news is still something that
communities value. Local news contributes to Contact department editor André Stork at andre.stork@igd
social cohesion—something that many feel is at .fraunhofer.de.
risk as people become increasingly involved in
their respective Internet worlds.
Hopefully, a dialog can begin between Readappeared
This article originally your subscriptions
in through
journalists and computer scientists about how to the myCS publications portal at
IEEE Computer Graphics
http://mycs.computer.org. vol.
and Applications,
work together to help local journalists provide the 37, no. 2, 2017.

54 March/April 2017
Computing Edge July 2017

Mutual Dependence Demands

Mutual Sharing

U nfortunately, interdependence, coop-

eration, and trust are poorly correlated.
This is the fundamental axiom of cyberinse-
small sample, to embrace non-cooperation
with public authorities.
History shows that nothing unites like a
curity risks for all of us. common enemy, and that failure to unite guar-
Interdependence makes risk transitive; if A antees defeat. Our shared security requires
depends on B to function and B depends on shared defenses.
C to function, then a failure of either B or C We need a broad reporting requirement for
induces a failure for A. By contrast, trust is not cyberattacks and increased information shar-
transitive; that A trusts B and B trusts C does ing among government, private sector actors
not impel A to trust C, nor would B’s loss of like ISPs, and private sector attack targets.
faith in C impel A to lose faith in B. Worse, the This imperative can be seen in other contexts.
fact that A vitally depends on B and C doesn’t Accident data must be reported when air-
necessarily induce cooperation with either. planes collide and workers are injured. Begin- Daniel E. Geer Jr.
In the digital world, our security is as con- ning in 1912, a diagnosis of plague, cholera, In-Q-Tel
nected as our devices. Your firewall might yellow fever, typhus, or smallpox obligated
be strong and your defenses might be active the doctor or clinic to share that informa-
and robust, but if a sophisticated and deter- tion (by telegraph) with public health offi-
mined opponent attacks your counterparties, cials. That list of five communicable diseases
spoofs your legitimate suppliers or customers, has grown to 40, but the same rule applies:
or infects your security providers, then that prompt, exceptionless disclosure for a lim-
opponent will find a way to undermine your ited set of priority conditions irrespective of
security. Most users don’t even discover their privacy rules. The principle? Transitive risk
penetration on their own—they learn it from above some threshold necessitates informa-
others.1 tion sharing.
In other words, not only is the problem The US is stumbling toward coerced infor-
technically and mathematically challenging, mation sharing about digital attacks. We do
but its operational impact has a social compo- this with different rationales in different con-
nent that compounds the complexity; you are texts, such as when we require prompt and
probably insecure and you are dependent on detailed attack information from defense con-
others to tell you that. tractors to Pentagon authorities, when state
According to a Wall Street Journal poll of laws force disclosure if customers’ credit card
its CEO Council: “9% of the respondents or other personal information is exposed, and
said they would never sufficiently trust the when the SEC requires the announcement of
[US] government with information to work security breaches that materially impair corpo-
with it during a cyberattack ... . Another 34% rate operations. But this is piecemeal improve-
said they would cooperate with the govern- ment, and we need to move beyond islands of
ment only if their own company was being insight. We should take advantage of oppor-
attacked.”2 We don’t think an obligation to tunities presented to us by digital systems for
share information is simple. Companies have immediate and comprehensive reporting. To Richard Danzig
competitive concerns and face legal vulner- benefit fully from the technical opportunity, Johns Hopkins University
abilities when they acknowledge problems. we need to complement it with a mandatory
Individuals and companies rightly value their reporting system that is comprehensive, inex-
privacy and fear overreaching by an overly pensive, adequately protective of confidential-
intrusive government. But we don’t think it’s ity, and valuably informative about the volume,
viable for nearly half of all CEOs, even in a pattern, and character of digital attacks.
2469-7087/17/$33.00 © 2017 IEEE Published by the IEEE Computer Society July 2017 55
1540-7993/17/$33.00 © 2017 IEEE Copublished by the IEEE Computer and Reliability Societies January/February 2017 87

Voluntary systems fill some of Diseases spread within jurisdic- 2016; www.whitehouse.gov/sites
the gaps. ISPs and equipment ven- tions before they become global, but /default/files/docs/cybersecurity
dors capture attack data and alert malware is global from the get-go. _report.pdf.
their users with varying consistency, Transmissible diseases can mutate, 6. D. Geer, “For Good Measure: Stress
speed, and detail. Federal and pri- but they have studied, predictable Analysis,” ;login:, vol. 39, no. 6,
vate Information Sharing and Ana- behaviors, whereas malware comes 2014, pp. 56–57.
lysis Centers (ISACs) encourage from sentient opponents who can
exchanges between industrial sec- be intentionally devious. Daniel E. Geer Jr. is the Chief Infor-
tors. Utilities representing perhaps mation Security Officer of In-Q-
two-thirds of America’s customers Tel. Contact him at dan@geer.org.
participate in an automated Cyber-
security Risk Information Shar-
ing Program (CRISP). Companies
O f course, our proposal is not
the only way to proceed, but
one can’t ignore that its cousins have
Richard Danzig is a senior advisor
to the Johns Hopkins University
share information with chosen oth- been proven successful in life-and- Applied Physics Laboratory and
ers, sometimes outside of their own death matters. We believe the digi- former Secretary of the Navy. Con-
industries to diminish competitive tal domain generates more widely tact him at rjdanzig@gmail.com.
and antitrust concerns. Former col- shared vulnerabilities than any other.
leagues commonly exchange infor- When dependence is mutual—that
mation across industry boundaries. is to say when for A to function B
However, these don’t suffice. must be operational and for B to
Large vulnerabilities remain when function A must be operational—
a third of the nation’s utilities don’t the brittleness of the combined sys-
yet participate in CRISP. In a con- tem to cascade failure requires other
nected grid, vulnerabilities for some coercive measures if either A or B is
have consequences for all. Though critically important to others beyond
industry-specific data compilation their mutual embrace.6
and analysis are commendable, soft-
ware and hardware attacks cross References
industries and exploit common vul- 1. Data Breach Investigations
nerabilities in ways we are not well Report, Verizon, 2016; www
positioned to understand, never .verizonenterprise.com/verizon
mind remediate.3 Two of the most -insights-lab/dbir/2016.
recent and insightful assessments 2. A. Cullison, “NSA Chief: ‘Uneven
have recommended improved vol- Cooperation between Pub-
untary reporting.4,5 But why not lic, Private Sectors Impedes
mandatory? Cyber Defenses,” Wall Street J.,
For its part, the federal govern- 15 Nov. 2016; blogs.wsj.com
ment needs to share more system- /washwire/2016/11/15/nsa-chief
atically. Often the FBI reveals to a -uneven-cooperation-between
surprised company that it has been -companies-government-impedes
penetrated, and then doesn’t divulge -cyber-defenses. This article originally
any more about what it knows or 3. P. Behr, “DOE Seeks to Offer appeared in
how. Our intelligence establishment Cyberthreat-Sharing Defenses IEEE Security & Privacy,
understandably guards its secrets, to Small Utilities,” E&E News, vol. 15, no. 1, 2017.
but its highest priority (to protect 6 Jul. 2016; www.eenews.net
us) is undervalued in an effort to /stories/1060039828.
protect its sources and methods. 4. “From Awareness to Action: Cyber-
We don’t think a regime of security Agenda for the 45th Presi-
broader, compelled sharing is an dent,” CSIS, 4 Jan. 2017; www.csis
easy one or without side effects. .org/analysis/awareness-action.
There is a great deal of thinking still 5. “Report on Securing and Growing
Read your subscriptions through
to be done. Biological infections are the Digital Economy,” Presiden-
the myCS publications portal at
treated by professionals, but malware tial Commission on Enhancing http://mycs.computer.org
infections are treated by amateurs. Nat’l Cybersecurity, 1 Dec.

56 Computing Edge July 2017

88 IEEE Security & Privacy January/February 2017

Careers Related to the

Internet of Things

F lorian Michahelles has run Siemens’

Web of Things research group—which
investigates the application of Seman-
tic Web technologies to the Internet of Things
(IoT)—since 2013. Having worked in the fields
ComputingEdge: What would you tell college
students to give them an advantage over the

Michahelles: Go beyond your major and think

of ubiquitous and wearable computing for more about also taking non-tech majors, such as by
than a decade, Michahelles’ current focus at combining computer science and psychology,
Siemens is leveraging Web and semantic tech- business and electrical engineering, or material
nologies to enable new business opportunities, science and sensors.
particularly in the fields of wearable sensing and
human-robot interaction. He wrote “Internet of ComputingEdge: What should applicants keep in
Things Reality Check” in IEEE Pervasive Comput- mind when applying for IoT-related jobs?
ing’s April—June 2017 issue. We asked Micha-
helles about IoT-related careers. Michahelles: Be an expert in one topic. While
breadth is welcome, depth in one topic is key. Breadth
ComputingEdge: Which IoT-related careers will then helps you effectively apply your expertise.
see the most growth in the next several years?
ComputingEdge: How can new hires make the
Michahelles: Any career bridging the disciplines strongest impression in a new position from the
of mechanical engineering, electrical engineering, beginning?
design, computer science, interactive design, and
communications will be in high demand because Michahelles: Listen and learn, get your hands
IoT reaches across these disciplines. dirty, be bold and courageous in proposing new

2469-7087/17/$33.00 © 2017 IEEE Published by the IEEE Computer Society July 2017 57

ideas. Play with technologies you haven’t used hard to create something innovative all by your-
before, and quickly build demos and prototypes to self. Therefore, it’s important to learn to work with
convey your ideas to others. others. And third, get a sense of what is required.
Find out what’s needed, where the opportunities
ComputingEdge: Name one critical mistake that are, and adjust your passion to this need.
young graduates should avoid when starting

their careers.
omputingEdge’s Lori Cameron inter-
Michahelles: Don’t be afraid of failing. Instead, be viewed Michahelles for this article. Con-
brave enough to fail often, but avoid failing twice tact her at l.cameron@computer.org if you
at the same thing. Keep improving. would like to contribute to a future ComputingEdge
article on computing careers. Contact Michahelles
ComputingEdge: Do you have any learning expe- at florian.michahelles@siemens.com.
riences that could benefit those just starting out in
their careers?

Michahelles: First, find your passion and develop

it. Passion is the prerequisite to being successful
at something. Second, learn how to deal with peo- Read your subscriptions through
the myCS publications portal at
ple. How do you present your ideas? How do you
explain your ideas to others? These days, it’s really

IEEE-CS Charles Babbage Award

Deadline 15 October 2017
Established in memory of Charles Babbage in recognition of significant
contributions in the field of parallel computation. The candidate would
have made an outstanding, innovative contribution or contributions to
parallel computation. It is hoped, but not required, that the winner will
have also contributed to the parallel computation community through
teaching, mentoring, or community service.

This award covers all aspects of parallel computing including
computational aspects, novel applications, parallel algorithms, theory of
parallel computation, parallel computing technologies, among others.


A certificate and a $1,000 honorarium presented to a single recipient.
The winner will be invited to present a paper and/or presentation at NOMINATION SITE
the annual IEEE-CS International Parallel and Distributed Processing awards.computer.org
Symposium (IPDPS 2017).
NOMINATION SUBMISSION www.computer.org/awards
Open to all. Nominations are being accepted electronically at www.
computer.org/web/awards/charles-babbage. Three endorsements are
required. The award shall be presented to a single recipient.

58 Computing Edge July 2017

SkillChoice™ Complete
Now with expanded libraries and an upgraded platform! Valued at

6,000+ Practice
VIDEOS Exams 15,000+ Books24x7 titles

OVER 20x as many resources as before

One membership.
Unlimited knowledge.
Did you know IEEE Computer Society membership comes with access to a
high-quality, interactive suite of professional development resources, available 24/7?

Powered by Skillsoft, the SkillChoice™ Complete library contains more than $3,000
worth of industry-leading online courses, books, videos, mentoring tools and exam
prep. Best of all, you get it for the one low price of your Preferred Plus, Training &
Development, or Student membership package. There’s something for everyone,
from beginners to advanced IT professionals to business leaders and managers.

The IT industry is constantly evolving. Don’t be left behind. Join the IEEE Computer
Society today, and gain access to the tools you need to stay on top of the latest
trends and standards.

Learn more at www.computer.org/join.


Looking for
the BEST Tech
Job for You?
Come to the Computer Society Jobs Board to meet the best
employers in the industry—Apple, Google, Intel, NSA, Cisco,
US Army Research, Oracle, Juniper...

Take advantage of the special resources for job seekers—

job alerts, career advice, webinars, templates, and resumes
viewed by top employers.