Home Area Network Performance and Reliability

Emerging Standards, Prototype Software

1018982

Home Area Network Performance and Reliability

Emerging Standards, Prototype Software 1018982 Technical Update, December 2010

EPRI Project Manager E. Ibrahim

ELECTRIC POWER RESEARCH INSTITUTE 3420 Hillview Avenue, Palo Alto, California 94304-1338 PO Box 10412, Palo Alto, California 94303-0813 USA 800.313.3774 650.855.2121 askepri@epri.com www.epri.com

DISCLAIMER OF WARRANTIES AND LIMITATION OF LIABILITIES

THIS DOCUMENT WAS PREPARED BY THE ORGANIZATION(S) NAMED BELOW AS AN ACCOUNT OF WORK SPONSORED OR COSPONSORED BY THE ELECTRIC POWER RESEARCH INSTITUTE, INC. (EPRI). NEITHER EPRI, ANY MEMBER OF EPRI, ANY COSPONSOR, THE ORGANIZATION(S) BELOW, NOR ANY PERSON ACTING ON BEHALF OF ANY OF THEM: (A) MAKES ANY WARRANTY OR REPRESENTATION WHATSOEVER, EXPRESS OR IMPLIED, (I) WITH RESPECT TO THE USE OF ANY INFORMATION, APPARATUS, METHOD, PROCESS, OR SIMILAR ITEM DISCLOSED IN THIS DOCUMENT, INCLUDING MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, OR (II) THAT SUCH USE DOES NOT INFRINGE ON OR INTERFERE WITH PRIVATELY OWNED RIGHTS, INCLUDING ANY PARTY'S INTELLECTUAL PROPERTY, OR (III) THAT THIS DOCUMENT IS SUITABLE TO ANY PARTICULAR USER'S CIRCUMSTANCE; OR (B) ASSUMES RESPONSIBILITY FOR ANY DAMAGES OR OTHER LIABILITY WHATSOEVER (INCLUDING ANY CONSEQUENTIAL DAMAGES, EVEN IF EPRI OR ANY EPRI REPRESENTATIVE HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES) RESULTING FROM YOUR SELECTION OR USE OF THIS DOCUMENT OR ANY INFORMATION, APPARATUS, METHOD, PROCESS, OR SIMILAR ITEM DISCLOSED IN THIS DOCUMENT. THE FOLLOWING ORGANIZATION, UNDER CONTRACT TO EPRI, PREPARED THIS REPORT Finite State Machine Laboratories, Inc. (FSMLabs)

This is an EPRI Technical Update report. A Technical Update report is intended as an informal report of continuing research, a meeting, or a topical study. It is not a final EPRI technical report.

NOTE
For further information about EPRI, call the EPRI Customer Assistance Center at 800.313.3774 or e-mail askepri@epri.com. Electric Power Research Institute, EPRI, and TOGETHERSHAPING THE FUTURE OF ELECTRICITY are registered service marks of the Electric Power Research Institute, Inc. Copyright 2010 Electric Power Research Institute, Inc. All rights reserved.

ACKNOWLEDGMENTS
The following organization, under contract to the Electric Power Research Institute (EPRI), prepared this report: FMSLabs, Inc. 5910 Blanco Pass Road Austin, TX Principal Investigators C. Rodine T. Godfrey This report describes research sponsored by EPRI.

This publication is a corporate document that should be cited in the literature in the following manner: Home Area Network Performance and Reliability: Emerging Standards, Prototype Software. EPRI, Palo Alto, CA: 2010. 1018982.
iii

ABSTRACT
The renewal of the North American electrical infrastructure the re-factoring of the present plant into a Smart Grid is among the most ambitious engineering feats to be undertaken in our time. The urgency, criticality, complexity, and cost of the endeavor are daunting, and can seem almost overwhelming. Faced with pressing societal needs capping, then decreasing carbon emissions; curbing per-capita energy consumption; raising the efficiency of energy production and delivery; integrating renewable energy sources and increasingly limited resources, leaders in government and industry are underlining the need to pursue Smart Grid goals with maximum overall efficiency and productivity. In response, over the past two years the technical community has focused on developing a layered Smart Grid architecture and a set of open standards to support end-to-end and top-tobottom interoperability. The advantages of this approach for rapid innovation; system integration, adaptability and scalability; and system stability and reliability over time, have been proven in similar large-scale infrastructure deployments (e.g. in the Internet and World Wide Web). In particular, to support Demand Response, Pre-Pay, Plug-in Hybrid/Electrical Vehicle Charging, and other customer-based Smart Grid applications, requirements and standards for communicating over a Home Area Network (HAN) have been formulated. Unfortunately, one essential capability went missing: metrics and methods for tracking the performance and reliability of HAN-based communications are so far not included in any NIST, SDO or industry standards. As of August 2009, when EPRI launched the R&D described in this report, there was still a clear need, but no support, for standards and tools enabling energy providers to verify the performance and reliability of current and emerging HAN technologies. In collaboration with utility sponsors, ERPI validated this need and began addressing it as part of its IntelliGrid program (Project 161D). This report describes our approach and results. In summary: We codified and validated three utility use cases for HAN performance monitoring: - verify proper device performance during installation; - diagnose a problem during operation; - pro-actively collect metrics and report network reliability (performance over time). We developed prototype software tools for two HAN technologies (IEEE 802.15.4/ZigBee and Homeplug AV) and validated the use of basic PHY-level metrics (RSSI) and available MAC-level metrics (Frame Re-tries, FCS failures) for performance monitoring purposes. We formally introduced metrics into two SDO or industry standards Task Group projects (IEEE 802.15.4e and Homeplug HP-SE), and socialized them in three others (IEEE P1901, IEEE 802.11 Ad-Hoc Smart Grid Group, Wi-Fi Alliance Smart Grid Marketing Group). We engaged with utilities (in addition to our sponsors), vendors, and industry alliance groups (Homeplug Powerline Alliance, Wi-Fi Alliance, and ZigBee Alliance) to promote the use and value of common HAN performance metrics in Smart Grid initiatives.

CONTENTS
1 INTRODUCTION ....................................................................................................................1-1 Smart Grid Communications Reference Model....................................................................1-1 Overview of Smart Grid Communications Technologies......................................................1-3 Communications Performance and Reliability .....................................................................1-5 Examples of Retail Energy Services ....................................................................................1-6 2 THE HOME AREA NETWORK (HAN) ...................................................................................2-1 HAN Reference Architecture................................................................................................2-2 Review of Leading LAN and HAN Technologies..................................................................2-3 Use Cases for HAN Performance Monitoring ......................................................................2-5 3 HAN COMMUNICATIONS PERFORMANCE: MODELS, STANDARDS, AND METRICS ...3-1 Models of Wireless and Wired HAN Communications .........................................................3-2 Proposed HAN Performance Metrics ...................................................................................3-3 PHY Layer Metrics .........................................................................................................3-4 MAC Layer Metrics.........................................................................................................3-5 MAC Transmit Metrics....................................................................................................3-5 MAC Receive Metrics.....................................................................................................3-6 Using Performance Metrics for Network Monitoring and Management................................3-7 4 PROTOTYPE HAN INSTRUMENTATION: SOFTWARE TOOLS .........................................4-1 Software Tools for IEEE 802.15.4 / ZigBee .........................................................................4-1 Measurement Set-up and Sensors ................................................................................4-1 Measurement Methodology..................................................................................................4-2 Example Applications: RSSI History Plotter; Whole House RF Survey .........................4-3 Software Tools for IEEE P1901 / Homeplug AV ..................................................................4-6 5 HAN METRICS: STANDARDS STATUS AND IMPLEMENTATION.....................................5-1 Principles Guiding our Effort ................................................................................................5-1 Introducing Metrics in IEEE 802.15.4e (MAC Enhancements TG) ......................................5-2 Exposing Metrics in Broadband Powerline Carrier (PLC) Standards...................................5-4 Homeplug AV (HP-AV)...................................................................................................5-5 Homeplug Green PHY (HP-GP).....................................................................................5-5 IEEE P1901....................................................................................................................5-5 Standardizing PLC Metrics.............................................................................................5-6 Exposing Metrics in IEEE 802.11 (Wireless LAN)................................................................5-7 IEEE 802.11k .................................................................................................................5-7 IEEE 802.11v .................................................................................................................5-7 Using IEEE 802.11 Metrics ............................................................................................5-8 Paths to Implementing 802.11 Metrics ...........................................................................5-8

vii

6 SYSTEM CONSIDERATIONS................................................................................................6-1 HAN Security........................................................................................................................6-1 HAN Internetworking ............................................................................................................6-2 HAN Service and/or Capability Discovery............................................................................6-3 7 END-TO-END PERFORMANCE MONITORING AND EPRIS HAN SUPPLEMENTAL PROJECT ..................................................................................................................................7-1 8 CONCLUSION........................................................................................................................8-1

viii

LIST OF FIGURES
Figure 1-1 Interfaces between Smart Grid Domains..................................................................1-2 Figure 1-2 Simplified Smart Grid-to-Customer Communications Model ....................................1-3 Figure 2-1 Demand Response Reference Architecture .............................................................2-2 Figure 3-1 Classical Communications System Model................................................................3-2 Figure 4-1 RSSI Plot #1 .............................................................................................................4-3 Figure 4-2 RSSI Plot #2 .............................................................................................................4-4 Figure 4-3 Relative RSSI Plot of a Representative Customer Premise .....................................4-5 Figure 4-4 3D Plot of Relative RSSI in a Representative Customer Premise............................4-6 Figure 4-5 How the Homeplug AV Monitoring Application Functions ........................................4-7 Figure 7-1 End-to-End HAN Monitoring System ........................................................................7-1 Figure 7-2 Architecture of Metric-Enabled HAN Devices...........................................................7-2

ix

LIST OF TABLES
Table 1-1 Examples of Utility Communications Technologies ...................................................1-4 Table 3-1 HAN Performance Metrics .........................................................................................3-3 Table 3-2 PHY Layer Metrics in HAN Standards .......................................................................3-5 Table 3-3 MAC Transmit Metrics ...............................................................................................3-6 Table 3-4 MAC Receive Metrics ................................................................................................3-7

xi

1
INTRODUCTION
The Smart Grid vision became increasingly focused and clear throughout 2009. The National Institute of Standards and Technology (NIST) orchestrated an industry-wide effort towards the development of a framework for Smart Grid interoperability and standardization. An impressive body of work is under way, organized around business and operational use cases; decomposition of the problem space into sub-domains and logical interfaces; information modeling and IT and communications architectures; and the systematic collation of requirements. Through open, collaborative and consensual NIST workshops facilitated by EPRI, thirty one technology standards covering key aspects of energy transmission, distribution and retail operations have been initially identified as relevant to the Smart Grid. The establishment of processes for evaluating additional candidate standards, and for converging towards a coherent interoperability framework, falls to the Smart Grid Interoperability Panel (SGIP) and the NIST Priority Action Plans (PAPs), in which EPRI actively participates. Two of the highest priority areas of Smart Grid development are Demand Response where an energy provider enrolls customers in programs for direct or indirect load control and Plug-in Hybrid/Electric Vehicle charging, where vehicle batteries can serve as manageable Distributed Energy Resources e.g., roaming storage and generation capacity in a utilitys distribution plant. These key applications require communication between energy service providers and their retail customers. A network located in the customer premise that can meet this need is designated in the Smart Grid architecture as a Home Area Network, or HAN. In this section, we situate the HAN within an overall conceptual model of Smart Grid communications. In Section 2, we survey some of the salient characteristics, advantages and drawbacks of the leading HAN technologies. Smart Grid Communications Reference Model Though the precise meaning and scope of the term Smart Grid are still matters of some debate, the conceptual model that emerged from the work of the 2009 NIST Smart Grid Interoperability Standards workshops provides a useful high-level representation (see Figure 1-1). Our main focus in Project 161D is on the Customer Domain, where the HAN is situated. In Figure 1-1, connections between the Customer Domain (e.g., the HAN) and four other Domains Service Provider, Operations, Markets, and Distribution are depicted by so many blue lines, representing logical (i.e. potential, eventual) connections. For the purposes of this report, however, we can model interactions between the HAN and the other Smart Grid domains as messages passing through a single interface that manages communications for all of the consumers retail energy services, no matter in which Domain the supporting back-end systems reside.

1-1

Figure 1-1 Interfaces between Smart Grid Domains

Not only does this simplify our model, it also represents the current technical and practical reality. In most if not all Advanced Metering Infrastructure/Smart Meter systems being tested or deployed today, utility-HAN interactions are implemented through a single services portal.1 While for some purposes it is useful to treat information flows between Domains discretely i.e., for end-to-end security analysis for the study of HAN performance and reliability, they can be understood to pass via a single logical portal between the service provide and the customer. Our purpose in measuring HAN communications is to determine how reliably messages that are sent through that single logical portal reach the devices that are involved in the delivery of Retail Energy Services. Another helpful abstraction, for the purpose of situating the HAN in the Smart Grid architecture, is from the networking layers at which services are implemented. Inter-Domain communication will naturally be governed by functions implemented at different levels, as appropriate. For example, one component of inter-device or inter-network security might be implemented as packet-level encryption (at network layer 3), and another might be an application-level access control mechanism (i.e. a password challenge at layer 7). The dynamic determination of which path (route) a packet should take through a mesh network would also take place at network layer 3. However, since our main concern is with the performance and reliability of HAN-based

Modeled as the Energy Services Interface (ESI) in the OpenHAN SRS [ref]. The ESI may be implemented within a Smart Meter, or it may reside in another HAN-connected gateway, for example a broadband service provider settop box, or a Home Energy Management System. For the purpose of monitoring communications flows between the energy service provider and the customer, its specific location is not critical.

1-2

communication, our focus will be on the physical communication medium and the channels it supports (layer 1), as well as the control logic governing access to and use of those channels (layer 2). These are the foundations on which all higher-level communications functions are built, so our modeling of HAN performance and our standards efforts are focused on the so2 called PHY and MAC layers of the networking protocol stack. The simplified, abstract model shown below (Figure 1-2) highlights the features of the Smart Grid-to-Customer Domain communications most relevant to our project.

Figure 1-2 Simplified Smart Grid-to-Customer Communications Model

Factors affecting the performance and reliability of this Service Provider-to-Customer link between a utilitys Advanced Metering Infrastructure (AMI) and the customers Home Area Network are the subject of a companion EPRI report [cit.]. After some additional introductory material in the next two subsections, the remainder of this report concerns the performance and reliability of intra-HAN communications, e.g. of the links between HAN-connected devices that can participate in retail energy service offerings. Overview of Smart Grid Communications Technologies The concept of a Smart Grid involves communications over a utilitys entire service territory, which can cover hundreds or even thousands of square miles. Some applications and operations for example, deployment of Wide Area Situation Awareness (WASA) will require communication over even larger regions, e.g. the multi-state territories of an ISO or RTO. At the other geographical extreme, the successful deployment of retail energy services might hinge on the link between a utilitys Smart Meter, installed on the side of a customers house, or a service provider-installed EMS inside the home, and devices located a few feet or a few meters away.

Layers one and two of the OSI networking reference model, see [ref].

1-3

The Smart Grids network speed, bandwidth, and other communications requirements can be as varied as the geographical aspects mentioned above. To support critical substation protection devices, the speed of round-trip communications might need to be faster than the duration of a single cycle of alternating current (AC) that is, 20mS for a 50 Hz system and 16.67 mS for a 60 Hz system. Demanding transmission protection applications may require round-trip performance orders of magnitude faster. In contrast, some applications may require data to be sent once an hour, or even less frequently. The volume of data produced by few dozen Phasor Measurement Units (PMUs), i.e. endpoints of an IOU-scale WASA application, can be considerable on the order of tens of Gigabytes per day; whereas the amount of data generated by a protection relay, or a retail Demand Response device, might be as low as a few tens of kilobytes per week. In the way of illustration, the table below (Table 1-1) includes a partial subset of communication technologies being used today in the listed utility applications.
Table 1-1 Examples of Utility Communications Technologies Application Technology
EIA RS-232 EIA RS-485

Medium
STP STP

Data rate
115.2 Kbps (15m) 35 Mbps (10m) 100 Kbps (1200m) Max range 4000 m 10/100/1000 Mbps (up to 100m) 10/100/1000 Mbps (up to 400m) Multiple E1/DS1 & 10/100BaseT; T3/DS3 (44.736 Mbps) Multiple E1/DS1; T3/DS3; 100/1000 Mbps (up to 20 Km) 4.8 19.2 Kpbs

Topology/Access
P2P*, Star Bus (multidrop)

Substation Automation

Ethernet over Copper

(10/100/1000BaseT)

STP (CAT 5e, Cat6) Single-/Multi-Mode Optical Fiber 5.8 GHz, 6-38 GHz RF

Bus, Star (CSMA/CD) Bus, Star (CSMA/CD) Multichannel, redundant P2P

Ethernet over Fiber

(10/100/1000BaseX)

LOS Microwave

Transmission Line Protection

Optical Fiber
(i.e., 100/1000BaseX)

Single-/Multi-Mode Optical Fiber

Distribution Automation

Proprietary Wireless Cellular

(GSM, GPRS, CDMA)

900 MHz (MAS, PCS)

P2P, P2MP P2P Mesh, Star Cellular multiple access

850/900/1800/1900/2100 MHz RF 902-928 MHz RF (ISM) 2.3, 2.5, 3.5 GHz 850/900/1800/1900/2100 MHz

9.6 153 Kbps 20 250 Kbps 256 Kbps 30+ Mbps

C&I Metering, AMI

P2MP Wireless 4G Wireless (WiMAX, LTE)

STP = Screened or Shielded Twisted Pair {STP, STP-A, ScTP, S/UTP, S/STP, S/FTP} * P2P = Point-to-Point; P2MP = Point-to-MultiPoint

With such a wide range of requirements across various applications and domains of the Smart Grid, no single communications technology will economically meet all the technical needs. The solution will have to involve multiple subsystems with differing characteristics (physical media, reliability, speed, bandwidth, etc.), all connected in an internetworking architecture. Luckily, the task of engineering this kind of hybrid solution is relatively well understood, thanks to experience gained in the design and deployment of similar industrial systems and modern

1-4

telecommunications infrastructure (e.g. national and international enterprise IT infrastructures; the worldwide Internet). Developing a well-balanced network architecture, which ties a utilitys disparate communications systems into a coherent whole, is one of the holy grails of Smart Grid design. Facing the number and diversity of legacy utility communications systems, designers need to consider the media (i.e. wired vs. wireless, licensed vs. unlicensed spectrum, RF propagation characteristics); modulation schemes and carrying capacity (i.e. data rate, bandwidth, channelization), latency and degradation under load, etc. of each underlying technology. To successfully deploy Smart Grid applications, utilities will require tools to monitor and manage the performance and reliability of the disparate constituent networks, in order to gain end-to-end visibility and control. Most relevant to our studies are the characteristics and performance of Advanced Metering Infrastructure and the Home Area Network. We cover critical aspects of the AMI-to-HAN connection in the companion report [ref], and focus on requirements for HAN performance and reliability in Sections 2 8 of this report. Communications Performance and Reliability A number of factors affect the performance and reliability of any communications medium or channel. Physical factors include the strength of the underlying signal that carries information (the carrier), a measure of radiated or received energy; and the ratio of a desired signals energy to that of noise and interference in the channel, due to the nature of the medium or other radiators. The radio spectrum and a vast range of common physical media (various types of cable, wire and optical fiber) have been well characterized, so published reports can inform choices among options for Smart Grid communications.3 By their nature, characterizations provide measures of average conditions and deviations, but do not address dynamic fluctuations in actual conditions. To provide insight into dynamic conditions, industrial and commercial communications systems are typically equipped with facilities for monitoring performance.4 They provide several levels of scrutiny and reporting, which are designed for use in system installation and calibration, during regular operations, for running periodic health checks, and diagnosing faults. Utility platforms such as microwave relay terminal stations, Ethernet switches for substation automation, and Advanced Metering Infrastructure systems may include this functionality via their device or element management interfaces. In the case of consumer electronics and home networking equipment, instrumentation is limited or non-existent. This is reasonable, since almost all consumer equipment is meant to operate stand-alone the service provided is just what the device delivers as configured out-of-thebox. Instrumentation adds cost and complexity to the design and manufacture of devices, and its usefulness and value to consumers, who dont want to have to manage the device, is doubtful.

In the case of wireless technologies, the choice is influenced by the cost and other conditions of obtaining a spectrum license, or contracting with a spectrum owner (i.e. a wireless carrier). Because this can be burdensome, systems designed for non-critical utility applications, i.e. AMI, typically run on unlicensed radio spectrum. 4 This is sometimes called instrumenting the system.

1-5

To verify that consumer-oriented communications products will work as planned and advertised, manufacturers test and characterize the device in the lab and in a range of typical environments for example, a new five-port Ethernet switch designed for home or office use will be deployed and monitored in a number of test homes. When the devices settings and software are deemed suitable and satisfactory, the vendor seals the box (i.e. issues final firmware) and ships the product. Thereafter, limitations on performance and reliability are determined largely by the environment in which the device operates.5 The advantages of the consumer electronics approach to communications performance and reliability are clear. The disadvantage of a completely service-free model is that consumers may not be able to get certain devices to work, leading to a certain (predictable) amount of dissatisfaction and missed opportunities.6 The consumer electronics industry can tolerate this, as long as the vast majority of users are satisfied with the service that an off-the-shelf, sealed box product delivers. In contrast, the electric power industrys regulated structure and strong record of service reliability set up an expectation of extremely reliable, near-universal service. Therefore, the success of retail energy services will depend on HANs performing reliably enough to escape customer notice (they should just work), in practically every premise where such services are desired. Despite its apparent simplicity, this is a very demanding requirement. Whether one pursues a brute-force hardware redundancy approach or leans towards a network management solution, various aspects of installation, provisioning, and maintenance will have to be mastered. One promising example of a hybrid business model a balance between the pure consumer electronics and the pure installed service models is the communication services model of broadband Internet and mobile telephony providers, featuring lightweight device management interfaces and remote monitoring capabilities. Service configuration, monitoring, and management are typically transparent, but when required, remote personnel can intervene and guide customers in maintaining their devices health and serviceability. The metrics and prototype tools we have developed lay a foundation for utilities to develop such a hybrid approach to managing HAN-based retail energy services. Examples of Retail Energy Services Customer-oriented services of most interest to utilities in the near term are the display of Energy Usage data and Pre-Pay status; the deployment of Demand Response programs; and the integration of Distributed Energy Resources, in particular Plug-in Hybrid/Electric Vehicle Charging services. While each of these could be implemented as a discrete system with its own communications support, there are obvious advantages to using a common Home Area Network for all utility-customer and energy device-to-device interactions. The functional communications

The ability reliably to download new versions of firmware to the device can improve performance and reliability. While it adds some cost and complexity to device design and manufacture, this capability is becoming common in mid- and high-range consumer networking devices. The ability to download apps, pioneered and proven in the mobile device market, is also now a feature of some home networking and energy management devices. 6 One can approximate this by the percentage of products returned to the retail outlet for replacement or credit; or customer account churn due to low-quality service (dropped or noisy calls, etc.) in the mobile telephony industry.

1-6

requirements of all these retail services are not extremely demanding, and can be met by the HAN technologies under consideration. If energy service providers knew that HANs existed or could be readily deployed in every customer premise, they could confidently develop new retail energy services. With the HAN concept and reality established, the emergence of platforms supporting retail energy service delivery, with associated business models and markets for application development and sales, would become more likely.7 In-Home Display (IHD) devices present metering, and other energy data and messages from the energy service provider to the consumer.8 If an IHD is meant to support Pre-Pay services, it might also need to support two-way communications, so customers can acknowledge receipt of account information and warnings, send payment data, etc. For Demand Response to be effective and fair (enabling all to participate with equal opportunity), the HAN must perform reliably and deliver price or direct load shed signals to every enrolled customer within a designated timeframe. Initially that might be on the order of ten hours or more (for day-ahead pricing), but movement towards near-real-time pricing would drive that down to an hour or less. At utility scale this could amount to a requirement for as many as 9 500,000 reliable transactions (acknowledged messages) in a matter of seconds. The integration of large numbers of small-scale, variable output DERs (such as Photovoltaic solar, PH/EVs, or small-scale storage) could be the most challenging retail energy service yet, with potentially significant impacts on distribution and transmission infrastructure as well as the HAN. For example, voltage regulation and distributed load balancing applications might require tighter coupling between controls on loads and sources, which (assuming some DERs will be connected to the grid via the HAN) would drive requirements for yet higher HAN performance (reliability, bandwidth, rapid response). These examples illustrate how energy retail services, separately and in aggregation, have the potential to become a major IT-based operation within a utilitys business and drive networking requirements from and in the HAN to the AMI system, the backhaul network, and the datacenter.

With mobile smartphone architectures and applications such as the iPhone (from Apple Computer) and Android (from Google), the platform approach has borne fruit beyond its origin in the Personal Computer field. From a consumer electronics market perspective, the concept of a Home Energy Management platform and associated application development environment seems compelling. See Platform Leadership (Gawer & Cusumano, 2002); and Product Platform and Product Family Design (Simpson, Siddique & Jiao, 2007). The Smart Energy Profile, v2 (ZigBee and Homeplug Alliances) might aspire to be an energy management application platform in this sense. 8 If such data and messages are hosted on a website, customers can retrieve them via the Internet but this may not obviate the need or justification for HANs. Not all utility customers have Internet access at home, and as discussed, many utilities have, in effect, a universal service mandate. In addition, transporting volumes of data from residential meters to servers in the utilitys back office plant puts a burden on AMI infrastructures. Finally, behavioral and ergonomic research might indicate a level of customer need or preference for an attached, alwayson device they dont need to power on an advanced thermostat, for example that has a comfortable, intuitive interface for controlling all aspects of home energy management. 9 Assuming a major IOU with 5M customer meters and 10% of customers enrolled in DR programs.

1-7

2
THE HOME AREA NETWORK (HAN)
During 2007-2008 a list of HAN requirements was established by the OpenHAN Task Force of the Utility Communications Alliance International Users Group (UCAIug). Published in mid2008, the OpenHAN SRS v1.0.4 was an important input to the DOEs Modern Grid Home-toGrid Domain Experts Working Group (DEWG). In 2009, as part of the NIST Standards Roadmap Phase I effort, the OpenHAN SRS requirements were incorporated in the Demand Response and PHEV Priority Action Plans (PAPs) as a baseline for NISTs interoperability framework for the Customer (HAN) domain. The OpenHAN SRS version 2.0 was ratified in mid-August, 2010 and the final document is being prepared for release.10 While the OpenHAN SRS includes requirements for application-level message integrity and confirmation of delivery, it has no provisions for measuring and verifying the performance and reliability of the HAN network or HAN devices. Nor was this requirement captured in any of the NIST PAPs. There might be an industry assumption that the underlying HAN media (PHY/MAC technologies and implementations) are reliable enough out of the box for utility purposes, or that HAN technologies already have adequate provisions for monitoring the health and reliability of physical and logical channels. For reasons given above (see the discussion in Section 1.3), and as we found in our research, these assumptions are not sound. Section 3 of this report presents a summary of the monitoring capabilities of leading HAN technologies, and describes proposed and emerging standards to meet the challenge of largescale utility operations. To set the context, in this section we present a HAN reference architecture that makes explicit the logical channels and physical connections between typical HAN devices and systems, and present three leading HAN technology candidates, namely: 1. IEEE 805.15 .4 Personal Area Network technology 2. Homeplug HP-AV/ HP-SE and IEEE P1901 (Broadband PLC) technologies 3. IEEE 802.1b/g/n Wireless LAN technology The principles, metrics, standards orientation, and software tools described in this report are directly applicable to any other HAN technology that utilities may choose to deploy. In Section 3 we present the classical channel model from which definitions of metrics used to monitor and manage communications systems are derived. We then propose a set of performance metrics that can be readily implemented (or already exist) in the three HAN technology candidates listed above. In Section 4 we describe prototype software tools that were developed under this project in the EPRI Intelligrid program, to demonstrate the technical feasibility and operational relevance of the proposed HAN metrics.

10

See [OpenHAN v2] available at http://osgug.ucaiug.org/sgsystems/openhan/default.aspx

2-1

HAN Reference Architecture The diagram below11 is a useful depiction of the logical links between entities within the Customer Domain, and with some relevant systems in other Domains. It represents an abstract or reference architecture that any HAN communications technology should be able to support. The connectors marked 1-3 have no direct association with the Customer Domain, and lie outside the scope of our report. The remaining connectors, marked 4-21, are links by which messages pertaining to a range of retail energy services can pass, and are therefore relevant to our study. The six entities (actors) depicted within the Customer Domain are HAN-connected devices12 that implement the message-passing and energy management capabilities required to deliver retail energy services.
Operations
8

Distribution Management
13

Grid Operations DR Services Provider

3 9 10 11

12 14

Service Provider Markets

Aggregator
5 1 2 6 15 19 4

EMS

16

Meter or Smart Meter

17

20 21

Device/ HAN Device

Customer

18

Smart Appliance

Retail Market

Display
7

Distributed Generation

Demand Response Use Cases: Actors and Logical Interfaces

HAN: Home Area Network EMS: Energy Management System DR: Demand Response

Key:
Actor 1
#

CommunicationsAssociation/Connector ID
Actor 2

LogicalInterfaces

Figure 2-1 Demand Response Reference Architecture

11 12

Appeared originally as: Figure 16, p. 55 in EPRI, Report to NIST on the SGISR, August 10, 2009. The connection need not be permanent. For example, the entity labeled Distributed Generation could designate a Plug-in Hybrid or pure Electric Vehicle, which might reside on the HAN at some times and be disconnected at others. That exception noted, other HAN-connected devices do not exhibit this roaming behavior and can, for our purposes, be considered permanent fixtures during their lifetime.

2-2

The entities in Figure 2-1 are shown with multiple logical links, which is helpful for associating their behaviors with use cases, developing interoperability test suites, etc. However, as in the case of the connections between the Customer Domain and other Smart Grid Domains (see Section 1), we can model the device as having a single HAN interface, perhaps with multiple logical channels or ports to facilitate the partitioning of application-level messaging. And once again, the single interface is a good representation of technical reality HAN devices typically have a single physical interface (wired or wireless transceiver) via which they connect to the network,13 and higher-level structures (e.g. software APIs) that enable the use of multiple logical channels and/or message streams over the physical medium. In addition, HANs can be organized in various network topologies, for example in bus, star and mesh configurations. These, too, are logical details that dont directly impact our model: since packets sent from one node to another in any of these topologies will have to pass through two or more physical interfaces, the end-to-end performance of a path through any network topology will be a function of the performance of the underlying physical links. Our model also works across multiple networks, each perhaps implemented on different media, connected via bridges, routers or gateways that is, for gauging end-to-end performance in an internetworking environment. What all this means is: by enabling each HAN physical interface to measure and report the channel conditions it experiences, we establish a foundation for measuring communications performance between any HAN devices regardless of the technology (medium), the network topology, or the higher-level applications they support. HAN monitoring and management tools that build on this foundation can track the performance and reliability of all the connections between entities within the Customer Domain, and most if not all of the connections between these devices and the entities in the other Domains.14 These tools can be used to verify performance during HAN device installation and to monitor performance during operation. The use of such tools will greatly enhance the utilitys operational awareness of the HAN infrastructure, and should result in increased customer satisfaction with a utilitys retail energy service offerings. Review of Leading LAN and HAN Technologies There are many potential candidates for HAN network technology they include any control or Local Area Network (LAN) that fits customer and utility needs (including cost, re-use of existing LAN, ease of installation, broad market support, and reliability). Thanks to developments in the personal computing and consumer electronics market segments, the LAN technology most widely deployed by customers in mature markets (the US, Japan, Korea, Europe, etc.) is Wi-Fi (based on IEEE 802.11). Some customer premises have structured wiring, which supports wired Ethernet (IEEE 802.3) over twisted pair (i.e. CAT5e). Low-power, low data-rate wireless
13

HAN devices with both wired and wireless interfaces, for connection redundancy and increased reliability or to serve as bridges, are likely to appear as the market matures and volumes grow (thus reducing costs). However, the complexity of manually configuring such a device might be a barrier to widescale adoption see Section 6 below on the desirability of Auto-configuration and Service Discovery. 14 As noted in Section 1.1, a prominent Smart Grid architecture has all the messages between energy service provider platforms and the HAN passing through a single Energy Services Interface, or a similar portal. Such a focal point offers significant advantages for implementing inter-network functions such as security enforcement, in addition to performance and reliability monitoring.

2-3

technologies, including ZigBee and Z-wave (both based on IEEE 802.15.4) are being used for home automation and energy control. There are several Powerline Carrier (PLC) options, featuring low to very high data-rates (6 Kbps 200 Mbps), which use premises electrical wiring as their physical medium. The range of HAN choice is wide, and utilities are presently engaged in testing and comparing the options. Ideally, all HAN technologies should be compatible and even interoperable, enabling plug and play of a wide variety of energy-aware devices and services. If that were the case, retail energy management systems, devices, and applications could discover and utilize whatever network(s) were deployed in the customer premises. This is the vision driving the Smart Grid industrys concerted architectural, standards and product development efforts. Based on a number of factors, primarily in our view the pressure that some utilities are under to roll out retail energy services in the very near term, there are different interpretations of that vision. Two main approaches have emerged. The first, driven by the ZigBee Alliance and some major US utilities, features medium bit rate (bandwidth), low cost wireless Personal Area Networking technology carefully adapted for energy management.15 A second, emerging with the arrival of major Internet-segment vendors in the Smart Grid arena,16 is looking towards broadband HANs using wired and wireless flavors of Ethernet, as the media on which to develop and deploy retail energy services. To help utilities mitigate the risk inherent in such an emerging market situation, we are carefully specifying HAN metrics that are common (comparable) across both approaches, while being closely aligned with the design philosophy and technical specifics of each. Along with technical factors, utilities are pondering issues of risk and responsibility that come with the advent of retail energy services. To minimize the potential for difficulties, some energy service providers might want directly to provide HAN devices that they have tested and are confident will meet their program requirements. For example, they would install specific PCTs (Programmable Communicating Thermostats) and other controllers using a chosen HAN technology, which allows the provider remotely to change settings (i.e. heating or cooling setpoints, or duty cycles) on command, or based on a schedule. In this scenario the HAN technology choice will be made by the utility, which would ostensibly assume a certain amount of responsibility for the reliable performance of the devices and HAN technology used in implementing the service. Other energy service providers might not want or be able to assume the costs and responsibilities described above. They would prefer a business model in which a utility qualifies, specifies, and/or certifies (or requires that vendors certify) HAN-based energy services devices. Customer would purchase designated, certified equipment through retail channels and install (or have a professional install) whatever is required to implement a service. This is widely thought to be the best business model, but is more ambitious in that it requires (from another perspective: presents an opportunity for) coordination between energy service providers and other sectors of the

15

The ZigBee Alliance is collaborating with the Homeplug Alliance on merging their solution with a low-power, moderate bitrate PLC technology: Homeplug Smart Energy, currently in development. 16 By Internet segment we mean vendors that supply Internet-based consumer, enterprise, and communications business segments, namely Cisco Systems, Google, et. al. More recent arrivals include telecoms carriers such as AT&T and Verizon, whose experience in the smartphone market and mobile data services may inform the retail energy services market segment as well.

2-4

economy, namely the homebuilding, consumer electronics, and automotive industries. As such, this has become a matter of regulatory interest at state and national levels. In any case, we believe the need for HAN performance and reliability monitoring is clear, and that our prototype tools and standards initiatives are relevant to any eventual outcome. By providing a common, fundamental capability on all the presently relevant HAN technologies, and an efficient path to implementation on others, they encourage the development of hardware and software tools supporting a lightweight approach to HAN performance monitoring, which will help utilities mitigate the business and technical risks of deploying retail energy services.17 Use Cases for HAN Performance Monitoring The HAN initial communications metrics and prototype software tools we have developed support three initial use cases, which were discussed and validated at a high level with our projects utility sponsors: 1. During HAN device installation and provisioning: on-site verification of connectivity and performance between the AMI system (i.e. a Smart Meter, where applicable) and HAN devices, and among HAN devices involved in utility-driven home energy management (including a retail energy or communications service provider-supplied device, where applicable). 2. During regular operation of retail energy services: remote execution of HAN device and network diagnostics, for example to enable issue disposition and resolution of HAN device connectivity or other communications problems. 3. During regular operation of retail energy services: periodic automated collection of HAN performance metrics by a Smart Meter, premise HAN device, or remote system, to enable trend monitoring and pro-active management of retail energy service reliability (i.e., automated reports and alarms).

17

See Section 3.4 below for a discussion of heavyweight and lightweight network management models, with examples.

2-5

3
HAN COMMUNICATIONS PERFORMANCE: MODELS, STANDARDS, AND METRICS
Fundamental to communication theory is the concept of a channel, and its capacity for delivering messages from a source to a target with a given statistical measure of reliability, in the presence of a given level of interference (noise).18 Based on this foundation, a massive body of work has been done in the laboratory and field settings to measure and characterize the behavior of various types of signals over all the media (cable configurations, radio spectrum bands, optical fiber, etc.) used in commercially available systems. Common measurement categories have arisen from theory and characterization that capture the salient aspects of channel behavior.19 As described below, our design of HAN metrics are derived directly from this theory and practical experience. In any particular communications technology, a set of pertinent technical parameters (in the case of FM radio, for example: carrier and stereo subcarrier frequencies and power levels, guard bands, modulation scheme, acceptable distortion levels, etc.) are codified in a specification setting forth, as clearly, completely and coherently as possible, the behaviors of the system. By allowing multiple parties to implement the technology, communications standards define the technical rules of the game and create market value by establishing an economic incentive for vendors to play the game and field products supporting the technology. However, if a standard did not allow for implementers to differentiate their standards-compliant products if it fully specified all behaviors and left no room for added value the game would never begin. One important aspect of the art of defining standards is to leave room for innovation in various areas of product design or manufacture. Performance is a factor with many dimensions, and vendors often make engineering trade-offs to create products that meet a standards requirements, which are also optimized for use in certain 20 applications, environments, conditions, etc. Since this allows for and even encourages a range of system behaviors that remain valid under a standard, it is practical to define methods and metrics for reporting performance as part of the standard specification. In this way, the standard can find its broadest range of applicability, while users of the standard from vendors of supporting silicon and software, to system builders and end customers will be able to monitor performance to verify its typical reliability, and manage exceptional conditions and cases.

18 19

Shannon, 1948. These include energy in the channel, relative or absolute signal strength, signal-to-noise ratio, bit and packet error rates. See Section 3.2. 20 One common trade-off is between cost/complexity of the product, and the conditions temperature range, power consumption, etc. under which it can meet the performance levels defined in the standard.

3-1

Models of Wireless and Wired HAN Communications The classic representation of a communications system and the components that comprise it is shown below (Figure 3-1). This simple scheme is the foundation for the remarkable advancement of communications theory and technologies that has occurred since it was published by Claude Shannon in 1948. Developments since then have resulted in a steady, better-than-linear growth in the amount of information that can be encoded in a channel of a given width, or with a given computational effort. Also increasing have been the channel capacity (bits per second per Hertz) of the physical media underlying those channels; the accuracy (decreased bit error rates) with which information is successfully sent and received; and the distance over which communications can occur with a given signal strength (electromagnetic energy); the ability to communicate in the presence of noise and interference; and device power consumption. To review just one component of the system, models of the communication channel (the center portion of the diagram) have come to include sub-models of noise, distortion, interference, fading, RF propagation, channelization, and other factors that affect the capacity and fidelity of the channel. Other components of the model, which involve functions like analog-to-digital conversion, coding and decoding, and carrier modulation schemes, have benefited from similar development.21

Figure 3-1 Classical Communications System Model22

A thorough characterization of particular media (a cable design or portion of the electromagnetic spectrum) can require very complex analysis and modeling. As these studies are published and comprehended by the engineering community, they inform the design of instruments, antennas, circuits, electron devices (i.e. interface ICs), and other components of communications systems. Communications standards like those behind the HAN technologies under review abstract from the variations and vicissitudes that signals can experience in the medium and capture its statistically predictable, useful behavior in some key parameters, such as channel width, signal to noise ratio, modulation (data) rate, etc. These key parameters are common among a wide range of standards, and provide a small set of core metrics for the purpose of gauging the performance and reliability of HAN communications. These metrics can be efficiently implemented in a given

21

On channel modeling, we have consulted Durgin, Gregory D. Space-Time Wireless Channels, Upper Saddle River, NJ: Prentice-Hall, 2003 [ISBN 0-13-065647-X]; and Goldsmith, Andrea. Wireless Communications, New York, NY: Cambridge University Press, 2005 [ISBN 0-521-83716-2]. 22 Shannon, 1948 (p. 2).

3-2

HAN technology by simply measuring or recording the conditions or states that each device experiences at the physical and medium access layers of the system. A summary of the HAN metrics, designed for their efficiency in implementation and their power to capture the essential measurements of system conditions, is given below in Table 3-1.
Table 3-1 HAN Performance Metrics Attribute (Metric) Description The number of transmitted frames that required exactly one retry before acknowledgement The number of transmitted frames that required more than one retry before acknowledgement The number of transmitted frames that did not result in an acknowledgement after max MAC retry parameter. The number of transmitted frames that were acknowledged within normal MAC frame timeout parameter The number of received frames discarded due to an incorrect FCS The number of received data frames that contained the same sequence number as a frame previously received (accounting for counter wrap-around) The number of received data frames that were received correctly PHYs ability to decode signal (data) in the presence of noise Fixed or adaptive data rate, in Mbps or b/S Background noise (wireless HAN technologies only) Provides additional information on PHY channel conditions (not mandatory)

MAC_TX_One_Retry

MAC_TX_Multiple_Retry

MAC_TX_Failure

MAC_TX_Success

MAC_RX_FCS_Error MAC_RX_Duplicate

MAC_RX_Success PHY_Link_Quality PHY_Data_Rate PHY_Idle_Channel_Noise PHY_Beacon_Reliability

Proposed HAN Performance Metrics Even though the three HAN standards are significantly different in various ways, it is advantageous to provide a consistent set of metrics across all three HAN technologies, to simplify the implementation of management functions and the operation of the network. We have researched the metrics that already exist in IEEE 802.11 and IEEE P1901, and proposed
3-3

equivalent metrics for inclusion in IEEE 802.15.4, in order to facilitate the development of common upper-layer network monitoring and management tools for use in utility deployments of retail energy services. The metrics developed for HAN performance monitoring fall into two categories. At the physical (PHY) layer, the metrics relate to signal strength, signal quality, channel noise, and usable data rate. These metrics are all interrelated, and analysis of the set provides the most complete picture. At the Medium Access Control (MAC) layer, the metrics relate to the processing (success or failure in transferring) of packets. This are measured as packets at the top of the MAC (MSDU or MAC Service Data Units) or at the bottom of the MAC (MPDU or Mac Protocol Data Units), depending on the standard. In the following sections we survey the PHY and MAC layer metrics and present their definitions and locations in the three leading HAN technologies (802.15.4, P1901, and 802.11).23 We also indicate how they can be used by higher-layer network monitoring or management applications to report network performance to the utility operator or customer. PHY Layer Metrics Although the various HAN standards implement their physical layers in significantly different ways, there are commonalities that indicate the physical layer conditions across the standards. All the standards have some concept of Link Quality, which is tied to the underlying PHYs ability to decode the signal (data) in the presence of noise and impairments. This is generally presented to higher layers as a single octet value, with the range of 0 to 0xFF representing an arbitrary dimensionless range from unusable to perfect. The use of the physical layers data rate is not universal among standards. In general, better channel conditions between peer PHY entities enable higher data rates to be used. However, data rate can be a misleading indicator: while 802.11 [1] offers a wide range of data rates, and operates at the highest rate allowed by channel conditions, some of the PHY options for 802.15.4 [2], which is used by ZigBee [3], support a single data rate. Power Line Carrier standards such as P1901 [4] and Homeplug AV [5] report PHY layer capacity in terms of a Bit Loading Estimate (BLE). BLE is expressed in bits per microsecond (b/uS), which corresponds to the more common megabits per second (Mb/S) units used by wireless standards. Background noise measurements are present in wireless standards for determining the optimum channel, but are not available as a single value for power line carrier. PLC standards continuously and dynamically adapt to noise through the use of tone maps [6]. The concept of a beacon is present in all the HAN standards being discussed, but it is not mandatory. When present, the reliability of the beacon (how often and how well it is received by a given device) can provide additional information on the network condition at the PHY layer. A list of PHY metrics and their MAC-level access methods (SAPs, APIs or MMEs) for the three leading HAN standards is given below in Table 3-2.

23

Since these are all IEEE standards, we will occasionally drop the IEEE moniker for simplicity.

3-4

Table 3-2 PHY Layer Metrics in HAN Standards Metric Link Quality 802.15.4 / ZigBee MCPS-DATA. indication parameter: mpduLink Quality. MCPS-DATA. indication parameter: DataRate P1901 / HomePlug CC_DISCOVER_LIST.CN F Stationinfo[n]: Signal Level CC_LINK_INFO.CNF parameters: Forward Link BLE, Reverse Link BLE. <N/A due to nature of PHY> CC_BEACON_RELIABIL ITY.CNF Reliability = 1 (NMB / NBP) 802.11 / Wi-Fi 7.3.2.38 RCPI (received channel power indicator)

Data Rate

PHY SAP: PHY-RXSTART. indication, DATARATE parameter in RXVECTOR

Idle Channel Noise

PLME-ED. PLMEED.confirm parameter: EnergyLevel Optional in 802.15.4; if present, MLMEBEACON-NOTIFY. indication. PAN Descriptor LinkQuality

7.3.2.21.5 Noise Histogram Request 7.3.2.21.6 Beacon Request, Beacon Report

Beacon Reliability

MAC Layer Metrics One function of the MAC layer is to provide acknowledgment and retransmission for unicast packets. While higher layer protocols may provide their own mechanisms for reliable delivery, they can interpret packet loss as an indicator of congestion rather than channel impairment. It is at the MAC layer that useful metrics concerning channel conditions can be collected. MAC layer acknowledgment is used by all of the HAN standards under review. While it hides occasional packet loss due to noise and channel impairments from the higher layers (at the cost of increased latency), it enables much higher overall throughput as a result. Metrics based on MAC acknowledgment and retry behavior are valuable indicators of overall network performance. MAC Transmit Metrics When transmitting, the MAC controls the acknowledgment and retry process. When a packet is transmitted, an acknowledgment (ACK) is expected within a set timeout period. If the ACK is not received within the timeout period, the transmission is retried. The process continues up to a maximum retry count. Useful metrics for the transmit process can be obtained by counting the number of first-time successes, the number of packets with one retry, the number of packets with more than one retry but eventual success, and those that were discarded due to exceeding the allowable number of retries.

3-5

Table 3-3 MAC Transmit Metrics Metric One Retry 802.15.4 / ZigBee PIB 0x81 macRetry Count P1901 / HomePlug CM_LINK_STATS.CNF LatBin(0) 802.11 / Wi-Fi 7.3.2.22.8 STA Statistics Report dot11 RetryCount 7.3.2.22.8 STA Statistics Report dot11Multiple RetryCount 7.3.2.22.8 STA Statistics Report dot11 FailedCount 7.3.2.22.8 STA Statistics Report dot11 TransmittedFrameCount

Multiple Retry

PIB 0x82 macMultipleRetryCount

CM_LINK_STATS.CNF Sum (LatBin(1) LatBin[N]) CM_LINK_STATS.CNF Tx_NumSeg_Dropped

Failure

PIB 0x83 macTX FailCount

Success

PIB 0x84 macTX SuccessCount

CM_LINK_STATS.CNF Tx_NumSeg_Suc

MAC Receive Metrics On the receive side of the MAC layer, the acknowledgment and retry behavior of the transmitter is not directly visible. The primary receiver-side metric is the Frame Check Sequence (FCS) which provides a means to detect errors within the packet. This function is described as the Integrity Check Value (ICV) in powerline standards. The FCS is calculated at transmission and appended to MAC packets (MSDUs). The FCS is implemented using Cyclic Redundancy Check sequences, which are generator polynomials designed to maximize the likelihood of detecting bit errors. The FCS is calculated again in the receiver, and if the calculated value does not match the value appended by the transmitter, the frame is discarded and is not acknowledged. The number of FCS errors is collected as a MAC metric along with the number of correctly received frames. The ratio of errors to correctly received packets is the significant metric that indicates the condition of the network.

3-6

Table 3-4 MAC Receive Metrics Metric FCS Error 802.15.4 / ZigBee PIB 0x85 macFCSErrorCount P1901 / HomePlug CM_LINK_STATS.CNF NumICV_FAILS 802.11 / Wi-Fi 7.3.2.22.8 STA Statistics Report dot11FCSErrorCount Duplicate Frame PIB 0x87 macDuplicateFrameCount < Due to the use of MSDU segmentation and Selective ACK, this metric does not apply> CM_LINK_STATS.CNF Rx_NumMSDUs 7.3.2.22.8 STA Statistics Report dot11Frame DuplicateCount dot11Received FragmentCount

Success

PIB 0x88 macRX SuccessCount

Using Performance Metrics for Network Monitoring and Management Utilities and service providers have a wide range of choice in network management protocols to communicate remote HAN network status and performance metrics back to a centralized control center. The common aspect is the use of the Internet Protocol (IP), which has become a de facto standard for Smart Grid communication. Simple Network Management Protocol (SNMP) has been widely adopted for managing enterprise LANs and WANs, as well as service provider and mobile network infrastructures. SNMP has gone through evolutionary development over many years. The current version is SNMP V3 [11], which includes a security model that addresses some shortcomings of previous versions. SNMP is a relatively heavyweight network management protocol. While such an approach is not a burden on enterprise class equipment, the code size and processing requirements increase the cost of high volume, price-sensitive devices such as Smart Meters, HAN devices, gateways, and similar products. There is a need for a lighter alternative to SNMP for network management, but there is no clear direction towards standards in this area. An optimal solution would re-use existing software architecture and infrastructure to support network management, allowing higher functionality at low implementation and development cost. Two initiatives in this direction are the Diagnostics Cluster being explored by the joint ZigBee-Homeplug Alliance Smart Energy Profile 2.0 project, and the loosely related effort to model HAN interfaces in a proposed Part 10 of the IEC 61968 Common Information Model (CIM). In any case, higher layer network management applications would use these HAN metrics by reading them on a regular basis and taking action based on the relative changes in the counters. (The counters wrap to zero when they exceed a maximum count, so comparison algorithms must take the wrapping into account in their calculations.) Of most interest to utilities are actions relating to network reliability, for example alarms when performance is decreasing or below a certain minimal threshold. The technical community is also interested in other, more technical functionality enabled by these metrics. For example, in an IEEE 802.15.4 stack, a segmentation process operating at layer 3 could respond to an increase in

3-7

macTXFailCount and macTXMultipleRetryCount by reducing the MTU size and increase the probability that frames could get through. Increases in macFCSErrorCount could cause the management entity to move the PAN to a different frequency, or change the routing in the case of a mesh network. MAC metrics can be used in conjunction with PHY metrics to gain further insight into network conditions. For example, if the PHY indicates good RSSI (Received Signal Strength Indication) and good LQI (Link Quality Indication), but the MAC is reporting a high ratio of failures, then it is likely that the problem is due to interference.

3-8

4
PROTOTYPE HAN INSTRUMENTATION: SOFTWARE TOOLS
In order to test our assumptions about the practicality and power of HAN metrics, we launched an initiative to develop prototype HAN monitoring tools. These consist of an adapted PC application (an Ethernet network analyzer tool) for controlling the testing and displaying results, and two new software modules: a simple RSSI/Energy Detection application for IEEE 802.15.4/ZigBee, and a PHY/MAC metrics reporting application for Homeplug AV. This work is still in progress, but the initial results are promising and have provided proof points in our discussions about our standard HAN metrics with utilities and Smart Meter and HAN interface device (IC, for Integrated Circuit) vendors. (See Section 6 below.) Our methodology was to work with commercially available HAN IC development kits. Our first prototype tools were for IEEE 802.15.4/ZigBee, and we had access to application-level software and APIs only. Thus we were limited to collecting the PHY level metrics exposed by the vendors ZigBee protocol stack.24 A richer set of metrics, reporting performance at both the PHY and MAC layers, are defined in the Homeplug AV standard but are not generally available to application developers. A supportive IC vendor provided us with a firmware build that exposes the interface (MAC Management Entities, or MMEs) needed to collect the metrics described in Section 3 above. Software Tools for IEEE 802.15.4 / ZigBee The reliability of RF-based HANs depends on devices, no matter where they are located, regularly receiving sufficiently strong signals over the air. At the first level of analysis, this is a function of the power of the transmitted signal, noise in the channel, and receiver sensitivity. Given that IEEE 802.15.4 is deployed over unlicensed radio bands, in which interference is to be expected, our main focus was on measuring the difference between signal and noise at the receiver position. If this becomes too small, the signal will be lost and communication will be impaired. Measurement Set-up and Sensors The IEEE 802.15.4 standard defines 16 channels in the 2.4 GHz ISM band. To determine channel characteristics such as energy level, signal strength, adjacent channel interference, distortion, etc., a signal generator and spectrum analyzer can be deployed to measure link

24

Other metrics are defined in the ZigBee standards, but are qualitative. See for example the Link Status Command, which is used by ZigBee routers to convey the cost (1-7) of sending packets via a neighboring routers ingoing and outgoing links. The cost algorithm is not specified, and is not obviously a measure of packet or bit error rate. See also Annex E, "Operation of Network Manager as Network Channel Manager for Interference Reporting and Resolution, which specifies some parameters related to channel quality, but they are not precisely defined.

4-1

performance in absolute values (e.g. in dBm units). Such a set-up and process is used in laboratory tests of devices (i.e. integrated circuits and modules supporting IEEE 802.15.4), and for detailed characterization of a given field environment (i.e. a single site of interest), but the expense and effort involved in such a test set-up is prohibitive for RF surveys of dozens or hundreds of customer premises. Another drawback of such a test bed is that it is extrinsic to the system it does not measure the behavior of the actual systems to be deployed in support of retail energy services. By using the communication interfaces own measurement capabilities, the characteristics of the actual systems that deliver the service are reported. For example, the radio front end will have specific channel filtering and out-of-channel rejection characteristics based on circuit design, choice of components, physical layout, shielding, etc., which differ from those of a lab-quality oscilloscopes probe. We wanted to account for such engineering trade-offs in the design and manufacture of HAN devices, which are driven by device type, price, and many other factors. Such trade-offs can affect in-situ behavior, so HAN devices that meet the minimal IEEE 802.15.4 performance requirements in the lab may perform quite differently from one another under field conditions. Finally, to get as close as we could to actual HAN device behaviors, we developed our prototype performance monitoring tools on engineering evaluation boards for IEEE 802.15.4-compliant products (radio ICs) that some Smart Meter and HAN device vendors are actually using in their products.25 Working with semiconductor evaluation boards also provides a way to prototype the proposed IEEE MAC-level metrics described in Section 3. Measurement Methodology A PC application was used to control an IEEE 802.15.4 transceiver, which sent ping (loopback) packets to an embedded ZigBee measurement application on a remote station. The PC application was used to send request packets, receive responses, aggregate statistics on received data, and display results. The control radio was calibrated in a nearly ideal environment (a remote Nevada lake site) and the manufacturers specifications of transmit power, receiver sensitivity, dynamic range, etc. were verified. This was critical to establishing baseline values for LQI/RSSI and Energy Level, which enabled us to calibrate reported receiver LQI/RSSI values with absolute power levels (dBm) and make accurate RSSI measurements in typical RF-noisy environments. Measurements were collected from the radio by the embedded ZigBee application using the IEEE 802.15.4-defined EnergyLevel and mpduLinkQuality PIB parameters. On the devices used, the latter value was delivered by the firmware as a measurement of signal-plus-noise energy, so the received signal strength (RSSI) was calculated by subtracting the channel noise energy, measured when no signal was present, from this composite value.

25

Our tests were done using evaluation platforms for the Ember 260 (Ember EM260-JMP-R) and Atmel ATmega128L (Meshnetics Meshbean WDB-A1281-P1) IEEE 802.15.4/Zigbee products. The characteristics of finished commercial products will likely differ from those of evaluation boards, for example in power amplifier and antenna design. However, our approach is the same as a vendor takes when evaluating HAN interface (IC) products, and provides a reasonable baseline for device performance assessment.

4-2

Example Applications: RSSI History Plotter; Whole House RF Survey As a first application of our HAN performance measurement software, we developed a graphing tool that plots received signal strength (RSSI) values over time. RSSI between two stations (i.e. proxies for HAN devices) is plotted as the scalar value (length) of a vector, whose radial value (angle) changes regularly with time. The resulting plot provides an intuitive representation of cumulative average values and changes in signal strength (a major factor in channel capacity) between two nodes in an IEEE 802.15.4 network.26 Example graphs are included below: Figure 4-1 shows relative RSSI measurements ranging around -50 to -55 dB, with differences of up to 30 dB, while Figure 4-2 shows a tighter density of average values (the test was run longer, in an environment with a quieter RF background) between -45 and -50 dB, but indicates a wider variation (~40 dB). The first test was run in an unoccupied home, so the plot shows RF background noise variations only. The second plot shows variations due to both background noise and typical human activity in the customer premises, namely foot traffic crossing the path between the test devices.

Figure 4-1 RSSI Plot #1

26

The changes in channel conditions are due to interference, distortion, and other factors that always have the potential to impact RF systems. See Section 3.1, above.

4-3

Figure 4-2 RSSI Plot #2

For our tests in this location, we turned off all appliances (Wi-Fi Access Points, microwave ovens, portable phones, baby room monitor intercoms, etc.) that would add interference in the 2.4 GHz ISM band channels used by IEEE 802.15.4-based radios. Thus, the background noise is due to other emitters in the vicinity (i.e. neighbors Wi-Fi Access Points). When such strong local sources are active in the environment, our RSSI plotting tool has shown interference that is within 10 dB of the radios sensitivity floor (operational limit), which could result in bit-level error rates that seriously hamper HAN communications. If such conditions are only temporary, then MAC frame retry logic, or higher-layer packet retry and routing logic, may be able to overcome the interference, albeit with delayed delivery times.27 Using the relative RSSI graphing tool, EPRI developed a second HAN application and procedure, to map relative signal strength between an AMI-to-HAN gateway (modeled as located in a Smart Meter) and locations in the customer premise. The result is an indication of where wireless HAN communication in the 2.4 GHz ISM band can be expected to be more and less reliable. Example plots for a typical San Jose/San Francisco/Oakland (California) Bay-area home are given below.

27

Since we are focusing on the foundational behavior of HAN physical and link layers, it is out of our scope to model the behavior of higher-level adaptive schemes, which introduce overhead i.e. for packet retransmission and re-routing. An alternative, layer 2 approach is automatic adaptive channel switching, though this is not defined in IEEE 802.15.4. In general, lower-level solutions are more efficient; in any case, the metrics we are driving into HAN standards provide data about channel conditions that could inform these higher-level algorithms.

4-4

Figure 4-3 Relative RSSI Plot of a Representative Customer Premise

The two-dimensional plot in Figure 4-3 shows the received signal strength for every 2 ft.-by-2 ft. square in and around the home. It indicates the location of typical household items that reflect or refract the HAN signal; signal strength is attenuated in the RF shadow of metal-backed mirrors, white-box appliances, and other items. If there were a need to reach HAN devices in the darker blue areas, where signal strength could easily fall below the receivers sensitivity floor, a ZigBee repeater node might be required to ensure reliable communications. Figure 4-4 provides an illustrative three-dimensional representation of the same relative RSSI data.

4-5

Figure 4-4 3D Plot of Relative RSSI in a Representative Customer Premise

Such an RSSI mapping application will be used to establish a baseline profile for each premise in a follow-on program for gathering aggregate data on the reliability of wireless (ZigBee and WiFi) HANs in utility employee homes. This HAN Supplemental program is described more fully in Section 6, below. Software Tools for IEEE P1901 / Homeplug AV We have been developing a similar network monitoring application for Homeplug AV. Since Homeplug standards specify an industry-standard interface (an Ethertype frame format) for communication of management information between end stations and network controllers, we were able to adapt a standard Ethernet monitoring tool to our purposes. We are developing this application to have essentially similar capabilities as the tools for IEEE 802.15.4/ZigBee wireless network monitoring described in the previous section. The figures below provide an initial indication of how the Homeplug AV monitoring application functions.

4-6

Figure 4-5 How the Homeplug AV Monitoring Application Functions

4-7

5
HAN METRICS: STANDARDS STATUS AND IMPLEMENTATION
For reasons given above, our goal is to define and promote equivalent (technically comparable) metrics in the three HAN technologies of most interest to North American utilities: ZigBee, Homeplug, and Wi-Fi. Of course, the relevant SDO and technology alliance projects each have their own goals, process, calendars, etc. Here we describe our approach to the specific work and task groups; the progress we made and the challenges we met; and a range of potential outcomes. Principles Guiding our Effort Our main guiding principle was that the Smart Grid market is best served by standards created by internationally recognized and renowned SDOs. The fact that NIST also embraced this principle reinforced this rationale. Our focus on metrics at the PHY/MAC networking layers naturally led us to engage with the IEEE 802 group, whose standards for wired and wireless communications dominate the LAN industry. Wi-Fi is based on the 802.11 family of Wireless LAN standards, and ZigBee is based on the 802.15 standards for so-called Wireless Personal Area Networks, or PANs.28 In addition, wired Ethernet over twisted copper and fiber optic media is defined the 802.3 sub-group, and the MAC interface and LLC data encapsulation defined in 802.2 are used at the top layer of both 802.3 and 802.11. Finally, 802.1 lays out a common architecture for (almost all) 802 standards, and specifies methods and protocols for intra-802 services including bridging, security (secure tunneling, access control, key management), quality of service, and network management.29 The fact that another IEEE standards group undertook to merge competing broadband powerline carrier technologies (HD-PLC, Homeplug AV, and UPA) in the P1901 standard, added support to our choice of prioritizing IEEE among potential SDO and alliance engagements. From extensive and intensive personal experience, the investigators knew that the IEEE Standards Association (SA) has well-established participation, access, voting, and Intellectual Property disclosure policies and processes, and a relatively good track record of fairness and

28

The PAN moniker is no longer an accurate indication of 802.15s scope, and can be confusing. HAN and AMI technologies were and are being developed in the dot15 group because the leaders of that effort, who are also involved in the ZigBee Alliance, took an early and aggressive position in the Smart Grid market. Towards the end of 2009 two other 802 entities, the dot11 WLAN and dot16 MAN groups, launched Smart Grid study groups that are gaining momentum in the wireless industry. 29 Notably, the 802.15 family of standards supports neither 802.2 LLC encapsulation nor the 802.1 architecture. A core choice and principle of dot15 is for lowest-power, lowest-complexity (cost); under those constraints, compatibility with the rest of 802 was deemed not feasible. This has had important consequences, namely that critical functions like security and routing have been defined in the ZigBee Alliance, and are therefore less visible and familiar to the broad community of wireless networking engineers than classic Ethernet and IETF architectures and protocols.

5-1

accountability. Both are members of the IEEE-SA, which qualifies them to participate first-hand in most IEEE standards-making processes. (P1901 is an entity-based process, see below.) Finally, the IEEE is deeply involved in the NIST Smart Grid Phase 2 Standards efforts (the SGIP, the PAPs, etc.) and has made strong investments in Smart Grid infrastructure, including coordinating groups (the IEEE Intelligent Grid Coordinating Committee, the P2030 effort), a new top-tier technical publication (IEEE Transactions on Smart Grid), and international workshops and conferences dedicated to Smart Grid Communications.30 Market and product design considerations also influenced our approach. ZigBee technology took an early HAN market lead and is adapting to requirements for end-to-end support of the Internet Protocol suite. By mid-2008, leading utilities included a requirement for a wired HAN solution, preferably one that would support the ZigBee Smart Energy Profile, Version 1.31 Homeplug was a logical PLC choice in the North American and European context, so utilities helped organize the Joint ZigBee-Homeplug Alliance Working Group to develop marketing and technical requirements for a combined wireless-wired Smart Energy Profile, Version 2. The strong preference, amounting to an implicit mandate, for SDO-originated standards led us to focus on the IEEE P1901 project, which incorporates Homeplug AV and HD-PLC, a broadband PLC standard created and used in Japan, in a dual PHY/MAC stack standard. We also tracked and contributed to the Homeplug Smart Energy Working Group effort on the so-called Green PHY specification, a low-power-consumption adoption of Homeplug AV that is designed to be 32 fully interoperable with IEEE P1901. Introducing Metrics in IEEE 802.15.4e (MAC Enhancements TG) A key design goal of the IEEE 805.15.4 Personal Area Networking group is to keep the protocols at a low degree of complexity, in order to minimize the power consumption and cost of interface ICs. Originally designed as a means to connect simple devices over short distances, 802.15.4 has very limited management capabilities; richer network management requirements may emerge as the protocol is extended to different applications (e.g. by the 802.15.4g Smart Utility Networks Task Group). There is always a question of where (at which layers of the network stack) its best to expose management functions: for example, the ZigBee standard and application profiles include some device and network management entities and functions. Some clarity on this design issues is emerging the ZigBee Smart Energy Profile, Version 2, defers to the underlying IEEE

30

Kathy Kowalenko, Smart Grid Projects Pick Up Speed, The Institute, 6 August 2009. Retrieved August, 2010: http://www.ieee.org/portal/site/tionline/index.jsp?pageID=institute_level1_article&TheCat=1044&article=tionline /legacy/inst2009/aug09/standards.xml ; http://smartgrid.ieee.org/ (Power and Energy Society); www.smartgridcomm.org (International Workshop, associated with the Communications Societys International Communications Conference); www.ieee-smartgridcomm.org (Joint PES-ComSoc International Conference); http://www.comsoc.org/Smart-Grid (ComSoc SG home) ; http://smartgrid.ieee.org/ieee-smartgrid-news/67introducing-ieee-smart-grid-world-forum (world-wide coordination of Smart Grid developments). 31 To be superseded by SEP Version 2, which is now expected to be ratified by mid-2011. 32 Intellon Corporation was awarded a Department of Energy Smart Grid Investment Grant of $5M to accelerate the production of Green PHY products, one of very few device vendors to benefit from this stream of federal stimulus funding. The completion of Intellons acquisition by Atheros Communications, Inc., a major supplier of 802.11 ICs for Wi-Fi applications, was announced on December 15, 2009.

5-2

802.15.4 specifications for access to PHY and MAC layer management capabilities.33 Whether and how these become exposed via APIs or management frames for use at higher layers is still to be determined. We engaged with IEEE 802.15.4 leadership in September, 2009 and were encouraged to present utility requirements and proposed metrics in the 4e Task Group, under the groups charter for driving MAC enhancements into the standard. Before reporting on our progress, current, and expected outcomes, an explanation of how changes to elements of the 802.15.4 standard are incorporated will be helpful. All IEEE 802 standards follow a similar process for incorporating amendments and revisions. A working group may develop multiple independent standards. Some working groups, such as 802.11, chose to apply all amendments to the base standard, so you see a progress of task groups from 802.11a through 802.11z, and on to 802.11af. The 802.15 working group has developed a number of independent standards, which are given sub-numbers such as 802.15.1, 802.15.2, and so on. A Task Group has a Project Authorization Request (PAR) that outlines the scope of the specific amendment to the base standard that it will develop. Task Groups are assigned sequential letters such as TG4a (which is the first amendment to 802.15.4), TG4b, etc. When an amendment has completed balloting and has been approved by the standards board, it is officially a part of the standard and has the same authority as if it were merged into the base standard. The amendment will exist as a separate document (the one that was balloted) for a period of time, until the base standard is updated. This update can be accomplished as a roll-up or as a revision. A roll-up is a purely editorial process where one or more amendments are merged into the base standard. A roll-up may be completed by IEEE staff, without the involvement of the working group. A revision is a maintenance activity which is conducted by a Task Group, under a PAR. Depending on the PAR, updates to the base standard (in addition to the merging of one or more amendments) are developed. The IEEE Standards Association may decline further PARs for additional amendments when three amendments have been approved but not incorporated via a roll-up or revision. As of mid 2010, 802.15.4 was last revised in 2006, resulting in 802.15.42006. There are now three outstanding approved amendments (802.15.4a, 802.15.4c, and 802.15.4d), so the standard is overdue for a revision. The revision will be conducted in a new task group 802.15.4i. A revision PAR has just been approved for 802.15.4(mid-2010). The work will be done in the 802.15.4i Task Group. The PAR allows maintenance changes (editorial and technical corrections) to 802.15.4-2006, in addition to incorporating the three amendments. Our proposal to introduce a work item was accepted by the 802.15 Task Group 4e at the September, 2009 802 Wireless LAN/MAN Interim Meeting (Waikoloa, HI). Our proposed text for inclusion in the 4e amendment (the resulting work product) was approved by the group at the November, 2009 802 Wireless LAN/MAN Plenary Meeting (Atlanta, GA). The substantive content of this proposal for IEEE 802.15.4 PHY and MAC metrics is included above in Table 31, Table 3-2, and Table 3-3.

33

Available at http://www.zigbee.org/Markets/ZigBeeSmartEnergy/Version20Documents.aspx.

5-3

The 802.15.4e Task Group finalized the first draft of its proposed amendment during the March, 2010 802 Wireless Plenary meeting (Orlando, FL). It then went to Letter Ballot, and work on comment resolution took place around the May, 2010 Wireless Interim meeting (Beijing, PRC) and into the summer. It is likely that a second Letter Ballot will be issued at the end of the September, 2010 802 Wireless Plenary meeting (Waikoloa, HI). If so, the group should be on track to go to Sponsor Ballot (the final stage of IEEE standards definition) at or around the November, 2010 802 Wireless Plenary meeting (Dallas, TX). When approved by the 802.15 Working Group (after a series of Letter Ballots and the resolution of substantive comments), and the IEEE Standards Board (after the completion of Sponsor Ballot and the resolution of substantive comments), the MAC Modifications amendment, including the specification of HAN metrics that EPRI introduced, will become part of the overall IEEE 802.15.4 standard. At a later time, the 802.15.4e amendment will be merged into the base standard, either by a rollup or revision process (at the discretion of the 802.15 Working Group). If the revision process is chosen, and the PAR allows technical changes, the entire standard is open for changes. Typically, only corrections and clarifications are incorporated, as there is a common presumption that the Task Group has done its work responsibly and well, using the IEEEs open, accountable process: 4e progress reports were shared at plenary sessions with the entire 802.15 community, and members were welcome to join calls, attend sessions, make contributions, submit comments, etc. Thus, changes proposed to ratified sub-sections during the revision process are typically limited to addressing unforeseen interactions between amendments done in different Task Groups, or other issues that are critical to the standards success or failure.34 They are submitted to careful scrutiny and rigorous debate at these later stages in the process. The 802.15.4i revision will need to be completed before a subsequent revision or roll-up can be started to incorporate 802.15.4e 802.15.4f, and 802.15.4g. Based on the pace of progress in 4g Smart Utility Networking Task Group), and other 802.15.4 TGs whose amendments will also be folded into the main standard document in a subsequent revision, the projection and expectation at this reports publication time (Summer, 2010) is that the next revision of IEEE 802.15.4 will be approved in the first half of 2011 (and thus be designated IEEE 802.15.4-2011). Then a new revision activity can begin to incorporate 802.15.4e, f, and g, which would likely be completed sometime late in 2012. As always in standards work, this is the best approximation of the Working Group and is, of course, subject to change based on circumstances. Exposing Metrics in Broadband Powerline Carrier (PLC) Standards The data elements that capture metrics in the Homeplug/IEEE P1901 family of broadband PLC are given in the third column of Table 3-2 through Table 3-4. They are defined in the Homeplug AV and Green PHY (HP-GP) industry standards, and in the draft IEEE P1901 SDO standard as part of the FFT stack. A brief explanation will help clarify the provenance, present status, and path to formal specification of metrics in these standards.
34

To avoid surprises later in the ratification of the next version of the full IEEE 802.15.4 standard, an ad-hoc group including members of the 4e and 4g subgroups has been meeting since the March, 2010 802 Wireless Plenary to manage some requests for changes in 4e coming from the 4g group, as part of the process of resolving comments on the first 4e Letter Ballot. This is a standard method of coordinating between the two subgroups, while allowing the faster one to drive towards meeting its PAR goals on the projected schedule.

5-4

Homeplug AV (HP-AV) The Homeplug AV standard was created to enable transport of streaming multimedia over premise electrical wiring. It inject an RF signal between 4 MHz 24 MHz onto premises electrical wiring, to encode more than 1,000 data channels using a version of OFDM (Orthogonal Frequency Domain Modulation), yielding a maximal raw PHY bit rate of 200 Mbps. This is at least twice the speed of widely-deployed wired and wireless LAN technologies used in offices and homes.35 As an indicator of its capacity, with compensation for channel noise, packet collisions, IP/UDP stack overhead, etc., Homeplug AV can in practice support payload data rates in the range of 15-30 Mbps enough to support one or two High-Definition Television (HDTV) streams. In order to support testing and validation, the designers of Homeplug AV specified a wide range of metrics in the standard. These include PHY channel- and MAC frame-level indicators comparable to those defined in IEEE 802.11 WLAN and to those we defined for IEEE 802.15.4.36 Thus, EPRI did not need to create new metrics for this family of PLC technologies. The challenge was rather to gain access to this data for our performance monitoring application: the Homeplug AV standard specifies that metrics are to be made available to applications via low-level software interfaces (so-called MAC Management Entities, or MMEs) that are not part 37 of IC vendors standard evaluation kits or reference designs. As noted in Section 4.2 above, we were able to obtain a firmware build that provides access to the required MMEs from a leading Homeplug vendor, who is supporting our monitoring application development effort. Homeplug Green PHY (HP-GP) Homeplug Green PHY (HP-GP) is a derivative of Homeplug AV specifically designed for Smart Grid applications. It can be thought of as a profile (set of operational parameters) for HP-AV that enables the same OFDM-FFT technology to run at much lower power consumption and much higher reliability, though at reduced bitrates (a minimum guaranteed 250 Kbps, projected typical rates of 0.5-2.0 Mbps). As such, the HP-GP specification inherits almost all the details of HP-AV; specifically it includes the desired performance metrics. IEEE P1901 The IEEE P1901 draft standard is a combination of two, very similar broadband PLC technologies: Homeplug AV and High-Definition PLC (HD-PLC). IEEE P1901 incorporates these two well-established and widely deployed industry alliance standards, referred to as Wavelet and FFT respectively for their PHY technology, modified minimally and only as

35

Fast Ethernet at 100 Mbps, used i.e. for PC wired access (where Gigabit Ethernet is not yet available), and of the IEEE 802.11g-based Wi-Fi networks at a maximum of 108 Mbps. (The maximum data rate specified in the IEEE 802.11g standard is 54 Mbps, but well-specified, interoperable enhancement techniques used in modern 802.11gbased Access Points often double that to 108 Mbps.) The now-finalized 802.11n specification defines raw bitrates of up to 300 Mbps (MIMO 2x2) or 600 Mbps (MIMO 4x4, rarely implemented). 36 The common, underlying models and methods used to characterize all communications channels facilitate this equivalence of metrics in the different technologies. See Section 3.1, above. 37 Vendors make the MMEs available to device vendors on request, especially for the purposes of product characterization and test. We found only one Homeplug vendor SpidCom whose product offering includes a network management application it is a proprietary solution developed for one of the companys customers (a telecoms service provider), and doesnt support a standard set of metrics or interfaces.

5-5

needed, in a dual stack standard. The main new feature of P1901 is the specification of coexistence schemes, which enable these two technologies and other PLC flavors to work harmoniously on the same physical medium.38 This provides the industry with an SDO standard that is backwards compatible with the two leading broadband PLC technologies, thereby protecting the installed base. Thus the P1901 standard enables vendors to support either technology stack, or both, without fear of performance being degraded beyond specified, verifiable levels due to the presence on the premise wiring of devices supporting other PLC technologies. Standardizing PLC Metrics Since mid-2009 EPRI engineers have been active participants in the Homeplug Powerline Alliance Smart Energy Technical Working Group, where the Green PHY specification is being developed, as well as in IEEE P1901 Working Group, Technical Work Subgroup #3, where the main work on that standard is being done. We have presented utility requirements and use cases for PHY/MAC metrics and HAN reliability monitoring to the groups, and the subject gained general interest and in-principle approval in both. Specifically, there has been no objection for technical or any other reasons over a sustained period, and EPRI has gained credibility as a [if not the] voice of the utility industry, which has become a critical customer base for all PLC vendors. For various reasons, for the entire period of our engagement these efforts have been under extreme time pressure and quite resource constrained. Under the auspices of the Homeplug Powerline Alliance, the Green PHY effort has been tightly focused and relatively unhampered by outside factors, allowing it to drive hard towards completion and earlier products to market. On the other hand, it represents a smaller, more focused market segment and a smaller number of vendors than the broader IEEE P1901 effort. As an SDO project, the P1901 group includes a larger and broader constituency, but has had to grapple with strong intervention by competing segments of the PLC industry whose intent was not always constructive. In both cases, the period since mid-2009 has required relentless focus on the narrow goals of resolving Letter Ballot comments (more than 3,000 in the case of P1901, which the group resolved in the incredibly short time of 3 months) and including essential features (i.e. the ISP and CX mechanisms in P1901; power saving and GP-mode compatibility with AV in HP-SE). Until the key project goals of a successful Sponsor Ballot (P1901) and publication of the final specification (HP-SE) are met, the groups are limited in their ability to add any new features. However, as described above, since both HP-GP and P1901-FFT inherit the metrics specified in HP-AV, these are not new features. EPRI has recommended and the groups have embraced steps towards their incorporation in the standards. In HP-GP we are helping to specify APIs that will provide standardized access to the required MMEs. In P1901 we have identified the need for a profile (an annex to the formal standard with recommendations for practical application) for using the HP-AV MMEs for utility HAN monitoring. Finally, as noted above we reached out to the HD-PLC community and invited them to document and provide access to similar metrics that

38

These are the so-called Inter-System Protocol, or ISP for inter-P1901 coexistence, and the so-called CX method for coexistence with non-P1901 PLC technologies. ISP is more efficient, yielding lower performance degradation of constituent P1901 technologies HP-AV and HD-PLC, while CX covers the case of coexistence with potential interference from other PLC devices at the cost of higher performance degradation.

5-6

could be made available for the P1901 Wavelet PHY/MAC option.39 There is interest from both constituent vendor communities in formally specifying metrics and methods for collecting them in follow-on and/or next-generation IEEE PLC standards addressing Smart Grid requirements.40 Exposing Metrics in IEEE 802.11 (Wireless LAN) IEEE 802.11 is a mature, proven Wireless LAN technology that most of us use every day, since it serves as the basis (PHY/MAC layers) for the extremely popular and widespread Wi-Fi services deployed in homes, offices, and public places throughout the world. Since it is already deployed in so many utility customer sites, and has become the de facto interface between personal computers, consumer electronics, smart phones, and so much else 802.11/Wi-Fi is an obvious and appealing HAN technology candidate. For our purposes, the most significant and useful aspect of 802.11 is the rich set of metrics and management methods that have been defined in amendments dot11k and dot11v. IEEE 802.11k IEEE 802.11k-2008 is an amendment that specifies Radio Resource Management for Wireless LANs. It defines data elements, interfaces, and wire formats enabling Stations (network endpoints, or devices) and Access Points to collect and report measurements of channel, link, and network characteristics, data which a network management platform can use to optimize network loading, transitions between Basic Service Sets (i.e. Access Points, or APs), and other operational aspects of an 802.11 WLAN. It is by far the most extensive set of PHY/MAC-level metrics available in an SDO-defined wireless LAN/HAN technology. It served as the reference and starting point for our development of the IEEE 802.15.4e HAN metrics described above.41 IEEE 802.11v IEEE 802.11v defines methods and mechanisms for configuring 802.11 Stations (end devices) in order to manage the overall health and performance of an 802.11 network. It is thus a companion to the 802.11k Radio Resource Management amendment. It has been in development for a number of years and is in its third re-circulation Sponsor Ballot. 802.11v is expected to be completed (ratified) and become an official 802.11 amendment in September, 2010.

39

EPRIs position as voice of the utility industry and its peer-to-peer participation in these standards is much appreciated by participants, who are typically competitors and may see proposals from a particular vendor as a potential threat. Thus, we were able to propose to some vendors the submission of a draft specification of HAN performance metrics in P1901 Wavelet PHY based on our engagement with the FFT-PHY technology. In recognition of EPRIs value and commitment, the P1901 Working Group elected EPRI to Invited Not for Profit Expert Entity status at its April, 2010 Working Group meeting (Osaka, Japan). 40 We have been following in particular the IEEE P1901.2 effort focused on Low Frequency (<500 kHz) Narrow Band Power Line Communications for Smart Grid Applications, sponsored by the IEEE Communications Society and established in March, 2010. A similar initiative that some P1901 members are pursuing is the ITU-T G.hnem project, launched in January, 2010. 41 One can consider the metrics defined in 802.15.4e as a small subset of 802.11k metrics.

5-7

Using IEEE 802.11 Metrics The IEEE 802.11k-2008 standard has met with industry acceptance; ICs that support it are available from multiple vendors. These chipsets are designed to be used in enterprise Access Points, where the additional complexity and cost are justified. They are deployed with a network management platform a software application running on remote computer that collects the 802.11k metrics and manage devices on the network to deliver more efficient use of infrastructure and a better user experience than what unmanaged networks can provide. While this capability may initially be vendor-specific and proprietary, standards-compliant platforms should become available with the ratification of IEEE 802.11v. The set of metrics defined in dot11k is extensive, to support fine-grained measurement of radio channel and modulation characteristics and enable sophisticated management of 802.11 networks. For utility HAN purposes, only a few of these metrics are needed to monitor basic network health and reliability. We identified these minimal elements and then looked for similar metrics in the specification of HP-AV/IEEE P1901. Since the two broadband technologies are similar e.g. they divide a region of radio spectrum into multiple channels, use variants of the OFDM modulation scheme, and support IEEE 802.2 Ethernet framing the metrics defined in P1901 include practical equivalents to the minimal set we identified in 802.11k. This common, minimal set of metrics derived from two IEEE standards served as our model for adding metrics in a third HAN standard (IEEE 802.15.4), and as our foundation for implementing a common utility HAN monitoring platform for use with both wired and wireless networks. Paths to Implementing 802.11 Metrics As explained, the 802.11k/v standards were designed for application in the enterprise environment, where the IT department actively manages the WLAN infrastructure. The deployment of retail energy services on customer HANs is quite different, based on a few simple use cases (as set forth in Section 2.3 above) and emphasizing network monitoring over management. This emerging, hybrid service model falls between the consumer electronics sealed box approach and the fully-managed enterprise case. Therefore, the implementation requirements for utility HAN will be quite different from mainstream, enterprise WLAN applications of 802.11k/v, for both chipsets and the network management platform. Broad Market Factors In our engagement with 802.11 chipset vendors and the Wi-Fi Alliance (WFA), weve learned that similar hybrid models are emerging in other market segments. For example, to deploy Voice over Internet Protocol (VoIP Vonage, Skype) service on Wi-Fi networks, metrics need to be supported on end devices and APs for proper network management and service delivery. The Wi-Fi Alliance captured this requirement, among others, in its Voice-Personal Program, and offers its members the opportunity to certify products against the formal VPP specification. The use case driving this development is offload of voice service from cellular networks; the opportunity for chipsets supporting these metrics is as large as the mid- to high-end mobile 42 phone market.

42

All high-end mobile handsets and Smart Phones coming to market now have Wi-Fi capability; the strong trend is for an increasing share of mid-range mobile phones to feature Wi-Fi as well.

5-8

The WFA Wireless Network Management Working Group is driving general support for almost all of the IEEE 802.11k specification; their recommendation includes all the 802.11 metrics listed in Tables 3-2 through 3-4 above. This bodes very well for the network monitoring and management capabilities that weve identified for Smart Grid HAN being implemented in all future 802.11 ICs. More than 500 million 802.11 chipsets shipped in calendar 2009; projections are that this number will top one billion by 2011. Annual shipments of the newest, most powerful 802.11n chipsets alone (excluding lower-cost, legacy 802.11b/g/a devices) are projected to exceed one billion in 2013, with total annual volume of all 802.11 devices considerably higher. It seems very likely that the metrics utilities require will be supported as part of these broad market developments. If not, the size of the worldwide Smart Grid segment might justify their inclusion in 802.11 chipsets designed specifically for HAN-based devices and applications. Path dependencies will. Micro-economic and technical factors Support for the 802.11k metrics requires some code and storage space that may increase the cost of chipsets or HAN devices. However, only the most cost-sensitive platforms will be challenged by this. If the ability to monitor HAN reliability is justified by total system and service cost delivery factors (i.e. lowering the cost of installation and customer support, facilitating pro-active network health, raising the uptake and effectiveness of retail energy services), an incremental increase in chipset cost and complexity needed to meet the requirements will be acceptable. If the metrics became widely supported in the broad market, as envisioned above, the desired functionality would become ubiquitous and the cost differential would shrink or vanish. It is challenging for vendors to decide when to start supporting a given feature. For instance, to date 802.11k metrics are supported on chipsets meant for enterprise platforms (APs and end-user devices, i.e. laptops). What if devices that dont support the metrics, for example older PCs or inexpensive handhelds, are used on the 802.11 network? Two factors mitigate concerns about deploying newer, metrics-enabled HAN devices over a customers legacy 802.11 infrastructure, and encourage the early introduction of this HAN monitoring feature: 1. 1. In 802.11, metrics and other network management information is exchanged point-to-point via management and action frames, which are separate (out of band) from user traffic. Deploying HAN metrics does not degrade network performance or otherwise affect network operation. 2. 2. It only takes one metrics-capable device to provide valuable HAN network health information. For example, if a customer installs a new Home Energy Management System (HEMS) that supports 802.11k, the utility could remotely determine whether packets are getting through to other devices, whether they are metrics-capable or not. Since metrics are collected at each interface, the HEMS can report statistics on packets sent to and received from other stations. A high percent of packets loss between the HEMS and the pool pump would be detectable, whether the pool pumps 802.11 chipset supported the 802.11k metrics or not.43

43

This scenario assumes the existence of certain HAN support services i.e. MAC-to-IP address mapping, device discovery and identification, and remote access control securing the HEMS metrics interface and service. See Section 6, below.

5-9

Our entire approach to designing and implementing performance and reliability metrics in three SDO-defined HAN standards, is meant to encourage vendors who want to differentiate their products by adding this capability with low effort and risk, and good prospects for technical and market success.

5-10

6
SYSTEM CONSIDERATIONS
In this section we will survey some system-level capabilities that we have not addressed, but that will need to be available, and preferably standardized (de jure or de facto), so that HANs will be able to support consumer-friendly retail energy services. Specifically, the performance/reliability measurement and monitoring functionality and standards described in this report will need to be integrated into and supported by this larger HAN framework in order to become practical features. This is not the place for a comprehensive treatment of HAN architecture. We will limit ourselves to describing the essential system-level features required to support the three HAN Performance Monitoring use cases driving our work. (See Section 2.3 above). HAN Security Each station (end device) that reports its performance metrics would need to be authenticated or otherwise identified as a valid HAN device. And although we believe metrics data are not very sensitive or valuable,44 if possible and practical they should be encrypted. There are provisions in all three of the HAN technologies under consideration for authorizing or authenticating end devices (also known as stations) and for encrypting payload data. IEEE 802.11 and IEEE P1901 (Homeplug AV) technologies have significantly stronger, more thoroughly and extensively proven, security provisions than IEEE 802.15.4/ZigBee. For example, in 802.11 and P1901, metrics are sent as management frames, which are out of band (not normal data payloads, e.g. separated from application software), whereas in IEEE 802.15.4 there is no standard provision for sending management data over the air. (In our prototype implementation we are sending the metrics as ordinary data frames.45) The IEEE 802.11w-2009 standard provides the highest level of security among the three HAN standards, with mechanisms for management frame confidentiality, integrity, authenticity, and replay protection. In practice, a metrics-enabled end device will establish trust with an external entity (a utilitysupplied system, i.e. an in-meter ESI, or in-home device such as HEMS) in the normal way, and the ESI or HEMS would collect metrics from end devices using MIB or PIB get operations. The EMI or HEMS would manage the granularity (frequency) of metrics collection, and sending them back to utility monitoring platforms. We have developed a prototype of such an end-to-end HAN reliability monitoring platform, as described in Section 7, below.

44

It seems the radio signal strength and frame re-try behavior that the metrics capture are not directly or obviously correlated with human presence or behavior, so we doubt that their collection or communication raise any material privacy concerns. If resources permit, we may be able to explore this in the context of the HAN Supplemental project. See Section 7, below. 45 As part of our research and standards activity, we have explored a few ways to fill this gap in 802.15.4 design and mooted an informal proposal to develop a new Ethertype, though 802.15.4 does not conform to IEEE 802.2 architecture. See Section 6.2.

6-1

The user interface and key management aspects of trust management are major challenges for utilities and vendors as they design and deploy HAN-based devices and retail energy services. However, the support of HAN monitoring use cases and implementation of performance metrics do not represent any significant additional complexity to the key management infrastructure, nor do they introduce additional security risk. HAN Internetworking There are a number of scenarios in which multiple networks will exist in the utility customers premises. In some cases, this will not be a concern the HAN will run independently and not be adversely affected by the environment. However, there will be environments in which multiple networks of the same or different types will be running simultaneously, often using the same physical medium (radio bands or home wiring). There are ways to separate HAN traffic if desired, with varying degrees of isolation, from creating logical subnets (address spaces), to establishing secure channels (VLANs), to deploying separate physical interfaces. There are costs and benefits to each approach; an architectural framework that provides abstraction from physical media, but unifies operations at a higher layer, can support bridging, routing, and other useful functionality. In many cases, the need to connect different kinds of networks dictates that interworking should be implemented at Layer 3 of the ISO stack. This requires a router, and more complex networking stacks (and consequently, the need for more processing power and memory) on end devices. If the networking requirements and desired functionality are well understood, much of what is needed for interworking can be implemented at Layer 2. Since this allows for a high level of integration (more functionality in communications chipsets), Layer 2 interworking generally has higher performance, is less expensive, and requires little to no manual configuration. As a result, almost all enterprise and public access networks datacenters, office and campus LANs, Wi-Fi access networks are implemented using Layer 2 interworking. The most successful architectural framework for LAN/MAN interworking at Layer 2 is the IEEE 802.1 family of standards. The unifying aspects of the 802 architecture include exclusively packet based communication, common bit and byte ordering, and a (mostly) common 48 bit addressing system. These common aspects allow different physical networks such as Ethernet and Wi-Fi to directly interoperate. Although the IEEE P1901 standards committee decided against establishing itself as an 802 Working Group, the resulting standard also adheres to these principles and supports the 802 architecture. Though it is also in the IEEE 802 family of network technologies, the 802.15.4 group decided not to adhere to or support the 802 architecture. Therefore, ZigBee, Z-wave, and other networks built on IEEE 802.15.4 (including 802.15.4g, the Smart Utility Network standard) cannot interoperate out of the box with mainstream LAN/MAN networks, nor take advantage of the rich set of security, service, and management standards that comprise IEEE 802. This nonconformance means that the effort to implement IEEE 802.15.4 metrics at a system level is much higher than the small, incremental effort needed to add system-level support for metrics for all the protocols in the 802 family specifically, IEEE 802.11 (Wi-Fi) and IEEE P1901 (Homeplug, HD-PLC).

6-2

HAN Service and/or Capability Discovery Ideally, energy management-enabled devices that were plugged into the HAN would benefit from some sort of Service Discovery and Auto-Configuration mechanism, perhaps similar to ZeroConf, Bonjour, etc. The ZigBee Alliance, in the SEP 2.0 IP Application Document, specifies a simple IP-based service discovery mechanism, and lists other options, including the IETF Service Location Protocol (SLP) and Multicast DNS (m-DNS) as candidates for this functionality.46 Other industry groups that have shown interest, or might be interested, in developing this kind of user-friendly functionality include the Digital Living Network Alliance (DLNA), the Universal Serial Bus Implementers Forum (USB-IF), the USNAP Alliance, the Universal Plug-n-Play (UPnP) Forum, and the Wi-Fi Alliance (WFA). A HAN performance monitoring service, as described in outline in the next section, could be included in any such HAN device and network auto-configuration architecture.

46

ZigBee Alliance (IP Stack Group Leslie Mulder), ZigBee IP Application, 095446r02ZB, November 12, 2009.

6-3

7
END-TO-END PERFORMANCE MONITORING AND EPRIS HAN SUPPLEMENTAL PROJECT
We have implemented a prototype HAN Monitoring platform that enables utilities to collect HAN communications reliability data the metrics described in this report from homes in which they are conducting trials. A HAN Communications Monitoring agent, which can be a PC application or a module embedded in a vendor platform (i.e. Home Energy Management System), collects metrics from any metrics-enabled Remote Device (thermostats, meters, pool pumps, appliances, Electric Vehicle Supply Equipment - charging stations, etc.) capable of participating in Demand Response, and sends them upon request via the Internet to a Utility Management System. The utility can monitor the signal strength, number of packets lost or subject to re-tries, etc. to every device in every home in the trial, no matter what kind of HAN technology (Homeplug, Wi-Fi, or ZigBee) is being used. The system is shown in Figure 7-1, below.

Figure 7-1 End-to-End HAN Monitoring System

The modular architecture of this solution, depicted in Figure 7-2 below, is available to any vendor to implement. By driving this solution in open standards, publishing this report, developing a complete system prototype, and collaborating with vendors on implementation, we hope to offer the industry an example of incremental, cost-effective, rapid feature development

7-1

that delivers essential functionality and lowers the risk of deploying novel Smart Grid technologies and services.

Figure 7-2 Architecture of Metric-Enabled HAN Devices

7-2

8
CONCLUSION
We found a significant gap between utility requirements and the use cases and system specification for HAN-based Demand Response and similar retail energy services. There was no way to verify that messages would reliably reach end devices on customer premise HANs. This represents a risk to utilities, who must rely on statistically dependable communications between their interface to the customer premise (i.e. the Energy Services Interface in a Smart Meter, or an Internet-based gateway or Home Energy Management System) and end devices. We identified the leading design and implementation of Local Area Network performance metrics and monitoring capability (IEEE 802.11k) and used it as our model for filling the gap in the three most popular and promising HAN technologies: IEEE 802.11/Wi-Fi, IEEE 802.15.4 /ZigBee, and IEEE P1901/Homeplug AV/GP. We found similar capabilities defined in the Homeplug AV, Green PHY, and IEEE P1901 specifications, and created practically equivalent metrics for IEEE 802.15.4. We engaged with the relevant IEEE standards working and task groups and industry alliances to socialize our approach and designs with the LAN/MAN communications industry leadership. We succeeded in getting our specification of IEEE 802.15.4 metrics accepted with mandatory status through first letter ballot, and have reason to be confident that it will gain final approval. We created this element of an IEEE communications standard in less than a years time. We formalized EPRIs status as Smart Grid experts within the Homeplug Alliance, the Wi-Fi Alliance, and the IEEE P1901 Working Group. We implemented prototype versions of the standard metrics on IEEE 802.15.4/ZigBee and IEEE P1901/Homeplug AV evaluation platforms. We also implemented a prototype end-to-end HAN Monitoring System for use in utility HAN trials. We supported one vendor implementation of the draft IEEE 802.15.4e metrics specification and are pursuing the formalization of Smart Grid profiles, including performance metrics, in the Wi-Fi and Homeplug Alliances. We authored a paper describing our work, which was accepted for presentation at the IEEE Smart Grid Workshop at the IEEE International Communications Conference, Cape Town, 47 South Africa, in May 2010.

47

Godfrey, Tim and Craig Rodine, Unified Metrics for Management of Smart Grid Home Area Networks, Proceedings of the First IEEE International Workshop on Smart Grid Communications, IEEE Xplore, Digital Object Identifier: 10.1109/ICCW.2010.5503925. Available at http://ieeexplore.ieee.org/Xplore/dynhome.jsp

8-1

Export Control Restrictions Access to and use of EPRI Intellectual Property is granted with the specific understanding and requirement that responsibility for ensuring full compliance with all applicable U.S. and foreign export laws and regulations is being undertaken by you and your company. This includes an obligation to ensure that any individual receiving access hereunder who is not a U.S. citizen or permanent U.S. resident is permitted access under applicable U.S. and foreign export laws and regulations. In the event you are uncertain whether you or your company may lawfully obtain access to this EPRI Intellectual Property, you acknowledge that it is your obligation to consult with your companys legal counsel to determine whether this access is lawful. Although EPRI may make available on a case-by-case basis an informal assessment of the applicable U.S. export classification for specific EPRI Intellectual Property, you and your company acknowledge that this assessment is solely for informational purposes and not for reliance purposes. You and your company acknowledge that it is still the obligation of you and your company to make your own assessment of the applicable U.S. export classification and ensure compliance accordingly. You and your company understand and acknowledge your obligations to make a prompt report to EPRI and the appropriate authorities regarding any access to or use of EPRI Intellectual Property hereunder that may be in violation of applicable U.S. or foreign export laws or regulations.

The Electric Power Research Institute Inc., (EPRI, www.epri.com) conducts research and development relating to the generation, delivery and use of electricity for the benefit of the public. An independent, nonprofit organization, EPRI brings together its scientists and engineers as well as experts from academia and industry to help address reliability, and challenges efficiency, in electricity, safety drive including and the health, to

environment. EPRI also provides technology, policy economic in analyses emerging long-range EPRIs research and development planning, and supports research technologies. members represent more than 90 percent of the electricity generated and delivered in the United States, and international participation extends to 40 countries. EPRIs principal offices and laboratories are located in Palo Alto, Calif.; Charlotte, N.C.; Knoxville, Tenn.; and Lenox, Mass. TogetherShaping the Future of Electricity

2010 Electric Power Research Institute (EPRI), Inc. All rights reserved. Electric Power Research Institute, EPRI, and TOGETHERSHAPING THE FUTURE OF ELECTRICITY are registered service marks of the Electric Power Research Institute, Inc. 1018982

Electric Power Research Institute 3420 Hillview Avenue, Palo Alto, California 94304-1338 PO Box 10412, Palo Alto, California 94303-0813 USA 800.313.3774 650.855.2121 askepri@epri.com www.epri.com