IBP1509_12 OPERATION AND MAINTENANCE OF THE TELECOMMUNICATIONS TECHNOLOGY PLATFORMS NG LNG PIPELINE SYSTEM Marcial A. Milla1, Neptali S.

Mayorga 2

Copyright 2012, Brazilian Petroleum, Gas and Biofuels Institute - IBP

This Technical Paper was prepared for presentation at the Rio Oi & Gas Expo and Conference 2012, held between September, 1720, 2012, in Rio de Janeiro. This Technical Paper was selected for presentation by the Technical Committee of the event according to the information contained in the final paper submitted by the author(s). The organizers are not supposed to translate or correct the submitted papers. The material as it is presented, does not necessarily represent Brazilian Petroleum, Gas and Biofuels Institute opinion, or that of its Members or Representatives. Authors consent to the publication of this Technical Paper in the Rio Oil & Gas Expo and Conference 2012 Proceedings.

Abstract
Telecommunications are vital for controlling and monitoring traffic on elements of SCADA networks (Supervisory Control and Data Acquisition) and the Corporation services in a Pipeline System. Therefore, having a complete and comprehensive management of what is happening on each links and Data Network Equipment it is important. This paper shows a successful case of Telecommunication Management Systems; the processes include the maintenance, operation and management of technology platforms. And also the importance use software for monitoring to achieve this goals. The paper show an actually experience in Gas Transportation Company in Peru, which has a buried pipeline for more than 730 km long and with a fiber optic cable (buried alongside the pipeline). The platform implemented over this cable is supported by 42 equipment SDH transmission technologies (Synchronous Digital Hierarchy). The characteristics of this platform give greater flexibility and high capacity. Therefore allows us to implement corporate and SCADA networks on the same platform. It covers the complete process of operation and maintenance of telecommunication systems. There will describe the different types of Network implemented such as: SCADA, Corporate, Maintenance, Telephony, etc. Additionally, it comments about the structure and staffing resources to meet all maintenance bases. It includes management indicators KPIs. Additionally, it covers management tools depending on the technology platforms of each telecommunications systems to monitor and followed objectives by each applications. This section will discuss the use of standard network management Protocol (SNMP). In conclusion, Telecommunications plays an important role in Gas & Oil Companies; it assures issues of communications in cases security, safety and budget, and also in cases of emergency and business continuity.

1. Introduction
Operating Gas Company of the Amazon S.A.C. (COGA), operates Camisea Pipeline Transportation System (Natural Gas NG and Liquefied Natural Gas LNG) owned by Transportadora de Gas del Peru SA (TGP). The Pipeline Transportation System (PTS) of Camisea has two pipelines, one that transports natural gas (NG) and the other Liquid Natural Gas (LNG). The PTS is initiated in the Amazon basin of Falkland, in the department of Cuzco, crosses the Andes and reaches the Pacific coast. The geographical, geological and climatic territories by crossing the Camisea PTS differ in complexity as others countries. The PTS has a Telecommunications System implemented on a Buried Fiber Optic Cable laid parallel to the pipeline (730 km) and a VHF Radio Communication System (22 Repeater Stations). The PTS has more than 42 Operating Facilities (OF), are controlled centrally and remotely by Lurin Control Room (CRL). In each Operating Facilities (Blocked valves, Pump Stations and Reduction Stations) there are fibers optic nodes, using SDH technology, similar to any Telecommunications Service Providers. Each of this Network Equipment (NE) has ADM characteristics

______________________________ 1 Electronics Engineer, Telecommunications Chief - Cia. Operadora de Gas del Amazonas S.A.C. 2 Electronics Engineer, IT Manager - Cia. Operadora de Gas del Amazonas S.A.C.

Rio Oil & Gas Expo and Conference 2012 (Add Drop and Multiplexer Circuits) to make a Network Access Point (NAP); through this point could enable data circuits to any point within the PTS. The distances of the Operational Facilities to each other are between 8 and 30 km depending of geographical location and design features of PTS. As the distribution of Team works, spare parts and contractor personnel are part of an operation and maintenance strategy aimed at achieving the highest levels of availability and reliability of this Telecommunications System. TgP and COGA are certified in ISO 9001, ISO 14001 and ISO 18001. Therefore, the processes of operation and maintenance includes improvements on identifying and management variables, optimization of routes in maintenance activities, resource allocation percentages by type of work, careful of waste management and data management.

2. Operation and Management of telecommunication systems

TgP has a wide technology infrastructure for their communications networks (SCADA and Corporate); similar to any Telecommunications Service Providers that offers data transport services. It made to include a group of professionals into COGAs Organizational Chart; this people are in charge for the Operation and Maintenance of this Telecommunications infrastructure (O&M). For this purpose COGA created a Telecommunications Department, and it reports directly to Information Technology Management (IT); Telecommunications Department has an organization structure like as other areas of Pipeline Maintenance. These maintenance crews are distributed according to geographical areas (Coast, High Sierra and Jungle). As shown in Figure 1.

Figure 1. IT Management Chart

The option taken to meet this need was to embrace a maintenance contract by a third party (ISI Solutions) with the structure shown in Figure 2. This structure is composed by a Chief of service, Networking engineer, Control Engineer, SCADA engineer, field coordinator and 7 field crews. With these people are performed visits to the facility Operational and execute field maintenance activities. Each field crew is comprised of two technical experts: one of these in optical fiber and the other one in VHF radio systems. It is important to consider each team has its truck and measurement equipment to execute appropriates maintenance activities.

Figure 2. Service Chart of the Field Support Contract by third 2

Rio Oil & Gas Expo and Conference 2012

Additionally, there is a team for especially remote operation and management of those different networks and technology platform; they are called Network Operation Center (NOC). By giving this carrier approach and its customers are other internal areas increases the options to maximize use of available resources, increasing communication options for new projects, modifications or expansions of STD. We identified 6 types of communication networks implemented on SDH TgPs Network, classified by the role it plays within the STD: (i) SCADA network (Include the communication of interconnected switches and routers that provide access to automation and control equipments within the Operational Facilities); (ii) Corporate Network (Include the communication of interconnected switches and routers that provide access to the equipments provided by corporate services within the Facilities Operating and Maintenance Bases), (iii) Maintenance Network (Include the communication of interconnected switches and routers that provide access to the different tools that bring support functions within the operating facilities and camps for monitoring and prevention of the ROW), (iv) Network Expansion Projects (Include the communication of interconnected switches and routers that provide access to Equipments providing corporate services to the camps and offices Expansion projects on STD), (v) Telephony Network (Include the communication of interconnected PBXs providing telephone communication between all Base Maintenance), and (vi) Networks and Support Agreements social (Include the communication of interconnected equipments that provide data transport capabilities that are part of conventions and social support programs, which is distributed from the Operational Facilities). In view of the complexity and scope that shows such a system of Telecommunications, concentrated and implemented management platforms centralized remote to have an optimal and flexible network operation, giving an efficient support to the expectations and new trends in the business of gas transport.

2.1. Systems and protocols for Network Management All the platforms and systems of remote management is controlled and administered by the NOC. It generated a needed to use management systems with a handle management protocols and standards that are supported by different brands and/or telecommunication products. It was done by the Simple Network Management Protocol (SNMP). SNMP is a set of network management protocols and functions that communicate using the Internet Protocol (IP) stack. SNMP allows network managers to isolate and troubleshoot faults on multi-vendor networks, configure devices on a network, and monitor network performance and status. SNMP is an implementation of a client/server relationship. The client application, called the network manager, makes virtual connections to an application program, called the SNMP agent, running on a remote network device. The database controlled by the SNMP agent is referred to as the MIB (Management Information Base), and is a standard set of statistical and control values. SNMP also allows the extension of these standard values with values specific to a particular agent or user requirement through the use of custom MIBs. Similar to the architecture of SCADA Systems, the SNMP architecture platform consists of a collection of network management stations and network elements. Network management stations execute management applications, which control and monitoring the network elements. Network elements are devices such as switches, routers, PBX, gateways and terminal servers that have management agents responsible for performing the network management functions required by the management stations. SNMP is used to communicate information between network management stations and the agents in the network elements. The Management Information Base contains data available to a network management program. They are created by management agents so that each machine with an agent will have an associated MIB. The network manager will query them and may even have a management MIB of its own. The management MIB contains general information and the individual MIB contains machine-specific information. Therefore, comparing with queries Modbus over SCADA networks, rather than consulting for a particular value in any Modbus address, you make a query for a value in particular OID into MIB Database from any telecommunication equipment. TGP implement its network management system (NMS) controlled from the NOC located in the Main Control Center (MCC). And there is a powerful tool that allows you to monitor the status and alarms of the equipments on the SDH Network. NOC can configure and assign data circuits for expansions of data networks from this application. It also has another application on a web platform where different monitors and manages the SNMP Agents (Network Equipment multivendors) of the different networks mentioned above.

Rio Oil & Gas Expo and Conference 2012

Figure 3. Application screens for managing the Network Equipment (SNMP Agents) The implementation of various network maps allows us to manage the wide range of handled TgP networks. Which allows us to quickly diagnose problems by using colors that represent the state of the equipment and links. This allows the scheduling of field work by event management tool. This Event Management tool not only useful in managing and allocating support staff activities in the field, if also as knowledge base for issues on COGA (be seen in 2.1 during the operation). TgP has also another tool that enables remote monitoring in real time to analyze what is happening and going through each of the network interfaces that comprise the System. This apliacin also uses the SNMP protocol as well as the other is the NetFlow protocol. This standard allows us to know the type of traffic that is causing congestion in parts of the network, indicating the Top Ten Talkers. This application is also used to obtain information on the rates and types of protocols that traverse the network, thus allowing to propose improvements to the Quality of Service (QoS) and prioritizing the most important traffic depending on the type of network that is being evaluated.

Figure 4. NetFlow application screens for managing Quality of Service (QoS) Another important tool, used from the NOC, which monitors is the Fiber Optic Cable by systems manufacturer JDSU. This system consists of connecting equipment OTDR, called OTU's in strategic positions to cover the full length of optical fiber cable. Those equipments are connected to the SCADA network and bring the information to a main server. then it sends measurement requirements in real time and then make the comparison with the Traces database. Where the application finds a significant variation on the comparison proceeds, it sends alarms to the NOC operators and field supervisors about a possible failure or strong attenuation of the FO cable. All tools have seen before with the option of sending alarm messages via SMS or EMAIL to pre-configured mailboxes.

Rio Oil & Gas Expo and Conference 2012

Figure 5. OFM and OMNS JDSU application screens for Fiber Optic Cable Management With all of these tools are able to operate efficiently Telecommunications network, but always supported on a strong knowledge support structure and geographically distributed field maintenance bases. Achieving with this cover 100% of the PTS. 2.2. Operation of Telecommunication Systems Using all the technology tools mentioned in the previous section we get the NOC can be fully aware of what happens in all networks. However, as mentioned above, you need an IT management tool for control, monitoring and write events, and then these events have to be assign in activities to field support. In this case, TgP has a web application where they generate "tickets" for each incidents, events and/or problems. The name of this application is Web Help Desk (WHD). This application also serves as a TI Base of Knowledge. This Application tool not only handles telecommunications events but it also handles the events and malfunctions of all systems and applications that are under IT Management. The application can handle the events and failures as individual events and / or consequential damages, which are defined and assigned to field staff for their care or execute them.

Figure 6. WHD application screens for managing maintenance of the tickets The implementation of field activities are documented and properly structured reports containing descriptive photos of the work performed in the field. These reports are stored in web sites created by Microsoft SharePoint. This application allows its use for queries easily and securely from anywhere in the network of TGP.

3. Maintenance of Telecommunication Systems

On the initial definition Maintenance is effective action to improve relevant operational aspects of a facility such as functionality, safety, productivity, corporate image, health and hygiene. Maintenance should be both regular and ongoing, preventive and corrective. Maintenance refers to works that are needed to make in order to provide a quality service stipulated. Thus separate it into two main types of maintenance: (i) Preventive Maintenance (This allows us to detect recurring failures, reducing deadlocks, increase equipment life, reduce costs of repairs and detect weaknesses in the system. (ii) Corrective Maintenance (Action ad hoc basis resulting from the operation, depletion of life or other 5

Rio Oil & Gas Expo and Conference 2012 external factors, components, parts materials and generally of what constitutes the physical infrastructure, allowing their recovery, restoration or renovation, without adding value to the property) that is divided into 2 types: Contingent Correction (Refers to activities that take place immediately, because it provides a vital service team has quit, for whatever reason, and we must act as emerging at best cases under a contingency plan.) and Planned Correction (refers to activities that take place on computers or machines that are providing a service and this trivial, though necessary, is not essential to give a good quality of service , so it is best to schedule your care, economic issues, in this way can be reconciled if these programs work with the maintenance or preservation). These activities are executed under an outsourcing contract with a major company in the middle. This participates in field support personnel with a level of technical knowledge half since the support and know-how is at the NOC to provide remote support for preventive and corrective activities. These activities impact the IT budget positively.

Figure 7. Microsoft SharePoint screens which shows Annual Preventive Plan and repository maintenance reports

3.1. Preventive Maintenance Key Performance Indicator (PM_KPI) Within the structures of management and control of maintenance activities we use as a baseline based on experience and best practice the following parameters: Compliance with maintenance schedule Indicator (ICMR): This indicator serves to monitor progress of work on a monthly. It gives us the amount of progress which is calculated according to Equation 1. Goal for this indicator is 100%. Trip Odometer Indicator (IKMR): This indicator serves to monitor distances covered to perform scheduled tasks. It gives us the amount of mileage which is calculated according to Equation 2. Where n is equals the number of trucks used for maintenance on this month. Worked Effective Hours Indicator (ICHET): The objective of this indicator is to maintain statistics for hours worked during the month for each working field group and appreciate the effectiveness of activities executed vs. activities planned in order to be optimizing maintenance. It gives us the amount of effectives worked hours which is calculated according to Equation 3. Where n is equals the number of Field Groups works for maintenance on this month. Quantity of Reports Executed Indicator (IREB): This indicator shows the number of completed maintenance report regarding the planned activities. It is calculated according to Equation 4.

ICMR

# OperatingFacilitiesV isited % # OperatingFacilities Pr ogrammed

(1)

IKMR Odometeri
n

(2)

Rio Oil & Gas Expo and Conference 2012

ICHET WorkedHoursi
n

(3)

IREB

# CompletedM a int enance Re ports % # PlannedAct ivities

(4)

3.1. Corrective Maintenance Key Performance Indicator (CM_KPI) Within the structures of management and control of maintenance activities we use as a baseline based on experience and best practice the following parameters: Average Repair Time Indicator (ITPR): This indicator shows the number of days that it takes to repair a breakdown on average by type of service. It gives us the amount of days to repair which is calculated according to Equation 5. Where n is equals the number of faults on this month by type of service.

ITPR

Re pairTime
n

# Faults

days

(5)

4. Continuous Improvement and Best Practices

Within the management system established in the COGA and be guided under the international standards ISO 9001, 18001 and 14001. The program of continuous improvement operates in supervision at the forefront and has got results in staff morale, since employees now have the assurance that they are not blamed for mistakes that are not under his government. Since established control system are focused on identifying the problem not the individual. We established the following indicators of continuous improvement and best practices: Excess of Velocity indicator (IEV): The purpose of this indicator is to keep track of traffic congestion rates of drivers assigned to the project and validate their behavior so it monthly (As shown an example in figure 8) take necessary actions when the behavior has abrupt changes. The Camisea project has route diversity where 70% of them are considered critical paths for the geographical area in which they find themselves.

Figure 8. Sample table of Excess of Velocity Indicator (IEV) Within the Integrated System Management (SGI) is carried out the registration and control of measuring equipment registered so it is controlled by traffic lights that allow us to keep better control of the periodicity of maintenance and calibration of such equipment as recommended the manufacturer.

5. References DOUGLAS MAURO, KEVIN SHMIDTH , Essential SNMP Eng, p. 6 - 17, 2001.