Journal of Pediatric Surgery (2011) 46, 115121

www.elsevier.com/locate/jpedsurg

Pediatric American College of Surgeons National Surgical Quality Improvement Program: feasibility of a novel, prospective assessment of surgical outcomes
Mehul V. Raval a,b,, Peter W. Dillon c , Jennifer L. Bruny d , Clifford Y. Ko a,e , Bruce L. Hall a,f , R. Lawrence Moss g , Keith T. Oldham h , Karen E. Richards a , Charles D. Vinocur i , Moritz M. Ziegler d on behalf of the ACS NSQIP Pediatric Steering Committee 1
a

Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL, USA Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, IL, USA c Division of Pediatric Surgery, Department of Surgery, Pennsylvania State University School of Medicine, Hershey, PA, USA d Department of Pediatric Surgery, University of Colorado, The Children's Hospital, Denver, CO, USA e Department of Surgery, University of California Los Angeles David Geffen School of Medicine and the VA Greater Los Angeles Healthcare System, Los Angeles, CA, USA f Department of Surgery, John Cochran Veterans Affairs Medical Center, and Barnes Jewish Hospital, Center for Health Policy and the Olin Business School, Washington University in St Louis, St Louis, MO, USA g Department of Surgery, Yale University, Yale New Haven Children's Hospital, New Haven, CT, USA h Department of Surgery, Medical College of Wisconsin, Milwaukee, WI, USA i Division of Pediatric Surgery, Thomas Jefferson University, A.I. DuPont Hospital for Children, Wilmington, DE, USA
b

Received 20 September 2010; accepted 30 September 2010

Key words:
ACS NSQIP; Pediatric surgery; Outcomes; Quality assessment

Abstract Purpose: The American College of Surgeons (ACS) National Surgical Quality Improvement Program (NSQIP) provides validated assessment of surgical outcomes. This study reports initiation of an ACS NSQIP Pediatric at 4 children's hospitals. Methods: From October 2008 to June 2009, 121 data variables were prospectively collected for 3315 patients, including 30-day outcomes and tailoring the ACS NSQIP methodology to children's surgical specialties.

Corresponding author. American College of Surgeons, Chicago, IL 60611-3211, USA. Tel.: +1 312 202 5489; fax: +1 312 202 5062. E-mail address: m-raval@md.northwestern.edu (M.V. Raval). 1 ACS NSQIP Pediatric Steering Committee: American College of Surgeons, Chicago, Ill (Mira Shiloach, MS; Charles Fischer, BS); The Children's Hospital, Aurora, Colo (Debra Liebrecht, RN); Children's Hospital of Wisconsin, Milwaukee, Wis (Yvonne Anderson, RN, MS); Yale New Haven Children's Hospital, New Haven, Conn (Marilyn Hirsch, RN; Debbie Ferrigno, RN); A.I. DuPont Hospital for Children, Wilmington, Del (Michael Marchildon, MD; Christine Schuck, RN); Colorado Health Outcomes Program, Denver, Colo (Karl Hammermeister, MD; William Henderson, PhD; Elaine Morrato, DrPH, MPH); QC Metrix (Kathy Rowell, MS; Janet Steeger, RN). 0022-3468/$ see front matter 2011 Elsevier Inc. All rights reserved. doi:10.1016/j.jpedsurg.2010.09.073

116

M.V. Raval et al.

Results: Three hundred seven postoperative complications/occurrences were detected in 231 patients representing 7.0% of the study population. Of the patients with complications, 175 (75.7%) had 1, 39 (16.9%) had 2, and 17 (7.4%) had 3 or more complications. There were 13 deaths (0.39%) and 14 intraoperative occurrences (0.42%) detected. The most common complications were infection, 105 (34%) (SSI, 54; sepsis, 31; pneumonia, 13; urinary tract infection, 7); airway/respiratory events, 27 (9%); wound disruption, 18 (6%); neurologic events, 8 (3%) (nerve injury, 4; stroke/vascular event, 2; hemorrhage, 2); deep vein thrombosis, 3 (b1%); renal failure, 3 (b1%); and cardiac events, 3 (b1%). Current sampling captures 17.5% of cases across institutions with unadjusted complication rates ranging from 6.8% to 10.2%. Completeness of data collection for all variables exceeded 95% with 98% interrater reliability and 87% of patients having full 30-day follow-up. Conclusion: These data represent the first multiinstitutional prospective assessment of specialtyspecific surgical outcomes in children. The ACS NSQIP Pediatric is poised for institutional expansion and future development of risk-adjusted models. 2011 Elsevier Inc. All rights reserved.

The National Surgical Quality Improvement Program (NSQIP) was developed in the 1990s in Veterans Health Administration hospitals and led to marked improvement in surgical quality [1]. Based on decreases in morbidity, mortality, and improved patient satisfaction, the program was expanded to the private sector in the Patient Safety in Surgery Study with support from the Agency for Healthcare Research and Quality [2]. Subsequently, the American College of Surgeons (ACS) adopted NSQIP and facilitated its expansion to more than 250 private sector institutions. Hospitals participating in the ACS NSQIP have demonstrated improved surgical outcomes and decreased health care expenditure for complications [3,4]. The Veterans Health Administration NSQIP, the Patient Safety in Surgery Study, and the ACS NSQIP are limited to adult patient populations. There has been a long-standing interest in developing a similar platform to examine the quality of surgical care being delivered to children in the United States. Since 2005, the ACS has partnered with the American Pediatric Surgical Association to create a pediatric ACS NSQIP module [5,6]. In October of 2008, phase 1 of the ACS NSQIP Pediatric was launched at 4 tertiary children's hospitals. The objective of ACS NSQIP Pediatric phase 1 was to demonstrate feasibility of the program in terms of adoption of clinically meaningful data points, implementation of a technical platform for data collection, completion of data collection including 30-day outcomes, validation of accurate data abstraction, and detection of clinically meaningful outcomes of interest in a multispecialty program. The purpose of this article is to document successful implementation of the ACS NSQIP Pediatric program and to report an interim analysis of data collected during phase 1.

1. Methods
The specific variables and outcomes collected in the ACS NSQIP Pediatric have been previously described [5]. In brief, approximately 121 variables were prospectively

collected: 6 demographic, 8 surgical profile, 46 clinical preoperative, 13 laboratory, 16 intraoperative, and 32 postoperative. In addition, variables of interest were collected for special populations such as neonates undergoing surgical procedures. Thirty-day outcomes of interest ranged from mortality to specific end points such as renal failure and surgical site infection. Data were abstracted at sites based on an 8-day cycle that allows for case selection with equal representation of each day of the week. Each year is, therefore, divided into 46 collection cycles, and cases must be submitted for 42 cycles to be compliant with program participation while accommodating data reviewer vacations and sick leave. Cases were identified for selection based on Current Procedural Terminology codes included in the program. The first 35 to 40 consecutive cases meeting inclusion criteria are captured. This methodology mirrored the systematic sampling methodology currently used in the ACS NSQIP and results in approximately 1600 cases submitted annually with a single full-time data collector. Several high-volume, low-risk procedures were limited during the sampling. For example, procedures such as pyloromyotomy and tonsillectomy are limited to 5 cases or fewer in a specific 8-day cycle. In addition, all trauma and transplant cases were excluded. Because the goal of the project was to examine surgical outcomes in children, ACS NSQIP Pediatric was designed as a multispecialty program to include the pediatric subspecialties of orthopedic surgery, urology, neurosurgery, otolaryngology, plastic surgery, gynecology, as well as general and thoracic pediatric surgery. The ACS NSQIP focuses on clinical data abstraction by trained data collectors who received extensive training before data collection [7]. This phase 1 study was planned as a 12-month pilot study for data collection at 4 sites: Yale New Haven Children's Hospital (New Haven, Conn), A.I. DuPont Hospital for Children (Wilmington, Del), Children's Hospital Colorado (Aurora, Colo), and Children's Hospital of Wisconsin (Milwaukee, Wis). Surgeons, data reviewers, and administrative staff involved in the phase 1 pilot were actively involved in the clarification and refinement of data

Pediatric ACS NSQIP variables and outcomes of interest. During the course of this study, the ACS NSQIP also performed an extensive audit of data accuracy and completeness [8]. Typically, the ACS NSQIP requires greater than 95% interrater reliability (or b5% disagreement rate) to approve data submission from participating hospitals. During an audit, 10 randomly selected medical records were expertly reviewed at each site and results were compared with the site's case abstractions. The overall number of cases performed during the period of interest at each institution was determined and the proportion of cases captured within the ACS NSQIP Pediatric was calculated. For the purpose of a feasibility analysis, outcomes were demonstrated in terms of complications detected. Descriptive results are provided with the number of complications or occurrences detected. The outcomes were grouped and studied in categories including mortality, intraoperative occurrences, and postoperative occurrences. Postoperative occurrences were categorized as infectious, airway related, wound disruptions, neurologic events, deep vein thrombosis, renal failure, and cardiac events. Infection occurrences included superficial, deep, or organ-space surgical site infections; sepsis or septic shock; pneumonia; and urinary tract infections. Neurologic events included nerve injury, stroke, cerebral-vascular accidents, and coma. Variability in outcomes was demonstrated across the 4 participating centers, but no attempt at risk adjustment was made in this study because of the limited sample sizes. Specialty-specific outcomes including volumes and occurrences captured within the ACS NSQIP Pediatric were tabulated. The ACS NSQIP Pediatric was conducted at each site after either full institutional review board (IRB) review or under the auspices of a quality improvement program exempt from formal IRB review. Representatives from all 4 centers participating in phase 1 have approved site de-identified data shared in this report. This analysis was retrospectively performed using de-identified data and was deemed not to qualify as human subjects' research after review of protocols by the Northwestern University IRB. Statistical analyses were performed using either SPSS 18 (SPSS Inc, Chicago, Ill) or Microsoft Excel 2003 (Microsoft Corp, Seattle, Wash).

117

2. Results
From October 2008 to June 2009, data variables were prospectively collected for 3315 patients. These patients represented cases from 183 surgeons, 685 cities, and 31 states. The ACS NSQIP Pediatric sampled 16.8% of eligible procedures across the 4 institutions (volume weighted), while the average sampling rate across the 4 was 17.5% (unweighted) with samples ranging from 502 cases to 1056 cases. Variation in sampling volumes was primarily because of variation in program start dates and vacation cycles by data abstractors. All phase 1 sites underwent interrater reliability auditing and were abstracting program data below the 5% threshold at 1.9%, with disagreement rates ranging from 0.75% to 3.19%. Completeness of data collection for all variables exceeded 95% with 87% of patients having full 30-day follow-up. The number of cases represented by specialty as well as associated complication rates by specialty are shown in Table 1. Pediatric neurosurgery represented a small proportion of cases (11.1%) but a high proportion of complications (29.4%). Conversely, pediatric otolaryngology represented a large proportion of cases (22.7%) but a relatively small proportion of complications (5.2%). Lists of the top 5 most frequent procedures in each specialty are shown in Table 2. A total of 307 postoperative complications/occurrences were detected in 231 patients representing 7.0% of the study population. There were 13 deaths (0.39%) and 14 intraoperative occurrences (0.45%) detected. Intraoperative occurrences included respiratory occurrences such as requirement for blood transfusion [3], accidental injury to an organ [3], cardiac arrest [1], nerve injury [1], or respiratory occurrences including organ injury bronchospasm [1], laryngospasm [1], hypoxia [3], or pneumothorax [1]. Of the patients with postoperative complications, 175 (75.7%) had a single complication, 39 (16.9%) had 2 complications, and 17 (7.4%) had 3 or more complications. Table 3 shows the various complications detected with rigorous 30-day followup. Most of the complications detected were infectious outcomes followed by airway/respiratory issues. Unadjusted complication rates ranged from 6.8% to 10.2% at the 4 sites.

Table 1 Specialty Pediatric Pediatric Pediatric Pediatric Pediatric Pediatric Others Total

Specialty specific representation and complication by specialty in the ACS NSQIP Pediatric No. of cases (%) general surgery otolaryngology orthopedics urology neurosurgery plastics 1009 (30.4) 754 (22.7) 599 (18.1) 391 (11.8) 368 (11.1) 183 (5.5) 12 (b1) 3315 (100) No. of cases with complications (%) 79 (7.8) 12(1.6) 46 (7.7) 8 (2.0) 68 (18.5) 17 (9.3) 1 (14.2) 231 (7.0) % of complications 34.2 5.2 19.9 3.5 29.4 7.4 0.4 100

118
Table 2 CPT code Top procedures by specialty contributed to the ACS NSQIP Pediatric Procedure description Frequency 998 139 62 46 39 33 599 69 46 22 22 19 520 183 89 27 24 15 338 82 19 16 13 13 229 23 18 13 12 12 183 21 9 8 7 7

M.V. Raval et al.

Percentage

Cumulative percentage 13.9 20.1 24.7 28.7 32.0 11.5 19.2 22.9 26.5 29.7 35.2 52.3 57.5 62.1 65.0

Pediatric general surgery 44970 Laparoscopic appendectomy 49500 Repair initial inguinal hernia, age 6 mo to younger than 5 y, reducible 49580 Repair umbilical hernia, younger than 5 y, reducible 49505 Repair initial inguinal hernia, age 5 y or older, reducible 43520 Pyloromyotomy Pediatric orthopedic surgery 20680 Removal of deep implant 22804 Spinal arthrodesis, 13 or more vertebral segments 22802 Spinal arthrodesis, 7-12 vertebral segments 27165 Osteotomy, intertrochanteric or subtrochanteric including internal or external fixation and/or cast 27395 Lengthening of hamstring tendon, multiple tendons, bilateral Pediatric otolaryngology 42820 Tonsillectomy and adenoidectomy, younger than age 12 42830 Adenoidectomy, primary, younger than age 12 69631 Tympanoplasty without mastoidectomy 31541 Laryngoscopy, direct, operative, with excision of tumor and/or stripping of vocal cords or epiglottis 21501 Incision and drainage, deep abscess or hematoma, soft tissues of neck or thorax Pediatric urology 54640 Orchiopexy, inguinal approach, with or without hernia repair 54326 One-stage distal hypospadias repair with urethroplasty 54324 One-stage distal hypospadias repair, with urethroplasty by local skin flaps 49500 Repair initial inguinal hernia, age 6 mo to younger than 5 y, reducible 53450 Urethromeatoplasty, with mucosal advancement Pediatric neurosurgery 62223 Creation of shunt, ventriculoperitoneal, ventriculopleural, other terminus 62230 Replacement or revision of cerebrospinal fluid shunt, obstructed valve, or distal catheter in shunt system 61343 Craniectomy, suboccipital with cervical laminectomy for decompression of medulla and spinal cord, with or without dural graft 61210 Burr hole(s), for implanting ventricular catheter, reservoir, EEG electrode(s), pressure recording device, or other cerebral monitor 62225 Replacement or irrigation, ventricular catheter Plastic Surgery 42200 Palatoplasty for cleft palate, soft and/or hard palate only 14300 Adjacent tissue transfer or rearrangement, N30 cm2, unusual or complicated, any area 42210 Palatoplasty for cleft palate, with closure of alveolar ridge, with bone graft to alveolar ridge (includes obtaining graft) 42225 Palatoplasty for cleft palate, attachment pharyngeal flap 42215 Palatoplasty for cleft palate, major revision
CPT indicates Current Procedural Terminology; EEG, electroencephalogram.

13.9 6.2 4.6 3.9 3.3 11.5 7.7 3.7 3.7 3.2 35.2 17.1 5.2 4.6 2.9

24.3 5.6 4.7 3.8 3.8 10.0 7.9 5.7 5.2 5.2 11.5 4.9 4.4 3.8 3.8

24.3 29.9 34.6 38.5 42.3 10.0 17.9 23.6 28.8 34.1 11.5 16.4 20.8 24.6 28.4

3. Discussion
This interim report demonstrates the successful implementation of the ACS NSQIP Pediatric phase 1 at 4 tertiary pediatric centers. Feasibility of the program implementation is shown through adoption of the standard ACS NSIQP systematic sampling of cases, collection of clinically relevant variables and outcomes as outlined before program initiation,

and creation of a data abstraction technology platform. The data collection process has been validated through rigorous auditing of each participating site. Each site surpassed 95% interrater reliability thresholds, and individual variables studied had more than 95% completion and greater than 87% successful 30-day follow-up. Uniform data collection with well-defined variables is a key component of any clinical registry. In addition, several

Pediatric ACS NSQIP

Table 3 Complications reported to the ACS NSQIP Pediatric Frequency 105 30 9 15 13 7 31 27 18 8 4 2 2 3 3 3 3 170 Percentage 0.62 0.18 0.05 0.09 0.08 0.04 0.18 0.16 0.11 0.05 0.02 0.01 0.01 0.02 0.02 0.02 0.02 1.00

119 end point of the ACS NSQIP Pediatric phase 1 [5]. Despite the limitations of the current design in terms of generic surgical outcomes and the decreased statistical power, the cases and complications captured by the ACS NSQIP Pediatric provide several valuable insights. Approximately a third of cases and complications were considered general pediatric surgery. Several specialties such as pediatric otolaryngology and pediatric urology represented a relatively high proportion of cases but a paucity of complications. Identifying the specific procedures of interest within these specialties may provide a more meaningful utilization of resources within the program, and therefore, the program must reach out to specialty surgical societies for direction. Using data provided by the ACS NSQIP Pediatric will provide valuable guidance for future development of variables and outcomes of interest. This process resembles the evolutionary advancements in the adult ACS NSQIP where specialty-specific modules for adult populations are already under development [10]. The vast majority of outcomes of interest collected in phase 1, which ended in December of 2009, were infectious and respiratory, which are generic surgical outcomes that apply to a broad range of surgical procedures. Nonetheless, more specific outcomes are needed, which address increasingly focused procedures and populations. Development of targeted, procedure-specific outcomes has been started and is a key component of the coming phases of the ACS NSQIP Pediatric. The phase 2 of the ACS NSQIP Pediatric began in January 2010 with expansion of the program from 4 sites to 24 additional sites. It is anticipated that the phase 2 will contribute approximately 40,000 cases and the development of several key modifications have been identified. First, data definitions will be further refined. The ACS NSQIP staff and definitions committee participants attempt to bring variable and outcomes definitions in line with consensus standards and published definitions provided by organizations such as the Centers for Disease Control. Second, data collection software will be advanced and updated. Special attention will be paid to built-in quality checks creating a more user-friendly interface and facilitating work flow for data abstractors. Online reports will be generated that allow individual sites to query their own data collection in a rapid manner and provide unadjusted comparison to all ACS NSQIP Pediatric sites based on procedure or time frame of interest. Third, the procedures and outcomes of interest will be targeted with feedback from various specialty representatives and experts. Last and most anticipated, risk-adjusted outcomes will be developed for participating institutions. Creating risk-adjusted models within children's surgery is a challenging endeavor primarily because of the low event rates of outcomes of interest. The diversity in patient populations because of age and developmental variability in the pediatric population as well as the variety of procedure types and specialties currently collected in the ACS NSQIP Pediatric make hospital-level quality assessment difficult.

Complication/occurrence Infection Superficial incisional infection Deep incisional infection Organ/space infection Pneumonia Urinary Tract Infection Systemic Sepsis Airway/respiratory events Wound disruption Neurologic events Nerve injury Stroke/vascular accident Hemorrhage Deep vein thrombosis Cardiac events Progressive renal insufficiency Graft or prosthesis failure Total

studies have demonstrated the superiority of rigorously collected clinical data such as those provided by the ACS NSQIP over standard administrative or claims data for the detection of complications and the subsequent development of risk-adjusted outcome measures [9]. The accuracy of data captured by the ACS NSQIP Pediatric is a testament to the key role that clinical data abstractors play in the process. The ACS NSQIP captures postoperative 30-day outcomes regardless of inpatient status. Data abstractors use a variety of techniques of reviewing medical record, phone calls, or e-mails to complete a follow-up. If needed, mortality is verified with query of the National Obituary Archives or local registry data. Our study demonstrates that ACS NSQIP Pediatric data is of high quality based on an overall interrater reliability demonstrating less than 2% discordance between hospital data abstractors and expert audit. The period of this interim phase 1 report spans October 2008 to June 2009 representing the initial 6 months of data collection. With more than 3000 cases included in the analysis, we project more than 6000 cases will be accumulated by the completion of phase 1. This is roughly 1500 cases per site per year and is on par with projections based on 40 cases per 8 day cycle for a single data abstractor dedicated to case abstraction in the established ACS NSQIP. There are several factors to consider when studying these data. There was a large discrepancy in the number of cases collected by each site. The 4 sites began data abstraction at various time points based on institution approval and availability of full staffing. Each site also has varying surgical volume that meets ACS NSQIP Pediatric inclusion. Our sampling methodology captured between 12% and 24% of all cases performed at varying institutions. Risk-adjusted hospital-level comparison as is currently provided by the adult ACS NSQIP was not an anticipated

120 The goal is to provide hospitals with an observed to expected (O/E) ratio. These ratios will be used to provide hospitals with an appropriate risk-adjustment model of outcomes. Options for future statistical analysis may include providing specialty- or procedure-specific reports. Additional options for feedback may include the use of control charts and cumulative sum charts that allow for portrayal of smaller sampling of cases and rare event rates. As the ACS NSQIP Pediatric continues to grow, interested centers must consider the resources and finances required to participate in the program. Whereas Phase 1 was fully funded by the ACS, those participating in Phase 2 of the ACS NSQIP Pediatric are required to pay a reduced programmatic fee to the ACS while the program remains in a development state. These fees cover all of the management and administration of the program, training of the site's clinical reviewer, data audits, technical platform development, and on-going support. Exact fees for participation in the program after development phases are complete remain to be determined. As noted, each site must also hire a dedicated clinical reviewer to enter the high-quality data that serve as the cornerstone of program success. It is important to recognize the ACS NSQIP aims to provide a broad-based, multidisciplinary sampling with assessment of short-term (30 day) outcomes. The program is therefore designed to comment on the quality of surgical care and to identify areas for targeted quality improvement at the hospital level. Those familiar with the care of children are cautioned against comparing the data provided by ACS NSQIP to comprehensive, prospective, diseasespecific registry data with long-term outcomes. In conclusion, this interim report represents the first multiinstitutional prospective assessment of specialtyspecific postoperative morbidity and mortality in children. Implementation of the ACS NSQIP Pediatric has been successful and has facilitated refinement of sampling methodology, data collection practices, and data definitions. The program is now poised for institutional expansion that will facilitate the development of riskadjusted models across surgical specialties. The ACS NSQIP Pediatric represents a long-awaited and exciting opportunity to advance children's surgery and measure the quality of care being delivered to children in the United States.

M.V. Raval et al.

References
[1] Khuri SF, Daley J, Henderson W, et al. The Department of Veterans Affairs' NSQIP: the first national, validated, outcomebased, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care. National VA surgical quality improvement program. Ann Surg 1998;228(4): 491-507. [2] Khuri SF, Henderson WG, Daley J, et al. Successful implementation of the Department of Veterans Affairs' National Surgical Quality Improvement Program in the private sector: the patient safety in surgery study. Ann Surg 2008;248(2):329-36. [3] Hall BL, Hamilton BH, Richards K, et al. Does surgical quality improve in the American College of Surgeons National Surgical Quality Improvement Program: an evaluation of all participating hospitals. Ann Surg 2009;250(3):363-76. [4] Dimick JB, Chen SL, Taheri PA, et al. Hospital costs associated with surgical complications: a report from the private-sector National Surgical Quality Improvement Program. J Am Coll Surg 2004;199(4): 531-7. [5] Dillon P, Hammermeister K, Morrato E, et al. Developing a NSQIP module to measure outcomes in children's surgical care: opportunity and challenge. Semin Pediatr Surg 2008;17(2):131-40. [6] Morrato EH, Dillon P, Ziegler M. Surgical outcomes research: a progression from performance audits, to assessment of administrative databases, to prospective risk-adjusted analysishow far have we come? Curr Opin Pediatr 2008;20(3):320-5. [7] ACS-NSQIP: surgical clinical nurse reviewer training available at http: //acsnsqip.org/main/program_nurse_training.asp, 2009. [8] Shiloach M, Frencher Jr SK, Steeger JE, et al. Toward robust information: data quality and inter-rater reliability in the American College of Surgeons National Surgical Quality Improvement Program. J Am Coll Surg 2010;210(1):6-16. [9] Steinberg SM, Popa MR, Michalek JA, et al. Comparison of risk adjustment methodologies in surgical quality improvement. Surgery 2008;144(4):662-7 discussion 662-7. [10] Birkmeyer JD, Shahian DM, Dimick JB, et al. Blueprint for a new American College of Surgeons: National Surgical Quality Improvement Program. J Am Coll Surg 2008;207(5):777-82.

Discussion
George Holcomb MD, (Kansas City, Kan): Mehul,that was a great presentation. Would you briefly tell us the changes that are becoming instead of a sampling of a variety of cases, aren't we going to be seeing more selected or defined cases that will be captured? Raval, MD (response): Thank you for the question. As Dr. Birkmeyer alluded to earlier in the week, the entire ACS NSQIP program is moving more towards a targeted platform where specific high-risk procedures are going to be collected and the greater benefit may be that the outcomes that are going to be followed are going to be procedure specific with procedure-specific variables as well as postoperative outcomes. The same thing is being planned in the pediatric module. We are beginning that process of identifying what groups of procedures are going to be of greatest benefit to track over time.

Acknowledgments
MVR participates in the American College of Surgeons Clinical Scholars in Residence Program. MVR is supported by the John Gray Research Fellowship and the Daniel F. and Ada L. Rice Foundation. The authors would like to thank the ACS NSQIP staff and the tremendous efforts toward high-quality, accurate data collection by the ACS NSQIP SCRs.

Pediatric ACS NSQIP Philip Glick, MD (Buffalo, NY): Can you give us a brief idea of how long it takes to collect this data? Who collects this data? How much does it cost to collect this data? And is there a cost captured or saved based on the evaluation and preventing these complications? Raval, MD (response): The way the program is set up is that each participating site has a full-time employee dedicated to clinical data extraction. This individual spends 40 hours per week collecting those 35 cases per eight-day cycle and some institutions in the adult program have added full-time employees on because they would like to have a higher sampling of the cases being collected. The overall expense of the program is a programmatic fee of roughly $17,000 and of course your institutional cost for that full-time employee. As far as return on investment, in the pediatric program it is a little bit too early to start assessing those kinds of figures, but the adult program has a lot of literature demonstrating that the program participation is resulting in fewer SSIs and UTIs and so on and that there is tremendous cost savings on the orders of millions of dollars for each institution. Philip Glick, MD: So the cost saving is institutional, so I would assume that the employee is an institutional employee. Raval, MD (response): Yes, sir.

121

Unidentified Speaker: I have a specific question about how NSQIP is handling surgical site infections. It is more common practice among pediatric practitioners to close dirty wounds and some of our adult colorectal surgeons do that as well. What I saw was when they implemented NSQIP in the adult environment in order to avoid surgical site infections of wounds they chose to close they started keeping the wounds open so that they would avoid that ding on the record keeping. Is there any difference to allow for that in terms of pediatric surgical site infections where we often chose to close dirty wounds anticipating that we will have a surgical site infection but that if you open it early you will save 50% of the kids from having to have their wounds packed. Raval, MD (response): The question you are asking I think goes down to clinically rigorous definitions. For the most part, we have always tried to stay in line with definitions provided by the CDC and others. But to answer your question directly, I think there is a little bit of gaming the system that occurs when you start to implement a quality improvement program and I anticipate that will be what we see with the pediatric program. After that initial gaming the system kind of levels out, I think that is when we truly see quality improvement being successful.