Estimating System Leak Rates
Using Spreadsheet Models to Measure Pressure Loss
By Don Meyer, P.E., and
Geoff Stephenson, P.E.
Introduction
Environmental regulations state that if some
materials leak from a system, it must be
reported if the leak exceeds some threshold
value. In a recent project, a refrigeration system
began leaking ammonia. If the ammonia leak
had exceeded 100 lb/day, the leak would have
been reportable and there would have been a
fine for noncompliance.
This article presents a calculation procedure that
was used to demonstrate that the leak was only
41 lb/day. It also demonstrates how parameters
of a nonlinear equation can be determined using
a least squares technique in a spreadsheet.
The calculations were based on a leak test using
nitrogen. The pressure loss in the system was
modeled as though the leak were through an
orifice. The leak data was fit to a pseudo orifice
coefficient. This coefficient was then used to
calculate the loss of ammonia.
The first step in the test was to fill the system
with nitrogen. The pressure in the system was
recorded as a function of time. This loss in
pressure was converted to a pseudo orifice
coefficient using the following model.
FO
w=lb/hr Vent Rate
V=System Volume ft3
W= Weight of Gas in System lb
Figure 1: Fixed Volume Vessel
TECHBriefs 2005 No. 1 6 Burns & McDonnell
The system can be viewed as a fixed volume
vessel with an opening at the top as shown
in Figure 1. The system is filled to an initial
pressure, P0, with nitrogen. Using the initial
pressure, system temperature, the gas molecular
weight and the system volume, the initial weight
of the gas in the system is given by:
V * P0 * MW
W = V* v =
10.73 * Z * (T + 460)
Where:
W = weight of gas in the system, pounds
V = volume of the system, cubic feet
v = vapor density, lb/cf
P0 = the initial system pressure, psia
MW = molecular weight of the gas
Z = gas compressibility
T = system temperature, ºF
For these calculations, the system was assumed
to be ideal. Therefore, Z is equal to 1. The
system was also assumed to be isothermal so
that T is constant.
The model assumes that at time equals 0,
material begins to leak out of a flow orifice,
which represents the leak from the system. This
leak rate is w. By materialVbalance,
* P0 * MW
the rate
W = V* v =
of change of the system
10.73 * Z * (T in
contents (W + lbs)
460)is
given by:
dW
= w(t)
dt
Where t is time in hours and w is flow in lb/hr.
 Nitrogen Venting Data Compared to Model
140

120

100

Pressure psia
80
Data
Source
60

40

20

0
0 2 4 6 8 10 12 14 16
Day

Figure 2: Test Data

The “t” in parenthesis after the “w” indicates
that the flow is a function of time.
Substituting in the above definition of weight in
the system yields:
P is the pressure in the system after time 0.
becomes sonic when the pressure drop is more
than 50% of the upstream pressure. At this
point, the flow rate is proportional to the square
root of the product of the density times the
upstream pressure. This can be expressed as:
K is the pseudo orifice coefficient.

This equation relates the rate of change of the

pressure in the system to the vent rate. Since at any given time w(t) = dW/dt, the above
The flow through a control device such as a two equations can be combined and integrated
flow orifice or control valve for compressible to yield the following expression relating
flow is given by the following equation: system pressure to time:
Where:
-Ln P = K * t 10.73 * (T+460)
P0 V MW
w = Cv * N6 * Y * X * P1 * 1
dW d v V * MW dP
=V* = * = w(t)
dt dt 10.73 * (T + 460) dt Figure 2 shows the data taken during the test.
Cv = flow device sizing coefficient The equation above relates the pressure in
N6 = constant depending on the units system the system versus time. The pressure curve
used is a function of the initial pressure, P0 and
Y = expansion factor K. The objective is to find the values of P0
X = ratio of absolute pressure drop to upstream and K, which minimize the error between
pressure the calculated and the predicted values. This
P1 = upstream pressure is fairly easy when the equations are linear.
1 = upstream density However, in this case the equations are
w = the rate of flow through the orifice nonlinear and the math required to complete
In general, the flow through a control device the least squares calculations is difficult.

Burns & McDonnell 7 TECHBriefs 2005 No. 1

However, Excel has a tool for solving these At 90 psig, the density of ammonia vapor is 0.38
types of problems. It does require some trial and lb/cf. The leakage rate of ammonia is given by
error work. The first step is to set up a column in the following equation:
the data spreadsheet that calculates the pressure
(104.7 * 0.38)0.5
at each point in the data set using assumed 2.17 = 1.73lb/hr
values for P0 and K. The errors are calculated
(112.7 * 0.555)0.5
as the square of the difference between the
calculated value using the assumed constants and Conclusion
the measured values. These errors are summed Based on the nitrogen data and this calculation
together to get the error for that set of constants. procedure, it was demonstrated that the ammonia
leak was less than the reportable 100 pounds per
In Excel there is a tool called “Solver” (most day.
spreadsheet programs have something similar). Don Meyer is a principal
This tool allows the spreadsheet to change one chemical engineer in the
cell to match a value in another cell. This tool Burns & McDonnell Process
& Industrial Group. He has
will also search for maximums and minimums. a master's in chemical
In the above case, the “Solver” tool is used to engineering from Purdue
adjust the K parameter to minimize the sum of University, and has been
the squares of the errors. involved in designing chemical
separation and heat transfer
systems for more than 30
The next step is to change the value of P0 and
years.
repeat the minimization to see if the result is a
lower error level. Since P0 should be close to the
initial point in the data set, it should be easy to
get a first approximation. Using this procedure,
the least error level occurred at P0 = 98 psig and
K = 0.2748. The curve in Figure 2 represents the
above equation using these constants.

In the nitrogen test, the leak rate is the highest at

the initial pressure. The maximum leak rate can
be calculated from K and the system volume.
For nitrogen, the initial leakage rate is 2.17 lb/
hr, the pressure is 112.7 psia and the density is
0.555 lb/cf. Geoff Stephenson is the
process technology manager
When the system is full of ammonia, there is for the Burns & McDonnell
both liquid and vapor present. At 60º F, the Process & Industrial Group.
He has a bachelor's in
pressure in the system is approximately 90
chemical engineering from
psig. As some of the ammonia vapor leaks out, the University of California,
ammonia liquid vaporizes to take its place. Santa Barbara and has been
Therefore, the ammonia system would have involved in the design of
maintained a constant pressure until all of the chemical and refining facilities
liquid had evaporated and then the pressure for over 15 years.
For more information on this subject please send an email to the following addresses:
would have fallen as in the nitrogen test. Geoff Stephenson <techbriefs@burnsmcd.com> or Don Meyer <techbriefs@burnsmcd.com

TECHBriefs 2005 No. 1 8 Burns & McDonnell