Académique Documents
Professionnel Documents
Culture Documents
2, MARCH 2015
773
Manuscript received August 29, 2013; revised December 2, 2013 and January 22, 2014; accepted March 18, 2014. Date of publication March 21, 2014;
date of current version March 2, 2015. This work was supported in part by
the Ministry of Science, ICT & Future Planning, Korea, under the Convergence
Information Technology Research Center support program (NIPA-2013-H040113-1007) supervised by the National IT Industry Promotion Agency, the Sogang
University Research Grant of 2013 (SRF-201314003.01), and the Leading Foreign Research Institute Recruitment Program through the National Research
Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future
Planning (MSIP) (2013K1A4A3055268).
J. H. Song is with the Medical Solutions Institute, Sogang Institute of
Advanced Technology, Sogang University, Seoul 121-742, Korea (e-mail:
shsong00@sogang.ac.kr).
C. Kim and Y. Yoo are with the Department of Electronic Engineering, Sogang University, Seoul 121-742, Korea (e-mail: choye@sogang.ac.kr;
ymyoo@sogang.ac.kr).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/JBHI.2014.2313145
I. INTRODUCTION
FFECTIVE vein visualization is clinically important for
various point-of-care applications, such as needle insertion [1], [2]. Needle insertion has been widely used for blood
sample acquisition and intravenous infusion, but successful insertion rates are not high, especially for obese patients and
children [3]. In addition, insertion failure may lead to several
adverse problems, such as blood clotting, allergic reactions and
bone and nerve damage [4]. As such, ultrasound imaging, which
visualizes the location of veins at various depths, has been utilized to improve the success rate of vascular accessing [5]. Due
to its superior guidance performance [6], ultrasound imaging
substantially decreases the rate of vascular access failure, particularly when used by less experienced clinicians [7]. However, ultrasound imaging may not be optimal for point-of-care
applications because of its high cost, time required and low
accessibility.
For subcutaneous veins, infrared vein visualization can also
be used as a guidance tool. Infrared vein visualization illuminates and detects infrared lights in order to differentiate veins
from other soft tissues based on their spectroscopic optical absorption coefficients [1]. Using this technique, clinicians can
achieve a higher success rate for vascular accessing [8], [9].
However, this infrared imaging guidance method suffers from
the high cost of the equipment required and the safety concerns
associated with the usage of lasers.
In this paper, a new vein visualization method based on multispectral Wiener estimation is proposed and its real-time implementation on a smart phone is presented. In the proposed
method, the RGB color image is acquired via the built-in camera on a commercial smart phone. The reflectance spectrum
information for the vein visualization is then achieved by utilizing a multispectral Wiener estimation method. In addition, post
image processing algorithms, such as histogram stretching and
average/median filtering, are applied to improve image contrast
while reducing noise. The performance of the developed system
was evaluated using color calibration chart experiments. Moreover, in vivo imaging experiments were conducted to confirm
the feasibility of the proposed vein visualization method by implementing it on a commercial smart phone (Galaxy Note 2,
Samsung Electronics Inc., Suwon, Korea) as an Android
application.
This paper is organized as follows. In Section II, the method
used to estimate the reflectance spectrum information from
RGB images is described. In Sections III and IV, the results
2168-2194 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications standards/publications/rights/index.html for more information.
774
IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, VOL. 19, NO. 2, MARCH 2015
Fig. 2. Vein visualization images obtained by the proposed method and their
histograms (a) before and (b) after post image processing. After histogram
stretching, image contrast improved considerably. The yellow boxes in the
middle of the images indicate the ROI used in the postprocessing.
from the performance evaluation and the in vivo imaging experiments are presented, respectively. In Section V, the advantages
and possible applications and limitations of the developed system are discussed, and finally in Section VI, we present the
conclusion.
II. METHODS
A. Reflectance Spectrum Estimation From a Color Image
Using Wiener Estimation
Fig. 1 illustrates the entire procedure for estimating the reflectance spectrum from the obtained RGB (red, green, and blue)
data. In the proposed method, the Wiener estimation method,
known as one of the simplest low- to high-dimension estimators,
which also boasts good performance, was applied [10][12]. As
shown in Fig. 1, in Wiener estimation, an RGB pixel value
from the built-in smart phone camera vi (i = R, G, B) can be
represented by
vi = fi () E () S () () d, i = R, G, B (1)
where fi () is the filter response for each red, green, and blue
wavelengths, E () is the spectrum of the light source, S () is
the camera sensitivity, and () is the targets real reflectance
spectrum [10], [11]. By utilizing matrix forms, (1) can be rewritten as
vi = Fit E S
=G
(2)
where t indicates the matrix transposition. The estimated reflectance spectrum est can be obtained from (2) if the inverse
matrix H = G1 is as follows:
est = H .
(3)
H = r v 1
vv
(4)
h1,G
h1,B
1
1,R
h2,G
h2,B R
2 h2,R
. .
..
..
. = .
(5)
.
. G .
. .
B
h
h
h
k 1
k 1,R
k 1,G
k 1,B
k
hk ,R
hk ,G
hk ,B
From (5), the reflectance spectrum rest = [1 2
k 1 k ]t can be estimated from the camera output v =
[R G B]t . The subscript k is an index representing specific
wavelengths which are to be estimated. The wavelength values
used in this paper are listed in Table II.
To estimate the spectral intensity of the kth wavelength, the
vector [hk ,R hk ,G hk ,B ] composed of Wiener estimator obtained by (4) is multiplied by the camera input vector [R G B]t .
Of the estimated wavelengths, 620 (620 nm) was selected and
its intensity was normalized so as to set the maximum value
to 255 (i.e., gray scale) for vein visualization. Note that the
light source, camera sensitivity, and filter responses are already
included in the estimation matrix H. As such, the reflectance
spectrum of any RGB images can be obtained using the predetermined estimation matrix if RGB images are taken under the
same circumstances, which is critical for real-time performance.
B. Postprocessing for Multispectral Images From Wiener
Estimation
Use as shown in the left panel of Fig. 2(a), the low contrast
of a multispectral image from Wiener estimation may not allow
SONG et al.: VEIN VISUALIZATION USING A SMART PHONE WITH MULTISPECTRAL WIENER ESTIMATION
Pin minin
=
(maxout minout ) + minout (6)
maxin minin
where Pout and Pin are the pixel values after and before histogram stretching, respectively, and maxin and minin are the
maximum and minimum values in the ROI before histogram
stretching, respectively. Since the intensity of the input signal can vary depending on image acquisition conditions and
surrounding environmental factors, such as light source, background light, and illuminating angle between the camera and
the target, both maxin and minin are determined on a frame-byframe basis using the input signals within the ROI. In addition, a
margin of 40% is given to minin to reduce the dark areas caused
by shadows and weak illumination. maxin and minin which
represent the maximum and minimum values after histogram
stretching are set to be 180 and 0, respectively.
With histogram stretching, as shown in Fig. 2(b), the distribution of the histogram is adjusted to improve image contrast. This
results in better visualization of blood vessels. Since noise can
be also emphasized by histogram stretching, 33 2-D average
and 55 median filters are sequentially applied to the histogram
stretched image.
III. EXPERIMENTS
A. Color Calibration Chart Experiments
The color calibration chart experiments were conducted to obtain the estimation matrix G1 and to evaluate the performance
of the procedure. As shown in Fig. 1, a color calibration chart
(ColorChecker Classic, X-rite, Grand Rapids, MI, USA) consists of various color samples for color adjustment. For increased
measurement accuracy, all experiments were conducted in a D65
illumination (i.e., standard daylight) environment created using
a lighting booth (SpectraLight III, X-rite, Grand Rapids, MI,
USA). To calculate the estimation matrix H [see Fig. 1 and (4)],
the camera output v and the real reflectance spectrum were
measured using the built-in smart phone camera (Galaxy Note
2, Samsung Electronics Inc., Suwon, Korea) and the spectrometer (Avaspect ULS2048 16, Avantes Inc., Broomfield, CO,
USA), respectively.
775
In order to access the real reflectance spectrum , the spectrum of the light source was first measured using a white reference tile (WS-2 Reference Tiles, Avantes Inc.), which reflected
98% of light within 3501800 nm in wavelength. Each sample
color spectrum in the color calibration chart was calculated by
dividing the measured sample color spectra by the light source
spectrum. The camera output v was measured by taking a photograph of the color calibration chart. In the performance evaluation, values were measured five separate times and then averaged
together.
To evaluate the obtained H, the estimated spectra were compared to the measured spectra from the spectrometer. For quantitative analysis, the root-mean-square error was calculated by
N
2
i=1 (xm sr,i xest,i )
(7)
RMSE =
N
where N was the number of samples, and xm sr,i and xest,i
denoted ith measured and estimated values, respectively.
B. In Vivo Imaging Experiments Using the Developed Vein
Visualization System
The proposed reflectance spectrum estimation method was
implemented on the smart phone as an Android application.
Using the developed system, in vivo imaging experiments were
conducted to demonstrate the clinical feasibility of the application. The eclipse SDK 2.72 was used to develop the Android
application. The built-in flash light was programmed to be kept
on continuously so that a sufficiently bright light would illuminate the targets. The screen was split in half in order to show
reference and vein images simultaneously. In the reference image placed on the left panel, the ROI, indicated by a yellow box,
was developed to allow users to place targets within the ROI.
With the developed system and the obtained estimation matrix
H, in vivo RGB and vein images were obtained from various
locations around the human body including the palm, wrist,
back of the hand, and inside elbow, which are the most common
locations for subcutaneous vein visualization applications. In
addition, to confirm that the patterns shown in the vein images
are real blood vessels, the locations of the veins were compared
to those in the ultrasound color Doppler images obtained from a
commercial ultrasound imaging system (V10, Samsung Medison, Seoul, Korea), which is a sensitive imaging method for
detecting blood vessels.
IV. RESULTS
A. Color Calibration Chart Experiments
Table I and Fig. 3 show the averaged camera output v and the
measured spectra values gathered from the spectrometer from
the color calibration chart, respectively. With v and , the estimation matrix H can be calculated by using (4) and the obtained
values of H are summarized in Table II. To evaluate the performance of Wiener estimation, the estimated spectra in Fig. 3 were
compared to the measured spectra from the spectrometer. In addition, for quantitative evaluation, the RMSE values between
the measured and estimated spectra were calculated by using
776
IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, VOL. 19, NO. 2, MARCH 2015
TABLE I
AVERAGED RED (R), GREEN (G), AND BLUE (B) VALUES OBTAINED FROM
EACH SAMPLE COLOR IN THE COLOR CALIBRATION CHART USING THE
BUILT-IN CAMERA ON THE GALAXY NOTE 2 SMART PHONE
Fig. 5. In vivo RGB and multispectral vein images obtained from elbows,
palms, wrists and backs of the hand of multiple volunteers. It can be seen that
veins are more clearly visualized on the multispectral images.
Fig. 3. Reflectance spectra obtained from each sample color in the color
calibration chart using the spectrometer.
TABLE II
CALCULATED ESTIMATION MATRIX FROM THE OBTAINED CAMERA OUTPUT
AND THE REAL REFLECTANCE SPECTRUM
the RGB image (left panel), a yellow ROI box is displayed. All
in vivo images shown in the results section were obtained using
the developed system.
Fig. 5 shows the RGB and relevant vein images of the palm,
wrist, back of the hand and inside elbow from multiple volunteers. It can be observed that veins were more clearly distinguishable in the multispectral images than in the original RGB
image. Even in several cases, veins invisible in the RGB image
were successfully revealed in the multispectral images. Consistently successful results were also seen in the palm, wrist, back
of the hand, and inside elbow as shown in Fig. 5.
To verify the accuracy of the proposed method, the locations
of the veins shown in the multispectral images were compared
to those shown in the ultrasound color Doppler images. Since
multispectral images exhibit a coronal plane whereas ultrasound
images show a transverse plane, three positions, in which ultrasound color Doppler images were to be obtained, were selected
and marked on skin in the vein image. Fig. 6 shows the multispectral vein image indicated by three marks. Fig. 6 also shows
the corresponding ultrasound color Doppler images obtained
from a commercial ultrasound imaging system (V10, Samsung
Medison, Seoul, Korea). As shown in Fig. 6(a) and (b), a single
thick vein (first mark), a single thin vein (second mark) and two
thinner veins (third mark) which were separated from the upper
vein were also shown in the ultrasound color Doppler images.
As such, we believe that patterns shown in the multispectral
images were revealing the positions of real blood vessels. For
further verification, ultrasound B-mode images were obtained to
SONG et al.: VEIN VISUALIZATION USING A SMART PHONE WITH MULTISPECTRAL WIENER ESTIMATION
777
Fig. 8. In vivo RGB and multispectral forearm vein images from two high
BMI volunteers when an interior light was turned (a) and (b) on and (c) and
(d) off, respectively. The BMI values of the two volunteers were 29.4 and 28.1,
respectively.
Fig. 6. Multispectral vein image and three ultrasound color flow images obtained at three marked positions. By comparing locations of blood vessels on
multispectral and ultrasound color Doppler images, it is concluded that the dark
pattern in the multispectral image visualized real blood vessels.
778
IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, VOL. 19, NO. 2, MARCH 2015
REFERENCES
[1] V. P. Zharov, S. Ferguson, J. F. Eidt, P. C. Howard, L. M. Fink, and
M. Waner, Infrared imaging of subcutaneous veins, Lasers Surg. Med.,
vol. 34, pp. 5661, 2004.
[2] H. D. Zeman, G. lovhoiden, C. Vrancken, and R. K. Danish, Prototype
vein contrast enhance, Opt. Eng., vol. 44, 086401, pp. 19, 2005.
[3] T. Maecken and T. Grau, Ultrasound imaging in vascular access, Crit.
Care Med., vol. 35, pp. S178S185, 2007.
[4] M. Mansoor, S. N. Sravani, S. Z. Naqvi, I. Badshah, and M. Saleem,
Real-time low cost infrared vein imaging system, in Proc. Int. Conf.
Signal Process., Image Process. Pattern Recog., Coimbatore, India, 2013,
pp. 14.
[5] The American Institute of Ultrasound in Medicine. (Aug. 20, 2013). Use
of ultrasound to guide vascular access procedures. [Online]. Available:
http://www.aium.org/resources/guidelines/usgva.pdf
[6] T. J. Milling, J. Rose, W. M. Briggs, R. Birkhahn, T J. Gaeta, J. J. Bove,
and L. A. Melniker, Randomized, controlled clinical trial of point-ofcare limited ultrasonography assistance of central venous cannulation:
The third sonography outcomes assessment program (SOAP-3) trial,
Crit. Care. Med., vol. 33, pp. 17641769, 2005.
[7] A. G. Randolph, D. J. Cook, C. A. Gonzales, and C. G. Pribble, Ultrasound guidance for placement of central venous catheters: a meta-analysis
of the literature, Crit. Care. Med., vol. 24, pp. 20532058, 1996.
[8] H. A. Hess, A biomedical device to improve pediatric vascular access
success, Pediatr. Nurs., vol. 36, pp. 259263, 2010.
[9] M. J. Kim, J. M. Park, N. Rhee, S. M. Je, S. H. Hong, Y. M. Lee,
S. P. Chung, and S. H. Kim, Efficacy of vein viewer in pediatric peripheral intravenous access: A randomized controlled trial, Eur. J. Pediatr.,
vol. 171, pp. 11211125, 2012.
Choye Kim received the B.S. and M.S. degrees in electronic engineering from
the Sogang University, Seoul, Korea, in 2004 and 2012, respectively.
She is currently a Researcher at LG Electronics.