2010 Second International conference on Computing, Communication and Networking Technologies

Effects of Variation of Patch Size and Seam Size in

Patch Based Texture Synthesis
Vinod Kumar.R.S

S.Arivazhagan

Department of Electronics & Communication Engineering,

Noorul Islam College of Engineering,, Tamil Nadu. India
rsvinodkumar69@yahoo.co.in

Department of Electronics & Communication Engineering,

Mepco Schlenk Engineering College, Tamil Nadu, India
s_arivu@yahoo.com

Abstract-

This paper presents a simple image-based method of

generating novel visual appearance in which a new image can be
synthesized by stitching together small patches of existing images.
This is a fast and very simple texture synthesis algorithm for
patches of uniform size which produces surprisingly good results
for a wide range of textures. Further, to improve the visual
perception of synthesized texture, a modified patch based
synthesis technique, in which the neighboring patches of the
synthesized output are made to overlap is proposed. Here instead
of picking out a random block, the algorithm searches for a
block that, by some measure agrees with its neighbors along the
region where overlap takes place. Experiments with different
sample textures for different patch sizes and seam sizes are
carried out and the quality of the synthesized outputs are
measured in terms of quantifying parameters like entropy,
histogram matching and computation time. Comparisons with
existing techniques demonstrate that these two approaches have
better performance than the previous texture synthesis
algorithms already proved before.
Keywords: Image Processing, Texture, Texture Synthesis,
Markov Random Fields, Image-based Rendering

I. INTRODUCTION
A. Terminology
Texture describes a variety of surfaces such as
terrain, plants, minerals and fur or skin. It is very difficult to
give a comprehensive definition for texture due to the
diversity of pattern in different natural and artificial textures.
According to Sklansky (1978), an image region has a constant
texture if a set of local properties in that region is constant,
slowly varying or approximately periodic [1]. Muath Sabha
(1998) says a texture is an image or part of an image or it can
be the characteristic appearance of a surface having a tactile
quality, made up of visually related elements distributed in a
repeated pattern [2].
Texture Synthesis is the process of algorithmically
constructing a large digital image from a small digital sample
image by taking advantage of its structural contents. The goal
of texture synthesis can be stated as follows: From a texture
sample, synthesize a new texture that, when perceived by a
human observer, appears to be generated by the same
underlying process. The two challenges involved in texture
synthesis are modeling and sampling. Modeling states how to
estimate the stochastic process from a given finite texture

978-1-4244-6589-7/10/$26.00 2010 IEEE

sample and sampling indicates how to develop an

efficient sampling procedure to produce new images with the
texture in question. [3].
The paper put forth an extremely simple algorithm to
address the texture synthesis problem. The main idea is to
synthesize new texture by taking patches of existing texture
and stitching them together in a consistent way. An effective
patch based texture synthesis method that tries to combine the
best aspects from several successful approaches is suggested.
The core algorithm resolves around the other patch-based
texture synthesis approaches [4]. An analysis of this
synthesized technique is carried out for different textures by
varying the input patch sizes and seam sizes.
B. Related Works
Texture synthesis has been an active research topic
since the middle of 1960s and it continues to enjoy wide
applications. Published techniques can be roughly subdivided
in two main categories: procedural texture synthesis and
texture synthesis by example [5].
Procedural Texture Synthesis is basically a
customized program which transforms some predefined signal
into a desired texture. This produces a high quality, continuous
texture. A major limitation of procedural texture synthesis
approaches is that, for creating a new texture a new program
has to be written.
Texture Synthesis by example generates a novel
texture that is similar to a given sample texture. There are
three classes of algorithms: texture synthesis by analysis, pixel
based texture synthesis and patch-based texture synthesis.
Texture Synthesis by analysis is usually characterized by a
sample texture by a limited number of statistics. A new texture
is synthesized such that the statistics of the sample texture are
maintained.
Pixel-based Texture Synthesis generates novel
textures by repeatedly selecting and copying a single pixel
from the sample texture, based on already synthesized pixels
in the novel texture. These algorithms are generally based on
the theory of Markov Random Fields, a two-dimensional
extension to Markov Chains [6]. Large numbers of researchers
are working in this area of pixel based texture synthesis. Efros
and Freeman has pointed out that pixel-based texture
synthesis algorithms perform excess computations when
dealing with structured textures [4].

Patch-based texture synthesis generates novel

textures by repeatedly selecting and copying a single patch
from the sample texture, based on already synthesized patches
in the novel texture. Some of the earlier works done in this
category are: Xu, et al (2000) presented a procedural method
for synthesizing large textures from an input texture sample.
The basis of their algorithm is the chaos mosaic. The chaos
mosaic is fast and facilitates memory efficient texture
rendering through procedural texturing but requiring a large
amount of storage [7]. Efros and Freeman (2001) presented a
simple image-based method of generating novel visual
appearance in which a new image was synthesized by stitching
together small patches of existing images called image
quilting. This method worked directly on the images and did
not require 3D information [4]. Nealen and Alexa (2003)
adaptively split patches so as to use large possible patches
while staying within a user-defined error tolerance for the
mismatch in the overlapping region. The remaining errors in
the overlapping regions are eliminated using pixel-based resynthesis. Their approach has improved over previous texture
synthesis algorithms, for textures with clearly visible, possibly
anisotropic structure such as natural stone wall or scales [8].
Tonietto, et al. (2005) presented a wavelet-based approach for
selecting patches for patch-based texture synthesis. They
showed that the use of wavelets as metric for selection of the
best patches has improved texture synthesis for samples which
previous work failed [9]. Li, et al. (2006) presented a novel
approach for texture synthesis with the help of texture
consistency" and matched patch along a spiral path. Compared
with image quilting [4] under the same condition of synthetic
quality, this approach was reported to be on an average 10
times faster [10]. Long and Mould (2007) presented an
improvement to the minimum error boundary cut. The
resulting artifacts tend to be less pronounced than those
created using the traditional cumulative distance metric [11].
Cheng, et al. (2009) presented a new patch-based texture
synthesis method. In this method, similarity measurement
calculation for selecting suitable patches in texture synthesis
has been greatly saved, and synthesis conflicts between
neighboring patches were substantially reduced [12].
The paper is organized in the following manner. It
begins with a brief introduction about texture synthesis and the
related works in patch based texture synthesis are given in this
Section. The motivation behind the paper is discussed in
Sections 2. Section 3 formalizes the detailed idea about the
Patch Based Texture Synthesis used. While texture synthesis
results are presented in Section 4 with an emphasis on the
texture quality, conclusions and future works are discussed in
Section 5.
II. MOTIVATION
Modern techniques, to meet the demand for highquality synthetic images, rely on the ability to generate
arbitrarily large images of a texture from smaller input images.
While stochastic and structured textures are challenging in
their own right, the texture synthesis problem is the most
difficult for semi-structured textures. Early successful

approaches at synthesizing structured and semi-structured

textures, like Fast based texture synthesis using Tree
structured Vector Quantization [13], were notable for their
computational intensity and long running times. Efros and
Freeman (2001) have stated that the full input image searches
of the approach described in [13] are unnecessary for many of
the pixels in structured textures. This patch based synthesis
approach has two main advantages over the pixel-at-a-timeapproach. First by transferring large patches at a time the
number of searches required can be reduced which leads to
significant decrease in run time. The second advantage is that,
by transferring large patches at a time, its visual consistency
within all transferred patches is guaranteed.
III. PROPOSED MODEL
The algorithm for the proposed model is explained in
this section using two different techniques. They are random
placement of blocks and neighboring blocks constrained by
overlap.
Random Placement of Blocks:
The initial idea is to tile together randomly selected
square patches from the input image. This very naive approach
is actually quite successful for a very limited set of perfectly
repetitive textures. In most semi-structure textures, the
resulting images will have clear borders where neighboring
patches meet. The schematic diagram for the synthesis of
textures under this technique in patch based texture synthesis
is shown on Fig.1 (a).

Fig. 1. Patch Based Texture Synthesis; (a) Random Placement of Blocks

(non-overlapping method); (b) Neighboring Blocks Constrained by Overlap
(overlapped regions are darkened).

Neighboring Blocks Constrained by Overlap:

The next step in the approach is to allow neighboring
patches of texture to overlap each other. Now, instead of
picking a block at random, the algorithm searches for such a
block that by some measure agrees with its neighbors along
the region of overlap as shown in the schematic diagram
shown in Fig.1 (b). In the obtained output a clear improvement
in the structure of the resulting texture is observed. The
detailed discussion of the algorithm consists of the following
steps:
i. Initialize the upper-left corner of the output image with a
randomly selected patch from the input image.
ii. Repeat the following step starting from the top-left position
and proceeding in scan-line order.

iii. During each iteration of processing, select the best-fit patch

using the following procedure to paste onto the current area of
texture according to L2 norm.
Perform a brute- force exhaustive search of all
possible patches in the input image.
Calculate the error for each patch, retain all patches
that meet the best-fit criteria.
Randomly select one of the retained patches.
iv. Select the next patch to be added from the input image
from set of the best-fit patches.
The heart of the algorithm is the left-to-right, top-tobottom scan of the output image. Because of the left-to-right,
top-to-bottom ordering of the scan, the area of overlap to
consider is always contained in an L-shape, which is
composed of a horizontal strip across the top of the patch and
a vertical strip along the left edge of the patch. Two special
cases arise in which the overlap does not take on this
characteristic L-shape. The first case is the processing of
patches of top row of the output image. The second case arises
in processing the first patch of every row.
The size of the block is the only parameter controlled
by the user and it depends on the properties of a given texture.
The block must be just big enough to capture the relevant
structures in the texture, but small enough so that the
interaction between these structures is left up to the algorithm.
The error is computed using the L2 distance between the two
overlapping regions of the adjacent neighborhoods. The L2
distance is the sum of squared distance between all pixels in
the overlapping regions which is given by

D( p1 , p 2 ) = (R1 R2 ) + (G1 G 2 ) + (B1 B2 )

2

2 1/ 2

where R1, G1 and B1 are color components of p1 while R2, G2

and B2 are the color components of p2 where p1 and p2
represent pixels.
IV. EXPERIMENTAL RESULTS AND DISCUSSIONS
The texture synthesis results obtained using random
placement of blocks or non overlapping method and
neighboring blocks constrained by overlap or random method
are presented and discussed in this section. Experiments are
conducted with five different texture images, say, mat,
wave, pebble, grass and tiles for three different input
patches of size 32x32, 64x64 and 128x128 with three different
L-shaped overlapped regions of size 4, 8 and 12 pixels. The
result obtained is enumerated: The outputs of only three types
of textures one each for irregular, near regular and regular are
shown in Fig.3, Fig. 4 and Fig.5. respectively for various patch
sizes and seam sizes.
A. Random Placement of Blocks:
The texture synthesis results obtained using random
placement of blocks or non overlapping method is presented
and discussed in this section. When the patches are of size 32
x 32, 64 patches are concatenated to form the synthesis output.

This results in 7 vertical and horizontal boundary lines of

patches within the output of size 256 x 256. When the patch
size is increased to 64 x 64, 16 patches are concatenated to
synthesize the output. Because of this, 3 vertical and
horizontal lines occur. One vertical and horizontal line passing
through the centre are seen during the synthesis of the output
texture of size 256 x 256, where 4 patches of size 128 x 128
are concatenated. The texture synthesis results obtained for the
three of the mentioned five texture inputs using the nonoverlapped method for three different patch sizes are shown in
the first row of Fig.3 - Fig.5 respectively. From these three set
of outputs it is observed that these discontinuities (visual
artifacts) are less prominent in irregular textures, slightly
visible in near regular textures and quite prominent in regular
textures as evident in the first row of Fig.3 - Fig.5.
B. Neighboring Blocks Constrained by Overlap:
Experiment is performed with input patches of size
32x32, 64x64 and 128x128 pixels, with all five input textures
with a seam size of 4, 8 and 12 pixels in which 4, 8 and 12
rows on the top and 4, 8 and 12 columns on the left of the tobe synthesized patch overlaps respectively with the previously
synthesized patches which lie above and on the left of the tobe-synthesized patch. Exceptions are given to the to-besynthesized patches which are in the first column and first row
of the output texture. For them only 4, 8 and 12 rows or 4, 8
and 12 columns respectively are overlapped. The texture
synthesis results obtained for the above mentioned five texture
inputs for the three different overlap regions are shown in the
second, third and fourth rows of Fig.3 - Fig.5. respectively.
From the figures it is observed that when the patch size is
32x32, 64x64 and 128x128, as in the case of random
placement of blocks, 7, 3 and 1 vertical and horizontal
boundary lines of patches are visible but in a subtle manner.
Since the adjacent patches are selected using similarity
measures, there is better matching between the boundaries of
patches and so the boundary lines are not clearly visible.
Moreover, as the patch size is increased the synthesized
outputs are better in quality. Also the output is more perfect
for random textures (Fig.3) and the output texture quality
gradually decreases for a near regular texture (Fig.4) and
becomes poor for a regular texture (Fig.5).
C. Observation with Different Patch Size under Different
Techniques for an Input Texture
From the output of the experiments with patch sizes
of 32x32, 64x64 and 128x128, performed for an input of
grass with the two synthesis techniques, the following
observations are made:
When the patch size is 32x32, the output of the first
technique is better than the second technique. It can be seen
clearly from the first image of row one and row two of Fig.4.
But, in both the cases the resemblance between the input
texture and the output texture is poor. When the patch size is
increased to 64x64, there is a vast improvement in the output
obtained by overlap method. Even though the visual quality of
the output obtained by random placement of blocks method

has increased, it is not as much as in the case of the

overlapping method which is shown in the second image of
row one and row two of Fig.4. Here there is a slight similarity
between the input texture and the output textures. When the
patch size is increased to 128x128, the quality of the outputs
obtained in both the methods has increased many fold. The
output textures are almost identical to the input texture and in
that the discontinuities are less prominent in the second
method than in the first method. This effect can be visualized
from the third image of row one and row two of Fig.4. The
same effect is observed in the experiments with mat, wave,
pebble and tiles as input textures. In short, the quality of
synthesized output texture is vastly improved when the patch
size and the overlapping regions are larger. However it is
important to note that the larger the overlapping regions the
greater will be the computations amply be compensated well
by larger patch size.
D. Limitations
Although the techniques discussed in this paper has
shown excellent performance for irregular and near regular
textures, it can not be so for regular textures. To show this,
experiments are performed with patch sizes of 32x32 and
64x64 of tiles texture under both the synthesis techniques and
the results are shown in Fig.5. From the results of random
placement blocks, it can be easily identified that there is only
less correlation between adjacent patches. In overlapping
method, though the adjacent patches almost match each other
based on the measured similarity value, the synthesized output
texture differs much from the visual input texture as shown in
the Fig.5.
E. Performance Comparison of the algorithm using various
quantifying parameters.
The quantifying parameters such as Entropy,
Computation Time and Histogram Matching are used to
measure the performance of the texture synthesis algorithm. It
can be observed from the Table.1, that as the patch size
increases, the computation time decreases. Also in most of the
cases the average difference in histogram decreases as the
patch size and seam size increase. These observations closely
match with the perceptual (visual) comparison of outputs done
in the previous section.
V. CONCLUSION AND FUTURE WORKS
The paper presents a method of synthesizing a new
image by stitching together small patches of existing images.
Two methods are discussed. In the first method the selection
of the patch is independent of the previously processed patch,
while in the second method the new patch is processed after
performing the similarity measurements. Despite its
simplicity, this method works remarkably well when applied

to texture synthesis, producing results that are equal or better

than the previous techniques with improved stability and with
less chance of growing garbage and at a fraction of the
computational cost.
The visual perception can still be increased by
allowing the blocks to have ragged edges which will allow
them to better the features in the texture. The computation
time can be reduced by searching new patches among the
neighborhoods of the previous matched patch along a spiral
path.
REFERENCES

[1] J. Sklansky, Image segmentation and feature extraction,

IEEE Transaction on system Man Cybernet 8, pp. 237-247,
1978.
[2] M Sabha, Sample Based Texture Synthesis, Ph.D Thesi,
Katholieke Universiteit Leuven, April 2008.
[3] L Wei, Texture Synthesis by Fixed Neighborhood
Searching, PhD Thesis, Stanford University, Nov. 2001.
[4] A.A.Efros and W.T. Freeman, Image Quilting for Texture
Synthesis and Transfer,Proceedings Siggraph, ACM Press,
2001, pp. 341-346.
[5] M. Sabha, P. Peers and P. Dutre, Texture Synthesis by
Exact Neighborhood Matching, Computer Forum, 2007.
[6] R. Paget, I. Longstaff, Texture Synthesis via a non
causal nonparametric multiscale Markov Random Fields,
IEEE Transaction on Image Processing, 1998, vol.7 (6),
IEEE, pp.347-354.
[7] Y.Xu B. Guo and H. Shum, Chaos mosaic: Fast and
memory efficient texture synthesis, In Technical Report,
Microsoft Research. MSR-TR-2000-32, 2000.
[8]. A Nealen and M Alexa, Hybrid texture synthesis. In
Eurographics Symposium on Rendering , Leuven, Belgium,
pp. 97-105, 2003.
[9] L. Tonietto., M. Walter and C. R. Jung.: Patch-Based
Texture Synthesis using Wavelets, Proceedings of the XVIII
Brazilian Symposium on Computer Graphics and Image
Processing, page 383, 2005.
[10] Y. Li B.Yin and D. Kong, Texture Synthesis Based
on Patch Matching
along Spiral Path, Journal of
Information & Computational Science 3: 4, 2006.
[11] J. Long and D. Mould, Improved Image Quilting in
GI2007: Proceedings of Graphics Interface-2007, pp- 257264, ACM Press, New York, NY, USA, 2007.
[12] W. W.Cheng1, L. F.Tong, H. P.Jie & W. E.Hua .:
Texture synthesis via the matching compatibility between
patches, Science in China Presss, 2009.
[13] I.Wei and M. Levoy, Fast texture synthesis using tree
structured vector
quantization, In
SIGGRAPH 00:
Proceedings of the 27th annual conference on Computer
Graphics and interactive techniques, 2000 ACM press/
Addison- Wesley Publishing Co., pp 479- 488.

Fig.2. Sample Inputs used for Patch Based Texture Synthesis: (a) wave. (b) mat. (c) Pebble. (d) grass. (e) tiles.
TABLE 1. PERFORMANCE COMPARISON OF PATCH BASED TEXTURE SYNTHESIS ALGORITHM FOR VARIOUS PATCH SIZES AND SEAM SIZES

Textu
re
Type

Patch Input
Size Entropy

Random Placement of
Blocks

Neighboring Pixels
Constrained by Overlap
(4pixels)

Averag
e
differen
ce in
Histogr
am

Averag
e
differen
ce in
Histogr
am

Outpu
t
Entro
py

Compu
tat
ion
Time

Outpu
t
Entro
py

pebbl
e

tiles

mat

grass

Outpu
t
Entro
py

in sec.

in sec.

wave

Compu
tat
ion
Time

Neighboring Pixels
Constrained by Overlap (8
pixels)

Averag
e
differen
ce in
Histogr
am

Comput
ation
Time

Neighboring Pixels
Constrained by Overlap (12
pixels)

Outpu
t
Entro
py

Averag
e
differen
ce in
Histogr
am

in sec.

Comput
atIon
Time
in sec.

32x32

6.499

6.259

0.0008

0.313

6.543

0.0018

0.479

6.471

0.0019

0.487

6.548

0.0023

0.492

64x64
128 x
128

6.499

6.228

0.0006

0.078

6.446

0.0009

0.328

6.445

0.0009

0.344

6.441

0.001

0.359

6.499

6.343

0.0004

0.062

6.330

0.0005

0.281

6.326

0.0005

0.289

6.316

0.0005

0.313

32x32

7. 926

7.889

0.0009

0.312

7.701

0.0018

0.453

7.625

0.0021

0.469

7.479

0.0024

0.484

64x64
128 x
128

7. 926

7.896

0.0006

0.078

7.873

0.0008

0.328

7.873

0.0008

0.344

7.864

0.0009

0.359

7. 926

7.895

0.0003

0.063

7.900

0.0004

0.297

7.899

0.0004

0.313

7.897

0.0003

0.328

32x32

6.875

6.658

0.0010

0.344

6.229

0.0024

0.484

6.066

0.0029

0.516

5.994

0.0004

0.531

64x64
128 x
128

6.875

6.677

0.0008

0.125

6.661

0.0015

0.383

6.661

0.0015

0.406

6.544

0.0018

0.422

6.875

6.715

0.0007

0.109

6.670

0.0009

0.336

6.64

0.0009

0.359

6.631

0.0009

0.375

32x32

7.390

7.314

0.0007

0.281

7.143

0.0013

0.453

7.066

0.0016

0.461

7.075

0.0017

0.469

64x64
128 x
128

7.390

7.303

0.0004

0.063

7.250

0.0008

0.328

7.25

0.0008

0.336

7.229

0.0008

0.359

7.390

7.313

0.0003

0.047

7.313

0.0004

0.289

7.314

0.0004

0.313

7.313

0.0004

0.328

32x 32

7.579

7.666

0.0011

0.250

7.503

0.0019

0.453

7.458

0.002

0.469

7.396

0.0023

0.477

64x64
128 x
128

7.579

7.682

0.0009

0.063

7.668

0.0011

0.328

7.668

0.0011

0.344

7.654

0.0012

0.344

7.579

7.698

0.0008

0.047

7.698

0.0008

0.289

7.697

0.0008

0.297

7.695

0.0008

0.313

(a)

(b)

(c)

Fig.3. Texture Synthesis Results using wave as input, e.g. for Irregular Texture
Column (a) 32x32 patch size; (b) 64x64 patch size; (c) 128x128 patch size; Row (1) Overlapped Region- NIL;
(2) Overlapped Region- 4pi; (3) Overlapped Region- 8pi; (4) Overlapped Region- 12pi.

(a)

(b)

(c)

Fig.4. Texture Synthesis Results using grass as input, e.g. for Near regular Texture
Column (a) 32x32 patch size; (b) 64x64 patch size; (c) 128x128 patch size; Row (1) Overlapped Region- NIL;
(2) Overlapped Region- 4pi; (3) Overlapped Region- 8pi; (4) Overlapped Region- 12pi..

(a)

(b)

(c)

Fig.5. Texture Synthesis Results using tiles.as input, e.g. for Regular Texture
Column (a) 32x32 patch size; (b) 64x64 patch size; (c) 128x128 patch size; Row (1) Overlapped Region- NIL;
(2) Overlapped Region- 4pi; (3) Overlapped Region- 8pi; (4) Overlapped Region- 12pi.