Meter mode setting : Helps in optimizing the load generation

by PATEL AKHILKUMAR on OCTOBER 3, 2011

Preface:
Oftentimes, we have observed that the Jmeter samples of load scenarios dont match with previous runs or dont match among different servers in standalone and client server environment (distributed) etc. The results/samples depend on the many parameters like SUT version, time of execution, data in back end, pacing of requests, think time between requests and iterations and jmeter server/client performance etc. But these situations are known and in general, load test engineers will try to setup and keep everything else consistent during all the test execution. However, of if you still notice differences in samples of hits per seconds, response time etc. during test execution from different systems, then you should first verify your load test setup. This blog will will help you to identify the initial steps for handling situations where you notice such differences:

Reduce sampling impact on load generation:

The load generation process can be impacted highly (or negligible) based on the usage of the memory, cpu, disk and network by the jmeter server and/or client. In all of these scenarios, the network traffic can impact load generation heavily if we are using distributed load execution scenario and system is sending all the samples to client immediately, depending on the network and network adapter traffic handling capacity as well as on the client system performance (which is obvious in all of the cases). Even jmeter sends the samples back to client as they are generated and client writes it to file on receipt as a default behavior means it uses network resources and client machine memory , disk and cpu for same. So, if client machines network or other resources performance are faulty, then it will impact on servers load generation activity adversely. To avoid this, set the sample sending mode appropriately and you will find more on mode as moving forward on this post. There are total 7 modes for sampling in JMeter: 1. Standard 2. Hold 3. Batch 4. Statistical

5. Stripped 6. StrippedBatch 7. Custom Implementation

Standard Default mode

This will send the samples to client as they are generated and client will write it to file on receipt in distributed environment. Same way it will try to write in file immediately in standalone environment. This will use less memory of load generating system but impact on load generation as the next requests will be sent after the samples sending is completed by network layer. So if the receiving clients network is slow then the sample sending will be slow and which can impact on requests sending to SUT. Still if the standard mode is required then try to reduce sampling interval means reduce the client server communication and sampling size. It is advisable to run small test in standalone mode and server mode and if you found difference then try to check the network and system performance.

Hold mode
In this mode, the sample data is held in an array in the memory and will be saved to file at the end of run by client after sending this data to client in distributed mode while it get saved at last in standalone environment (not advisable to use this mode in standalone if disk IO is performing well). But as you must have already guessed, it is a memory consuming mode and it can be a major cause for system crash . So, it is advisable tthink thrice before using this mode. For small load test it is among best load generation modes. Run small tests and watch the increase in memory usage to understand how risky it is to use this mode and how best to use this mode.

Batch mode
The system send samples in batch as per the defined threshold of samples or time. So set following property appropriately: num_sample_threshold number of samples in a batch (default 100) time_threshold number of milliseconds to wait (default 60 seconds) But sometimes it impact advertly if client network and system performance is not good.

Statistical mode
This mode is mainly for summarized sampling and do not samples all the fields. Also, the sampling rate is depends on the properties described with batch mode. The samples would be grouped as per thread group name and sample label. It accumulates only following fields and other fields that changes between samples will be ignored: Fields which will be accumulated are: 1. Elapsed time, 2. Latency, 3. Bytes, 4. Sample count and 5. Error count. This mode somewhat reduces the samples data impact over the network and will use very less resources of client as well in distributed environment. So it is advisable to setup effiecient thresholds after consideration of client system performance, network performance etc.

Stripped mode
This mode is nothing but standard mode without response data of successful samples.

StrippedBatch mode
This is nothing but the Batch mode without response data of successful samples.

Custom implementation mode

We can write our own custom sample sender class to customize the sampling behavior and set the mode parameter to this custom sample sender class name. This must implement the interface SampleSender and have a constructor which takes a single For example:
public class PdfResponseSampleSender implements SampleSender, Serializable { //The code will just show the basic implementation and may be missing some required implementation to fit in standard practice. private static final Logger h log pave = a LoggingManager.getLoggerForClass(); //SampleSender

parameter

of

type

RemoteSampleListener.

constructor

which

takes

single

parameter

of

type

RemoteSampleListener. So following will be used for that. private final RemoteSampleListener PdfResponseSampleSender(RemoteSampleListener listener; listener)

{ this.listener = listener; } //It will be fired while sample started and stopped public void sampleOccurred(SampleEvent event) { //Basically it is advicable to create enum of mime types. but here we just using it directoly the { { in string variable type String pdfType="application/pdf"; is byte[0]); } catch not } pdf. try e) SampleResult result = event.getResult(); //Empty the response data if media if(result.getMediaType().compateTo(pdfType)!=0) result.setResponseData(new listener.sampleOccurred(event); (RemoteException

{ log.error("Error sending sample result over network ",e) } } }

We can set mode=org.example.load.PdfResponseSampleSender into jmeter.properties and make sure that this class is reachable to jmeter.

Conclusion:
I would say that we can resolve any load generation issue easily as jmeter is very flexible and supports customization for most of the activities of L & P testing. If you have any questions, please post those in the comments below, and I will try to answer those as soon as I can! PS: All opinions in this post are my personal and it may or may not be of Infostretch. Also, I have taken help of JMeter user manual and source code for this blog post.

Regular Expressions
20.1 Overview JMeter includes the pattern matching software Apache Jakarta ORO There is some documentation for this on the Jakarta web-site, for example a summary of the pattern matching characters There is also documentation on an older incarnation of the product at OROMatcher User's guide , which might prove useful. The pattern matching is very similar to the pattern matching in Perl. A full installation of Perl will include plenty of documentation on regular expressions - look for perlrequick, perlretut, perlre, perlreref. It is worth stressing the difference between "contains" and "matches", as used on the Response Assertion test element:

"contains" means that the regular expression matched at least some part of the target, so 'alphabet' "contains" 'ph.b.' because the regular expression matches the substring 'phabe'. "matches" means that the regular expression matched the whole target. So 'alphabet' is "matched" by 'al.*t'.

In this case, it is equivalent to wrapping the regular expression in ^ and $, viz '^al.*t$'. However, this is not always the case. For example, the regular expression 'alp|.lp.*' is "contained" in 'alphabet', but does not match 'alphabet'. Why? Because when the pattern matcher finds the sequence 'alp' in 'alphabet', it stops trying any other combinations - and 'alp' is not the same as 'alphabet', as it does not include 'habet'. Note: unlike Perl, there is no need to (i.e. do not) enclose the regular expression in //. So how does one use the modifiers ismx etc if there is no trailing /? The solution is to use extended regular expressions , i.e. /abc/i becomes (?i)abc. See also Placement of modifiers below. 20.2 Examples

Extract single string

Suppose you want to match the following portion of a web-page:
name="file" value="readme.txt">

and you want to extract readme.txt . A suitable regular expression would be:
name="file" value="(.+?)">

The special characters above are:

( and ) - these enclose the portion of the match string to be returned . - match any character + - one or more times ? - don't be greedy, i.e. stop when first match succeeds

Note: without the ?, the .+ would continue past the first "> until it found the last possible "> - which is probably not what was intended. Note: although the above expression works, it's more efficient to use the following expression:
name="file" value="([^"]+)">

where

[^"] - means match anything except " In this case, the matching engine can stop looking as soon as it sees the first " , whereas in the previous case the engine has to check that it has found ">rather than say " > .

Extract multiple strings

Suppose you want to match the following portion of a web-page:
name="file.name" value="readme.txt" both file.name and readme.txt .

and you want to extract

A suitable reqular expression would be:

name="([^"]+)" value="([^"]+)"

This would create 2 groups, which could be used in the JMeter Regular Expression Extractor template as $1$ and $2$. The JMeter Regex Extractor saves the values of the groups in additional variables.

20.3 Line mode The pattern matching behaves in various slightly different ways, depending on the setting of the multi-line and single-line modifiers. Note that the single-line and multi-line operators have nothing to do with each other; they can be specified independently.

Single-line mode
Single-line mode only affects how the '.' meta-character is interpreted. Default behaviour is that '.' matches any character except newline. In single-line mode, '.' also matches newline.

Multi-line mode
Multi-line mode only affects how the meta-characters '^' and '$' are interpreted. Default behaviour is that '^' and '$' only match at the very beginning and end of the string. When Multi-line mode is used, the '^' metacharacter matches at the beginning of every line, and the '$' metacharacter matches at the end of every line. 20.4 Meta characters Regular expressions use certain characters as meta characters - these characters have a special meaning to the RE engine. Such characters must be escaped by preceeding them with \ (backslash) in order to treat them as ordinary characters. Here is a list of the meta characters and their meaning (please check the ORO documentation if in doubt).

( ) - grouping [ ] - character classes { } - repetition * + ? - repetition . - wild-card character \ - escape character | - alternatives ^ $ - start and end of string or line

Please note that ORO does not support the \Q and \E meta-characters. [In other RE engines, these can be used to quote a portion of an RE so that the meta-characters stand for themselves.] The following Perl5 extended regular expressions are supported by ORO. (?#text) An embedded comment causing text to be ignored. (?:regexp)

Groups things like "()" but doesn't cause the group match to be saved. (?=regexp) A zero-width positive lookahead assertion. For example, \w+(?=\s) matches a word followed by whitespace, without including whitespace in the MatchResult. (?!regexp) A zero-width negative lookahead assertion. For example foo(?!bar) matches any occurrence of "foo" that isn't followed by "bar". Remember that this is a zero-width assertion, which means that a(?!b)d will match ad because a is followed by a character that is not b (the d) and a d follows the zero-width assertion. (?imsx) One or more embedded pattern-match modifiers. i enables case insensitivity, m enables multiline treatment of the input, s enables single line treatment of the input, and x enables extended whitespace comments. Note that (?<=regexp) - lookbehind - is not supported. 20.5 Placement of modifiers Modifiers can be placed anywhere in the regex, and apply from that point onwards. [A bug in ORO means that they cannot be used at the very end of the regex. However they would have no effect there anyway.] The single-line (?s) and multi-line (?m) modifiers are normally placed at the start of the regex. The ignore-case modifier (?i) may be usefully applied to just part of a regex, for example:
Match ExAct case or (?i)ArBiTrARY(?-i) case

Perl5 regular expressions

Here we summarize the syntax of Perl5.003 regular expressions, all of which is supported by the Perl5 classes in this package. However, for a definitive reference, you should consult the perlre man page that accompanies the Perl5 distribution and also the book Programming Perl, 2nd Edition from O'Reilly & Associates. We are working toward implementing the features added after Perl5.003 up to and including Perl 5.6. Please remember, we only guarantee support for Perl5.003 expressions in version 2.0.

Alternatives separated by | Quantified atoms

{n,m} Match at least n but not more than m times. {n,} Match at least n times. {n} Match exactly n times. * Match 0 or more times. + Match 1 or more times. ? Match 0 or 1 times.

Atoms
o o o

o o

regular expression within parentheses a . matches everything except \n a ^ is a null token matching the beginning of a string or line (i.e., the position right after a newline or right before the beginning of a string) a $ is a null token matching the end of a string or line (i.e., the position right before a newline or right after the end of a string) Character classes (e.g., [abcd]) and ranges (e.g. [a-z]) Special backslashed characters work within a character class (except for backreferences and boundaries). \b is backspace inside a character class Special backslashed characters

\b null token matching a word boundary (\w on one side and \W on the other) \B null token matching a boundary that isn't a word boundary \A Match only at beginning of string \Z Match only at end of string (or before newline at the end) \n newline \r carriage return \t

tab \f formfeed \d digit [0-9] \D non-digit [^0-9] \w word character [0-9a-z_A-Z] \W a non-word character [^0-9a-z_A-Z] \s a whitespace character [ \t\n\r\f] \S a non-whitespace character [^ \t\n\r\f] \xnn hexadecimal representation of character \cD matches the corresponding control character \nn or \nnn octal representation of character unless a backreference. a \1, \2, \3, etc. match whatever the first, second, third, etc. parenthesized group matched. This is called a backreference. If there is no corresponding group, the number is interpreted as an octal representation of a character. \0 matches null character Any other backslashed character matches itself

Expressions within parentheses are matched as subpattern groups and saved for use by certain methods.

By default, a quantified subpattern is greedy . In other words it matches as many times as possible without causing the rest of the pattern not to match. To change the quantifiers to match the minimum number of times possible, without causing the rest of the pattern not to match, you may use a "?" right after the quantifier. *? Match 0 or more times +?

Match 1 or more times ?? Match 0 or 1 time {n}? Match exactly n times {n,}? Match at least n times {n,m}? Match at least n but not more than m times Perl5 extended regular expressions are fully supported. (?#text) An embedded comment causing text to be ignored. (?:regexp) Groups things like "()" but doesn't cause the group match to be saved. (?=regexp) A zero-width positive lookahead assertion. For example, \w+(?=\s) matches a word followed by whitespace, without including whitespace in the MatchResult. (?!regexp) A zero-width negative lookahead assertion. For example foo(?!bar) matches any occurrence of "foo" that isn't followed by "bar". Remember that this is a zero-width assertion, which means that a(?!b)d will match ad because a is followed by a character that is not b (the d) and a d follows the zero-width assertion. (?imsx) One or more embedded pattern-match modifiers. i enables case insensitivity, m enables multiline treatment of the input, s enables single line treatment of the input, and x enables extended whitespace comments.

Ajax Testing with Jmeter

Initially when I tried to do Load testing with Ajax I found that it was not an easy job to do, especially when DWR concept is involved, I didn't find any good solution on web, so decided to dig in and give it a try on my desk top and found that handling Ajax with DWR in Jmeter is pretty simple but not that easy. Here I am assuming that Ajax is implemented using DWR. Before start recording you need to consider which version of DWR is used, based on the version of DWR we need to choose the recording approach. This approach will vary with DWR 1.3 and above 1.3

So considering the steps involved for DWR version 1.3 1. Record the scripts as you do normally. 2. Have the listener "View result tree" which will help in debugging the scripts.

Fig 1 From the above figure we can see that few of the DWR calls have been captured. One thing to be noticed is that right hand panel shows Name/Value pair and the method used is "POST". The value captured is in single line and initially default to understand what it is. NOTE: - Dont edit the value from Jmeter UI, It will screw up the scripts. We need to edit it in JMX file, Use Text Pad or any other editor to edit the JMX file. After editing notice that each DWR calls will have unique id, this ID will differ for each run. c0-id=323_1456789521 Handling this id in such a way that new value will be generated for each run. In order to do this generate random numbers, We can see that id is split in to 2 parts separated by "_" so for first 3 digit generate 1 random number and for last 10 digit generate another random number. After modification it will look some thing like this. c0-id=${dwr1}_${dwr2} Make sure you edit in JMX file. In Order to generate the Ramdom numbers use "Random Function". This function looks like this ${__Random(100,400,test)} for dwr1 variable ${__Random(100000000000,200000000000,test1)} for dwr2 variable. In order to access these value use an "User Defined Variable" you can see in the below figure

Fig 2 dwr1 and dwr2 is the variable name which should be used to generate the id value. Now save the Jmx file and restart the Jmeter. when u run the script 1st DWR call will run with out any problem (figure 1 you can see that there are 3 DWR calls recorded) but fallowing DWR calls may fail some times, If you face the problem for running the next DWR calls, You need to change the content type of the request to "text/plain". In order to do these add the "HTTP Header Manager" config element and set the content type to text/plain Name field=Content-Type value=text/plain

Fig 3 If you fallow above instruction you can easily handle the Ajax calls with DWR version 1.3 For DWR version greater than 1.3 needs to be handled in little bit different way 1. Record the scripts for the pages where Ajax is implemented. 2. Once recording is completed you should be able to see the request some thing like xyz/dwr/engine.js, if you don't find this request then close the browser and open it newly and start recording again. This request will only be recorded when new browser is opened. 3. This version of DWR uses two session variable JSESSIONID and SCRIPTSESSIONID

You need to capture these values and substitute it in JMX file where ever it is being used. In order to do this, add 2 RegularExpressionExtractor under dwr.engine.js request For Jsessionid Reference Name: jsessionid Regular expression: dwr.engine._origJSessionId = "(.*)"; You can get this jsessionid from cookie manager also. For ScriptSessionid Reference Name: scriptsessionid Regular expression: dwr.engine._origScriptSessionId = "(.*)"; Above things is w.r.t DWR 2.0 version. 4. Now use these variable to substitute the value and run the script. Tip:-If you are not able to record any of the DWR calls then use HTTP Analyzer or any tool and create your own HTTP request and method should be POST.
Posted by Pramod at 4:28 AM

JMeter Ant Task

We all know that the word Automation means, there should not be any human intervention for the task. Similarly w.r.t Jmeter we can achieve this by integrating with the ant build process. When ever a new code is checked in and build has happened then Jmeter should kick off immediately, run the scripts and produce the performance report, So now the question is how to do this? Answer is simple, 1.To use the task, you must have JMeter installed. You must also include ant-jmeter-1.0.9.jar in your Ant classpath. Adding the jar to $ANT_HOME/lib will make this happen automatically. 2.Set the jmeterhome parameter to your JMeter install location, and the resultlog parameter to the name of a file to log the test results to. You can either specify a single test plan using the testplan parameter, or multiple test plans using the testplans nested element. The testplans element is a standard Ant FileSet element. This is an Ant task for automating running JMeter test plans. The task executes one or more JMeter test plans, and logs the results to a file. Start by defining the task to make it available to your build script: <taskdef name="jmeter" classname="org.programmerplanet.ant.taskdefs.jmeter. JMeterTask"/>

Set the jmeterhome parameter to your JMeter install location, and the resultlog parameter to the name of a file to log the test results to. You can either specify a single test plan using the testplan parameter, or multiple test plans using the testplans nested element. The testplans element is a standard Ant FileSet element. <jmeter jmeterhome="c:\jakarta-jmeter-1.8.1" testplan="${basedir}/loadtests/JMeterLoadTest.jmx" resultlog="${basedir}/loadtests/JMeterResults.jtl"/> <jmeter jmeterhome="c:\jakarta-jmeter-1.8.1" resultlog="${basedir}/loadtests/JMeterResults.jtl"> <testplans dir="${basedir}/loadtests" includes="*.jmx"/> </jmeter> Optional JMeter arguments supported include specifying an alternate jmeter properties file (jmeterproperties), running remote servers specified in jmeter properties file (runremote), and running the tests through a proxy or firewall (proxyhost, proxyport, proxyuser, proxypass). Setting the failureProperty attribute will set the specified property to "true" in the event of a JMeter test failure. This gives you the opportunity to take further action such as send an email or fail the and build. You can override JMeter properties (instead of modifying jmeter.properties) like this: <jmeter jmeterhome="c:\jakarta-jmeter-1.8.1" testplan="${basedir}/loadtests/JMeterLoadTest.jmx" resultlog="${basedir}/loadtests/JMeterResults.jtl"> <property name="request.threads" value="1"/> <property name="request.loop" value="10"/> </jmeter> You may also specify additional JVM arguments to the JVM launched to run JMeter. Here is an example of how to specify JVM arguments: <jmeter jmeterhome="c:\jakarta-jmeter-1.8.1" testplan="${basedir}/loadtests/JMeterLoadTest.jmx" resultlog="${basedir}/loadtests/JMeterResults.jtl"> <jvmarg value="-Xincgc"/> <jvmarg value="-Xmx128m"/> <jvmarg value="-Dproperty=value"/> </jmeter> You can find XSLT file in "Jmeter\extras" folder jmeter-results-report.xsl, for generating a summary report from the result log file. The summary report is very similar to the default report created by the junitreport task. You can use the xslt task to create the report: <xslt in="${basedir}/loadtests/JMeterResults.jtl" out="${basedir}/loadtests/JMeterResults.html" style="${basedir}/loadtests/jmeter-results-report.xsl"/>

Note: If you are using JMeter 2.1 or later, you must use the new xslt stylesheet(s) included in the JMeter extras directory. The new stylesheets have been modified to support the new JMeter log file format. If you would like failure detail messages in the report output, you must configure JMeter to output that information to the result log. To do this, set the following property in your jmeter.properties file before running the test plans: jmeter.save.saveservice.assertion_results=all Note: As of JMeter 1.9RC2(?), the default results output format is now csv. It must be changed to xml in order to use the xslt task to create the html report: jmeter.save.saveservice.output_format=xml There is also another XSLT file that was contributed which generates an enhanced report that includes expandable details.

The report will look something like this:

Parameters
Attribute jmeterhome testplan resultlog resultlogdir Description JMeter install location. The location of the test plan file. The location of the result log file.

The directory to place result log files. When used, result log file names will match the test plan names, with the extension renamed from .jmx to .jtl. failureproperty The name of a property to set to "true" in the event of a test plan failure. jmeterproperties The location of an alternate jmeter.properties file to use. runremote If "true", runs remote servers specified in jmeter.properties. Default is "false". proxyhost Host name of a proxy to run the tests through. proxyport Port of the proxy host specified.

proxyuser proxypass Element testplans property jvmarg jmeterarg

Username of the proxy host specified. Password of the proxy host specified.

Nested Elements

Description Use instead of testplan attribute when you want to specify multiple test plan files. This element is a s Use to specify additional JMeter properties (instead of modifying jmeter.properties file). Attributes in Use to specify JVM arguments to the JVM launched to run JMeter. The only attribute is value. Use to specify additional JMeter command line arguments. The only attribute is value.
Links to this post

Posted by Pramod at 10:48 PM 0 comments

Sunday, April 12, 2009

JDBC Testing with Jmeter

Data base testing with Jmeter is simple. Here I am using Jmeter2.3 and Oracle 10g. Some of the things needs to be taken care before start testing. 1. Place the JDBC JAR file in --> Jmeter\lib folder (For Oracle 10g it will be classes12.jar) 2.Need to know the JDBC Driver class, If you don't know where to find the Driver class information, Open the Jar file using winzip or any tool for that matter and edit the OracleDriver.class in text file and there you can find the JDBC Driver Class. ( oracle.jdbc.driver.OracleDriver) 3. Create Test Plan and add thread group. 4. Add the Config Element-->JDBC Connection Configuration and fill in all the required information, It should look like below figure.

Fig 1 5. From the above figure notice that Variable name is highlighted, what ever name you give here will be shared with the "Jdbc request default". In my example I have given variable name as "SQL". Provide the JDBC Drive Class information which you got from OracleDriver.class file. 6. Add the Config element "JDBC Request"

Fig 2 7. Give the variable name as "SQL" this name should match with "JDBC Connection Configuration" variable name. 8. If you are executing select statement, Update statement chooses appropriate Query type and put your query there. 9. Add view result tree for debugging purpose and run the script. Hope this information will help!!!!