MyLoadTest Blog

If you are running LoadRunner scripts from the command line using mmdrv.exe, then you will know that mmdrv.exe has a large number of command line options. Unfortunately, the options are not really documented apart from what is displayed in the user interface. Read on for some hints… […]

LoadRunner Analysis is a powerful tool for understanding exactly what happened during a load test. You can use it to slice and dice your performance test results data, then you can export selected graphs/charts for your Performance Test Summary Report. Some people may want to use a different tool to create charts from their LoadRunner test results. Fortunately, test results data is stored in a database, and there is an SQL query behind each graph/chart you see in LoadRunner Analysis. If you understand the LoadRunner Analysis database tables, you can write your own queries (unfortunately you can’t do this directly [...]

The idea of static code analysis has been around since at least the 1970s. These days, some kind of static analysis is usually built into most good quality IDEs (or it is available as a plug-in). Static analysis tools find bugs in software by looking at the program’s source code rather than executing the program. While the compiler will pick up syntax errors, there are thousands of other types of errors that a developer can accidentally create while programming – and, unlike compiler errors, there is nothing to tell you that you have made a mistake unless you are using [...]

If you are version controlling your VuGen scripts using Git, then you will want to prevent certain files from being added to your Git repository (e.g. replay logs, binary files outputted by the compiler, etc.) Adding a .gitignore file to your repository allows you to specify files or directories that should not be tracked by Git. I have created a LoadRunner-specific .gitignore file to get you started… […]

Since the release of Performance Center 11.0, the product has been based on the HP Application Lifecycle Management (HP ALM) platform (i.e. Quality Center). This architectural change means that functional testers and performance testers can share a combined testing platform – a single repository for requirements, test cases, and defects – giving full visibility across the functional, non-functional and performance testing areas. It sounds great, but there are pros and cons to combining Performance Center and ALM/Quality Center… […]

One of the most poorly understood VuGen Runtime Settings is the one that allows a performance tester to treat “non-critical resource errors as warnings”. The majority of LoadRunner users I have spoken to do not realise that enabling this setting can hide serious load-related defects in the system under test. LoadRunner automatically detects HTTP 4xx or 5xx responses but, if this runtime setting is enabled, LoadRunner will ignore the HTTP 4xx or 5xx error code when requesting a resource (i.e. the current transaction will not fail). […]

Whenever a team of performance testers grows larger than three people, then it is time to assemble the team and create some LoadRunner scripting standards (or guidelines if you prefer to be less strict). Documenting your team’s scripting standards helps junior performance testers avoid common mistakes and follow best practices. Experienced performance testers benefit from having consistency between scripts, so there is less confusion when work is transferred between team members. I have developed a short quiz so you can test your knowledge of some of the VuGen scripting best practices. […]

If you ever need to disguise a password in a VuGen script, you will no doubt have used the lr_decrypt() function. If you have stopped to think for a second or two, you will have realised “encrypting” the password in your script doesn’t make it more secure in any meaningful way. Anyone with access to the script can decode the password with a single line of code: // As a security measure, LoadRunner password encryption is about as effective as rot13. lr_output_message(“The secret password is: %s”, lr_decrypt(“50de7ec44d2ff7033c2fcb9e5cf7307799d7”)) So, if this feature doesn’t really make anything more secure, why was it [...]

A friend was performance testing a new SAP system that had a self-service portal where employees could view and download a PDF version of their payslip. He asked me how he could add a check to his LoadRunner script to make sure that the information in the PDF was correct (employee name, pay grade, etc). Manually checking all the PDFs at the end of the run was not a scalable solution (and just checking a sample of them would have missed any intermittent errors). Fortunately, it was a simple matter to write some code in VuGen that would solve his [...]

Since I wrote about my ideas for VuGen add-ins, a few people have spoken to me about their need for an add-in that would integrate VuGen with their revision control/source code management system. The interesting thing is that when I ask what version control features they are missing, they don’t have any specific features in mind (unless you count “backups”, which has nothing to do with version control). Most fall back on “VuGen scripts are source code, and all source code should be managed in a version control tool”. Well, VuGen 11.50 allows basic version control features (check in/out, version [...]

LoadRunner 11.50 was released on June 5th, during HP Discover 2012 in Las Vegas. This release was originally going to be called LoadRunner 12.00, but this would have put it out of step with the version numbers for ALM/Quality Center and QTP, which might have created some marketing confusion. Significant R&D effort has been spent on VuGen, with only bug fixes (or very minor functionality) for LoadRunner Analysis and the Controller. Those of us who participated in the Beta program have already been playing with the new VuGen for the last 9 months, and should be fairly familiar with these [...]

Every time someone uploads a script to the (web-based version of the) VuGen Validator, I get emailed a copy of the validation report for their script. Needless to say, I see a lot of really bad scripts…and hopefully the script developers learn something from the failing grade their scripts are given by the Validator. One thing that worries me is that lately I have seen a lot of BPM scripts containing the following code… Action() { web_cache_cleanup(); web_cleanup_cookies(); web_set_sockets_option(“CLOSE_KEEPALIVE_CONNECTIONS”, “1”); // script body removed from example. return 0; } […]

Imagine that you are load testing the website for a new loyalty card scheme. Millions of people will be mailed a loyalty card, and they will need to register their details online, along with their card number. Each card number may only be used once. So, every time you run your test, you will consume a large amount of card numbers, and you must be sure to use unregisterd card numbers for each test. […]

You might have already heard that it will be possible to add your own features to VuGen when version 11.5 is released mid-year. I will be talking about this in more detail in my “Extending VuGen 11.5 with Custom Add-ins” talk at HP Discover 2012 in June. In the meantime, I have decided to share some of my add-in ideas. Please feel free to develop these add-ins yourself. After all, ideas can’t be copyrighted, and I would prefer to buy or download someone else’s add-in rather than spend time developing them myself. Here are my ideas… […]

Recently a client asked me “What’s the difference between LoadRunner and Performance Center? They’re both sold by the same company, but they seem to do much the same thing.” I think he could see that I was about to launch into a half-hour lecture on the topic because he quickly added “Just give me the elevator pitch”. I thought for a second and explained “Well, obviously they are both used for performance testing, but Performance Center is the Enterprise version of LoadRunner; so it suits larger companies who need to run performance testing for multiple projects at the same time [...]

Imagine that you need to load test a Network Management System (NMS), which might receive hundreds of SNMP traps every second from tens of thousands of devices on the network. Clearly it is impractical to actually deploy thousands of devices in a test environment, so you need a tool to generate the SNMP traps at the required volume. A quick search for SNMP load testing tools shows that they are all fairly primitive, with poor support for creating SNMP traps with dynamic field values, and no support for integrated system monitoring. There are definitely no tools with the same level [...]

If you need to run a Unix command on a remote server with a LoadRunner script (running on Windows), then there is a neat program called plink that you will find very useful. If you use any variety of Unix, you will probably already by using PuTTY; well plink is is distributed with PuTTY, and allows you to run remote Unix commands from your DOS prompt. In this example, I have used it to collect disk I/O metrics from a remote Solaris server. […]

Everyone should already know that you can run a command-line program from a LoadRunner script using the system() function. If you have used this, you would already know that things become a little awkward when need to capture the output of the program. The common solution is to pipe the output of the command to a file (i.e. system(“my_command > output_file.txt”)), and then read the contents of the file. This is clunky and awkward, and not safe when multiple virtual users are writing to the same file. A much better solution is to run all your command-line (DOS) programs using [...]

On increasingly rare occasions, you might be required to load test an application that has a Java-based client. While it’s nice for users to have an application that has a rich GUI, it will probably lead to a world of pain for you as a performance tester. Don’t immediately reach for LoadRunner’s Java Record-Replay virtual user type. It might not be the right solution. This article will guide you through your first steps when selecting the correct vuser type to test an application with a Java-based client… […]

I like posting LoadRunner code snippets on My Load Test and, judging by the emails and comments that I get, a lot of people find them really useful. To make my code more readable, I have added syntax highlighting that uses the same colours you see in VuGen. If you would like to download a copy of the syntax highlighting code to use on your own blog or forum, or you would like to learn more about how I did it, read on… […]

LoadRunner 9.51 is a patch for for LoadRunner 9.50 which adds some minor functionality (including SiteScope 10.0 compatibility) and, according to the release notes, fixes bugs raised in 48 service requests. Readers interested in the major additions to the most recent major release of LoadRunner should read the Tech Tip on new features in LoadRunner 9.50. […]

You might have noticed that once you have started your load test, it is possible to increase the number of transactions per hour by adding virtual users, but not by changing the pacing time. The VuGen Pacing runtime setting is only read at the start of the test so, to change it, it is necessary to stop the test, then make the change and restart. Fortunately (if you need to), you can write code to control the pacing of your virtual users instead of using the runtime settings. If you write code to pick up a pacing setting from a [...]

There are a couple of options for testing a WebSphere MQ-based system using LoadRunner, but this blog post shows my preferred approach: write some simple Java code to put a message on a queue (and get a message from a queue). Note that this approach requires that you have a Java-based LoadRunner license such as the Java Tempate vuser or the Java Record-Replay vuser. You are not likely to need many virtual users, even if you want to generate a lot of messages. If you need to test a message-based system, you might also be interested in reading about testing [...]

Subject: HTML return value Hi Stu, you probably know the answer, what I need to know if there is an option in Vugen to run a url and to save the url content to a string, something like XML apis in Vugen. Later I can parse the string to see if I got the right value. There is surely an add-on to web_url to save the return value to a string. and I need to save it as a variable in Vugen and not writing it to a file. Tell me if you know if it achievable. Thanks Txxxx Here [...]

I frequently receive emails with LoadRunner-related questions. I try to reply to questions when I have time. Maybe posting my answers to MyLoadTest will be an incentive for me to answer more questions, and maybe it will save other people some time. Subject: Sending email from loadrunner script Hi Stuart, We would like to run a LoadRunner script (HTTP WEB Protocol) which will send an email to the user whenever the transaction response time for web page is more than the specified time, or when the application is down for some reason. Basically we would like to monitor the application [...]

Subject: LR JDBC Protocol Hi Stuart, I have a Java Swing-based client that communicates directly with a database server using JDBC. What are my options for recording this application with LoadRunner Thanks, Jxxxx Cxxxx. As you probably noticed, there is no native vuser type which will record your application successfully. Fortunately, you still have several options for recording your application. Option #1 - Winsock (a bad option) If you record your application with a Windows Sockets vuser, you will see uninteligible data being sent and received. Just because you can record something with a Winsock vuser doesn't mean it is [...]

The Virtual Table Server is great for most situations where your virtual users need a common data pool, but the limitations of the API mean that it is a bad fit in some cases. Say you need to find a row in the data table with a specific value. On a real database you could just do a simple SELECT statement. With VTS, you would have to write code to iterate through every row in the table and check the column value each time. So sometimes you need to use a real database. Interfacing with a real database is not [...]

Correlation is one of the fundamental LoadRunner scripting skills; and LoadRunner novices are usually not very good at it (which is expected), but people who think they are LoadRunner experts are sometimes not very good at it either. Mercury has done just about everything it can to remove the need for complicated correlation (correlation rules, the "scan script for correlations" option in VuGen, HTML mode recording, the Click and Script vuser type), but there will always be web applications that will require you to perform manual correlation when scripting. Mercury's training material kind of glosses over correlation, and makes it [...]

This is a quick walkthrough of a LoadRunner script I created to load test a "web" service that communicates with SOAP messages sent over JMS. Note that this script supports the queuing/point-to-point model of JMS (rather than the publish and subscribe model), works with JMS version 1.1 rather than 1.02b (as it uses the BytesMessage.getBodyLength() method among others), and has been tested with Java 1.5 using the Tibco implementation of JMS. The LoadRunner script can be downloaded here, the source code is available here, and a mirror of the JMS 1.1 specification can be downloaded here. People using Tibco's JMS [...]

A LoadRunner feature that has made my life a lot easier has been ContentCheck rules, which are available in the script runtime settings. If you are using a web-based vuser type, you can configure your LoadRunner script to search through all returned pages for strings that match known error messages. Using web_reg_find functions is fine, but when you get an error LoadRunner reports it as "failed to find text" instead of something more descriptive. I will always create rules for any error messages I find during scripting and, if I receive an error while running a scenario, I will add [...]

Imagine that you have to create a script for the "Create New User Account" business process. There are several constraints that will make your job difficult... The username field has the following properties: valid characters are a-z, A-Z, 0-9 usernames are not case sensitive. "Stuart" is the same as "stuart". usernames must start with a letter, not a number. usernames may be a maximum of 8 characters long. username must not be a duplicate Some other information about your testing: It is unclear when the database in the test environment will be refreshed, but it should happen at least yearly. [...]

When running a training or mentoring session, people often ask what runtime settings they should use; as if there is a magical list of settings that will always be correct for any testing situation. Obviously you select runtime settings that are appropriate for what you are trying to achieve with your test, but the funny thing is that there are actually a small list of settings that are usually appropriate for most situations. Read on... Whenever I am using a vuser type that allows multiple actions in a single script, I will create a separate action for each business process [...]

As of LoadRunner 8.1, Feature Pack 3, the following Vuser types are available (note that descriptions are provided by Mercury). Application Deployment Solutions Citrix_ICA Represent the Citrix ICA protocol communication between the Citrix client and server as GUI events. Client/Server DB2 CLI The IBM Call Level SQL Interface to the DB2 family of databases. Domain Name Resolution (DNS) A system which translates domain names into internet addresses. Informix IBM's Informix Database servers. MS SQL Server Microsoft's SQL Server using the Dblib interface. ODBC Open Database Connectivity - a protocol providing a common interface for accessing databases. Oracle (2-Tier) Oracle database [...]

The Virtual Table Server (VTS) is an in-memory database that can be used to share data between virtual users. I have found this useful when I need my VUsers to pass data to each other; it's certainly a much neater solution than writing to a file and having each VUser check the value in the file on every iteration. Here is what the documentation has to say about it... VTS Virtual Table Server (version 2.10) Virtual Table Server (VTS) first edition, introduced the capability for LoadRunner virtual users, WinRunner and XRunner, to communicate in real time. Data extraction and communication [...]

A common question on the LoadRunner support forums is "I've got an application that uses this new thing called AJAX. Will LoadRunner work with my application?" ...and the answer is "yes...but...". The "but" is a big one . An "AJAX" application can mean anything from a simple webpage that updates some fields with information from the server (without reloading the whole page), to a horribly complicated rich GUI interface created with JavaScript. Of course LoadRunner is going to be able to handle anything that is sent over HTTP, but scripting might not be the usual simple Record-Correlate-Playback, and the chances [...]

Useless trivia time... Back in 1983, Broderbund released a simple (and hugely popular) platform game called Lode Runner. Many years later, Mercury Interactive released a hugely popular load testing tool called LoadRunner. Notice that the animated figures look very similar. Do you think that this is subtle homage, or just a case of all stick figures looking alike?

There's a saying in software circles that "writing the first 90 percent of a computer program takes 90 percent of the time... The remaining ten percent also takes 90 percent of the time." A year ago, Mercury started telling people about their new GUI level virtual users for the web. Their promise was an impressive one - no more correlation. Anyone who has scripted for the web knows that correlating a web script can be either incredibly easy or painfully difficult if, for example, the web application constructs HTTP POSTs with an unholy mess of JavaScript. Mercury was promising to [...]

A question that regularly comes up in the Mercury Support forum is how to get a LoadRunner script to handle websites that require you to type in the characters from a picture of a funny-looking piece of text before you can perform an action such as creating a new account. "Surely someone has come across this before, and has figured out how to correlate these values?" Well...uh...no they haven't. These are Captchas - they are used to prevent bots from doing nasty things like signing up for thousands of Hotmail accounts or leaving comment spam. If it was possible to [...]

I don't ever like to feel that I am wasting a client's time by doing unnecessary load tests, but a long time ago I had an experience that altered my definition of what a necessary test is. I had come to this project after the scope had been decided and the Detailed Test Plan had been written. The overall system was made up of four applications; privately I queried why we were bothering to test ApplicationX, which was an exceedingly low volume application. ApplicationX was not business-critical or customer-facing. It was performing acceptably with a production-sized dataset and a small [...]

Okay, this is a bit of a tease because I haven't actually written this function, but it would be nice if someone out there would (maybe it will be me when I get some more free time). web_reg_save_param() is a useful little function for easily extracting text from the last HTTP response received by the virtual user. You specify left and right boundaries, the occurrence of the text you want to find and a large number of other options which, admittedly, don't give you the power of regular expressions but are a lot easier to use. The only problem is [...]

People don't generally read click-through licenses on the software they install. The only license I've read with a lawyer-like (or tester-like) attention to detail has been the GPL. The rest have just had a cursory check for any unduly onerous conditions...like the right to spy on my computer or the soul of my first-born child or something equally sneaky. The LoadRunner license has a couple of surprises in it. I'm not sure how enforceable the license would be, but it's probably in the best interests of anyone who uses Mercury tools that they are aware of the terms and do [...]

I think that the following question from the Mercury Support forum neatly illustrates one of the major problems with LoadRunner. Hi all, I have a financial application that submits the data in $ amounts. When I looked in the extended log the value is in number format like (43251) but during submission it is converted into dollar format $43,251.00. How to convert the string into $ format? Is there any function? Thanks in advance, The majority of people using LoadRunner aren't great programmers, and the ones who are aren't necessarily great programmers in C. Fortunately, the level of skill required [...]

Background I recently finished my first CORBA project using LoadRunner 8.0, and this is a quick brain-dump of some of my impressions. The official documentation does not contain all the information that I needed to know, and I found it hard to track down any people who had worked with the LoadRunner CORBA protocol before, so these notes are largely comprised of information that I wish I had known before I started. If you want to do performance testing of a Java client-server application using CORBA, LoadRunner seems to be the only viable option. CORBA is not supported by Rational [...]

When working with any kind of LoadRunner Java-based vuser type, any string that appears within the script may be parameterised. This differs significantly from all of the C-based vuser types, where only functions that are labelled as "parameterizable" may be parameterised without explicitly calling lr_eval_string(). This is a simple example of parameterisation with a Java-based script: lr.save_string("jumped", "myParam"); String message = "the quick brown fox <myParam> over the lazy dog"; lr.output_message(message); Rather than trying to do something that is theoretically impossible such as overloading the String class to handle this extra functionality, the LoadRunner developers did the simplest thing that [...]

LoadRunner uses WinDiff to compare recorded scripts while manually correlating. I always wondered why WinDiff would throw an error when I invoked it from LoadRunner. It turns out that this only happens if your script directory contains spaces (like the default LoadRunner script directory does). WinDiff is showing its age and is not too good with long filenames or filenames containing special characters.

LoadRunner Knowledge Base article 5218 "Generating Unique Names", posted by John Sawbridge, gives a function to generate a "unique" string [a-zA-Z] up to 20 characters long. The function accepts a string length argument and always writes the output to a parameter called {rname}. He notes "If you invoke the function successively in the same second, you will get the same name." Clearly this is a weak solution. As the function writes to the same parameter each time, it is not possible to use the function more than once in a LoadRunner script without overwriting your original value and if you [...]

This entry appears in the Mercury Interactive Knowledge Base as entry 5012. It was posted by me on 2003-11-30. Description It is easy to view the number and type of errors as they occur during your test from LoadRunner Controller. It is difficult to determine error information with the same level of detail after the test unless the errors are exported to a file before the Controller is closed. Solution All errors are written to a Microsoft Access file in the results directory (output.mdb). Knowledge Base article #4164 provides an executable that extracts this information in the same format as [...]