Question: Monitoring and alerting with VuGen?

I frequently receive emails with LoadRunner-related questions. I try to reply to questions when I have time. Maybe posting my answers to MyLoadTest will be an incentive for me to answer more questions, and maybe it will save other people some time.

Subject: Sending email from loadrunner script

Hi Stuart,

We would like to run a LoadRunner script (HTTP WEB Protocol) which will send an email to the user whenever the transaction response time for web page is more than the specified time, or when the application is down for some reason. Basically we would like to monitor the application continuously. How can we achieve this using LoadRunner? Or can you suggest the best solution for monitoring application which would send email notifications to the monitoring team.

Thanks,
Rxxx

The best solution for monitoring a web application’s availability and performance is going to be HP Business Availability Center (BAC). BAC has a component called Business Process Monitor (BPM). BPMs are VuGen scripts that create synthetic transactions against your application. BAC handles the alerting, collection of historical data, and dashboard functionality.

The advantage of using BPMs as well as traditional infrastructure-based monitoring is that a BPM will tell you if your web application is really available because it is performing a real business process. Pinging a server every 5 minutes doesn’t tell you that the database service has hung. Requesting just the front page of the website every 5 minutes doesn’t tell you that all of your online store functions are working except the “checkout” function.

As well as alerting on response times which are outside of SLAs, capturing response time data shows historical performance trends – maybe you will see a jump in response times for searches after all those extra products were added to the catalog? Or maybe a slow degradation in response times as more and more old orders sit in the database?

But let’s assume that you are unable to convince your management that they should do things the correct way, and they decide to go with the unreliable but cheap solution of monitoring an application with a copy of VuGen which is running constantly on a workstation.

Problem #1 – stability of the monitor over time

Using VuGen by itself as a monitor is a solution you would have to babysit. You would probably want to restart it every day as logs will accumulate and eventually start slowing everything down.

Problem #2 – accuracy of response time measurements

As well as the possibility of very large logs distorting your response time measurements by slowing down VuGen, running a script in VuGen is far slower than running the same script through the LoadRunner agent (mmdrv.exe which is used by LoadRunner Controller and by BAC). The overhead can be reduced by disabling “Animated Run” and “Output Window”, but there should still be a significant difference.

Problem #3 – historical data

I don’t generally like collecting data if I am not going to keep a copy of it somewhere. I like to see trends over time. Extracting data out of the logs would probably be painful so, if I were doing this, I would probably write some extra code to write to a CSV file so that I could graph the data in Excel.

Problem #4 – email alerting on slow response times

If you have licensed the SMTP vuser type, then you can create a Multiple Protocol (Web + SMTP) script so that you can send emails with your script. Alternatively you might be able to find a command line tool that can send email (like Blat), and call it with the system() function; or you could send an email using a web-based interface – maybe set up a private web service on a completely separate server.

Actually detecting that the response time for a transaction does not meet your SLA would require you to set a timer at the start of each transaction, and write some conditional logic at the end of each transaction e.g. if ((endTime – startTime )> SLA) { raiseAlert(); }. Using BAC is obviously a much more robust way of doing this, and eliminates all of the extra code you would have to write.

Problem #5 – alerting on errors

Alerting on slow response times is reletively simple, but alerting on errors becomes much more complicated with my hacky VuGen solution. The problem is that control is passed from your code back to VuGen and its Run Logic as soon as you fail a Content Check Rule or a web_reg_find. The only way to get around this is to enable the “Continue on error” runtime setting and do all of your error handling manually. This will mean writing big ugly chunks of code like this for every request that you want to check…

lr_continue_on_error(LR_ON_ERROR_CONTINUE);

// Check for known error messages
web_reg_find("Text=Your session has expired", "SaveCount=errorSessionExpired", LAST);
web_reg_find("Text=A runtime error occurred", "SaveCount=errorRuntimeError", LAST);

// Check for message to indicate success (positive text check)
web_reg_find("Text=Your order has been submitted successfully", "SaveCount=checkOrderSubmitted", LAST);

web_submit_data(...); // snip!

if (atoi(lr_eval_string("{errorSessionExpired}")) > 0) {
    sendEmailAlert("Error: Session expired for user %s", lr_eval_string("{UserName}"));
} else if (atoi(lr_eval_string("{errorRuntimeError}")) > 0) {
    sendEmailAlert("Error: Runtime error for user %s", lr_eval_string("{UserName}"));
} else if (atoi(lr_eval_string("{errorSessionExpired}")) > 1) {
    sendEmailAlert("Error: Text Check failed for \"Your order has been submitted successfully\".");
}