APM

In recent years, IT projects seem to have stopped asking “which APM solution should we buy”, and have started asking “should we buy New Relic or AppDynamics?” Given the speed at which these two companies are innovating, the many product comparisons available on the web quickly become outdated. This comparison is a little more “high level”, so hopefully it will help give people some context for further research for several years to come. Please read some (recent) technical comparisons too, and take advantage of the free trial periods to install and try both products. […]

All performance testing tools that I know of are bad at collecting system metrics from Unix-like system (AIX, Solaris, Linux, BSD etc). If you are lucky you might (might!) be able to collect CPU utilisation and maybe one or two memory utilisation counters. Generally you have to create a shell script to run perfstat (or similar) then, when your load test is over, you process the output file and import it into your analysis tool. Yesterday I learnt that you can write whatever data you want to an SNMP MIB so, if your performance test tool is able to gather [...]

Following a load test, I often need to perform some additional analysis on the web server logs. It is not practical to use any commercial tools, and the free tools are all aimed at people who want graphs and statistics for entire weeks or months, rather than a few hours of high load. So far, I have usually been forced into a roll-your-own solution. While it is easy to create graphs with Microsoft Excel, it's 65536 row limit makes analysing any sort of non-trivial load futile unless the log files have been filtered before importing them. Even with filtering, it [...]

Some changes were made to a web application to reduce the amount of bandwidth used. To determine the impact of the changes, a previous benchmark test was compared with a new test. Both tests measured Bytes Received/sec on IIS, Bytes Sent/sec on IIS and Throughput (in Bytes/sec) from the perspective of the load testing tool. Comparing the tests showed that Bytes Received/sec had been reduced by 1/3 but the Total Throughput had been reduced by 2/3 (Bytes Received/sec was a trivially small value). Where were the missing bytes? Throughput: Displays the amount of throughput (in bytes) on the Web server [...]