HP Software Universe 2010 (recap)

I just got back from the US after presenting at HP Software Universe 2010. I will save the content of my presentation for another time, so this is a quick overview of what I saw at the Conference and some of the things I learned from the people who attended.

Conference Overview

HP really know how to run a conference. The Gaylord Hotel in Washington D.C. was a great venue, and the event was well catered and superbly organised. The format was perfect; each session was 1 hour long (including questions), and there was always at least a 30 minute gap between sessions, which was perfect for talking to the presenter or other people in the session who seemed to have something to contribute.

A nice change from last year’s conference was that there were monitors everywhere showing the latest Twitter posts with the #HPSWU hashtag.

Conference Sessions

There were lots of interesting sessions, and there was usually more than one presentation I would have liked to see in each timeslot. I was mostly interested in the presentations in the Performance Validation track, so here they are in chronological order…

JCPenney: successful implementation of a performance center of excellence on HP Performance Center (1665)

Someone from JCPenney was loudly trying to attract people to this session by promising that an iPad would be given away at the end. “Center of Excellence” was on my Bullshit Bingo card, so I passed on the chance to win a cool device that I would use for about 5 minutes, and went to the Digital Marketing session instead.

Taking the mystery out of digital marketing (2420)

Kathy Malchi from Allegro gave a great overview for people who want to connect with potential customers through their company website (she also plugged Zift’s HP Content Syndication service, but that is a topic for another time).

Useful takeaways were:

• The idea of “touch” in sales/marketing, with each contact (email, phone call, newsletter) counting as a touch. It usually takes 15 touches before you get a result.
• 70% of customers find you, rather than you make the first contact.
• Think why you are doing SEO. If it is to generate leads, then optimise for this.
• An audience member who used their website to generate leads generally got 4-6 leads per month through the site.
• Useful tools: Google Webmaster Central has a feature that will give you SEO suggestions for your site.
• Great story about someone who landed their dream job by buying Google ads for the names of people who worked at the company. Someone did a vanity search and clicked through to his resume. Landing his dream job cost $6 in advertising.
• Sales guys aim to be a “trusted advisor”. Marketers should aim to the the “trusted educator”.
• Potential customers want to be prepared before talking to a sales guy. Put content online that is less sales focussed, and more about answering their questions.
• A/B split testing: can have 2 different Google ads. Google will tell you which is more effective at getting people to click through.
• Bad websites: can someone go to your site and find out what your company does (or do you have something fuzzy like “we provide solutions”).
• “Solutions” is where good product companies go to die.
• Email Marketing: success is weighted by the following factors: 60% list quality (is it targetted, or did you just buy a list of email addresses from someone), 25% offer/call to action, 15% copy (text in the email).
• Email open rate is determined by combination of FROM + SUBJECT. Click through rate is determined by the OFFER. Conversion rate is determined by the landing page.
• Make sure emails have a mobile view, because people use their iPhone to view emails.
• Lead nurturing is important, because they will buy from someone in the next 18 months.
• Social media: Youtube video from disgruntled United Airlines customer went viral. See: United Breaks Guitars (8 million views).

Southwest Airlines: using HP Diagnostics to drive value (1187)

I enjoyed this presentation by Subuhi Ali from Southwest, because I have spent a lot of time using HP Diagnostics, and it is great to hear about someone else’s experiences. I had also helped Subuhi in the past by sending her some example code, so it was nice to meet her in real life.

• Bypass a painful change control process by putting the Diagnostics software and configuration files on a share drive (neat idea for test environment you own, not so cool for Production environments). This also allows you to make config changes without having RDP or SSH access to servers.
• Subuhi generally estimates 2 hours – 1 day to configure Diagnostics (interesting, because this is very different to my experience where fully configuring Diagnostics can take weeks – especially when you start writing code snippets).
• Had some nice examples of problems that the development team had tried to solve for a long time, that were solved very quickly with Diagnostics (“A problem that the developer was looking at for 2 days was resolved in 2 minutes”).
• Subhi really loved the Diagnostics integration with LoadRunner Analysis, which I am not a fan of as it gives much less information than the historical view in the Diagnostics Server ().
• Southwest Airlines looks like a fun place to work. 🙂
• If you have to use Diagnostics, you might also be interested in my articles on Installing HP Diagnostics and Scaling HP Diagnostics.
• Some parts reminded me of my Mercury World 2006 presentation on Diagnostics for SAP

Seagate Technologies: an insider’s guide to best practices for establishing a Performance Testing Center of Excellence (1322)

This session clashed with the Diagnostics presentation above.

Thomson Reuters: using HP software and WAN Emulation to test the performance of your applications under real-world network conditions (1474)

This session clashed with the “Center of Excellence” presentation below.

Drinking our own Merlot: how HP Software’s Performance Center of Excellence (PCoE) team uses Performance Center for testing in an agile environment (1648)

Lior Reuven from HP

• Nothing much in this presentation was actually related to Agile, but that didn’t really matter.
• Lior’s key components of Performance Testing:
  1. Test early
  2. Risk comes first
  3. Built the right performance team
  4. Test early (mentioned twice because it is so important)
• Performance test

InterContinental Hotels Group: how we collaborate with our operations teams to get comprehensive information and set the stage for better performance testing (1594)

???

• An audience member through he was asking a great question about how the

Oracle: capturing data from production-monitoring tools to build realistic load tests that reflect production usage (1421)

Alex
TODO

HP: running massive load tests via HP Software-as-a-Service (1720)

Missed this one as it clashed with Alex’s presentation.

Announcement: HP LoadRunner in the Cloud (1760)

This session was presented by Priya Kothari (LoadRunner Product Marketing), Mark Tomlinson (LoadRunner Product Manager). Minimal notes as this is in Beta (no NDA was signed).

• Uses Amazon EC2 cloud, so can scale up to huge numbers of concurrent vusers.
• Will probably be a great fit for customers who want to do infrequent, very high volume load tests of applications on the public Internet (i.e. not intranet apps).
• Is currently in Beta (US customers only), so you cannot buy this right now.
• Best to just check out Mark Tomlinson’s video here: HP LoadRunner Amazon Cloud View
• Licensing model undecided (existing LoadRunner Term and VUD licenses are not a good fit for this model), but The Cloud suits a “billing by usage” model with fine granularity (think “Virtual User Hours”).
• Undecided on whether HP Partners will be able to re-sell this.
• I would imagine that this will make Performance Center SaaS irrelevant.
• I think that monitoring may be painful to set up as sysadmins don’t really like opening their servers up to the Internet.

HP Software & Solutions: HP LoadRunner and HP Performance Center roadmaps (1761)

I didn’t sign an NDA for this session, but I did sign one to be part of the Beta Program, so minimal details for this one (contains forward-looking statements that may change etc etc)…

• I loved Mark Tomlinson’s Guiding Statments:
  1. Performance Testing is not for beginners or newbies
  2. Performance validation is not enough (need to be more useful than just saying “it is too slow”)
  3. Performance Testing can be done much sooner
• Major changes in the upcoming version. Expect something better than Click and Script.
• As HP is typically a bit slow to release new versions of their training materials, there is probably a market opportunity for someone to develop LoadRunner 11 training (my thoughts, not something Mark said).

Expedia: extreme performance for extreme Web services: taking load testing to the next level (1520)

This session was presented by Rex Black (the other Rex Black; not the one who has written books on software testing, and used to be the president of the ISTQB and the ASTQB), who was very knowledgable but, unfortunately, does not blog or use Twitter.

• Lots of good info for people testing web services (people should also read my Testing Web Services With a Standard Web Vuser article).
• web_service_set_security() – adds security tokens etc to future SOAP requests. Becoming more common.
• Rex has a whitepaper available on some of the undocumented functions available in LoadRunner:
  • util_base64decode() – is in micsocket.dll. Must include micsocket.h and call lr_load_dll before using.
  • uncompress() – an unzip function, found in zlib.dll.
  • It would make a really cool future blog post to use a tool (Dependency Walker, dia2dump – if the DLLs have PDBs, or PE Explorer) to find all the undocumented functions in LoadRunner and publish a full list.

ING Direct: how we do performance testing in agile environments and bridge the gaps between IT teams (1543)

???

Digital River: predictive performance testing—integrating statistical tests into Agile development lifecycles (1431)

???

MKS Instruments: validating the performance impact of migrating from Oracle E-Business to Oracle RAC and Linux (1653)

Unfortunately clashed with the above presentation.

Tabcorp: communicating performance test results to stakeholders (1823)

• This was the session that I co-presented with Peter Dunkinson from Tabcorp.
• I swear that it was more interesting than the title makes it sound. The working title for the presentation was Communication Skills for LoadRunner Nerds (PDF download).
• I will put the rest of the content from this presentation online soon.

Note that the photos is from before the presentation, not during. 🙂

Tufts Health Plan: test strategies for performance testing complex applications on a immutable schedule (1202)

TODO

Other Conference Highlights

LoadRunner BBQ

Well known LoadRunner community member James Pulley organised a meetup of various LoadRunner folks at a bar near the conference hotel (the original plan was to have a BBQ, but this didn’t happen). It was great to have a beer (and talk geeky with) Scott Moore, Tim Chase, James Pulley, Mark Tomlinson, Andy Bearsley, Peter Dunkinson, and a whole bunch of other expert Performance Testers.

Performance Engineering Roundtable Session

• Support for IE 8 is available in LoadRunner 9.5 as a patch (not in a service pack). If you raise a support ticket, they will give you a link to download the patch.

Keynote Sessions

I think I would have appreciated another couple of track sessions, rather than the keynote sessions. The HP people were a bit too “big picture” and fuzzy on concrete details, and the “general interest” speakers (an Olympic silver medalist, and a journalist) were both people I had never heard of.

My nomination for Keynote Speaker next year is Steve Wozniak (who is an ex-HP employee, and) who should have a story or two that will interest anyone who works with technology.

Trade Booths

Packed up on Thursday at lunchtime

TODO
HP Test Data Management – http://www.hp.com/go/tdm
JDS does no business in the American market,

The Aftermath

Washington DC is not exactly a cultural hub. It’s got some nice museums (the National Air and Space Museum was pretty special), but not much in the way of fun things to do after dark.

New York was much more fun; I spent a week there. My wife and I hung out (avoided the queues for the iPhone 4), caught up with some friends, and celebrated our wedding anniversary by going to see The Dwarves play at The Bowery Electric.

Going home was a lengthy 32 hours travel time (6 hours NY to LA, 15 hours LA to Sydney, and 1.5 hours Sydney to Melbourne, plus a whole lot of waiting around at airports). Jetlag ensured that I was wide awake at 4am writing a VuGen script to download all the presentations from the conference, and converting my notes into this summary for you…