4 New Cloud Regions

I’m happy to announce that you can now load test from four additional Loadster Cloud regions:

  • Mumbai, India
  • London, United Kingdom
  • Montréal, Canada
  • Ohio, USA

You can now run your on-demand load tests from any 14 AWS regions spread across five continents.

We’ll continue adding more locations in the upcoming months. We’re also considering expanding to other cloud providers such as Google Cloud and Azure, so if you have a preference drop a line to support@loadsterperformance.com.

Custom Hostname Resolution

Most of us run our applications in multiple environments.

For example, you might do your regression/integration testing in a staging environment, and once all seems good, promote the code to production. Or you might spin up a temporary environment for a special purpose, like running load and performance tests against a replica of production, without the risk of impacting real users.

Because switching load test scripts and infrastructure between multiple environments can be a bit of a headache, we’ve introduced a new feature into Loadster to make this a bit easier.

With custom hostname resolution, you can now leave your scripts untouched, and switch quickly between environments. It’s like an /etc/hosts file, except that Loadster automatically takes care of distributing the configuration between all the engines or cloud regions when you run your test.

Overriding DNS

You can customize how hostnames are resolved by your load tests with a DNS Override. DNS Overrides take effect at the virtual user group level. To override host resolution for a specific host name, expand the +… menu and select Override DNS.

DNS Override

Enter one or more key-value pairs to map your hostnames to IP addresses. Again, this is very similar to a hosts file but with the added benefit that Loadster will automatically distribute the configuration for you.

Are you load testing against multiple environments of your application? Do you spin up temporary environments as part of your build/deployment process? If so, we’d like to hear about it and how we can make your life easier. Drop us a line at info@loadsterperformance.com!

Script Functions for Random Data

You can use Loadster’s Variables & Expressions to inject dynamic data into your scripts.

variables

We just added a few new ones for generating random test data: randomalphanumeric(len), randomalpha(len), and randomnumeric(len).

There’s also a uuid() function for generating a UUID of the format “77124923-bad6-4bac-b71b-791c6347b814″.

These functions are useful anytime you need to generate randomized data for your load tests (which is a good idea, especially if there’s a database or server-side caching involved).

Loadster Recorder Chrome Extension

The Loadster Recorder Chrome Extension is a new way to record Loadster scripts, and it’s live now on the Chrome Web Store.

If you’re a Chrome user, it takes the hassle out of messing with web proxy settings and HTTPS root certificates. It’s also easier to enable and disable the recorder with a single click in your browser.

Enabling Chrome Recording

Of course, the Recording Web Proxy is still a viable alternative… if you’re not a Chrome user or have to record non-browser HTTP traffic (say, from a library or command line client) the proxy is still your best bet.

Learn more about recording with the Loadster Recorder Chrome Extension and give it a try!

Loadster 3.7 Charts & Team Sync

We’ve just released our first minor version upgrade to Loadster in a couple months.

Most of the improvements in this release are under the hood, laying the groundwork for some awesome features coming your way soon. For now though…

Better, more responsive charts

A huge part of load testing is analyzing and presenting the results of your test. With that in mind, we’ve greatly improved the interactive charts that Loadster uses to display your response times, throughput and other metrics.

Loadster 3.7 Chart

These new charts appear in the live test dashboard, and also in the final test reports that you can review and share afterwards.

Sortable Chart Legends

In addition to just looking good, these charts now have sortable legends so you can quickly pinpoint your fastest and slowest pages, zero in on errors, and troubleshoot hot spots while your load test is running.

Easier XML parsing for JavaScript validators & capturers

Our JSON support is great, but sometimes you just gotta parse XML. We’ve made that easier in Loadster with the help of Nick Farina’s slick but simple xmldoc parser. XML parsing is available anywhere you use a JavaScript snippet in Loadster, which pretty much means in your validators and capturers (check out the documentation for some usage examples).

Capturing XML with JavaScript

Import/export projects

This feature has been asked for a dozen or more times, and we finally got around to it. You can now export projects to zip files and import them elsewhere, making it a lot easier to collaborate with other people using Loadster Workbench.

Loadster Project Import/Export

Share your repository with cloud sync

Also new in 3.7, for the first time ever you can easily sync your Loadster repository with other members of your team.

This is great because you can collaborate on scripts and design test scenarios together, and they’ll all be stored in the same place for anyone in your organization who wants to run them.

Although the shared repository is considered a public beta feature to start with, it’s the beginning of many more team-oriented features coming soon.

Recording iOS Traffic for Load Testing

Many apps are being built with a native mobile front-end, and an HTTP web service on the back-end. Load testing apps like these requires you to create scripts that simulate the actions of the front-end hitting the back-end.

Recording HTTP(S) traffic from an iOS app is generally pretty easy with Loadster Workbench’s recording proxy.

Configuring your iOS Device

To record traffic from your iPhone or iPad, you’ll need to install Loadster’s Root Certificate on your device, and then point the device at Loadster’s recording proxy running on your workstation.

Installing Loadster’s Root Certificate

If your mobile app uses HTTPS to communicate with its back-end, you will need to install Loadster’s Root Certificate. This step is optional if you only need to record regular HTTP traffic.

  1. On the iOS device, open your mobile Safari browser.
  2. Download the Loadster Root Certificate. You may need to temporarily disable the proxy settings you just entered in order to install the root certificate.
  3. Your iOS device will ask you whether to install the “Loadster Root Certificate” profile.
  4. If you trust Loadster’s root certificate, choose Install and then verify this choice by entering your passcode.
  5. The profile is now installed. Tap Done.

Now that the root certificate is installed on your iOS device, it will trust the SSL certificates created by Loadster’s recording proxy and you’ll be able to record HTTPS traffic through the proxy.

iOS Certificate Profile Installed

It’s very important that you remove the certificate when you’re done recording. Not doing so could expose you to man-in-the-middle attacks, if bad guys were to someday use Loadster’s Root Certificate to imitate legitimate websites. You can disable the certificate by navigating to General >> Profile >> Loadster Root Certificate >> Delete Profile.

Setting the Recording Proxy

You’ll need to tell your iPhone or iPad to use Loadster’s recording proxy.

  1. Start Loadster Workbench and begin recording a script.
  2. Find out the IP address of your workstation. This will typically be an internal IP address on your local network, for example, 10.0.1.111.
  3. In iOS, open the Settings app and navigate to Wi-Fi >> [Your Network] and tap the little (i) icon for advanced settings.
  4. Scroll down to the HTTP Proxy section and choose “Manual”.
  5. Enter the IP address of your workstation that’s running Loadster Workbench.
  6. Enter the port Loadster’s recording proxy is listening on (normally port 1997).

iOS Proxy Settings

That’s it! Now your iOS device will send all web traffic through Loadster’s recording proxy. Remember to unset the proxy settings when you finish recording.

Recording Mobile App Traffic

Open the iOS app you want to record and use it normally.

You’ll see Loadster Workbench recording the traffic on your workstation. You can then use this recording to create a script, just as you would for ordinary website traffic recorded from your desktop.

You will likely need to edit your script and add dynamic data, just as you would when testing ordinary web apps recorded from a browser.

Simulating Mobile Apps in a Load Test

When running a load test, keep in mind that each virtual user playing your scripts represents a single user running the mobile app on their own device.

Load testing means you are measuring the overall impact of all these simulated users, to see how the load affects performance and to determine scalability of the back-end. Front-end performance concerns, such as rendering and animations, are out of scope for this type of testing.

Loadster Cloud: Bigger and Badder

Loadster Cloud has expanded into 9 global regions on 5 continents!

cloud-map-9-regions

With our recent expansion into Japan, Germany, and two US West Coast data centers, you can now generate load closer to home, or spread it across multiple regions to simulate an even broader distribution of global traffic.

Each cloud cluster is capable of simulating tens of thousands of concurrent users, and with Loadster Workbench you can commandeer multiple clusters at a time in a single, massively distributed load test.

On-demand cloud testing is a cost-effective way to test your public facing websites, web apps, and web services — without setting up your own hardware or virtual infrastructure, and without purchasing expensive licenses. Even for smaller tests, you’re only charged for the exact amount of Cloud Fuel you use.

Start running huge distributed load tests, literally within minutes. To get started, sign into your Loadster Dashboard to purchase some Cloud Fuel!

Advanced Load Test Scripting with JavaScript

A few months back, we quietly rolled out JavaScript support in Loadster’s script editor.

This is kind of a big deal, because one of the things Loadster is known for is the no-nonsense scripting environment that makes the “easy things easy”. Our goal has always been to make it simple and straightforward to create and customize HTTP requests, and just as easily, to parse and validate the response. In fact, for load testing most websites and web apps, you’ll never need (or want) to write a single line of code.

JavaScript validators and capturers, used sparingly, can make things even easier. Writing a custom JavaScript function lets you parse more complicated responses (iterating through a JSON object or XML document, obtaining the n’th value of a type, and so on). You can even write custom JavaScript to generate random or unique values when you need them.

How does it work?

There are two places in a script command where you can use JavaScript: capturing and validation.

Validation with JavaScript

Validating an HTTP response with JavaScript means creating a validation function. To add a JavaScript validator to your HTTP command, just add it as you would any other validator:

Creating a JavaScript Validator

By default, the validation function will simply return true. A “true” response means the response is valid. A “false” (or null) response means it is invalid and Loadster should raise an error.

When the HTTP request is made and a response returned, Loadster calls your validation function and passes it a response object. Within the validation function, you can do fancy things with the response, like evaluating HTTP headers or parsing the response body as JSON:

As long as your function returns true, validation will succeed; otherwise Loadster will report the error.

Capturing with JavaScript

You can also create capturing functions, to extract values from the response and save them as script variables. You can then reuse the values later in your script.

Capture from a response with JavaScript

To capture a name from a JSON document is simple:

While capturing rules have access to the response object, you can just as easily ignore it and return random values or timestamps:

Loadster saves the result of your capturing function to the variable you designate.

Examples

We’ve published quite a few other examples of JavaScript capturing and validation (many of which can also be found in the Loadster Manual):

So far, being able to use JavaScript validators and capturers has really accelerated my own testing and I’m looking forward to hearing from you how they work for you! Feel free to share any other examples or unique problems/solutions you come across.

HTTP Response Validation

Testing dynamic web applications requires a lot more than just sending requests to the server and recording how long it takes to respond. It’s equally crucial to validate that the response came back as expected. A response with errors is not the same as a successful response.

Loadster automatically checks for certain types of errors. If the HTTP status code was 400 or higher, it’s universally regarded as an error. If the response time exceeds the globally configured timeout, that’s an error too. The same thing goes for socket errors such as connection refused… Loadster automatically detects and reports on them.

But what about application-specific errors? For example:

  • You tried to complete an order, and the test credit card was declined.
  • You tried to register an account, and the email address was already in use.
  • You tried to log in, but the password was incorrect.
  • You attempted a search, but no results were returned.
  • A third-party system was unreachable.
  • A JSON/REST web service returned an error object.
  • An XML document had the wrong number of child elements.

Errors like these are not always accompanied by an HTTP 4xx or 5xx status code. To detect them, you can create your own Validation Rules.

Validation Rules

Adding validation rules to your script commands is a good way to guarantee the application is behaving the way you want it to, and the responses are coming back as expected.

Response Time Validation

Let’s say you have a requirement that a certain key page must load in less than 2.5 seconds. Adding a response time validator to your script will check this every time that page is loaded, and raise an error if the load time exceeds that. For pages with additional resources attached (images, JavaScript, etc) you can specify whether to include page resources in the response time, or only count the load time of the primary request.

Response Size Validation

A response that is significantly larger or smaller than expected is often a key indicator that something went wrong. Usually, error pages have less content than successful content or confirmation pages. Creating response size validators for key responses is often an easy way to check for abnormalities in the response — particularly with Ajax POSTs, form submissions and other dynamic requests.

Content Validation

Loadster also has validators that can scan the response content itself. You can create content validators that require a certain string in the response (such as “Thank you for your purchase!”), or content validators that make sure a certain string is not in the response (such as “Fatal Error”). Of course, these strings will be unique to your system.

When should I use validation rules?

The more dynamic your application (and test scripts), the more likely you are to get unexpected results, and the more important it is to add validation rules so they don’t go unnoticed.

When scripting for dynamic web applications in particular, it’s always a good idea to add some extra validation to make sure the responses are as expected. If you’re pressed for time, start with any POST/PUT commands, since those are typically the most likely to fail and produce something unexpected.


Read more about Validation Rules in the Loadster Manual. 

HTTP Response Capturing

Real web apps are data driven.

To test them, it’s not enough to just throw a bunch of identical requests at the server until it falls over. Your scripts actually need to evaluate the responses from your application, and react accordingly.

Capturing Rules

Loadster’s Capturing Rules let you extract a value from an HTTP response and save it to a variable for later use.

Examples of when capturing is necessary:

  1. Placing an order and receiving a confirmation number. You need to hold on to the confirmation number and use it later, to check the status of your order.
  2. Uploading a forum post which is assigned a unique URL. You’ll then need to navigate to that URL to edit the post.
  3. Submitting a web form that uses a CSRF token to prevent cross-site request forgery.

In Loadster, dealing with these dynamically generated values means capturing them from an HTTP response, saving them to a variable, and using the variable later in the script.

Creating a Capturing Rule

You can configure your Loadster scripts to capture dynamic values from a response using a Capturing Rule.

Capturing rules are a combination of two things:

  1. The name of the variable that will receive the captured value
  2. A recipe for how to find the value in the response

For example, you could create a capturing rule to scan the HTTP response and capture a string into a variable called ${OrderNumber}. The scanning could be done in several ways:

  • Look for text sandwiched between two strings, like [Your confirmation number is ] and [.]
  • Look for text matching a regular expression, such as /Your order # is: (\d+)./
  • Apply a custom JavaScript function to the response

Here’s what it looks like in Loadster Workbench:

Regex Capturing

Using Captured Variables

Once you’ve captured a piece of the HTTP response into a variable, what now?

You can edit subsequent commands to send that variable instead of the original recorded value.

Using a Captured Variable

Keep in mind that every virtual user will have a unique copy of this captured variable. This allows you to realistically simulate many users at the same time on a dynamic web application.


Read more about Capturing Rules in the Loadster Manual. For my next post, I’ll introduce HTTP Response Validation, another way to ensure the application under test is responding to your requests correctly.