1 - Product Overview

RealLoad: Unique Key Features

What is RealLoad?

RealLoad is an Enterprise Tool to perform next generation Load Testing, Regression Testing, and Synthetic Monitoring.

Due to the universal product architecture, a Test of any Type supported by RealLoad can be executed as:

  • Load Test
  • Regression Test
  • Synthetic Monitoring Job

The following Types of Tests are supported::

  • ‘HTTP Test Wizard’ Tests - Web Surfing Sessions and HTTP API Tests.
  • JUnit Tests - Testing any type of Network Protocol (such as DNS, SMTP or UDP).
  • Selenium IDE and Playwright Tests - Execution of Real Web Browser Tests.
The product is highly scalable and can be used to run tests from one simulated user up to an almost unlimited strength of millions of concurrent users ¹ - and this for Load Tests, Regression Tests, and for Synthetic Monitoring Jobs.
¹ = does not apply to Selenium IDE and Playwright Tests (max. 5..50 users per load generator due to the high CPU usage of such tests)

Regardless of what type of test you are running, all measured data are displayed directly in real time in form of statistics and charts.

All work steps can be carried out within the web interface of the RealLoad Portal which:

  • contains simple and powerful dialogs.
  • contains numerous wizards, so no programming is required in most cases.
  • does not require to configure any JSON and XML files.

In addition, using the RealLoad Portal offers many other advantages, for example:

  • Web surfing sessions can be recorded with a ready-for-use Remote HTTP/S Proxy Recorder and post-edited with a powerful wizard.
  • JUnit tests can measure Additional Statistics and Charts, declared on the fly.
  • Selenium IDE tests can be converted into RealLoad tests using a wizard.
  • A powerful graphical Test Suite Editor supports the compilation of Regression Tests.
  • The Synthetic Monitoring Dashboard gives you a complete overview of the last 24 hours by displaying real-time sparklines (micro charts) per each monitoring job.
  • Multiple load generators can be combined into a cluster, with the test results of the cluster members are combined into a single test result - displayed in real time - for all types of tests.

Complete Overview of All Product Features

Our mission is to provide you with a powerful, professional testing tool - at a competitive price.
Try the RealLoad product for free by signing up to the Portal. Or email support@realload.com to request a quote or a 1:1 demo that takes your specific needs into account.

1.1 - Technical Product Features

RealLoad: Technical Product Features
Image Title

Universal ‘RealLoad Tests’

‘RealLoad Tests’ are universal, abstract tests which can created from any type of test. All types of tests can be executed as:

  • Load Test
  • Regression Test
  • and as Synthetic Monitoring Job

Creating a ‘RealLoad Test’ is usually easy and supported by wizards and powerful dialogs.

Types of Tests

The following Types of Tests are supported:

  • ‘HTTP Test Wizard’ Tests - Web Surfing Sessions and API Tests
    • HTTP Test Wizard’s tests simulate only the network traffic from an HTTP client to HTTP servers - without running a web browser’s rendering engine. This allows extremely efficient testing of the web servers.
    • Web Sessions can be either created manually - by entering URL HTTPS GET/POST… requests - or by recording a Web Browser Surfing Session using a RealLoad HTTP/S Proxy Recorder.
    • No Programming Required - All functionalities can be carried out using powerful dialogs with just a few mouse clicks and form entries.
    • Support of handling dynamically exchanged Session Variables using a wizard and by editing the session in the Remote Debugger.
    • Support of HTTP response verification
    • Support of Session Cookies
    • Support of PKCS12 Client Certificates
    • Support of parallel executed HTTP requests
    • Support of User Input Fields
    • Support of Input Files (from which variables can be extracted)
    • Support of Conditional Jumps to Session Elements (conditional GoTo)
    • Support of ‘HTTP Test Wizard’ Plug-Ins which are invoked during test execution
    • Support of multiple client TCP/IP addresses during test execution (requires special OS configuration on Measuring Agents)
    • Can be debugged using a powerful Remote Debugger
    • The ‘HTTP Test Wizard’ generates powerful, performance optimized code that does not need to be edited manually. Depending on the hardware resources of a load generator (Measuring Agent), up to 5000 virtual users can be executed in parallel per load generator. In addition, by combining multiple load generators into a cluster, an almost unlimited load can be achieved.
  • ‘RealLoad’ JUnit Tests - Testing any type of network protocol
    • ‘RealLoad’ JUnit Tests can be converted from any JAR file which contains original JUnit Tests by using a wizard, or can be written from scratch.
    • Due to the generic nature of JUnit tests, any type of network protocol can be tested while taking full advantage of the RealLoad infrastructure (e.g. running as load tests, part of a regression test, and executed as monitoring job).
    • Additional Features of RealLoad JUnit Tests written from scratch:
      • Support of User Input Fields
      • Support of Additional Measured Statistics and Charts, declared on the fly inside the JUnit test (in addition to the @Test execution time).
  • ‘RealLoad’ Selenium IDE and Playwright Tests - Execution of Real Web Browser Tests
    • ‘RealLoad’ Selenium IDE Test can be converted from any exported Selenium IDE Test by using a wizard, or can be written from scratch.
    • Support of Firefox and Google Chrome browser.
    • Support of User Input Fields.
    • The executed tests take a final screenshot of the simulated web browser at the end of each surfing session.
    • The tests can take full advantage of the RealLoad infrastructure (e.g. running as load tests, part of a regression test, and executed as monitoring job). However, due to the high CPU load when executing such tests, only a few simulated users can run per load generator at the same time (typically 5 to 50).

Load Tests

  • Support of all types of ‘RealLoad Tests’.
  • All measured data (and errors) are displayed in real-time during test execution.
  • Support Clusters of Load Generators with up to several hundred cluster members (Measuring Agents). This enables you to execute load tests with an almost unlimited strength of load, with more than 1,000,000 concurrent VUs, for ‘HTTP Test Wizard’ Tests and for JUnit Tests.
  • Cluster Features:
    • Assembling a cluster of load generators is easy and can be done with just a few mouse clicks.
    • The content of Input Files can be split among the cluster members.
    • The real-time view of the measured data is also available at cluster level.
    • The test results of the cluster members are combined into a single test result.

Regression Tests

  • Support of all types of ‘RealLoad Tests’.
  • With a powerful Graphical Editor you can create and edit Test Suites and their tests with just a few mouse clicks and form entries.
  • All tests of a Test Suite support to be executed with multiple of concurrent users (same as load tests).
  • The tests of a Test Suite are grouped into “Execution Groups”, where the Execution Groups can be executed in sequential or parallel order.
  • The progress of a Test Suite “run” can be viewed in real-time.
  • The results of the Test Suite “runs” are archived and can be compared with each other.
  • A “run” of a Test Suite can be triggered either interactively or via an API.

Synthetic Monitoring

  • Support of all types of ‘RealLoad Tests’.
  • Multiple “Monitoring Groups” can be defined, each containing multiple “Monitoring Jobs”.
  • Each “Monitoring Job” references an (abstract) ‘RealLoad Test’, whereby the same properties as for a Load Test Job can be configured per “Monitoring Job”, i.e. the number of executed virtual users.
  • Multiple “Measuring Agents” can be configured per “Monitoring Group” which means that the corresponding “Monitoring Jobs” are executed from multiple geographical locations. The measurement results of the Measuring Agents are combined into a single result, whereby the individual result of each Measuring Agent is still available.
  • The Real Time Dashboard displays the status of all Monitoring Groups and Jobs, and displays also real time sparklines (micro charts) per Monitoring Job about the availability and the response times of the last 24 hours.
  • Statistics about the availability and performance of the Monitoring Groups and Jobs can be displayed in a freely selectable time period, and can be exported to a PDF and MS Word document. This also allows you to check whether the requirements of an SLA (Service Level Agreement) have been met.
  • Detailed data for each individual test execution of a Monitoring Job can be displayed.
  • Support for Multiple Alert Groups that refer to Alert Devices - which can be configured differently depending on the type of alert.
  • Supported Alert Devices: Email, Mobile Companion App (installed on a mobile phone), SMS and Webhooks (RealLoad native, Pagerduty and Slack).
  • In addition to release Alert Notifications about measured warnings and errors, an alert can also be triggered if the Performance of a Monitoring Job is poor.
  • Support for Delayed Alert Notifications: Alert notifications can be triggered immediately or only after one or several repeats.
  • A fully documented API enables developers to implement their own version of a Synthetic Monitoring “Real Time Dashboard”.

RealLoad Portal - Features and Licenses

  • Fully integrated functionalities. All work steps can be carried out within the portal.
  • Simple and powerful dialogs. All functions can be carried out via the web user interface. And all of this without the need to configure any JSON and XML files.
  • Multiple team member accounts can be created per main user account, although certain team members can only have read rights.
  • After Sign Up to the portal, a remote HTTP/S Proxy Recorder and two shared Measuring Agents (load generators with limited capacity) are immediately available to you for free. In addition you will get 20 free ‘Cloud Credits’.
  • Additional Measuring Agents (load generators), Cluster Controllers and Remote Proxy Recorders that are needed for a short time can be easily started via the portal in the AWS cloud by using of ‘RealLoad Cloud Credits’. The advantage of using ‘Cloud Credits’ is that you do not have any cost risk from forgetting to terminate the instances, and that you don’t need an own AWS account. Depending of your license, you can also start additional (private) Measuring Agents under your own AWS account, by using our Desktop Companion App, or by sign in directly into the AWS console.
  • According to your license, you can also operate private Measuring Agents, Cluster Controllers and Remote Proxy Recorders yourself or have us operate them. You can register such components directly in the portal in your account.
  • Powerful API with over 70 functions. Such as creating new projects, uploading and downloading files, starting load test jobs and downloading their results, starting regression tests and downloading their results, implementing your own synthetic monitoring dashboard.

RealLoad Core Components

All core components are independent OS processes that can be executed on Linux and Windows:

  • Portal Server
  • Measuring Agent ¹ - Execution of tests.
  • Cluster Controller ¹ - Combines multiple Measuring Agents to a cluster.
  • Remote HTTP/S Proxy Recorder ¹ - Recording of HTTP/S web surfing sessions.
  • Table Server - Stores values of (previous) tests, which can be used as input for next tests.

¹ = Available as public AWS/EC2 image

RealLoad Apps

Using locally installed RealLoad applications is optional, but can be quite useful:

  • Desktop Companion - can be installed on Mac OS X and Windows
    • Generate HTTP Load Test scripts from HTTP archives (.har) files.
    • Locally run the RealLoad HTTP/S Proxy Recorder (local recording of HTTP Test Wizard ‘Tests’).
    • Perform some basic editing of test scripts.
    • Upload test scripts directly to the RealLoad Portal.
    • Manage AWS Measuring Agents by using your own AWS account (launch, terminate, register with RealLoad Portal).
     
  • Mobile Companion - can be installed on Android mobile phones
    • Displays a simplified real-time view of the Synthetic Monitoring Dashboard.
    • The mobile phone can be used as a Synthetic Monitoring Alert Device. You will receive instant push notifications when a monitoring job detects an error or warning - and when this issue have been resolved.

RealLoad Browser Extensions

The use of RealLoad browser extensions is optional, but allows you to conveniently control the recording of a web surfing session in combination with a Remote HTTP/S Proxy Recorder.

  • Browser Extensions are available for Firefox and Google Chrome.

Our mission is to provide you with a powerful, professional testing tool - at a competitive price.
Try the RealLoad product for free by signing up to the Portal. Or email support@realload.com to request a quote or a 1:1 demo that takes your specific needs into account.

2 - Getting Started

Getting started with RealLoad

This section of the documentation will walk you through the first steps in order to getting started with the RealLoad Product.

Follow the next sections of the document and you should be able to run your first basic test script using the SaaS Evaluation scenario within 20 minutes.

Let us know if encounter any issues while getting started, as that will help us updating this documentation to make it as clear and user friendly as possible. Please email us at support@realload.com

2.1 - Deployment Types & Architecture

Chose the right deployment type for you.

First of all you’ll have to decide which RealLoad deployment type better suits your needs. These guidelines should assist you in making an informed decision.

Architecture

There a few components that make up the RealLoad solution. Most components are available under an SaaS model which doesn’t require any infrastructure commitment from your side.

If required by security or other constraints, some or all components can be deployed on your own infrastructure. Please reach out to us to discuss requirements if you’re planning to go down this road.

The key components that make up RealLoad’s solution are:

The Portal Server

  • Exposes the main GUI to end users.
  • All tasks to maintain Load Testing or Synthetic Monitoring scripts are performed here.
  • Load Tests are can be triggered via the Portal.
  • The Synthetic Monitoring dashboard is visible here and shows the current state of configured tests.

Measuring Agent

  • Load Test and Synthetic Monitoring scripts are executed from Measuring Agents.
  • For Synthetic Monitoring purposes, you can use shared or dedicated agents to execute your tests from.
  • The Measuring Agent needs to be reachable from RealLoad’s portal server and needs to be able to reach the servers to be load tested or monitored.
  • You can have multiple Measuring Agents, for example to execute Synthetic Monitoring tests from different locations.

There are 2 options for your to run AWS cloud based Measuring Agents:

  • Using our Cloud Credits: The relevant Virtual Machines are spun up from our portal under our own AWS account and you pay a usage fee using Cloud Credits you’ve previously purchased from us.
  • By starting pre-configured AWS AMIs under your own AWS account. You’ll be charged directly for AWS related costs.

Cluster Controller (Optional)

  • If your Load Testing requirements exceed the load generating capacity of a single Measuring Agent, you can combine multiple agents in a cluster using a Cluster Controller.
  • Like measuring agents, Cluster Controllers can be hosted in the cloud or on your own infrastructure, as per your requirements.

Proxy Recorder (Optional)

  • You can use the Proxy Recorder component in order to develop test scripts by recording HTTP traffic while it transits through it.
  • The Proxy Recorder can be run in the cloud, as a service on your own infrastructure or directly on your Desktop using the Desktop Companion.

Typical deployment scenarios

Load Testing using Cloud Credits

The RealLoad portal gives you the ability to launch load generators (Measuring Agents) in the AWS cloud with a click of button and the execute a load test.

Key features and requirements are:

  • You’re not required to have an AWS account, as the Measuring Agent is run under RealLoad’s AWS account.
  • You pay for the AWS and RealLoad licensing costs using previously purchased Cloud Credits.
  • The application to be load tested needs to be reachable from the Measuring Agent’s public IP address, assigned by AWS at launch.

Load Testing using Cloud Credits

Load Testing by running the Measuring Agent behind perimeter firewall

Using Cloud Credits is not an option if the servers to be performance tested are not exposed to the internet. In this scenario, the Measuring Agent will need to be deployed behind perimeter firewalls as follows:

  • In AWS: You can launch our pre-configured AMIs under your AWS account and attach them to your VPCs.
  • If preferred, Measuring Agent instances can be launched using our convenient Desktop Companion application.
  • If running on non-AWS infrastructure: Install the Measuring Agent on a supported Linux Distribution.

Key features and requirements are:

  • The Measuring Agent needs to be reachable from the internet based Portal Server. You’ll need to configure firewall and NAT rules on your perimeter firewall to allow communication.
  • You’ll need to update the Measuring Agent software from time to time, to stay compatible with updates on the Portal Server.
  • You’ll need to purchase a Load Testing license to suit your load requirements.

Load Testing using managed agent

Synthetic Monitoring of Internet exposed services

Typically Synthetic monitoring is use to monitor services that are exposed to the internet. RealLoad uses Measuring Agents to implement the monitoring functionality.

We recommend executing Synthetic Monitoring tests from at least 2 Measuring Agents, to cater for transitory local disruptions (like network issues or agent software updates) which might trigger false positive alerts.

There are two types of agents you can use for monitoring purposes:

  • Shared Agents: These are Monitoring Agents shared by multiple customers. Using these agents is included in the basic cost of the monitoring service.
  • Dedicated Agents: If you need to execute monitoring from a specific location or you don’t want to use a shared agent, we can setup a dedicated agent for you. This will incur additional costs, plz reach out to us to discuss.

Key features and requirements are:

  • The application or service to be monitored needs to be reachable from the internet.
  • You’ll need to subscribe to a Synthetic Monitoring plan. Reach out to us for pricing.

Synthetic Monitoring using Shared Agents

Synthetic Monitoring of services running behind perimeters firewalls

Should you have the need to monitor services within your perimeter, this can be arranged by deploying Measuring Agents within your networks.

Key features and requirements are:

  • You’ll need to deploy two measuring agents behind the firewall perimeter. For AWS deployments your can use our existing AMIs, for other infrastructure install the Measuring Agent on a supported Linux Distribution.
  • You’ll need to update the Measuring Agent software from time to time, to stay compatible with updates on the Portal Server.
  • The Measuring Agents need to be reachable from the internet based Portal Server. You’ll need to configure firewall and NAT rules on your perimeter firewall to allow communication.
  • You’ll need to subscribe to a Synthetic Monitoring plan. Reach out to us for pricing.

Synthetic Monitoring using Internal Agents

2.2 - Portal Sign Up and Login

To get started you’ll need to setup and account for yourself at the RealLoad portal. See here how…

If you already have an account, you can login at https://portal.realload.com

Signing Up - Free Account Features

In order to login to the portal you’ll first have to setup an account. Go to the portal URL and click on the Sign Up button or go this URL: https://portal.realload.com/SignUp

You’ll need to provide:

  • Email address.
  • Mobile phone number.
  • No credit card required, it’s completely free.

Account deletion

To have your account deleted, please send an email to support@realload.com. We’ll then reach out to you to confirm deletion.

Your account will be immediately disabled and all your data will be removed within 90 days, including backups.

Sign Up Process

Step 1 of 4

Provide your details, including email and mobile number. All data you enter in this form are only required to verify your identity and will not be shared with any third parties.

Step 2 of 4

Validate your email address.

Step 3 of 4

Validate your mobile number

Step 4 of 4

Configure your nickname and password

Welcome Page with Examples

Once you signed up, you will be automatically loggedIn and the welcome page will be displayed. Three examples are shown which are also available online:

2.3 - Execute a Simple HTTP/S Test

This section gets you kickstarted with a simple HTTP/S test.

This example shows how a simple HTTP/S Test can be defined as a RealLoad ‘Test’ and executed as both a Load Test Job and a Monitoring Job.

The test sequence is manually entered into the HTTP Test Wizard to keep this example simple. However, you can also record HTTP/S tests using an HTTP/S Proxy Recorder and convert the recording into a HTTP Test Wizard session.

Pre-Requisites

To configure and execute a simple test you’ll need:

  • Access to the RealLoad portal. If you haven’t signed up, do so first Sign Up
  • Approx. 20..30 minutes of your time
  • A cup of tea or coffee

1. Invoke the HTTP Test Wizard and Create a New Session

Invoke HTTP Test Wizard and Create New Session

Enter the Headline that briefly describes what the test does.

Enter Session Headline

2. Save the (empty) Session

It’s best to save the empty session now. To do this, you must select a Resource Set of a Project. Alternatively, you can also create a new ‘Project’ and/or ‘Resource Set’. ‘Resource Sets’ are something like sub-directories of a Project, which contain all the files necessary to define and execute a test.

Click ‘Save Session’

Enter the file name for the session, e.g. HttpSession_TestDownloadServer.json and save the session.

Enter File Name of Session

In this example, a new Project named ‘HTTP Tests’ is created which also contains a new Resource Set named ‘Download Server’.

Create new Project and Resource Set

Result: Empty saved Session

3. Add URLs to the Session

Click on the ‘Add’ dropdown and select ‘URL’.

Dropdown ‘Add’ - Select ‘URL’

Enter the URL and click on the ‘+’ icon at ‘Verify HTTP Response’.

Enter URL

At ‘Verify HTTP Status Code’ select ‘200 OK’ and then click the ‘Add URL’ button.

Select ‘200 OK’ and add URL

You can then add additional URLs to the session. In this example it looks like this:

Session with 3 URLs

4. Debug the Session

Before converting the session into a RealLoad ‘Test’ you should Debug the Session first. This way you can check whether all URL calls are working correctly. Click ‘Debug Session’.

Invoke the Session Debugger

This invokes a powerful Remote Debugger that runs always on a Measuring Agent. As shown at the top right of the following image you can also select an alternative Measuring Agent on which the debugger will run.

In the debugger you can also extract values from HTTP responses and assign them to succeeding HTTP requests, whereby the session is automatically synchronized between the debugger and the HTTP Test Wizard.

In the debugger, click Next Step until all URLs have been executed successfully and the end of the session has been reached - or until an error occurs. If an error occurs, exit the debugger, then repair the URL (= HTTP/S request), and then invoke the debugger again.

Debug the Session

After debugging is successful, return to the HTTP Test Wizard.

Return to HTTP Test Wizard

5. Save the (debugged) Session

Save the debugged Session

6. Generate and Compile the Code, and Define the RealLoad ‘Test’

Click ‘Generate Code’

Generate and Compile the Code

After the source code is compiled, click ‘Define New Test’.

Click ‘Define New Test’

Enter the Test Description, and click ‘Define New Test’.

Enter Test Description

The RealLoad ‘Test’ is now defined. From here you can now create both a Load Test Job and a Monitoring Job.

Result: RealLoad ‘Test’ Defined

Note: In case you (later) modify the session in the HTTP Test Wizard, you will have to generate and compile the source code again.

7. Define and Execute a Load Test Job

In the RealLoad ‘Test’ click ‘Define Test Job’.

Click ‘Define Test Job’

In next menu click ‘Continue’ …

Click ‘Continue’

… And select the Measuring Agent on which the Load Test Job will be executed, then click ‘Define Load Test Job’.

Click ‘Define Load Test Job’

The Load Test Job is now defined. Click ‘Start Test Job’.

Click ‘Start Test Job’

Enter the settings of the Load Test Job here and then click ‘Start Test Job’. We recommend the following settings for the first load test run:

  1. Number of Users: 5
  2. Max. Test Duration (Seconds): 60
  3. Max. Loops per User: [unlimited]
  4. Loop Iteration Delay (Milliseconds): 1000
  5. Ramp Up Time (Seconds): 10

Enter Settings and Click ‘Start Test Job’

After a few seconds the Load Test Job is in the state ‘Running’. Click ‘Monitor Jobs’.

Job is ‘Running’, Click ‘Monitor Jobs’

The real-time data of the running Load Test Job is now displayed.

Real-Time Data of running Load Test Job

Click on ‘Analyze Result’ after the Load Test Job is completed.

Load Test Job completed, Click ‘Analyze Result’

The Load Test Job result is then displayed.

Load Test Job Result

To view the Job Output Files navigate to ‘Load Test Jobs’ and click in the dropdown of the Job on ‘Job Log Files’.

Click ‘Job Log Files’

Select the file users.out which contains the Log Output of the Job.

Select File ‘users.out’

If you would like to run the Load Test Job again with the same or changed settings, click on ‘Clone Job’ in the Job dropdown and then start the cloned job.

Clone Load Test Job

8. Define a Monitoring Job

If this is your first Monitoring Job, you must first create a Monitoring Group. Navigate to Monitoring, click the ‘Configuration’ tab and then click ‘Add Monitoring Group’. There is also a help available for configuring the Synthetic Monitoring.

Navigate to Monitoring Configuration

Enter the ‘Group Title’ and select at least one ‘Measuring Agent’ on which the Monitoring Job(s) will be executed. Then click ‘Add Monitoring Group’.

Add Monitoring Group

In the ‘Monitoring Group’ click ‘Monitoring Jobs’ and then click ‘Add Monitoring Job’.

Add Monitoring Job

Select the ‘Test’ of the Monitoring Job.

Select ‘Test’ of Monitoring Job

Click ‘Define Monitoring Job’.

Define Monitoring Job

Enable the execution of the Monitoring Group and navigate to ‘Dashboard’.

Enable Monitoring Group and Navigate to Dashboard

The Monitoring Job is now defined and will be executed periodically. For additional help configuring monitoring (e.g. adding ‘Alert Devices’), see Monitoring Help.

Monitoring Job is Defined

Tip: If you modify the RealLoad ‘Test’, the corresponding Monitoring Job is not automatically updated to avoid unexpected corruption of the Monitoring Job. After you have verified the modified ‘Test’ by running a small Load Test Job, you must manually update the Monitoring Job.

Update the Monitoring Job

Done, congrats, you’ve run your first test with Real Load.

9. Conclusion and Prospects

As you have seen, a RealLoad Test can be run as both a Load Test Job and a Monitoring Job.

This example was deliberately kept simple so that you can become familiar with the RealLoad product. To create a test with a web browser session across multiple web pages, you can use a Remote Proxy Recorder to record the test case and then convert the recording into an HTTP Test Wizard session.

Additionally, a Test Job Template can also be defined from any ‘Load Test Job’, which can then be part of a Test Suite. This means that you can add multiple ‘Test Job Templates’ to a Test Suite and execute them in a single run as a Regression Test.

To learn more, we recommend you read the User Guide. There you will find detailed documentation on the steps described in this document.

2.4 - Convert a Selenium IDE Test to a RealLoad Test

This example shows how to convert a Selenium IDE Test to RealLoad Test.

This example shows how a Selenium IDE Test - which was recorded with a Chrome or Firefox web browser - can be converted into a RealLoad ‘Test’. The converted RealLoad ‘Test’ can then be executed both as a Load Test Job and as a Monitoring Job. In addition, the converted test can also be part of a Test Suite for performing Regression Tests.

In most cases, the conversion can be done easily with just a few mouse clicks - without the need for any special settings.

At the end of each executed test run (of the converted RealLoad test), a final screenshot of the simulated web browser session is taken for each virtual user so that you can check whether the web browser session of the virtual user has been fully executed.

Tip: If possible, always use the Chrome web browser for Selenium IDE recording, as Chrome sessions are simulated much faster (= more realistic) than Firefox sessions.

Pre-requisites

To configure and execute this test you’ll need:

  • Access to the Real Load portal. If you haven’t signed up, do so first: Sign Up
  • Approx. 20..30 minutes of your time
  • A cup of tea or coffee

1. Export the Selenium IDE Test as JUnit Test

Export the your recorded Selenium IDE Test as Java JUnit file and save the file to any folder on your local device. Note that the 3-dot dropdown icon may initially be hidden and may only appear when you are near to the Selenium ‘Test Case’ area. Do not choose any export option. Export Java JUnit File

Export Java JUnit File

Export Java JUnit File

2. Upload the exported Selenium JUnit File

At Projects menu click on the ‘Developer Tools’ dropdown and select ‘Convert Selenium IDE Test to Real Load Test’. Select ‘Convert Selenium IDE Test to Real Load Test’

Then upload the file by ‘Upload File’ or by dragging and dropping the file into the dashed area. Upload the File

3. Convert the Uploaded JUnit File

Click on the ‘Convert’ button after uploading the file. The converted Java code is then displayed. Then click on the ‘Save & Compile’ button. Convert Selenium IDE Test

You have to save the converted code in a Resource Set of a Project. You can also first create a new Project with a new Resource Set. Save Converted JUnit File

After saving the converted code, you will be automatically redirected to the Compile Java File dialog.

4. Compile the Converted JUnit File

The JAR files required to compile are already preselected, i.e. you need only click the ‘Compile’ button. Compile Converted JUnit File

After successful compilation, click ‘Define or Update Selenium by JUnit Test’. Define or Update Selenium by JUnit Test

You will then be automatically redirected to the Define JUnit & Selenium Test dialog.

5. Define the Selenium RealLoad ‘Test’

Select the JUnit @Test method (in this case only one will be shown) and click ‘Define Test’. Define Selenium RealLoad ‘Test’

In an intermediate step you can enter an (optional) test description. Enter Test Description

Now the Selenium RealLoad ‘Test’ is defined from which you can define a Load Test Job and/or a Monitoring Job. Result: Defined RealLoad ‘Test’

6. Note about Measuring Agents

Please note that not all Measuring Agents support the execution of Selenium jobs. You can ‘ping’ a Measuring Agent at application level to find out whether Selenium jobs are supported: Ping Measuring Agent

Example 1: Measuring Agent ‘Demo Agent 1 (CH)’ does not support Selenium jobs Ping ‘Demo Agent 1 (CH)’

Example 2: Measuring Agent ‘Demo Agent 2B (CH)’ supports Selenium jobs Ping ‘Demo Agent 2B (CH)’

3 - Release Notes

RealLoad Release Notes

4.8.38 | 2024-04-21

  • New Features: The User Guide has been completely revised. In this context, the portal’s online documentation was also revised and enhanced, which now refers to the corresponding chapters of the user guide.
  • Changes:
    • HTTP Test Wizard ‘Output Files’ are now stored in the in the output directory of the test job.
  • Bug Fixes:
    • Compiling the code generated by the HTTP Test Wizard failed if the content of an HTTP request was very large.
  • Portal Server version 4.8.38 requires now Measuring Agent and Cluster Controller version 4.5.12

4.8.37 | 2024-03-12

  • New Features:
    • New Mobile Companion App released. The Mobile Companion is a small Android App that displays a simplified view of the Synthetic Monitoring Dashboard and can receive and display Synthetic Monitoring Alert Notifications.
      • This App gives you an overview of the status of your Synthetic Monitoring Jobs at any time - without having to sign in to the Portal Server.
      • The App also notifies you immediately if a Monitoring Job detects an error or warning.
      • By installing this App on your mobile device, you are always informed whether the operation of your services and servers is working reliably.
      • Documentation and Installation Instructions
  • Announcement: Currently the Mobile Companion App can be installed on Android devices only. Support for Apple iOS will be available in around the next 2 months.

4.8.34 | 2024-02-20

  • Synthetic Monitoring API Documentation
    • Full Synthetic Monitoring API documentation is now available to support developers to implement their own version of a real-time dashboard for Synthetic Monitoring that can be displayed on their companies screen.
    • The powerful API also supports receiving data for drawing charts and sparklines. In addition, the API documentation contains helpful information about implementation and runtime behavior.
    • See https://kb.realload.com/docs/api/remoteusermonitoringapi/

4.8.31 | 2024-01-18

4.8.30 | 2023-12-27

  • New Features:
    • Synthetic Monitoring: Added configurable performance alert thresholds for the measured time per loop (= average passed session time). Two performance alert threshold values (in milliseconds) can be configured per monitoring job. One to trigger a warning and another to trigger an error. If one of these thresholds is reached or exceeded, a performance warning or performance error is added to the test result, which has the same effect as measuring a “normal” error with a severity of “Warning” or “Error”.
  • Portal Server version 4.8.30 requires now Measuring Agent and Cluster Controller version 4.5.11

4.8.29 | 2023-12-17

  • New Features:
    • Synthetic Monitoring: Added real-time Sparklines (microcharts) to the Real Time Dashboard. The sparklines with a time range of the last 24 hours show the measured availability for monitoring groups, and for monitoring jobs the measured availability and the measured response times (milliseconds per loop). The displayed time range of the sparklines can be zoomed by clicking on the chart while holding down the mouse button and panning to the right or left in the chart. After releasing the mouse button, the zoom of the corresponding area appears. If necessary, you can then zoom in again to narrow down the area further.

4.8.28 | 2023-12-14

  • New Features:
    • Slightly redesigned Synthetic Monitoring user interface.
      • Real-Time Dashboard: The dashboard supports now to show only monitoring groups or only monitoring jobs - or both of them (as before).
      • Real-Time Dashboard: The displayed availability of Monitoring Groups and Jobs for the last 24 hours is now a hyperlink that, when clicked, displays the corresponding statistical chart.
      • All Synthetic Monitoring menus and popups have been revised to make them more descriptive and user-friendly.
  • Bug Fixes:
    • Synthetic Monitoring: Webhook alert calls which return a HTTP status code 202 (instead of 200) are no longer logged as failed.

4.8.27 | 2023-12-07

  • New Features:
    • Synthetic Monitoring: Support for delayed alert notifications: Alert notifications can now be triggered immediately (as before) or (new) only after one or several repeats. This means that short-period errors and downtimes can be excluded from alert notifications. This does not affect the measured statistics such as availability and detected errors. Delayed alert notification settings can be configured separately for each “Alert Group”, whereby multiple alerts groups can be assigned to one or multiple “Monitoring Groups” and “Monitoring Jobs”.

4.8.26 | 2023-11-28

  • New Features:
    • Synthetic Monitoring: Added support for ‘Monitoring Downtimes’. A Monitoring Downtime is a window of time during which synthetic monitoring is temporarily disabled. Multiple monitoring downtimes can be defined for all monitoring groups, or for specific monitoring groups and also for specific monitoring jobs. In addition, a monitoring downtime can be repeated daily, weekly or monthly.

4.8.24 | 2023-11-12

With this new release we have reached the point where a general load testing tool has been crossed with a monitoring tool and also with a regression testing tool - with both the monitoring tool and the regression testing tool are capable of executing test jobs with hundreds or thousands of concurrent users.

  • New Features:
    • Real Load Test Suites
      • Test suites are the grouping of multiple test job templates and are executed as a single “run” with different test scenarios.
      • Due to the universal architecture, test jobs of any kind can be part of a test suite (HTTP Test Wizard Jobs, JUnit Jobs, Selenium Jobs ..).
      • The results of the test suite “runs” are archived and can be compared with each other.
      • In addition, the result of a particular test job of a test suite run can be compared with the result of the same test job executed by other test suite runs.
      • All test jobs of a test suite support the powerful ability to be executed with hundreds or thousands of concurrent users and can even executed on a cluster of measuring agents.
      • The test jobs of a test suite are grouped into “execution groups”, where the test jobs of an execution group can be executed in sequential or parallel order.
      • The execution groups can be nested, whereby a separate measuring agent or measuring agent cluster can be defined for each execution group.
      • A “run” of a test suite can be triggered either interactively or via the Remote User API.
    • The following functions have been added to Remote User API:
      • getTestExecutionSuites
      • getCurrentlyRunningTestExecutionSuiteIds
      • startTestExecutionSuiteTestRun
      • getTestExecutionSuiteTestRunResultDetail
      • getTestExecutionSuiteTestRunTestJobOutputFileContent
      • deleteTestExecutionSuiteTestRun
    • Tests: Delete Test: A warning message will now be displayed if the test is referenced by a Monitoring Job or by a Test Suite.
    • Tests: Option “Delete All Unreferenced Tests” added.
    • Load Test Jobs: The display of load test jobs can now filtered by Project and Resource Set.
  • Bug Fixes:
    • Starting a load test job failed in rare cases due to a synchronization problem.
  • Portal Server version 4.8.24 requires now Measuring Agent and Cluster Controller version 4.5.10

4.8.23 | 2023-09-18

  • New Features:
    • Added support for ‘Test Job Templates’.
    • The following functions have been added to Remote User API:
      • getTestjobTemplates
      • defineNewTestjobFromTemplate
      • submitTestjob
      • makeTestjobReadyToRun
      • startTestjob
      • getMeasuringAgentTestjobs
      • getTestjobOutDirectoryFilesInfo
      • getTestjobOutDirectoryFile
      • saveTestjobOutDirectoryFileToProjectTree
      • deleteTestjob

4.8.21 | 2023-09-13

  • New Features:
    • Improved integration of JUnit Test with the Real Load architecture. As already described earlier, any JAR file that contains JUnit test can be uploaded to the portal server and executed directly as a test, i.e. without conversion. However, if you are programming a JUnit test from scratch, you can benefit from the additional enhanced features of the Real Load architecture. These additional features include better log output of the executed JUnit test, support for “User Input Fields” (configurable test input parameters), and the measurement and display of additional statistics obtained during test execution.
    • After logging into the portal server, a new project called “Developer Examples” will appear, which contains an example of a JUnit test and shows how a JUnit test can be fully integrated into the Real Load architecture.

4.8.20 | 2023-09-10

  • New Features:
    • Added support for executing Selenium IDE tests with Google Chrome.
    • When a Measuring Agent is pinged at application level, it is displayed whether the Measuring Agent supports the execution of Selenium tests.
  • Changes:
    • The Selenium version sent to Measuring Agents is now 4.12.1 instead of 4.10.0. However, this only applies to tests that are newly created.
  • Portal Server version 4.8.20 requires now Measuring Agent version 4.5.9 when Selenium IDE tests with Google Chrome are executed.

4.8.19 | 2023-09-04

  • New Features:
    • Added support for executing Selenium IDE tests. Selenium IDE tests recorded on your local device can be executed as monitoring job and as load test job - by exporting them in the Selenium IDE as ‘Java JUnit’ test and then importing them into the Real Load portal. The imported tests support also ‘User Input Fields’ (configurable options when starting a test) and take a final screenshot of the (virtual) Web browser display just at the end of each test run - which can then be viewed in the Real Load portal. Currently only tests recorded with Firefox are supported. Support for tests recorded with Chrome and Edge will be added soon.
    • Added support for executing JUnit 4 tests. JAR files which contain ready-to-use JUnit @Test(s) can be uploaded to the Real Load portal and run directly as a monitoring job and as a load test job. The benefit of running JUnit tests on the Real Load infrastructure is firstly that they can be run as a “monitoring job” and secondly that they can be run as a “load test job” with many hundreds of JUnit tests, running concurrently in parallel (or even with many thousands of JUnit tests running in parallel, through the use of a cluster of Real Load Measurement Agents). However, this is usually only useful for JUnit tests which are testing or stressing network services of any kind.
  • Portal Server version 4.8.19 requires now Measuring Agent and Cluster Controller version 4.5.8

4.8.17 | 2023-07-29

  • New Features:
    • Synthetic Monitoring: Support of ‘Pagerduty V2’ and ‘Slack’ Webhook alert devices added.
    • Test Results: The selected test result can now exported to a PDF and to a MS Word document.
    • ‘HTTPS Server Certificate Tool’ added.
  • Changes:
    • HTTP Test Wizard: The URL HTTP timeout can now specified in milliseconds (instead of seconds).

4.8.14 | 2023-06-29

  • New Features:
    • Revised Sign-Up: When sign-up, two shared Measuring Agents are now assigned to the new user account.
    • For customers (user accounts) who do not operate their own HTTP/S Remote Proxy Recorder, a temporary HTTP/S Remote Proxy Recorder from a pool is now assigned each time they sign-in.
    • Synthetic Monitoring: A warning sign is now displayed for Monitoring Groups for which no Measuring Agent have been defined.
  • Portal Server version 4.8.14 requires now HTTP/S Remote Proxy Recorder version 1.1.3, and Measuring Agent and Cluster Controller version 4.5.7

4.8.13 | 2023-06-20

  • New Features:
    • HTTP Test Wizard ‘URL Plugins’ can now be assigned in one step to multiple or all URLs of a HTTP/S session.
  • Changes:
    • Sent SMS messages now have the sender ID ‘RealLoad’ instead of a (technical) phone number.
  • Bug Fixes:
    • Synthetic Monitoring: No alert was send at monitoring group level when an execution timeout occurred.

4.8.12 | 2023-06-07

  • New Features:
    • Measuring Agents which are shared with other Portal Server user accounts (other customers) are now marked with [P] = ‘partial shared’ or [S] = ‘fully shared’ in the Measuring Agents menu. Measuring Agents that are operated exclusively for a specific customer do not have such a mark.
    • The ‘My Account -> Price Plan’ menu shows now the number of sent SMS monitoring alerts of the current month, and also of the last 12 months.
    • Price Plan limits for Synthetic Monitoring have been implemented and are now shown in the ‘My Account -> Price Plan’ menu.
  • Changes:
    • The primary Mobile Number for main user accounts and team member accounts no longer needs to be unique across all Real Load accounts.
    • Measuring Agent Housekeeping: The data of executed jobs older than 30 days are now deleted on Measuring Agents (monitoring jobs + load test jobs).
  • Bug Fixes:
    • Team member accounts with read/only privileges could make changes to the Synthetic Monitoring configuration.

4.8.10 | 2023-05-11

  • New Features:
    • HTTP Test Wizard
      • ‘Conditional Jump’ session element added. For example, this new session element allows a webpage to be called in a loop until the received content contains a certain value, or until a maximum number of conditional jumps is exceeded. Adding multiple conditional jumps per session is supported, with the ability to jump backwards or forwards. The condition of whether to jump and where to jump can be determined by variable values.
      • Java JVM warm-up before starting a test. Tests that run with only one or a few users and loops are now more accurately measuring URL response times by performing a Java JVM warm-up before starting the test.
      • The log-data of measured errors have been improved (less but more meaningful data).
    • Projects Menu
      • The preview of the test results shows now additionally the ‘Avg. Passed Session Time’.
    • Load Test Jobs Menu
      • A new delete option ‘All Completed Jobs - Except the Last Ones’ has been added.
    • Test Results Menu
      • The ‘Avg. Passed Session Time’ is now show in the Summary Statistic.
      • Searching for Text values in an Error Detail ‘Log’ and in an Error Detail ‘Context’ is now supported.
  • Changes:
    • HTTP Test Wizard
      • Errors reported at URL level are no longer additional reported at Measurement Group level.
    • Test Execution
      • The number of stored errors is now limited to 2000 fully stored errors plus 8000 partial stored errors (partial stored: without error log and without error context).
  • Portal Server version 4.8.10 requires now Measuring Agent and Cluster Controller version 4.5.6
  • To take advantage of the new features you have to re-generate and re-compile the Java source code in the HTTP Test Wizard.

4.8.8 | 2023-04-05

  • New Features:
    • Menu Measuring Agents
      • Measuring Agent Log File: The log file is displayed faster in the browser and supports now searching for text fragments. The time zone of the log file is now also displayed.
      • List Remote Test Jobs: The display of the Job Type was added (‘Load Test’ or ‘Monitoring’).
  • Bug Fixes:
    • In rare cases the start a job failed on Measuring Agents.
    • In rare cases the Synthetic Monitoring Availability Chart did show wrongly a green bar (OK) instead of a red bar (Error).
  • Portal Server version 4.8.8 requires now Measuring Agent and Cluster Controller version 4.5.5

4.8.7 | 2023-03-26

  • New Features:
    • Synthetic Monitoring
      • “Statistics” Submenu added which shows statistics and charts of monitoring groups and monitoring jobs, such as availability, cumulated error time and performance. The time range can be selected arbitrary. In addition, single, historical measurements can be selected and their details displayed (e.g. to analyze a measured error).
      • The Real Time Dashboard shows now for the monitoring jobs the availability of the last 24 hours in percent.

4.8.4 | 2023-03-03

  • New Feature:
    • The Synthetic Monitoring Real Time Dashboard shows now the (average) performance of the last executed monitoring jobs in the unit of milliseconds per passed loop. Background information: A monitoring job can be defined with a strength of up to 5 concurrent users and with up to 5 test-loops per user. This value shows the average execution time of the successful executed test-loops overall concurrent users.

4.8.3 | 2023-02-28

  • New Features:
    • Synthetic Monitoring added: The same scripts used for load testing can now also be used to periodically check the uptime and performance of your critical business transactions. This means that a Monitoring Job can be defined from an already existing Test (similar as to define a Load Test Job). The Monitoring Jobs can be combined to Monitoring Groups whereby each Monitoring Group can have its own settings. Multiple Measuring Agents can be assigned to a Monitoring Group in order that the assigned Monitoring Jobs are executed from several, different locations (for example from New York, Berlin and Singapore). Supported Monitoring Alert Devices are Email, SMS and Webhooks.
    • WebSocket API for Synthetic Monitoring added: This WebSocket API enable developers to implement their own version of a Synthetic Monitoring “Real Time Dashboard”.
  • Portal Server version 4.8.3 requires now Measuring Agent and Cluster Controller version 4.5.4

4.7.8 | 2022-11-06

  • New Features:
    • Desktop Companion: An ‘OS X Intel Installation Kit’ and an ‘OS X ARM Installation Kit’ are now available at https://download.realload.com
    • Remote Proxy Recorder
      • Support of Client Certificates (PKCS12 files) that can uploaded to the Portal Server and applied for target server authentication on all Remote Proxy Recorders. The uploaded client certificates are automatically transmitted to the Remote Proxy Recorder(s) when connecting from the Portal Server to a Remote Proxy Recorder.
      • The log files of Remote Proxy Recorders can now be viewed in the Portal Server Web UI.
      • ‘Ping’ and ‘Connect’ to a Remote Proxy Recorder in the Portal Server Web UI now show an error message for outdated/incompatible Remote Proxy Recorder versions.
      • Any changes made at Remote Proxy Recorder settings for ‘Hosts Recording Filter’, ‘URL Filter’ and ‘URL Filter Quick Setting’ are now saved on the Portal Server and reapplied when connecting from the Portal Server to any Remote Proxy Recorder.
  • Bug Fixes:
    • HTTP/S session recording (with Remote Proxy Recorder and Desktop Companion) failed on Apple macOS and iOS client devices.
    • HTTP Test Wizard URL Explorer - Variables Wizard: variable names for JSON HTTP-request ‘variable assigners’ were not displayed.
    • HTTP Test Wizard Generate Code: The “Java Class Name” can now have a lowercase letter at the beginning.
  • Portal Server version 4.7.8 now requires Remote Proxy Recorder version 1.1.1

4.7.6 | 2022-10-17

  • New Features:
    • HTTP Test Wizard: Variable Assigner for ‘General Request Header Field’ added.
    • HTTP Test Wizard: Select Multiple Session Elements - ‘URL Filter Quick Settings’ menu added.
    • HTTP Test Wizard Debugger: Support to set/update the ‘Recorded Data’ of an executed URL (shown in the HTTP Test Wizard URL Explorer). This functionality can be used for further test optimization (e.g. if the content of a tested URL has been changed on the server side, or if the URL response is as yet unknown).
    • HTTP Test Wizard Debugger: Menu ‘Search in executed URLs’ added.
  • Bug Fixes:
    • HTTP Test Wizard URL Explorer | Variables Wizard: Bug Fix for automatically applied variable assigners which replace URL-Encoded/Decoded values.
    • Measuring Agent - Data Collector: Bug Fix for supporting large measured Error Details.
  • Portal Server version 4.7.6 requires now Measuring Agent and Cluster Controller version 4.5.2

4.7.3 | 2022-8-28

  • New Features:
    • HTTP Test Wizard: New menu “URL Explorer” added. The URL Explorer shows all the details of the recorded HTTP session data and also supports searching within it. Furthermore, the URL Explorer also contains a “Variables Wizard” which displays all distinct values clearly sent in an HTTP session and makes it much easier to extract and assign session variables.
    • HTTP Test Wizard / Review and Edit URL Settings: This menu has been revised and enhanced. Two new auto-configurations wizards are available with which the content of HTTP responses can be automatically checked for keywords and with which the parallel execution of HTTP calls can be configured automatically.
    • HTTP Test Wizard / Debugger: UI improved.
    • A new “Text Phrase Tool” has added. This tool determines the most meaningful human-readable text phrases or keywords from any given content (HTML/JSON/XML). This tool has also been integrated into the HTTP Test wizard.
    • Remote Proxy Recorder: “URL Filter Quick Setting” menu added.
    • Remote Proxy Recorder: “Convert Recording to HTTP Test Wizard Session” improved and enhanced.
    • Portal Server (all menus): Large screen support has been improved.
    • Admin Menu: Mobile Companion API integrated and “Mobile Companion App - Test & Logs” menu added.
  • Bug Fixes:
    • HTTP Test Wizard / Generate Code: The measured statistic IDs of executed load tests have now the same values as the indexes of the HTTP Test Wizard session elements.
    • Load test programs generated by the HTTP Test Wizard did not correctly measure the response time in some cases when an HTTP response had no response content (bug fix in com.dkfqs.tools.jar, new version 2.3.0).
  • Portal Server version 4.7.3 requires now Measuring Agent and Cluster Controller version 4.5.0, and Remote Proxy Recorder version 1.0.0

4.6.8 | 2022-5-19

  • Bug Fixes:
    • Adding of cluster members to AWS/EC2 ‘Cluster Controller’ instances which have launched by cloud credits did fail.
    • The status of cluster jobs was not updated by the real-time monitor.
    • Deleting of test jobs did fail if the corresponding cluster was no longer defined.
    • Starting of cloud instances is now prevented if the portal’s internal monitoring system is no longer working.
  • Portal Server version 4.6.8 requires now Measuring Agent version 4.2.0 (bug fix: incorrect measurement results were reported sporadically)

4.6.7 | 2022-5-18

  • New Features:
    • AWS/EC2 ‘Remote Proxy Recorders’ can now be started by cloud credits.
    • Preview of ‘Remote Proxy Recorder CA Root Certificate’ added when downloading a remote proxy recorder CA root certificate.
    • 40 seconds countdown to left navigation added when launching EC2 instances by cloud credits.

4.6.5 | 2022-5-10

  • New Features:
    • Support for team member accounts added:
      • Depending on the license (price plan), a normal user (main user) can also define sub-users (so-called team member accounts).
      • Team members can login as usual, with their email and password.
      • Team members can either have the same rights as the main user or alternatively only have read rights.
      • The team members (including the main user) can communicate with each other via push messages. The portal also shows which team member is currently online. Offline team members receive an email instead of an online push notification.
      • A list of all team members is provided for each team member (inclusive the main user), which contains the profile text and profile image of each team member.
      • Team member accounts are not portal-wide ‘public users’, and cannot act as public technical expert.
  • Bug Fixes:
    • Deleting of test jobs did fail if the corresponding measuring agent was no longer defined.

4.6.4 | 2022-4-18

  • New Features:
    • Support of special licenses to top up the amount of ‘Cloud Credits’.
    • License restrictions on the maximum number of Measuring Agents and Remote Proxy Recorders no longer apply when such components are launched by spending Cloud Credits.
    • The Web Browser interface has been optimized to support large screens.

4.6.2 | 2022-4-12

  • New Features:
    • Support of ‘Sign in with Microsoft’ added.

4.6.0 | 2022-4-10

  • New Features:
    • The list of excluded AWS/EC2 regions can now be edited in the admin menu.
    • The additional data transfer costs of AWS/EC2 instances which are started via the Portal Server are now billed via Cloud Credits.
    • A new Measuring Agent version 4.1.0 is available which is able to automatically configure the Java memory of the ‘Data Collector’ and the Java memory of executed Java test programs.
  • Portal Server version 4.6.0 supports Measuring Agent version 4.0.4 and version 4.1.0

In order to enable the automatic Java memory configuration of a Measuring Agent 4.1.0, the following Java program arguments must be set in the startup script: -autoAdjustMemory -osReservedMemory 1GB

Furthermore, the Java memory of the Measuring Agent should be set in the startup script as shown in the table below:

OS Physical Memory Measuring Agent Java -Xmx setting
<2 GiB 256m
2..3 GiB 512m
4..7 GiB 512m
8..15 GiB 1536m
16..31 GiB 3072m
32..63 GiB 4096m
64..96 GiB 6144m
>96 GiB 8192m
Odd number of GiB should be rounded up (e.g. 7.7 = 8 = 1536m).

Example: sudo -H -u dkfqs bash -c ‘CLASSPATH=/home/dkfqs/agent/bin/bcpkix-jdk15on-160.jar:/home/dkfqs/agent/bin/bcprov-jdk15on-160.jar:/home/dkfqs/agent/bin/bctls-jdk15on-160.jar:/home/dkfqs/agent/bin/DKFQSMeasuringAgent.jar;export CLASSPATH;nohup java -Xmx512m -DdkfqsMeasuringAgentProperties=/home/dkfqs/agent/config/measuringagent.properties -Dnashorn.args="–no-deprecation-warning" com.dkfqs.measuringagent.internal.StartDKFQSMeasuringAgent -autoAdjustMemory -osReservedMemory 1GB 1>/home/dkfqs/agent/log/MeasuringAgent.log 2>&1 &’

4.5.0 | 2022-3-27

  • New Features:
    • Amazon AWS/EC2 instances of ‘Measuring Agents’ and ‘Cluster Controllers’ can now be started directly in the Portal Server - without the customer needing an Amazon AWS account. The costs of such started AWS/EC2 instances are charged to the customer via so-called ‘Cloud Credits’, whereby the number of ‘Cloud Credits’ can be purchased as an integral part of a license or can also be purchased separately. For customers who register for the first time at the Portal Server, a free amount of ‘Cloud Credits’ will be credited to allow them try out this functionality. Note: It is still fully supported to launch Amazon AWS/EC2 instances of ‘Measuring Agents’ and ‘Cluster Controllers’ with your own Amazon AWS account by using the standalone tool ‘Desktop Companion’.
    • User Top Navigation: Link to Download Server and to Real Load Store added (both links are manageable by the administrator menu).
    • HTTP Test Wizard: Review and Edit URLs menu added.
    • Launched Cloud Instances menu added.
    • Cloud Credit Statement menu added.
    • Measuring Agents & Cluster Controllers menu: The ‘List of Predefined AWS/EC2 Measuring Agent AMIs’ shows no longer incompatible AMI versions.
    • The following new functions have been added for administrators:
      • User Accounts menu: Cloud Credit Statement menu per user added.
      • Server Settings menu: AWS/EC2 Cloud Provider API Settings section added.
      • Customize Auth Users Navigation menu: Download Components and Shop sections added.
      • Cloud Providers & Instance Type Credit Costs menu added.
      • Inspect AWS/EC2 Cloud Instances menu added.
      • Launched Cloud Instances menu added.
      • Cloud Credit Transactions menu added.

Advantages and disadvantages of ‘Cloud Credits’ versus launching AWS/EC2 instances by the ‘Desktop Companion’:

Benefits of ‘Cloud Credits’:

  • Full cost control. Cloud instances are automatically terminated after a selected time.
  • Easy to use. Seamless integration with the portal server.
  • No Amazon AWS account required and therefore no security issue for unauthorized AWS access.

Advantages of the ‘Desktop Companion’:

  • Lower costs for cloud instances compared to ‘Cloud Credits’(AWS self-cost are charged only).
  • Suitable for customers who use cloud instances frequently and are aware of AWS/EC2 costs involved.

4.4.4 | 2022-1-30

  • New Features:
    • Load-releasing Clusters of Measuring Agents are now supported. In order to operate a cluster, a separate process/component is required (so-called “Cluster Controller”) which is not part of the portal server and is operated by the customer himself. Cluster-Features:
      • Multiple clusters can be registered in the portal server using the UI, and the cluster members (measuring agents) can be assigned to the clusters with simple clicks.
      • Depending on the power of the Cluster Controller, a cluster can have several 100 members, which allows to perform load tests of almost unlimited strength (up to millions of concurrent users).
      • A cluster job can be started as easily as a normal job. The only difference is that a cluster is selected for execution instead of a single measuring agent.
      • Optionally, the content of input files can be automatically divided among the cluster members (e.g. if they contain user account credentials).
      • Cluster jobs support all runtime features like normal jobs: suspend job, resume job, stop job and kill job. Entering annotations at runtime is also supported.
      • The real-time display of cluster jobs is exactly the same as for normal jobs, but the statistics displayed at real time already contain the combined values of all cluster members. In addition, a table about the current operating status of each cluster member is available.
      • After a cluster job has been completed, the test result contains the combined values of all cluster members (analogous to the real-time display). In addition, the individual test results of the cluster members can also be displayed.
    • The following functions have been added to the “Remote User API”:
      • getMinRequiredMeasuringAgentVersion
      • getMinRequiredClusterControllerVersion
      • getMinRequiredProxyRecorderVersion
      • getMeasuringAgentClusters
      • getClusterControllers
      • pingClusterController
      • addMeasuringAgentCluster
      • addMemberToMeasuringAgentCluster
      • removeMemberFromMeasuringAgentCluster
      • pingMeasuringAgentClusterMembers
      • setMeasuringAgentClusterActive
      • deleteMeasuringAgentCluster
    • Portal Server “Measuring Agents” menu: The AWS/EC2 popup highlights now incompatible AMI versions.
    • Portal Server “Test Jobs” menu: The log files of executed jobs can now directly copied to the project tree.
  • Bug Fixes:
    • A test job can now also have the state “execution failed”.
  • Portal Server version 4.4.4 requires now Measuring Agent version 4.0.4

4.3.23 | 2021-11-13

  • New Features:
    • HTTP Test Wizard: New Session Element ‘Outbound IP Address’ added. In order that this element can be used, multiple valid IP addresses must be assigned to the network interface of the Measuring Agent(s).
    • HTTP Test Wizard: Support of specific HTTP processing timeout per URL added.
    • Test Results: Support of annotation and annotation events added. The annotation and annotation events can be added at runtime during the test and are shown in the test result. In addition, annotation events can also be reported via the All Purpose Interface and generated, for example, by HTTP Test Wizard plug-ins.
  • Bug Fixes:
    • The cached HTTP Test Wizard Session in the Portal Server UI is now updated when a variable is defined, modified or deleted, and when a variable extractor or a variable assigner is deleted.
    • Invalid Java code was generated by the HTTP Test Wizard if the error handling of URLs was set to ‘Continue as usual’.
    • Performance bottleneck fixed in com.dkfqs.tools.http.HTTPClient (occurred since previous version 4.3.22).
    • An incorrect error exception was thrown in com.dkfqs.tools.crypto.EncryptedSocket when an SSL handshake timeout occurred
    • Realtime charts sometimes showed inexact measurement results.
  • Portal Server version 4.3.23 requires now Measuring Agent version 3.9.33 and DKFQS Tools version 2.2.25

4.3.22 | 2021-10-24

  • Documentation: All Purpose Interface added.
  • The All Purpose Interface has been extended by 3 new statistics types that can be declared at runtime:
    • average-and-current-value : An average and current value
    • efficiency-ratio-percent : An efficiency ratio in percent (0..100%)
    • throughput-time-chart : A chart of a throughput per second
  • The measurement results of the HTTP Test Wizard now contain the following additional test-specific values (if the executed HTTP Test Wizard session contains URL session elements):
    • Total Bytes Sent
    • Total Bytes Received
    • Network Throughput in Mbps (real-time: current value, test result: average value and chart)
    • Average TCP Connect Time in milliseconds (real-time: + latest value)
    • Average SSL Handshake Time in milliseconds (real-time: + latest value)
    • HTTP Keep-Alive Efficiency (0..100%)
  • Bug fix: The time in the name of the test result files is now always set in the time zone of the portal server, regardless of which time zone the Measuring Agents are operated.
  • Portal Server version 4.3.22 requires now Measuring Agent version 3.9.32 and DKFQS Tools version 2.2.24

4.3.21 | 2021-09-18

  • Searching for a text in all files of the project tree is now supported.
  • Support to upload files by drag and drop added.
  • Bug fix for assigning variables to HTTP requests in HTTP Test Wizard debugger.
  • Bug fix on real time statistics and test results if the measured unit is other than ms.
  • User profile images can now have a max size of 400 KB instead of 200 KB.
  • CA Root Certificates of HTTP/S Proxy Recorder(s) can now be downloaded via the Portal Server UI.
  • Documentation: User Guide added.
  • SNMP Plug-In published at https://portal.realload.com/publicPublishedPlugins

Portal Server Installation / Ubuntu 20: The “fontconfig” package has to be installed in order that the captcha generator is working:

sudo apt-get update
sudo apt-get install fontconfig

4.3.20 | 2021-08-07

  • HTTP Test Wizard Plug-Ins can now be published and are then available to other users.
  • The following HTTP Test Wizard Plugin fields are now protected by algorithms and cannot be manually modified by the JSON editor:
    • pluginId
    • authorNickname
  • A (new) “Resources Library” project is automatically created for each user in the Project Tree which contains by default the following “Resource Sets”:
    • “HTTP Test Wizard Plug-Ins”: by default empty
    • “Java”: contains always the latest version of com.dkfqs.tools.jar
    • “PowerShell”: contains always the latest version of DKFQSLibrary2.psm1
  • The “Resources Library” project contains common resources which are used by multiple projects. The users can add additional resource sets and files as needed to this project.
  • The file com.dkfqs.tools.jar is no longer copied to the corresponding resource set when a HTTP Test Wizard test is generated. Instead of this the generated test contain now a reference to “Resources Library / Java / com.dkfqs.tools.jar”.
  • New created HTTP Test Wizard Plug-Ins contain now by default a reference to the resource file “Resources Library / Java / com.dkfqs.tools.jar”.
  • More than 50 minor bugs have been fixed and some improvements have been made to the portal user interface.
  • Portal Server version 4.3.20 requires now Measuring Agent version 3.9.31
  • Existing tests and plug-ins should be upgraded to use com.dkfqs.tools.jar version 2.2.21 (located at “Resource Sets / Java”). This means that the tests have to be generated and defined once again.
  • The tuning parameters of Linux operating systems on which “Measuring Agents” run must be increased:

in /etc/security/limits.conf add or modify:

# TCP/IP Tuning
# =============
* soft     nproc          262140
* hard     nproc          262140
* soft     nofile         262140
* hard     nofile         262140
root soft     nproc          262140
root hard     nproc          262140
root soft     nofile         262140
root hard     nofile         262140

Enter: systemctl show -p TasksMax user-0

output: TasksMax=8966

if you get a value less than 262140 then add in /etc/systemd/system.conf

# System Tuning
# =============
DefaultTasksMax=262140

Reboot the system and verify the settings. Enter: ulimit -n

output: 262140

Enter: systemctl show -p TasksMax user-0

output: TasksMax=262140

4.3.18 | 2021-07-06

  • Support of Licenses added. Multiple licenses per user account are supported. New licenses can be entered via the Admin Menu and via the User Menu. Users whose license has expired can enter a new license during sign-in.
  • Pluggable Architecture for License Providers implemented.
  • Remote Admin API and Remote User API added. The corresponding API Authentication Tokens can be generated via the Admin Menu and via the User Menu. See API Documentation.
  • Unix Time Tool in User Menu added.
  • User accounts that are since a long time expired are now automatically deleted - including all user specific data. The number of days between the expiry date and the deletion date can be configured in the Admin Menu.
  • Deleted user accounts can now be viewed in the Admin Menu.
  • Portal Server version 4.3.18 requires now Measuring Agent version 3.9.30

4.3.14 | 2021-04-08

  • Support of multiple Price Plans / Limits for User Accounts such as Disk-Space, Number of Measuring Agents, Account expires time … (Admin Menu).
  • Access to Measuring Agents can now (optional) protected by an Authentication Token (password).
  • Configurable Default Price Plan for Sign Up (Admin Menu).
  • Configurable HTML content for Sign Up steps 1 to 4 and for (new) Sign Up completed “Welcome Page” (Admin Menu).
  • Test jobs are now digitally signed. This means that the following job settings on a Measuring Agent cannot be modified after the job has been transmitted to the Measuring Agent: type of job, local test job ID, number of users, maximum test duration.

4 - Synthetic Monitoring

Synthetic Monitoring User’s Guide

Synthetic Monitoring is an essential part of Digital Experience Monitoring, where organizations can detect their service outages or performance degrade proactively by periodically running the tests.

RealLoad provides an easy way of configuring synthetic tests with an outcome of accurate metrics, alerting and SLA details.

RealLoad Synthetic Monitoring Features

  • Due to the universal architecture, RealLoad Tests of any kind can be run as a Monitoring Job (HTTP Test Wizard tests, JUnit tests, Selenium IDE tests).
  • Monitoring Jobs can be assigned to multiple Measuring Agents, which means that the job is executed from different geographical locations. The measurement results of the Measuring Agents are combined into a single result, whereby the individual result of each Measuring Agent is still available.
  • Support for performance warning thresholds.
  • Support for delayed alarm notifications.
  • Supported alert devices: Email, Mobile Companion App (installed on a mobile phone), SMS and WebHooks (including Pagerduty and Slack).
  • Synthetic monitoring jobs can also run with hundreds of virtual users ¹
    ¹ = does not apply to Selenium IDE and Playwright Tests (max. 5..50 users per Measuring Agent due to the high CPU usage)
  • Custom dashboards can be created for third-party products like Grafana using the WebSocket API.

Prerequisite: Define a Test

You have first to define a RealLoad ‘Test’ before you can add it to Synthetic Monitoring.

Configure Synthetic Monitoring

First add a Monitoring Group and then add one or more Monitoring Jobs to the Monitoring Group. If you want to run Monitoring Jobs with different intervals, you can also define several Monitoring Groups. Measuring Agents will be available for selection based on how many agents are already configured on different geographical locations.

Add Monitoring Group

“alt attribute”

“alt attribute”

Input Fields:

  • Group Title : Must be unique across all monitoring groups.
  • Group Description : Optional description.
  • Max. Data Storage Time : The maximum retention period of the measured data.
  • Execution Interval : The execution interval of the Monitoring Jobs.
  • Execution Timeout : The execution timeout of the Monitoring Group. Tip: do not choose this too short, otherwise the Monitoring Jobs will be aborted. The execution timeout can also be greater than the execution interval. The monitoring jobs of a monitoring group are executed in sequential order.
  • Primary Measuring Agent & Additional Measuring Agents : Select the Measuring Agent(s) on which the Monitoring Jobs will be executed.

After you have added a monitoring group, its execution is switched off. Switch on the execution now:

“alt attribute”

Add Monitoring Job

In a Monitoring Group, first click on ‘Monitoring Jobs’ and then in the Monitoring Jobs area on ‘Add Monitoring Job’:

“alt attribute”

Then select a Test …

“alt attribute”

… And Configure and Define the Monitoring Job:

“alt attribute”

Input Fields:

  • Monitoring Job Title : Required.
  • Monitoring Job Description : Optional description.
  • Job Execution Enabled : Controls whether the job is executed.
  • Number of Users : The number of simulated users that are started.
  • Max. Test Duration : The maximum test duration.
  • Max. Loops per User : The maximum number of sessions executed per user (the number of iterations of the test per user).
  • Loop Iteration Delay : The delay time after a session of a user has ended until the next session of the user is started.
  • Ramp Up Time : The length of time at the beginning of the test until all simulated users are started. Example: with 20 simulated users and a time of 10 seconds, a new user is started each 0.5 seconds.
  • Additional Arguments : Additional values which are transferred on the command line when the test script is started. These arguments are test specific. For tests that were created with the “HTTP Test Wizard” you can specify for example the following values: “-tcpTimeout 8000 -sslTimeout 5000 -httpTimeout 60000” (TCP connect timeout / SSL handshake timeout / HTTP processing timeout) which are considered by the executed URL calls and override the default values.
  • Debug Execution : This option effects that detailed information are written to the log file of the test. For example variable values which have been extracted from input files or from HTTP responses as well as variable values which are assigned at runtime. Only activate this option if you have problems with the execution of the test.
  • Debug Measuring : Effects that the Data Collector of the Measuring Agent writes the JSON objects of the All Purpose Interface to its log file. This option can be enabled to debug self-developed tests that have been written from scratch.
  • (User Input Fields) : If the test contains User Input Fields you can enter their values here.
  • Performance Warning Alert Threshold : If switched on, a warning alert is triggered if the configured time per user loop (per user session) is exceeded.
  • Performance Error Alert Threshold : If switched on, a error alert is triggered if the configured time per user loop (per user session) is exceeded.

Normally you do not have to enter any “Additional Arguments” and leave “Debug Execution” and “Debug Measuring” switched off.

Tip: We recommend that you leave options ‘Performance Warning Alert Threshold’ and ‘Performance Error Alert Threshold’ switched off for now and configure them later - after the new defined Monitoring Job has been running for a few hours.

The following limits apply for Monitoring Jobs which are executed on shared Measuring Agents:

  • Number of Users: max. 5
  • Max. Test Duration (Seconds): max. 300
  • Max. Loops per User: max. 5

If you operate your own, private Measurement Agents (or have us to operate them), these limits do not apply - meaning you can run Monitoring Jobs with hundreds or thousands of concurrent users.

Alerting

Proceed as follows:

  1. Add one or more Alert Groups.
  2. Define the Alert Devices and assign them to the Alert Group(s).
  3. Assign the Alert Groups to Monitoring Groups and/or to Monitoring Jobs

Add Alert Group

First click on the ‘Alert Groups & Devices’ tab and then click on ‘Add Alert Group’.

“alt attribute”

“alt attribute”

Input Fields:

  • Alert Group Title : Must be unique across all alert groups.
  • Alert Group Description : Optional description.
  • Alert Group Enabled : Controls whether the alert group is enabled or disabled.
  • Report Measured Errors : Controls whether errors measured by monitoring jobs are reported.
  • Report Measured Warnings : Controls whether warnings measured by monitoring jobs are reported.
  • Report System Failures : Controls whether malfunctions of the monitoring system are reported (for example if a Measuring Agent is not reachable).
  • Report Execution Failures : Controls whether execution failures of monitoring jobs on Measuring Agents are reported (meaning that the start of the corresponding test job failed).
  • Repeat Alerts : Controls whether the alerts for the same issue are reported repeatedly.
  • Report Alerts ‘Instantly’ or ‘After n Repeat’ : Controls whether alerts are reported immediately or delayed.

Tip: For example, you can define two alert groups, one for operations and another for developers. In the developer alert group, you could turn off the switches ‘Report System Failures’ and ‘Report Execution Failures’.

Add Alert Devices

First expand the area of the alert device type and then click the ‘+’ icon:

“alt attribute”

“alt attribute”

After adding the alert device, you should test it:

“alt attribute”

“alt attribute”

Assign Alert Device to Alert Groups

After an Alert Device has been added, you can assign it to one or more Alert Groups.

“alt attribute”

“alt attribute”

Assign Alert Groups to Monitoring Groups and Monitoring Jobs

To receive Alert Notifications, you must assign the Alert Groups to Monitoring Groups and/or to Monitoring Jobs.

“alt attribute”

If you click on a bell icon you can only assign one Alert Group. However, you can click the icon multiple times to assign multiple Alert Groups.

Synthetic Real-Time Dashboard

Now the created monitoring groups with its monitoring jobs will be available under the Real-Time Dashboard like below with the important metrics and also have the option to drill down into test results

Custom dashboards can be created on third party products like Grafana, using our WebSocket API

Statistics with Performance & Availability

Next explains about the statistics data, which will be the source for many KPIs , for example to calculate the defined SLA for an application or an API service

5 - API

Portal Server APIs

The portal server has three APIs:

  • Remote Admin API
  • Remote User API
  • Remote User Monitoring API

All APIs require so-called “Authentication Tokens” to verify the access authorization. If 5 or more invalid “Authentication Tokens” are sent within 60 seconds, the corresponding remote IP address is blocked for 30 minutes.

5.1 - Remote Admin API

Portal Server Remote Admin API Specification

Generating an Authentication Token

To perform a Remote Admin API call, you must first generate an “Admin API Auth Token” in the Administrator Menu of the portal. When generating the token you can enter a purpose (only used as information) and also restrict the remote IP addresses for which the token is valid. You can also specify whether the token has read/write access or read/only access.

“alt attribute”

API Functions

The API supports the following functions (so-called “actions”):

  • getAllUserAccounts (Get all user accounts of the portal server)
  • getAllPricePlans (Get all price plans of the portal server)
  • addLicenseCertificateToUser (Add a (new) license certificate to an user)
  • getServerSettings (Get the server settings)
  • setServerMaintenanceMode (Turn the server maintenance mode on or off)
URL HTTP Method
https://portal.realload.com/RemoteAdminAPI POST
All data are sent and received in JSON data format. The “authTokenValue” and the “action” must always be sent when an API call is made.

Example

API HTTP/S Request

curl -v --request POST --header "Content-Type: application/json" --data "@getAllUserAccounts.json"  https://portal.realload.com/RemoteAdminAPI

API Request Data

{
  "authTokenValue": "8mKSz1UzaQg17kfu",
  "action": "getAllUserAccounts"
}

API Response Data

{"allUserAccountsArray":
[{"userId":13,"nickname":"DKF","firstName":"Max","lastName":"Fischer","primaryEmail":"max@dkfqa.com","primarySMSPhone":"+41771111111","secondaryEmail":"","secondarySMSPhone":"","accountBlocked":false,"accountCreateTime":1538556183756,"lastLoginTime":1625181623869,"lastLoginIP":"127.0.0.1","pricePlanId":1,"accountExpiresTime":-1,"pricePlanTitle":"Unlimited"},{"userId":18,"nickname":"AX","firstName":"Alex","lastName":"Fischer","primaryEmail":"alexfischer66@yahoo.com","primarySMSPhone":"+41781111111","secondaryEmail":"","secondarySMSPhone":"","accountBlocked":false,"accountCreateTime":1539874749677,"lastLoginTime":1616111301975,"lastLoginIP":"127.0.0.1","pricePlanId":1,"accountExpiresTime":-1,"pricePlanTitle":"Unlimited"},{"userId":22,"nickname":"Kes","firstName":"Kesorn","lastName":"Fischer","primaryEmail":"gsklsta@yahoo.com","primarySMSPhone":"+66000000000","secondaryEmail":"","secondarySMSPhone":"","accountBlocked":false,"accountCreateTime":1605303204754,"lastLoginTime":1624389324770,"lastLoginIP":"127.0.0.1","pricePlanId":6,"accountExpiresTime":-1,"pricePlanTitle":"BASIC1"},{"userId":48,"nickname":"BET","firstName":"Bettina","lastName":"Meier","primaryEmail":"b123456@lucini.id.au","primarySMSPhone":"+61404905702","secondaryEmail":"","secondarySMSPhone":"","accountBlocked":false,"accountCreateTime":1623719604561,"lastLoginTime":-1,"lastLoginIP":"","pricePlanId":6,"accountExpiresTime":1625061600000,"pricePlanTitle":"BASIC1"}],
"isError":false}

If the API call is successful, then the response field “isError” is false. If a numeric field has a value of -1 (minus one), this means “no data” or “unlimited” depending on the context.

getAllUserAccounts

Specific Request Fields:

  • [none]

Specific Error Flags:

  • [none]

getAllPricePlans

Specific Request Fields:

  • [none]

Specific Error Flags:

  • [none]

addLicenseCertificateToUser

Specific Request Fields:

  • mapToUserEmailAddress
  • mapToUserMobilePhone
  • licenseProvider
  • licenseCertificate

The license is successfully assigned to a user if either mapToUserEmailAddress or mapToUserMobilePhone matches to a user account.

Specific Error Flags:

  • writeAccessError
  • mapToUserError
  • licenseProviderError
  • licenseCertificateError
  • licenseCertificateAlreadyAddedError
  • pricePlanError

JSON Request Example:

{
  "authTokenValue":"8mKSz1UzaQg17kfu",
  "action":"addLicenseCertificateToUser",
  "licenseProvider": "Real Load Pty Ltd / nopCommerce",
  "mapToUserEmailAddress": "max@dkfqa.com",
  "mapToUserMobilePhone": "+41771111111",
  "licenseCertificate": "-----BEGIN CERTIFICATE-----\r\nMIIEnjCCA4agAwIBAgIEyDnukzANBgkqhkiG9w0BA ...... Hn/UMGAGRB6xF4w+TewYqTAZrdhi/WLyYwg==\r\n-----END CERTIFICATE-----"
}

JSON Response Example (Success Case):

{"licenseId":12,"cloudCreditLicenseId":-1,"userId":13,"isCloudCreditsLicense":false,"isError":false}

JSON Response Example (Error Case):

{"isError":true,"genericErrorText":"","writeAccessError":false,"licenseProviderError":false,"mapToUserError":false,"pricePlanError":false,"licenseCertificateAlreadyAddedError":false,"licenseCertificateError":true}

getServerSettings

Specific Request Fields:

  • [none]

Specific Error Flags:

  • [none]

JSON Response Example:

{
  "isServerMaintenanceMode":false,
  "isSignInSelectPricePlanFromMultipleValidLicenseCertificates":true,
  "isSignInExpiredAccountCanEnterLicenseCertificate":true,
  "isSignUpEnabled":true,
  "isSignUpRequiresInvitationTicket":false,
  "signUpDefaultPricePlanId":2,
  "signUpDefaultAccountExpiresInDays":14,
  "deleteExpiredUserAccountsAfterDays":183,
  "isError":false
}

setServerMaintenanceMode

Specific Request Fields:

  • serverMaintenanceMode

Specific Error Flags:

  • writeAccessError

JSON Request Example:

{
  "authTokenValue":"8mKSz1UzaQg17kfu",
  "action":"setServerMaintenanceMode",
  "serverMaintenanceMode":true
}

JSON Response Example (Success Case):

{"isServerMaintenanceMode":true,"isError":false}

5.2 - Remote User API

Portal Server Remote User API Specification

Generating an Authentication Token

To perform a Remote User API call, you must first sign in into the portal and generate an “API Authentication Token”. When generating the token you can enter a purpose (only used as information) and also restrict the remote IP addresses for which the token is valid.

“alt attribute”

“alt attribute”

API Functions

The API supports the following functions (so-called “actions”):

Common Functions:

  • getUserAccountInfo (Get information about the own user account)
  • getPricePlanInfo (Get information about the current price plan)

Projects, Resource Sets and Files Functions:

  • getProjectTree (Get the project tree inclusive all resource sets and all file information)
  • createProject (Create a new project)
  • deleteProject (Delete a project)
  • getResourceSetsOfProject (Get all resource sets of a project, inclusive all file information)
  • createResourceSet (Create a new resource set)
  • deleteResourceSet (Delete a resource set)
  • getFilesInfoOfResourceSet (Get all files information of a resource set)
  • createFile (Create or overwrite a file)
  • getFile (Get the content of a file and the file information)
  • deleteFile (Delete a file)

Measuring Agents Functions:

  • getMeasuringAgents (Get all defined measuring agents)
  • getMinRequiredMeasuringAgentVersion (Get the minimum required measuring agent version)
  • addMeasuringAgent (Add a new measuring agent)
  • pingMeasuringAgent (Ping a measuring agent)
  • setMeasuringAgentActive (Set the state of a measuring agent to active or inactive)
  • deleteMeasuringAgent (Delete a measuring agent)

Measuring Agent Cluster Functions:

  • getMeasuringAgentClusters (Get all defined measuring agent clusters)
  • getClusterControllers (Get all cluster controllers and for each cluster controller a list of measuring agent clusters which are referencing the cluster controller)
  • getMinRequiredClusterControllerVersion (Get the minimum required cluster controller version)
  • pingClusterController (Ping a cluster controller)
  • addMeasuringAgentCluster (Add a new measuring agent cluster)
  • addMemberToMeasuringAgentCluster (Add a member to a measuring agent cluster)
  • removeMemberFromMeasuringAgentCluster (Remove a member from a measuring agent cluster)
  • pingMeasuringAgentClusterMembers (Ping the cluster members of a measuring agent cluster via cluster controller)
  • setMeasuringAgentClusterActive (Set the state of a measuring agent cluster to active or inactive)
  • deleteMeasuringAgentCluster (Delete a measuring agent cluster)

HTTP/S Remote Proxy Recorders Functions:

  • getProxyRecorders (Get all defined HTTP/S proxy recorders)
  • getMinRequiredProxyRecorderVersion (Get the minimum required HTTP/S proxy recorder version)
  • addProxyRecorder (Add a new HTTP/S proxy recorder)
  • pingProxyRecorder (Ping a HTTP/S proxy recorder)
  • deleteProxyRecorder (Delete a HTTP/S proxy recorder)
  • getProxyRecordersClientCertificates (Get all client certificates for HTTP/S proxy recorders)
  • addProxyRecordersClientCertificate (Add a client certificate for HTTP/S proxy recorders)
  • setProxyRecordersClientCertificateActive (Set a client certificate for HTTP/S proxy recorders to active or inactive)
  • modifyProxyRecordersClientCertificate (Modify a client certificate for HTTP/S proxy recorders)
  • deleteProxyRecordersClientCertificate (Delete a client certificate for HTTP/S proxy recorders)

Load Test Jobs Functions:

  • getTestjobTemplates (Get all test job templates)
  • defineNewTestjobFromTemplate (Define new test job from template)
  • submitTestjob (Submit a ‘defined’ test job to the measuring agent or to the cluster controller)
  • makeTestjobReadyToRun (Turn a remote test job which is in the state ‘submitted’ into the state ‘ready to run’)
  • startTestjob (Turn a remote test job which is in the state ‘ready to run’ into the state ‘running’)
  • getMeasuringAgentTestjobs (Get from a measuring agent all test jobs, inclusive their states)
  • getTestjobOutDirectoryFilesInfo (Get from test job information about all files located in a measuring agent job or cluster controller job ‘out’ subdirectory)
  • getTestjobOutDirectoryFile (Get from a measuring agent or from a cluster controller the content of a file located in a testjob/cluster-job out subdirectory in base64 format)
  • saveTestjobOutDirectoryFileToProjectTree (Get from a measuring agent or from a cluster controller the content of a file located in a testjob/cluster-job out subdirectory and save it in a resource set of the project tree)
  • deleteTestjob (Delete a test job locally and also on the measuring agent or cluster controller)

Test Execution Suites Functions:

  • getTestExecutionSuites (Get all test execution suites with a summary of all results of the executed test runs)
  • getCurrentlyRunningTestExecutionSuiteIds (Get the ID(s) of all currently running test execution suites)
  • startTestExecutionSuiteTestRun (Start a new run of a test execution suite)
  • getTestExecutionSuiteTestRunResultDetail (Get a test run result detail of a test execution suite)
  • getTestExecutionSuiteTestRunTestJobOutputFileContent (Get the content of a test job output file of a test execution suite run)
  • deleteTestExecutionSuiteTestRun (Delete a test run of a test execution suite)
URL HTTP Method
https://portal.realload.com/RemoteUserAPI POST
All data are sent and received in JSON data format. The “authTokenValue” and the “action” must always be sent when an API call is made.

Example

API HTTP/S Request

curl -v --request POST --header "Content-Type: application/json" --data {\"authTokenValue\":\"jPmFClqeDUXaEk8Q274q\",\"action\":\"getUserAccountInfo\"}  https://portal.realload.com/RemoteUserAPI

API Request Data

{
  "authTokenValue": "jPmFClqeDUXaEk8Q274q",
  "action": "getUserAccountInfo"
}

API Response Data

{
  "userAccountInfo":{
    "userId":48,
    "nickname":"BET",
    "firstName":"Bettina",
    "lastName":"MeierHans",
    "primaryEmail":"bettina@meierhans.id.au",
    "primarySMSPhone":"+61401111111",
    "secondaryEmail":"",
    "secondarySMSPhone":"",
    "accountBlocked":false,
    "accountCreateTime":1623719604561,
    "lastLoginTime":1625348376450,
    "lastLoginIP":"127.0.0.1",
    "pricePlanId":6,
    "accountExpiresTime":1625953109397,
    "pricePlanTitle":"BASIC1"
    },
  "isError":false
}

If the API call is successful, then the response field “isError” is false. If a numeric field has a value of -1 (minus one), this means “no data” or “unlimited” depending on the context.

getUserAccountInfo

Specific Request Fields:

  • [none]

Specific Error Flags:

  • [none]

getPricePlanInfo

Specific Request Fields:

  • [none]

Specific Error Flags:

  • [none]

JSON Response Example:

{
  "pricePlanInfo":{
    "pricePlanId":6,
    "title":"BASIC1",
    "description":"",
    "isDeprecated":false,
    "lastModified":1625348413042,
    "maxDiskSpaceMB":1024,
    "maxSubUserAccounts":0,
    "maxMeasuringAgentsOwnedByUser":3,
    "maxRemoteProxyRecordersOwnedByUser":3,
    "executeLoadJobsEnabled":true,
    "executeMonitoringJobsEnabled":false,
    "apiAccessEnabled":true,
    "maxStartLoadJobsLast24h":24,
    "maxUsersPerLoadJob":500,
    "maxDurationPerLoadJob":1800
  },
  "isError":false
}

The unit for “maxDurationPerLoadJob” is seconds.

getProjectTree

Specific Request Fields:

  • [none]

Specific Error Flags:

  • [none]

JSON Response Example:

{
  "projectsArray": [
    {
      "projectId": 97,
      "projectName": "Common",
      "projectDescription": "",
      "resourceSetsArray": [
        {
          "resourceSetId": 154,
          "resourceSetName": "Input Files",
          "resourceSetDescription": "",
          "filesArray": [
            {
              "fileName": "InputFile.txt",
              "fileSize": 233,
              "fileHashCode": 1873256029,
              "fileLastModified": 1613835992073
            }
          ]
        },
        {
          "resourceSetId": 155,
          "resourceSetName": "Jar Files",
          "resourceSetDescription": "",
          "filesArray": [
            {
              "fileName": "com.dkfqs.tools.jar",
              "fileSize": 578087,
              "fileHashCode": -2033420926,
              "fileLastModified": 1613838181727
            }
          ]
        },
        {
          "resourceSetId": 156,
          "resourceSetName": "Plug-Ins",
          "resourceSetDescription": "",
          "filesArray": [
            {
              "fileName": "HttpSessionPlugin_ChangeCopyright.json",
              "fileSize": 5321,
              "fileHashCode": 1958407366,
              "fileLastModified": 1613838287871
            }
          ]
        }
      ]
    },
...
...
...
  ],
  "isError": false
}

createProject

Specific Request Fields:

  • projectName
  • projectDescription (optional)

Response Fields:

  • projectId

Specific Error Flags:

  • projectNameError
  • diskSpaceLimitExceededError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"createProject",
  "projectName":"My New Project",
  "projectDescription": "Created by API call"
}

JSON Response Example (Success Case):

{"projectId":113,"isError":false}

JSON Response Example (Error Case):

{"isError":true,"genericErrorText":"","diskSpaceLimitExceededError":false,"projectNameError":true}

deleteProject

Specific Request Fields:

  • projectId
  • moveToTrash (optional, default: false)

Specific Error Flags:

  • projectIdError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"deleteProject",
  "projectId":113,
  "moveToTrash":false
}

getResourceSetsOfProject

Specific Request Fields:

  • projectId

Specific Error Flags:

  • projectIdError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getResourceSetsOfProject",
  "projectId":97
}

JSON Response Example (Success Case):

{
  "resourceSetsArray": [
    {
      "resourceSetId": 154,
      "resourceSetName": "Input Files",
      "resourceSetDescription": "",
      "filesArray": [
        {
          "fileName": "InputFile.txt",
          "fileSize": 233,
          "fileHashCode": 1873256029,
          "fileLastModified": 1613835992073
        }
      ]
    },
    {
      "resourceSetId": 155,
      "resourceSetName": "Jar Files",
      "resourceSetDescription": "",
      "filesArray": [
        {
          "fileName": "com.dkfqs.tools.jar",
          "fileSize": 578087,
          "fileHashCode": -2033420926,
          "fileLastModified": 1613838181727
        }
      ]
    },
    {
      "resourceSetId": 156,
      "resourceSetName": "Plug-Ins",
      "resourceSetDescription": "",
      "filesArray": [
        {
          "fileName": "HttpSessionPlugin_ChangeCopyright.json",
          "fileSize": 5321,
          "fileHashCode": 1958407366,
          "fileLastModified": 1613838287871
        }
      ]
    }
  ],
  "isError": false
}

createResourceSet

Specific Request Fields:

  • projectId
  • resourceSetName
  • resourceSetDescription (optional)

Response Fields:

  • resourceSetId

Specific Error Flags:

  • projectIdError
  • resourceSetNameError
  • diskSpaceLimitExceededError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"createResourceSet",
  "projectId":97,
  "resourceSetName":"My New Resource Set",
  "resourceSetDescription": "Created by API call"
}

JSON Response Example (Success Case):

{"resourceSetId":172,"isError":false}

JSON Response Example (Error Case):

{"isError":true,"genericErrorText":"","resourceSetNameError":true,"projectIdError":false,"diskSpaceLimitExceededError":false}

deleteResourceSet

Specific Request Fields:

  • projectId
  • resourceSetId
  • moveToTrash (optional, default: false)

Specific Error Flags:

  • projectIdError
  • resourceSetIdError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"deleteResourceSet",
  "projectId":97,
  "resourceSetId":172,
  "moveToTrash":false
}

getFilesInfoOfResourceSet

Specific Request Fields:

  • projectId
  • resourceSetId

Specific Error Flags:

  • projectIdError
  • resourceSetIdError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getFilesInfoOfResourceSet",
  "projectId":23,
  "resourceSetId":143
}

JSON Response Example:

{
  "filesArray": [
    {
      "fileName": "DKFQSLibrary2.psm1",
      "fileSize": 16339,
      "fileHashCode": -1503445747,
      "fileLastModified": 1603566144851
    },
    {
      "fileName": "powershell-http-bern2.ps1",
      "fileSize": 12900,
      "fileHashCode": -1174212096,
      "fileLastModified": 1603566162094
    },
    {
      "fileName": "TestResult_powershell-http-bern2Neu_2020-10-24@21-06-04.json",
      "fileSize": 14395,
      "fileHashCode": -951574615,
      "fileLastModified": 1603566379097
    },
    {
      "fileName": "TestResult_powershell-http-bern2Neu_2020-10-24@21-09-45.json",
      "fileSize": 55128,
      "fileHashCode": 1499924815,
      "fileLastModified": 1603566591322
    }
  ],
  "isError": false
}

createFile

Specific Request Fields:

  • projectId
  • resourceSetId
  • fileName
  • fileContentB64 (the content of the file, in Base64 format)

Response Fields:

  • fileName
  • fileSize
  • fileHashCode
  • fileLastModified

Specific Error Flags:

  • projectIdError
  • resourceSetIdError
  • fileNameError
  • diskSpaceLimitExceededError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"createFile",
  "projectId":23,
  "resourceSetId":143,
  "fileName":"test.txt",
  "fileContentB64":"VGhpcyBpcyB0aGUgY29udGVudCBvZiB0aGUgZmlsZS4=" 
}

JSON Response Example (Success Case):

{
  "fileName":"test.txt",
  "fileSize":32,
  "fileHashCode":-1460278014,
  "fileLastModified":1625423562384,
  "isError":false
}

JSON Response Example (Error Case):

{"isError":true,"genericErrorText":"","projectIdError":false,"resourceSetIdError":false,"diskSpaceLimitExceededError":false,"fileNameError":true}

getFile

Specific Request Fields:

  • projectId
  • resourceSetId
  • fileName

Response Fields:

  • fileName
  • fileContentB64 (the content of the file, in Base64 format)
  • fileSize
  • fileHashCode
  • fileLastModified

Specific Error Flags:

  • projectIdError
  • resourceSetIdError
  • fileNameError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getFile",
  "projectId":23,
  "resourceSetId":143,
  "fileName":"test.txt"
}

JSON Response Example (Success Case):

{
  "fileName":"test.txt",
  "fileContentB64":"VGhpcyBpcyB0aGUgY29udGVudCBvZiB0aGUgZmlsZS4=",
  "fileSize":32,
  "fileHashCode":-1460278014,
  "fileLastModified":1625423562384,
  "isError":false
}

deleteFile

Specific Request Fields:

  • projectId
  • resourceSetId
  • fileName
  • moveToTrash (optional, default: false)

Response Fields:

  • fileDeleted (a flag which is true if the file was deleted)

Specific Error Flags:

  • projectIdError
  • resourceSetIdError
  • fileNameError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"deleteFile",
  "projectId":23,
  "resourceSetId":143,
  "fileName":"test.txt",
  "moveToTrash":false
}

JSON Response Example (Success Case):

{"fileDeleted":true,"isError":false}

JSON Response Example (Error Case):

{"isError":true,"genericErrorText":"","projectIdError":false,"resourceSetIdError":false,"fileNameError":true}

getMeasuringAgents

Specific Request Fields:

  • [none]

Response Fields:

  • agentId (the unique measuring agent id)
  • createdBySystem (normally false, true = the user cannot modify or delete the measuring agent)
  • ownerUserId (always the same as the user account id)
  • agentActive (flag: if false then the availability of the measuring agent is not monitored)
  • agentDescription
  • agentHost
  • agentPort
  • authToken (the authentication token to access the measuring agent, or an empty string = no access protection | don’t confuse it with the API authTokenValue)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getMeasuringAgents"
}

JSON Response Example (Success Case):

{
  "measuringAgentsArray": [
    {
      "agentId": 46,
      "createdBySystem": false,
      "ownerUserId": 13,
      "agentActive": true,
      "agentDescription": "Local Agent",
      "agentHost": "127.0.0.1",
      "agentPort": 8080,
      "authToken": "OrKmpkbyNWEHok"
    },
    {
      "agentId": 49,
      "createdBySystem": false,
      "ownerUserId": 13,
      "agentActive": false,
      "agentDescription": "Rasberry 1",
      "agentHost": "192.168.0.51",
      "agentPort": 8080,
      "authToken": ""
    },
    {
      "agentId": 50,
      "createdBySystem": false,
      "ownerUserId": 13,
      "agentActive": true,
      "agentDescription": "Ubuntu 10",
      "agentHost": "192.168.0.110",
      "agentPort": 8080,
      "authToken": ""
    },
    {
      "agentId": 51,
      "createdBySystem": false,
      "ownerUserId": 13,
      "agentActive": true,
      "agentDescription": "Ubuntu 11",
      "agentHost": "192.168.0.111",
      "agentPort": 8080,
      "authToken": ""
    }
  ],
  "isError": false
}

getMinRequiredMeasuringAgentVersion

Specific Request Fields:

  • [none]

Response Fields:

  • minRequiredMeasuringAgentVersion (the minimum required measuring agent version)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getMinRequiredMeasuringAgentVersion"
}

JSON Response Example (Success Case):

{
  "minRequiredMeasuringAgentVersion":"3.9.34",
  "isError":false
}

addMeasuringAgent

Specific Request Fields:

  • agentDescription (must be unique across all measuring agents of the user, cannot be an empty string)
  • agentHost
  • agentPort
  • agentActive (flag: if false then the availability of the measuring agent is not monitored)
  • agentAuthToken (the authentication token to access the measuring agent, or an empty string = no access protection)

Response Fields (JSON object “measuringAgent”):

  • agentId (the unique measuring agent id)
  • createdBySystem (always false for this function)
  • ownerUserId (always the same as the user account id)
  • agentActive (flag: if false then the availability of the measuring agent is not monitored)
  • agentDescription
  • agentHost
  • agentPort
  • authToken (the authentication token to access the measuring agent, or an empty string = no access protection)

Specific Error Flags:

  • agentDescriptionError
  • agentHostError
  • agentPortError
  • maxNumberMeasuringAgentsLimitExceededError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"addMeasuringAgent",
  "agentDescription":"Ubuntu 12",
  "agentHost":"192.168.0.112",
  "agentPort":8080,
  "agentActive": true,
  "agentAuthToken": "nixda"
}

JSON Response Example (Success Case):

{
  "measuringAgent": {
    "agentId": 53,
    "createdBySystem": false,
    "ownerUserId": 13,
    "agentActive": true,
    "agentDescription": "Ubuntu 12",
    "agentHost": "192.168.0.112",
    "agentPort": 8080,
    "authToken": "nixda"
  },
  "isError": false
}

pingMeasuringAgent

Specific Request Fields:

  • agentId

Response Fields (JSON object “agentResponse”):

  • pingFromRemoteIp
  • pingFromRemoteUserId
  • productVersion (measuring agent version | don’t confuse with portal server version)
  • limitMaxUsersPerJob (limit of the measuring agent, -1 = unlimited | don’t confuse with price plan limit)
  • limitMaxJobDurationSeconds (limit of the measuring agent, -1 = unlimited | don’t confuse with price plan limit)
  • osName
  • osVersion
  • javaVersion
  • javaVendor
  • javaMaxMemory
  • systemTime
  • deltaTimeMillis
  • agentStartupTimeStamp
  • httpExecuteTimeMillis

Specific Error Flags:

  • agentIdError
  • agentAccessDeniedError
  • agentVersionOutdatedError
  • agentNotReachableError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"pingMeasuringAgent",
  "agentId":48
}

JSON Response Example (Success Case):

{
  "agentResponse": {
    "pingFromRemoteIp": "83.150.39.44",
    "pingFromRemoteUserId": 13,
    "productVersion": "3.9.30",
    "limitMaxUsersPerJob": 500,
    "limitMaxJobDurationSeconds": 300,
    "osName": "Linux",
    "osVersion": "4.15.0-136-generic",
    "javaVersion": "11.0.1",
    "javaVendor": "Oracle Corporation",
    "javaMaxMemory":"2048 MB",
    "systemTime": 1625513238236,
    "deltaTimeMillis": 841,
    "agentStartupTimeStamp": 1622836702172,
    "httpExecuteTimeMillis": 247
  },
  "isError": false
}

JSON Response Example (Error Case 1):

{
  "isError": true,
  "genericErrorText": "API V1 request to 192.168.0.51:8080 timed out",
  "agentIdError": false,
  "agentAccessDeniedError": false,
  "agentNotReachableError": true,
  "agentVersionOutdatedError": false
}

JSON Response Example (Error Case 2):

{
  "isError": true,
  "genericErrorText": "Min. measuring agent version required: 3.9.30",
  "agentIdError": false,
  "agentAccessDeniedError": false,
  "agentNotReachableError": false,
  "agentVersionOutdatedError": true
}

setMeasuringAgentActive

Specific Request Fields:

  • agentId
  • agentActive

Response Fields:

  • [none]

Specific Error Flags:

  • agentIdError
  • agentAccessDeniedError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"setMeasuringAgentActive",
  "agentId":46,
  "agentActive":false
}

JSON Response Example (Success Case):

{"isError":false}

deleteMeasuringAgent

Specific Request Fields:

  • agentId

Response Fields:

  • [none]

Specific Error Flags:

  • agentIdError
  • agentAccessDeniedError
  • agentDeleteDeniedError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"deleteMeasuringAgent",
  "agentId":54
}

JSON Response Example (Success Case):

{"isError":false}

getMeasuringAgentClusters

Specific Request Fields:

  • [none]

Response Fields (JSON array “measuringAgentClustersArray”):

  • clusterId (the unique cluster id)
  • createdBySystem (normally false, true = the user cannot modify or delete the cluster)
  • ownerUserId (always the same as the user account id)
  • clusterActive (flag: if false then the availability of the cluster is not monitored)
  • clusterDescription
  • controllerHost (the hostname or IP address of the cluster controller)
  • controllerPort (the IP port of the cluster controller)
  • controllerAuthToken (the authentication token to access the cluster controller, or an empty string = no access protection | don’t confuse it with the API authTokenValue)
  • clusterMembersArray
    • clusterMemberId (the unique cluster member id)
    • loadFactor (integer 0..100: the default load factor of this cluster member)
    • agentId (the referenced measuring agent id)
    • agentActive (flag: if false then the availability of the measuring agent is not monitored)
    • agentDescription
    • agentHost
    • agentPort
    • agentAuthToken (the authentication token to access the measuring agent, or an empty string = no access protection | don’t confuse it with the API authTokenValue)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getMeasuringAgentClusters"
}

JSON Response Example:

{
  "measuringAgentClustersArray": [
    {
      "clusterId": 11,
      "createdBySystem": false,
      "ownerUserId": 13,
      "clusterActive": true,
      "clusterDescription": "C1",
      "controllerHost": "192.168.0.50",
      "controllerPort": 8083,
      "controllerAuthToken": "aberaber",
      "clusterMembersArray": [
        {
          "clusterMemberId": 59,
          "loadFactor": 1,
          "agentId": 64,
          "agentActive": true,
          "agentDescription": "Agent 1",
          "agentHost": "192.168.0.10",
          "agentPort": 8080,
          "agentAuthToken": "OrKmAAbyNWEHok"
        },
        {
          "clusterMemberId": 60,
          "loadFactor": 1,
          "agentId": 59,
          "agentActive": true,
          "agentDescription": "Ubuntu 10",
          "agentHost": "192.168.0.110",
          "agentPort": 8080,
          "agentAuthToken": "asc7jhacab"
        },
        {
          "clusterMemberId": 61,
          "loadFactor": 1,
          "agentId": 60,
          "agentActive": true,
          "agentDescription": "Ubuntu 11",
          "agentHost": "192.168.0.111",
          "agentPort": 8080,
          "agentAuthToken": "66ascascsdac"
        }
      ]
    },
    {
      "clusterId": 14,
      "createdBySystem": false,
      "ownerUserId": 13,
      "clusterActive": true,
      "clusterDescription": "C2",
      "controllerHost": "192.168.0.50",
      "controllerPort": 8083,
      "controllerAuthToken": "aberaber",
      "clusterMembersArray": [
        {
          "clusterMemberId": 66,
          "loadFactor": 1,
          "agentId": 56,
          "agentActive": true,
          "agentDescription": "Test System",
          "agentHost": "192.168.0.60",
          "agentPort": 8080,
          "agentAuthToken": "aberdoch"
        },
        {
          "clusterMemberId": 67,
          "loadFactor": 1,
          "agentId": 59,
          "agentActive": true,
          "agentDescription": "Ubuntu 10",
          "agentHost": "192.168.0.110",
          "agentPort": 8080,
          "agentAuthToken": "asc7jhacab"
        }
      ]
    }
  ],
  "isError": false
}

getClusterControllers

Specific Request Fields:

  • [none]

Response Fields (JSON array “clusterControllersArray”):

  • controllerHost (the hostname or IP address of the cluster controller)
  • controllerPort (the IP port of the cluster controller)
  • controllerAuthToken (the authentication token to access the cluster controller)
  • measuringAgentClustersArray (an array of measuring agent clusters which are referencing this cluster controller)
    • clusterId
    • clusterDescription
    • clusterActive

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getClusterControllers"
}

JSON Response Example:

{
  "clusterControllersArray": [
    {
      "controllerHost": "192.168.0.33",
      "controllerPort": 8083,
      "controllerAuthToken": "2fasdtfffe",
      "measuringAgentClustersArray": [
        {
          "clusterId": 11,
          "clusterDescription": "C1",
          "clusterActive": 1
        },
        {
          "clusterId": 13,
          "clusterDescription": "C2",
          "clusterActive": 1
        },
        {
          "clusterId": 14,
          "clusterDescription": "C3",
          "clusterActive": 1
        }
      ]
    },
    {
      "controllerHost": "192.168.0.50",
      "controllerPort": 8083,
      "controllerAuthToken": "asfsdgh763",
      "measuringAgentClustersArray": [
        {
          "clusterId": 15,
          "clusterDescription": "C4",
          "clusterActive": 1
        },
        {
          "clusterId": 16,
          "clusterDescription": "C7",
          "clusterActive": 1
        }
      ]
    }
  ],
  "isError": false
}

getMinRequiredClusterControllerVersion

Specific Request Fields:

  • [none]

Response Fields:

  • minRequiredClusterControllerVersion (the minimum required cluster controller version)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getMinRequiredClusterControllerVersion"
}

JSON Response Example:

{
  "minRequiredClusterControllerVersion":"4.0.4",
  "isError":false
}

pingClusterController

Specific Request Fields:

  • controllerHost (the cluster controller host name or IP address)
  • controllerPort (the cluster controller IP port)
  • controllerAuthToken (the authentication token to access the cluster controller, or an empty string = no access protection)

Response Fields (JSON object “controllerResponse”):

  • pingFromRemoteIp
  • pingFromRemoteUserId
  • productVersion (cluster controller version | don’t confuse with portal server version)
  • osName
  • osVersion
  • javaVersion
  • javaVendor
  • javaMaxMemory
  • systemTime
  • deltaTimeMillis
  • controllerStartupTimeStamp
  • httpExecuteTimeMillis
  • clusterControllerOutdated

Specific Error Flags:

  • controllerHostError
  • controllerPortError
  • controllerVersionOutdatedError
  • controllerNotReachableError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"pingClusterController",
    "controllerHost":"192.168.0.50",
    "controllerPort":8083,
    "controllerAuthToken":"hagsajjs99"
}

JSON Response Example (Success Case):

{
  "controllerResponse": {
    "pingFromRemoteIp": "192.168.0.100",
    "pingFromRemoteUserId": 13,
    "productVersion": "4.0.4",
    "osName": "Linux",
    "osVersion": "4.15.0-135-generic",
    "javaVersion": "11.0.1",
    "javaVendor": "Oracle Corporation",
    "javaMaxMemory": "512 MB",
    "systemTime": 1643406118552,
    "deltaTimeMillis": 1120,
    "controllerStartupTimeStamp": 1643322597013,
    "httpExecuteTimeMillis": 249,
    "clusterControllerOutdated": false
  },
  "isError": false
}

JSON Response Example (Error Case 1):

{
  "isError": true,
  "genericErrorText": "API call pingGetControllerInfo failed. Error code = 18, Error message = Invalid authentication token",
  "controllerHostError": false,
  "controllerVersionOutdatedError": false,
  "controllerNotReachableError": true,
  "controllerPortError": false
}

JSON Response Example (Error Case 2):

{
  "isError": true,
  "genericErrorText": "Min. cluster controller version required: 4.0.4",
  "controllerHostError": false,
  "controllerVersionOutdatedError": true,
  "controllerNotReachableError": false,
  "controllerPortError": false
}

addMeasuringAgentCluster

Specific Request Fields:

  • clusterActive (flag: if false then the availability of the cluster is not monitored)
  • clusterDescription (must be unique across all measuring agent clusters and all measuring agents of the user, cannot be an empty string)
  • controllerHost (the cluster controller host name or IP address)
  • controllerPort (the cluster controller IP port)
  • controllerAuthToken (the authentication token to access the cluster controller, or an empty string = no access protection)
  • clusterMembersArray (an array of cluster members - can also be empty)
    • agentId (the referenced measuring agent id)
    • loadFactor (integer 0..100: the load factor of this cluster member, recommended value = 1)

Response Fields:

  • clusterId (the unique cluster id)
  • clusterMembersArray (the array of cluster members)
    • clusterMemberId (the unique cluster member id)
    • agentId (the referenced measuring agent id)
    • loadFactor (integer 0..100: the load factor of this cluster member)

Specific Error Flags:

  • clusterDescriptionError
  • controllerHostError
  • controllerPortError
  • agentIdError
  • loadFactorError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"addMeasuringAgentCluster",
  "clusterActive":true,
  "clusterDescription":"C7",
  "controllerHost":"192.168.0.50",
  "controllerPort":8083,
  "controllerAuthToken":"aberaber",
  "clusterMembersArray":[
    {
      "agentId":59,
      "loadFactor":1
    },
    {
      "agentId":60,
      "loadFactor":1
    }
  ]
}

JSON Response Example (Success Case):

{
  "clusterId":16,
  "clusterMembersArray":[
    {
      "clusterMemberId":71,
      "agentId":59,
      "loadFactor":1
    },
    {
      "clusterMemberId":72,
      "agentId":60,
      "loadFactor":1
    }
  ],
  "isError":false
}

JSON Response Example (Error Case):

{
  "isError": true,
  "genericErrorText": "Invalid agentId = 101",
  "controllerHostError": false,
  "agentIdError": true,
  "controllerPortError": false,
  "loadFactorError": false,
  "clusterDescriptionError": false
}

addMemberToMeasuringAgentCluster

Specific Request Fields:

  • clusterId
  • agentId (the referenced measuring agent id)
  • loadFactor (integer 0..100: the load factor of this cluster member, recommended value = 1)

Response Fields (JSON object “clusterMember”):

  • clusterMemberId (the unique cluster member id)
  • agentId
  • loadFactor

Specific Error Flags:

  • clusterIdError
  • clusterAccessDeniedError
  • clusterModifyDeniedError
  • agentIdError
  • agentAccessDeniedError
  • loadFactorError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"addMemberToMeasuringAgentCluster",
  "clusterId":17,
  "agentId":64,
  "loadFactor":1
}

JSON Response Example:

{
  "clusterMember": {
    "clusterMemberId": 75,
    "agentId": 64,
    "loadFactor": 1
  },
  "isError": false
}

removeMemberFromMeasuringAgentCluster

Specific Request Fields:

  • clusterId
  • clusterMemberId

Response Fields:

  • [none]

Specific Error Flags:

  • clusterIdError
  • clusterAccessDeniedError
  • clusterModifyDeniedError
  • clusterMemberIdError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"removeMemberFromMeasuringAgentCluster",
  "clusterId":17,
  "clusterMemberId":75
}

JSON Response Example:

{"isError":false}

pingMeasuringAgentClusterMembers

Specific Request Fields:

  • clusterId

Response Fields (JSON object “controllerResponse”):

  • productVersion (cluster controller version)
  • clusterConnectResult (the connect information to the cluster members)
    • measuringAgentClusterMemberArray (the array of cluster members)
      • clusterMemberId
      • loadFactor
      • agentId
      • agentActive
      • agentDescription
      • agentHost
      • agentPort
      • agentAuthToken
    • connectSuccessfulClusterMemberArray (the array of cluster member ids to which the connection was successful established)
    • connectFailedClusterMemberArray (the array of cluster members to which the connection has failed)
      • clusterMemberId
      • errorMessage
    • clusterConnectStartTimestamp
    • clusterConnectDurationMillis
  • clusterActionResult (the ping result of the cluster members)
    • actionSuccessfulClusterMemberArray (the array of cluster member ids which have performed the ping to the measuring agent)
    • actionFailedClusterMemberArray (the array of cluster members which have not performed the ping to the measuring agent)
      • clusterMemberId
      • errorMessage
    • jsonResponseClusterMemberArray (the array of cluster member which have performed the ping)
      • clusterMemberId
      • jsonResponseObject (the pong response of the cluster member)
        • productVersion (measuring agent product version)
        • systemTime
        • deltaTimeMillis (the OS time difference in milliseconds between the cluster controller and the measuring agent)
        • osName
        • osVersion
        • javaVersion
        • javaVendor
        • javaMaxMemory
        • samplingGranularityMillis (the data collector sampling granularity in milliseconds)
        • isError (boolean flag, normally always false)
        • measuringAgentOutdated (a boolean flag, true = measuring agent product version is outdated)
    • clusterActionStartTimestamp
    • clusterActionDurationMillis
  • httpExecuteTimeMillis
  • clusterControllerOutdated

Specific Error Flags:

  • clusterIdError
  • clusterAccessDeniedError
  • controllerVersionOutdatedError
  • controllerNotReachableError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"pingMeasuringAgentClusterMembers",
  "clusterId":16
}

JSON Response Example (Success Case):

{
  "controllerResponse": {
    "productVersion": "4.0.4",
    "clusterConnectResult": {
      "measuringAgentClusterMemberArray": [
        {
          "clusterMemberId": 71,
          "loadFactor": 1,
          "agentId": 59,
          "agentActive": true,
          "agentDescription": "Ubuntu 10",
          "agentHost": "192.168.0.110",
          "agentPort": 8080,
          "agentAuthToken": "agsdhagsj"
        },
        {
          "clusterMemberId": 72,
          "loadFactor": 1,
          "agentId": 60,
          "agentActive": true,
          "agentDescription": "Ubuntu 11",
          "agentHost": "192.168.0.111",
          "agentPort": 8080,
          "agentAuthToken": "nvbjnvbnn"
        }
      ],
      "connectSuccessfulClusterMemberArray": [
        71,
        72
      ],
      "connectFailedClusterMemberArray": [],
      "clusterConnectStartTimestamp": 1643410829270,
      "clusterConnectDurationMillis": 79
    },
    "clusterActionResult": {
      "actionSuccessfulClusterMemberArray": [
        71,
        72
      ],
      "actionFailedClusterMemberArray": [],
      "jsonResponseClusterMemberArray": [
        {
          "clusterMemberId": 71,
          "jsonResponseObject": {
            "productVersion": "4.0.4",
            "systemTime": 1643410829340,
            "deltaTimeMillis": -10,
            "osName": "Linux",
            "osVersion": "5.4.0-92-generic",
            "javaVersion": "11.0.1",
            "javaVendor": "Oracle Corporation",
            "javaMaxMemory": "2048 MB",
            "samplingGranularityMillis": 4000,
            "isError": false,
            "measuringAgentOutdated": false
          }
        },
        {
          "clusterMemberId": 72,
          "jsonResponseObject": {
            "productVersion": "4.0.4",
            "systemTime": 1643410829351,
            "deltaTimeMillis": -10,
            "osName": "Linux",
            "osVersion": "5.4.0-92-generic",
            "javaVersion": "11.0.1",
            "javaVendor": "Oracle Corporation",
            "javaMaxMemory": "2048 MB",
            "samplingGranularityMillis": 4000,
            "isError": false,
            "measuringAgentOutdated": false
          }
        }
      ],
      "clusterActionStartTimestamp": 1643410829349,
      "clusterActionDurationMillis": 43
    },
    "httpExecuteTimeMillis": 1778,
    "clusterControllerOutdated": false
  },
  "isError": false
}

JSON Response Example (Error Case / Partly failed):

{
  "controllerResponse": {
    "productVersion": "4.0.4",
    "clusterConnectResult": {
      "measuringAgentClusterMemberArray": [
        {
          "clusterMemberId": 71,
          "loadFactor": 1,
          "agentId": 59,
          "agentActive": true,
          "agentDescription": "Ubuntu 10",
          "agentHost": "192.168.0.110",
          "agentPort": 8080,
          "agentAuthToken": "marderzahn"
        },
        {
          "clusterMemberId": 72,
          "loadFactor": 1,
          "agentId": 60,
          "agentActive": true,
          "agentDescription": "Ubuntu 11",
          "agentHost": "192.168.0.111",
          "agentPort": 8080,
          "agentAuthToken": "marderzahn"
        }
      ],
      "connectSuccessfulClusterMemberArray": [
        72
      ],
      "connectFailedClusterMemberArray": [
        {
          "clusterMemberId": 71,
          "errorMessage": "Connection refused (Connection refused)"
        }
      ],
      "clusterConnectStartTimestamp": 1643414272214,
      "clusterConnectDurationMillis": 97
    },
    "clusterActionResult": {
      "actionSuccessfulClusterMemberArray": [
        72
      ],
      "actionFailedClusterMemberArray": [],
      "jsonResponseClusterMemberArray": [
        {
          "clusterMemberId": 72,
          "jsonResponseObject": {
            "productVersion": "4.0.4",
            "systemTime": 1643414272310,
            "deltaTimeMillis": -8,
            "osName": "Linux",
            "osVersion": "5.4.0-92-generic",
            "javaVersion": "11.0.1",
            "javaVendor": "Oracle Corporation",
            "javaMaxMemory": "2048 MB",
            "samplingGranularityMillis": 4000,
            "isError": false,
            "measuringAgentOutdated": false
          }
        }
      ],
      "clusterActionStartTimestamp": 1643414272311,
      "clusterActionDurationMillis": 21
    },
    "httpExecuteTimeMillis": 1769,
    "clusterControllerOutdated": false
  }
}

setMeasuringAgentClusterActive

Specific Request Fields:

  • clusterId
  • clusterActive

Response Fields:

  • [none]

Specific Error Flags:

  • clusterIdError
  • clusterAccessDeniedError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"setMeasuringAgentClusterActive",
  "clusterId":16,
  "clusterActive":true
}

JSON Response Example (Success Case):

{"isError":false}

deleteMeasuringAgentCluster

Specific Request Fields:

  • clusterId

Response Fields:

  • [none]

Specific Error Flags:

  • clusterIdError
  • clusterAccessDeniedError
  • clusterDeleteDeniedError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"deleteMeasuringAgentCluster",
  "clusterId":16
}

JSON Response Example (Success Case):

{"isError":false}

getProxyRecorders

Specific Request Fields:

  • [none]

Response Fields (JSON array “proxyRecordersArray”):

  • recorderId (the unique proxy recorder id)
  • createdBySystem (normally false, true = the user cannot modify or delete the proxy recorder)
  • ownerUserId (always the same as the user account id)
  • recorderDescription
  • recorderProxyHost
  • recorderProxyPort (HTTP and HTTPS port of the proxy)
  • recorderProxyAuthUsername (proxy authentication username, or an empty string = no proxy authentication required)
  • recorderProxyAuthPassword (proxy authentication password)
  • recorderControlPort (the proxy recorder control port)
  • recorderControlAuthToken (the authentication token to access the proxy recorder control port, or an empty string = no access protection | don’t confuse it with the API authTokenValue)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getProxyRecorders"
}

JSON Response Example:

{
  "proxyRecordersArray": [
    {
      "recorderId": 3,
      "createdBySystem": false,
      "ownerUserId": 13,
      "recorderDescription": "Erster",
      "recorderProxyHost": "192.168.0.40",
      "recorderProxyPort": 8082,
      "recorderProxyAuthUsername": "",
      "recorderProxyAuthPassword": "",
      "recorderControlPort": 8081,
      "recorderControlAuthToken": ""
    },
    {
      "recorderId": 4,
      "createdBySystem": false,
      "ownerUserId": 13,
      "recorderDescription": "proxy.realload.com",
      "recorderProxyHost": "proxy.realload.com",
      "recorderProxyPort": 8082,
      "recorderProxyAuthUsername": "max.meier",
      "recorderProxyAuthPassword": "123456",
      "recorderControlPort": 8081,
      "recorderControlAuthToken": "aZujkl97zuwert"
    }
  ],
  "isError": false
}

getMinRequiredProxyRecorderVersion

Specific Request Fields:

  • [none]

Response Fields:

  • minRequiredProxyRecorderVersion (the minimum required HTTP/S proxy recorder version)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getMinRequiredProxyRecorderVersion"
}

JSON Response Example (Success Case):

{
  "minRequiredProxyRecorderVersion":"0.2.2",
  "isError":false
}

addProxyRecorder

Specific Request Fields:

  • recorderDescription (must be unique across all HTTP/S proxy recorders of the user, cannot be an empty string)
  • recorderProxyHost
  • recorderProxyPort (HTTP and HTTPS port of the proxy)
  • recorderProxyAuthUsername (proxy authentication username, or an empty string = no proxy authentication required)
  • recorderProxyAuthPassword (proxy authentication password, applied if recorderProxyAuthUsername is not an empty string)
  • recorderControlPort (the proxy recorder control port)
  • recorderControlAuthToken (the authentication token to access the proxy recorder control port, or an empty string = no access protection)

Response Fields (JSON object “proxyRecorder”):

  • recorderId (the unique HTTP/S proxy recorder id)
  • createdBySystem (always false for this function)
  • ownerUserId (always the same as the user account id)
  • recorderDescription
  • recorderProxyHost
  • recorderProxyPort
  • recorderProxyAuthUsername
  • recorderProxyAuthPassword
  • recorderControlPort
  • recorderControlAuthToken

Specific Error Flags:

  • recorderDescriptionError
  • recorderProxyHostError
  • recorderProxyPortError
  • recorderControlPortError
  • maxNumberProxyRecordersLimitExceededError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"addProxyRecorder",
  "recorderDescription":"My New Proxy Recorder",
  "recorderProxyHost":"192.168.0.148",
  "recorderProxyPort":8082,
  "recorderProxyAuthUsername":"max.meier",
  "recorderProxyAuthPassword":"123456",
  "recorderControlPort":8081,
  "recorderControlAuthToken":"aZujkl97zuwert"
}

JSON Response Example (Success Case):

{
  "proxyRecorder": {
    "recorderId": 10,
    "createdBySystem": false,
    "ownerUserId": 13,
    "recorderDescription": "My New Proxy Recorder",
    "recorderProxyHost": "192.168.0.148",
    "recorderProxyPort": 8082,
    "recorderProxyAuthUsername": "max.meier",
    "recorderProxyAuthPassword": "123456",
    "recorderControlPort": 8081,
    "recorderControlAuthToken": "aZujkl97zuwert"
  },
  "isError": false
}

pingProxyRecorder

Specific Request Fields:

  • recorderId

Response Fields (JSON object “pongResponse”):

  • pingFromRemoteIp
  • pingFromRemoteUserId
  • productVersion (the remote proxy recorder version | don’t confuse with portal server version)
  • recorderComponentVersion (the proxy recorder component version | don’t confuse with portal server version)
  • isRecording
  • recordHostFilter
  • numRecordedElements
  • osName
  • osVersion
  • javaMemoryMB
  • javaVersion
  • javaVendor
  • systemTime
  • deltaTimeMillis
  • httpExecuteTimeMillis

Specific Error Flags:

  • recorderIdError
  • recorderAccessDeniedError
  • recorderNotReachableError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"pingProxyRecorder",
  "recorderId":4
}

JSON Response Example (Success Case):

{
  "pongResponse": {
    "pingFromRemoteIp": "83.150.39.44",
    "pingFromRemoteUserId": 13,
    "productVersion": "0.2.0",
    "recorderComponentVersion": "1.1.0",
    "isRecording": false,
    "recordHostFilter": "www.dkfqa.com",
    "numRecordedElements": 0,
    "osName": "Linux",
    "osVersion": "5.4.0-74-generic",
    "javaMemoryMB": 2048,
    "javaVersion": "11.0.1",
    "javaVendor": "Oracle Corporation",
    "systemTime": 1625529858405,
    "deltaTimeMillis": 790,
    "httpExecuteTimeMillis": 88
  },
  "isError": false
}

JSON Response Example (Error Case):

{
  "isError": true,
  "genericErrorText": "connect timed out",
  "recorderNotReachableError": true,
  "recorderIdError": false,
  "recorderAccessDeniedError": false
}

deleteProxyRecorder

Specific Request Fields:

  • recorderId

Response Fields:

  • [none]

Specific Error Flags:

  • recorderIdError
  • recorderAccessDeniedError
  • recorderDeleteDeniedError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"deleteProxyRecorder",
  "recorderId":10
}

JSON Response Example (Success Case):

{"isError":false}

getProxyRecordersClientCertificates

Specific Request Fields:

  • [none]

Response Fields:

  • certificateId (unique ID)
  • ownerUserId (always the own user ID)
  • recorderId (always -1 = apply for all proxy recorders)
  • originalCertFileName (without disk path)
  • certFileType (always PKCS#12)
  • certFileDataB64 (the file data in base64 format)
  • certPassword
  • certDescription
  • certActive (if false then do not transmit the certificate to the proxy recorders)
  • applyForHost
  • applyForPort

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getProxyRecordersClientCertificates"
}

JSON Response Example (Success Case):

{
  "clientCertificatesArray": [
    {
      "certificateId": 1,
      "ownerUserId": 13,
      "recorderId": -1,
      "originalCertFileName": "fischer@dkfqa.com.p12",
      "certFileType": "PKCS#12",
      "certFileDataB64": "MIIRQQ.....CAA=",
      "certPassword": "12345678",
      "certDescription": "Fischer's Cert",
      "certActive": true,
      "applyForHost": "192.168.0.100",
      "applyForPort": 499
    },
    {
      "certificateId": 3,
      "ownerUserId": 13,
      "recorderId": -1,
      "originalCertFileName": "miller@dkfqa.com.p12",
      "certFileType": "PKCS#12",
      "certFileDataB64": "MIIRXB.....CCAA=",
      "certPassword": "abcdefgh",
      "certDescription": "",
      "certActive": false,
      "applyForHost": "www.werwaswo.com",
      "applyForPort": 443
    }
  ],
  "isError": false
}

addProxyRecordersClientCertificate

Specific Request Fields:

  • recorderId (always -1 = apply for all proxy recorders)
  • certFileName (without disk path)
  • certFileType (always PKCS#12)
  • certFileDataB64 (the file data in base64 format)
  • certPassword
  • certDescription (optional, but not null)
  • certActive (if false then do not transmit the certificate to the proxy recorders)
  • applyForHost
  • applyForPort

Response Fields:

  • certificateId (the new unique ID)
  • ownerUserId (always the own user ID)
  • recorderId (always -1 = apply for all proxy recorders)
  • originalCertFileName (without disk path)
  • certFileType (always PKCS#12)
  • certDescription
  • certActive
  • applyForHost
  • applyForPort

Specific Error Flags:

  • certDescriptionError
  • applyForHostError
  • applyForPortError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"addProxyRecordersClientCertificate",
  "recorderId": -1,
  "certFileName": "felix@dkfqa.com.p12",
  "certFileType": "PKCS#12",
  "certFileDataB64": "MIIRQQIBAzCCEQ.....3dUWQICCAA=",
  "certPassword": "mammamia",
  "certDescription":"",
  "certActive": true,
  "applyForHost": "www.dkfqa.com",
  "applyForPort": 443
}

JSON Response Example (Success Case):

{
  "addedClientCertificate": {
    "certificateId": 5,
    "ownerUserId": 13,
    "recorderId": -1,
    "originalCertFileName": "felix@dkfqa.com.p12",
    "certFileType": "PKCS#12",
    "certDescription": "",
    "certActive": true,
    "applyForHost": "www.dkfqa.com",
    "applyForPort": 443
  },
  "isError": false
}

setProxyRecordersClientCertificateActive

Specific Request Fields:

  • certificateId
  • certActive (if false then do not transmit the certificate to the proxy recorders)

Response Fields:

  • certificateId
  • ownerUserId (always the own user ID)
  • recorderId (always -1 = apply for all proxy recorders)
  • originalCertFileName (without disk path)
  • certFileType (always PKCS#12)
  • certDescription
  • certActive
  • applyForHost
  • applyForPort

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"setProxyRecordersClientCertificateActive",
  "certificateId": 5,
  "certActive": false
}

JSON Response Example (Success Case):

{
  "updatedClientCertificate": {
    "certificateId": 5,
    "ownerUserId": 13,
    "recorderId": -1,
    "originalCertFileName": "felix@dkfqa.com.p12",
    "certFileType": "PKCS#12",
    "certDescription": "",
    "certActive": false,
    "applyForHost": "www.dkfqa.com",
    "applyForPort": 443
  },
  "isError": false
}

modifyProxyRecordersClientCertificate

Specific Request Fields:

  • certificateId
  • certDescription (optional but not null)
  • applyForHost (required)
  • applyForPort (required)

Response Fields:

  • certificateId
  • ownerUserId (always the own user ID)
  • recorderId (always -1 = apply for all proxy recorders)
  • originalCertFileName (without disk path)
  • certFileType (always PKCS#12)
  • certDescription
  • certActive
  • applyForHost
  • applyForPort

Specific Error Flags:

  • certDescriptionError
  • applyForHostError
  • applyForPortError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"modifyProxyRecordersClientCertificate",
  "certificateId": 5,
  "certDescription": "My Cert",
  "applyForHost": "www.dkfqa.com",
  "applyForPort": 444
}

JSON Response Example (Success Case):

{
  "updatedClientCertificate": {
    "certificateId": 5,
    "ownerUserId": 13,
    "recorderId": -1,
    "originalCertFileName": "felix@dkfqa.com.p12",
    "certFileType": "PKCS#12",
    "certDescription": "My Cert",
    "certActive": false,
    "applyForHost": "www.dkfqa.com",
    "applyForPort": 444
  },
  "isError": false
}

deleteProxyRecordersClientCertificate

Specific Request Fields:

  • certificateId

Response Fields:

  • [none]

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"deleteProxyRecordersClientCertificate",
  "certificateId": 5
}

JSON Response Example (Success Case):

{"isError": false}

JSON Response Example (Error Case):

{"isError": true, "genericErrorText": "Invalid certificate Id"}

getTestjobTemplates

Specific Request Fields:

  • [none]

Response Fields:

  • testjobTemplatesArray

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getTestjobTemplates"
}

JSON Response Example (Success Case):

{
  "testjobTemplatesArray": [
    {
      "testjobTemplate": {
        "templateId": 10,
        "templateName": "SimpleTest 1VU",
        "templateDescription": "",
        "testjobProperties": {
          "testjobResultFileName": "",
          "executeMeasuringAgentClusterInfo": "",
          "testjobConcurrentUsers": "1",
          "testjobDescription": "SimpleTest",
          "testjobDebugMeasuring": "false",
          "testjobMaxTestDuration": "30",
          "isClusterJob": "",
          "testjobRampUpTime": "10",
          "localTestjobId": "",
          "testjobMaxLoopsPerUser": "1",
          "productVersion": "4.8.23",
          "remoteTestjobId": "",
          "testjobAdditionalArguments": "",
          "definedFromTestId": "384",
          "testjobSignature": "",
          "testjobDelayPerLoop": "1000",
          "testjobExitCode": "",
          "createTimeStamp": "1694549684261",
          "userInputFieldsFileName": "",
          "testjobType": "load test",
          "enableAutomaticFileSync": "true",
          "testjobState": "invalid",
          "isClusterControllerJobProperties": "",
          "testjobDebugExecution": "false",
          "executeOnUserMeasuringAgentId": "",
          "definedFromTimeZoneId": "Europe/Berlin",
          "executeMeasuringAgentInfo": "",
          "executeOnMeasuringAgentClusterId": ""
        },
        "hasUserInputFields": false
      },
      "testIsHealthy": true
    },
    {
      "testjobTemplate": {
        "templateId": 11,
        "templateName": "SimpleTestWithInputFields 10VU",
        "templateDescription": "",
        "testjobProperties": {
          "testjobResultFileName": "",
          "executeMeasuringAgentClusterInfo": "",
          "testjobConcurrentUsers": "10",
          "testjobDescription": "SimpleTestWithInputFields",
          "testjobDebugMeasuring": "false",
          "testjobMaxTestDuration": "60",
          "isClusterJob": "",
          "testjobRampUpTime": "10",
          "localTestjobId": "",
          "testjobMaxLoopsPerUser": "5",
          "productVersion": "4.8.23",
          "remoteTestjobId": "",
          "testjobAdditionalArguments": "",
          "definedFromTestId": "385",
          "testjobSignature": "",
          "testjobDelayPerLoop": "1000",
          "testjobExitCode": "",
          "createTimeStamp": "1695047319785",
          "userInputFieldsFileName": "InputFields_SimpleTestWithInputFields.json",
          "testjobType": "load test",
          "enableAutomaticFileSync": "true",
          "testjobState": "invalid",
          "isClusterControllerJobProperties": "",
          "testjobDebugExecution": "true",
          "executeOnUserMeasuringAgentId": "",
          "definedFromTimeZoneId": "Europe/Berlin",
          "executeMeasuringAgentInfo": "",
          "executeOnMeasuringAgentClusterId": ""
        },
        "hasUserInputFields": true,
        "userInputFieldsFile": {
          "useShortNames": false,
          "magicPattern": "UserInputFields",
          "productVersion": "4.8.23",
          "userInputFieldsArray": [
            {
              "useShortNames": false,
              "productVersion": "4.8.23",
              "label": "Input Host",
              "variableName": "vHost",
              "inputType": "string",
              "defaultValue": "126.3.4.5",
              "currentValue": "126.3.4.5"
            },
            {
              "useShortNames": false,
              "productVersion": "4.8.23",
              "label": "Input Port",
              "variableName": "vPort",
              "inputType": "integer",
              "defaultValue": "444",
              "currentValue": "444"
            }
          ]
        }
      },
      "testIsHealthy": true
    }
  ],
  "isError": false
}

defineNewTestjobFromTemplate

Specific Request Fields:

  • templateId
  • measuringAgentOrClusterId
  • isCluster (boolean)
  • jobDescription

Response Fields:

  • newTestjobId

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"defineNewTestjobFromTemplate",
  "templateId":10,
  "measuringAgentOrClusterId":65,
  "isCluster":false,
  "jobDescription": "Simple Test"
}

JSON Response Example (Success Case):

{ 
  "newTestjobId":33533,
  "isError":false
}

submitTestjob

Specific Request Fields:

  • localTestjobId

Response Fields:

  • agentResponse (for measuring agents), or
  • controllerResponse (for cluster controllers)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"submitTestjob",
  "localTestjobId":33534
}

JSON Response Example (Success Case):

{
  "isClusterJob": false,
  "agentResponse": {
    "remoteTestjobId": 24129,
    "httpExecuteTimeMillis": 338
  },
  "isError": false
}

makeTestjobReadyToRun

Specific Request Fields:

  • localTestjobId

Response Fields:

  • agentResponse (for measuring agents), or
  • controllerResponse (for cluster controllers)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"makeTestjobReadyToRun",
  "localTestjobId":33534
}

JSON Response Example (Success Case):

{
  "isClusterJob": false,
  "agentResponse": {
    "testjobProperties": {
      "testjobResultFileName": "",
      "executeMeasuringAgentClusterInfo": "",
      "testjobConcurrentUsers": "1",
      "testjobDescription": "Simple Test",
      "testjobDebugMeasuring": "false",
      "testjobMaxTestDuration": "30",
      "isClusterJob": "false",
      "testjobRampUpTime": "10",
      "localTestjobId": "33534",
      "testjobMaxLoopsPerUser": "1",
      "productVersion": "4.8.18",
      "remoteTestjobId": "24129",
      "testjobAdditionalArguments": "",
      "definedFromTestId": "384",
      "testjobSignature": "Z9B963yIj232nht1DIX5aSRt8m77QuXpvVA5736Tl9Wc7UZ7jOiU3l6UC7y3Nx6CXDDZB4qUg4FHDtTuHsD9xd0T/CSq/g==",
      "testjobDelayPerLoop": "1000",
      "testjobExitCode": "",
      "createTimeStamp": "1695050460395",
      "userInputFieldsFileName": "",
      "testjobType": "load test",
      "enableAutomaticFileSync": "true",
      "testjobState": "ready to run",
      "isClusterControllerJobProperties": "",
      "testjobDebugExecution": "false",
      "executeOnUserMeasuringAgentId": "65",
      "definedFromTimeZoneId": "Europe/Berlin",
      "executeMeasuringAgentInfo": "agent2.realload.com:8080",
      "executeOnMeasuringAgentClusterId": ""
    },
    "httpExecuteTimeMillis": 2871
  },
  "isError": false
}

startTestjob

Specific Request Fields:

  • localTestjobId

Response Fields:

  • agentResponse (for measuring agents), or
  • controllerResponse (for cluster controllers)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"startTestjob",
  "localTestjobId":33534
}

JSON Response Example (Success Case):

{
  "isClusterJob": false,
  "agentResponse": {
    "testjobProperties": {
      "testjobResultFileName": "",
      "executeMeasuringAgentClusterInfo": "",
      "testjobConcurrentUsers": "1",
      "testjobDescription": "Simple Test",
      "testjobDebugMeasuring": "false",
      "testjobMaxTestDuration": "30",
      "isClusterJob": "false",
      "testjobRampUpTime": "10",
      "localTestjobId": "33534",
      "testjobMaxLoopsPerUser": "1",
      "productVersion": "4.8.18",
      "remoteTestjobId": "24129",
      "testjobAdditionalArguments": "",
      "definedFromTestId": "384",
      "testjobSignature": "Z9B963yIj232nht1DIX5aSRt8m77QuXpvVA5736Tl9Wc7UZ7jOiU3l6UC7y3Nx6CXDDZB4qUg4FHDtTuHsD9xd0T/CSq/g==",
      "testjobDelayPerLoop": "1000",
      "testjobExitCode": "",
      "createTimeStamp": "1695050460395",
      "userInputFieldsFileName": "",
      "testjobType": "load test",
      "enableAutomaticFileSync": "true",
      "testjobState": "running",
      "isClusterControllerJobProperties": "",
      "testjobDebugExecution": "false",
      "executeOnUserMeasuringAgentId": "65",
      "definedFromTimeZoneId": "Europe/Berlin",
      "executeMeasuringAgentInfo": "agent2.realload.com:8080",
      "executeOnMeasuringAgentClusterId": ""
    },
    "httpExecuteTimeMillis": 249
  },
  "isError": false
}

getMeasuringAgentTestjobs

Specific Request Fields:

  • measuringAgentId
  • synchronizeLocalTestjobsState (boolean)

Response Fields:

  • agentResponse

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getMeasuringAgentTestjobs",
  "measuringAgentId":65,
  "synchronizeLocalTestjobsState":true
}

JSON Response Example (Success Case):

{
  "agentResponse": {
    "allTestjobsArray": [
      {
        "testjobProperties": {
          "testjobResultFileName": "TestResult_RealloadTest_2023-09-09@23-45-34.json",
          "executeMeasuringAgentClusterInfo": "",
          "testjobConcurrentUsers": "1",
          "testjobDescription": "RealloadTest",
          "testjobDebugMeasuring": "false",
          "testjobMaxTestDuration": "60",
          "isClusterJob": "false",
          "testjobRampUpTime": "10",
          "localTestjobId": "30717",
          "testjobMaxLoopsPerUser": "1",
          "productVersion": "4.8.18",
          "remoteTestjobId": "15240",
          "testjobAdditionalArguments": "",
          "definedFromTestId": "379",
          "testjobSignature": "NeYJG4vf8/joAwz4nR4Fie3W0Qr7ijps25ONuk6F5BLghuD8/clPL9mBKc/ogQiic7wI8OXOf53Mi4YDCpg4dQzuaWQ==",
          "testjobDelayPerLoop": "1000",
          "testjobExitCode": "0",
          "createTimeStamp": "1694295923799",
          "userInputFieldsFileName": "",
          "testjobType": "load test",
          "enableAutomaticFileSync": "true",
          "testjobState": "completed",
          "isClusterControllerJobProperties": "",
          "testjobDebugExecution": "true",
          "executeOnUserMeasuringAgentId": "65",
          "definedFromTimeZoneId": "Europe/Berlin",
          "executeMeasuringAgentInfo": "agent2.realload.com:8080",
          "executeOnMeasuringAgentClusterId": ""
        },
        "testProperties": {
          "referencedResourceFilesList": [
            {
              "projectId": 130,
              "testplanId": 203,
              "fileName": "com.dkfqs.selenium.jar"
            },
            {
              "projectId": 130,
              "testplanId": 203,
              "fileName": "commons-io-2.13.0.jar"
            },
            {
              "projectId": 130,
              "testplanId": 203,
              "fileName": "hamcrest-core-1.3.jar"
            },
            {
              "projectId": 130,
              "testplanId": 203,
              "fileName": "com.dkfqs.tools.jar"
            },
            {
              "projectId": 130,
              "testplanId": 203,
              "fileName": "junit-4.13.2.jar"
            },
            {
              "projectId": 130,
              "testplanId": 203,
              "fileName": "selenium_bundle-4.12.1.jar"
            }
          ],
          "jsonConfigObject": {
            "productVersion": "4.8.20",
            "jUnitConfigExecutionClassArray": [
              {
                "productVersion": "4.8.20",
                "executingClassName": "RealloadTest",
                "testMethodNamesArray": [
                  "realload"
                ]
              }
            ],
            "seleniumWebBrowserType": "Firefox"
          },
          "executingScript": "RealloadTest.jar",
          "executingScriptResourceFileList": "",
          "createTimeStamp": "1694293491984",
          "testDescription": "",
          "definedFromTestplanId": "251",
          "productVersion": "4.8.20",
          "definedFromProjectId": "137",
          "visibleTestName": "RealloadTest",
          "scriptType": "Selenium4byJUnit4",
          "testId": "379"
        }
      },
      {
        "testjobProperties": {
          "testjobResultFileName": "TestResult_SimpleTest_2023-09-18@17-01-08.json",
          "executeMeasuringAgentClusterInfo": "",
          "testjobConcurrentUsers": "1",
          "testjobDescription": "Simple Test",
          "testjobDebugMeasuring": "false",
          "testjobMaxTestDuration": "30",
          "isClusterJob": "false",
          "testjobRampUpTime": "10",
          "localTestjobId": "33533",
          "testjobMaxLoopsPerUser": "1",
          "productVersion": "4.8.18",
          "remoteTestjobId": "24112",
          "testjobAdditionalArguments": "",
          "definedFromTestId": "384",
          "testjobSignature": "JYstx+myVNqbyVMr24u4Pi8IXkLGH0iXUquRh5+NL0veKs4DRJX/w2ZT77wLXF70J9YmTPzR7g==",
          "testjobDelayPerLoop": "1000",
          "testjobExitCode": "0",
          "createTimeStamp": "1695049264730",
          "userInputFieldsFileName": "",
          "testjobType": "load test",
          "enableAutomaticFileSync": "true",
          "testjobState": "completed",
          "isClusterControllerJobProperties": "",
          "testjobDebugExecution": "false",
          "executeOnUserMeasuringAgentId": "65",
          "definedFromTimeZoneId": "Europe/Berlin",
          "executeMeasuringAgentInfo": "agent2.realload.com:8080",
          "executeOnMeasuringAgentClusterId": ""
        },
        "testProperties": {
          "referencedResourceFilesList": [
            {
              "projectId": 130,
              "testplanId": 203,
              "fileName": "com.dkfqs.tools.jar"
            }
          ],
          "jsonConfigObject": {},
          "executingScript": "SimpleTest.jar",
          "executingScriptResourceFileList": "",
          "createTimeStamp": "1694549112155",
          "testDescription": "",
          "definedFromTestplanId": "251",
          "productVersion": "4.8.21",
          "definedFromProjectId": "137",
          "visibleTestName": "SimpleTest",
          "scriptType": "OpenJDK11Jar",
          "testId": "384"
        }
      },
      {
        "testjobProperties": {
          "testjobResultFileName": "TestResult_SimpleTest_2023-09-18@17-51-53.json",
          "executeMeasuringAgentClusterInfo": "",
          "testjobConcurrentUsers": "1",
          "testjobDescription": "Simple Test",
          "testjobDebugMeasuring": "false",
          "testjobMaxTestDuration": "30",
          "isClusterJob": "false",
          "testjobRampUpTime": "10",
          "localTestjobId": "33534",
          "testjobMaxLoopsPerUser": "1",
          "productVersion": "4.8.18",
          "remoteTestjobId": "24129",
          "testjobAdditionalArguments": "",
          "definedFromTestId": "384",
          "testjobSignature": "Z9B963yIj232nht1DIX5aSRt8m77QuXYjP9rn4OBSq/g==",
          "testjobDelayPerLoop": "1000",
          "testjobExitCode": "0",
          "createTimeStamp": "1695050460395",
          "userInputFieldsFileName": "",
          "testjobType": "load test",
          "enableAutomaticFileSync": "true",
          "testjobState": "completed",
          "isClusterControllerJobProperties": "",
          "testjobDebugExecution": "false",
          "executeOnUserMeasuringAgentId": "65",
          "definedFromTimeZoneId": "Europe/Berlin",
          "executeMeasuringAgentInfo": "agent2.realload.com:8080",
          "executeOnMeasuringAgentClusterId": ""
        },
        "testProperties": {
          "referencedResourceFilesList": [
            {
              "projectId": 130,
              "testplanId": 203,
              "fileName": "com.dkfqs.tools.jar"
            }
          ],
          "jsonConfigObject": {},
          "executingScript": "SimpleTest.jar",
          "executingScriptResourceFileList": "",
          "createTimeStamp": "1694549112155",
          "testDescription": "",
          "definedFromTestplanId": "251",
          "productVersion": "4.8.21",
          "definedFromProjectId": "137",
          "visibleTestName": "SimpleTest",
          "scriptType": "OpenJDK11Jar",
          "testId": "384"
        }
      }
    ],
    "httpExecuteTimeMillis": 141
  },
  "isError": false
}

getTestjobOutDirectoryFilesInfo

Specific Request Fields:

  • localTestjobId

Response Fields:

  • filesInfoArray (for measuring agents), or
  • controllerResponse (for cluster controllers)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getTestjobOutDirectoryFilesInfo",
  "localTestjobId":33534
}

JSON Response Example (Success Case):

{
  "isClusterJob": false,
  "measuringAgentData": {
    "agentId": 65,
    "createdBySystem": false,
    "ownerUserId": 13,
    "agentActive": true,
    "agentDescription": "Demo Agent 2",
    "agentHost": "agent2.realload.com",
    "agentPort": 8080,
    "authToken": "************",
    "sharedMeasuringAgentTemplateId": -1,
    "isLaunchedByCloudCredits": false
  },
  "filesInfoArray": [
    {
      "fileName": "DataCollector.err",
      "fileSize": 0,
      "lastModified": 1695051456261,
      "mimeType": "application/octet-stream"
    },
    {
      "fileName": "DataCollector.out",
      "fileSize": 5647,
      "lastModified": 1695052318516,
      "mimeType": "application/octet-stream"
    },
    {
      "fileName": "DataCollector.pid",
      "fileSize": 8,
      "lastModified": 1695051456273,
      "mimeType": "application/octet-stream"
    },
    {
      "fileName": "DataCollector.port",
      "fileSize": 6,
      "lastModified": 1695051457729,
      "mimeType": "application/octet-stream"
    },
    {
      "fileName": "DataCollector.state",
      "fileSize": 2,
      "lastModified": 1695052318512,
      "mimeType": "application/octet-stream"
    },
    {
      "fileName": "TestResult_SimpleTest_2023-09-18@17-51-53.json",
      "fileSize": 7421,
      "lastModified": 1695052315488,
      "mimeType": "application/json"
    },
    {
      "fileName": "user_1_statistics.out",
      "fileSize": 5037,
      "lastModified": 1695052313788,
      "mimeType": "application/octet-stream"
    },
    {
      "fileName": "users.out",
      "fileSize": 1923,
      "lastModified": 1695052314000,
      "mimeType": "application/octet-stream"
    }
  ],
  "isError": false
}

getTestjobOutDirectoryFile

Specific Request Fields:

  • localTestjobId
  • clusterMemberId (optional, -1 = no cluster member file)
  • fileName

Response Fields:

  • filesInfoArray (for measuring agents), or
  • controllerResponse (for cluster controllers)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getTestjobOutDirectoryFile",
  "localTestjobId":33534,
  "fileName":"users.out"
}

JSON Response Example (Success Case):

{
  "isClusterJob": false,
  "measuringAgentData": {
    "agentId": 65,
    "createdBySystem": false,
    "ownerUserId": 13,
    "agentActive": true,
    "agentDescription": "Demo Agent 2",
    "agentHost": "agent2.realload.com",
    "agentPort": 8080,
    "authToken": "************",
    "sharedMeasuringAgentTemplateId": -1,
    "isLaunchedByCloudCredits": false
  },
  "fileContentB64": "MjAyMy0wOS0xOCAxNTo1MTo1Mi42MjYgfCBtYWluIHwgSU5GTyB8IE1heC4gSmF2YSBNZW1vcnkgPSAzMzAwIE1CCjIwMjMtMDktMTggMTU6NTE6NTIuNjU1IHwgbWFpbiB8IElORk8gfCBUaW1lIFpvbmUgPSBDb29yZGluYXRlZCBVbml2ZXJzYWwgVGltZSArMDA6MDAKMjAyMy0wOS0xOCAxNTo1MTo1Mi42NjggfCBtYWluIHwgSU5GTyB8IC0tLSB2dnYgLS0tIHJlc291cmNlIGZpbGVzIC0tLSB2dnYgLS0tCjIwMjMtMDktMTggMTU6NTE6NTIuNjY5IHwgbWFpbiB8IElORk8gfCAtLS0gXl5eIC0tLSByZXNvdXJjZSBmaWxlcyAtLS0gXl5eIC0tLQoyMDIzLTA5LTE4IDE1OjUxOjUyLjY2OSB8IG1haW4gfCBJTkZPIHwgLS0tIHZ2diAtLS0gZ2VuZXJpYyBhcmd1bWVudHMgLS0tIHZ2diAtLS0KMjAyMy0wOS0xOCAxNTo1MTo1Mi42NjkgfCBtYWluIHwgSU5GTyB8IHVzZXJzID0gMQoyMDIzLTA5LTE4IDE1OjUxOjUyLjY2OSB8IG1haW4gfCBJTkZPIHwgZHVyYXRpb24gPSAzMAoyMDIzLTA5LTE4IDE1OjUxOjUyLjY3MCB8IG1haW4gfCBJTkZPIHwgbWF4TG9vcHMgPSAxCjIwMjMtMDktMTggMTU6NTE6NTIuNjcwIHwgbWFpbiB8IElORk8gfCBkZWxheVBlckxvb3AgPSAxMDAwCjIwMjMtMDktMTggMTU6NTE6NTIuNjcwIHwgbWFpbiB8IElORk8gfCByYW1wdXBUaW1lID0gMTAKMjAyMy0wOS0xOCAxNTo1MTo1Mi42NzAgfCBtYWluIHwgSU5GTyB8IGRhdGFPdXRwdXREaXIgPSAvaG9tZS9ka2Zxcy9hZ2VudC91c2Vyc0RhdGEvVXNlcl8xMy9UZXN0Sm9icy9UZXN0am9iXzI0MTI5L291dAoyMDIzLTA5LTE4IDE1OjUxOjUyLjY3MCB8IG1haW4gfCBJTkZPIHwgZGVidWdFeGVjID0gZmFsc2UKMjAyMy0wOS0xOCAxNTo1MTo1Mi42NzEgfCBtYWluIHwgSU5GTyB8IGRlYnVnRGF0YSA9IGZhbHNlCjIwMjMtMDktMTggMTU6NTE6NTIuNjcxIHwgbWFpbiB8IElORk8gfCBkZXNjcmlwdGlvbiA9IFNpbXBsZSBUZXN0CjIwMjMtMDktMTggMTU6NTE6NTIuNjcxIHwgbWFpbiB8IElORk8gfCAtLS0gXl5eIC0tLSBnZW5lcmljIGFyZ3VtZW50cyAtLS0gXl5eIC0tLQoyMDIzLTA5LTE4IDE1OjUxOjUyLjkyMSB8IG1haW4gfCBJTkZPIHwgSlZNIFdhcm11cCBwZXJmb3JtZWQgaW4gMjM2IG1zCjIwMjMtMDktMTggMTU6NTE6NTIuOTc3IHwgbWFpbiB8IElORk8gfCAtLS0gdnZ2IC0tLSBzcGVjaWZpYyBhcmd1bWVudHMgLS0tIHZ2diAtLS0KMjAyMy0wOS0xOCAxNTo1MTo1Mi45OTQgfCBtYWluIHwgSU5GTyB8IHRjcFRpbWVvdXQgPSAxMDAwMAoyMDIzLTA5LTE4IDE1OjUxOjUyLjk5NSB8IG1haW4gfCBJTkZPIHwgc3NsVGltZW91dCA9IDUwMDAKMjAyMy0wOS0xOCAxNTo1MTo1Mi45OTcgfCBtYWluIHwgSU5GTyB8IGh0dHBUaW1lb3V0ID0gMzAwMDAKMjAyMy0wOS0xOCAxNTo1MTo1Mi45OTcgfCBtYWluIHwgSU5GTyB8IC0tLSBeXl4gLS0tIHNwZWNpZmljIGFyZ3VtZW50cyAtLS0gXl5eIC0tLQoyMDIzLTA5LTE4IDE1OjUxOjUzLjAwMiB8IG1haW4gfCBJTkZPIHwgW1N0YXJ0IG9mIFRlc3RdCjIwMjMtMDktMTggMTU6NTE6NTMuMDAzIHwgVXNlci0xIHwgSU5GTyB8IFVzZXIgc3RhcnRlZAoyMDIzLTA5LTE4IDE1OjUxOjUzLjAyNiB8IHBlcmlvZGljLXRocmVhZCB8IElORk8gfCBUaHJlYWQgc3RhcnRlZAoyMDIzLTA5LTE4IDE1OjUxOjUzLjAyNiB8IFVzZXItMSB8IElORk8gfCBTdGFydCBleGVjdXRlIHNlc3Npb24gbm8uIDEgLi4uCjIwMjMtMDktMTggMTU6NTE6NTMuNTk3IHwgVXNlci0xIHwgSU5GTyB8IC4uLiBFbmQgZXhlY3V0ZSBzZXNzaW9uIG5vLiAxIHN1Y2Nlc3NmdWwgcGFzc2VkCjIwMjMtMDktMTggMTU6NTE6NTMuNjQ5IHwgVXNlci0xIHwgSU5GTyB8IFVzZXIgbm9ybWFsIHRlcm1pbmF0ZWQKMjAyMy0wOS0xOCAxNTo1MTo1NC4wMDMgfCBtYWluIHwgSU5GTyB8IFtFbmQgb2YgVGVzdF0K",
  "isError": false
}

saveTestjobOutDirectoryFileToProjectTree

Specific Request Fields:

  • localTestjobId
  • clusterMemberId (optional, -1 = no cluster member file)
  • fileName
  • saveAsFileName
  • projectId
  • resourceSetId

Response Fields:

  • fileHashCode

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"saveTestjobOutDirectoryFileToProjectTree",
  "localTestjobId":33534,
  "fileName":"users.out",
  "saveAsFileName":"users.out",
  "projectId":137,
  "resourceSetId":251
}

JSON Response Example (Success Case):

{
  "isClusterJob": false,
  "fileHashCode": 4557110,
  "isError": false
}

deleteTestjob

Specific Request Fields:

  • localTestjobId
  • deleteAlsoRemoteTestjob (boolean)

Response Fields:

  • localTestjobState (before the testjob was delete)
  • agentResponse (for measuring agents), or
  • controllerResponse (for cluster controllers)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"deleteTestjob",
  "localTestjobId":33534,
  "deleteAlsoRemoteTestjob":true
}

JSON Response Example (Success Case):

{
  "isClusterJob": false,
  "localTestjobState": "completed",
  "agentResponse": {
    "httpExecuteTimeMillis": 143
  },
  "isError": false
}

getTestExecutionSuites

Specific Request Fields:

  • [none]

Response Fields:

  • testExecutionSuitesArray
    • testExecutionSuite
    • testRunResultsArray

Specific Error Flags:

  • [none]

Values of testRunResultsArray[n].executionState :

  • EXECUTION_STATE_INITIALIZED = 0;
  • EXECUTION_STATE_RUNNING = 1;
  • EXECUTION_STATE_RUN_COMPLETED = 2;
  • EXECUTION_STATE_RUN_FAILED = 3;
  • EXECUTION_STATE_RUN_ABORTED = 4;

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getTestExecutionSuites"
}

JSON Response Example (Success Case):

{
  "testExecutionSuitesArray": [
    {
      "testExecutionSuite": {
        "testExecutionSuiteId": 2,
        "testExecutionSuiteName": "First Suite",
        "testExecutionSuiteDescription": "",
        "topExecuteOnMeasuringAgentOrClusterId": "m-145",
        "topExecutionOrder": 2,
        "totalNumberOfTestjobs": 3,
        "isTestExecutionSuiteHealthy": true
      },
      "testRunResultsArray": [
        {
          "userId": 13,
          "subUserId": -1,
          "testRunId": 1,
          "executionState": 2,
          "annotationArray": [],
          "testRunStartTimestamp": 1699716014207,
          "testRunLastUpdatedTimestamp": 1699716054368,
          "numTestjobTemplatesToExecute": 3,
          "numTestjobTemplatesExecutionPassed": 3,
          "numTestjobTemplatesExecutionFailed": 0,
          "totalSessionsPassed": 8,
          "totalSessionsFailed": 0,
          "totalSessionsPassedTimeSum": 5295,
          "totalSamplesPassed": 12,
          "totalSamplesFailed": 0,
          "maxConcurrentUsers": 1,
          "maxPendingSamples": 1
        },
        {
          "userId": 13,
          "subUserId": -1,
          "testRunId": 2,
          "executionState": 2,
          "annotationArray": [],
          "testRunStartTimestamp": 1699716108571,
          "testRunLastUpdatedTimestamp": 1699716151826,
          "numTestjobTemplatesToExecute": 3,
          "numTestjobTemplatesExecutionPassed": 3,
          "numTestjobTemplatesExecutionFailed": 0,
          "totalSessionsPassed": 8,
          "totalSessionsFailed": 0,
          "totalSessionsPassedTimeSum": 5503,
          "totalSamplesPassed": 12,
          "totalSamplesFailed": 0,
          "maxConcurrentUsers": 1,
          "maxPendingSamples": 1
        }
      ]
    },
    {
      "testExecutionSuite": {
        "testExecutionSuiteId": 3,
        "testExecutionSuiteName": "Second Suite Long Run",
        "testExecutionSuiteDescription": "",
        "topExecuteOnMeasuringAgentOrClusterId": "c-27",
        "topExecutionOrder": 1,
        "totalNumberOfTestjobs": 2,
        "isTestExecutionSuiteHealthy": true
      },
      "testRunResultsArray": [
        {
          "userId": 13,
          "subUserId": -1,
          "testRunId": 1,
          "executionState": 2,
          "annotationArray": [],
          "testRunStartTimestamp": 1699715808192,
          "testRunLastUpdatedTimestamp": 1699715867468,
          "numTestjobTemplatesToExecute": 2,
          "numTestjobTemplatesExecutionPassed": 2,
          "numTestjobTemplatesExecutionFailed": 0,
          "totalSessionsPassed": 44,
          "totalSessionsFailed": 0,
          "totalSessionsPassedTimeSum": 4400,
          "totalSamplesPassed": 44,
          "totalSamplesFailed": 0,
          "maxConcurrentUsers": 4,
          "maxPendingSamples": 4
        },
        {
          "userId": 13,
          "subUserId": -1,
          "testRunId": 2,
          "executionState": 2,
          "annotationArray": [],
          "testRunStartTimestamp": 1699715952212,
          "testRunLastUpdatedTimestamp": 1699716012124,
          "numTestjobTemplatesToExecute": 2,
          "numTestjobTemplatesExecutionPassed": 2,
          "numTestjobTemplatesExecutionFailed": 0,
          "totalSessionsPassed": 44,
          "totalSessionsFailed": 0,
          "totalSessionsPassedTimeSum": 4400,
          "totalSamplesPassed": 44,
          "totalSamplesFailed": 0,
          "maxConcurrentUsers": 4,
          "maxPendingSamples": 3
        },
        {
          "userId": 13,
          "subUserId": -1,
          "testRunId": 3,
          "executionState": 2,
          "annotationArray": [],
          "testRunStartTimestamp": 1699717843777,
          "testRunLastUpdatedTimestamp": 1699717905674,
          "numTestjobTemplatesToExecute": 2,
          "numTestjobTemplatesExecutionPassed": 2,
          "numTestjobTemplatesExecutionFailed": 0,
          "totalSessionsPassed": 43,
          "totalSessionsFailed": 0,
          "totalSessionsPassedTimeSum": 4300,
          "totalSamplesPassed": 43,
          "totalSamplesFailed": 0,
          "maxConcurrentUsers": 4,
          "maxPendingSamples": 3
        }
      ]
    }
  ],
  "isError": false
}

getCurrentlyRunningTestExecutionSuiteIds

Specific Request Fields:

  • [none]

Response Fields:

  • testExecutionSuiteIdArray

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getCurrentlyRunningTestExecutionSuiteIds"
}

JSON Response Example (Success Case):

{
  "testExecutionSuiteIdArray": [
    3
  ],
  "isError": false
}

startTestExecutionSuiteTestRun

Specific Request Fields:

  • testExecutionSuiteId
  • testRunAnnotation (optional, can also be an empty string)
  • topExecuteOnMeasuringAgentOrClusterId (the ’top execution’ measuring agent id or cluster id)
  • topExecuteOnCluster (boolean, if true: top execution = cluster id, or false: top execution = measuring agent id)

Response Fields:

  • newTestRunId

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"startTestExecutionSuiteTestRun",
  "testExecutionSuiteId":3,
  "testRunAnnotation":"",
  "topExecuteOnMeasuringAgentOrClusterId":143,
  "topExecuteOnCluster":false
}

JSON Response Example (Success Case):

{
  "newTestRunId": 5,
  "isError": false
}

JSON Response Example (Error Case):

{
  "isError": true,
  "genericErrorText": "Test execution suite currently running"
}

getTestExecutionSuiteTestRunResultDetail

Specific Request Fields:

  • testExecutionSuiteId
  • testRunId

Response Fields:

  • testRunResultDetail
    • testRunResult
    • executedByUserNickname
    • testjobsOutFilesArray

Specific Error Flags:

  • [none]

Values of executionState :

  • EXECUTION_STATE_INITIALIZED = 0;
  • EXECUTION_STATE_RUNNING = 1;
  • EXECUTION_STATE_RUN_COMPLETED = 2;
  • EXECUTION_STATE_RUN_FAILED = 3;
  • EXECUTION_STATE_RUN_ABORTED = 4;

Values of logLevel :

  • LOG_DEBUG = 4;
  • LOG_INFO = 7;
  • LOG_WARN = 8;
  • LOG_ERROR = 9;
  • LOG_FATAL = 10;

Values of elementType :

  • ELEMENT_TYPE_TESTJOB_TEMPLATE = 1;
  • ELEMENT_TYPE_EXECUTION_GROUP = 10;

Values of executionOrder :

  • EXECUTION_ORDER_SEQUENTIAL = 1;
  • EXECUTION_ORDER_PARALLEL = 2;

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getTestExecutionSuiteTestRunResultDetail",
  "testExecutionSuiteId":3,
  "testRunId":5
}

JSON Response Example (Success Case):

{
  "testRunResultDetail": {
    "testRunResult": {
      "userId": 13,
      "subUserId": -1,
      "testExecutionSuite": {
        "testExecutionSuiteId": 3,
        "testExecutionSuiteName": "Second Suite Long Run",
        "testExecutionSuiteDescription": "",
        "topExecuteOnMeasuringAgentOrClusterId": "c-27",
        "topExecutionOrder": 1,
        "testExecutionElementsArray": [
          {
            "productVersion": "4.8.24",
            "elementType": 10,
            "elementId": 737867696,
            "elementInactive": false,
            "executionGroupName": "Top Group",
            "executionGroupDescription": "",
            "executionOrder": 2,
            "executeOnMeasuringAgentOrClusterId": "",
            "executionElementsArray": [
              {
                "productVersion": "4.8.24",
                "elementType": 1,
                "elementId": 1040705161,
                "elementInactive": false,
                "templateId": 18
              },
              {
                "productVersion": "4.8.24",
                "elementType": 1,
                "elementId": 493179849,
                "elementInactive": false,
                "templateId": 18
              }
            ]
          }
        ]
      },
      "testjobTemplatesArray": [
        {
          "templateId": 18,
          "templateName": "Simple Test 2VU Long Run",
          "templateDescription": "",
          "testjobProperties": {
            "testjobResultFileName": "",
            "executeMeasuringAgentClusterInfo": "",
            "testjobConcurrentUsers": "2",
            "testjobDescription": "Simple Test 2VU Long Run",
            "testjobDebugMeasuring": "false",
            "testjobMaxTestDuration": "30",
            "isClusterJob": "",
            "testjobRampUpTime": "1",
            "localTestjobId": "",
            "testjobMaxLoopsPerUser": "-1",
            "productVersion": "4.8.24",
            "remoteTestjobId": "",
            "testjobAdditionalArguments": "",
            "definedFromTestId": "384",
            "testjobSignature": "",
            "testjobDelayPerLoop": "3000",
            "testjobExitCode": "",
            "createTimeStamp": "1697224057753",
            "userInputFieldsFileName": "",
            "testjobType": "load test",
            "enableAutomaticFileSync": "true",
            "testjobState": "invalid",
            "isClusterControllerJobProperties": "",
            "testjobDebugExecution": "false",
            "executeOnUserMeasuringAgentId": "",
            "definedFromTimeZoneId": "Europe/Berlin",
            "executeMeasuringAgentInfo": "",
            "executeOnMeasuringAgentClusterId": ""
          },
          "hasUserInputFields": false
        }
      ],
      "testRunId": 5,
      "executorLog": {
        "userId": 13,
        "testExecutionSuiteId": 3,
        "testRunId": 5,
        "logEntriesArray": [
          {
            "timestamp": 1699726961703,
            "logLevel": 7,
            "threadName": "TestSuiteExecutor-S:3-R:5",
            "testExecutionElementId": -1,
            "logMessage": "Test run #5 started",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699726963704,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-G:737867696",
            "testExecutionElementId": 737867696,
            "logMessage": "Execution Group 'Top Group' started",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699726963706,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:1040705161",
            "testExecutionElementId": 1040705161,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' execution started",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699726963708,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:493179849",
            "testExecutionElementId": 493179849,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' execution started",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699726963731,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:1040705161",
            "testExecutionElementId": 1040705161,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' test job defined with id = 34442",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699726963753,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:493179849",
            "testExecutionElementId": 493179849,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' test job defined with id = 34443",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699726964081,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:1040705161",
            "testExecutionElementId": 1040705161,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' test job submitted to 'Shared Agent 2', remote test job id = 791",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699726964128,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:493179849",
            "testExecutionElementId": 493179849,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' test job submitted to 'Shared Agent 2', remote test job id = 792",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699726968458,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:493179849",
            "testExecutionElementId": 493179849,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' started on 'Shared Agent 2', remote test job id = 792",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699726968466,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:1040705161",
            "testExecutionElementId": 1040705161,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' started on 'Shared Agent 2', remote test job id = 791",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699727005910,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:493179849",
            "testExecutionElementId": 493179849,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' execution successful",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699727006912,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:1040705161",
            "testExecutionElementId": 1040705161,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' execution successful",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699727012111,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:1040705161",
            "testExecutionElementId": 1040705161,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' execution terminated",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699727012112,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:493179849",
            "testExecutionElementId": 493179849,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' execution terminated",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699727012113,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-G:737867696",
            "testExecutionElementId": 737867696,
            "logMessage": "Execution Group 'Top Group' terminated",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699727014114,
            "logLevel": 7,
            "threadName": "TestSuiteExecutor-S:3-R:5",
            "testExecutionElementId": -1,
            "logMessage": "Test run #5 done. Execution state = completed",
            "stacktraceArray": []
          }
        ]
      },
      "executionState": 2,
      "annotationArray": [],
      "testRunStartTimestamp": 1699726961701,
      "testRunLastUpdatedTimestamp": 1699727012113,
      "numTestjobTemplatesToExecute": 2,
      "numTestjobTemplatesExecutionPassed": 2,
      "numTestjobTemplatesExecutionFailed": 0,
      "totalSessionsPassed": 44,
      "totalSessionsFailed": 0,
      "totalSessionsPassedTimeSum": 4400,
      "totalSamplesPassed": 44,
      "totalSamplesFailed": 0,
      "maxConcurrentUsers": 4,
      "maxPendingSamples": 2,
      "testRunExecutionGroupResultArray": [
        {
          "testExecutionSuiteId": 3,
          "testRunId": 5,
          "executionGroupElementId": 737867696,
          "containsMeasuringAgent": true,
          "measuringAgent": {
            "agentId": 143,
            "createdBySystem": true,
            "ownerUserId": 13,
            "agentActive": true,
            "agentDescription": "Shared Agent 2",
            "agentHost": "192.168.0.50",
            "agentPort": 8081,
            "sharedMeasuringAgentTemplateId": 5,
            "isLaunchedByCloudCredits": false
          },
          "containsMeasuringAgentCluster": false,
          "executionState": 2,
          "executionStartTimestamp": 1699726963704,
          "executionEndTimestamp": 1699727012113,
          "numTestjobTemplatesExecutionPassed": 2,
          "numTestjobTemplatesExecutionFailed": 0,
          "totalSessionsPassed": 44,
          "totalSessionsFailed": 0,
          "totalSessionsPassedTimeSum": 4400,
          "totalSamplesPassed": 44,
          "totalSamplesFailed": 0
        }
      ],
      "testRunTestjobResultArray": [
        {
          "testExecutionSuiteId": 3,
          "testRunId": 5,
          "testExecutionGroupElementId": 737867696,
          "testjobTemplateElementId": 493179849,
          "testjobTemplate": {
            "templateId": 18,
            "templateName": "Simple Test 2VU Long Run",
            "templateDescription": "",
            "testjobProperties": {
              "testjobResultFileName": "TestResult_SimpleTest_2023-11-11@19-22-49.json",
              "executeMeasuringAgentClusterInfo": "",
              "testjobConcurrentUsers": "2",
              "testjobDescription": "Simple Test 2VU Long Run",
              "testjobDebugMeasuring": "false",
              "testjobMaxTestDuration": "30",
              "isClusterJob": "false",
              "testjobRampUpTime": "1",
              "localTestjobId": "34443",
              "testjobMaxLoopsPerUser": "-1",
              "productVersion": "4.8.24",
              "remoteTestjobId": "792",
              "testjobAdditionalArguments": "",
              "definedFromTestId": "384",
              "testjobSignature": "",
              "testjobDelayPerLoop": "3000",
              "testjobExitCode": "",
              "createTimeStamp": "1697224057753",
              "userInputFieldsFileName": "",
              "testjobType": "load test",
              "enableAutomaticFileSync": "true",
              "testjobState": "invalid",
              "isClusterControllerJobProperties": "",
              "testjobDebugExecution": "false",
              "executeOnUserMeasuringAgentId": "143",
              "definedFromTimeZoneId": "Europe/Berlin",
              "executeMeasuringAgentInfo": "Shared Agent 2",
              "executeOnMeasuringAgentClusterId": ""
            },
            "hasUserInputFields": false
          },
          "executionState": 2,
          "executionStartTimestamp": 1699726963708,
          "executionEndTimestamp": 1699727005710,
          "jobStartTimestamp": 1699726969618,
          "jobEndTimestamp": 1699727003517,
          "totalSessionsPassed": 22,
          "totalSessionsFailed": 0,
          "totalSessionsPassedTimeSum": 2200,
          "totalSamplesPassed": 22,
          "totalSamplesFailed": 0,
          "maxConcurrentUsers": 2,
          "maxPendingSamples": 1
        },
        {
          "testExecutionSuiteId": 3,
          "testRunId": 5,
          "testExecutionGroupElementId": 737867696,
          "testjobTemplateElementId": 1040705161,
          "testjobTemplate": {
            "templateId": 18,
            "templateName": "Simple Test 2VU Long Run",
            "templateDescription": "",
            "testjobProperties": {
              "testjobResultFileName": "TestResult_SimpleTest_2023-11-11@19-22-49.json",
              "executeMeasuringAgentClusterInfo": "",
              "testjobConcurrentUsers": "2",
              "testjobDescription": "Simple Test 2VU Long Run",
              "testjobDebugMeasuring": "false",
              "testjobMaxTestDuration": "30",
              "isClusterJob": "false",
              "testjobRampUpTime": "1",
              "localTestjobId": "34442",
              "testjobMaxLoopsPerUser": "-1",
              "productVersion": "4.8.24",
              "remoteTestjobId": "791",
              "testjobAdditionalArguments": "",
              "definedFromTestId": "384",
              "testjobSignature": "",
              "testjobDelayPerLoop": "3000",
              "testjobExitCode": "",
              "createTimeStamp": "1697224057753",
              "userInputFieldsFileName": "",
              "testjobType": "load test",
              "enableAutomaticFileSync": "true",
              "testjobState": "invalid",
              "isClusterControllerJobProperties": "",
              "testjobDebugExecution": "false",
              "executeOnUserMeasuringAgentId": "143",
              "definedFromTimeZoneId": "Europe/Berlin",
              "executeMeasuringAgentInfo": "Shared Agent 2",
              "executeOnMeasuringAgentClusterId": ""
            },
            "hasUserInputFields": false
          },
          "executionState": 2,
          "executionStartTimestamp": 1699726963706,
          "executionEndTimestamp": 1699727006711,
          "jobStartTimestamp": 1699726969515,
          "jobEndTimestamp": 1699727003517,
          "totalSessionsPassed": 22,
          "totalSessionsFailed": 0,
          "totalSessionsPassedTimeSum": 2200,
          "totalSamplesPassed": 22,
          "totalSamplesFailed": 0,
          "maxConcurrentUsers": 2,
          "maxPendingSamples": 1
        }
      ]
    },
    "executedByUserNickname": "DKF",
    "testjobsOutFilesArray": [
      {
        "elementId": 1040705161,
        "outFileInfoArray": [
          {
            "fileName": "DataCollector.err",
            "fileSize": 0,
            "lastModified": 1699727006724,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "DataCollector.out",
            "fileSize": 6537,
            "lastModified": 1699727006726,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "DataCollector.pid",
            "fileSize": 7,
            "lastModified": 1699727006736,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "DataCollector.port",
            "fileSize": 6,
            "lastModified": 1699727006737,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "DataCollector.state",
            "fileSize": 1,
            "lastModified": 1699727006738,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "TestResult_SimpleTest_2023-11-11@19-22-49.json",
            "fileSize": 11913,
            "lastModified": 1699727006739,
            "mimeType": "application/json"
          },
          {
            "fileName": "users.out",
            "fileSize": 5507,
            "lastModified": 1699727006741,
            "mimeType": "application/octet-stream"
          }
        ]
      },
      {
        "elementId": 493179849,
        "outFileInfoArray": [
          {
            "fileName": "DataCollector.err",
            "fileSize": 0,
            "lastModified": 1699727005722,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "DataCollector.out",
            "fileSize": 6264,
            "lastModified": 1699727005725,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "DataCollector.pid",
            "fileSize": 7,
            "lastModified": 1699727005733,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "DataCollector.port",
            "fileSize": 6,
            "lastModified": 1699727005734,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "DataCollector.state",
            "fileSize": 1,
            "lastModified": 1699727005735,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "TestResult_SimpleTest_2023-11-11@19-22-49.json",
            "fileSize": 11850,
            "lastModified": 1699727005736,
            "mimeType": "application/json"
          },
          {
            "fileName": "users.out",
            "fileSize": 5507,
            "lastModified": 1699727005738,
            "mimeType": "application/octet-stream"
          }
        ]
      }
    ]
  },
  "isError": false
}

getTestExecutionSuiteTestRunTestJobOutputFileContent

Specific Request Fields:

  • testExecutionSuiteId
  • testRunId
  • testjobTemplateElementId (don’t confuse with testjobTemplateId)
  • fileName

Response Fields:

  • fileContentB64

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getTestExecutionSuiteTestRunTestJobOutputFileContent",
  "testExecutionSuiteId":3,
  "testRunId":5,
  "testjobTemplateElementId":1040705161,
  "fileName":"users.out"
}

JSON Response Example (Success Case):

{
  "fileContentB64": "MjAyMy0xMS0xMSAxO ... uZCBvZiBUZXN0XQo=",
  "isError": false
}

deleteTestExecutionSuiteTestRun

Specific Request Fields:

  • testExecutionSuiteId
  • testRunId

Response Fields:

  • [none]

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"deleteTestExecutionSuiteTestRun",
  "testExecutionSuiteId":3,
  "testRunId":5
}

JSON Response Example (Success Case):

{
  "isError": false
}

5.3 - Remote User Monitoring API

Portal Server Remote User Monitoring API Specification

Abstract

The “Remote User Monitoring API” is a WebSocket API that supports developers to implement their own version of a Synthetic Monitoring “Real Time Dashboard”.

Similar to the Remote User API there is also an “API Authentication Token” required to access the API.

“alt attribute”

API Functions

The API supports the following functions (so-called “actions”):

  • pingPortalServer : Ping the Portal Server at API level.
  • subscribeToHeartbeat : Subscribe to a periodic timestamp which is received every second.
  • unsubscribeFromHeartbeat : Cancel subscription to ‘subscribeToHeartbeat’.
  • subscribeToPeriodicAliveThread: Same functionality as ‘subscribeToHeartbeat’, but the time interval (in seconds) can be specified arbitrarily.
  • unsubscribeFromPeriodicAliveThread: Cancel subscription to ‘subscribeToPeriodicAliveThread’.
  • subscribeToRealtimeMonitor : Subscribe to events when a Monitoring Group or a Monitoring Job is created, modified or deleted, and subscribe to be notified each time when a Monitoring Group or a Monitoring Job is executed.
  • unsubscribeFromRealtimeMonitor : Cancel subscription to ‘subscribeToRealtimeMonitor’.
  • subscribeToMonitoringAlerts : Subscribe to monitoring alerts.
  • unsubscribeFromMonitoringAlerts : Cancel subscription to ‘subscribeToMonitoringAlerts’.
  • subscribeToGroupExecutionHistoryCache : Subscribe to a monitoring group’s execution history cache, which contains the measured data over the last 24 hours. The received data can be used to draw a chart or sparkline of the corresponding monitoring group.
  • unsubscribeFromAllGroupExecutionHistoryCaches : Cancel subscription to ‘subscribeToGroupExecutionHistoryCache’ for all monitoring groups.
  • subscribeToJobExecutionHistoryCache : Subscribe to a monitoring job’s execution history cache, which contains the measured data over the last 24 hours. The received data can be used to draw a chart or sparkline of the corresponding monitoring job.
  • unsubscribeFromAllJobExecutionHistoryCaches : Cancel subscription to ‘subscribeToJobExecutionHistoryCache’ for all monitoring jobs.
WebSocket URL
wss://portal.realload.com/RemoteUserMonitoringAPI
HTTP Request Header Fields
x-api-key: <API Authentication Token> required
subscribeToHeartbeat: true optional
subscribeToRealtimeMonitor: true optional

The subscribed data are received in JSON format. The received JSON values are never null.

Runtime Behavior

  • When the WebSocket connection is closed, all subscriptions are terminated.
  • When you send an “action”, the action is confirmed with a response frame with the name of the action repeated (except for “pingPortalServer”).
  • If you send an invalid “action”, the WebSocket connection will be closed.
  • The WebSocket connection doesn’t last forever. It will be closed after a few hours by the Portal Server. You then have to reestablish the WebSocket connection and renew the subscriptions.

Example:

Send: {"action":"subscribeToHeartbeat"}
Received: {"action":"subscribeToHeartbeat","isError":false}
Received: {"action":"heartbeat","timestamp":1708466543962,"isError":false}
Received: {"action":"heartbeat","timestamp":1708466544963,"isError":false}
Received: {"action":"heartbeat","timestamp":1708466545963,"isError":false}

Implementation Recommendations

  • We recommend that you implement two hash maps on the client side which contain the current configuration data of the monitoring jobs and monitoring groups - based on the actions of onMonitoringGroupEvent and onMonitoringJobEvent received from subscribeToRealtimeMonitor (hash map keys = Monitoring Group ID, Monitoring Job ID).
  • Preferably use a WebSocket client framework that can detect the closure of the WebSocket connection to the server. If this is not possible, use “subscribeToPeriodicAliveThread” or “pingPortalServer”.

pingPortalServer

Example:

Send: {"action":"pingPortalServer"}
Received: {"action":"pongFromPortalServer","portalServerVersion":"4.8.35","portalServerTimestamp":1708468300516,"isError":false}

subscribeToHeartbeat

  • Send action : “subscribeToHeartbeat”
{
  "action":"subscribeToHeartbeat"
}
  • Received action: “heartbeat”
{
  "action":"heartbeat",
  "timestamp":1677521478771,
  "isError":false
}

subscribeToPeriodicAliveThread

  • Send action : “subscribeToPeriodicAliveThread”
{
  "action":"subscribeToPeriodicAliveThread",
  "periodicIntervalSeconds":60
}
  • Received action: “periodicAlive”
{
  "action":"periodicAlive",
  "timestamp":1708469366167,
  "isError":false
}

subscribeToRealtimeMonitor

  • Send action : “subscribeToRealtimeMonitor”
{
  "action":"subscribeToRealtimeMonitor"
}
  • Received actions:
    • onMonitoringGroupEvent(The configuration data of a monitoring group)
    • onMonitoringJobEvent(The configuration data of a monitoring job)
    • onMonitoringGroupExecuted(The execution result of a monitoring group)
    • onMonitoringJobExecuted(The execution result of a monitoring job)
    • onMonitoringGroupTemporarilyEvent(Notification whether a monitoring group is temporarily disabled)
    • onMonitoringJobTemporarilyEvent(Notification whether a monitoring job is temporarily disabled)

As first you will receive instantly the configuration data of all already defined Monitoring Groups and Monitoring Jobs, and also the data of the last Execution Result of the Monitoring Groups and Monitoring Jobs (if they have been executed in the past).

After that you will receive update information of all Monitoring Groups and Monitoring Jobs when they are new-defined/modified/deleted, and you will receive also update information about the result of executed Monitoring Groups and Monitoring Jobs.

Because monitoring groups and jobs can be temporarily disabled by configuring one or multiple “Monitoring Downtime(s)” you receive the actions “onMonitoringGroupTemporarilyEvent” and “onMonitoringJobTemporarilyEvent”. “alt attribute”

Notifications for Monitoring Groups

  • Received action: “onMonitoringGroupEvent”
  • monitoringGroupAction:
    • “groupExists”
    • “groupAdded”
    • “groupModified”
    • “groupDeleted”
{
  "action": "onMonitoringGroupEvent",
  "monitoringGroupId": 34,
  "monitoringEvent": {
    "eventTimestamp": 1708541771774,
    "monitoringGroup": {
      "monitoringGroupId": 34,
      "ownerUserId": 13,
      "title": "First Group",
      "description": "",
      "maxDataStorageDays": 14,
      "agentType": "agent",
      "executionEnabled": true,
      "executionInterval": 300,
      "executionTimeout": 300,
      "currentExecutionState": "executing",
      "lastExecutionStartTime": 1708541763897
    },
    "monitoringGroupAction": "groupExists"
  },
  "isError": false
}

The executionInterval and the executionTimeout are in seconds. The lastExecutionStartTime has a value of -1 (minus one) if the Monitoring Group was never executed. The agentType has currently no meaning.

Any modification of a Monitoring Group made by the Portal Server Web Interface such as modify executionEnabled or title effects that a new frame with monitoringGroupAction = “groupModified” is received.

Notification for Monitoring Jobs

  • Received action: “onMonitoringJobEvent”
  • monitoringJobAction:
    • “jobExists”
    • “jobAdded”
    • “jobModified”
    • “jobDeleted”
{
  "action": "onMonitoringJobEvent",
  "monitoringGroupId": 36,
  "monitoringJobId": 79,
  "monitoringEvent": {
    "eventTimestamp": 1708541771790,
    "monitoringJobProperties": {
      "performanceErrorAlertThresholdMillis": "-1",
      "enableJobExecution": "true",
      "performanceWarningAlertsEnabled": "false",
      "testjobConcurrentUsers": "1",
      "testjobDelayPerLoop": "1000",
      "testjobDebugMeasuring": "false",
      "performanceErrorAlertsEnabled": "false",
      "enableAutomaticFileSync": "false",
      "performanceWarningAlertThresholdMillis": "-1",
      "testjobMaxTestDuration": "60",
      "testjobDebugExecution": "false",
      "testjobRampUpTime": "5",
      "definedFromMonitoringGroupId": "36",
      "testjobMaxLoopsPerUser": "1",
      "productVersion": "4.8.35",
      "monitoringJobId": "79",
      "monitoringJobTitle": "Local_Cinema",
      "lastExecutionStartTime": "1708541731138",
      "testjobAdditionalArguments": "",
      "definedFromTestId": "399",
      "testjobUserInputFieldsValuesJsonArray": "[]",
      "monitoringJobDescription": ""
    },
    "monitoringTestProperties": {
      "referencedResourceFilesList": [
        {
          "projectId": 130,
          "testplanId": 203,
          "fileName": "com.dkfqs.tools.jar"
        }
      ],
      "jsonConfigObject": {},
      "executingScript": "Local_Cinema.jar",
      "executingScriptResourceFileList": "",
      "createTimeStamp": "1708293851731",
      "testDescription": "",
      "definedFromTestplanId": "287",
      "productVersion": "4.8.34",
      "definedFromProjectId": "137",
      "visibleTestName": "Local_Cinema",
      "scriptType": "OpenJDK11Jar",
      "testId": "399"
    },
    "monitoringTestIsHealthy": true,
    "monitoringJobAction": "jobExists"
  },
  "isError": false
}

The testjobMaxTestDuration and the testjobRampUpTime are in seconds. The testjobDelayPerLoop are in milliseconds. The lastExecutionStartTime has a value of -1 (minus one) if the Monitoring Job was never executed.

Any modification of a Monitoring Job made by the Portal Server Web Interface such as modify enableJobExecution or monitoringJobTitle effects that a new frame with monitoringJobAction = “jobModified” is received.

Notification for Monitoring Group Execution

  • Received action: “onMonitoringGroupExecuted”
  • oldEvent: true or false | Note: the oldEvent is true at the first time (if the monitoring group was executed in the past) and then will always be false each time when the monitoring group is executed.
{
  "action": "onMonitoringGroupExecuted",
  "monitoringGroupId": 34,
  "monitoringEvent": {
    "eventTimestamp": 1708541771806,
    "oldEvent": true,
    "groupExecutionResult": {
      "magicPattern": "MonitoringGroupExecutionResult",
      "productVersion": "4.8.35",
      "userId": 13,
      "monitoringGroupId": 34,
      "groupExecutionId": 39412,
      "resultTimestamp": 1708541492973,
      "monitoringJobIdToLocalTestjobIdArray": [
        {
          "monitoringJobId": 60,
          "localTestjobId": 73932
        },
        {
          "monitoringJobId": 61,
          "localTestjobId": 73934
        },
        {
          "monitoringJobId": 62,
          "localTestjobId": 73933
        }
      ],
      "groupSystemStatus": 0,
      "groupCombinedTestjobState": "completed",
      "groupExecutionLogArray": [
        "2024-02-21 19:51:03.915 |  | INFO | Monitoring Group Execution Thread started",
        "2024-02-21 19:51:32.972 |  | INFO | Monitoring Job Id 60 executed",
        "2024-02-21 19:51:32.972 |  | INFO | Monitoring Job Id 62 executed",
        "2024-02-21 19:51:32.972 |  | INFO | Monitoring Job Id 61 executed",
        "2024-02-21 19:51:32.973 |  | INFO | Monitoring Group Execution Thread terminated"
      ],
      "numberOfPassedSamples": 10,
      "numberOfFailedSamples": 0,
      "numberOfPassedLoops": 6,
      "numberOfFailedLoops": 0,
      "sumErrorsSeverityWarning": 0,
      "sumErrorsSeverityError": 0,
      "sumErrorsSeverityFatal": 0
    }
  },
  "hasGroupExecutionStatistic": true,
  "groupExecutionStatistic": {
    "userId": 13,
    "monitoringGroupId": 34,
    "fromTimestamp": 1708455371812,
    "toTimestamp": 1708541771812,
    "systemAvailabilityPercent": 100,
    "measuredAvailabilityPercent": 100,
    "measuredFaultyTimeSumSeconds": 0
  },
  "isError": false
}

groupSystemStatus:

  • 0: Healthy = All monitoring jobs of the monitoring group have been started on all measuring agents.
  • -1 (minus one): Partial Malfunction = Some monitoring jobs of the monitoring group have not been started on some measuring agents.
  • -2 (minus two): Malfunction = No monitoring jobs of the monitoring group have been started on any measuring agent.

groupCombinedTestjobState

  • “completed”: All test jobs of the monitoring group have been successfully executed on all measuring agents.
  • “partial failed”: The execution of some of the test jobs of the monitoring group did fail on some measuring agents.
  • “failed”: The execution of all of the test jobs of the monitoring group did fail on all measuring agents.

Hint for groupCombinedTestjobState : Note that is value does not reflect if any warnings or errors have been measured by the test jobs.

The Reference Implementation on the Real Load Portal Server shows the measured (sumErrorsSeverityError + sumErrorsSeverityFatal) simple as “number of measured errors”.

Notification for Monitoring Job Execution

  • Received action: “onMonitoringJobExecuted”
  • oldEvent: true or false | Note: the oldEvent is true at the first time (if the monitoring job was executed in the past) and then will always be false each time when the monitoring job is executed.
{
  "action": "onMonitoringJobExecuted",
  "monitoringGroupId": 36,
  "monitoringJobId": 79,
  "monitoringEvent": {
    "eventTimestamp": 1708541771849,
    "oldEvent": true,
    "jobExecutionResult": {
      "magicPattern": "MonitoringJobExecutionResult",
      "productVersion": "4.8.35",
      "userId": 13,
      "monitoringGroupId": 36,
      "groupExecutionId": 39417,
      "monitoringJobId": 79,
      "localTestjobId": 73939,
      "resultTimestamp": 1708541748100,
      "systemStatus": 0,
      "jobExecutionLogArray": [
        "2024-02-21 19:55:31.137 |  | INFO | Monitoring Job Execution Thread started",
        "2024-02-21 19:55:31.495 | Shared Agent 1 | INFO | Job submitted to measuring agent, remote testjob id = 19015",
        "2024-02-21 19:55:38.034 | Shared Agent 1 | INFO | Job executed on measuring agent, exit code = process executed successfully",
        "2024-02-21 19:55:40.014 | Shared Agent 2 | INFO | Job submitted to measuring agent, remote testjob id = 4864",
        "2024-02-21 19:55:46.446 | Shared Agent 2 | INFO | Job executed on measuring agent, exit code = process executed successfully",
        "2024-02-21 19:55:48.100 |  | INFO | Monitoring Job Execution Thread terminated"
      ],
      "combinedTestjobState": "completed",
      "hasCombinedTestResult": true,
      "combinedTestResultFileName": "TestResult_MonitoringJob_79_2024-02-21@19-55-48.json",
      "combinedTestResultSummaryStatistic": {
        "productVersion": "4.8.35",
        "localTestjobId": 73939,
        "remoteTestjobId": 39417,
        "samplingGranularity": 4000,
        "jobStartTimestamp": 1708541736385,
        "jobEndTimestamp": 1708541744994,
        "osCpuLoadPercent": -1,
        "numberOfPassedLoopsTimeSum": 178,
        "numberOfConcurrentUsers": 0,
        "currentlyPendingSamples": 0,
        "numberOfPassedSamples": 2,
        "numberOfFailedSamples": 0,
        "numberOfPassedLoops": 2,
        "numberOfFailedLoops": 0,
        "numberOfMeasuredErrors": 0,
        "numErrorsSeverityWarning": 0,
        "numErrorsSeverityError": 0,
        "numErrorsSeverityFatal": 0
      },
      "averagePassedLoopTime": 89,
      "sumErrorsSeverityWarning": 0,
      "sumErrorsSeverityError": 0,
      "sumErrorsSeverityFatal": 0
    }
  },
  "hasJobExecutionStatistic": true,
  "jobExecutionStatistic": {
    "userId": 13,
    "monitoringGroupId": 36,
    "monitoringJobId": 79,
    "fromTimestamp": 1708455371849,
    "toTimestamp": 1708541771849,
    "systemAvailabilityPercent": 100,
    "measuredAvailabilityPercent": 100,
    "measuredFaultyTimeSumSeconds": 0,
    "averagePassedLoopTime": 93,
    "hasPassedLoopTimePercentileArray": false
  },
  "isError": false
}

systemStatus:

  • 0: Healthy = The monitoring job have been started on all measuring agents.
  • -1 (minus one): Partial Malfunction = The monitoring job was not started on some measuring agents.
  • -2 (minus two): Malfunction = The monitoring job was not started on any measuring agent.

combinedTestjobState

  • “completed”: The testjob was successfully executed on all measuring agents.
  • “partial failed”: The execution of the testjob did fail on some measuring agents.
  • “failed”: The execution of the testjob did fail on all measuring agents.

Hint for combinedTestjobState : Note that is value does not reflect if any warnings or errors have been measured by the test jobs.

The Reference Implementation of the Monitoring Dashboard on the RealLoad Portal Server shows the measured (sumErrorsSeverityError + sumErrorsSeverityFatal) simple as “number of measured errors”.

Notification for Monitoring Group Temporarily Event

  • Received action: “onMonitoringGroupTemporarilyEvent”
  • monitoringGroupTemporarilyAction: “groupExecutionTemporarilyNotDisabled” or “groupExecutionTemporarilyDisabled”
{
  "action": "onMonitoringGroupTemporarilyEvent",
  "monitoringGroupId": 34,
  "monitoringEvent": {
    "eventTimestamp": 1708541771851,
    "monitoringGroup": {
      "monitoringGroupId": 34,
      "ownerUserId": 13,
      "title": "First Group",
      "description": "",
      "maxDataStorageDays": 14,
      "agentType": "agent",
      "executionEnabled": true,
      "executionInterval": 300,
      "executionTimeout": 300,
      "currentExecutionState": "executing",
      "lastExecutionStartTime": 1708541763897
    },
    "monitoringGroupTemporarilyAction": "groupExecutionTemporarilyNotDisabled"
  },
  "isError": false
}

Notification for Monitoring Job Temporarily Event

  • Received action: “onMonitoringJobTemporarilyEvent”
  • monitoringJobTemporarilyAction: “jobExecutionTemporarilyNotDisabled” or “jobExecutionTemporarilyDisabled”
{
  "action": "onMonitoringJobTemporarilyEvent",
  "monitoringGroupId": 36,
  "monitoringJobId": 79,
  "monitoringEvent": {
    "eventTimestamp": 1708541771854,
    "monitoringJobProperties": {
      "performanceErrorAlertThresholdMillis": "-1",
      "enableJobExecution": "true",
      "performanceWarningAlertsEnabled": "false",
      "testjobConcurrentUsers": "1",
      "testjobDelayPerLoop": "1000",
      "testjobDebugMeasuring": "false",
      "performanceErrorAlertsEnabled": "false",
      "enableAutomaticFileSync": "false",
      "performanceWarningAlertThresholdMillis": "-1",
      "testjobMaxTestDuration": "60",
      "testjobDebugExecution": "false",
      "testjobRampUpTime": "5",
      "definedFromMonitoringGroupId": "36",
      "testjobMaxLoopsPerUser": "1",
      "productVersion": "4.8.35",
      "monitoringJobId": "79",
      "monitoringJobTitle": "Local_Cinema",
      "lastExecutionStartTime": "1708541731138",
      "testjobAdditionalArguments": "",
      "definedFromTestId": "399",
      "testjobUserInputFieldsValuesJsonArray": "[]",
      "monitoringJobDescription": ""
    },
    "monitoringJobTemporarilyAction": "jobExecutionTemporarilyNotDisabled"
  },
  "isError": false
}

subscribeToMonitoringAlerts

  • Send action : “subscribeToMonitoringAlerts”
{
  "action":"subscribeToMonitoringAlerts"
}
  • Received action: “onMonitoringAlert”
  • alertLevel: “Monitoring Group”, “Monitoring Job” or “Monitoring Job Measuring Agent”
  • alertType: “New Alert”, “Modified Alert”, “Repeated Alert” or “Canceled Alert”
{
  "action": "onMonitoringAlert",
  "monitoringEvent": {
    "eventTimestamp": 1708547148410,
    "monitoringAlert": {
      "userId": 13,
      "alertLevel": "Monitoring Group",
      "alertType": "New Alert",
      "alertId": 579,
      "alertContext": {
        "alertIdUpdateCounter": 0,
        "currentSystemStatus": "Healthy",
        "currentExecutionState": "Successful",
        "currentNumMeasuredWarnings": 0,
        "currentNumMeasuredErrors": 2,
        "previousSystemStatus": "Healthy",
        "previousExecutionState": "Successful",
        "previousNumMeasuredWarnings": 0,
        "previousNumMeasuredErrors": 0
      },
      "monitoringGroupId": 36,
      "monitoringGroupExecutionResult": {
        "magicPattern": "MonitoringGroupExecutionResult",
        "productVersion": "4.8.35",
        "userId": 13,
        "monitoringGroupId": 36,
        "groupExecutionId": 39525,
        "resultTimestamp": 1708547148390,
        "monitoringJobIdToLocalTestjobIdArray": [
          {
            "monitoringJobId": 79,
            "localTestjobId": 74083
          }
        ],
        "groupSystemStatus": 0,
        "groupCombinedTestjobState": "completed",
        "groupExecutionLogArray": [
          "2024-02-21 21:25:31.365 |  | INFO | Monitoring Group Execution Thread started",
          "2024-02-21 21:25:48.383 |  | INFO | Monitoring Job Id 79 executed",
          "2024-02-21 21:25:48.390 |  | INFO | Monitoring Group Execution Thread terminated"
        ],
        "numberOfPassedSamples": 0,
        "numberOfFailedSamples": 2,
        "numberOfPassedLoops": 0,
        "numberOfFailedLoops": 2,
        "sumErrorsSeverityWarning": 0,
        "sumErrorsSeverityError": 2,
        "sumErrorsSeverityFatal": 0
      }
    }
  },
  "isError": false
}

subscribeToGroupExecutionHistoryCache

  • Send action : “subscribeToGroupExecutionHistoryCache”
{
  "action": "subscribeToGroupExecutionHistoryCache",
  "monitoringGroupId": 36
}

Error Case

{
  "action": "subscribeToGroupExecutionHistoryCache",
  "isError": true,
  "genericErrorText": "subscribeToGroupExecutionHistoryCache failed: java.lang.IllegalArgumentException: Invalid monitoringGroupId : 35"
}

Received actions:

  • onMonitoringGroupExecutionHistoryInitialEntries(history data of the last 24 hours)
  • onMonitoringGroupExecutionHistoryNewEntry(last update of history data)
{
  "action": "onMonitoringGroupExecutionHistoryInitialEntries",
  "monitoringGroupId": 36,
  "monitoringEvent": {
    "eventTimestamp": 1708548777246,
    "historyEntriesArray": [
      {
        "t": 1708463444623,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "w": 0,
        "e": 0,
        "f": 0,
        "gid": 37851
      },
      {
        "t": 1708463504557,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "w": 0,
        "e": 0,
        "f": 0,
        "gid": 37853
      },
      {
        "t": 1708463563635,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "w": 0,
        "e": 0,
        "f": 0,
        "gid": 37854
      },
      {
        "t": 1708548588515,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "w": 0,
        "e": 2,
        "f": 0,
        "gid": 39554
      },
      {
        "t": 1708548648466,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "w": 0,
        "e": 2,
        "f": 0,
        "gid": 39555
      },
      {
        "t": 1708548708464,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "w": 0,
        "e": 2,
        "f": 0,
        "gid": 39557
      },
      {
        "t": 1708548768449,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "w": 0,
        "e": 2,
        "f": 0,
        "gid": 39558
      }
    ]
  },
  "isError": false
}
{
  "action": "onMonitoringGroupExecutionHistoryNewEntry",
  "monitoringGroupId": 36,
  "monitoringEvent": {
    "eventTimestamp": 1708549188525,
    "historyEntry": {
      "t": 1708549188468,
      "ge": 60,
      "ss": 0,
      "ct": "completed",
      "w": 0,
      "e": 2,
      "f": 0,
      "gid": 39566
    }
  },
  "isError": false
}
  • t : Unix-like timestamp
  • ge : Monitoring group execution interval (seconds)
  • ss : Group system status: 0 = HEALTHY, -1 = PARTIAL MALFUNCTION, -2 = MALFUNCTION
  • ct : Group combined test job state: “completed”, “partial failed” or “failed”
  • w : Sum of errors with severity warning
  • e : Sum of errors with severity error
  • f : Sum of errors with severity fatal
  • gid : Group execution ID

subscribeToJobExecutionHistoryCache

  • Send action : “subscribeToJobExecutionHistoryCache”
{
  "action": "subscribeToJobExecutionHistoryCache",
  "monitoringJobId": 79
}

Error Case

{
  "action": "subscribeToJobExecutionHistoryCache",
  "isError": true,
  "genericErrorText": "subscribeToJobExecutionHistoryCache failed: java.lang.IllegalArgumentException: Invalid monitoringJobId : 80"
}

Received actions:

  • onMonitoringJobExecutionHistoryInitialEntries(history data of the last 24 hours)
  • onMonitoringJobExecutionHistoryNewEntry(last update of history data)
{
  "action": "onMonitoringJobExecutionHistoryInitialEntries",
  "monitoringGroupId": 36,
  "monitoringJobId": 79,
  "monitoringEvent": {
    "eventTimestamp": 1708551216713,
    "historyEntriesArray": [
      {
        "t": 1708464823243,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "al": 118,
        "w": 0,
        "e": 0,
        "f": 0,
        "lid": 71889,
        "h": true
      },
      {
        "t": 1708464882694,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "al": 107,
        "w": 0,
        "e": 0,
        "f": 0,
        "lid": 71890,
        "h": true
      },
      {
        "t": 1708464943399,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "al": 96,
        "w": 0,
        "e": 0,
        "f": 0,
        "lid": 71891,
        "h": true
      },
      {
        "t": 1708550987656,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "al": -1,
        "w": 0,
        "e": 2,
        "f": 0,
        "lid": 74186,
        "h": true
      },
      {
        "t": 1708551047722,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "al": -1,
        "w": 0,
        "e": 2,
        "f": 0,
        "lid": 74187,
        "h": true
      },
      {
        "t": 1708551108599,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "al": -1,
        "w": 0,
        "e": 2,
        "f": 0,
        "lid": 74191,
        "h": true
      },
      {
        "t": 1708551167586,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "al": -1,
        "w": 0,
        "e": 2,
        "f": 0,
        "lid": 74192,
        "h": true
      }
    ]
  },
  "isError": false
}
{
  "action": "onMonitoringJobExecutionHistoryNewEntry",
  "monitoringGroupId": 36,
  "monitoringJobId": 79,
  "monitoringEvent": {
    "eventTimestamp": 1708551228189,
    "historyEntry": {
      "t": 1708551228183,
      "ge": 60,
      "ss": 0,
      "ct": "completed",
      "al": -1,
      "w": 0,
      "e": 2,
      "f": 0,
      "lid": 74193,
      "h": true
    }
  },
  "isError": false
}
  • t : Unix-like timestamp
  • ge : Monitoring group execution interval (seconds)
  • ss : System status: 0 = HEALTHY, -1 = PARTIAL MALFUNCTION, -2 = MALFUNCTION
  • ct : Combined test job state: “completed”, “partial failed” or “failed”
  • al : Average passed loop time (milliseconds), -1 = no data
  • w : Sum of errors with severity warning
  • e : Sum of errors with severity error
  • f : Sum of errors with severity fatal
  • lid : Local test job ID
  • h: Has combined test result: true or false

Other Received Actions

You will also receive other actions than previously described above. These are confirmations that you can ignore. Example:

{
    "action":"subscribeToRealtimeMonitor",
    "isError":false
}

6 - User Guide

How to use RealLoad

Thank you for using the RealLoad product.

This User Guide gives you an overview of how to use RealLoad and also contains numerous links to relevant details.

The order of the chapters in this guide corresponds to the order of the menus in the Main Navigation Bar.

“alt attribute”

Projects Menu

“alt attribute”

After you have signed in to the RealLoad Portal, you will see a Main Navigation Bar whose first menu on the left is Projects. This is a file browser that displays all of your files. The view is divided into Projects and Resource Sets which contain all the files for executing your tests and the test results of the executed load tests. You can also store additional files in a Resource Set which contain e.g. instructions on how a test should be performed.

Among other things, you can:

  • Create, clone and delete projects.
  • Create, clone and delete resource sets (subdirectories of projects).
  • Searching for files by file name and by content.
  • Upload and download files, create new files, display the content of files, edit and delete files.

There is also a recycle bin from which you can restore deleted projects, resource sets and files.

In the Developer Tools dropdown menu, there is a wizard that supports converting a Selenium IDE Test to a RealLoad Test, and an example of developing a JUnit Test from scratch.

Tests Menu

The Test Menu shows all tests that you have defined. You can also filter the view according to Projects and Resource Sets, and sort the tests in different ways.

Tests can defined:

  • By the HTTP Test Wizard (Web Surfing Sessions and HTTP API Tests)
  • From exported Selenium IDE Tests (Execution of real Web Browser Tests)
  • From imported or self-developed JUnit Tests (Testing any type of Network Protocol such as DNS, SMTP or UDP)

…and can be executed:

  • As Load Test Job
  • As Regression Test (Part of a Test Suite)
  • As Synthetic Monitoring Job

“alt attribute”

Note that a test is something like a bracket that only contains references to the files that are required for the test execution. There a no files stored inside the test itself.

Each test has a base location from which it was defined (Project + Resource Set) and to where also the load test results are saved.

A test itself can reference its resource files in two ways:

  • Resource Files that reside in the base location of the test (the test core files).
  • Referenced Files that are located in other Projects / Resource Sets. These are typically libraries, plug-ins and input files which are used/shared by several, different tests.

Defining a Load Test Job

After you have clicked on a test on “Define Test Job”, the intermediate “Define Test Job” menu is displayed. “alt attribute”

“alt attribute”

In contrast to a “test”, all referenced files of the test are copied into the “load test job”. For this reason there is also the option “Synchronize Files at Start of Job” which you should always have switched on. At “Additional Job Arguments” you can enter test-specific command line arguments that are transferred directly to the test program or test script. However, you usually do not need to enter anything.

After you have clicked on “Define Load Test Job” the test job is created and the view changes from the “Tests” menu to the Load Test Jobs menu.

Monitoring Menu

This menu is documented at Synthetic Monitoring.

Load Test Jobs Menu

In this menu all load test jobs are displayed with their states. The green point at the top right next to the title indicates that all measurement agents and cluster controllers which are set to ‘active’ can also be reached. The color of the point changes to yellow or red if one or more ‘active’ measuring agents and cluster controllers cannot be reached / are not available.

“alt attribute”

A load test job can have one of the following state:

  • invalid: The job is damaged and can only be deleted.
  • defined: The job is defined locally in the portal server, but has not yet been transmitted to a measuring agent or cluster controller. Jobs that are in this state can still be modified.
  • submitted: The job was transmitted to a measuring agent or cluster controller, but is not yet ready to be started.
  • ready to run: The job on the measuring agent or cluster controller is ready to start.
  • start failed: The start of the job has failed.
  • running: The job is currently running and can be monitored at real time.
  • completed: A job that was previously running is now completed (done).

As soon as a job is in the “defined” state, it has a “local job Id”. If the job is then submitted to a measuring agent or cluster controller, the job has additionally a “remote job Id”.

From this menu you can also create Test Job Templates and edit Test Suites - which can be executed as Regression Tests.

“alt attribute”

“alt attribute”

Starting a Load Test Job

After you have clicked on “Start Test Job” in a load test job you can select/modify the Measuring Agent or Cluster of Measuring Agents on which the job will be executed, and you can configure the job settings.

“alt attribute”

Input Fields:

  • Measuring Agent | Cluster: The Measuring Agent or Cluster of Measuring Agents on which the load test is executed.
  • Number of Users: The number of simulated users that are started.
  • Max. Test Duration: The maximum test duration.
  • Max. Loops per User: The maximum number of sessions executed per user (the number of iterations of the test per user). Each time a new session of a user is started, the user’s session context is reset.
  • Loop Iteration Delay: The delay time after a session of a user has ended until the next session of the user is started.
  • Ramp Up Time: The length of time at the beginning of the test until all simulated users are started. Example: with 20 simulated users and a time of 10 seconds, a new user is started each 0.5 seconds.
  • Additional Arguments: Additional values which are transferred on the command line when the test script is started. These arguments are test specific. For tests that were created with the “HTTP Test Wizard” you can specify for example the following values: “-tcpTimeout 8000 -sslTimeout 5000 -httpTimeout 60000” (TCP connect timeout / SSL handshake timeout / HTTP processing timeout) which are considered by the executed URL calls and override the default values.
  • Debug Execution: This option effects that detailed information are written to the log file of the test. For example variable values which have been extracted from input files or from HTTP responses as well as variable values which are assigned at runtime. Only activate this option if you have problems with the execution of the test.
  • Debug Measuring: Effects that the Data Collector of the Measuring Agent writes the JSON objects of the All Purpose Interface to its log file. This option can be enabled to debug self-developed tests that have been written from scratch.
  • (User Input Fields): If the test contains User Input Fields you can enter their values here.

Normally you do not have to enter any “Additional Arguments” and leave “Debug Execution” and “Debug Measuring” switched off.

After clicking Start Test Job, the job is started on the Measuring Agent or Custer of Measuring Agents and is then in the state “Running”. Then click on “Monitor Jobs” and the view changes from the “Load Test Jobs” menu to the Real Time View menu.

“alt attribute”

Real Time View Menu

The real time view shows all currently running load test jobs including their measured values and errors.

“alt attribute”

You can also suspend a running job for a while and resume it later. However this has no effect on the “Max. Test Duration”.

“alt attribute”

After the job is completed you can click on “Analyze Result”. The view changes then to the Test Results menu.

“alt attribute”

Test Results Menu

The “Test Results” menu is a workspace into which you can load multiple test results from any type of test. You can also switch back and forth between the test results. All measured values and all measured errors are displayed. In addition, Percentile Statistics and Diagrams of Error Types are also displayed in this menu.

This menu enables you also to combine several test results into a so-called Load Curve - to determine the maximum number of users that a system such a Web server can handle (see next chapter).

“alt attribute”

The Summary Statistic of a test result contains some interesting values:

  • ∅ Measured Samples per Second: These are the number of successful measured values per second, counted over the whole test (in other products also so-called as “hits per second”).
  • Max. Concurrent Users: These are the maximum number of concurrent user that have been really reached during the test (which may differ from the “Number of Users” defined when starting the test).
  • Max. Pending Samples: This is the maximum amount of requests to the server, for which no immediately response has been received, measured over the whole test (the maximum traffic jam of the requests).
  • ∅ CPU Load on Measuring Agent: This is the average CPU load in percent on the measuring agent (load generator) or on the members of a measuring agent cluster - which was captured during the execution of the test. If this value is greater than 95% the test is invalid because the measuring agent was overloaded.

Since several thousand to several million response times can be measured in a very short time during a test, the successfully measured response times are summarized in the response time diagrams at 4-second intervals. For this reason, a minimum value, a average value and a maximum value is displayed for each 4-second interval in such diagrams.

However, this summarization is not performed for Percentile Statistics and for Measured Errors. Every single measured value is taken into account here.

Determining System Capacity - Load Curve

The maximum capacity of a system, such as the maximum number of users that a web server can handle, can be determined by a so-called Load Curve.

To obtain such a Load Curve, you must repeat the same load test several times, by increasing the number of users with each test. For example a test series with 10, 50, 100, 200, 400, 800, 1200 and 1600 users.

The easiest way to repeat a test is to Clone a Load Test Job. You can then enter a higher number of users when starting the cloned load test job.

“alt attribute”

“alt attribute”

A measured Load Curve looks like this, for example:

“alt attribute”

As you can see, the throughput of the server increases linearly up to 400 users - with the response times remaining more or less the same (Avg. Passed Session Time). Then, with 800, 1200 and 1600 users, only individual errors are measured at first, then also many errors, with the response times now increasing sharply.

This means that the server can serve up to 400 users without any problems.
But could you operate the server with 800 users if you accept longer response times? With 800 users, 745,306 URL calls were successfully measured, with only 50 errors occurring. To find it out, let’s compare the detailed response times of “Page 1” of 400 users with 800 users.

Response Times of “Page 1” at 400 Users: “alt attribute”

Response Times of “Page 1” at 800 Users: “alt attribute”

The 95% percentile value at 400 users is 224 milliseconds and increases to 1952 milliseconds at 800 users. Now you could say that it just takes longer. However, if you look at the red curve of the outliers, these are only one time a little bit more than 1 second at 400 users, but often more than 8 seconds at 800 users. Conclusion: The server cannot be operated with 800 users because it is then overloaded.

Now let’s do one last test with 600 users. Result:

“alt attribute”

The throughput of the server at 600 users is a little bit higher than at 400 users and also little bit higher than at 800 users. No errors were measured.

Response Times of “Page 1” at 600 Users: “alt attribute”

The 95% percentile value at 600 users is 650 milliseconds, and there are only two outliers with a little bit more than one second.

Final Conclusion: The server can serve up to 600 Users, but no more.

Troubleshooting a Load Test

In rare cases it can happen that a test does not measure anything (neither measurement results nor errors). In this case you should either wait until the test is finished or stop it directly in the “Real Time View” menu.

Then you can then acquire the test log files in the “Load Test Jobs” menu and search for errors.

“alt attribute”

“alt attribute”

Network Components Menu

“alt attribute”

The Network Components menu allows you to manage Measuring Agents, Cluster Controllers and HTTP/S Remote Proxy Recorders. In addition, you can also start additional components that you only need for a short time as AWS Cloud Instances by spending RealLoad Cloud Credits.

Measuring Agents & Cluster Controllers

On the one hand, depending on your license (Price Plan), you can add and manage Measuring Agents and Cluster Controllers that you operate yourself. On the other hand, you can also launch additional Measuring Agents and Cluster Controllers using RealLoad Cloud Credits.

How to launch and operate Measuring Agents and Cluster Controllers using your own AWS account is documented at AWS Measuring Agents.

“alt attribute”

HTTP/S Remote Proxy Recorders

“alt attribute”

HTTP/S Remote Proxy Recorders can be used to record Web Surfing Sessions, which can then be post-processed with the HTTP Test Wizard and converted into a RealLoad Test.

The most important points here are:

  • You must import the CA Root Certificate of the Proxy Recorder into the web browser with which you record the web surfing session (Firefox recommended).
  • To control the Proxy Recorder in the RealLoad Portal, you need a second web browser from another manufacturer (Goggle Chrome recommended), or alternatively installation of the RealLoad Browser Extension.
  • Optional: When recording, HTTPS/X509 client certificates are also supported to authenticate yourself to the corresponding web server(s) - if required. You can upload the client certificates directly in this menu. The client certificates are then automatically transferred to the proxy recorder and applied during recording.

The complete documentation is available at HTTP/S Remote Proxy Recorder.

Launched Cloud Instances (by Cloud Credits)

This menu shows (only) all AWS instances that were launched using RealLoad Cloud Credits. Here you can also terminate such instances early - whereby Cloud Credits for complete unused hours are refunded.

“alt attribute”

Wizards Menu

This menu only displays the wizards that can be accessed via direct navigation. In addition, the RealLoad product contains numerous other wizards that are accessible depending on the context.

“alt attribute”

HTTP Test Wizard

The functionalities of the HTTP Test Wizard (+ Plug-Ins and Published Plug-Ins) is documented at HTTP Test Wizard.

User Input Fields Wizard

“User Input Fields” are additional values that are entered when starting or defining a test execution and that are available as (global) variables during test execution.

With this wizard you can create or edit JSON files containing definitions of User Input Fields. By convention, the names of such files must always start with ‘InputFields_’ and end with ‘.json’.

A User Input Field contains the following attributes:

  • GUI Label : The label of the User Input Field that is displayed in the user interface.
  • Variable Name : The name of the global variable at test execution.
  • Input Type : The datatype of the input value (String, Integer, Real or Boolean). Please note that the Input Type is only used to validate the input. During test execution, the value of the variable is always converted to a string.
  • Default Value : The default value of the variable.

“alt attribute”

User Input Fields are supported by All Types of RealLoad Tests.

Tools Menu

“alt attribute”

This menu contains tools that are helpful for developing and debugging tests:

  • Base64 Tool : Converts plain text to Base64 format or vice versa.
  • DNS Tool : This allows you to resolve DNS hostnames through one or more specific DNS servers.
  • HTTPS Server Certificate Tool : With this tool you can check the validity of HTTPS server certificates and also create your own (client) Trust Stores.
  • JSON Editor : This allows you to create and edit JSON files, as well as display JSON data in a formatted view. Tip: You can also copy raw JSON data into the JSON Editor and then view it as formatted JSON data with a mouse click - without having to save the data to a file.
  • PKCS12 Client Certificate Tool : With this tool you can view the contents of an SSL/X509 client certificate which is in PKCS12 format.
  • Query String Tool : With this tool you can convert URL Query Strings to the unescaped format or vice versa. See Wikipedia - Query String
  • Text Phrase Tool: Determines the most meaningful human-readable text phrases or keywords from any given URL with HTML/JSON/XML response content. This tool’s algorithm is also used by the HTTP Test Wizard to verify the content of HTTP responses.
  • Unix Time Tool : Converts a Unix time to a formatted local time - or vice versa.
  • WebSocket Tool : This allows you to test a WebSocket connection to a web server. The tool also supports SSL/X509 client certificates in PKCS12 format.
  • XML Editor : This allows you to create and edit XML files.

User Account Menu

“alt attribute”

Submenus:

  • Sign Out : This will sign out you out without asking. Any tests that may be running are not affected by this and will continue.
  • To Welcome Page : Displays the welcome page analogously after sign up.
  • Personal Data & Profile Settings : Here you can edit your personal data and also determine whether parts of it are publicly displayed in the portal.
  • Team Members : Displays the list of all team members.
  • Team Members Accounts : Management of team member accounts by the main user account. Team members have no access to this submenu.
  • Mobile Companion : Displays a list of all mobile devices that have logged in to the portal using the Mobile Companion App. Furthermore, all push notifications that were sent to the mobile devices are displayed.
  • API Authentication Tokens : Here you can create and manage authentication tokens that are used for authentication against the portal server API.
  • Price Plan : Displays your current price plan and your access permissions to RealLoad resources.
  • Manage Licenses : Displays your purchased licenses and allows you to add additional licenses and cloud credits.
  • Cloud Credit Statement : Displays your Cloud Credit balance and a list of all Cloud Credit transactions.
  • Set New Password : Here you can set a new password for your portal account.
  • Delete Account : This will permanently delete your account in the portal - including deleting all data associated with your account.

The following submenus are only accessible depending on your configured “Personal Data & Profile Settings”:

  • Public In-App Users : Displays a list of all portal users which are visible to other portal users. You can also contact them within the portal. This list is only shown if you are also a public portal user yourself.
  • In-App Message Box : Here you can sent and display received messages to/from all portal users which are visible to other portal users.
  • Chat with Signed-In Users : Opens the chat popup that allows chatting with other in the portal signed in users who have enabled this functionality in their profile settings.
  • Received Contact Messages : If you have enabled “Publish My Profile as Technical Expert:” in your profile settings then all received contact messages will be displayed here.

Downloading RealLoad Components and RealLoad Libraries Java Doc

“alt attribute”

Clicking this icon will open a new browser window or tab to download.realload.com

Purchase of Licenses and Cloud Credits

“alt attribute”

Clicking this icon will open a new browser window or tab to shop.realload.com

Here you can Purchase Licenses and additional Cloud Credits.

If you have any questions, please email us at support@realload.com

6.1 - AWS Measuring Agents

How to launch AWS based Measuring Agents

This document describes how to launch Measuring Agents in AWS as EC2 instances. Readers are assumed to be familiar with AWS EC2 terms used in here, in particular if launching Measuring Agents manually.

AWS based Measuring Agents can be launched in three ways:

  1. Via the RealLoad Portal
  2. Via our Desktop Companion App
  3. Via the AWS Console

1. Launching AWS Measuring Agents via the RealLoad Portal

Launching AWS Measuring Agents via the RealLoad portal is the easiest of the three options. You don’t need an AWS account for this, but RealLoad ‘Clouds Credits’ are required. If you run out of Cloud Credits, you can purchase new ones in the RealLoad Shop: https://shop.realload.com/

Launching an AWS Measuring Agent using Cloud Credits

Navigate in the Portal to ‘Measuring Agents & Cluster Controllers’.

Navigate to ‘Measuring Agents & Cluster Controllers’

Them click the AWS icon:

click the AWS icon

Click on the rocket icon of an AWS region:

Click the rocket icon of an AWS region

Select the ‘AWS/EC2 Instance Type’ (in this example ’t3.xlarge’) and the number of hours until the instance will be terminated (in this example 3 hours). Then click on the ‘Launch Instance’ button:

Select Instance Type and number of hours

You will then receive a confirmation notification that the instance has been started. Now you have to wait at least 40 seconds until the instance gets into the running state.

Launch instance confirmation notification

The Measuring Agent is now registered in the portal. You should then ‘Ping’ it to make sure it is functional. If the Ping fails, wait another minute and then try again.

Measuring Agent registered in the portal

Ping Measuring Agent

Premature Termination of an Instance - Refund of Cloud Credits

In this example, the Measuring Agent (cloud instance) was started for three hours, but the load test was performed successfully after just one and a half hours. In order to save Cloud Credits, the instance can be terminated early.

To terminate an instance launched by Cloud Credits navigate to ‘Launched Cloud Instances’:

Navigate to ‘Launched Cloud Instances’

At the relevant instance, click the ‘Terminate Instance’ icon:

Click ‘Terminate Instance’ icon

Click the ‘Terminate Instance’ button:

Click ‘Terminate Instance’ button

The instance will now terminate. In this example you will receive 4 out of a total of 12 cloud credits back.

Refund of Cloud Credits

Verifying the Cloud Credit Billing

Navigate to ‘Cloud Credit Statement’ to check your Cloud Credit Balance and Transactions:

Refund of Cloud Credits

Refund of Cloud Credits

2. Launching AWS Measuring Agents via the Desktop Companion App

Launching AWS Measuring Agents via the Desktop Companion App requires that you have an AWS account.

No additional costs (beyond the costs charged by AWS) apply when using our Measuring Agent images.

Note that you need the rights to log in to the AWS Console to check whether all Measuring Agent instances that are no longer needed have been terminated.

First download, install and configure the Desktop Companion. The download links are published on https://download.realload.com

Configuring the Desktop Companion

To configure the Desktop Companion for launching AWS Measuring Agents you need:

  1. An AWS ‘Access Key’ of an AWS User which have the permission AmazonEC2FullAccess
  2. A RealLoad ‘API Authentication Token’

To generate a RealLoad ‘API Authentication Token’ sign in to the RealLoad Portal and navigate to ‘My Account’ - ‘API Authentication Tokens’:

Navigate to ‘My Account’ - ‘API Authentication Tokens’

Click the ‘Add API Authentication Token’ button:

Click ‘Add API Authentication Token’

Enter as purpose ‘Desktop Companion’ and click the ‘Add API Authentication Token’ button. Then copy the shown Token Value to the clipboard or to a text editor:

Copy the shown Token Value

Now start the Desktop Companion and configure it. Click on the File tab and then click Settings.

Start and configure Desktop Companion

In the General Tab enter:

  1. The Agent Secret: This is something like a password that protects the launched AWS Measuring Agents from unauthorized access. You can enter any text here, but it must not be empty.
  2. The Portal URL: This is the URL of the Portal Server API and should always be https://portal.realload.com/RemoteUserAPI
  3. The Authentication Token: Enter here the RealLoad ‘API Authentication Token’.
  4. Set the Refresh Interval to a value that is greater than 60 seconds.

Then click the ‘Test’ button near ‘Authentication Token’:

Configure General Settings

Switch then to the AWS Tab and enter:

  1. The AWS Access Key ID
  2. The AWS Secret Access Key
  3. The Preferred Instance Type: We recommend that you enter t3.medium for your first attempt.
  4. Set the AWS EC2 Refresh Interval to a value that is greater than 60 seconds.
  5. Select at My AWS Regions your preferred AWS region(s): Multiple regions can be selected by CTRL-mouse-click. We recommend that you select eu-central-1 and/or us-west-1 for your first attempt.

Then click the ‘Test’ button near ‘AWS Secret Access Key’:

Configure AWS Settings

Finally, click the ‘OK’ button. The configuration is now complete.

Launching an AWS Measuring Agent

In the Desktop Companion, go to the AWS tab and select an AMI of type ‘MA’ (MA = Measuring Agent). Then launch the instance with a right mouse click:

Launch Instance

A window will appear in which you can check the configuration of the instance and set the instance auto-terminate time period. You can also choose a value of zero for the auto-terminate time period, which means that the instance will continue to run until you stop it.

Confirm Launch Instance

The instance is then launched and in the state ‘pending’:

Instance in ‘pending’ state

Now wait until the instance reaches the ‘running’ state. Then register the instance in the portal with a right mouse click:

Register ‘running’ instance

The instance is now also displayed in ‘Portal Agents Registration’:

Portal Agents Registration

Check in the RealLoad Portal whether the instance is running correctly by pinging the Measuring Agent at application level:

Portal: Navigate to Measuring Agents

Portal: Ping Measuring Agent

Portal: Ping Measuring Agent - Result

Terminating an AWS Measuring Agent

Note that this section only applies to Measuring Agent instance which have launched via the Desktop Companion.

Two steps are necessary to terminate an instance:

  1. Terminate the instance in AWS
  2. Deregister the Measuring Agent in the portal server

To terminate an instance in AWS, perform a right mouse click in the Desktop Companion - Running AWS Images in “My Regions” at the corresponding instance and select ‘Terminate’:

Terminate AWS Instance

The instance is then in the state ‘shutting-down’. Now you can deregister the Measuring Agent in the portal server by a right mouse click:

Deregister Measuring Agent

Double-check no longer needed Instances

Because the Desktop Companion only shows Running AWS Images in “My Regions”, some instances launched in other AWS Regions may not be displayed.

Sign in to the AWS console to review Measuring Agent instances across all AWS regions. Select EC2 Global View:

AWS Console: EC2 Global View

In EC2 Global View click Global search and select the the tag REAL_LOAD_AGENT:

AWS Console: Global search

Select the operator Equals and set the value to true, then hit the return key:

AWS Console: Global search

If the list displayed contains one or more instances, click on the instance(s):

AWS Console: Global search

Memorize the instance ID and terminate the instance:

AWS Console: Terminate Instance

Sign in into the RealLoad Portal and delete the corresponding Measuring Agent:

RealLoad Portal: Delete Measuring Agent

3. Launching AWS Measuring Agents via the AWS Console

In EC2, first select the AWS Region where you want to launch the Measurement Agent instance.

Creating the Security Group

In order for the AMI to be reachable from the RealLoad Portal you’ll need to configure a Security Group allowing inbound TCP/IP V4 connections to port 8080 as a minimum. Optionally, you can also configure SSH access.

In the following example the security group “Real Load Agent Security Group” was created:

AWS Console: Security Group

Launching a Measuring Agent Instance

When launching the instance click Browse more AMIs:

AWS Console: Browse more AMIs

Then click Community AMIs, enter our (RealLoad) AWS account 775953462348 and select the Measuring Agent image:

AWS Console: Community AMIs - Measuring Agent

Add the tag REAL_LOAD_AGENT = true

AWS Console: Launch Instance - Add Tag

Select the Instance Type and set the (existing) Security Group “Real Load Agent Security Group”. Optionally you may also set a Key Pair:

AWS Console: Launch Instance - Instance Type and Security Group

Scroll down and click Advanced Details:

AWS Console: Launch Instance - Advanced Details

In Advanced Details scroll down to User Data. In order to protect access to your Measuring Agent we strongly recommend setting a non-default secret when launching the instance.

The secret can be set at launch time by providing User Data to the AMI as follows (replace “secret” with the secret of your choice):

#!/bin/sh
echo "AGENT_SECRET=secret" > /home/ec2-user/agent_secret.sh

AWS Console: Launch Instance - Agent Secret

Scroll down to the bottom of the AWS Console and click Launch Instance:

AWS Console: Click Launch Instance

After launching the instance click on the instance ID:

AWS Console: Click on Instance ID

Copy the Public IPv4 address of the instance to a text editor or to the clipboard:

AWS Console: Copy Public IPv4 address

Sign In to the RealLoad Portal and add (=register) the Measuring Agent instance:

RealLoad Portal: Add Measuring Agent

The Agent IP Port wil always be 8080. Enter the Auth Token which you have set as AGENT_SECRET when launching the instance.

RealLoad Portal: Add Measuring Agent

Then ping the Measuring Agent at application level:

RealLoad Portal: Ping Measuring Agent

RealLoad Portal: Ping Measuring Agent

And check the Measuring Agent Log File to verify that the Agent Secret was configured:

RealLoad Portal: Measuring Agent Log File

RealLoad Portal: Measuring Agent Log File

The Measuring Agent is now ready for operation. Don’t forget to terminate the instance when you no longer need it.

Once you have terminated the instance, you should delete the corresponding Measuring Agent in the RealLoad Portal.

6.2 - Recording of a Web Surfing Session

Recording and Post-Processing of a Web Surfing Session

This document describes how a Web Surfing Session can be recorded by using a HTTP/S Remote Proxy Recorder and then post-processed in the HTTP Test Wizard. After post-processing, a RealLoad Test can be generated.

Preconditions

The following assumes that a Firefox web browser is used for recording and that the CA Root Certificate of the HTTP/S Remote Proxy Recorder has already been imported into Firefox.

You must also have the ‘RealLoad Test Recorder’ browser extension installed in Firefox. It is recommended to only turn on this browser extension when recording a Web Surfing Session.

Preparing Firefox for Recording

The following text describes how to copy and paste a HTTP/S Remote Proxy Recorder Configuration into the Firefox ‘RealLoad Test Recorder’ browser extension.

You now need two web browsers from different manufacturers:

  • The Firefox browser
  • A Chrome browser that is signed-in into the RealLoad portal

In Firefox navigate to Manage Extensions

“alt attribute”

At the RealLoad Test Recorder extension click Options

“alt attribute”

“alt attribute”

Now use the Chrome browser, sign in to the RealLoad portal, navigate to the HTTP/S Remote Proxy Recorders menu, and Connect to the Proxy Recorder

“alt attribute”

After you are connected to Proxy Recorder with the Chrome browser, click on the ‘Red Eye Icon’ and copy the ‘Proxy Configuration Data’ to the Clipboard by clicking the corresponding icon

“alt attribute”

Then paste the ‘Proxy Configuration Data’ to the Firefox ‘RealLoad Test Recorder’ browser extension and click ‘Save’

“alt attribute”

Firefox is now ready for recording.

Tip: You can also Pin the ‘RealLoad Test Recorder’ browser extension to the Firefox Toolbar:

“alt attribute”

“alt attribute”

Recording a Web Surfing Session

The following example shows the recording of the purchase of a Cinema Voucher in the RealLoad Cinema Demo Shop.

cinema.realload.com is a public server available to you for educational purposes.

In the Chrome browser connect via portal to the Proxy Recorder and click on the ‘Red Eye Icon’:

“alt attribute”

In Firefox click the ‘RealLoad Test Recorder’ browser extension. When you are asked for the Proxy Authentication then paste the username and password shown in Chrome.

“alt attribute”

In the ‘RealLoad Test Recorder’ browser extension first enable Clear Cache & Cookies and then click Start Recording

“alt attribute”

In Firefox enter the URL https://cinema.realload.com/shop which is the starting point of the recorded session.

“alt attribute”

Optional: Have a look at Chrome and you will see that the first web page is recorded:

“alt attribute”

A cinema voucher will now be added to the shopping cart. In Firefox first insert a ‘Page Break’ with the comment ‘Add Cinema Voucher’, and then click on ‘Add to Chart’ in the shop:

“alt attribute”

Add another Page Break with the comment ‘Goto Chart’, and then click the Cart Icon:

“alt attribute”

“alt attribute”

The shopping cart is now displayed. Add another Page Break with the comment ‘Checkout and Pay’, and then click ‘Checkout and Pay’:

“alt attribute”

The receipt will be displayed and the recording is now complete.

In the ‘RealLoad Test Recorder’ browser extension, click Stop Recording:

“alt attribute”

Now return to the Chrome browser and click on ‘Refresh’. As you can see, a lot of URLs were recorded that were unwanted in your test. Click on URL Filter - Quick Settings to filter out the unwanted URLs:

“alt attribute”

Exclude any web servers that you do not want to be part of your test. In particular, you should exclude any tracking servers.

“alt attribute”

Then save the recorded session. You can later re-load the recorded session to (any) proxy recorder and filter it again:

“alt attribute”

After completing the recording, it is strongly recommended to switch off the ‘RealLoad Test Recorder’ browser extension. Only turn this browser extension on again when you record a new web surfing session.

“alt attribute”

Converting the Recording to an HTTP Test Wizard Session

In the HTTP/S Remote Proxy Recorder click ‘Convert’

“alt attribute”

Enter the Headline of the HTTP Test Wizard session (in this case ‘Purchase Cinema Voucher’) and click ‘Convert’:

“alt attribute”

The recording will now be converted and appears in the HTTP Test Wizard. We recommend that you save the HTTP Test Wizard session first before you start post-processing.

“alt attribute”

“alt attribute”

Post-Processing of the HTTP Test Wizard Session

Handling of Dynamic Session Values

If a dynamic value such as a order number is part of an HTTP response, and that value is then used in subsequent HTTP requests, you must handle this value using a dynamic session variable.

To identify such dynamic values, you can use the Variables Wizard, which you can access via the HTTP Test Wizard. First click on URL Explorer

“alt attribute”

Then in the URL Explorer click Variables Wizard

“alt attribute”

Now a list of Distinct Transmitted Values is shown in the Variables Wizard, which is extracted from all HTTP request of the session. The challenge now is to estimate which of these values are dynamic (static values do not need to be handled).

As a rule of thumb:

  • if the value is a simple speaking word, or a combination of simple speaking words, or a small number, it is probably a static value.
  • However, if the value is a complex, random string, it is probably a dynamic value.

In the image below there is only one dynamic value (receiptId = e1bjlSo9CoJLR9TL). Click on the magnifying glass icon to handle this value:

“alt attribute”

The session is now searched for this dynamic value e1bjlSo9CoJLR9TL. Click on the first search result of an HTTP response:

“alt attribute”

In this example, the HTTP response is in JSON data format. So click on the JSON tab:

“alt attribute”

Then click on the value of receiptId, and then click Test Extractor and enter a new Variable Name ‘(vReceiptId’ in this case). Finally enable the switch Assign Extracted Value as Variable to All Subsequent HTTP Requests and click the Define Variable Extractor button.

“alt attribute”

The handling of the dynamic session variable is now complete. Click the ‘Variables’ icon to check the variable extractor and variable assigners:

“alt attribute”

After you have processed all dynamic values, you should save the session again.

Reviewing the HTTP Response Content Validations

When the recording was converted, the verification of the HTTP responses were also automatically configured.

The received HTTP status code of each URL is always checked for the value it had at the time of recording. However, some URLs may also be automatically configured to additionally validate the response content against a text fragment. You should review these URL-specific text fragments and correct them if necessary.

In the HTTP Test Wizard click on Review and Edit URL Settings

“alt attribute”

Take a look at table column ‘V. Text’ and click on the text fragment wizard icons:

“alt attribute”

For example, for the URL https://cinema.realload.com/shop, it is better to choose the text fragment ‘RealLoad Cinema Demo Shop’ instead of ‘clone the product card template’:

“alt attribute”

After you have reviewed the content validations of the HTTP responses, you should save the session again - if you have made any changes.

Debugging an HTTP Test Wizard Session

After you have completed the post-processing, we recommend that you debug the HTTP Test Wizard session.

“alt attribute”

Since this is a Remote Debugger, you can choose which Measuring Agent the debugger runs on. This means you can also debug the session from different geographical locations.

“alt attribute”

Click on ‘Next Step’ until you reach the end of the session - or until an error is displayed. During debugging you will also see when the value of dynamic session variable is extracted from an HTTP response and when it is assigned to an HTTP request.

“alt attribute”

“alt attribute”

Generating a RealLoad ‘Test’ from an HTTP Test Wizard Session

If you were able to debug the session without errors, you can now generate a RealLoad Test from it. Click on ‘Generate Code’:

“alt attribute”

First click on ‘Generate Source Code’ and then on ‘Compile & Generate JAR’:

“alt attribute”

Then click ‘Define New Test’:

“alt attribute”

You can adjust the name of the test a little (it can also contain spaces). Then click on ‘Define New Test’:

“alt attribute”

Your browser will now be redirected to the ‘Tests’ menu and the test will appear at first position (if the sorting is set to descending test ID).

“alt attribute”

From here you can continue as described at the Tests Menu.

6.3 - HTTP Test Wizard

User Guide | HTTP Test Wizard

“alt attribute”

Overview and Functionality

The HTTP Tests Wizard supports you to create sophisticated tests in an easy way. You can:

As the name suggests, the HTTP Tests Wizard is optimized for the execution of HTTP/S tests. However, by using HTTP Tests Wizard Plug-Ins, any other protocols can also be tested and measured (similar to JUnit Tests).

HTTP Tests Wizard Features:

  • No programming required (except for self written plug-ins). You can create your test through the powerful user interface by assembling all required functionalities via simple dialogs.
  • Variables can be defined and their values can be extracted and assigned directly via the user interface.
  • A powerful debugger helps you to perfect your test.
  • The defined test can be automatically converted into a performance-optimized RealLoad Test.

Session and Session Elements

The test sequence is referred to as a so-called “Session”, whereby each simulated user repeatedly executes the session in a loop (so called “User Session” or “Session Loop”).

“alt attribute”

In order to define a test sequence, you can add various Session Elements to a session:

  • User Input Field : An additional input field whose value can be entered when starting the test and that is defined as a global variable during the test.
  • Measurement Group : Measures the execution time over several session elements.
  • URL : Executes an HTTP request and receives and verifies the HTTP response. The URL can executed either synchronously or in parallel with other URLs. (see also URL Synchronisation Point). One or more “Bound to URL Plug-Ins” can also be added to a URL which, among other things, can modify the HTTP request and post-process the HTTP response.
  • URL Synchronisation Point : Synchronizes URLs executed in parallel (waits until the last HTTP response has received).
  • Delay Time : Delays the execution of the user session for an amount of time.
  • Basic Authentication : Adds a Basic Authentication username and password to the URLs, for all Web servers or for a specific Web server.
  • SSL/TLS Client Certificate : Adds SSL Client Certificate(s) in PKCS12 format for a specific HTTPS server.
  • General Request Header Field : Adds a HTTP request header field (for example “User-Agent”) to the URLs, for all Web servers or for a specific Web server.
  • Cookie Modifier : Sets/Extracts/Deletes HTTP cookies.
  • Outbound IP Address : Supports to use multiple outbound IP addresses on Measuring Agent(s) when executing the test.
  • Conditional Jump : Performs a conditional jump (conditional GoTo) to any other session element.
  • Plug-In : Executes a “Normal Session Element Plug-In”.
  • Input File : Reads an input file line by line, tokenizes each line and extracts the values of the tokens into variables.
  • Output File : Stores the values of variables into an output file.

Position of an Added Session Element

If an (existing) session element is selected, the new session element is inserted after it. Or if no session element is selected, the new session element will be added at the end of the session.

Copy / Cut / Paste and Delete of a Single Session Element

By clicking on the index of a session element, you can copy, cut, paste, and delete it. You can also disable and re-enable the execution of the session element.

“alt attribute”

Copy / Cut / Paste and Delete of Multiple Session Elements

By clicking on the ‘Select Multiple Session Elements’ icon, you can copy, cut, paste, and delete multiple session elements:

“alt attribute”

Displaying and Defining Variables

By clicking on the ‘Variables’ icon the list of all variables is shown. Here you can also define new variables and modify their scope and default value. Furthermore all extractors and assigners for each variable are displayed:

“alt attribute”

Scopes of Variables

A variable can have the following Scope:

  • Global : There is only one instance of this variable during test execution. Such a variable is visible across all simulated users and their session loops. The initial (default) value is assigned at the start of the test.
  • User : There are as many (different) instances of this variable as there are users simulated during a test - this means that each simulated user has its own instance. Such a variable is visible across all session loops of the simulated user. The initial (default) value is assigned when the simulated user starts.
  • User Session : For each session loop of a user, a new instance of the variable is created, which is only valid and visible during the corresponding session loop. The initial (default) value is assigned at the start of each session loop of a user.

Extractors and Assigners for Variables

  • A Variable Extractor defines how the value of a variable is extracted from a session element. The value is typically extracted from an HTTP response of a URL, or from a User Input Field, or from an Input File. Values can also be extracted from many other session elements, such as from HTTP Cookies or from Plug-Ins.
  • A Variable Assigner defines how the value of a variable is assigned to a session element. The value is typically assigned to an HTTP request of a URL, but can also be assigned to many other session elements, such as to Plug-Ins or to HTTP Cookies.

“alt attribute”

See also Handling of Dynamic Session Values.

URL Explorer

With the URL Explorer you can view and search the HTTP data of the URLs, and also post-edit (extract and assign) dynamic session values.

“alt attribute”

“alt attribute”

Post-Editing Recorded Sessions

See Handling of Dynamic Session Values.

Post-Editing Manually Entered Sessions

If you entered the session manually (i.e. did not record it with a proxy recorder), the HTTP data for the URLs will be empty because they are currently unknown. However, you can use the debugger and handle dynamic session values directly in the debugger:

“alt attribute”

All definitions regarding variables that you make in the debugger are transferred directly to the HTTP Test Wizard session.

There is another option in the debugger where you can transfer the HTTP data from the URLs into the URL Explorer by clicking on each executed URL at the HTTP response to the ‘Set/Update Recorded URL’ icon:

“alt attribute”

“alt attribute”

Afterwards it will look something like this in the URL Explorer and you can continue as described at Handling of Dynamic Session Values.

“alt attribute”

Reviewing and Editing multiple URLs

After you’ve finished post-editing the session - and the session is working with the debugger - you should take a look at the content verification of the HTTP responses of all URLs. For example, maybe you forgot to verify the HTTP response status code for some URLs. For this purpose invoke the ‘Review and Edit URL Settings’ wizard:

“alt attribute”

You can also select several URLs here and edit them together:

“alt attribute”

“alt attribute”

We also recommend you take a look at Reviewing the HTTP Response Content Validations.

Configuring Parallel URL Execution

A ’normal’ browser loads certain URLs of a website in parallel. For example, images are loaded in parallel. The same behavior can also be achieved in the HTTP Test Wizard by calling the ‘AutoConfig Asynchronous URL Execution’ wizard. However, please note that this wizard is not perfect - you will probably need to manually adjust the execution of the URLs

“alt attribute”

After calling the wizard, the execution of the URLs will look like this, for example:

“alt attribute”

Of course, not all URLs are executed in parallel at the same time. As with a web browser, the HTTP requests are first entered into a queue and then processed by a few working threads (typically 4 threads).

By calling the wizard again you can undo the configuration.

Generating a RealLoad ‘Test’ from an HTTP Test Wizard Session

This is documented at Recording of a Web Surfing Session.

Detailed Doc of Session Elements

User Input Field

“User Input Fields” are additional values that are entered when starting or defining a test execution and that are available as (global) variables during test execution.

A User Input Field contains the following attributes:

  • GUI Label : The label of the User Input Field that is displayed in the user interface.
  • Variable Name : The name of the global variable at test execution.
  • Input Type : The datatype of the input value (String, Integer, Real or Boolean). Please note that the Input Type is only used to validate the input. During test execution, the value of the variable is always converted to a string.
  • Default Value : The default value of the variable.

See also Starting a Load Test Job

Measurement Group

Adding Measurement Group(s) is optional and causes the execution time of all subsequent session elements to be displayed in aggregate. The end of a measurement group is defined by either adding another measurement group or reaching the end of the session.

Note: If you Record a Web Surfing Session using a Proxy Recorder, a measurement group is automatically added to the session every time you enter a Page Break.

URL

When adding an URL you have at least to select the HTTP Method (for example GET) and to enter the absolute URL (https://<host>/path).

Checking the HTTP response code is optional, but strongly recommended, as otherwise the test result may contain false positive results.

“alt attribute”

The following fields can be entered or selected:

  • General Area
    • HTTP Method: Select the HTTP method (GET, POST, …).
    • URL: Enter an absolute URL, inclusive (optional) query string parameters. Example: https://www.dkfqa.com/?v=1&w=2
    • Execute - synchronous or asynchronous: Determines whether the URL is executed synchronously or in parallel with other URLs (= asynchronous). If you choose “asynchronous” then you have also to add an URL Synchronisation Point to the test sequence after the asynchronous executed URLs.
    • Error Handling: Select what happens if the HTTP request fails or if the HTTP response is invalid. “Final End” means that the whole test is aborted.
    • Enable Implicit Assigners ${<variable-name>}: If checked, then placeholders for variables are considered in the URL as well as in the HTTP request header and content. Example: URL = https://${vHost}/?v=1&w=2, Variable vHost = “www.realload.com”, Result = https://www.realload.com/?v=1&w=2 . Note: instead of using implicit Assigners you can also define Variable Assigners.
  • HTTP Request Content Area | Only fill in this area if the HTTP request contains a content, for example for POST requests.
    • Request Content Type: Select or enter the request content type. For example “application/x-www-form-urlencoded” or “application/json”.
    • Request Content Charset: Select or enter the request content charset - or let the field blank if not applicable.
    • Direct Value or Read From File: Select if the request content data are directly entered here or are read from a file.
    • Request Content Data: Enter the request content data (if they are not read from a file). Note: select first the Request Content Type before you enter the Request Content Data.
    • Also send Zero Content Length: If checked, but no content is available, the HTTP request header field “Content-Length: 0” is sent.
  • Additional HTTP Header Fields Area
    • You can enter additional HTTP request header fields here which are only applied for this URL. Note: additional HTTP request header fields that apply to all URLs should instead be defined by using a General Request Header Field session element.
  • Verify HTTP Response Area | Note: you should configure at least a HTTP response code in order to verify the test result.
    • Verify HTTP Status Code: Select the expected HTTP response code(s).
    • Verify Content Type: Select the expected HTTP response content type(s).
    • Verify Content Text: Enter text fragments that must be present in the HTTP response content, or let the fields blank if not applicable.
  • Plug-Ins Area

URL Synchronisation Point

If you have manually added URLs and configured them to run in parallel (asynchronous) with other URLs, you must add a URL Synchronisation Point after the end of all URLs running in parallel.

See also Configuring Parallel URL Execution

Delay Time

This session element delays execution for a certain time.

The following fields can be entered or selected:

  • Title : The title or description (optional).
  • Delay Time : The delay time in milliseconds.
  • Random Deviation : The random deviation in percent of the Delay Time. Example: if the delay time is 4000 ms and the random deviation is 50% then the randomized delay time that is applied is between 2000 ms and 6000 ms.

Basic Authentication

This session element adds a Authorization: Basic <credentials> HTTP request header field to the HTTP requests of the URLs.

The following fields can be entered or selected:

  • Apply for Host : The DNS hostname or the IP Address of the Web Server to which the authentication is sent, or ‘*’ = all Web Servers.
  • Apply for Port : The TCP port of the Web Server to which the authentication is sent, or ‘*’ = all Web Servers.
  • HTTPS only : If checked, basic authentication will only be sent over encrypted HTTPS connections.
  • Direct Value : If checked, the Username and Password can be entered directly. The same value is applied for all simulated users.
  • Read Values from Input File : If checked, the Usernames and Passwords are read from an (already defined) Input File. The Scope of the Input File determines how the usernames and passwords are assigned to the simulated users.
  • Username : The username (direct value).
  • Password : The password (direct value).
  • Input File : Select an Input File session element here (if ‘Read Values from Input File’ is checked).
  • Username Token No. : Select the token number of the Input File from which the Username is extracted.
  • Password Token No. : Select the token number of the Input File from which the Password is extracted.

SSL/TLS Client Certificate

Using this session element, one or multiple SSL/TLS Client Certificates can be configured, which are used for HTTPS server authorization.

The following fields can be entered or selected:

  • Apply for Host : The DNS hostname or the IP Address of the HTTPS server.
  • Apply for Port : The TCP port of the HTTPS server.
  • Single Certificate : If checked, the PKCS12 File can be selected and the Password of the Certificate can be entered directly. The same certificate is applied for all simulated users.
  • Multiple Certificates referenced by Input File : If checked, the File Names of the PKCS12 Files and the corresponding Passwords of the Certificates are read from an (already defined) Input File. Note that the PKCS12 Files must be located in the same Project/Resource Set where the Input File is located. The Scope of the Input File determines how certificates are assigned to the simulated users.
  • Certificate Password : The password of the certificate (if ‘Single Certificate’ is checked).
  • Input File : Select an Input File session element here (if ‘Multiple Certificates referenced by Input File’ is checked).
  • PKCS12 File Token No. : Select the token number of the Input File from which the PKCS12 File Name is extracted.
  • Certificate Password Token No. : Select the token number of the Input File from which the Password of the Certificate is extracted.

General Request Header Field

This session element adds a HTTP request header field to the HTTP requests of the URLs.

The following fields can be entered or selected:

  • Header Field Name : The name of the HTTP request header field. Example: ‘User-Agent’
  • Apply for Host : The DNS hostname or the IP Address of the Web Server to which the HTTP request header field is sent, or ‘*’ = all Web Servers.
  • Apply for Port : The TCP port of the Web Server to which the HTTP request header field is sent, or ‘*’ = all Web Servers.
  • Direct Value : If checked, the value of the HTTP request header field can be entered directly. The same value is applied for all simulated users.
  • Assign Value from Variable : If checked, the value of the HTTP request header field is assigned by a Variable. The Scope of the Variable determines how the value of the HTTP Request Header Field is assigned to the simulated users.
  • Direct Value : The value of the HTTP Request Header Field (if ‘Direct Value’ is checked).
  • Variable : Select a Variable here (if ‘Assign Value from Variable’ is checked).

Using this session element, HTTP Cookies can be manipulated - regardless of whether they are “HttpOnly cookies” or not.

“alt attribute”

Tip: You can use the Debugger to examine the cookies.

“alt attribute”

“alt attribute”

Extracts the value of a cookie into a variable. The following fields can be entered or selected:

  • From Cookie Name : The name of the cookie.
  • Cookie Domain : The domain of the cookie.
  • Cookie Path : The path of the cookie.
  • To Variable : Select a Variable here to which the value of the cookie is assigned.

Assigns the value of a variable to the value of a cookie. The following fields can be entered or selected:

  • From Variable : Select a Variable here whose value is assigned to the cookie.
  • To Cookie Name : The name of the cookie.
  • Cookie Domain : The domain of the cookie.
  • Cookie Path : The path of the cookie.

Adds or overwrites a cookie. The following fields can be entered or selected:

  • Cookie Name : The name of the cookie.
  • Cookie Domain : The domain of the cookie.
  • Cookie Path : The path of the cookie.
  • Variable: Value : Select a Variable here whose value is assigned to the cookie value.
  • Secure Only : If checked, the cookie will only be sent over encrypted HTTPS connections.
  • Session Cookie : If checked, the cookie is only valid in the session loop of the simulated users (all session cookies will be cleared before each iteration of a session loop).
  • Persistent Cookie : If checked, the cookie is a permanent cookie.

Deletes a cookie. The following fields can be entered or selected:

  • Delete Cookie Name : The name of the cookie.
  • Cookie Domain : The domain of the cookie.
  • Cookie Path : The path of the cookie.

Delete All Session Cookies

Deletes all session cookies.

Delete All Cookies

Deletes all cookies.

Outbound IP Address

The dynamic assignment of multiple outbound IP addresses for the executed URLs of the simulated users is only possible if you operate specially configured Measuring Agents yourself - or have them operated by us.

Basically, every internet device only has a single routable IP address. However, each operating system can be reconfigured so that it supports multiple IP addresses - if the network environment allows it and the additional IP addresses are also routable.

You can only use this session element if you have access to such specially configured Measuring Agents.

The Outbound IP Addresses are set by a Variable (often extracted from an Input File), where the Scope of the Variable determines how the Outbound IP Addresses are assigned to the simulated users.

“alt attribute”

Conditional Jump

This session element performs a conditional jump (conditional GoTo) to any other session element, based on a fixed value or on the value of one or several variables. The number of jumps is counted and can also control the conditional execution.

“alt attribute”

Conditional Jumps are based on a Conditional Expression whose result is always true or false. If the condition is true, the test execution of the currently executed user session jumps to the addressed (target) session element.

Two types of Condition Interpreters are supported:

  • Boolean Expression, such as 1 <= 2
  • Boolean Regex, such as (.)Zurich(.)

One or multiple Placeholder for Variables in the format ${variable-name} can be used at any point in a Conditional Expression:

  • Example of Boolean Expression with a placeholder variable: ${X} <= 2
  • Example of Boolean Regex with a placeholder variable: (.)${Y}(.)

Functionality of the Boolean Expression Interpreter

The Boolean Expression Interpreter supports:

  1. The simple keywords true and false (without further text) which effects that the interpreter always returns true or false.
  2. Simple conditional expressions, such as 10 > 9 | Note: simple expressions are not enfolded by brackets.
  3. Boolean expressions with two conditional expressions, such as (5 > 6) || (6 > 5)
  4. Multiple boolean expression, such as ((5 > 6) or (6 > 5)) and ("Meier" contains "ei")

Supported comparison operators:

  • <
  • <=
  • ==
  • >=
  • >
  • !=
  • contains
  • !contains

Supported boolean operators:

  • && or alternatively the keyword and
  • || or alternatively the keyword or

Parsing of values:

  1. First the interpreter tries to convert values into numbers. If this succeeds for both values of a conditional expression, the comparison is performed with the numbers.
  2. If one of the values cannot be converted into a number, both values are interpreted as text and compared as text.
  3. Simple text values cannot contain spaces. However, it is supported for text values to be enclosed in double quotes, which circumvents this restriction.
  4. Text values enclosed in double quotes are not converted to a number.

Functionality of the Boolean Regex Interpreter

The Boolean Regex Interpreter requires two arguments:

  1. The input to which the Regex is applied. This is always referenced by a variable whose value is interpreted as Regex input.
  2. The Boolean Regex itself which can contain placeholder for variables. The values of the placeholder variables are replaced first, before the Boolean Regex is evaluated.

Jump To : Target Session Element

A target session element can be:

  • A fixed session element : That means that the jump is always made to this session element, regardless of whether new session elements are added or existing session elements are deleted.
  • A Session element addressed by element index : That means that the jump is performed to the index of a session element. In case if session elements are added or deleted, the jump target may change - you may need to reconfigure the conditional jump settings in such a case.
  • A Session element addressed by a variable whose value is interpreted as (integer number) element index. This allows dynamic jumps.

Jump Action

The jump action is performed when the jump is executed (if Conditional Expression == true):

  • None (no specific action)
  • Set Variable Value : When jumping, the variable is set to this value.
  • Incr/Decr Variable : When jumping, the value of the variable is interpreted as (long) integer number and incremented or decremented by the configured amount.

Max. Number of Jumps

This is the limit of the maximum executed jumps per user session (per session loop) - applied to this conditional jump. For jumps back to previous session elements, this limit should never be ‘unlimited’, otherwise this can end in an endless loop.

After this limit has been reached, the Action: If Max. Number of Jumps Exceeded rules what happens next.

Action: If Max. Number of Jumps Exceeded

One of the following actions can be configured if the limit of ‘Max. Number of Jumps’ is exceeded for this conditional jump:

  • No Further Jump : Recommended for simple inner loops.
  • Abort User Session and Report Error : The simulated user aborts the current session and will continue with the next session (if the maximum number of outer session-loops per user is not reached).
  • Abort User and Report Error : The simulated user is terminated - but all other simulated users continue to run.
  • Abort Test and Report Error : All simulated users will be stopped and the test execution will be aborted prematurely.

Plug-In

See Plug-Ins.

Input File

Input files are text files (*.txt, *.csv) whose content is read line by line during the test. Each line is divided into tokens from which values of variables are extracted. Empty lines are skipped. Note that the variables whose values will be extracted from the input file must first be defined before you can add an input file.

The following fields can be entered or selected:

  • Token Delimiter: The token delimiter. This is usually a single character, but can be also a string.
  • Comment Tag: Lines which are starting with the comment tag are skipped. You can place also the comment tag within a line which means that the remaining part of the line is ignored. The comment tag is usually a single character, but can be also a string.
  • Cyclic Read: If not checked, then the test will end when no further line can be read (at eof). On the other hand, if checked, the file is re-read after the end of file was reached.
  • Randomize Lines: if checked then the order of the lines is randomized each time when the file is read.
  • Trim Values: If checked then whitespace characters are removed from the start and end of the extracted values (tokens).
  • Remove Double Quotes: If checked then double quotes are removed from the extracted values (tokens).
  • Scope: Get Next Line per:
    • Global: Reads only a single one line of the file. The line is read at the start of the test.
    • User: Reads each time a line from the file when a simulated user is started.
    • User Session: Reads each time a line from the file when a simulated user starts a new session loop iteration.
  • 6x Variable - Token[Index]: You can configure up to 6 token indexes whose values are extracted into variables.

“alt attribute”

Output File

Output files are text files (*.txt, *.csv) into which values of variables are written. The file is saved in the output directory of the test job.

The following fields can be entered or selected:

  • Output File Name: The name of the output file.
  • Scope: Write Line per:
    • Global: Only a single line is written to the file at the end of the test.
    • User: Each time a simulated user’s execution ends, a line is written to the file.
    • User Session: At the end of each execution of a session loop of any simulated user, a line is written to the file. If the session loop is aborted (for example because an error was measured), no line is written.
  • Value Separator: The separator between the values.
  • Enclose Values: Here you can configure how the values are enclosed.
  • 6x Variable: Up to 6 Variables can be selected

“alt attribute”

“alt attribute”

“alt attribute”

Plug-Ins

Plug-Ins are reusable HTTP Test Wizard extensions that are manually programmed. They can also be published by manufacturing users and imported by other users. Therefore, before you start programming your own plug-in, take first a look at the already published plug-ins.

Plug-In Types

There are 3 types of plug-ins:

  • Normal Session Element Plug-Ins: As the name suggests, such plug-ins are executed as a normal session element. This type of plug-in is versatile and can, among other things, process variables, abort the current session, the user or the entire test, or even perform independent measurements with any protocol.
  • Bound to URL Plug-Ins: Such plug-ins are linked to a URL session element and can change the HTTP request before sending it to the server and post-process the received HTTP response - such as extracting variable values or checking the content of the HTTP response. Such plug-ins can also abort the current session, the user, or the entire test.
  • Java Source Code Modifier Plug-Ins: These are special plug-ins that can subsequently modify the source code generated by the HTTP Test Wizard. Usually such plug-ins are only used as a workaround if the HTTP Test Wizard generates faulty or incomplete source code.

Importing Published Plug-Ins

“alt attribute”

“alt attribute”

After you have imported the plug-in you have to load and compile it - then save the compiled plug-in.

“alt attribute”

“alt attribute”

“alt attribute”

“alt attribute”

After you have saved the compiled plug-in you can add it to your HTTP Test Wizard session. Plug-ins of the type “Java Source Code Modifier Plug-Ins” do not have to be added, but can be called directly after the source code of the test program has been generated.

Developing own Plug-ins

Own plug-ins can be developed in Java 8 or 11. The following example shows a plug-in which decode a base64 encoded string.

“alt attribute”

“alt attribute”

After clicking at “New Plug-In” the plugin type has to be selected (in this example “Normal Session Element Plug-In”):

“alt attribute”

At the “General Settings” tab you have at least to enter the Plug-In Title and the Java Class Name. You should als enter a Plug-In Description which can be formatted by simple HTML tags. The onInitialize Scope can be set to “Global”, as this plug-in does not require any initialization. The onExecute Scope is set to “User Session” in order that all kind of variable scopes are supported.

“alt attribute”

At the next tab - “Input & Output Values” - a plug-in input and a plug-in output (Java-)variable is defined for the plug-in method onExecute. The values of this two Java variables will later correspond with two HTTP Test Wizard variables (which may have a different variable name):

“alt attribute”

When you click at the next tab - “Resources Files” - you will see that the Java library com.dkfqs.tools.jar is already added to the plugin. This is normal as all kind of plug-ins require this library. Here you may add also other Java libraries required by the plug-in - but in this case no other libraries are needed.

“alt attribute”

Now at the “Source Code & Compile” tab, you can first generate the plug-in template. Then you have to extend the generated code with your own code, ie in this example you have to complete the Java import definitions and program the inner logic of the Java method onExecute. Then compile the plug-in.

“alt attribute”

“alt attribute”

At the last tab “Test & Save” you can first test the plug-in (remotely) on any Measuring Agent. To perform the test enter for example “SGVsbG8gV29ybGQ=” as input and you will see as output “Hello World”.

“alt attribute”

“alt attribute”

Finally save the plug-in:

“alt attribute”

After you have saved the plug-in click at the “Close” button. Then you will see the new plug-in in the plug-in list:

“alt attribute”

Publishing Plug-Ins

If your plug-in can be used universally, it would be nice if you also publish it - to make it available to other users.

Note that you have to enable in your Profile Settings the option “Public In-App User” in order that you entitled to publish plug-ins.

Publishing plug-ins is especially useful for users who have additionally activated the option “Publish My Profile as Technical Expert” in their Profile Settings. This will significantly improve your visibility and competence.

“alt attribute”

“alt attribute”

Debugger

“alt attribute”

The HTTP Test Wizard Debugger is a powerful tool with some special features:

  • It is a Remote Debugger, which means the instructions are always executed on a Measuring Agent. This also allows you to debug the session from different geographical locations.
  • The debugger works directly with the definitions made in the HTTP Test Wizard - so there is no code to compile before debugging a session.
  • During debugging, variables can also be defined, modified, and deleted.
  • Values of variables can extracted from HTTP responses and assigned to HTTP requests (in addition/alternative to the Variables Wizard of the URL Explorer).
  • All definitions made in the debugger regarding variables are synchronized with the HTTP Test Wizard session.
  • You can force the debugger to jump over lines of Input Files.
  • The value of HTTP Cookies can be examined and modified.
  • The value of variables can be monitored in real-time.
  • You can search for a text over all (so far) executed HTTP requests and responses.

“alt attribute”

Selecting the Measuring Agent

The executing Measuring Agent can be selected in the upper right corner. The debugger is reset when you select another Measuring Agent.

“alt attribute”

Values of User Input Fields

If your session contains User Input Fields, you will be prompted to enter their values when the debugger is initialized or reset.

“alt attribute”

Jumping over Lines of Input Files

Reading the lines of Input Files is automatically processed by the debugger. However you can force the debugger to jump over the lines of an Input File.

“alt attribute”

“alt attribute”

Debugging the Session

Click Next Step repeatedly to debug the session.

If you want to assign the value of a variable to an HTTP request, briefly enable HTTP Requests Breakpoint before the URL is executed. After that, you should disable this option.

“alt attribute”

Extracting Values from HTTP Responses

Click the received HTTP Response Header or Response Content of a URL to extract a value into a variable:

“alt attribute”

This example extracts the value of a JSON response into a variable. If the variable does not yet exist, you can define it first.

“alt attribute”

Assigning Values to HTTP Requests

Proceed as follows

  1. Enable ‘HTTP Requests Breakpoint’ before the URL is executed.
  2. Click ‘Next Step’. The data of the HTTP request is then initialized, but the request is stopped ‘in the middle’, meaning it is not yet sent to the HTTP(S) server.
  3. Assign the value to the URL, or to the HTTP Request Header, or to the HTTP Request Content.
  4. Disable ‘HTTP Requests Breakpoint’.
  5. Click ‘Continue’. The HTTP request is now sent to the HTTP(S) server.

“alt attribute”

This example assigns the value of a variable to a JSON HTTP request content:

“alt attribute”

6.4 - HTTP/S Remote Proxy Recorder

User Guide | HTTP/S Remote Proxy Recorder

“alt attribute”

By using an HTTP/S Remote Proxy Recorder, the HTTP/S traffic from Web browsers and technical Web clients can be recorded and easily converted into a HTTP Test Wizard session.

The HTTP Test Wizard session can then be post-processed and debugged. Finally, an executable test can be generated from the recorded session.

Functional Overview: Remote Proxy Recorder

“alt attribute”

An HTTP/S Remote Proxy Recorder has two service ports:

  • A Control Port that is addressed by the portal server.
  • A Proxy Port that records the data traffic from a Web browser or from a technical Web client.

All data traffic that passes through the proxy port is first decrypted by the Proxy Recorder and then encrypted again before it is forwarded to the target Web server(s).

In order to record a Web surfing session by a Web browser you have to start two different Web browser products on your local machine. For example:

  • A Chrome browser to operate the portal server (Web Browser 1), and
  • A Firefox browser to record the Web surfing session (Web Browser 2)

We recommend to use always Firefox as Web Browser 2.

To be able to record a Web surfing session, you have to reconfigure Web Browser 2

Additional note: Before you start recording a Web surfing session in Web Browser 2, you must always clear the browser cache.

Once the recording is completed you should undo the configuration changes in Web Browser 2 (restore the original network settings and delete the Proxy Server CA Root Certificate).

Importing the Proxy Recorder CA Root Certificate into the Web Browser

The following describes how you can import the CA Root Certificate of a Proxy Recorder in the Firefox Web Browser.

Sign in to the portal, navigate to the HTTP/S Remote Proxy Recorders menu, and download the CA Root Certificate of the Proxy Recorder to any folder on your computer:

“alt attribute”

“alt attribute”

Note: The image above shows the certificate of a shared proxy recorder (whereby all shared proxy recorders use the same CA root certificate). However, the ‘Subject’ of the certificate could also have other values, depending on the proxy recorder. So that you can find the certificate in Firefox and delete it if necessary, you should remember the values of the ‘Certificate Subject’ (or take a photo of the certificate properties with your mobile phone).

In Firefox call Settings:

“alt attribute”

Click Privacy & Security:

“alt attribute”

Scroll down to Security - Certificates, and click ‘View Certificates…’

“alt attribute”

In the Firefox Certificate Manager click first on Authorities and then on ‘Import…’

“alt attribute”

Select the CA Root Certificate you downloaded before and check the “Trust this CA to identify websites” box. Then click OK

“alt attribute”

The CA Root Certificate is now imported into Firefox:

“alt attribute”

Important Security Notice

After you have installed a Proxy Recorder’s CA Root Certificate in a web browser product (in this case, Firefox), you can record web surfing sessions, but you can no longer trust the web browser itself, because if someone else has accesses to the same CA Root Certificate, you can be cheated - without the corresponding web browser issuing a warning.

This is especially important if you use a Shared Proxy Recorder or the Desktop Companion application. This means that you should never use such a web browser for financial-critical transactions such as e-banking - or for logging into personal websites, as long as a proxy recorder’s CA root certificate is installed in the browser. Only after removing the CA root certificate from the browser is it safe again.

This security problem is not so critical if you operate your own HTTP/S Remote Proxy Recorder (or have us operate it). But even in this case, you could theoretically be cheated by your team members.

So the most secure way is to remove the proxy recorder’s CA root certificate from the web browser as soon as your recording is done.

Recording of Web Surfing Sessions

Once you have Imported the Proxy Recorder CA Root Certificate into the Web Browser and have installed the ‘RealLoad Test Recorder’ browser extension you are ready to Record Web Surfing Sessions and convert them into RealLoad Tests.

7 - Browser Recorder extensions for both Chrome and Firefox

Browser Extension - Introduction

This browser extension will ask for the RealLoad proxy configurations, then automatically set Proxy in the browser. Using the extension user can clear cache, start recording, stop recording, clear recording and add page breaks while recording the user journeys in the browser.

Before you start installing the browser extension, Please make sure that you have already followed the steps mentioned in the doc HTTP/S Remote Proxy Recorder) , Registering a Proxy Recorder at the Portal and Import the Proxy Server CA Root Certificate.

Browser extension for Chrome

Please open the link https://chrome.google.com/webstore/detail/mlindhmkpdadnkdoicbmijgdecpbioai/ in Chrome and Add to Chrome.

After adding the extension to Chrome, the next step will be setting the proxy details for recording in the extension options. In order to do that first we need to copy the proxy configuration data from the RealLoad portal https://portal.realload.com/ like below (under the HTTP/S Remote Proxy Recorder screen).

Once you have entered all the necessary RealLoad proxy details and save, you are ready to use the Browser Recorder extension for Chrome

Now you can go ahead and record your browser sessions by pressing Start Recording, You can add page breaks while recording by adding the page break name and press icon marked in the below screenshot.

Once you are done with your recording, you can press Stop Recording, which will revert the configured RealLoad Proxy to the Browser Automatic Proxy.

Now when you go to portal https://portal.realload.com/, you can see the recordings under the configured HTTP/S Remote Proxy Recorder screen.

The same steps are described below for the extension for Firefox.

Browser extension for Firefox

Please open the link https://addons.mozilla.org/en-US/firefox/addon/realload-test-recorder/ in Firefox and Add to Firefox. Please make sure that you have checked the option “Allow this extension to run in Private Windows” during the installation.

After adding the add-ons to the firefox, the next step will be setting the proxy details for recording in the add-on preferences like below.

Once you have entered all the necessary RealLoad proxy details and save, you are ready to use the Browser Recorder extension for Firefox.

Now you can go ahead and record your browser sessions by pressing Start Recording, You can add page breaks while recording by adding the page break name and press icon marked in the below screenshot.

Once you are done with your recording, you can press Stop Recording, which will revert the configured RealLoad Proxy to the Browser Automatic Proxy.

Now when you go to portal https://portal.realload.com/, you can see the recordings under the configured HTTP/S Remote Proxy Recorder screen.

8 - Desktop Companion

Real Load Desktop companion - Introduction

The Desktop Companion is a small application that allows you to manage some features related to the Real Load Platform.

In particular the desktop companion allows you to:

  • Generate HTTP Load Test scripts from HTTP archives (.har) files
  • Locally run the Real Load Proxy Recorder
  • Perform some basic editing of test scripts
  • Upload test scripts directly to the Real Load Portal
  • Manage AWS Measuring Agents (launch, terminate, register with Real Load Portal)

See sections below to learn more about the Desktop Companion.

8.1 - Release Notes

Desktop Companion change log

0.26 | 6 Nov 2022

0.25 | 28 Oct 2022

0.24 | 21 Mar 2022

0.23 | 25 Jan 2022

0.22 | 13 Dec 2021

0.21 | 9 Dec 2021

0.20 | 3 Dec 2021

  • Usability: Added help links to online documentation.
  • Usability: Added various configuration checks and related alert dialogs.
  • Usability: On exit, added alert in relation to running AWS EC2 instances.
  • Usability: List of AWS instances is updated via a background thread, periodically.
  • Usability: List of registered Measuring Agents is updated via a background thread, periodically.
  • Usability: The AWS region where an EC2 instance is launched is automatically added to “My Regions” list.
  • Installer: Created a Windows full installer that includes a JRE.
  • Bugfix: Fixed an issue with HAR files containing POST requests with no content type.
  • Bugfix: Updated JavaFX to v17
  • Download URL JAR: https://download.realload.com/desktop_companion/RealLoadDesktopCompanion-0.20.jar
  • Download URL Windows full installer: https://download.realload.com/desktop_companion/RLDTC_windows-x64_0_20.exe

0.10 | 8 Nov 2021

8.2 - Desktop Companion - Pre-requisites

Read this first.

Before running the Desktop Companion make sure you satisfy the following pre-requisites:

JRE 11 (Mandatory to execute JAR file)

The Desktop Companion is a Java application currently delivered as a .jar file. At this stage there is no full installer for the application, which means that as a pre-requisite to run the application you’ll need to install a Java 11 JRE.

While it should run on any systems that support Java 11 and FXML, it was only tested on the following platforms:

Operating System JRE 11 Vendor and Build JRE download link Comments
Windows 10 Microsoft “11.0.11” 2021-04-20 https://docs.microsoft.com/en-au/java/openjdk/download
Windows 11 Microsoft https://docs.microsoft.com/en-au/java/openjdk/download
Apple OS X Microsoft “build 11.0.13+8-LTS” https://docs.microsoft.com/en-au/java/openjdk/download

AWS API Security Credentials (Optional)

In order to use the AWS integration features, you’ll need to prepare Security Credentials with AWS IAM.

Seach for “IAM” in the AWS search bar and click on “Users”:

Then add a new user:

Enter a suitable user name and select “Access key” as the AWS credential type.

On the next page set an appropriate policy for the user. Usin the “AmazonEC2FullAccess” provides sufficinet permissions.

Skip the “Add tags” screen and go to the “Review” screen and click on “Create user”. Make sure the permissions you’ve assigned to the user appear on this screen.

Finally take a note of the Access Key ID and Secret Access Key as thess need to be configured in the Real Load Desktop Companion.

Real Load Portal User API Authentication Token (Optional)

In order to register with the Real Load portal the AWS EC2 instances you’ve launched, the Desktop Companion requires that you configure an authentication token.

To create an authentication token proceed as follows:

Login to the Real Load Portal and click on the User -> API Authenticaiton Tokens menu.

Then click on the “Add API Authentication Tokens” button and enter a suitable purpose description. Optionally you can restrict the src IP address from where this token can be used.

Take note of the authentication Token Value as you’ll need to configure it in the Desktop Companion.

8.3 - Desktop Companion - Download and run

Where to donwload the application from and how to run.

Downloading the full installer (Windows and OS X)

The full installer includes a JRE as well as the Desktop Companion application itself. This is the preferred method of deploying the application, as it avoids issues that might occur because of JRE customization, etc…

Additionally the application can be started from the Windows start menu, as any other application.

The latest version of the full installer can be downloaded from here:

Downloading standalone JAR file only (Intel x64 architectures only)

You can download the latest version of the Desktop Companion as an executable .jar file from here: https://download.realload.com/desktop_companion/latest

There is no installer to be executed, the application can be launched by double clicking on the .jar file. If that doesn’t work, consider launching from the command line.

Windows: Launch from file explorer

If you’ve only got one JDK/JRE version installed on your Windows computer then it’s very likely that .jar files are associated with the correct JRE. In that case you should be able to launch the application by double clicking on the .jar file.

If the application doesn’t start it’s possible the .jar extensions is associated with the incorrect JRE or another application altogether. You can try to right click on the .jar file and the “Open with” from the context menu.

Then selecte the OpenJRE launcher:

… or alternatively navigate to the JRE 11 binary, as shown in this screenshot:

Similarly on Windows 11:

s

If the application doesn’t start, consider starting it from the command line as explained below.

Windows: Launch from command line

Open a Terminal window. First validate that the java binary in your path is a JRE 11 binary by running this command:

java -version

The output should indicate you’re running a JRE 11 binary, as shown here:

If the version of the JRE is 11 then you can run the following command from the folder where the Desktop Companion .jar file was downloaded:

java.exe -jar .\RealLoadCompanionFXML-0.1.jar

OS X (x64 only): Launch from finder

If you only have JRE 11 installed on your Mac, you should be able to launch the application simply by double clicking on its icon:

Alternatively right-click on the icon and “Opne With” Jar Launcher.

If this doesn’t work, refer below on how to launch from the command line.

OS X: Launch from command line

Open a Terminal window. First validate that the java binary in your path is a JRE 11 binary by running this command:

java -version

The output should indicate you’re running a JRE 11 binary, as shown here:

If a different JRE version is returned (like for example a JRE 8 binary) then you’ll need to specify the full path to the java 11 command. For example, assuming you installed the Microsoft OpenJDK 11, you would typically find it at this location:

/Library/Java/JavaVirtualMachines/microsoft-11.jdk/Contents/Home/bin/java

To run the Desktop Companion, assuming the .jar file is present in the current directory, run the following command from a Terminal window:

/Library/Java/JavaVirtualMachines/microsoft-11.jdk/Contents/Home/bin/java -jar RealLoadCompanionFXML-0.1.jar

8.4 - Desktop Companion - Settings Menu

Configure the Desktop Companion for integration with AWS and the Real Load Portal.

Various application settings are configurable in the File -> Settings menu.

General settings

  • Default HAR input folder: Location where to look for .har (HTTP Archives) files by default. This should be the location where your browser writes files to.

  • Default export folder: This is the location where Real Load HTTP test scripts will be exported to in JSON format.

  • Agent Secret: The Measuring Agent secret to authenticate connections from the Real Load Portal to the Cloud Agent instance (AWS EC2 and Azure instances in future).

User API settings

  • Portal URL: The endpoint of the Real Load portal User API. Unless you have an on-premise installation, use the default value https://portal.realload.com/RemoteUserAPI

  • Authentication Token: Enter here the authentication token that was generated by the Portal as part of the pre-requisites steps. Click on the “Test” button to test the API token.

  • Refresh Interval: How frequently should the list of Measuring Agents registered with the Real Cloud portal be refreshed. Any value 60 seconds or lower indicates no background refresh, you’ll need to manually trigger a refresh via context menu. By default set to 61 seconds (background refresh enabled).

AWS settings

  • AWS Access Key: Paste the AWS Access Key that was obained as part of the preparation steps.

  • Secret Access Key: Paster the AWS Secret Access key that was obained as part of the preparation steps. Use the “Test” button to validate the credentials.

  • Preferred Instance Type: The EC2 instance type to use when launching a new Mesauring Agent instance.

  • AWS EC2 Refresh Interval: How frequently should the list of AWS EC2 instances be refreshed. Any value 60 seconds or lower indicates no background refresh, you’ll need to manually trigger a refresh via context menu. By default set to 61 seconds (background refresh enabled).

  • My AWS Regions: Select the AWS regions you commonly use. To select multiple regions hold the CTRL key while selecting.

Proxy Recorder settings

  • Proxy Port: The TCP port on the local machine that will listen for incoming proxy connections. You’ll need to configure this port as the HTTP proxy in your browser.

  • Export CA Certificate: This exports the CA certificate used by reverse proxy to sign SSL certificates. The exported CA should be added as a trusted CA to the browser you’re planning to use to record HTTP requests.

The next settings should be left to default values, unless there are specific reasons for changing them.

  • Proxy Backend Server Start Port Range: Starting TCP source port range for connections generated by the Proxy.
  • Proxy Backend Server End Port Range: Ending (high) TCP source port range for connections generated by the Proxy.
  • Debug HTTP Headers: Enables capturing HTTP headers values in Proxy debug log.

8.5 - Desktop Companion - File Menu

Import HTTP archives and export test scripts locally or to the Portal.

Import HAR file

To import an HTTP Archive generate by a browser use the corresponding option in the file menu and select the relevant .har file. All requests present in the file will appear in the editor tab.

Export test script to JSON file

You can export your test script to a Real Load JSON session file by selecting the corresponding option in the file menu. The file will be saved in the folder you select.

You’ll then need to manually upload the file to a Resource Set in the Real Load Portal using the upload function.

Export test script to Portal

Test scripts can be directly uploaded to the Real Load Portal via the “Export session to portal…” menu. You’ll need to select the resource set to save the load test to:

You have 2 options:

  • Overwrite existing test script: If you select and existing HTTP test script without modifying the filename this will effectively overwrite it.
  • Select a resource set: On the other hand if you select a resource set (… “Web Test” in the above screenshot for example) you’ll then have to enter a filename for your new test script.

8.6 - Desktop Companion - Editor

Simple request editor.

In the request editor tab you’ll be able to perform some basic editing of the requests imported from an HAR file or recorded by the proxy recorder.

For more complex editing please use the HTTP wizard in the Real Cloud portal after uploading your test script there.

Overview

The Editor tab is split in 3 main parts:

  • HTTP Requests lists: This section shows all HTTP requests imported from an HAR or recorded by the Recording Proxy. The context menu accessible via a right click allows you to perform some basic editing functions.
  • Request details: When selecting a request some additional details about the request will appear, like values extracted from HTTP headers, etc…
  • Unique domains: In this section all unique domain names will be listed. Clicking on a domain will select all relevant rows in the requests table.

Delete Requests

You can mark requestes in the main editor window by clicking on them. Hold the ctrl keyty to mark multiple request or use the shift key to mark a range of requests.

Once marked you can delete the requests by right-clicking on a marked request and selecting “Delete selected” from the context menu:

Adding time delays

To add a time delay (currently hardcoded to 1 second) before or after a request, select the relevant context menu item while hovering over a request.

Time delays rows appear as requests of type “T” in the main editor window:

Domain based selection

In order to bulk select all requests belonging to specific domains select one or more domains (by holding the CTRL key). This will select corresponding requests in the requests list which can then be easilly deleted.

It is also possible to directly delete requests belonging to one or multiple domains using the context menu.

8.7 - Desktop Companion - Measuring Agents

Launch and terminate AWS based measuring agents and manage their registration with the Portal.

The Measuring Agents tab allows you to manage Cloud Based (… currently AWS) Measuring Agents. This section of the application will be of most use if you configured AWS credentials in the preferences section of the application.

If no AWS credentials are configured, you won’t be able to start and terminate AWS instances.

Listing Measuring Agents AMIs

In the left pane you’ll see a list of available AWS AMIs. You can further filter the list of AMIs by selecting a specific version and/or by selecteing an AWS region.

Launching an instance

To launch an new EC2 instance right-click on the relevant AMI and select “Launch”. A screen to confirm the launch will be displayed. Confirm by clicking on the “Launch” button.

When launching an instance in an AWS EC2 region that is not yet part of your “My Regions” list, the region will be automatically added to the list.

Launching a new instaance will automatically trigger a refresh of the AWS Measuring Agents list. It might take a few second for the list to update and the new instance to be reflected in it.

Then confirm the launch action:

Listing running EC2 instances

After launching an instance go to the top right part of the window listing the running instances and right click on “Refresh”. This will retrieve all running EC2 instances from the preferred AWS regions and display them in the table.

Registering an instance

To register an AWS instance with the Real Load Portal right-click on the instance and then select the “Register with portal” menu item. The instance ID will be used as the description in the Real Load Portal.

List registered instances

In the left bottom part of the window you’ll see the Measuring Agents currently registered on the Portal. To update the list right click and select the “Refresh” option.

De-register an instance

In order to de-register an instance from the Portal right click on the instance name and select De-register.

Terminate an instance

To terminate an AWS EC2 instance right click on the instance name.

8.8 - Desktop Companion - Proxy Recorder

Run the proxy recorder locally on your desktop.

The Desktop Companion allows you to run the Proxy Recorder locally on your desktop. Recorded requests will then appear in the Editor tab. Refer to the Proxy Recorder documentation for further information on how it works.

There are two main sections in this chapter:

  • How to configure your browser.
  • Hot to start/stop the recording process.

Configure your browser

You’ll have to configure the port the Real Load Proxy listens on (default: 18080) in your browser. We recommend to use a browser that allows you to configure proxy settings independently from the Operating System proxy settings. FireFox is a good candidate for this and we’ve documented its configuration in this section.

Import Reverse Proxy CA certificate

For the browser to trust SSL certificates issued by the Recording Proxy, you’ll need to import the Proxy’s CA certificate as follows:

  • Export the certificate to a file: In the Desktop Companion’s Proxy Setting tab, click the “Export CA Certificate. Then select a folder where the certificate is to be exported. It will be saved in file “RecProxyCert.cer”.

  • Import certificate: Navigate to the certificate settings in FireFox:

Then click on the “Auhtorities” tab, “Import” button and select the “RecProxyCert.cer” that you’ve just exported from the Desktop Companion.

Make sure you trust the certificate to identify websites:

Configure proxy settings

In the FireFox “Settings” and scroll to the bottom of the page where “Network Settings” are located . The below window should be displayed and configure the Port to be 18080 for both HTTP and HTTPS. Configure the host and port as shown in the below screenshot. Also tick the “Also use this proxy for HTTPS”.

Test browser configuration

Navigate to any SSL enabled page. You shouldn’t see any warnings about untrusted SSL certificates beings used.

If you check the certificate of the site you’re visiting, the issuer should be “Real Load Pty Ltd”, as shown in this screenshot.

Start the recording process

To start the recording process go to the Proxy Recorder tab and click on the “Start Recorder” button. Note that when the recording process is started, previous Proxy Recorder logs are purged.

Stop the recording process

To stop the recording process clik on the “Stop Recorder” button. Then navigate to the “Editor” window and you should see all recorded requests as in this example:

9 - Mobile Companion App

RealLoad Mobile Companion App

Abstract

The Mobile Companion is a small Android App that displays a simplified view of the Synthetic Monitoring Dashboard and can receive and display Synthetic Monitoring Alert Notifications.

App Features

  • This App can be installed on any Android Device and gives you an overview of the status of your Synthetic Monitoring Jobs at any time - without having to sign in to the Portal Server.
  • The App also notifies you immediately if a Monitoring Job detects an error or warning.
  • The use of multiple Mobile Devices per Portal Server main user account and its team member accounts is supported.
  • Each Mobile Device can be a member of one or multiple Monitoring Alert Groups, thereby causing different types of Alert Notifications to be sent to different Mobile Devices.

Prerequisites:

  • Any device running Android version 9 or newer.
  • You need a RealLoad Portal Server account, i.e. the App cannot be operated “stand-alone”. If you don’t have a RealLoad Portal Server account you can Sign Up for Free and follow the instructions to create your first Synthetic Monitoring job.
  • You should have at least one Synthetic Monitoring Job configured.

Installation

Get it on Google Play

In order to receive notifications at any time, we recommend that you switch off the App option “Pause app activity if unused”.

App Configuration - Login to the Portal Server

When you start the App for the first time, you have to log in to the portal server. Enter the same username and password as for a normal portal server sign-in.

In the background, during the login process on the portal server, a so-called “API Authentication Token” is generated with the purpose “RealLoad Mobile App” which is permanently stored in the Mobile Companion App.

Portal Server:

Testing the App

After you have installed and configured the App, you should test the receipt of push notifications.

  1. Sign in to the Portal Server using a web browser
  2. Navigate to User - Mobile Companion
  3. Send a Test Notification
  4. Check the receiving of the Notification

Synthetic Monitoring Dashboard - Simplified View

  1. Click “Monitoring” at the bottom of the App. This will display the status of your Monitoring Groups.
  2. Click on a Monitoring Group. This displays the status of the corresponding Monitoring Jobs.

If Synthetic Monitoring warnings or errors were detected, the background color changes to yellow or red:

Configuration as Alert Device

  1. Sign in to the Portal Server using a web browser.
  2. Navigate to Synthetic Monitoring - Alert Groups & Devices.
  3. In the Mobile Companion section, click the Expand icon.
  4. In the Mobile Companion section, click the Add icon.
  5. Select the Mobile Companion Device and click the “Add Mobile Companion Alert Device” button.
  6. The Mobile Companion Device is now shown in the Mobile Companion section. Click the “Add to Alert Groups” icon.
  7. Select one or multiple Alert Groups and click the “Add Alert Device to Alert Groups” button.

The Synthetic Monitoring Alert Notifications are shown when click on “Messages” at the bottom of the App. You can also delete individual or all messages by long-clicking on a message. Please note that the message(s) will only be deleted on the mobile device - but not in the RealLoad portal.

When you set up a mobile device as an alert device for the first time, we recommend that you double check whether all alert notifications have been received. Proceed as follow:

  1. Sign in to the Portal Server using a web browser.
  2. Navigate to User - Mobile Companion.
  3. Select the “Transmitted Push Notifications” tab.
  4. Compare the number and type(s) of the transmitted notifications of your Device Id with the number of shown messages in the Mobile Companion App.

Tip: You can get the unique Device Id at any time by clicking “Settings” at the bottom of the App.

10 - Regression Testing - Test Suites

Regression Testing - Test Job Templates and Test Suites

Abstract

Regression testing is supported by Real Load using “Test Job Templates” and “Test Suites”.

Features of Real Load Test Suites:

  • Test suites are the grouping of multiple test job templates and are executed as a single “run” with different test scenarios.
  • Due to the universal architecture, test jobs of any kind can be part of a test suite (HTTP Test Wizard Jobs, JUnit Jobs, Selenium Jobs ..).
  • The results of the test suite “runs” are archived and can be compared with each other.
  • In addition, the result of a particular test job of a test suite run can be compared with the result of the same test job executed by other test suite runs.
  • All test jobs of a test suite support the powerful ability to be executed with hundreds or thousands of concurrent users and can even executed on a cluster of measuring agents.
  • The test jobs of a test suite are grouped into “execution groups”, where the test jobs of an execution group can be executed in sequential or parallel order.
  • The execution groups can be nested, whereby a separate measuring agent or measuring agent cluster can be defined for each execution group.
  • A “run” of a test suite can be triggered either interactively or via the Remote User API.

This seems a bit complicated at first glance. However, as you will see, creating and running test suites is fairly easy.

Test Job Templates

Test job templates can be created by clicking ‘Save as Template’ in the dropdown menu of any load test job:

You can freely choose the name of the template, but it must be unique.

The template is then created. For verification purposes, the test job properties are displayed which you can also change here.

The list of all templates will then be displayed. Here you can delete, clone and modify individual templates and you can create also new load test jobs from a template.

Test Suites

Adding a Test Suite

As soon as one or more test job templates are created, you can add a new test suite.

You can freely choose the name of the test suite, but it must be unique.

After that the test suite is created and the “Test Suite Editor” is automatically invoked.

Editing a Test Suite

It is recommended that you configure a measuring agent or cluster on which the test suite runs will be executed so that you are not asked about it every time you start a test suite run.

Only “Execution Groups” can be defined at the main level of a test suite. This means that you have first to add one or more execution groups to the test sequence.

At the top right of the editor area you can configure the “Main Execution Order”, which controls whether the execution groups at the main level are executed one after the other (= sequential ) or concurrently (= parallel).

Adding an Execution Group

Here you can specify on which measuring agent or cluster the execution group will be executed, whereby the default option “inherit from parent” is recommended. By setting the “Execution Order” you can determine whether the elements of the execution group are executed sequentially or in parallel.

Note: Execution groups can also be nested, which means that an execution group can contain several sub-execution groups, which can then contain sub-sub execution groups. The advantage of nesting execution groups is, among other things, that the measured data of the executed the test jobs are presented in aggregated form at the level of each execution group.

Adding Test Job Templates to an Execution Group

Finally, click the icon at the top left corner to return to the Test Suites Dashboard.

Executing a Test Suite “Run”

A test suite “run” can be triggered either interactively or via the Remote User API.

To start a “run” click on the green icon of the test suite - which means that all test jobs of the test suite are executed.

In the upper area of the green window, summarized/aggregated measurement results over all executed test jobs are displayed.

In the left area of the green window, the progress of the of the test suite execution is displayed in real time, whereby the measured values of the currently executed (load) test jobs are updated every 4 seconds.

Multiple test suites can be executed concurrently, but the same test suite cannot be executed twice at the same time. Note that closing the green window will not abort the execution of the test suite “run”.

Tip: If a load test job of a test suite takes several minutes to execute, you can also view the detailed statistics of the load test job in a second Web Browser window in the Real Time View. All you have to do is select the Real Time View in the second browser window.

After a test suite run is completed you can click on “Display Result”.

Test Suite “Run” Results

Test Suite Run Results are always historical data, whereby the execution groups and the test jobs are painted as they were at the time of execution.

In the upper area summarized/aggregated measurement results over all executed test jobs are displayed.

Clicking on the chart icon of a test job displays the “Test Result Details”. By clicking the file icon, you can view the contents of the test job output files.

Comparing Test Suite Run Results

The summarized/aggregated measurement results of the test suite runs can be compared across all or a selection of runs.

Furthermore, the measurement results of the individual test jobs can be compared across all or a selection of runs.

11 - Table Server

Table Server - Introduction

It is an API Service which holds all the API requests, which can be used to store the test data in table formats in a Database, which can be consumed/produced by both load test and synthetic monitoring tests. So it acts like a Test Data database where multiple scripts can consume common test data.

Currently, the Tests need Plugins to communicate with the Table Server API, but in near future we will come up with a solution where performance Engineers can write inline javascript in the test to communicate with the Table Server APIs.

Below are the API Requests that Table Server supports

  • Create a Table with the JSON input, an overwriteFlag flag will be optional here, if set to true, it will overwrite the existing table with the same name.
  curl -k --location --request POST 'https://{{tableserverURL}}:8084/Api' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "apiAuthToken": "{{apiAuthToken}}",
  "action": "createNewTable",
  "tableName": "Products",
  "columnNamesAndValues": [
        {
            "ProductGroup": "Icecream",   
            "ProductName": "Vanilla"
        },
        {
            "ProductGroup": "Pizza",   
            "ProductName": "Margareta"
        }
    ]
  }'
  • Create a Table with the CSV file data, an overwriteFlag flag will be optional here, if set to true, it will overwrite the existing table with the same name.
  curl -k --location --request POST 'https://{{tableserverURL}}:8084/Api' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "apiAuthToken": "{{apiAuthToken}}",
  "action": "createNewTableFromCSVFile",
  "uploadFileName": "Products.csv",
  "delimiter": ",",
  "uploadFileDataB64": "UHJvZHVjdEdyb3VwLFByb2R1Y3ROYW1lLFByaWNlDURyaW5rcyxDb2NhQ29sYSwxMApEcmlua3MsUGVwc2ksNwpEcmlua3MsN1VwLDgKRHJpbmtzLEZhbnRhLDkKUGl6emEsTWFyZ2FyZXRhLDEwDQ=="
  }'
  • Get All the Tables from Table Server with or without column details.
  curl -k --location --request POST 'https://{{tableserverURL}}:8084/Api' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "apiAuthToken": "{{apiAuthToken}}",
  "action": "getAllTables",
  "includeColumns": true
  }'
  • Create a single test data, can also pass an optional parameter “uniqueFlag” to 1, so that it will make sure that the column value is unique.
  curl -k --location --request POST 'https://{{tableserverURL}}:8084/Api' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "apiAuthToken": "{{apiAuthToken}}",
  "action": "insertColumn",
  "table": "Products",
  "column": "ProductGroup",
  "value": "Sweets"
  }'
  • Get a single test data, returns first row value for the given column and marks that as retrieved in the table. Supports an additional parameter rowIndex for getting the same value multiple times during the execution of the tests.
  curl -k --location --request POST 'https://{{tableserverURL}}:8084/Api' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "apiAuthToken": "{{apiAuthToken}}",
  "action": "retrieveColumn",
  "table": "Products",
  "column": "ProductName"
  }'
  • Update a single test data.

    curl -k --location --request POST 'https://{{tableserverURL}}:8084/Api' \
    --header 'Content-Type: application/json' \
    --data-raw '{
    "apiAuthToken": "{{apiAuthToken}}",
    "action": "updateColumn",
    "table": "Products",
    "column": "ProductGroup",
    "value": "Fruits",
    "rowIndex":1
    }'
    
  • Create multiple test data.

  curl -k --location --request POST 'https://{{tableserverURL}}:8084/Api' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "apiAuthToken": "{{apiAuthToken}}",
  "action": "insertRow",
  "table": "Products",
  "columnsAndValues": [
        {
            "ProductGroup": "Vegetables",      
            "ProductName": "Carrot"
        }
    ]
  }'
  • Get multiple test data, returns first row values for given columns and marks that as retrieved in the table. Supports an additional parameter rowIndex for getting the same value multiple times during the execution of the tests.
  curl --location --request POST 'https://{{tableserverURL}}:8084/Api' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "apiAuthToken": "{{apiAuthToken}}",
  "action": "retrieveRow",
  "table": "Products",
  "columns": [
            "ProductGroup",    
            "ProductName"
    ]
  }'
  • Query a table with column names and values, if not provided Column Names and Values, the entire table data will be retrieved.
  curl -k --location --request POST 'https://{{tableserverURL}}:8084/Api' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "apiAuthToken": "{{apiAuthToken}}",
  "action": "queryTable",
  "table": "Products",
  "columnNamesAndValues": [
        {
            "ProductGroup": "Vegetables",   
            "ProductName": "Carrot"
        }
    ]
  }'
  • Export a table as CSV file data.
 curl --location --request POST 'https://{{tableserverURL}}:8084/Api' \
 --header 'Content-Type: application/json' \
 --data-raw '{
 "apiAuthToken": "{{apiAuthToken}}",
 "action": "exportTableToCSV",
 "table": "Products"
 }'

12 - Developer Guide

All Purpose Interface | Developer Guide

Abstract

This document explains:

  1. How to develop a test program from scratch.
  2. How to add self-programmed measurements to the HTTP Test Wizard (as plug-ins).

The product’s open architecture enables you to develop plug-ins, scripts and programs that measure anything that has numeric value - no matter which protocol is used!

The measured data are evaluated in real time and displayed as diagrams and lists. In addition to successfully measured values, also errors like timeouts or invalid response data can be collected and displayed in real time.

At least in theory, programs and scripts of any programming language can be executed, as long as such a program or script supports the All Purpose Interface.

In practice there are currently two options for integrating your own measurements into the Real Load Platform:

  1. Write an HTTP Test Wizard Plug-In in Java that performs the measurement. This has the advantage that you only have to implement a subset of the “All Purpose Interface” yourself:

    • Declare Statistic
    • Register Sample Start
    • Add Sample Long
    • Add Sample Error
    • [Optional: Add Counter Long, Add Average Delta And Current Value, Add Efficiency Ratio Delta, Add Throughput Delta, Add Test Result Annotation Exec Event]

    Such plug-ins can be developed quite quickly, as all other functions of the “All Purpose Interface” are already implemented by the HTTP Test Wizard.

    Tip: An HTTP Test Wizard session can also only consist of plug-ins, i.e. you can “misuse” the HTTP Test Wizard to only carry out measurements that you have programmed yourself: Plug-In Example

  2. Write a test program or from scratch. This can currently be programmed in Java or PowerShell (support for additional programming languages will be added in the future). This is more time-consuming, but has the advantage that you have more freedom in program development. In this case you have to implement all functions of the “All Purpose Interface”.

Interface Specification

Basic Requirements for all Programs and Scripts

The All Purpose Interface must be implemented by all programs and scripts which are executed on the Real Load Platform. The interface is independent of any programming language and has only three requirements:

  1. The executed program or script must be able to be started from a command line, and passing program or script arguments must be supported.
  2. The executed program or script must be able to read and write files.
  3. The executed program or script must be able to measure one or more numerical values.

All of this seems a bit trivial, but has been chosen deliberately. So that the interface can support almost all programming languages.

Generic Program and Script Arguments

Each executed program or script must support at least the following arguments:

  • Number of Users: The total number of simulated users (integer value > 0).
  • Test Duration: The maximum test duration in seconds (integer value > 0).
  • Ramp Up Time: The ramp up time in seconds until all simulated users are started (integer value >= 0). Example: If 10 users are started within 5 seconds then the first user is started immediately and then the remaining 9 users are started in (5 seconds / 9 users) = 0.55 seconds intervals.
  • Max Session Loops: The maximum number of session loops per simulated user (integer value > 0, or -1 means infinite number of session loops).
  • Delay Per Session Loop: The delay in milliseconds before a simulated user starts a next session loop iteration (integer value >= 0) – but not applied for the first session loop iteration.
  • Data Output Directory: The directory to which the measured data have to be written. In addition, other data can also written to this directory like for example debug information.

Implementation Note: The test ends if either the Test Duration is elapsed or if Max Session Loops are reached for all simulated users. Currently executed sessions are not aborted.

In addition, the following arguments are optional, but also standardized:

  • Description: A brief description of the test
  • Debug Execution: Write debug information about the test execution to stdout
  • Debug Measuring: Write debug information about the declared statistics and the measured values to stdout
Argument Java PowerShell
Number of Users -users number -totalUsers number
Test Duration -duration seconds -inputTestDuration seconds
Ramp Up Time -rampupTime seconds -rampUpTime seconds
Max Session Loops -maxLoops number -inputMaxLoops number
Delay Per Session Loop -delayPerLoop milliseconds -inputDelayPerLoopMillis milliseconds
Data Output Directory -dataOutputDir path -dataOutDirectory path
Description -description text -description text
Debug Execution -debugExec -debugExecution
Debug Measuring -debugData -debugMeasuring

Single-Threaded Scripts vs. Multiple-Threaded Programs

For scripts which don’t support multiple threads the Real Load Platform starts for each simulated user a own operating system process per simulated user. On the other hand, for programs which support multiple threads, only one operating system process is started for all simulated users.

Scripts which are not able to run multiple threads must support the following additional generic command line argument:

  • Executed User Number: The currently executed user (integer value > 0). Example: If 10 scripts are started then 1 is passed to the first started script, 2 is passed to the second started script, .. et cetera.
Argument PowerShell
Executed User Number -inputUserNo number

Specific Program and Script Arguments

Additional program and script specific arguments are supported by the Real Load Platform. Hoverer, their values are not validated by the platform.

Job Control Files

During the execution of a test the Real Load Platform can create and delete at runtime additional control files in the Data Output Directory of a test job. The existence, and respectively the absence of such control files must be frequently checked by the running script or program, but not too often to avoid CPU and I/O overload. Rule of thumb: Multi-threaded programs should check the existence of such files every 5..10 seconds. Single-threaded scripts should check such files before executing a new session loop iteration.

The following control files are created or removed in the Data Output Directory by the Real Load Platform:

  • DKFQS_Action_AbortTest.txt If the existence of this file is detected then the test executions must be aborted gracefully as soon as possible. Currently executed session loops are not aborted.
  • DKFQS_Action_SuspendTest.txt If the existence of this file is detected then the further execution of session loops is suspended until the file is removed by the Real Load Platform. Currently executed session loops are not interrupted on suspend. When resuming the test then the Ramp Up Time as passed as generic argument to the script or program must be re-applied. If a suspended test runs out of Test Duration then the test must end.

Testjob Data Files

When a test job is started by the Real Load Platform on a Measuring Agent, then the Real Load Platform creates at first for each simulated user an empty data file in the Data Output Directory of the test job:

Data File: user_<Executed User Number>_statistics.out

Example: user_1_statistics.out, user_2_statistics.out, user_3_statistics.out, .. et cetera.

After that, the test script(s) or test program is started as operating system process. The test script or the test program has to write the current state of the simulated user and measured data to the corresponding Data File of the simulated user in JSON object format (append data to the file only – don’t create new files).

The Real Load Platform component Measuring Agent and the corresponding Data Collector are listening to these data files and interpret the measured data at real-time, line by line as JSON objects.

“alt attribute”

Writing JSON Objects to the Data Files

The following JSON Objects can be written to the Data Files:

JSON Object Description
Declare Statistic Declare a new statistic
Register Execute Start Registers the start of a user
Register Execute Suspend Registers that the execution of a user is suspended
Register Execute Resume Registers that the execution of a user is resumed
Register Execute End Registers that a user has ended
Register Loop Start Registers that a user has started a session loop iteration
Register Loop Passed Registers that a session loop iteration of a user has passed
Register Loop Failed Registers that a session loop iteration of a user has failed
Register Sample Start Statistic-type sample-event-time-chart: Registers the start of measuring a sample
Add Sample Long Statistic-type sample-event-time-chart: Registers that a sample has measured and report the value
Add Sample Error Statistic-type sample-event-time-chart: Registers that the measuring of a sample has failed
Add Counter Long Statistic-type cumulative-counter-long: Add a positive delta value to the counter
Add Average Delta And Current Value Statistic-type average-and-current-value: Add delta values to the average and set the current value
Add Efficiency Ratio Delta Statistic-type efficiency-ratio-percent: Add efficiency ratio delta values
Add Throughput Delta Statistic-type throughput-time-chart: Add a delta value to a throughput
Add Error Add an error the test result
Add Test Result Annotation Exec Event Add an annotation event to the test result

Note that the data of each JSON object must be written as a single line which end with a \r\n line terminator.

Program Sequence

“alt attribute”

“alt attribute”

“alt attribute”

JSON Object Specification

Declare Statistic Object

Before the measurement of data begins, the corresponding statistics must be declared at runtime. Each declared statistic must have a unique ID. Multiple declarations with the same ID are crossed out by the platform.

Currently 5 types of statistics are supported:

  • sample-event-time-chart : This is the most common statistic type and contains continuously measured response times or any other continuously measured values of any unit. Information about failed measurements can also be added to the statistic. Statistics of this type are added to the ‘Overview Statistic’ area and can also displayed as a chart (see picture below).
  • cumulative-counter-long : This is a single counter whose value is continuously increased during the test. Statistics of this type are added to the ‘Test-Specific Values’ area.
  • average-and-current-value : This is a separately measured mean value and the last measured current value. Statistics of this type are added to the ‘Test-Specific Values’ area.
  • efficiency-ratio-percent : This is a measured efficiency in percent (0..100%). Statistics of this type are added to the ‘Test-Specific Values’ area.
  • throughput-time-chart : This is a measured throughput per second. Statistics of this type are added to the ‘Test-Specific Values’ area.

“alt attribute”

It’s also supported to declare new statistics at any time during test execution, but the statistics must be declared first, before the measured data are added.

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "DeclareStatistic",
  "type": "object",
  "required": ["subject", "statistic-id", "statistic-type", "statistic-title"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'declare-statistic'"
    },
    "statistic-id": {
      "type": "integer",
      "description": "Unique statistic id"
    },
    "statistic-type": {
      "type": "string",
      "description": "'sample-event-time-chart' or 'cumulative-counter-long' or 'average-and-current-value' or 'efficiency-ratio-percent' or 'throughput-time-chart'"
    },
    "statistic-title": {
      "type": "string",
      "description": "Statistic title"
    },
    "statistic-subtitle": {
      "type": "string",
      "description": "Statistic subtitle | only supported by 'sample-event-time-chart'"
    },
    "y-axis-title": {
      "type": "string",
      "description": "Y-Axis title | only supported by 'sample-event-time-chart'. Example: 'Response Time'"
    },
    "unit-text": {
      "type": "string",
      "description": "Text of measured unit. Example: 'ms'"
    },
    "sort-position": {
      "type": "integer",
      "description": "The UI sort position"
    },
    "add-to-summary-statistic": {
      "type": "boolean",
      "description": "If true = add the number of measured and failed samples to the summary statistic | only supported by 'sample-event-time-chart'. Note: Synthetic measured data like Measurement Groups or Delay Times should not be added to the summary statistic"
    },
    "background-color": {
      "type": "string",
      "description": "The background color either as #hex-triplet or as bootstrap css class name, or an empty string = no special background color. Examples: '#cad9fa', 'table-info'"
    }
  }
}

Example: 
{
  "subject":"declare-statistic",
  "statistic-id":1,
  "statistictype":"sample-event-time-chart",
  "statistic-title":"GET http://192.168.0.111/",
  "statistic-subtitle":"",
  "y-axis-title":"Response Time",
  "unit-text":"ms",
  "sort-position":1,
  "add-to-summarystatistic":true,
  "background-color":""
}

After the statistics are declared then the activities of the simulated users can be started. Each simulated user must report the following changes of the current user-state:

  • register-execute-start : Register that the simulated user has started the test.
  • register-execute-suspend : Register that the simulated user suspend the execution of the test.
  • register-execute-resume : Register that the simulated user resume the execution of the test.
  • register-execute-end : Register that the simulated user has ended the test.

Register Execute Start Object

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "RegisterExecuteStart",
  "type": "object",
  "required": ["subject", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'register-execute-start'"
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{"subject":"register-execute-start","timestamp":1596219816129}

Register Execute Suspend Object

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "RegisterExecuteSuspend",
  "type": "object",
  "required": ["subject", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'register-execute-suspend'"
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{"subject":"register-execute-suspend","timestamp":1596219816129}

Register Execute Resume Object

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "RegisterExecuteResume",
  "type": "object",
  "required": ["subject", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'register-execute-resume'"
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{"subject":"register-execute-resume","timestamp":1596219816129}

Register Execute End Object

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "RegisterExecuteEnd",
  "type": "object",
  "required": ["subject", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'register-execute-end'"
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{"subject":"register-execute-end","timestamp":1596219816129}

Once a simulated user has started its activity it measures the data in so called ‘session loops’. Each simulated must report when a session loop iteration starts and ends:

  • register-loop-start : Register the start of a session loop iteration.
  • register-loop-passed : Register that a session loop iteration has passed / at end of the session loop iteration.
  • register-loop-failed : Register that a session loop iteration has failed / if the session loop iteration is aborted.

Register Loop Start Object

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "RegisterLoopStart",
  "type": "object",
  "required": ["subject", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'register-loop-start'"
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{"subject":"register-loop-start","timestamp":1596219816129}

Register Loop Passed Object

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "RegisterLoopPassed",
  "type": "object",
  "required": ["subject", "loop-time", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'register-loop-passed'"
    },
    "loop-time": {
      "type": "integer",
      "description": "The time it takes to execute the loop in milliseconds"
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{"subject":"register-loop-passed","loop-time":1451, "timestamp":1596219816129}

Register Loop Failed Object

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "RegisterLoopFailed",
  "type": "object",
  "required": ["subject", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'register-loop-failed'"
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{"subject":"register-loop-failed","timestamp":1596219816129}

Within a session loop iteration the samples of the declared statistics are measured. For sample-event-time-chart statistics the simulated user must report when the measuring of a sample starts and ends:

  • register-sample-start : Register that the measuring of a sample has started.
  • add-sample-long : Add a measured value to a declared statistic.
  • add-sample-error : Add an error to a declared statistic.

Register Sample Start Object (sample-event-time-chart only)

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "RegisterSampleStart",
  "type": "object",
  "required": ["subject", "statistic-id", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'register-sample-start'"
    },
    "statistic-id": {
      "type": "integer",
      "description": "The unique statistic id"
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{"subject":"register-sample-start","statisticid":2,"timestamp":1596219816165}

Add Sample Long Object (sample-event-time-chart only)

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "AddSampleLong",
  "type": "object",
  "required": ["subject", "statistic-id", "value", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'add-sample-long'"
    },
    "statistic-id": {
      "type": "integer",
      "description": "The unique statistic id"
    },
    "value": {
      "type": "integer",
      "description": "The measured value"
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{"subject":"add-sample-long","statisticid":2,"value":105,"timestamp":1596219842468}

Add Sample Error Object (sample-event-time-chart only)

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "AddSampleError",
  "type": "object",
  "required": ["subject", "statistic-id", "error-subject", "error-severity",
  "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'add-sample-error'"
    },
    "statistic-id": {
      "type": "integer",
      "description": "The unique statistic id"
    },
    "error-subject": {
      "type": "string",
      "description": "The subject or title of the error"
    },
    "error-severity": {
      "type": "string",
      "description": "'warning' or 'error' or 'fatal'"
    },
    "error-type": {
      "type": "string",
      "description": "The type of the error. Errors which contains the same error
    type can be grouped."
    },
    "error-log": {
      "type": "string",
      "description": "The error log. Multiple lines are supported by adding \r\n line terminators."
    },
    "error-context": {
      "type": "string",
      "description": "Context information about the condition under which the error occurred. Multiple lines are supported by adding \r\n line terminators."
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{
  "subject":"add-sample-error",
  "statistic-id":2,
  "error-subject":"Connection refused (Connection refused)",
  "error-severity":"error",
  "error-type":"java.net.ConnectException",
  "error-log":"2020-08-01 21:24:51.662 | main-HTTPClientProcessing[3] | INFO | GET http://192.168.0.111/\r\n2020-08-01 21:24:51.670 | main-HTTPClientProcessing[3] | ERROR | Failed to open or reuse connection to 192.168.0.111:80 |
 java.net.ConnectException: Connection refused (Connection refused)\r\n",
  "error-context":"HTTP Request Header\r\nhttp://192.168.0.111/\r\nGET / HTTP/1.1\r\nHost: 192.168.0.111\r\nConnection: keep-alive\r\nAccept: */*\r\nAccept-Encoding: gzip, deflate\r\n",
  "timestamp":1596309891672
}

Note about the error-severity :

  • warning : After the error has occurred then the simulated user continues with the execution of the current session loop. Error color = yellow.
  • error : After the error has occurred then the simulated aborts the execution of the current session loop iteration, and starts the execution of the next session loop iteration. Error color = red.
  • fatal : After the error has occurred then the simulated user aborts any further execution of the test, which means that the test has ended for this simulated user. Error color = black.

Implementation note: After an error has occurred, the simulated user should wait at least 100 milliseconds before continuing his activities. This is to prevent that within a few seconds several thousand errors are measured and reported to the UI

Add Counter Long Object (cumulative-counter-long only)

For cumulative-counter-long statistics there is no such 2-step mechanism as for ‘sample-event-time-chart’ statistics. The value can simple increased by reporting a Add Counter Long object.

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "AddCounterLong",
  "type": "object",
  "required": ["subject", "statistic-id", "value"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'add-counter-long'"
    },
    "statistic-id": {
      "type": "integer",
      "description": "The unique statistic id"
    },
    "value": {
      "type": "integer",
      "description": "The value to increment"
    }
  }
}

Example: 
{"subject":"add-counter-long","statistic-id":10,"value":2111}

Add Average Delta And Current Value Object (average-and-current-value only)

To update a average-and-current-value statistic the delta (difference) values of the cumulated sum and the delta (difference) of the cumulated number of values has to be reported. The platform calculates then the average value by dividing the cumulated sum by the cumulated number of values. In addition, the last measured value must also be reported.

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "AddAverageDeltaAndCurrentValue",
  "type": "object",
  "required": ["subject", "statistic-id", "sumValuesDelta", "numValuesDelta", "currentValue", "currentValueTimestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'add-average-delta-and-current-value'"
    },
    "statistic-id": {
      "type": "integer",
      "description": "The unique statistic id"
    },
    "sumValuesDelta": {
      "type": "integer",
      "description": "The sum of delta values to add to the average"
    },
    "numValuesDelta": {
      "type": "integer",
      "description": "The number of delta values to add to the average"
    },
    "currentValue": {
      "type": "integer",
      "description": "The current value, or -1 if no such data is available"
    },
    "currentValueTimestamp": {
      "type": "integer",
      "description": "The Unix-like timestamp of the current value, or -1 if no such data is available"
    }
  }
}

Example: 
{
  "subject":"add-average-delta-and-current-value",
  "statistic-id":100005,
  "sumValuesDelta":6302,
  "numValuesDelta":22,
  "currentValue":272,
  "currentValueTimestamp":1634401774374
}

Add Efficiency Ratio Delta Object (efficiency-ratio-percent only)

To update a efficiency-ratio-percent statistic, the delta (difference) of the number of efficient performed procedures and the delta (difference) of the number of inefficient performed procedures has to be reported.

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "AddEfficiencyRatioDelta",
  "type": "object",
  "required": ["subject", "statistic-id", "efficiencyDeltaValue", "inefficiencyDeltaValue"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'add-efficiency-ratio-delta'"
    },
    "statistic-id": {
      "type": "integer",
      "description": "The unique statistic id"
    },
    "efficiencyDeltaValue": {
      "type": "integer",
      "description": "The number of efficient performed procedures to add"
    },
    "inefficiencyDeltaValue": {
      "type": "integer",
      "description": "The number of inefficient performed procedures to add"
    }
  }
}

Example: 
{
  "subject":"add-efficiency-ratio-delta",
  "statistic-id":100006,
  "efficiencyDeltaValue":6,
  "inefficiencyDeltaValue":22
}

Add Throughput Delta Object (throughput-time-chart only)

To update a throughput-time-chart statistic, the delta (difference) value from a last absolute, cumulated value to the current cumulated value has to be reported, whereby the current time stamp is included in the calculation.

Although this type of statistic always has the unit throughput per second, a measured delta (difference) value can be reported at any time.

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "AddThroughputDelta",
  "type": "object",
  "required": ["subject", "statistic-id", "delta-value", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'add-throughput-delta'"
    },
    "statistic-id": {
      "type": "integer",
      "description": "The unique statistic id"
    },
    "delta-value": {
      "type": "number",
      "description": "the delta (difference) value"
    },
    "timestamp": {
      "type": "integer",
      "description": "The Unix-like timestamp of the delta (difference) value"
    }
  }
}

Example: 
{
  "subject":"add-throughput-delta",
  "statistic-id":100003,
  "delta-value":0.53612,
  "timestamp":1634401774410
}

Add Error Object

Add an error to the test result.

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "AddError",
  "type": "object",
  "required": ["subject", "statistic-id", "error-subject", "error-severity",
  "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'add-error'"
    },
    "statistic-id": {
      "type": "integer",
      "description": "The unique statistic id, or -1 if this error is not bound to any statistic"
    },
    "error-subject": {
      "type": "string",
      "description": "The subject or title of the error"
    },
    "error-severity": {
      "type": "string",
      "description": "'warning' or 'error' or 'fatal'"
    },
    "error-type": {
      "type": "string",
      "description": "The type of the error. Errors which contains the same error type can be grouped."
    },
    "error-log": {
      "type": "string",
      "description": "The error log. Multiple lines are supported by adding \r\n line terminators."
    },
    "error-context": {
      "type": "string",
      "description": "Context information about the condition under which the error occurred. Multiple lines are supported by adding \r\n line terminators."
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{
  "subject":"add-error",
  "statistic-id":-1,
  "error-subject":"Connection refused (Connection refused)",
  "error-severity":"error",
  "error-type":"java.net.ConnectException",
  "error-log":"2020-08-01 21:24:51.662 | main-HTTPClientProcessing[3] | INFO | GET http://192.168.0.111/\r\n2020-08-01 21:24:51.670 | main-HTTPClientProcessing[3] | ERROR | Failed to open or reuse connection to 192.168.0.111:80 |
 java.net.ConnectException: Connection refused (Connection refused)\r\n",
  "error-context":"HTTP Request Header\r\nhttp://192.168.0.111/\r\nGET / HTTP/1.1\r\nHost: 192.168.0.111\r\nConnection: keep-alive\r\nAccept: */*\r\nAccept-Encoding: gzip, deflate\r\n",
  "timestamp":1596309891672
}

Note: Do not use this error object for sample-event-time-chart(s).

Add Test Result Annotation Exec Event Object

Add an annotation event to the test result.

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "AddTestResultAnnotationExecEvent",
  "type": "object",
  "required": ["subject", "event-id", "event-text", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'add-test-result-annotation-exec-event'"
    },
    "event-id": {
      "type": "integer",
      "description": "The event id, valid range: -1 .. -999999"
    },
    "event-text": {
      "type": "string",
      "description": "the event text"
    },
    "timestamp": {
      "type": "integer",
      "description": "The Unix-like timestamp of the event"
    }
  }
}

Example: 
{
  "subject":"add-test-result-annotation-exec-event",
  "event-id":-1,
  "event-text":"Too many errors: Test job stopped by plug-in",
  "timestamp":1634401774410
}

Notes:

  • The event id must be in the range from -1 (minus one) to -999999.
  • Events with the same event id are merged to one event.

[End of Interface Specification]

Example

HTTP Test Wizard Plug-In

This plug-in “measures” a random value, and is executed in this example as the only part of an HTTP Test Wizard session.

The All Purpose Interface JSON objects are written using the corresponding methods of the com.dkfqs.tools.javatest.AbstractJavaTest class. This class is located in the JAR file com.dkfqs.tools.jar which is already predefined for all plug-ins.

import com.dkfqs.tools.javatest.AbstractJavaTest;
import com.dkfqs.tools.javatest.AbstractJavaTestPluginContext;
import com.dkfqs.tools.javatest.AbstractJavaTestPluginInterface;
import com.dkfqs.tools.javatest.AbstractJavaTestPluginSessionFailedException;
import com.dkfqs.tools.javatest.AbstractJavaTestPluginTestFailedException;
import com.dkfqs.tools.javatest.AbstractJavaTestPluginUserFailedException;
import com.dkfqs.tools.logging.LogAdapterInterface;
import java.util.ArrayList;
import java.util.List;
// add your imports here

/**
 * HTTP Test Wizard Plug-In 'All Purpose Interface Example'.
 * Plug-in Type: Normal Session Element Plug-In.
 * Created by 'DKF' at 24 Sep 2021 22:50:04
 * DKFQS 4.3.22
 */
@AbstractJavaTestPluginInterface.PluginResourceFiles(fileNames={"com.dkfqs.tools.jar"})
public class AllPurposeInterfaceExample implements AbstractJavaTestPluginInterface {
	private LogAdapterInterface log = null;
	
	private static final int STATISTIC_ID = 1000;
	private AbstractJavaTest javaTest = null;       // refrence to the generated test program
	
	/**
	 * Called by environment when the instance is created.
	 * @param log the log adapter
	 */
	@Override
	public void setLog(LogAdapterInterface log) {
		this.log = log;
	}
	
	/**
	 * On plug-in initialize. Called when the plug-in is initialized. <br>
	 * Depending on the initialization scope of the plug-in the following specific exceptions can be thrown:<ul>
	 * 	<li>Initialization scope <b>global:</b> AbstractJavaTestPluginTestFailedException</li>
	 * 	<li>Initialization scope <b>user:</b> AbstractJavaTestPluginTestFailedException, AbstractJavaTestPluginUserFailedException</li>
	 * 	<li>Initialization scope <b>session:</b> AbstractJavaTestPluginTestFailedException, AbstractJavaTestPluginUserFailedException, AbstractJavaTestPluginSessionFailedException</li>
	 * </ul>
	 * @param javaTest the reference to the executed test program, or null if no such information is available (in debugger environment)
	 * @param pluginContext the plug-in context
	 * @param inputValues the list of input values
	 * @return the list of output values
	 * @throws AbstractJavaTestPluginSessionFailedException if the plug-in signals that the 'user session' has to be aborted (abort current session - continue next session)
	 * @throws AbstractJavaTestPluginUserFailedException if the plug-in signals that the user has to be terminated
	 * @throws AbstractJavaTestPluginTestFailedException if the plug-in signals that the test has to be terminated
	 * @throws Exception if an error occurs in the implementation of this method
	 */
	@Override
	public List<String> onInitialize(AbstractJavaTest javaTest, AbstractJavaTestPluginContext pluginContext, List<String> inputValues) throws AbstractJavaTestPluginSessionFailedException, AbstractJavaTestPluginUserFailedException, AbstractJavaTestPluginTestFailedException, Exception {
		// log.message(log.LOG_INFO, "onInitialize(...)");
		
		// --- vvv --- start of specific onInitialize code --- vvv ---
		if (javaTest != null) {
		    this.javaTest = javaTest;
		    
		    // declare the statistic
		    javaTest.declareStatistic(STATISTIC_ID, 
            		                  AbstractJavaTest.STATISTIC_TYPE_SAMPLE_EVENT_TIME_CHART,
            		                  "My Measurement",
            		                  "",
            		                  "My Response Time",
            		                  "ms",
            		                  STATISTIC_ID,
            		                  true,
            		                  "");
		}
		// --- ^^^ --- end of specific onInitialize code --- ^^^ ---
		
		return new ArrayList<String>();		// no output values
	}

	/**
	 * On plug-in execute. Called when the plug-in is executed. <br>
	 * Depending on the execution scope of the plug-in the following specific exceptions can be thrown:<ul>
	 * 	<li>Initialization scope <b>global:</b> AbstractJavaTestPluginTestFailedException</li>
	 * 	<li>Initialization scope <b>user:</b> AbstractJavaTestPluginTestFailedException, AbstractJavaTestPluginUserFailedException</li>
	 * 	<li>Initialization scope <b>session:</b> AbstractJavaTestPluginTestFailedException, AbstractJavaTestPluginUserFailedException, AbstractJavaTestPluginSessionFailedException</li>
	 * </ul>
	 * @param pluginContext the plug-in context
	 * @param inputValues the list of input values
	 * @return the list of output values
	 * @throws AbstractJavaTestPluginSessionFailedException if the plug-in signals that the 'user session' has to be aborted (abort current session - continue next session)
	 * @throws AbstractJavaTestPluginUserFailedException if the plug-in signals that the user has to be terminated
	 * @throws AbstractJavaTestPluginTestFailedException if the plug-in signals that the test has to be terminated
	 * @throws Exception if an error occurs in the implementation of this method
	 */
	@Override
	public List<String> onExecute(AbstractJavaTestPluginContext pluginContext, List<String> inputValues) throws AbstractJavaTestPluginSessionFailedException, AbstractJavaTestPluginUserFailedException, AbstractJavaTestPluginTestFailedException, Exception {
		// log.message(log.LOG_INFO, "onExecute(...)");
		
		// --- vvv --- start of specific onExecute code --- vvv ---
		if (javaTest != null) {
		    
		    // register the start of the sample 
		    javaTest.registerSampleStart(STATISTIC_ID);
		    
		    // measure the sample
		    final long min = 1L;
		    final long max = 20L;
		    long responseTime = Math.round(((Math.random() * (max - min)) + min));
		    
		    // add the measured sample to the statistic
		    javaTest.addSampleLong(STATISTIC_ID, responseTime);
		    
		    /*
		    // error case
		    javaTest.addSampleError(STATISTIC_ID,
                                    "My error subject",
                                    AbstractJavaTest.ERROR_SEVERITY_WARNING,
                                    "My error type",
                                    "My error response text or log",
                                    "");
            */
		}
		// --- ^^^ --- end of specific onExecute code --- ^^^ ---
		
		return new ArrayList<String>();		// no output values
	}

	/**
	 * On plug-in deconstruct. Called when the plug-in is deconstructed.
	 * @param pluginContext the plug-in context
	 * @param inputValues the list of input values
	 * @return the list of output values
	 * @throws Exception if an error occurs in the implementation of this method
	 */
	@Override
	public List<String> onDeconstruct(AbstractJavaTestPluginContext pluginContext, List<String> inputValues) throws Exception {
		// log.message(log.LOG_INFO, "onDeconstruct(...)");
		
		// --- vvv --- start of specific onDeconstruct code --- vvv ---
		// no code here
		// --- ^^^ --- end of specific onDeconstruct code --- ^^^ ---
		
		return new ArrayList<String>();		// no output values
	}

}

“alt attribute”

“alt attribute”

Debugging the Interface

  1. In order to debug the processing of the reported data of the interface, activate the “Debug Measuring” checkbox when starting the test job.
  2. After the test job has completed, select in the Test Jobs menu at the corresponding test job the option “Job Log Files” and then select the file “DataCollector.out”.
  3. Review the “DataCollector.out” file for any errors. Lines which contains “| Tailer data” reflect the raw reported data.

“alt attribute”

“alt attribute”

12.1 - Developing a JUnit Monitoring Test

This example shows a JUnit Test that executes a DNS query to resolve and verify a hostname of a domain.

Abstract

This example shows a JUnit Test that executes a DNS query to resolve a hostname of a domain. In addition, the received IP V4 address is verified. You can use this example to verify that a DNS hostname is currently defined and pointing to the correct IP V4 address.

If you scroll through the code below, you will notice that the Java class is not part of a Java Package and extends AbstractJUnitTest, and that there is small code at @Before and @After. This is necessary so that the JUnit test can run on the RealLoad infrastructure.

The code at @Test corresponds to a normally programmed JUnit test, with the only exception that a special logger is used (class MemoryLogAdapter, method log.message).

TestDnsARecord.java[Source Code]

To compile the code, the JAR files com.dkfqs.tools.jar and junit-4.13.2.jar are required. The Java Doc of com.dkfqs.tools is published at https://download.realload.com

A simple test case was deliberately chosen so that you can become familiar with the RealLoad product. We recommend that you carry out the steps described here yourself. After you know how to get a JUnit test running, you can create any JUnit test yourself, or migrate existing JUnit tests to RealLoad - simply by replacing the @Test method of this example.

1. Create the File TestDnsARecord.java in the Project Tree

You must first create an empty file named TestDnsARecord.java in a Project’s ‘Resource Set’. ‘Resource Sets’ are something like sub-directories of a project, which contain all the files necessary to define and execute a test. New Projects and Resource Sets can be created in the Projects Menu.

Create a new Project and Resource Set: JUnit Tests / Test DNS A Record. Create new Project with Resource Set

Create a new (empty) file: TestDnsARecord.java in the Resource Set. Create Empty File

After the empty file has been created, click on ‘Edit File’ and paste the source code shown above into the editor. Edit File

Then save the file and close the editor. Save File

2. Compile the TestDnsARecord.java File and Define the RealLoad ‘Test’

Click on the ‘Compile Java File’ icon. Compile File

Select the JAR files com.dkfqs.tools.jar and junit-4.13.2.jar and click the ‘Compile’ button. Compile File

After the file has been successfully compiled click on ‘Define or Update JUnit Test’. Define or Update JUnit Test

Select the @Test method (in this case resolveARecord) and click ‘Select’. Leave the switches at ‘Additional Required JAR Libraries’ as shown (default setting) and then click on ‘Define Test’. Define Test

Enter the ‘Test Description’ and click on ‘Define Test’. Define Test

The RealLoad ‘Test’ is now defined. From here you can now create both a Load Test Job and a Monitoring Job. RealLoad ‘Test’ is Defined

3. Verify the Test

To verify that the test works correctly, first start it as a Load Test Job with only one simulated user and one loop. Click ‘Define Test Job’. Define Test Job

Click ‘Continue’. Define Load Test Job

Select the Measuring Agent on which the Load Test Job will be executed and turn on the switch ‘Debug Execution’. Then click on ‘Define Load Test Job’. Define Load Test Job

The Load Test Job is then created and in the state ‘Defined’. Click on ‘Start Test Job’. Start Test Job

The settings of the Load Test Job are displayed, which can also be modified here (not necessary in this case). Click ‘Start Test Job’. Start Test Job

Wait a few seconds until you receive the notification that the Load Test Job has completed. Then delete the notification. Load Test Job completed Notification

From the Load Test Job drop-down menu, select ‘Job Log Files’. Select ‘Job Log Files’

Select the Job Log File ‘users.out’. Select ‘users.out’ File

If no error or Java stack trace is displayed, the Load Test Job was executed successfully. Check ‘users.out’ File

4. Troubleshooting a Test

If the job log file users.out shows an error, you need to modify the Java code of the test, or you need to add additional JAR files required to run the test. After editing the Java code, compile the Java file again and then click the ‘Define or Update JUnit Test’ button after successful compilation. Then jump directly into the Load Test Jobs menu, ‘clone’ the Load Test Job and run it again. ‘Clone’ the Load Test Job

5. Define the Monitoring Job

If this is your first Monitoring Job, you must first create a ‘Monitoring Group’. Navigate to Monitoring, click the ‘Configuration’ tab and then click ‘Add Monitoring Group’. Note: There is also separate help for monitoring. Navigate to Monitoring Configuration

Enter the ‘Group Title’ and select at least one ‘Measuring Agent’ on which the Monitoring Job(s) will be executed. Then click ‘Add Monitoring Group’. Add Monitoring Group

In the ‘Monitoring Group’ click ‘Monitoring Jobs’ and then click ‘Add Monitoring Job’. Add Monitoring Job

Select the ‘Test’ of the Monitoring Job. Select ‘Test’ of Monitoring Job

Turn on the switch ‘Debug Execution’ and click ‘Define Monitoring Job’. Define Monitoring Job

Enable the execution of the Monitoring Group and navigate to ‘Dashboard’. Enable Monitoring Group and Navigate to Dashboard

The Monitoring Job is now defined and will be executed periodically. For additional help configuring monitoring (e.g. adding ‘Alert Devices’), see Monitoring Help. Monitoring Job is Defined

6. Optional: Create a Generic DNS A-Record Test

As you saw in the code before, the following 3 variables are defined as constants:

private static final String dnsServer = "8.8.8.8";  // use Google LLC public DNS server
private static final String dnsHostnameToResolve = "www.realload.com";  // the DNS host name to resolve
private static final String expectedIpV4Address = "83.150.39.46";       // the expected IP V4 address of the resolved host name

This section now describes how you can dynamically initialize these variables using User Input Fields.

The ‘User Input Fields’ values can be entered when starting a Load Test Job and when defining a Monitoring Job. This makes the test reusable, i.e. you can add the same ‘Test’ multiple times to multiple Monitoring Jobs, but use different values for the ‘User Input Fields’ for each Monitoring Job.

First you have to create a file that contains the definitions of the User Input Fields. To do this, invoke the User Input Fields Wizard. Invoke the ‘User Input Fields Wizard’

Enter (add) the 3 User Input Fields and save the file at JUnit Tests / Test DNS A Record / InputFields_TestDnsARecordGeneric.json

GUI Label Variable Name Input Type Default Value
DNS Server dnsServer String 8.8.8.8
Hostname to Resolve dnsHostnameToResolve String www.realload.com
Expected IP Address expectedIpV4Address String 83.150.39.46

Add the 3 User Input Fields

Then copy the file TestDnsARecord.java to TestDnsARecordGeneric.java and then edit TestDnsARecordGeneric.java Copy Java File

Copy Java File

Edit TestDnsARecordGeneric.java

At the beginning of the Java class replace the class name and modify the 3 variables:

public class TestDnsARecordGeneric extends AbstractJUnitTest
{
    private String dnsServer;  // example: 8.8.8.8
    private String dnsHostnameToResolve; // example: www.realload.com
    private String expectedIpV4Address; // example: 83.150.39.46

Replace also the class name in the constructor:

    /**
     * Constructor.
     */
    public TestDnsARecordGeneric() {
        // disable DNSJava to search for default DNS servers (DNSJava is integrated in com.dkfqs.tools)
        System.setProperty("dnsjava.noDefaultDnsServers", "true");
    }

And modify the code @Before as follows:

    /**
    * Prepare the test.
    */
    @Before
    public void setUp() {
        if (isArgDebugExecution()) {
            log.setLogLevel(LOG_DEBUG);
        }

        // get the user input fields
        dnsServer = getUserInputField("dnsServer", null);
        if (dnsServer == null) {
            throw new RuntimeException("User input field 'dnsServer' missing");
        }
        dnsHostnameToResolve = getUserInputField("dnsHostnameToResolve", null);
        if (dnsHostnameToResolve == null) {
            throw new RuntimeException("User input field 'dnsHostnameToResolve' missing");
        }
        expectedIpV4Address = getUserInputField("expectedIpV4Address", null);
        if (expectedIpV4Address == null) {
            throw new RuntimeException("User input field 'expectedIpV4Address' missing");
        }

        openAllPurposeInterface();
    }

Then save and compile TestDnsARecordGeneric.java Compile TestDnsARecordGeneric.java

At ‘Define Test’ turn on the switch InputFields_TestDnsARecordGeneric.json at ‘Additional Required Resource Files’. Turn on Switch InputFields_TestDnsARecordGeneric.json

Define the Test and the corresponding Load Test Job as usual, and then start the Load Test Job. You can now enter the values for the 3 User Input Fields. Load Test Job with User Input Fields

Check the log file users.out after the Load Test Job is completed. Check Job Output File

When defining a Monitoring Job, you can also enter the values for the User Input Fields. Monitoring Job with User Input Fields

Final Dashboard

7. Conclusion and Prospects

As you have seen, a JUnit Test can be run as both a Load Test Job and a Monitoring Job.

Additionally, a Test Job Template can also be defined from any ‘Load Test Job’, which can then be part of a Test Suite that is executed as Regression Test. This means that you can add multiple JUnit Tests to a Test Suite and execute them in a single run as a Regression Test.

Last but not least, note that JUnit Tests can also be executed with many, up to several thousands, virtual users - for Monitoring Jobs, for Load Tests and for Regression Tests. This is a generic feature of the RealLoad architecture which applies to all kind of RealLoad ‘Tests’.

13 - Installation

How to install RealLoad

Usually you just need a Portal Server account and to install the “Desktop Companion” on your laptop which enables you to record and upload HTTP/S sessions and to launch Measuring Agents on Amazon EC2. You don’t need a special installation license for the Desktop Companion. However, user licenses are required for using the Portal Server and to perform tests on Measuring Agents - see https://shop.realload.com/.

The RealLoad components Measuring Agent(s), Cluster Controller(s) and Remote Proxy Recorder(s) can also installed/operated on your own hosted machines.

The installation and operation of an own dedicated Portal Server requires a contract with us and a special, commercial license.

The software can be downloaded from https://download.realload.com

Prerequisites

Supported operating systems / for all RealLoad components

  • Windows 10 / Windows Server 2012 or newer.
  • Centos 8
  • Red Hat Linux 6
  • Ubuntu 16, 18, 20 or 22
  • OS X

Installation

Install using the installer

Desktop Companion Windows installer: https://download.realload.com/desktop_companion/latest_win64

Manual installation

Follow the links below to perform a manual installation.

13.1 - Ubuntu: Measuring Agent manual install

Ubuntu: Measuring Agent Install Instructions

Prerequisites

Supported Hardware

  • Amazon EC2 Cloud instances, or
  • Own hosted Servers with any Intel or AMD CPU, or
  • Own hosted Raspberry Pi 4 Model B / 8 GB (ARM CPU) / Ubuntu 20 only / for weak load tests up to max. 100 concurrent users (with loop iteration delay = 1000 ms)

Minimum Requirements

  • Minimum required CPU Cores of Processor: 4
  • Minimum required Memory: 8 GB
  • Minimum required Disk: 64 GB
  • Minimum required Network Speed: 100 Mbps (1000 Mbps or faster strongly recommended)

Usual Requirements

  • Suggested Hardware for performing load tests up to 500 concurrent users: Intel CPU i3 / 16 GB Memory / Disk: 256 GB
  • Suggested Hardware for performing load tests up to 1000 concurrent users: Intel CPU i5 / 16 GB Memory / Disk: 512 GB
  • Suggested Hardware for performing load tests up to 5000 concurrent users: Intel CPU i7 / 64 GB Memory / Disk: 1024 GB

Rule of Thumb for Amazon EC2 Instances

  • Per EC2 vCPU, 100 virtual users can simulated
  • Required Memory: 5 GB + (1 GB per 100 virtual users)

Environment and Location

Tests performed from ‘Measuring Agents’ which are virtualized or which run in a container environment measure often incorrect results. Because additional CPU and Network delays occur at virtualization/container level. It’s recommended that you use BARE-METAL-SERVERS to perform your tests. Alternatively you can also use Amazon EC2 Cloud instances.

You can place your ‘Measuring Agents’ at any location (anywhere at the internet or inside your local DMZ). Depending on which kind of traffic you have to test. Note that your Measuring Agents - usually running on TCP/IP port 8080 (HTTPS) - must be reachable form the ‘Portal Server’, and that you have to enable the corresponding inbound firewall rule.

Network & System Tuning

In /etc/sysctl.conf add:

# TCP/IP Tuning
# =============
fs.file-max = 524288
net.ipv4.tcp_max_syn_backlog = 2048
net.ipv4.tcp_synack_retries = 3
net.ipv4.tcp_max_orphans = 65536
net.ipv4.tcp_fin_timeout = 30
net.ipv4.ip_local_port_range = 16384 60999
net.core.somaxconn = 256
net.core.rmem_max = 1048576
net.core.wmem_max = 1048576

in /etc/security/limits.conf add:

# TCP/IP Tuning
# =============
* soft     nproc          262140
* hard     nproc          262140
* soft     nofile         262140
* hard     nofile         262140
root soft     nproc          262140
root hard     nproc          262140
root soft     nofile         262140
root hard     nofile         262140

Enter: systemctl show -p TasksMax user-0

output: TasksMax=8966

if you get a value less than 262140 then add in /etc/systemd/system.conf

# Ubuntu Tuning
# =============
DefaultTasksMax=262140

Reboot the system and verify the settings. Enter: ulimit -n

output: 262140

Enter: systemctl show -p TasksMax user-0

output: TasksMax=262140

Install Dependencies

Install haveged

sudo apt-get update
sudo apt-get install haveged

Configure the UFW Firewall (optional)

sudo ufw allow ssh
sudo ufw allow 8080/tcp
sudo ufw logging off
sudo ufw enable

Enter: sudo ufw status verbose

Status: active
Logging: off
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW IN    Anywhere
8080/tcp                   ALLOW IN    Anywhere
22/tcp (v6)                ALLOW IN    Anywhere (v6)
8080/tcp (v6)              ALLOW IN    Anywhere (v6)

Install OpenJDK Java 8 and 11 / For Intel and AMD CPUs

Get the Java Installation Kits

wget https://download.java.net/openjdk/jdk8u41/ri/openjdk-8u41-b04-linux-x64-14_jan_2020.tar.gz
wget https://download.java.net/java/GA/jdk11/13/GPL/openjdk-11.0.1_linux-x64_bin.tar.gz

Install OpenJDK Java 8

gunzip openjdk-8u41-b04-linux-x64-14_jan_2020.tar.gz
tar -xvf  openjdk-8u41-b04-linux-x64-14_jan_2020.tar
rm openjdk-8u41-b04-linux-x64-14_jan_2020.tar
sudo bash *******
mkdir /opt/OpenJDK
mv java-se-8u41-ri /opt/OpenJDK
cd /opt/OpenJDK
ls -al
chown root -R java-se-8u41-ri
chgrp root -R java-se-8u41-ri
exit # end sudo bash

Verify the Java 8 installation.

/opt/OpenJDK/java-se-8u41-ri/bin/java -version

openjdk version "1.8.0_41"
OpenJDK Runtime Environment (build 1.8.0_41-b04)
OpenJDK 64-Bit Server VM (build 25.40-b25, mixed mode)

Install OpenJDK Java 11

gunzip openjdk-11.0.1_linux-x64_bin.tar.gz
tar -xvf openjdk-11.0.1_linux-x64_bin.tar
rm openjdk-11.0.1_linux-x64_bin.tar
sudo bash
mv jdk-11.0.1 /opt/OpenJDK
cd /opt/OpenJDK
ls -al
chown root -R jdk-11.0.1
chgrp root -R jdk-11.0.1

Execute the following commands (still as sudo bash):

update-alternatives --install "/usr/bin/java" "java" "/opt/OpenJDK/jdk-11.0.1/bin/java" 1
update-alternatives --install "/usr/bin/javac" "javac" "/opt/OpenJDK/jdk-11.0.1/bin/javac" 1
update-alternatives --install "/usr/bin/keytool" "keytool" "/opt/OpenJDK/jdk-11.0.1/bin/keytool" 1
update-alternatives --install "/usr/bin/jar" "jar" "/opt/OpenJDK/jdk-11.0.1/bin/jar" 1
update-alternatives --set "java" "/opt/OpenJDK/jdk-11.0.1/bin/java"
update-alternatives --set "javac" "/opt/OpenJDK/jdk-11.0.1/bin/javac"
update-alternatives --set "keytool" "/opt/OpenJDK/jdk-11.0.1/bin/keytool"
update-alternatives --set "jar" "/opt/OpenJDK/jdk-11.0.1/bin/jar"
exit # end sudo bash

Verify the Java 11 installation.

java -version

openjdk version "11.0.1" 2018-10-16
OpenJDK Runtime Environment 18.9 (build 11.0.1+13)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.1+13, mixed mode)

Install OpenJDK Java 8 and 11 / For Raspberry Pi 4 Model B / ARM CPU

sudo apt install openjdk-8-jre-headless
sudo apt install openjdk-8-jdk-headless
sudo apt install openjdk-11-jre-headless
sudo apt install openjdk-11-jdk-headless

Verify the Java installation.

java -version

openjdk version "11.0.10" 2021-01-19
OpenJDK Runtime Environment (build 11.0.10+9-Ubuntu-0ubuntu1.20.10)
OpenJDK 64-Bit Server VM (build 11.0.10+9-Ubuntu-0ubuntu1.20.10, mixed mode)

Install PowerShell (optional)

You only need to install powershell if you run load tests with powershell scripts.

# Install PowerShell
sudo snap install powershell --classic

# Start PowerShell
pwsh
exit

Install Ubuntu Desktop for Selenium (optional)

Install the Display Manager

sudo apt install slim

Install Ubuntu Desktop (which includes also Firefox)

sudo apt install ubuntu-desktop

Then reboot the machine.

Install Xvfb

sudo apt-get install xvfb

Install Chrome

wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
sudo sh -c 'echo "deb https://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list'
sudo apt-get update
sudo apt-get install google-chrome-stable

Install the Measuring Agent

Create the DKFQS account which is running the Measuring Agent

sudo adduser dkfqs    # follow the questions, remember or write down the password

Install the Measuring Agent

Login with the dkfqs account (SSH) - or - Enter: sudo -u dkfqs bash | OR: Install Samba to get convenient access to /home/dkfqs as Samba dkfqs user

Create the directory /home/dkfqs/agent (as dkfqs user):

cd /home/dkfqs
mkdir agent

Create the following sub-directories at /home/dkfqs/agent (as dkfqs user):

  • bin
  • config
  • internalData
  • log
  • scripts
  • usersData
cd /home/dkfqs/agent
mkdir bin config internalData log scripts usersData

Copy the following files to the bin directory /home/dkfqs/agent/bin

  • bcpkix-jdk15on-160.jar
  • bcprov-jdk15on-160.jar
  • bctls-jdk15on-160.jar
  • DKFQSMeasuringAgent.jar
  • chromedriver (optional, for Selenium)
  • geckodriver (optional, for Selenium)
chmod 755 chromedriver
chmod 755 geckodriver

Copy the following files to the config directory /home/dkfqs/agent/config

  • datacollector.properties
  • measuringagent.properties

Modify the measuringagent.properties file. Set the following properties:

  • HttpsCertificateCN (set the public DNS name or the IP address for the automatically generated SSL/TLS server certificate)
  • HttpsCertificateIP (set the public IP address for the automatically generated SSL/TLS server certificate)
  • PowerShellCore6Path
  • OpenJDK8JavaPath
  • OpenJDK8JavaJobDefaultXmx (set around 20% of total OS memory - example: 1024m)
  • OpenJDK11JavaPath
  • OpenJDK11JavaJobDefaultXmx (set around 20% of total OS memory - example: 1024m)

Example: datacollector.properties

# local TCP/HTTPS data collector ports
DataCollectorPortStartRange=44444
DataCollectorPortEndRange=45000
DataCollectorPortExcludeList=

LogLevel=info
MaxLifeTimeMinutes=240

MaxWebSocketConnectTimeSeconds=14400
MaxInboundWebSocketTrafficPerConnection=67108864
MaxInboundWebSocketPayloadPerFrame=1048576
MaxInboundWebSocketFramesPerIPTimeFrame=10
MaxInboundWebSocketFramesPerIPLimit=1000

RealtimeStatisticsSamplingGranularityMillis=4000

Example: measuringagent.properties

HttpsPort=8080
HttpsCertificateCN=agent2.realload.com
HttpsCertificateIP=83.150.39.43
LogLevel=info

# AuthTokenEnabled: true or false, if true = the AuthTokenValue must be configured at portal server measuring agent settings
AuthTokenEnabled=false
# If AuthTokenEnabled is true, but AuthTokenValue is undefined or an empty string, then the (permanent) AuthTokenValue is automatically generated and printed at the log output
# AuthTokenValue=

MeasuringAgentLogFile=/home/dkfqs/agent/log/MeasuringAgent.log
MeasuringAgentInternalDataDirectory=/home/dkfqs/agent/internalData
MeasuringAgentUsersDataRootDirectory=/home/dkfqs/agent/usersData

ApiV1MaxRequestSizeMB=256
ApiV1WorkerThreadBusyTimeoutSeconds=330
ApiV1WorkerThreadExecutionTimeoutSeconds=300

MaxWebSocketConnectTimeSeconds=14400
MaxInboundWebSocketTrafficPerConnection=160000000
MaxInboundWebSocketPayloadPerFrame=80000000
MaxInboundWebSocketFramesPerIPTimeFrame=10
MaxInboundWebSocketFramesPerIPLimit=1000

DataCollectorProcessJavaPath=java
DataCollectorProcessJavaXmx=512m
DataCollectorPropertiesPath=/home/dkfqs/agent/config/datacollector.properties

# Settings for Supported Scripts / Programming Languages
PowerShellCore6Path=/snap/bin/pwsh
OpenJDK8JavaPath=/opt/OpenJDK/java-se-8u41-ri/bin/java
OpenJDK8JavaJobDefaultXmx=512m
OpenJDK11JavaPath=/opt/OpenJDK/jdk-11.0.1/bin/java
OpenJDK11JavaJobDefaultXmx=512m

# Limits
# LimitMaxUsersPerJob=500
# LimitMaxJobDurationSeconds=300

# Settings for Synthetic Monitoring
SyntheticMonitoringDataCollectorProcessJavaXmx=128m
SyntheticMonitoringOpenJDK8JavaJobXmx=256m
SyntheticMonitoringOpenJDK11JavaJobXmx=256m
SyntheticMonitoringLimitMaxUsersPerJob=5
SyntheticMonitoringLimitMaxJobDurationSeconds=300
SyntheticMonitoringLimitMaxLoopsPerUser=5

# Settings for Tests Jobs which require a (virtual) display. If VirtualDisplayPerRemoteUserIdEnabled = false then the value of DebugDisplay is used
DebugDisplay=:0.0
VirtualDisplayPerRemoteUserIdEnabled=true
VirtualDisplayType=Xvfb
XvfbPath=Xvfb
SeleniumTestJobsEnabled=true
SeleniumLimitMaxUsersPerJob=20
SeleniumWebBrowserTypesSupported=Chrome,Firefox
SeleniumChromeDriverPath=/home/dkfqs/agent/bin/chromedriver
SeleniumGeckoDriverPath=/home/dkfqs/agent/bin/geckodriver
SeleniumEdgeDriverPath=

First Test - Start the Measuring Agent manually (as dkfqs user)

cd /home/dkfqs/agent/bin
export CLASSPATH=bcpkix-jdk15on-160.jar:bcprov-jdk15on-160.jar:bctls-jdk15on-160.jar:DKFQSMeasuringAgent.jar
java -Xmx512m -DdkfqsMeasuringAgentProperties=../config/measuringagent.properties -Dnashorn.args="--no-deprecation-warning" com.dkfqs.measuringagent.internal.StartDKFQSMeasuringAgent

Data Collector service port range from 44444 to 45000
LimitMaxUsersPerJob = unlimited
LimitMaxJobDurationSeconds = unlimited
X509 TLS server certificate generated for CN = 192.168.0.51
Internal RSA 2048 bit keypair generated in 373 ms
2021-03-11 18:20:27.947 | QAHTTPd | WARN | QAHTTPd V1.3-U started
2021-03-11 18:20:27.990 | QAHTTPd | INFO | HTTPS server starting at port 8080
2021-03-11 18:20:28.089 | QAHTTPd | INFO | HTTPS server ready at port 8080

Create the Measuring Agent Startup Script (as root)

sudo bash # become root
cd /etc/init.d
vi MeasuringAgent

Edit - create /etc/init.d/MeasuringAgent

#!/bin/sh
# /etc/init.d/MeasuringAgent
# install with: update-rc.d MeasuringAgent defaults

### BEGIN INIT INFO
# Provides:          MeasuringAgent
# Required-Start:    $local_fs $network $time $syslog
# Required-Stop:     $local_fs $network
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start MeasuringAgent daemon at boot time
# Description:       MeasuringAgent daemon
### END INIT INFO

case "$1" in
  start)
    if [ -f /home/dkfqs/agent/log/MeasuringAgent.log ]; then
       mv /home/dkfqs/agent/log/MeasuringAgent.log /home/dkfqs/agent/log/MeasuringAgent.log_$(date +"%Y_%m_%d_%H_%M")
    fi
    sudo -H -u dkfqs bash -c 'CLASSPATH=/home/dkfqs/agent/bin/bcpkix-jdk15on-160.jar:/home/dkfqs/agent/bin/bcprov-jdk15on-160.jar:/home/dkfqs/agent/bin/bctls-jdk15on-160.jar:/home/dkfqs/agent/bin/DKFQSMeasuringAgent.jar;export CLASSPATH;nohup java -Xmx512m -DdkfqsMeasuringAgentProperties=/home/dkfqs/agent/config/measuringagent.properties -Dnashorn.args="--no-deprecation-warning" com.dkfqs.measuringagent.internal.StartDKFQSMeasuringAgent -autoAdjustMemory -osReservedMemory 1GB 1>/home/dkfqs/agent/log/MeasuringAgent.log 2>&1 &'
    ;;
  stop)
       PID=`ps -o pid,args -e | grep "StartDKFQSMeasuringAgent" | egrep -v grep | awk '{print $1}'`
       if [ ! -z "$PID" ] ; then
          echo "MeasuringAgent stopped with pid(s) : $PID"
          kill -9 ${PID} 1> /dev/null 2>&1
       fi
    ;;
  status)
       PID=`ps -o pid,args -e | grep "StartDKFQSMeasuringAgent" | egrep -v grep | awk '{print $1}'`
       if [ ! -z "$PID" ] ; then
          echo "MeasuringAgent running with pid(s) : $PID"
       else
          echo "No MeasuringAgent running"
       fi
    ;;
  *)
    echo "Usage: /etc/init.d/MeasuringAgent {start|stop|status}"
    exit 1
    ;;
esac

exit 0

The Java memory of the Measuring Agent should be set in the startup script as shown in the table below:

OS Physical Memory Java -Xmx setting
<2 GiB 256m
2..3 GiB 512m
4..7 GiB 512m
8..15 GiB 1536m
16..31 GiB 3072m
32..63 GiB 4096m
64..96 GiB 6144m
>96 GiB 8192m
Odd number of GiB should be rounded up (e.g. 7.7 = 8 = 1536m).

Change owner and file protection of /etc/init.d/MeasuringAgent (root at /etc/init.d):

chown root MeasuringAgent
chgrp root MeasuringAgent
chmod 755 MeasuringAgent

Register /etc/init.d/MeasuringAgent to be started at system boot (root at /etc/init.d):

update-rc.d MeasuringAgent defaults

Reboot the system. Login as dkfqs and check /home/dkfqs/agent/log/MeasuringAgent.log

Register and Verify the Measuring Agent

  1. Sign-in at the ‘Portal Server’
  2. Select at Top Navigation ‘Measuring Agents & Cluster Controllers’
  3. Add your new Measuring Agent
  4. Ping the Measuring Agent at application level

“alt attribute”

“alt attribute”

13.2 - Ubuntu: Cluster Controller manual install

Ubuntu: Cluster Controller Install Instructions

Prerequisites

Supported Hardware

  • Amazon EC2 Cloud instance, or
  • Own hosted server with any Intel or AMD CPU

Minimum Requirements

  • Minimum required CPU Cores of Processor: 2
  • Minimum required Memory: 8 GB
  • Minimum required Disk: 64 GB
  • Minimum required Network Speed: 100 Mbps (1000 Mbps or faster strongly recommended)

Environment and Location

In terms of network technology, the cluster controller should be as close as possible to the cluster members.

The simultaneous operation of a cluster controller together with one or more measuring agents on the same machine is possible, although not recommended. This means that the cluster controller should be operated on its own machine - especially if a cluster contains more than 100 members.

The time difference of the operating system time between the cluster controller and the cluster members must not be greater than one second (1000 ms). It is recommended to use the same time server for the cluster controller and the cluster members.

The Portal Server supports the use of multiple cluster controllers. Each cluster controller can manage multiple clusters. And each measuring agent can be a member of multiple clusters.

Network & System Tuning

In /etc/sysctl.conf add:

# TCP/IP Tuning
# =============
fs.file-max = 524288
net.ipv4.tcp_max_syn_backlog = 2048
net.ipv4.tcp_synack_retries = 3
net.ipv4.tcp_max_orphans = 65536
net.ipv4.tcp_fin_timeout = 30
net.ipv4.ip_local_port_range = 16384 60999
net.core.somaxconn = 256
net.core.rmem_max = 1048576
net.core.wmem_max = 1048576

in /etc/security/limits.conf add:

# TCP/IP Tuning
# =============
* soft     nproc          262140
* hard     nproc          262140
* soft     nofile         262140
* hard     nofile         262140
root soft     nproc          262140
root hard     nproc          262140
root soft     nofile         262140
root hard     nofile         262140

Enter: systemctl show -p TasksMax user-0

output: TasksMax=8966

if you get a value less than 262140 then add in /etc/systemd/system.conf

# Ubuntu Tuning
# =============
DefaultTasksMax=262140

Reboot the system and verify the settings. Enter: ulimit -n

output: 262140

Enter: systemctl show -p TasksMax user-0

output: TasksMax=262140

Install Dependencies

Install haveged

sudo apt-get update
sudo apt-get install haveged

Install OpenJDK 11

Get the Java Installation Kit

wget https://download.java.net/java/GA/jdk11/13/GPL/openjdk-11.0.1_linux-x64_bin.tar.gz

Install OpenJDK Java 11

gunzip openjdk-11.0.1_linux-x64_bin.tar.gz
tar -xvf openjdk-11.0.1_linux-x64_bin.tar
rm openjdk-11.0.1_linux-x64_bin.tar
sudo bash
mv jdk-11.0.1 /opt/OpenJDK
cd /opt/OpenJDK
ls -al
chown root -R jdk-11.0.1
chgrp root -R jdk-11.0.1

Execute the following commands (still as sudo bash):

update-alternatives --install "/usr/bin/java" "java" "/opt/OpenJDK/jdk-11.0.1/bin/java" 1
update-alternatives --install "/usr/bin/javac" "javac" "/opt/OpenJDK/jdk-11.0.1/bin/javac" 1
update-alternatives --install "/usr/bin/keytool" "keytool" "/opt/OpenJDK/jdk-11.0.1/bin/keytool" 1
update-alternatives --install "/usr/bin/jar" "jar" "/opt/OpenJDK/jdk-11.0.1/bin/jar" 1
update-alternatives --set "java" "/opt/OpenJDK/jdk-11.0.1/bin/java"
update-alternatives --set "javac" "/opt/OpenJDK/jdk-11.0.1/bin/javac"
update-alternatives --set "keytool" "/opt/OpenJDK/jdk-11.0.1/bin/keytool"
update-alternatives --set "jar" "/opt/OpenJDK/jdk-11.0.1/bin/jar"
exit # end sudo bash

Verify the Java 11 installation.

java -version

openjdk version "11.0.1" 2018-10-16
OpenJDK Runtime Environment 18.9 (build 11.0.1+13)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.1+13, mixed mode)

Install the Cluster Controller

Create the DKFQS account which is running the Cluster Controller

sudo adduser dkfqs    # follow the questions, remember or write down the password

Install the Cluster Controller

Login with the dkfqs account (SSH) - or - Enter: sudo -u dkfqs bash | OR: Install Samba to get convenient access to /home/dkfqs as Samba dkfqs user

Create the directory /home/dkfqs/controller (as dkfqs user):

cd /home/dkfqs
mkdir controller

Create the following sub-directories at /home/dkfqs/controller (as dkfqs user):

  • bin
  • config
  • internalData
  • log
  • scripts
  • usersData
cd /home/dkfqs/controller
mkdir bin config internalData log scripts usersData

Copy the following files to the bin directory /home/dkfqs/controller/bin

  • bcpkix-jdk15on-160.jar
  • bcprov-jdk15on-160.jar
  • bctls-jdk15on-160.jar
  • DKFQSMeasuringAgent.jar

Copy the following files to the config directory /home/dkfqs/controller/config

  • clustercontroller.properties

Modify the clustercontroller.properties file. Set the following properties:

  • HttpsCertificateCN (set the public DNS name or the IP address for the automatically generated SSL/TLS server certificate)
  • HttpsCertificateIP (set the public IP address for the automatically generated SSL/TLS server certificate)
  • AuthTokenValue

Example: clustercontroller.properties

HttpsPort=8083
HttpsCertificateCN=192.168.0.50
HttpsCertificateIP=192.168.0.50
LogLevel=info

# AuthTokenEnabled: true or false, if true = the AuthTokenValue must be configured at portal server measuring agent cluster settings
AuthTokenEnabled=true
# If AuthTokenEnabled is true, but AuthTokenValue is undefined or an empty string, then the (permanent) AuthTokenValue is automatically generated and printed at the log output
AuthTokenValue=aberaber

ClusterControllerLogFile=/home/dkfqs/controller/log/ClusterController.log
ClusterControllerInternalDataDirectory=/home/dkfqs/controller/internalData
ClusterControllerUsersDataRootDirectory=/home/dkfqs/controller/usersData

ApiMaxRequestSizeMB=256
ApiWorkerThreadBusyTimeoutSeconds=330
ApiWorkerThreadExecutionTimeoutSeconds=300

MaxWebSocketConnectTimeSeconds=14400
MaxInboundWebSocketTrafficPerConnection=83886080
MaxInboundWebSocketPayloadPerFrame=20971520
MaxInboundWebSocketFramesPerIPTimeFrame=10
MaxInboundWebSocketFramesPerIPLimit=1000

First Test - Start the Cluster Controller manually (as dkfqs user)

cd /home/dkfqs/controller/bin
export CLASSPATH=bcpkix-jdk15on-160.jar:bcprov-jdk15on-160.jar:bctls-jdk15on-160.jar:DKFQSMeasuringAgent.jar
java -Xmx512m -DdkfqsClusterControllerProperties=../config/clustercontroller.properties -Dnashorn.args="--no-deprecation-warning" com.dkfqs.measuringagent.clustercontroller.StartDKFQSClusterController

Cluster Controller V4.0.4
Max. Memory = 512 MB
AuthTokenEnabled = true
AuthTokenValue = ********
X509 TLS server certificate generated for CN = 192.168.0.50
Internal RSA 2048 bit keypair generated in 305 ms
2022-01-29 20:45:20.118 | QAHTTPd | WARN | QAHTTPd V1.3-Y started
2022-01-29 20:45:20.219 | QAHTTPd | INFO | HTTPS server starting at port 8083
2022-01-29 20:45:20.278 | QAHTTPd | INFO | HTTPS server ready at port 8083

Create the Cluster Controller Startup Script (as root)

sudo bash # become root
cd /etc/init.d
vi ClusterController

Edit - create /etc/init.d/ClusterController

#!/bin/sh
# /etc/init.d/ClusterController
# install with: update-rc.d ClusterController defaults

### BEGIN INIT INFO
# Provides:          ClusterController
# Required-Start:    $local_fs $network $time $syslog
# Required-Stop:     $local_fs $network
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start ClusterController daemon at boot time
# Description:       ClusterController daemon
### END INIT INFO

case "$1" in
  start)
    if [ -f /home/dkfqs/controller/log/ClusterController.log ]; then
       mv /home/dkfqs/controller/log/ClusterController.log /home/dkfqs/controller/log/ClusterController.log_$(date +"%Y_%m_%d_%H_%M")
    fi
    sudo -H -u dkfqs bash -c 'CLASSPATH=/home/dkfqs/controller/bin/bcpkix-jdk15on-160.jar:/home/dkfqs/controller/bin/bcprov-jdk15on-160.jar:/home/dkfqs/controller/bin/bctls-jdk15on-160.jar:/home/dkfqs/controller/bin/DKFQSMeasuringAgent.jar;export CLASSPATH;nohup java -Xmx6144m -DdkfqsClusterControllerProperties=/home/dkfqs/controller/config/clustercontroller.properties -Dnashorn.args="--no-deprecation-warning" com.dkfqs.measuringagent.clustercontroller.StartDKFQSClusterController 1>/home/dkfqs/controller/log/ClusterController.log 2>&1 &'
    ;;
  stop)
       PID=`ps -o pid,args -e | grep "StartDKFQSClusterController" | egrep -v grep | awk '{print $1}'`
       if [ ! -z "$PID" ] ; then
          echo "ClusterController stopped with pid(s) : $PID"
          kill -9 ${PID} 1> /dev/null 2>&1
       fi
    ;;
  status)
       PID=`ps -o pid,args -e | grep "StartDKFQSClusterController" | egrep -v grep | awk '{print $1}'`
       if [ ! -z "$PID" ] ; then
          echo "ClusterController running with pid(s) : $PID"
       else
          echo "No ClusterController running"
       fi
    ;;
  *)
    echo "Usage: /etc/init.d/ClusterController {start|stop|status}"
    exit 1
    ;;
esac

exit 0

Change owner and file protection of /etc/init.d/ClusterController (root at /etc/init.d):

chown root ClusterController
chgrp root ClusterController
chmod 755 ClusterController

Register /etc/init.d/ClusterController to be started at system boot (root at /etc/init.d):

update-rc.d ClusterController defaults

Reboot the system. Login as dkfqs and check /home/dkfqs/controller/log/ClusterController.log

Define a Cluster and Verify the Cluster Controller

  1. Sign-in at the ‘Portal Server’
  2. Select at Top Navigation ‘Measuring Agents & Cluster Controllers’
  3. Add the Cluster
  4. Ping the Cluster Controller at application level
  5. Add the Cluster Members
  6. Ping the Cluster Members by the Cluster Controller, and verify that the absolute value of OS Δ Time for each cluster member is not greater than 1000 ms

“alt attribute”

“alt attribute”

“alt attribute”

“alt attribute”

“alt attribute”

“alt attribute”

“alt attribute”

“alt attribute”

13.3 - Ubuntu: Portal Server manual install

Ubuntu: Portal Server Install Instructions

Prerequisites

Supported Hardware

  • Amazon EC2 Cloud instances
  • Own hosted Servers with any Intel or AMD CPU

Minimum Hardware Requirements

  • Minimum required CPU Cores of Processor: 4
  • Minimum required RAM: 16 GB
  • Minimum required Disk: 512 GB
  • Minimum required Network Speed: 1000 Mbps

Email Server

The Portal Server sends its emails via SMTP. You need an email server which receive and forward these SMTP messages.

Twilio SMS Gateway

If the Portal Server will be operated/configured in such a way that any person can “sign up” (= self registration) you need a customer account for the Twilio SMS Gateway www.twilio.com/docs/sms

Network & System Tuning

In /etc/sysctl.conf add:

# TCP/IP Tuning
# =============
fs.file-max = 524288
net.ipv4.tcp_max_syn_backlog = 2048
net.ipv4.tcp_synack_retries = 3
net.ipv4.tcp_max_orphans = 65536
net.ipv4.tcp_fin_timeout = 30
net.ipv4.ip_local_port_range = 16384 60999
net.core.somaxconn = 256
net.core.rmem_max = 1048576
net.core.wmem_max = 1048576

In /etc/security/limits.conf add:

# TCP/IP Tuning
# =============
* soft     nproc          262140
* hard     nproc          262140
* soft     nofile         262140
* hard     nofile         262140
root soft     nproc          262140
root hard     nproc          262140
root soft     nofile         262140
root hard     nofile         262140

Enter: systemctl show -p TasksMax user-0

output: TasksMax=8966

if you get a value less than 262140 then add in /etc/systemd/system.conf

# Ubuntu Tuning
# =============
DefaultTasksMax=262140

Reboot the system and verify the settings. Enter: ulimit -n

output: 262140

Enter: systemctl show -p TasksMax user-0

output: TasksMax=262140

Forward the external TCP/IP server port 80 (HTTP) to port 8000, and forward external port 443 (HTTPS) to port 8001

Install ifupdown

sudo apt-get update
sudo apt-get install ifupdown

Create/edit the file DKFQSiptables in /etc/network/if-pre-up.d/ and add:

#!/bin/sh
iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8000
iptables -t nat -I OUTPUT -p tcp -d 127.0.0.1 --dport 80 -j REDIRECT --to-ports 8000
iptables -t nat -A PREROUTING -p tcp --dport 443 -j REDIRECT --to-port 8001
iptables -t nat -I OUTPUT -p tcp -d 127.0.0.1 --dport 443 -j REDIRECT --to-ports 8001
exit 0

Then give execute permission to /etc/network/if-pre-up.d/DKFQSiptables : sudo chmod 755 /etc/network/if-pre-up.d/DKFQSiptables

Reboot the machine and check with:

sudo iptables -L -t nat

> Chain PREROUTING (policy ACCEPT)
> target     prot opt source               destination
> REDIRECT   tcp  --  anywhere             anywhere             tcp dpt:http redir ports 8000
> REDIRECT   tcp  --  anywhere             anywhere             tcp dpt:https redir ports 8001
> REDIRECT   tcp  --  anywhere             anywhere             tcp dpt:http redir ports 8000
> REDIRECT   tcp  --  anywhere             anywhere             tcp dpt:https redir ports 8001
>
> Chain INPUT (policy ACCEPT)
> target     prot opt source               destination
>
> Chain OUTPUT (policy ACCEPT)
> target     prot opt source               destination
> REDIRECT   tcp  --  anywhere             localhost            tcp dpt:https redir ports 8001
> REDIRECT   tcp  --  anywhere             localhost            tcp dpt:http redir ports 8000
> REDIRECT   tcp  --  anywhere             localhost            tcp dpt:https redir ports 8001
> REDIRECT   tcp  --  anywhere             localhost            tcp dpt:http redir ports 8000
>
> Chain POSTROUTING (policy ACCEPT)
> target     prot opt source               destination

Install Dependencies

Install fontconfig

sudo apt-get update
sudo apt-get install fontconfig

Install haveged

sudo apt-get update
sudo apt-get install haveged

Install SQLite

sudo apt-get update
sudo apt install sqlite

Install OpenJDK Java 8 and 11

Get the Java Installation Kits

wget https://download.java.net/openjdk/jdk8u41/ri/openjdk-8u41-b04-linux-x64-14_jan_2020.tar.gz
wget https://download.java.net/java/GA/jdk11/13/GPL/openjdk-11.0.1_linux-x64_bin.tar.gz

Install OpenJDK Java 8

gunzip openjdk-8u41-b04-linux-x64-14_jan_2020.tar.gz
tar -xvf  openjdk-8u41-b04-linux-x64-14_jan_2020.tar
rm openjdk-8u41-b04-linux-x64-14_jan_2020.tar
sudo bash *******
mkdir /opt/OpenJDK
mv java-se-8u41-ri /opt/OpenJDK
cd /opt/OpenJDK
ls -al
chown root -R java-se-8u41-ri
chgrp root -R java-se-8u41-ri
exit # end sudo bash

Verify the Java 8 installation.

/opt/OpenJDK/java-se-8u41-ri/bin/java -version

openjdk version "1.8.0_41"
OpenJDK Runtime Environment (build 1.8.0_41-b04)
OpenJDK 64-Bit Server VM (build 25.40-b25, mixed mode)

Install OpenJDK Java 11

gunzip openjdk-11.0.1_linux-x64_bin.tar.gz
tar -xvf openjdk-11.0.1_linux-x64_bin.tar
rm openjdk-11.0.1_linux-x64_bin.tar
sudo bash
mv jdk-11.0.1 /opt/OpenJDK
cd /opt/OpenJDK
ls -al
chown root -R jdk-11.0.1
chgrp root -R jdk-11.0.1

Execute the following commands (still as sudo bash):

update-alternatives --install "/usr/bin/java" "java" "/opt/OpenJDK/jdk-11.0.1/bin/java" 1
update-alternatives --install "/usr/bin/javac" "javac" "/opt/OpenJDK/jdk-11.0.1/bin/javac" 1
update-alternatives --install "/usr/bin/keytool" "keytool" "/opt/OpenJDK/jdk-11.0.1/bin/keytool" 1
update-alternatives --install "/usr/bin/jar" "jar" "/opt/OpenJDK/jdk-11.0.1/bin/jar" 1
update-alternatives --set "java" "/opt/OpenJDK/jdk-11.0.1/bin/java"
update-alternatives --set "javac" "/opt/OpenJDK/jdk-11.0.1/bin/javac"
update-alternatives --set "keytool" "/opt/OpenJDK/jdk-11.0.1/bin/keytool"
update-alternatives --set "jar" "/opt/OpenJDK/jdk-11.0.1/bin/jar"
exit # end sudo bash

Verify the Java 11 installation.

java -version

openjdk version "11.0.1" 2018-10-16
OpenJDK Runtime Environment 18.9 (build 11.0.1+13)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.1+13, mixed mode)

Install the Portal Server

Create the DKFQS account which is running the Portal Server

sudo adduser dkfqs    # follow the questions, remember or write down the password

Install the Portal Server

Login with the dkfqs account (SSH) - or - Enter: sudo -u dkfqs bash | OR: Install Samba to get convenient access to /home/dkfqs as Samba dkfqs user

Create the directory /home/dkfqs/portal (as dkfqs user):

cd /home/dkfqs
mkdir portal

Create the following sub-directories at /home/dkfqs/agent (as dkfqs user):

  • backup
  • bin
  • config
  • db
  • developerExamples
  • htdocs
  • javaUsersLib
  • jks
  • log
  • reportTemplate
  • scripts
  • temp
  • usersLib
  • usersData
cd /home/dkfqs/portal
mkdir backup bin config db developerExamples htdocs javaUsersLib jks log reportTemplate scripts temp usersLib usersData

Copy the following files to the bin directory /home/dkfqs/portal/bin

  • bcpkix-jdk15on-160.jar
  • bcprov-jdk15on-160.jar
  • bctls-jdk15on-160.jar
  • DKFQS.jar
  • jasperreports-fonts-6.20.5.jar

Copy the following files to the db directory /home/dkfqs/portal/db

  • CreateNewAdminDB.sql
  • CreateNewMonitoringDB.sql
  • CreateNewOperationsDB.sql
  • CreateNewUsersDB.sql

Edit the file CreateNewUsersDB.sql and modify the following line to set the nickname, the email, the phone number and the temporary password of the Admin account. Note: The nickname must always start with “Admin-”

insert into AdminAccountsTable (adminUserId, nickname, adminPrimaryEmail, adminPrimaryPhone, initialPassword) values (1, "Admin-One", "falarasorn@yahoo.com", "+43123456789", "ginkao1234");

Navigate to /home/dkfqs/portal/db and create the Admin, Operations and the Users DB (as dkfqs user):

sqlite3 AdminAccounts.db < CreateNewAdminDB.sql
sqlite3 Monitoring.db < CreateNewMonitoringDB.sql
sqlite3 Operations.db < CreateNewOperationsDB.sql
sqlite3 Users.db < CreateNewUsersDB.sql

Copy the following file to the htdocs directory /home/dkfqs/portal/htdocs

  • htdocs.jar

Navigate to /home/dkfqs/portal/htdocs and execute (as dkfqs user):

jar -xvf htdocs.jar
rm htdocs.jar   # delete the jar)
rm -R META-INF  # delete the META-INF directory)

Copy the following files to the javaUsersLib directory /home/dkfqs/portal/javaUsersLib

  • com.dkfqs.selenium.jar
  • commons-io-2.13.0.jar
  • hamcrest-core-1.3.jar
  • junit-4.13.2.jar
  • selenium_bundle-4.10.0.jar
  • selenium_bundle-4.12.1.jar

Copy the following file to the jks directory /home/dkfqs/portal/jks

  • dkfqscom.jks

Copy the following files to the usersLib directory /home/dkfqs/portal/usersLib

  • com.dkfqs.tools.jar
  • DKFQSLibrary2.psm1 (optional)

Copy the following files to the bin directory /home/dkfqs/portal/reportTemplate

  • AnnotationEvent.jasper
  • Annotations.jasper
  • ErrorCharts.jasper
  • ErrorTypeDistribution.jrxml
  • ErrorTypes.jasper
  • JobProperties.jasper
  • MonitoringGroup.jasper
  • MonitoringGroupAvailability.jasper
  • MonitoringJob.jasper
  • MonitoringJobAvailability.jasper
  • MonitoringJobPerformance.jasper
  • OverviewStatistics.jasper
  • SamplesThroughputPerSecond.jasper
  • SummaryStatistic.jasper
  • TestResults.jasper
  • TestSpecificData.jasper

Copy the following files to the bin directory /home/dkfqs/portal/config

  • dkfqs.properties
  • rewrite.properties
  • twilio.properties

Modify the dkfqs.properties file. Set the following properties:

  • ServerName
  • ServerDNSName
  • DNSJavaDefaultDNSServers
  • UsersMailServerHost
  • UsersMailFrom
  • UsersMailServerAuthUser
  • UsersMailServerAuthPassword
  • ServerStatusPageEnabledIPList
  • AlertMailServerHost
  • AlertMailFrom
  • AlertMailToList
  • AlertMailBounceAddress
  • AlertMailServerAuthUser
  • AlertMailServerAuthPassword

Example: dkfqs.properties

IsProduction=true
ServerName=192.168.0.50
ServerDNSName=192.168.0.50
DiskDocumentRootDirectory=/home/dkfqs/portal/htdocs
SQLiteDBDirectory=/home/dkfqs/portal/db
ReportTemplateDirectory=/home/dkfqs/portal/reportTemplate/
JasperReportsTemporaryDirectory=/home/dkfqs/portal/temp/
UsersDataRootDirectory=/home/dkfqs/portal/usersData
OSProcessLogFile=/home/dkfqs/portal/log/DKFQS.log
LogLevel=info
StaticContentMaxAgeTime=7200
MaxHTTPRequestSize=20240000
MaxInvalidAnonymousSessionsPerIPLimit=32
AnonymousSessionTimeout=1200
MaxAnonymousSessionTime=21600
MaxWebSocketConnectTimeSeconds=14400
MaxInboundWebSocketTrafficPerConnection=67108864
MaxInboundWebSocketPayloadPerFrame=1048576
MaxInboundWebSocketFramesPerIPTimeFrame=10
MaxInboundWebSocketFramesPerIPLimit=1000
HTTPExternalServerPort=80
HTTPInternalServerPort=8000
HTTPSExternalServerPort=443
HTTPSInternalServerPort=8001
HTTPSKeyStoreFile=/home/dkfqs/portal/jks/dkfqscom.jks
HTTPSKeyStorePassword=topsecret
#
FileTreeApiMaxRequestSizeMB=256
FileTreeApiWorkerThreadBusyTimeoutSeconds=330
FileTreeApiWorkerThreadExecutionTimeoutSeconds=300
TestjobsApiMaxRequestSizeMB=256
TestjobsApiWorkerThreadBusyTimeoutSeconds=330
TestjobsApiWorkerThreadExecutionTimeoutSeconds=300
#
DNSJavaDefaultDNSServers=8.8.8.8,8.8.4.4
#
JavaSDK8BinaryPath=/opt/OpenJDK/java-se-8u41-ri/bin
JavaSDK11BinaryPath=/opt/OpenJDK/jdk-11.0.1/bin
HTTPTestWizardJavaCodeLibraries=/home/dkfqs/portal/usersLib/com.dkfqs.tools.jar
#
UserSignInURL=/SignIn
UsersMailServerHost=192.168.1.4
UsersMailFrom=xxxxxxxxx@xxxxxxx.com
UsersMailServerAuthUser=xxxxxxxxx@xxxxxxx.com
UsersMailServerAuthPassword=*********
UsersMailTransmitterThreads=2
UsersMailDebugSMTP=false
#
smsGatewaysClassNames=com.dkfqs.server.sms.twilio.TwilioSMSGateway
#
# ServerStatusPageEnabledIPList=127.0.0.1,192.168.0.99
ServerStatusPageEnabledIPList=*.*.*.*
AdminSignInURL=/AdminSignIn
AlertMailEnabled=true
AlertMailServerHost=192.168.1.4
AlertMailFrom=xxxxxxxxx@xxxxxxx.com
AlertMailToList=yyyyyyyyy@xxxxxxx.com,zzzzzzzzz@xxxxxxx.com
AlertMailBounceAddress=bbbbbbb@xxxxxxx.com
AlertMailServerAuthUser=xxxxxxxxx@xxxxxxx.com
AlertMailServerAuthPassword=*******
AlertMailDebugSMTP=false
AlertMailNotifyStartup=false
#
SecurityIpBlacklistBlockTime=900
SecurityMaxRequestsPerIpLimit=200
SecurityMaxRequestsPerIpTimeFrame=10
SecurityMaxInvalidRequestsPerIpLimit=12
SecurityMaxInvalidRequestsPerIpTimeFrame=60
SecurityMaxAnonymousFormSubmitPerIpLimit=8
SecurityMaxAnonymousFormSubmitPerIpTimeFrame=60
SecurityMaxAuthenticationFailuresPerIpLimit=5
SecurityMaxAuthenticationFailuresPerIpTimeFrame=60
#
MeasuringAgentConnectTimeout=10

# Support of license provider 'Real Load Pty Ltd / nopCommerce', if enabled then the LicenseProviderRealLoadPtyLtdNopCommerceCARootFileName is located in the config directory of the portal server
LicenseProviderRealLoadPtyLtdNopCommerceEnabled=true
LicenseProviderRealLoadPtyLtdNopCommerceCARootFileName=nopCommerceCARoot.pem

# Synthetic Monitoring settings
MonitoringJobSchedulerMainThreadIntervalSeconds=10
MonitoringJobSchedulerInitialDBLogLevel=warn
MonitoringJobSchedulerInitialStdoutLogLevel=info
MonitoringJobSchedulerInitialUserFileLogLevel=info
MonitoringGroupThreadsStartDelayMillis=1000

# Support for compiling Junit and Selenium Tests
JavaUsersLibDirectory=/home/dkfqs/portal/javaUsersLib
JavaJUnit4CompilerLibraries=hamcrest-core-1.3.jar,junit-4.13.2.jar
JavaJUnit4TestExecutionLibraries=com.dkfqs.selenium.jar,com.dkfqs.tools.jar,commons-io-2.13.0.jar,hamcrest-core-1.3.jar,junit-4.13.2.jar
JavaSelenium4CompilerLibraries=hamcrest-core-1.3.jar,junit-4.13.2.jar,selenium_bundle-4.12.1.jar,com.dkfqs.selenium.jar
JavaSelenium4TestExecutionLibraries=selenium_bundle-4.12.1.jar,com.dkfqs.selenium.jar,com.dkfqs.tools.jar,commons-io-2.13.0.jar,hamcrest-core-1.3.jar,junit-4.13.2.jar

Modify the twilio.properties file. Set the following properties:

  • sid
  • authToken
  • fromTwilioPhoneNumber

Example: twilio.properties

apiURLMainPath=https://api.twilio.com/2010-04-01/Accounts/
sid=********************************
authToken=********************************
fromTwilioPhoneNumber=+1123456789
fromTwilioAlphanumericSenderID=RealLoad
tcpConnectTimoutMillis=10000
sslHandshakeTimeoutMillis=5000
httpProcessingTimeoutMillis=10000
debugAPI=false

First Test - Start the Portal Server manually (as dkfqs user)

cd /home/dkfqs/portal/bin
export CLASSPATH=bcpkix-jdk15on-160.jar:bcprov-jdk15on-160.jar:bctls-jdk15on-160.jar:DKFQS.jar
java -Xmx2048m -DdkfqsProperties=../config/dkfqs.properties -DrewriteProperties=../config/rewrite.properties -Dnashorn.args="--no-deprecation-warning" com.dkfqs.server.internal.StartDKFQSserver

Internal RSA 2048 bit keypair generated in 1220 ms
2021-03-10 22:27:25.040 | QAHTTPd | INFO | SQL connection pool for DB "UsersDB" initialized
2021-03-10 22:27:25.062 | QAHTTPd | INFO | SQL connection pool for DB "AdminAccountsDB" initialized
2021-03-10 22:27:25.068 | QAHTTPd | INFO | Alarm adapter "IP Blacklist Alarm Adapter" started
2021-03-10 22:27:25.069 | QAHTTPd | WARN | QAHTTPd V1.3-U started
2021-03-10 22:27:25.071 | QAHTTPd | INFO | Execute PreUpStartupLoadIPRangeBlacklist
2021-03-10 22:27:25.082 | QAHTTPd | INFO | HTTP server starting at port 8000
2021-03-10 22:27:25.109 | QAHTTPd | INFO | HTTP server ready at port 8000
2021-03-10 22:27:25.110 | QAHTTPd | INFO | HTTPS server starting at port 8001
2021-03-10 22:27:25.124 | QAHTTPd | INFO | HTTPS server ready at port 8001
2021-03-10 22:27:25.821 | EMAIL-1 | INFO | Email transmitter thread started
2021-03-10 22:27:25.822 | EMAIL-2 | INFO | Email transmitter thread started
2021-03-10 22:27:25.828 | main | INFO | Twilio SMS Gateway registered
2021-03-10 22:27:25.853 | main | INFO | Twilio SMS Gateway initialized
2021-03-10 22:27:25.857 | SMS-Dispatcher | INFO | Thread started

Create the Portal Server Startup Script (as root)

sudo bash # become root
cd /etc/init.d
vi DKFQS

Edit - create /etc/init.d/DKFQS

#!/bin/sh
# /etc/init.d/DKFQS
# install with: update-rc.d DKFQS defaults

### BEGIN INIT INFO
# Provides:          DKFQS
# Required-Start:    $local_fs $network $time $syslog
# Required-Stop:     $local_fs $network
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start DKFQS daemon at boot time
# Description:       DKFQS daemon
### END INIT INFO

case "$1" in
  start)
    if [ -f /home/dkfqs/portal/log/DKFQS.log ]; then
       mv /home/dkfqs/portal/log/DKFQS.log /home/dkfqs/portal/log/DKFQS.log_$(date +"%Y_%m_%d_%H_%M")
    fi
    sudo -H -u dkfqs bash -c 'CLASSPATH=/home/dkfqs/portal/bin/bcpkix-jdk15on-160.jar:/home/dkfqs/portal/bin/bcprov-jdk15on-160.jar:/home/dkfqs/portal/bin/bctls-jdk15on-160.jar:/home/dkfqs/portal/bin/jasperreports-fonts-6.20.5.jar:/home/dkfqs/portal/bin/DKFQS.jar;export CLASSPATH;cd /home/dkfqs/portal/temp;nohup java -Xmx3072m -DdkfqsProperties=/home/dkfqs/portal/config/dkfqs.properties -DrewriteProperties=/home/dkfqs/portal/config/rewrite.properties -Dnashorn.args="--no-deprecation-warning" com.dkfqs.server.internal.StartDKFQSserver 1>/home/dkfqs/portal/log/DKFQS.log 2>&1 &'
    ;;
  stop)
       PID=`ps -o pid,args -e | grep "StartDKFQSserver" | egrep -v grep | awk '{print $1}'`
       if [ ! -z "$PID" ] ; then
          echo "DKFQS stopped with pid(s) : $PID"
          kill -9 ${PID} 1> /dev/null 2>&1
       fi
    ;;
  status)
       PID=`ps -o pid,args -e | grep "StartDKFQSserver" | egrep -v grep | awk '{print $1}'`
       if [ ! -z "$PID" ] ; then
          echo "DKFQS running with pid(s) : $PID"
       else
          echo "No DKFQS running"
       fi
    ;;
  *)
    echo "Usage: /etc/init.d/DKFQS {start|stop|status}"
    exit 1
    ;;
esac

exit 0

Change owner and file protection of /etc/init.d/DKFQS (root at /etc/init.d):

chown root DKFQS
chgrp root DKFQS
chmod 755 DKFQS

Register /etc/init.d/DKFQS to be started at system boot (root at /etc/init.d):

update-rc.d DKFQS defaults

Reboot the system. Then check /home/dkfqs/portal/log/DKFQS.log

Administrator Sign In

Enter in your browser https://admin-portal-host/admin .

You will get a browser warning because the SSL server certificate is expired. Ignore the warning and enter in the Sign In the email address and the password as you have set in CreateNewAdminDB.sql.

You will now asked to set a new password. Then you are signed in.

“alt attribute”

Disable Sign Up if you don’t have an SMS gateway:

“alt attribute”

User accounts can be added directly:

“alt attribute”

Replace the SSL Server Certificate

If you or your company can already issue SSL server certificates you can skip the next sub-chapter. Continue in such a case with “Convert and Install the SSL Server Certificate”.

Get a Let's Encrypt SSL Server Certificate | Ubuntu 20

Make sure that your portal server has a public, valid DNS name.

Install certbot:

sudo snap install --classic certbot
sudo ln -s /snap/bin/certbot /usr/bin/certbot

Stop the Portal Server:

sudo /etc/init.d/DKFQS stop

To get the SSL server certificate enter:

sudo certbot certonly --standalone   # enter your email address and the DNS name ou your portal server, follow the instructions

On success certbot generates the following two files:

  • fullchain.pem
  • privkey.pem

Start the Portal Server:

sudo /etc/init.d/DKFQS start

Convert and Install the SSL Server Certificate

Become root and navigate to the directory where the fullchain.pem and privkey.pem files are located. Enter:

sudo bash
openssl pkcs12 -export -in fullchain.pem -inkey privkey.pem -out your-certificate-name.p12  # convert cert to PKCS12 file
keytool -importkeystore -srckeystore your-certificate-name.p12 -srcstoretype PKCS12 -destkeystore your-certificate-name.jks -deststoretype JKS  # convert PKCS12 file to Java keystore file

Copy the Java keystore file to /home/dkfqs/portal/jks

cp your-certificate-name.jks /home/dkfqs/portal/jks

Edit /home/dkfqs/portal/config/dkfqs.properties and replace:

HTTPSKeyStoreFile=/home/dkfqs/portal/jks/your-certificate-name.jks
HTTPSKeyStorePassword=*********

Restart the Portal Server

sudo /etc/init.d/DKFQS stop
sudo /etc/init.d/DKFQS start

Create a Cron Job to Renew the Let's Encrypt SSL Certificate

Create at your home directory the sub-directory system_cronjobs_scripts and add/edit the file “DKFQS_certbot_renew” in this directory. Replace the ********* placeholders with your real values.

#!/bin/sh
#
# renew the letsencrypt DKFQS certificate
# =======================================
certbot renew
#
# set the default working directory
cd /home/*********/system_cronjobs_scripts
#
# cleanup in any case
rm -f *.jks
rm -f *.p12
#
# convert the letsencrypt certificate to PKCS12 and place it in the default directory
openssl pkcs12 -export -in /etc/letsencrypt/live/*********/fullchain.pem -inkey /etc/letsencrypt/live/*********/privkey.pem -out ./*********.p12 -passin pass:******** -passout pass:********
#
# convert the PKCS12 certificate to a Java keystore
echo ******** | keytool -importkeystore -srckeystore *********.p12 -srcstoretype PKCS12 -destkeystore *********.jks -deststoretype JKS -storepass ********
#
# update DKFQS keystore file
cp *********.jks /home/dkfqs/portal/jks
chown dkfqs /home/dkfqs/portal/jks/*********.jks
chgrp dkfqs /home/dkfqs/portal/jks/*********.jks
chmod 600 /home/dkfqs/portal/jks/*********.jks
#
# restart DKFQS server
/etc/init.d/DKFQS stop
sleep 5
sudo /etc/init.d/DKFQS start
#
# cleanup again in any case
rm -f *.jks
rm -f *.p12
#
# display DKFQS log file
sleep 5
cat /home/dkfqs/portal/log/DKFQS.log
#
# all done
exit 0
sudo bash # become root
chmod 700 DKFQS_certbot_renew # change file protection and set execute bit
./DKFQS_certbot_renew  # try out manually

Add the file to crontab. Important: The last line in crontab must be an empty line!

sudo crontab -e
1 1 1 * * /home/*********/system_cronjobs_scripts/DKFQS_certbot_renew > /home/*********/system_cronjobs_scripts/DKFQS_certbot_renew.log 2>&1

13.4 - Ubuntu: Remote Proxy Recorder manual install

Ubuntu: Remote Proxy Recorder Install Instructions

Prerequisites

Supported Hardware

  • Amazon EC2 Cloud instance, or
  • Own hosted server with any Intel or AMD CPU

Minimum Requirements

  • Minimum required CPU Cores of Processor: 2
  • Minimum required Memory: 8 GB
  • Minimum required Disk: 32 GB
  • Minimum required Network Speed: 100 Mbps (1000 Mbps or faster strongly recommended)

Environment and Location

The Remote Proxy Recorder can be placed at any network location, but the control port must be reachable from the Portal Server.

Network & System Tuning

In /etc/sysctl.conf add:

# TCP/IP Tuning
# =============
fs.file-max = 524288
net.ipv4.tcp_max_syn_backlog = 2048
net.ipv4.tcp_synack_retries = 3
net.ipv4.tcp_max_orphans = 65536
net.ipv4.tcp_fin_timeout = 30
net.ipv4.ip_local_port_range = 16384 60999
net.core.somaxconn = 256
net.core.rmem_max = 1048576
net.core.wmem_max = 1048576

in /etc/security/limits.conf add:

# TCP/IP Tuning
# =============
* soft     nproc          262140
* hard     nproc          262140
* soft     nofile         262140
* hard     nofile         262140
root soft     nproc          262140
root hard     nproc          262140
root soft     nofile         262140
root hard     nofile         262140

Enter: systemctl show -p TasksMax user-0

output: TasksMax=8966

if you get a value less than 262140 then add in /etc/systemd/system.conf

# Ubuntu Tuning
# =============
DefaultTasksMax=262140

Reboot the system and verify the settings. Enter: ulimit -n

output: 262140

Enter: systemctl show -p TasksMax user-0

output: TasksMax=262140

Install Dependencies

Install haveged

sudo apt-get update
sudo apt-get install haveged

Install OpenJDK 11

Get the Java Installation Kit

wget https://download.java.net/java/GA/jdk11/13/GPL/openjdk-11.0.1_linux-x64_bin.tar.gz

Install OpenJDK Java 11

gunzip openjdk-11.0.1_linux-x64_bin.tar.gz
tar -xvf openjdk-11.0.1_linux-x64_bin.tar
rm openjdk-11.0.1_linux-x64_bin.tar
sudo bash
mv jdk-11.0.1 /opt/OpenJDK
cd /opt/OpenJDK
ls -al
chown root -R jdk-11.0.1
chgrp root -R jdk-11.0.1

Execute the following commands (still as sudo bash):

update-alternatives --install "/usr/bin/java" "java" "/opt/OpenJDK/jdk-11.0.1/bin/java" 1
update-alternatives --install "/usr/bin/javac" "javac" "/opt/OpenJDK/jdk-11.0.1/bin/javac" 1
update-alternatives --install "/usr/bin/keytool" "keytool" "/opt/OpenJDK/jdk-11.0.1/bin/keytool" 1
update-alternatives --install "/usr/bin/jar" "jar" "/opt/OpenJDK/jdk-11.0.1/bin/jar" 1
update-alternatives --set "java" "/opt/OpenJDK/jdk-11.0.1/bin/java"
update-alternatives --set "javac" "/opt/OpenJDK/jdk-11.0.1/bin/javac"
update-alternatives --set "keytool" "/opt/OpenJDK/jdk-11.0.1/bin/keytool"
update-alternatives --set "jar" "/opt/OpenJDK/jdk-11.0.1/bin/jar"
exit # end sudo bash

Verify the Java 11 installation.

java -version

openjdk version "11.0.1" 2018-10-16
OpenJDK Runtime Environment 18.9 (build 11.0.1+13)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.1+13, mixed mode)

Generate the Certificate Authority (CA) Root Certificate

For technical reasons, the Remote Proxy Recorder generates “fake” web server certificates during operation in order to break the encryption between the web browser and the web servers and to record the data exchanged.

In order for this to work, the Remote Proxy Recorder needs its own CA root certificate, which you then have to import into your browser.

Thus for security reasons, never use a CA root certificate from us or someone else for the Remote Proxy Recorder root certificate. Always create your own CA root certificate.

Example:

C:\Scratch2>openssl genrsa -des3 -out myCAPrivate.key 2048
Generating RSA private key, 2048 bit long modulus
.......................+++
.............................+++
unable to write 'random state'
e is 65537 (0x10001)
Enter pass phrase for myCAPrivate.key:
Verifying - Enter pass phrase for myCAPrivate.key:

C:\Scratch2>openssl pkcs8 -topk8 -inform PEM -outform PEM -in myCAPrivate.key -out myCAPrivateKey.pem -nocrypt
Enter pass phrase for myCAPrivate.key:

C:\Scratch2>openssl req -x509 -new -nodes -key myCAPrivate.key -sha256 -days 3700 -out myCARootCert.pem
Enter pass phrase for myCAPrivate.key:
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:CH
State or Province Name (full name) [Some-State]:Bern
Locality Name (eg, city) []:Bern
Organization Name (eg, company) [Internet Widgits Pty Ltd]:My Pty Ltd
Organizational Unit Name (eg, section) []:QA
Common Name (e.g. server FQDN or YOUR name) []:DKFQS Proxy Recorder Root
Email Address []:

C:\Scratch2>dir
 Volume in drive C is OS
 Volume Serial Number is AEF7-CFB1

 Directory of C:\Scratch2

06 Feb 2022  20:44    <DIR>          .
06 Feb 2022  20:44    <DIR>          ..
06 Feb 2022  20:40             1.743 myCAPrivate.key
06 Feb 2022  20:41             1.704 myCAPrivateKey.pem
06 Feb 2022  20:44             1.350 myCARootCert.pem
               3 File(s)          4.797 bytes
               2 Dir(s)  310.772.580.352 bytes free

Install the Remote Proxy Recorder

Create the DKFQS account which is running the Remote Proxy Recorder

sudo adduser dkfqs    # follow the questions, remember or write down the password

Install the Remote Proxy Recorder

Login with the dkfqs account (SSH) - or - Enter: sudo -u dkfqs bash | OR: Install Samba to get convenient access to /home/dkfqs as Samba dkfqs user

Create the directory /home/dkfqs/proxy (as dkfqs user):

cd /home/dkfqs
mkdir proxy

Create the following sub-directories at /home/dkfqs/proxy (as dkfqs user):

  • bin
  • config
  • log
cd /home/dkfqs/proxy
mkdir bin config log

Copy the following files to the bin directory /home/dkfqs/proxy/bin

  • bcmail-jdk15on-168.jar
  • bcpg-jdk15on-168.jar
  • bcpkix-jdk15on-168.jar
  • bcprov-jdk15on-168.jar
  • bctls-jdk15on-168.jar
  • com.dkfqs.remoteproxyrecorder.jar

Copy the following files to the config directory /home/dkfqs/proxy/config

  • config.properties
  • myCAPrivateKey.pem (the private key of your self generated CA root certificate)
  • myCARootCert.pem (your self generated CA root certificate)

Modify the config.properties file. Set (modify) the following properties:

  • ControlServerHttpsCertificateCN (set the public DNS name or the IP address of the Remote Proxy Recorder)
  • ControlServerHttpsCertificateIP (set the public IP address of the Remote Proxy Recorder)
  • ControlServerAuthToken
  • ProxyServerDefaultCaRootCertFilePath (set the CA root certificate of the proxy)
  • ProxyServerDefaultCaRootPrivateKeyFilePath (set the private key of the CA root certificate)

Example: config.properties

ControlServerLogLevel=info
ControlServerHttpsPort=8081
ControlServerHttpsCertificateCN=proxy2.realload.com
ControlServerHttpsCertificateIP=83.150.39.45
#Note: the control server authentication token is required to connect to the control server
ControlServerAuthToken=krungthep

ProxyServerLogLevel=warn
ProxyServerPort=8082
ProxyServerDefaultCaRootCertFilePath=/home/dkfqs/proxy/config/myCARootCert.pem
ProxyServerDefaultCaRootPrivateKeyFilePath=/home/dkfqs/proxy/config/myCAPrivateKey.pem
#Note: the proxy authentication credentials are replaced on the fly when the portal user connects via the control interface to the control server
ProxyServerDefaultAuthenticationUsername=max
ProxyServerDefaultAuthenticationPassword=meier

First Test - Start the Remote Proxy Recorder manually (as dkfqs user)

cd /home/dkfqs/proxy/bin
export CLASSPATH=bcmail-jdk15on-168.jar:bcpg-jdk15on-168.jar:bcpkix-jdk15on-168.jar:bcprov-jdk15on-168.jar:bctls-jdk15on-168.jar:com.dkfqs.remoteproxyrecorder.jar
java -Xmx2048m -DconfigProperties=../config/config.properties -Dnashorn.args="--no-deprecation-warning" com.dkfqs.remoteproxyrecorder.main.StartRemoteProxyRecorder

> Remote Proxy Recorder V0.2.0
> Max. Memory = 2048 MB
> Internal RSA 2048 bit keypair generated in 85 ms
> 2021-06-05 23:24:37.710 | QAHTTPd | WARN | QAHTTPd V1.3-V started
> 2021-06-05 23:24:37.710 | QAHTTPd | INFO | HTTPS server starting at port 8081
> 2021-06-05 23:24:37.726 | QAHTTPd | INFO | HTTPS server ready at port 8081
> 2021-06-05 23:24:38.722 | Proxy | WARN | ProxyRecorder V1.1.0 started at port 8082

Create the Remote Proxy Recorder Startup Script (as root)

sudo bash # become root
cd /etc/init.d
vi RemoteProxyRecorder

Edit - create /etc/init.d/RemoteProxyRecorder

#!/bin/sh
# /etc/init.d/RemoteProxyRecorder
# install with: update-rc.d RemoteProxyRecorder defaults

### BEGIN INIT INFO
# Provides:          RemoteProxyRecorder
# Required-Start:    $local_fs $network $time $syslog
# Required-Stop:     $local_fs $network
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start RemoteProxyRecorder daemon at boot time
# Description:       RemoteProxyRecorder daemon
### END INIT INFO

case "$1" in
  start)
    if [ -f /home/dkfqs/proxy/log/RemoteProxyRecorder.log ]; then
       mv /home/dkfqs/proxy/log/RemoteProxyRecorder.log /home/dkfqs/proxy/log/RemoteProxyRecorder.log_$(date +"%Y_%m_%d_%H_%M")
    fi
    sudo -H -u dkfqs bash -c 'CLASSPATH=/home/dkfqs/proxy/bin/bcmail-jdk15on-168.jar:/home/dkfqs/proxy/bin/bcpg-jdk15on-168.jar:/home/dkfqs/proxy/bin/bcpkix-jdk15on-168.jar:/home/dkfqs/proxy/bin/bcprov-jdk15on-168.jar:/home/dkfqs/proxy/bin/bctls-jdk15on-168.jar:/home/dkfqs/proxy/bin/com.dkfqs.remoteproxyrecorder.jar;export CLASSPATH;nohup java -Xmx4096m -DconfigProperties=/home/dkfqs/proxy/config/config.properties -Dnashorn.args="--no-deprecation-warning" com.dkfqs.remoteproxyrecorder.main.StartRemoteProxyRecorder 1>/home/dkfqs/proxy/log/RemoteProxyRecorder.log 2>&1 &'
    ;;
  stop)
       PID=`ps -o pid,args -e | grep "StartRemoteProxyRecorder" | egrep -v grep | awk '{print $1}'`
       if [ ! -z "$PID" ] ; then
          echo "RemoteProxyRecorder stopped with pid(s) : $PID"
          kill -9 ${PID} 1> /dev/null 2>&1
       fi
    ;;
  status)
       PID=`ps -o pid,args -e | grep "StartRemoteProxyRecorder" | egrep -v grep | awk '{print $1}'`
       if [ ! -z "$PID" ] ; then
          echo "RemoteProxyRecorder running with pid(s) : $PID"
       else
          echo "No RemoteProxyRecorder running"
       fi
    ;;
  *)
    echo "Usage: /etc/init.d/RemoteProxyRecorder {start|stop|status}"
    exit 1
    ;;
esac

exit 0

Change owner and file protection of /etc/init.d/RemoteProxyRecorder (root at /etc/init.d):

chown root RemoteProxyRecorder
chgrp root RemoteProxyRecorder
chmod 755 RemoteProxyRecorder

Register /etc/init.d/RemoteProxyRecorder to be started at system boot (root at /etc/init.d):

update-rc.d RemoteProxyRecorder defaults

Reboot the system. Login as dkfqs and check /home/dkfqs/proxy/log/RemoteProxyRecorder.log

Register and Verify the Remote Proxy Recorder

  1. Sign-in at the ‘Portal Server’
  2. Select at Top Navigation ‘HTTP/S Remote Proxy Recorders’
  3. Add your new Remote Proxy Recorder
  4. Ping the Remote Proxy Recorder at application level

“alt attribute”

“alt attribute”