The Designer allows simulating performance tests, by the means of running multiple sequential configurations for one or multiple users.

These tests are intended to globally review model performances after the release of a new version of the model.

• Make sure that no other processes are running (CPU and memory have to be as similar as possible throughout multiple runs)
• Make sure that no other users are modeling in the Designer
• Make sure that no other users are running/testing models in the Configurator
• Put on hold all modeling activities when conducting performance tests

In order to inspect a test case, proceed as follows:

How-To:

• [In the menu toolbar]
  • Click on the step “Run”
• [In the explorer]
  • Open the test case by clicking on its description
    • The test case opens up
• [In the menu toolbar]

  Enter the performance test parameters:

  • Enter the number of simultaneous users: the number of concurrent users that will be simulated
  • Enter the number of sequential runs: the number of sequential configuration sessions that each user will go through
  • Enter the ramp up time: the number of seconds to wait between one user launching and the next one.
  • Enter the wait time mode: soonest or specified. Soonest means that the configuration will run “as fast as possible”; specified means that the latency registered on the “Test case” step will be used.
• [In the menu toolbar]

  Execute the “Inspect Test Case” function.

  • The “status” shows the progress of the test run (blue: running, green: complete & valid, red: errors).

In order to inspect a group of test cases, proceed as follows:

How-To:

• [In the menu toolbar]
  • Click on the step “Run”
• [In the explorer]
  • Open the test campaign by clicking on its description
    • The test campaign opens up
• [In the working section]
  • Select all test cases that need to be validated
• [In the menu toolbar]

  Enter the performance test parameters:

  • Enter the number of simultaneous users: the number of concurrent users that will be simulated
  • Enter the number of sequential runs: the number of sequential configuration sessions that each user will go through
  • Enter the ramp up time: the number of seconds to wait between one user launching and the next one.
  • Enter the wait time mode: soonest or specified. Soonest means that the configuration will run “as fast as possible” ; specified means that the latency registered on the “Build test case” step will be used.
• [In the menu toolbar]

  Execute the “Inspect Test Case” function.

  • The “status” shows the progress of the test run (blue: running, green: complete & valid, red: errors).
  • For each test case, the following information is registered:
    • “min”: the minimum performance (cumulated response time) of the test case in milliseconds
    • “max”: the maximum performance (cumulated response time) of the test case in
      milliseconds:

      “avg”: the average performance (cumulated response time) of the test case in milliseconds

In order to obtain a report for a test campaign, proceed as follows:

How-To:

Proceed to the performance test of one or multiple test cases. Once the performance test has finished:

• [In the explorer]
  • Execute the function “Download report”
• [In the popup]

  Enter the parameters:

  • CSV delimiter
  • File encoding Finally, click on “OK”
    • The report is now generated and can be downloaded.

Reports contain the following information:

• Software version
• Model name & version
• Run date & time
• Test parameters (simultaneous users, sequential configurations, ramp up time, wait mode)
• For each test case: min/max/avg response time
  Warning: In order to obtain comparable results, please ensure that the reports are stored together with a model export. For obvious reasons, performance test results can only be compared when knowing the differences in the models between different tests.