Load testing Open WebUI deployments

Setup

pyenv virtualenv 3.12.2 locust-llm-3.12.2
pyenv local locust-llm-3.12.2

pip install locust==2.43.0
pip install python-dotenv

locust -V

Copy the .env.example as .env and edit as needed

cp .env.example .env

Run tests

Configuration options

Configure the Locust tests in the locust.conf file as needed

For options, see the Locust docs

You can override all of the settings set in the configuration file by setting the same on the command line. For example, to override the log level settings use --loglevel as in example:

locust --headless -f ./tests/test_verify_setup.py --loglevel debug

Running the tests

The majority of the tests in this project are designed to be executed using the Locust UI. To use, execute the provided locust command in a terminal and then open the Locust UI in a browser.

Open http://localhost:8089

Capturing test results

Each of the test cases (test files) will generate an HTML report file upon completion. Reports can be disabled in the UI.

• The report files are saved in the ./reports/ directory.
• There is one report filename per test file.
• The HTML reports are uniquely saved with a timestamp.

To also capture test result stats in csv files, append a command line argument and name the filename prefix, e.g.:

--csv ./stats/test_api

Run web tests

Run this test with 10 users for 1 minute.

locust -f ./tests/test_owui_web.py --users 10 --run-time 1m

Run API tests

Run this test with 2 users for 1 minute.

locust -f ./tests/test_owui_api.py --users 2 --run-time 1m

Run the basic LLM chat tests

Run this test with 1 user for 1 minute.

locust -f ./tests/test_llm_basic.py --users 1 --run-time 1m --loglevel debug

Run the Biochatterbasic LLM chat tests

Parts of these tests are based on the Biochatter project

Run this test with varying amounts of users for 3 minutes. For example, to run with 2 concurrent users:

locust -f ./tests/test_llm_biochatter_med_exam.py --users 2 --run-time 3m

To run a selection of the test tasks, you can filter by tag(s), e.g.

locust -f ./tests/test_llm_biochatter_med_exam.py --users 2 --run-time 3m --tags bool

or exclude tag(s), e.g.

locust -f ./tests/test_llm_biochatter_med_exam.py --users 2 --run-time 3m --exclude-tags bool

Verify setup of Locust

locust --headless -f ./tests/test_verify_setup.py --users 1 --run-time 10s --loglevel debug

Development

Install and use ruff

python -m pip install -U ruff
ruff check .
ruff format --check .