A FunkLoad test is made of a typical unittest and a configuration file. Let’s look at a simple test script that is comming with the FunkLoad examples.
To get the demo examples you just need to run:
fl-install-demo
# Extract FunkLoad examples into ./funkload-demo : ... done.
cd funkload-demo/simple
Here is an extract of the simple demo test case test_Simple.py:
import unittest
from random import random
from funkload.FunkLoadTestCase import FunkLoadTestCase
class Simple(FunkLoadTestCase):
"""This test use a configuration file Simple.conf."""
def setUp(self):
"""Setting up test."""
self.server_url = self.conf_get('main', 'url')
def test_simple(self):
# The description should be set in the configuration file
server_url = self.server_url
# begin of test ---------------------------------------------
nb_time = self.conf_getInt('test_simple', 'nb_time')
for i in range(nb_time):
self.get(server_url, description='Get url')
# end of test -----------------------------------------------
if __name__ in ('main', '__main__'):
unittest.main()
The Simple test case extend FunkLoadTestCase and implement a test case named test_simple. this test case loop on a get request.
The FunkLoadTestCase extends the unittest.TestCase to add methods:
The target url, the number of requests are defined in the configuration files.
By convention the name of the configuration file is the name of the test case class with ”.conf” extension in our case: Simple.conf.
It is a plain text file with sections:
# main section for the test case
[main]
title=Simple FunkLoad tests
description=Simply testing a default static page
url=http://localhost/index.html
# a section for each test
[test_simple]
description=Access %(nb_time)s times the main url
nb_time=20
<<snip>>
# a section to configure the test mode
[ftest]
log_to = console file
log_path = simple-test.log
result_path = simple-test.xml
sleep_time_min = 0
sleep_time_max = 0
# a section to configure the bench mode
[bench]
cycles = 50:75:100:125
duration = 10
startup_delay = 0.01
sleep_time = 0.01
cycle_time = 1
log_to =
log_path = simple-bench.log
result_path = simple-bench.xml
sleep_time_min = 0
sleep_time_max = 0.5
Check that the url present in the main section is reachable, then invoking fl-run-test will run all the tests present in the test_Simple module:
$ fl-run-test -dv test_Simple.py
test_simple (test_Simple.Simple) ... test_simple: Starting -----------------------------------
Access 20 times the main url
test_simple: GET: http://localhost/index.html
Page 1: Get url ...
test_simple: Done in 0.006s
test_simple: Load css and images...
test_simple: Done in 0.002s
test_simple: GET: http://localhost/index.html
Page 2: Get url ...
<<snip>>
Page 20: Get url ...
test_simple: Done in 0.000s
test_simple: Load css and images...
test_simple: Done in 0.000s
Ok
----------------------------------------------------------------------
Ran 1 test in 0.051s
OK
To run a benchmark you invoke fl-run-bench instead of the test runner, you also need to select which test case to run.
The result of the bench will be saved in a single xml file simple-bench.xml, the name of this result file is set in the configuration file in the bench section.
You can override the configuration file using command line option, here we ask for 3 cycles with 1, 10 and 20 concurrents users (CUs).
$ fl-run-bench -c 1:10:20 test_Simple.py Simple.test_simple
========================================================================
Benching Simple.test_simple
========================================================================
Access 20 times the main url
------------------------------------------------------------------------
Configuration
=============
* Current time: 2011-01-26T23:22:51.267757
* Configuration file: /tmp/funkload-demo/simple/Simple.conf
* Log xml: /tmp/funkload-demo/simple/simple-bench.xml
* Server: http://localhost/index.html
* Cycles: [1, 10, 20]
* Cycle duration: 10s
* Sleeptime between request: from 0.0s to 0.5s
* Sleeptime between test case: 0.01s
* Startup delay between thread: 0.01s
Benching
========
* setUpBench hook: ... done.
Cycle #0 with 1 virtual users
-----------------------------
* setUpCycle hook: ... done.
* Start monitoring localhost: ... failed, server is down.
* Current time: 2011-01-26T23:22:51.279718
* Starting threads: . done.
* Logging for 10s (until 2011-01-26T23:23:01.301664): .. done.
* Waiting end of threads: . done.
* Waiting cycle sleeptime 1s: ... done.
* tearDownCycle hook: ... done.
* End of cycle, 14.96s elapsed.
* Cycle result: **SUCCESSFUL**, 2 success, 0 failure, 0 errors.
Cycle #1 with 10 virtual users
------------------------------
* setUpCycle hook: ... done.
* Current time: 2011-01-26T23:23:06.234422
* Starting threads: .......... done.
* Logging for 10s (until 2011-01-26T23:23:16.360602): .............. done.
* Waiting end of threads: .......... done.
* Waiting cycle sleeptime 1s: ... done.
* tearDownCycle hook: ... done.
* End of cycle, 16.67s elapsed.
* Cycle result: **SUCCESSFUL**, 14 success, 0 failure, 0 errors.
Cycle #2 with 20 virtual users
------------------------------
* setUpCycle hook: ... done.
* Current time: 2011-01-26T23:23:06.234422
* Starting threads: .......... done.
* Logging for 10s (until 2011-01-26T23:23:16.360602): .............. done.
* Waiting end of threads: .......... done.
* Waiting cycle sleeptime 1s: ... done.
* tearDownCycle hook: ... done.
* End of cycle, 16.67s elapsed.
* Cycle result: **SUCCESSFUL**, 14 success, 0 failure, 0 errors.
* tearDownBench hook: ... done.
Result
======
* Success: 40
* Failures: 0
* Errors: 0
Bench status: **SUCCESSFUL**
The xml result file can be turn into an html report this way:
$ fl-build-report --html simple-bench.xml
Creating html report: ...done:
/tmp/funkload-demo/simple/test_simple-20110126T232251/index.html
Note that there were no monitoring in our simple benchmark.
The process to write a new test is the following:
Use the recorder to initialize the test case and the configuration files and to grab requests.
Play the test and display each response in firefox, this will help you to add assertion and check the response:
fl-run-test -dV test_BasicNavigation.py
Implement the dynamic part:
For each request add an assertion to make sure the page is the one you expect. this can be done by checking if a term is present in a response:
self.assert_('logout' in self.getBody(), "Login failure")
Generates random input, you can use the FunkLoad.Lipsum module:
from FunkLoad import Lipsum
...
lipsum = Lipsum()
# Get a random title
title = lipsum.getSubject()
Extracts a token from a previous response:
from FunkLoad.utils import extract_token
...
jsf_state = extract_token(self.getBody(), ' id="javax.faces.ViewState" value="', '"')
Uses a credential server if you want to make a bench with different users or simply don’t want to hard code your login/password:
from funkload.utils import xmlrpc_get_credential
...
# get an admin user
login, pwd = xmlrpc_get_credential(host, port, "admin")
Configure the monitoring and automate your benchmark using a Makefile.