author: | Benoit Delbosc |
---|---|
address: | bdelbosc _at_ nuxeo.com |
abstract: | This document describes changes between FunkLoad versions. |
Table of Contents
git: | https://github.com/nuxeo/FunkLoad |
---|---|
Target: | 1.16.2 |
Package: | http://pypi.python.org/packages/source/f/funkload/funkload-1.16.1.tar.gz |
---|---|
github: | https://github.com/nuxeo/FunkLoad/tree/1.16.1 |
Released date: | 2011-07-28 |
Package: | http://pypi.python.org/packages/source/f/funkload/funkload-1.16.0.tar.gz |
---|---|
github: | https://github.com/nuxeo/FunkLoad/tree/1.16.0 |
Released date: | 2011-07-21 |
New monitoring plugins contribution from Krzysztof A. Adamski. You can extend the monitoring by adding new plugins checkout the Munin and Nagios examples.
Extends HTTP methods with HEAD and OPTIONS, enable to send any methods using the new method api, adding PROPFIND DAV exemple
# requests for options
self.options(server_url, description="options")
self.propfind(server_url + '/dav/foo', depth=1, description="propfind")
# put a lock using the new method api
data = Data('text/xml', """<?xml version="1.0" encoding="utf-8"?>
<D:lockinfo xmlns:D="DAV:">
<D:lockscope><D:exclusive/></D:lockscope>
<D:locktype><D:write/></D:locktype>
<D:owner>
<D:href>http://www.webdav.org/webdav_fs/</D:href>
</D:owner>
</D:lockinfo>""")
self.method("LOCK", server_url + "/somedocpath", params=data, description="lock")
GH-9: New options for distributed mode: you can specify the Python binary to use with --distribute-python=PYTHON_BIN option and additional packages to install on slaves with the --distributed-packages=DISTRIBUTED_PACKAGES option. Contribution from Adrew McFague.
GH-9: Added the ability to monitor hosts from the central host, The monitoring will also retroactively match up the concurrent users for graphing purposes. Contribution from Adrew McFague.
GH-5: fl-record now checks for tcpwatch command in addition to tcpwatch.py and tcpwatch-httpproxy.
GH-6: Filter out .swfs when building funkload tests with fl-record.
GH-10: Support of python2.7.
GH-15: Exposed the load_auto_links parameter to the get/put/post/method methods so that the option to not parse the payload is available when writing a test. Provided by Ali-Akber Saifee.
Package: | http://pypi.python.org/packages/source/f/funkload/funkload-1.15.0.tar.gz |
---|---|
github: | https://github.com/nuxeo/FunkLoad/tree/1.15.0 |
Released date: | 2011-03-11 |
Now supporting emacs Org-mode text format as report output. This enable to edit a report as plain text and to produce professional PDF reports througth the Org-mode LaTeX export.
Here is an example of a PDF and an Org-mode report.
You need to build the HTML report first:
fl-build-report --html funkload.xml
Creating html report: ...done:
/tmp/test_foo-20110304T160328/index.html
# then create the org file
fl-build-report --org funkload.xml > /tmp/test_foo-20110304T160328/index.org
emacs /tmp/test_foo-20110304T160328/index.org
# then export as PDF C-c C-e d
# refer to the org-mode site for latex exports requirements
New trend report to display evolution of performances over time. Just use the --trend option of the fl-build-report command.
The credential server can serve a sequence. Using xmlrpc_get_seq threads can share a sequence:
from funkload.utils import xmlrpc_get_seq
...
seq = xmlrpc_get_seq()
Source migrated from svn to git, hosted in gigthub https://github.com/nuxeo/FunkLoad
Bug tracker moving to github: https://github.com/nuxeo/FunkLoad/issues
New site and documentation using sphinx: http://funkload.nuxeo.org/
CPSTestCase and ZopeTestCase have been moved to the demo folder.
Removing deprecated gdchart support, now relying only on gnuplot. Note that a mathplotlib support is on the TODO list.
Package: | http://pypi.python.org/packages/source/f/funkload/funkload-1.14.0.tar.gz |
---|---|
Svn: | http://svn.nuxeo.org/pub/funkload/tags/1.14.0 |
Released date: | 2011-02-14 |
Switch to Sphinx for documentation, this work is in progress, the draft can be found in http://funkload.nuxeo.org/sphinx/. Thanks to Ali-Akber Saifee.
Support of HTTP PUT and DELETE method, provided by Ali-Akber Saifee.
Distributed mode (beta), provided by Ali-Akber Saifee, visit the new FAQ http://funkload.nuxeo.org/sphinx/faq.html#how-to-run-multiple-bencher for more information.
Support of the --simple-fetch option using the configuration file. This can be useful when using benchmaster http://pypi.python.org/pypi/benchmaster. Just add simple_fetch = 1 in the bench section.
Add setUpBench and tearDownBench hooks, they are called only once per bench before and after all cycles.
Note that these hooks are called only using fl-run-bench and no fl-run-test.
This comes in addition to the existing setUpCycle and tearDownCycle hooks that are run before and after each bench cycle.
Enable to add metadata for a bench using addMetadata(kw), metadata are stored into the bench result file and are displayed on the bench configuration section.
A typical usage is to add metadata during setUpBench or tearDownBench hooks.
Handling FunkLoad todo list with an Org-mode file TODO, replacing the old trac.
Mark CPSTestCase as deprecated will be removed in 1.15.
Mark GDChart support as deprecated, it will be removed in 1.15.
Package: | http://pypi.python.org/packages/source/f/funkload/funkload-1.13.0.tar.gz |
---|---|
Svn: | http://svn.nuxeo.org/pub/funkload/tags/1.13.0 |
Released date: | 2010-07-27 |
Adding Apdex (Application Performance inDEX) on reporting.
The Apdex is a numerical measure of user satisfaction, it is based on three zones of application responsiveness:
- Satisfied: The user is fully productive. This represents the time value (T seconds) below which users are not impeded by application response time.
- Tolerating: The user notices performance lagging within responses greater than T, but continues the process.
- Frustrated: Performance with a response time greater than 4*T seconds is unacceptable, and users may abandon the process.
By default T is set to 1.5s this means that response time between 0 and 1.5s the user is fully productive, between 1.5 and 6s the responsivness is tolerating and above 6s the user is frustrated.
The T variable can be set using the fl-build-report option --apdex-T.
The Apdex score converts many measurements into one number on a uniform scale of 0-to-1 (0 = no users satisfied, 1 = all users satisfied).
To ease interpretation the Apdex score can be converted to a rating:
By looking at the Page stats in the report you can take the maximum throughput with a Good/Excellent rating as an overall performance score.
Note that this new metric can be generated with old funkload results file.
Here is a report example: http://funkload.nuxeo.org/report-example/test_seam_java6/#page-stats
Visit http://www.apdex.org/ for more information on Apdex.
The selection of the best cycle to sort slowest requests in the report is found using the product of throuthput and apdex score instead of simple maximum.
Package: | http://pypi.python.org/packages/source/f/funkload/funkload-1.12.0.tar.gz |
---|---|
Svn: | http://svn.nuxeo.org/pub/funkload/tags/1.12.0 |
Released date: | 2010-05-26 |
fl-build-report is now able to create a report using many result files. The goal of this feature is to create bench report for a distributed bench. To be merged the result files must have the same cycles and cycle duration.
The hostname and python version is now logged in the result file.
Try to use psyco if installed. Note that psyco works only on 32-bit system.
Miscellaneous speed improvments: removing minimal sleep time between actions, removing useless logs.
New --as-fast-as-possible or -f option to fl-run-bench to remove sleep any times between requests and test cases, this is equivalent to -m0 -M0 -t0.
New --enable-debug-server option for fl-run-bench command. This option run a debug HTTP server which exposes an interface using which parameters can be modified at run-time.
Currently supported parameters:
You can load a server and be able to change the number of concurrent users during the run without pre-specifying it through ‘cycles’ argument, this is usefull during debugging or profiling.
This feature has been provided by Goutham Bhat.
Package: | http://pypi.python.org/packages/source/f/funkload/funkload-1.11.0.tar.gz |
---|---|
Svn: | http://svn.nuxeo.org/pub/funkload/tags/1.11.0 |
Released date: | 2010-01-26 |
Package: | http://pypi.python.org/packages/source/f/funkload/funkload-1.10.0.tar.gz |
---|---|
Svn: | http://svn.nuxeo.org/pub/funkload/tags/1.10.0 |
Released date: | 2009-03-23 |
Package: | http://pypi.python.org/packages/source/f/funkload/funkload-1.9.0.tar.gz |
---|---|
Svn: | http://svn.nuxeo.org/pub/funkload/tags/1.9.0 |
Released date: | 2008-11-07 |
Switch to gnuplot 4.2 for charts generation. Gnuplot script and data are kept with the report enabling charts customization. No more gdchart charting woes.
Enhanced charts displaying more information, view a report example.
New diff report option, comparing 2 reports is a long and error prone task, fl-build-report has been enhanced to produce a differencial report. This new feature requires gnuplot.
To produce a diff report:
fl-build-report --diff path/to/report-reference path/to/report-challanger
View a diff report example.
Post method handles custom data and content type. For example to post a xmlrpc by hand
from funkload.utils import Data
...
data = Data('text/xml', """<?xml version='1.0'?>
<methodCall>
<methodName>getStatus</methodName>
<params>
</params>
</methodCall>
""")
self.post(server_url, data, description="Post user data")
The recorder translates properly application/xml or any content type using the new Data class (see above exemple).
New test script provided with fl-install-demo to bench the JBoss Seam Booking application.
Package: | http://pypi.python.org/packages/source/f/funkload/funkload-1.8.0.tar.gz |
---|---|
Svn: | http://svn.nuxeo.org/pub/funkload/tags/1.8.0/ |
Released date: | 2008-07-28 |
Package: | http://funkload.nuxeo.org/funkload-1.7.0.tar.gz |
---|---|
Svn: | http://svn.nuxeo.org/pub/funkload/tags/1.7.0/ |
Released date: | 2008-07-23 |
Package: | http://funkload.nuxeo.org/funkload-1.6.2.tar.gz |
---|---|
Svn: | http://svn.nuxeo.org/pub/funkload/tags/1.6.2/ |
Released date: | 2007-04-06 |
Package: | http://funkload.nuxeo.org/funkload-1.6.1.tar.gz |
---|---|
Svn: | http://svn.nuxeo.org/pub/funkload/tags/1.6.1/ |
Released date: | 2007-03-09 |
Package: | http://funkload.nuxeo.org/funkload-1.6.0.tar.gz |
---|---|
Svn: | http://svn.nuxeo.org/pub/funkload/tags/1.6.0/ |
Released date: | 2007-02-27 |
Package: | http://funkload.nuxeo.org/funkload-1.5.0.tar.gz |
---|---|
Svn: | http://svn.nuxeo.org/pub/funkload/tags/1.5.0/ |
Released date: | 2006-01-31 |
# 1284: TestCase: support of doctest
There is a new FunkLoadDocTest class that ease usage of FunkLoad from doctest:
>>> from funkload.FunkLoadDocTest import FunkLoadDocTest
>>> fl = FunkLoadDocTest()
>>> response = fl.get('http://localhost/')
>>> 'HTML' in response.body
True
>>> response
<response url="http://127.0.0.1:80//" code="200" message="OK" />
If you use python2.4, the test runner fl-run-test is able launch doctest from a plain text file or embedded in python docstring:
$ fl-run-test -v doctest_dummy.txt
Doctest: doctest_dummy.txt Ok
-----------------------------------------------------
Ran 1 test in 0.077s
OK
And the --debug option makes doctests verbose:
$ fl-run-test -d doctest_dummy.txt
...
Trying:
fl = FunkLoadDocTest()
Expecting nothing
ok
Trying:
fl.get('http://localhost/')
Expecting:
<response url="http://127.0.0.1:80//" code="200" message="OK" />
ok
Ok
----------------------------------------------------------------------
Ran 1 test in 0.051s
OK
Test runner can use a negative regex to select tests. For example if you want to launch all tests that does not ends with ‘foo’
fl-run-test myFile.py -e '!foo$'
# 1282: TestRunner: more verbosity
The new fl-run-test option --debug-level=2 will produce debug output on each link (images or css) fetched.
Improve firefox view in real time by using approriate file extention for the content type.
CPSTestCase is up to date for 3.4.0, use CPS338TestCase for a CPS 3.3.8.
Package: | http://funkload.nuxeo.org/funkload-1.4.1.tar.gz |
---|---|
Svn: | http://svn.nuxeo.org/pub/funkload/tags/1.4.1/ |
Released date: | 2005-12-16 |
Package: | http://funkload.nuxeo.org/funkload-1.4.0.tar.gz |
---|---|
Svn: | http://svn.nuxeo.org/pub/funkload/tags/1.4.0/ |
Released date: | 2005-12-08 |
New --loop-on-pages option for fl-run-test.
This option enable to check response time of some specific pages inside a test without changing the script, which make easy to tune a page in a complex context. Use the debug option to find the page numbers. For example:
fl-run-test myfile.py MyTestCase.testSomething -l 3 -n 100
Run MyTestCase.testSomething, reload one hundred time the page 3 without concurrency and as fast as possible. Output response time stats. You can loop on many pages using slice -l 2:4.
New --accept-invalid-links option for fl-run-test and fl-run-bench
Don’t fail if css/image links are not reachable.
New --list option for fl-run-test to list the test names without running them.
# 936: TestRunner: use regexp to load test
New --regex or -e option for fl-run-test to filter test names that match a regular expression.
# 939: Browser: Provide an option to disable image and links load
New --simple-fetch option for fl-run-test and fl-run-bench.
# 937: TestRunner: Add an immediate fail option
New --stop-on-fail option for fl-run-test that stops tests on first failure or error.
# 933: Report: Add global info
Adding total number of tests, pages and requests during the bench.
CPSTestCase.listDocumentHref is renamed into cpsListDocumentHref
FunkLoadTestCase.xmlrpc_call is renamed into xmlrpc (xmlrpc_call is still working)
Some code cleaning, cheesecake index 460/560 ~82%.
fl-run-test is now able to run standard unittest.TestCase.
Package: | http://funkload.nuxeo.org/funkload-1.3.1.tar.gz |
---|---|
Svn: | http://svn.nuxeo.org/pub/funkload/tags/1.3.1/ |
Released date: | 2005-11-10 |
Package: | http://funkload.nuxeo.org/funkload-1.3.0.tar.gz |
---|---|
Svn: | http://svn.nuxeo.org/pub/funkload/tags/1.3.0/ |
Released date: | 2005-11-08 |
# 944: Recorder: replace TestMaker recorder.
Providing a fl-record command that drive a TCPWatch proxy. See INSTALL_ to setup TCPWatch.
# 1041: Browser: implement an addHeader method.
FunkLoadTestCase provides new methods setUserAgent, addHeader and clearHeaders.
# 1088: TestRunner / BenchRunner: use compatible command line option
- All fl-* executables have a --version option to display the FunkLoad version.
- All fl-run-* are now in color mode by default. Use --no-color options for monochrome output. You need to remove the -c option for fl-run-test and -C for fl-run-bench in your scripts.
- Changing fl-run-bench short option -d into -D for duration, keeping -d for debug mode.
- Removing fl-run-test short option -D to not conflict with new -D option of fl-run-bench, you now have to use the long format --dump-directory.
Package: | http://funkload.nuxeo.org/funkload-1.2.0.tar.gz |
---|---|
Svn: | http://svn.nuxeo.org/pub/funkload/tags/1.2.0/ |
Released date: | 2005-10-18 |
Package: | http://funkload.nuxeo.org/funkload-1.1.0.tar.gz |
---|---|
Svn: | http://svn.nuxeo.org/pub/funkload/tags/1.1.0/ |
Released date: | 2005-10-07 |
Location: | http://funkload.nuxeo.org/funkload-1.0.0.tar.gz |
---|---|
Released date: | 2005-09-01 |
First public release.
More information on the FunkLoad site.