mardi 3 mars 2015

Using performance measurement in functional testing

I'm trying to add performance measurement to my unit testing. My general idea is as follow:



  • I have predefined rest api calls that do some logic behind the scenes, some of the logic is used to query the DB

  • I want to fill the DB with mock data (10K rows for example) taken from production system

  • I'm calling the rest api randomly with various parameters (again taken from production). Doing the same batch of calls multiple times and calculating the average time

  • The first time running this test will produce a base number which will be used for comparison in subsequent runs, this one is being done manually and not part of the unit tests.

  • If the amount of time it takes to run all these queries exceeds a certain threshold (5% for example) above the base number I calculated in the previous bullet then this test fail.


The biggest problem is that the number varies on each computer that it runs, therefore I'll have to calculate the base number on each new computer



  • Does this make sense ?

  • Time of execution is not very "portable" unit of measurement, each environment will have different numbers, any other unit of measurement that you can think of that can be shared ?


Aucun commentaire:

Enregistrer un commentaire