lundi 3 août 2015

How to decide where Python debugger stops and which line is to be blamed?

Background: I write Squish GUI tests in Python. I tried to make test code as Pythonic and DRY as I could and hence I moved all repeating code to separate classes / modules.

Problem definition: test.verify or assert statement tells the debugger to stop at the very line where the statement is and that's in most cases the module with details of single test step. This line is shown in eclipse during manual run and output by automatic test in Jenkins.

To actually see what failed in test it would be far better to stop the debugger at the invocation point of the procedures with asserts inside. Then tester / GUI developer can spot what actions on GUI lead to a problem and what was checked.

Example:

test_abstract.py

class App():
    def open_file(self, filename):
        pass # example

    def test_file_content(content):
        # squish magic to get file content from textbox etc.
        # ...
        test.verify(content in textBoxText)

test_file_opening.py

def main():
   app = App()
   app.open_file('filename.txt')
   app.test_file_content('lorem')

As the test fails on test.verify() invocation the debugger stops and directs to test_abstract.py file. It actually say nothing about test steps that lead to this test failure.

Is there a way to tell debugger to ignore direct place of test failure and make it show where the procedure with test was invoked. I'm looking for elegant way that would not need too much of code in generic test file itself.

Not ideal solution which works: For now I'm not using test.verify inside of abstract modules and invoke this in the particular test case code. Generalized test functions return a tuple (test_result, test_descriptive_message_with error) which is unpacked with *:

def test_file_content(content):
    # test code
    return (result, 'Test failed because...')

and test case code contains:

test.verify(*test_file_content('lorem'))

which works fine, but the each and every test case code has to contain a lot of test.verify(*... and test developers have to remember about it. Not to mention that it looks wet... (not DRY).

Aucun commentaire:

Enregistrer un commentaire