Unit-testing a boost::python library in python

So I have a shared library created with boost::python (C++). For the C++ functions inside I have unit-tests that check that they are working. Now I would like to use unit-test to see if I implemented the python interface correctly. For this I thought about using using the python package unittest.

Now my Folder setup is roughly:

project
  |
   -- C++ source (library and boost::python stuff)
  |
   -- build (here the shared library is located)
  |
   -- Test (here I have the python classes that should test the interface)

The test folder has some subfolders that mirror the structure of the python interface, containing lots of small python modules testing the different aspects of the library.

So the question now:

How do I import the shared library into the test?

What I tried so far was in my test_main.py

import sys
sys.path.insert(0,'../build')

But this does not help for the modules inside the test folder. And anyways hardcoding this path into the test-code seems to be hackish. I also don't want to install an untested library just to figure out the tests failed to then uninstall it again.