Unit testing with Core Data created from remote data source

I'm starting to build tests into a legacy code base. It relies heavily on Core Data and all of the information in the datastore is fetched from a server when the app first runs. The model is very complex - in other words I can't really create entities manually for each test.

I'm an experienced iOS dev but have little testing experience. What is the best way to handle this? Should I download the required data from the server in my setUp() method? I should also mention that the data is downloaded via an NSOperationQueue so is asynchronous.