C# MSTest - Delete file in TestInitialize

When choosing to run all tests, and there is logic within [TestInitialize] method to delete a file, anything after the first completed test receives the following error:

The process cannot access the file 'C:\root\var\MonRequestGeneratorTests.sqlite' because it is being used by another process..

Here is the Initialize method:

[TestInitialize]
public override void Initialize()
{
    string sqliteFilePath = "myPath";
    if (File.Exists(sqliteFilePath)) {
        File.Delete(sqliteFilePath);
    }
}

How do I get around this with tests? This is a SQLite database, and the means of dropping it is deleting a file. I need that file to be deleted prior to each test running.