Test always fails with Test Explorer Run All, but always passes with Debug Test

I have the following piece of code:

public override async Task<string> ExecuteAsync(string googleAuthToken)
{
    string apikeyKeyname = "token";

    dynamic response = await BuildFullApiPath()
            .SetQueryParams(new { token = googleAuthToken })
            .GetJsonAsync();

    string receivedApiKey = ((IDictionary<string, object>) response)[apikeyKeyname].ToString();
    Debug.WriteLine("GetApiKey: received apikey is " + receivedApiKey);

    return receivedApiKey;
}

And the xUnit test for that code:

    [Fact]
    public async void ShouldBeAbleToGetApiKeyFromToken()
    {
        // Arrange
        using (var httpTest = new HttpTest())
        {
            var jsonResponse = JsonConvert.DeserializeObject(@"{""token"":""abcdef""}");
            string expectedApiKey = "abcdef";
            httpTest.RespondWithJson(jsonResponse);
            var api = new GetApiKey();

            // Act
            var receivedApiKey = await api.ExecuteAsync("mockToken");
            output.WriteLine("Received apikey = " + receivedApiKey);

            // Assert
            Assert.Equal(expectedApiKey, receivedApiKey);
        }
    }

The test above only succeeds when I set a breakpoint inside ExecuteAsync(), debug the test (right click on the test in Test Explorer and debug), and step over line by line. But when I press Run All, the test always fail with System.Collections.Generic.KeyNotFoundException : The specified key 'token' does not exist in the ExpandoObject. When I debug the test and step over line by line, receivedApiKey always has "abcdef" as passed in by the unit test, but when I Run, receivedApiKey in the test is always null.