How to alter invocation history with mockito?

Note: mockito 1.10.17

The scenario is as follows; I have an abstract class as such:

/**
 *
 * @param <M> metadata class
 */
@ParametersAreNonnullByDefault
public abstract class AttributeWriter<M>
    implements AttributeWriterByName
{
    protected final MetadataDriver<M> driver;
    protected final Path path;

    protected AttributeWriter(final Path path, final MetadataDriver<M> driver)
    {
        this.driver = driver;
        this.path = path;
    }
}

The AttributeWriterByName interface is pretty simple:

@ParametersAreNonnullByDefault
public interface AttributeWriterByName
{
    void setAttributeByName(String name, Object value)
        throws IOException;
}

An example of an implementation is this (the most simple there is):

@SuppressWarnings("DesignForExtension")
@ParametersAreNonnullByDefault
public abstract class FileOwnerAttributeWriter<M>
    extends AttributeWriter<M>
{
    protected FileOwnerAttributeWriter(final Path path,
        final MetadataDriver<M> driver)
    {
        super(path, driver);
    }

    @Override
    public void setAttributeByName(final String name, final Object value)
        throws IOException
    {
        if (!"owner".equals(Objects.requireNonNull(name)))
            throw new NoSuchAttributeException(name);
        setOwner((UserPrincipal) Objects.requireNonNull(value));
    }

    @SuppressWarnings("MethodMayBeStatic")
    public void setOwner(final UserPrincipal owner)
        throws IOException
    {
        throw new ReadOnlyAttributeException();
    }
}

The matching test file relies on an abstract class which reads as this:

public abstract class AttributeWriterTest<W extends AttributeWriter<Object>>
{
    protected final Path path = mock(Path.class);
    @SuppressWarnings("unchecked")
    protected MetadataDriver<Object> driver = mock(MetadataDriver.class);

    protected W writer;

    protected static FileTime nullFileTime()
    {
        return isNull(FileTime.class);
    }

    @BeforeMethod
    public abstract void init()
        throws IOException;
}

As to the actual test class, it is:

public final class FileOwnerAttributeWriterTest
    extends AttributeWriterTest<FileOwnerAttributeWriter<Object>>
{
    @BeforeMethod
    @Override
    public void init()
        throws IOException
    {
        writer = spy(new FileOwnerAttributeWriter<Object>(path, driver)
        {
        });

        doNothing().when(writer).setOwner(any(UserPrincipal.class));
    }

    @Test
    public void setOwnerTest()
        throws IOException
    {
        final UserPrincipal owner = mock(UserPrincipal.class);

        writer.setAttributeByName("owner", owner);

        verify(writer).setOwner(same(owner));
    }
}

Now, the problem I have is as follows: I basically want to check that "after" the invocation of .setAttributeName(), nothing else happens other than .setOwner().

But if I add to the test that I want to verifyNoMoreInteractions(writer), this will fail since in this same test I invoke .setAttributeByName()...

Notwithstanding that this MAY be a problem with my design, is there a way to, for instance, tell mockito to ignore, somewhat, the call to setAttributeByName() in such a test so that I can write this?

verify(writer, only()).setOwner(same(owner));