Unit Test Parametric Abstact Fatory

Consider the following design:

public class PhoneAbstractFactory
    {
        public IPhoneFactory GetPhoneFactory(string type, bool needSmartPhone)
        {
            if (type == "Samsung")
                return new SamsungFactory(needSmartPhone);

            if (type == "Apple")
                return new AppleFactory(needSmartPhone);

            else throw new Exception("Unknown phone type: " + type);
        }
    }

    public interface IPhoneFactory  { }

    public class SamsungFactory : IPhoneFactory
    {
        public SamsungFactory(bool needSmartPhone)
        {
        }
    }

    public class AppleFactory : IPhoneFactory
    {
        public AppleFactory(bool needSmartPhone)
        {
        }
    }

How can I test only PhoneAbstactFactory ? If I want to test it, inevitably an instance of either AppleFactory or SamsungFacory should get be instantiated; which means that if the test is to be passed, it is needed that constructing both of the factories must be always assumed truth-worthy. Otherwise, test scope spans over the two factories which is not good.