Any thoughts about the defensibility of early-stage GPT-powered products?

The GPT craze is going strong. The accessibility of LLMs and generative models creates a new wave of verticalized AI-powered product possibilities. When such technology gets widely accessible, and the cost to run them gets closer to zero, any individual anywhere in the world can compete and build the exact product you're making. Today with GPT, any idea you have is replicated ten times the next day. What are your thoughts on defensibility when/if starting a GPT or open-source LLM-based product?