Bear in mind, what has become more accepted is that ballistic lithopanspermia is in principle possible; no non-controversial evidence that it has actualy happened has come to light as far as I know.
I can follow your logic wrt interstellar panspermia, expecially considering that conditions were much more favorable for it when our solar system was young and part of a dense packed open cluster, but again, there is no evidence that is has actualy occured. Great theory without evidence is ultimately just very clever spectulation.
If the island is all you have ever known, and the sea is dead as far as you can tell, then
yes the logical assumption is that the island life is all there is.
Doesn't mean you close your mind to the idea of extra-island life, refuse to theorize about it, or refuse to accept incontrovertable proof ( a lizard on a drift wood branch, the QE2, whatever) if it does come along, but to assume that other life exists without seeing it is just as illogical as denying it when you do see it.