It doesn't have to conspire. There are known outcomes, and they take into account
the efforts of meddling time travellers.
And it doesn't have to be complicated or cartoon-like, although you can probably envisage the wannabe grandfather killer slipping on a banana skin on his way to the time machine. Or he gets the coordinates wrong. Or he succeeds in killing the man, only to discover the man wasn't his grandfather after all. Or he's gunned down by a time traveller from further in the future.
It's trivially easy to come up with scenarios that keep time consistent.
Not at all.
Forget time travel for a moment, and imagine this:
You see someone's death certificate. Then a bit later you discover that the events leading up to his death were recorded on a video camera. You watch the video of a medical team doing what they can to save his life. No matter how promising their actions appear to be, you know how it's going to end. It might appear contrived, it might appear as if the universe is conspiring to kill that man, but of course it isn't really. It merely seems that way because you've seen events in the wrong order.
The only difference with time travel is that you are potentially a participant
in a situation with a known outcome.