Yes,it would fly off in a straight line on a tangent because once there is no centripetal force,then at the same time centrifugal force cease to exist.
Newton has not explained eveything.He cannot explain gravity which is also considered a fictitious force.If you jump off a building no force acts on you but you will still be accelerated towards the ground and when you hit the ground,you better hope gravity is not a real force either.
Newton explained attraction between masses. Gravity is the force between the mass of the planet and any other mass. Plugging the size and mass of the Earth into F=GMm/r^2 breaks down to F=mg as weight = mass multiplied by constant g.
Jumping off that building you are accelerating towards the Earth because you have mass. (If I remember, I think the Earth will also move towards you as well, but only a tiny amount because your mass is so small relative to the Earth, assuming there are no other masses moving anywhere else on the planet.)
Why there is a force of attraction between two masses is another story, but this doesn't seem ficticious to me.
A passenger in a car driving really quickly in a circle will feel like they are subject to a force away from the centre of the circle. Their mass wants to fly off in a straight line and all that is stopping them (besides negligible friction, etc.) is the outside door of the car; they are only following a circular path because the door is there. If the car drove in a square, they would feel the force against the door four times, five for a pentagon, six for a hexagonal path and so on. Keep going and it would soon feel like a constant force, but I can't see that it then becomes a new force with a different name.
This is force, but I don't see this as centrifugal force, so might need it explaining again.
