Alan Dershowitz

Good character consists of recognizing the selfishness that inheres in each of us and trying to balance it against the altruism to which we should all aspire. It is a difficult balance to strike, but no definition of goodness can be complete without it.

You can hold yourself back from the sufferings of the world, that is something you are free to do and it accords with your nature, but perhaps this very holding back is the one suffering you could avoid.