Bitter pill to swallow
A bitter pill to swallow is something that is hard to accept.