Some people feel pride at the history of their society. I'm among them. Maybe it's irrational, I'm not sure; it probably is. But there you are.
It's obviously irrational and illegitimate to only pay attention to the good stuff. In my society, conservatives (rightly) insist we "remember the ANZACs", that is to say the hundreds of thousands of people who fought to defend the world against German aggression in the First World War. The line is literally "lest we forget"; that's literally chiselled into stone all over the damn country. Often in the same breath, they demand Indigenous Australians "get over" all the nasty things we did to them. Much of our Indigenous population was still alive at the time of the Stolen Generations - which basically ended in the 60s - and 100% of them continue to feel the consequences of all those nasty things conservatives demand we forget ever happened and do nothing about.
White guilt is basically a natural reaction to history, as natural as pride. If you feel pride in your country but not guilt, you're almost certainly a racist fuck (or, perhaps, ignorant).
Now, as to whether it is useful, I feel that it is. What is the logical consequence of white guilt? We as a society did these nasty things, they had these consequences, I feel bad, I can assuage my bad feelings by doing something to make it better. No?