Washington's National Cathedral is facing a difficult decision. It has quietly removed images of the Confederate flag from its windows as its display has become increasingly unacceptable. But what about the much larger windows displaying portraits of famous Confederate generals?

The symbols of the Southern states during the Civil War have always raised mixed feelings, but it's since the Charleston shootings of black churchgoers by white supremacist Dylann Roof that they've become a flashpoint in conversations about race. The South, in spite of revisionist attempts to depict the conflict as all about states' rights, fought to maintain slavery; the North fought to get rid of it.

So when the Robert E Lee window describes him as "a Christian soldier without fear and without reproach" and the Jackson window says he "walked humbly before his Creator whose word was his guide", a very large section of the American population is entitled to say, "Seriously?"

Whatever their private virtues, these are men whose cause was abominable. Why should they be commemorated in a cathedral?

The row in Washington echoes others around the world. The Rhodes Must Fall protest movement started in South Africa and was focused on removing a statue of Cecil Rhodes, the colonial hero who is far from a hero to the black majority population who suffered the consequences of his actions for generations. It spread to Oxford University, whose Oriel College also has a statue of Rhodes. In Cambridge the movement inspired a campaign for the return of the okukor cockerel at Jesus College to Nigeria, from where it was stolen during the Benin expedition in 1897.

All over the world, educational and other institutions are facing demands to expunge the memories of the past. Given the nature of that past – the brutality of colonialism and racism, the casual oppression of minorities in the name of perverted nationalism or naked commercial self-interest ­­– it's impossible not to have some sympathy.

But at the same time, it's important not to give in, and to continue making the case for history. The fact that someone has a statue erected to them or a window dedicated to them doesn't mean we approve of what they did. It means that the people who first created and displayed it did. That statue of Rhodes or the windows to Lee and Jackson don't have to say we approve of colonialism or slavery. They can remind us that people once did, and warn us how easy it is to fall into ways of thinking that are deeply and appallingly un-Christian.

Take the images away, and you've robbed future generations of learning those lessons.

There's a biblical parallel here. Parts of the Bible are appalling. They are full of extreme violence, and they are profoundly troubling. They are what the theologian Phyllis Trible called the 'Texts of terror', like the rape and dismemberment of the Levite's concubine in Judges 19 or the sacrifice of Jephthah's daughter in Judges 11. We read them and we're appalled at how men treated women.

But take out of the Bible all the stories that are hard, or offensive, and you end up with an anaemic collection of feel-good writings that are ultimately meaningless. The 'good' parts of the Bible shine out because of the 'bad' parts, not in spite of them. The gospel is glorious because it redeems fallen humanity, not because it denies humanity was ever fallen. The most offensive story of all in the Bible is the crucifixion: take that out, and there's nothing left.

In Worcester Cathedral there is the tomb of the worst king England ever had. Since he died in 1216, there has never been another king named John. A rapist and a murderer, he is the last person to deserve a place of honour in a great place of worship. But his tomb there reminds us that we are all sinners, and warns us that we will all come to judgment.

We don't face the future by denying the past. We face it by repenting of it – and these memorials to sinners can help us to do it.