An Anglican archbishop in South Sudan is accusing his government of failing to take seriously women who accuse soldiers of raping them.

The Archbishop of Central Equatorial Internal Province in the Episcopal Church of South Sudan, Paul Yugusuk, said army officials had not arrested a soldier accused of raping a pregnant woman at gunpoint that his office had reported last week.

The soldier is in the South Sudan army and remains at large, Yugusuk said, free to continue raping women.

'The woman was threatened by gun. The husband was also threatened. One soldier raped the woman. The woman is now in the hospital. She has been in a critical situation,' he told South Sudan in Focus.

It comes after President Salva Kiir ordered the defence minister and the army's chief of staff to execute soldiers who commit rape or murder.

But military authorities are covering up such crimes to shield soldiers from arrest, the archbishop said.

'When the case goes up [to military authorities], this is where it also dies. So it encourages the soldiers to do what they want to do because they will not be held accountable,' he said.

It is not the first time Yugusuk has spoken out the army's record of rape, despite the obvious dangers to his own life.

Last year he heavily criticised the military authorities after four soldiers accused of raping teenage girls in a village in South Sudan were not charged a year after the incident.

President Kiir warned South Sudanese not to send messages that will tarnish the image of South Sudan during his New Year message.

The archbishop responded: 'You [President Kiir] cannot tell me that messages should not be sent out when our people are being raped. So we keep quiet?'