The Bishop of Manchester has criticised the "overzealous" and inefficient operations of the Department for Work and Pensions which he said result in innocent benefits claimants suffering from sanctions and late payments.

Speaking at a fringe event at the Conservative Party Conference, Rt Rev David Walker said that too many innocent people were being trapped in the drive to catch fraudsters.

"The way that policy is being implemented – there are too many delays, there are too many wrong decisions and that does lead to too many innocent people effectively being punished."

He added: "We have to let a few manipulative people get away with it if we're not going to have too many innocent people punished. I don't think we can put the burden of proof so far over to making sure nobody ever pulls one over our eyes that we trap lots of innocent people."

Walker said that he backed the new system of Universal Credit, designed to simplify the labyrinthine system of benefits, but said that the DWP had got its priorities wrong.

"My first priority would be to drive down the unacceptable delays in benefit payments and the wrongful or overzealous imposition of sanctions," he said.

"I think they have got out of hand and at times it's almost Kafkaesque to have a situation that is dysfunctional, [which] dissuades people from getting involved with it – is that in itself meant to be a disincentive?"

Walker was widely reported as saying that benefits cheats should be "let off". He hit back on Twitter, saying that he had argued that "You can't catch every cheat without punishing too many innocents".

Walker is an outspoken campaigner on anti-poverty causes and has fiercely criticised the Chancellor's plans to reduce the tax credits many poor families rely on to survive. He wrote when the proposals were first announced: "Reining back public support for those who are trying to work their way out of poverty, often by holding down several jobs at once, is a huge leap backwards. It asks those already struggling to keep their heads above water to take on an extra burden." He stressed that children would be affected particularly badly.

A DWP statement said: "Our vital reforms are restoring fairness to the system for claimants as well as the taxpayer."