The head of the Anglican church in Scotland has warned the Church of England against treading on his ecclesiastical territory in an historic agreement with the Presbyterian Church of Scotland.

Bishop of St Andrew's David Chillingworth, known as the "blogging bishop", who is also primus of the Scottish Episcopal Church, said: "The Church of England is not a Scottish Church nor does it have any jurisdiction in Scotland. The Anglican way is to recognise the territorial integrity of each province – they are autonomous but inter-dependent."

He said the document had already caused damage to long-established relationships, and called for its publication to be delayed to allow a fuller consultation to take place.

The Columba Declaration commits the Church of England and the Church of Scotland to growing closer together in communion and mission and to recognising each other's clergy and laity. The document appears to take little account of the Anglican province in Scotland, the Scottish Episcopal Church, which withdrew from the talks early on but remained present as an observer.

The Church of Scotland and the Church of England announced the Columba Declaration on the morning of Christmas Eve, stating they had reached an historic agreement to work more closely together. Both churches will debate it later this year, at General Synod in York and the General Assembly in Edinburgh.

It also looks certain to be yet another cause of division in Canterbury this week, when Bishop Chillingworth represents the Scottish province of the Anglican Communion at the primates' meeting called by the Archbishop of Canterbury in a bid to prevent splits over homosexuality.

Bishop Chillingworth writes: "The aspect of the Columba Declaration which will cause most concern to the Scottish Episcopal Church is the potential involvement of the Church of England in the ecclesiastical life of Scotland."

Because the launch has been so public, he said, this has stirred considerable feeling and concern in his church.

He says it strays into areas which are "properly the concern of the Scottish Episcopal Church" as the church which represents the Anglican Communion in Scotland.

Whether or not it becomes independent at some stage in the future, Scotland is becoming a more distinct place, more sure of its own identity, he writes. "The Columba Declaration turns the Church of Scotland towards the Church of England in a way which to me seems to be a misreading of our context. The ecumenical family of churches in Scotland needs the leadership and active involvement of the Church of Scotland at this critical time in our national life."

He also says that the agreement suggests that the Church of England will respond warmly to the idea that its members will worship in Church of Scotland Churches when they visit Scotland. "Yet the Church of England's Anglican Communion partner in Scotland is the Scottish Episcopal Church."

Recognition of each other's ministries is also an issue. "The Scottish Episcopal Church now seems to be faced with the possibility that Church of England clergy will minister in Scotland under the authorisation of the Church of Scotland and without reference to the Scottish Episcopal Church," the bishop writes.

A spokesperson for the Scottish Episcopal Church said: "We fully understand the desire of the Church of Scotland and the Church of England as national churches to discuss and explore matters of common concern. However certain aspects of the report which appear to go beyond the relationship of the two churches as national institutions cause us concern. The Scottish Episcopal Church, as a member of the worldwide Anglican Communion, represents Anglicanism in Scotland, and we will therefore look forward to exploring the suggestions within the report more fully in due course."

The Church of Scotland said: "The joint study group report and the Columba Declaration are the outcome of ecumenical work that has been under way since 2010. The Scottish Episcopal Church was a full partner in the joint study group until it chose to withdraw from active participation in 2013. From that point on, the Scottish Episcopal Church continued to attend as an observer."