I delivered a first draft of my biography of Bethlehem to my editor in late 2016, knowing that she would have a problem. For all the attention I had lavished on the archaeology, the church builders and ancient wars, I had said nothing about the present-day city. I left the story in 1917 with a squadron of Welsh soldiers capturing the city. 'This is not going to fly,' she told me. I had to bring the story up-to-date.

Bethlehem is my wife's hometown, and I have lived there on-and-off for twenty-five years. But I felt I had too much to say, and nothing that was not despairing. Who wants to hear that? My editor made it clear I had no choice. I based the final chapters on a road trip I made using the settler-only road that surrounds Bethlehem. My tour guide found a car with Israeli plates, which he was forbidden from driving. While he sat in the passenger seat, I took the wheel and together we got to see the city from a settler's perspective.

Bethlehem is the largest of a number of small towns and villages south of Jerusalem. The urban conurbation has grown to swallow four small towns and a handful of villages, but beyond this built-up area, there is still a verdant and fertile landscape of orchards and olive groves, arranged over ancient terraces. The hill villages are based around natural springs, fed by a large aquifer in the limestone layers below.

In 200 BCE, the Bethlehem region was transformed from a farming district to an important strategic centre with the building of an aqueduct to convey water to the temple in Jerusalem. The aqueduct remained in use for an astonishing 2300 years. It was only when Israel seized the West Bank in 1967 that the aqueduct was replaced by Israeli engineers who tapped straight into the aquifer. The new pumping station, Kfar Etzion, became the first Jewish settlement in the occupied West Bank in 1967. Today, the Gush Etzion block, named after Kfar Etzion, comprises thirty-three of the forty-one settlements around Bethlehem. Gush Etzion forms a ring around the city, almost touching the edges of the urban Bethlehem area. The line between the settlements and the last Palestinian homes is marked by a long stretch of concrete wall, and by the settler-only bypass road. I planned our trip to explore all the ancient villages, on the 'wrong' side of this road: Husan, Beit Fajjar, Nahalin, Wadi Fuqin, Ain Faghur and others. These are farming villages, growing fruit and salad vegetables in polytunnels, overlooked by the Israeli government-built projects of Gush Etzion.

The isolation means the villagers struggle to get food to market, or find other work. Youth unemployment rates of 80 per cent are not uncommon. At one time the villagers would have looked to Jerusalem as a market place and a source of new work, but the city is now closed to the West Bank. So the villagers have turned to Bethlehem instead. As poverty and harassment drives them off their lands, the farmland has been depopulated, while Bethlehem has become ever more crowded.

We got back to Bethlehem in rush hour. The city can often feel like a pressure cooker environment. The settlements add to the tension: they are so close, and visible from any street corner. With no room to expand, land in Bethlehem fetches insane prices, despite the city's permanent economic crisis. The newcomers from the villages are poorer and more conservative, less educated, and often devoutly Muslim. Bethlehem has long enjoyed a strong secular culture, combined with a Christian ascendancy that owns the major businesses and much of the property. When ill-feeling breaks out, it can be portrayed as a class conflict, or as a struggle between liberal and conservative values, or as religious tension. At heart, however, the crisis is man-made, driven by Israel's unchecked demands for land and water that has turned Bethlehem into an urban ghetto.

The traffic is at its worst as the schools close. As the population grows, the city needs better schools, better health care, more jobs. It is impossible to see how this can be done under what is legally termed 'a hostile military occupation', a status reaffirmed at the UN in December 2016. Yet Bethlehem has a long history of welcoming incomers. The Armenian and Syriac communities arrived as refugees in the early 20th Century, escaping the Holocaust in Turkish-controlled Anatolia. 1948 saw a new wave of refugees escaping the ethnic cleansing in areas controlled by Israeli forces. The 1960s and 70s saw Bedouin communities forced to settle in new homes at the edges of the city. The latest arrivals brings new challenges, but one that Bethlehem has risen to in the past.

It is a special city. As the rush hour quietened and sunset bathed everything in amber, I can believe the future will not be as gloomy as I often fear.

Nicholas Blincoe is the author of Bethlehem: The Biography, published by Constable & Robinson.