Andrea Leadsom appeared to come down on the side of parents in an ongoing dispute over the teaching of LGBT issues in schools.
Parkfield Community School in Birmingham dropped its No Outsiders teaching programme, which covers LGBT relationships, after dozens of parents protested outside the school gates and withdrew their children from classes.
Four more schools in Birmingham have since decided to stop teaching the programme after complaints from parents, the BBC reports.
Leadsom, the Leader of the House, told LBC radio's Nick Ferrari that while she believed it was right that children learn about gay and lesbian relationships, parents should be entitled to have a say on whether their kids "are exposed to that kind of information".
"I think that it's right that the government should have passed legislation that requires that relationships and sex education is taught in schools," she said.
"But at the same time, I also agree that it's right that parents should be able to choose the moment at which their children become exposed to that information.
"And there are steps taken to enable parents to withdraw their children from classes up to a certain age."
Asked whether she believed it was appropriate for children as young as five to receive such lessons, she said: "I would be entirely happy for my children to grow up finding that their LGBT classmates are exactly the same as them.
"One of my own kids was in a class with a friend who had two mums and that was absolutely normal right from a very young age.
"I think it's important that we absolutely accept equality in every area whilst at the same time respecting that parents may have concerns about how young their children are when they become aware of these things."
Ofsted chief inspector Amanda Spielman angered people of faith last month when she came out in defence of the No Outsiders programme and said that schools must teach children that some families "have two mummies or two daddies" irrespective of their basis of faith.
"It's making sure they know just enough to know that some people prefer not to get married to somebody of the opposite sex and that sometimes there are families that have two mummies and two daddies," she told the BBC.
"It's about making sure that children who do happen to realise that they themselves may not fit a conventional pattern know that they're not bad or ill."
She went on to say that most faith schools were managing to meet their legal obligations, "even those which clearly teach that homosexuality is not right in their faith."
The Christian Institute, however, said lessons on lesbian and gay relationships should be left until secondary school.
Christian Institute Director Colin Hart accused Ofsted of being "side-tracked into identity politics" when instead it should be focusing on the quality of children's education.
"Not every controversial issue has to be covered in primary schools," he said.
"The idea of teaching ever more detail about sex to ever younger children is deeply worrying. Treating parents who object as homophobes is not the answer. They just want to protect their child's innocence."
He added: "Instead of listening to parents or allaying their concerns, the Chief Inspector of Schools seems to be casting doubts on their integrity. Implying that parents' desire to protect their own children is from fear, ignorance or possibly even homophobia."
