Tory ministers secretly back holding a second Brexit referendum, it has been claimed.

Phillip Lee, the former justice minister who quit in June in protest at the prime minister’s Brexit strategy, told HuffPost UK he had “direct” conversations with three ministers who wanted to back the campaign for a so-called People’s Vote.

And he said others were “on the cusp” of going public with the same view. 

The Conservative Party conference in Birmingham has been dominated by divisions within the party over Brexit and the potential for a leadership challenge.

Currently only a handful of Tory MPs have called in public for a referendum on the final Brexit deal. The prime minister has ruled out holding another vote. 

Former Education Secretary Justine Greening and pro-EU backbencher Anna Soubry joined Lee at a packed meeting on the fringes of the conference in Birmingham to push for another referendum.

Soubry told the meeting to loud cheers: “We only have six months to save our country.”

She also slammed the current state of the party for abandoning its pro-EU agenda, moving to the right, and not having enough “women”, “brown faces” and “black faces” in it.

Lee mocked some hardline Brexiteer Tories such as Jacob Rees-Mogg for pursuing a “Dad’s Army, Rule Britannia” Brexit.

“How has the Conservative Party got itself on the side of angry men and against women and and young people?” he asked.

The former minister said May’s Chequers Brexit deal did not have the support of the majority of MPs and would be unlikely to get through the Commons.

He warned Tory MPs that voting for it, or a harder Brexit, would lead to Jeremy Corbyn being elected prime minister.

“Every single social economic ill that takes place between now and the next general election is going to be blamed on Tory Brexit,” he said.

“It’s about time, particularly Conservative colleagues who think that, stood up and were counted.”

Asked how many Tory MPs privately supported holding another referendum, Lee told HuffPost UK: “I suspect there are significant numbers of colleagues who can see the arguments for a second vote.

“I know of three ministers who do. I’ve had direct conversations with them.

“But it is a difficult environment for them to come out. I understand that and I respect that. Because I didn’t find it easy at all.

“I think some others are on the cusp. But it needs public support and business support. And I think you will see more come out and support us.”

The Conservative Party, which is officially opposed to another referendum, had not allowed the event to take place inside the official secure zone of the conference.

It took place as a rival rally of Brexiteers including ex-Environment Secretary Owen Paterson and former Development Secretary Priti Patel was held to attack Chequer’s from the eurosceptic side. 

In a blow to Tories calling for another vote, Ruth Davidson, the Scottish Tory leader who campaigned strongly for Remain, said she would be a “hypocrite” to demand a referendum.

“You don’t get to demand a re-run just because you didn’t get what you want,” she told party members.

“That way leads to more division, more rancour and a politics trapped in the past.

“The people voted. And there is no way to bring this country back together that doesn’t respect the vote.”