The majority of Labour MPs are “desperate to rejoin” the EU “at heart”, a former shadow minister has said.

Rosie Duffield told HuffPost UK’s Commons People podcast that Labour MPs, including frontbenchers, voted for Boris Johnson’s deal “with a heavy heart” and “haven’t given up” on EU membership.

She said they would eventually “try and shift the leadership” of the party but conceded they must “let the dust settle” on Brexit, acknowledging: “We’re not talking in the next five years realistically”.

But the Canterbury MP suggested Keir Starmer may not be at the helm when the time comes.

“Possibly we might even have a different sort of set up, a different leader by then, who knows,” she said.

She also warned Starmer that he cannot “dismiss” voters from the “liberal elite” who are anti-Brexit.

Duffield has previously served in as a parliamentary private secretary (PPS) in Labour’s women and equalities team and as a party whip.

Duffield’s comments came after Starmer told The Guardian last week that Labour would not seek major changes to the UK’s relationship with the EU if he is elected prime minister. 

Asked about the interview, Duffield told HuffPost UK’s Commons People podcast: “Yeah but you know, us backbench rebels don’t necessarily have to pay too much attention to that.

“Every day groups of Labour MPs and opposition MPs were meeting about People’s Vote and trying to take it to the absolute last minute, those people haven’t gone away.

“And most people on the frontbench who voted for this deal last week did it with a very heavy heart and they haven’t given up either.

“We will try and shift the leadership, as and when it needs to shift towards rejoining I would imagine.

“Possibly we might even have a different sort of set up, a different leader by then, who knows. 

“We’re not talking in the next five years realistically.”

She went on: “The majority of the parliamentary Labour party wanted us to remain and were campaigning really hard on that and towards the end almost all of those people had signed up towards a People’s Vote.

“So the majority of us, we don’t need converting again, we’ve done all that gelling together if you like, it was pretty hellish, voting at two, three in the morning on these deals and things, and all the amendments we put forward.

“All of that hurt is still there and we are still desperate to rejoin if we possibly can I think, at heart.

“But it does depend on negotiating with Labour Party policy.”

Put to her that rejoining the EU would be anathema to voters in the so-called “red wall” of working class constituencies in the north and Midlands that Labour lost at the last election, and who Starmer is trying to win back, Duffield said: “Look at London and the south-east, we could see them turning against Labour if we don’t start to be much more honest about these trade deals and our relationship with Europe.

“The kind of liberal elite if you like, represented by people like me, they still vote and campaign and they march and they make their voices heard – so Keir can’t afford to completely dismiss that group of people as well.”

Duffield conceded that Labour remainers may have to settle for rejoining the single market, rather than full EU membership.

“Certainly a closer relationship, and then there are people like me who want to rejoin,” she said.

“Getting those bits back even in increments is better than nothing.

“We are all going to see that this is not going to be very good for trade.”

But she said campaigners in Canterbury had already started to think about a rejoin campaign.

“All the groups I was involved with are already calling themselves rejoiners and are starting to think about that,” Duffield said.

“I think maybe it’s a little bit too soon but we might as well start to build a movement and look at that.”