On the very day that Matt Hancock was fighting to save his political career in Westminster, Labour’s battle to hang onto one of its own parliamentary seats was being played out nearly 200 miles north in West Yorkshire.

Amid ugly, unsettling scenes outside a mosque in Batley as worshippers left Friday prayers, the party’s candidate Kim Leadbeater was being harangued and chased by an anti-gay rights activist. The man, who had urged people to vote for George Galloway, was from Birmingham.

Keir Starmer tweeted that the abuse Leadbeater faced was disgraceful and the best way to counter Galloway’s “poisonous politics” was to vote Labour in the Batley and Spen by-election next Thursday.

But with one Survation poll putting the Tories in first place, Leadbeater second and Galloway third, the incident proved that Starmer is still struggling with problems in his party’s backyard at precisely the moment when Boris Johnson’s government needs a robust opposition more than ever.

And as much as Labour insiders mock Galloway’s grand claim that his entire campaign objective is to unseat Starmer as leader, there is now intense debate within the party about just what happens “after Batley”.

Among the questions swirling on Labour MPs WhatsApps are these. Will Starmer finally use the result as a vital trigger for change? Or will he confirm to his critics that he simply lacks the raw politics needed to become prime minister? And either way, isn’t he running out of time ahead of a possible early election in 2023?

The unease among both Labour backbenchers and frontbenchers has been palpable since May 6, when the party lost not just the symbolic by-election in Hartlepool but also lost councils and council seats in once-reliably Red areas in the north and midlands.

Hartlepool was meant to be a wake-up call for the Labour leader, but some believe he’s instead just hit the snooze button, with a lack of grip that his critics say defines his leadership. Allies counter that in the past week some key figures have left their roles, not least political director Jenny Chapman, chief of staff Morgan McSweeney and communications director Ben Nunn.

Despite worries about a damaging hiatus, none of their replacements are expected to be appointed before the Batley and Spen election. One insider said it was unclear whether the power vacuum was accidental or deliberate, but the new team will be able to credibly claim they are a break with the past.

What is striking is that, unlike Hartlepool, almost everyone within the party appears to have written off Batley and Spen. Leadbeater, sister of former MP the late Jo Cox, is seen as a superb candidate, the only one born and bred in the seat. There has been a huge influx of party activists trying to help (in Hartlepool, the campaign office had one sheet of A2 paper for signed up volunteers, but Batley has a whole wall of them). Yet the mood is of grim fatalism.

“Unlike in Hartlepool, there’s just seems to be an acceptance this time that we should believe what we’re seeing in Batley, which is if people are not talking to you and lifelong voters are saying they’re not going to vote for you, then you accept you’ve lost it,” one MP says.

In what feels like a dry-run of the arguments to be deployed once the result comes in, some argue that Galloway’s candidacy actually creates the alibi that this is a very particular set of circumstances, and not a generic ‘Red Wall’ problem as with Hartlepool. One shadow cabinet minister says the white working class vote appears to be holding up in some places, but the Asian vote has been either tempted by Galloway or will stay at home.

Some believe the by-election may have been lost for Labour the moment that Paul Halloran, the independent ‘Heavy Woollen District’ candidate who polled 6,000 votes in 2019, decided not to stand. His support is expected to almost all go to the Tories. Add in Galloway eating into Labour’s majority too, and it’s a lethal cocktail. “We’ve gone from their vote divided in 2019 to our vote divide now,” says one MP.

Old hands also counsel another message likely to be heard after the count: that Batley has always been a marginal and not a safe seat, especially as the Tories held it from 1983 to 1997. Yet it’s precisely for that reason that some are complaining bitterly that defeat would be a self-inflicted wound.

Former MP Tracy Brabin, who triggered the contest after her successful election as the new metro mayor for West Yorkshire, should never have been allowed to quit, several sources said. Unlike in Hartlepool where Teesside mayor Ben Houchen was an electoral asset, Brabin is not. “She was already seen as an absentee MP, and now she’s sodded off to Leeds, that’s how some of our voters see it,” one said.

One former minister says: “We had a saying under Blair and Brown: ‘never inflict an unnecessary election on the voters’. Tracy got the nod from Corbyn to go for it, but Keir could have stopped it if people around him had spotted the danger.

“What that really reflects is that Labour’s corporate memory is deteriorating, 11 years into opposition. Not knowing what you don’t know is the problem. And that’s more serious than people think. It’s not so much about judgement as experience.”

Another MP adds: “So many MPs and members of the NEC said to her [Jenny Chapman] ‘Do not allow this. An MP is trading in their job for what they think will be a better job? Serve out your term and then leave’. She wasn’t alive to the catastrophe, just as she didn’t think we would lose Hartlepool.”

“The vast majority of people on the doorstep quite like Kim, they tell us she’s a nice woman. But it’s Keir that they are unsure about. So Jenny’s genius has just resulted in a by election where Keir Starmer’s name is on the ballot. They’ve elevated the by-election to some fucking make or break thing.”

One former Corbyn staffer said that it was “astonishing” that the loss of Batley was seen as inevitable. “No opposition leader has ever lost two by elections to the governing party. Jeremy had four defensive by-elections by the same point in his leadership and we won them all. You just think, my God, the double standards.”

Already a “script” for what Starmer should say after a defeat is being worked on, some insiders say. No MP wants a repeat of the shell-shocked TV clip he gave after the Hartlepool loss. “He can’t just say ‘I’m listening’. He said that after Hartlepool. He has to say what he’s going to do about it,” one ally says.

Several MPs and aides believe Starmer’s problems have been both organisational and political. The shake-up of the leader’s office operation has been welcomed by the parliamentary Labour party (PLP), as has the appointment of Shabana Mahmood as national campaign coordinator and Conor McGinn as her deputy.

The pair this week instituted a daily 8am conference call (10am on Sundays, after Marr) with all frontbench advisers and the leader’s team. They run through what’s moving in the media and parliament and the party. Everyone is expected to arrive with ideas, but some policy advisers have been given a rude awakening, not least as McGinn’s blunt and direct style makes clear rapid improvements are needed.

“We’ve been told we have to be relentless in the war of attrition against the government, hungrier than them, faster than them, more brutal than them,” one MP says. “Our mission every day should be to make some Tory somewhere wish they’d never been born today,” another says. “Only after that can we get onto our highfalutin’ political strategy for winning a general election.”

MPs hope the changes in organisation will ensure a smarter approach to both policy and communications. When Rachel Reeves made a presentation to the shadow cabinet in her new Treasury role, she mentioned the party’s current biggest spending pledge was £15bn for schools catch-up funding. But for some around the shadow cabinet, that was the first they had even heard of the promise. “If we haven’t heard of it, how the hell do we expect the public to?” one MP said.

The pledge from shadow education secretary Kate Green, based on demands from former catch-up czar Kevan Collins, barely registered when it was dropped on the media the day after the late May bank holiday. “We were like, OK, so we’ve done that then, great we have £15bn for schools, shame we didn’t know about it,” one frontbencher said.

Some progress is being made, but it seems tortuous, some say. When the by-election team in Batley told the leader’s office that crime and anti-social behaviour was a big issue on the doorstep and they needed a big intervention from Starmer on it, his office reacted. In the end, it was decided Labour would scrap the Tory plan for a new Royal Yacht and spend the £200m on fighting anti-social behaviour instead.

The policy, a way of ridiculing Johnson’s competence while saying crime was Labour’s priority, made the pages of the Daily Mail. “It was spiky and painting in primary colours,” one MP said. “But the amount of effort it takes to get to the point of a single thing like that is astonishing.

“He’s detached from it because he only comes in at the end. It’s like you’re walking through treacle to make a small thing happen. Worse, there was no follow-up, it was just dropped into the pond and left.”

But it’s the need for Starmer to inject some politics, not just managerial ability, into his leadership that is the most common demand. One insider says the problem of a lack of definition for the leader stems from his slick leadership campaign in early 2020.

“They turned him into a bit of a blank sheet of paper onto which people could project what they wanted, left or right,” they say. “It was perfect for keeping everyone happy. But when you face the electorate, you have to pick some fights to let people know what you stand for.”

One MP says that Starmer needs a sense of urgency above everything. “There’s a hesitancy that he has, which he’s got to get rid of. His team tie themselves in knots, they worry a lot about whether something is the right ‘fit’ for different audiences. We just want him to make an emotional connection with the voters. People want him to show leadership, say in his own words what he thinks. That will go a long way.

“He still doesn’t understand the difference between a court of law and the court of public opinion and hasn’t made the emotional leap from one to the other. In a court of law, you say something once, that’s it. But with public opinion, you have to say it seven million times before even half of it is heard.”

A senior backbencher says the problem has been a lack of equals in his office. “He doesn’t have a peer to peer relationship with someone with the authority to just go and say ‘right, get your shit together, you need to do this right now’.”

“The ones who have been close to him were in awe of him, they do think he’s amazing, that he’s intellectually their superior, and that’s not a challenge to his brain. It’s not like Blair with Jonathan Powell or Campbell and Mandelson, or Ed Miliband and Ed Balls.”

One former minister says: “I don’t doubt his determination, and his ability. There’s a more fundamental problem: does he know what work he has to do? Does he know what he doesn’t know? He’s got the necessary ruthless streak, but has he got the judgment to know when to deploy it?”

Others believe that while the right staff in his office is crucial, it’s ultimately down to Starmer to lead and to define himself and his party. “Part of the problem is he seems to think that you just need to hire the right person and somehow they’ll make him a future prime minister. He doesn’t seem to understand most of that is down to him. They need to know what his gut is telling him.

“He will acknowledge you’re basically right, nod, and then somehow he just doesn’t make it happen. And no one can understand what the hell that’s about. Some people are more despairing than others, it’s like ‘just make a fucking decision, man’.

If Batley is indeed lost, Starmer will indeed have to make some decisions. Several figures on different wings of the party believe his leadership will inevitably come under question. “The big question is what the tone will be after Batley,” says one. “If it is apologetic or introspective or anything like that, I think the clock’s ticking.

“He needs to come out swinging on this somehow. We can’t keep reacting to failure, explaining why we’ve lost. There needs to be a moment where we’re going to tell people how we win. And if we can’t do that now after 14 months of his leadership, I don’t know when we’re going to start. How he comes out of this now is going to shape whether Keir buys himself six months or 24 months.”

Under party rules, more than 40 MPs are needed to trigger any leadership challenge. Over the past week, there has been chatter among backbenchers that Dawn Butler was being lined up as a candidate who could get the backing of the 34-strong Socialist Campaign Group, as well as some disaffected Black and Asian MPs.

The idea of some instant Butler challenge the day after Batley has been dismissed, not least by the Left. But perhaps more worrying for Starmer is the level of grumbling around those who have been desperate to make his leadership work. “Some people are thinking, can this ever work and have become very down about it,” one MP said.

“They’re the people he needs to impress when he addresses the PLP after the defeat. He will have to move quite fast, have a really good summer and a really good conference.”

 One senior party figure says a leadership challenge from the Left would be defeated easily by 80% to 20%. “It could provide a sort of reset moment, in the way that the best thing that ever happened to Corbyn was MPs challenging him in 2016,” they say.

With disloyalty a toxic charge, Starmer allies are also confident that neither Lisa Nandy nor Angela Rayner would want to ruin any future leadership hopes of their own by exploiting any “stalking horse” challenge by the Left.

Others believe a challenge would be a huge distraction and the focus should be on connecting with the public this summer, not Labour members. Starmer’s team have long planned a summer of meet-the-people events, and have been itching for Covid restrictions to lift to make the strategy workable.

Starmer tested the water in Ipswich last week, meeting business leaders. While the Commons is still sitting until late July, the plan is for him to every Wednesday travel to a key seat, meet families in a pub or restaurant, then spend the next day in a workplace canteen or old people’s lunch club. Once parliament rises, he would do Q&A sessions regularly too with voters.

But one Tory minister says the Batley by-election is proving that Starmer is the Tories’ biggest asset. “His problem is, he fails the pub test,” the minister says. “With Tony Blair and with Boris, whether you’re working class or whether you are posh, you could imagine them having a conversation with you, and that they enjoy the job. Starmer’s a bit like Gordon Brown or Theresa, they looked haggard, tired, grumpy, uncomfortable. Worst of all, he looks a bit aloof.”

A senior Starmer ally hits back: “Well, the pubs aren’t bloody fully open so it’s a bit hard for him to pass or fail a pub test. That’s exactly why we think that once he’s out and about, people will see the real Keir and like what they see. The [Piers Morgan] ‘Life Stories’ [where Starmer talked about his family life and upbringing] was just the start.”

Another shadow cabinet source says: “The biggest thing he’s got going for him is that the majority of people in the majority of pubs don’t know who he is. There is still a huge chunk of the population that hasn’t made up their mind about Keir. That’s either because he’s been so hidden away or there’s been a sort of blandness that they don’t have anything to make their mind up about.”

Some MPs are more sceptical. One warns that a ‘listening tour’ won’t work and that what’s needed is a ‘telling tour’, telling the voters what he stands for. “He’s going to use the summer tour as a reboot of his leadership, his own brand and the party’s brand,” they say.

“It’s not the worst idea in the world but it won’t work if it’s just a bit nicey, nicey and doesn’t change the dial on what people’s fears of the Labour Party are, which is that we are the soft guys of British politics, we say yes to everyone and everything.”

There will be a summer campaign on workers’ rights, but some Starmer allies believe that a focus on crime and anti-social behaviour is the best way of uniting Labour’s fractured voting coalition of working class and middle class voters.

The issue has cropped up repeatedly in Hartlepool and Batley and the May 6 elections and aides think Starmer’s unique background as a former Director of Public Prosecutions ought to be deployed. “This is a guy who effectively locked up serious criminals and terrorists, but few of the public know that.” one insider says. “We’ve got to tell them.”

Any summer tour may or may not register on the voters’ radar, but the party conference in Brighton in September is seen as Starmer’s first real chance to properly grab the public’s attention. Efforts to ram home that the Corbyn era really is over may depend on what one NEC member refers to as “the most important set of elections and no one’s ever heard of”.

These are local constituency party delegate elections, the deadline for which is July 9. Some 40% of local parties will be holding Zoom meetings to pick the delegates who vote at the annual conference. Allied with more centrist trade union leaderships, the hope is that internal reforms or policy changes will be much easier.

Some centrists believe that Starmer’s best hope of proving his party has changed would be to tear up its member-led leadership rules and return to an electoral college that restored MPs alongside unions and members. MPs represent millions of voters, whereas party members represent only themselves, allies say. The Left would be sure to react with fury.

It’s too late to mobilise such a radical reform for this autumn, and it may need Unite the union to elect centrist Gerard Coyne to have a chance. But there is chatter of a special conference next spring to carry it out.

Although Starmer likes to talk about tackling anti-Semitism as defining his leadership, some MPs believe the public either don’t know or or uninterested in the issue. A big bang change to water down members’ hold over the party leader would be more bold.

There are however downsides, as one senior aide points out. “It’s a terrible dilemma. One of the big, most essential ways that Keir has to define himself is by changing the leadership rules to ensure that this great party never goes back to being run by cranks. But the problem he’s got is as soon as he does that is he could face a challenge from the Right or competent centre. It’s a total Catch-22.

“Everyone recognises that that if you remove the threat from the Left, it gives you the chance to govern. At the same time, it makes you massively vulnerable because suddenly people can challenge you without worrying about Richard Burgon ending up in charge.”

The main focus for Starmer will be his conference speech and themes. “Conference is absolutely crucial now,” a shadow cabinet minister says. “He needs to set out what he believes in, what he’s doing, his priorities for being Prime Minister, set out in an ambitious, bold, enthusiastic, passionate way in front of a live audience.”

One MP says: “He needs some drama, a big moment to show what he stands for. He can show leadership by saying there are some people in my party who believe things the country will never accept and I’m going to take them on. The country will understand that.”

Some believe he could grab the public’s attention with a policy pledge, just as the Cameron-Osborne inheritance tax cut in 2007 gave the Tories definition, turning round months of poor poll ratings (which are now forgotten) and scaring Gordon Brown into postponing the election. “It wasn’t the policy itself, it was that they were actually saying what they stood for,” one aide says.

The key has got to be both a clean break with Corbyn and a positive vision of Britain, one frontbencher says. “Nobody I talk to tells me you know what ‘Labour has changed too much’. And when I ask Tory MPs privately what they fear most, they say it’s if we got rid of the loonies” they say.

“If we lose in Batley, and the expectation very much is we will, we have to get the PLP and the wider party to accept that without a big change things could get even worse than 2019, despite that election being absolutely catastrophic for us.

“It’s a huge achievement that for the first time since Tony Blair we have a leader who passes the prime minister test. But the toxicity that there still is around the brand and the damage that has been done, it’s not going to be solved just by a guy who looks like he could stand outside Number 10, it’s got to be much more fundamental.”

One senior party figure says Starmer still has the chance to have a fresh start. “It’s not about losing this or that by-election, that’s just a welcome to the real world of irrelevant opposition. The big sense he thinks he could lead through the power of his great intellect, that’s not gonna happen.

“He’s basically got to accommodate Corbynism, or move on from Corbynism. And he’s got from Batley to conference to decide what he does on that or signal what he does on that.”

And a frontbencher puts it even more simply: “We just need to stop acting like a vehicle or a client agency for campaigns and stakeholder groups. We need to speak like the voters, think like the voters, about how they get up every day and just want to make ends meet, to let their kids get on in life. We just need to be, you know, normal.”

One senior MP worries that Starmer’s real problem is that he only entered politics because he wanted to be prime minister, and now that looks a distant possibility. “I think he’s had a loss of confidence post-May. He looked like a man in shock, and he’s not quite recovered. He’s got on in life by being self-sufficient and betting on himself, so why is he not now betting on himself?

“Is the crisis of confidence really because he thought he was going to be Blair and maybe now the best he can hope for is being Kinnock? Maybe. But he needs to realise that’s OK, that’s a great life story, ‘I saved the Labour Party’ is a historic thing in the national interest. Maybe he thinks that’s a bit beneath him. If I was close enough to him I would tell him there is such honour in that, in dragging us closer to power.”

One former minister warns that with possible by-elections in Tory-held Delyn and Wakefield, there’s a chance to counter the Hartlepools and Batleys. “If after Batley, he comes out fighting I think the PLP will say at last he’s listened, better late than never, and there’s no alternative, let’s give him more time. It’s basically a case of using the summer and conference properly. But if things are still like this next May, I think he really is in trouble.”