Kemi Badenoch has said she believes up to 10% of civil servants are so bad they should be in prison, claiming they leak official secrets and “agitate” against ministers.

At a fringe event at the Conservative party conference in Birmingham, the leadership contender said she did not want to criticise all civil servants but said there were a few who were obstructive. “There’s about 5-10% of them who are very, very bad. You know, should-be-in-prison bad,” Badenoch said.

“Leaking official secrets, undermining their ministers … agitating. I had some of it in my department, usually union-led, but most of them actually want to do a good job. And the good ones are very frustrated by the bad ones.”

A senior civil servant told the Guardian: “After the last 14 years, the country might want to say this about Conservative MPs.”

The general secretary of the FDA union, Dave Penman, called on Badenoch to withdraw the comments. “These are serious accusations from a former secretary of state, who is now standing to be leader of her party. If she has evidence to back up those claims she should publish it, otherwise withdraw.”

Badenoch, a former business secretary, said of civil servants: “About 10% of them are absolutely magnificent. And the trick to being a good minister is to find the good ones quickly, bring them close and try and get the bad ones out of your department as quickly as possible.”

Her comments come as the four remaining leadership contenders prepare to close the party’s conference with 20-minute speeches on Wednesday. The two candidates with the most backing from MPs – Robert Jenrick and Badenoch – were both condemned on Tuesday for comments they made during the conference, leaving the former home secretary James Cleverly with a place among the final two firmly in his sights.

Jenrick received criticism from the former defence secretary Grant Shapps and his two other rivals, Cleverly and Tom Tugendhat, over comments he made about UK special forces killing terrorists.

MPs will vote again next week to whittle down the candidates before a vote by members to pick the new leader. Jenrick and Tugendhat will make their speeches with no notes on an empty stage, with candidates rehearsing until the early hours of the morning as well as attending packed fringe events.

In her speech on Wednesday, Badenoch will claim wealth creators are fleeing the UK and vow to dismantle key aspects of the state, saying: “The truth is the left never left.”

The former business secretary, whose conference has been dominated by a row over comments she made on maternity pay, will say that the Conservatives’ 14 years of government were hamstrung by changes made by Tony Blair and Gordon Brown, which she describes as “ever increasing social economic and legal control”.

Jenrick, who is ahead with MPs’ votes and has led a hyperactive campaign with dozens of fringes and receptions at the conference in Birmingham, will urge his party to “build something new” out of the historic defeat it suffered in July.

Channelling Blair, he will say: “If we’re to tackle the immense challenges we face, if we’re to restore the public’s trust, we must build something new. A new Conservative party. That is what I call for today. Nothing less than a new Conservative party, built on the rock of our oldest values and best traditions.”

Jenrick, who has staked his leadership campaign on a pledge to cut migration and leave the European convention on human rights, will set out five principles he would stand on as Conservative leader: rejecting mass migration, campaigning for cheaper energy, housebuilding, public sector reform and “building a more united country”.

Cleverly will say in his speech: “I want to get the heel of the state off your neck, get their noses out of your business and their hands out of your pocket. Let’s sell the benefits of a Conservative government with a smile. We will not win back voters by pretending to be something we’re not. We win back voters by being honest, by being professional, by being Conservative.

“Never forget – Reform didn’t deliver Brexit, we did. Reform didn’t cut immigration, I did. And mark my words, we will beat Reform by being the best version of ourselves.”

Tugendhat will use his speech to set out what he describes as a “fundamental change in leadership” and will say he is intending to speak directly to voters who had previously voted Tory but turned against the party in July.

“If you went to Reform, I want to show you the Conservative values we share. If you went to the Lib Dems, I want you to see the opportunities only we can deliver. If you went to Labour, I want to show you why freedom, not state control, is how we build. If you stayed at home, I want to make you proud to vote Conservative again,” he will say.

New polling suggests that all four of the candidates have a long road to cut through with the public. The latest poll from Ipsos found two-thirds of voters (64%) do not care who becomes Tory leader, which includes almost a third of those who voted Conservative in July. Keir Starmer leads Badenoch by 25 points, Cleverly by 21 points, Jenrick by 22 points and Tugendhat by 22 points.

The chancellor of the Duchy of Lancaster, Pat McFadden, said: “When the Tories were in power they were more focused on stoking up divisions than delivering for the public. This shows they have not changed.

“These continuity candidates still won’t take responsibility for their appalling record and are trying to find anyone to blame instead. Civil servants work hard for the country every day and deserve better.”