Entrepreneur Luke Johnson says the Institute of Directors (IoD) is “so obsessed by its internal rows that it has lost its way”.
Businesswoman Lady Judge resigned on Friday as chair of the business group after allegations of racism, sexism, and bullying. She denies the claims.
Deputy chairman Sir Ken Olisa and non-executive director Arnold Wagner have also resigned amid the fallout.
Mr Johnson said the ongoing rows at the IoD were a “great shame”.
He said the organisation should be “championing business”, but had instead lost its way “in terms of governance”.
“I don’t know the truth of the allegations and counter-allegations, [but] it is clearly not setting a good example to the business community,” added Mr Johnson, who was a candidate for the IoD chairmanship in 2015 when Lady Judge was chosen.
One of the main purposes of the IoD is to promote good corporate governance.
Lady Judge’s resignation came after she reportedly made derogatory comments to colleagues when discussing cutting staff at the IoD.
She allegedly told the IoD’s director general last April that “the problem is we have one black and one pregnant woman [on the IoD secretariat] and that is the worst combination we could possibly have”.
Her remarks are understood to have been recorded by the business lobby group’s director general, Stephen Martin, because he thought she might make racist remarks.
Mr Martin made the recording without her knowledge.
Lady Judge said the recording was made to entrap her.
She said on Friday she wanted to make clear that she was seeking to have those individuals removed from their positions “solely on the basis of their performance” rather than their ethnicity or gender.
Other parts of the recording, heard by the BBC, indicate she is in favour of promoting diversity.
Earlier in the recording, Lady Judge said she had arranged women-only “cocktail party” networking events, and she participated in IoD women’s conferences.
“It’s all part of this agenda, which is to bring in more women,” she said.
Mr Johnson said: “If I were chair of an organisation and the chief executive was secretly taping conversations I would find it hard to carry on.”
He said that control and power at the organisation were “sometimes opaque.”
“I don’t know how things unwind from here,” he added.