If you want to hear JPMorgan Chase CEO Jamie Dimon launch into a tirade, just ask him about proxy advisory firms.

“Anyone who uses them, shame on you. I mean, shame on you,” he said at a Semafor forum in March.

“Yeah, they certainly got under his skin,” said Robert Bartlett, faculty co-director of the Rock Center for Corporate Governance at Stanford.

The “they” Bartlett’s talking about are two dominant proxy advisory firms — Institutional Shareholder Services, or ISS, and Glass Lewis.

The White House is reportedly considering limiting the power of proxy advisory firms, such as ISS and Glass Lewis. These firms put out recommendations on how large investors should vote during corporate shareholder meetings. The Wall Street Journal says the Trump administration is thinking about even banning those recommendations altogether.

Bartlett said they’re often involved in advising large firms like mutual fund managers on how to vote on various corporate issues, from who should be on the board of directors to reducing carbon footprints.

“Management is relying on shareholders to approve various items, and when ISS and Glass Lewis advise against voting for management-supported shareholder items, that obviously makes life difficult,” Bartlett said.

Tao Li, finance professor of the University of Florida, worries that new limits on proxy firm recommendations could be counterproductive.

“Having proxy advisors, advising the largest institutional investors, in general, is working really well,” he said. “Without any recommendations, how would people vote?”

Li added that there could even end up being fewer shareholders voting at all, leaving company insiders with more power.