WASHINGTON (AP) — Hurrying to check a growing controversy, President Barack Obama ousted the acting commissioner of the Internal Revenue Service late Wednesday amid an outcry over revelations that the agency had improperly targeted tea party groups for scrutiny when they filed for tax-exempt status.

Obama said Treasury Secretary Jacob Lew had asked for and accepted Steven T. Miller's resignation. Obama made no public criticism of Miller but spoke of inexcusable "misconduct" by IRS employees and said new leadership at the agency was critical.

"Americans are right to be angry about it, and I am angry about it," Obama said in a televised statement from the White House. "I will not tolerate this kind of behavior in any agency but especially in the IRS, given the power that it has and the reach that it has into all of our lives."

Miller's ouster came five days after an IRS supervisor publicly revealed that agents had improperly targeted groups with "tea party" or "patriots" in their applications for tax exempt status. It came a day after an inspector general's report blamed ineffective management in Washington for allowing it to happen for more than 18 months.

The report said tea party groups were asked inappropriate questions about their donors, their political affiliations and their positions on political issues, resulting in delays averaging nearing two years for applications to be processed.

Miller's departure hardly ends the matter. Three congressional committees are investigating, and the FBI is looking into potential civil rights violations at the IRS, Attorney General Eric Holder said earlier Wednesday.

Other potential crimes include making false statements to authorities and violating the Hatch Act, which prohibits federal employees from engaging in some partisan political activities, Holder said.

Miller, a 25-year IRS veteran, took over the agency in November when the five-year term of Commissioner Douglas Shulman ended. Shulman was appointed by President George W. Bush.

Obama has yet to nominate a permanent successor. A new acting commissioner was not announced Wednesday evening.

In an email to employees, Miller said, "This has been an incredibly difficult time for the IRS given the events of the past few days, and there is a strong and immediate need to restore public trust in the nation's tax agency. I believe the service will benefit from having a new acting commissioner in place during this challenging period."

In Lew's letter asking for Miller's resignation, Lew wrote that the inspector general's report "has created an urgent need to restore public trust and confidence in the IRS by installing new leadership for the service."

At the time when tea party groups were targeted, Miller was a deputy commissioner who oversaw the division that dealt with tax-exempt organizations.

The report by the Treasury inspector general for tax administration does not indicate that Miller knew conservative groups were being targeted until after the practice ended. But documents show that Miller repeatedly failed to tell Congress that tea party groups were being targeted, even after he had been briefed on the matter.

The IRS said Miller was first informed on May, 3, 2012, that applications for tax-exempt status by tea party groups were inappropriately singled out for extra, sometimes burdensome scrutiny.

At least twice after the briefing, Miller wrote letters to members of Congress to explain the process of reviewing applications for tax-exempt status without revealing that tea party groups had been targeted. On July 25, 2012, Miller testified before the House Ways and Means oversight subcommittee but again was not forthcoming on the issue — despite being asked about it.

In all, members of Congress sent at least eight letters to the IRS over the past two years, asking about complaints from conservative groups that they were being harassed by the IRS. None of the IRS responses acknowledged that conservative groups were targeted.

Miller was scheduled to testify Friday at a Ways and Means hearing. A committee aide said Wednesday evening that Miller was still expected to attend the hearing.

"More than two years after the problem began, and a year after the IRS told us there was no problem, the president is beginning to take action," said Senate Republican leader Mitch McConnell of Kentucky. "These allegations are serious — that there was an effort to bring the power of the federal government to bear on those the administration disagreed with, in the middle of a heated national election. We are determined to get answers."

The Justice department opened its criminal investigation on Friday, Holder said.

"I can assure you and the American people that we will take a dispassionate view of this," Holder told the House Judiciary Committee at a hearing Wednesday. "This will not be about parties, this will not be about ideological persuasions. Anybody who has broken the law will be held accountable."

But, Holder said, it will take time to determine if there was criminal wrongdoing.

Legal experts, however, said it could be difficult to prove that IRS officials or employees knowingly violated the civil rights of conservative groups. If there is a violation, the experts said, investigators can sometimes prove more easily that officials made false statements or obstructed justice in some other way.

"I think it's doubtful that any of these knuckleheads who engaged in the conduct that gave rise to this controversy knowingly believed that they were violating the law," said David H. Laufman, a former Justice Department lawyer. "But that remains to be seen. That's what investigations are for."

The IRS started targeting groups with "Tea Party," ''Patriots" or "9/12 Project" in their applications for tax exempt status in March 2010, the inspector general's report said. The criteria later evolved to include groups that promoted the Constitution and the Bill of Rights.

Wednesday's hearing was the first of several in Congress that will focus on the issue.

The House Oversight Committee announced Wednesday that it would hold a hearing May 22, featuring Lois Lerner, the head of the IRS division that oversees tax exempt organizations, and Shulman, the former commissioner.

The Senate Finance Committee announced a hearing for next Tuesday.

Colleen M. Kelley, president of the National Treasury Employees Union, said Wednesday that no union employees had been disciplined, as far as she knew. She noted that the IG's report said agents were not motivated by political bias.

Kelley told The Associated Press that low-level workers could not have specifically targeted conservative groups for long without the approval of supervisors. However, she noted, there are many levels of supervisors at the IRS.

"No processes or procedures or anything like that would ever be done just by front-line employees without any management involvement," Kelley said. "That's just not how it operates."