Mr Rove, nicknamed "Bush's brain", will step down as Mr Bush's deputy chief of staff at the end of the month to spend more time with his family in Texas.
He will remain covered by the executive privilege of the White House, protecting him from testifying before the Democrat-controlled Congress over a string of political controversies.
But Democrat Senator Charles Schumer said Mr Rove, arch-enemy of the Left for years, would still be required to answer questions over the alleged politically motivated firings of US prosecutors.
"He has every bit as much of a legal obligation to reveal the truth once he steps down as he does today," he said.
His voice faltering, Mr Rove - the most powerful unelected man in Washington - said he would remain unswervingly loyal to Mr Bush, his trusted friend of 34 years.
He said he would be "a fierce and committed advocate on the outside".
Their political partnership spans 14 years from Mr Bush's plan to run for Texas governor.
The surprise surrounding the resignation of the master political and policy tactician 18 months before the end of the presidency cannot be overstated.
It adds to the perception that the Bush Administration, suffering record low approval ratings on the back of the unpopular Iraq war, is running out of steam.
Mr Rove, 56, denied he was forced out.
His departure follows a string of high-profile resignations. The White House officials described it as a "big loss" and said he was "irreplaceable".
Mr Rove, a key architect of the Iraq war and the US response to the September 11, 2001, terror attacks, had widely been expected to stay on to help shape and protect the President's legacy.
The timing means he will be missing from the White House's expected showdown with Democrats in September, when a report on the progress of the war strategy is delivered.
He will still have Mr Bush's ear, and while he will not actively work for any 2008 Republican presidential candidates, he will offer advice.
Mr Bush calls him the "architect", "boy genius", and "turd blossom", a Texan term for a flower that grows in cow dung.
Mr Rove's reputation as a political mastermind was tarnished after the 2006 US midterm elections, when the Democrats won control of Congress.
Mr Rove escaped prosecution over the Valerie Plame-CIA leak case, but is being investigated over the firing of eight federal prosecutors, alleged misuse of White House email accounts, and alleged improper political briefings to government agencies.
An emotional Mr Rove, standing beside a grave-faced Mr Bush, said it was not an easy decision and he was grateful to have been a witness to history.
Mr Bush, who gave a farewell handshake and hug, applauded his service and sacrifice, adding:
"I will be on the road behind you here in a little bit."