The Supreme Court on Tuesday rejected an emergency appeal to remove Robert F. Kennedy Jr. from the presidential ballot in two battleground states.

Kennedy wanted to get off the ballot in Wisconsin and Michigan after dropping his independent bid and endorsing Republican Donald Trump in the tight contest. He argued that keeping him on violated his First Amendment rights by wrongly implying he still wanted to be elected president.

Michigan and Wisconsin said removing his name now, with early voting underway days before the election, would be impossible. More than 1.5 million people in Michigan have already returned absentee ballots, and another 264,000 have voted early, state attorneys wrote in court documents. In Wisconsin, over 858,000 people have returned absentee ballots.

The justices did not detail their reason in an order rejecting the emergency appeal, as is typical. One justice, Neil Gorsuch, publicly dissented in the Michigan case.

The presence of independent and third-party candidates on the ballot in swing states could be a key factor in the close presidential race. The high court previously rejected Kennedy’s separate effort to stay on the ballot in New York, a state where his presence is unlikely to make a difference in the race between Trump and Democrat Kamala Harris.

Kennedy has been working to get off the ballot in the seven key swing states since endorsing Trump. Wisconsin and Michigan are the last two where his name is expected to appear.

In Michigan, he notched an appeals court win but courts ultimately found he couldn’t withdraw as the candidate of the Natural Law Party, which had wanted him to stay on.

In his dissent, Gorsuch pointed to lower court judges who wrote that the timing of Kennedy’s original request to be removed wasn’t so unreasonable that it should be denied.

In Wisconsin, courts rejected Kennedy’s argument that major parties unfairly get more time to switch nominees. Judges there found candidates who miss deadlines to change nomination papers must remain on the ballot unless they die, and a plan to cover Kennedy’s name with stickers was unworkable.