I kindly ask we keep George W. Bush and Barack Obama out of this discussion to give it more of a historical context, but you are free to discuss things as you please.
Thomas Jefferson - great philosopher ; mediocre president. embargo policies caused a depression, naval budget cuts left US unprepared for War of 1812, practically took credit for his predecessor's work, and doesn't have a good record on racial equality to say the least.
James Madison - Disagreed with his decision to enter the War of 1812. Nearly had the US completely destroyed had it not been for the luck of Britain having other problems in Europe.
Woodrow Wilson - Yes, he was present during an important time in history, but he totally waffled when it came to his policy to enter WWI. And his refusal to compromise made his League of Nations an embarrassing failure.
Andrew Jackson - Yes, he was a strong leader, but his policies were truly awful. Vetoing the national bank did contribute to a depression. Then, there's the whole indian removal thing and political patronage thing he supported.
Franklin Roosevelt - I'm not saying he isn't a great president or that his administration has no merit, but Japanese Internment was a horrendous policy and we should certainly dock some points off his legacy for that.
Ronald Reagan (by conservatives) - He was much smarter than liberals gave him credit for but not quite deserving of the deification conservatives gave him. No doubt he was an overall strong military strategist and succeeded in Cold War policy. But then there's also Beirut, Iran-Contra, and the cocaine epidemic that he turned a blind eye to.
John Adams - The Alien and Sedition Acts are more of a talking point. He rarely (like almost never) actually enforced it. With a strong defense policy as well as diplomacy, he proved to be a great but much underappreciated leader
John Quincy Adams - The premise behind his presidency being bad (in the 1800's) was that the government didn't need to be building libraries and infrastructure. Clearly, history proved his opponent's wrong and Quincy Adams was quite farsighted.
George H.W. Bush - He was above partisanship and supported compromise. He added a tone of sobriety after the grandiose drunken stupor of the Reagan years. Most of all, he was level-headed, mature and practical about things. Very rare for a president.
(In all three cases, they were overshadowed by a more charismatic president)