Jefferson stocked his Cabinet with people he knew and trusted, with the most significant members being James Madison as SecState and Albert Gallatin as SecTreas. This triumvirate of Jefferson, Madison, and Gallatin would stay together and united, unlike the first trio of Washington, Hamilton, and Jefferson (and the nightmare that was John Adams' Cabinet). One benefit that Jefferson experienced by having Madison in his Cabinet is that his Secretary of State became the "Lighting Rod of Hate" for the Federalists, taking much of the attention away from the new President (e.g. Marbury v. Madison).
James Callendar, however, kept his focus (and newfound contempt and hatred) on Jefferson; he was out for political revenge against a President that did not find a position for his loyal newspaper publisher in his adminstration. Callendar, now writing for a Federalist newspaper, "outed" Jefferson in terms of his illicit relationship with one of his African slaves, Sally Hemings. In an event that reeked even then of conspiracy, James Callendar was found dead in 1803, having drowned in the James River. While it was true in those days that journalists were often attacked and assaulted, many Americans (especially Federalists) believed that Callendar's death was not an accident.
As Secretary of State, Madison shared two goals with President Jefferson: peace and expansion. They were also in agreement in terms of how they viewed nations in Europe, most importantly France (awesome!) and England (root of all evil!). The main advantage of their relationship was consistency, with the main disadvantage being that their mutual agreement / world view blinded them to difficulties and failures.
Two events changed France's empire-building west of the Mississippi River: a "Polar Vortex" so severe that all of France's ports on the Atlantic actually froze solid in 1803, and an African slave revolt in the Caribbean sugar cane island of Santo Domingo (led by Toussaint L'Ouverture). The impulsive Napoleon decided that the stars were no longer in alignment for his goals in North America, and he offered the Louisiana Territory to the United States for purchase. Robert Livingston and James Monroe signed the document agreeing to purchase the territory from France, and Jefferson and Madison started to work to get votes in Congress to finalize the transfer of land (ironically, the LA Purchase DID NOT include West Florida or New Orleans; it wasn't until Madison was President in 1810 that the U.S. gained what it most desired in the Gulf region).
In 1804, Jefferson easily won re-election, receiving 162 Electoral Votes, with the Federalist candidate, Charles Pinckney, garnering only 14. The Republicans also gained an astounding 80% of the seats in the House and the Senate. The purchase of the Louisiana Territory and the results of the Election of 1804 went to the heads of Jefferson and Madison. In short, they both believed that they could do no political wrong; they would not be ready for the foreign difficulties that lay ahead in Jefferson's second term as President.
In 1805, Jefferson's and Madison's relatively idyllic time in office came to a halt, when in July the British Government changed the rules of the game in the Atlantic Ocean. In recent years, the U.S. had been able to trade under what was called "The Broken Voyage" rule, which meant that U.S. ships could sail the expanse of the Atlantic, but had to stop at specified British ports on their way to their final destination. It was a win-win situation for both the U.S. and Britain, but with the threat from Napoleon, Great Britain reinstituted the "Rule of 1756", which in essence put up trade barriers exclusively to Britain's benefit; it was a de facto return to Mercantilism.
Madison published an argument that laid the basis for "Freedom of the Seas" in America's foreign policy, and instructed the U.S. Minister to Britain, James Monroe, to make an issue of impressment. Impressment now became an additional "affront" when combined with the British trade barriers. By late-1806, Britain and France had created a situation where the U.S. couldn't trade with either nation. The main source of revenue (and jobs in cities) was revenue from free trade, and it was in sharp decline; the Jefferson/Madison vision of a totally self-sufficient nation was taking some painful body-blows.
In July of 1807, the British warship Leopard attacked the U.S. frigate Chesapeake, killing three Americans and wounding eighteen more; the British took four sailors, and one was hanged. To the Federalists (and a growing number of Republicans), this situation seemed to epitomize the foreign policy of Jefferson / Madison . . . it seemed to be "Wussyville."
The Embargo Act of 1807 showed the "I'm too good for this world" streak in Jefferson's character. His mountaintop mansion (Monticello), his extreme dislike of face-to-face arguments, and his small invitation-only dinner parties also confirmed that aspect of his character (as would his ultimate reaction to the failure of the Embargo of 1807). While Jefferson supported the embargo by signing it into law, the embargo was actually Madison's brainchild - he (and Jefferson) believed in the power of commercial warfare. SecTreas Gallatin objected to the embargo; he understood that the U.S. was not truly self-sufficient, and was not in a position of power to benefit from the trade barrier. He also (correctly) pointed out that enforcing the embargo would in-and-of-itself be a nightmare. But, Gallatin was loyal to his President and his party, and he publicly supported the Embargo of 1807.
Before the Presidential Election of 1808, Madison had to defeat James Monroe within his party without burning any political bridges in the process. Dolley Madison's skill as a hostess-extraordinaire helped her husband the most, since he needed as many Republicans in Congress to support him as possible to get the nomination. In addition, Madison was able to secure the nomination during the caucuses in that he had the support of SecTreas Gallatin, and the effects of the embargo hadn't become a reality (or political liability) yet. When the full negative impact of the embargo came to fruition, the Republican delegates were already committed to Madison. In the Electoral College, Madison easily won re-election with 122 votes to Charles Pinckney's 47 as the Federalist candidate.
The Embargo of 1807 did not end until the Republicans and Federalists in Congress acted in concert on their own (it took that political and economic disaster to get both parties to work together); both Houses voted to end the embargo on 4 March, 1809 (Jefferson's last day in office). Jefferson signed the bill into law, ending the embargo, attended Madison's Inauguration, and then retired to Monticello for good.
(Below: The Presidency of Thomas Jefferson, from the History Channel's "The Presidents")