He began his legal education by reading lawbooks and by studying law in 1780 with George Wythe at the College of William and Mary to acquire a secure foundation in English common law. As an officer in the Continental Army, he served as a judge advocate in 1777 and 1778. Marshall was admitted to the bar in 1780 and steadily built a law practice. He represented Fauquier County in the General Assembly for one term and served on the Council of State from 1782 to 1784. Marshall represented Henrico County in the House of Delegates from 1784 to 1787, and in June of 1788 was a delegate to the state convention called to ratify the U.S. Constitution. Marshall was also a delegate to the 1829-1830 Virginia constitutional convention.

In 1785 Marshall settled in Richmond and promptly offered himself as a candidate to city council. Finishing second in the balloting, he was chosen city recorder, a position that enabled him to sit as a magistrate on the Richmond Hustings Court which handled minor civil and criminal cases. Although his formal legal education consisted of a few months of lectures, Marshall learned his trade by experience. He developed an ability to focus on and address the issues in clear language. Marshall gained a reputation for adherence to a strict republican ideal of subordinating self-interest to the public good, controlling oneself by reason, and maintaining a sense of duty.

Author Samuel Mordecai, in his Richmond in By-Gone Days (1856), recalled Marshall carrying a basket to do the family marketing. Marshall was a frequent visitor to Richmond's theaters and enjoyed reading the novels of Jane Austen. In addition to being an able chess player, Marshall was renowned among the gentlemen of Richmond as a formidable quoits player. An ancient game of skill that required tossing a brass or iron ring to a meg, the iron stake placed in the ground, quoits resembles the modern game of horseshoes. Marshall was a member of the Buchanan Spring Quoits Club, which was known also as the Richmond or Fairfield Sociable Club. At this gathering of the club, the city's influential men, such as Marshall, John Wickham, William Wirt, Benjamin Watkins Leigh, John Buchanan, and John D. Blair, enjoyed food and drink and pleasant conversation. Political discussion was strictly prohibited. As a member of the Barbecue Club, Marshall was known for imbibing the punch laced with brandy, rum, and Madeira.

In 1812 the General Assembly appointed a twenty-two-man commission to survey the James River from its headwaters to its terminus at the Chesapeake Bay to determine a means to connect the eastern and western parts of the commonwealth. Marshall's reputation as a fair-minded jurist resulted in his appointment as chairman. Andrew Alexander, surveyor for Rockbridge County, was the principal surveyor on the commission. The map was engraved in Philadelphia, and Marshall was the principal author of the published report. Marshall performed one last service to the commonwealth in 1829-1830 when he joined James Monroe and James Madison as delegates to the Virginia constitutional convention.

In 1800 Marshall agreed to write a biography of George Washington. The general had left his papers to his nephew, Bushrod Washington, who approached Marshall about writing the biography. As a friend of the first president, Marshall had announced Washington's death in 1799, had offered the eulogy, had chaired the committee that arranged the funeral rites, and had led the commission that planned a monument in the nation's capital. Marshall began writing in 1801 and continued to add to the manuscript for five years. The resulting biography in five volumes totaled more than 3,200 pages. More than 7,000 copies were sold for one dollar each volume.

Marshall had served previously as Adams's envoy to France during the XYZ Affair and in 1799 in the U.S. House of Representatives. Marshall accepted the appointment on 2 February 1801 and served as chief justice for thirty-four years. Marshall heard cases and offered groundbreaking opinions that continue to guide the Supreme Court and the United States government today. The Marshall court established the principle of judicial review, in which the court ruled that the Supreme Court had the power to declare invalid any act of Congress that was in conflict with the U.S. Constitution. The Marshall court also ruled that state judiciaries could set aside state legislative acts if they conflicted with the federal Constitution and that the U.S. Supreme Court could reverse a decision of a state court. By his opinions, Marshall increased the power of the Supreme Court as a branch of the federal government, emphasized the role of the judiciary in the states, and reinforced the national supremacy of the federal government.

John Marshall died on 6 July 1835 in Philadelphia. He was buried next to his wife, Mary Ambler Marshall, in Shockoe Cemetery in Richmond. Today the John Marshall House, located behind the Library of Virginia, is open to the public as a historic house museum.