James Madison President of the United States from 1809 to 1817 (1751–1836) (1751–1836) Parents Eleanor Rose Conway James Madison, Sr. Children Siblings Ambrose Madison unknown son Madison William Madison Frances Taylor Madison Rose Reuben Madison Catlett Madison Elizabeth Madison Francis Madison Sarah Catlett Madison Nelly Conway Madison Partners Dolley Madison