Dolley Madison was born in North Carolina on May 20, 1768. Her two parents, John Payne and Mary Coles Payne, were both Quakers. She lived in North Carolina with her two older brothers, Walter and William Temple. Issac would soon join the Payne family. After four years of living in North Carolina the family moved to Virginia. While living in Virginia Dolley's mom was able to give birth to 3 more daughters, Lucy, Anna, and Mary, and 1 son, John Coles. Nobody knows where the Paynes lived in Virginia. Two places that the Paynes might have lived were the Scotchtown Plantation (shown above) or with the family of Dolley's mother at Coles Hill Plantation. In 1775 the American Revolution began, causing slavery. John Payne didn't believe in slavery, he was a very religious man. When Dolley was 15 her family moved again, this time to Philadelphia to start a new life. When Dolley's father's business, a laundry starch business, failed and the family was in debt he became very depressed. He died on October 24, 1792.