James Foster Scott was born in India at Futteguhr, 50 miles from Agra, on January 22, 1863. His closest relatives were the Fosters of Chile, where his mother's brother, Julius Mulford Foster, of Milford, carried on a large milling business and left numerous descendants. His son, Manuel Foster, of Chile, was one of the noted lawyers of South America. Dr. Scott's father was the Rev. James Long Scott, a Presbyterian minister, and his mother was Eliza Jane Foster, a Presbyterian missionary, as well as a pioneer in women's education.

James Foster Scott attended Hastings Academy in Philadelphia and took an A.B. at Yale in 1884, where he was an athlete who rowed on the varsity crew and developed his great love of rivers and boats. A book written about him, Sourdough Gold, by Mary Lee Davis and published in 1933, tells chiefly of his solo trip down the Yukon in a small boat in 1898 when he piloted his boat over the famous White Horse Rapids and other dangerous places in Alaska.

Dr. Scott received an M.B. and C.M. from Edinburgh University and later an M.D., completing his post-graduate work at Vienna. For three years he was a member of the Volunteer Medical Staff Corps of the British army and spent three years in the British West Indies. At the age of twenty-seven he was made head of the Columbia Hospital in Washington, D.C. and was elected vice president of the Medical Society. He was considered one of the most brilliant young doctors in the profession and wrote a medical book that is still quoted by authorities. At the height of this early success and in the midst of a brilliant career and fashionable life in Washington D.C., Dr. Scott astonished his friends by giving it up and going to the wilds of the Yukon, after which he lived a retired life in the Virginia woods and spent time rowing the Potomac. As the book Sourdough Gold explains, he did not care to be a distinguished gentleman with a silky bedside manner but set out to find the real man beneath all the over-education he had received. Friends and neighbors say that no two people were ever more ideally happy than Dr. Scott and his wife in their "Kalokagathon" retreat, the Greek term for the Greek Ideal of the "Good and the Beautiful and True" which Dr. Scott made the principle of his life.

Dr. Scott wrote numerous volumes on philosophy with special emphasis on ethics and Christianity. Dr. Scott’s interests in these topics were partly derived from attitudes found his mother's family, enlightened Puritans that had migrated to America in 1635. These writings analyzed the relationship between Christian ethics and religion and the modern sciences.

Dr. Scott died on Wednesday, February 20, 1946 following an operation at the Emergency Hospital. Funeral services were held Friday at his home near McLean, Virginia. A physical culturist at 83, Dr. Scott was cutting down trees only a few days before, as was his daily exercise, together with shooting the rapids in his boat in the Potomac below his house on the hill. His death came before he was able to complete his writing, and he left in a fireproof safe the product of nearly 50 years of study and research intended for publication as a series of works on philosophy.

For over 30 years, Dr. Scott lived near McLean, Virginia where he was sometimes called the "Hermit of Scott's Island," an island he bought along the Potomac, upon which he built a house. Dr. Scott married Karen Gram on March 13, 1940 and they had no children.