Robert E. Lee and His Family
General Lee and his wife, Mary Anne Randolph Custis, had four daughters, born at Arlington: Mary Custis, in 1835; Anne Carter, June 18, 1839; Eleanor Agnes, between January and March, 1841; and Mildred Chiles, February 10, 1846.
Mildred was about three years old when Colonel Lee returned from Mexico. Some of her earliest recollections were associated with Baltimore where the family lived for about three years, on Madison Street, while Colonel Lee was engaged in the construction of Fort Carroll, near the city. Her older sisters and brothers had spent several years at Fort Hamilton on Long Island while their father was in charge of the defenses of New York harbor. When Mildred was six years old Colonel Lee was made superintendent of the U.S. Military Academy at West Point, N.Y., and lived there for the next four years. She was fifteen when the family had to leave Arlington in the spring of 1861.
In February, 1853, Colonel Lee wrote a long letter to his daughter Anne Carter, whom he always called Annie. She at this time was nearly 14, and as it appears, she and her younger sister Agnes were at the time at Arlington with their grandmother Custis. He gave her much good advice, some at least of which she was to pass on to Agnes. “I am told,” he wrote, “you are growing very tall, and I hope very straight.”
From Arlington, in May 1861, Mrs. Lee went to Ravensworth and the girls to Fauquier County to visit relatives. During the next year or two they all moved around rather frequently. Early in June Annie was at the White House in New Kent County with Rooney's wife. In August two of the girls were in Richmond.
Photo by André Studio, Lexington.
MARY CUSTIS LEE
Photo about 1865
From the collection of Frederick H. Meserve on the walls of
Lee Chapel, Washington and Lee University.
In September, 1861, Mrs. Lee went to Hot Springs in Bath County, Va., accompanied by Mary and Rob. The other girls, apparently, were at that time with Charlotte, Rooney's wife, in New Kent County, or in Richmond. Mildred was thinking of going to school at Raleigh, N.C. On November 15 the General, at Charleston, S.C., wrote to Mildred—”Life” was his pet name for her. “Life” at this time was at school in Winchester, and had written to her father not long before telling about a visit she had enjoyed at Kinloch, the home of Mr. Edward Turner in Fauquier County. Three days later, November 18, General Lee wrote from Savannah, Ga., to his eldest daughter Mary, acknowledging a letter she had written him from Shirley, his mother's old home. Mary, in the General's letters to other members of the family, was usually referred to as “Daughter.” Annie and Agnes at this time were at Clydale, the summer home of Dr. Richard Stuart, in King George County. From there they had made a trip over to Stratford, their father's birthplace, in Westmoreland County, and had written to him about it. From Savannah, on November 22, he replied, expressing pleasure in their description of Stratford, but seemed disappointed that they did not mention the spring, which was one of the objects cherished in his early recollections. He expressed gratification that Agnes had become “so early a riser.”
On December 8 he wrote to Annie from Coosawhatchie, S.C., and again to her on March 2, 1862, from Savannah. He wrote frequently to Mrs. Lee, and, of course, there were other letters to his daughters in addition to those mentioned.
On October 20, 1862, Anne Carter Lee (Annie) died at White Sulphur Springs, Warren County, N.C., at the age of 23 years and four months. To her the General was specially devoted; she, as he declared, never gave him aught but pleasure. It may be that his affection for her was heightened by the fact that she in early life had suffered an injury to one of her eyes—to this he referred in his will which was made when she was seven years old. At the close of the war the people of Warren County marked her grave with a plain shaft of native granite. General Lee was invited to be present at the unveiling of this monument, but was unable to attend. It may be that Mildred was with her sister at the time of her death, for she was at school in North Carolina in December, 1862. The General wrote to her on Christmas Day; the next day he wrote to Agnes, who was in Richmond with her mother. To Agnes he wrote again on February 6, 1863. At that time he was still in the vicinity of Fredericksburg.
Photo by Cook of Richmond.
HOUSE ON FRANKLIN STREET, RICHMOND
Home of the Lees in the closing years of the War.
Now the home of the Virginia Historical Society.
During the last year or two of the war the daughters, apparently, were with their mother in Richmond at the house on Franklin Street. There General Lee found them on April 15, 1865, when he arrived in Richmond from Appomattox. In the latter part of June the girls, with their father and mother and brother Custis, moved out to Derwent, the quiet country home in Powhatan County. About the first of December Mrs. Lee, with Rob, Mildred, and perhaps Mary, went up to Lexington. Agnes had gone from Bremo to Richmond to attend the wedding of a Miss Warwick. She later joined the family in Lexington.
Photo by André Studio, Lexington.
MILDRED C. LEE
Daguerrotype given to her school friend, Katherine Seymour
Green (Mrs. John Paul of Harrisonburg, Va.); presented to
Washington and Lee University by Mrs. Paul's daughters, Miss
Virginia Paul and Mrs. Greenlee D. Letcher.
After Captain Rob Lee brought the riding mare Lucy Long to Lexington in the autumn of 1866, one of the girls frequently mounted her to accompany General Lee on his rides on Traveler out through the surrounding country. In the summer of 1867 he and Mildred rode down to the Peaks of Otter in Bedford County. Later the same summer he spent some time at Greenbrier White Sulphur Springs with Mrs. Lee and Agnes. In the summer of 1868 the family went to Warm Springs in Bath County where Mildred had a long siege with typhoid fever. She was still weak and nervous when the family returned to Lexington early in September. In the spring of 1870 Agnes accompanied her father' on an extended trip to Georgia, undertaken for the benefit of the General's health, upon the urgent advice of his physicians.
ELEANOR AGNES LEE
From page 217 of Rose M. MacDonald's
Life of Mrs. Robert E. Lee, by permission.
Agnes Lee died at Lexington on October 15, 1873, almost exactly three years after the death of her father; and her death was followed by that of her mother, 21 days later. Mary and Mildred survived.
A gentleman who was a student at Washington College when General Lee was president, writing in later years, gives interesting particulars relating to the Lees. He says:
The Lee family living in the home in Lexington as I knew them consisted of General and Mrs. Lee, their son, George Washington Custis, who succeeded his father as President of the College, and three daughters, Mary, Agnes, and Mildred. Agnes was an invalid and was away from home on health trips most of the time and I saw very little of her. The oldest daughter, Mary, was a person of strong, but somewhat eccentric character. She was wholly devoid of fear and was fond of taking long walks in the country alone.”
The same gentleman then goes on to relate a characteristic incident. On one of her walks, on the road leading to House Mountain, Miss Mary came upon a man, bringing a load of apples to town, who was savagely beating his horse, which seemed unable to pull the wagon out of a mudhole. Miss Mary walked up to him, demanding that he stop beating the horse, and promising to show him how to get out of the hole. This she did. Later she induced some of the workers in church and Sunday-school to look him up. Under their influence he reformed his habits, came to church with his family, and became an upstanding member of the community.
In 1912, when Fred W. Alexander published his book on Stratford Hall and the Lees, Miss Mary was traveling in Europe. She died November 22, 1918. Her sister Mildred had died at New Orleans on March 27, 1905. Her funeral was held at Lexington four days later.
 See “Lexington in Old Virginia,” by Henry Boley, 1936 page 123.
Return to Robert E. Lee and His Family