Robert E. Lee and His Children
Virginia Louise Lee


It is a desirable thing to be well descended by the glory belongs to the ancestors.


It was a cold, raw winter day, that January 19, 1807. Chill winds blew across the Potomac River that flowed at the rear of Stratford, the ancestral home of the Lees, in Westmoreland County, Virginia. In a high-ceilinged bedroom at its southeast corner an unwelcome baby boy had just been born. He was named Robert Edward, after his maternal uncles, Robert and Edward Carter of Shirley.1

The mother was Anne Hill Carter Lee. She had recently returned from a visit to her old home during which time her beloved father had died. This recent sorrow, plus the genteel poverty to which her husband’s financial indiscretions had reduced the family, left her in no state of mind to anticipate with any joy the coming of another child. In addition, the journey back to Stratford had been made in the open carriage that was the only one remaining of the many that had formerly been there, and traveling thus through the bleak countryside the wind and cold air had penetrated through the flaps, chilling her to the bone. By the time she arrived home she had a bad chest cold.2 With all these troubles, and grieving over the illness of her favorite sister Mildred, it is little wonder that she felt depressed. As Anne Carter Lee lay in the big four poster bed in a room warmed only by a few sticks in the shallow fireplace, and gazed out into the brown, wintry garden, her only consolation must have been that at last she had been relieved of her burden.

The prospects were not bright for the future of this child, nor indeed for any of his brothers and sisters.3 The heritage of a great name was about all they then had left. Even the old house, filled with memories and portraits of other Lees who had occupied leading positions in the colony since the first Lee came to Virginia, was not theirs. It was willed to their older half-brother Henry, and upon his coming of age it would pass into his possession.4

But if they were poor financially they were still wealthy in honor and achievement. It is not hard to imagine Anne Lee, in the troubled years following the birth of her fourth son, when chains were placed across the front door to keep out clamoring creditors, taking the small boy by the hand, leading him up and down the great hall hung with pictures and telling him of those other Lees whose stories of accomplishment and distinction were an incentive to all who bore the name. Those Lees who had lived there before him, and who bewigged and bepowdered, watched him from the walls to see that he lived worthy of his race. It was all she could do to assuage her pride; it would help her son, too, she hoped.

The Lees had a proud record of prominence and performance in England long before the original immigrant came to America. Launcelot Lee had, according to family tradition, fought under William the Conqueror at Hastings, and Lionel Lee had ridden with Richard the Lion-Heart in the Third Crusade. For bravery at the siege of Acre he had been created Earl of Litchfield and his armor hung in the Tower of London. A Sir Henry Lee was knighted by Queen Elizabeth, while two other Lees were made members of England’s highest order, the Order of the Garter, and their banners hung in St. George’s Chapel, Windsor Castle, burial place of English Kings, and chapel of the royal family. Above the banners are displayed to this day the family arms emblazoned with their motto, Non Incautus Futuri, “Not incautious of the future.”5

Richard Lee, a merchant who may or may not have descended from this line, first of the family in Virginia, went there during the reign of Charles the First as Secretary to the Colony and member of the King’s Privy Council. He was a man of some means in England, where his home was located in Shropshire.6

Richard soon acquired great estates in the colony. The earliest land grant recorded in his name bears the date of August 10, 1642.7 It consisted of one thousand acres to which he gave the name “Paradise.” Judicious additions and close adherence to the Fairfax family interests made him one of the wealthiest landowners in Virginia.

John, the eldest son of Colonel Richard Lee, died unmarried so that his second son, another Richard, succeeded to his Virginia estates. He was a graduate in law at Oxford, distinguished for his learning, and spent almost his whole life in study.8

This Richard Lee’s fifth son, Thomas, being a younger son, received but a limited patrimony.9 However, by industry and ability, particularly as an agent for the Fairfax lands, he acquired a considerable fortune, became a member of the Council, and was widely known and respected. Thomas had such strong natural “parts” that with only a common Virginia education (not much in those days), and by dint of his own diligence and efforts, he also became a good Latin and Greek scholar long after he was a grown man. Thomas Lee was the first native born acting governor of Virginia and the builder of Stratford.10 His granddaughter, Matilda Lee, heiress of Stratford, became the first wife of Henry Lee, father of Robert E. Lee. Fearful of her children’s future she put the estate in trust for them, to protect them against her husband’s creditors, a circumstance which lead (sic) to Henry Lee and his second wife and children, including Robert, leaving Stratford to live in Alexandria when Robert was four years old. Although he left it at such an early age, Robert’s recollections of Stratford were to remain as vivid as they were tender and pleasant.

Many years later, during the Civil War, his daughters visited the old place and when he heard of it he wrote them on 22 November 1861

I am much pleased at your description of Stratford and your visit there. It is endeared to me by many recollections, and it has always been the desire of my life to be able to purchase it. Now that we have no other home, and the one we loved has been for ever desecrated, that desire is stronger with me than ever.11

Again, in a letter to his wife, 25 December 1861, he had expressed a wish to purchase Stratford, as “the only other place that I could go to, now accessible to us, that would inspire me with feelings of pleasure and local love. I wonder if it is for sale and at how much.12

But if Robert E. Lee was the inheritor of a proud lineage on his father’s side, he was no less the fortunate heir to a tradition of nobility of character on his mother’s side.

She was the great-granddaughter of the celebrated “King” Robert Carter, so called because of his wealth and regal style of living. He had been a Speaker of the House of Burgesses, Rector of the College of William and Mary, and Governor of Virginia.13

By marriage the Carters had acquired the Shirley plantation on the James River where Anne Hill Carter was born, one of thirteen children of Charles Carter’s second marriage.

The Carters were deeply religious, with a simple straightforward piety that expressed itself in deeds as well as words. Wealthy though they were, waste was abhorrent to them, although never failing in Christian charity and generosity. Charles Carter, Anne’s father, yearly gave a certain amount of the sales of his tobacco to the poor of London, giving as his reason that there were not enough indigent persons in Virginia for him to fulfill his Christian duty of charity.14 His hospitality and benevolence were commensurate with his station, and all the Carters were known throughout Virginia for their piety and prudence.

And as the Lees had produced statesmen and patriots, the prolific Carter daughters became the ancestresses of an extraordinary number of likewise gifted men, including three signers of the Declaration of Independence, three governors, and two Presidents of the United States.15

Anne Carter Lee was typically a Carter in her unfeigned faith and her example of a living Christianity. There were always prayers at home, and when the family moved to Alexandria, regular attendance at Christ Church, where the boy Robert came under the influence of the Reverend William Meade, a kinsman and later Bishop of Virginia. His influence was to measurably reinforce the religious turn of mind already evident in Robert, and to develop into a lifelong friendship as well.

One more Lee for the boy to look up to and venerate was his own father, General Henry Lee, better known as “Light-Horse Harry.”16 Much fairer than the prospects for Roebert (sic) E. Lee had been those of Henry Lee, born at Leesylvania in Prince William Country, January 29, 1756. He was the eldest son of Henry Lee and Lucy Grymes. She had been famous as the “Lowland Beauty” and tradition has it that George Washington at one time was greatly enamored of her if not actually a suitor for her hand.

From boyhood Henry had the high intelligence of his father’s family and the physical charm of his beautiful mother. After receiving the usual rudimentary education at home, at thirteen he was sent to Princeton College where he was an excellent student and exhibited high moral character. The profession of law was thought best for the display of his talents and after graduation in 1773, at the age of seventeen, it was intended that he should go to England to prepare for the bar. But already the shadow of war was threatening and the prospective lawyer became a soldier instead.

He took an active part in organizing and drilling the militia and at the age of nineteen was appointed by Governor Patrick Henry a captain of cavalry in the Virginia regiment commanded by his kinsman, Theodoric Bland.

A vigorous man, five feet nine inches in height, of cool courage, great ability, and unceasing activity, he became attached to Washington’s command during the Revolution, and soon won the attention and esteem of the Commander in Chief. He earned a commendation for gallant behavior, and for beating off a surprise attack at Spread Eagle Tavern, New York in January, 1778, he was promoted to major and given a mixed command of infantry and cavalry which was officially designated “Lee’s Partisan Corps.”

When Major Lee stormed Paulus Hook, New Jersey, in July, 1779, he evinced so much prudence, bravery and tactical skill he was praised by Washington in unstinted terms. Congress voted him thanks and a gold medal, a distinction conferred on no other officer below the rank of general during the war.17 He was privileged to address his dispatches directly to Washington who had ordered him to mark them “private.” Lafayette held him in the highest regard as did the other officers of the army.

After three years of service in the northern states, Lee was promoted to lieutenant colonel of dragoons and transferred to the Southern Department under General Nathaniel Greene’s command. Although its commander was only twenty-five years old, “Lee’s Legion” was already famous and he one of the most renowned of American soldiers. In the southern campaigns he won an even greater measure of fame. That the Carolinas were recovered for Congress was due in no small part to the talents and energy of the dashing cavalry leader. He was a typical beau sabreur, delighting in nothing so much as to lead a charge, saber flashing, straight into the British troops.

The war over, something happened to the brilliant young officer. His mental outlook changed and the tragedy of his life began.18 He became sensitive, resentful and imperious, jealous of his own reputation, and convinced that his brother officers were envious and hostile. In this confused frame of mind he debated whether to remain in the army or return to civil life. He chose civil life. He resigned from the army early in 1782 and that same year married his cousin Matilda Lee. She was the heiress to Stratford, left to her by her father, Philip Ludwell Lee, eldest of the patriot sons of Thomas Lee. It was a happy marriage although only two of their four children survived childhood.

Following the example of public service of the Lees before him, Henry Lee began an auspicious civil career, being successively a member of the state legislature, member of Congress, and Governor of Virginia for three terms. In the first open challenge to federal authority, the “Whiskey Rebellion” in western Pennsylvania, 1794, he led the troops that quelled the mutiny. He vigorously opposed attacks on the United States Constitution, and ardently urged his friend and hero, Washington, to accept the presidency. When Washington started for New York to be inaugurated it was Lee who wrote the farewell speech.

A man of letters, devoted to the classics, as renowned with the pen as with the sword, he was also a master of oratory. When Washington died he was chosen to deliver the funeral oration and it was in this eulogy that he uttered the famous “first in war, first in peace, and first in the hearts of his countrymen.”

For all his military and civil achievements, Harry Lee had a weakness in his character which manifested itself in a mania for speculation. When he lost his money in one venture, to recoup it he would try another. When that failed, and another, and another he got so heavily in debt that parts of Stratford had to be sold, some of the rooms closed off, its doors chained against the sheriff, and the whole estate fell into a sad state of disrepair.

In 1790 the death of his wife, followed soon after by that of his son Philip, added grief to his financial woes, and he realized that he should have stayed in the army. Casting about for suitable duty he sought to command the forces being sent to the northeast frontier but failed. He then decided to throw in his lot with the revolutionary party in France which had offered him the rank of major general, but the excesses of the Reign of Terror and the advice of Washington dissuaded him from this step. Another reason he gave up the idea was the opposition of Charles Carter whose daughter Anne he was then courting.

Promising the father that he had put all such ideas out of his mind, he obtained consent to the marriage, and on June 30, 1793, Henry Lee was united in marriage to Anne Hill carter. This, too, was a happy marriage, despite being blighted by poverty, six children being born of the union.

His desperate ventures persisted, however, and his income became so reduced that many lawsuits ensued seeking to force him to meet his obligations. Finally he was locked up as a common debtor in the Westmoreland County jail at Montrose, April, 1809.19 Here, to while away his time, he began writing his Memoirs of the War in the Southern Department of the United States, hoping, no doubt, with his usual optimism, that it would be a great success and provide him with an income.

The next year he was released and returned to Stratford where his son Robert, now three, was absorbing his impressions of the family home and lessons of past glory from his mother. But Stratford was now in the possession of Henry Lee, eldest son of Henry and Matilda, who had inherited it from his mother. Besides, the children were growing up and needed an education which was unavailable in that vicinity.

In 1811 the family removed to Alexandria, substituting for the broad acres and huge rooms of Stratford a small brick house on Cameron Street, with a tiny garden in back. Here the sixth and last child, Catherine Mildred, was born.

All that the family had to sustain them was a trust fund left to Mrs. Lee by her prudent father and which was barely sufficient to provide food, clothing and shelter.20

A year and a half later another tragedy befell the already tragic “Light-Horse Harry.” As a die-hard Federalist and an active opponent of war with Great Britain, he had gone to Baltimore to visit a young editor whose anti-war editorials inflamed a “patriotic” mob. The mob attacked the editor and his friends, killing one and severely beating eleven others, including Henry Lee. While these were lying on the ground, presumed dead, the drunken rioters set upon them, mutilating them with pen knives, pouring hot candle grease into their eyes, and attempting to cut off their noses.21

At last, on the pretext of giving him a decent burial, Lee was carried off by friends but so near death that he was weak, crippled, disfigured, and doomed to invalidism for the rest of his life. Unable to work, and cut off by this ghoulish act from accepting the military commission that would certainly have been offered him when the war occurred, the almost incredible optimism that had sustained him all during his futile schemes and dreams at last deserted him. Sick in body and mind, he wanted only peace. Pride prompted him to seek that peace outside the confines of his own country, where he would not be an object of pity or derision.

In the summer of 1813, he said farewell to his wife and children in Alexandria and sailed for the West Indies. He was never to see them again. He remained in the tropics for five years, wondering (sic) from island to island in search of health. Then, despairing of recovery, he determined to return home to see his family for the last time. But on the voyage he became so much worse he was put ashore at Cumberland Island, off the coast of Georgia, where his former commander and friend, General Nathaniel Greene had had his estate and where his daughter still lived. Although tenderly and skillfully cared for, his sufferings were intense, and two months later he died. There, on March 25, 1818, with full military honors, he was buried close to the grave of General Greene.22

So ended the life of the father of Robert E. Lee. A life begun with high promise, carried through to great achievements, deceived by too great expectations, and destroyed by too much optimism.

Return to Robert E. Lee and His Children