Washington and Lee University

Robert E. Lee and His Children
Virginia Louise Lee


He who commands others is powerful, he who commands himself is mighty.


At Stratford the boy Robert had been free to wander at will over the extensive grounds. There had been animals to watch, a pond to wade in, open fields where the young Lees could chase each other after the manner of all children, gardens full of beautiful flowers, and trees everywhere: oaks, cedars, maples, poplars, horse-chestnut and beech. His first years were spent in close contact with nature and ever afterward the deepest desire of his heart was to remove to some quiet place in the country where he could again experience the peace and joy which such communion with nature alone gives. In this he was a true Lee as the crest of the Lee coat of arms bears a squirrel, denoting a lover of the woods and emblematic of the thought that sylvan retirement was the delight of its bearer.1

Alexandria was a vastly different place from Stratford. It was a handsome, bustling town, full of sights and sounds and places calculated to attract and enthrall the mind of any little boy, especially an alert, inquisitive boy like Robert.2 Even though he missed the openness of Stratford, Alexandria was full of Lee relations. One of these was his cousin Cassius Lee who became his close friend and playmate.

Growing up in such a place was a fascinating experience, there was always so much to see and to do. He and Cassius could spend hours on the docks watching the ships of the busy port, and in quiet coves along the Potomac they could go swimming. During the summer they would have to hunt out a place to play ball and fly their kites, as the city fathers would not allow the playing of games in the streets.3 For this reason they often went to a place called “King George's Meadows.” Much of it was not a meadow at all but a marshy place of great charm for the boys. There was plenty of tall grass, rabbits without number hidden within it, and fish in all the streams.4 In the fall, in the wooded countryside beyond Alexandria, Robert would often follow the hounds all day, running after them on foot because he owned no horse.5 In the winter there was ice skating on the frozen Potomac and interesting places to visit, like the museum on the upper floor of the market house where for twelve-and-one-half cents one could spend hours looking over all its marvels.6 Robert's love for sports and his boyhood participation in strenuous outdoor activities helped endow him with a superb constitution and to give him the physical health and strength which later enabled him to stand the strain of exhausting marches, loss of sleep, hunger, thirst, heat, cold and all the other hardships of war.

Other places to see were the old Carlyle house where Braddock's expedition against the Indians had been formed. Gadsby's Tavern, where so often Washington had visited, was still doing business. The Masonic Lodge to which he had belonged was there. It was always fun to stand outside the Friendship Fire Company and look at “Washington's” engine. The addition of a tower, a steeple and a bell to Christ Church was a procedure interesting to small boys.7 Everywhere and over everything in Alexandria lay the spirit of George Washington, for this was his home town. Not far down the river was his home, Mount Vernon, and on pleasant spring days Robert and his mother would drive down there to look at the stately white-pillared mansion and its lovely lawns.

Up the Potomac just a few miles was another mansion, Arlington, built by the adopted son of Washington, and full of Washington articles and treasures.

Robert's own father had been the protege and friend of the great commander, and pride in that friendship had been his consolation during his days of trial. Some of his comrades in “Lee's Legion” lived in Alexandria and Robert delighted to walk with his father and see the deference and respect they accorded him.8

The boy was steeped in the lore of Washington and the Revolution from his father, uncles and cousins, and at the age of seven he experienced his own first taste of war. The War of 1812 brought the British fleet up the Potomac to burn the city of Washington. He could see the smoke from across the river. Three days later ships appeared off Alexandria and British sailors swarmed ashore. Alexandria was not burned because she paid tribute to spare the town.9

His elder half-brother Henry paid the family a call on his way to the Canadian frontier, smart in his new major's uniform.10 There was constant movement and commotion all over town as soldiers were mustered in and drilled.

Alexandria became Robert E. Lee's home town as it had been that of Washington. After the Civil War, in 1869, he spent several days there renewing old acquaintances and memories, and although no formal reception had been planned, half the town came to honor him, remembering him as a boy, as a schoolmate, as a neighbor, as well as the revered Confederate chieftain. The Alexandria Gazette, speaking of the spontaneous outpouring of well-wishers said, “It was more like a family meeting than anything else for we regard General Lee as one of our Alexandria boys.”11

That he reciprocated this affection is evident in a letter he wrote the following year to some citizens of Alexandria who had requested his aid in a project concerning the city. “There is no community to which my affections more strongly cling than that of Alexandria,” he told them, “composed of my earliest and oldest friends, my kind schoolfellows, and faithful neighbors. Its interests and prosperity are of such paramount importance to me that it is my desire to do anything to promote them.”12

He often left Alexandria for many vacations to his numerous kinfolk situated all over the Northern Neck of Virginia, and others to Stratford and Shirley on the James, his mother's old home. On these vacations he indulged in the outdoor exercises he enjoyed so much, but preferred riding as he loved horses above all other animals.

On these visits he also renewed his acquaintanceship with a multitude of cousins, for if geniality was a prime family characteristic of the Carters—and they liked everybody, most of all a Carter liked his kinspeople. Since practically all the Virginia families were related in one way or another, the Custis family was numbered among them, including cousin Mary Anne Custis. She was frequently a guest at the same time Robert was visiting and they were attracted to each other even when children.

His mother was his first teacher and sporadically his father imparted some knowledge, but the first school he attended was at Eastern View in Fauquier County. The Carters were so numerous and intimate that they maintained two schools for their children, one for the girls at Shirley and one for the boys at Eastern View, the home of his mother's sister, Mrs. Elizabeth Randolph.13

The school was more like a friendly home and did not have the discipline that Robert was used to in his own home. After a term there he returned to Alexandria a little too self-willed, his mother thought, so she wrote to her sister thanking her but noting that Robert was getting difficult to handle. Her sister replied that she had always found Robert a most engaging child, not difficult at all, but that if he had become so the only advice she could give was that which she applied to her own boys: to “whip and pray, and pray and whip.”

Outgrowing Eastern view, at thirteen Robert began attending the Alexandria Academy where he was taught by Mr. William Leary. His teacher became his firm friend and the pupil had enduring respect for him.15 Three years were spent under Mr. Leary's tutelage, learning the rudiments of a classical education. He read Homer and Longinus in Greek and became well grounded in Latin. In mathematics he shone for he had the type of mind that delighted in the precise reasoning of algebra and geometry.16

At the end of three years arose the question of his career. A family conference decided that he should make personal application to the Secretary of War for an appointment to West Point.17 Competition for such an appointment was keen but with endorsements by five senators and three representatives, plus letters of character and ability from teachers and kinsmen, the latter strongly advancing Robert's claim on the basis of his father's military services, Secretary of War John C. Calhoun, on March 11, 1824, notified the anxious applicant that he was accepted. But owing to the long list of candidates he could not be admitted until July 1825, a whole year away.

During the intervening time Robert studied at the school next door to his home which had recently been opened by a young Quaker named Hallowell. Here he could brush up on his mathematics and prepare himself for the more advanced mathematics taught at West Point.

He was an exemplary student, exhibiting such traits of neatness and exactness that Mr. Hallowell said of him, “His specialty was finishing up. He imparted a finish and a neatness, as he proceeded, to everything he undertook.[”]18

This was Robert E. Lee at seventeen, on the threshold of manhood. An accident of birth had made him the heir to a great tradition, had given him a name and a standard to live up to, a legacy of character and intellect, and a handsome and healthy body. Understanding that, he can be understood as Lee the Virginian. But it is his utilization of these basic factors, plus external influences, that the progress and development into Robert E. Lee the man, a unique and individual personality, is best understood.

Of first importance in this expansion of his individual character was the influence of a remarkable mother. From her he inherited the Carter traits of piety, prudence and industry which she reinforced by teaching him how to apply them to daily life. She early taught him to practice self-denial and self-control as well as the strictest economy in all financial matters because as a Carter it was her nature to do so and because she was determined that her children would not repeat the mistakes of their father. These principles were bred in Robert so early and deeply that he retained them all his life.

Typically a Carter in her intense faith she laid the basis on which Robert E. Lee was to become, in the eyes of the world, the very model of a Christian, respected and revered more for his Christian example than even his acknowledged military ability. His father had once written of her faith, “Your dearest mother is singularly pious from love to Almighty God and love of virtue, which are synonymous; not from fear of hell,—a low, base influence.”19 This religious bedrock was to determine to a great extent the character and the destiny of her youngest son, and to give him the unfailing, unshaken confidence in the inscrutable ways of God that would carry him through days when everything seemed hopeless and all those around him despaired. Never, in a life full of anguish and sorrow, did he depart from the faith of his mother, nor did he ever forget that he was “upheld by a righteous, omnipotent hand.”20

If religion was the guiding principle of her life, as it was to become that of her son, Anne Carter Lee was also an intelligent, practical woman. She knew how to make do with little, as is shown by the discretion with which she administered her small income, which alone supported the Lee family during the years of impoverishment. She would take Robert down to the market place in Alexandria with her and show him how to select the best vegetables and fruits, the freshest and plumpest chickens. She taught him the value of money and how to live within his means, so that later he could write to his children, “if you spend only for necessities you will always have something left over.” She taught him to be neat about his person and careful with his clothing, which stood him in good stead for the inspections at West Point.

Perhaps her greatest contribution to him, next to a sublime faith, was to habituate him to a rigorous self-control. For by nature he was of a positive temperament, and of strong passions, not one of those invariably amiable men whose tempers are never ruffled and whose calmness results from lack of sensibility rather than any exercise of will.21 On the contrary he was extraordinarily sensitive. This characteristic, when considered in the light of the immense burdens that rested upon him and the numberless excuses he had for giving vent to his tempter, but did not, only reveal that Anne Carter Lee had done her work well.

Not much really is known of the mother of Robert E. Lee. The women of her time lived only in the shadow of their husbands' reputations and even fame of such proportion as Harry Lee had did not suffice to bring her out of them. That she was black-eyed and dark-haired, and the pet of her father among all his twenty-three children we know. She was lively and intelligent. Her father had her tutored in spelling, writing, grammar, and other subjects. She was talented in music, skillful at the harpsichord, with a voice flatteringly compared to a bird.

The only picture extant has not been fully authenticated.22 It shows a woman of rare beauty, wearing a miniature of Washington, and bearing a very marked resemblance to Robert E. Lee in coloring and features.

Though little else is known of her, and none of her letters to Robert survive, a few that she wrote to his brother Smith have been preserved. It is in these and in the lives of her children that Anne Carter Lee is revealed as a woman of marked piety, devoted to her friends and kin, patient in misfortune, wise and skillful in employing her means to best advantage, and consecrated to the upbringing of her children in the nurture and admonition of the Lord and in all those other virtues calculated to make their earthly lives as successful as she hoped their heavenly lives would be. In spite of continued adversity she remained courageous and uncomplaining.

In one of her letters is found another of the legacies she left to her children but which Robert evidently took more to heart than Smith did. She chides him for not writing more and says that

exclusive of my desire to hear from you I lament your dislike of writing because it will be such a disadvantage to you through life. A man that cannot write a good letter on business or on the subject of familiar letters will make an awkward figure in every situation and will find himself greatly at a loss on any occasion. Indeed I cannot imagine how he could pass through life with satisfaction and respectability.23

All his life Robert E. Lee maintained a voluminous correspondence, letters which are models of the “good style” she enjoined on her sons, and to which we are indebted for much of the knowledge that we have about him.

Since his father had died when he was eleven and his brothers and sisters were either away or too small to help much, as his mother's health deteriorated into invalidism the management of the household devolved almost entirely upon Robert. As the old Virginia phrase has it, he “carried the keys.”24 He took care of the marketing, the horses, supervised the housework, managed the finances, and was nurse-companion to his ailing mother.25

If the boy had lacked a close and intimate companionship with a father he scarcely knew, his care and devotion to his mother earned him compensation in the love and dependence she placed on him, who was “both son and daughter” to her she said. His filial duties to her became the first obligation of his youth, as the care of an invalid wife was to be one of the chief duties of his mature years. Yet this close association with his mother did not make him effeminate although it did give him a fondness for the company of women.26

He did not have much time for sports or play because of his household chores. While the other boys would run off to King George's Meadows he would hurry home from school to do the marketing before it got dark, or before it was too late or too cold to take his mother for a drive. He would carry her to the carriage, arrange the cushions around her like an experienced nurse, then cheerfully entertain her with small talk saying the drive would do her no good unless she enjoyed it. If she complained of drafts he would take out his jacknife and improvise curtains from a newspaper and stuff the cracks with it.27

All her efforts to instill habits of self-denial, self-control and prudence into her son were given ample chance to be practiced by him during these years of responsibility. Living in close intimacy with such a mother, as her physical condition made necessary, almost alone her stay and companion during his formative years, small wonder that Robert E. Lee absorbed those virtues she “was most desirous he possess,” and became the very embodiment of the greatness of spirit and Christian humility that was Anne Carter Lee's in such goodly measure.

Although the years with his father had been few, nonetheless he was a cherished memory to his son. Often away from home on one crusade or another, in prison for debts, the boy very young, Henry Lee did not have time to develop an intimate relationship with Robert. But such a vivid, dramatic personality as “Light-Horse Harry” Lee, who stirred even the somnolent members of Congress with his fiery oratory, could not fail to impress his own impressionable adoring young son. The dashing exploits of the Revolution he recounted to Robert, the glowing terms of love and affection he used in describing his idol Washington, the respect and esteem that the soldiers of his father's old corps paid him as father and son walked the streets of Alexandria, all sank deep into the little boys' mind.

Then during the War of 1812 his half-brother had appeared in his major's uniform, off to fight that war in the best tradition of his hero father. Both father and brother were to write books on military history, one on the Revolution in the South, the other on the life of Napoleon, and Robert must have heard much during the preparation of his father's book and later have pored over it when published, so that more and more the military life appealed to him, personalized as it was by his father, his brother, and Washington, the idol of the Lees and of all Alexandria.28 If, in the pious Lee home, God was worshipped first, Washington was second.29

To his wife, whatever else he may have been, and whatever the life he subjected her to, Henry Lee was the man she loved and respected in spite of his faults. She taught her children to love and respect him, too, and he, for all his misfortunes, reciprocated her devotion and respect. In exile he remembered the anniversary of their marriage and frequently referred to her in his letters to his son Charles Carter Lee. These letters are full of longing for his wife and children and anxiety for their well-being.

In one he writes, “You know how I love my children and how dear Smith is to me. . . . Robert was always good, and will be confirmed in his happy turn of mind by his ever-watchful and affectionate mother. . . .”30 Many of the letters have been preserved and are literary models, the object being to impress religion, morality and learning upon his children, as well as to express great affection for them. The high principles he sought to inculcate in his children while with them and later through letters were made manifest in the character of Robert E. Lee, who in turn wrote in much the same vein to his own children, so who can say that the influence of the father was negligible, though the time together was so short?

Especially did the absent father dwell on the theme that the pursuit of virtue and truth is man's greatest business in life, and all else should be subordinated to that end. “I would rather see you unlettered and unnoticed, if virtuous in practice as well as theory, than see you equal in glory the great Washington.”31 Again he writes, “Fame in arms or art, however conspicuous, is naught, unless bottomed on virtue.”32

Along with these precepts he imparted much advice on such practical matters as immoderate sleeping, cleanliness of person, participation in sports, care of one's health, letter writing and many other good habits he wished his children to cultivate.33

The letters of love and wisdom Henry Lee wrote from exile were meant for family consumption and his pathetic exhortations to a good life imprinted themselves on the mind of his young son Robert, already sensitive to the tragedy and pity of a life that had not practiced what it preached. They reinforced the similar teachings of his mother, if in a more worldly vein. Where she taught him self-denial and self-control, his father, after love of virtue and truth, also emphasized the acquirement of complete self-command as “the pivot upon which the character, fame, and independence of us mortals hang.” He wanted his children to turn their attention steadily and closely to this cardinal quality, and to “habituate yourself at once to reject with disdain every temptation which may assail your self-dominion.”34

That his parents succeeded in their efforts to develop self-control as a fundamental basis of his character is amply borne out in the life of Robert E. Lee, for of all mortal men few have equalled such mastery of self as he had.

Perhaps the most notable influence on the boy from his father, other than character and the military tradition, was political. A Federalist, and strong supporter of President Washington, Henry Lee was first and foremost a Virginian. In 1792, when he was Governor of Virginia, James Madison had written to him to sound him out on the subject of resigning his office to take command of the forces then being considered for the western frontier. Governor Lee had replied

One objection I should only have. . . . and that is, abandoning my native country, to whose goodness I am so much indebted. No consideration on earth could induce me to act a part, however gratifying to me, which could be construed into disregard or forgetfulness of this Commonwealth.35

And in a debate on the Virginia-Kentucky Resolutions of 1798-1799, he had put it even more strongly: “Virginia is my country; her will I obey, however lamentable the fate to which it may subject me.”36

This doctrine was inherited by his son and when the test came for him in 1861 he reacted in the only way possibly that the son of “Light-Horse Harry” Lee of Virginia could have reacted.

Robert E. Lee's entire life demonstrated his father's influence on him, an influence heightened by the imagination of youth, intensified by emotions of pity and sympathy, and dramatized by separation and distant death. To a man sensitive enough to recall the scenes and pleasures of a home he had left at the age of four, in spite of a life of changing surroundings and full of heavy cares and responsibilities, time is not the yardstick of memory, so similarly with a father he knew but a few years, the impression went deeper than the brief span together would seem to warrant.

For to young Robert “Light-Horse Harry” Lee must have appeared as the perfect combination of father and hero. High-spirited and enthusiastic, full of stories of the war which his talent for words and oratory could recreate so vividly for his wide-eyed son, personal friend of the great Washington, himself a soldier decorated by his country and esteemed by his comrades, always fired by a new crusade, handsome, charming and sociable, Henry Lee, when at home, had been a kind and loving father, interested in and concerned with every detail of his children's lives. How could such a vital, inspiring and devoted father not impress his personality upon the “tabula rasa” of his likewise high-spirited, intelligent son?

After Henry Lee's departure in 1813, although the family shared a close relationship, each member genuinely fond of the others and loyal to the family per se in the clannish way amounting almost to idolatry so typical of Virginia society (a factor which must be taken into consideration in the light of later events in the life of Robert E. Lee), its unity was soon broken again. In 1816, when Robert was only nine, his brother Charles Carter left for Harvard, and when he was thirteen his other brother Smith entered the navy. His sister Ann was frequently away seeking medical aid, and Mildred, the youngest child, was four years his junior. It was because of these circumstances that the task of running the household fell on Robert, the only child available to do it.37 The family disintegration further destroyed what little there was of normal living in the fatherless, almost destitute, illness-ridden home.

Constantly worried about finances, with chronic invalidism compounding her burden, Anne Carter Lee in her thankfulness for such a dutiful, able son, seems never to have realized that saddling him with such adult responsibilities denied him a normal, carefree boyhood, a boyhood which to some extent might have compensated him for the lack of a father.

For all Robert's love and care of his mother and his cheerful acceptance of the weight of the household upon his immature shoulders he must have missed the inner security and outward pleasure his father's presence had given him, and in his deprivation, realized the need for a father to maintain a normal, happy home.

Significantly, it was when his oldest son Custis was the same age he had been when his father left, that Robert E. Lee, absent from home on an army assignment, first gave evidence in his letters of his regret at the separation because it prevented him from supervising in person the upbringing of his children. He had not known a father's guiding hand and counsel and he did not want his children to grow up with the same gap in their lives. More cause for his concern was the fact that where the rigid training of his mother had kept him in check, Mary Custis Lee was much more indulgent and easy-going with her children.38 Doting grandparents only complicated the problem, making it seem imperative to him that he do all in his power to instill self-discipline and self-control into his headstrong, self-willed children.

He knew how necessary that was for he himself had inherited the imperiousness of the Lees, and coupled with the independence natural to an alert, intelligent child, was inclined to having his own way. In great part it was Anne Carter Lee's unceasing vigil in her determination to strengthen her children in the particular areas of their father's weaknesses that made Robert E. Lee the strongly self-disciplined man of high moral principles that the world in general venerates. But to accept the stereotyped picture of him as always calm, always meek, bowing unquestioningly if courageously at whatever fate had to offer him, a man something more than human and only slightly less than divine, is to do him an injustice. To acknowledge that he was a man of strong passions and emotions, to which he gave way at times, tempted like as we are in all things, though he overcame temptation more often than most, detracts nothing from his stature. On the contrary, it makes him more human, and correctly evaluated, enhances his reputation. So wilfullness in his children most worried their father and the theme of self-control and self-mastery runs through all of his letters to them.

As he grew older Robert must have become aware of the stain on his father's military reputation due to his financial excesses; and as a natural result of his mother's training, family pride and loyalty, and youthful idealism, he leaned over backward in an effort to erase that stain by his own conduct. When his half-brother Henry Lee further disgraced that name in more financial excesses and by becoming involved in an illicit love affair which made him the center of a bitter fight in the United States Senate over his appointment as consul to Morocco, the young honor graduate of West Point must have been more determined than ever not to emulate his father and brother but to do everything possible to redress the name of Lee.39 In reaction to them Robert E. Lee became a man of almost stern morality, with a rigid exactitude in all money matters.

This heritage he passed on to his own children, for in many ways it was because of what his father was not, that Robert E. Lee became the man and father he was. With his children, however, he endeavored to teach the same lessons by a positive example, not negatively as he had had to learn them, and he coupled them with generous doses of the fatherly love and guidance he had so sorely missed in his own childhood.

Yet he followed his father's profession, Washington became his idol as he had been that of his father, when the test came he chose Virginia regardless of consequences as “Light-Horse Harry” Lee had once chosen, and all his life Robert regarded his father not as a failure but as a hero who had fallen on misfortune.

During the Civil War, while on an inspection trip along the coasts of Florida and Georgia, General Lee visited the grave of his father, something he had always wished to do. General Long, in his Memoirs, describes it simply:

Passing on we came to a dilapidated wall enclosing a neglected cemetery. The general then, in a voice of emotion, informed me that he was visiting the grave of his father. He went alone to the tomb, and after a few moments of silence plucked a flower and slowly retraced his steps, leaving the lonely grave to the guardianship of the crumbling stones and the spirit of the restless waves that perpetually beat against the neighboring shore. We returned in silence to the steamer and no reference was ever made to this act of filial devotion.40

General Lee later wrote to his wife of this visit, telling her in more detail of the plants and trees in the garden as she was interested in things horticultural, and describing the condition of General Greene's house.

Following in the footsteps of this cherished father, the long year of waiting finally ended for the anxious cadet, and in June, 1825, the son of the distinguished soldier, General “Light-Horse Harry” Lee, entered upon his preparation for the profession of arms.

Return to Robert E. Lee and His Children