In 1653 Sir Robert built the well known church at Staunton Harold in Leicestershire, one of the few built during the Commonwealth. An inscription on the church reads 'He did the best of things in the worst of times and hoped them in the most calamitous'. He was imprisoned many times as a Royalist. In 1656 arms were found in his dwelling and he was taken to the Tower where he died. Charles II in exile wrote his widow an expressive letter of sympathy.For this rare signature, not in the British Library, see Stemmata Shirleiana, 1841. His son, Robert junior, was born in 1650 and was the father of the first Earl Ferrers.