In spite having high hopes and looking forward to his renewed career in England when he returned home in 1806, Elgin faced a series of misfortunes. First, Lady Elgin deprived Elgin of his conjugal rights as she claimed to have suffered much since she had been through pregnancy and lost a child. Also, she had an affair with Robert Ferguson of Raith. After Elgin started proceedings for a divorce, he was also affected by how his public career went through a virtual termination. Although he initially had a relatively good career as an ambassador and in the field of both politics and in the army, none of these ares of his life proved to be sustainable. However, Lord Grenville still honoured him by giving him the green ribbon of the Knights of the Thistle. This chapter tells of Elgin's life and misfortunes upon returning from Athens.