Job loss has a permanent negative effect on life-time earnings and, in general, on labor market outcomes of workers. This negative effect is larger if job loss happens in recession than the case of job loss during expansion. In this paper, using linked employer--employee data of Germany from 1975 to 2014, I show that about $55\%$ of the variation in earning loss is accounted for by negative sectoral performance. Intuitively, earning losses associated with job loss are greater during recessions, because the average displaced worker in a recession comes from an industry with poor performance relative to the average industry which makes sector-specific skills less valuable. In other words, recessions are not only periods of more job loss but also periods with more sectors of very poor performance. Hence, workers displaced in recessions are, on average, more likely to struggle with lack of demand for their skills. By building a model of sector-specific human capital, it is shown how loss of sector-specific human capital can explain huge and persistent earning and wage loss of displaced workers. In addition, it is shown that concentration of displaced workers in severely declining sectors during recessions is a potential explanation for larger earning loss in recessions than job-losses in expansions.