Radiation-induced pneumonitis and fibrosis are dose-limiting side effects of thoracic irradiation. Thoracic irradiation triggers acute and chronic environmental lung changes that are shaped by the damage response of resident cells, by the resulting reaction of the immune system, and by repair processes. Although considerable progress has been made during the last decade in defining involved effector cells and soluble mediators, the network of pathophysiological events and the cellular cross talk linking acute tissue damage to chronic inflammation and fibrosis still require further definition. Infiltration of cells from the innate and adaptive immune systems is a common response of normal tissues to ionizing radiation. Herein, lymphocytes represent a versatile and wide-ranged group of cells of the immune system that can react under specific conditions in various ways and participate in modulating the lung environment by adopting pro-inflammatory, anti-inflammatory, or even pro- or anti-fibrotic phenotypes. The present review provides an overview on published data about the role of lymphocytes in radiation-induced lung disease and related damage-associated pulmonary diseases with a focus on T lymphocytes and B lymphocytes. We also discuss the suspected dual role of specific lymphocyte subsets during the pneumonitic phase and fibrotic phase that is shaped by the environmental conditions as well as the interaction and the intercellular cross talk between cells from the innate and adaptive immune systems and (damaged) resident epithelial cells and stromal cells (e.g., endothelial cells, mesenchymal stem cells, and fibroblasts). Finally, we highlight potential therapeutic targets suited to counteract pathological lymphocyte responses to prevent or treat radiation-induced lung disease.