Photoreceptors and their supporting retinal pigment epithelium constitute the key functional parts of the retina. Here, a study was undertaken to show how aging and lifestyle factors affect the photoreceptor layer and the retinal pigment epithelium and Bruch's membrane complex (RPE-BM) in vivo in a healthy Danish population using spectral-domain optical coherence tomography. This was a cross-sectional study of healthy humans aged ≥50 years. All participants were interviewed for medical history and lifestyle factors. Maculae of all participants were scanned using spectral-domain optical coherence tomography. The thickness of the photoreceptor layer and the RPE-BM was measured on one eye from each participant. In 150 eyes of 150 participants, it was found that aging was associated with a decrease in the thickness of the photoreceptor layer (-0.143 μm/year, P = 0.031) and an increase in the thickness of the RPE-BM layer (0.100 μm/year, P = 0.029) at the foveal minimum. Regarding lifestyle factors, alcohol intake or BMI were not associated with any significant trend, but physical inactivity and smoking had effects on the photoreceptor layer (decreased thickness) and the RPE-BM layer (increased thickness) indicating an accelerated aging process of the macula. Taken together, aging affects photoreceptors and the RPE-BM, and these aging trends are accelerated in smokers and the physically inactive.