I always thought LaFarga looks like he's in his 40s somewhere so I don't really see them all as looking young (though apparently that's just me! also as I get closer to 40 myself I'm starting to think maybe that's not so old... maybe...) also when Clef's talking about the Cephiran's trying to save Cephiro but being unable to in the first volume of the manga, one of the Random Background People is what looks like an old dude with long beard, so I don't think they all stay or choose to stay young, even those with strong will-power (as that dude is togged up like a mage)? But I do think a lot of the older people may have gone to fight Zagato and died from it. Also that the cast focuses on the few people with the energy and will and circumstances to be involved with the magic knights somehow, and that that makes them more likely to feel and appear younger?
Anyway! I tend to think that there is a 'natural' age for each individual to look, which is based partially on age but also partially on how they feel they are, age-maturity wise, and because there's not that much in the way of milestones physically-emotionally between young-adult when someone is say the equivalent of earth 25, and then like menopause-and-metabolism-slowing-down at 50ish, people don't tend to appear between those ages unless they're transitioning (I have LaFarga as getting older in response to the responsibility of his job as he's now in charge of a bunch of people and so his physical appearance reflects it even though he's not consciously doing that with will, because that's how he feels about his maturity/age?), and they're more likely to look like a young adult until they change to look like they're 50-60, and as they get older it takes more force of will and/or willpower and/or belief in This Is What I Look Like to counter it, so people tend to leap to 60ish and then slowly age from there, but also! there are probably people who want to look 60ish because they believe they are mature and wise and stuff!
ANYWAY I think that when they're dying most anyone who is countering their 'natural' body image would loose control of it, so their appearance may well change in the weeks/months up to death, and that's a thing people notice and so know something is happening?