I agree with your general thoughts but you will find driver risk is at a peak when the driver is young/new/inexperienced. Predominantly male, with risk taking, aggression, poor attitude, recklessness, inexperience being key reasons. As drivers get more experience and mature personally, the risk drops significantly in the 30 to 50 age range, when it starts to rise again, to similar risk level of the young, inexperienced group. Your logic would suggest new drivers be taken off the road, I think. However, the older group have different type of crash. They've got lots of experience, so 'practice' clearly 'doesn't make perfect'. Weakening mental function , eyesight, co-ordination, concentration, confusion all start to have a great impact. Elderly drivers, therefore are not killing people with risk taking, aggression etc., but with increasing inability to cope with traffic situations. Confusion, lack of awareness are the cause of many crashes involving elderly drivers. If re-assessments were required for everyone, every ten years, perhaps, then more regularly beyond, say, 70<?> then we'd save lots of lives, as driver weaknesses (skills, attitudes etc) would be identified earlier and could be dealt with through advice or training. The cost of an accident is massive. In UK it's calculated a fatal crash costs society about $2million), so there's a massive saving to be made by reducing crashes.
I've voluntarily had re-tests at advanced level every five years or so, for many years and surely that makes me safer. Perhaps those people who argue against re-testing are good but not confident, or confident they're not good.