Wait a second, do you pick up any call you receive? IME Australia is one of the worst places with regards to spam calls (by far the worst out of all countries I lived in), if I allow calls from numbers not already in my contacts, I get an average of around 10 spam calls per week. If it's something important like my daughter's school or strata or the lawyer, they do leave voicemails.
Honestly pre-COVID I got spam calls maybe once a quarter. After giving out my number every second time I left the house, that number is more like a few times a week.
I would say I am pretty privacy conscious (when possible), so this may not be the average person's experience.
The problem with this strategy is that sooner or later, the spammers will find your "trusted" number. After all, phone numbers are easily enumerable...