Other way around. only 1 copy of rank 1,2,3 ect will give a bonus. Mastery of different weapons is however required because only highest rank attained for given weapon is applied.
So maximum bonus is rank 1 with handcannon, rank 2 with whip, rank 3 with fisticuffs, ect. But all weapon at 3 would only be one instance of rank 3 bonus.