I agree, this is incredibly frustrating, as it makes bossing, a chore in itself, take way longer than it should. It's worse when the random stays in the party and expects a free carry.
If you are the leader, you can get rid of that unwanted guest. The random becoming the new party leader is unacceptable.
The current function of the Joker items (scarlet items and lvl 140 hats dropped from Chaos Root Abyss) is to substitute for a vacant piece of a set with the highest set ID (when you have 2 sets which are both the same level to each other) The idea of automatically subbing in for a piece can be very annoying sometimes.
Wearing a Chaos Vellum Helm, a CRA top and bottom as well as the fafnir weapon will count as a full Root Abyss Set . The Chaos Vellum Helm will automatically fill in for the missing Root Abyss hat. However, adding a item such as an Absolab shoulder will alter the combination. The Chaos Vellum Helm will then substitute itself into the Absolab Set. The player will be wearing three quarters of the CRA set (Pants, Shirt and Weapon but no Hat) and two pieces of the Absolab Set (Absolab Shoulder + Chaos Vellum Helm substituting for Absolab Hat.)
It would be better if the Scarlet/Joker items allowed the player to choose which items they want to sub in for instead of SET ID being the priority setting. It could lead to better endgame gear combinations.