Its working a little bit different to what you think how its working.
If you are wearing sets, you'll get set boni for every set piece that you are wearing.
Wearing one jester item gives you -25% xp bonus and -25% loot bonus, wearing two items of the set gives you the second level of the set bonus which is -50% in xp and loot bonus and some other stuff.
Or, just take one of those of the Jester set off. It's less -xp, and still usable, what level are you? People can help you more if they know your level.
Comments on Profile Post by XxMilk_the_CowxX