From earlier posts I gathered that dynamic difficulty is adjusted per worker connection, so I could have a fast miner doing high difficulty shares and a slower miner doing diff 1 shares on the same worker account. However, all my miners are doing diff 1 shares when vardiff is set to -1. The fastest is just 2GH/s, so it might not be enough to trigger a change. I tried manually adjusting vardiff to 4, but my miners are still doing difficulty 1 work after reconnecting. Do I have to disconnect all miners simultaneously before the change takes effect, or could there be some other issue?
That is how it should work, yep.
2Gh/s isnt enough to trigger higher diff
I set 2 on mine

when you say reconnect, do you mean restarting miner or leaving miner running and reconnecting?
EDIT: or you might have just found a bug. looking into it now
