Removing somebody from the userlist also removes them from the user group

I don't know if this has been mentioned elsewhere or if it's by design. I have a separate user group for people who are permanently on my userlist. Others I temporarily add to my userlist. It seems like when you remove somebody from your userlist it also removes them from your user group(s).

It would be nice to be able to add somebody who hasn't logged in for more than 30 days. If you remove somebody who is no longer active, it's not possible to re-add them since it says they no longer exist.

Is this possible? it will be much faster to remove multiple users than one at a time. thanks, Colin