There doesn't seem to be any option to remove users from the client portal as a whole.