Restricting data visibility to assigned data is pretty limited. We need to be able to give users access to see specific smart lists where there may be unassigned contacts or contacts assigned to other users.