So right now when you look at the calendar you don't know between which hours is particular user available. You can only see booked slots.

Add "shade color" to show period of time in the day when user is available and then on top of that show booked slots.

The color of the shade can be the calendar color, just lighter.