We need the ability to decide if any given page on a site is "live" and accessible externally, or not.

There are many use cases where this may be necessary, but the most obvious is when you have a live site and you're creating a new page on it. Once you start to build the new page, there's going to be some amount of time to get it 100% complete, tested, and approved, and you wouldn't want that page to be accessible, crawled, etc. in the mean time.

There are also plenty of circumstances where you may want to take a page offline temporarily and turn it back on later, or make it inaccessible long term but not delete it for reference purposes.

Another way of looking at this feature or something like it is the ability to make a page "private", or "internally accessible".