My parents friends bought a house with two extra bedrooms for when their adult children and their grandchildren come to visit. Pool too for the grandkids.
They use the bedrooms and the pool for the grandkids once a year.
For the extra in up keep of the pool, additional cost of cooling that big of a house, they could of easily saved the monies and spent the money on a nice hotel with a pool.
MarketWatch had an article that mapped out which rooms were used the most in a average American home. The kitchen came in first, followed by the bathroom, then the bedrooms. The formal dinning room, family room saw little or no use at all.
I agree with you about the pantry.
IF we had to rebuild, or the farm was not available at the time we bought it (one of the Amish guys was looking at it, thought it was a good deal, but didnt have the money), we would of gotten something smaller. But at less than $1k an acre for 160acres, could not pass it up.