I recently had this question presented to me by a client. The short answer is No. The AC system is sized for a certain square footage, and reducing that can lead to efficiency issues.
The long answer is complicated. Since this customer had three rooms shut off both from the vents and the doors, and the master bedroom door was constantly shut, the return air was only coming from the hall and the hall bath. There wasn’t a lot of air circulation. What I noticed at the compressor was that the head pressure was excessively high. In layman terms, the compressor was working harder than its design. When the compressor is working hard, it is not very efficient. The electric bill will be higher than normal. It also shortens the life of the A/C system. You might see a locked compressor or leaks build up in the system down the road. Letting the system breathe as designed is the most efficient way to run your A/C.
The other thing to keep in mind is that you are allowing heat to penetrate the insulated space of the house creating a heat source inside the shut off rooms. The interior walls are not well insulated and the heat from one room will penetrate the other rooms easier. Imagine having an oven in one of your rooms. Yikes. But that is exactly what you are doing. Having a heat source inside the insulation is going to make cooling the other rooms much more difficult.
If each room has a return (newer construction) then shutting the doors will not have a significant impact on the air circulation.
It is best to ventilate every room and leave the doors open if there is only one central return vent.
Bottom Line:
DON’T CLOSE THOSE VENTS.