I noticed that once you deploy headless windows servers (domain controllers as an example) you can manage everything remotely (mmc, event viewer, rsat, etc.) except Windows Firewall right out of the box.

To be able to manage the firewall remotely, you need to open additional ports on the headless instance using:

netsh advfirewall firewall set rule name="Windows Firewall Remote Management (RPC)" new enable=yes
netsh advfirewall firewall set rule name="Windows Firewall Remote Management (RPC-EPMAP)" new enable=yes