Should I do a Push to Production for any content changes?

Yes, any content and settings change require you to do a Push to Production.

When you save a change, it gets saved on the Whatfix server. But for your end users to see it, the change needs to be pushed across to the CDN they are using.

All changes like modifying Whatfix content, Activating/Deactivating Widgets, setting different Visibility rules or changing Global configurations, etc. may appear when you use the Preview mode or See Live. But for the changes to reflect on your customers' application you must do a Push to Production.