Are configuration changes retroactive?

In general, no. Most configuration changes (such as modifying a page name/page group/traffic segment/etc. rule) will update btt.js and new users (without a cached copy of btt.js) will have to receive the updated tag in order for configuration changes to propagate.

Configuration changes that modify “Display Names” in the portal, rather than the data values, will take place immediately and be available everywhere in the portal. This applies to Display Names for Custom Numbers, Custom Variables, and Custom Categories, as well as modifications made in the Custom Portal Terms page.