[Looking for Charlie's main web site?]

How can I keep CF Admin settings in sync between multiple servers or instances?

Note: This blog post is from 2020. Some content may be outdated--though not necessarily. Same with links and subsequent comments from myself or others. Corrections are welcome, in the comments. And I may revise the content as necessary.
Updated, Feb 2022

This question, "How can I keep CF Admin settings in sync between multiple servers or instances?" was asked today on the Adobe CF forums. The person had CF instances on multiple servers and lamented having to login to each CF admin to make changes that would apply equally to all instances, in particular creating or changing datasources.

They wondered if in fact there was a feature in the CF Admin to "cluster" datasource definitions, like there is (since CF2016 Enterprise) the feature to "cluster" scheduled tasks.

While I explained that there was not such a "feature" to sync CF admin settings generally across instances, I added that there WERE at least a couple of options one could consider using to achieve the goal. (And as an update after I wrote this in Apr 2020, note that CF2021 came out a few months later offering still one more.) My answer elaborating things (as is my wont) was long enough that I should have probably created a blog post instead. After submitting it, I decided to do just that, here (and I have tweaked here what I said, with some more elaboration and links).

Short answer: there are different tools that might be considered to help with this task (especially automating it), and a couple more that some may consider:

  • the CFConfig tool within CommandBox, a powerful and automatable command line tool which uses json files and could allow you to sync things, even across CF versions and between CF and Lucee
  • the CF2021 cfsetup tool, a new command line tool which also uses json files and could allow you to sync things, for CF2021 and even earlier version
  • the CF Admin API (minimalist and manual), which you'd have to code to sync settings or set them in a synced manner
  • the CF CAR file feature deserves a mention also, though its use can't be automated (it seems)
  • I would advise AGAINST the seeming "shortcut" of copying neo xml files, especially between different servers or CF versions (same with using symlinks, etc.)

For more on all this, read on.

BTW, about the ability the CF Admin has (in Enterprise) to "cluster scheduled tasks", it may be worth noting that was something INHERITED by Adobe (not CREATED by them). As some may know, it's the open source Quartz package which provides the sched task mechanism within CF, since CF10. And it's Quartz which offered the feature to cluster tasks, based on their definition being stored (and accessed by CF) in a db that's shared between the instances. That's nice that Adobe did enable it, but to be clear there's no other feature in the CF admin that works like that (to sync settings generally).

We can hope that perhaps in the future, Adobe may consider adding such a capability. Until then, how might you try to synchronize cf admin settings? Let's look at them in the order I offered above, which is generally in the order of what's ) most able to be automated and would work for more CF versions than other solutions.

The Ortus-provided command-line tool: CFConfig within CommandBox

First up, note that you could leverage the cfconfig tool offered as an optional package in the free, open source Ortus CommandBox tool. Some of you may stop right here and say, "I don't use Commandbox to manage my CF instances".

Well let me stop YOU right there and say that cfconfig can be used even with servers that are NOT started/controlled by CommandBox. All you need to do (after installing commandbox and cfconfig) is to point the cfconfig tool's FROM (or TO) attribute to name the folder within your CF instance containing the lib folder and its neo xml files (more on those files in a moment).

Even better, and specifically for the notion of "syncing" settings across instances, CFConfig offers options to export and import settings (as json), and even compare settings among instances. And there are of course features to set and get settings, as well, and more. CFConfig (like Commandbox) can be used with Lucee as well as CF (and you can even use it to compare/export/import settings BETWEEN CF and Lucee!)

For more on the tool, including examples, see this great starting page, as well as the CFConfig manual.

While to some CF/Lucee devs these are two vital tools they use daily (especially for local development), they are still unknown or unused by many. And like the Admin API below, I'ave long planned to do posts with more on leveraging cfconfig. But this question on the forums today at least prompted this post, to get things started.

A new option from Adobe, since CF2021: cfsetup command-line tool

Again, I'm adding this section since posting this originally in Apr 2020. When CF2021 came out (in Nov 2020), one of its many new features is something of a competitor to the CFconfig tool above. Called cfsetup, it's offered in the config/cfsetup folder, such as C:\ColdFusion2021\config\cfsetup, as either a bat file for Windows or an sh file for *nix deployments. It can be run as an interactive CLI or using arguments passed to the tool (the latter being better suited to automation).

Like cfconfig, it provides many features including the ability to import and export (via json) as well as list CF Admin settings, whether individually, by category, or in total. There is substantial documentation on the tool (38 pages, if printed), including examples and reference info.

There are a couple of important things I don't find documented there: one is that I find the tool can be used with ANY version of CF. I've tested it processing CF Admin settings for as far back as CF10!: but you have to be able to point to that CF instance from the location where CF2021 is installed. (You can't just copy the folder holding the tool to a machine with an earlier instance. It will fail to run.) The other thing not documented is that there's a config.log file which tracks all commands run using the tool. Curiously, it's stored not in the normal CF logs folder, but in that same cfsetup folder.)

As a quick overview, note that you can see all settings (or all in a category using its "show" command, or view individual values using its "get" command, or export/import to json using the "export" and "import" commands, to name just a few. And with each, you can either name the folder holding the admin settings (the neo.xml files), or you can define an "alias" that points to one. You can even switch among aliases (using its "select" and "delesect" commands). Here's a quick example of listing the datasources defined in your CF instance (the default instance name is "cfusion", but the tool requires you to must name it):

cfsetup show datasource cfusion

You can also point to the full path to an instance, as in:

cfsetup show datasource C:\ColdFusion2021\cfusion

And as I noted above, I find this tool can be used even with older CF versions, so you can also point to the full path of an earlier version's instance, as in:

cfsetup show datasource C:\ColdFusion2016\cfusion

And to see the details of any one datasource, I could view the details of one of the sample datasources in my CF2021 instance:

cfsetup show datasource cfartgallery C:\ColdFusion2021\cfusion

You can then switch from the "show" to the "get" command, to view any particular setting for such a given datasource. Since datasources have the option of a "validation query" (under the "advanced settings" for a dsn), you can view if any is set for a given DSN, using:

cfsetup get datasource cfartgallery validationQuery C:\ColdFusion2021\cfusion

To find what the names are of the settings you can "get" this way, use the "help" command and that category, as in:

cfsetup help datasource

Notice that although the show command may render settings in a nested way (suggesting subcategories, as it were), each setting in the cfsetup tool has just a single name to use. More specifically, the show datasource will indicate a "urlmap" section with a nested "connectionprops" having the DSN "port" as "connectionprops-port". To see that value, you simply use that alone as th property to "get" (get datsource cfartgallery connectionprops-port cfusion)

Of course, "datasource" is just one of many categories of settings. To see them all, use "show category".

For instance, you can get info such as would be on the CF Admin "java and jvm" page, with the "jvm" category. And let me note in this example how you can use the "get" to get multiple settings at once (in this case the javahome and maxheapsize values):

cfsetup get jvm javahome maxheapsize cfusion

(FWIW, I find that we cannot use multiple items on the SHOW command, such as if we wanted to see all the settings for two named datasources at once.)

And besides doing all this sort of "viewing" of info with "show" and "get", you can also ADD/DELETE/MODIFY settings, whether with the cfsetup "add", "delete", and "set" commands. Again, see the docs for more, including examples.

Finally, many commands offer an "all" argument, to get all settings in all categories. As such, to export ALL the CF Admin settings to a json file, use simply:

cfsetup export all cf2021settings_exported.json cfusion

And to export all the CF2016 admin settings, assuming you have CF installed at C:\ColdFusion2021\cfusion, use this:

cfsetup export all cf2016settings_exported.json C:\ColdFusion2021\cfusion

As the tool will report, you should secure this file, whether setting a passphrase for its use or just protectingit (and deleting it once used, to be safe). The issue is that the file has what is needed not only to import the settings into any other CF instance, but also what's needed to decrypt any passwords (such as datasource passwords, mail server passwords) that might have been stored in various CF admin settings.

You can also use these exported settings to automate deployment of such Admin settings into the Adobe CF Docker images, at least the CF2021 images, using their available environment variables importCFSettings and importCFSettings.

Again, there are more commands (like export, which is better suited to the need prompting this blog post), and an interactive CLI instead. I recommend CF users check out the tool and its substantial docs.

As is often the case, the cfsetup tool doesn't do quite ALL that the predecessor Ortus cfconfig alternative offers, but for those on CF2021 or above--and who may prefer or even be required to use only built-in tools, it's good to see that they now have a built-in alternative to consider.

Another Adobe-provided solution: CF Admin API

While not providing a built-in "sync" capability, as for how one could add or change some CF Admin setting (in an automated way), the Adobe-provided cf "admin API" could be considered. It was introduced in CF7 but has generally failed to be noticed by most. It's a set of CFCs provided in CF, within the CFIDE folder in the CF wwwroot. Nearly anything one can do in the Admin can be done by calling a certain method (a certain way) in one of the various CFCs, each devoted to a different part of the CF admin.

One could use it to automate a process of creating (and changing) settings, via CFML, across instances, such as if one wanted to change the port, server name, or password for a datasource, as an example.

Sadly, the Admin API has been poorly documented over the years. I think the original developers presumed people would use the feature in CF that allows anyone to browse any CFC, but many never knew of even that.

There was an effort started by Adobe just in recent years to better document the Admin API for us, as I noted in a post in early 2018. Sadly, the effort is still only about half done, two years later. (They ought to open-source that effort! Or someone else may want to take that on.) And if you see the "related blog entries" at the bottom of that post, I have other posts I've done on using the Admin API for various purposes.

If you just google that phrase, CF adminapi, you'll find other discussions about it from over the years, including examples. (I will update this post later with some more specific recommended resources, and I've long wanted to do another post with some more thoughts on leveraging the Admin API.)

Finally, do beware just blindly following examples shown (even from Adobe) for creating a dsn. Make sure you are setting the RIGHT settings for your own dsn needs. Sometimes their examples show settings that would NOT be what you would get in a CF DSN left to its defaults shown in the Admin DSN page.

Another Adobe-provided solution: CAR files

After I first wrote this post, Mark Gregory commented below that the CF CAR (ColdFusion Archive) mechanism could be used. I hadn't thought of it because I was thinking more of "syncing" settings and in a programmatic way, especially if wanting to "make a change" in some setting across multiple servers. But it's a fair point that I should have at least mentioned this option. Thanks, Mark.

It's a feature (available in CF Standard since CF11, but Enterprise-only before that) which allows one to use a UI (within the CF Admin, "packaging and deployment" section) to identify any of the many settings in the current CF Admin, to save those into an export file (a .car file). Then you can use the same Admin page to IMPORT those settings into a CF server (even a different version). (I'm not aware of there being an automated way to do the export, but I'd not be surprised that there would be an admin api call to do an import.)

FWIW, I've written more about it here, as well as a post on the Adobe CF Portal, where I also point to other resources introducing the feature in more detail.

Other alternatives, that I would advise against (or with great caution)

Finally, some may know that CF admin settings are nearly all saved in a set of neo*.xml files, stored in the lib folder under your CF instance. As such, they may wonder if they can just "manually" sync the Admin settings of CF instances, using either of two means, which I would advocate against.

Don't just copy the neo xml files

The most obvious thought would be, "why can't I just copy the neo xml files between the servers?, or perhaps at least just the neo-datasource.xml where DSN's are stored?"

Copying those neo xml files between servers is not wise, for various reasons.

First, even between two otherwise identical CF instances (same version, same updates, etc.), there are aspects of the files that may NOT be suited to be copied from one instance to another. For instance, I am pretty sure the password stored for a DSN is encrypted using info specific to the instance (and based on info in files other than those neo xml files. Indeed, consider that the neo-datasource.xml is also related directly to the neo-drivers.xml, so they should be regarded as a pair.)

But second, as for instances running DIFFERENT versions of CF, I would STRONGLY recommend against copying the neo xml files between those, as the layout (elements, attributes, values) of the xml files can change. I realize some "have done it" and had "no problem", but I can tell you I have helped people overcome problems that were due to this practice.

Despite these warning, some folks have indeed (out of need) dug in and confirmed that it WAS ok for them to copy some neo xml file, to make it easier to "keep them in sync". Just be very careful.

Don't try to force CF to "share" the setting location between instances

This is a variation on the above. Some admins may try either of a couple possible ways to to somehow point multiple CF instances at a single location for the neo xml files, again whether for all the settings or perhaps the DSNs only. Again, I would strongly advocate against these ideas.

First, more savvy admins will know that both Linux and Windows (and MacOS) offer a way to create a symlink (or junction or alias) for any folder, so that while an app may look for files in a folder named x, there will be no such "real" folder and the OS will instead find its files in some folder y.

And this has been used to "fake out" some app that hard-coded a path, so that instead its files were found elsewhere. While that's a great feature in many cases, it's unwise for the concerns I raised above (about copying neo xml files between two otherwise identical servers). Still another concern would be possible race conditions if somehow changes were made to two instances at once (though I realize one could implement protections against that).

And note that there's no provision in CF for it to auto-reload changes made to the neo xml files, so any change would require that CF be restarted for the change to be picked up by the instance pointing to it. (Some may notice that in the CF Admin "settings" page there's a setting called, "Check Configuration Files for Changes Every (Seconds)", but see the description there which indicates that's a feature unique to Websphere deployments, for some reason.)

Second, some may wonder if there's perhaps a way to configure CF to name/change the folder where it should look for the neo xml files. I am not aware of any such setting, even in an underlying XML file or something. And I'd argue that's for our own good, because if Adobe DID offer that, some would to try to point CF at a folder intending precisely TO share it among more than one CF instance, which would come with all the same risks discussed above.

Conclusion

Anyway, there are several ideas. I think the best choices for this need to sync (or implicitly "cluster") CF admin settings would be the first two, the Admin API or cfconfig. But I share the rest for consideration (and mostly recommending against them, for nearly all folks).

Let me know what you think, and if you may try it, how it goes for you.

For more content like this from Charlie Arehart: Need more help with problems?
  • If you may prefer direct help, rather than digging around here/elsewhere or via comments, he can help via his online consulting services
  • See that page for more on how he can help a) over the web, safely and securely, b) usually very quickly, c) teaching you along the way, and d) with satisfaction guaranteed
Comments
I believe one could also create a .car file and apply it, in theory, but I am not certain it would get 100% of the settings one might be applying manually.
Would get tedious with more than a handful of boxes to update though.
# Posted By Mark Gregory | 4/7/20 12:47 PM
Oh, of course. Very good point, Mark. I certainly should have mentioned that. I was thinking more of syncing than migrating, but it's close enough.

I will tweak the post, thanking you for the suggestion.
Copyright ©2024 Charlie Arehart
Carehart Logo
BlogCFC was created by Raymond Camden. This blog is running version 5.005.
(Want to validate the html in this page?)

Managed Hosting Services provided by
Managed Dedicated Hosting