You will find that you can no longer INSTALL CF updates via the CF admin, if CF is using this new Java version. And even if the CF update is run from the command line, if using this newer Java version that also will fail. In either case, there is a new JVM argument that solved the problem, as I discuss below.
This is happening in CF2023, 2021, and 2018. (And this may continue to happen with future JVM updates, until Adobe otherwise addresses the problem.)
As an update, when I first created this post on Jul 21, another problem was that you would find that you could no longer use the CF Administrator to download CF updates, if CF was runnign this new Java version. You would get an error reporting, "Failed Signature verification"--or in some cases you may see only "error failed". But within a couple of weeks, I found that the CF Admin COULD now download updates (including the August 2023 CF update) but the CF update STILL fails to install correctly, as discussed in this post, unless the workaround offered is used.
FWIW, Adobe has also updated the technotes for CF2021 update 10 and CF2023 update 4 with a text box at the top that acknowledges this issue and points to this post for more detail.
In this post, I explain a) what this is all about, then b) how you can fix the problem of INSTALLING the update using the CF Admin, I'll explain how it seems we HAVE to workaround that problem (for now). I also offer a link to a bug report I've filed. I even offer a thought on how this new JVM update may prove over time to affect MORE than just this, and even MORE than just CF (and Lucee) but many java apps. Read on for more.
TLDR: For now, to perform any CF updates (once CF is set to use this JVM updated in July), you will need run the CF update install from the command line, AND you will need to add this argument (before the -jar arg) when doing that:
Or as a workaround, you could change CF to use a JVM prior to July 2023 (to the install within the CF Admin_, or you could use such a prior jvm to run the command-line installer without need of this new arg. Complicated, yes. Solvable, though, yes.
Many people will need more detail, and they can read on, for these topics:
- Overview of the problem
- How the CF logs offer a clue
- How the JVM updates starting in July 2023 add a new zip validation feature, and a way to disable it
- Solving DOWNLOAD of CF updates, after this JVM is implemented
- Option 1 (to download): Add the new JVM arg (to disableZip64ExtraFieldValidation) to CF's startup args to allow download in CF Admin
- Option 2 (to download): Download/install the CF updates manually
- Solving INSTALL of CF updates, after this JVM is implemented
- How adding this JVM arg to CF will NOT fix the problem INSTALLING the CF update from the CF Admin<
- Option 1 (to install): Do a manual install of the CF update, adding the disableZip64ExtraFieldValidation arg at command line
- Option 2 (to install): Do a manual install of the CF update, using an (even only slightly) older JVM update
- Option 3 (to install): Change CF to use an (even only slightly) older JVM update<
- Hoping that Adobe might solve this, and a bug report to vote/comment on
- Should you consider leaving that disableZip64ExtraFieldValidation in CF, rather than not update the jvm?
- How this problem might crop up in other ways, in CF, Lucee, or other Java
And if I learn of new information on this issue, I'll update this post.
Updates:And indeed, I have now updated the post.
- Again, with regard to the August 2023 CF update, I can confirm that while having CF set to use this new Java update, the CF Admin WAS able to download that update, even WITHOUT the new JVM arg. And that's nice, but sadly the INSTALL of that update fails (if CF is set to use that new JVM update), so the rest of this post still applies--and the JVM arg needs to be used to run the CF update install from the command line, if using this new Java version
- On 7/27, I created the new TOC at the top, and made some modest wording changes. I also added the new section on considering whether to leave in that disableZip64ExtraFieldValidation rather than not go to the update.
OverviewAs some will know (and I blogged about earlier this week), there is a new Oracle Java (JVM/JDK/JRE) update, and CF shops running CF2021 or 2018 could consider moving to the new Java 11.0.20, while CF2023 shops could consider going to the new Java 17.0.8. And normally Adobe would encourage us to update to that: in fact the Adobe Security Bulletins for CF updates (3 in the last week) also indicate we should do that.
But then people started reporting (and I confirmed) the problems above, first with downloading then with installing the CF update, once CF was updated to use this new JVM. I hadn't noticed anything in the July 2023 JVM update technotes that should have led to any such problem, but when people reported it, I started digging in...
And I found the issue and a "solution". (And the problem isn't about the latest CF updates, per se. I found the problem would happen even with older CF updates.) And the rest of this post explains what I found and what you can do.
How the CF logs offer a clue
(Again, when I first wrote this post it was mid-July, when the problem was BOTH being unable to download AND install CF updates. I leave this detail for the record, though again downloads now do work as of mid-August.)
When attempting to download the CF updates (in mid July), the "Failed Signature verification" error was shown in the UI, and I also found it was written to the CF logs in the update.log Here's an example of the line for the CF2021 update 9 from that week in mid-July:
Error While Downloading File From https://cfdownload.adobe.com/pub/adobe/coldfusion/2021/updates/hotfix-005-330109.jar at D:\ColdFusion2021\bundles\updateinstallers - Failed Signature verification"
But when I looked at the coldusion-out.log, I found this line coinciding with the above:
That might seem unrelated: the downloaded CF update files are jar files, not zips. But technically, a jar IS a zip (just packaged a certain way).
Anyway, as for solving this problem, I did remember seeing something about "zip64" in the update technotes, so I went back to those.
How the JVM updates starting in July 2023 add a new zip validation feature, and a way to disable it
Sure enough, among several things discussed in the technotes for the July 2023 JVM update is this:
Improved ZIP64 Extra Field Validation (JDK-8302483 (not public))
java.util.zip.ZipFile has been updated to provide additional validation of ZIP64 extra fields when opening a ZIP file. This validation may be disabled by setting the system property jdk.util.zip.disableZip64ExtraFieldValidation to true.
So that's the problem. What's the solution? For downloading and then installing CF updates, once that new JVM is used?
Solving DOWNLOAD of CF updates, after this JVM is implemented
Update: now that the problem of DOWNLOADING CF updates from the CF Admin (after this JVM is implemented) has been resolved by Adobe, I will strike out this section. The rest of what I say below it applies. And I leave this, rather than delete it, as it provides context for some of what I saw elsewhere in this post.
Most readers can skip to the next section on Solving INSTALL of CF updates, after this JVM is implemented.
How can you get CF updates in the CF Admin, if you move to this new JVM? I offer a couple of solutions, to suit different needs/interests.
Option 1 (to download): Add the new JVM arg (to disableZip64ExtraFieldValidation) to CF's startup args to allow download in CF Admin
So given what I found in that previous section here, we can add that new system property as a JVM arg for CF's startup (which one can do in the CF Admin "Java and JVM" page or the jvm.config file, as found in the cfusion/bin or [instance]/bin folder). The actual arg to add would be:
And of course, one should backup that jvm.config file before making this change, whether changing the setting in the file or in the CF Admin. If you make any mistake, CF may not restart the next time anyone tries to do that.
Also, beware when adding it that this should be added to the end of the list of current JVM args, not as a new line. You can do it in the CF Admin and the Java & JVM page, and its "java args" field, or in that underlying jvm.config file and ITS java.args line.
After a CF restart, having this setting in place now DID allow the download of CF updates from within the Admin to proceed. (Again, this is only an issue if you update CF to use the JVM as updated in July 2023 and beyond.)
Sadly, it did NOT allow use of the CF Admin to INSTALL the update, as I will explain in a moment. (And as I explain toward the end of this post, there's an argument for us LEAVING this argument in--as there may be other CF features that could be impacted by this JVM change, and using this arg would negate that change for better or worse.)
Option 2 (to download): download/install the CF updates manually
I should note as well that one does not HAVE to use the CF Admin to download CF updates.
The CF update technotes always offer a section on manually obtaining and installing the update, which is needed especially when folks are running their CF servers on machines that have no internet access. So sure, you could just download the update via that link offered in the technote.
BTW, note that sometimes Adobe offers a link to a jar (the same one that the CF Admin downloads), and sometimes they offer a link to a zip. Again, see the discussion in the CF update technote for how to USE that downloaded file (and I will address it in a moment here.)
Solving INSTALL of CF updates, after this JVM is implemented
With the problem of DOWNLOADING CF updates in the CF Admin resolved, the next challenge is INSTALLING CF updates--whether via the CF Admin OR via the command line, which will ALSO fail to work if this new JVM update is used. Let me explain what I found, and again different solutions, to suit different people.
How INSTALLING CF updates from the CF Admin will fail
Once people changed CF to use this new JVM (from July), they found that in using the CF Admin to perform CF updates, they would fail. On clicking that install button for the update, they'd get the popup message that it's proceeding....but it never does do the update. There's no folder created for that update in the hf-updates folder of the CF instance, and therefore no logging about the update.
And for those who know that this Install button kicks off a java process in the background (running as the user that CF is configured to run as), you will NOT see such a java process start. It's like "nothing happens".
Here again, in digging through all the CF logs, I found that there IS a log entry reflecting this attempted install. In the coldfusion-out.log and application.log files, we will see a line like the following coinciding with this attempt to run the update installer:
Again, I see this with CF2023 or CF2018, as well as with CF2021, if they have been updated to use the JVM version from July 2023.
Sadly, there was no further info than that one line, which didn't help much.
(BTW, this in ability to install updates from the CF Admin happens not ONLY with the NEW CF updates but also with any PREVIOUS CF updates you may have downloaded but not yet applied.)
Adding the new jvm arg to CF's startup args has no effect on this install failure
Before I go further, let me note here that adding the new JVM arg (-Djdk.util.zip.disableZip64ExtraFieldValidation=true) to CF will NOT help with the failing CF update install from within the CF Admin.
When the CF Admin "install" button kicks off the java process to do the update for us, it's not using the JVM args within CF. And it's NOT (yet) itself using this particular JVM arg. I hope that may change. See more about the tracker ticket I created, below.
Finally, while it was tempting to consider other options offered the JVM update technotes (like jdk.jar.maxSignatureFileSize and jdk.nio.zipfs.allowDotZipEntry), I confirmed that neither helped.
So what are people to do, to get these CF updates installed, if they do update their JVM (as the Adobe APSB suggests we should)? Fortunately, there are at least a couple of options....
Option 1 (to install): Do a manual install of the CF update, adding the disableZip64ExtraFieldValidation arg if using the new JVM update
I have confirmed that in doing a command line install of the CF update, the update WILL work--if I add that new arg. Here would be an example of doing that, for CF2021 update 9 (Windows users should be sure to use "run as admin" in opening the command line). This should be regarded as being a single line:
Note that using "java" as a command won't run unless you have implemented Java as a JRE, or have first done a "change directory" into the JVM's bin folder (or have otherwise put that folder on the "path" for your OS). This post is not the right place to elaborate further on that. (I can assist directly via my remote consulting services.)
Of course, you could (and may need to) be more specific about where to find both the Java and the CF update jar file, such as this example for CF2021 on Windows, assuming the 11.0.20 JVM update was installed by the JDK installer, and assuming the CF9 update was downloaded via the CF Admin (as discussed above), which puts it in CF's bundles/updateinstallers folder. (Again this should be regarded as being a single line:
(If you may never have yet run a manual CF update using the jar this way, the UI or CLI presents a series of a couple of screens walking one through the CF update process.)
I will also note that I tried the above command line against even the previous CF update jar files: they ALL reported this error, so the problem (that makes this new JVM balk at the CF update jar files) is NOT something NEW with the recent CF updates.
Option 2 (to install): Do a manual install of the CF update, using an (even only slightly) older JVM update
Recall that this entire problem only arises once you install a new JVM update after July 2023. If somehow you're still running an older JVM version, you could use THAT instead (in the steps from the previous section).
Finally, you could even run the commandline install using the java that's implemented within the original JRE that CF comes with, which you'll find in the /coldfusion2021/jre/bin folder (if using cf2021, I mean). Use the CF2023 jre/bin folder if using CF2023, of course.
(Someone else may even want to point out that, "the Adobe update technotes or APSB even TELL you that when doing a manual update you SHOULD use the JVM that CF came with". While that's so, there are mixed messages about that in different places. The main point being stressed in that sort of discussions is that you should NOT use a major Java version that is not SUPPORTED by your CF version, such as trying to use Java 17 with CF2021 or 2018, or even Java 11 with CF2023.)
Option 3 (to install): change CF to use an (even only slightly) older JVM update
Another alternative (to get the CF Admin install of a CF update to work) would be to just change CF back to using a JVM that is NOT the one updated in July 2023, and that is true. As above, you could point CF back to the jvm in the /coldfusion2021/jre/bin folder (if using cf2021, I mean).
Someone may also want to point out that one could change CF back to using an older JVM JUST for doing this CF update, which may seem a bit tedious but is an option.
(Before leaving this subject of INSTALLING CF updates, some good news is that I did test running the CF Admin update "uninstall" feature while CF was running with this updated JVM. Fortunately, that uninstall process worked. It, too, kicks off a java process, running against a jar--but it's the uninstaller.jar that's buried within the hf-updates folder for whatever CF update you are uninstalling. Somehow, that jar doesn't have the problem that these installer jars do. That may be a useful clue for Adobe.)
Hoping that Adobe might solve this, and a bug report to vote/comment on
There's still one more "option" we can hope for: that Adobe will do something to resolve this. I'm sharing this to get the explanation and workarounds out, but really I hope Adobe might somehow just make the problem go away. I can see 2 possibilities:
- Perhaps they can fix whatever it is about the update jars (old and new) that keeps them from working with this newer JVM, unless that new arg is used
- As for the CF admin update installer, perhaps they could modify the java process it launches to add this JVM arg
- Finally, perhaps they need to do some serious vetting of all of CF to make sure everything works still with this new protection. Maybe they can somehow confirm what things WOULD trip over this Zip64ExtraFieldValidation, and whether we should leave in the setting (see the next section here)
As I said at the opening, I've filed a bug report. If you want to add a vote and/or follow along there, it's at CF-4218719.
Should you consider leaving that disableZip64ExtraFieldValidation in CF, rather than not update the jvm?
Update: I've added this section since the original post
So clearly, there's at LEAST the problem of the CF update installer failing to work with this new JVM, unless that new disableZip64ExtraFieldValidation JVM arg is used. Clearly this new "protection" can have an impact on some java processing, and who knows what other CF features might suffer for this new "protection".
Should we perhaps be considering leaving that disableZip64ExtraFieldValidation in the startup args for CF, for benefits beyond this one known problem?
Some may even naturally ask: "perhaps we should just NOT MOVE to this latest JVM update". That would be a "throwing the baby out with the bathwater" situation: there are OTHER protections added by this JVM update. Worse, what about when the NEXT JVM update comes out (in October: they're quarterly). Unless Oracle changes direction, this changed behavior/protection will still be there.
So another option is to consider leaving that disableZip64ExtraFieldValidation JVM arg in place in CF's startup args (or for other Java apps). At least that's better than NOT implementing the latest JVM update. (And once it's in place, it would remain in place as you may update the JVM, since it's a startup arg rather than a change in the java.security file, like some other recent "protections".)
Of course, a reasonable counterpoint would be, "Oracle added that new protection for a reason, so disabling it seems unwise". That's a fair point also.
My lament again is that the Oracle technote for the update again offers nothing more for us to know about what the protection really is for, so we can't know. And since the bug report for it is private, we can't see it, so I can't say WHAT to do. I'm just presenting this as an option to consider, for now.
How this problem might crop up in other ways, in CF, Lucee, or other Java
Before wrapping up, and expanding on that previous section, it makes me wonder: might similar problems crop up in even other Java software beyond ColdFusion....and I don't just mean Lucee. I mean all KINDS of Java-based software. Again, it's unclear what it is that makes things fail this "ExtraFieldValidation", and since the bug fix number indicated in the Oracle technote is indicated to be "not public", we can't find anything more about it. And trust me, I've looked for it, in various ways.
I will say finally that we may well hear more (outside of the CF community) in time: often the news spreads faster (and new installers are made available sooner) with the Oracle JVM than with OpenJDK variants of the JVM. As more folks move to using EITHER the Oracle JVM OR such openjdk variants implementing this new JVM, we may hear more people reporting and talking about problems like this one. (BTW, no, Adobe does not formally support using openJDK JVMs with CF. They formally support only the Oracle JVM/JDK.)
Update: And on that last point, as I expected, there has indeed a lot of other cases of people hitting or raising issues with this recent JVM change and considering that jvm arg as a solution:
- Install Anywhere issue
- a liferay issue
- apktool issue
- someone also complaining (with detail) about this change
- a chromium issue (one of a few pointed to by that last post)
- and more
So yeah, as I suspected, this issue is having wide-ranging impact beyond CF--and certainly could affect CF (and Lucee) BEYOND just this inability to install (and at first even download) CF updates. So I'm leaning toward "keep the arg in" until we hear otherwise. But that's not my formal recommendation of what YOU should do. I'm just a messenger. I'm not even committed to ensuring that I keep this updated with EVERY discovery that others may come out with as better solutions or info about this mess. But I certainly welcome comments and will update the post accordingly.
Hope all that's helpful. Looking forward to news as it may evolve, and I'll update this post with any major developments. Again, if you face challenges with all this and want direct assistance, I can offer that via my remote consulting services. But I'm happy to try to help everyone via comments here, of course.
For more content like this from Charlie Arehart:
Need more help with problems?
- Signup to get his blog posts by email:
- Follow his blog RSS feed
- View the rest of his blog posts
- View his blog posts on the Adobe CF portal
- If you may prefer direct help, rather than digging around here/elsewhere or via comments, he can help via his online consulting services
- See that page for more on how he can help a) over the web, safely and securely, b) usually very quickly, c) teaching you along the way, and d) with satisfaction guaranteed