[Looking for Charlie's main web site?]

CF911: Solving problem in #ColdFusion Admin getting "error accessing this page" on certain actions

Here's a real CF911 challenge (and solution): You may find that when using the CF Admin, especially in CF10 but it can happen in CF 9 or 8 depending on security hotfixes applied, when performing certain Admin operations (like making a change, or verifying datasources, or checking for server updates) you get an error:

"There was an error accessing this page. Check logs for more details."

And your operation fails. You're then prompted to "Click here to login", but even if you back up or client another link, you'll be prompted with the CF Admin login.

What gives? Why is it happening? And how can you fix things? Is CF broken? No, not in the sense that you need to reinstall or anything. The good news is that there is a quite simple solution. Well, there are several, depending on your goals.

The simple solution: delete the duplicate cfid/cftoken or jsessionid cookies that you will find your browser is sending to CF. But there is much more to this, as well as other solutions, which would be worth most readers taking a few minutes to read on here.

BTW, the same root problem can be the cause of your own application's users finding that they can't stay logged in. More on that in a moment.

What is happening?

First, let's note that the problem doesn't happen when you login to the Admin, nor even when you click around visiting different pages in the Admin. It happens when you click on features that involve a form submission, such as when you make pretty much any change, or when you click buttons like "verify all connections" in the Datasources page, or click the "check for updates" button on the Server Updates page. All these cause CF to do some processing that's different from just visiting a page. (I suspect it's an issue with CFLOGIN, but I'm just guessing.)

Second, let's point out that the problem of the CF Admin getting that error above need not be as mysterious and irreconcilable as some make it out to be. There actually is more information in the logs, if you just go looking for it, either the application.log or the coldfusion-out.log (if using Windows, or it may be the cfserver.log on *nix, or the console log if launching CF from the command line or CF Builder).

In the coldfusion-out.log, you will find that this error appears at the time of the error:

"Apr 9, 2014 22:36:55 PM Warning [catalina-exec-7] - There was an error while verifying the token. Either the session timed out or un-authenticated access is suspected."

And in the application.log, this one appears:

"Information","catalina-exec-7","04/09/14","22:36:52",,"Session rotated successfully."
"Warning","catalina-exec-7","04/09/14","22:36:55","CFADMIN","There was an error while verifying the token. Either the session timed out or un-authenticated access is suspected."

Now, both messages give a bit of a clue as to what's going on. Your session has rotated, but then there was a problem with CF verifying the "token". Any guesses what that "token" is?

Spotting the problem

Some will recognize that the "token" is (typically) the cookies used by CF for managing sessions, which may be CFID and CFTOKEN or JSessionID if using the "J2EE sessions" feature (which is settable in the CF Admin, on the "memory variables" page).

The problem here is that (for any of various reasons) CF sees duplicate cookies for the tokens it is wanting to use, and being confused, it is just giving up.

The good news is that there are several workaround, as I will show in a moment. (Sadly, the Adobe bugbase entry where someone raised this issue says that there are "no workarounds", when in fact there are several, as I will show.)

If you use any sort of browser dev tool or "proxy/sniffer tool" (now built-into all the major browsers, as I've spoken about before), you'll most likely see you have 2 CFID and 2 CFTOKEN cookies, or 2 jsessionid cookies being sent from your browser to the server.

The issue is that they are different enough, whether in the "case" (upper or lower) of the cookie name, or with different domain values, path, or other attributes, by which the browser thinks they're different enough to send both cookies (of the same name) to your server.

And CF gets confused by this.

How the duplicates got there is a subject for another day. (Someone in the bugbase suggested it happened for them because they had CF9 and 10 on their one machine. I just had it happen on a machine with only CF10, but I do happen to have Railo on the machine also, for some testing purposes, and I can confirm that it was setting the lowercase cfid and cftoken cookies, in conflict with CF's uppercase CFID and CFTOKEN.) You may also have the duplicate cookies because YOUR OWN CODE may be setting a CFID/CFTOKEN or jsessionid cookie, and you may be doing it differently from CF, thus ending up with the duplicates one way or another.

An impact of this problem beyond the CF Admin

And as I mentioned in the opening, this problem of duplicate CF session cookies can actually also be at the root of some other problems (besides your use of the CF Admin), such as your own CF web application's users finding that they can't stay logged into their applications, being forced to login again unexpectedly, etc. (And no, you do NOT need to be using CFLOGIN to suffer the problem. I was guessing above that that could be the issue with who only form submissions fail in the CF Admin. I've seen people have their CF applications cause loss of sessions in CF10, or again 9 or 8 depending on hotfixes applied, even if they do NOT use CFLOGIN, for the reasons I outline here.)

As above, you'll find often that in such cases, if you could get the users to look at their cookies (using the same browser dev tools), they would see the duplicate session token cookies, and they could remove them.

And if you are reluctant to try to get your users to confirm this, you could do it yourself by dumping/logging the value of cgi.http_cookie, which holds the value of cookies sent from the browser to the web server. (Note that I am NOT proposing you dump/log the CF "cookie" scope, which could hold cookies created in the life of the request on CF itself, which could be different.) And you could try in your own code to find and remove the duplicate cookies, but that too goes beyond the scope of this entry.

My focus here is to help you get around the error in the Admin, both to explain why it's happening and some easy workarounds you could consider.

Some workarounds

As I said at the opening, DO NOT freak out and think that you need to reinstall CF or something (as many have asserted if you look around the bugbase or indeed the web on this problem).

There are any of several easy workarounds you can use.

The goal of course is to enable you to be able to do whatever operation you had wanted to do, and to do it, you need to deal with that duplicated cookie issue. Some workarounds will be easier for folks than others, depending on the browser/tools you have to manage cookies. You can either:

  • Clear just those duplicate cookies, the pair of CFID/CFTOKEN/jsessionid cookies (again, watch the case). CF will recreate them on the next request. (Do a google search to find the way to clear cookies for your browser)
  • Or if it may be easier, clear all the cookies for the domain (if you're using localhost, clear all those for localhost), or worst case (I hate this advice) clear all cookies from your browser. That's a last resort.
  • You could also simply try visiting your CF Admin using a URL that's different from what you've been using, but which gets you to the same CF instance (such as using localhost instead of 127.0.0.1, or vice-versa, or using an IP address rather than your domain, etc.) The browser will only send cookies to a server that the server (that named domain or IP) has sent to the browser in the first place. If you've been visiting the CF Admin with one ip/domain name, and not some other that would work, you will find in visiting from that "other one" that it may not have the duplicate cookies (yet), and so you may now get into the Admin, no problem. Of course, if you HAVE visited the admin or other pages on your site using that domain/IP, you may run into the same issue in accessing the Admin
  • You could try visiting the same Admin URL in a different browser. For instance, if the failure happens when using Chrome, try Firefox, IE, Safari, or Opera, and vice-versa. The key is, if the other browser is one that you've not visited the CF Admin/site using, then it won't have the cookies (or may not have the duplicates), so won't have the same problem.
  • If you only have one browser installed, if it's Chrome or Firefox, you could try visiting the CF Admin in an "incognito window" (in Chrome) or a "private window" in Firefox (search for more on these features). These features open a new window in your browser which is supposed to make you appear rather anonymous, such as not tracking your history, not sending cookies that have been sent before. I find the tools to be uneven on that last point, but it's worth a try.
  • If you ARE using "J2EE sessions" in CF, since the jsessionid (at least that created by CF) is a "session cookie" (meaning it lives only in the memory of the browser), you may also find that if you simply close the browser and re-open it, that may clear the cookie (whether you have to close all windows of the browser will depend on the browser and how it shares such session cookies between windows. I discussed this in an article on J2EE sessions when they were added to CF6, in 2002. The characteristics of IE and Firefox (and Chrome) have not really changed in this respect since then.
  • Following on the previous point, if using J2EE sessions, you may find that simply opening a new window of your browser it may not bring in the jsessionid from the other window. Again, see the 2002 article for more on that.

Any of these should get you past the problem, at least temporarily, so that you can proceed to do whatever you wanted to in the CF Admin that was failing to work.

Long-term solutions

Again, though, the long term solution is to find and resolve whatever is creating the duplicate cookies, or put in place some code to find and remove them. That's beyond the scope of this entry, but I wanted to get the above out as I see folks often hit the problem on their CF Admin, including customers of my CF server troubleshooting services.

Could be your application

I'll also note, as you may want to explore this more, that the problem may be due to your own application's manipulation of the CF session token cookies. People for years have done this with the CFID/CFTOKEN cookies, for instance, such as to set them "again" using CFCOOKIE but with no EXPIRES attribute, which causes CF to set the cookie without any expiration value (to prevent it being stored on disk) which makes it into what the browsers consider a "session cookie" (like was described above with respect to CF's "J2EE session" cookie, jsessionid).

But in doing that, they may leave out some attribute that causes the cookie to be different than the one CF sets. Again, this is where those browser dev tools can be handy: you can see what cookies ARE being sent to your browser from CF, and maybe you will spot what's amiss, and where it's happening.

It can be difficult to solve, though, because the cookie may be set in some application (on your domain) that you're not thinking of or testing. Nothing tracks what page in your site sets a given cookie (that would be a pretty cool addition for some browser to add), so it could be anywhere in your domain.

This is also why this problem is so challenging when a user gets it (not with respect to them accessing your CF Admin, of course, but when they're having problems staying logged into your application.) You may have a hard time finding first which users, if any, have the problem. Then you may have a hard time finding what page is setting the cookie that's creating the duplicate. It can be a real pain.

You might consider switching session variable approaches

Finally, still another long-term "solution" might which you might consider is that if you were NOT using J2EE sessions, you'd find that if you DID change to using them, it could seem that the problem was "solved". Same with if you WERE using them and switched to NOT using them. (But to be clear, it's possible that it may only hide the problem temporarily.)

What's happening is that now your browser will be given a different session cookie (jsessionid for j2ee sessions, cfid/cftoken for regular CF sessions), and initially that "other" kind of cookie would not be duplicated like your first kind was.

But whatever caused the duplication of the other kind would likely lead to duplication of the new session cookie eventually. So making this switch may or not really be a good long term solution.

(I would point out, also, that switching to or from J2EE sessions is not something to do lightly. There are important implications of doing it which could affect your application and how it works for users. I discussed this in more detail in an article when the feature came out, New Possibilities for Session and Client Variable Handling in CFMX (CFDJ, August 2002), which should be as relevant today as then.)

You might want to reconsider the "session fixation protection" feature

All this problem really is rooted in a change that Adobe made (in 2011, available as a hotfix for CF 8 and 9, but included automatically in CF9.0.2 and CF10) related "session fixation". It's that feature, a protection mechanism meant to help prevent session hijacking, which really seems to make CF so sensitive to this issue of duplicate cookies. You can do a google search for that topic to find many others in the CF community who have talked about that, and some workarounds available to disable that, which involves setting a jvm argument to coldfusion.session.protectfixation=false. It's a rather drastic solution to the problem, but especially if the issue is more about your application users having problems, versus just CF admin access issues, then it may be worth considering.

And while one may want to consider removing the security hotfix (more later) which introduced the issue for CF 9 or 8, that may be overkill. And note that there's no hotfix you can "remove" in CF10 to address this. The session fixation protection is built-in from the first CF10 installation. But one could tell CF to disable the session fixation protection. Again, I think that's overkill for the problem described here (the CF Admin issue), but you may want to consider if if it's the cause of your session loss problems in your applications. Again, just see the other discussions you find googling as mentioned in the previous paragraph for more on that.

Reconsider the CF cookie settings

Finally, at the same time CF added new features that changed how it (CF itself, setting cookies) and your own code goes about creating cookies, which might increase the chances of your own code (unexpectedly) creating the duplicate cookies.

Some are changes in the CF admin Session Variables page, others are in the cfapplication tag or variables in the "this" scope of application.cfc, and some are in the CFOOKIE tag itself.

For instance, CF10 has added a new encodevalue attribute for CFCOOKIE, and its default is true, so that CF now creates the value of a cookie as a URLEncoded value, which means that if it has special characters, those get converted into a URL-safe character entity. That's fine, except that if it causes your code to create a value that's not quite the same as CF would create itself, that may start this problem of your having duplicates.

I don't want to belabor this point. This entry has gone long enough. I just wanted to share the observation in case it may help some readers take the solution still further for their own needs.

Conclusion

I hope that one of the far simpler workarounds above will help anyone who runs into this core problem, of the CF Admin seeming to be "broken". But when it comes to solving the problem with respect to your application users, it's definitely a thornier one to decide the best solution.

Whether you're one of my consulting customers, or one of my regular readers, or someone who perhaps finds this later trying to search a solution, I hope this this was helpful, both to understand the nature of the problem and its available workarounds, and to consider the larger context.

This is why my blog entries tend to be longer than most. If the problem really was "simple", it wouldn't be so hard for people to get a clear understanding of the problem, its solutions, its broader ramifications, and so on. :-)

Comments
Thank you for this great post
# Posted By Deepa | 4/10/14 8:31 AM
This may also be related to this bug: Domain attribute of cfcookie is always trimmed to .domain.tld - https://bugbase.adob...
# Posted By Henry Ho | 4/10/14 3:53 PM
Very interesting! Next time I see you I'll buy you a beer!!

I ran into something similar and am now wondering if it had anything at all to do with fonts... http://thecrumb.com/...

I've since trashed the install I first ran into this but I'm wondering if it's the version I'm using - I'm not sure if my last install (local) was updated to 9.0.2 but that must have been it - I never recalled running into this issue until recently...

Seems to be something with recent CF installers because I
# Posted By Jim Priest | 4/10/14 4:39 PM
Using 9.2 I am seeing this. I tried downgrading IE10 to IE9 per another solution online, then clearing cookies for the browser, but it had no effect. I did not access anything with the browser except for the admin page. I even tried 127.0.0.1, <ip>, and the dns name, all did the same thing.
# Posted By Mike | 4/14/14 10:27 AM
Charlie, until a few minutes ago, my day seemed to be turning into a nightmare, one where I was cascading deeper and deeper into a pit of unresolvable unknowns. Thanks for pulling me out!
# Posted By Nando | 6/19/14 9:16 AM
@Nando, really glad to hear I helped then! Welcome back to the light of day. :-)
# Posted By charlie arehart | 6/20/14 12:11 PM
Charlie, you saved my week! Thank you! This post also explains some really odd behaviour experienced by some clients over recent years that we were never able to get to the bottom of. Thank you!!! Hugely Appreciated!
# Posted By Jason | 7/1/14 1:52 AM
@Jason, great to hear and glad to help.
# Posted By charlie arehart | 7/1/14 8:55 AM
Thanks Charlie, you are the best...(new CF11 install, Chrome, OS X 10.9).
# Posted By Pierre Chaillet | 7/10/14 10:19 PM
@Pierre, thanks for the kind regards. Glad to help.
# Posted By charlie arehart | 7/10/14 10:24 PM
thanks for the tips, the cookies was the problem, I flushed the cookies, and it worked fine.

regards..
# Posted By Orlando | 8/13/14 5:53 PM
This didn't work for me at all. Tried from many different browsers and different computers, flushed the cookies. Nothing helps.
# Posted By Peter Daams | 9/5/14 10:04 PM
@Peter, I suggested a lot more than those 2 things. So when you say "this didn't work for me at all", do you mean you tried every thing in the blog entry? Including disabling the protectfixation issue? or tweaking the Admin-, app-, or code-level settings for cookies? Did you use a browser proxy tool and did you see changeing and/or duplicate session cookies?

If so, I'll repeat what I said that this can be a difficult challenge to solve, especially if it's affecting your applications.

But you don't clarify which it is: is it that (as the blog entry focuses on) you have trouble posting pages in the CF Admin? Or is it about your applications (which I go on to discuss later)?

Or is it perhaps that you can't even login at all to the Admin (which is another problem I didn't think to mention here)? In that case (or if any future reader sees this), that problem happens often when you first install FR on a Windows server and are using Internet Explorer on the server itself. It's so locked down that it can't even run the Javascript in the login page that's doing some validation. Just add your current site (whatever the IP or domain name is you are using for the Admin) to the "trusted sites" feature in IE. Then refresh the page and try logging in again.

Finally, to Peter or anyone else, I'll repeat that if time is of the essence and you don't want to wait for back and forth here in the blog post, I can provide direct, remote support. For more on my rates, approach, satisfaction guarantee, and more, see http://www.carehart....

Hope something there helps, Peter.
# Posted By charlie arehart | 9/6/14 10:57 AM
Sorry Charlie, I had the specific problem you are describing in this blog post when trying to post any changes in the admin:

"There was an error while verifying the token. Either the session timed out or un-authenticated access is suspected."

I've googled around for this question and inevitably there is an "Accepted answer" from you on almost all the forum entries about this that "Basically, it's a duplicate cookie problem for the CF session cookie(s)" with a link back to this blog entry.

Well, it's not in my case.

Tried:
* clearing cookies (there were no duplicates that I could see, but hey - I'll give anything a shot)
* accessed admin from a different domain
* visited admin url from different browser
* used incognito
* reopened browser

These were all the temporary solutions provided which you said "should" get me past the problem.

I looked at the longer term solutions, but they all seem to revolve around fixing the same duplicate cookie problem fixed by the temporary ones. And seeing as I can't see any duplicate cookies, I just don't think this is my problem. Oh, and I'm not using IE.

I don't mean to be contrary on your blog, but I just see your answer posted in so many places as an accepted truth, that I felt it important to add my own comment saying that not *everyone* who experiences this error has the duplicate cookie problem.

And here is the latest bugbase report on this: https://bugbase.adob... . Adobe doesn't seem to think this problem exists at all!
# Posted By Peter Daams | 9/6/14 9:20 PM
@Peter, I hear your frustration, and while I'm tempted to think you may not think I can help you at all, I'm not going to give up so easily.

First, no, the long-term fixes do not revolve around solely the duplicate cookie issue. They also address "rotation" issue, which is in fact what the error in the application.log points to (as noted above).

Can you confirm if you see that in your log? If so, then you might possibly want to try disabling the session fixation protection. But then I'll note that such a change would apply to all your CF apps, so it may be overkill.

Do you find that any other app users complain of any issues related to sessions? Or is this problem only in the CF admin? If so, is this CF admin in fact a production server? If not, then you may want to give the change a try just to confirm if it helps, in which case you'd know you're on the right track.

Similarly, you could try changing to using J2EE sessions (another CF-server-wide change), or revert from using them if you are using them already. If either helped (and you cleared cookies on your testes, to remove any previous remnants), it would not only offer some relief but could point to the real root cause here.

And I'll say again (as I do in the post): if you ARE using J2EE sessions, and you try to clear cookies, you do have to CLOSE the browser (all instances of it) to be sure all cookies related to j2ee sessions variables are removed: they're stored in memory, not on disk, and some browser tools don't clear them even when they assert that they do, in my experience.

All this comes down (in my experience, at least) to cookies, and in your list of things you say you did, you don't say whether you viewed the session cookie values with a tool that exposes them. It would seem very important for you to to confirm first that rotation of cookie values is indeed the problem, but that specific error you quote does seem always to be connected to an error about rotation in the application.log.

These are all things you can still investigate.

But I realize you're probably more in the "adobe has a bug and I just want it fixed" frame of mind. If so, I can help no further. Obviously I don't work for Adobe. But I am confident there is an explanation that you're not yet seeing, and it may not reflect so much a "bug" but just a change of behavior that needs to be understood. I realize that's no consolation when your admin does not work.

In fact, it made me just think of yet another possibility (which could explain why your problem seems different than others): could it be that you have the wrong version of the CF Admin code running, for the version of CF you're running? I have seen that problem plenty of times.

Look in your CF Admin mappings. Where is the CFIDE pointing to? And then if you are using an external web server (like IIS or Apache), where is THAT pointing to for its CFIDE? Or do you have CFIDE code in your web site's docroot? If this is CF10 or 11, there ought instead to be a virtual directory pointing to the CFIDE in the CF10 or 11 wwwroot. I have seen lots of "odd, unexplainable" errors happen when the version of the CF admin code does not match the version of CF that's running. Sadly, it's easy to have happen for a variety of reasons (which may be have been caused by someone else on your server if you're not the sole CF or web server admin).

You also don't mention what version of CF this is. If it's 9 or 8, did you perhaps apply any CF hotfixes recently (or in recent months), since which you've seen the problem? It was a security hotfix (or the latest CHF for 9.x) that would have added this session fixation protection. It's even possible someone made a mistake in applying a hotfix (or CHF) in CF 9 or earlier. I point out the details of how that can happen easily, at http://www.carehart....

Hope that helps. And again, if this is important for you and you only continue to struggle, I could help more directly. I don't work for Adobe, though, so I can't offer that help for free, nor offer a refund if it's proven to be a bug in CF. I realize that for both those reasons you may not be interested. Just making the offer, though I'll repeat that I DO offer a satisfaction guarantee, so that if you don't think some part of time we spend together is valuable, you won't pay for it.

Anyway, hope perhaps some of the additional info above can help, especially the recommendation to look at the cookie values. That has been the bottom line pointer to the problem. The cookies may not look like they should (domain- adn sub-domain wise), or they may have attributes that are not expected. I mentioned how there are settings that can influence those, but there are a lot of moving parts with all this, and cruft in CF that has built-up over the years, as well as changes based on Tomcat (if you may be on CF10 or 11).

If you may look for more help here (from me or others), you may want to clarify not only what CF version (and updates) you have, but what OS and web server. Just trying to help, if possible from afar like this.
# Posted By charlie arehart | 9/8/14 9:11 AM
Hi Charlie, Thanks for your response and your patience. I'm on CF10, using J2EE sessions and yes, that is the error I see in my logs when this happens. I did look at the cookies' values. And there were no duplicates there.

I have a lead to chase up which may resolve this. I had to adjust my server.xml file (I'm using CF10) to adjust so my app could handle domain cookies. The standard setdomaincookies=yes option doesn't seem to work on JSESSIONIDs, which is why I had to go down that path. I have a feeling that the change there messed up the admin area since it specifies my domain and I'm not actually accessing the admin on the same domain. I'll follow down that path and see if it resolves things.
# Posted By Peter Daams | 9/8/14 7:31 PM
This is great! It really shows me where to expand my blog. I think that sometime in the future I might try to write a book to go along with my blog, but we will see.Good post with useful tips and ideas. http://www.exams-sof... - http://www.exams-sof...
# Posted By john | 9/18/14 2:48 AM
BlogCFC was created by Raymond Camden. This blog is running version 5.005.

Managed Hosting Services provided by
Managed Dedicated Hosting