[Looking for Charlie's main web site?]

CF911: Is your ColdFusion service taking too long to shut down? Find out why

Note: This blog post is from 2011. Some content may be outdated--though not necessarily. Same with links and subsequent comments from myself or others. Corrections are welcome, in the comments. And I may revise the content as necessary.
Have you ever found (or heard others complain) that sometimes ColdFusion doesn't stop (or it takes a long time to do so)? It can be especially challenging when you're running CF on Windows as a Service, for reasons I'll explain here and in a follow-up entry.

First, this one will help you perhaps find why it's so slow. You may just have been looking in the wrong place for that diagnostic information. Second, the next entry will offer tips to handle on better handling the situation (that the Windows service is slow to stop, and timeouts in the Windows Services panel itself) which you may need to consider until you do solve the root cause problem (or in case it happens again for other reasons).

(Note that most of this info will apply as well if your problem is that CF's taking a long time to start, also.)

As some of you know, I do CF server troubleshooting as an independent consultant. In helping several people a week, this is a fairly common complaint. This also came up on a mailing list today, so I decided to offer these thoughts here.

Check the logs--but not the "CF" logs

Generally, the explanation for why CF is taking so long to stop (and perhaps also, to start) will be logged (if you are patient enough to wait for it to really go down. More on that in a moment.)

Of course, if you run CF from the command line, then the error would be shown in the command line console. But nearly everyone running CF on Windows will instead be running it as a service. In that case, things get more challenging.

The challenge is that the information about startup/showdown of CF is NOT detailed in the "CF" logs (those shown in the CF Admin, or found in the [cf]\logs directory or buried deep within an instance in Multiserver mode). Those logs (like application.log and exception.log) are often not helpful for system-level problems.

Instead, you need to look in the equivalent of the "console" logs, which in CF 9 or before are found in [cf]\runtime\logs\*out*.log or [jrun]\logs\*out*.log. (In Zeus (the next version, not yet released, which has been publicly shown as running on Tomcat), the will be in the "logs" directory, but as a new file: [zeus]\logs\start.log.)

I refer to these as CF's "console logs" (though as you can see that's not not the formal name) because again this is where CF writes messages that would have been written to the console if you'd started CF from the command line.

(Some will note that there is also info written to the Windows Event logs, but nearly always it just says to see the CF logs. Sadly, it doesn't clarify which logs, or this need to let it end, which is why I'm writing this entry.)

Others will note that if you did start and stop CF (not the service, but CF itself) from the command line, rather than a a service, then you would see any such error messages in the command prompt window. That's true, but it doesn't help you if you already have CF running as a service when this happens. It also would not be an effective way for most to start/stop CF on a continuous basis. But it may be useful for debugging the problem if it happens consistently. See the "configuring and administering CF" docs for more on running CF from the command line.

The key point in this entry is that you will nearly always find something in the log just before CF finally shuts down.

Note that you DO need to let CF shut down, to get that log message

Given what I just said, you will generally need to be patient and to let CF shut down completely to see the message (written in the log just before it finally shuts down). I know this will dishearten some: in the pressure of the moment they just want CF to go down ASAP and so many will just kill the task to be done with it (though of course, sometimes that doesn't work either).

The problem with killing the task is that you then may not see the needed log message explaining why it was waiting so long.

When my customers are complaining of this issue, I try to persuade them to make the painful sacrifice (at least one time) to just go ahead wait and let CF really stop completely on its own, so that they get that log info.

Yes, I realize it may take a few minutes, or even several in some situations. And there are a couple of problems related to that (who can afford that time? and what about the Windows Services timeout). Let's take them one at a time. Because if you need to solve the problem of CF taking a long time to shut down, you have to at least try to get this diagnostic info.

What if you can't "afford the time" to let it stop completely?

As for "who can afford the time" to wait for it to shutdown, here's a tip: if the problem happens fairly consistently (pretty much whenever you stop CF), then just wait to do it (this experiment, where you'll let it take as long as it needs) later in your evening or on the weekend (assuming that's a slower time for you.)

Of course, if you have a cluster of servers, then taking one instance offline is less a problem.

But what about the Windows services panel timeout?

The bigger problem with waiting for it to completely shut down is that you can't "wait forever" for a Windows service to stop. The Windows control panel has a fixed time that it's willing to wait, and if you exceed that it gives an error message (or worse).

The good news is that there are solutions for that, but that's really a whole different subject, and I'll tackle that as a separate entry next (coming up right away).

So what sort of reasons might you find that CF doesn't shut down quickly?

As for what errors that you may find to explain why CF is taking so long to stop, there are many and that would be good fodder for a follow-up blog post. I'll try to get to that in coming days/weeks.

Still, I suspect that most of you will be able to understand what the messages say on your own, now that perhaps you may finally be able to see them. :-)

But certainly, until I do get another entry together on the possible reasons it's slow to shut down, feel free to ask here in the comments or email me directly (charlie at@carehart.org).

For more content like this: Need more help with problems?
  • If you may prefer direct help, rather than digging around here/elsewhere or via comments, I can help via my consulting services
  • See that for more on how I can help a) over the web, safely and securely, b) usually very quickly, c) teaching you as we go, and d) with satisfaction guaranteed
Comments
Thanks, this is really helpful!
# Posted By Tim Cunningham | 10/20/11 4:54 PM
Thanks, Charlie this was great. Helped me track down a problem a lot faster than using the regular logs found in the CF Admin
# Posted By matt | 10/27/11 9:00 PM
Thanks so much, Matt. Always glad to hear when something is helpful.
# Posted By Charlie Arehart | 10/28/11 6:03 AM
Hi Charlie,

I am a regular follower of your blogs, specially the ones on Performance Tuning and System Administration. This particular topic is of some importance to me because we are ending up in this situation quite often and need to reboot our server to actually get ColdFusion services started again.

One observation is that, we get into this situation where we are not able to successfully stop ColdFusion Services and then restart it only when the server has become unresponsive for some reason (another issue that we are trying to find a root cause).

When the server is stable, we do not see issues with stopping and starting ColdFusion services, its only when the server is unresponsive.

Let me know if you have any insights to such a situation and have some quick solutions to look at.
# Posted By Rupesh Rajan | 2/5/15 3:04 AM
@Rupesh, first, thanks for your kind regards, and happy to have helped.

As for your problems (both why you find you sometimes have to restart the server to restart CF, and why sometimes CF hangs up in the first place), I'm sure the problem can be solved. As you indicate that you've read my blog for sometime, you know then that I do this sort of CF server troubleshooting every day for folks as an independent consultant.

But I'm afraid I can't really help solve your particular problem here in the comments, based solely on the information you have provided. And honestly, I doubt your problem could be solved in an extended interchange of comments (or forum posts, or emails, whether to me or even to Adobe). There can just be so many variables: so many places to look and things to consider.

Instead, this is a situation where you really need someone to be on-hand with you looking your server. And by on-hand, I don't mean "on-site" but via the sort of remote troubleshooting I do offer, all day every day to thousands of clients. You can find more about my services, rates, approach, satisfaction guarantee, and more at http://www.carehart....

You'll see there that the approach I take is not that you give me remote desktop access (which would require you to give me a login, and likely to open ports, etc.). Instead, I propose we use a desktop sharing solution (I use join.me, or we could use gotomeeting, webex, etc if you preferred). That way, I just see your screen while YOU remote into your server and YOU login. I'm just "looking over your shoulder" and would guide the process of finding and resolving your problem.

And I'm confident we could solve it, as I've helped many in your situation. So if you're interested, please check out that page and get back to me using the contact mechanisms there and we can proceed, even today if you may be able to arrange it.
# Posted By Charlie Arehart | 2/5/15 9:53 AM
Copyright ©2019 Charlie Arehart
Carehart Logo
BlogCFC was created by Raymond Camden. This blog is running version 5.005.
(Want to validate the html in this page?)

Managed Hosting Services provided by
Managed Dedicated Hosting