[Looking for Charlie's main web site?]

CF911: Are you finding performance problems with CFDOCUMENT? Aware of the important LOCALURL attr.?

This is something that I find nearly no one has talked about, as a problem and solution. Did you know that by default, a single request doing a CFDOCUMENT may cause CF to execute several additional requests, each doing a CFHTTP to grab any images on the page? even if the images could be found locally on the server? This can be quite tragic.

The good news is that the problem can be solved using the simple LOCALURL attribute. The bad news is that you have to do it at all, and that if you don't do it, it can have such unfortunate and unexpected impact. (And just as bad, again, is that hardly anyone has talked about it.) This entry will elaborate on the issue (and a couple of other possible CFDocument performance issues, as a bonus.)

I've been meaning to write about the importance of this problem and solution (the LocalURL attribute) for a long time (it came out in CF8). Often when I'm helping people with CF troubleshooting problems, whether on mailing lists or in my consulting services, I've been able to show that long-running requests (or an unexpectedly excessive number of requests) were sometimes due to this very problem.

Basics of the LocalURL attribute

Before we go any further, let's start with the details of the attribute (which may surprise many). The following is extracted from a larger blog entry about various CFDocument improvements in CF8 from Adobe Engineer, Rupesh Kumar, back in 2007:

When CFDocument body contains a relative URL, ColdFusion will resolve the relative URL to an absolute URL and will send an HTTP request for this url [emphasis, mine]. A side effect of this is Server ends up sending HTTP request even for local URL or images that are lying on the local file system which obviously hurts the performance. In ColdFusion 8, we have added a new attribute "localURL" to cfdocument tag which if enabled, will try to resolve the relative URLs as file on the local machine.

  • localURL : "true" | "false" It should be enabled if the images used in cfdocument body are on the local machine. This would make the cfdocument engine retrieve the images directly from the file system rather then asking the server for it over http

This attribute helps reducing the load from the server so that the same web server thread can now serve user request instead of serving local images to CFDocument. This also addresses some of the "missing image" problems which I mentioned here. Here is a sample code using this attribute.

<cfdocument format="PDF" localUrl="true">
<table>
<tr>
<td>bird</td>
<td><image src="images/bird.jpg"></td>
</tr>
<tr>
<td>Rose</td>
<td><image src="images/rose.jpg"></td>
</tr>
</table>
</cfdocument>

So why is this an issue?

It would help to make a point of clarification: CFDOCUMENT builds a document (perhaps a PDF or Word doc) on the server, from whatever CFML and/or HTML is within the tags.

As such, CF will need to "get" whatever images (or scripts or css files) are defined on the generated HTML page (as img src, script src, link href, etc.) so it can build the resulting "document" file on the server.

And the point of the dilemma (identified on this page) is that a single CFDOCUMENT with many such img src (or similar) tags will not just cause CF to look to the file system to get the files they point to. Instead, CF will get them via CFHTTP.

Technically, the issue is that CF doesn't necessarily "know" that the location pointed to in the img src and similar tags *is* on the local server. So it presumes it has to get ANY of them using a CFHTTP call. That's what causes the problem, if there are many of them. Using the LOCALURL attribute tells CF instead that it SHOULD look for the files on the local filesystem and NOT do a CFHTTP to get them.

Sadly, this issue is only barely mentioned in the CF Docs age on CFDocument, and it would be easy to miss the point it's trying to make. (It says only that CF "requests the server for images over HTTP even though the image files are stored locally".)

How some tried to workaround the problem with file:// references

While looking around to see who else maybe had talked about this fix (which, as I noted, I found virtually no other references to it), I did find that some people had effectively "solved" the problem by telling CF to use "file:///" protocol references in the img src and similar tags.

Among the entries discussing this were:

It's understandable that people figured it out as a work-around, and perhaps someone even started sharing the idea before CF8 added this attribute, but the LOCALURL would seem generally the way to go.

Some other CFDOCUMENT performance resources

Though not related to the LocalURL attribute, there are some other things that may lead to performance problems.

First, those on CF Standard (as opposed to Enterprise or Developer edition) should know that CFDocument is one of several tags that are single-threaded through CF's "Enterprise Feature Router" (EFR).

Second, there are certainly still other possible CFDocument performance issues, and the following other resources address some of those:

Hope any or all of the info above may be helpful to some readers. Let me know what you think.

CF911: Have you updated your #ColdFusion JVM to _24 yet? Important security fix for CF 8/9

This isn't new info, but you may have missed it. If you're running CF 8 or 9, did you know you can and should update the JVM that came with it? And that you have Adobe's blessing to do this update? This is because of a serious bug in the JVM that is not fixed until 1.6.0.24.

Both CF 9.0 and 9.01 run on older JVMs (and therefore need this update). And are you on CF8? You're not left out: Adobe even has confirmed this update can be applied to CF 8 and 8.01, too!

Update: Since posting this note some have kindly pointed out that Java has been updated beyond update 24, and indeed to Java 7. I did realize that, but didn't mention it simply because u24 is all that Adobe has (for now) certified CF to support. Still, for the sake of completeness, I want to clarify this now and I'll address it more in the final section below.

Old news, but not everyone knows

Someone mentioned today on a list that they'd seen news from Oracle of this important bug (and fix) for the JVM, which can cause someone to crash the JVM using a particular URL string. He also noted that while Oracle had released the fix some months ago, he lamented that "This issue doesn't seem to have gotten too much interest from Adobe."

I explained that in fact Adobe had addressed it, also some months ago.

Adobe's response

Adobe offered a technote on the issue in March 2011.

In fact, they not only confirmed support for (and recommended updating to) the fixed JVM version, 1.6.0_24, but they even (for the first time in years) approved updating to this JVM version for CF 8 and 8.01. Since those ran on very old JVM releases, which had problems not fixed until JVM update 10, this was really good news.

But as his note conveys, word just had not gotten out as much as it could. Beside his thread on that list, I hope now that this blog entry will help reach more people.

More about the bug

If you want to know more about the bug from a CF perspective, check out this blog entry (from several months ago by David Stockton, of cfconsultant.com). He explains the problem/risk as well as shows how to cause the problem (to confirm when it's fixed), as well as some useful extra info on using FusionReactor to help diagnose it.

How to update the JVM for CF

So if you're now persuaded, you may wonder how you go about updating the JVM. You may fear it's like doing open-heart surgery. It's more like getting a mole removed. :-) It could go wrong, but will barely be noticeable if done right.

There are many blog entries that walk through the few simple steps. Find out more from Ryan Stille, Mark Kruger, and Adobe, to name just a few.

While it's not too hard to do, there are just a couple of potential gotchas: be sure to get the JDK not the JRE, and pay attention to the special path format that's needed when pointing to the JVM on Windows.

So if you're on CF and have not yet updated the JVM, seriously consider it.

What about updates beyond u24?

This is an update since I posted the entry above. As you'll see in the comments below, some folks were kind enough to write in and point out that there have been JVM updates since u24.

I did realize that, and do appreciate their wanting to share the info. I'd not mentioned updates beyond 24 simply because I would not propose (myself) that anyone now update beyond the release for which Adobe has certified CF. (At least not until compelling reasons arise and substantial community testing suggests it may be safe, as happened with CF8 and the u10 update. Even then, you are risking being out of bounds for support by Adobe, so should make such a change with caution.)

The whole point of this entry was that u24 HAD in fact received Adobe's blessing, for CF 8 and 9, which was pretty compelling and seemed to have been missed by some when it happened earlier in the year.

Anyway, one more tip: in case anyone may find it hard to locate the older updates (like u24) on the Oracle site, here is a link (which works, at least, for now):

http://www.oracle.com/technetwork/java/javasebusiness/downloads/java-archive-downloads-javase6-419409.html

Again, be sure to get the JDK (development kit) not the JRE (runtime environment) among the options listed there.

CF911: Are you finding CF (or CF Admin) busted after applying a hotfix? Three possible reasons

Many people have reported that they find after applying certain ColdFusion cumulative hotfixes (CHFs), security hotfixes (SHFs) and hotfixes (HFs) that either CF Admin or perhaps even some feature of CF is busted.

The "good" news is that there are at least two really common explanations of what may have happened, which I'll explain them here. Once you consider them, you may find either that you can confirm this is what happened (and fix it), or if nothing else you can keep it in mind if you're ever applying them yourself in the future.

I help people solve this very problem often, whether on CF mailing lists or in my independent consulting as a CF troubleshooter. And I've been meaning to post these thoughts for months but as you'll see below, there's a fair bit to consider, if you look beyond what one may try to communicate in a tweet or simple mailing list answer. After doing just that today on yet another forum, I decided to offer the expanded explanation, and hope that it may help someone.

First, some background.

What's different about recent CHFs, SHFs, and HFs (past couple of years)

Before addressing the problem, it would help to understand what's different about hotfixes (for 8 and 9) the past couple of years which can make this an issue in the first place.

It used to be that most hotfixes (CHF, SHF, HF) were just a single jar file, that you would (per the technote for the hotfix) just either upload using the CF admin "system information" page or drop into the appropriate \lib\updates folder.

But in recent years, the hotfixes have gotten more complicated (solving important security problems, often), including now often needing to update entire directories, whether in the WEB-INF or the CFIDE (where both the CF Admin and also various UI features for CFML are stored).

Well, if you make a mistake in doing that (updating those directories, or doing it in the wrong place), then you may find that some part of CF is busted, whether the Admin or perhaps even part of your app (which relies on finding those properly updated files within the CFIDE directory).

Let me make one important note before, some jump in here to (reasonably) bemoan the situation.

Note that Adobe is addressing this with the next version of CF

Yes, it is unfortunate that these problems can happen. And the good news is that Adobe is addressing it in the next release of CF. They have pointed out in a number of recent public presentations and blog entries that they will be adding an automated hotfix mechanism in the next release, for now code-named Zeus.

Also, at the 2011 RIACon conference, during the Adobe keynote, it was indicated that they planned to offer a form of this hotfix installer for CF 8, 8.0.1, 9, and 9.0.1 as well.

But until then, the following may prove helpful.

Now, back to the problem at hand: how you may make mistakes in applying hotfixes that may leave things busted. My observation (in helping lots of people with this problem) is that the challenge is primarily three-fold.

Challenge 1: Updating the wrong directory CFIDE

The first problem could be that you have installed hotfixes related to CFIDE into the wrong place. It's surprisingly easy.

I noted earlier that often the hotfixes may now have you update code in the CFIDE directory. Most know that to be where the CF Admin is stored. But it's also where various files are stored (by default, unless you move them) to be used by various UI elements of CF, in CFIDE\scripts. For instance, all the ajaxy things are stored there, as are the older cfform elements. There's also CSS, XSL, and other stuff there. That's why those things can break (silently, since it's the client's browser that now can't find what it wants, or finds the wrong version of a file, based on CF generating code that expects to use the "updated" versions.)

Well, here's the problem: when the technotes tell you to extract the zip into the CFIDE directory, they often leave you to figure out where that should go.

And the problem is that you may have many. For one thing, when you install CF, you're asked if you want to use the built-in web server, or an external one. If you do the former, then the CFIDE is put in the [cf]\wwwroot. if you do the latter, it's stored wherever you tell it is your web server root, whether inetpub\wwwroot, or htdocs, or wherever.

And if you use the Multiserver form of deployment, you may find that even if you install CF initially to integrate with your web server (in which case it would put the CFIDE in perhaps the inetpub\wwwroot, if/when you create new instances, they by default each have their own CFIDE deep within the instance, such as [JRun4]\servers\[instancename]\cfusion.ear\cfusion.war\CFIDE.

So when you run the update, you need to be POSITIVE that you are updating the one that your CF instance is really using, which means you may need to update many.

And if you're not careful, you may actually find you currently have a CFIDE directory in both kinds of places (and more), either because you have had multiple versions of CF installed, or someone has copied the files from one place to another to "fix" a problem. Then also, some admins (and hosting providers) even repeatedly copy the CFIDE (or perhaps just CFIDE/scripts) to the docroot for different web sites, trying to help with some other issues.

The upshot then is that you could easily have more than one CFIDE.

But it gets better: you can also set things up so that you can access the CF Admin either through the built-in web server, or through the external web server, or both. So besides the question of where the CFIDE lives, there is also the matter of where your web server (whichever you're using) is expecting to find the CFIDE.

And this goes for the CFIDE/scripts. Whether an admin has made multiple copies of it, or just manually added a virtual directory definition to their web server, this means that what matters most (for "finding" the CFIDE to update) is where the web server says it's looking.

Now, before someone gets too burdened with these prospects, note that I'm just explaining some of the many things that CAN happen. You may have a simple environment, and only one (or maybe two) places where the CFIDE lives.

So where to look to know where to update the CFIDE?

Given all the above, what are you to do?

Well here's one tip (not mentioned in the technotes): if you look in the CF Admin, you will find in the Mappings page that there is in fact a mapping for where *CF* at least thinks that the CFIDE is located. So first thing before applying the hotfix, check out that mapping, and at least be sure to update THAT location. That may be all you need.

But second (and even despite doing the above), do look at your web server (IIS or Apache), and see where THEY think that the CFIDE is located. Remember, that's what your browser will end up using (whether for the Admin, if you access it via those web servers, or for your end users running apps on your server). The point again is that they may show different CFIDE mappings, either for different sites, or within different sites.

Yes, it's lamentable. But the problem is caused by people doing things on their own that they need to remember they've done. And no, I doubt Adobe really will be able to fully resolve this, since they'd have to go digging in your web server to find the info. (They could, but should they? since they didn't cause the problem?)

And consider too that it may be that the different CFIDEs are leftovers from different CF versions you may have installed, and each may have been installed differently (one using the built-in web server for the CF Admin, one using the external web server).

So as the captain in the old show, "Hill Street Blues", used to say to his patrolmen after their morning briefing, "be careful out there".

Challenge 2: Extracting the zips at the wrong level

This next challenge is different from the above, and yet applies to that step (extracting the zip for the CFIDE). But it also applies to extracting the zip for the WEB-INF, or any other directory where a hotfix says to extract a zip.

Here's the problem: it's possible that even if you point your zip extracting tool (winzip, izarc, winrar, copy/paste, whatever) to the "right" directory, you may mistakenly (or the tool may) extract the zip in such a way that what you extract is not at the level you point to, but one level below it.

I've seen it happen, to myself and others. (And some zip tools are more prone to it than others, especially winrar, in my experience.)

Here's the thing, if the technote says, for instance, to extract from the zip the web-inf directory, if you're not careful you may end up putting a NEW \web-inf under your current one! So you end up with \web-inf\web-inf. That will NOT, of course, update the needed files within the "real" \web-inf directory.

And of course the same could happen when you extract the CFIDE. You could end up with CFIDE\CFIDE.

Indeed, I've seen people have it happen when they extract the jar to the lib\updates.

So again be VERY careful, and double check things as you proceed.

Challenge 3: Be sure to do ALL the steps in the technote

Finally, another mistake I've seen people make is that they miss one or more steps in the technote. Again, these recent hotfixes have gotten more and more complicated, and have many steps.

Again, the automated hotfix mechanism in Zeus should address that.

But until then, it's incumbent upon you to make sure you follow each and every step, carefully and completely. Hope the above helps.

(BTW, there's a slightly related issue: you may also think you can skip over intermediate CHFs, like from having none to going to CHF 2, or going from CHF 1 to CHF4, etc. I addressed that separate problem in a past entry, "CFMyths: "If/when I apply Cumulative Hotfixes, I need apply only the latest CHF, right?"">.)

What you can do, if it's broken

So the steps above are what to be aware of as you're going through the hotfix update process. But what if you did it and it's busted, and you found this note after the fact?

Well, I hope you can use the info above to simply double-check what was done. For instance, check if the directories have mistakenly got things in the wrong place (like if you see a web-inf\web-inf).

Look also at the dates of the files in the zips, and make sure that the dates you show (where files SHOULD BE) are in fact the same dates.

As for the CFIDE issues, besides the above, you may also benefit from just doing a search of your hard drive to find any and all CFIDE directories. You may find many. Again you can check for which are being used by your web server and make sure they are correct for whatever version of CF you mean to be running (for that URL on that web server).

Hope that helps. Or did I leave anything out? Any questions? Comments? Let me know, either way. Like I said, I'd been holding off publishing this because I knew there was a lot that deserved to be covered and it would take time to write. I felt pressed to get it out today, and am trying to remember now all that I've had in mind over the past couple of years on this topic. :-)

Let me know how I did.

CF911: Tips for dealing with Windows service timeout, useful when CF's taking too long to stop/start

In my last entry, CF911: Is your ColdFusion service taking too long to shut down? Find out why, I discussed the first of a two part answer to helping people who are finding that sometimes ColdFusion takes too long to shut down (or startup). That one talks about how to find out where CF may be logging info to explain why it's taking so long.

In this second entry, I'll address the separate but related problem, particularly if you're running CF as a Windows service, that you may find you get timeout errors from the Windows Services panel itself. I'll share some tips to help with that, which I share often with clients of my independent consulting as a CF troubleshooter.

The first thing to understand is that there is indeed a timeout (configurable) for how long the Windows Services control panel is willing to wait. Beyond that, though, there are some 3 more tricks you can use to avoid that timeout.

Update: I should add that a commenter suggested that another solution was just to kill the task. Um, yeah. I knew that, of course. :-) But the point of this entry, like the last one, is when you DON'T just want to kill it. In particular, I made the point in the last entry that if you want to find out *why* CF is taking so long to shutdown, then you need to let it run to completion to see if it may log some diagnostics. See the last entry for more on that.

But yes, as Russ Michaels noted, sure, you can also just kill the task. I will say I try to avoid doing that whenever possible, to let CF end gracefully, unless I'm forced to kill it for urgency sake.

Note that these tips can be helpful for everyone who runs CF as a Windows service, and indeed for anyone running any application as a Windows service.

Tip 1: There is a timeout set for how long the Windows Services panel will wait

So the first thing to note is that, yes, the Windows Services control panel has a fixed duration for how long it's willing to wait for an action (start, stop, or restart) to take, which is configurable in the registry. If the service control action you request does not finish in time, you'll get an error like "The service did not respond to the start or control request in a timely fashion", with associated event log errors like 1053, 7000, 7009, 7011, and so on.

And technically, you should be able to control that timeout via a registry key, called ServicesPipeTimeout, which is discussed in this Microsoft technote and defined in HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control.

Sadly, though I've tried it on multiple machines, I'm not finding it to work.

The technote shows clearly how to either update or add it (if it doesn't exist), including noting that you need to choose the "decimal" option to see the value, which is in milliseconds. The technote does note that you must restart Windows for that change to take effect (and I have done that, and double-checked the value, keyname, and location.)

If anyone has more thoughts on this, why it may not be working for me, feel free to share. Still, you will find many blog entries and discussions proposing this, for CF and other servers, so maybe it will work for you.

The technote doesn't clarify what the default is if the key is not there at all, though I've found some mention that it's 30 seconds. I suppose it could vary by version of Windows (anyone know for sure?).

(Note that there is another key you may see (or hear about) set in the same location: WaitToKillServiceTimeout. That is NOT related to this discussion. That key, instead, controls how long Windows should wait for any/all services to stop when you shutdown Windows itself, which is interesting to consider for itself, but not related to this particular blog entry about Services control panel timeouts.)

Of course, all the usual warnings apply about messing with the registry. But it's the solution, if you need it.

So again, this ServicesPipeTimeout registry key would seem to be the one that controls how long the Services panel is willing to wait for a requested action to finish.

Do beware that it's system-wide setting, controlling all services. What may be good for you (CF) may not be good for all services, so be careful. Second, again, note that the server will require a restart for that to take effect.

Whether it works for you or not, the remaining tips should be helpful.

For instance, if you can't change that registry settings, or don't want to, or you find some day the Services panel is still taking too long to stop a service and you worry it may timeout (the control panel), then there are 3 more tips I often share with my customers (and now you, dear readers).

Tip 2: Use stop/start, instead of restart

Again, if you're being plagued by a service start or stop taking a long time, here's something you may not have considered.

Most often when people want to restart CF (or some other service) they will use the "restart" action in the Services panel. That's logical, of course.

But consider that when you have a problem with the action taking too long (stop and/or start, since a restart does both), both have to finish in that configured timeout time (above).

You can help matters by doing not a restart but a stop and start separately. That way, each action can take whatever the max time is that's configured (per above) to do just the one action.

Sure, it's a little painful. You do have to wait for (and notice) the stop before then doing the start. There's also the risk that even the stop may take too long. I'll offer two more tips related to that.

Tip 3: Close the progress bar before its "times out"

This one may not be as obvious, but it does really help. When you have a start/stop/restart action (for the Windows Services control panel) taking a long time, you risk hitting that timeout above.

But notice that Services panel shows a progress bar indicating the duration of the action you've requested. Here's the tip: if/when it's about to reach the end of its indicated time, note that you can hit the "close" button.

That will prevent it complaining about not being able to stop the service. (And BTW, I've seen situations where hitting that timeout limit not only gives an error but even causes the start/stop/restart buttons to be disabled, requiring a restart of the box!)

Once you close the progress bar, you can use the refresh button in the Services panel (or f5) to watch for when the service goes from "stopping" to blank, meaning it's down. Same for when you "close" it while starting a service: it will go from "starting" to "started".

Tip 4: Consider not using the Services Panel at all

Still another tip (which I use regularly) is that you don't actually HAVE to use the Windows services panel to stop/start Windows services. You can do it from the command line (or a shortcut or batch file).

To be clear, I'm not talking about starting the program itself from the command line (CF, as I've been discussing in these last 2 entries), but the Windows service that starts the program. It's a subtle difference.

As an alternative to starting the Windows Service (for CF) using the Service control panel (Control Panel>Administrative Tools>Services, to be clear), you can use a command line command like this, which I use to stop CF:

net stop "ColdFusion 9 Application Server"

The service name is whatever is shown in the properties for the service you want to stop. (And of course you would use a "start" command instead to start the service. There is no "restart" command, buy you could create a batch file to do both.)

You can run this from the command line (Start>Run>cmd) or as a batch file, but I prefer instead to just setup a Windows shortcut, which you can do by right-clicking the desktop and choosing new>shortcut.

That will prompt you for the command (it just asks you to "type the location of the item", but just enter that command above). It will then ask for a name for the shortcut. I call mine simply "stop cf9".

And while you can then execute it by going to the desktop and double-clicking it, a niftier tip is that with more modern Windows versions (2008, 7, Vista), when you hit the the Window Start menu, it now offers a search feature, and you should be able to type "stop cf9" (or just type "stop " and leave it to pop any shortcuts you've created this way to stop various services.)

So why do I like this approach, of stop/starting the service from the command line instead? Because what happens is that a window pops opens (the traditional black background for DOS windows, so easy to observe) and it shows the stop (or start) taking place, however long it takes (which may be very fast). The key benefit is that the window will close on its own when the action is done (however long it takes). I just watch for the stop to be done, and then I do the start.

There is a possibility, of course, that there will be an error in the shortcut or batch file (or perhaps as a response to the correct command), in which case the window will close leaving no chance to see the error. In that case, I just go to the command line (Start>Run>cmd) and execute the same net stop or start command to see what the error is.

To be clear, using this command-line doesn't stop you or others from using the Services panel.

Conclusion

All that said, though, again, if the reason you need to worry about all this is that your CF Service is taking a long time to stop, and you want to find out why, then I would point you again to the last entry I did, In my last entry, CF911: Is your ColdFusion service taking too long to shut down? Find out why.

Hope all that was helpful, whether you use CF or not. Please do let me know in the comments below.

CF911: Is your ColdFusion service taking too long to shut down? Find out why

Have you ever found (or heard others complain) that sometimes ColdFusion doesn't stop (or it takes a long time to do so)? It can be especially challenging when you're running CF on Windows as a Service, for reasons I'll explain here and in a follow-up entry.

First, this one will help you perhaps find why it's so slow. You may just have been looking in the wrong place for that diagnostic information. Second, the next entry will offer tips to handle on better handling the situation (that the Windows service is slow to stop, and timeouts in the Windows Services panel itself) which you may need to consider until you do solve the root cause problem (or in case it happens again for other reasons).

(Note that most of this info will apply as well if your problem is that CF's taking a long time to start, also.)

As some of you know, I do CF server troubleshooting as an independent consultant. In helping several people a week, this is a fairly common complaint. This also came up on a mailing list today, so I decided to offer these thoughts here.

Check the logs--but not the "CF" logs

Generally, the explanation for why CF is taking so long to stop (and perhaps also, to start) will be logged (if you are patient enough to wait for it to really go down. More on that in a moment.)

Of course, if you run CF from the command line, then the error would be shown in the command line console. But nearly everyone running CF on Windows will instead be running it as a service. In that case, things get more challenging.

The challenge is that the information about startup/showdown of CF is NOT detailed in the "CF" logs (those shown in the CF Admin, or found in the [cf]\logs directory or buried deep within an instance in Multiserver mode). Those logs (like application.log and exception.log) are often not helpful for system-level problems.

Instead, you need to look in the equivalent of the "console" logs, which in CF 9 or before are found in [cf]\runtime\logs\*out*.log or [jrun]\logs\*out*.log. (In Zeus (the next version, not yet released, which has been publicly shown as running on Tomcat), the will be in the "logs" directory, but as a new file: [zeus]\logs\start.log.)

I refer to these as CF's "console logs" (though as you can see that's not not the formal name) because again this is where CF writes messages that would have been written to the console if you'd started CF from the command line.

(Some will note that there is also info written to the Windows Event logs, but nearly always it just says to see the CF logs. Sadly, it doesn't clarify which logs, or this need to let it end, which is why I'm writing this entry.)

Others will note that if you did start and stop CF (not the service, but CF itself) from the command line, rather than a a service, then you would see any such error messages in the command prompt window. That's true, but it doesn't help you if you already have CF running as a service when this happens. It also would not be an effective way for most to start/stop CF on a continuous basis. But it may be useful for debugging the problem if it happens consistently. See the "configuring and administering CF" docs for more on running CF from the command line.

The key point in this entry is that you will nearly always find something in the log just before CF finally shuts down.

Note that you DO need to let CF shut down, to get that log message

Given what I just said, you will generally need to be patient and to let CF shut down completely to see the message (written in the log just before it finally shuts down). I know this will dishearten some: in the pressure of the moment they just want CF to go down ASAP and so many will just kill the task to be done with it (though of course, sometimes that doesn't work either).

The problem with killing the task is that you then may not see the needed log message explaining why it was waiting so long.

When my customers are complaining of this issue, I try to persuade them to make the painful sacrifice (at least one time) to just go ahead wait and let CF really stop completely on its own, so that they get that log info.

Yes, I realize it may take a few minutes, or even several in some situations. And there are a couple of problems related to that (who can afford that time? and what about the Windows Services timeout). Let's take them one at a time. Because if you need to solve the problem of CF taking a long time to shut down, you have to at least try to get this diagnostic info.

What if you can't "afford the time" to let it stop completely?

As for "who can afford the time" to wait for it to shutdown, here's a tip: if the problem happens fairly consistently (pretty much whenever you stop CF), then just wait to do it (this experiment, where you'll let it take as long as it needs) later in your evening or on the weekend (assuming that's a slower time for you.)

Of course, if you have a cluster of servers, then taking one instance offline is less a problem.

But what about the Windows services panel timeout?

The bigger problem with waiting for it to completely shut down is that you can't "wait forever" for a Windows service to stop. The Windows control panel has a fixed time that it's willing to wait, and if you exceed that it gives an error message (or worse).

The good news is that there are solutions for that, but that's really a whole different subject, and I'll tackle that as a separate entry next (coming up right away).

So what sort of reasons might you find that CF doesn't shut down quickly?

As for what errors that you may find to explain why CF is taking so long to stop, there are many and that would be good fodder for a follow-up blog post. I'll try to get to that in coming days/weeks.

Still, I suspect that most of you will be able to understand what the messages say on your own, now that perhaps you may finally be able to see them. :-)

But certainly, until I do get another entry together on the possible reasons it's slow to shut down, feel free to ask here in the comments or email me directly (charlie at@carehart.org).

I'm speaking this evening on the Adobe CF Developer Week webinars: mine on CF Server Monitor

Hey folks, just a heads up (for those who may not have seen all the tweets and list messages) that this week is the Adobe CF Developer Week series of free webinars.

Update, Recording: Note that this session was recorded. You can view it here, but note that you must login with an Adobe ID to see it.

And I'm presenting a session tonight, Tuesday September 13, at 7pm Eastern, on "Understanding and Using the ColdFusion Server Monitor".

As many of you know, I'm pretty much a fanatic about the monitor, especially about truly understanding elements of it that many miss. And so in my talk this will not be just a dog and pony show, but I will talk about practical experiences with it, though presented to either those new to it or experienced with it.

Note that the times for all these devweek sessions is shown (on the Adobe site) as being Pacific time, so again mine is at 7pm, not 4pm, Eastern.

And yes, the sessions are being recorded and seem to be made available the next day.

Finally, beware that there is no one URL you can use to join in on all the Connect sessions, nor can you get the Connect session URL by going to the event page (via the first link above). Instead, you must register for each event (free) from that first page, to get each session's Connect URL--and you'll want to do that at least several minutes in advance of any session to have time to register, get the email, login, etc.

See you then.

PS Hey, while we're talking monitoring, note as well that if you've not heard, FusionReactor has come out with its new release 4, which has lots of great additions, especially FREC (or the FR Extensions for CF) which cause FR to grab and log lots of great info that the CF Server Monitor only shows and never logs. I'll be blogging about FR 4 soon, but plenty to see on their site. and FusionAnalytics is also just about to release, really!

I won't be discussing these at this talk, focused solely on the server monitor, but as I always tell folks, each tool has its use and often a single shop can benefit from having both (like I do, as do many of the clients I help with troubleshooting). You can find more from me about FR here in my blog. And I'll have lots more to say about FA and FR4 more soon.

I'll be presenting at RIAcon next weekend: "CF911 ColdFusion Performance Report 2011"

I'm delighted to announce that I'll be presenting at next week's RIAcon conference in the DC area, August 6-7 2011.

My session will be "CF911 ColdFusion Performance Report 2011", a new talk/concept I've created. Here's the description:

Starting a new tradition, veteran CF troubleshooter Charlie Arehart will present a review of the performance aspects of making various choices when working with ColdFusion, especially in recent version(s) of ColdFusion. Leveraging the important value of real load testing (as opposed to the less accurate conclusions from "large loop" testing), Charlie's annual report will help attendees appreciate the performance-related improvements of new/changed features, as well some older features where choices can make an important impact. Depending on the timing of the release of "CF next", the session may cover its new features, but it will certainly cover some things new in CF 9 and 9.0.1.

As I note there, I hope this may become an annual event which I might present at this and/or other conferences. (It's an idea rooted in a similar presentation made by a former colleague, from my first IT career from 1982-1997, where he presented the annual "Jim Damon model 204 Performance Report".)

About RIACon

As for RIAcon, I hope you're considering it. Phil Nacelli and the folks at AboutWeb have been working hard to put together the conference, which in some minds is kind of picking up where CFUnited left off. It will be a more intimate event, much like CFunited was when it first started.

Indeed, some will recognize that the location for RIAcon is across the street from one of the hotels where CFunited was held in its early years, right next to the Twinbrook Metro station in Rockville, MD.

A Personal Connection to the Hotel Location

Even more of a delight for me personally is that the hotel (The Legacy Hotel) is right on the land that was once the location of Congressional Roller Skating Rink (until the late 70's), where my sister and I (and many friends) spent our teen years pretty much whenever we weren't in school. Yep, I was a skating nerd: dance, figures, freestyle, and more. Here's incriminating evidence!

cf.Objective(): I'll be there, and I'll be busy with 4 sessions

I've gotten word from the folks running the cf.Objective() conference that besides the 2 talks I'm giving, they've also recently accepted my proposals for a BOF (birds of a feather) session and a slot in their lightning talk session. Phew! I'll be busy.

Here are the details.

First are the two talks:

(I had put in just the first talk originally, and then a few weeks ago a slot opened and they asked if I could do the other, which I was happy to offer, as an reprisal/update to my talk from the first release.)

Then for the Lightning Round (or what was originally referred to as the Pecha Kucha), my talk will be:

  • Lies, damned lies, and CF request timeouts (in which I'll share in 5 minutes some information that even experienced developers admit having never known)

Finally, for the Birds of a Feather (no page on their site listing them yet), the session I will be leading will be:

  • CF911: Share your CF server troubleshooting tips (come share some ideas, or learn from myself and others)

Sense a theme? Yep, other than the CFBuilder talk, the other sessions are all focused on the topic that is now most near and dear to my heart (and livelihood): CF Server Troubleshooting. It's what I do, and more important it's how I feel I can best help the most people.

There's one last aspect of my involvement at the conference that I'll mention: they started a new sponsorship program this year called "Friends of cf.Objective()", and I'll be participating in that. No mention of it yet on their site, but there should be more news at the event.

So hope to see you there, or if you won't be there, I'll post if any of these are recorded, or if not then I would likely record them myself in the future.

CF911: Looking for info on handling IIS 7 integration in CF9 Updater 1? Not in the docs

Are you looking for info on the change in handling of IIS 7 integration as of CF9 Updater 1? Sadly, it's not in the primary docs you may think to look at. There's also another gotcha that I will explain in a follow-up entry.

What's changed about IIS 7 support in 9.01?

Some folks will know that ColdFusion 9 Updater 1 (9.0.1) has finally added full support for IIS 7, without need to rely upon enabling IIS 6 Compatibility (which IS still required for CF 9.0, 8.0, and 8.0.1). This is indeed great news, whether you're running on Vista, Windows 7, or Windows Server 2008, which all have IIS 7 by default, and for which you can enable IIS 6 Compatibility mode, but it's not on by default and not always straightforward to enable.

So bottom line, if you're on 9.0.1, you no longer need to go through the hoops described in the CF9 Admin/Install docs (nor in helpful blog entries like here and here, though those are still great for those running on CF 9.0 and 8, and may even offer tips of value to anyone setting up CF to run on IIS, which has other challenges on Server 2008.)

Gotcha 1: The IIS 7 Compatibility change is NOT documented where expected

Sadly, though, if you go looking for help on this in the CF docs, such as Chapter 6 of the manual, Installing ColdFusion 9, you will find that it has NOT been updated with the info that is new in the updater.

It opens, "If you are configuring IIS 7 ... ensure that you have the options IIS Metabase and IIS 6 configuration compatibility ... and ISAPI Extensions ... selected".

What a shame.

Solution 1: Where the change IS documented

So with respect to change in 9.0.1, where you no longer need to enable IIS 6 compatibility, the details are covered instead in the installation guide that was created just for installing CF9 updater 1 itself, called "Installing the Coldfusion 9 Update", available online at:

http://www.adobe.com/support/documentation/en/coldfusion/901/cf901install.pdf

It's certainly reasonable to expect that the primary install manual would have been updated with the info above.

But that's why I'm pointing this out. I have also added a comment explaining it on the page pointed to in the first link above. Hope that may help someone.

(The same info is also offered as a chapter in the manual, "ColdFusion 9 Updater 1: New Feature Notes", available at http://www.adobe.com/support/documentation/en/coldfusion/901/cf901features.pdf, which is of course very interesting if you may have missed it.)

I'll explain the second gotcha in a follow-up blog entry.

Did you know there's a "request execution limit" on IIS? It's 3, 10, 25, or unlimited, depending...

Did you know there's a "request execution limit" on IIS? It's 3, 10, or unlimited, depending on the version of Windows (Vista, 7, 2008) and edition (such as Starter/Home/Basic/Pro/Server).

I'll detail the limits per version/edition below.

I'll also offer a (possibly surprising) workaround that can allow you to get even more requests through IIS, even for a single web site.

(Before I elaborate on that, note that there is a separate issue if you're finding that CF doesn't let you see more than 25 requests at once. That's instead due to a setting in CF/JRun, the maxworkerthreads setting. For more on that, see this blog entry.)

That said, this is a problem which could affect anyone regardless of the app server they may be running behind IIS. (And yes, I do realize that for some, the answer to this problem will be, "see, that's another reason to run Apache." We get that. Let's just focus on this problem for those who choose/have to remain on IIS.)

Background

I was doing some load testing noticed that I couldn't get beyond 3 requests running at a time. I suspected the problem may be IIS.

So at first I tried just making the same requests against CF's built-in web server (such as might be running at port 8500, which can be enabled/configured during CF installation or manually).

Sure enough, CF could execute more requests at once (up to the "simultaneous requests" setting) through the built-in web server.

So that told me the problem had to be in IIS. I did some digging and found the documentation that affirmed what I was seeing.

(Some may recall there used to be a limit of 10 for IIS on Windows XP, so it may surprise some to run into an even lower limit on later Windows versions.)

IIS 7 does have a "request execution limit"

The first resource I found was with respect to the limits in Vista. This page says specifically: "With Starter and Home Basic editions of Windows Vista, the simultaneous request execution limit for IIS is three for Windows Vista Home Premium. ...For Windows Vista Business, Enterprise, and Ultimate edition...the simultaneous request execution limit is ten. ..With server editions of Window, IIS 7.0 has no request execution limit."

But I'm sure some reading this will be using Windows 2008. I found a page discussing its limits: for the Basic & Starter editions: 3 requests, for Premium: 3, for Pro: 10, and for Server: unlimited.

I happened to be running Windows 7 when I hit the limit, and while I've not found a document stating the limits specifically for Windows 7, I can again confirm the same limit of 3 on my copy of Windows 7 Home Premium, so I'm sure things are the same, to whatever degree the edition names parallel the other versions. (It's curious that the learn.iis.net page above does discuss both Vista and Windows 2008, but makes no mention of Windows 7.)

A workaround for this limit: multiple worker processes

With respect to this limit, here's a valuable bonus tip that I've not seen documented anywhere: the limit is really per application pool, or more technically per worker process. So you can certainly get more request simultaneously against your box by using multiple app pools.

But perhaps you want to have more requests against a single site, which obviously can be connected only to a single app pool. There's still good news: you can increase the number of worker processes per app pool in the "advanced settings" for a given app pool (right-click on the app pool), increasing "Maximum Worker Processes" from the default of 1. (Some will recognize that as being the same as what was known as "web gardens" for app pools in IIS 6.)

For those new to them, whether creating new app pools or more worker processes for an app pool, for each new worker process, you'll see a new w3wp.exe in task manager.

A caution, for those using ASP.NET, about setting more than one worker process per app pool

Finally, there's a caution to consider if you decide to increase the number of worker processes. At least back in IIS 6, I documented that if you're running an ASP.NET app using sessions that are "inproc" (or in memory), the default, there is a problem with using multiple worker processes (web gardens), in that the sessions are not replicated among the worker processes. That may not be an issue for the OP, so I'll say you can learn more at an entry I did a few years ago.

Did any of this help you? Let me know below.

More Entries

BlogCFC was created by Raymond Camden. This blog is running version 5.005.

Managed Hosting Services provided by
Managed Dedicated Hosting