[Looking for Charlie's main web site?]

Sending HTTP headers in a CFHTTP request? Name them correctly

If you ever try to use CFHTTPPARAM (inside a CFHTTP) to set HTTP headers for the request you're calling, be aware that you need to be careful to specify the name of the headers as they're known in the HTTP specification. Don't be misled by what you see in a CFML dump of the CGI scope.

(This is a reprisal and update of an old blog entry I'd done back in 2003, on a blog site that will soon no longer exist. I'll be reprising a few such blog entries in coming days/weeks, to keep them around for posterity [and to save some having to dig for them in the archive.org site] since often the info offered then may be just as valuable now. I hope that in time these new versions would come up if people do searches that would have found the old entry.)

So, about these http headers, while CFML exposes them in a dump of the CGI scope, such as the user-agent field which shows up as cgi.http_user_agent, the issue is that you would not use that name, nor even "user_agent", when specifying it in a CFHTTPPARAM. The proper way to pass the user agent in a CFHTTPPARAM would be as follows:

<cfhttpparam name="user-agent" value="somevalue" type="CGI">

Why is this an issue, and how may it happen?

An easy mistake to be lulled into

The point is that the name of the header you specify must be a name that is correct according to the http specifications, not the headername as it appears in CFML when you dump the CGI scope. The problem is that CF tries to "help" you by how it labels the headers in these scopes, but it does not show them in the actual name you must specify if you're going to be sending it in a CFHTTPPARAM request.

It's is an easy mistake to be lulled into, especially if you aren't careful to confirm the results in the page you are calling. And if you are trying to call some non-CF server, and things aren't working, this may well be the reason.

Other possible headers of interest

Other headers of possible interest that you may want to specify when doing a CFHTTP request are Accept, Accept-Encoding, Referer, and so on. So if you were wanting to send/spoof what CF would show as the cgi.http_referer variable, you'd want to send it with the name just as "referer".

Alternative type="cgi" and type="header"

FWIW, note that I am using here the type="CGI" value. CFML also supports a type="header". The difference is in whether CF urlencodes the result before sending it (it does not for type="header"), which could be another issue to consider. This encoding can also be turned on or off with the available "encoded" attribute, at least for type="cgi".

Yes, you don't need to use cfhttpparam for sending the user agent

Finally, someone may want to point out that you don't even need to use CFHTTPPARAM to send the user agent, since CFHTTP offers a separate built-in USERAGENT attribute which can be used instead. Yes, this is do.

I suspect it's an older approach that either existed before CFHTTPPARAM permitted passing it, or it was put in as a shortcut.

But the point of this entry is that if you do need to set this or any other headers within the CFHTTPPARAM, be aware of using the correct http header name.

Applying CF security hotfixes: do it from oldest to newest

If you may be applying several security hotfixes to a new implementation of CF (or one where none have been applied before), you may wonder if there's any significance to applying them in either chronological order (newest to oldest, or oldest to newest). The technotes don't really clarify this.

I will propose that they should be done from oldest to newest. Here's why.

If you look at the list of CF security hotfixes offered here, you may wonder whether to start with the most recent ones, or the oldest ones. If you look at their respective technotes and zip files closely, you may find that some are truly independent of the others. But it's also a reality that some do supercede the others. That may be obvious to some, but I wanted to offer a case in point.

Consider the hotfixes for APSB11-14 and APSB11-04. If you look at the zips for these, you may notice that they offer a few files (in the web-inf/lib directory) that appear identical in name (log4j.properties, esapi.properties, validation.properties, commons-fileupload-1.2.jar, and so on.) If you had applied the later one first, you may then wonder if it's significant if you then want to apply the older one.

Well, those files are in fact identical (I did a compare). But the files in the CFIDE associated with each hotfix are in fact quite different and so the later ones would supercede the earlier ones.

So you would NOT want to do them in the wrong order (older before newer). It's just not always clear from the technotes whether this is an issue to worry about, so I wanted to share it here.

This is not the place for discussion of how complicated CF hotfixes are. That's been discussed to death in many places. If you didn't know, Adobe is indeed addressing the problem in the next release, currently code-named Zeus. You can read about this and 50 other currently publcized Zeus features in a talk I did. And note that they have said they will be back-porting some of that to earlier releases.

Hope it's helpful to someone. If you may have a different experience or opinion about the order of applying security hotfixes, feel free to share it.

PS I have written previously about easy mistakes you can make in applying any CF hotfixes, and how to avoid them. See CF911: Are you finding CF (or CF Admin) busted after applying a hotfix? Three possible reasons. I have also written about the challenges of trying to "skip" cumulative hotfixes. You may want to read them if you've not yet done so.

But again, please hold off on any comments about the complications of CF hotfixes. There's nothing more to do than deal with the situation as it is and await Adobe offering a better solution.

CF911: Are you finding performance problems with CFDOCUMENT? Aware of the important LOCALURL attr.?

This is something that I find nearly no one has talked about, as a problem and solution. Did you know that by default, a single request doing a CFDOCUMENT may cause CF to execute several additional requests, each doing a CFHTTP to grab any images on the page? even if the images could be found locally on the server? This can be quite tragic.

The good news is that the problem can be solved using the simple LOCALURL attribute. The bad news is that you have to do it at all, and that if you don't do it, it can have such unfortunate and unexpected impact. (And just as bad, again, is that hardly anyone has talked about it.) This entry will elaborate on the issue (and a couple of other possible CFDocument performance issues, as a bonus.)

I've been meaning to write about the importance of this problem and solution (the LocalURL attribute) for a long time (it came out in CF8). Often when I'm helping people with CF troubleshooting problems, whether on mailing lists or in my consulting services, I've been able to show that long-running requests (or an unexpectedly excessive number of requests) were sometimes due to this very problem.

Basics of the LocalURL attribute

Before we go any further, let's start with the details of the attribute (which may surprise many). The following is extracted from a larger blog entry about various CFDocument improvements in CF8 from Adobe Engineer, Rupesh Kumar, back in 2007:

When CFDocument body contains a relative URL, ColdFusion will resolve the relative URL to an absolute URL and will send an HTTP request for this url [emphasis, mine]. A side effect of this is Server ends up sending HTTP request even for local URL or images that are lying on the local file system which obviously hurts the performance. In ColdFusion 8, we have added a new attribute "localURL" to cfdocument tag which if enabled, will try to resolve the relative URLs as file on the local machine.

  • localURL : "true" | "false" It should be enabled if the images used in cfdocument body are on the local machine. This would make the cfdocument engine retrieve the images directly from the file system rather then asking the server for it over http

This attribute helps reducing the load from the server so that the same web server thread can now serve user request instead of serving local images to CFDocument. This also addresses some of the "missing image" problems which I mentioned here. Here is a sample code using this attribute.

<cfdocument format="PDF" localUrl="true">
<table>
<tr>
<td>bird</td>
<td><image src="images/bird.jpg"></td>
</tr>
<tr>
<td>Rose</td>
<td><image src="images/rose.jpg"></td>
</tr>
</table>
</cfdocument>

So why is this an issue?

It would help to make a point of clarification: CFDOCUMENT builds a document (perhaps a PDF or Word doc) on the server, from whatever CFML and/or HTML is within the tags.

As such, CF will need to "get" whatever images (or scripts or css files) are defined on the generated HTML page (as img src, script src, link href, etc.) so it can build the resulting "document" file on the server.

And the point of the dilemma (identified on this page) is that a single CFDOCUMENT with many such img src (or similar) tags will not just cause CF to look to the file system to get the files they point to. Instead, CF will get them via CFHTTP.

Technically, the issue is that CF doesn't necessarily "know" that the location pointed to in the img src and similar tags *is* on the local server. So it presumes it has to get ANY of them using a CFHTTP call. That's what causes the problem, if there are many of them. Using the LOCALURL attribute tells CF instead that it SHOULD look for the files on the local filesystem and NOT do a CFHTTP to get them.

Sadly, this issue is only barely mentioned in the CF Docs age on CFDocument, and it would be easy to miss the point it's trying to make. (It says only that CF "requests the server for images over HTTP even though the image files are stored locally".)

How some tried to workaround the problem with file:// references

While looking around to see who else maybe had talked about this fix (which, as I noted, I found virtually no other references to it), I did find that some people had effectively "solved" the problem by telling CF to use "file:///" protocol references in the img src and similar tags.

Among the entries discussing this were:

It's understandable that people figured it out as a work-around, and perhaps someone even started sharing the idea before CF8 added this attribute, but the LOCALURL would seem generally the way to go.

Some other CFDOCUMENT performance resources

Though not related to the LocalURL attribute, there are some other things that may lead to performance problems.

First, those on CF Standard (as opposed to Enterprise or Developer edition) should know that CFDocument is one of several tags that are single-threaded through CF's "Enterprise Feature Router" (EFR).

Second, there are certainly still other possible CFDocument performance issues, and the following other resources address some of those:

Hope any or all of the info above may be helpful to some readers. Let me know what you think.

BlogCFC was created by Raymond Camden. This blog is running version 5.005.

Managed Hosting Services provided by
Managed Dedicated Hosting