[Looking for Charlie's main web site?]

CF2016: What's deprecated and/or 'no longer supported' (note: nothing 'removed')

Continuing a series of posts I started last week on ColdFusion (2016 Release), aka CF 2016, I'd like to highlight the items that Adobe has chosen to deprecate and/or declare as "no longer supported", as of CF 2016.

(Note that for now, nothing is "obsoleted" or "removed" in CF2016, so this is just about our being made to know that certain features that DO work now in CF 2016 may NOT in future releases and won't be receiving any additions, enhancements, or modifications. )

While some of these features listed as deprecated will be things that most won't miss (and some may even wonder why they weren't deprecated sooner), some of the features listed as deprecated may really surprise some folks, as I'd not heard discussion of their deprecation myself until finding them discussed in the CF docs.

[....Continue Reading....]

Sending HTTP headers in a CFHTTP request? Name them correctly

If you ever try to use CFHTTPPARAM (inside a CFHTTP) to set HTTP headers for the request you're calling, be aware that you need to be careful to specify the name of the headers as they're known in the HTTP specification. Don't be misled by what you see in a CFML dump of the CGI scope.

(This is a reprisal and update of an old blog entry I'd done back in 2003, on a blog site that will soon no longer exist. I'll be reprising a few such blog entries in coming days/weeks, to keep them around for posterity [and to save some having to dig for them in the archive.org site] since often the info offered then may be just as valuable now. I hope that in time these new versions would come up if people do searches that would have found the old entry.)

So, about these http headers, while CFML exposes them in a dump of the CGI scope, such as the user-agent field which shows up as cgi.http_user_agent, the issue is that you would not use that name, nor even "user_agent", when specifying it in a CFHTTPPARAM. The proper way to pass the user agent in a CFHTTPPARAM would be as follows:

[....Continue Reading....]

Struggling with using the XML features of CFML? Here's where to learn more

Are you struggling with using the XML features in CFML? Or do you help people who are?

It's not something most people use often, and there is far more to the feature set than meets the eye. If you don't use the right resources to help you, you could waste time trying to piece together a solution using only scant examples you may find.

In this entry, I'll point to several resources you should consider to help get you quickly up to speed in using--and appreciating--the power of the XML features in CFML.

Even if you don't need them today, you (or someone you know) may need them some day, so keep this in mind. (Or if you find this page doing a google search some years down the road, drop a note to let me know if it helped then!)

Background

[....Continue Reading....]

Bug with Query of Query: using LIKE incorrectly finds records with null values

Someone reported on a mailing list that they'd noticed a mistake in the results of a Query or Query operation when using a LIKE. It finds a record whose value for the column is null, which is of course wrong (that doesn't match the LIKE pattern--indeed a null never equals or doesn't equal anything).

The problem can be worked around by adding "and column is not null", but it really surprised me (and him, obviously) to discover the bug. Maybe someone else has written of it before.

But I tested it in 8.01 and can confirm it's still an issue. While I've recommended that he file it as a bug, I wanted to share it here until it's resolved. Hope this may help someone. (If we're missing something, feel free to comment.)

Here's some code you can run which uses the example app database that comes with CF. See the comments for what's wrong:

<CFQUERY NAME="demoq" DATASOURCE="cfdocexamples">
   select * from employees
</CFQUERY>

<!--- note that by default, all records in the example employees table have values in all columns --->

<cfdump var="#demoq#" label="before insert">

<!--- insert a record with only one column (causing the others to be null)
--->

<CFQUERY NAME="insert" DATASOURCE="cfdocexamples" >
   insert into employees (firstname) values ('charlie') </CFQUERY>

<CFQUERY NAME="test2" DATASOURCE="cfdocexamples">
   select * from employees
</CFQUERY>

<cfdump var="#test2#" label="after insert">

<!--- do a Q of Q of that query, for a column which would have a null for that last added records, so you'd expect it not to find it --->

<cfquery dbtype="query" name="test3">
   select * from test2
   where department like 'Sales%'
</cfquery>

<!--- that record with the null value still shows up! --->
<cfdump var="#test3#" label="dump of Q of Q">

Again, the problem is that the record with the null value still shows up in the Q of Q result, and it definitely should not. A solution for now is that if you change the Where clause to "department is not null and department like 'Sales%'", then it properly does not show the null-valued record(s) anymore.

(If anyone plays with this on their own code, note as well that Q of Q is case-sensitive, so no records would be found at all if you mistakenly used 'sales%' rather than 'Sales%', above.)

CFMAIL being detected as spam? Some solutions for CF 6, 7, and 8

Someone asked on a list about how to resolve the problem where messages sent via CFMAIL have a message-id value that can cause mail to be flagged as spam. I mentioned in an earlier entry how to solve this in CF8. In this entry, I offer a couple of solutions for CF 6 and 7.

To backup and explain the problem, some have lamented that in CFMX 6 and 7, CFMAIL used the name of your server where CF is installed, which might be something like "server1" or "bingo", as in:

Message-ID: <23070863.1197039960343.JavaMail.SYSTEM@Server1>

This might cause a recipient mail server to flag the mail as spam, if the mail server was a different domain name (like "yourcompany.com"). The bummer was that many found no way to fix this. Sure, in CF5 you could set it in a CFMAILPARAM to set a mail header, but CF 6 and above ignores that.

The good news for those on CF 8 is that a change makes the problem go away, which I talked about in a blog entry back in December.

But for those still on 6 or 7, I'd mentioned that there's an available code or (preferably) configuration fix to resolve the problem. Apologies for not posting it at the time, so here it is. I also explain what the issue is all about, for those not aware of it.

Simple CF5 solution doesn't work in 6, 7, or 8

First, let me point out that some may propose that one can just use CFMAILPARAM to set that message-id header, as was mentioned in this FAQ:

http://www.developer.be/index.cfm/fuseaction/faqDetail/FaqId/201.htm

Unfortunately, that's a very old FAQ and it no longer applies after CF5.

CFMX 6 (and 7 and 8) ignore that header you set and set the message-id themselves to the name of the physical server from which the mail's sent. Here's an example, from my laptop (which is named charlied620):

Message-ID: <23070863.1197039960343.JavaMail.SYSTEM@CharlieD620>

On your server, it could be that the problem is that the server is a real domain name, but it's not the same name as the SMTP server through which you're sending email via CFMAIL, and some servers won't let mail through with FROM addresses having a different domain name.

Why doesn't CF just use the SERVER set in CFMAIL or the CF Admin?

You might wish/expect it would just use name of the mail server as specified in the CF Admin mail server setting or the CFMAIL SERVER attribute, but it does not (at least, not prior to CF8, as discussed in my earlier entry.)

But while researching this problem for the person, I found this, where a very compelling solution was proposed. I tested it, and it does indeed work.

It turns out that you can get CFMAIL to use a specific mail server, either of 2 ways, for CF 6, 7, or 8 (again, on CF8, the easier solution is that I discussed in the earlier entry). Ken Smith of Adobe offered it also in this Adobe forum thread. If you make this change, either server-wide or per page/app, you will now find that the CFMAIL sets the message-id to use a desired servername.

Solution for 6, 7, or 8: Configuring it at CF Startup

If you're on a server where you want to change this for all CFMAIL (and javamail) done from within CF, you can change the JVM.config for your server to add this to the java.args line:

-Dmail.host=desiredservername

That java.args line is one long line. Don't introduce any line breaks. You need to restart the server after making that change. As always, when editing the JVM.config (in the runtim/bin directory of CF), make a backup first, because if you get it wrong, your server won't restart. (Or leave the file open and be prepared to do an Undo in your editor.)

But if you don't want to (or can't) mess with the startup file, or more important, if you are on a server where each app may need to do its own setting, you're not out of luck.

Solution for 6, 7, or 8: Changing it programatically

You can also make the needed change on the fly, programatically. You could issue this set of code just before your CFMAIL:

<cfscript>
sysObj = CreateObject("java", "java.lang.System");
sysObj.setProperty("mail.host", "desiredservername");
</cfscript>

I tried this per-page setting in in 6 and 7 and it worked fine. For CF8, the code will run, but it won't affect the CFMAIL for reasons I explain in the next section. But there's a real problem with this approach, if you're on a shared CF server. You're still setting the value for the entire CF server, but you're just doing it on the fly. It has at least a few problems you need to think about:

  • you'll affect all who use CFMAIL on this server (at least in 6, and 7, since CF8 ignores this, as discussed below)
  • you'll also affect anyone who uses Javamail on this instance of CF
  • and if someone else on the server issue the same code with a different server name, that of course would override your setting

There are a few things you can do to mitigate the problem, but they're not entirely perfect.

First, you could get the current mail.host value (in case it's different), then change it to what you want, and then change it back. You could do that with this code:

<cfscript>
   sysObj = CreateObject("java", "java.lang.System");
   // if there is no mail.host set already, this next variable simply won't be created. need to know that for later.    oldmailserver = sysObj.getProperty("mail.host");
</cfscript>

Note that if the mail.host property has not been set in the startup config or by someone else running such code, then the getproperty will return nothing, and the oldmailserver variable literally will not be created (a curiosity of working with some java methods).

Once you have the oldmailserver (or know that it did not exist), you can do the set of the property and the cfmail as above, then you could set it back with the following:

<cfscript>
// reverse the setting of the mail.host, so as not to affect others on this server if (not isdefined("oldmailserver")) {
   // if there was no previously set mail.host, remove the property
   sysObj.getProperties().remove("mail.host");
}
else{
   // set it back to what it was before
   sysObj.setProperty("mail.host", oldmailserver);
}
</cfscript>

There's still a problem. Because of CF's multi-threaded nature, it's entirely possible that between your setting the host and doing your CFMAIL, someone else could also set the mail server to something different. (This is called a "race condition".)

Now, Mr. Smith in the threads above suggested that you could wrap the code doing the change and the CFMAIL in a named lock, but that's an incomplete solution. It will prevent other other code (that ALSO does the same named lock) from running until yours is complete, but it won't help if others do the set of the property without bothering to use the same named lock (or use a differently named lock).

That's a frequent misconception about locks. They don't prevent other code doing what you're doing in the lock: they only tell other code that IS using the same lock not to run while this lock is held--and even then, only if you use an EXCLUSIVE lock, which he didn't indicate.

So the bottom line is that as useful as the feature is to set the property dynamically, it's fraught with peril in an environment where multiple apps may try to use the code. Only if you can guarantee that all use the same named lock will you be able to protect against this problem is held up while you have the value changed:

<cflock name="setmail-servername" timeout="10" type="EXCLUSIVE">

   <cfscript>
   sysObj = CreateObject("java", "java.lang.System");
   // if there is no mail.host set already, this next variable simply won't be created. need to know that for later.    oldmailserver = sysObj.getProperty("mail.host");
   sysObj.setProperty("mail.host", "desiredserver");
   </cfscript>

   <cfmail ...>
   ...
   </cfmail>
   
   <cfscript>
   // reverse the setting of the mail.host, so as not to affect others on this server    if (not isdefined("oldmailserver")) {
      // if there was no previously set mail.host, remove the property
      sysObj.getProperties().remove("mail.host");
   }
   else{
      // set it back to what it was before
      sysObj.setProperty("mail.host", oldmailserver);
   }
   </cfscript>
</cflock>

Now, someone may propose that all this could be wrapped up into UDFs or CFC methods, and perhaps it could, but since you need to do the setting of the value, and the mail, and the resetting of the value, all within a CFLOCK, it would be challenging to wrap all this up into a generically callable method (unless you wanted to pass in as well all the CMAIL attribute values and the body). Just seems not worth it, since this is a pretty esoteric problem and solution. But others can comment if they feel differently.

Solution for CF8 is much easier

All this is obviated on CF8, because it now properly uses the name of the server specified in the CF Admin mail server setting (or CFMAIL SERVER attribute, which overrides the CF Admin setting.) This is AWESOME news for those challenged by this, and hasn't gotten much press.

Now, what was the caveat I mentioned above? Well, if you use the approach of setting the mail.host servername in the java property, CF8 no longer pays attention to that. It JUST uses the CF Admin mail server setting, or the CFMAIL SERVER attribute. So that code above "won't work". It will work, but it won't affect CFMAIL. But that was a hack to work around CF not honoring these other attributes. I'm not surprised (or bummed myself) to see that it no longer regards the mail.host property for CFMAIL.

(I should say I'm saying this as of CF8. I've not tested it on 8.0.1.)

Some related notes

If you add the SERVER attribute, you may need to add the PORT, Username, and Password as well.

Here's a little bonus tip, in case you try to use the CFMAIL SERVER attribute for the first time on an existing CFMAIL tag to check this out. Note that using the SERVER attribute on CFMAIL requires you then to specify any other attributes required for the mail connection such as USERNAME and PASSWORD (if needed) and PORT (if not 25). What I mean is that it will no longer pick up the values in the CF Admin. If you override the SERVER, then you override the other config settings as well and need to specify them.

I'm not getting the emails now. Where are they?

If you make a mistake in your setting of the mail server arguments, then CF will move the failed emails to the [coldfusion]/mail/Undelivr directory where CF is installed. They're just plain text files. You can open them to see what got set and perhaps can figure out why they failed. You may also find information in the [coldfusion]/logs/mail.log.

When I look at mail in the spool, it looks fine (if I use CFMAILPARAM). Why doesn't it get through?

Don't be misled. The email you generate in CFMAIL may look fine in the [coldfusion]/mail/spool directory, but when it gets sent to the mail server, CF will change that message-id (and some other headers). You really need to look at the email as it's RECEIVED. You can't even look at the message in the Undelvr folder as an indication. It doesn't have those added headers.

So how DO I observe the mail headers?

You need to receive the email and then look at its headers. I'll show you how to do that in Outlook and gmail in a moment. Let me point out that it can be very englightening to view the message headers: not only the message-id but possibly also other headers as well as messages that CF, your mail server, or the mail server of your recipient may have added, which may include indications of whether your email was detected to be spam (perhaps by tools like SpamAssassin).

In Outlook, you can use View>Options while reading an email, to see the values in the "Internet Headers" box of the window that opens. Scroll down in that to find the message-id and other headers.

In gmail, when you open the message, look to the top right of the pane showing your email, to the right of the indicator of the time the message was received. There's a drop-down box, with options like "reply" and "forward". Choose "show original". That will show the complete message including all the headers at the top.

Other CFMAIL alternatives/replacements

Finally, I'll point out that if you run into other problems with CFMAIL, there are always alternative CF mail server solutions, like those I list in the "CFMAIL replacement/enhancement tools" section of my "Tools to Consider for CFML developers" page.

Hope that helps some.

Stopping multiple form submissions with CF 7/8 and "submitonce" validation

What happens in your app if a user hits the submit button more than once before the form is processed, such as when the form didn't come back quickly enough, so the user submitted the form again? Would you be interested to know that CF has a feature to prevent them doing that? It's an often-missed hidden gem of CF7.

Have you ever considered this prospect of multiple form submissions? What could happen? It could cause multiple inserts to a database, or multiple charges to a card, or unexpected increases in some session variable counter, etc., which could be real trouble for you or your users.

It's a subject that comes up often in developer circles (even outside of CF). The good news is that there's a very simple solution in the available validate="submitonce" option of CFINPUT, which as of CF7 can be used for submit buttons. You'd use CFINPUT TYPE="submit" VALIDATE="submitonce" NAME="somename", as demonstrated in a complete code example below.

In this entry, I offer more info on the feature. I've not found too many other resources discussing it, so I hope this will help folks. There are some interesting challenges you should understand as well.

Problem already solved for you in Firefox--but do all your users use it?

For instance, if you've read about the feature and tried it on Firefox, you may have been surprised that you couldn't see what it did differently than a normal submit button. That's because it turns out Firefox already solves the problem itself, by preventing double-submission of a given form. It doesn't hurt to use the CF-based feature on FF. It just isn't needed. But you certainly still need the CF-based feature, though, if you may have users visiting your site with other browsers. (I can confirm that even IE 7 does not prevent double form submissions. Haven't tested Safari yet. Any takers want to report in the comments?)

Demonstrating the problem of multiple form submissions

It's not too difficult to demonstrate the problem of multiple form submissions. All you need is a means to track how many times the server-side form processing takes place for a given form's submission(s). In your real application, again, this may lead to multiple inserts in a database or multiple charges to a customer's credit card. To keep things simple, I'll demonstrate it using a session variable that's incremented on each form submission.

But then more important (to demonstrate the effect), we need the form processing to simulate "taking too long" so that a series of rapid submissions of a form will be able to be sent before the processing of the page "completes". We can do that in CF6+ by calling the Java thread sleep method, or in CF8 we can use the new sleep built-in function.

The example code is offered at the bottom here. Before you take a look at that, or try it out, note a few things.

Applies to form, not submit button

Besides the point above about Firefox being immune automatically to the multiple form submission problem, note that this multiple submission protection is enabled on the form itself, not on the submit button, so it does apply just as readily to forms submitted by pressing enter (when permitted by the browser) as by forms submitted by pressing the submit button.

The fact that it's enabled for the entire form (when the CFINPUT above is used) means also that you can't test this by having a "regular" submit button and a "protected" one in the same form. Again, it's not the submit button that gets the protection, but the form in which the CFINPUT type="submit" appears. You'll notice, therefore, that my demo code above uses 2 forms. (It's not at all important whether I use or don't use CFFORM in the "unprotected" form example. It just makes no difference.)

Does require use of CFFORM And Javascript

That makes a point though: the CFINPUT tag does indeed need to be used within a CFFORM tag, as shown in the second form. And further the feature is indeed relying on Javascript (generated by CF) to perform the multiple submit prevention. Neither should be a showstopper for most. You don't need to convert any other HTML tags within a form to their CFML CFFORM equivalents (like CFSELECT or CFTEXTAREA) just to enable the CFINPUT for form submission.

Some may want to point out that you don't need to use this particular approach to solve the problem: there are other Javascript-based approaches, as well as some that assert to work without JS. I'll leave it to commenters to mention them here, if interested.

My goal was to demonstrate the submitonce functionality, since it's not been discussed much. Do let me know if this was interesting to you.

Some final observations about my code example

I've provided comments in my code example code, about things that aren't related to the multiple submissions feature, but which may raise questions for some.

First, note that because I'm using sessions to demonstrate the feature, I've just gone ahead and put the CFAPPLICATION tag right into this template, so it doesn't matter where you put the code (won't be helped or hurt by an existing application.cfm or application.cfc, with respect to the session var created.)

As for the forms, note that I'm using self-posting forms. It doesn't matter if you do or don't for this multiple submission prevention feature to work.

And note that I don't use any means to force a filename into the form ACTION attribute, to make it post back to itself. If you leave the action empty, that makes the form self-posting. This is a legitimate HTML-specified use, if not widely known. No need to force the current file into it using #cgi.script_name# or the like.

And I determine if the form is submitted using a test for cgi.request_method being "post", which it will be on a form submission (versus it being a "get" when the page is first loaded). I don't use a test for whether the submit button is defined (a technique that is taught by some but will fail when the page is run on IE, if the user presses enter rather than the submit button, at least when there's only a single input field). Again, these choices have NO effect on the validity of the tests.

Finally, while I don't *need* to specify the empty action or post method on a CFFORM (since they're the default), I do it to avoid any confusion from those looking at the code and not familiar with that

The example code

You should be able to drop this code, verbatim, into any CFML page in any CF7 or 8 server to see the effect.

<cfapplication sessionmanagement="Yes" name="submitoncedemo">
<cfparam name="session.submitted" default="0">
<cfset interval=2>

<h4>Demonstration of New submitonce validation in CF7+</h4>

Try clicking each of the submit buttons below multiple times, in rapid succession, before the page returns from its #interval# second delay. Wait for that delay before noticing the count indicated after the 2nd form.
<p>
Notice how the "uncontrolled submit" will cause execution of multiple submissions (as tracked by a session variable). (The problem is not apparent in Firefox 1.5 and above, as it includes its own feature to prevent multiple form submissions. Try it on IE, though, even IE 7.)

<form name="test" action="" method="post">
<input type="Submit" value="Uncontrolled Submit">
<input type="button" onclick="location.href='<cfoutput>#cgi.script_name#</cfoutput>'" value="Reset Counter">
</form>
<!--- the use of no value for ACTION, to do a self-post of the form to itself, is legitimate HTML-specified functionality, if not widely known. No need to force the current file into it using #cgi.script_name# or the like. --->

<hr>
The "controlled submit" will not. Regardless of how many times you press it, it will only register a single increment in the session count, demonstrating that CF prevented the form processing from being executed more than once until the form processing page was completed.

<!--- This needs to be in a separate form from above, because the submitonce validation applies to the form, not the button. And while I don't *need* to specify the empty action or post method on a CFFORM (since they're the default), I do it to avoid any confusion from those looking at the code and not familair with that. --->
<cfform name="test2" action="" method="POST">
<cfinput type="Submit" validate="submitonce" name="submit" value="Controlled Submit">
<input type="button" onclick="location.href='<cfoutput>#cgi.script_name#</cfoutput>'" value="Reset Counter">
</cfform>

<cfif cgi.request_method is "get">
   <!--- if the form is requested the first time, or via the HREF below, reset the session variable. --->
   <cfset session.submitted=0>
<cfelseif cgi.request_method is "post">
   <!--- if the form is submitted, process it. (There are other ways to test form submissions, but this is a very good one for many reasons.) --->
   <!--- put thread to sleep for the number of seconds indicated above, to simulate a wait while form submission is being processed --->
   <cfset thread = createObject("java", "java.lang.Thread")>
   <cfset thread.sleep(javaCast("long", 1000*interval))>
   <!--- in CF8, could use sleep function:    <cfset sleep(interval)> --->

   <!--- yes, perhaps we should lock the sesion variable below, but it's not critical to help or hurt this demo--->
   <cfset session.submitted=session.submitted+1>
</cfif>

<cfoutput>
Submitted: #session.submitted# times.

<hr size="3" color="##000000">

<h4>Discussion</h4>
The form above has 2 submit buttons to demonstrate the new SubmitOnce validation in CF7+.
<p>
The first button, "Uncontrolled Submit", is a normal submit button (<input type="submit">).
<p>
The second button, "Controlled Submit", uses (<cfinput type="submit" validate="submitonce" name="somename">) which causes CF to prevent it being clicked twice to cause submission of a single form.
<p>
To demonstrate the effect, the action processing of this form is set to wait #interval# seconds before completing. It also outputs (using a session variable) how many times the button led to submission of a request to the form processing portion of the page.
<p>
Note that clicking the uncontrolled submit multiple times in rapid succession (on other than Firefox--haven't tested on Safari) will lead to multiple submissions. Even though you don't see the output of each submission (because each new submission starts a new request to the server), the increase in the session variable at a rate higher than just one per the #interval# second interval it takes for the form to complete. This demonstrates that multiple submissions are being caused.
<p>
Do the same with the "controlled submit" button, and that doesn't happen. It only ever increases by one per 3 second period.
<p>
</cfoutput>

How can I process Excel files in CFML? Let me count the ways

Most of us have seen over the years many requests and many ways to process (read or create) Excel (.xls) and other spreadsheet files. In fact, there have been so many different ways (some simple, some powerful; some new and some quite old) that I fear some may do a quick Google search and try the first thing they find. Sadly, what works for some may not work for all.

One place to list all the approaches

I don't recall ever seeing any single listing of all the possible ways to do CFML to Excel integration, so I decided to create one. I've created a new section in my Tools to Consider for CFML developers on the topic of "Excel File Processing Tools".

You'll see I've gathered a range of articles and tools from over the years showing how to do CF-Excel integration (both directions) any of several of the following ways (the links go to the section within my tools list on this topic):

  • You can easily create them using HTML tables and CSV generation (as well as read them using CSV)
  • You can create them with the more powerful XML-based approach
  • You can create and read them using Java-based APIs like POI, JExcel, and jXLS
  • You can create and read them using COM and now in CF8, .NET
  • You can read them using ODBC and JDBC drivers
  • Let's not forget also that you can create them using the Report Builder in CF7 and 8, which can output as Excel
  • And finally there's the old Excel Web Query feature for populating a spreadsheet from a CFML request

In the listing, I offer dozens of links to articles, blog entries, downloadable tools, any relevant CF docs, and more, all broken down by the approaches listed above.

Different Strokes for Different Folks

So you see, there are not only different solutions but different needs (create a spreadsheet CFML, read one into CFML, or populate one within Excel from a CFML request). It seems helpful to gather a single resource to organize them all and help people decide.

I need your feedback

So let me know what you think of all this. Did I leave out any other important approaches? I'm not claiming to "know it all". This is just a result of some research I did this morning (and/or recollection from past experiences). I'm sure I could have missed something. Certainly I know I'm not listing *all* possible references on each of the subjects. I just wanted to give folks something to start with, but I'm certainly open to adding any useful references or tools that I left out. Please leave a comment below.

Update: Indeed, already, in response to a comment, I've tried to make more clear the fact that I'm not just talking here about creating spreadsheets from CFML but was referring to reading them as well. That's why I chose "processing" in the title, and I referred several times to "reading them", but to make it stand out, I've bolded all the references to reading them.

More important, because it seems he had missed the 2 links to the lists where I offer more details, I've also changed the words in the list above to link into the discussion of the topic in the tools list. Hope that helps avoid people missing the real point of the listing. I was torn about just offering the listing itself as a blog entry and went with this approach instead of a summary that points to the details. I know some don't like really long blog entries. Also, listing them in the bigger tools list seemed helpful, but I may split it out into its own file. We'll see.

Not just about Excel or even Office

BTW, one may argue that I could have a generic section on processing all sorts of file types (or even just all kinds of MS Office) file types, but there are so many requests for Excel file processing specifically, and so many approaches/tools to suit those needs, that I just figured I'd start out with this and consider other filetypes later (and may rename the section then). It's also certainly true that most of the techniques/tools shown here can be used with any spreadsheet tool that reads xls files or processes CSV/HTML table files in a similar way (like OpenOffice).

Update

Again, note that since creating this blog entry, I may have updated the section on this in my CF411 tools list, "Excel File Processing Tools". Be sure to check that out to see if there are still more new options.

CF8 Hidden Gem: CFMAIL auto-generated message-id uses specified mail server name

Here's a neat hidden gem for CF8 that will delight some: CFMAIL now uses the mail server name you specify in CFMAIL SERVER attribute or the CF Admin mail setup page, in creating the message-id header that's generated when your mail is sent.

Some have lamented that in CFMX 6 and 7, it instead used the name of your server where CF is installed, which might be something like "server1" or "bingo", as in:

Message-ID: <23070863.1197039960343.JavaMail.SYSTEM@Server1>

This might cause a recipient mail server to flag the mail as spam, if the mail server was a different domain name (like "yourcompany.com"). The bummer was that many found no way to fix this. Sure, in CF5 you could set it in a CFMAILPARAM to set a mail header, but CF 6 and above ignores that.

So the good news is that CF8 now does use that CF Admin or CFMAIL SERVER value to create the message-id. If I use CFMAIL SERVER="mail.carehart.org"..., for instance, I might get something like:

What about CF 6 or 7?

But what if you're on 6 or 7? Well, there's a solution for you, too, by way of a tweak in the CF startup script (jvm.config) or (less preferably) in code. Since the change for CF 6 or 7 deserves some explanation to do it justice, I'll take that up in a part 2 message next. I'll also lay out the whole problem with the auto-generated message-id and why it's a concern for some.

For now, I just wanted to get this word out to those who understand the problem already that CF8 solves it. Woo hoo.

CF and OO and blogs, oh my (a flush of CF-OO blog tutorials)

Step right up ladies and gentlemen, get your CF-OO tutorials here. :-)

I've noticed a sudden flush of multipart CF-OO tutorial blog series being done, to help people make the leap to using CFML in a more Object-Oriented way. I thought I'd point them out for those who may have missed them:

Don't know if there's a connection between the rather simultaneous releases, or if it's just pure coincidence. Charlie clarified that there indeed was no connection between the two.

And for those who want to get their CF-OO freak on, there are still more resources on the topic.

And there are certainly many more. I'd welcome hearing of those to add to the list. And there may well be more multi-part CF-OO blog series' created before or after now. Feel free to make note in the comments here and I'll update the list for posterity.

Maybe we're starting to reach a tipping point. Of course, there are some who will lament a push to OO (too complicated for them), while others will say you're a hopeless loser if you don't get with the program.

I don't want to debate that here, nor do I really want to see it blow up in my comments, please! :-) Feel free, instead, to point people to where else it may have taken place so that they can evaluate the discussions there instead.

Resources for getting a much greater understanding of the SQL Server query plan/procedure cache

Ever wanted to understand the SQL Server query plan/procedure cache better? It can be vital to good performance of SQL statements, especially with respect to the concept of "bind variables" or "parameterized queries" (using CFQUERYPARAM in a CFML context, or Parameters.Add() in a C# context, among other ways).

Update: If you may wonder why any of this is important, see some subsequent talks I gave on the topic, that give far more context on the problem. While you rarely hear about it from others, it is a source of potentially significant problems so is worth understanding more completely. See the "My Own Talks" section at the bottom of this entry.

Yet we so often just kind of assume the database will do what's best. Or we blindly use (or recommend use of) bind variables without fully understanding why, nor understanding some implications when it may not always be such a good idea.

And did you know that if you can't (or don't) change your code to cause parameterization, did you know that the database can do it for you, either automatically in some simple cases, or by way of an enforced parameterization, either for an entire table of for queries that meet certain "plan guides"? There are many implications to understand in all this.

And how do you track the plan cache (using DMVs, the profiler, perfmon)? And how does it work (allocation of memory, flushing the cache automatically or manually)?

Fortunately, I've found many great resources to help you understand, and I've not found them listed (all together) anywhere else yet.

The info applies to 2005, 2000, and 7, though some aspects may differ, as the resources indicate. (There are even substantive changes in 2005 SP2 that are important to note over the RTM and SP1 releases.)

Huge Plan Caching Article Series by MS Engineers

First, I'll point out that some MS engineers have put together a long and very resourceful series of article-length blog posts on the topic, with explanations, code, demos, troubleshooting techniques, and lots more.

They do kindly offer a "table of contents" page listing all the topics covered, with bullet points about topics within each entry.

Sadly, that page doesn't offer URLs to the articles, nor have any that I found there, nor do the articles link to each other. You can dig around and find the URLs, even finding a category page that lists them in rather random order. But I've looked around the web and can't find any page that lists them all with their links, in order, so here you go:

  1. Structure of the Plan Cache and Types of Cached Objects
  2. Sql_Handle and Plan_Handle Explained
  3. How Cache Lookups Work
  4. Query Parameterization
  5. Retrieving Query Plans from Plan Cache DMV's
  6. Best Programming Practices
  7. Costing Cache Entries
  8. Factors that affect Batch Cache-ability
  9. Memory Pressure Limits
  10. Plan Cache Flush
  11. Temporary Tables, Table Variables and Recompiles
  12. Plan Cache Trace Events and Performance

They also offer a series of extensions to that, on troubleshooting:

Chapter in "Practical SQL Server 2005 Troubleshooting" book

Next, I'd point out that one of the contributors to that, Bart Duncan, is a Microsoft Support engineer who I saw speak on the topic at the SQL Server PASS 2006 conference, where I first learned a lot of this info.

He identified then that a lot of the info was in a chapter of a new book, SQL Server 2005 Practical Troubleshooting: The Database Engine, which I've since gotten and was indeed very useful.

If you're a member of the O'Reilly Safari service (or join for their trial), you can find the chapter online.

Of course, both the talk and the book came out before the article series above, and certainly before SP2, so I would recommend you consider both. (I've not yet done a comparison to determine what may be in the chapter that's not in the articles.)

SQL Server BooksOnline

Of course, it always pays to read the docs, and there are indeed some discussions of this concept and related features in the SQL Server Books Online, but I honestly found the info above either easier to find or just more complete.

My Own Talks

I've made my own attempts to communicate some of this info myself in a couple of talks I've given to both CFML and SQL Server audiences:

But really, there is just too much to communicate in any one hour. If this topic seems of interest, take a few hours to digest all the above. I think you'll be AMAZED at what you learn.

Did you know about the NULL attribute of CFQUERYPARAM? I didn't, until today.

Will the riches of CFML ever cease to amaze me? :-) Did you know about the NULL attribute of CFQUERYPARAM? I didn't, until today. Consider that you use CFQUERYPARAM and point its VALUE to a variable. What would happen if the value was empty, but your database required a null? How would you solve this? If you didn't know better, you might use an IF test to say "if it's null, use a null, else use the variable", but there a much more elegant solution.

Did you know that CFQUERYPARAM has a NULL attribute that is just for this purpose? It takes a boolean to indicate whether and when to use a NULL rather than the VALUE.

It's not new, having been around since 4.5 according to the CFML language history file. I've just never noticed it before. I learned about it today from a couple of folks on the great CFAUSSIE list.

Now, to be honest, the docs (CFML Reference for the tag) just don't make it as clear as it could be, I don't think (otherwise I'd like to think I'd have noticed it before). I don't see any mention of it in the Developer's Guide, for instance.

Anyway, you can read more about it in a blog entry from Michael Sharman, who I see also just happened to blog about it last month, with a lot more detail:

http://www.chapter31.com/2007/02/04/cfqueryparam-and-conditional-handling-of-nulls/

You might also want to read the comments there as well as at the CFMX 7 docs for the tag, both of which have people sharing their experiences using the tag, over time.

Using a range of hex values in a CF regular expression

If you were asked by someone to strip a string of all characters having ASCII codes from some value and higher, how would you do it? If you'd think to use a regular expression, you get 5 points. :-) But how would you get it to take a range? If you're ever stumped, here's how.

Assume you have a str1 variable with the string to be processed, and you want a resulting str2 holding the result. Let's assume as well that you've been asked to strip any codes with ASCII values of 160 or higher. This will do it:

<cfset str2 = rereplace(str1,"[\xa0-\xff]","","all")>

The CF docs do discuss using the \xnn option to search for a hex value, but they don't show how to specify a range. You might, as I did, try a few variations until you stumble upon it. That's the format.

Oh, and as for the xa0, that's the hex equivalent of decimal 160, and ff is the top of the range. I didn't find that it performed any more slowly with ff or some lower number.

Hope all that's helpful to someone. As for why you may do the above, I'd rather take that up in a separate entry which I hope to write today.

Did you know you can store CFDOCUMENT and CFREPORT output in a variable?

This may not be news to some, but it was to me. I just noticed that the CF7 tag, CFDOCUMENT (as well as the new form of CFREPORT), has an available NAME attribute. This allows you to save the resulting PDF or FlashPaper (or Excel, in the case of CFREPORT) output (in binary form) to a variable.

Some may know the available FileName attribute, which allows you to save the output to a file. This NAME attribute is simply an interesting alternative.

Here's a quick example:

<cfdocument format="PDF" name="test">
test
</cfdocument>

This creates a variable named test. How would you output it? CFCONTENT, of course, with the appropriate content type. If your page has created any other output, don't forget the Reset attribute as well:

<cfcontent variable="#test#" type="application/pdf" reset="Yes">

To output a Flashpaper format report, use a CFCONTENT type of "application/x-shockwave-flash", and for a CFREPORT Excel spreadsheet, use a CFCONTENT type of "application/vnd.ms-excel".

Why might you do this?

Of course, if you just did that above, you might as well not have bothered, right? The point is that you may do something between when you create the variable and when you output it. For one thing, you may create it in one CFC, custom tag, or CFFUNCTION-based UDF and then output it later in the request.

More likely, you may choose this approach to facilitate caching and later reusing the generated output. Just as with creating a variable with any tag, you could have instead provided a prefix scope like session, application, or server. Then you could manage that cached result any of the many ways that have long existed for caching other tag results in shared scope variables.

Posting a form to itself without trickery, using an empty ACTION attribute

Did you know that if a form has an empty ACTION attribute (or none at all), it will post back to itself (to the page that presented the form)? This can be very useful, and it's a lot easier than more complicated code that struggles to build the ACTION attribute to hold the current filename and any query string.

In the first iteration of this entry, I referred to the two approaches of either providing an empty ACTION or none at all, but as the comments below show, the former violates the HTML spec. So let's stick with the notion of an empty ACTION. Same result, though.

How often have we all seen code along the lines of:

<form action="<cfoutput>#cgi.script_name#?#cgi.query_string#</cfoutput>" method="post">
...

or more involved:

<cfset action=CGI.SCRIPT_NAME>
<cfif CGI.QUERY_STRING NEQ "">
   <cfset action=action & "?" & xmlformat(CGI.QUERY_STRING)>
</cfif>
<form action="<cfoutput>#action#</cfoutput>" method="post">
...

All this could be replaced very simply with:

<form method="post" action="">
...

The form will post back to itself. I'll offer another post that shows a unique way to take advantage of this. In any case, I hope that this observation may help some folks.

(Update: I never got around to that other entry in 2007, but see a my reply to a comment below where someone asked for more info on the idea I had in mind.

Is this reliable?

Now, there are some who will argue that this is a violation of the HTTP HTML spec, and so it may be, but I've never found a browser in which it didn't work.

Again, a clarification over what I wrote originallyhere. As was refined in the comments below, it's a violation to have *no* ACTION, but it's perfectly legit according to the URI spec (section 4.2 at http://www.ietf.org/rfc/rfc2396.txt) to have an empty ACTION, which is interpreted as a "same-document reference. (Thanks, Christopher Bradford, for that info.) Given that, even the following cautions seem needless, but I'll leave them for any still concerned.

If you have any hesitation, because you have to support multiple browsers and you can't test all possibilities, I'll understand if you choose to pass on this. But certainly if you only need to support browsers you can test, then if it works as expected, enjoy.

If anyone reading this can offer where this is the case, I'd appreciate hearing it. If you want some simple code to test, try this:

<form method="post" action="">
   <input type="Submit">
</form>
<cfif cgi.request_method is "post">
   Posted to itself
</cfif>

If it shows the text within the IF, then it worked as expected.

What about query string info?

You may wonder about the earlier more involved examples that showed passing the query string, in case any had been passed to the form. No problem. This technique passes any query string along just fine. Try it yourself (add ?test=test to the form and view in the debugging info that it's still in the URL scope after submission.)

CFFundamentals: Mimicking a form submission using CFHTTP

While developers who've used CFML for a long time will regard some topics as old hat, we have to remember that there are folks who either have come along recently or simply never used some feature. This is one of those topics, and since I answered it on a list, I offer it here.

I'd like to start doing that more often (reformatting some answer I give on some list, so that others may learn from it.) As in this case, I'll refer to such as CFFundamentals and create a category for them so that interested readers may find them more readily.

The problem

Someone wrote wanting to mimic (in CFML code) the submission of some data to a form that they saw happening within an Airborne Trackship (http://track.dhl-usa.com/TrackEmail.asp) page.

They saw that the form submitting to that allowed one to enter an email address and some tracking numbers to have a status email sent back to that address. Using "view source" in the browser, they observed that the form actually called a Javascript form which submitted the form to TrackEmail.asp?nav=TrackByEmail. Since this was on a page at http://track.dhl-usa.com/, they realized they'd need to submit to http://track.dhl-usa.com/TrackEmail.asp?nav=TrackByEmail.

They also noticed that the form had two input fields, txtTracknbrs and txtEmailAddress, so they tried to put together a CFHTTP request that tried to submit to the form using this URL:

http://track.dhl-usa.com/TrackEmail.asp?nav=TrackByEmail&txtTracknbrs=nnnnnnnnn&txtEmailAddress=someemail@address.com

It didn't work. They asked if anyone knew why.

My Response: He was close, but needed to understand a couple of points

Here's what I wrote:

I think your problem is in presuming that the Javascript method builds a request that looks like the one you tried in CFHTTP. I just reviewed the page, and I see that it's a form, with a METHOD=POST, and so your CFHTTP needs to send its info in the way a form post would. And that's NOT by passing the variables on the URL as you have. (One may argue that they can write CFML apps that accept either FORM or URL variables, but the page being called is an ASP page, so we can't make that presumption.)

With that, I changed your CFHTTP to the following and it worked (you should be getting an email soon, as it told me the request was accepted). Try it yourself:

<cfhttp url="http://track.dhl-usa.com/TrackEmail.asp" method="POST">
   <cfhttpparam type="URL" name="nav" value="TrackByEmail">
   <cfhttpparam type="FORMFIELD" name="txtTrackNbrs"
value="nnnnnnn">

   <cfhttpparam type="FORMFIELD" name="txtEmailAddress"
value="someemail@address.com">

</cfhttp>

<cfoutput>
#cfhttp.filecontent#
</cfoutput>

Note that it uses a TYPE="URL" for the nav query string value (as that WAS passed on the URL), but uses TYPE="FormField" for the form fields, since those are passed as form fields. You were right in getting the exact names of the form fields as used in the form fields of the form. That's critical as well.

I'll point out as well that when trying to mimic such forms in CFHTTP, it's also sometimes critical to send along any hidden form fields. There were two on that form, but I left them off and it seems to work. IF you found you needed to add them, just add them as more TYPE="FormField" values.

I should note as well that sometimes when trying to simulate a form submission, you will also need to send along any cookie values that might be being sent by your browser to the server. Those aren't shown in the form. You could use browser/server proxy tools (like the free Fiddler or Firebug tools) to detect what's being sent along.

Hope that may help other readers.

I'll add in this blog entry that regarding that last point, about using HTTP proxies to study the browser/server communication, I wrote about those previously in:

Alternatives HTTP debugging proxies, for debugging

Programmatically accessing allowed IP Addresses in Developer Edition (and understanding the limits)

Have you ever needed to know programmatically what IP addresses, besides localhost, ARE allowed to access your Developer Edition of ColdFusion? I'll show you how here.

The message above does show the allowed IP addresses, but what if you need that list programmatically for any reason? Since I couldn't find it discussed anywhere, and I discovered the feature today, I just figured I'd share it in case it helps anyone searching in the future.

Along the way, I also explain for any who need to know, both what the "allowed IPs" limits are and how they've changed in 5, 6, and 7, as well as how to reset the allowed IP addresses.

Background

Most folks know that the Developer Edition of ColdFusion is intended for just local (as opposed to production) development. But to be specific, it's limited not JUST to requests from your localhost but both localhost and 1 other IP address (in CFMX 6 or 6.1), or localhost and 2 other IP addresses in CFMX 7.

Once that limit is reached, if you or anyone else tries to access a CF page from an IP address other than localhost or that 1 (in 6/6.1, or 2 in 7) permitted to that point, the user will get a message:

A License Exception has been thrown.

You tried to access the developer edition from a disallowed IP (nnn.nnn.nnn.nnn). The developer edition can only be accessed from 127.0.0.1 and two additional IP addresses. The additional IP addresses are: yyy.yyy.yyy.yyy,zzz.zzz.zzz.zzz

Of course, in a real message the nnn, yyy, and zzz would be real numbers, and in CFMX 6/6.1 it would list only "one additional IP address".

But the key point for the purpose of this blog entry is that the end of the message lists the IP address(es) that have "gotten in", such that any requests from other IP addresses will fail to run and will get the error message.

So it's sometimes useful to know the list of allowed IP addresses, but what if you need to access it programmatically? And how do you reset it to allow other IP addresses to get in?

Accessing that List of Allowed IP Addresses Programmatically

If you have need to know the list of allowed IP address(es) programmatically for any reason, the following code using the undocumented and unsupported ServiceFactory will show it:

<cfobject action="CREATE" type="JAVA" class="coldfusion.server.ServiceFactory" name="ServiceFactory">

<cfoutput>#ServiceFactory.LicenseService.getAllowedIp()#</cfoutput>

Unfortunately, there is no equaivalent to this in CF Admin API (in CF 7 or 8).

Resetting the Allowed IP Address(es)

Finally, some have wondered how they can reset the list of permitted IP addresses. Just as in CF5 (where you could access the developer edition from any 1 IP address, either localhost or another), in CFMX 6.1 and 7 you just need to restart the server. That will clear the list of allowed addresses.

In CFMX 6.0, however, you needed to actually edit a license.properties file while the server was stopped (as I blogged about back in 2002. Thankfully, that was fixed in 6.1.

For more information on that, as well as for a documented reference of the localhost/localhost+1 licensing in CF 4.5/5/6.0/6.1, see http://www.adobe.com/cfusion/knowledgebase/index.cfm?id=tn_17832.

For documented reference to the localhost+2 support in CF 7, see the bottom of the table at http://www.adobe.com/products/coldfusion/productinfo/product_editions/#s2.

Of course, the error message itself also clearly explains it in CFMX 6 and 7.

Reloading CF web services programmatically, using the CF7 Admin API

I'm surprised to not see much out there about how to reload or refresh CF's cached WSDL proxy for calling a web service, at least programmatically using the new CF 7 Admin API. Perhaps it's because people have been tripped up, or simply haven't explored it. Either way, I'd like to offer here the code you need, and also point out some tips and traps.

Update for CF8: As an update to this entry from 2006, which is focused on CF7, I'll note that there was yet another approach that was added in CF8. Both still work and have their own value. More on the CF8 feature in my later blog entry. Still, the feature added in CF7 is important to understand, too, so please read on.

Introduction: Why You Would Want to

As background, someone reported having a problem calling a web service from CFML, and a solution suggested was that the person reload or refresh the web service using the CFMX Admin console (in the "Data & Services" > "Web Services" nav bar tab. There, you'll find any web services that have been called from CF (which is often a surprise to folks that they're tracked there). For each listed web service, there are 3 buttons and the middle one does a refresh (which means it goes and grabs the WSDL and builds a local java proxy stub, which is used when you then invoke the web service).

But one may then wonder how to do that programmatically, without having to open the Admin console. In CFMX 6.1, you could use the undocumented ServiceFactory, as I'll show below to those still using that. But since that's being deprecated, you really ought to learn the new CF7 approach.

The CF 7 Admin API Approach

As of CFMX 7, we are expected to use the new Admin API, a set of CFCs provided with CFMX 7, which offer a formal, secured API for accessing functionality otherwise offered in the Admin console.

So how would one do that to refresh a web service? Well, there is an extensions.cfc in the Admin API (a set of CFCs in the webroot's /CFIDE/adminapi/ directory), and it has a reloadWebService method that's just the trick. How did I know that? You can call the built-in CFC documentor by browsing the CFC directly:

http://<em>[servername]</em>/cfide/adminapi/extensions.cfc

Before you can call that, though, note that you do need to "login" to the Admin API by calling the login method of the administrator.cfc first. (Check out its docs to learn more.)

But to save you that effort, here's some code. (As always there are several ways to call a CFC and its methods (using CFINVOKE or CFOBJECT, or createObject either within CFSCRIPT or not), but here's at least one approach.):

<cfset createObject("component","cfide.adminapi.administrator").login("youradminpw")>
<cfset ws = createobject("component","CFIDE.adminapi.extensions")>
<cfset ws.reloadWebService(name="<em>webservicename</em>",path="<em>WSDLurl</em>")>

Note that you need to specify your own admin password in the first line, and in the last line you need to specify a web service name and its WSDL URL.

What's with this notion of passing in a web service "name"?

As you contemplate that code, you will certainly know what the WSDL URL is, since it's the same one you'd use in a CFINVOKE or CFOBJECT/createobject call of the web service itself. But what's the "name" requested here? Well, that can trip you up and it deserves further discussion, as it has several ramifications as I'll explain here.

The name is the name shown in the Admin console for the given web service. The trick/trap is that if you never open and change the Admin console entry for this web service, then the name will simply be the same value as the WSDL URL. But there's more to understand.

First, if you didn't know it, one can edit that "name" in the Admin console, and then one can even use that "name" as an alternative (or "alias") to the web service WSDL URL when invoking the web service from CFML. That's a whole separate subject which I've covered in user group talks in the past.

But assuming that no one has modified the web service name (or for reasons I'll explain in a moment, if you are not using such an alias name when you invoke the web service), then you can presume the name and WSDL URL to be the same. As an example, one could change the last line above to:

Getting Web Service Names Programmatically

Now may wonder, "can I get the web service name programmatically?" You can. But here's where it gets a little confusing. There is an available getwebservices method of the extensions.cfc. And according to the docs, you can either pass in the "name" of the webservice, or leave it off to get all web services. If we don't know the name, then we may think we'd want to use the latter approach. But I find that it doesn't quite work as straightforwardly as it seems.

First, I tried calling getwebservices() without a name:

<cfset createObject("component","cfide.adminapi.administrator").login("youradminpw")>
<cfset ws = createobject("component","CFIDE.adminapi.extensions")>
<cfdump var="#ws.getwebservices()#">

Sadly, it returned an empty dump as a result. Yet I had several web services listed in my admin console. Here's the thing: none had a name. I then renamed one of them (to "test"), and tried it again, and suddenly the call did return an array of structures (1, in my case) with the name and WSDL URL.

Hmm. So it seems instead that the getwebservices() ought perhaps instead be named getnamedwebservices(), since it only returns web services whose names have been changed (been given an alias).

Still, though, if I do pass in a name, as the docs suggest, then I do indeed get the same result:

<cfset createObject("component","cfide.adminapi.administrator").login("youradminpw")>
<cfset ws = createobject("component","CFIDE.adminapi.extensions")>
<cfdump var="#ws.getwebservices("test")#">

Now, you may wonder: "if I can't a listing unless I know the name, or can only get a list of 'all' of them if they are named, then how might I get the info for ones that have no name or whose name I don't know?"

Good question. And guess what I've found? A couple of important things.

First, I've found that you can also pass in the WSDL URL, even for a web service that's been renamed with an alias like "test", such that this works:

<cfset createObject("component","cfide.adminapi.administrator").login("youradminpw")>
<cfset ws = createobject("component","CFIDE.adminapi.extensions")>
<cfdump var="#ws.getwebservices("http://ws.invesbot.com/stockquotes.asmx?WSDL")#">

Again, that's not the web service name but the WSDL URL. And the resulting dump shows the name and that URL. So the API docs on this are a little misleading.

Be sure to refresh the "right" webservice: the one you'd really try calling

But perhaps a more important thing is that I found that you can have a web service entry for the SAME WSDL URL but with different names/aliases. Why is this important? Because you want to make sure you refresh whichever one you're using.

This goes back to my point above when I introduced the refreshWebService method: you need to give it the name as YOU call the web service, otherwise you'll be refreshing a different proxy stub and won't see the benefit you expect.

If you use the WSDL URL when you invoke the web service, then that will create a proxy stub with that "name", and therefore you want to use that as the "name" when you refresh it.

If you rename a web service in the Admin console, and then use that when you invoke the web service, then you want to use that as the "name" when you refresh it.

Refreshing Web Services Using the ServiceFactory

Since some folks reading this may not have moved to CF 7, let me show how you could do the same using CFMX 6/6.1 (or indeed 7, since it still works there). It's important to note, however, that using the ServiceFactory is not only not documented but it's also not supported. It has security problems (since there's no need to provide the admin password as you must in the Admin API). Also, it may eventually be obsoleted or otherwise restricted.

Still, since it's been documented by others in the past and is readily available on the web, I'll offer it here:

<cfobject action="CREATE" type="JAVA" class="coldfusion.server.ServiceFactory" name="ServiceFactory">
<cfset ServiceFactory.getXMLRPCService().refreshWebService("<em>webservicename</em>")>

Now, you'll notice that I've indicated that the value passed into the refreshWebService method is the webservicename. That's because it works just like the Admin API reloadWebService discussed at the top here. Be sure to specify the name as you would use it in the invocation of the web service (whether an alias or the full URL).

How to Confirm That Refreshing is Working

As you try these three approaches (Admin button, Admin API, and ServiceFactory), you will probably benefit greatly from being able to see for sure that the refresh/reload is updating the java proxy/stub class files. Where do you find them? They're stored in a "stubs" directory under your CFMX install directory, such as C:\CFusionMX\stubs or C:\CFusionMX7\stubs.

Then, under that, you will find directory names such as WS141836989, or a name that's the same as the alias/name you give to a web service in the Admin console. Inside those directories you will find other subdirectories, eventually finding some that hold the .class files representing the objects available in the web service. It's those .class files whose date/time stamp you want to see changing when you do a refresh/reload.

I'll note that there's no mapping or indication of that WSnnnn directory name, to know which one holds the web service you're interested in. I guess you just have to find the right one by looking for one whose class names map to the web service object you're calling. (If anyone knows a better connection, please do share it.)

Finally, I think it may be worth clarifying that when you do a refresh/reload of a web service using the approaches above, you need to have first made a call to that web service from CFML (or entered it manually in the admin console).

Hope all that's helpful.

Summary of Notes for Adobe Folks

Before I end, in case any Adobe folks are listening, here is a restatement (and expansion) of the couple of observations I made about wrong or undocumented functionality. This is for both the docs group and the engineers, since this is also about internal API documentation returned by the CFCs and their functionality:

  • getwebservices() only returns web services that have a name that was changed, whereas the API doc says it returns "all web services", so it either is working incorrectly or ought to be called getnamedwebservices instead
  • though not documented, getwebservices("wsdlurl") works also. The API docs say that it should only take a name (and I tested this against a web service where I had renamed it, so it was not getting it "by name")
  • if you do consider renaming the getwebservices() method to be called getnamedwebservices(), you might then also want to rename getwebservices(name) to getwebservice (singular), since it just gets one webservice
  • it would be nice to be able to refresh ALL webservices that use a given WSDL URL at once, perhaps by new method that accepts URL rather than name (and works for all occurrences of that URL in the cache, whether named with an alias or not)

Are you using CFCOMPONENT NAME? I'll argue that you should not be. Here's why

Are you using CFCOMPONENT NAME? I'll argue that you should not. Here's why. Someone asked me what I knew about the CFCOMPONENT NAME attribute. They were finding it being used but couldn't find any explanation of its purpose. I'd never really noticed it before, but as I started researching it, I can see some sources of potential confusion. From my testing, I'm going to propose that it's not something you should be using at all. It serves no purpose and can instead lead to confusion. I realize that's going to surprise those who use it, and perhaps it will cause a stir, but let me make the case, and you can decide for yourself.

Supported versus Unsupported Attributes on CFCOMPONENT, and GetMetaData()

First, there's no NAME attribute listed for CFCOMPONENT in the CFMX docs. If one wants to give a friendly name or description to a CFC, that's the purpose of the DisplayName or Hint attributes instead.

More important, whenever you use any attribute on CFCOMPONENT that's not otherwise supported, the only programmatic motivation would seem to be to take advantage of the fact that a GetMetaData() call against that CFC will return that attribute as a key in its result.

But it turns out there is ALREADY a getmetadata key called "name" that's always returned from a getmetadata call against a CFC, and it always returns an internally generated value indicating the name and location of the CFC, and more to the point, this value returned is NEVER affected by your use of a NAME attribute on CFCOMPONENT.

Curiously, I do find that different versions of CF and BD return slightly different values for that (more on that in a moment). And for those who may wonder, neither the use of NAME nor any unsupported attribute affects the output of the CF Component browser.

That's why I'll argue you shouldn't be using NAME at all. It not only does nothing (is not returned by getmetadata), but it may lead someone to be confused that it should. I don't know if I will go so far as to propose that CF (and BD) should raise an error when it's used. Sadly, a google search for "cfcomponent name" shows hundreds of pages using it (and those aren't real code running on people's servers but instead code people are sharing on the web for demonstration purposes), so restricting that could have a lot of impact.

If one wonders how people started using it, I'll note that (sadly) the CF7 docs user guide section on "documenting CFCs" does shows using the NAME attribute. I will leave a comment there pointing to this blog for folk's consideration (and perhaps to get Adobe to remove that, if all this observation holds true).

Further, the person asking me about this did so because they noticed that the CFEclipse CFC wizard also implements the NAME attribute in CFCs it creates. Again, unless comments to this suggest otherwise, it seems that perhaps that should be changed.

I am open to hearing from others. It sure seems that all is as I see it, but if I've missed something, I welcome comments below. If there is any egregious problem or misunderstanding I've created, I will offer a follow-up entry to correct things.

What getmetadata's NAME key returns on different CFML engines

I mentioned above that if you use GetMetaData against a CFC instance, it does return a NAME value. What does that value hold? Well, the docs for CFMX 6.1 simply indicate that it's "the component name", with no further explanation. The CF7 docs reference on GetMetadata does elaborate further, now saying that it's the "Component name, including the period-delimited path from a component search root such as the web root or a directory specified in the administrator Custom Tag Paths page."

I mentioned before that I'd found differences between the two versions of CF as well as between editions of BlueDragon. Here's a demonstration. I setup a simple test hello.cfc file in a /demo directory off my webroot:

<CFCOMPONENT name="hellox">
<CFFUNCTION NAME="GetHello" ACCESS="REMOTE" RETURNTYPE="string">
<CFRETURN "Hello World">
</CFFUNCTION>
</CFCOMPONENT>

Note that in the CFC I have set the value of NAME to "hellox". I did also test it using simply "hello" or "hello.cfc", and it made no difference at all in the following results. I then called it with a template that used Getmetadata:

<cfobject component="hello" name="hello">
<cfdump var="#getmetadata(hello)#">

The dump of the getmetadata result always showed a NAME key and it was always the same whether I provided one or not, and regardless of what value I provided. But there was a slight difference among the engines, which showed the following values for the getmetadata name key:

versionvalue
CFMX 6.1hello
CFMX 7.02demo.hello
BDJX 6.2.302 (JX)demo.hello *
BDJX 6.2.302 (.NET)demo.hello

The big difference is that CFMX 6.1 does not show the location of the CFC as found, just its name. Both CFMX 7 and BD show the directory name in which the CFC was found, as explained in the CF7 help above. This value can be useful for times when you wonder which CFC you have found on an invocation (since both CFMX and BD have a variety of search locations that they use), although note there is also a "path" key returned by getmetadata which returns the absolute rather than relative path.

You'll note I've put an asterisk next to the BDJX test. To do this testing among the 4 engines, I'm running on Windows XP with IIS, so I can't have multiple web sites. (Yes, I know about tools to get around that and that Apache would let me have multiple sites.) Instead, I setup multiple virtual directories, each pointing to the different engines (as I document in a CFDJ article), so that I could call the page invoking the CFC using each of the different versions of CF and BD. On my laptop, I use virtual directories named _cfmx, _cf7, _bdjx, and _bdnet (so calling my demo page above might be http://localhost/_cf7/demo/invoke_hello.cfm.) I was surprised to find that in BDJX (only), the resulting name key returned did also show the name of the virtual directory as a prefix to the value returned, so the getmetadata name value returned by BDJX in my example was _bdjx.demo.hello. Of course, you won't see that difference if you don't use virtual directories.

Those who delight in the discover of the new value returned in the pre-defined getmetadata name key may find the above details useful. The more important point, again, is simply that the use of a NAME attribute on CFCOMPONENT has no programmatic purpose at all. It's not a supported attribute, and unlike any other unsupported attribute, it's not returned by GetMetaData, so it seems it should not be used at all. Let me know if you think otherwise.

Simplifying the captcha graphic in Lyla Captcha (and BlogCFC)

Wish you could simplify your captcha's? If you use Peter Farrell's Lyla Captcha, as I do because it's embedded in Ray's BlogCFC, I'll show a few quick changes you can make that will make them much easier for your users to read.

Sound counter-intuitive? Aren't captcha's supposed to be difficult to read, to hamper spammers? In my last entry, I made a call for simplifying captchas and why they aren't all bad. As a blog owner who uses them to weed out the random spambots who would otherwise clog my comments and feedback mechanisms, I like captchas, and I'm grateful for the work Peter's done.

That said, I have to admit that as I've encountered them in the blogs of others, I've grown a tad weary of their complexity. They require the user to type several characters and have several swirly ovals, random lines, and a wavy background. Frankly they're quite hard to read, and it would be a shame to lose commenters for that reason.

hard captcha

Again, the intent is to make it hard for some spammer to scan the captcha request somehow and figure out what's being requested so as to automate around it. Fair enough, but as I said in my last entry I'm really not that concerned about protecting my site from determined break-ins. I'm not a bank. I just want to keep out the automated pests.

With just a couple of changes to Lyla's captcha.xml file, you'll have a much simplified captcha, if you want one.

hard captcha

Lyla is highly customizable

On a lark, I decided to try to find out if Lyla might just be modifiable to dial down the intensity. Turns out it is, by simple changes in the lyla captcha.xml file, as documented in this PDF. Thanks again, Peter! :-)

After a few simple tweaks, I reduced my captcha to just asking for 3 characters, all lowercase, without all the swirly ovals, lines, and wavy background.

Changing Lyla's captcha.xml

In BlogCFC, the captcha.xml file is located in blog\client\includes (or just \includes if you've installed the blog client directory as your webroot.)

To effect the change I wanted, I ended up with the following values for the following entries. Again, see the docs for more info:

<config name="randStrType" value="alphaLcase"/>
<config name="randStrLen" value="3"/>
<config name="fontColor" value="dark"/>
<config name="backgroundColor" value="light"/>
<config name="useGradientBackground" value="false"/>
<config name="backgroundColorUseCyclic" value="false"/>
<config name="useOvals" value="false"/>
<config name="useBackgroundLines" value="false"/>
<config name="useForegroundLines" value="false"/>

You can change them to suit your taste. Note that if you do change the randStrLen, the value selected represents the "average" length of the string that users will be asked to enter, and may vary by +/- 1 from that.

Make the changes, and check 'em out for yourself. Note that with Ray's BlogCFC, you need to reinitialize the blog (add ?reinit=1 to your blog URL) to see the changes. What I did was had one browser page open to do that, and another sitting on a blog comment form. After running the reinit, I could then just reload the comment page to see the impact. (If there's a still-simpler way to test changes to the captcha.xml, let me know.)

If you don't use BlogCFC, then you have to re-instantiate the captcha object after making changes to the XML file. If you've stored it in a shared scope (like application), you need to run some code that reloads it. Of course, restarting ColdFusion will also reload the CFC in whatever scope you stored it in.

Conclusion

Making these changes won't solve the accessibility problems some have with captchas, and it certainly could increase the risk of a determined spammer more easily breaking your captcha. As I said in the last entry, I doubt that's a real concern for most of us. If it proves to be so, then you can dial the intensity back up.

I just want to keep from annoying my readers, and I hope others will consider these changes to keep from annoying us all. :-)

PS: I do realize that one could skip the captcha graphic entirely and just go to prompting the user for a random string. That may just a bit "too" easy for a spambot to get around. To each his own.

Thanking Peter

One last note: while Peter certainly appreciates your kind comments (and do share them, as I'm sure many don't bother), those who REALLY appreciate his work should note that he gratefully accepts contributions by way of his Amazon Wishlist or you may may make a donation with PayPal, using his address, pjf@maestropublishing.com.

Captchas: making them simpler, and dialing down the angst against them

Most by now understand what captchas are. Some love 'em, some hate 'em. I want to dial down the rhetoric some with this perspective: as a blog owner fighting frequent spam in comments and trackbacks, captchas (in some form, not necessarily a graphic) have their place to keep out spambots, and they can indeed be simplified (even the graphics ones) and at no loss of benefit. My bottom line: I don't use them as a double-key deadbolt lock to keep out intruders, I just use them as a screendoor to keep out random pests.

If you use Peter Farrell's Lyla Captcha, which I use because it's embedded in Ray's BlogCFC, in the next entry I'll show a few quick changes you could make in the Lyla captcha.xml file to make them much easier to read, going from this
hard captcha
to this
simple captcha.

Before that, I just want to expand on those thoughts above on the general angst against captcha's, and why I think it's ok to make them easier to read.

The Haters

I realize that some have gone to great lengths to decry captchas primarily because they are not "accessible" (to those using screenreaders), though audio ones help solve that.

Others simply hate them because they're too darned difficult to read. I've surely seen that, even in the ones created by default in Lyla (thus my next entry on addressing that).

Now, while most use a graphic that a user must read, it's not the only approach. As the previous link discusses, other approaches include simpler approaches like asking the reader to add some numbers or answer a question (that only a human could reasonably do).

But the other complaint is that they give those who use them a false sense of security, because they can be easily broken, even the graphic ones.

But my Blog is Not a Bank

Here's the thing: my blog is not a bank. While the difficulty in breaking a captcha may be important to a bank or commercial site trying to use them for authentication, I just want to make it hard for an automated spambot to post crap in my blog comments and trackback forms. If you have any similar king of input form on a publicly accessible site, you may suffer similar problems.

I really can't believe anyone would go to the lengths of scanning and breaking the captcha on my site (random as it is) to get a crap spam comment into my lil' ol' blog. And some of the comments are just nonsense; it's not like they're trying to drive traffic to another site or something--so the popularity of my (or your) site isn't the issue. It's just the annoyance factor (both to me as I get notified of comments and to readers who would have to sift through them if I didn't delete them as I do now).

Having made the case for why a simpler captcha may suffice for some purposes, in the next entry I'll show how to control the degree of difficulty in reading them for captchas built using Lyla Captcha.

More Entries

Copyright ©2017 Charlie Arehart
Carehart Logo
BlogCFC was created by Raymond Camden. This blog is running version 5.005.
(Want to validate the html in this page?)

Managed Hosting Services provided by
Managed Dedicated Hosting