[Looking for Charlie's main web site?]

Version numbers of libraries underlying ColdFusion 2016

Are you wondering what updates (in terms of version numbers) have been made to the libraries underlying CF2016?

For instance, what's the version of Solr? (It's considerably upgraded over what was in CF11 and 10). What about Java, Tomcat, Hibernate, Hibernate, Quartz, jQuery, and so on?

In this post, I offer a rundown of what seem the most significant libraries and their versions, as deployed in the first release of ColdFusion (2016 Release). I also explain how I found these version numbers, which isn't always obvious, in case that may help others. But I also offer some commentary on why this is an important issue to some, and I offer some counterpoints to the demands some have that every library should always be the absolute latest version (and why that's just not practicable).

If you just want the version numbers without the "waffle", look for the bulleted list of them below. :-)

[....Continue Reading....]

Solving seeming ColdFusion / MySQL 5.6 incompatibility, by updating CF's MySQL driver

If you're running ColdFusion 10 (and perhaps also CF 9 or earlier), you will find that if you update your MySQL installation to version 5.6, you'll get the following error from any SQL you try to run from CFQUERY (and perhaps other CF querying tags, like CFSTOREDPROC):

You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'OPTION SQL_SELECT_LIMIT=DEFAULT'

Summary: There is a reasonable explanation and a rather simple solution: update the MySQL driver that CF is using to at least version 5.1.22 of the driver, the first to support MySQL 5.6, because the one built into CF 10 (driver version 5.1.17) not only does not. More important, that older driver uses something that causes the failure above in 5.6.

That explanation of the "solution" may be enough for some to take the ball and run with it (and if not, I will offer more details on how to do that), though it should be noted that updating the driver is not formally supported, nor is MySQL 5.6 technically supported at all in CF10 (or 9). But for those who will press on knowing that risk, you now know what you need to do.

But as often, there's much more to this than meets the eye, so I hope you will follow along to learn more. I have broken this into two parts:

  • the problem (with what I hope is helpful explanation of what the real root of the problem is),
  • who's to blame (not Adobe, I will argue)
  • and the solution (with some caveats that even experienced folks, or those who don't care about "the problem", should still read.

And again, while I discuss this in the context of CF10, where I've seen the problem happen, it could apply also to CF9 (and it seems reasonable that it would), so all the information still applies, it would seem.

[....Continue Reading....]

New for CF9 (and 9.0.1): a query timeout that may really work, with a caveat

This is a very interesting change in CF9 (and 9.0.1), which has slipped under the radar for the most part it seems.

Did you know there is now a setting in the DSN page of the CF Admin (for most of the Adobe-provided DB drivers) which allows you to set a maximum timeout for queries against that DSN?

It's a new feature enabled for the DataDirect drivers, as updated in CF 9. (You will not see it if you use an "other" datasource type, such as when using a downloaded JDBC driver that you implement on CF.)

The caveat? This timeout is ONLY settable there in the DSN definition, not in CFQUERY (or CFSTOREDPROC) itself, which is a shame. The existing TIMEOUT attribute for those is not the same and generally does not work. Still, the value of this even at the DSN level is too important to ignore for some challenges. More on that (and some other thoughts) in a moment.

[....Continue Reading....]

New DataDirect JDBC Type 5 drivers (for SQL Server, MySQL, Oracle, and more)

I think most folks know that the underlying database drivers in CF are from DataDirect. Well, they've announced new "Type 5" drivers. While you would have to buy and install them separately from those built-into CF (for now, as Adobe has not yet certified CF for use with them), I think some people may want to give them serious consideration even before then.

Several Performance Advantages, and Failover As Well

[....Continue Reading....]

Spying on ORM database interactions: Hibernate, Transfer, etc. on any CFML engine

As people use CF9's ORM feature (or other ORMs like Transfer and Reactor, or indeed Hibernate, on any version of CF6+ or indeed any other CFML engine), they may be left wondering what sort of SQL interactions happen "under the covers" between the ORM framework and the database engine (whether in a given request, or perhaps at startup of CF).

Well, there are several ways you can watch them, as this entry will discuss, and some may be better suited to the job than others. It can be very interesting to discover what's going on, especially if you're having any suspected performance problems which you think may be related to ORM processing (or just if you wonder what all it does for you).

As for spying on the SQL, of course ORM support is just a different way that the CFML engine (through the ORM framework) sends SQL to a database via a regular DSN, just like any other request, so there's nothing really "tricky" about this. It's just about realizing that while you don't write the SQL yourself, it's still generated by the CFML engine/ORM framework, and you may not realize/consider the available tools which can spy on it, just like any other DB processing from within CF. Indeed, some people may not even realize how many options exist to spy on JDBC interactions from their CFML engine to the database engine.

The good news is that there are several approaches, some included in CF (some depending on the edition), and some available separately which would work in any edition of CF or the other CFML engines (Open BlueDragon, Railo, etc.), and with any of the ORM frameworks. And again, some may be better than others for certain challenges.

(FWIW, besides the aforementioned Transfer and Reactor, there are still other ORM solutions for CFML, which I mention in my CF411 list as CFML ORM Frameworks. Indeed, note that you can run Hibernate on CF prior to CF9, if you want to. This is a recovery of a blog entry that no longer exists, recovered via archive.org.)

Built-in ORM Logging Option

First, note that for those using CF9+ ORM, there is indeed a built-in option in the CF ORM setup where one can enable logging, settable in the application.cfc: see the this.ormsettings option and its available key/value pair, logSQL="true".

There are several resources where you can learn more on that (and a related log4j property file approach to logging this). Besides the CF9 docs page on the ORM settings, there is also a blog entry by Adobe engineer Rupesh Kumar.

The default is to log this information to the console, but you can manipulate those log4j settings to tell it to use a file (see the links above). Even so, this will result in quite a lot of data being logged, which you will then need to connect back to your specific requests. The following approaches may be preferable.

Using FusionReactor or SeeFusion

Users of any CF edition (6+) or any CFML engine (Railo, OpenBD, or BD 7+) can use tools like SeeFusion and FusionReactor, which have always had the ability to monitor database interactions by "wrapping" the datasource to be monitored. FusionReactor engineer John Hawksley has posted a recent article specifically on monitoring CF9's ORM interaction, in the FR Devnet site, Using FusionReactor's JDBC Driver Wrapper With ColdFusion 9 ORM. Its concepts would apply to any ORM, of course.

Similarly, I've written generically about FusionReactor's database monitoring feature in What is the FusionReactor datasource monitoring feature? Why would I use it? Powerful stuff. As I point out in that article, the concepts discussed apply as well to SeeFusion's ability to monitor queries by wrapping datasources.

That said, it's worth noting that FusionReactor does have a couple of advantages, in that it provides for the display of all queries for a given request (while viewing the details of that request), whereas SeeFusion only lets you see the slowest query in a given request. FusionReactor also provides a separately available display of all the slowest queries (across all requests). It also logs every query (connecting it to a given request as well), while SeeFusion (Enterprise, at least) can also log the slowest queries to a database.

And note that both of these track any requests coming out of CF, not just those associated with a given request. So if there is ORM SQL that is associated with the startup of CF, that's tracked too. (And for those aware of issues with CF's Client Variables, such DB activity is also tracked, even that done by the hourly purge, which takes place on a background, non-jrpp thread.)

CF Enterprise Server Monitor

Those running CF 8 or 9 (Enterprise only) will find that its available Server Monitor does offer built-in monitoring of the SQL executed against CF datasources, at least, as long as you enable "Start Profiling" (which also enables other features, and overhead, as well). In this way, the Enterprise Server Monitor can monitor database interactivity, including ORM interactions.

Unlike FusionReactor (and like SeeFusion), it focuses only on showing queries that exceed certain limits, and at that it shows them only in a "Slowest Queries" interface, tracking the slowest queries among all requests. The CF Enterprise Server Monitor also has no logging ability at all.

Being able to see every single DB interaction for a given request (or across all requests) may be all the more interesting for discovering/observing what's happening with ORM interactivity.

Another alternative CF feature

Still another little-known feature for spying on JDBC interactions in CF is by way of the JDBC "spy" feature, which does in fact allow logging of all JDBC interactions mde from within CF. This feature was first enabled by way of the DataDirect 3.5 driver update which was made available (as an optional upgrade for 6 and 7) in the CF 7.02 timeframe. I wrote about the Spy feature back back in Aug 2006.

Since then, CF 8 (and now 9) offer it instead as a new "log activity" option in the "advanced settings" for a datasource definition in the CF Admin (which is disabled by default). I pointed this out in another entry from 2007 as one of many easily missed changes for the CF 8 Admin.

This "log activity" output is not as easy to interpret as FusionReactor's logs, and can indeed be voluminous (moreso than FR's), so be careful. Anyway, it's one of the several ways you can monitor JDBC interactions between CFML and your DB engine. Again, any of these may be useful for monitoring any of your CFML/database interactions.

Generic DB Monitoring tools

Indeed, it's worth noting finally that while the focus here has been watching the DB interaction from CF (and the ORM framework) to the database (by watching the JDBC traffic going out of CF and returning), you could just as well watch the DB interactivity from the DB's perspective instead (watching it coming and and being returned).

There are many tools that can monitor database processing, available for each of the major databases (free and commercial). I list several such tools in one of my CF411 section, Database/SQL Monitoring Tools.

Hope all that's helpful, whether you use ORM or not.

Determining Your JDBC Driver Version: Pick from 3 ways

Following onto my 2 previous entries related to updating JDBC drivers in CFMX 6 and 7, some readers may appreciate a little more help on the matter of determining just what version of the drivers they do now have installed.

This news has been shared in various blog entries over the past couple of years. I'd like to reprise them here for your edification. Just pay close attention to whether you mean to have them look at CFMX or CFMX 7. They each point to a directory such as C:\CFusionMX\lib or C:\CFusionMX7\lib. Make sure you use the value appropriate to your configuration:

Hope that helps someone as much as it did me.

More on the new SpyAttributes feature of the"new" 3.5 JDBC drivers for CFMX

In my last entry, I mentioned briefly the new "JDBC Spy" feature that's available in the new 3.5 JDBC drivers for CFMX. I want to point out some more info for those interested in investigating the Spy feature further.

First, notice that there is a site at the DataDirect site (from whom Adobe purchases the JDBC drivers):

http://media.datadirect.com/download/docs/jdbc/alljdbc/reference/spy.html

It discusses some additional attributes. And still more useful info is offered at another page on their site:

http://media.datadirect.com/download/docs/jdbc/alljdbc/reference/diagnostics2.html

This actually shows what the log of info would look like, if you want to decide first if its info is worth the bother setting up.

This other page also talks about ways to turn the spy logging on and off at runtime, using Java method calls. Perhaps someone with more familiarity with the underlying Java classes for these drivers can translate the code there into something we can use in CFML. (It's late, and I don't have the energy.)

As such, be aware that the spy feature could generate a lot of data, so you may want to consider creating two datasource for each DB you may want to monitor. Either rename them to enable whichever you want to be in effect, or change if your code lets you specify a DSN via a variable, you can change the value to the "spying" version when you need it.

I did test and determined that also you can just add/remove the connection string from the end of the JDBC URL (see the technote for more), and immediately the logging stopped against the datasource. As a trick, I stuck the connectionstring value into the "description" field of the datasource definition, to keep it readily accessible if I wanted to add it back. (See my previous entry for a warning about putting it into the ConnectionString field instead.)

Finally, I should clarify that this Spy feature is not to be confused with the much older (and more powerful) p6Spy project, which has gone from open source to commercial to kind of stagnant. It still has great use, and in fact has a useful related SQL Profiler that's worth looking at. Still more at http://www.p6spy.com/, and maybe I'll write more about it later.

Did you see the 3.5 JDBC update for CFMX 6.1 and 7: Anyone tried it? Seen the new features it adds

Had you noticed the JDBC updater at http://www.adobe.com/cfusion/knowledgebase/index.cfm?id=42dcb10a? It applies updates to both the Merant DataDirect and Sequelink (ODBC) drivers.

I have a few questions/observations for others who have run it or may consider it, and they apply to 7.02 and 6.1 users, such as whether you need to apply it manually and about the various new features it adds, like integrated NT authentication for SQL Server, and a new JDBC Spy feature for any DBMS? More on them in a moment.

Do you need to update this for 7.02? 6.1?

First, do you need to apply the update manually if you're running 7.02, or even one of the later cumulative hotfixes (CHFs)? It doesn't say specifically. While it may seem we should presume it was included in 7.02, I think we should not. Some features it adds (below) are not in the C:\CFusionMX7\lib\macromedia_drivers.jar (whose date is also from 2004, at least on my 7.02 install.)

(UPDATE: In fact, it appears that the updated drivers were NOT in fact included in 7.02, or any CHF. First, as mentioned in the comments below a discussion on the Adobe Forums confirmed that these drivers were NOT included in 7.02. Also, note that CHF 1 for 7.02 lists the link for the driver updates as something you need to do yourself, under "additional information". There's also no mention of the JDBC drivers in the technotes for 7.02 CHF's 2 or 3, so it seems safe to conclude that 7.02 users do need to apply the update individually to get the updated JDBC drivers.

But better still, if you have 7.02 installed and wonder what version you have, you can find out for yourself. Check out this later entry of mine which helps you determine what version of the DB drivers you are indeed now running.)

As for 6.1, it's not clear but do we need to apply it if we have the latest 6.1 updater (6,1,0,83762 )? Again that build's macromedia_drivers.jar also lacks some of the files added by this updater, so I would assume so. I applied the update to my CFMX 6.1 updater release including also the important updating of the Sequelink ODBC Server as indicated in the technote. Don't miss that, if you're running on 6.1 (not needed on 7). After restarting, things are working.

Warning for those with 6.1 and 5 installed

I should note that I did find that my CF5 server (also running on this same dev box) suddenly, and for the first time ever since this box was created a few months ago, started raging with a persistent 55% cpu utilization for ntconsolejava.exe. I certainly would have assumed that the problem was in my CFMX 6.1 services, but by process of elimination it was the stopping of "ColdFusion Management Repository Server" (from CF5) that did the trick. That runs a JRun instance in the CF5 server.

I can't see how this would be connected, but I point it out in case it happens to others. I don't know what to do about it. I may just stop my CF5 server until I ever need it. I suppose I could also just set the management service to manual. I don't recall what the mgt service did in CF5. A quick review of the docs shows it had to do with archiving and admin reporting.

New Features in the Update Driver

But perhaps more important, as for the features it adds, did anyone notice the "sql server nt authentication" it enables? It seems that would solve a useful problem, but I could not get it to work. I followed all the steps including extracting the DLL (it's in the zip) and creating an "other" datasource type with the provided "authenticationmethod" in the JDBC URL. Has it worked for anyone?

Note also the interesting new spy feature, which can provide some useful diagnostics (think SQL Profiler, but for any DB and all the info tracked in a log file on the CF server). I was able to get it working successfully with a SQL Server database (using the "other datasource type" as explained in the technote), but when I tried to add it to an odbc dsn against an Access DB, using the connection string approach listed, nothing happened (no spy log created, until I used the "other datasource approach).

Note as well that the updates offer various fixes for SQL Server, Oracle, and Informix. It's also an update that pure JRun 4 users can/should run. I will point out that one blogger has found a change in behavior regarding multiple statements on a single SQL line: http://www.feed-squirrel.com/blog/index.cfm/2006/6/8/CREATE-TABLE-from-CF-7.

See the technote (linked at the top here) to learn more. I've also opened a question in the Adobe forums with this same info: http://www.adobe.com/cfusion/webforums/forum/messageview.cfm?catid=143&threadid=1180864&enterthread=y, as I can't assume as many people read my blog as will read that.

I'll be curious to hear (here or there) what others have experienced and what they think of these new features.

I should warn that the technote indicates a couple of times that you should not try to put these new connection string values into the "connection string" field of a DSN, instead creating an "other" datasource type and providing them on the JDBC URL for that DSN. When I made the mistake, I found that I could not query any ODBC datasources nor could I even start the CFMX Admin or get debugging on any page. All complained that the "DataSource service is not available.". I found no neo-query.bak to recover from, so was quite stuck. A careful analysis of the new-query file found that inappropriate CRLF character codes had made their way into the file (As 0a and 0d values that made for bad xml). I removed them (not trivial to do correctly) to get things back in order, so FOREWARNED IS FOREARMED.

BlogCFC was created by Raymond Camden. This blog is running version 5.005. (Want to validate the html in this page?)

Carehart Logo

Managed Hosting Services provided by
Managed Dedicated Hosting