I'll be presenting two talks, "Hidden Gems In ColdFusion 2016" (as my personal talk) and the sponsor talk for the Integral (FusionReactor) folks, "Keep Applications Online And Detect & Isolate Application Issues & Performance Bottlenecks In No Time With FusionReactor 6".
See the descriptions for each, via the links above. My Hidden Gems talk should be as classically useful as ever, for those either new to or experienced with CF 2016. And the FR 6 talk will be more than "just another intro" to FR, highlighting as it will the important new features of FR6. Again, see the description [I wrote the description, they picked the title :-)] to appreciate what it will be about.
Also, let me give a heads-up to all attending: do be sure to look at the email that was sent out today or yesterday, with all kinds of important information to be aware of, from timing of events, to ground transport, to the very important map to see how to get from the conference hotel to the conference location at the college just a few minutes away, and more.
While the conference name has changed (from cf.Objective()), there are still plenty of CF-oriented topics, and of course as nearly everyone would point out, it's good for everyone to expand their reach and focus.
To that end, I'll be doing a topic a bit different than my normal focus of CF server troubleshooting. Instead, recognizing that there will be folks there who either use other servers, or develop web apps or mobile apps, I'll do a bit of a "soft" topic on how to troubleshoot performance problems more generically, in:
Hey, my web app's slow. Where's the problem?
Looking to buy ColdFusion? Get it at a discount from the FR folks, and an additional discount off FR
Even better, for a limited time, folks making such a purchase of CF there will get 20% off a purchase of FusionReactor Standard or Enterprise Subscription, if purchased at the same time.
So if you're looking to buy CF11, save money on CF and get a great monitoring solution for CF. I work with it daily, using it on my sites and nearly always recommending it as the best tool for solving CF problems. (It also works with Railo, Lucee, and BlueDragon Server JX, as well as Tomcat, JBoss, Jetty, GlassFish, and indeed any Java application.)
See my FusionReactor category of blog posts for more I've shared on FR over the years.
But more than that basic info, I'd like in this post to take advantage of the chance to share a few things related to the topic of upgrading FusionReactor, including some common questions I often am asked as I assist people in using it:
- Where do I get the update?
- How do I find out what's new in the update?
- Should I hesitate about applying the very latest FR update as soon as it comes out?
- How would you I know there WAS an update to FR available?
- Do I just need to run the full installer or can I update just one file perhaps?
- Do I need to restart CF (or Railo/Lucee/Tomcat, etc.) for the update to take effect?
- Can I update FRAM and choose NOT to update my monitored CF/Railo/Lucee/Tomcat/etc instance?
- Why do you keep referring to "CF/Railo/Lucee/Tomcat/etc"? I thought FR was a CF monitor tool?
- What if I'm on an FR version older than 5.0?
For these and more, read on.
One of the most important features is the stack tracing feature, used to understand what's holding up a long-running request.
Also, sorry for the long delay in blogging. Just been so busy doing my CF server troubleshooting consulting.
As for my sessions at CFSummit (next week), I'll be doing the following (and you can follow the links to learn more about the talks, their dates and times, etc.):
At CFCamp, the following week, I'll be offering:
- Hidden Gems in ColdFusion 11
- Monitor, Troubleshoot & Protect Your CF/Railo Servers with FusionReactor 5
I had also presented the Hidden Gems talk at NCDevCon 2014.
These are all great conferences, and, in addition to cf.Objective 2014 (where I spoke also, on different topics), they are each great ways to keep up on what's going on in the world of ColdFusion and related technologies.
Finally, if someday you're visiting this blog entry and find that one of the conference links no longer work, you can find my own link to all my presentations, at all conferences the past 15+ years (as well as to any recordings made available) at my presentations page.
Come say hello if you're at any of these events.
But as is often the case in a lot of the CF server troubleshooting consulting I do, I find the causes to be far less often what most people seem to suspect. So what would I look for when someone reported high CPU in ColdFusion (or Railo)? Read on.
Anyway, here is the answer I wanted to offer to that question...
You can find the recording here. Note that you need to login with an Adobe ID, just like when you download Adobe software or participate in their forums. (I have no control over that.)
Since that link just goes right to the recording, here is the description I'd used for the session, to help decide if the recording may interest you. BTW, I clarify on the session that FR is useful for more than just ColdFusion, in that FusionReactor can be used for Railo, BlueDragon, and OpenBlueDragon, as well as in fact any Java server (Tomcat, JBoss, Jetty, Glassfish, Websphere, etc.), and the session applies just as well to folks using those.
My session: Monitoring ColdFusion with FusionReactor
Yet people will reasonably want to be able to have some mechanism that "watches" CF "from the outside", to know when it's gone down. How can you do that? That's what I'll point out in this entry.
And beyond talking about what goes along with the CF Enterprise Server Monitor, I'll also point out options for those who are NOT running CF 8, 9, or 10 Enterprise and therefore do not have the Enterprise Server Monitor. This also includes those CF 6 or 7. There are solutions for you, and also for those running Railo, BD, or indeed any Java server. More on all that in a moment.
This is part 4 of an unexpected series of entries today on the CF Enterprise Server Monitor. :-) I got on a roll, and each seemed deserving of its own topic. See the "Related Blog Entries" below this entry for links to those.
What the CF Server Monitor is, and is not
The two session topics will be:
- What's Next In Zeus, aka CF10
- Continuously improve CF code quality, server availability & application stability
The descriptions for each of those is on that page for the conference program". As you'll also note there, the other speakers are Mark Drew, Gert Franz, Gary Gilbert, Luis Majano, and Bilal Soylu.
The day-long class I'll be doing (separately purchased, and nearly sold out) is:
If you haven't heard, both Fusionreactor 4 and FusionAnalytics have been released in recent weeks. They're powerful tools that I help people use all the time in my independent CF troubleshooting consulting. If you haven't checked them out yet, do. And note the availability of both a live demo (nothing to download and install) and a free 10-day demo for each.
See you in Munich, or in the futureIf you may be in the area and interested in attending, see that page (top right) for more on registering.
I'll note that I will likely give both the talks in other venues and formats (whether in-person or over the web) in the future. If you may be interested, let me know.
And if you'll be in Germany next week, I hope to see you there. (Sadly, my wife didn't get to come this time.)
Thanks to all the sponsors for helping make the event happen, for me and for all who will be attending.
Update, Recording: Note that this session was recorded. You can view it here, but note that you must login with an Adobe ID to see it.
And I'm presenting a session tonight, Tuesday September 13, at 7pm Eastern, on "Understanding and Using the ColdFusion Server Monitor".
As many of you know, I'm pretty much a fanatic about the monitor, especially about truly understanding elements of it that many miss. And so in my talk this will not be just a dog and pony show, but I will talk about practical experiences with it, though presented to either those new to it or experienced with it.
Note that the times for all these devweek sessions is shown (on the Adobe site) as being Pacific time, so again mine is at 7pm, not 4pm, Eastern.
And yes, the sessions are being recorded and seem to be made available the next day.
Finally, beware that there is no one URL you can use to join in on all the Connect sessions, nor can you get the Connect session URL by going to the event page (via the first link above). Instead, you must register for each event (free) from that first page, to get each session's Connect URL--and you'll want to do that at least several minutes in advance of any session to have time to register, get the email, login, etc.
See you then.
PS Hey, while we're talking monitoring, note as well that if you've not heard, FusionReactor has come out with its new release 4, which has lots of great additions, especially FREC (or the FR Extensions for CF) which cause FR to grab and log lots of great info that the CF Server Monitor only shows and never logs. I'll be blogging about FR 4 soon, but plenty to see on their site. and FusionAnalytics is also just about to release, really!
I won't be discussing these at this talk, focused solely on the server monitor, but as I always tell folks, each tool has its use and often a single shop can benefit from having both (like I do, as do many of the clients I help with troubleshooting). You can find more from me about FR here in my blog. And I'll have lots more to say about FA and FR4 more soon.
Or perhaps you've seen this error from ColdFusion, in your logs or on-screen:
Do you know what this means? It's usually not what you think. I've even seen experienced CF developers who get thrown by this challenge. In this entry I'll try to help explain a very common problem and correct some misconceptions. I'll even contend that this info is often useless and indeed misleading (and therefore the feature producing it ought not be relied upon, and should even be turned off). Along the way, I'll share some things that I've not seen documented elsewhere.
Strap on your seatbelts. We're going for a bit of a ride (if it was easy and could be understood in the length of a tweet, then perhaps everyone would already understand it!) As always, I welcome feedback.
FusionReactor is one of the leading CF Server Monitor tools, which works not only with CF 6/7/8/9, either Standard or Enterprise, but it also works with Railo, Open BlueDragon, and even BlueDragon JX 7.1. In fact, it works with any J2EE/JEE server or servlet engine.
If you're running a site on any of those platforms and ever have problems of slowness, instability, or any other "curious" problems, or just need to better understand the nature of requests that CF is processing, and how well (or poorly) it's doing it, FusionReactor is a great tool, for the reasons I outline. It's like having x-rays into the app server.
"CF911: Stack Tracing CFML Requests to Solve Problems"
Regardless of what CFML server monitoring tool(s) you have, or even if none, did you know that you can use a feature called "stack traces" to be able to pinpoint the exact line of code that a CFML request is running at any time? Did you know how to use that information to troubleshoot performance/stability problems? Do you know how to obtain that information either manually or automatically (such as during a crash while you're not watching)? Do you know how to obtain that information in any of the CFML Server Monitors (FusionReactor, SeeFusion, the CF8/9 Enterprise Server Monitor), or with free command line tools? And how to do this for any CFML engine (CF, Railo, BlueDragon, etc.)? Do you know how to interpret the information once you get it?
In this session, veteran CF troubleshooter Charlie Arehart will help remove the mystery from using stack traces. It really is amazingly simple with the right tools, and it can be incredibly useful to solve otherwise thorny problems, once you understand how to interpret the information.
Of course, I'm thrilled to be heading back to Minneapolis. I spoke there previously in 2008 and 2007 but couldn't attend in 2009. It'll be great to see all the fine folks who run and attend this unique conference.
BTW, I just saw also that CFUnited announced another round of topics accepted today and I see a topic whose title if very similar, "How to Read a Stack Trace", by the inimitable Daryl Banttari. It's hard to tell from his brief description how similar these will be, but Daryl is awesome so I'm sure I'll learn much from his. (I was literally just about to offer mine as another CFUnited submission but now won't of course. :-) Hopefully another of my submissions will be accepted, so I can keep my streak of having spoken at every CFUnited since they started.)
Anyway, the good news is that whichever conference you go to, this important (and often misunderstood) topic will be covered! :-)
Well, there are several ways you can watch them, as this entry will discuss, and some may be better suited to the job than others. It can be very interesting to discover what's going on, especially if you're having any suspected performance problems which you think may be related to ORM processing (or just if you wonder what all it does for you).
As for spying on the SQL, of course ORM support is just a different way that the CFML engine (through the ORM framework) sends SQL to a database via a regular DSN, just like any other request, so there's nothing really "tricky" about this. It's just about realizing that while you don't write the SQL yourself, it's still generated by the CFML engine/ORM framework, and you may not realize/consider the available tools which can spy on it, just like any other DB processing from within CF. Indeed, some people may not even realize how many options exist to spy on JDBC interactions from their CFML engine to the database engine.
The good news is that there are several approaches, some included in CF (some depending on the edition), and some available separately which would work in any edition of CF or the other CFML engines (Open BlueDragon, Railo, etc.), and with any of the ORM frameworks. And again, some may be better than others for certain challenges.
(FWIW, besides the aforementioned Transfer and Reactor, there are still other ORM solutions for CFML, which I mention in my CF411 list as CFML ORM Frameworks. Indeed, note that you can run Hibernate on CF prior to CF9, if you want to. This is a recovery of a blog entry that no longer exists, recovered via archive.org.)
Built-in ORM Logging Option
First, note that for those using CF9+ ORM, there is indeed a built-in option in the CF ORM setup where one can enable logging, settable in the application.cfc: see the this.ormsettings option and its available key/value pair, logSQL="true".
There are several resources where you can learn more on that (and a related log4j property file approach to logging this). Besides the CF9 docs page on the ORM settings, there is also a blog entry by Adobe engineer Rupesh Kumar.
The default is to log this information to the console, but you can manipulate those log4j settings to tell it to use a file (see the links above). Even so, this will result in quite a lot of data being logged, which you will then need to connect back to your specific requests. The following approaches may be preferable.
Using FusionReactor or SeeFusion
Users of any CF edition (6+) or any CFML engine (Railo, OpenBD, or BD 7+) can use tools like SeeFusion and FusionReactor, which have always had the ability to monitor database interactions by "wrapping" the datasource to be monitored. FusionReactor engineer John Hawksley has posted a recent article specifically on monitoring CF9's ORM interaction, in the FR Devnet site, Using FusionReactor's JDBC Driver Wrapper With ColdFusion 9 ORM. Its concepts would apply to any ORM, of course.
Similarly, I've written generically about FusionReactor's database monitoring feature in What is the FusionReactor datasource monitoring feature? Why would I use it? Powerful stuff. As I point out in that article, the concepts discussed apply as well to SeeFusion's ability to monitor queries by wrapping datasources.
That said, it's worth noting that FusionReactor does have a couple of advantages, in that it provides for the display of all queries for a given request (while viewing the details of that request), whereas SeeFusion only lets you see the slowest query in a given request. FusionReactor also provides a separately available display of all the slowest queries (across all requests). It also logs every query (connecting it to a given request as well), while SeeFusion (Enterprise, at least) can also log the slowest queries to a database.
And note that both of these track any requests coming out of CF, not just those associated with a given request. So if there is ORM SQL that is associated with the startup of CF, that's tracked too. (And for those aware of issues with CF's Client Variables, such DB activity is also tracked, even that done by the hourly purge, which takes place on a background, non-jrpp thread.)
CF Enterprise Server Monitor
Those running CF 8 or 9 (Enterprise only) will find that its available Server Monitor does offer built-in monitoring of the SQL executed against CF datasources, at least, as long as you enable "Start Profiling" (which also enables other features, and overhead, as well). In this way, the Enterprise Server Monitor can monitor database interactivity, including ORM interactions.
Unlike FusionReactor (and like SeeFusion), it focuses only on showing queries that exceed certain limits, and at that it shows them only in a "Slowest Queries" interface, tracking the slowest queries among all requests. The CF Enterprise Server Monitor also has no logging ability at all.
Being able to see every single DB interaction for a given request (or across all requests) may be all the more interesting for discovering/observing what's happening with ORM interactivity.
Another alternative CF feature
Still another little-known feature for spying on JDBC interactions in CF is by way of the JDBC "spy" feature, which does in fact allow logging of all JDBC interactions mde from within CF. This feature was first enabled by way of the DataDirect 3.5 driver update which was made available (as an optional upgrade for 6 and 7) in the CF 7.02 timeframe. I wrote about the Spy feature back back in Aug 2006.
Since then, CF 8 (and now 9) offer it instead as a new "log activity" option in the "advanced settings" for a datasource definition in the CF Admin (which is disabled by default). I pointed this out in another entry from 2007 as one of many easily missed changes for the CF 8 Admin.
This "log activity" output is not as easy to interpret as FusionReactor's logs, and can indeed be voluminous (moreso than FR's), so be careful. Anyway, it's one of the several ways you can monitor JDBC interactions between CFML and your DB engine. Again, any of these may be useful for monitoring any of your CFML/database interactions.
Generic DB Monitoring tools
Indeed, it's worth noting finally that while the focus here has been watching the DB interaction from CF (and the ORM framework) to the database (by watching the JDBC traffic going out of CF and returning), you could just as well watch the DB interactivity from the DB's perspective instead (watching it coming and and being returned).
There are many tools that can monitor database processing, available for each of the major databases (free and commercial). I list several such tools in one of my CF411 section, Database/SQL Monitoring Tools.
Hope all that's helpful, whether you use ORM or not.
For readers not familiar with FusionReactor, it's a server monitor (and more) for CF, Railo, OpenBD, and indeed any J2EE/Java EE server. I've written and spoken about it a lot. Here is the link to the category of my blog entries about it.
As for 3.5, here is what has been shared in an email to their customers. I wanted to pass it along to everyone:
EXTENDED PLATFORM SUPPORT - FusionReactor 3.5 now supports ColdFusion 9 plus a range of new servers and operating system platforms - including Windows 7, Windows 2008 Server R2, Mac OS X 10.6 "Snow Leopard", Railo 3.1.1 and JBoss 5.1.
IMPROVED INSTALLER - supporting 64 bit Windows machines
INCREASED MONITORING PERFORMANCE - FusionReactor continues to be the ColdFusion production monitor of choice because of its incredibly LOW overhead of less than 1%!
EXTENDED FRAPI INTERFACE - FRAPI, the FusionReactor API, gives you the ability to access FusionReactor functionality from your ColdFusion pages. This interface has now been extended to include additional Request information.
NEW AMF PROCESSOR - Action Message Format (AMF) is the file format used with Flash Remoting and applications such as Flex 2 and 3. FusionReactor has a completely new AMF processor supporting externalizable Objects.
A NUMBER OF MINOR ENHANCEMENTS AND IMPROVEMENTS - With every new release we continue to extend FusionReactor to make it even more stable and secure.
Click here to see the FusionReactor 3.5 Release Notes and Resolved Issues http://www.fusion-reactor.com/support/kb/FRS-230.cfm.
To upgrade, please download FusionReactor 3.5 from the FusionReactor download pag.e
Click here to download 3.5: http://www.fusion-reactor.com/fr/downloads.cfm
If you have any questions or feedback please email email@example.com.
(If you're not familiar with the value of thread dumps or stack traces, read on. The resources I point to get help you to appreciate their usefulness.)
Basic CF Monitoring in Nagios
First let me share that if you've not explored monitoring CF with Nagios at all, here's at least one blog entry from 2005 discussing that. (A more recent one, at http://profec.net/2009/01/12/monitoring-coldfusion-with-nagios/, which I used to list here, is no longer responding, nor could I find it on archive.org.)
Of course, you can easily use Nagios primarily to report whether CF's up or down, but you can also monitor basic statistics such as are provided via CFSTAT (also available on Windows via Perfmon, which also exposes generic stats about all processes), as demonstrated in that blog entry.
But you may have noticed that, if you run the Multiserver (multiple instances) version of CF, those CFSTAT and Windows Perfmon Stats aren't available on that version of CF.
So what to do if you wanted to monitor more about what's going on inside of CF?
FusionReactor Nagios Plug-in
If you're running FusionReactor Enterprise, you can now get that additional information via the free FusionReactor Nagios Plugin. It's a perl app that when implemented will expose certain key statistics in a way that Nagio can process them (and some are things that CFSTAT and the CF Permon stats don't report):
- Count of current running requests, and total count of all requests run
- Count of request queued by FusionReactor
- Average request runtime (since server start), and Recent request runtime (in past 60s, by default)
- Count of recent slow pages
- Memory free in bytes and percentage
- Memory Total and Max
- Instance and System CPU Use percentage
- Count of recently completed JDBC requests
Note that this is technically community-contributed feature (from an Intergral employee, David Stockton), not a supported feature.
If you're running or are interested in FusionReactor Enterprise, check it out.