[Looking for Charlie's main web site?]

Problem creating Derby DSN? It's a bug in CF8 Standard

Here's some important news I've not seen shared elsewhere. We occasionally hear of people having problems creating Derby datasources in the CF Admin. They report that they get taken to the "other" datasource page by mistake. I've done some digging and can report the root cause: it's a bug in CF 8 Standard. You don't see the problem if you run the Developer or Enterprise edition, whether running standalone or multiserver (or "multi-instance"). It only happens on Standard (which only supports a standalone/server deployment.)

I think this is why it's been so confusing to resolve. Instead, I've seen all kinds of guesses as to the root cause. The challenge is that if you don't have a Standard license, you can't see how things work in Standard. Fortunately, I have one and just ran a test and can confirm this.

How I've demonstrated the problem

I started out on Enterprise (same on Developer), and in the CF Admin chose to create a new datasource, choosing "Apache Derby Embedded" in the "select a valid driver type", and it took me to a page asking just for the "Database folder" where the Derby DB would live, and it offered the optional "create database" checkbox. That and a description field were it. This is indeed what one SHOULD see for a Derby Embedded DSN.

But when I switched my CF license (clicking the blue "i" icon at the top right for the System Information page) to use a Standard license key, and repeated the steps above, I was taken instead to the "other" driver page, where one is prompted to enter a JDBC URL, driver class and name, and many other things that would clearly confuse someone trying to use Derby. What a shame.

Does NOT affect those adding a Derby Client DSN

I'll add that this does NOT happen when choosing the "Apache Derby Client" driver option, even in Standard, which is odd. (And again, it doesn't affect those using Enterprise or Developer edition, even if they're using the Server/Standalone mode that's closest in implementation to Standard.)

What's this Derby thing anyway?

If you're not familiar with the difference between Derby Embedded and Client, or indeed don't know much about Derby at all (the Apache database embedded in CF 8) and may wonder why one may bother to try to use it, you can check out various Derby resources I have offered on my site, including most recently a talk I gave at Max 2008, the recording of which is available here.

Bug report filed

This problem of redirecting to the wrong page seems, then, to be clearly just a bug in CF 8 Standard. I've just filed a bug report on the CF wish form, which I discuss more here for those not familiar with it. Let's hope that perhaps this is addressed in the Centaur release.

"Using Apache Derby", watch my Max presentation

Did you know that CF8 (Enterprise, Standard, and Developer) has an embedded Apache database called Derby? Have you wondered what it's about? Why you might consider it? How to use it, and how to use it with other tools? Well now you can watch/listen to my Max presentation, "Using Apache Derby: the Open Source DB Embedded in CF 8", which is one of many Max NA 2008 videos which Adobe has released at tv.adobe.com. You can find a description of the talk here.

I actually did 2 Max talks (in addition to my Unconference talks), and I mentioned the other in a previous entry.

I'll note as well that I list this and all presentations I do, with links to PDFs or recordings, at the presentations section of my site.

Finding other CF Videos at tv.adobe.com

Finally, note that you can find other CF videos by selecting the "all product" category and choosing CF. Of course, you can also use the search field as well to find specific speakers or topics.

CF/Derby: Some common questions about Derby's suitability for use

Some of the most common questions people wonder as they first consider using Derby (the embedded database in CF8) are the following: is it a development-only DB? Does it perform and scale well? Isn't it only a single-user DBMS? Is it the same database embedded in AIR? There's a lot of misinformation out there and there are surprisingly positive answers to all these potential criticisms, which I address these below.

I'll also address these points and more in my talk at Adobe Max, in my session "Using Apache Derby, the Open Source Database Embedded in ColdFusion 8", on Tuesday, November 18, from 4:30 pm - 5:30 pm. If you've not yet booked that slot, or are on the fence, I think you'll be VERY surprised to hear all that Derby can do. I'll share some tidbits over coming days.

Is it a development-only DB? How does it perform and scale?

No, it's not just a development DB. First, yes, of course you can build production applications with it (though redistribution would be according to the Apache license). But as for its performance capability, there's in fact a PDF of a presentation comparing Derby to MySQL and others. Still another is available from IBM.

I've heard that Derby is a single-user DBMS

That's a common misconception, and some of it stems from the fact that it can run on its own, embedded in another Java application or server (like ColdFusion), or it can operate using another feature called the Derby Network Server.

Let's look first at the simpler embedded form: Derby itself provides no communications abilities of its own, which helps keep it lightweight. As such, it can't be communicated with from outside applications, indeed it can accept requests from within that same JVM process. Does that mean it's single-user? Of course not. CF (like most web app servers) is multi-user, and IT handles that multi-user processing. Derby itself has no problem processing multiple requests within the CF process. It just can't receive them from outside of CF.

So why might a CF user care about the Network Server feature? Well, if you wanted to talk to Derby from another Java process, then you need it. You may think this applies only to other Java applications on the server or clients, but here's one that may trip you up: an IDE. The ij tool (mentioned later), for example, can't talk to your embedded database without enabling the Network Server.

That latter point, and indeed more on this whole question of "is it a multi-user database or not?" is covered in Chapter 1 of an available Derby book I'll discuss in a later entry, and that chapter is available online (and in pdf form). The discussion of interest here is on the next to last page. I'd like to quote a bit:

When developers refer to Apache Derby as an embeddable database, they are referring to the fact that the Apache Derby database runs within a JVM process. Without the Apache Derby network server, there would be no networking services, data access outside of the embedded JDBC driver in the database engine, or other infrastructure requirements; this accounts for its small footprint.

Understanding what the embedded concept entails is critical when developing applications. For example, one common misconception that developers have when they work with Apache Derby as a standalone database is that it's only a single-user database and does not have communication capabilities. They believe that it is a single-user, single-connection, single-threaded system and develop their applications accordingly. This is not true. Apache Derby as a standalone database can support as many connections as desired, so long as they are established from the same JVM hosting the Apache Derby engine.

For an Apache Derby database to be accessed from a process that resides outside the hosting JVM that loaded the Apache Derby database initially (even if the JVM process resides on the same server), you need to load the Apache Derby network server. Read that last sentence twice to ensure you understand it because it is often a source of confusion for Apache Derby developers when multiple JVMs reside on the same machine. The Apache Derby network server allows for communications between JVM processes. This means that this communication infrastructure isn't solely required to communicate between machines; it is needed even if two different JVM processes reside on the same machine and want to talk to the same database.

Is this the same database engine that's embedded in Adobe Air?

No, that's yet another open source (indeed, public domain) DB, called SQLite (not a typo: it's spelled with one "L"). One may ask why the different Adobe teams chose different open-source embedded DBMSs. It could be that the different groups weren't aware of each other's decision and simply chose what seemed best for them. (Is Derby embedded in CF the same way SQLite is in Air? Well, that seems semantics. Yes, Derby is embedded in CF. The full DBMS is there. Nothing to install. Yes, you have to create a DSN (more later), but that's it.)

Could Derby have been used for Air? Sure. Could SQLite have been used as the embedded DB for CF? It seems so. The two are very similar in being small, embedded, yet highly functional multiuser database engines.

There's a comparison of the two on the SQLite site, though obviously it has a SQLite perspective. For instance, it says "Derby only allows a single process to have the database open at a time in its embedded mode. However, Derby also offers a full client/server mode." That's a common misconception, as discussed in my previous point above.

Signup for my Derby talk at Max

Again, I'll have lots more to say on Derby. But I realize it can be hard to keep up on any blog entry or series of entries. If I've piqued your interest at all, or if you want to show your support for CF (and Derby) talks at Max, consider signing up for my talk. I'll have more to say and show there, for sure. Also, if you have colleagues who maybe don't read blogs, let them know that Derby is a lot more than they may have been led to believe, and have them sign up. The Max folks use signups to measure interest in topics. This may be one that slipped under the radar of many considering talks. I'm just trying to help promote it, while sharing more Derby goodness along the way.

Have you considered using the Derby DB embedded in CF 8?

One of the less heralded new features in CF8 is that fact that there is now a DBMS embedded within it, called Apache Derby. Yes, I called it a DBMS. Some have misconstrued it to be some simplistic file-system data file. It's much more (multithreaded, multiuser), and quite capable of being a production DB for you, with far more features and capability than you probably imagined.

I'll be talking about it more at Adobe Max, in my session "Using Apache Derby, the Open Source Database Embedded in ColdFusion 8", on Tuesday, November 18, from 4:30 pm - 5:30 pm.

If you've not yet booked that slot, or are on the fence, I think you'll be VERY surprised to hear all that Derby can do. I'll share some tidbits over coming days.

If you've not yet started to create your Max schedule, please do so. The planners use the information provided to juggle rooms, etc. Derby may not seem like a sexy talk, but I hope you show you there that it's very much worth your attention. It also opens the doors to some interesting that only a DBMS embedded in CF can enable.

Even if you don't want to book your whole Max schedule yet, if you think this would interest you, please add it to your schedule as a start. Thanks.

Getting Started with the Apache Derby Open Source Database in CF8

Had you noticed that one of the many hidden gems in CF8 is the Apache Derby open-source, cross-platform database engine embedded in CF8. You're free to use it for production. Why would you? How does it differ from other free databases like MySQL?

To answer those questions and more, I've created a new resource called, "Getting Started with the Apache Derby Open Source Database in CF8". It's a resource compendium for those interested in using the DB with CF, pointing you to general resources to learn more, linking to many CF-specific articles and blog entries on it (from both before and after the final CF8 release), and lots more to help you get started--as well as respond to some common misconceptions.

It will continue to evolve, but for now, the sections are:

  • General Resources for Learning More about Derby
  • Is it a pure development only DB?
  • Is this the same database engine that's embedded in Adobe Air?
  • I've heard that Derby is a single-user DBMS
  • How can we use it/enable it in ColdFusion?
  • Are there any IDE or query tools for talking to Derby?
  • Why use Derby database verses using something like MySQL?
  • Where can I find a reference for supported SQL, etc.?
  • Is there a way to automate creation of a new Derby Database in CF?
  • Where else can I learn more from other CFers using Derby?

I'll also point out that those interested in Derby, especially in discussing it with others, will want to know that a couple of weeks ago I started a Derby "community" on the coldfusioncommunity.org site. Anyone may join in. I'll note that I rolled all my answers to questions there, prior to December, into the getting started page above.

Let me know what you think of it, and certainly any additions or corrections you'd suggest.

BlogCFC was created by Raymond Camden. This blog is running version 5.005. (Want to validate the html in this page?)

Managed Hosting Services provided by
Managed Dedicated Hosting