[Looking for Charlie's main web site?]

Part 2: Serious security threat for ColdFusion servers [now covered by a hotfix]

Since I posted my entry earlier today about a Serious security threat for #ColdFusion servers [not now covered by a hotfix], I have had many questions and discussions which lead me to share more info.

At first I was adding these as updates to the previous entry, but I fear that some who may have read it earlier in the day may then miss some of this new info, thus this "Part 2". You will definitely want to read part 1 before proceeding here.

[Update: And since writing this entry 2 weeks ago, Adobe has indeed now come out with a hotfix. I have more to say about that in the new Part 3: Adobe hotfix released for "Serious security threat for #ColdFusion servers". While you should proceed to get that fix in place, you'll likely benefit from reading parts 1, 2, and 3, as there's more discussed than just the thread and fix, itself, which could benefit you down the road.]

Among the new information shared below are such things as how the hack worked (not too much detail, though), how to determine what the exploit may have exposed, how to handle resolving things for many sites via scripting, how to lock down the /adminapi, /administrator, and /componentutils directories, and most important, why you should not skip all this just because "we already block all access to the CFIDE/adminapi" (and /administrator and /componentutils)". There may be exposure you're not considering.

How did the hack work?

[Update (on Jan 15) Since posting this originally, the technote from Adobe indicates that we should also block unfettered public access to the CFIDE/componentutils directory, used for the component browser. I'll update the other references to these directories, below. And while certainly any current known vulnerabilities in those folders are indeed now fixed, there could be others that might appear later, so better to protect them now.]

I didn't want to share too much info, but again I've been asked enough so I'll share at least this: the adminapi exploit allowed the hacker to create a scheduled task, and then that task called a page on another server, and the scheduled task was configured to "save" the output of that to this h.cfm file. Well, the page did a CFHTTP of a page that returned CFML, and so what was "saved" was in fact CFML code into that file. And it's THAT code (and that page) which then allowed the hacker to attempt further exploits (the file manipulation and db access I discussed previously), mostly leveraging a file manager interface (written in CFML) and some of the undocumented java objects (servicefactory).

The good news is that depending on other security hotfixes or updates you may have applied, some operations may have been prevented. More on that later here.

Also, if you followed some of the standard precautions about running CF with a user having limited rights and privileges, that also would have limited potential damage. Again, see the lockdown guides (mentioned in the last entry) for more on that.

What to do if you have been hacked? what did the hackers get?

I've had some people ask what to do if they have been exploited. Of course, the original forum entry explained the need to delete the file (h.cfm) and the scheduled task (if there), and I added in reply (and clarified above) that really one needs to close down public unfettered access to the /CFIDE/adminapi, /CFIDE/administrator, and /CFIDE/componentutils folders to prevent them being exploited again. That's THE MOST IMPORTANT thing to do first. I discuss more about "how" to do that below (if you do go do that first, be sure to come back to read the rest of this section).

So if you DID find the file ended up on your server (and have deleted it and closed the hole), how can you find out what they actually "did" with the hack in place.

Well, first, if you have web server logging turned on, you can see their calls to the h.cfm file. If that gets a 200, then they were able to access the page. And then that page let them do things like access/manipulate files, and access DB info, including passwords. Most operations on the page would lead to GET requests to the h.cfm file, with a fuseaction indicating what operation to perform. So you could search your web server logs for requests to "/h.cfm fuseaction" (assuming your logs have the requested file and then the query string next to each other, separated by a space, as is true for IIS logs).

Be careful, though, when interpreting the logs: Not all the operations in the page lead to "gets". Some lead to "posts", which then have no query string.

You can even run the page (if you didn't delete it, or only renamed it), to see what sort of operations it performs. The file is open CFML source (not precompiled or encoded), so you can see what it does, and at least with the version I had it did not do any operations that sent info off the server, etc.) It would be wise to rename or move it before running it, so no one else can find it.

Update: when you run it, you will be asked for a code. The code is in the file itself. Just look for "code" in the source to find the value, and use that to get in.

You can then see that operations you perform lead to these web server log entries. (Do beware that sometimes web servers are slow to flush their buffers, so don't expect an operation you do to lead to an immediately updated web server log.)

Now, as for the database stuff, when I ran that page , I found that (at least on my server, running 9.0.2) the page would NOT expose the datasource info. I suspect some hotfix included in 9.0.2 (and available for 9.0.1) blocked that. So if that's true for you, then you don't need to worry about db info/password exposure.

But if that page DID let you see DB/password info, then the hacker could see it, too. You'll want to change your DB passwords, which means changing your CF DSNs and any CFML code if you may have hard-coded passwords in the code (bad practice, but some do it.)

Then you should also look to see if any files were created or modified, anywhere on the server, since the date of the exploit, to see if any look suspicious. There are tools that enable that. I blog about a couple at this blog entry. As for deletes, that's a lot harder to track.

Caution: Don't be lulled into a false sense of security/confidence!

Based on some private feedback I've gotten, I want to make this caution. Before you may say, "oh, we're not exposed to this problem because we block all access to the CFIDE/adminapi" (and /administrator and /componentutils), I'll say just be very careful before moving on, as there are a couple ways you might get burned that might not have considered. I did explain this in the forum thread, but since some may not bother reading it if they thought themselves "already covered", I wanted to add this caution here

First, be sure you've done that protection on ALL sites.

Besides the fact that you may have a real or virtual CFIDE defined in more than one site, note as well that in CF10, if you have it configure "all sites" for IIS or Apache, note that it puts a CFIDE virtual mapping in all sites. Be sure to have protected it then on all those.

Second, you may have things set so that only the "default site" (at least as its known in IIS) has the Admin directories open. And you may bind your web site to another site, assuming then that you can "only get to the default site" on the server. Beware: if you have that default site bound to "all ip addresses", then any request that gets to your server (whether for a domain name not otherwise bound, or using an IP address that does exist on your server) WILL then get to that site.

So as discussed above, do be sure to protect those directories, as discussed variously in the lockdown guides.

A real gotcha: implicit access to the built-in web server root

Third, there's another real potential gotcha. You may find you can get into the CF Admin on sites where you don't even HAVE a CFIDE folder as a real or virtual directory (such as would be shown in IIS, but this can happen on Apache as well)!

How could a request for /CFIDE/administrator/index.cfm work when your web server docroot doesn't have that folder (nor a virtual folder), you might reasonably ask?

This is a real surprise to some: CF has always had a capability whereby if you have the built-in web server enabled (the JRun web server in CF 6-9, or the Tomcat web server in CF10), then when a request is made for a CF page via the external web server (IIS or Apache, for instance), CF looks for the file FIRST in the external web web server's docroot, and THEN in the built-in web server's docroot, such as \coldfusion9\wwwroot or C:\JRun4\servers\cfusion\cfusion-ear\cfusion-war\wwwroot.

Now, you may think you don't have the built-in web server enabled, but you may. And note that if you are running CF multiserver (multiple instances), then by DEFAULT all new instances have the built-in web server enabled (specifically for accessing the Admin, before you may connect the instance to the external web server.)

So bottom line: if you have the built-in web server enabled for CF, then a request to any CFM page (including the CFIDE folder's pages) can and will be sought and executed if found, even if it's only in the built-in web server root. So test all your sites to see if they respond to http://servername/CFIDE/administrator/index.cfm, for instance. If it responds, even though there's no CFIDE defined for or existing in the IIS/Apache site/docroot itself, then this built-in web server root is where and how it's finding one.

Now, one way to stop this would be to disable the built-in web server, but some (especially CF multiserver users) may not want to do that.

And one may also wonder if they could lock down the CFIDE in the built-in web server--but that won't work in this case, because we're talking about when you're accessing the CFIDE through an external web server (so lockdown features in the built-in web server won't be honored during external web server page processing.) And of course because the built-in web server runs on a non-standard port, by default, it will generally already not be possible for anyone "from the outside" to access these Admin directories using that built-in web server.

So we're talking about how to block access to the internal web server docroot by way of requests made through the external web server. The only solutions (to me, which have worked for some I've helped today) is to do either of the following:

  • If on IIS 7, use the new "request filtering" feature, to block access to the to-be-protected /CFIDE paths (/CFIDE/adminapi, /CFIDE/administrator, and /CFIDE/componentutils directories at a minimum, for all but the sites where you may still want access to these). There may be a corresponding way to globally block access to these paths for all sites within Apache, but I am not familiar with that.
  • If not on IIS 7 (if on IIS 6, Apache, etc), you could specifically ADD a virtual directory to your web server (in each site that is connected to allow processing of CFML files). This virtual directory could be a folder with no files or folders at all. And then you could use the techniques alluded to previously (and later) for locking down the to-be-protected /CFIDE paths.

    Remember, we are talking (in this section) about the problem of your having a site which has NO CFIDE, so you would assume that it should NOT allow any access to the CF Admin, but you find that it unexpectedly does, because of this implicit invocation of the CFIDE within the internal CF web server's docroot.

    I know it may seem counterintuitive to add a CFIDE directory to such a site (for which you're trying to block access), but in this situation where the built-in web server folder is being accessed, and you don't have a global means to block these paths, it seems the only solution.

Don't lock down the entire CFIDE, without due consideration

Finally, do be careful that before you go locking down the entire CFIDE directory, be sure that your apps are not using the CFIDE/scripts or other directories, which are referred to by HTML code that CF generates when using many tags--and not just CFFORM-related ones, such as the new Ajax features since CF8, and more. Again, the lockdown guide talks about how you can use the CF Admin to change the location for that scripts directory, and then create a new virtual directory for it, if you may want to create better separation/access among these CFIDE folders.

How to lock down the /adminapi, /administrator, and /componentutils directories

Some people have asked for more info on how to lock down the /CFIDE/adminapi, /CFIDE/administrator, and /CFIDE/componentutils directories. There have been many blog entries discussing it, and I'll point to some, but do be careful that some are old (so things may have changed).

More than that, be sure to have read the previous sections here before going off to tackle this, as there may be nuances I share that these other entries do not consider.

More important, again, note the warning above on not stopping at what you "think" is full protection of your "sites that have a CFIDE" or not.

Finally, for those on IIS 7 or above, you definitely want to consider the "request filtering" feature, mentioned in some of the blog entries below, and the CF9 and 10 lockdown guides. Note that on IIS 7.0, there is no UI to configure these, but in IIS 7.5 there is. The CF9 lockdown guide was written in the time of IIS 7.0, so does not mention the request filtering UI. If you're on CF9 and IIS 7.5, do be sure to look at the CF10 lockdown guide for its section on setting up request filtering, as it's mostly not specific to CF10.

But with those caveats, here are some blog entries with traditional recommendations on locking down the Admin (beyond the discussion in the lockdown guides, mentioned in the last post):

And again, of course, see the lockdown guides, linked to in part 1.

I welcome if anyone has any more to propose. (I may just create a new one of my own at some point.)

How to protect dozens or hundreds of sites

If you find that you have dozens of IIS sites which you need to modify (to add IP or web server authentication), you may wonder if you have to go through and manually. Well, no. There are ways to script these operations, such as appcmd (an exe built-into windows), powershell (free from MS to install into Windows, if not already installed), or programmatic APIs like ADSI, WMI, and more.

Comments
Well, this is serious, but no different to historical access bypass attacks on the CFIDE - and as you say the advice has always been to restrict access to CFIDE's *.cfm files, or if you don't use the CF8+ AJAX tags or elderly CFFORM stuff, just remove the global CFIDE mapping all together, which would halt this attack.

In general terms administrator functions should never be wide open to the internet on any system, even if you have a password prompt.
# Posted By Tom Chiverton | 1/3/13 4:02 AM
Yes to all that, Tom. I don't know if you're saying you'd seen that I conveyed much the same points in various earlier discussions on this (whether in the forum thread where it first broke, or in the previous blog entry).

There is indeed nothing new under the sun. And the lockdown guides have existed (for good reason) since CF8. People just get lazy, or figure "it can't happen to me".

Still, this vector seems indeed new, so worth the alarm (especially judging from the several dozen people I've heard from by email, phone, twitter, and blog comments, who have been hit and did greatly appreciate being made aware of it.) It's understandable that they don't want to indicate on a public forum that they were hit, though it does mean that others don't see their expressions of thanks--especially those who may perhaps think I'm making too much of this. ;-}

While we're here, let me also make a plug for Pete Freitag's HackMyCF.com service (free and commercial), which can help folks test for and identify other security risks.
# Posted By Charlie Arehart | 1/3/13 10:04 AM
Charlie,

All excellent information, but really can not stress how important it is to use the published lockdown guides. Just some other notes, I haven't had any run ins with this attack, but have cleaned up several other that had similar payloads as described.

1) If you are running ColdFusion 9.0.1 or higher set a new Password Seed for datasources in Security > Administrator. Most of the scripts I have seen are using the default seed that was published many years ago. Changing the seed nullifies the scripts ability to decrypt the datasource password without doing extra work.

2) Consider enabling Disable access to internal ColdFusion Java Components in Server Settings > Settings. Again most of the scripts seen use createobject("java","coldfusion.server.ServiceFactory") so disabling access to ColdFusion Java Components can help.

3) If on Enterprise, use Security Sandboxes and restrict IP/server and ports that ColdFusion can connect to. In general, I set up a highly restricted policy on the root only allowing it to connect to 127.0.0.1 on port 80 and only open access to known IP/server and port as needed on a directory level. The Adobe default of allowing all IP/server and ports is wrong and should be changed. Also the same full sandboxing functionality that is in Enterprise should be made available in Standard.

There is absolutely a need for Adobe to separate the ColdFusion Administrator and AdminAPI from the other scripts (AJAX, CFFORM, etc).

I definitely +1 the recommendation for HackMyCF service.

There have been changes through the security updates, APSB12-21 did update both /CFIDE/adminapi/security.cfc and /CFIDE/administrator/security/_cffunctionsoptions.cfm, but I can't say if that would have stopped it since they are compiled. It does highlight that people do need to stay on top of applying the security hotfixes when released by Adobe.
# Posted By David Epler | 1/3/13 11:04 AM
@Dave, thanks.

As for how one "really can not stress how important the lockdown guides" are, I'll just note that I have indeed mentioned them several times already (4 times in this entry, with links to all 3 versions in the last entry, and having mentioned the guides 6 times in my first reply on the forum thread.)

I do appreciate your pointing out the specifics, though. I had pointed to the lockdown guides so many times so that folks would find those details there. Still, the guides are rather long (35 and 80+ pages for cf9 and 10, respectively), and I've had some people complain that they wanted to know exactly what to focus on for this hack. (As they say, "you can please some of the people some of the time...")

Anyway, because of that I'd been considering doing a part 3 today highlighting the discussion of some of these very things, so good on ya for thinking of it, too.

And admittedly, the discussion in the guides on some things is rather brief, and the seed feature is only mentioned in the cf10 guide (perhaps the 9 guide was written before 9.0.1 came out).

Fair point (in 3) about CF locking down access to all sites/ports. Let's be clear for readers: what you're talking about is the ability of tags like CFHTTP, CFFTP, etc to make calls out to other servers. That said, this one made a call to a remote server using a scheduled task. Not sure if that would have been blocked by sandboxing. Do you know? Have not had time to test.

Naturally, many would balk that too many things would break if all remote access was blocked from the server. Still, as you may know, CF10 introduced the new "secure profile", which DID lock things down, even if painful. I don't think it did what you're proposing, but it may have. Do you know? If not, it should be considered as another feature.

It would be nice if somehow there was a way to more readily highlight when failures happened due to such blocks (such as if someone had legit code that did a CFHTTP to a remote server that now failed because of such a block), rather than relying solely on error messages to be tracked or having to scour code up front.

Of course, it couldn't work like a firewall, asking for permission, because it should be an admin (not the executing user) who grants permission. I'm just saying something could log it or notify an admin. Still, I don't hold my breath for such.

And of course, some still-more security conscious people would propose instead that the whole server should be setup to prevent outbound access. There's certainly an argument for that. And while it does then complicate things like using the new CF10 autoupdate feature, there are ways to deal with it.

As with all things security, it comes down to a risk/reward (or cost/benefit, pain/pleasure) decision.

Finally, about the latest security fix, no, it seems it would not have blocked things. Adobe has said they are working on yet another for this very one.
# Posted By Charlie Arehart | 1/3/13 11:29 AM
Thanks Charlie and Dave for the recommendation of HackMyCF.com - I have made some updates to the service today to further help identify / lock down with respect to this attack.

First HackMyCF will now tell you if the Admin API is public.

Second, using our probe (a feature only available to paid subscribers) we are now alerting customers that may have been attacked by this, but did not yet know it.

Finally big thanks to Charlie for all his work in helping folks hit with this issue.
# Posted By Pete Freitag | 1/3/13 2:55 PM
What was the name of the scheduled task?
Thanks
# Posted By Steven Benjamin | 1/3/13 3:19 PM
@Steven, in my case it was cfprobe.

@Pete, great to hear, and thanks for the update.

BTW, I realize it may be tough to do but if you consider my observation about the implication of sites still exposing the CFIDE via the built-in web server root, I wonder if you might have some way (with the commercial version's internal probe) to find all sites and IP addresses for a given server and scan for whether any of those respond to a request for /CFIDE.

Sadly, for those using the free version of hackmycf.com, where they provide you the URL to scan, it would be up to them to think to try to scan all sites/IPs, and I'm sure most would never think of it.
# Posted By Charlie Arehart | 1/3/13 3:47 PM
Charlie - that's a great suggestion, I'll look into that because I want to solve that problem. One issue with that approach might be that some servers are on an internal IP so they might have IP's like 10.0.0.80 which we can't hit from the outside, but we can filter them out.

I think in the mean time doing a DNS lookup on the hostname and checking that IP will at least catch a good number more of these cases.
# Posted By Pete Freitag | 1/3/13 3:58 PM
@Pete, sure, and if they can't be hit, then they're not a problem, right? :-) My point is that something needs to probe all possible domains/iPs that a given server might possibly receive responses on, to really ensure that "the CFIDE" is not exposed.

Of course, once Adobe closes this specific hole, this problem will be averted for those who apply the fix, but another may arise inevitably, and I'm really fearful that a lot of people are blithely thinking, "oh, our site is protected", when they may not realize this "back door" that is still "open".

But, sure, anything you can do to help them will be great. And in the meantime, I hope that readers will take care to go this extra mile themselves. Certainly hackers will be trying to!
# Posted By Charlie Arehart | 1/3/13 4:10 PM
In relation to the administrator and adminapi, dont you need to password to get into those files through CF?

Also, as a temp solution? Would moving those 2 folders to another location be a temp fix or would that break anything?
# Posted By steve | 1/3/13 4:31 PM
@steve,

That would work in theory, but you wouldn't be able to use the CF Administrator. The CF Administrator uses those CFC's in the /adminapi folder to perform it's actions it looks like. At least I think so at least.
# Posted By Scott | 1/3/13 5:45 PM
@Steve, the hack got around the password prompt, so no, that's not sufficient protection. As I said many times in the 3 resources so far, you need to lock down those directories, whether by IP or requiring addition web server authentication.

And no, you can't remove them unless, as Scott notes, you're willing to forgo the Admin. That said, I know some customers who choose to do just that: they remove the two dirs, and put them back when wanting to use the Admin. I think that's overkill, and just better-protecting the folders should suffice. But there's no doubt that going to that extent is still more secure.
# Posted By Charlie Arehart | 1/3/13 5:52 PM
If I restrict all access from the cfadministrator using Pete's technique...
http://www.petefreit...
Do that protect me from this attack?
# Posted By Steven Benjamin | 1/3/13 7:09 PM
Steven - you will also want to block /CFIDE/adminapi if you are not already, and make sure that your Request Filtering Rules are global to all IIS sites and not just one as Charlie has pointed out is a common mistake.
# Posted By Pete Freitag | 1/3/13 7:20 PM
Thanks for the help with this guys. I ran a HackMyCF scan today and are almost there. I am have trouble finding pages with examples of how to set up IIS 7.5 to block the two important directories. Does anyone know of any instructions for the non-IIS experts? Thanks again.
# Posted By Jeff | 1/3/13 9:56 PM
@Jeff, for IIS 7, to lockdown by IP address (such as to limit to localhost) will require installing a feature not installed by default. I plan to do a part 3, where I'd planned to address that, but until then, check out any of these blog entries which are not CF-specific (and don't need to be) but are also rather spartan (but enough to get you going):

- http://toastergremli...
- http://serverfault.c...
- http://tisupport.fr....

The other approach is to lock it down by requiring web server authentication. The best choice for most would be Windows Authentication, where someone accessing these directories would be prompted for a valid Windows login (safely and encrypted, even over the web, what has in the past been called NTLM authentication).

This, too, requires using a feature not enabled by default. For more, see:

http://www.iis.net/c...
# Posted By Charlie Arehart | 1/3/13 10:38 PM
Just to clarify on the comment by Scott about CF admin using the adminapi, I am not sure if it does use it but even if you block it on the web server it doesn't prevent CF from using it internally, you can still use the admin API in your code if you block it on the web server using something like Request Filtering.
# Posted By Pete Freitag | 1/3/13 11:10 PM
Thank you Charlie. That is perfect. Looking forward to Part 3.
# Posted By Jeff | 1/4/13 9:47 AM
This command line will search for the h.cfm file on Windows. Run it from the root, such as D:\
dir /S /P "h.cfm"
# Posted By Troy | 1/4/13 12:28 PM
While it has been reported that the file is written as h.cfm, it could be written as any filename, anywhere the ColdFusion process/server can write.

I have also encountered compromised servers (not this attack) where the attacker has buried files in the system. One occasion was in /CFIDE/scripts/ajax/FCKeditor/editor/dialog/common and the file was called fck_dialog_common.cfm. Without knowing the file shouldn't be there it is nearly impossible to find. Really the only way to make sure something hasn't been modified or placed on the system is to compare against a good known copy or an intrusion detection system (IDS) is monitoring the file system.

The sad reality is that just cleaning the server up might not be sufficient and fully rebuilding it (properly configured and locked down) might be the best approach.
# Posted By David Epler | 1/4/13 3:25 PM
Would have been nice to remotely log in and check our servers. But upgraded to Windows 8 and all that was rooted....
# Posted By Peter Tilbrook | 1/5/13 12:31 AM
Yes, the file can be called anything the hacker wants. However, the script to exploit your server contains a few specific variables within the CFM file. To be sure you have not been attacked, search/grep all of your .CFM files for any of these words.

Fserverinfo
FcmdOpp
Ffileopp

(for our servers we simply searched for Fserverinfo) This script is nasty and reveals plenty from your server.
# Posted By RJ | 1/5/13 1:11 AM
Charlie:
thanks for keeping on top of this. I have heard from several people that they found the h.cfm file and the scheduled tasks. This is being actively exploited so the word needs to get out as fast as possible.

I have put a few thoughts out regarding how to detect code compromise that may be helpfull:
http://boncode.blogs...

Best.
Bilal
# Posted By Bilal | 1/5/13 5:30 PM
Adobe interestingly say (in http://www.adobe.com...) that the unauthorised access is only possible if you don't have a CFIDE password (!).
# Posted By Tom Chiverton | 1/7/13 4:21 AM
@Tom
I read the same advisory which left me confused. The sites I know of have all had their admin CFIDE password set. By the same token, I like to believe that Charlie would have a password for CFIDE admin access, but he may be able to comment. What I can puzzle together, the, "no password" bit surround RDS. However, if you follow best practice and disable RDS during install, you will not be asked for a password. This may be the "no password" thing that is being referenced. It may be advisable to assign an RDS password after the install even if you have disabled it initially.
# Posted By Bilal | 1/7/13 8:58 AM
@Bilal

I believe the RDS vulnerability was one that I reported. It allowed a user to bypass the login for the CF Administrator. I believe it was separate from Charlie's vulnerability, I can't be sure though because I haven't seen the details of it. When Charlie reported his vulnerability, I looked in to administrator.cfc myself and was able to bypass the login entirely.
# Posted By Scott | 1/7/13 9:09 AM
Sorry folks. I was just slammed last week (helping people deal with this) so simply could not keep up with the comments (here and in other entries). Let me now take care of the several I missed since I last replied here.

First, though, I do want to point out (for you getting these comments by email or RSS) that I have posted a Part 3 entry just tonight, offering info about Adobe's hotfix released today, and still more: http://www.carehart....

Second, note that the Adobe bulletins clarify that we want to also protect /CFIDE/componentutils as something not to leave open as unprotected. I just updated the blog entries to reflect that new info. While any current known vulnerabilities in there are indeed now fixed, there could be others that might appear later, so better to protect it now.

Now to the comments:

@Pete, yep, good point to clarify that locking down the /CFIDE/adminapi does not lock down use of the Admin. In that CFML code, the CFCs would be invoked internally. Web-accessibility to these adminapi would seem a very limited use case.

Indeed, to Adobe's credit, most of the methods in most of the CFCs were already marked as not remote (so couldn't be called via the web.) Sadly, as is now being acknowldedged or documented by others, it was a call to the login method of the administrator.cfc, which is in fact "remote", which had the vulnerability.

I'll discuss that more (to the degree that I can without exposing the exploit to be used against others not yet protected) in a planned part 4/post mortem entry I have planned.

@Jeff, thanks, and to your question about Part 3, again, I have since done that just tonight. :-)

@Troy, cool to see the offer of the command-line tool to search for a file across the server. Still, to that point...

@Dave, yes you're right that people ought not restrict their searching to just a file called h.cfm, or to look in a particular directory. indeed, I'll note that I had said above (in the original part 2 entry) that "you should also look to see if any files were created or modified since the date of the exploit." By that I did mean "anywhere on the server". I'll tweak that now in case someone finds it later.

But you're absolutely right that the best protection would be an IDS. And indeed, i see that 2 days later, you blogged about this, with respect specifically to CFIDE files, at http://www.dcepler.n... Check that out, folks.

BTW, about your opening sentence there, kindly referring to my blog entries: I think it could be misconstrued. You say, "there was a comment that got me a bit concerned where the post said all you had to do is search for h.cfm". Again, I did not say that. :-) Maybe you mean someone else did in the comments, but when you refer to "a comment" and "the post", the implication to most readers I think would be that it was my blog entry saying that, rather than a commenter.

#Peter, bummer about your Wndows 8 experience. (Somehow, I think that will be a common refrain for some time to come, as often is the case with new Windows desktop releases.) That said, aren't there any third party alternatives to allow you to remote into a server from Windows 8?

@RJ, that's interesting to hear. Again, yes, the file can be called anything. As for your saying that "the script to exploit your server contains a few specific variables within the CFM file", and that among those are Fserverinfo, I will just point out that the version of the file I found on the server (h.cfm) did not have those in it at all. Perhaps there was more than one kind of file that the exploit caused to be loaded, which shouldn't surprise us really. Once they could get on the server, they could cause anything to be put anywhere (assuming one didn't have other protections in place to limit where CF could write, as discussed again in the lockdown guides.)

@bilal, thanks for the kind regards, and for the additional info in your blog entry. And I see you also offer some IDS approaches, especially for the CFIDE and other directories. You also point out the benefits of your BonCode connector (alternative way to connect IIS to CF, Railo, or OpenBD.) Folks should check out his entry for more info on these and other things.

On your approach to an IDS, Bilal, here's a question for you and Dave: can you guys confirm that your approaches (of creating signatures for the files already there, to detect changes) will work no problem for these Adobe-provided CFM pages that are there, encoded (not plain text)? I would assume so, but since no one has asked on your blogs, I thought I'd ask here.

@Tom C, yes, that was indeed an interesting thing to observe in their bulletin. But let's be clear, it's not so much a "CFIDE password" (which I know many use as their term for the CF Admin) but specifically a discussion of both the RDS AND the CF Admin passwords. What is offered is a bit confusing, as Bilal also notes in his reply to you. I am hoping to have more to say on that matter in the planned part 4.

@Bilal, to be clear, I DID of course have a CF Admin password set, but I DID NOT have an RDS password set, because (as you have surmised) I had not chosen to install RDS on my prod server, so was not asked to create one. I suspect that this is indeed the root cause of the vulnerability, which I'm sure is among the things that were corrected in the hotfix today.

At least I hope it is, because the technote says only, "If RDS is enabled, password protection should also be set." It does NOT say that if it's NOT enabled, we should set one anyway. And the bulletin (pointed to by the technote) makes no mention of it at all. It's only that first advisory (http://www.adobe.com..., which Tom points to) which indicates merely that "customers using an unsupported version of ColdFusion (including ColdFusion 8.x and earlier) can protect themselves"...by..."Setting the password for Remote Development Services (even if RDS is disabled)." That said, this is not repeated in the "mitigations" section of that document, so many may not see it (whether it applies to them or not).

I will ask Adobe about this.

Finally, @Scott, that administrator.cfc login vulnerability was in fact the very one that I was hit by, as was everyone whose system I helped with. (Either the hackers tried it and got it, or thankfully for those on CF10, they were unable to get in.)

Again, I plan to share more in a part 4. For now, I need some rest! :-)
# Posted By charlie arehart | 1/16/13 12:49 AM
Charlie,
The suggested signature approach will work on encrypted and compiled files. If you generated a signature for cfide it should match across your servers.
It is pretty fast as well if you limit yourself to cfide only. However, my suggestion would be to profile your app as well and periodically check it.
# Posted By Bilal | 1/16/13 6:41 AM
@Charlie,

I clarified my post to indicate that it was a comment left on this post, not the posting itself. I do see how the was original phrasing could be confusing. Sorry about that.

While my post was limited to validating CFIDE, it is possible to extend it to other directories. It is just a matter of updating the hashsets when known changes are made. The intent was to provide a quick was to check CFIDE and to show what tools are out there for file integrity checking and possible IDS solutions.

I do plan on updating the hashsets I published for CFIDE at https://github.com/d... later today with APSB13-03 applied.
# Posted By David Epler | 1/16/13 7:36 AM
@Bilal and @David, thanks for your followup clarifications.
# Posted By Charlie Arehart | 1/16/13 12:33 PM
@Charlie,

W/o describing any implementation details, I'd just like to say:

Regarding "for those on CF10, they were unable to get in", I agree! And that's awesome news! I just wish Adobe realized this.. CVE-2013-0632 still says CF10 is affected :/ Another upside is that the componentutils issue is safely guarded from empty passwords in CF10.

Also, CF10 restricts cfschedule to only saving output as .txt/log. Meaning, *even if* CF10 was vulnerable to the empty-password issue, CVE-2013-0632's current rating of 10 (full compromise) would not be applicable to CF10 (the h.cfm hack is *absolutely* a non-issue on CF10).

I *really* hope Adobe corrects APSA13-01.

Thanks,
-Aaron
# Posted By Aaron Neff | 1/21/13 6:22 AM
Hi Charlie, we've found the h.cfm fileon a CF 801 box, but it does not execute unless called from localhost due to IP address restrictions. Calls to adminapi fckeditor all resulted in 403.6 errors. We do not see any evidence of an executable compromise. My question is the mechanism by which the file write happened in the first place-obviously that part worked and we want to understand why/how and understand how to block it.
# Posted By Mike | 1/30/13 5:59 PM
I'm sure Charlie is better at this than me, but I understand there is an issue around RDS- even if it was off, the password and username could still be used via the public CFIDE mapping, and if you never turned RDS on, these passwords would have some default value.
This access bypass seemed to be used to create a scheduled task to fetch content, write it, and execute it.
# Posted By Tom Chiverton | 1/31/13 3:57 AM
I followed the CF9 lockdown guide to the extent I could, as I will describe in detail below. But I would like to suggest a revision. The guide suggests the following:

To lock down /CFIDE, add the following lines to your /etc/httpd/httpd.conf file. This blocks all requests that for all IP addresses that start with /CFIDE, except 127.0.0.1. You might want to change this configuration to the IP address of an administration workstation instead to allow yourself access to the ColdFusion administrator.

<Location /CFIDE>
Order Deny,Allow
Deny from all
Allow from 127.0.0.1
</Location>

Yeah, right. But most of us are administrating servers that we do NOT have physical access to. The guide suggests changing 127.0.0.1 to the IP address of your location, but again, how many of us always connect to the internet from a single fixed IP address?

I had followed the rest of the guide, so I have /CFIDE/administrator behind a SSL login, but this left the rest of the directories under /CFIDE/ exposed, and I didn't grasp the risk.

Just now, I've added the following to my apache configuration ...

<Location ~ "/CFIDE/adminapi">
   Order Deny,Allow
   Deny from All
</Location>

<Location ~ "/CFIDE/componentutils">
   Order Deny,Allow
   Deny from All
</Location>

That, and following the recommendations in the lockdown guide for securing /CFIDE/administrator, seems to have locked down the vulnerable areas of CFIDE, and still allows me access from any IP to the CF admin panel.

If sufficient, I think this should be added to the lockdown guide to cover likely real world usage.

Until I get the patch installed, I'll remain anonymous ... as I'm late to discover this. ;-)
# Posted By An Anonymous CF Admin ... | 2/4/13 3:06 PM
I also note now, after looking again at what Pete Freitag suggests in the linked posts above, that these recommendations clearly are no longer enough as far as Apache goes. Don't know what can be done about that to publicize it more widely, but perhaps this is a start.
# Posted By An Anonymous CF Admin | 2/4/13 3:29 PM
@Anonymous CF Admin - can you clarify which linked article you are referring to, and which recommendation is falling short. Often my blog entries are referring to one specific risk, and do not mean to be all inclusive, if I need to clarify that in places I'd be happy to.

If you are referring to the CF9 lockdown guide -- If you had restricted /CFIDE access to 127.0.0.1 as the guide suggested you would not be vulnerable to this attack. I agree that it probably could have been clearer to spell out risks of not doing so, but to be fair this exploit was not known at the time of writing.

My stance has long been to block /CFIDE and only let through any URI which absolutely must be allowed.
# Posted By Pete Freitag | 2/4/13 3:50 PM
@Pete - In the first post, I'm referring to the lockdown guides, in the second to the linked blog posts that simply touch on the issue of locking down CFIDE.

The important detail here, I think, is that many of us don't have physical access to the servers we admin, nor do we access the internet from a fixed IP. Hence the following recommendation:

<Location /CFIDE>
Order Deny,Allow
Deny from all
Allow from 127.0.0.1
</Location>

is impractical, and that's where I got hung up following the lockdown guide. Hence the suggestion I made in the first post, which is definitely open to be improved upon, but for the moment, it seems to work.
# Posted By Anonymous CF Admin | 2/4/13 4:15 PM
@Pete ... would have been better if I had written "In the first comment I posted here, I'm referring to the lockdown guides, and in the second comment ... "
# Posted By An Anonymous CF Admin | 2/4/13 4:17 PM
@Anonymous CF Admin
It's trivial to add "Allow from" lines for all the hosts and/or IP ranges you are ever likely to need (office gateway, home ADSL, umm.. that's about it)
Permissive access (white list) to the Administrator is the way forward - you can do this at the Apache level, in your internet facing firewall, or whatever way best suits. Don't get hung up on the specifics.
# Posted By Tom Chiverton | 2/5/13 5:23 AM
@Tom

+1, It's pretty simple really, there are so many different ways to do it. Disallow anonymous users, IP Ranges, Request Filtering, Firewall..
# Posted By Scott Buckel | 2/5/13 9:21 AM
Wow, I've fallen behind on responding to comments here the past week. Just been really (really) busy helping people (in my CF server troubleshooting consulting)--and hardly any of it's been related to this hack. Just typical challenges that people face and often need help with.

I'll respond to each of the comments here (since Jan 30) in one reply (rather than annoy folks with lots of emails if I replied individually to each.)

@Mike, you asked about "the mechanism by which the file write happened in the first place-obviously that part worked and we want to understand why/how and understand how to block it."

I did address this above in an update I added Jan 15 (2 weeks before your comment here): "the adminapi exploit allowed the hacker to create a scheduled task, and then that task called a page on another server, and the scheduled task was configured to "save" the output of that to this h.cfm file."

Some good news is that folks on CF10 were not vulnerable to that specific hack, and those on CF 9 or below who block the adminapi (as I discuss above) or those on 9 who apply the Adobe hotfix (which came out and i discussed in my part 3) are now protected against this hack.

@Tom, yep, there's a specific connection with RDS security, as discussed in the Adobe technote.

@Anonymous, regarding your first note, you comment on the lockdown guide, and I see that Pete and others replied. But you also raised the concern that your having left the adminapi and componentutils open left you vulnerable. I'll just say that, if you didn't notice it above, i had specifically said that "one needs to close down public unfettered access to the /CFIDE/adminapi, /CFIDE/administrator, and /CFIDE/componentutils folders to prevent them being exploited again. That's THE MOST IMPORTANT thing to do first."

And as for your concern that you don't have access to the server itself (to lock things down to just localhost access) and that you need to access the Admin from more than any single IP address, I'll note that I had pointed out in both Part 1 and in comments above (before you wrote) that besides IP locking, one could (and should) consider adding additional web server authentication for accessing the CF Admin. That would address your concern.

But if your point is more about what is or is not in the lockdown guide, I'll leave that for Pete to address, as has has offered to do.

And @Tom's next comment (after your last one) and Scott's are good ones: there are indeed many ways to lock things down: at the web server, firewall, or other levels. It's tough for the lockdown guide (already now over 80 pages in the CF10 version) to get too detailed into the many ways people might lock things down.

The main point, as I'm sure you're asserting, is that it just needs to be clear about what should be locked down, and can point folks to any of the many options (in addition to offering details where it does).
# Posted By Charlie Arehart | 2/6/13 8:40 PM
I had a good suggestion come my way regarding locking down CFIDE, namely access only from 127.0.0.1 as below:

<Location /CFIDE>
Order Deny,Allow
Deny from all
Allow from 127.0.0.1
</Location>

And then use SSH to tunnel into your server with your browser to access the CF admin panel. Until today, I didn't know you could do that. For details, see "Use SSH tunneling to make your web browser think it is at Fermilab" http://fermilinux.fn...

As to Tom's suggestion to white IP ranges I might be on, I could call up my ISP and ask, but then my thought was that this might open this directory to everyone else in the country using my ISP. That might not be true, but I don't know how they rotate their IPs. And it doesn't preclude my ISP from changing IPs in the future. So unless I had a fixed IP, and colleagues to back me up while away (my applications and servers have always waited until I'm on vacation to fail), this solution doesn't seem optimal to me.

I like the Fermilab solution!
# Posted By An Anonymous CF Admin | 2/8/13 8:46 AM
Hi Charlie,
Thank you for these useful informations.

You said that "...I can without exposing the exploit to be used against others not yet protected" but I wanted to know why you cannot give implementation details about the hack ?

When you say that Adobe has already fixed it that means : Now Adobe has no responsibility against customers that does not use their hotfix. Feel free to correct if I'm wrong about it.

Kind regards,
# Posted By cfdev | 3/25/13 10:10 AM
First, we run MX7 and IIS6. Due to economic reasons, we can't afford to upgrade after we obtained 7 through a nonprofit discount that was eventually removed. So we are at risk. We simply can't afford to upgrade. It is what it is. Adobe was interested in nonprofit for a while but not anymore.

The h.cfm and i.cfm hit in early March but we reloaded the software, ran Norton and Kaspersky and Webroot, and the system is now apparently clean. Knock on wood.

Locked down CFIDE completely and now use Administrator after disconnecting from the Net. Better safe than sorry.

The problem now is we can't use charts.

Can we leave the CFIDE root open while locking down the individual directories such as Administrator and the others?
# Posted By Hal | 3/27/13 12:25 PM
In order to use CFChart you need to allow the URI /CFIDE/GraphData.cfm
# Posted By Pete Freitag | 3/27/13 12:29 PM
@cfdev

You can leave just the files in /CFIDE that you need. So, like Pete said, you can leave /CFIDE/GraphData.cfm in there. You could also just move the "adminapi" and "administrator" folders to other directories outside of your web root and then copy them back in when needed.

-Scott Buckel
# Posted By Scott | 3/27/13 12:33 PM
Thanks.

The CDFIDE/GraphData.cfm is generated when the call is made, but isn't there physically. Is there any other way to have that file generated somewhere else?

We don't use CFForm from Scripts.

If we move the other directories as suggested by Scott and/or lock them down, can CFIDE be exploited or a new h.cfm or i.cfm file be inserted?
# Posted By Hal | 3/27/13 1:08 PM
I'm not aware of any way to make it resolve to a different URI other than doing a cfsavecontent around the generated cfchart content and then replacing it with a new URI, you would then need to modify the servlet mapping to listen to the new URI as well. I've never done that in practice, but in theory it should work.

What I usually do if one of the URI's is needed within /CFIDE is to create an exception for that URI -- if you are using Request Filtering in IIS to block /CFIDE then you will need to change it to block all folders and files except for the ones you need. This approach is the one that is used in the CF9/10 lockdown guides, so you can reference that for more info.

You may also want to block /CFIDE/main/ide.cfm which is RDS's ServletMapping, like GraphData.cfm it is not really there, but CF responds to requests for it.

You can delete /CFIDE/administrator and /CFIDE/adminapi but keep in mind that there are other folders which could potentially be used for attack (and have in the past), it is best to block everything except for what your application requires.
# Posted By Pete Freitag | 3/27/13 1:40 PM
We've gotten the h.cfm hack under control. They continue to hit the h.cfm and help.cfm as well are running transverse attempts through Administrator and Adminapi but those are locked down as well as the entire CFIDE. We created some filter blocks I won't go into in the open, but our logs files give me a good indication what they are looking for so we can plan accordingly.

Our main problem now continues to be the Chart ability. Pete mentioned a modified servelet mapping, but that is above my ability to figure out.

By the way, this blog saved the bacon here. You guys get our new award, The Order of Keyboard.
# Posted By Hal | 4/17/13 12:37 PM
Would like to add a bit more information around: "A real gotcha: implicit access to the built-in web server root"

It seem that ColdFusion will serve any cfm file located in [ColdFusion Home]\wwwroot if the file is not found in the IIS or Apache webroot directory.

What happens is that when you install CF with the built-in webserver, it install the CFIDE directory in [Coldfusion Home]\wwwroot directory.
# Posted By John Shin | 5/23/13 6:11 PM
@John, yep, that is in fact what I was describing in that section. It is very important, as otherwise people may think they're protected when they could still be exposed.
# Posted By Charlie Arehart | 6/12/13 1:19 AM
Thx
# Posted By calumbo | 6/26/13 10:48 PM
Hi Charlie,

Would disabling CFHTTP stop the attack working?
# Posted By Paul | 10/14/13 5:59 PM
It might stop this particular package, but it's not the attack vector, only something done after someone else is already do anything they want on your server. The exploit could easily fall back to invoking Java directly or downloading wget
# Posted By Tom Chiverton | 10/15/13 4:12 AM
Yep, what Tom said. :-)

But Paul, if you have no need of CFHTTP on your server, then disabling it would be wise just to prevent any unexpected use of it in rogue code. Just beware that if someday you or another developer on your server tries to use it, and it fails, you/they will need to remember both how you disabled it (sandbox or resource security) and why. :-)
# Posted By Charlie Arehart | 10/21/13 7:08 PM
Yes but say if all they had managed to do was upload the script somewhere (i could not find) and i had then locked down CFIDE, and then disabled CFHTTP. Then they would not able to invoke JAVA then would they?
# Posted By Paul | 10/22/13 4:35 AM
@Paul, before I address your questions, I want to make a point.

I think it's risky trying to sort out all your needs here. I shared above some specific details about a specific threat. When you ask now about other ways that exploits can be perpetrated, we are going beyond the scope.

More than that, you are sharing only some info and we (Tom and I) are having to draw it out of you to try to give a good answer. Comments on a blog just aren't the best forum for that kind of troubleshooting, limited as it is (and as also can be mailing lists and forums).

Sometimes, there's just no substitute for you hiring someone to help with your specific challenges, addressing your specific environment, and your level of expertise. I point to such consultants (including myself) at cf411.com/cfconsult.

But in answer to your question, you make a proposition and end with "then they would not able to invoke JAVA then would they." Your conclusion doesn't follow your premise. The ability to invoke java has nothing to do with your locking down the CFIDE and CFHTTP. They're entirely separate. I suspect you know this, but you're not saying it, and so I want to be clear (for you and other readers).

If you are saying "I have enabled the CF Admin option to prevent access to Java objects", then yes, they would THEN not be able to invoke java.

But back to your original question, you ask if locking down CFHTTP would have helped prevent the attack above (that described in this blog entry from January, which did indeed hit so many people). Actually, I should have said (in following up Tom) that in fact, no, it would not have prevented it.

The attack above was perpetrated by two steps: first, the bad guys obtained CF Admin rights by accessing it (technically, the AdminAPI) over the web using a particular URL (which now fails to work if one has applied the update that Adobe came out with, also mentioned above).

Second, now that they had access, they created a scheduled task, and THAT is what pointed to the URL on another server which returned some CFML code as text, and which the scheduled task was set to save as a CFM file. (That behavior, of being able to save a cfm file as output of a scheduled task, is ALSO now prevented after applying the fix above.)

So you see, CFHTTP really had nothing to do with this one exploit. But as Tom said, once the bad guys could get code on your server (whether with this exploit or another), then in fact you MAY help yourself by locking things down so that CF code is more limited in what it can do (as discussed in the lockdown guide).

And Tom further wanted you to understand that just locking down CFHTTP didn't stop someone (who could run code on your server) from being able to run the equivalent of CFHTTP (whether as a java object call to the java httpclient library, or a cfexecute of wget, or whatever). Again, the lockdown guide discusses several things one could lockdown to gain maximum security.

But as I was trying to say in my last reply, whether you should lock such things down is a choice based on your needs. Obviously you can't lock down CFHTTP if your code uses it (though in CF Enterprise, with its Sandbox Security feature, you can assign such locking down of tags/function/features at a folder level so that it affects only code run within that folder, whereas with CF Standard its Resource Security feature applies such tag/function/feature restrictions at the entire CF server level).

Hope that helps. But I stand by my caveat above: I'm only able to offer info in reply to what you have shared, and what I can think right now to offer in response. This may or may not be a complete answer to your particular situation, and I don't warrant it to be such. :-)
# Posted By Charlie Arehart | 10/22/13 10:32 AM
I found another part of this attack that I don't think was mentioned.
Check your neo-security.xml for an added user.
I had my admin set to single user, so I didn't see until I switch to managed users.
# Posted By Steven Benjamin | 4/16/14 10:58 PM
At the bottom of my neo-security.xml file I found an entry that looked like this:
<var name="ghostboss9191">
<struct type="java.util.HashMap">
<var name="username">
<string>ghostboss9191</string>
</var>
<var name="exposedServices">
<array length="1">
<string>mail</string>
</array>
</var>
<var name="description">
<string>vide</string>
</var>
<var name="roles">
<array length="3">
<string>coldfusion.administrator</string>
<string>coldfusion.rds</string>
<string>coldfusion.adminapi</string>
</array>
</var>
<var name="sandboxes">
<array length="1">
<string>C:\ColdFusion9\wwwroot\WEB-INF\</string>
</array>
</var>
<var name="password">
<string>3A0B94AEF9CF31D5F066E75ACAEC351EBA36EE78</string>
</var>
# Posted By Steven Benjamin | 4/17/14 7:35 AM
BlogCFC was created by Raymond Camden. This blog is running version 5.005. (Want to validate the html in this page?)

Managed Hosting Services provided by
Managed Dedicated Hosting