[Looking for Charlie's main web site?]

CFFundamentals: Mimicking a form submission using CFHTTP

While developers who've used CFML for a long time will regard some topics as old hat, we have to remember that there are folks who either have come along recently or simply never used some feature. This is one of those topics, and since I answered it on a list, I offer it here.

I'd like to start doing that more often (reformatting some answer I give on some list, so that others may learn from it.) As in this case, I'll refer to such as CFFundamentals and create a category for them so that interested readers may find them more readily.

The problem

Someone wrote wanting to mimic (in CFML code) the submission of some data to a form that they saw happening within an Airborne Trackship (http://track.dhl-usa.com/TrackEmail.asp) page.

They saw that the form submitting to that allowed one to enter an email address and some tracking numbers to have a status email sent back to that address. Using "view source" in the browser, they observed that the form actually called a Javascript form which submitted the form to TrackEmail.asp?nav=TrackByEmail. Since this was on a page at http://track.dhl-usa.com/, they realized they'd need to submit to http://track.dhl-usa.com/TrackEmail.asp?nav=TrackByEmail.

They also noticed that the form had two input fields, txtTracknbrs and txtEmailAddress, so they tried to put together a CFHTTP request that tried to submit to the form using this URL:

http://track.dhl-usa.com/TrackEmail.asp?nav=TrackByEmail&txtTracknbrs=nnnnnnnnn&txtEmailAddress=someemail@address.com

It didn't work. They asked if anyone knew why.

My Response: He was close, but needed to understand a couple of points

Here's what I wrote:

I think your problem is in presuming that the Javascript method builds a request that looks like the one you tried in CFHTTP. I just reviewed the page, and I see that it's a form, with a METHOD=POST, and so your CFHTTP needs to send its info in the way a form post would. And that's NOT by passing the variables on the URL as you have. (One may argue that they can write CFML apps that accept either FORM or URL variables, but the page being called is an ASP page, so we can't make that presumption.)

With that, I changed your CFHTTP to the following and it worked (you should be getting an email soon, as it told me the request was accepted). Try it yourself:

<cfhttp url="http://track.dhl-usa.com/TrackEmail.asp" method="POST">
   <cfhttpparam type="URL" name="nav" value="TrackByEmail">
   <cfhttpparam type="FORMFIELD" name="txtTrackNbrs"
value="nnnnnnn">

   <cfhttpparam type="FORMFIELD" name="txtEmailAddress"
value="someemail@address.com">

</cfhttp>

<cfoutput>
#cfhttp.filecontent#
</cfoutput>

Note that it uses a TYPE="URL" for the nav query string value (as that WAS passed on the URL), but uses TYPE="FormField" for the form fields, since those are passed as form fields. You were right in getting the exact names of the form fields as used in the form fields of the form. That's critical as well.

I'll point out as well that when trying to mimic such forms in CFHTTP, it's also sometimes critical to send along any hidden form fields. There were two on that form, but I left them off and it seems to work. IF you found you needed to add them, just add them as more TYPE="FormField" values.

I should note as well that sometimes when trying to simulate a form submission, you will also need to send along any cookie values that might be being sent by your browser to the server. Those aren't shown in the form. You could use browser/server proxy tools (like the free Fiddler or Firebug tools) to detect what's being sent along.

Hope that may help other readers.

I'll add in this blog entry that regarding that last point, about using HTTP proxies to study the browser/server communication, I wrote about those previously in:

Alternatives HTTP debugging proxies, for debugging

Comments
Hey - thanks for the little lesson... i've been trying to do something like this but with a form that requires login... is there something special i have to do to handle that?
# Posted By Mitch | 2/19/07 8:39 PM
Well, you just need to think (or make your CFHTTP processing work) the way the form would be processed.

If the form requires a login, does it do it by redirecting to a login page? If so, then you need to CFHTTP to that login page instead. If that sets a session variable, then the CFHTTP needs to capture the cookie sent back to the client which is used as the key to the session, and then it needs to pass that cookie on a subsequent page visit (after the login).

As I mentioned, this is where using a proxy/sniffer can be useful, to watch the kind of back and forth from the server to the browser to see what happens. Hope that's helpful.
# Posted By Charlie Arehart | 2/19/07 11:43 PM
+1 thanks
# Posted By cyberman | 9/2/07 1:11 PM
I have a question.
I use cfhttp to make a search to a web site and i'm getting back the filecontent allright, but how can i get the url of the result page?

For example:
i use the cfhttp page to make a search like this http://www.site.com/...=john doe

And the form submit's to a page like:
http://www.site.com/...

how can i get this ^^^ url?
# Posted By Spame | 1/25/08 6:25 AM
Hi Spame, you don't (typically) want to use CFHTTP to call the search page (if it shows a form). Rather, you want to use CFHTTP to submit your search request to the form's action page (as the form would have done.) That's the point of the blog entry above, though I was focused on a particular challenge that a list member brought up. I see now that I didn't explain that one point about getting the form's ACTION attrubute, as I kind of presumed people reading this had experience with CFHTTP. My bad. Thanks for asking.

So, you want to view the source of the search form page (use "view source" in your browser, as I'd mentioned above). Identify what the form's ACTION attribute is, and you would use that as the URL for the CFHTTP, passing in the form fields with any values to simulate the form passing them in (using cfhttpparam type="FORMFIELD" as I show above). And again if the form ACTION you find passes in any querystring (as shown above with nav=TrackByEmail), then you pass that in cfhttpparam type="URL".

You mention passing in "john doe" (though you seem to have left off the query string variable name for it). If doing that would show the form with that "john doe" value filled in, then again you don't want to CFHTTP to it. You want to view the source of that form to see what URL the form itself calls, and CFHTTP to that.

Now, you show that the result page you'd expect to see would have a URL like http://www.site.com/... I don't think that's the URL you want to use for the CFHTTP. In fact, I'm betting that the form's action page does its lookup (based on what's passed in) and then redirects (does a CFLOCATION) to that page (having found the "id" based on the input search criteria). If that's the case, you still want to do everything I say above, but add REDIRECT="yes" to the CFHTTP, so that CF will follow the redirection from the action page to this results page. I'm just guessing here.

Hopefully it will now make sense.
# Posted By Charlie Arehart | 1/25/08 2:06 PM
Charlie thank you for the answer.
What i'm really trying to do is the following:
I'm using a form so the user can search by giving a search term, like "Small Barnacled Clam", then i use cfhttp to send the term to wowhead.com to search for this term.

In fact i'm doing something similar to this:
http://www.wowhead.c...?search=Small Barnacled Clam

If you follow the above link you will see that when wowhead finds the term, it redirects to another page placing the id (item) of the item to the url: http://www.wowhead.c.../?item=5523" target="_blank">http://www.wowhead.c...

That's what i'm trying and need to grab... the item number or the full URL. Is it possible to get this url? (even if the target cfhttp page redirected)
# Posted By Spame | 2/11/08 3:22 AM
Spame, as I said earlier, if you request one URL but get back another, then your browser is being told to redirect to another page. In CF, we'd cause that (if we were writing the page doing the redirection) using CFLOCATION.

As I alluded to above, the REDIRECT attribtue of CFLOCATION is the ticket to dealing with issues of redirection in pages you're requesting. Since you just want the URL that it WOULD redirect to, you would use REDIRECT="NO", then inspect the resulting CFHTTP struct to see what it showed. It turns out that in CF 6, 7, and 8 (at least) you will find the page they're redirecting to in the cfhttp.responseheaders structure as a key named location. The following code shows how you could see this. The first cfdump dumps the entire cfhttp struct, and the second shows you the value of the location field specifically. You'd use THAT for whatever it is you're wanting to do.

<cfhttp url="http://www.wowhead.c...?search=Small Barnacled Clam" redirect="No" >

<cfdump var="#cfhttp#">

<cfdump var="#cfhttp.responseheader.location#">

If I've helped you make some money with this, I would point you to my Amazon wish list button located on the right of the screen here. :-)
# Posted By Charlie Arehart | 2/11/08 2:38 PM
Thank you Charlie,
i'm gonna try it right now.
And LOL man, no it's just a test for my guild site :) It could be great if we could make some earn some from that tip! :p I need it to!
# Posted By Spame | 2/12/08 3:28 AM
Copyright ©2017 Charlie Arehart
Carehart Logo
BlogCFC was created by Raymond Camden. This blog is running version 5.005.
(Want to validate the html in this page?)

Managed Hosting Services provided by
Managed Dedicated Hosting