following the HTTP dialog in curl and browsers like Firefox and Chrome

I honestly wonder, why curl and the browsers do not show you the HTTP status code during your web interactions "by default".

Usually I personally employ curl on the command line with --write-out '… %{http_code} %{url_effective}  …'.

Add-ons for Firefox:

• LiveHTTPheaders
• HTTP Resource Test

Add-ons for Chrome:

• HTTP Headers
• REST console
• Advanced Rest client
• this search will lead you to quite some different tools

Actually in Chrome you can follow the description here, it only employs internal means.

my approach to HTTP scripting, web harvesting, page scraping, and all that

• use LiveHTTPheaders in Firefox
• resp. ieHTTPHeaders in IE for extracting the relevant HTTP traffic;
• run my script scan_HTTP_log_file.pl on that and create a raw perl script (raw because e.g. it doesn't know, that it includes session IDs etc. as constant literals),
• that makes use of my JHwis toolkit around libcurl, it also includes a truely working cookie machine.
• on the Curl web-site you can also find formfind.pl, that helps a lot in finding all the "input" tags.
• now you have a raw perl script, that you can enrich will all necessary condition handling – there is all kinds of stuff in that code (like in any code generated from visiting a web-site), that you need to replace by dedicated handling, like session IDs and similar fields and URL ingredients; associate coordinate names with the resp. image names.
• on CPAN lives a module by the name of HTML::TableExtract for extracting HTML tables;
  I wrapped it up a little, so that I can make use of it on a command line.
• that command line utility supplies me with all necessary options for related tasks regarding navigating through HTML and all its tables, just what HTML::TableExtract actually does.
• I love writing "small" utilities in perl resp. ruby, that I wrap up bash resp. zsh scripts.

Years ago (I guess the situation is not so much different nowadays) libcurl was just so much more powerful than LWP, that I simply had to got for libcurl. Read "Using cURL to automate HTTP job" aka "The Art Of Scripting HTTP Requests Using Curl", that's IMHO the major evergreen in that area. libcurl does a lot more than just simple PUT, GET and that sort of thing.
All that makes my swiss army knife of web harvesting and page scraping.
Reality is seriously more challenging than text book examples, trust me!
Right, I could make all of that open source. I just recently started my open sourcerer career, after SzabGab had stayed in my place for a couple of days around LinuxTag.org/2010 at Berlin.

I should also mention Daniel Stenberg, the father of curl. IMO without his great work the art of HTTP scripting would not stand, where it stands today.

Right: last not least: no, I am not into dealing with AJAX and all that. For the last couple of years my approach has been: with the toolset I described above I can still manage "all" tasks without caring for AJAX. It does not matter enough.

Firefox-3.6 and certain extensions

Whenever we are offered another nice major release of Firefox, I always fear, certain extensions won't run any more and I have to find replacements for them. Now, when 3.6 and it RCs got released, there were a couple of weeks of big uncertainty (at least for me), during which I really didn't know, whether these extensions would follow this release step.

There is Sage, the lightweight RSS and Atom feed reader, (article on wikipedia (I just updated it), its home page). If you follow more than two or three feeds, what other tool can you actually use? I hope, its developers will never abandon this very nice piece of software.

There is "Live HTTP headers" (aka "LHH"), an extension for tracing the HTTP protocol traffic between you browser and the HTTP servers you are talking to. It has a mozdev home page.
This is a Firefox extension intended to be used by software developers.
There are also extensions to this extension like LHHreplay.
There is an extension to Microsoft's IE by the name of ieHTTPHeaders, that is rather similar to LHH.
I heavily depend on both of them (LHH and ieHTTPHeaders), as I use their trace files in order to generate (raw) perl scripts from them to mechanize web server accesses.
I started several years ago with several such projects to run a lot of my daily tasks, tasks like downloading market data or downloading bank account statements. And I add a few such scripts every year. Just recently I created a script to download the call record from my telephone+Internet router, an AVM fritz box.
A couple of weeks ago, there was a chap from Microsoft, who told me, he wanted to try my software for personal use. After I sent him my software, I never heard from him again.
Maybe my software is just too bad read and too ugly (a colleague of mine (a rather ego-centric guy) keeps saying so), but maybe he is just not very polite.