Get the Source of Webpages in AppleScript

There is a shell program that allows you to get the source of a page called curl.

To use curl in a script, just use:

set the_source to do shell script "curl http://www.theurl.com"

Obviously, replace the URL in the above sample with the URL you want to get the data from. Unfortunately, this doesn't work with every site, such as Google search results.

Once you have the source, you then need to use AppleScript's text item delimiters to parse the text.