Monday, September 27, 2010

Retrieve web pages using perl

Here is the sample script to fetch web pages using perl.

#!/usr/local/bin/perl -w
use strict;
use LWP 5.64;
use URI;

my $browser = LWP::UserAgent->new;
$browser->cookie_jar({});

my @browserHeaders = (
'User-Agent' => 'Mozilla/4.0; (compatible; MSIE 6.0; Windows NT 5.1; en-US)',
'Accept-Language' => 'en-US',
);

my $response;
$response = $browser->get("http://vishnuagrawal.blogspot.com",@browserHeaders);

if ($response->is_success) {
print $response->content;
} else {
print $response->error_as_HTML, "\n";
print $response->status_line, "\n";
}

print $response->code(), "\n";
print $response->message() , "\n";
print $response->header('content-type'), "\n";

my $url = URI->new("http://www.softwareqa.com");
$response = $browser->get($url,@browserHeaders);

print $response->content;

No comments: