Retrieve web pages using perl

Here is the sample script to fetch web pages using perl.

#!/usr/local/bin/perl -w
use strict;
use LWP 5.64;
use URI;

my $browser = LWP::UserAgent->new;
$browser->cookie_jar({});

my @browserHeaders = (
'User-Agent' => 'Mozilla/4.0; (compatible; MSIE 6.0; Windows NT 5.1; en-US)',
'Accept-Language' => 'en-US',
);

my $response;
$response = $browser->get("http://vishnuagrawal.blogspot.com",@browserHeaders);

if ($response->is_success) {
print $response->content;
} else {
print $response->error_as_HTML, "\n";
print $response->status_line, "\n";
}

print $response->code(), "\n";
print $response->message() , "\n";
print $response->header('content-type'), "\n";

my $url = URI->new("http://www.softwareqa.com");
$response = $browser->get($url,@browserHeaders);

print $response->content;

Find the day diff between two days

Following is the shell script solution for getting number of days between two dates:

#/bin/sh
date1=`date +%s -d $1`
date2=`date +%s -d $2`
diff=`expr $date2 - $date1`
diff_days=`expr $diff / \( 60 '*' 60 '*' 24 \)`
echo $diff_days


[vishnu@ shell]# ./daydiff.sh 2010-08-02 2010-09-15
44
[vishnu@ shell]#

Here is the perl solution for the same:

#!/bin/perl
use strict;
use warnings;
use Time::Local;

my $date1 = $ARGV[0];
my $date2 = $ARGV[1];

my @formatted_date1 = split(/-/, $date1);
my @formatted_date2 = split(/-/, $date2);

my $local_date1 = timelocal(0, 0, 0, $formatted_date1[2], $formatted_date1[1], $formatted_date1[0]);
my $local_date2 = timelocal(0, 0, 0, $formatted_date2[2], $formatted_date2[1], $formatted_date2[0]);

my $diffSeconds = $local_date2- $local_date1;
my $diffDays = $diffSeconds / (60 * 60 * 24);
print "Day diff :: $diffDays\n";

[vishnu@ shell]# ./daydiff.pl 2010-08-02 2010-09-15
Day diff :: 43
[vishnu@ shell]#

Here is modified version of above shell script, if you want to find out time difference in seconds

#!/bin/bash

d1=`date +%s -d "$1"`
d2=`date +%s -d "$2"`
((diff_sec=d2-d1))
echo "Diff Seconds : $diff_sec"

[vishnu@ shell]#./date.sh "2011-07-19 22:44:34" "2011-07-20 02:04:14"
Diff Seconds : 11980
[vishnu@ shell]#

Dump cpu and memory usage of a process

Here is the automated shell script which can dump the memory and cpu usage of a process. Helpful in cases when you want to monitor a process for a long time.

#!/bin/bash

if [ "$#" -ne 1 ]; then
echo "Usage ... [$0 <pid>]"
exit
fi

pid=$1

# maxRunTime is used to control the script, so that it run only for specified number of seconds
maxRunTime=72000
# sleepTime is used to sleep for specified seconds, to take the memory dump again
sleepTime=120

logFile=${pid}_process.log
commandFile=${pid}_command.log

`ptree $pid >> $commandFile`;
`pargs $pid >> $commandFile`;

currentRunTime=0

while [ $currentRunTime -lt $maxRunTime ]
do
time=`date +%F::%X | tr -d '[\n]'`
line=`ps -eo pid,pcpu,vsz | grep $pid`;
echo "$time $line" >> $logFile
sleep $sleepTime
currentRunTime=`expr $currentRunTime + $sleepTime`
echo "Execution Time : ${currentRunTime} seconds [max ${maxRunTime}]"
done;