Sunday 29 January 2012

xargs


xargs
Xargs  is a command which can pass arguments from the output of one command to another. It can run multiline commands by passing arguments in a single file. The  most important feature is that it  creates a  list of arguments to be passed to the command and runs them.
Example1)
Consider that you have many files  and  each  have to be  renamed  with a common subscript letter.

/home/Krishna $ ls    account_[0-9].txt
account_4
account_5
account_7

Then to rename  all these files  as  < filename>_old  , use xargs as follows.

/home/Krishna$ ls  account_[0-9].txt  | xargs  -I  { }   mv   { }   { }_prev
/home/Krishna $ls    account_[0-9]*
account_4_prev
account_5_prev
account_7_prev
..

Here, xargs passed the output of ls, which is  a list  as  an argument list which is denoted by 
‘{  }’   to  the  mv  command .the  the entire line can be reconstructed as
mv  account_4   account_4_prev
mv   account_5  account_5_prev
….
..

Example2)
On few occasions, many unwanted processes  of a common type  may be running and  it is necessary to kill all of them without killing any other  process. One method would be to kill all of them with their individual PIDs, but  it may be not possible to do this in scripts, xargs does the job.

When  it is required to kill all processes run by your login name that have names containing  ftp in it,use
ps  -fu    $LOGNAME  |  grep  ftp  |  awk  ‘{ print  $2 }’  | xargs  -I   { }   kill  -9  { }


Example 3)
 You want to copy  a large number of  files  into  a directory  placing all the filenames  in a list file.
Consider a file containing  list of all the filenames .end line contains the destination directory

/home/Krishna$ cat   candidates_logs.txt
Arjuna.log
Yudhistir.log
Bhim.log
Nakul.log
Sahadev.log
..
/var/tmp/logs

You want to copy all these files to a directory  /var/tmp/logs with timestamp. use  xargs as follows.

/home/Krishna$ xargs   cp  –p  <  candidates_logs.txt
The command  structure created by xargs was
cp  -p   Arjuna.log  Yudhistir.log  Bhim.log  Nakul.log  Sahadev.log    candidates_logs.txt

This avoided the typing of all the files in an entire line. You may use this form of xargs in scripts where it is easier to make the list file using  simple echo and  “>>” .


Example 4)
You are redirecting  output containing contents of a file to another and you want a delimiting character  after every  n words , where n is any natural number, use xargs as follows.
a point to note here is that  a word can be a single line if it contains  only one word per line.

/home/Krishna$  cat   Train-Time.list
S29        F             12           Mumbai_CST     Karjat                   7:00pm  
T113     F             12 (x)   Mumbai_CST     Thane                   7:04pm  
K95        S              9 (x)      Mumbai_CST     Kalyan                  7:06pm  
A59        S              9             Mumbai_CST     Ambernath         7:15pm  
BL39     F             12           Mumbai_CST     Badlapur             7:17pm  
T115     S              9 (x)      Mumbai_CST     Thane                   7:20pm  
N27       F             12           Mumbai_CST     Kalyan                  7:21pm  

You can use awk to delimit the fields with  a “|” or any other delimiting character, but  xargs can  perform it too.
/home/Krishna$ cat   Train-Time.list | xargs  –n1  | xargs   -I  { }  echo  “{ }|”
S29|       F|            12|         Mumbai_CST|   Karjat|                  7:00pm|  
T113|   F|            12|         Mumbai_CST|   Thane|                 7:04pm|  
K95|      S|            9|            Mumbai_CST|   Kalyan|                7:06pm|  
A59|      S|            9|            Mumbai_CST|   Ambernath         7:15pm|  
BL39|   F|            12|         Mumbai_CST|   Badlapur|            7:17pm|  
T115|   S|            9|            Mumbai_CST|   Thane|                                 7:20pm|  
N27|      F|            12|         Mumbai_CST|   Kalyan|                7:21pm|  

You can trim the ending  “|” character by using sed .

Here you could have also used values greater than 1 after  n  so that the character “|” might be inserted after   those  many   words instead of 1.

Example 5)
When running commands by  passing multiple arguments , it sometimes becomes necessary to interactively  ask you to run it for each argument as seen in the commands  cp  -i  and  mv  -i.
xargs  using  its  -p option, can perform this on any command you want to run  interactively.

Consider a case where you need to take  tar  of files in a directory  by adding  every file  interactively  into the archive file.

/home/Krishna$ ls  |  xargs   -p   -n 1   tar -cvf   /backup/ALL_KRISHNA.tar
tar -cvf   /backup/ALL_KRISHNA.tar  account_4_prev   ?...y
tar -cvf   /backup/ALL_KRISHNA.tar  account_5_prev   ?...y
tar -cvf   /backup/ALL_KRISHNA.tar  account_7_prev   ?...n  ----------> your reply.
tar -cvf   /backup/ALL_KRISHNA.tar   jokes_old   ?...y
..
..

who


who
who  gives information regarding the users logged in the system, terminals, the  client IP addresses,the start and end times of previous logins and  their other  command  details. It s output is different from that of finger  command. “who” can give the  old details of users processes which  finger cannot


Example 1)
To know the details of users currently logged in a machine,  along with the header, use  who  command  with  –Hu.

home/kamlesh$who   -Hu
Name        Line                Time              Activity          PID                Hostname
kamlesh     pts/0       Jan 07 23:07    15:25        5206202             (169.173.25.64)
yadav         pts/1       Jan 08 07:40     0:22         3735576              (169.8.25.100)
stella          pts/2       Jan 08 14:47     0:35         5124214              (169.8.25.114)
jaffer          pts/5       Jan 08 16:17      .               4108396              (169.8.25.100)


Name
            Identifies the user's login name.
   
Line  
            Identifies the line name as found in the /dev directory. It is actually the terminal number  of   the user.

 Time
            Represents the time when the user logged in.
     
 Activity
            Represents the hours and minutes since activity last occurred on that user's line. A . (dot) here       indicates line activity within the last minute. If the line has been quiet more than 24 hours or has not been used since  the last system startup, the entry is marked as old.
     
 PID
            Identifies the process ID of the user's login shell.
    
 Hostname
            Indicates the name of the machine the user is logged in from.


Example2)
Who command gets the  information from  /etc/utmp file. If you wish it to use another file, specify the  filename  after who.

To get the output  from    /var/adm/wtmp.

home/kamlesh $who  -Hu    /var/adm/wtmp
Name                    Line                      Time                Activity                  PID              Hostname
kamlesh               pts/2                Dec 28 00:26                .                   930160        (indra)
sneha                  pts/2                  Dec 28 01:09                .                  156080        (indra)
krishna                pts/1                  Dec 28 02:33                .                  660214        (169.8.25.100)
krishna                pts/2                  Dec 28 02:37                .                  942618        (169.8.25.117)
justin                   pts/2                   Dec 28 02:51               .                 934590        (169.154.25.65)
varsha                 pts/2                  Dec 28 03:02                 .                  672308        (indra)
jaffer                   pts/2                   Dec 28 04:27               .                  267016        (169.8.25.100)
steve                  pts/3                    Dec 28 05:33               15:16           610818        (indra)
..
..
^c

All such  old sessions are displayed . so to keep a track on people who logged in and out ,the command helps.

Example 3)
To know the details of your  current session , ie the session from which you are running the command , an easier to remember command is used

home/kamlesh $ who am i
kamlesh    pts/1       Jan 08 01:30     (169.173.25.64)


Example 4)
In multi user work  environments, you may  require to know  whether your colleague is   logged in from a remote machine or  check continuously  whether  he/she has logged in ,
 Here is a script to do that.

#!/usr/bin/ksh
while [ 1 ]
do
who|grep  10.120.245.60
if [ $? -eq 0 ]
then
      while [ 1 ]
      do
      echo "\a Logged in"
      sleep 1
      done
fi
echo "Not logged in yet"
sleep 5
done



wc


wc
wc is a very useful utility which can count lines, characters, words, bytes etc in a plain text file
or standard input. In shell scripts, It helps to  store a value for the total lines of a file or output of
a command  into a variable for subsequent use. The examples below  show how these things can be achieved.

Example 1)
To  get  the counts of lines,  words  and  bytes of a file  use wc as follows.

/home/mark$ wc    Bulk-SMS-file.txt
 45      140      990------------------------------> No   of   bytes
 |              |---------->  No  of  Words                   
No of lines                                                   
                                


 The individual values can be retrieved as field through awk , but wc provides options.

wc   -l      :   Total   Lines

wc   -w    :   Total   words

wc    -c     :  Total   bytes

wc   -k     :  Total   characters

The wc  command considers a word to be a string of characters of non-zero length which are delimited by a white space and  lines are counted when  newline characters occur.


Example 2)
Using  wc  on multiple files.

/home/mark$ ls  file_list-201111*
file_list-20111105.txt
file_list-20111113.txt
file_list-20111120.txt
file_list-20111128.txt

  /home/mark$ wc   -l    file_list-201111*
  98164   file_list-20111105.txt
  531665 file_list-20111113.txt
  527303 file_list-20111120.txt
  564207 file_list-20111128.txt
 1721339 total

This  gives  you the  no of lines  in  every file as well as the sum of  all the individual  line counts.

Same command  can be  run without the  -l option to get the counts of various parameters and their totals.

Example 3)
wc   can also count these  values  from standard  output through pipes.

/home/mark$ echo  "I  Love  You" | wc
      1       3      17

Hmm!   wc  achieved  something cool  this time…

Similarly you can use wc  -l  as an alternative to grep  -c.
/home/mark$  grep  –c   tremendous    Director-speech.txt 

Is same as

/home/mark$ grep   tremendous    Director-speech.txt  | wc  -l 


Example 4)
wc  -l   may consume a lot of time on counting  the line numbers when the size of the file is huge.
If  we are sure that every line in a file  contains equal number of characters, there  is a faster method to achieve it.

Assume that  you have a continuous file, i.e. a file which  has same number of characters (or bytes ) in each line.
/home/mark$  ls   -lrt   All_Customers.lst
-rw-r--r--    1 mark   Administ    1321655460912 Dec  16 14:37 All_Customers.lst

/home/mark$  wc   -l   All_Customers.lst
23456898

The following steps explain the method.

Step1)
save the  first thousand lines of the file in a separate  file.
/home/mark$  head   -1000   All_Customers.lst   > All_Customers_1000.lst`

Step2)
get  the ratio of  the  total size of the file to the  line count(1000) of the file.
/home/mark$  ls  -lrt  All_Customers_1000.lst
-rw-r--r--    1 mark   Administ    56344000 Dec  16 14:37 All_Customers.lst

/home/mark$  fsize=` ls  -lrt  All_Customers_1000.lst | awk  ‘{ print  $5}’`
/home/mark$  fratio=`expr   $fsize  /  1000`
/home/mark$  echo  $fratio 
56344

Now ,fratio actually stores the number of bytes per file.

Step3)
Now divide  the total size of  All_Customers.lst with  fratio.
/home/mark$tot_lines=` expr  1321655460912 /  $fsize`
/home/mark$echo  $tot_lines
23456898

Which is same as calculated by wc  -l.
All These steps can be used in a shell script  for any given file of this type.

If   there are multiple files of such types (such  as file_list-201111*)  and all have the same fratio.(defined above),then  you can  write a script to get the similar output  as that of
 wc  –l   file_list-201111*

The script is as shown.
fratio=313
tot_val=0
for stream in `ls file_list-201111*`
do
str=`ls -lrt $stream|awk '{print  $5 " + "}'|tr -d "\n"|sed 's/$/0/'`
val=`echo "($str)/$fratio"|bc`
echo  “$val  $stream "
tot_val=`expr $tot_val + $val`
done
echo  " $tot_val   total " 


Here ,between the output  of ls  giving sizes of all the files ,a ‘ +’  symbol  is placed and 0 at the end and is passed to bc command as an expression whose value (the sum)  is divided by the
ratio  just as explained in the 3 steps above.