Sunday, 7 August 2011

Unix tool : make

The make utility is a software engineering tool for managing and maintaining computer programs.Make  provides most help when the program consists of many component files.
By creating a descriptor file containing dependency rules,macros and suffix rules,you can instruct make to automatically rebuild your program whenever one of the program's component files is modified.

To prepare to use make ,You must write a file called makefile the describes the relationships among files in your program,and the states the commands for updating each file.In a program,typically the executable file is updated from object files to decide which of the file needs to be updated.For each of those files,it issues the commands recorded in the data base.

Simple example:

This is an example descriptor file to build an executable file called requires the source file,,and include file mydefs.h,is required by files and you wanted to compile this file from the command line using C++ the command would be
        %  CC -o prog1
This command line is rather long to be entered many times as a program is developed and is prone to typing errors. A descriptor file could run the same command better by using the simple command
    % make prog1
or if prog1 is the first target defined in the descriptor file
    % make
No we are going to see how to write a descriptor file,this example descriptor file is much longer than the necessary but it is useful for describing what is gong on.

prog1 : file1.o file2.o file3.o
        CC -o prog1 file1.o file2.o file3.o
file1.o : mydefs.h
        CC -c
file2.o : mydefs.h
        CC -c
file3.o :
        CC -c
clean :
        rm file1.o file2.o file3.o

Let's go through the example to see what make does by executing with the command make prog1 and assuming the program has never been compiled.
  1. Make finds the target prog1 and sees that it depends on the object files file1.o file2.o file3.o

  2. make next looks to see if any of the three object files are listed as targets. They are so make looks at each target to see what it depends on. make sees that file1.o depends on the files andmydefs.h.

  3. Now make looks to see if either of these files are listed as targets and since they aren't it executes the commands given in file1.o's rule and compiles to get the object file.

  4. make looks at the targets file2.o and file3.o and compiles these object files in a similar fashion.

  5. make now has all the object files required to make prog1 and does so by executing the commands in its rule.

Unix tool : AWK

What is AWK?

AWK is a programming language designed for processing text-based data, either in files or data streams. It is an example of a programming language that extensively uses the string datatype, associative arrays , and regular expressions.

There are three variations of AWK:

AWK - the original from AT&T
NAWK - A newer, improved version from AT&T
GAWK - The Free Software foundation's version

AWK is an excellent filter and report writer.AWK also has string manipulation functions, so it can search for particular strings and modify the output.

The essential organization of an AWK program follows the form:

pattern { action }

The pattern specifies when the action is performed.i.e the pattern specifies a test that is performed with each line read as input. If the condition is true, then the action is taken. 

Lets look some examples,First of all create a file named "file1.txt", and write somthing like "unix tool awk " in the body and then save,then type the following command in terminal

$awk '{print $1}' file1.txt

The output will be
'unix', i.e it will print the first word in the body of the file.

Two other important patterns can be specified by the keywords "BEGIN" and "END.",these two words specify actions to be taken before any lines are read.and after the last line is read.THe AWK program below:

  BEGIN  ( print "START" )
                 ( print                    )
  END       ( print "STOP"    )

We can make this into a typical AWK program :

Firt of create two files like "sample1.txt "with body " sam 15 groupA" & "sample2.txt" with body "neil 13 groupB" ,then write our AWK program as described below and save the file with extention (.awk) in the same directory that your sample file resides  ,Here my awk program will be "file1.awk"

 AWK is also an interpreter,Here we making the file executable by adding the first line in the beginning of the file i.e (" #!/bin/awk  -f ").

programe name : file1.awk

#!/bin/awk -f

  BEGIN { print "Name\tAge\tGroup" }

  { print $1, "\t" ,$2, "\t" ,$3 }

  END { print " - DONE -" }

Then change the permission with the chmod  command (i.e "chmod +x file1.awk" ),and then the script becomes a new command.Here the "-f" option following "#!/bin/awk"  make the file execute directly i.e " awk -f file1.awk".

Then run the file by typing "$ awk -f file1.awk sample1.txt sample2.txt" in your terminal.then you will get the following output.
sreehari@~/sed awk make/awk09:54:02$ $awk -f file1.awk sample1.txt sample2.txt

    Name     Age   Group

    sam      15    groupA

    neil     13    groupB

    - DONE -

This is just a brief introduction to on AWK,You can find more about AWK by using the following document. AwK-Tutorial

Unix tool : Sed

What is Sed?

This post will help you to get the answer of the above question,The same question persuaded me to understand the concept and also to write this post about what i have learned about Sed.

sed(stream editor) is a tool which allows you to perform large-scale, noninteractive editing tasks. For example, sed can replace all the occurrences of a given word in a file with a different word, or delete all lines containing a target string from a file.Sed is particularly used when working with large files .

Substitution command  S:

Sed has several commands,but the essential command i.e used by most is substitute command : S
The substitute command changes all occurrences of the regular expression into a new value.

To try this command , create a file named "file1.txt" and write something like "sed is fun" in the body of that file,then run the following command in terminal.

      $sed s/fun/cool/ <file1.txt >file2.txt

After you run the command you can see a new file created in the same directory with a name file2.txt , when you open the file you can see that the text is changed to "sed is cool".here the command will substitute the content "fun" in the old file (file1.txt) to "cool" in the new file (file2.txt).

Try the following command those who want to test the working in terminal.
      $echo fun | sed s/fun/cool/

This will output as "cool" in terminal.

      $echo sed is fun | sed 's/fun/cool/'

This will output as "sed is cool" in terminal.

There are four parts to this substitute command:

s                      Substitute command.
/ . . / . . /          Delimiter.
fun                  Regular Expression Pattern Search Pattern
cool                 Replacement string. 

The search pattern is on the left hand side and the replacement string is on the right hand side.

Using "&"as the matched string :

By using the "&" symbol you can match the patterns as you like,

      $echo sed is fun | sed 's/[a-z][a-z][a-z]/(&)/'

This will ouput as " (sed) is fun ",  [a-z] matches single letter ,here 3 [a-z] will match 3 letters like(sed).

You can have any number of  "&"  in the replacement string,like

      $echo sed is fun | sed 's/[a-z][a-z][a-z]/(&&)/'

This will output as "(sedsed) is fun".

If you want to match the entire string then you have to make changes like,

      $echo sed is fun | sed 's/[a-z]* [a-z]* [a-z]*/(&)/'

  It will output as " (sed is fun)".

If you want to display the matched string 3 times then try,

      $echo sed is fun | sed 's/[a-z]* [a-z]* [a-z]*/(&)(&)(&)/'

The output will be "(sed is fun)(sed is fun)(sed is fun)".

If we have a text file like "sed is fun 123",then you should change the code like this to get the whole text inside the bracket.
      $echo "sed is fun 123" | sed 's/[a-z]* [a-z]* [a-z]* [0-9]*/(&)/'

Then the output will be "(sed is fun 123)".

Using \1 to keep part of the pattern:-

If you want to remove the numbers from the above example,then you should try like,

     $echo sed is fun 123 | sed 's/\([a-z]* [a-z]* [a-z]*\).*/\1/'

It will remove the numbers and the output will be "sed is fun".

If you want to switch the words around, you can remember the patterns and change the order around:

     $echo sed is fun 123 | sed 's/\([a-z]*\) \([a-z]*\) \([a-z]*\).*/\3 \2 \1/'

The output will be "fun is sed".

Substitute flags:

You can add additional flags after the last delimiter. These flags can specify what happens when there is more than one occurrence of a pattern on a single line, and what to do if a substitution is found.

      $sed 's/[^ ]*/(&)/' <file1.txt>file2.txt

This will put parenthesis around the first word, i.e the first word in the body of your second file have a parenthesis around it.

If you want it to make change of  every word add a "g" after the last delimiter and use the work-around:

      $sed 's/[^ ]*/(&)/g' <file1.txt>file2.txt

The output will be "(sed) (is) (fun)".

These are some basic commands ,there are several other commands that can be used with Sed.