#How to print/display the first line of a file?

There are many ways to do this. However the easiest way to display the first line of a file is using the [head] command. $> head -1 file.txt If you specify [head -2] then it would print first 2 records of the file. Another way can be by using [sed] command. [Sed] is a very powerful text editor which can be used for various text manipulation purposes like this. $> sed '2,$ d' file.txt

#How to print/display the last line of a file?

The easiest way is to use the [tail] command. $> tail -1 file.txt If you want to do it using [sed] command, here is what you should write: $> sed -n '$ p' test From our previous answer, we already know that '$' stands for the last line of the file. So '$ p' basically prints (p for print) the last line in standard output screen. '-n' switch takes [sed] to silent mode so that [sed] does not print anything else in the output.

#How to display n-th line of a file?

The easiest way to do it will be by using [sed] I guess. Based on what we already know about [sed] from our previous examples, we can quickly deduce this command: $> sed n ' p' file.txt You need to replace with the actual line number. So if you want to print the 4th line, the command will be $> sed n '4 p' test Of course you can do it by using [head] and [tail] command as well like below: $> head - file.txt | tail -1 You need to replace with the actual line number. So if you want to print the 4th line, the command will be $> head -4 file.txt | tail -1

#How to remove the first line / header from a file?

We already know how [sed] can be used to delete a certain line from the output by using the'd' switch. So if we want to delete the first line the command should be:

$> sed '1 d' file.txt But the issue with the above command is, it just prints out all the lines except the first line of the file on the standard output. It does not really change the file in-place. So if you want to delete the first line from the file itself, you have two options. Either you can redirect the output of the file to some other file and then rename it back to original file like below: $> sed '1 d' file.txt > new_file.txt $> mv new_file.txt file.txt Or, you can use an inbuilt [sed] switch 'i' which changes the file in-place. See below: $> sed i '1 d' file.txt

#How to remove the last line/ trailer from a file in Unix script?
Always remember that [sed] switch '$' refers to the last line. So using this knowledge we can deduce the below command: $> sed i '$ d' file.txt

#How to remove certain lines from a file in Unix?

If you want to remove line to line from a given file, you can accomplish the task in the similar method shown above. Here is an example: $> sed i '5,7 d' file.txt The above command will delete line 5 to line 7 from the file file.txt

#How to remove the last n-th line from a file?

This is bit tricky. Suppose your file contains 100 lines and you want to remove the last 5 lines. Now if you know how many lines are there in the file, then you can simply use the above shown method and can remove all the lines from 96 to 100 like below: $> sed i '96,100 d' file.txt # alternative to command [head -95 file.txt] But not always you will know the number of lines present in the file (the file may be generated dynamically, etc.) In that case there are many different ways to solve the problem. There are some ways which are quite complex and fancy. But let's first do it in a way that we can understand easily and remember easily. Here is how it goes: $> tt=`wc -l file.txt | cut -f1 -d' '`;sed i "`expr $tt - 4`,$tt d" test As you can see there are two commands. The first one (before the semi-colon) calculates the total number of lines present in the file and stores it in a variable called tt. The second command (after the semi-colon), uses the variable and works in the exact way as shows in the previous example.

#How to check the length of any line in a file?

We already know how to print one line from a file which is this: $> sed n ' p' file.txt Where is to be replaced by the actual line number that you want to print. Now once you know it, it is easy to print out the length of this line by using [wc] command with '-c' switch. $> sed n '35 p' file.txt | wc c The above command will print the length of 35th line in the file.txt.

#How to get the nth word of a line in Unix?

Assuming the words in the line are separated by space, we can use the [cut] command. [cut] is a very powerful and useful command and it's real easy. All you have to do to get the n-th word from the line is issue the following command: cut f -d' ' '-d' switch tells [cut] about what is the delimiter (or separator) in the file, which is space ' ' in this case. If the separator was comma, we could have written -d',' then. So, suppose I want find the 4th word from the below string: A quick brown fox jumped over the lazy cat, we will do something like this: $> echo A quick brown fox jumped over the lazy cat | cut f4 d' ' And it will print fox

#How to reverse a string in unix?

Pretty easy. Use the [rev] command. $> echo "unix" | rev xinu

#How to get the last word from a line in Unix file?

We will make use of two commands that we learnt above to solve this. The commands are [rev] and [cut]. Here we go. Let's imagine the line is: C for Cat. We need Cat. First we reverse the line. We get taC rof C. Then we cut the first word, we get 'taC'. And then we reverse it again. $>echo "C for Cat" | rev | cut -f1 -d' ' | rev Cat

#How to get the n-th field from a Unix command output?

We know we can do it by [cut]. Like below command extracts the first field from the output of [wc c] command $>wc -c file.txt | cut -d' ' -f1 109 But I want to introduce one more command to do this here. That is by using [awk] command. [awk] is a very powerful command for text pattern scanning and processing. Here we will see how may we use of [awk] to extract the first field (or first column) from the output of another command. Like above suppose I want to print the first column of the [wc c] output. Here is how it goes like this: $>wc -c file.txt | awk ' ''{print $1}' 109 The basic syntax of [awk] is like this: awk 'pattern space''{action space}' The pattern space can be left blank or omitted, like below: $>wc -c file.txt | awk '{print $1}' 109 In the action space, we have asked [awk] to take the action of printing the first column ($1). More on [awk] later.

#How to replace the n-th line in a file with a new line in Unix?
This can be done in two steps. The first step is to remove the n-th line. And the second step is to insert a new line in n-th line position. Here we go. Step 1: remove the n-th line $>sed -i'' '10 d' file.txt # d stands for delete

Step 2: insert a new line at n-th line position $>sed -i'' '10 i This is the new line' file.txt # i stands for insert

#How to show the non-printable characters in a file?

Open the file in VI editor. Go to VI command mode by pressing [Escape] and then [:]. Then type [set list]. This will show you all the non-printable characters, e.g. Ctrl-M characters (^M) etc., in the file.

General Unix Commands -Very Useful

Command alias Action Creates a temporary name for a Unix command. To remove an alias: ansiprint assets Bg Cat Cd Prints a file to a printer connected to your computer. Displays your account resources, use, and threshold. Places a suspended job in the background. Displays a specified file. Changes current directory to a different directory. To change back to your home directory: Directory abbreviations: ~ Home directory .. Parent directory . Current working directory chmod Changes permission status for a file or directory. To keep a file private: To give everyone read permission: compress cp Reduces the size of a file and adds .Z to the file's name. To restore a compressed file: Makes a copy of a file. To copy a file into a different directory: To inquire before overwriting an existing file: diff fg finger ftp grep Compares two files and displays the differences. Restarts a suspended job in the foreground. Displays user information at the specified computer. To change your finger entry information: Starts the file transfer program with a remote computer. Finds lines in a file matching a character pattern. -i Ignores case -c Lists count of lines that contain pattern -v Lists all lines except those with pattern -n Lists line number for each found pattern head Displays the first ten lines of a file. To display the first n number of lines: head file head -n file finger userid@computer chfn ftp address grep -icvn pattern file chmod code file chmod code directory chmod 600 file chmod 664 file compress file uncompress file.Z cp file1 file2 cp file directory cp -i file1 file2 cat file cd directory cd Example alias name command unalias name ansiprint file

help history

Displays an online help note. To display a list of help notes: Displays a list of your most recent commands. To repeat the nth command from the list: To repeat your most recent command:

help note help -l history ! n !! jobs job & fg %number z bg kill pid kill -9 pid ls -Fltra

jobs

Displays active jobs and their corresponding job numbers. To start a job in the background: To return a job to the foreground by the job number: To send a running job to the background:

kill

Terminates a process by its process identification number. To use the strongest form of the kill command: Ends your computer session. Lists the files and subdirectories in your current directory. -l Lists long format -t Lists by modification time -r Lists in reverse order -F Lists file type with special character -a Lists all files including dot (.) files

logout ls

lynx man mesg mkdir more mv

Starts a text-only Web browser session Starts a Web browser session at a specific Web address. Displays Unix Manual entry for a command. To search Manual Index for a keyword or topic: Displays your talk and write message access status. To set your message status to yes or no: Creates a new directory within your current directory. Displays a file one screen at a time. Use to scroll forward and q to quit. Moves a file to a different directory. Moves a directory to a second directory. To rename file1 as file2: To rename directory1 as directory2: To inquire before overwriting an existing file:

lynx lynx url man command man -k keyword mesg mesg y (or n) mkdir directory more file mv file directory mv directory1 directory2 mv file1 file2 mv directory1 directory2 mv -i file1 file2

passwd pico pine ps pwd rm rmdir spell telnet vi w wc

Starts program to change account password. Starts the Pico text editor with a file. Starts the Pine electronic mail program. Displays processes and their corresponding process identification numbers. Displays absolute pathname of current directory. Permanently deletes a file. To verify prior to removal: Deletes an empty directory. To verify prior to removal: Checks the spelling in a file. Starts a telnet session with a remote computer. Starts the vi text editor with a file. Displays a list of who is on system. Counts lines, words, and characters in a file. wc file rm file rm -i file rmdir directory rmdir -i directory spell file telnet address vi file pico file

Special commands and characters: Redirection : < Routes input to command from file > >> | Routes output from command to file Appends output to existing file Routes output between commands

Wildcards used in filenames : * Matches any number of characters ? Matches one character

Control codes : s Suspends the screen display q d c z u Restarts suspended display Signals end of file Cancels a command Suspends a process or job: use fg to restart Clears the command line

FILE COMMANDS:
Touch Create a new file. Usage: touch FileName cp Copy files. Usage: cp [options] FileName1 File Name2 cp [options] Options: -r recursively copy directory structures. mv Move or Rename files or directories. Usage: mv [options] file1 ./tmp/file1 mv [options] Options: -i query user for confirmation. rm Remove files. Usage: rm [options] filename Options: -r recursively remove directory structures. -i query user for confirmation. cat View complete file content. cat more View file contents in sections determined by the size of the terminal. Usage: more less View file contents in sections determined by the size of the terminal. Has more options and search features than more. Usage: less [options] compress Reduces the size of the file. and adds the extension .Z Usage: compress . uncompress /zcat : Restores a compressed file. Usage: uncompress Usage: zcat DIRECTORY COMMANDS cd Change directory. Usage: cd Eg: cd my-directory cd go to home directory cd .. go up one directory pwd Print working directory on the terminal.

ls List the content of a directory. Usage: ls [options] or ls [options] Options: -l list all files in long format. (permissions, users, filesize,date, and time are displayed). -a list all files including those beginning with a . -F list files distinguishing directories/ executables* symbolic links@ -R recursively list subdirectories encountered. mkdir Create a new directory. Usage: mkdir rmdir Remove a directory if its empty. Usage: rmdir SYMBOLIC LINKS ln Create symbolic links between files or between directories. Usage: ln [options] ln [options] Options: -s allows linking across file systems and allows the display of the links name upon ls -l. Eg: ln -s course-file myfile Eg: ln -s course-directory myspace TERMINAL COMMANDS clear Clears the terminal. echo: Write a string to standard output. Usage: echo string or echo string repeat Repeats commands. Usage: repeat HELP COMMANDS man Displays the manual page for the selected command. Usage: man rtfm Displays the man page and help files ont he terminal. Usage: rtfm INFORMATION COMMANDS history Lists the commands typed during the session. Options: -r displays the list in reverse. hostname Displays the computers or servers name on the terminal. who Displays who is on the system. who am i Displays the invoking user.

# How to zip/tar a file in Linux? Use inbuilt [zip/tar] command in Linux $> zip file.gz /Path/FilesToZip
$> tar -czvf myfiles.tar.gz /Path/FilesToZip

# How to unzip/untar/tar a file in Linux/Unix? Use inbuilt [unzip] command in Linux. $> unzip j file.zip
$> tar -zxvf backup.tar.gz $> tar -xvf backup.tar

1. 2.

-z : Work on gzip compression automatically when reading archives. -x : Extract archives.

3. -v : Produce verbose output i.e. display progress and extracted file list on screen. 4. -f : Read the archive from the archive to the specified file. In this example, read backups.tar.gz archive. 5.
-t : List the files in the archive.

# How to test if a zip file is corrupted in Linux? Use -t switch with the inbuilt [unzip] command $> unzip t file.zip

# How to check if a file is zipped in Unix? In order to know the file type of a particular file use the [file] command like below: $> file file.txt file.txt: ASCII text If you want to know the technical MIME type of the file, use -i switch. $>file -i file.txt file.txt: text/plain; charset=us-ascii If the file is zipped, following will be the result $> file i file.zip file.zip: application/x-zip

# How to connect to Oracle database from within shell script?

You will be using the same [sqlplus] command to connect to database that you use normally even outside the shell script. To understand this, let's take an example. In this example, we will connect to database, fire a query and get the output printed from the unix shell. Ok? Here we go $>res=`sqlplus -s username/password@database_name < SET HEAD OFF; select count(*) from dual; EXIT; EOF` $> echo $res 1 If you connect to database in this method, the advantage is, you will be able to pass Unix side shell variables value to the database. See below example $>res=`sqlplus -s username/password@database_name < SET HEAD OFF; select count(*) from student_table t where t.last_name=$1; EXIT; EOF` $> echo $res 12 # How to execute a database stored procedure from Shell script? $> SqlReturnMsg=`sqlplus -s username/password@database< BEGIN Proc_Your_Procedure( your-input-parameters ); END; / EXIT; EOF` $> echo $SqlReturnMsg

# How to check the command line arguments in a UNIX command in Shell Script? In a bash shell, you can access the command line arguments using $0, $1, $2, variables, where $0 prints the command name, $1 prints the first input parameter of the command, $2 the second input parameter of the command and so on.

# How to fail a shell script programmatically? Just put an [exit] command in the shell script with return value other than 0. this is because the exit codes of successful Unix programs is zero. So, suppose if you write exit -1 inside your program, then your program will thrown an error and exit immediately. # How to list down file/folder lists alphabetically? Normally [ls lt] command lists down file/folder list sorted by modified time. If you want to list then alphabetically, then you should simply specify: [ls l]

# How to check if the last command was successful in Unix? To check the status of last executed command in UNIX, you can check the value of an inbuilt bash variable [$?]. See the below example: $> echo $?

# How to check if a file is present in a particular directory in Unix? Using command, we can do it in many ways. Based on what we have learnt so far, we can make use of [ls] and [$?] command to do this. See below: $> ls l file.txt; echo $? If the file exists, the [ls] command will be successful. Hence [echo $?] will print 0. If the file does not exist, then [ls] command will fail and hence [echo $?] will print 1. # How to check all the running processes in Unix? The standard command to see this is [ps]. But [ps] only shows you the snapshot of the processes at that instance. If you need to monitor the processes for a certain period of time and need to refresh the results in each interval, consider using the [top] command. $> ps ef If you wish to see the % of memory usage and CPU usage, then consider the below switches $> ps aux If you wish to use this command inside some shell script, or if you want to customize the output of [ps] command, you may use -o switch like below. By using -o switch, you can specify the columns that you want [ps] to print out. $>ps -e -o stime,user,pid,args,%mem,%cpu

# How to tell if my process is running in Unix?

You can list down all the running processes using [ps] command. Then you can grep your user name or process name to see if the process is running. See below: $>ps -e -o stime,user,pid,args,%mem,%cpu | grep "opera" 14:53 opera 29904 sleep 60 14:54 opera 31538 grep opera 0.0 0.0 0.0 0.0 14:54 opera 31536 ps -e -o stime,user,pid,arg 0.0 0.0

# How to get the CPU and Memory details in Linux server? In Linux based systems, you can easily access the CPU and memory details from the /proc/cpuinfo and /proc/meminfo, like this: $>cat /proc/meminfo $>cat /proc/cpuinfo Just try the above commands in your system to see how it works.

SED COMMAND IN UNIX AND LINUX EXAMPLES

Sed is a Stream Editor used for modifying the files in unix (or linux). Whenever you want to make changes to the file automatically, sed comes in handy to do this. Most people never learn its power; they just simply use sed to replace text. You can do many things apart from replacing text with sed. Here I will describe the features of sed with examples. Consider the below text file as an input.

>cat file.txt unix is great os. unix is opensource. unix is free os. learn operating system. unixlinux which one you choose.

SED COMMAND EXAMPLES

1. Replacing or substituting string Sed command is mostly used to replace the text in a file. The below simple sed command replaces the word "unix" with "linux" in the file.

>sed 's/unix/linux/' file.txt linux is great os. unix is opensource. unix is free os.

learn operating system. linuxlinux which one you choose.

Here the "s" specifies the substitution operation. The "/" are delimiters. The "unix" is thesearch pattern and the "linux" is the replacement string. By default, the sed command replaces the first occurrence of the pattern in each line and it won't replace the second, third...occurrence in the line. 2. Replacing the nth occurrence of a pattern in a line. Use the /1, /2 etc flags to replace the first, second occurrence of a pattern in a line. The below command replaces the second occurrence of the word "unix" with "linux" in a line.

>sed 's/unix/linux/2' file.txt unix is great os. linux is opensource. unix is free os. learn operating system. unixlinux which one you choose.

3. Replacing all the occurrence of the pattern in a line. The substitute flag /g (global replacement) specifies the sed command to replace all the occurrences of the string in the line.

>sed 's/unix/linux/g' file.txt linux is great os. linux is opensource. linux is free os. learn operating system. linuxlinux which one you choose.

4. Replacing from nth occurrence to all occurrences in a line. Use the combination of /1, /2 etc and /g to replace all the patterns from the nth occurrence of a pattern in a line. The following sed command replaces the third, fourth, fifth... "unix" word with "linux" word in a line.

>sed 's/unix/linux/3g' file.txt unix is great os. unix is opensource. linux is free os. learn operating system. unixlinux which one you choose.

5. Changing the slash (/) delimiter You can use any delimiter other than the slash. As an example if you want to change theweb url to another url as

>sed 's/http:\/\//www/' file.txt

In this case the url consists the delimiter character which we used. In that case you have to escape the slash with backslash character, otherwise the substitution won't work. Using too many backslashes makes the sed command look awkward. In this case we can change the delimiter to another character as shown in the below example.

>sed 's_http://_www_' file.txt >sed 's|http://|www|' file.txt

6. Using & as the matched string There might be cases where you want to search for the pattern and replace that pattern by adding some extra characters to it. In such cases & comes in handy. The & represents the matched string.

>sed 's/unix/{&}/' file.txt {unix} is great os. unix is opensource. unix is free os. learn operating system.

{unix}linux which one you choose.

>sed 's/unix/{&&}/' file.txt {unixunix} is great os. unix is opensource. unix is free os. learn operating system. {unixunix}linux which one you choose.

7. Using \1,\2 and so on to \9 The first pair of parenthesis specified in the pattern represents the \1, the second represents the \2 and so on. The \1,\2 can be used in the replacement string to make changes to the source string. As an example, if you want to replace the word "unix" in a line with twice as the word like "unixunix" use the sed command as below.

>sed 's/\(unix\)/\1\1/' file.txt unixunix is great os. unix is opensource. unix is free os. learn operating system. unixunixlinux which one you choose.

The parenthesis needs to be escaped with the backslash character. Another example is if you want to switch the words "unixlinux" as "linuxunix", the sed command is

>sed 's/\(unix\)\(linux\)/\2\1/' file.txt unix is great os. unix is opensource. unix is free os. learn operating system. linuxunix which one you choose.

Another example is switching the first three characters in a line

>sed 's/^\(.\)\(.\)\(.\)/\3\2\1/' file.txt inux is great os. unix is opensource. unix is free os. aelrn operating system. inuxlinux which one you choose.

8. Duplicating the replaced line with /p flag The /p print flag prints the replaced line twice on the terminal. If a line does not have the search pattern and is not replaced, then the /p prints that line only once.

>sed 's/unix/linux/p' file.txt linux is great os. unix is opensource. unix is free os. linux is great os. unix is opensource. unix is free os. learn operating system. linuxlinux which one you choose. linuxlinux which one you choose.

9. Printing only the replaced lines Use the -n option along with the /p print flag to display only the replaced lines. Here the -n option suppresses the duplicate rows generated by the /p flag and prints the replaced lines only one time.

>sed -n 's/unix/linux/p' file.txt linux is great os. unix is opensource. unix is free os. linuxlinux which one you choose.

If you use -n alone without /p, then the sed does not print anything. 10. Running multiple sed commands. You can run multiple sed commands by piping the output of one sed command as input to

another sed command.

>sed 's/unix/linux/' file.txt| sed 's/os/system/' linux is great system. unix is opensource. unix is free os. learn operating system. linuxlinux which one you chosysteme.

Sed provides -e option to run multiple sed commands in a single sed command. The above output can be achieved in a single sed command as shown below.

>sed -e 's/unix/linux/' -e 's/os/system/' file.txt linux is great system. unix is opensource. unix is free os. learn operating system. linuxlinux which one you chosysteme.

11. Replacing string on a specific line number. You can restrict the sed command to replace the string on a specific line number. An example is

>sed '3 s/unix/linux/' file.txt unix is great os. unix is opensource. unix is free os. learn operating system. linuxlinux which one you choose.

The above sed command replaces the string only on the third line. 12. Replacing string on a range of lines. You can specify a range of line numbers to the sed command for replacing a string.

>sed '1,3 s/unix/linux/' file.txt linux is great os. unix is opensource. unix is free os. learn operating system. linuxlinux which one you choose.

Here the sed command replaces the lines with range from 1 to 3. Another example is

>sed '2,$ s/unix/linux/' file.txt linux is great os. unix is opensource. unix is free os. learn operating system. linuxlinux which one you choose.

Here $ indicates the last line in the file. So the sed command replaces the text from second line to last line in the file. 13. Replace on a lines which matches a pattern. You can specify a pattern to the sed command to match in a line. If the pattern match occurs, then only the sed command looks for the string to be replaced and if it finds, then the sed command replaces the string.

>sed '/linux/ s/unix/centos/' file.txt unix is great os. unix is opensource. unix is free os. learn operating system. centoslinux which one you choose.

Here the sed command first looks for the lines which has the pattern "linux" and then replaces the word "unix" with "centos". 14. Deleting lines. You can delete the lines a file by specifying the line number or a range or numbers.

>sed '2 d' file.txt >sed '5,$ d' file.txt

15. Duplicating lines You can make the sed command to print each line of a file two times.

>sed 'p' file.txt

16. Sed as grep command You can make sed command to work as similar to grep command.

>grep 'unix' file.txt >sed -n '/unix/ p' file.txt

Here the sed command looks for the pattern "unix" in each line of a file and prints those lines that has the pattern. You can also make the sed command to work as grep -v, just by using the reversing the sed with NOT (!).

>grep -v 'unix' file.txt >sed -n '/unix/ !p' file.txt

The ! here inverts the pattern match. 17. Add a line after a match. The sed command can add a new line after a pattern match is found. The "a" command to sed tells it to add a new line after a match is found.

>sed '/unix/ a "Add a new line"' file.txt unix is great os. unix is opensource. unix is free os. "Add a new line" learn operating system. unixlinux which one you choose. "Add a new line"

18. Add a line before a match The sed command can add a new line before a pattern match is found. The "i" command to sed tells it to add a new line before a match is found.

>sed '/unix/ i "Add a new line"' file.txt "Add a new line" unix is great os. unix is opensource. unix is free os. learn operating system. "Add a new line" unixlinux which one you choose.

19. Change a line The sed command can be used to replace an entire line with a new line. The "c" command to sed tells it to change the line.

>sed '/unix/ c "Change line"' file.txt "Change line" learn operating system.

"Change line"

20. Transform like tr command The sed command can be used to convert the lower case letters to upper case letters by using the transform "y" option.

>sed 'y/ul/UL/' file.txt Unix is great os. Unix is opensoUrce. Unix is free os. Learn operating system. UnixLinUx which one yoU choose.

Here the sed command transforms the alphabets "ul" into their uppercase format "UL"