Shell Script for log file cleanup

Problem :

lets find the list of file that are older than 30days. That has extension of *.log

-rw-r--r-- 1 oracle oinstall        513 Jul 28 08:27 change_pass.sql
-rw-r----- 1 oracle oinstall     188416 Jul 30 11:37 testnetlink.dmp
-rw-r--r-- 1 oracle oinstall       1477 Jul 30 11:37 impi_temp.log
-rw-r--r-- 1 oracle oinstall       1433 Jul 30 12:58 impi1_temp.log

Requirement :

Now I want to remove the files that are older than 30 days and at same time I want keep a log of the files that are being deleted.


### Script starts here ###

#!/usr/bin/ksh

cd /u01/app/oracle/dpump; ## location where I want to cleanup the log files
echo " we are now in : `pwd`"


for FILE in `find /u01/app/oracle/dpump/*.log -mtime +30`
   do echo $FILE
   rm -f $FILE ##2>/dev/null # Gets rid of directory or permission errors
done  2>&1 1> /u01/app/oracle/dpump/log.txt




sample output of log file :

lets see what is captured in cleanup log file.

cat u01/app/oracle/dpump/log.txt

u01/app/oracle/dpump/AUDIT_PART1.log
/u01/app/oracle/dpump/impi1_temp.log
/u01/app/oracle/dpump/impi_temp.log
/u01/app/oracle/dpump/test1.log

shell script to find files in a directory

####This script starts here ####
###This script cheks if the a file exist current Directory ##
## script will propmt for filename ##
#!bin/ksh

echo "eneter Filename :"
read DUMPFILE;
echo " **** checking if the File exists **** "
if [ -e ${DUMPFILE} ];
then
echo " **** Given file exists in directory `pwd` ****"
else
echo " **** ERRR : Given file does not exist in directory `pwd` ****"
fi

Lets see how to use this code :

atoorpu@Linux01:[~/scripts] $ ls -al testfile
ls: testfile: No such file or directory

atoorpu@Linux01:[~/scripts] $ . check_file.sh
eneter Filename :
testfile
 **** checking if the File exists ****
 **** ERRR : Given file does not exist in directory /home/atoorpu/scripts ****

atoorpu@Linux01:[~/scripts] $ touch testfile

atoorpu@Linux01:[~/scripts] $ ls -al testfile
-rw-rw-r-- 1 atoorpu atoorpu 0 Oct 23 14:03 testfile

atoorpu@Linux01:[~/scripts] $ . check_file.sh
eneter Filename :
testfile
 **** checking if the File exists ****
 **** Given file exists in directory /home/atoorpu/scripts ****

atoorpu@Linux01:[~/scripts] $

atoorpu@Linux01:[~/scripts] $

using UNIX pipes to export oracle table and compress

Script explanation:

This script is to export the dump file and compress the exported data using UNIX pipes. This method is best suited when there is not enough space on disk. The file exported and compressed at same time, this method will not require much space while exporting the dumpfile.

Note: Using of UNIX pipe's will also speed up the export process & yes there will consume a good amount of CPU.



#!/usr/bin/ksh:

#### This script will export the dumpfile and compress at same time using linux pipe

#### You can use normal gunzip cmd to unzip the zipped file

####



export mydate=`date "+%d%b%Y"`;

export TEST_BAK=TEST_MKNOD_"$mydate";

DUMPFILE=$TEST_BAK".dmp";    ## Setting dumpfile name here

LOGFILE=$TEST_BAK".log";               ## Setting logfile name here

DPUMP=/u01/app/oracle/dpump;                   ## Setting target directory for dump & log







## create exp

## you can use the pipe for compress the file

mknod $DPUMP/exp.pipe p



## compress dmp with file using pipe

gzip < $DPUMP/exp.pipe > $DPUMP/$DUMPFILE.gz &



exp USERNAME/PASSWORD@ORCL file="$DPUMP"/exp.pipe log="$DPUMP/$LOGFILE" tables=TEST statistics=none



## remove the pipe created

rm $DPUMP/exp.pipe



echo $' \n ';



echo " export completed "

Shell script - Check if the files exists in a file

This is a simple script to look for a file in a given location.

#!/usr/bin/ksh   echo " File name please : " read FILENAME; dataDir=/u01/app/oracle/dpump; ### Directory to look for a file ### sourceFile=SCOTT.dmp; ### file name to search for ###   cd /u01/app/oracle/dpump; echo " Present directory : " `pwd`"";   ##Did we get a file?   if [ -e ${dataDir}/${sourceFile} ]; then echo "Found file !! " else echo " No file found !!! " fi

Backup unix files and folders using tar - compress and backup files

Because of various reasons some of which hearken back to the era of tape drives, Unix uses a program named tar to archive data, which can then be compressed with a compression program like gzip, bzip2, 7zip, etc.



In order to "zip" a directory, the correct command would be



tar -zcvf archive.tar.gz directory/

This will tell tar to c (create) an archive from the files in directory (tar is recursive by default), compress it using the z (gzip) algorithm, store the output as a f (file) named archive.tar.gz, and v (verbosely) list all the files it adds to the archive.



To decompress and unpack the archive into the current directory you would use



----------------------------------

ARCHIVE FOLDER CONTENTS & CREATE NEW BACKUP FILE:

----------------------------------



atoorpu@linux01:[~] $ tar -zcvf Output_File.tar.gz SourceFile

ziptest/

ziptest/readme1.txt



--->>> Output_File.tar.gz    >>>   output file

--->>> SourceFile    >>>  source directory



atoorpu@linux01:[~] $ ls

oswbb733.tar                       patches                readme.txt  sys_checker.sh  ziptest.tar.gz

p19543113_851499_Linux-x86-64.zip  pitt_4_21_2014.dmp.gz  scripts     ziptest1



LIST CONTENTS OF TAR FILE:



atoorpu@linux01:[~] $ tar -ztvf ziptest.tar.gz

drwxrwxr-x atoorpu/atoorpu   0 2015-08-27 14:16:41 ziptest/

-r--r--r-- atoorpu/atoorpu  50 2015-08-27 14:14:20 ziptest/readme1.txt



----------------------------------

EXTRACT/UNTAR THE FILE USE :

----------------------------------



To extract one or more members from an archive, enter:

$ tar -zxvf {file.tar.gz}



If your tarball name is backup.tar.gz, enter the following at a shell prompt to extract files:

$ tar -zxvf backup.tar.gz



To extract resume.doc file from backup.tar.gz tarball, enter:

$ tar -zxvf backup.tar.gz resume.doc

Simple Shell script to execute a sql from UNIX

This is a simple shell script to execute a sql from UNIX shell script.





#!/usr/bin/ksh



export ORACLE_SID=ORCL

export ORACLE_HOME=/u01/app/oracle/product/11.2.0/db_1

export PATH=$ORACLE_HOME/bin:/usr/local/bin:$PATH

export mydate=`date "+%d%b%Y"`





echo " ************** EXECUTING SQL STARTS HERE ************ "



sqlplus -s USERNAME/PASSWORD@ORCL > /u01/app/oracle/dpump/test.log  <<EOF



SET SERVEROUTPUT ON;

SET FEED OFF;



select name from v\$database;



SET SERVEROUTPUT OFF;

EOF



echo " ************** END OF SQL HERE ************ "

Shell script to export a schema and move it to different server

#!/usr/bin/ksh



### local_path is the data pump directory in local server.

### remote_path is the target location in the remote server.

### RMTSERVER is the ip adress (or) hostname of remote host.



export mydate=`date "+%d%b%Y"`;

local_path=/u01/oracle/DPUMP;

RMTSERVER=10.176.33.99;  ### We are hard coding the remote server here ###

remote_path=/u01/app/oracle/dpump;



## echo "Enter Password:"

##read PASSWORD;



echo "Enter SCHEMA name:"

read SCHEMA;





expdp EXPORT_USER/PASSWORD directory=DPUMP dumpfile="$SCHEMA"_"$mydate".dmp logfile="$SCHEMA"_exp"$mydate".log schemas="$SCHEMA" COMPRESSION=ALL



echo "Export Completed"



cd /u01/oracle/DPUMP;

echo `pwd`

export DUMP_FILE="$SCHEMA"_"$mydate".dmp

sftp $RMTSERVER  <<EOF



cd $remote_path;

put $DUMP_FILE



EOF

Unix Permissions and values

Change UNIX permissions by using the simple command.


Syntax:


chmod 775 filename or foldername


Example:



chmod 775 test.txt


In the above cmd we are changing the permission of test file to 775 (Read/Write/Execute).


Below is the table of permissions.



UNIX Permissions:




Value
Permission
Directory Listing

0
No read, no write, no execute
---
1
No read, no write, execute
--x
2
No read, write, no execute
-w-
3
No read, write, execute
-wx
4
Read, no write, no execute
r--
5
Read, no write, execute
r-x
6
Read, write, no execute
rw-
7
Read, write, execute
rwx


rwx rwx rwx = 111 111 111
rw- rw- rw- = 110 110 110
rwx --- --- = 111 000 000

and so on...

rwx = 111 in binary = 7
rw- = 110 in binary = 6
r-x = 101 in binary = 5
r-- = 100 in binary = 4

exception handling in shell script

Problem :

I have a a situation where I want to execute script1 and then script 2 in sequence.. But at same time want to make sure if script1 fails, script2 doesn't get executed. In this case I want to explain how to break the script if there were some errors in

lets create twp scripts script1.sh as below

#!bin/ksh
echo " this is with in script 1 "

create script2.sh as below

#!bin/ksh
echo " this is with in script 2 "


now lets create another script handle_errors.sh as below

#!bin/ksh
script1.sh
rc=$?
if [ ${rc} -eq 0 ];then
echo "script1 pass, starting script2"
  script2.sh
  rc=$?
  if [ ${rc} -eq 0 ];then
   echo "script2 pass"
  else
   echo "script2 failed"
  fi
else
 echo "script1 failed"
fi

Sample output :

atoorpu@Linux01:[~/scripts] $ . handle_errors.sh
this is from script1
script1 passed, starting script2 now !!
 this is from script2
script2 has passed
atoorpu@Linux01:[~/scripts] $


now if  you want to mess around you can update one of script1 or script2 :

I changed script1.sh to below now :

#/bin/sh
this is from script1"

If you see here the script has failed on script1 and it didn't go to script2 

atoorpu@Linux01:[~/scripts] $ . handle_errors.sh

-bash: /home/atoorpu/scripts/script1.sh: line 2: unexpected EOF while looking for matching `"'

-bash: /home/atoorpu/scripts/script1.sh: line 4: syntax error: unexpected end of file

script1 failed !!