Skip navigation

Tag Archives: bash script

I haven’t written a scripting post in a while, but I’ve wanted to.  So in keeping with the spirit of my stick to the script posts lets look at something that is common among all scripting languages (even if the syntax isn’t).

Let’s talk about strings…….

 kitty-yarn

Awwww, but no.  These kind of strings.  In the case of scripting, I think the best way to think about it is, text, what you are reading or able to read.  They aren’t used mathematically (usually), but can and will be a huge component in your scripting.  Especially when automating things around a desktop or server environment.

Oh really?  Yes, really.  General uses for strings in a script are:

  • User messages
  • Reporting or logging
  • Comparisons
  • Explicit paths
  • Application execution
  • Conditionals

Ok, so maybe that list doesn’t look that impressive, but when you consider how much of that is done within a script, it becomes obvious the importance of string values to scripting.  It’s also important to recognize that in certain scripting environments, it’s important to define a string value as such so that it can be properly used. 

(Powershell for instance, requires you to properly define a value type to use the relevant functions… but I’m getting ahead of myself.)

So wait?  There are more than strings in a script?  Yes; Strings, Integers, and Booleans are your standard value types. Integers are numbers (math!) and Booleans are True or False.  So given those value types, perhaps it is a bit more obvious how frequently you will use string values?

So lets get into some sample code and evaluate strings some shall we?


VBScript:

strTest = "Hey, I'm a string value"

wscript.echo strTest
'Shows the string value
wscript.echo strTest&", and I've been concatenated to the value."
'& operator joins values
wscript.echo lcase(strTest)
'Lower case
wscript.echo ucase(strTest)
'Upper case
wscript.echo strReverse(strTest)
'Reverses the string
wscript.echo len(strTest)
'Gives the total length of a string
wscript.echo mid(strTest,10,8)
'Returns fix number of characters in string
wscript.echo left(strTest,11)
'11 chars from the left
wscript.echo right(strTest,13)
'13 chars from the right
'----------------------------------------------------------------
wscript.echo inStr(strTest,"a")
'Returns position of string from left to right
wscript.echo inStrRev(strTest,"a")
'Returns position of string from right to left
'----------------------------------------------------------------
a = split(strTest)
'Splits strTest by it's spaces
for each item in a
    wscript.echo item
   'echoes each dimension from the split array
next

wscript.echo a(3)&" "&a(4)

'echoes the split arrays dimensions 3 and 4
'---------------------------------------------------------------

strReturn=inputbox("Here, you try!")

if strReturn = "" then
   wscript.echo "Fine, don't play along"
else
    wscript.echo "So you said: "&trim(ucase(strReturn))&vbcrlf _
    &"Sheesh, no need to shout!"
end if
'---------------------------------------------------------------


Running the above script will give you a better understanding of what I’m about to explain.  I wanted to show some common functions in vbscript (syntax is different but these will be universal functions you will use).  The above are common string manipulation tools

 

Code explained… line by line

First we are defining our string to a variable strTest.  Now “in the wild” as it were, this string could be pulled from an object property, read from a file, registry, user input, output from another application, etc.  It’s best to define a string to a variable though, no matter the method for input.  This of course is the most direct way to do it for our example

Now we begin with the simplest string usage, output.

Now we raise the stakes a bit by joining an additional string value to our current strTest.  This action is known as concatenation.  This is a very common thing with string usage and manipulation.  Building complex values/messages/logs from various predefined and/or dynamically pulled string values.

Our next two examples have to do with manipulating case between upper and lower.  This is fairly self explanatory, and in the interest of string comparisons it’s usually a good practice (and often necessary) to force a case set, especially if the comparison function is case sensitive.

String reversal, this may not seem important initially, but makes a huge difference when you are forced to chop strings up.  The ability to reverse a string can go a long way for string chopping.  Especially if you are dealing with filenames.

The length function is another that may seem arbitrary to some, but allows for great flexibility as well in chopping up strings such as file names.  If you have a fixed number of characters to remove it’s sometimes simpler then splitting the string.  (so I wasn’t completely honest about the math stuff and strings)

Mid, Left, and Right.  These 3 can be used in conjunction with length to return a fixed number of characters from the left right, or middle (specified) of a string.  Here’s a quick easy example:

test="I am 18 chars long"
wscript.echo test
count=len(test)-4
wscript.echo right(test,count)

Very simply, we take the total length of Test and subtract it by 4, then return the sum of remaining characters from right to left of the original string.  In this example we, had a fixed value to subtract, be mindful you could use the length value of multiple strings to achieve the same type of results.

Now, InStr and InStrRev, or “In String” and “In String Reverse”.  These two functions make for great conditionals.  They, along with strComp, are excellent for determining like strings and taking action.  Especially when parsing through files and directories looking for specific returns. 

One of my favorites, especially in PowerShell, split.  Split takes a string, looks for a delimiter (space by default) and breaks the string up into an array.  Why do I like it so much?  Put simply, it allows you to quickly whittle down long path names into a single filename.  It also allows you to quickly and efficiently modify lists of data into manageable formats.  And last but not least, it can easily turn files like csv’s into an array for manipulation.

Finally, user input.  This is pretty self explanatory.  Prompt for input, receive and control input, use input.


In PowerShell, string functions are called like this:

$a=”This is a string value”

$a.ToString()

$a.ToUpper()

$a.ToLower()

$a.Replace(“a “,”my “)

$a.split(“ “)

$a.contains(“string”)

$a.StartsWith(“t”)

$a.EndsWith(“e”)

$a.Length

$a.CompareTo(“this is a string value”)

$a.Length.Equals(22)

$a.CharArray()

$a.PadLeft(“30”)

$a.PadRight(“30”)

$a.Trim()

$a.TrimEnd()

$a.TrimStart()

Given the previous examples in vbscript, you should be able to easily adapt your knowledge to using these in PowerShell.  The idea and purpose is still the same, again, the syntax is just different.


String manipulation inside Bash is, admitedly, a bit more convoluted so I won’t be touching on it in this post.  However I’d highly recommend an online source like: Mendel Cooper’s guide.  Again, the methodology will still be fairly the same, but the syntax will differ.  The largest issue with Bash is the myriad of ways of performing the string manipulation.

 

Anyway, I hope this has been informative for you.  Good luck, and happy scripting!

Advertisements

Alright, so continuing our discussion on scripting with Bash scripting in Unix and Linux.  In the same way that command interprets a script to perform actions in order so does bash.  What’s the difference then?  The sheer volume or tools available to script and manipulate the environment is greatly increased.  The manageability of output, the variety of conditionals, arrays, and integration of the shell into the environment.

It’s worth mentioning up front, I could spend days on the subject of any of these scripting languages.  However for the sake of space and interest, I will still keep things high level.  It’s ALSO worth mentioning that Bash/shell scripting is something of a nix requirement.  Not that you have to know shell scripting to use the OS, but it’s certainly a primary part of the culture.  Bash scripting also works within OSX environments (since it’s built on BSD), however some research and testing will be required to find the similarities between their nix and others which leads to my next topic…

Linux and Unix {

If you are reading this, I will assume you know what Linux and Unix are.  If not, here’s some reading material for Unix and Linux.  What you may or may not be aware of is that although there is a great level of standardization as maintained by IEEE POSIX standards, there is also a lot of distribution specific tools or practices.  In other words, a person can generally port work from one distribution to another, but should never assume a one size fits all solution across the board.  Which is precisely why there are so many distributions, because one size does not fit all, at least not to everyone’s tastes.

I also want to take a moment to point out that this is not a flaw. Although standardization is a good thing, limited options are not. There are other standards that GPL licensed software (generally linux)adheres to that gives it reliable standards or compliance. They also maintain partial POSIX compliance as well. It’s an interesting subject, and the differences are fairly minor in most cases. Do not be scared by the options, determine what it is you want and go from there if you are interested in trying.

Now a days the user experience one has come to expect from Windows or OSX is easily found with a popular Linux distribution (Ubuntu and Fedora immediately come to mind).

     Bash Scripting {

Now lets take a look at a very simple script that I’ve briefly modified from Mendel Cooper’s guide for an example on here:


#!/bin/bash#Log Cleanup
#Author: Daniel Belcher
#4/30/11
#Built for tutorial purposes
#Idea for script from beginner example
#in advanced bash scripting guide:
#http://tldp.org/LDP/abs/html/sha-bang.html
#Variable declaration
LDIR=“/var/log/”
DATE=`date ‘+%m%d%y’`
RUID=0
E_USER=
1#Checking for root user, exiting with error code 1 if not root
if [ “$UID” ne “$RUID” ]
then
          echo
“User needs to be root”
exit $E_USER
fi
#Copying the current messages and wtmp logs to an archive with date code
cp $LDIR“messages” $LDIR“messages.”$DATE;
     cp $LDIR“wtmp” $LDIR“wtmp.”$DATE;
#Taking the last 10 lines from that log and writing it to the current
#messages and wtmp to reset them.
tail n 10 $LDIR“messages.”$DATE > $LDIR“messages”
          tail n 10 $LDIR“wtmp.”$DATE > $LDIR“wtmp”               
   sleep
.5
#Finding all messages and wtmp files with an
#access time of 7 days or more and deleting them.
find $LDIR -iname messages.\* atime 7 exec rm {} \; #could also swap –exec rm etc for –delete 
             find $LDIR -iname wtmp.\* atime 7 | xarg rm

Save as a .sh

then

chmod +x scriptname.sh

Now same as with our batch script, lets focus on 4 things.  Hopefully some similarities begin to present themselves.

  1. Sequence
  2. Command usage
  3. Output usage
  4. Conditional logic

Sequence:

Now similarly to the command shell, Bash reads the script line for line.  However instead of GOTO statements in Bash, we would actually build functions. (I do not have an example in this script but the syntax is generally name_of_function() { work to perform }.  I’ll discuss this more with PowerShell and VB script).  All lines are read as if they were typed at the console.  Here you will see I also do something that we didn’t do within the batch file.  Variable declaration (LDIR=”/var/log/”) then I can call it again in the script as $LDIR.

This allows you to take a commonly used command, object, integer, or string and set it to something easily repeatable inside the script.  Variables can be declared at run time, as an argument, or produced from script output to name a few methods for declaration.  They are integral to programming in general.  As far as sequence is concerned, as long as the variable is declared before it’s called you are fine.

Command usage and Output usage:

Now hopefully you are seeing that the commands available in a shell are what does most of the heavy lifting in terms of work.  Something worth learning in command usage is piping.  Piping is taking the output of one command and using it as the input for another (to put it simply).  A visual I’ve always liked to use in understanding what is happening is this:

A lake brings water to your home.  A pipe exists between the lake and your faucet (very simple). On the course to your house a series of filters are applied to weed out what you don’t want from the water by the time it reaches your house.  In this case the lake is the source or original input, and the filters are the series of commands applied to the pipe before the final output at your faucet.

So in pseudo code: lake-water | filter-fish | filter-algae | filter-dirt > glass-of-water.drk

The end result is that water is transported through that pipe and all unwanted elements are removed until we finally have our desired glass of water to drink.

In this case we use:

find $LDIR -iname wtmp.\* atime 7 | xarg rm

This will take the findings from the FIND command and pass it to xarg which initiates rm and deletes these files.

Hopefully this is beginning to paint a clearer picture of pipe and command usage.  Shells that allow for command piping are a God send and allow for some very elegant solutions to complex problems in simple one liners that a user could easily alias, but that’s another subject entirely.

Conditional Logic:

Now in an identical fashion to our batch file we are performing an IF statement.  IF statements are one of, if not the most, commonly used conditional statements.  Notice there is a very distinct difference in the syntax used here in comparison to the one we used in our batch.  The testing method and outcome are still the same however, so do not be deceived.  IF statements put simply are: if it’s this then do that.  Since conditional tests are capable of greater complexity (and will be with all other script engines we discuss going forward) it needs to be terminated so the script knows when the conditions or work statements are final. We close this IF statement in Bash script by spelling IF backwards FI.

The same is true for all conditionals within bash.  They will open spelled forward and close in reverse.  Elif … file case … esac; when you begin to learn what and how logic is applied in scripts you will know how to appropriately apply the syntax.

So what does this Bash script do?

The script opens with a shabang! (#!/bin/bash) which tells the shell that it’s a script to be interpreted by the bash shell found in the bin folder.

Next it declares 4 variables, LDIR (log directory path), DATE (current date no special characters month date year), RUID (Root user id), and E_USER (error code to use if the user fails our user check).

Now we check a shell variable $UID (current user’s ID #) and see if it’s different than the $RUID (-ne = not equal). If it’s not the root user we exit with the error code # assigned to $E_USER.

If we pass that check we will now copy the messages and wtmp files under the /var/log/ folder to their same folder as filename.currentdate.  Example: messages.050111

Now we take the last 10 lines of that file and output it to the primary log files messages and wtmp effectively overwriting the previous data and truncating that file.  > operator on the command line is a redirect (as we saw with our > nul in the batch file).

Finally we take and search that folder for all files following the archived file syntax and search for an access time greater than 7 days then we delete them.  I’ve used two different formats for deletion to show the different possible ways of doing this.

     } gnitidE shaB

} xinU dna xuniL

In my next post I’m going to break into vb scripting, which is a considerable departure from the shell relative scripting we’ve looked at so far.  However I will return to another form of shell scripting with PowerShell, stay tuned.


Just for another example, how would this script look utilizing a function?

#!/bin/bash#Log Cleanup
#Author: Daniel Belcher
#5/1/11
#Variable declaration
LDIR=“/var/log/”
DATE=`date ‘+%m%d%y’`
RUID=0
E_USER=
1#Checking for root user, exiting with error code 1 if not root
if [ “$UID” ne “$RUID” ]
then
         echo
“User needs to be root”
exit $E_USER
fi
#Function to create backup, truncate, and purge old log files.
cleanup () {
                 cp $LDIR$1 $LDIR$1“.”$DATE;
                         tail n 10 $LDIR$1“.”$DATE > $LDIR$1
                                 sleep .7
                         find $LDIR iname $1.\* –atime 7 exec rm {} \;
                 return 0
                 }

#Calling function cleanup () and passing 1 argument

cleanup messages
cleanup wtmp

exit 0

A lot cleaner.  Hopefully you can see the benefits of functions now, or even loops as we will examine in the future.