OK this is a time about central logging.
This is one of the script that a lot of places have it in place and a lot of companies would like to
have some kind of automation tool around the logging.
How is how it works.
You have a real Linux machine has waterlog messages right.
And you would actually have to go into the machine log into the machine to view those those logs and
to see if there are any error messages or failures or anything.
So you were you probably would have to do that multiple times if you have multiple servers or does you
have to do probably a thousand times over a thousand servers and you have to go in vi the file tested
it so many things.
What if I tell you you could write a script a central log or script what it's going to do is going to
trace the var log messages file not just trace like entire file.
It actually is going to sit and wait for the new line generation to log in to the wire log messages
and as soon as it comes in it's actually going to filter out whether it has error messages or not and
it's going to put it out if it matches your criteria if you are looking for only let's say error messages
or failures then it's only going to grow up for that keywords and want Once it finds those keywords
is going to push that out to our file.
And you could read their file later on by yourself or then.
Or you could actually leave that script to run it in the background while it's running it's keep putting
your error messages or your failures related lines from you or log messages to a file and then you could
go in and read it and you just have to do is run in the background or you could third step you could
do is you could as soon as those file is created you could create another script and tell it was going
to wake up like every 15 minutes or 20 minutes and tell the users or administrators or managers that
hey there is this file has been generated and it has these many errors and failures or of course you
could go another step which is to schedule the script through the cron job.
So if you look at it backward you have a script that is scheduled to wake up every 15 minutes.
That script will have to notify a user if a file exist and that file that exists that has your information
that you're looking for like errors failures shut down off line or anything that you have specified.
And those messages are coming from where from waterlog messages.
So we will create this script.
It's a little long script you'll create a script step by step and you could actually implement that
script in your job if you are at Lenox capacity right now and you will see definitely that this has
this will make your life so much easier when you actually have these error messages coming in through
the e-mail without you having to log into every machine.
So let's go straight again to our machine let's clear our screen.
And I want you to at this time become root because in order to read waterlog messages you have to be
root Bedau root you cannot read Whitelock messages.
It does not have permissions for anyone else but root.
OK here you have to start writing a script but before you write a script I just have to tell you that
the way you look at the logs is by doing tail let's say minus 10 or log messages right.
Most the time people do that to see if there are any messages or what they do is they do CAT scan or
log messages and they all just go up for minus.
I had to skip uppercase lowercase her.
You see I have so many messages on my machine that came up.
I would known if this error messages exist unless I come in and log into the machine and actually run
this command.
But is that something that you want to do every day and is spending so much of your time just looking
for log messages.
And if you have 100 system then forget about it you're going to spend all day and it's going to affect
your performance significantly and you don't want to do that.
OK.
So for you for that you have to write a script and write a script.
I'm going to pick which directory I'm in and slash route and here I have a few other ones I already
have one script that I created earlier it's called Long Tail and I will actually use that script but
I will tell you exactly how it works.
So let me clear the screen so long tail is a script that I've written.
Hit her.
And then you will see here I skipped it real quick but I want you to put it in the information that
we always need.
Description This script will log on the defined key word
modifier.
OK.
So let's get into this.
This is what this group would do.
First the script is going to tell minus F and zero option tail is a command that will tail the last
error messages from a file.
So not error message any messages from afar.
And then you specify minus f.
This is going to tail live incoming traffic with minus minus and zero option what is telling it only
look for the latest new incoming messages.
Don't look forward messages that previously in this in this log which is fine because when you're going
to run it you don't want to start from the beginning.
You don't want to keep getting the same error messages.
OK so I'm sure you're familiar with this command you do.
Our log messages.
Next thing right here you put a pipe and you do while read line.
What this is doing is while it is doing tail.
If you leave it right here then it is going nowhere because you are running a shell but you want to
put it somewhere right.
So you do a wild script.
The reason you do while is because you want to run the script in a loop.
Meaning it doesn't stop.
It just keeps running and running and running until you actually go in and stop the script.
So what is doing is you are telling the script to run the script while a while loop and read the line.
And what is the line line is variable and what is coming inside the line whatever the output from this
is going to go into this variable.
OK.
Now be clear here.
Now once we have all those messages coming in in this line what should we do.
Right.
What you do do echo that line.
First of all whatever is inside of this variable echo it again once you echo it which is going to it's
going to read it right then it will grep e-group egrep is another command.
It's efficient command that will crap multiple keywords.
So e-group minus.
I and I have keywords here refused.
Invalid error.
Fail last shut down off line.
These are the key words that I have gathered in the past from my experience and I know these keywords
actually work.
And these are the type of keywords that mostly come in to the system and valuable to the system and
soon find out.
So anyway whatever comes in and it will grow for all these messages how are we putting it in what are
we saying is if statement is if dollar question mark meaning if the exit status is equal to zero meaning
if it does find something any of these keywords only then only then echoed that output from this to
a file called Temp folder messages.
That's my own created file.
I wanted to put into this filtered file once it's been put in fee and done so that's saved that file.
When I run the file you'll see it will run an infinite loop unless I actually put it in some kind of
log or message in it and then it will put it into the folder files.
So that's Control-C out of it.
And if you notice where is it putting it's putting into sashed temp second filter for messages right
here.
I ran this command previously and it has some of the error messages.
And then you do cat
filter messages you will see some of the error messages that I sent to the lawyer.
It came in here the logger command to run it.
I think it's for me too for my history grow for longer and you see right here this is a logger if you
run any of the command will go into the war log messages because let's see if you run logger Dashti
error test entry or put in like Emran is testing and hit enter.
This entry is going into the whois log messages.
How do you verified you do tayle minus 5.
Var log messages.
You see it's right here.
It is being recorded because logger tells it to record it.
So now if I do the same thing again I run the log tail script and now I opened up the terminal
root
and here if I run the logger command I will put in Imraan is testing again and hit enter.
Sorry it has to be the exact log or test
log.
Mine is testing OK.
OK.
You see as soon as I ran this the long tail also through the message on the screen and I'm sure that
it has also logged into our
this file.
Cat Oh sorry slash temp's.
I felt a mess so let's verify it.
You see it has added that right here into orbit.
So that's what happens every time an error message comes in.
It will be logged in here.
So if I also have let's do our cat log tail if I do invalid so let's do a logger instead of error.
Put in invalid.
Let's see.
Picks up that Imraan is testing invalid
key word enter as soon as it hit enter.
You see in the back it also listed that and two are log messages and when you read the file the file
where is pushing it to.
Sorry.
Clear.
You see it has also added that to your slash slash filter.
OK.
Now you see one step is done.
You have actually if you notice right here is minimized.
You have actually created a script to trace a log.
You run the script in the bag.
Next step is now you see my prompt is not given me back.
It's stuck.
Why.
Because this script that I've written is going in a loop and it's now going to give me my prompt back.
So in order for me to run the script in the background I would have to run the script with no help and
am percent so just do Control-C out of it and hit up arrow key.
And this time I'm going to specify no HUP meaning don't bother my terminal get me my terminal back log
tail.
So the and just put and person.
Why is that a no hub and percent that's how it is that's how you run the commands in the background.
So you hit enter and say Did it work.
Log 1 second.
No slash route log tail ampersand percent
yes sorry.
One one percent.
OK.
So once you run it you could actually do P.S. minus E.F and gruffed for log tail your script and you'll
see it is running right here.
You know he's running two scripts because it's one is it's reading and second it's putting into a file
and grepping for error messages.
So if you want to bring it to the foreground it believe it's for ground something and it brings it back
anyway so whenever you run this script you have to run it with this command slash routes that long tail
and percent and it will put it in the bag.
The next thing is right here we have user admin notification.
Now what if it actually generates the file and it put an end to the user.
It actually throws that message to a user e-mail.
So for that I believe I also have a script.
No actually I don't.
I would have to create a new script and I will name it
log alert
before I do that.
Let me copy that law tale to log alert.
So I have to type the variable and description the log alert description descript will send an e-mail
to administrator
that's modified.
OK so now let's get rid of everything.
So this way we are typing it from scratch.
First thing is if you have multiple users that you want to send this log to instead of putting that
e-mail one by one you could put them in a variable.
So the way I do it is I put it like if your group name is let's say I-T
you put equal and now put code and the e-mail address Imraan underscore and at hotmail dot com comma.
Let's say Peter dot
says
e-mail com code close.
So I'm just picking two users right now if you want to specify more than that you just put comma and
the e-mail address.
Another advantage of putting the e-mails in in a container or in a variable is because if a user leaves
the company or a new user joins the company you don't have to modify your script.
You can make a mistake you could simply modify this variable OK.
So now our variable is defined.
Now you put if statement if is always very strong.
The next the new wire option of picking is minus S..
So what we're doing at minus as is slash temp
filled turd messages.
What I'm saying is first make sure if that file exists I don't want to send an empty file right.
If the file is there then I want it to send it.
But before that let me make sure this is the correct PO to see this grip and just two Ls minus cell
Yes it is so V-I log that Lerch OK.
So if you find a file there in temp and filename is filtered messages then
you cat that file
sort it unique it
mail it to who you could smell it to.
Right.
But also put in some subject is sist log
messages and then send it to
remove this else.
Don't do anything.
OK.
Let me go over it again.
I know probably typed it fast and or fast you probably don't know what I just did.
Again I specified the e-mail addresses I put it in the container then I'd send the command if and what
he's doing is making sure this file exists because again you don't want to send an empty file if it
does exist.
Then what you want to do what you want script to do is to catch that file.
OK.
Did cat then sort it.
Why am I sorting it.
Because I want to unique it right.
But why are you seeking it.
Because what happens if you have 10 messages 10 error messages that are the same one that's a error.
User can log an average user can a logging error user can log in.
You don't want to view the three repetitive same error messages.
You just want to know one time.
So that's why I'm putting in sort here.
So it will sort of unique it just give me one message and I would know you'd have to turn it to real
time.
OK.
So once you do that then you mail it the command for mail it's simple mail minus.
As for subject and the subject is going to say this log message.
OK.
And where is it going to go to is going to go to dollar I.T. and who is dubiety.
These are the people of dollars.
OK.
All right.
And then once you are done sending it then I want you to remove them all.
Why are we removing the file now.
The reason we are removing their file because we do not want new messages coming in and adding it to
the same messages.
And then what will happen is we will get new messages as well as all messages.
So that's why I'm removing all else do nothing if there is no file.
Then of course don't do anything don't send any error message.
I don't even want to know hey they'll file slash temp files their message is not created.
You could do that but you don't want to get that message every 15 minutes if the script is running.
So basically else and fee and you are done again if you run the script in your real environment you
will start getting messages.
I do not have actual mail relay server which is going to actually send e-mails outside to my lab environment
so I can not test the script.
But again I have tested this script in my job in my previous jobs and it works perfectly.
It save it and change my code a.
X log alert.
All right.
So now you you know we have two script.
One is longtail second one in log alert lock tail will continue running forever until you actually stop
log alert will run only when you want to view those messages and you wanted to send an e-mail with those
messages.
Now you could go into your machine every time and run log alert.
But again it's a repetitive tasks so the last thing we have to do right here is schedule the script
through cron job and to schedule the script to Cronje up.
I'm sure you know you just have to type Crohn tab minus E to edit it and you specify let's say if you
wanted to run the script every 15 minutes you'd do Asterix slash 15 every day every month.
And so and so slash fruit slash log
ups alert and the script would run every 15 minutes.
Of course I don't want to run it because this is my test environment.
I'll just added it and I'll put a comment out of it.
And this is how you create a central logging script.
It took me a while when I first created it but eventually I really refined it many times.
Now you have this script already done created for you.
I want you to create this group from scratch don't just copy mine from the text that I have attached
with this lesson.
This is just for your reference.
If you skip something in my video then you could always go back to my text file and look look from there.
Otherwise tried everything from scratch and it will work perfectly.