[rsyslog] forwarding mysqld log

[Josh Bitto]

I'm having a problem with how the output is happening. Its filling up my messages file with repetitive information.

Should my /etc/rsyslog.d/forwardFile.conf look like this?

$ModLoad imfile
# File 1
$InputFileName /var/log/mysqld.log
$InputFileTag mysqld:
$InputFileStateFile stat-file1
$InputFileSeverity notice
$InputFileFacility local0
$InputRunFileMonitor

(no /etc/rsyslog.d/forwardRules.conf was set)
And to have this forwarded from a remote server to a central server. Should there be a /etc/rsyslog.d/forwardRule.conf?

I'm not quite sure how to put the forwarding rule in rsyslog.conf
If $programname == 'mysqld' then @forward.to.this.ip:514

I'm kinda confused about this....I don't know if mysqld.log (centos box) falls under something that you have to monitor...or if I can just forward the file that is created by the program to another server.



On my central syslog server the rsyslog.conf I want to use a template to receive this data and put it in a file so
$template DYNmysqld,"/var/log/hosts/%HOSTNAME%/mysqld.log"

If \
$source != 'myhostname'\
And \
$programname == 'mysqld' \
Then ?DYNmysqld

Should this be working?


Joshua Bitto
Information Technologist
KCC

[David Lang]

Post by Josh Bitto
I'm having a problem with how the output is happening. Its filling up my messages file with repetitive information.

you need to explain the problem a bit more

1. what are you seeing

2. what do you expect to see

Post by Josh Bitto
Should my /etc/rsyslog.d/forwardFile.conf look like this?
$ModLoad imfile
# File 1
$InputFileName /var/log/mysqld.log
$InputFileStateFile stat-file1
$InputFileSeverity notice
$InputFileFacility local0
$InputRunFileMonitor
(no /etc/rsyslog.d/forwardRules.conf was set)
And to have this forwarded from a remote server to a central server. Should there be a /etc/rsyslog.d/forwardRule.conf?

include files are completely optional.

I prefer not to use them, because it makes it harder to understand the
processing flow (especially if you discard messages at some point so that they
won't show up in the other log files)

Post by Josh Bitto
I'm not quite sure how to put the forwarding rule in rsyslog.conf
I'm kinda confused about this....I don't know if mysqld.log (centos box) falls
under something that you have to monitor...or if I can just forward the file
that is created by the program to another server.

The first thing that you should be doing is making sure that rsyslog on the
mysql machine is seeing the logs. The best way to do this is to configure mysql
to log to syslog. If you can't do that, then using imfile to check the file
periodically and read new items from it is a distant second choice.

Once you are sure that you are getting the logs in syslog on the first machine,
then you can worry about getting them to a second machine and determining where
they go.

troubleshooting 101, start at the beginning and make sure things are working
each step of the way.

Post by Josh Bitto
On my central syslog server the rsyslog.conf I want to use a template to receive this data and put it in a file so
$template DYNmysqld,"/var/log/hosts/%HOSTNAME%/mysqld.log"
If \
$source != 'myhostname'\
And \
$programname == 'mysqld' \
Then ?DYNmysqld
Should this be working?

it depends on what the logs look like. If they are logs generated from the
imfile config above, I think they probably will, although I'll point out that
doing mixed case on commands "if" "and" "then" is highly questionable at best.

David Lang

[Josh Bitto]

David,

Ok so by default mysqld uses /var/log/mysqld.log as its log file. I haven't found anything on the net that says that mysql will talk with syslog at all. So I don't know. I did get it working with the imfile though. Basically the same as I mentioned below, but on the central server I put it like this.

If it's anything, but these programs (httpd, mysqld, etc) then send it to this template as messages. If the programname is mysqld then put it here.

I don't have the education for programming, but it does come natural. I can pretty much understand coding, but if you sat me down in front of a blank script page I'd have no idea how to form it. oO

/end rant.

Thanks for your insight and help. All of your knowledge does help me out a lot.


Josh


-----Original Message-----
From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of David Lang
Sent: Friday, April 05, 2013 10:27 PM
To: rsyslog-users
Subject: Re: [rsyslog] forwarding mysqld log

Post by Josh Bitto
I'm having a problem with how the output is happening. Its filling up my messages file with repetitive information.

you need to explain the problem a bit more

1. what are you seeing

2. what do you expect to see

Post by Josh Bitto
Should my /etc/rsyslog.d/forwardFile.conf look like this?
$ModLoad imfile
# File 1
$InputFileName /var/log/mysqld.log
$InputFileStateFile stat-file1
$InputFileSeverity notice
$InputFileFacility local0
$InputRunFileMonitor
(no /etc/rsyslog.d/forwardRules.conf was set) And to have this
forwarded from a remote server to a central server. Should there be a /etc/rsyslog.d/forwardRule.conf?

include files are completely optional.

I prefer not to use them, because it makes it harder to understand the processing flow (especially if you discard messages at some point so that they won't show up in the other log files)

Post by Josh Bitto
I'm not quite sure how to put the forwarding rule in rsyslog.conf If
I'm kinda confused about this....I don't know if mysqld.log (centos
box) falls under something that you have to monitor...or if I can just
forward the file that is created by the program to another server.

The first thing that you should be doing is making sure that rsyslog on the mysql machine is seeing the logs. The best way to do this is to configure mysql to log to syslog. If you can't do that, then using imfile to check the file periodically and read new items from it is a distant second choice.

Once you are sure that you are getting the logs in syslog on the first machine, then you can worry about getting them to a second machine and determining where they go.

troubleshooting 101, start at the beginning and make sure things are working each step of the way.

Post by Josh Bitto
On my central syslog server the rsyslog.conf I want to use a template
to receive this data and put it in a file so $template DYNmysqld,"/var/log/hosts/%HOSTNAME%/mysqld.log"
If \
$source != 'myhostname'\
And \
$programname == 'mysqld' \
Then ?DYNmysqld
Should this be working?

it depends on what the logs look like. If they are logs generated from the imfile config above, I think they probably will, although I'll point out that doing mixed case on commands "if" "and" "then" is highly questionable at best.

David Lang

[David Lang]

A google search for "mysql logging to syslog" found

http://shinguz.blogspot.com/2010/01/mysql-reporting-to-syslog.html

http://help.papertrailapp.com/kb/configuration/configuring-centralized-logging-from-mysql-query-logs

MySQL 5.1.20+ only

MySQL versions 5.1.20 and later support logging to syslog natively. Two
arguments to mysqld_safe control logging: --syslog and
--log-error="/path/to/file"

Both, either, or neither argument can be used. mysqld_safe is generally started
from /etc/init.d/mysqld. Edit that file (or its equivalent for your
distribution) and confirm that --syslog is included on the command line, or add
it.

Here is a sample which makes MySQL log to syslog and directly to a file:

/usr/bin/mysqld_safe --datadir="$datadir" --socket="$socketfile" \
--syslog --log-error="/var/log/mysql.err" --pid-file="$mypidfile" \

/dev/null 2>&1 &
Date: Sat, 6 Apr 2013 23:01:07 -0700
From: Josh Bitto <jbitto at onlineschool.ca>
Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
To: rsyslog-users <rsyslog at lists.adiscon.com>
Subject: Re: [rsyslog] forwarding mysqld log
David,
Ok so by default mysqld uses /var/log/mysqld.log as its log file. I haven't found anything on the net that says that mysql will talk with syslog at all. So I don't know. I did get it working with the imfile though. Basically the same as I mentioned below, but on the central server I put it like this.
If it's anything, but these programs (httpd, mysqld, etc) then send it to this template as messages. If the programname is mysqld then put it here.
I don't have the education for programming, but it does come natural. I can pretty much understand coding, but if you sat me down in front of a blank script page I'd have no idea how to form it. oO
/end rant.
Thanks for your insight and help. All of your knowledge does help me out a lot.
Josh
-----Original Message-----
From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of David Lang
Sent: Friday, April 05, 2013 10:27 PM
To: rsyslog-users
Subject: Re: [rsyslog] forwarding mysqld log

Post by Josh Bitto
I'm having a problem with how the output is happening. Its filling up my messages file with repetitive information.

you need to explain the problem a bit more
1. what are you seeing
2. what do you expect to see

Post by Josh Bitto
Should my /etc/rsyslog.d/forwardFile.conf look like this?
$ModLoad imfile
# File 1
$InputFileName /var/log/mysqld.log
$InputFileStateFile stat-file1
$InputFileSeverity notice
$InputFileFacility local0
$InputRunFileMonitor
(no /etc/rsyslog.d/forwardRules.conf was set) And to have this
forwarded from a remote server to a central server. Should there be a /etc/rsyslog.d/forwardRule.conf?

include files are completely optional.
I prefer not to use them, because it makes it harder to understand the processing flow (especially if you discard messages at some point so that they won't show up in the other log files)

Post by Josh Bitto
I'm not quite sure how to put the forwarding rule in rsyslog.conf If
I'm kinda confused about this....I don't know if mysqld.log (centos
box) falls under something that you have to monitor...or if I can just
forward the file that is created by the program to another server.

The first thing that you should be doing is making sure that rsyslog on the mysql machine is seeing the logs. The best way to do this is to configure mysql to log to syslog. If you can't do that, then using imfile to check the file periodically and read new items from it is a distant second choice.
Once you are sure that you are getting the logs in syslog on the first machine, then you can worry about getting them to a second machine and determining where they go.
troubleshooting 101, start at the beginning and make sure things are working each step of the way.

Post by Josh Bitto
On my central syslog server the rsyslog.conf I want to use a template
to receive this data and put it in a file so $template DYNmysqld,"/var/log/hosts/%HOSTNAME%/mysqld.log"
If \
$source != 'myhostname'\
And \
$programname == 'mysqld' \
Then ?DYNmysqld
Should this be working?

it depends on what the logs look like. If they are logs generated from the imfile config above, I think they probably will, although I'll point out that doing mixed case on commands "if" "and" "then" is highly questionable at best.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.