Thank you David, this really helps, but if wildcards do not work for
Post by David Langnot examining for syntax errors in the logrotate config, this looks mostly
correct.
One important thing, the move before the HUP needs to remain on the same
filesystem (a mv to a different filesystem is copy + delete, which would
cause logs to be lost as they get written after the copy completes)
so it's more
/define/my/logfiles
{
prerotate or sharescript
mv /define/my/logfiles /define/my/logfiles.YYMMDD-HHMM
endscript
postrotate
/usr/bin/killall -HUP rsyslogd
mv /define/my/logfiles.YYMMDD-HHMM /final/destination
endscript
compress
}
It's very common to have the logs arrive to a local (fast) disk and then
rotate them off to slower (frequently NFS) storage. On many of my systems, I
have a fast (SCSI/SAS or SSD) disk and then a slow, but huge SATA disk for
the archives.
I also avoid using wildcards in this process. It gets _really_ ugly if it's
not quite finished processing one logfile when the next one rotates if you
have a wildcard and the mv/copy/compress intended for the new file finds the
old one still around and trys to do it again as well.
If you have a long rotation period, this isn't as likely to happen, but I
have systems where I rotate the files every minute and at peak times the
system may fall behind once in a while. with wildcards this turns into a
'box down' result while without it nothing bad happens.
David Lang
Post by robert s:) thanks David, I really appreciate the explanation, it was confusing
me because I couldn't grasp the idea of files being written to while
moved.
So I would just do a logrotate.d for rsyslog,
/define/my/logfiles
{
prerotate or sharescript
mv /logfiles/to/new/YYYYMMDD/format
endscript
postrotate
/usr/bin/killall -HUP rsyslogd
endscript
compress
}
and hopefully it will work :)
Robert
Post by David Langno, it really is move then HUP
the way things work is that rsyslog opens the file, and after that it
accesses the file via a filehandle. That filehandle remains until rsyslog
closes the file. It doesn't matter if you rename, or even delete the file,
rsyslog will still write to the file though that filehandle.
So you rename the file to something new, and rsyslog keeps writing happily
to that file. Then you send rsyslog a HUP and rsyslog closes all outputs and
when the next messages arrive, it opens the files as needed, and when it
goes to open them it finds they do not exist, so it creates new ones (some
old versions of syslog would not create new files, you had to do that before
issuing the HUP)
David Lang
Date: Thu, 14 Nov 2013 15:42:29 -0500
From: robert s <rsyslogrobert at gmail.com>
Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
To: rsyslog-users <rsyslog at lists.adiscon.com>
Subject: Re: [rsyslog] rsyslog log rotation
thanks David,
Lastly, on your first email that you replied to this, you said to move
the files as YYYYMMDD, then hup the process, the question I have here
is this if I move the files while rsyslog is writing to them, won't I
loose files? should I hup the process then move the files instead?
I have many logs that will need to be rotated and zipped, roughly 200
logfiles every 4 hours, I have a script that defines which logs I want
to rotate, the script is a little messy and kinda confusing, there is
error checking and a lot of logic, I am wondering if it would be best
to start clean with logrorate and just do the mv and hup?
Robert
Post by David Langthe command is kill -HUP pid where pid is the pid of the process you
want
to
kill
you can also do killall -HUP processname (in this case processname is
rsyslogd) to send the signal to all processes with that name.
when rsyslog gets a HUP, it logs it, look for logs from rsyslog.
David Lang
Date: Wed, 13 Nov 2013 17:02:03 -0500
From: robert s <rsyslogrobert at gmail.com>
Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
To: rsyslog-users <rsyslog at lists.adiscon.com>
Subject: Re: [rsyslog] rsyslog log rotation
Hello guys,
regarding the HUP, is there a particular way that the commands needs
to be executed? I ran rsyslogd kill -HUP but it didnt do anything,
then I read a man page that said to run it like : kill -HUP $(sudo cat
/var/run/syslogd.pid), but im not sure how to determine whether it ran
it or not, is there a log that I can view whether this command ran?
...Im just a bit confused, thanks in advanced for the help.
Robert
Post by robert sOh Im sorry, I was thinking it was something relating to rsyslog.
thanks
David
Robert
Post by David Langhttp://linuxcommand.org/man_pages/logrotate8.html
or 'man logrotate'
David Lang
Post by robert sThanks for the response Lucas,
I am not sure what you are referring to the logrotate program?
Robert
On Fri, Nov 8, 2013 at 5:59 PM, Lucas Tobey <lucas.tobey at 1and1.com>
Post by Lucas TobeyHi,
Have you considered using the 'logrotate' program for this? This
may
be
a better/easier solution?
Sincerely,
-Lucas
Post by robert sHello Everyone,
I am full of questions today, I am reading a few things on the
documentation regarding log rotation and I am curious if this can be
I would like to rotate logs all the logs every 4 hours, but at the
same time not stop the rsyslog service, so there is no loss of data
during the rotation process, is this possible?
The only place that I was able to find some info was to use a template
$template
DailyPerHostLogs,"/var/log/logdir/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%_messages.log"
*.info;mail.none;authpriv.none;cron.none
-?DailyPerHostLogs
If there might be any more documentation that I may be pointed to I
would really appreciate it.
Robert
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
you DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
you
DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
you
DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.