Discussion:
[rsyslog] Multiline logging option not working as expected
Bijohn Bijohn
12 years ago
Permalink
Hi,

I am using rsyslog to ship logs to centralized log server. Having *issues
with logs having more than one line*.

Tried $InputFileReadMode 1 in client.
--- If I send logs to a centralized log server, it will send the first line
only. All the other lines are ignored.
--- If I write logs to another file in same client machine, it appends
time stamp to first line but, *still the log is multi-line. I believe it
should be a one liner here escaping new line char.*

client version : rsyslogd 5.8.6
Server version: rsyslogd 5.8.11


*My client config:*
$ModLoad imfile
$InputFileName /tmp/ne.error.log
$InputFileTag test.ne.ne-error
$InputFileStateFile test.ne.ne-error
$InputFileReadMode 1
$InputRunFileMonitor
if $syslogtag == 'test.ne.ne-error' then @@172.17.7.9:514
#& ~
if $syslogtag == 'test.ne.ne-error' then /tmp/ne.error.new
& ~

*My server (*172.17.7.9*) config:*
$template test.ne.ne-error,"/tmp/ne-test.log"
if $syslogtag == 'test.ne.ne-error' then ?test.ne.ne-error
& ~

Please point out if I am doing anything wrong.

--------------------

Tried with following as log:
Newline 1
test1
test2
test3

Newline 2
test4
test5

------------------

In same server, syslog write it as:

Aug 30 17:47:50 myhost test.ne.ne-error Newline 1
test1
test2
test3
Aug 30 17:47:50 myhost test.ne.ne-error Newline 2
test4
test5

--------------------

In remote log server it was like this:
Aug 30 17:47:50 myhost test.ne.ne-error Newline 1
Aug 30 17:47:50 myhost test.ne.ne-error Newline 2
Radu Gheorghe
12 years ago
Permalink
Hello,

AFAIK this is a design flaw of syslog over TCP (at least the RFC 3164).
Because log lines sent over TCP are delimited by a new line.

So far I've sent multi-line logs successfully via UDP, by disabling control
character escaping on the receiving end:
http://www.rsyslog.com/doc/rsconf1_escapecontrolcharactersonreceive.html

But other options should also work. I didn't test them all, but this thread
might give you the pointers you need:
http://www.gossamer-threads.com/lists/rsyslog/users/9619

Best regards,
Radu


2013/9/2 Bijohn Bijohn <bijohnbijohn at gmail.com>
...
Rainer Gerhards
12 years ago
Permalink
If you want to ship messages with embedded LF in them, you can use
octet-counted framing, which solves that problem. It's optional in rsyslog
and turned off by default as most syslogd's do not understand it.

HTH
Rainer
...
David Lang
12 years ago
Permalink
we really do need an option that will cause the imfile module to output an
escape sequence instead of newline when reading in multiline logs, I have been
meaning to throw something together to do this for a while, but haven't gotten
around to it (in part, because I wasn't sure if it was a real problem or just a
theoretical problem)

David Lang
...
Rainer Gerhards
12 years ago
Permalink
Post by David Lang
we really do need an option that will cause the imfile module to output an
escape sequence instead of newline when reading in multiline logs, I have
been meaning to throw something together to do this for a while, but
haven't gotten around to it (in part, because I wasn't sure if it was a
real problem or just a theoretical problem)
Will be part of 7.5.3. Blog posting explains the details:

http://blog.gerhards.net/2013/09/imfile-multi-line-messages.html

Rainer

Loading...