Discussion:
[rsyslog] Missing messages in our databases
Rainer Gerhards
2018-11-19 17:44:38 UTC
Permalink
Do you have any rsyslog error messages "in front of" the quoted ones?
They look like the are second in an error report - but may
unfortunately be the only ones.

Nothing looks obviously wrong to me.

Rainer
El lun., 19 nov. 2018 a las 18:02, Mike Schleif
What does the following mean?
How can we correct this?
We just discovered missing rsyslog messages in our databases, going back to
February 2018.
Feb 3 02:16:33 hermes rsyslogd[30458]: The error statement was: insert
into SystemEvents (Message, Facility, FromHost, Priority,
DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('Connection
reset by 172.99.99.100 port 39596 [preauth]', 10, 'hermes', 6,
'20180203021157', '20180203021157', 1, 'sshd[17977]:') [v8.32.0 try
http://www.rsyslog.com/e/2218 ]
Nov 15 11:24:23 hermes rsyslogd[701]: The error statement was: insert into
SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime,
ReceivedAt, InfoUnitID, SysLogTag) values ('pam_unix(sshd:session): session
opened for user ms50013 by (uid=0)', 10, 'hermes', 6, '20181115112422',
'20181115112422', 1, 'sshd[12048]:') [v8.39.0 try
http://www.rsyslog.com/e/2218 ]
Nov 15 11:39:43 hermes rsyslogd[701]: The error statement was: insert into
SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime,
ReceivedAt, InfoUnitID, SysLogTag) values ('pam_unix(sshd:session): session
closed for user ms50013', 10, 'hermes', 6, '20181115113943',
'20181115113943', 1, 'sshd[12048]:') [v8.39.0 try
http://www.rsyslog.com/e/2218 ]
Below are ommysql configurations for two connections to same host,
different
databases. We do not find any of these failures for ActionName Ftp, only
Sftp.
$ActionName Ftp
$ActionQueueDequeueSlowdown 1000 # How long (in microseconds) dequeueing
should be delayed
$ActionQueueFileName dbFtpQueue # Set file name, also enables disk mode
$ActionQueueSaveOnShutdown on # Save messages to disk on shutdown
$ActionQueueType LinkedList # Use asynchronous processing
$ActionResumeRetryCount -1 # Infinite retries on insert failure
ftp.*
:ommysql:172.99.99.125,vsftplog,hermesvsftplog,_PASSWORD_
$ActionName Sftp
$ActionQueueDequeueSlowdown 1000 # How long (in microseconds) dequeueing
should be delayed
$ActionQueueFileName dbSftpQueue # Set file name, also enables disk mode
$ActionQueueSaveOnShutdown on # Save messages to disk on shutdown
$ActionQueueType LinkedList # Use asynchronous processing
$ActionResumeRetryCount -1 # Infinite retries on insert failure
authpriv.*
:ommysql:172.99.99.125,sftplogDB,hermesvsftplog,_PASSWORD_
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE a
Rainer Gerhards
2018-11-19 20:04:52 UTC
Permalink
El lun., 19 nov. 2018 a las 19:47, Mike Schleif
Sorry, I didn't notice this earlier :(
Post by Rainer Gerhards
Do you have any rsyslog error messages "in front of" the quoted ones?
They look like the are second in an error report - but may
unfortunately be the only ones.
Nothing looks obviously wrong to me.
Rainer
Feb 3 02:16:33 hermes rsyslogd[30458]: ommysql: db error (1172): Result
consisted of more than one row [v8.32.0]
Feb 3 02:16:33 hermes rsyslogd[30458]: The error statement was: insert
into SystemEvents (Message, Facility, FromHost, Priority,
DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('Connection
reset by 172.31.128.100 port 39596 [preauth]', 10, 'hermes', 6,
'20180203021157', '20180203021157', 1, 'sshd[17977]:') [v8.32.0 try
http://www.rsyslog.com/e/2218 ]
I admit I do notnow what that means in the context here. More than one
row ... I don't understand why this can happen with an insert
statement. Anyone?

Rainer
Nov 15 11:39:43 hermes rsyslogd[701]: ommysql: db error (1172): Result
consisted of more than one row [v8.39.0]
Nov 15 11:39:43 hermes rsyslogd[701]: The error statement was: insert into
SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime,
ReceivedAt, InfoUnitID, SysLogTag) values ('pam_unix(sshd:session): session
closed for user ms50013', 10, 'hermes', 6, '20181115113943',
'20181115113943', 1, 'sshd[12048]:') [v8.39.0 try
http://www.rsyslog.com/e/2218 ]
_What_ is more than one row?
What can we do about this?
Please, advise. Thank you.
~ Mike
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
David Lang
2018-11-19 19:25:06 UTC
Permalink
Try writing the log message to a file using the same template that you are using
to send it to the database, you may find that there's an embedded newline or
other puncutation in the failing message.

David Lang

On Mon, 19 Nov 2018, Rainer Gerhards
Date: Mon, 19 Nov 2018 21:04:52 +0100
Subject: Re: [rsyslog] Missing messages in our databases
El lun., 19 nov. 2018 a las 19:47, Mike Schleif
Sorry, I didn't notice this earlier :(
Post by Rainer Gerhards
Do you have any rsyslog error messages "in front of" the quoted ones?
They look like the are second in an error report - but may
unfortunately be the only ones.
Nothing looks obviously wrong to me.
Rainer
Feb 3 02:16:33 hermes rsyslogd[30458]: ommysql: db error (1172): Result
consisted of more than one row [v8.32.0]
Feb 3 02:16:33 hermes rsyslogd[30458]: The error statement was: insert
into SystemEvents (Message, Facility, FromHost, Priority,
DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('Connection
reset by 172.31.128.100 port 39596 [preauth]', 10, 'hermes', 6,
'20180203021157', '20180203021157', 1, 'sshd[17977]:') [v8.32.0 try
http://www.rsyslog.com/e/2218 ]
I admit I do notnow what that means in the context here. More than one
row ... I don't understand why this can happen with an insert
statement. Anyone?
Rainer
Nov 15 11:39:43 hermes rsyslogd[701]: ommysql: db error (1172): Result
consisted of more than one row [v8.39.0]
Nov 15 11:39:43 hermes rsyslogd[701]: The error statement was: insert into
SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime,
ReceivedAt, InfoUnitID, SysLogTag) values ('pam_unix(sshd:session): session
closed for user ms50013', 10, 'hermes', 6, '20181115113943',
'20181115113943', 1, 'sshd[12048]:') [v8.39.0 try
http://www.rsyslog.com/e/2218 ]
_What_ is more than one row?
What can we do about this?
Please, advise. Thank you.
~ Mike
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
David Lang
2018-11-19 19:52:31 UTC
Permalink
How can I do that?
in your config for writing to mysql you specify a template to use, use that same
template to write to a file

old syntax

/var/log/foo_file;bar_template

new syntax

action(type="omfile" filename="foo_file" template="bar_template")

in general, if things aren't working the way you think they should, look at the
actual data.

you can also use the template RSYSLOG_Debug_Format to get a dump of everything
that rsyslog knows about the log message to understand what your templates are
doing.

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Rich Megginson via rsyslog
2018-11-20 15:15:14 UTC
Permalink
David,
Post by David Lang
How can I do that?
in your config for writing to mysql you specify a template to use, use that same
template to write to a file
old syntax
/var/log/foo_file;bar_template
new syntax
action(type="omfile" filename="foo_file" template="bar_template")
I'm trying to do this; but, the only occurrence of the word "template" in
# Use default timestamp format
module(load="builtin:omfile" template="RSYSLOG_TraditionalFileFormat")
How ought I modify this?
You could try this:

# debug output

module(load="builtin:omfile" file="/var/log/rsyslog-debug.log" template="RSYSLOG_TraditionalFileFormat")

If RSYSLOG_TraditionalFileFormat doesn't give you enough detail, try RSYSLOG_DebugFormat
~ Mike
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Jacob Steinberger via rsyslog
2018-11-20 15:56:47 UTC
Permalink
What am I missing?
More than likely, you have a message coming across the wire with a
errant character in it, that when pushed to MySQL causes the error
you're seeing. The question is, what does that message look like, what
character is causing the problem, and then what kind of setup you can do
to strip that errant character. The previous suggestions are to attempt
to find those messages.
All of those events to be sent to the database are also going to
/var/log/secure.
What is the purpose of this rsyslog-debug.log?
Will this "debug output" rsyslog-debug.log be in _addition_ to regular
logfiles, or replace my current configuration?
Depends on your entire configuration and the placement of the action.
The goal would be to add a debug file, that may contain the problem
message, so you can then move forward with a technical fix.
We currently have ~10 different logfiles - if this debug configuration
replaces the original, won't that concatenate all logs into one file?
I'd suggest you pick a file name that isn't used, so it doesn't
*replace* anything.

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Rich Megginson via rsyslog
2018-11-20 15:58:59 UTC
Permalink
Jacob's answer is better, but I'll answer your questions below.
What am I missing?
On Tue, Nov 20, 2018 at 9:15 AM Rich Megginson via rsyslog <
Post by Rich Megginson via rsyslog
# debug output
module(load="builtin:omfile" file="/var/log/rsyslog-debug.log"
template="RSYSLOG_TraditionalFileFormat")
If RSYSLOG_TraditionalFileFormat doesn't give you enough detail, try RSYSLOG_DebugFormat
All of those events to be sent to the database are also going to
/var/log/secure.
Hmm - strange - it is ignoring file="/var/log/rsyslog-debug.log" ?
What is the purpose of this rsyslog-debug.log?
To see what is being sent to mysql
Will this "debug output" rsyslog-debug.log be in _addition_ to regular
logfiles, or replace my current configuration?
Yes.
We currently have ~10 different logfiles - if this debug configuration
replaces the original, won't that concatenate all logs into one file?
It doesn't _replace_ the original, it is _in addition to_ the original.

Yes - so if that isn't acceptable, you'll have to add some sort of logic to say "only send the logs I'm sending to mysql to the debug log file"
~ Mike
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Jacob Steinberger via rsyslog
2018-11-20 16:01:42 UTC
Permalink
Post by Rich Megginson via rsyslog
It doesn't _replace_ the original, it is _in addition to_ the original.
Yes - so if that isn't acceptable, you'll have to add some sort of
logic to say "only send the logs I'm sending to mysql to the debug log
file"
Can't reinforce that last sentence enough. Since we're working off
snippets of a configuration, who knows what the rest of the file looks
like. Stringing together a few dozen filters and `&~` could make
placement of any extra action to log out data critical.

Only you know what your entire configuration looks like, and only you
can prevent forest fires.


_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Jacob Steinberger via rsyslog
2018-11-20 15:32:20 UTC
Permalink
I'm trying to do this; but, the only occurrence of the word "template" in
# Use default timestamp format
module(load="builtin:omfile" template="RSYSLOG_TraditionalFileFormat")
How ought I modify this?
That template is used for just writing to a file, not to a database.
What does your configuration file look like for inserting data into a
database?

Jacob

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Jacob Steinberger via rsyslog
2018-11-20 15:53:19 UTC
Permalink
That is in the original post. There I showed two configurations to the same
database and host - one that has never failed, and the errant one.
So you did.

Old school config. Based on the errors, I assume you're running 8.39.
Would recommend using "action" type syntax, as it's much prettier for us
poor part-timers.

https://www.rsyslog.com/doc/v8-stable/configuration/modules/ommysql.html

And I quote:

Rsyslog contains a canned default template to write to the MySQL
database. It works on the MonitorWare schema. This template is:

$template tpl,"insert into SystemEvents (Message, Facility, FromHost,
Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values
('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%,
'%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%,
'%syslogtag%')",SQL

Try this in your config:

$template tpltext, "insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values
('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%,'%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%,
'%syslogtag%')"
module(load="builtin:omfile" file="/var/log/rsyslog-debug.log" template="tpltext")

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Jacob Steinberger via rsyslog
2018-11-20 16:30:38 UTC
Permalink
On Tue, Nov 20, 2018 at 9:53 AM Jacob Steinberger via rsyslog <
Post by Jacob Steinberger via rsyslog
$template tpltext, "insert into SystemEvents (Message, Facility, FromHost,
Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values
('%msg%', %syslogfacility%, '%HOSTNAME%',
%syslogpriority%,'%timereported:::date-mysql%',
'%timegenerated:::date-mysql%', %iut%,
'%syslogtag%')"
module(load="builtin:omfile" file="/var/log/rsyslog-debug.log" template="tpltext")
What am I missing now?
What I'm giving may not be 100%, it may require you to do some work for
syntax checking. I'm basically working off memory and quick google searches.

https://www.rsyslog.com/doc/v8-stable/configuration/modules/omfile.html
138:$template tpltext, "insert into SystemEvents (Message, Facility,
FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag)
values ('%msg%', %syslogfacility%, '%HOSTNAME%',
%syslogpriority%,'%timereported:::date-mysql%',
'%timegenerated:::date-mysql%', %iut%, '%syslogtag%')"
139:module(load="builtin:omfile" file="/var/log/rsyslog-debug.log"
template="tpltext")
Technically the documentation says file should be "File", you could give
that a try.

Are you running Rsyslog 8.39?

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Joe Blow via rsyslog
2018-11-20 21:40:53 UTC
Permalink
You don't need to load the omfile module at all, unless you're changing
config parameters:
<snip>
Configuration Parameters
<https://www.rsyslog.com/doc/v8-stable/configuration/modules/omfile.html#configuration-parameters>

*Omfile is a built-in module that does not need to be loaded.* In order to
specify module parameters, use

module(load="builtin:omfile" ...parameters...)

</snip>

Don't even bother loading that, it should work without it.

This is how I use omfile:

<snip1>

template(name="json_syslog"


type="list") {


constant(value="{")


constant(value="\"@timestamp\":\"")
property(name="timereported" dateFormat="rfc3339")


constant(value="\",\"type\":\"syslog_json")


constant(value="\",\"tag\":\"")
property(name="syslogtag" format="json")


constant(value="\",\"relayhost\":\"")
property(name="fromhost")


constant(value="\",\"relayip\":\"")
property(name="fromhost-ip")


constant(value="\",\"logsource\":\"") property(name="source")


constant(value="\",\"hostname\":\"")
property(name="hostname" caseconversion="lower")


constant(value="\",\"program\":\"")
property(name="programname")


constant(value="\",\"priority\":\"") property(name="pri")


constant(value="\",\"severity\":\"")
property(name="syslogseverity")


constant(value="\",\"facility\":\"")
property(name="syslogfacility")


constant(value="\",\"severity_label\":\"")
property(name="syslogseverity-text")


constant(value="\",\"facility_label\":\"")
property(name="syslogfacility-text")


constant(value="\",\"message\":\"")
property(name="rawmsg" format="json")


constant(value="\",\"end_msg\":\"")


constant(value="\"}\n")


}

action(type="omfile" file="/var/log/fail2ban.log" template="json_syslog")


action(type="omfile" file="/var/log/suricata.log"
template="json_syslog")

</snip1>

The answer to your question #1 is in green. You do not need to load
the module at all for it to work, so your answer to #2 is "nothing".

HTH

Cheers,

JB
On Tue, Nov 20, 2018 at 10:30 AM Jacob Steinberger via rsyslog <
Post by Jacob Steinberger via rsyslog
Technically the documentation says file should be "File", you could give
that a try.
Are you running Rsyslog 8.39?
Yes, we are running the latest: v8.39.0
1) How to configure two (2) instances of omfile?
parameters for built-in module builtin:omfile already set - ignored
[v8.39.0 try http://www.rsyslog.com/e/2220 ]
2) How to configure the module line?
parameter 'template' not known -- typo in config file?
parameter 'File' not known -- typo in config file?
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Flo Rance via rsyslog
2018-11-21 14:35:12 UTC
Permalink
This should do what you want.

authpriv.* action(type="omfile" file="/var/log/rsyslog-debug.log"
template="RSYSLOG_DebugFormat")

Flo
On Tue, Nov 20, 2018 at 3:41 PM Joe Blow via rsyslog <
Post by Joe Blow via rsyslog
You don't need to load the omfile module at all, unless you're changing
<snip>
Don't even bother loading that, it should work without it.
<snip>
action(type="omfile" file="/var/log/suricata.log" template="json_syslog")
<snip>
The answer to your question #1 is in green. You do not need to load
the module at all for it to work, so your answer to #2 is "nothing".
HTH
Cheers,
JB
How can I use omfile _only_ for authpriv.* ?
~ Mike
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Flo Rance via rsyslog
2018-11-21 14:40:27 UTC
Permalink
Btw, this syntax is working, too.

authpriv.* /var/log/rsyslog-debug.log;RSYSLOG_DebugFormat

Flo
On Tue, Nov 20, 2018 at 3:41 PM Joe Blow via rsyslog <
Post by Joe Blow via rsyslog
You don't need to load the omfile module at all, unless you're changing
<snip>
Don't even bother loading that, it should work without it.
<snip>
action(type="omfile" file="/var/log/suricata.log" template="json_syslog")
<snip>
The answer to your question #1 is in green. You do not need to load
the module at all for it to work, so your answer to #2 is "nothing".
HTH
Cheers,
JB
How can I use omfile _only_ for authpriv.* ?
~ Mike
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Joe Blow via rsyslog
2018-11-21 17:48:40 UTC
Permalink
Try reading through some of the use cases implemented here:

https://github.com/SparkITSolutions/phoenix/tree/legacy/install/rsyslog

That will have examples for many things you want to do.

Cheers,

JB
On Tue, Nov 20, 2018 at 9:53 AM Jacob Steinberger via rsyslog <
Post by Jacob Steinberger via rsyslog
$template tpltext, "insert into SystemEvents (Message, Facility,
FromHost,
Post by Jacob Steinberger via rsyslog
Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values
('%msg%', %syslogfacility%, '%HOSTNAME%',
%syslogpriority%,'%timereported:::date-mysql%',
'%timegenerated:::date-mysql%', %iut%,
'%syslogtag%')"
module(load="builtin:omfile" file="/var/log/rsyslog-debug.log" template="tpltext")
module(load="builtin:omfile" template="RSYSLOG_TraditionalFileFormat")
$template tpltext, "insert into SystemEvents (Message, Facility, FromHost,
Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values
('%msg%', %syslogfacility%, '%HOSTNAME%',
%syslogpriority%,'%timereported:::date-mysql%',
'%timegenerated:::date-mysql%', %iut%, '%syslogtag%')\n"
authpriv.* action(type="omfile" file="/var/log/rsyslog-debug.log"
template="tpltext")
rsyslog-debug.log
Now, how can I send all log entries containing a fixed string to a new
logfile?
For example, this is an intermittent problem, and I discovered it on
The error statement was: insert into SystemEvents
If I can get all of those messages into a separate logfile, I only need see
if that file has been updated, to know that another intermittent problem
occurred.
Please, advise. Thank you.
~ Mike
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Rainer Gerhards
2018-11-21 18:00:34 UTC
Permalink
El mié., 21 nov. 2018 a las 16:23, Mike Schleif
On Tue, Nov 20, 2018 at 9:53 AM Jacob Steinberger via rsyslog <
Post by Jacob Steinberger via rsyslog
$template tpltext, "insert into SystemEvents (Message, Facility, FromHost,
Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values
('%msg%', %syslogfacility%, '%HOSTNAME%',
%syslogpriority%,'%timereported:::date-mysql%',
'%timegenerated:::date-mysql%', %iut%,
'%syslogtag%')"
module(load="builtin:omfile" file="/var/log/rsyslog-debug.log" template="tpltext")
module(load="builtin:omfile" template="RSYSLOG_TraditionalFileFormat")
$template tpltext, "insert into SystemEvents (Message, Facility, FromHost,
Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values
('%msg%', %syslogfacility%, '%HOSTNAME%',
%syslogpriority%,'%timereported:::date-mysql%',
'%timegenerated:::date-mysql%', %iut%, '%syslogtag%')\n"
authpriv.* action(type="omfile" file="/var/log/rsyslog-debug.log"
template="tpltext")
rsyslog-debug.log
Now, how can I send all log entries containing a fixed string to a new
logfile?
https://www.rsyslog.com/doc/v8-stable/configuration/filters.html
For example, this is an intermittent problem, and I discovered it on
The error statement was: insert into SystemEvents
If I can get all of those messages into a separate logfile, I only need see
if that file has been updated, to know that another intermittent problem
occurred.
Please, advise. Thank you.
~ Mike
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRI
Rainer Gerhards
2018-11-21 19:55:02 UTC
Permalink
Post by Rainer Gerhards
El mié., 21 nov. 2018 a las 16:23, Mike Schleif
Now, how can I send all log entries containing a fixed string to a new
logfile?
https://www.rsyslog.com/doc/v8-stable/configuration/filters.html
How to configure _actions_ with Property-Based Filters?
You can only configure actions with filters. For all other objects it makes
no sense.

It's

Filter action

Or

Filter { action1, action 2, ...}

If you have multiple for the same filter.

Overview:
https://www.rsyslog.com/doc/v8-stable/configuration/basic_structure.html#quick-overview-of-message-flow-and-objects

Sample:

If $rawmsg contains "bla" Action(...)

HTH
Rainer
~ Mike
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE an
Rainer Gerhards
2018-11-21 19:57:31 UTC
Permalink
Or

authpriv.* action(...)

Or any other filter.

But you cannot do

Authpriv.* Input()

Because this makes no sense at all.

Rainer

Sent from phone, thus brief.
Am Mi., 21. Nov. 2018, 20:44 hat Mike Schleif <
On Wed, Nov 21, 2018 at 12:01 PM Rainer Gerhards <
Post by Rainer Gerhards
El mié., 21 nov. 2018 a las 16:23, Mike Schleif
Now, how can I send all log entries containing a fixed string to a new
logfile?
https://www.rsyslog.com/doc/v8-stable/configuration/filters.html
How to configure _actions_ with Property-Based Filters?
You can only configure actions with filters. For all other objects it
makes no sense.
It's
Filter action
Or
Filter { action1, action 2, ...}
If you have multiple for the same filter.
https://www.rsyslog.com/doc/v8-stable/configuration/basic_structure.html#quick-overview-of-message-flow-and-objects
If $rawmsg contains "bla" Action(...)
HTH
Rainer
~ Mike
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POS
David Lang
2018-11-21 19:57:19 UTC
Permalink
I saw this example: :msg, contains, "informational" ~
The tilde is _not_ documented, except anecdotally. I wonder if there are
other possibilities for that ~ position?
~ is the same as stop, that is the action position, and any action type (either
the new action() statement or a legacy action) can be put here. You can also put
a series of actions inside braces {} there.

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Rainer Gerhards
2018-11-21 20:59:46 UTC
Permalink
Sent from phone, thus brief.
Am Mi., 21. Nov. 2018, 20:44 hat Mike Schleif <
On Wed, Nov 21, 2018 at 12:01 PM Rainer Gerhards <
Post by Rainer Gerhards
El mié., 21 nov. 2018 a las 16:23, Mike Schleif
Now, how can I send all log entries containing a fixed string to a
new
Post by Rainer Gerhards
logfile?
https://www.rsyslog.com/doc/v8-stable/configuration/filters.html
How to configure _actions_ with Property-Based Filters?
You can only configure actions with filters. For all other objects it
makes
no sense.
Yes, I understand that.
However, are _Property-Based Filters_ filters, or something not filters?
A filter is a filter, its just different types
if $msg contains 'The error statement was: insert into SystemEvents' then
/var/log/errInsertSystemEvents.log
:msg,contains,"The error statement was: insert into SystemEvents" ???
I saw this example: :msg, contains, "informational" ~
The tilde is _not_ documented, except anecdotally. I wonder if there are
other possibilities for that ~ position?
~ is the old name for stop (action). So

:msg, contains, "informational" /var/log/errInsertSystemEvents.log

Or

:msg, contains, "informational" {
/var/log/errInsertSystemEvents.log
Action(...) # other actions
}

So it's just a filter... Any filter works

*.*
If
...

Doc PRs are welcome if the wording can be clarified.

Rainer
~ Mike
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE U
Flo Rance via rsyslog
2018-11-20 07:55:46 UTC
Permalink
I would advise to take a look at the mysql logs, there may be more verbose
about the statement used in that case.

Regards,
Flo
Sorry, I didn't notice this earlier :(
Post by Rainer Gerhards
Do you have any rsyslog error messages "in front of" the quoted ones?
They look like the are second in an error report - but may
unfortunately be the only ones.
Nothing looks obviously wrong to me.
Rainer
Feb 3 02:16:33 hermes rsyslogd[30458]: ommysql: db error (1172): Result
consisted of more than one row [v8.32.0]
Feb 3 02:16:33 hermes rsyslogd[30458]: The error statement was: insert
into SystemEvents (Message, Facility, FromHost, Priority,
DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('Connection
reset by 172.31.128.100 port 39596 [preauth]', 10, 'hermes', 6,
'20180203021157', '20180203021157', 1, 'sshd[17977]:') [v8.32.0 try
http://www.rsyslog.com/e/2218 ]
Nov 15 11:39:43 hermes rsyslogd[701]: ommysql: db error (1172): Result
consisted of more than one row [v8.39.0]
Nov 15 11:39:43 hermes rsyslogd[701]: The error statement was: insert into
SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime,
ReceivedAt, InfoUnitID, SysLogTag) values ('pam_unix(sshd:session): session
closed for user ms50013', 10, 'hermes', 6, '20181115113943',
'20181115113943', 1, 'sshd[12048]:') [v8.39.0 try
http://www.rsyslog.com/e/2218 ]
_What_ is more than one row?
What can we do about this?
Please, advise. Thank you.
~ Mike
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Loading...