[rsyslog] Usign DynaFile option with custom message template

Discussion:

Ferenc Zuellich

2018-11-16 20:21:55 UTC

Hi all,

I'm having trouble using a dynamic file option in combination with a custom
message template. In my scenario multiple applications talk to rsyslog via
a UDP port. I would like to separate the incoming log messages by
application name. To simplify configuration and have it extensible I'd like
to define the log name dynamically based on the app name.

I'm running rsyslog in version 8.3.3 on Debian 8 inside a VirtualBox.

My configuration looks as follows:

module(load="imudp")
input(type="imudp" address="*" port="8514" ruleset="writeToFile")

template(name = "customMessageTemplate" type = "list") {
property(name="syslogpriority-text" caseConversion="upper")
constant(value=" ")
property(name="structured-data")
constant(value=" ")
property(name="timegenerated")
constant(value=" ")
property(name="msg")
constant(value="\n")
}

template(name = "DynaFileTemplate" string="/var/log/apps/%programname%.log"
type="string")
ruleset(name="writeToFile") {
action(
type="omfile"
template="customMessageTemplate"
DynaFile="DynaFileTemplate"
dirCreateMode="0740"
dirGroup="admin"
fileGroup="admin")
}

I could already confirm that:

- Removing the line *template="customMessageTemplate"* leads to the
creation of log files with a dynamic file name. In other words it works.
The only thing missing is my custom message format.
- Removing the line *DynaFile="DynaFileTemplate"* and adding a file
property leads to the creation of the log file. The messages inside are
formatted according to the custom format.

So in theory my configuration is correct, except that the combination of
DynaFile and Template doesn't seem to work. I have checked in
*/var/log/syslog* for error messages in between rsyslogd restarts, but I
could find none. I ran *rsyslogd -dn* and looked for errors. I can't say I
found any. But I noticed this output:

6336.150931206:main thread : name: (unset)
6336.150933518:main thread : type: 'omfile'
6336.150936227:main thread : action.writeallmarkmessages: (unset)
6336.150938269:main thread : action.execonlyeverynthtime: (unset)
6336.150940198:main thread : action.execonlyeverynthtimetimeout: (unset)
6336.150942119:main thread : action.execonlyonceeveryinterval: (unset)
6336.150944078:main thread : action.execonlywhenpreviousissuspended:
(unset)
6336.150945997:main thread : action.repeatedmsgcontainsoriginalmsg:
(unset)
6336.150947914:main thread : action.resumeretrycount: (unset)
6336.150949818:main thread : action.reportsuspension: (unset)
6336.150951763:main thread : action.reportsuspensioncontinuation: (unset)
6336.150953678:main thread : action.resumeinterval: (unset)
6336.150957462:main thread : newActInst (omfile)
6336.150959979:main thread : nvlstGetParam: name 'dirgroup', type 3,
valnode->bUsed 0
6336.151009164:main thread : param 'dirgroup': uid 1000 obtained for
group 'admin'
6336.151012079:main thread : nvlstGetParam: name 'filegroup', type 3,
valnode->bUsed 0
6336.151131952:main thread : param 'filegroup': uid 1000 obtained for
group 'admin'
6336.151135928:main thread : nvlstGetParam: name 'dircreatemode', type
5, valnode->bUsed 0
6336.151137769:main thread : nvlstGetParam: name 'dynafile', type 14,
valnode->bUsed 0
6336.151139400:main thread : nvlstGetParam: name 'template', type 13,
valnode->bUsed 0
6336.151141015:main thread : action param blk in omfile:
6336.151142066:main thread : dynafilecachesize: (unset)
6336.151144050:main thread : ziplevel: (unset)
6336.151145998:main thread : flushinterval: (unset)
6336.151147907:main thread : asyncwriting: (unset)
6336.151149823:main thread : veryrobustzip: (unset)
6336.151151744:main thread : flushontxend: (unset)
6336.151153611:main thread : iobuffersize: (unset)
6336.151155528:main thread : dirowner: (unset)
6336.151157425:main thread : dirownernum: (unset)
6336.151159310:main thread : dirgroup: 1000
6336.151161496:main thread : dirgroupnum: (unset)
6336.151163394:main thread : fileowner: (unset)
6336.151165262:main thread : fileownernum: (unset)
6336.151167133:main thread : filegroup: 1000
6336.151169166:main thread : filegroupnum: (unset)
6336.151171064:main thread : dircreatemode: 480
6336.151173151:main thread : filecreatemode: (unset)
6336.151175091:main thread : failonchownfailure: (unset)
6336.151176990:main thread : createdirs: (unset)
6336.151178907:main thread : sync: (unset)
6336.151180805:main thread : file: (unset)
6336.151182721:main thread : dynafile: 'DynaFileTemplate'
6336.151185071:main thread : sig.provider: (unset)
6336.151186960:main thread : cry.provider: (unset)
6336.151188861:main thread : closetimeout: (unset)
6336.151190739:main thread : template: 'customMessageTemplate'
6336.151202063:main thread : janitor: entry 0xfeec70, id
'omfile:dynafile:DynaFileTemplate:0xff0370' added

*6336.151204258:main thread : Module builtin:omfile processes this
action.6336.151217998:main thread : template: 'customMessageTemplate'
assigned6336.151219385:main thread : template: 'DynaFileTemplate'
assigned*

Which looks like rsyslog configures the action and overwrites the template
setting. Which I wouldn't expect.

Does somebody have any idea? I scanned the changelogs, too, and where
unable to find anything regarding bugs when using DynaFile and template
together.

Greetings

Ferenc
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

Rainer Gerhards

2018-11-17 10:03:37 UTC

Permalink

Your config should work. 8.3.3. is an unstable devel version from 2014
- probably it simply has a bug. We are currently at 8.39.0.

HTH
Rainer
El vie., 16 nov. 2018 a las 22:22, Ferenc Zuellich

Post by Ferenc Zuellich
Hi all,
I'm having trouble using a dynamic file option in combination with a custom
message template. In my scenario multiple applications talk to rsyslog via
a UDP port. I would like to separate the incoming log messages by
application name. To simplify configuration and have it extensible I'd like
to define the log name dynamically based on the app name.
I'm running rsyslog in version 8.3.3 on Debian 8 inside a VirtualBox.
module(load="imudp")
input(type="imudp" address="*" port="8514" ruleset="writeToFile")
template(name = "customMessageTemplate" type = "list") {
property(name="syslogpriority-text" caseConversion="upper")
constant(value=" ")
property(name="structured-data")
constant(value=" ")
property(name="timegenerated")
constant(value=" ")
property(name="msg")
constant(value="\n")
}
template(name = "DynaFileTemplate" string="/var/log/apps/%programname%.log"
type="string")
ruleset(name="writeToFile") {
action(
type="omfile"
template="customMessageTemplate"
DynaFile="DynaFileTemplate"
dirCreateMode="0740"
dirGroup="admin"
fileGroup="admin")
}
- Removing the line *template="customMessageTemplate"* leads to the
creation of log files with a dynamic file name. In other words it works.
The only thing missing is my custom message format.
- Removing the line *DynaFile="DynaFileTemplate"* and adding a file
property leads to the creation of the log file. The messages inside are
formatted according to the custom format.
So in theory my configuration is correct, except that the combination of
DynaFile and Template doesn't seem to work. I have checked in
*/var/log/syslog* for error messages in between rsyslogd restarts, but I
could find none. I ran *rsyslogd -dn* and looked for errors. I can't say I
6336.150931206:main thread : name: (unset)
6336.150933518:main thread : type: 'omfile'
6336.150936227:main thread : action.writeallmarkmessages: (unset)
6336.150938269:main thread : action.execonlyeverynthtime: (unset)
6336.150940198:main thread : action.execonlyeverynthtimetimeout: (unset)
6336.150942119:main thread : action.execonlyonceeveryinterval: (unset)
(unset)
(unset)
6336.150947914:main thread : action.resumeretrycount: (unset)
6336.150949818:main thread : action.reportsuspension: (unset)
6336.150951763:main thread : action.reportsuspensioncontinuation: (unset)
6336.150953678:main thread : action.resumeinterval: (unset)
6336.150957462:main thread : newActInst (omfile)
6336.150959979:main thread : nvlstGetParam: name 'dirgroup', type 3,
valnode->bUsed 0
6336.151009164:main thread : param 'dirgroup': uid 1000 obtained for
group 'admin'
6336.151012079:main thread : nvlstGetParam: name 'filegroup', type 3,
valnode->bUsed 0
6336.151131952:main thread : param 'filegroup': uid 1000 obtained for
group 'admin'
6336.151135928:main thread : nvlstGetParam: name 'dircreatemode', type
5, valnode->bUsed 0
6336.151137769:main thread : nvlstGetParam: name 'dynafile', type 14,
valnode->bUsed 0
6336.151139400:main thread : nvlstGetParam: name 'template', type 13,
valnode->bUsed 0
6336.151142066:main thread : dynafilecachesize: (unset)
6336.151144050:main thread : ziplevel: (unset)
6336.151145998:main thread : flushinterval: (unset)
6336.151147907:main thread : asyncwriting: (unset)
6336.151149823:main thread : veryrobustzip: (unset)
6336.151151744:main thread : flushontxend: (unset)
6336.151153611:main thread : iobuffersize: (unset)
6336.151155528:main thread : dirowner: (unset)
6336.151157425:main thread : dirownernum: (unset)
6336.151159310:main thread : dirgroup: 1000
6336.151161496:main thread : dirgroupnum: (unset)
6336.151163394:main thread : fileowner: (unset)
6336.151165262:main thread : fileownernum: (unset)
6336.151167133:main thread : filegroup: 1000
6336.151169166:main thread : filegroupnum: (unset)
6336.151171064:main thread : dircreatemode: 480
6336.151173151:main thread : filecreatemode: (unset)
6336.151175091:main thread : failonchownfailure: (unset)
6336.151176990:main thread : createdirs: (unset)
6336.151178907:main thread : sync: (unset)
6336.151180805:main thread : file: (unset)
6336.151182721:main thread : dynafile: 'DynaFileTemplate'
6336.151185071:main thread : sig.provider: (unset)
6336.151186960:main thread : cry.provider: (unset)
6336.151188861:main thread : closetimeout: (unset)
6336.151190739:main thread : template: 'customMessageTemplate'
6336.151202063:main thread : janitor: entry 0xfeec70, id
'omfile:dynafile:DynaFileTemplate:0xff0370' added
*6336.151204258:main thread : Module builtin:omfile processes this
action.6336.151217998:main thread : template: 'customMessageTemplate'
assigned6336.151219385:main thread : template: 'DynaFileTemplate'
assigned*
Which looks like rsyslog configures the action and overwrites the template
setting. Which I wouldn't expect.
Does somebody have any idea? I scanned the changelogs, too, and where
unable to find anything regarding bugs when using DynaFile and template
together.
Greetings
Ferenc
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNS

Ferenc Zuellich

2018-11-19 11:21:15 UTC

Permalink

Dear Rainer,

I update rsyslog to 8.23 and changed my template from %programname% to
%app-name%. Now it works.

Thanks for your help!

Greetings

Ferenc

Am Sa., 17. Nov. 2018 um 11:04 Uhr schrieb Rainer Gerhards <

Post by Rainer Gerhards
Your config should work. 8.3.3. is an unstable devel version from 2014
- probably it simply has a bug. We are currently at 8.39.0.
HTH
Rainer
El vie., 16 nov. 2018 a las 22:22, Ferenc Zuellich

Post by Ferenc Zuellich
Hi all,
I'm having trouble using a dynamic file option in combination with a

custom

Post by Ferenc Zuellich
message template. In my scenario multiple applications talk to rsyslog

via

Post by Ferenc Zuellich
a UDP port. I would like to separate the incoming log messages by
application name. To simplify configuration and have it extensible I'd

Post by Ferenc Zuellich
to define the log name dynamically based on the app name.
I'm running rsyslog in version 8.3.3 on Debian 8 inside a VirtualBox.
module(load="imudp")
input(type="imudp" address="*" port="8514" ruleset="writeToFile")
template(name = "customMessageTemplate" type = "list") {
property(name="syslogpriority-text" caseConversion="upper")
constant(value=" ")
property(name="structured-data")
constant(value=" ")
property(name="timegenerated")
constant(value=" ")
property(name="msg")
constant(value="\n")
}
template(name = "DynaFileTemplate"

string="/var/log/apps/%programname%.log"

Post by Ferenc Zuellich
type="string")
ruleset(name="writeToFile") {
action(
type="omfile"
template="customMessageTemplate"
DynaFile="DynaFileTemplate"
dirCreateMode="0740"
dirGroup="admin"
fileGroup="admin")
}
- Removing the line *template="customMessageTemplate"* leads to the
creation of log files with a dynamic file name. In other words it

works.

Post by Ferenc Zuellich
The only thing missing is my custom message format.
- Removing the line *DynaFile="DynaFileTemplate"* and adding a file
property leads to the creation of the log file. The messages inside

are

Post by Ferenc Zuellich
formatted according to the custom format.
So in theory my configuration is correct, except that the combination of
DynaFile and Template doesn't seem to work. I have checked in
*/var/log/syslog* for error messages in between rsyslogd restarts, but I
could find none. I ran *rsyslogd -dn* and looked for errors. I can't say

Post by Ferenc Zuellich
6336.150931206:main thread : name: (unset)
6336.150933518:main thread : type: 'omfile'
6336.150936227:main thread : action.writeallmarkmessages: (unset)
6336.150938269:main thread : action.execonlyeverynthtime: (unset)

(unset)

Post by Ferenc Zuellich
6336.150942119:main thread : action.execonlyonceeveryinterval: (unset)
(unset)
(unset)
6336.150947914:main thread : action.resumeretrycount: (unset)
6336.150949818:main thread : action.reportsuspension: (unset)

(unset)

Post by Ferenc Zuellich
6336.150953678:main thread : action.resumeinterval: (unset)
6336.150957462:main thread : newActInst (omfile)
6336.150959979:main thread : nvlstGetParam: name 'dirgroup', type 3,
valnode->bUsed 0
6336.151009164:main thread : param 'dirgroup': uid 1000 obtained for
group 'admin'
6336.151012079:main thread : nvlstGetParam: name 'filegroup', type 3,
valnode->bUsed 0
6336.151131952:main thread : param 'filegroup': uid 1000 obtained for
group 'admin'
6336.151135928:main thread : nvlstGetParam: name 'dircreatemode', type
5, valnode->bUsed 0
6336.151137769:main thread : nvlstGetParam: name 'dynafile', type 14,
valnode->bUsed 0
6336.151139400:main thread : nvlstGetParam: name 'template', type 13,
valnode->bUsed 0
6336.151142066:main thread : dynafilecachesize: (unset)
6336.151144050:main thread : ziplevel: (unset)
6336.151145998:main thread : flushinterval: (unset)
6336.151147907:main thread : asyncwriting: (unset)
6336.151149823:main thread : veryrobustzip: (unset)
6336.151151744:main thread : flushontxend: (unset)
6336.151153611:main thread : iobuffersize: (unset)
6336.151155528:main thread : dirowner: (unset)
6336.151157425:main thread : dirownernum: (unset)
6336.151159310:main thread : dirgroup: 1000
6336.151161496:main thread : dirgroupnum: (unset)
6336.151163394:main thread : fileowner: (unset)
6336.151165262:main thread : fileownernum: (unset)
6336.151167133:main thread : filegroup: 1000
6336.151169166:main thread : filegroupnum: (unset)
6336.151171064:main thread : dircreatemode: 480
6336.151173151:main thread : filecreatemode: (unset)
6336.151175091:main thread : failonchownfailure: (unset)
6336.151176990:main thread : createdirs: (unset)
6336.151178907:main thread : sync: (unset)
6336.151180805:main thread : file: (unset)
6336.151182721:main thread : dynafile: 'DynaFileTemplate'
6336.151185071:main thread : sig.provider: (unset)
6336.151186960:main thread : cry.provider: (unset)
6336.151188861:main thread : closetimeout: (unset)
6336.151190739:main thread : template: 'customMessageTemplate'
6336.151202063:main thread : janitor: entry 0xfeec70, id
'omfile:dynafile:DynaFileTemplate:0xff0370' added
*6336.151204258:main thread : Module builtin:omfile processes this
action.6336.151217998:main thread : template: 'customMessageTemplate'
assigned6336.151219385:main thread : template: 'DynaFileTemplate'
assigned*
Which looks like rsyslog configures the action and overwrites the

template

Post by Ferenc Zuellich
setting. Which I wouldn't expect.
Does somebody have any idea? I scanned the changelogs, too, and where
unable to find anything regarding bugs when using DynaFile and template
together.
Greetings
Ferenc
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad

of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.