Discussion:
[rsyslog] nanoseconds
Jason Nordwick
2018-10-26 21:42:44 UTC
Permalink
Is there a way to get nanoseconds in and out of rsyslog? I'm using
dateformat rfc3339, and it works find for micros, but at nanos, the
timestamp gets correupted:

property(name="timereported" dateFormat="rfc3339")

Also, when writing out json formatted fields, is there a way to print out a
numeric property? I would like to make syslogseverity a number, but I have
been unsuccessful in creating a numeric json field. My workaround is to
handcraft the entire json payload instead of using the jsonf option on the
template.

Thanks,
Jason
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Peter Viskup via rsyslog
2018-10-29 09:34:07 UTC
Permalink
It might be possible to extend the rfc3339 time format to rfc3339nano,
but that will break rfc5424 which allow up to microseconds precision
only. Similar already in use when rfc3164 syslog messages used with
rfc3339 timestamps.

https://github.com/rsyslog/rsyslog/search?p=1&q=date-rfc3339&unscoped_q=date-rfc3339
https://github.com/rsyslog/rsyslog/search?q=formatTimestamp3339&unscoped_q=formatTimestamp3339
https://github.com/rsyslog/rsyslog/search?q=tplFmtRFC3339Date&unscoped_q=tplFmtRFC3339Date
https://tools.ietf.org/html/rfc5424

Consider opening github request for the implementation.

Peter
Post by Jason Nordwick
Is there a way to get nanoseconds in and out of rsyslog? I'm using
dateformat rfc3339, and it works find for micros, but at nanos, the
property(name="timereported" dateFormat="rfc3339")
Also, when writing out json formatted fields, is there a way to print out a
numeric property? I would like to make syslogseverity a number, but I have
been unsuccessful in creating a numeric json field. My workaround is to
handcraft the entire json payload instead of using the jsonf option on the
template.
Thanks,
Jason
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
David Lang
2018-10-29 19:46:54 UTC
Permalink
I think it would be a resonable extension to rfc5424 to allow it's parser to
accept more digits in the timestamp.

I'm not sure that digits beyond microseconds really represent valid time, but I
don't think it's a big deal to support it.

David Lang
Post by Peter Viskup via rsyslog
It might be possible to extend the rfc3339 time format to rfc3339nano,
but that will break rfc5424 which allow up to microseconds precision
only. Similar already in use when rfc3164 syslog messages used with
rfc3339 timestamps.
https://github.com/rsyslog/rsyslog/search?p=1&q=date-rfc3339&unscoped_q=date-rfc3339
https://github.com/rsyslog/rsyslog/search?q=formatTimestamp3339&unscoped_q=formatTimestamp3339
https://github.com/rsyslog/rsyslog/search?q=tplFmtRFC3339Date&unscoped_q=tplFmtRFC3339Date
https://tools.ietf.org/html/rfc5424
Consider opening github request for the implementation.
Peter
Post by Jason Nordwick
Is there a way to get nanoseconds in and out of rsyslog? I'm using
dateformat rfc3339, and it works find for micros, but at nanos, the
property(name="timereported" dateFormat="rfc3339")
Also, when writing out json formatted fields, is there a way to print out a
numeric property? I would like to make syslogseverity a number, but I have
been unsuccessful in creating a numeric json field. My workaround is to
handcraft the entire json payload instead of using the jsonf option on the
template.
Thanks,
Jason
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Jason Nordwick
2018-10-29 23:19:23 UTC
Permalink
It's most useful when you get multiple log messages per microseond as a way
of ordering them as you can do with multiple cpu cores generating log
traffic simultaneously.
Post by David Lang
I think it would be a resonable extension to rfc5424 to allow it's parser to
accept more digits in the timestamp.
I'm not sure that digits beyond microseconds really represent valid time, but I
don't think it's a big deal to support it.
David Lang
Post by Peter Viskup via rsyslog
It might be possible to extend the rfc3339 time format to rfc3339nano,
but that will break rfc5424 which allow up to microseconds precision
only. Similar already in use when rfc3164 syslog messages used with
rfc3339 timestamps.
https://github.com/rsyslog/rsyslog/search?p=1&q=date-rfc3339&unscoped_q=date-rfc3339
https://github.com/rsyslog/rsyslog/search?q=formatTimestamp3339&unscoped_q=formatTimestamp3339
https://github.com/rsyslog/rsyslog/search?q=tplFmtRFC3339Date&unscoped_q=tplFmtRFC3339Date
Post by Peter Viskup via rsyslog
https://tools.ietf.org/html/rfc5424
Consider opening github request for the implementation.
Peter
Post by Jason Nordwick
Is there a way to get nanoseconds in and out of rsyslog? I'm using
dateformat rfc3339, and it works find for micros, but at nanos, the
property(name="timereported" dateFormat="rfc3339")
Also, when writing out json formatted fields, is there a way to print
out a
Post by Peter Viskup via rsyslog
Post by Jason Nordwick
numeric property? I would like to make syslogseverity a number, but I
have
Post by Peter Viskup via rsyslog
Post by Jason Nordwick
been unsuccessful in creating a numeric json field. My workaround is to
handcraft the entire json payload instead of using the jsonf option on
the
Post by Peter Viskup via rsyslog
Post by Jason Nordwick
template.
Thanks,
Jason
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
you DON'T LIKE THAT.
Post by Peter Viskup via rsyslog
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
David Lang
2018-10-30 01:33:27 UTC
Permalink
I question how real the time is, even across cores in a single machine, when you
get down to that sort of timekeeping.

David Lang
Date: Mon, 29 Oct 2018 16:19:23 -0700
Subject: Re: [rsyslog] nanoseconds
It's most useful when you get multiple log messages per microseond as a way
of ordering them as you can do with multiple cpu cores generating log
traffic simultaneously.
Post by David Lang
I think it would be a resonable extension to rfc5424 to allow it's parser to
accept more digits in the timestamp.
I'm not sure that digits beyond microseconds really represent valid time, but I
don't think it's a big deal to support it.
David Lang
Post by Peter Viskup via rsyslog
It might be possible to extend the rfc3339 time format to rfc3339nano,
but that will break rfc5424 which allow up to microseconds precision
only. Similar already in use when rfc3164 syslog messages used with
rfc3339 timestamps.
https://github.com/rsyslog/rsyslog/search?p=1&q=date-rfc3339&unscoped_q=date-rfc3339
https://github.com/rsyslog/rsyslog/search?q=formatTimestamp3339&unscoped_q=formatTimestamp3339
https://github.com/rsyslog/rsyslog/search?q=tplFmtRFC3339Date&unscoped_q=tplFmtRFC3339Date
Post by Peter Viskup via rsyslog
https://tools.ietf.org/html/rfc5424
Consider opening github request for the implementation.
Peter
Post by Jason Nordwick
Is there a way to get nanoseconds in and out of rsyslog? I'm using
dateformat rfc3339, and it works find for micros, but at nanos, the
property(name="timereported" dateFormat="rfc3339")
Also, when writing out json formatted fields, is there a way to print
out a
Post by Peter Viskup via rsyslog
Post by Jason Nordwick
numeric property? I would like to make syslogseverity a number, but I
have
Post by Peter Viskup via rsyslog
Post by Jason Nordwick
been unsuccessful in creating a numeric json field. My workaround is to
handcraft the entire json payload instead of using the jsonf option on
the
Post by Peter Viskup via rsyslog
Post by Jason Nordwick
template.
Thanks,
Jason
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
you DON'T LIKE THAT.
Post by Peter Viskup via rsyslog
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Peter Viskup via rsyslog
2018-11-06 08:57:21 UTC
Permalink
On modern systems the quality of time on all CPU cores should be
synced and "guaranteed".

Some readings related to this topic:
http://jijithchandran.blogspot.com/2014/06/linux-kernel-time-calculation.html
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux_for_real_time/7/html/reference_guide/sect-posix_clocks
https://aufather.wordpress.com/2010/09/08/high-performance-time-measuremen-in-linux/
https://en.wikipedia.org/wiki/Time_Stamp_Counter
https://en.wikipedia.org/wiki/High_Precision_Event_Timer
http://man7.org/linux/man-pages/man7/time.7.html

Peter
Post by David Lang
I question how real the time is, even across cores in a single machine, when you
get down to that sort of timekeeping.
David Lang
Date: Mon, 29 Oct 2018 16:19:23 -0700
Subject: Re: [rsyslog] nanoseconds
It's most useful when you get multiple log messages per microseond as a way
of ordering them as you can do with multiple cpu cores generating log
traffic simultaneously.
Post by David Lang
I think it would be a resonable extension to rfc5424 to allow it's parser to
accept more digits in the timestamp.
I'm not sure that digits beyond microseconds really represent valid time, but I
don't think it's a big deal to support it.
David Lang
Post by Peter Viskup via rsyslog
It might be possible to extend the rfc3339 time format to rfc3339nano,
but that will break rfc5424 which allow up to microseconds precision
only. Similar already in use when rfc3164 syslog messages used with
rfc3339 timestamps.
https://github.com/rsyslog/rsyslog/search?p=1&q=date-rfc3339&unscoped_q=date-rfc3339
https://github.com/rsyslog/rsyslog/search?q=formatTimestamp3339&unscoped_q=formatTimestamp3339
https://github.com/rsyslog/rsyslog/search?q=tplFmtRFC3339Date&unscoped_q=tplFmtRFC3339Date
Post by Peter Viskup via rsyslog
https://tools.ietf.org/html/rfc5424
Consider opening github request for the implementation.
Peter
Post by Jason Nordwick
Is there a way to get nanoseconds in and out of rsyslog? I'm using
dateformat rfc3339, and it works find for micros, but at nanos, the
property(name="timereported" dateFormat="rfc3339")
Also, when writing out json formatted fields, is there a way to print
out a
Post by Peter Viskup via rsyslog
Post by Jason Nordwick
numeric property? I would like to make syslogseverity a number, but I
have
Post by Peter Viskup via rsyslog
Post by Jason Nordwick
been unsuccessful in creating a numeric json field. My workaround is to
handcraft the entire json payload instead of using the jsonf option on
the
Post by Peter Viskup via rsyslog
Post by Jason Nordwick
template.
Thanks,
Jason
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
you DON'T LIKE THAT.
Post by Peter Viskup via rsyslog
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Loading...