Scot Kreienkamp
2018-12-04 16:49:46 UTC
Hi everyone,
I have a program that is sending long lines into the logs.... they are arriving at the local server and into the logs correctly but are being truncated when being received on a remote server, and I can't understand why.
In crontab I have a program being run like so:
*/5 * * * * java foo |& nc -w 240 localhost 1516
Basically it's running the java program and redirecting all IO to netcat, which is sending it to port 1516 on localhost via the default TCP. That is working fine, and the local logfile has all the content in it. The problem comes when I'm trying to forward to a remote host. The local file has a line that goes on for pages, however the remote end writes the line into the file but truncates that line after approximately 950 characters and then does really weird things with the rest of that incoming line. Sometimes it shows up under other servers (the destination separates by hostname and tag), sometimes it creates new folders with partial names of servers, etc.
Here's my rule that encompasses the local file write and the forward:
template(name="RMS-ForwardDMA" type="list") {
constant(value="<")
property(name="pri")
constant(value=">")
property(name="timestamp" dateFormat="rfc3339")
constant(value=" ")
constant(value="retv6030.na.lzb.hq")
constant(value=" ")
constant(value="RMS-DMA:")
property(name="msg" spifno1stsp="on" )
property(name="msg")
}
ruleset(name="RMS-DMA-1516-1531"
queue.filename="Net-1516-1531"
queue.dequeuebatchsize="1024"
queue.discardmark="45000"
queue.DiscardSeverity="0"
queue.maxDiskSpace="1g"
queue.saveOnShutdown="on"
queue.size="50000"
queue.spoolDirectory="/var/spool/rsyslog"
queue.type="LinkedList"
){
action(
name="omfile-RMS-ForwardDMA"
type="omfile"
dynafilecachesize="5"
DynaFile="DailyPerHostLogDMA"
template="msgonly-no1sp"
ioBufferSize="64k"
flushOnTXEnd="off"
asyncWriting="on"
dirCreateMode="0755"
)
action(
name="fwd-monvsyslog-1531-RMS-DMA"
queue.dequeuebatchsize="1024"
queue.discardmark="45000"
queue.DiscardSeverity="0"
queue.maxDiskSpace="1g"
queue.saveOnShutdown="on"
queue.size="50000"
queue.spoolDirectory="/var/spool/rsyslog"
queue.type="LinkedList"
queue.filename="fwd-monvsyslog-1531-RMS-DMA"
type="omfwd"
TCP_Framing="octet-counted"
protocol="tcp"
Target="monvsyslog.na.lzb.hq"
Port="1531"
template="RMS-ForwardDMA"
)
stop
}
#setup local port listeners for ecomm
input(type="imudp" port="1516" address="127.0.0.1" ruleset="RMS-DMA-1516-1531")
input(type="imptcp" port="1516" address="127.0.0.1" ruleset="RMS-DMA-1516-1531")
And here's my receiving line:
if $syslogtag startswith "RMS-DMA:" then {
action(name="Net-1531-DMA" type="omfile" dynafilecachesize="50" DynaFile="DailyPerHostLogDMA" ioBufferSize="128k" flushOnTXEnd="off" asyncWriting="on" dirCreateMode="0755" template="msgonly")
stop
}
Scot Kreienkamp | Senior Systems Engineer | La-Z-Boy Corporate
One La-Z-Boy Drive | Monroe, Michigan 48162 | * 734-384-6403 | | * 7349151444 | * ***@la-z-boy.com<mailto:%7BE-mail%7D>
www<http://www.la-z-boy.com/>.la-z-boy.com<http://www.la-z-boy.com/> | facebook.<https://www.facebook.com/lazboy>com<https://www.facebook.com/lazboy>/<https://www.facebook.com/lazboy>lazboy<http://facebook.com/lazboy> | twitter.com/lazboy<https://twitter.com/lazboy> | youtube.com/<https://www.youtube.com/user/lazboy>lazboy<https://www.youtube.com/user/lazboy>
[cid:lzbVertical_hres.jpg]
This message is intended only for the individual or entity to which it is addressed. It may contain privileged, confidential information which is exempt from disclosure under applicable laws. If you are not the intended recipient, you are strictly prohibited from disseminating or distributing this information (other than to the intended recipient) or copying this information. If you have received this communication in error, please notify us immediately by e-mail or by telephone at the above number. Thank you.
I have a program that is sending long lines into the logs.... they are arriving at the local server and into the logs correctly but are being truncated when being received on a remote server, and I can't understand why.
In crontab I have a program being run like so:
*/5 * * * * java foo |& nc -w 240 localhost 1516
Basically it's running the java program and redirecting all IO to netcat, which is sending it to port 1516 on localhost via the default TCP. That is working fine, and the local logfile has all the content in it. The problem comes when I'm trying to forward to a remote host. The local file has a line that goes on for pages, however the remote end writes the line into the file but truncates that line after approximately 950 characters and then does really weird things with the rest of that incoming line. Sometimes it shows up under other servers (the destination separates by hostname and tag), sometimes it creates new folders with partial names of servers, etc.
Here's my rule that encompasses the local file write and the forward:
template(name="RMS-ForwardDMA" type="list") {
constant(value="<")
property(name="pri")
constant(value=">")
property(name="timestamp" dateFormat="rfc3339")
constant(value=" ")
constant(value="retv6030.na.lzb.hq")
constant(value=" ")
constant(value="RMS-DMA:")
property(name="msg" spifno1stsp="on" )
property(name="msg")
}
ruleset(name="RMS-DMA-1516-1531"
queue.filename="Net-1516-1531"
queue.dequeuebatchsize="1024"
queue.discardmark="45000"
queue.DiscardSeverity="0"
queue.maxDiskSpace="1g"
queue.saveOnShutdown="on"
queue.size="50000"
queue.spoolDirectory="/var/spool/rsyslog"
queue.type="LinkedList"
){
action(
name="omfile-RMS-ForwardDMA"
type="omfile"
dynafilecachesize="5"
DynaFile="DailyPerHostLogDMA"
template="msgonly-no1sp"
ioBufferSize="64k"
flushOnTXEnd="off"
asyncWriting="on"
dirCreateMode="0755"
)
action(
name="fwd-monvsyslog-1531-RMS-DMA"
queue.dequeuebatchsize="1024"
queue.discardmark="45000"
queue.DiscardSeverity="0"
queue.maxDiskSpace="1g"
queue.saveOnShutdown="on"
queue.size="50000"
queue.spoolDirectory="/var/spool/rsyslog"
queue.type="LinkedList"
queue.filename="fwd-monvsyslog-1531-RMS-DMA"
type="omfwd"
TCP_Framing="octet-counted"
protocol="tcp"
Target="monvsyslog.na.lzb.hq"
Port="1531"
template="RMS-ForwardDMA"
)
stop
}
#setup local port listeners for ecomm
input(type="imudp" port="1516" address="127.0.0.1" ruleset="RMS-DMA-1516-1531")
input(type="imptcp" port="1516" address="127.0.0.1" ruleset="RMS-DMA-1516-1531")
And here's my receiving line:
if $syslogtag startswith "RMS-DMA:" then {
action(name="Net-1531-DMA" type="omfile" dynafilecachesize="50" DynaFile="DailyPerHostLogDMA" ioBufferSize="128k" flushOnTXEnd="off" asyncWriting="on" dirCreateMode="0755" template="msgonly")
stop
}
Scot Kreienkamp | Senior Systems Engineer | La-Z-Boy Corporate
One La-Z-Boy Drive | Monroe, Michigan 48162 | * 734-384-6403 | | * 7349151444 | * ***@la-z-boy.com<mailto:%7BE-mail%7D>
www<http://www.la-z-boy.com/>.la-z-boy.com<http://www.la-z-boy.com/> | facebook.<https://www.facebook.com/lazboy>com<https://www.facebook.com/lazboy>/<https://www.facebook.com/lazboy>lazboy<http://facebook.com/lazboy> | twitter.com/lazboy<https://twitter.com/lazboy> | youtube.com/<https://www.youtube.com/user/lazboy>lazboy<https://www.youtube.com/user/lazboy>
[cid:lzbVertical_hres.jpg]
This message is intended only for the individual or entity to which it is addressed. It may contain privileged, confidential information which is exempt from disclosure under applicable laws. If you are not the intended recipient, you are strictly prohibited from disseminating or distributing this information (other than to the intended recipient) or copying this information. If you have received this communication in error, please notify us immediately by e-mail or by telephone at the above number. Thank you.