[rsyslog] Ruleset : send to server over UDP instead of TCP

Discussion:

sophie.loewenthal--- via rsyslog

2018-11-12 13:44:13 UTC

Hi,

Will this rule send the messages to the server over UDP or TCP? I would like this to be UDP.

# RuleSet
*.info { action (type="omelasticsearch"
server="el8"
serverport="10514"
searchIndex="unix"
bulkmode="on"
template="ElasticSearchTemplate"
name="el8-514-out"
queue.size="1024000"
queue.filename="el8-10514.queue"
queue.spoolDirectory="/soft/rsyslog/queues"
queue.maxdiskspace="512m"
queue.type="FixedArray"
queue.maxfilesize="20m"
queue.saveonshutdown="on"
queue.discardseverity="6"
Action.ResumeInterval="1"
Action.ResumeRetryCount="-1"
)
}

This page gives examples in the old format, but not for the new format:
https://www.rsyslog.com/doc/v8-stable/configuration/actions.html

Best wishes,
Sophie

This message and any attachments (the "message") is
intended solely for the intended addressees and is confidential.
If you receive this message in error,or are not the intended recipient(s),
please delete it and any copies from your systems and immediately notify
the sender. Any unauthorized view, use that does not comply with its purpose,
dissemination or disclosure, either whole or partial, is prohibited. Since the internet
cannot guarantee the integrity of this message which may not be reliable, BNP PARIBAS
(and its subsidiaries) shall not be liable for the message if modified, changed or falsified.
Do not print this message unless it is necessary, consider the environment.

----------------------------------------------------------------------------------------------------------------------------------

Ce message et toutes les pieces jointes (ci-apres le "message")
sont etablis a l'intention exclusive de ses destinataires et sont confidentiels.
Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
merci de le detruire ainsi que toute copie de votre systeme et d'en avertir
immediatement l'expediteur. Toute lecture non autorisee, toute utilisation de
ce message qui n'est pas conforme a sa destination, toute diffusion ou toute
publication, totale ou partielle, est interdite. L'Internet ne permettant pas d'assurer
l'integrite de ce message electronique susceptible d'alteration, BNP Paribas
(et ses filiales) decline(nt) toute responsabilite au titre de ce message dans l'hypothese
ou il aurait ete modifie, deforme ou falsifie.
N'imprimez ce message que si necessaire, pensez a l'environnement.

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

sophie.loewenthal--- via rsyslog

2018-11-12 14:11:57 UTC

Permalink

I thought this could work, but nope:

protocol="tcp" / protocol="udp"

rsyslogd: error during parsing file /etc/rsyslog.conf, on or before line 70: parameter 'protocol' not known -- typo in config file? [v8.24.0 try http://www.rsyslog.com/e/2207 ]

-----Original Message-----
sophie.loewenthal--- via rsyslog
Sent: Monday, November 12, 2018 2:44 PM
To: rsyslog-users
Cc: LOEWENTHAL Sophie
Subject: [rsyslog] Ruleset : send to server over UDP instead of TCP
Hi,
Will this rule send the messages to the server over UDP or TCP? I would like this
to be UDP.
# RuleSet
*.info { action (type="omelasticsearch"
server="el8"
serverport="10514"
searchIndex="unix"
bulkmode="on"
template="ElasticSearchTemplate"
name="el8-514-out"
queue.size="1024000"
queue.filename="el8-10514.queue"
queue.spoolDirectory="/soft/rsyslog/queues"
queue.maxdiskspace="512m"
queue.type="FixedArray"
queue.maxfilesize="20m"
queue.saveonshutdown="on"
queue.discardseverity="6"
Action.ResumeInterval="1"
Action.ResumeRetryCount="-1"
)
}
https://www.rsyslog.com/doc/v8-stable/configuration/actions.html
Best wishes,
Sophie
This message and any attachments (the "message") is
intended solely for the intended addressees and is confidential.
If you receive this message in error,or are not the intended recipient(s),
please delete it and any copies from your systems and immediately notify
the sender. Any unauthorized view, use that does not comply with its purpose,
dissemination or disclosure, either whole or partial, is prohibited. Since the internet
cannot guarantee the integrity of this message which may not be reliable, BNP PARIBAS
(and its subsidiaries) shall not be liable for the message if modified, changed or
falsified.
Do not print this message unless it is necessary, consider the environment.
--------------------------------------------------------------------------------------------------
--------------------------------
Ce message et toutes les pieces jointes (ci-apres le "message")
sont etablis a l'intention exclusive de ses destinataires et sont confidentiels.
Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
merci de le detruire ainsi que toute copie de votre systeme et d'en avertir
immediatement l'expediteur. Toute lecture non autorisee, toute utilisation de
ce message qui n'est pas conforme a sa destination, toute diffusion ou toute
publication, totale ou partielle, est interdite. L'Internet ne permettant pas d'assurer
l'integrite de ce message electronique susceptible d'alteration, BNP Paribas
(et ses filiales) decline(nt) toute responsabilite au titre de ce message dans l'hypothese
ou il aurait ete modifie, deforme ou falsifie.
N'imprimez ce message que si necessaire, pensez a l'environnement.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.

Rainer Gerhards

2018-11-12 14:17:42 UTC

Permalink

where did you place this? It's not in the config you posted. And, no,
omelastisearch does not UDP (neither does Elasticsearch AFIK).

Rainer
El lun., 12 nov. 2018 a las 15:12, sophie.loewenthal--- via rsyslog

Post by sophie.loewenthal--- via rsyslog
protocol="tcp" / protocol="udp"
rsyslogd: error during parsing file /etc/rsyslog.conf, on or before line 70: parameter 'protocol' not known -- typo in config file? [v8.24.0 try http://www.rsyslog.com/e/2207 ]

-----Original Message-----
sophie.loewenthal--- via rsyslog
Sent: Monday, November 12, 2018 2:44 PM
To: rsyslog-users
Cc: LOEWENTHAL Sophie
Subject: [rsyslog] Ruleset : send to server over UDP instead of TCP
Hi,
Will this rule send the messages to the server over UDP or TCP? I would like this
to be UDP.
# RuleSet
*.info { action (type="omelasticsearch"
server="el8"
serverport="10514"
searchIndex="unix"
bulkmode="on"
template="ElasticSearchTemplate"
name="el8-514-out"
queue.size="1024000"
queue.filename="el8-10514.queue"
queue.spoolDirectory="/soft/rsyslog/queues"
queue.maxdiskspace="512m"
queue.type="FixedArray"
queue.maxfilesize="20m"
queue.saveonshutdown="on"
queue.discardseverity="6"
Action.ResumeInterval="1"
Action.ResumeRetryCount="-1"
)
}
https://www.rsyslog.com/doc/v8-stable/configuration/actions.html
Best wishes,
Sophie
This message and any attachments (the "message") is
intended solely for the intended addressees and is confidential.
If you receive this message in error,or are not the intended recipient(s),
please delete it and any copies from your systems and immediately notify
the sender. Any unauthorized view, use that does not comply with its purpose,
dissemination or disclosure, either whole or partial, is prohibited. Since the internet
cannot guarantee the integrity of this message which may not be reliable, BNP PARIBAS
(and its subsidiaries) shall not be liable for the message if modified, changed or
falsified.
Do not print this message unless it is necessary, consider the environment.
--------------------------------------------------------------------------------------------------
--------------------------------
Ce message et toutes les pieces jointes (ci-apres le "message")
sont etablis a l'intention exclusive de ses destinataires et sont confidentiels.
Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
merci de le detruire ainsi que toute copie de votre systeme et d'en avertir
immediatement l'expediteur. Toute lecture non autorisee, toute utilisation de
ce message qui n'est pas conforme a sa destination, toute diffusion ou toute
publication, totale ou partielle, est interdite. L'Internet ne permettant pas d'assurer
l'integrite de ce message electronique susceptible d'alteration, BNP Paribas
(et ses filiales) decline(nt) toute responsabilite au titre de ce message dans
l'hypothese
ou il aurait ete modifie, deforme ou falsifie.
N'imprimez ce message que si necessaire, pensez a l'environnement.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.

Flo Rance via rsyslog

2018-11-12 14:18:41 UTC

Permalink

Hi,

It seems that the primary purpose of omelasticsearch is to send logs to
elesticsearch rest, running on http or https, thus using tcp. It's not
intended to use udp.

https://www.rsyslog.com/doc/v8-stable/configuration/modules/omelasticsearch.html

But I might be wrong...

Flo

On Mon, Nov 12, 2018 at 3:12 PM sophie.loewenthal--- via rsyslog <

Post by sophie.loewenthal--- via rsyslog
protocol="tcp" / protocol="udp"
rsyslogd: error during parsing file /etc/rsyslog.conf, on or before line
70: parameter 'protocol' not known -- typo in config file? [v8.24.0 try
http://www.rsyslog.com/e/2207 ]

like this

to be UDP.
# RuleSet
*.info { action (type="omelasticsearch"
server="el8"
serverport="10514"
searchIndex="unix"
bulkmode="on"
template="ElasticSearchTemplate"
name="el8-514-out"
queue.size="1024000"
queue.filename="el8-10514.queue"
queue.spoolDirectory="/soft/rsyslog/queues"
queue.maxdiskspace="512m"
queue.type="FixedArray"
queue.maxfilesize="20m"
queue.saveonshutdown="on"
queue.discardseverity="6"
Action.ResumeInterval="1"
Action.ResumeRetryCount="-1"
)
}
https://www.rsyslog.com/doc/v8-stable/configuration/actions.html
Best wishes,
Sophie
This message and any attachments (the "message") is
intended solely for the intended addressees and is confidential.
If you receive this message in error,or are not the intended

recipient(s),

please delete it and any copies from your systems and immediately notify
the sender. Any unauthorized view, use that does not comply with its

purpose,

dissemination or disclosure, either whole or partial, is prohibited.

Since the

internet
cannot guarantee the integrity of this message which may not be

reliable, BNP

PARIBAS
(and its subsidiaries) shall not be liable for the message if modified,

changed or

falsified.
Do not print this message unless it is necessary, consider the

environment.
--------------------------------------------------------------------------------------------------

--------------------------------
Ce message et toutes les pieces jointes (ci-apres le "message")
sont etablis a l'intention exclusive de ses destinataires et sont

confidentiels.

Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
merci de le detruire ainsi que toute copie de votre systeme et d'en

avertir

immediatement l'expediteur. Toute lecture non autorisee, toute

utilisation de

ce message qui n'est pas conforme a sa destination, toute diffusion ou

toute

publication, totale ou partielle, est interdite. L'Internet ne

permettant pas

d'assurer
l'integrite de ce message electronique susceptible d'alteration, BNP

Paribas

(et ses filiales) decline(nt) toute responsabilite au titre de ce

message dans

l'hypothese
ou il aurait ete modifie, deforme ou falsifie.
N'imprimez ce message que si necessaire, pensez a l'environnement.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad

sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.

sophie.loewenthal--- via rsyslog

2018-11-12 14:39:38 UTC

Permalink

Flo wrote:> But I might be wrong...
You are right. I had thought that the elasticsearch type just sent messages in a specific format.

Rainer wrote:> where did you place this? It's not in the config you posted. And, no,omelastisearch does not UDP (neither does Elasticsearch AFIK).
The logstash is listening on UDP, and it's just waiting for some json formatted syslog messages. This is then fed in to ESearch.

I have replaced the type() with omfwd, which I think is more appropriate. My template (not included below) is a json template that logstash expects. I've included this below. Should the files in the queues directory decrease?

*.info { action (type="omfwd"
server="el8"
serverport="10514"
protocol="udp"
searchIndex="unix"
bulkmode="on"
template="ElasticSearchTemplate"
name="el8-514-out"
queue.spoolDirectory="/soft/rsyslog/queues"
queue.size="1024000"
queue.filename="el8-10514.queue"
queue.maxdiskspace="512m"
queue.type="FixedArray"
queue.maxfilesize="20m"
queue.saveonshutdown="on"
queue.discardseverity="6"
Action.ResumeInterval="1"
Action.ResumeRetryCount="-1"
)
}

Contents of queues directory:
total 179M
-rw------- 1 rsyslog rsyslog 21M Nov 8 10:24 el7-10514.queue.00000001
-rw------- 1 rsyslog rsyslog 21M Nov 12 10:50 el7-10514.queue.00000002
-rw------- 1 rsyslog rsyslog 21M Nov 12 10:50 el7-10514.queue.00000003
-rw------- 1 rsyslog rsyslog 21M Nov 12 15:10 el7-10514.queue.00000004
-rw------- 1 rsyslog rsyslog 9.3M Nov 12 15:20 el7-10514.queue.00000005
-rw------- 1 rsyslog rsyslog 579 Nov 12 15:20 el7-10514.queue.qi
-rw------- 1 rsyslog rsyslog 21M Nov 8 10:24 el8-10514.queue.00000001
-rw------- 1 rsyslog rsyslog 21M Nov 12 10:50 el8-10514.queue.00000002
-rw------- 1 rsyslog rsyslog 21M Nov 12 10:50 el8-10514.queue.00000003
-rw------- 1 rsyslog rsyslog 21M Nov 12 15:10 el8-10514.queue.00000004
-rw------- 1 rsyslog rsyslog 9.3M Nov 12 15:20 el8-10514.queue.00000005
-rw------- 1 rsyslog rsyslog 579 Nov 12 15:20 el8-10514.queue.qi

From: Flo Rance [mailto:***@gmail.com]
Sent: Monday, November 12, 2018 3:19 PM
To: rsyslog-users
Cc: LOEWENTHAL Sophie
Subject: Re: [rsyslog] Ruleset : send to server over UDP instead of TCP

Hi,

It seems that the primary purpose of omelasticsearch is to send logs to elesticsearch rest, running on http or https, thus using tcp. It's not intended to use udp.

https://www.rsyslog.com/doc/v8-stable/configuration/modules/omelasticsearch.html

But I might be wrong...

Flo

On Mon, Nov 12, 2018 at 3:12 PM sophie.loewenthal--- via rsyslog <***@lists.adiscon.com> wrote:

I thought this could work, but nope:

protocol="tcp" / protocol="udp"

rsyslogd: error during parsing file /etc/rsyslog.conf, on or before line 70: parameter 'protocol' not known -- typo in config file? [v8.24.0 try http://www.rsyslog.com/e/2207 ]

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POS

Flo Rance via rsyslog

2018-11-12 14:54:45 UTC

Permalink

Hi,

Most probably not. The syntax for the omfwd module is different.

action(type="omfwd" target="el8" port="10514" protocol="tcp" ...)

https://www.rsyslog.com/doc/v8-stable/configuration/modules/omfwd.html

Flo

Post by sophie.loewenthal--- via rsyslog
Flo wrote:> But I might be wrong...
You are right. I had thought that the elasticsearch type just sent
messages in a specific format.
Rainer wrote:> where did you place this? It's not in the config you
posted. And, no,omelastisearch does not UDP (neither does Elasticsearch
AFIK).
The logstash is listening on UDP, and it's just waiting for some json
formatted syslog messages. This is then fed in to ESearch.
I have replaced the type() with omfwd, which I think is more appropriate.
My template (not included below) is a json template that logstash expects.
I've included this below. Should the files in the queues directory
decrease?
*.info { action (type="omfwd"
server="el8"
serverport="10514"
protocol="udp"
searchIndex="unix"
bulkmode="on"
template="ElasticSearchTemplate"
name="el8-514-out"
queue.spoolDirectory="/soft/rsyslog/queues"
queue.size="1024000"
queue.filename="el8-10514.queue"
queue.maxdiskspace="512m"
queue.type="FixedArray"
queue.maxfilesize="20m"
queue.saveonshutdown="on"
queue.discardseverity="6"
Action.ResumeInterval="1"
Action.ResumeRetryCount="-1"
)
}
total 179M
-rw------- 1 rsyslog rsyslog 21M Nov 8 10:24 el7-10514.queue.00000001
-rw------- 1 rsyslog rsyslog 21M Nov 12 10:50 el7-10514.queue.00000002
-rw------- 1 rsyslog rsyslog 21M Nov 12 10:50 el7-10514.queue.00000003
-rw------- 1 rsyslog rsyslog 21M Nov 12 15:10 el7-10514.queue.00000004
-rw------- 1 rsyslog rsyslog 9.3M Nov 12 15:20 el7-10514.queue.00000005
-rw------- 1 rsyslog rsyslog 579 Nov 12 15:20 el7-10514.queue.qi
-rw------- 1 rsyslog rsyslog 21M Nov 8 10:24 el8-10514.queue.00000001
-rw------- 1 rsyslog rsyslog 21M Nov 12 10:50 el8-10514.queue.00000002
-rw------- 1 rsyslog rsyslog 21M Nov 12 10:50 el8-10514.queue.00000003
-rw------- 1 rsyslog rsyslog 21M Nov 12 15:10 el8-10514.queue.00000004
-rw------- 1 rsyslog rsyslog 9.3M Nov 12 15:20 el8-10514.queue.00000005
-rw------- 1 rsyslog rsyslog 579 Nov 12 15:20 el8-10514.queue.qi
Sent: Monday, November 12, 2018 3:19 PM
To: rsyslog-users
Cc: LOEWENTHAL Sophie
Subject: Re: [rsyslog] Ruleset : send to server over UDP instead of TCP
Hi,
It seems that the primary purpose of omelasticsearch is to send logs to
elesticsearch rest, running on http or https, thus using tcp. It's not
intended to use udp.
https://www.rsyslog.com/doc/v8-stable/configuration/modules/omelasticsearch.html
But I might be wrong...
Flo
On Mon, Nov 12, 2018 at 3:12 PM sophie.loewenthal--- via rsyslog <
protocol="tcp" / protocol="udp"
rsyslogd: error during parsing file /etc/rsyslog.conf, on or before line
70: parameter 'protocol' not known -- typo in config file? [v8.24.0 try
http://www.rsyslog.com/e/2207 ]

like this

recipient(s),

please delete it and any copies from your systems and immediately notify
the sender. Any unauthorized view, use that does not comply with its

purpose,

dissemination or disclosure, either whole or partial, is prohibited.

Since the

internet
cannot guarantee the integrity of this message which may not be

reliable, BNP

PARIBAS
(and its subsidiaries) shall not be liable for the message if modified,

changed or

falsified.
Do not print this message unless it is necessary, consider the

environment.
--------------------------------------------------------------------------------------------------

--------------------------------
Ce message et toutes les pieces jointes (ci-apres le "message")
sont etablis a l'intention exclusive de ses destinataires et sont

confidentiels.

Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
merci de le detruire ainsi que toute copie de votre systeme et d'en

avertir

immediatement l'expediteur. Toute lecture non autorisee, toute

utilisation de

ce message qui n'est pas conforme a sa destination, toute diffusion ou

toute

publication, totale ou partielle, est interdite. L'Internet ne

permettant pas

d'assurer
l'integrite de ce message electronique susceptible d'alteration, BNP

Paribas

(et ses filiales) decline(nt) toute responsabilite au titre de ce

message dans

sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.

Flo Rance via rsyslog

2018-11-12 15:01:15 UTC

Permalink

My bad, UDP not TCP.

action(type="omfwd" target="el8" port="10514" protocol="udp" ...)

Post by Flo Rance via rsyslog
Hi,
Most probably not. The syntax for the omfwd module is different.
action(type="omfwd" target="el8" port="10514" protocol="tcp" ...)
https://www.rsyslog.com/doc/v8-stable/configuration/modules/omfwd.html
Flo

Post by sophie.loewenthal--- via rsyslog
Flo wrote:> But I might be wrong...
You are right. I had thought that the elasticsearch type just sent
messages in a specific format.
Rainer wrote:> where did you place this? It's not in the config you
posted. And, no,omelastisearch does not UDP (neither does Elasticsearch
AFIK).
The logstash is listening on UDP, and it's just waiting for some json
formatted syslog messages. This is then fed in to ESearch.
I have replaced the type() with omfwd, which I think is more
appropriate. My template (not included below) is a json template that
logstash expects. I've included this below. Should the files in the
queues directory decrease?
*.info { action (type="omfwd"
server="el8"
serverport="10514"
protocol="udp"
searchIndex="unix"
bulkmode="on"
template="ElasticSearchTemplate"
name="el8-514-out"
queue.spoolDirectory="/soft/rsyslog/queues"
queue.size="1024000"
queue.filename="el8-10514.queue"
queue.maxdiskspace="512m"
queue.type="FixedArray"
queue.maxfilesize="20m"
queue.saveonshutdown="on"
queue.discardseverity="6"
Action.ResumeInterval="1"
Action.ResumeRetryCount="-1"
)
}
total 179M
-rw------- 1 rsyslog rsyslog 21M Nov 8 10:24 el7-10514.queue.00000001
-rw------- 1 rsyslog rsyslog 21M Nov 12 10:50 el7-10514.queue.00000002
-rw------- 1 rsyslog rsyslog 21M Nov 12 10:50 el7-10514.queue.00000003
-rw------- 1 rsyslog rsyslog 21M Nov 12 15:10 el7-10514.queue.00000004
-rw------- 1 rsyslog rsyslog 9.3M Nov 12 15:20 el7-10514.queue.00000005
-rw------- 1 rsyslog rsyslog 579 Nov 12 15:20 el7-10514.queue.qi
-rw------- 1 rsyslog rsyslog 21M Nov 8 10:24 el8-10514.queue.00000001
-rw------- 1 rsyslog rsyslog 21M Nov 12 10:50 el8-10514.queue.00000002
-rw------- 1 rsyslog rsyslog 21M Nov 12 10:50 el8-10514.queue.00000003
-rw------- 1 rsyslog rsyslog 21M Nov 12 15:10 el8-10514.queue.00000004
-rw------- 1 rsyslog rsyslog 9.3M Nov 12 15:20 el8-10514.queue.00000005
-rw------- 1 rsyslog rsyslog 579 Nov 12 15:20 el8-10514.queue.qi
Sent: Monday, November 12, 2018 3:19 PM
To: rsyslog-users
Cc: LOEWENTHAL Sophie
Subject: Re: [rsyslog] Ruleset : send to server over UDP instead of TCP
Hi,
It seems that the primary purpose of omelasticsearch is to send logs to
elesticsearch rest, running on http or https, thus using tcp. It's not
intended to use udp.
https://www.rsyslog.com/doc/v8-stable/configuration/modules/omelasticsearch.html
But I might be wrong...
Flo
On Mon, Nov 12, 2018 at 3:12 PM sophie.loewenthal--- via rsyslog <
protocol="tcp" / protocol="udp"
rsyslogd: error during parsing file /etc/rsyslog.conf, on or before line
70: parameter 'protocol' not known -- typo in config file? [v8.24.0 try
http://www.rsyslog.com/e/2207 ]

would like this

recipient(s),

please delete it and any copies from your systems and immediately notify
the sender. Any unauthorized view, use that does not comply with its

purpose,

dissemination or disclosure, either whole or partial, is prohibited.

Since the

internet
cannot guarantee the integrity of this message which may not be

reliable, BNP

PARIBAS
(and its subsidiaries) shall not be liable for the message if modified,

changed or

falsified.
Do not print this message unless it is necessary, consider the

environment.
--------------------------------------------------------------------------------------------------

--------------------------------
Ce message et toutes les pieces jointes (ci-apres le "message")
sont etablis a l'intention exclusive de ses destinataires et sont

confidentiels.

Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
merci de le detruire ainsi que toute copie de votre systeme et d'en

avertir

immediatement l'expediteur. Toute lecture non autorisee, toute

utilisation de

ce message qui n'est pas conforme a sa destination, toute diffusion ou

toute

publication, totale ou partielle, est interdite. L'Internet ne

permettant pas

d'assurer
l'integrite de ce message electronique susceptible d'alteration, BNP

Paribas

(et ses filiales) decline(nt) toute responsabilite au titre de ce

message dans

myriad of

sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you

DON'T

LIKE THAT.

sophie.loewenthal--- via rsyslog

2018-11-12 15:16:27 UTC

Permalink

Hi Flo,

Thanks, I had already changed it from tcp to udp. Now I have this, and the queue files in the queue directory have disappeared :)

*.info { action (type="omfwd"
target="el7"
port="10514"
protocol="udp"
template="ElasticSearchTemplate"
queue.spoolDirectory="/soft/rsyslog/queues"
queue.size="1024000"
queue.filename="el7-10514.queue"
queue.maxdiskspace="512m"
queue.type="FixedArray"
queue.maxfilesize="20m"
queue.saveonshutdown="on"
queue.discardseverity="6"
Action.ResumeInterval="1"
Action.ResumeRetryCount="-1"
)
}

From: Flo Rance [mailto:***@gmail.com]
Sent: Monday, November 12, 2018 4:01 PM
To: LOEWENTHAL Sophie
Cc: rsyslog-users; Rainer Gerhards
Subject: Re: [rsyslog] Ruleset : send to server over UDP instead of TCP

My bad, UDP not TCP.

action(type="omfwd" target="el8" port="10514" protocol="udp" ...)

On Mon, Nov 12, 2018 at 3:54 PM Flo Rance <***@gmail.com> wrote:
Hi,

Most probably not. The syntax for the omfwd module is different.

action(type="omfwd" target="el8" port="10514" protocol="tcp" ...)

https://www.rsyslog.com/doc/v8-stable/configuration/modules/omfwd.html

Flo

On Mon, Nov 12, 2018 at 3:39 PM ***@externe.bnpparibas.com <***@externe.bnpparibas.com> wrote:
Flo wrote:> But I might be wrong...
You are right. I had thought that the elasticsearch type just sent messages in a specific format.

Rainer wrote:> where did you place this? It's not in the config you posted. And, no,omelastisearch does not UDP (neither does Elasticsearch AFIK).
The logstash is listening on UDP, and it's just waiting for some json formatted syslog messages. This is then fed in to ESearch.

I have replaced the type() with omfwd, which I think is more appropriate. My template (not included below) is a json template that logstash expects. I've included this below. Should the files in the queues directory decrease?

*.info { action (type="omfwd"
server="el8"
serverport="10514"
protocol="udp"
searchIndex="unix"
bulkmode="on"
template="ElasticSearchTemplate"
name="el8-514-out"
queue.spoolDirectory="/soft/rsyslog/queues"
queue.size="1024000"
queue.filename="el8-10514.queue"
queue.maxdiskspace="512m"
queue.type="FixedArray"
queue.maxfilesize="20m"
queue.saveonshutdown="on"
queue.discardseverity="6"
Action.ResumeInterval="1"
Action.ResumeRetryCount="-1"
)
}

Contents of queues directory:
total 179M
-rw------- 1 rsyslog rsyslog 21M Nov 8 10:24 el7-10514.queue.00000001
-rw------- 1 rsyslog rsyslog 21M Nov 12 10:50 el7-10514.queue.00000002
-rw------- 1 rsyslog rsyslog 21M Nov 12 10:50 el7-10514.queue.00000003
-rw------- 1 rsyslog rsyslog 21M Nov 12 15:10 el7-10514.queue.00000004
-rw------- 1 rsyslog rsyslog 9.3M Nov 12 15:20 el7-10514.queue.00000005
-rw------- 1 rsyslog rsyslog 579 Nov 12 15:20 el7-10514.queue.qi
-rw------- 1 rsyslog rsyslog 21M Nov 8 10:24 el8-10514.queue.00000001
-rw------- 1 rsyslog rsyslog 21M Nov 12 10:50 el8-10514.queue.00000002
-rw------- 1 rsyslog rsyslog 21M Nov 12 10:50 el8-10514.queue.00000003
-rw------- 1 rsyslog rsyslog 21M Nov 12 15:10 el8-10514.queue.00000004
-rw------- 1 rsyslog rsyslog 9.3M Nov 12 15:20 el8-10514.queue.00000005
-rw------- 1 rsyslog rsyslog 579 Nov 12 15:20 el8-10514.queue.qi

From: Flo Rance [mailto:***@gmail.com]
Sent: Monday, November 12, 2018 3:19 PM
To: rsyslog-users
Cc: LOEWENTHAL Sophie
Subject: Re: [rsyslog] Ruleset : send to server over UDP instead of TCP

Hi,

It seems that the primary purpose of omelasticsearch is to send logs to elesticsearch rest, running on http or https, thus using tcp. It's not intended to use udp.

https://www.rsyslog.com/doc/v8-stable/configuration/modules/omelasticsearch.html

But I might be wrong...

Flo

On Mon, Nov 12, 2018 at 3:12 PM sophie.loewenthal--- via rsyslog <***@lists.adiscon.com> wrote:

I thought this could work, but nope:

protocol="tcp" / protocol="udp"

rsyslogd: error during parsing file /etc/rsyslog.conf, on or before line 70: parameter 'protocol' not known -- typo in config file? [v8.24.0 try http://www.rsyslog.com/e/2207 ]