[rsyslog] root .bash_history

[Josh Bitto]

I'm wanting to get an opinion. Would it be a smart play to monitor /root/.bash_history and log the file to a remote server?

Joshua Bitto
Information Technologist
KCC

[Champ Clark III]

Bash has a built in "history to syslog" patch. I think it's standard
now. That'd be a better way to archive the commands.

Post by Josh Bitto
I'm wanting to get an opinion. Would it be a smart play to monitor /root/.bash_history and log the

file to a remote server?

Post by Josh Bitto
Joshua Bitto
Information Technologist
KCC
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a

myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
if you DON'T LIKE THAT.


- --
- - Quadrant Information Security
Champ Clark III
o: 800.538.9357 x 101
c: 850.443.2440

[Josh Bitto]

I did some searching with google and can't find any adequate information on it. Do you happen to know where I should be looking? I'm using centos 6.4 w/ bash 4.1



-----Original Message-----
From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Champ Clark III
Sent: Monday, April 08, 2013 10:37 AM
To: rsyslog at lists.adiscon.com
Subject: Re: [rsyslog] root .bash_history


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Bash has a built in "history to syslog" patch. I think it's standard now. That'd be a better way to archive the commands.

Post by Josh Bitto
I'm wanting to get an opinion. Would it be a smart play to monitor
/root/.bash_history and log the

file to a remote server?

Post by Josh Bitto
Joshua Bitto
Information Technologist
KCC
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a

myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.


- --
- - Quadrant Information Security
Champ Clark III
o: 800.538.9357 x 101
c: 850.443.2440

[Philippe Muller]

it should be built with SYSLOG_HISTORY defined. I guess you can do it with
"make -DSYSLOG_HISTORY=1".

Philippe Muller

Post by Josh Bitto
I did some searching with google and can't find any adequate information
on it. Do you happen to know where I should be looking? I'm using centos
6.4 w/ bash 4.1
-----Original Message-----
rsyslog-bounces at lists.adiscon.com] On Behalf Of Champ Clark III
Sent: Monday, April 08, 2013 10:37 AM
To: rsyslog at lists.adiscon.com
Subject: Re: [rsyslog] root .bash_history
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Bash has a built in "history to syslog" patch. I think it's standard now.
That'd be a better way to archive the commands.

Post by Josh Bitto
I'm wanting to get an opinion. Would it be a smart play to monitor
/root/.bash_history and log the

file to a remote server?

Post by Josh Bitto
Joshua Bitto
Information Technologist
KCC
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a

myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
you DON'T LIKE THAT.
- --
- - Quadrant Information Security
Champ Clark III
o: 800.538.9357 x 101
c: 850.443.2440
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQEcBAEBAgAGBQJRYwA0AAoJENnmXt7Lmc3K3k8H/Re3NndMauH64QlUvhU2tMuy
tYrdSFLdzKpnWcLDyWLivcKxX5hhSdYk6IplAWHU5HaTbu22pzeN6eICgd6sv7EM
UXBujutxEMPsoLdTQxCA6SF94cikvgXzSfj+An7lWVjVjhhfk2x1JNd/detZYO5x
8OAsa+gZtqTic2YSL1yiFOL+ZUO97nKHGCANKe1MDfAaz0FOQsmgw+59lzvY2j/D
T/rURyvFQHZ2eQ/A3Hb9wq5KWFisdkZq6r8ebMFY1Aucy+7yogR/4Gx/tkKv8lMM
37E9ujs2CaxjXk4b4S3eMom+CGi3I7ZQu6VcxEuweuGM24quj+1l0bgikmR0+xY=
=y/IH
-----END PGP SIGNATURE-----
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites
beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.

[Josh Bitto]

I just found this....I tested it out and it works for me. So now I just gonna add this as a monitored file and forward the file to my central syslog server.

http://linuxmycommand.blogspot.ca/2012/06/how-to-log-all-bash-commands-by-all.html



-----Original Message-----
From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Philippe Muller
Sent: Monday, April 08, 2013 11:15 AM
To: rsyslog-users
Subject: Re: [rsyslog] root .bash_history

it should be built with SYSLOG_HISTORY defined. I guess you can do it with "make -DSYSLOG_HISTORY=1".

Philippe Muller

Post by Josh Bitto
I did some searching with google and can't find any adequate
information on it. Do you happen to know where I should be looking?
I'm using centos
6.4 w/ bash 4.1
-----Original Message-----
rsyslog-bounces at lists.adiscon.com] On Behalf Of Champ Clark III
Sent: Monday, April 08, 2013 10:37 AM
To: rsyslog at lists.adiscon.com
Subject: Re: [rsyslog] root .bash_history
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Bash has a built in "history to syslog" patch. I think it's standard now.
That'd be a better way to archive the commands.

Post by Josh Bitto
I'm wanting to get an opinion. Would it be a smart play to monitor
/root/.bash_history and log the

file to a remote server?

Post by Josh Bitto
Joshua Bitto
Information Technologist
KCC
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a

myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
if you DON'T LIKE THAT.
- --
- - Quadrant Information Security
Champ Clark III
o: 800.538.9357 x 101
c: 850.443.2440
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQEcBAEBAgAGBQJRYwA0AAoJENnmXt7Lmc3K3k8H/Re3NndMauH64QlUvhU2tMuy
tYrdSFLdzKpnWcLDyWLivcKxX5hhSdYk6IplAWHU5HaTbu22pzeN6eICgd6sv7EM
UXBujutxEMPsoLdTQxCA6SF94cikvgXzSfj+An7lWVjVjhhfk2x1JNd/detZYO5x
8OAsa+gZtqTic2YSL1yiFOL+ZUO97nKHGCANKe1MDfAaz0FOQsmgw+59lzvY2j/D
T/rURyvFQHZ2eQ/A3Hb9wq5KWFisdkZq6r8ebMFY1Aucy+7yogR/4Gx/tkKv8lMM
37E9ujs2CaxjXk4b4S3eMom+CGi3I7ZQu6VcxEuweuGM24quj+1l0bgikmR0+xY=
=y/IH
-----END PGP SIGNATURE-----
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites
beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.

[Igor Sverkos]

Hi,

Post by Josh Bitto
I'm wanting to get an opinion. Would it be a smart play to monitor
/root/.bash_history and log the file to a remote server?

When you come to the conclusion that you want to log the shell history,
please check first for any privacy issues:

- Are you allowed to do that?

- Are your users informed?

- How do you deal with scenarios like someone's sensitive data you are
definitely not allowed to log and store become part of the history? Are
you prepared to remove these data?

- If you want a logged shell history for security, be aware that there
maybe ways to bypass the log (maybe user can turn it off; maybe user can
user sftp to spawn another shell which doesn't log...)

[Josh Bitto]

When you come to the conclusion that you want to log the shell history, please check first for any privacy issues:

- Are you allowed to do that?

(yes I'm allowed)

- Are your users informed?

(the main purpose for it is for server access...Only admins have access to it.)

- How do you deal with scenarios like someone's sensitive data you are definitely not allowed to log and store become part of the history? Are you prepared to remove these data?

(What sensitive data are you inferring to? It logs command line input.)

- If you want a logged shell history for security, be aware that there maybe ways to bypass the log (maybe user can turn it off; maybe user can user sftp to spawn another shell which doesn't log...)

(This wouldn't be a complete solution only a layer. There are other steps you must take in hardening a server from intrusion. I just want to include this.)

[Igor Sverkos]

Hi,

Post by Josh Bitto

Post by Igor Sverkos
How do you deal with scenarios like someone's sensitive data you
are definitely not allowed to log and store become part of the
history? Are you prepared to remove these data?

What sensitive data are you inferring to? It logs command line
input.

Right. An application which supports logging will log prepared data
(=chances are high, that sensitive data are removed/masked). A command
line gets unfiltered raw input.

For example you can connect to your mysqld via

# mysql -h foo -u myuser -p

and you will be prompted for myuser's password. But you can also pass
the password to the command:

# mysql -h foo -u myuser -pmysecretpasswordisnowinthelogs

Now your mysql password for the user "myuser" is in the logs.

Maybe that's not a problem at first view, but people tend to keep there
logs unprotected, at least less protected. So when someone get access to
your logs (you compressed your log files, put the archive in your htdocs
folder to grab it from another machine and your forget to remove..., now
somebody found the file), you may have more problems like when you did
not have logged the command.

Please, don't get me wrong. I don't say you should not log shell
histories. You just asked for opinions and I want to share some issues I
think you should be aware of when you want to do that. That's all :)

[Josh Bitto]

I greatly appreciate the insight. The only way for that to log is like you said, but if you make a practice of just doing mysql -u user -p........Then when prompted for password.....it doesn't log.



-----Original Message-----
From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Igor Sverkos
Sent: Tuesday, April 09, 2013 10:41 AM
To: rsyslog at lists.adiscon.com
Subject: Re: [rsyslog] root .bash_history

Hi,

Post by Josh Bitto

Post by Igor Sverkos
How do you deal with scenarios like someone's sensitive data you are
definitely not allowed to log and store become part of the history?
Are you prepared to remove these data?

What sensitive data are you inferring to? It logs command line input.

Right. An application which supports logging will log prepared data (=chances are high, that sensitive data are removed/masked). A command line gets unfiltered raw input.

For example you can connect to your mysqld via

# mysql -h foo -u myuser -p

and you will be prompted for myuser's password. But you can also pass the password to the command:

# mysql -h foo -u myuser -pmysecretpasswordisnowinthelogs

Now your mysql password for the user "myuser" is in the logs.

Maybe that's not a problem at first view, but people tend to keep there logs unprotected, at least less protected. So when someone get access to your logs (you compressed your log files, put the archive in your htdocs folder to grab it from another machine and your forget to remove..., now somebody found the file), you may have more problems like when you did not have logged the command.

Please, don't get me wrong. I don't say you should not log shell histories. You just asked for opinions and I want to share some issues I think you should be aware of when you want to do that. That's all :)


--
Regards,
Igor

[David Lang]

Post by Igor Sverkos

Post by Josh Bitto

Post by Igor Sverkos
How do you deal with scenarios like someone's sensitive data you
are definitely not allowed to log and store become part of the
history? Are you prepared to remove these data?

What sensitive data are you inferring to? It logs command line
input.

Right. An application which supports logging will log prepared data
(=chances are high, that sensitive data are removed/masked). A command
line gets unfiltered raw input.
For example you can connect to your mysqld via
# mysql -h foo -u myuser -p
and you will be prompted for myuser's password. But you can also pass
# mysql -h foo -u myuser -pmysecretpasswordisnowinthelogs
Now your mysql password for the user "myuser" is in the logs.
Maybe that's not a problem at first view, but people tend to keep there
logs unprotected, at least less protected. So when someone get access to
your logs (you compressed your log files, put the archive in your htdocs
folder to grab it from another machine and your forget to remove..., now
somebody found the file), you may have more problems like when you did
not have logged the command.
Please, don't get me wrong. I don't say you should not log shell
histories. You just asked for opinions and I want to share some issues I
think you should be aware of when you want to do that. That's all :)

This is a good point, but you are missing the fact that you are already logging
passwords.

You are logging failed login attempts, right?

I guarantee you that at some point a user will get out of sync with the login
prompt and type their password into the userid field, and therefor you will have
that user's password in the logs (usually followed almost immediatly by the
userid as the user realizes their mistake and logs in correctly)

So you really need to be protecting your log data and/or implement something
better than simple password authentication.

David Lang

[Josh Bitto]

The reason this works for me is not because of the scenario's you have outlined, but because command line interaction with production servers are only limited to admins (3 people). Where I'm coming from is more of an audit trail. I want to know (if by some miracle) that if a server is broken into I can see what commands were put in and what was done. That's it....I do see the points of view on it. If I had regular users that needed access to the command line or what not, then yeah I could see that being an issue.


----------------------------------------------------------------
This is a good point, but you are missing the fact that you are already logging passwords.

You are logging failed login attempts, right?

I guarantee you that at some point a user will get out of sync with the login prompt and type their password into the userid field, and therefor you will have that user's password in the logs (usually followed almost immediatly by the userid as the user realizes their mistake and logs in correctly)

So you really need to be protecting your log data and/or implement something better than simple password authentication.

David Lang

[David Lang]

Actually, your protection is that the only people with access to the logs are
your three admins, they are protected from everyone else :-)

But I was really explining this for the other poster because if he is thinking
that adding the bash history to the logs creates this sort of risk, he's missed
the possibility I outlined below.

David Lang

Post by Josh Bitto
The reason this works for me is not because of the scenario's you have outlined, but because command line interaction with production servers are only limited to admins (3 people). Where I'm coming from is more of an audit trail. I want to know (if by some miracle) that if a server is broken into I can see what commands were put in and what was done. That's it....I do see the points of view on it. If I had regular users that needed access to the command line or what not, then yeah I could see that being an issue.
----------------------------------------------------------------
This is a good point, but you are missing the fact that you are already logging passwords.
You are logging failed login attempts, right?
I guarantee you that at some point a user will get out of sync with the login prompt and type their password into the userid field, and therefor you will have that user's password in the logs (usually followed almost immediatly by the userid as the user realizes their mistake and logs in correctly)
So you really need to be protecting your log data and/or implement something better than simple password authentication.
David Lang

[Igor Sverkos]

Hi,

Post by Josh Bitto
The reason this works for me is not because of the scenario's you
have outlined, but because command line interaction with production
servers are only limited to admins (3 people).

And that was your first mistake. Administrators are smart? They don't
make errors like normal users do? ;)

Every log archive you will find on the net was made available (and
forgotten) by someone with administrator privileges... remember that.

Post by Josh Bitto
Where I'm coming from is more of an audit trail. [...]

Well, then I hope you are aware of what kind of sensitive data could be
in an audit log and that you have to take care.

I don't need to mention, this wasn't part of your question, but I want
to write it down for completion, if we stick to the mysql example, that
you could also log the mysql client history (similar to the shell
history). So if one of your three smart administrators set/changes a
password, the password is stored encrypted by mysqld itself, but I hope
they are aware of the fact, the the plain unencrypted password can be
found in the log. That's why mysql recommend to disable logging per
session, when doing such things... but you already know that. right?
Fine. :-)

I completed my mission. I just wanted to point out, that you also have
to think about log protection.

[Josh Bitto]

Igor,

Thank you for all your insight. I have taken those suggestions already into consideration before starting this topic. Mysql history has already been taken care of ;)
Anywho...I think it would still be a good practice audit/analysis technique to do this type of logging.


Best Regards,

Josh Bitto

---------------------------------------------------------------------------------------


Hi,

Post by Josh Bitto
The reason this works for me is not because of the scenario's you have
outlined, but because command line interaction with production servers
are only limited to admins (3 people).

And that was your first mistake. Administrators are smart? They don't make errors like normal users do? ;)

Every log archive you will find on the net was made available (and
forgotten) by someone with administrator privileges... remember that.

Post by Josh Bitto
Where I'm coming from is more of an audit trail. [...]

Well, then I hope you are aware of what kind of sensitive data could be in an audit log and that you have to take care.

I don't need to mention, this wasn't part of your question, but I want to write it down for completion, if we stick to the mysql example, that you could also log the mysql client history (similar to the shell history). So if one of your three smart administrators set/changes a password, the password is stored encrypted by mysqld itself, but I hope they are aware of the fact, the the plain unencrypted password can be found in the log. That's why mysql recommend to disable logging per session, when doing such things... but you already know that. right?
Fine. :-)

I completed my mission. I just wanted to point out, that you also have to think about log protection.


--
Regards,
Igor

[Champ Clark III]

Post by Josh Bitto
Igor,
Thank you for all your insight. I have taken those suggestions already

into consideration before starting this topic. Mysql history has already
been taken care of ;)

Post by Josh Bitto
Anywho...I think it would still be a good practice audit/analysis

technique to do this type of logging.
We do this in production environments as well (full syslog command line
logging). We protect the logs (ie - "root" readable only) and send
syslog off to a centralized syslog server. We do this not only for
auditing purposes, but so we can also analyze the syslog/commands line
via Sagan (plug, I'm the author of Sagan -
http://sagan.quadrantsec.com). For example, we don't expect our
admin's to compile code on production system. So, someone attempting
to execute gcc/make/etc, might raise a flag. Or perhaps someone trying
to set the command line history to /dev/null.... or execution of nmap..
might raise a alert... Things a normal "admin" wouldn't be doing on our
production systems.

See https://github.com/beave/sagan-rules/blob/master/bash.rules for some
examples.

Is it appropriate everywhere? Probably not and there are limitation.
You just have to make that call per/environment basis.

- --
- - Quadrant Information Security
Champ Clark III
o: 800.538.9357 x 101
c: 850.443.2440

[Igor Sverkos]

Hi,

Post by David Lang
This is a good point, but you are missing the fact that you are already
logging passwords.
You are logging failed login attempts, right?

We are logging login attempts, but we don't log the used credentials
(only the account name). So we see things like

Post by David Lang
Jan 13 00:19:09 ws337 sshd[6972]: SSH: Server;Ltype: Authname;Remote: 221.174.50.141-57911;Name: ts2 [preauth]
Jan 13 00:19:09 ws337 sshd[6972]: Invalid user ts2 from 221.174.50.141
Jan 13 00:19:09 ws337 sshd[6972]: input_userauth_request: invalid user ts2 [preauth]
Jan 13 00:19:09 ws337 sshd[6972]: Received disconnect from 221.174.50.141: 11: Bye Bye [preauth]

in logs, but we don't know what password 221.174.50.141 for user ts2 tried.

Are you really logging the used full credentials from failed logins?

Post by David Lang
So you really need to be protecting your log data and/or implement
something better than simple password authentication.

Exactly, that's the point! To be honest, I don't know any application,
which logging mechanism will log full credentials. You can hack and
modify them, to do that, but this is not the default. So normal logs
aren't at the same risk in my opinion.

But when you log such data, you should take care... no question.

[Rainer Gerhards]

Post by Igor Sverkos
Hi,

Post by David Lang
This is a good point, but you are missing the fact that you are already
logging passwords.
You are logging failed login attempts, right?

We are logging login attempts, but we don't log the used credentials
(only the account name). So we see things like

David's point was subtle, you should re-read his mail very carefully. In
short, he said users sometimes mistype the password when the account is
asked for (so it gets logged) and then immediately correct it (So that
you can guess the account name).

Rainer

Post by Igor Sverkos

Post by David Lang
Jan 13 00:19:09 ws337 sshd[6972]: SSH: Server;Ltype: Authname;Remote: 221.174.50.141-57911;Name: ts2 [preauth]
Jan 13 00:19:09 ws337 sshd[6972]: Invalid user ts2 from 221.174.50.141
Jan 13 00:19:09 ws337 sshd[6972]: input_userauth_request: invalid user ts2 [preauth]
Jan 13 00:19:09 ws337 sshd[6972]: Received disconnect from 221.174.50.141: 11: Bye Bye [preauth]

in logs, but we don't know what password 221.174.50.141 for user ts2 tried.
Are you really logging the used full credentials from failed logins?

Post by David Lang
So you really need to be protecting your log data and/or implement
something better than simple password authentication.

Exactly, that's the point! To be honest, I don't know any application,
which logging mechanism will log full credentials. You can hack and
modify them, to do that, but this is not the default. So normal logs
aren't at the same risk in my opinion.
But when you log such data, you should take care... no question.

[Miloslav Trmač]

Hello,
I'll just randomly hijack the thread to note the existence of tty_audit in Linux; see pam_tty_audit(8). This logs all input, not only bash commands.

(And yes, currently it will log even passwords; some work is happening upstream to make that optional.)
Mirek

[David Lang]

Post by Rainer Gerhards

Post by Igor Sverkos
Hi,

Post by David Lang
This is a good point, but you are missing the fact that you are already
logging passwords.
You are logging failed login attempts, right?

We are logging login attempts, but we don't log the used credentials
(only the account name). So we see things like

David's point was subtle, you should re-read his mail very carefully. In
short, he said users sometimes mistype the password when the account is
asked for (so it gets logged) and then immediately correct it (So that
you can guess the account name).

Exactly, if you just log the username that they try to login as, someone will
type the userid into the password field and the password into the username
field.

I find this especially common with GUI logins, the user goes to type their
userid and password, but end up in the wrong box and reverse them

But even with command line logins, I've seen it happen.

So, if you are logging the userid of failed logins, at some point you will
record someone's password instead.

David Lang