Posts tagged file
Stdout or a Pipe and a Tee
Mar 9th
If you have a binary that is outputting information to standard output, you can save the information 2 ways. You can direct the output to a file using >> or you can use a pipe with a tee. When you use >> you will not see the output, it will directly get dumped into the file, which is why the pipe and the tee is useful. Lets say you want to monitor your your mail.log. Open it with tail -f, and watch it on the screen:
sudo tail -f /var/log/mail.log
I find it interesting sometimes to watch spam and viruses get processed by the server.
Now if you want to save this output to a file other than the default mail.log you can direct the stdout to a file using >>:
sudo tail -f /var/log/mail.log >> ~/mail_bak.log
If you want to view the tail, while you are also saving it to a backup log file, use:
sudo tail -f /var/log/mail.log | tee ~/mail_bak.log
Two options, I use them both on occasion.
Secure Remote Connections – Tech Support
Jan 23rd
If you are going to be providing technical support, to be professional you should implemented a secure ssh connection. There is no excuse to potentially give terminal access to a password snooper. Using ssh the terminal commands themselves are encrypted, so disabling password login truly minimized risks of a security breach. Using a RSA key passwords are not transmitted and the login process is truly encrypted. Disabled password login by changing its option to “no” in the sshd configuration file. The config file is located in /etc/ssh/sshd_config. The default authorized key file is located in the users home directory. ~/.ssh/authorized_keys2. You need to place the id_rsa.pub information in the authorized_keys2 file. You can open the file manually or your can cat it in. You can also replace the entire authorized_keys2 file with id_rsa.pub; but maybe you want multiple authorized keys so use cat or editing manually may be better.
Create a new RSA key with:
ssh-keygen
It will put the new id_rsa and id_rsa.pub in ~/.ssh. When logging in as a client the default location for the id_rsa is in ~/.ssh, but you can place it anywhere using the ssh -i option. Then include the path to the id_rsa key. An example is:
ssh -i /home/user/Desktop/id_rsa user@host
Server Side Spam Filter – Sendmail – Dovecot – Procmail
Dec 6th
I tried for a long time to get sieve to work with sendmail. I could not get it to work. I would try and create a custom .m4 mailer, as per the sieve pages on the dovecot website, but nothing would work. Ultimately I gave up and used client side mail filtering built into Thunderbird. But this has a big drawback. A desktop session has to be logged on to keep the graphical application running. Because of this large amounts of system resources are consumed and end up being swapped out. As I would sit down at night it would take a significant amount of time for the desktop session to become responsive as it is retreived from swap. I tried disabling swap, but this was a no no because of MythTV’s commercial flagging. It would cause problems and lead to random system instability. Even with 8 gigs of ram my virtual server would sometimes crash. I don’t want this. Now I keep swap enabled and figured out how to get procmail to do server side mail filtering.
It was easy. Simply go into your sendmail.mc file and change the mailer. The default mailers are likely:
MAILER(local)
or
MAILER(smtp)
Install procmail and change the mailer accordingly:
MAILER(procmail)
Now go into your users home directory and touch a new file:
touch .procmailrc
Open it:
nano .procmailrc
Add:
GNU nano 2.0.7 File: .procmailrc
:0:
* ^X-Spam-Flag: YES
$HOME/mail/Junk
With this particular rule all messages flagged as spam, as per spamassassin, will be deposited into the Junk folder in the mail directory. It works as expected. For more information about sendmail, and how to configure spamassassin and clamav, visit a previous post.
Piping Data and Redirection
Dec 5th
There are several commands that export information in the terminal. two that come to mind as being rather useful are:
1. tail
2. cat
Tail prints the last several lines of a file. It is extremely useful in monitor logs. Even more useful is using tail with the -f option. Using tail you can monitor a log in real time, keeping a terminal window open to see whats going on in a program. I like to keep a terminal open to monitor the mail.log associatd with sendmail, dovecot, spamassassin, and clamav. There are some useful error, and you can see clearly if local relaying is enabled or disabled. One error that threw me off for a while was dovecot quitting because NTP was stepping the time back to large a number. Basically the error message said dovecot had no idea what to do, so it simply quit.
If you want you can dump the entire tail to a file, other than the log file, in order to separate a particulars days errors messages. Use > save.txt and the output of tail will be added to a new file called save.txt in the directory from which you called the binary. There are several other commands that can be used including >> which adds the output to the text file. There are other options but I don’t see then as being useful for default file operations; the other options including storing standard error, and accecpting info from the file into standard input.
Ok, lets say you want to run tail -f, output the info to a file save.txt and also view it as normal displaying in the terminal. Use the tee command instead of the > parameter. Use:
tail -f | tee save.txt
The above code will display in the terminal the standard output of tail, and also pipe the info to the file. This is the purpose of |. It runs the binary, and connects the output to a subsequent binary.
The cat command also prints info to standard output. For example if you want to check the status of your raid arrays, in the terminal, type:
cat /proc/mdstat
Using cat like this will print a static screen. To monitor the contents of the file, like tail -f use watch.
watch cat /proc/mdstat
The binary cat is also used to merge two text files together. It will print them in standard output consecutively, therefore standard output can be recorded to effectively merge the files into one:
cat file1.txt file2.txt > merged.txt
This will keep a screen up much like top. Using ps will give you information about running processes individually, but top will continueously poll and print useful information. Ultimately top has grown into the graphical gnome system monitor.
Server Memory Setup – To Swap or Not to Swap
Oct 28th
For a while I turned off the swap file on my virtual machine. It worked, but then I began to use more php pages and the server began acting funny, especially with database queries. After a day or two of the system crashing I turned on a swap file and the system stability came back real quick. The virtual machine has 1024 megs of ram and 2048 megs of swap and the system appears to be managing memory much better. Review your system performance and status with the top binary.
As for the host, I turned the swap partition back on, but I dont think it is needed. I will try server configurations on my days off this week.
