Install server certificates for WHM services - mail, ftp and web interface

Even though this is simple task I decide to describe it. WHM / cPanel system is really complex and overlooking some settings can easily happen. That's what happened also to me. I think there are 2 reasons:
1. This task is not on daily basis - certificates are generated for longer periods - year or more years
2. Settings for SSL in WHM is not on one place and that is bit confusing.

To secure communication on WHM's important services:

- Web interface (WHM, cPanel, Webmail on SSL ports 2083, 2087, 2096)
- SMTP (Exim on port 25 - event. aditionally another)
- Mail (IMAP, POP3 -Dovecot on ports 993 and 995)
- SFTP (Pure-FTPD on port 20, 21,  event. aditionally another)

it is required to get some certificate from CA (certificate authority). There are lot of CA - you can Google it and compare their prices. Proper type of certificate is Class 2 - organizations, for which proof of identity is required. But if you are individualist or small project you can save some money here. Class 1 certificate is sufficient which is intended for securing web-mail and simillar applications like cPanel / WHM is. Some CA providing Class 1 certificates for free of charge like StarCom CA.

Specific steps:

1. Generate an SSL Certificate and Signing Request:
Go to WHM > SSL / TLS > Generate an SSL Certificate and Signing Request.
Fill all fields with your data. I recommend also filling last field to get certificate to email to have backup.

2. Sign your certificate by CA - cpecifics steps depends on CA.

3. As soon as CA sign the certificate you can install it to all services mentioned above under this navigation:
WHM > Service Configuration > Manage Service SSL Certificates

Note:

In order to configure secure communication without warnings it is also required to use proper hostname on client side for which was certificate issued. Otherwise user will see security warning - domain mismatch. Connection itself remains encrypted.

How to avoid 'repair with keycache' process in MySQL

This would be another quick post. I have encountered this problem few times and I hope that this could help other people making database recovery faster and more reliable.

If you see long running process 'repair with keycache' in MySQL 'process list' during database recovery that means your mysqld have insufficient resources to use standard method - 'repair by sorting'. Repair by Keycache is very slow - 5x - 10x slower (or even worse) depending on your setup, hardware performance and database contents.

You have to change you configuration following way:

1. Your setting 'tmpdir' in my.cnf points to file-system with low space storage. Be sure that you have at least 2x or more availabe space than database backup alone (in some scenarios maybe more). On some systems, administrators creating separate file-system for /tmp dir, which is good idea, but mostly it's very small, therefore insufficient for large database recovery. In this case just change your setting for example to /var/tmp/mysql

2. Check permissions on file-system of 'tmpdir'. Usually mysqld running under 'mysql' user. This user need to have r/w permissions to that directory.

3. Another variables need to be checked:
myisam_max_sort_file_size (several times larger than biggest table inside database)
myisam_sort_buffer_size (try 64MB or more)

I'm not explaining everything. I'm just trying to advice which things need to be tweaked.
Any comments appreciated.

Restoring single table from full MySQL dump file

That would be quick post. To restore single table from full MySQL backup it is required to filter all other sql commands which is not belonging to certain table.

 # sed -n -e '/CREATE TABLE.*Tablename/,/UNLOCK TABLES/p' full_backup.sql > table_recovery.sql

This command read content of  'full_backup.sql' and search for line containing "CREATE TABLE" followed with specified table name (just replace it with desired name). From this line starts sending to standard output. Then search for line "UNLOCK TABLES" - on this line stops sending contents. At the end of this command is redirection to file ">"

It is better to create intermediate file to be sure, that the filtering working as you expected, because this is based on sql commands. After "CREATE TABLE" statements comes pair of statements "LOCK TABLES" and "UNLOCK TABLES" which are used in 99% if not in 100%. You don't need to create intermediate file and you are sure you can pipe it directly to MySQL:

 # sed -n -e '/CREATE TABLE.*Tablename/,/UNLOCK TABLES/p' full_backup.sql | mysql -u user -p dbname

Force database name on import MySQL even if CREATE DATABASE and USE statements are specified

Many advices over Internet are:
You have to preform proper backup first with right options like --no-create-db. Otherwise dump file contains SQL commands which says to server 'create database with this name' and 'use that database'. That's super advice, damn, I wouldn't know that without you! What if you need to use dump which is already done before time ago (already containing these lines) and you can't create new backup with new options because the data you need are only in that old backup.

Importing options in MySQL doesn't support ignore or something similar to avoid these SQL commands. It is necessary to alter or remove these lines containing mentioned commands.

If the database dump file is small, you can edit it by hand, find lines 'CREATE DATABASE' and 'USE', then replace with desired name or even remove them.

If the file is big, then it's not so easy to edit file by hand. Opening file bigger than available free operating memory will cause problems. You can do it by using sed from command line:

 #  sed '/^CREATE DATABASE\|^USE/s/old-name/new-name/' original.sql > new.sql

This command use original.sql as input, will find lines beginning with CREATE DATABASE or USE and alter old-name to new-name and all together goes to new.sql file.

Checkings:
Check content of new.sql if the names are properly set.:

 #  grep '^CREATE DATABASE\|^USE' new.sql
 CREATE DATABASE /*!32312 IF NOT EXISTS*/ `new-name` /*!40100 DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci */;
 USE `new-name`;

Some time-saving hints:
If you will be importing database anyway, you can save some time by piping output directly to mysql instead of creating new file and then importing in to database. And as it was in my case, I need to recover database from gzipped package. But I wanted to keep backup file untouched and do not create intermediate file. Then the command will be like this:

 #  gunzip < /path/to/backup-file.sql.gz | sed '/^CREATE DATABASE\|^USE/s/old-name/new-name/' | mysql -u user -p

This commands read backup-file.sql.gz which goes to gunzip to decompress the content then on the fly piping to sed which I described above and one more pipe directly to mysql to import the database. With this command you need to be careful - you need to be sure that you are not going to rewrite existing data in database if you want to keep them. It is better to use MySQL user with limited permissions to run this command safely.

How to remove ads from notification area on Android

Most of Android users already experienced this issue. Each day annoying ads appears in notification bar and you don't have clear way how to get rid of it. Some of applications doesn't ask for permissions and target of developers of these applications is to distribute ads without users permissions. Why? Because these applications are for free and developers wants to earn money somehow. Google Ads doesn't provide such rates like pushing notifications ads.

Where they come from?

I googled out that behind this stand 2 companies - AirPush and SlingLabs, providing developers ability to push messages to notification area.

Investigating each application before installation simply takes too much time and the beauty of Android is its versatility thanks to wide spectrum of applications. Users want to try and use many applications.

Now what to do. Solution for advanced users with rooted devices is bit tricky and it also provides more options. But I want to find easy way for common users.

Solution is relatively easy:

For AirPush visit optout page where you can download simple application which reads device identifications nombers (IMEI...) and send them to system to remove your phone from their database. Or in Google Play is another application called "AirPush Detector"

For SlingLabs has optout page too. But currently not working. I hope they will fix it soon.

SVN server with SASL on Debian

Reason

I thought that I will not have to write this post, but i took much time. I just hope it will help other people  to save some time. I stuck for few hours by preparing of this and at the end I discovered that's not possible what I exactly want - encrypt password when they are sent over network. About this at bottom of this article. Now how to get SASL working:

Configuration:

Assuming that you have already SVN installed on your system.
This configuration could be applied also for Debian derivates like Ubuntu.

 # apt-get install libsasl2-2 libsasl2-modules sasl2-bin 

Enable SASL:

 # vim /etc/default/saslauthd
 START=yes

Create new repository:

 # cd /var/svn
 # svnadmin create myrepo
 # vim myrepo/conf/svnserve.conf
        # insert configuration of repository
        [general]
        anon-access = none #we don't want to allow public access
        auth-access = write #only authorized users
        realm = realmname
        [sasl]
        use-sasl = true
        min-encryption = 256
        max-encryption = 256

Configure SASL with basic settings:

 # vim /usr/lib/sasl2/svn.conf
        pwcheck_method: auxprop
        auxprop_plugin: sasldb
        sasldb_path: /etc/svn/sasldb
        mech_list: DIGEST-MD5

Create users using saslpasswd2 tool (issue command as many times as much users you need):

 # saslpasswd2 -f /etc/svn/sasldb -c -u realmname username

Start SASL daemon and restart svnserve:

 # /etc/init.d/svnserve restart # /etc/init.d/saslauthd start

Common problems:

When connecting to SVN server client respond this error message

  svn: Could not obtain the list of SASL mechanisms  

Most common reason of this is missing libsasl2. To resolve this problem you must have this library installed on both system - client and server too. If you are installing SVN from sources then do not forget to:

 # ./configure –with-sasl

Another issue which I mentioned at the beginning - keep encrypted password when they are sent over network. This simply not possible in this configuration. This is known issue because svn:// protocol doesn't support TLS yet. More on this: https://svn.apache.org/repos/asf/subversion/trunk/notes/sasl.txt - section 7. To keep password secure you will have to add another layer between svnserve and its clients like VPN or tunnel - for example stunnel (http://www.stunnel.org/index.html). SASL covers only password storage encryption. Another option is deploy ssh tunnel which is most easiest way. Impressive choice is apache + dav + svn + ssl modules which is more complex. But in both cases (ssh or apache) users will be not controlled by svnserve. For ssh you will have to create system users, custom file permissions and for apache are many options.

Here are sources which used:

http://serverfault.com/questions/226586/subversion-1-6-sasl-only-works-with-plaintext-userpassword
http://www.dm9.se/?p=518
https://svn.apache.org/repos/asf/subversion/trunk/notes/sasl.txt