Friday, 3 February 2012

Linux - Installing and configuring Proftpd with SFTP (mod_sftp)

I recently upgraded Proftpd on a CentOS box and added mod_sftp as a static module... This post will explain what the hell I have just said, and also, how to set it setup and running. I will include gotcha's (plenty!) and some general helpful advice along the way.

We will be building from source in this guide, so I recommend grabbing that one from the download site. Download Proftpd from their website Here. Take ProFTPD 1.3.3rc1 or later as this includes mod_sftp.


Prerequisites

1. You will need to install a copy of the OpenSSL source, if you haven't already. If you have it installed, check it's in your PATH. When compiling Proftpd, it requires some header files from the OpenSSL-dev release, so this is a must!

You can download OpenSSL from Here. Grab the latest non-beta if its going to go into production. That's the rule!

Here is a guide on installing OpenSSL



2. You also need the dev libraries for zlib. These can be found Here. Install them using the supplied ReadMe file. It will be something like this...

Code Snippet
  1. ./configure
  2. make
  3. make install
End of Code Snippet


Things to note


Modules

There are some things to note before we get onto the configuration phase of Proftpd... Proftpd has numerous modules, each which aide different aspects of the FTP server. FOr example, for FTPS we use the mod_tls module.... for SFTP, we iuse mod_sftp module... To integrate FTP user accounts with a backend mysql database, we use mod_mysql and mod_sql_mysql. etc....

Note: Some modules are not compiled by default! mod_sftp being one of them...

While there are a lot of modules included as standard, you can download other modules Here


Static Modules vs. DSO's (Dynamic Shared Objects)

Modules can either be statuc or in DSO form. A static module basically means you are compiling it into the Proftpd compilation. On the other hand, a DSO Will be a seperate physical file of the library that is loaded in dynamically. DSO's allow you to drop modules in quickly- without re-compiling Prtoftpd each time.

Read more on DSO's in Proftpd Here


Installation

1. First we need to configure the installation of Proftpd. This is where you will need to decide

- How you want your modules to be represented (Static or DSO)
- What modules you actually want to include with the installation

I will be including mod_sftp in this installation, and im going to make it static.


2. Browse to the directory where the Proftpd source is, and execute the following command... Check the file paths beforehand though!

Code Snippet
  1. ./configure --prefix=/usr --with-includes=/usr/local/ssl/include --with-libraries=/usr/local/ssl/lib --enable-openssl --with-modules=mod_sftp --enable-dso
End of Code Snippet


--prefix
Where Proftpd will be installed to. I'm using CentOS as a root user, so it will be placed in /usr/sbin

--with-includes
You need to specify the include directory for the dev installation of OpenSSL. This include directory will include the development library headers which Proftpd requires to build in mod_sftp support. You can add more paths to this parameter as you wish, seperated by a colon.

--with-libraries
Similar to above but these will point to the OpenSSL libraries.

--enable-openssl
Required to enable OpenSSL for mod_sftp

--with-modules=mod_sftp
Include the sftp module. You can specify more by adding a colon after each module name.

--enable-dso
This is for DSO modules only, but I have included it as I already had some modules built with a previous version, and wanted to include them.

Note: If you wish to use DSO's (Shared Modules)... then replace --with-modules with --with-shared and ensure --enable-dso is included. Simple!


More info on compiling can be found Here


3. Produce the Makefile
Code Snippet
  1. make
End of Code Snippet

4. Install
Code Snippet
  1. make install
End of Code Snippet


Installation - Troubleshooting

In file included from mod_sftp.c:30:
mod_sftp.h:83:18: error: zlib.h: No such file or directory
make: *** [mod_sftp.lo] Error 1


You didn't install the zlib-dev development libraries. See prerequisites above!


/usr/bin/ld: cannot find -lcrypto
collect2: ld returned 1 exit status
make: *** [shared] Error 1


or

/usr/bin/ld: openssl/crypto.h: No such file or directory
make[1]: *** [support.o] Error 1
make[1]: Leaving directory `/src/proftpd-1.3.2b/src'
make: *** [src] Error 2



You didn't install the OpenSSL development libraries. See prerequisites above! Ensure the libraries AND headers are present.


Configuration

1. Check proftpd has registered itself correctly and that your static modules have been installed. Execute the following command...

Code Snippet
  1. proftpd -l
End of Code Snippet

mod_sftp should be included in the list here.


2. Check the correct version has been installed and verify the location of the proftpd configuration file. Execute the following command...

Code Snippet
  1. proftpd -V
End of Code Snippet


The version should match (Just incase an upgrade went wrong!) and you can see which configuration file Proftpd is using here... It should reside within the /etc directory of the --prefix specified when configuration the installation. I.e. /usr/etc/proftpd.conf


3. Open proftp.conf with your favourite text editor. The config file has the same syntax as Apache server configs, so it's easier to get to grips with.

View sample configurations Here

If you take a look at the Complex Virtual example, you can see how different VirtualHosts are configured.

Note: The Global tag contains settings that wiull be included across all VirtualHost's within the configuration file. Therefore, there is only ever one instance of the Global tag.

4. Take a copy of this file, and setup your Virtual hosts based on this format. You can even use the simple config file to start with. However, it is useful to understand what is possible here.

5. Save your proftpd.conf file and restart the proftpd service.
See this link for restarting services on various distros of Linux: http://theos.in/desktop-linux/tip-that-matters/how-do-i-restart-linux-network-service/

This ensures the latest config is read and is compatible before we continue.

[root@servername proftpd]# service proftpd restart
Shutting down proftpd: [ OK ]
Starting proftpd: [ OK ]



6. Keep the proftpd.conf file open and add the following for SFTP support.

Code Snippet
  1. <IfModule mod_sftp.c>
  2.         <VirtualHost xx.xx.xx.xx>
  3.       SFTPEngine on
  4.       SFTPLog /etc/proftpd/sftp/sftp.log
  5.  
  6.       # Configure the server to listen on port
  7.       Port 2222
  8.  
  9.       # Configure both the RSA and DSA host keys, using the same host key
  10.       # files that OpenSSH uses.
  11.       SFTPHostKey /etc/ssh/ssh_host_rsa_key
  12.       SFTPHostKey /etc/ssh/ssh_host_dsa_key
  13.  
  14.  
  15.       # Configure the file used for comparing authorized public keys
  16.       SFTPAuthorizedUserKeys file:/root/.ssh/authorized_keys
  17.  
  18.       # Enable compression
  19.       SFTPCompression delayed
  20.  
  21.       # Allow the same number of authentication attempts as OpenSSH.
  22.       #
  23.       # It is recommended that you explicitly configure MaxLoginAttempts
  24.       # for your SSH2/SFTP instance to be higher than the normal
  25.       # MaxLoginAttempts value for FTP, as there are more ways to authenticate
  26.       # using SSH2.
  27.       MaxLoginAttempts 6
  28.  
  29.         </VirtualHost>
End of Code Snippet


Note: Replace xx.xx.xx.xx with IP or hostname on which to accept incoming connections.

Note: Set the port for SFTP. If you have OpenSSH installed, you will either need to change the port OpenSSH uses, or change the port within this VirtualHost. I have opted for the latter, and use port 2222 for SFTP connections. You can't have two things on one port! Check what is running on certain ports using netstat.

Code Snippet
  1. netstat -tulpn
End of Code Snippet


Note: The SFTPAuthorizedUserKeys attribute points to an authorized_keys file or public key authentication. Huh?? See my guide here on what it is and how to get setup

Note: You will notice the VirtualHost is wrapped within an IfModule tag. This is a condition that will granted true if the mod_sftp loaded correctly.


7. Restart the Proftpd service and test the connectivity using an FTP client.
Code Snippet
  1. lftp sftp://user@server:port    (I.e. lftp sftp://sean@server:2222)
  2.  
  3. or
  4.  
  5. sftp user@server:port    (I.e. sftp sean@server:2222)
End of Code Snippet


You can also use FileZilla by setting the connectivity type to SFTP and specifying your configured port (I.e. 2222).


OPTIONAL: Module Configuration (DSO Configuration Only)

If you would like to include DSO's into the proftpd configuration, then open the Proftpd.conf file and add the following line...

Code Snippet
  1. Include                         /path/to/modules.conf
End of Code Snippet

This path should point to an empty file called modules.conf. You can create this using your favourite text editor.

Now open up your newly created modules.conf file and add the following...

Code Snippet
  1. # This is the directory where DSO modules reside
  2. ModulePath /usr/libexec/proftpd
  3.  
  4. # Allow only user root to load and unload modules, but allow everyone
  5. # to see which modules have been loaded
  6. ModuleControlsACLs insmod,rmmod allow user root
  7. ModuleControlsACLs lsmod allow user *
  8.  
  9. # Load the modules
  10. LoadModule MODULENAMEHERE1.c
  11. LoadModule MODULENAMEHERE2.c
  12. LoadModule MODULENAMEHERE3.c
End of Code Snippet


The directory you specified within the ModulePath attribute should point to the directory where your DSO's reside. Now replace MODULENAMEHERE with the name of your modules.
Once you have done this, you may need to restart the Proftpd service for the changes to take effect.



Useful features and Debugging help


Proftpd Debug mode

You can place Proftpd in debugging mode using the following command...
Code Snippet
  1. proftpd -nd10
End of Code Snippet


Please note, that you will have to stop the Proftpd service first, or you will get an 'address in use' error. Makes sense!

How to stop a service: http://theos.in/desktop-linux/tip-that-matters/how-do-i-restart-linux-network-service/

This is level 10 debugging and will show you connecting clients and all activity within the console output. Very useful to troubleshoot connectivity issues and auth.

Full documentation on debugging Here


Trace Logging
The mod_sftp module supports different forms of logging. The main module logging is done via the SFTPLog directive. For debugging purposes, the module also uses trace logging, via the module-specific "scp", "sftp", and "ssh2" log channels. Thus for trace logging, to aid in debugging, you would use the following in your proftpd.conf:

Code Snippet
  1. TraceLog /path/to/sftp-trace.log
  2. Trace scp:20 sftp:20 ssh2:20
End of Code Snippet


This trace logging can generate large files; it is intended for debugging use only, and should be removed from any production configuration.

mod_sftp Documentation Here


Logging

See the official guide on how logging can help you debug Proftpd. Logging documentation can be viewed Here

Where is proftpd?
Code Snippet
  1. which -a proftpd
End of Code Snippet

Any problems or issues, please leave a comment and I will assist you where I can. I wrote this up from my notes, and im confident I have included most of the vital parts!

Shoutouts for the 200th BLOG POST!!!

3 comments:

Asava Samuel said...

This is a reasonable priced SFTP Library you can use with VB.NET:
https://www.kellermansoftware.com/p-41-net-sftp-library.aspx

Anonymous said...

I followed your direction as closely as possible. I am unable to do "# service proftpd restart"...

What do I do now?

gaurav kulshrestha said...

Hi i am using proftpd and also configured virtual users , able to login also for the new user created with the password but when for the key based authentication i am getting some issue . what i ahve done for this is as follows.
1. generated the keys using puttygen
2.configure public key in proftpd.cof file for the "SFTPAuthorizedUserKeys file" attribute and have restarted the proftpd .

3. now use winscp to login with one of the virtual users loginId and use private key here which i generated using puttygen but not able to login.

here the confusion is i have not done any kind of mapping of these keys with the virtual user which i have created so please suggest if anyone has some solution for that