February 22nd, 2011 by Josh

Optimising Phusion Passenger for a VPS

Having recently launched a new site we've been keen to look into the finer details of getting more out of our server instances. We use Slicehost (amongst other providers) to host some of our applications on Virtual Private Servers. Depending on the type of applications we were hosting we never did too much to the standard configuration that comes shipped with Phusion Passenger.

The standard configuration of Passenger assumes a small to medium sized box. We usually start our clients off on a 256MB slice and then build up that specification before or after launch (depending on how much traffic they expect). Below are a few tweaks we found helped squeeze a bit more performance out of a smaller VPS before having to scale it up.

NB: We use NGINX to serve our sites, which we prefer over Apache due to its lower memory footprint. Each directive can easily be translated to its Apache counterpart by looking it up in the Apache version of the documentation.

passenger_max_pool_size

This value needs to reflect the specification of the server you're running. The larger the server the higher this number can be. For a small VPS (256MB) you'll want to set this at 2. Any more than this and you'll struggle to free up enough resources for other software running on the server. We found this to be particularly beneficial as we tend to run MySQL alongside NGINX for the standard Rails setup. This will also give you some leeway if you need to run cron jobs and other background tasks.

http {
  ...
  
  passenger_max_pool_size 2;
}

It's also worth nothing that if you run rake tasks through your cron jobs that these might load the Rails environment before executing the task. This may result in a spike in performance due to the decreased amount of memory available.

passenger_pool_idle_time

This directive controls how long an application instance will remain dormant before shutting down. This figure will most likely correlate to the amount of traffic the site will be receiving. If you expect a constant stream of traffic you'll want to set this to a higher value. If the amount of traffic is infrequent and relatively small you might want to set it to a lower value. As Passenger will be shutting down the application processes after the given time period, users visiting the site after this period will notice a short pause whilst the new application process is spawned, however this will reduce the overall memory usage as processes that are not being used are shut down. Setting the value to 0 will mean the process is never shut down. The documentation suggests using the formula 2 * X, where X is the average number of seconds a user is expected to remain on the site.

http {
  ...
  
  passenger_pool_idle_time 120;
}

passenger_pre_start

Whenever we restart Passenger we want our application instances available as soon as the web server is ready. To enforce this you can use the passenger_pre_start directive. This will send a local request to a given URL when the web server is started, similar to when a user hits the site for the first time. This will then go through the normal spawn process.

http {
  ...
  
  passenger_pre_start http://example.com/;
}

You can read about the rationale for using a URL instead of a flag for this setting in the Passenger documentation.

Inspecting memory usage to test your changes

Passenger comes with a handy utility called passenger-memory-stats. This allows you to view realistic statistics surrounding the memory footprint of everything Passenger is up to. It's useful to keep this tool nearby in a separate SSH session so you can keep track of what your changes are actually doing.

someone@somewhere:~$ sudo passenger-memory-stats
[sudo] password for someone: 
============== Apache processes ==============
*** WARNING: The Apache executable cannot be found.
Please set the APXS2 environment variable to your 'apxs2' executable's filename, or set the HTTPD environment variable to your 'httpd' or 'apache2' executable's filename.


============== Nginx processes ===============
PID    PPID   VMSize   Private  Name
==============================================
20779  1      35.5 MB  0.0 MB   nginx: master process /opt/nginx/sbin/nginx
20780  20779  36.0 MB  0.3 MB   nginx: worker process
### Processes: 2
### Total private dirty RSS: 0.32 MB


============== Passenger processes ==============
PID    VMSize    Private  Name
=================================================
20755  87.0 MB   ?        PassengerWatchdog
20758  97.2 MB   0.6 MB   PassengerHelperAgent
20760  39.2 MB   3.3 MB   Passenger spawn server
20763  70.5 MB   0.0 MB   PassengerLoggingAgent
20851  177.6 MB  80.2 MB  Rack: /home/someone/sites/example.com/current
21025  162.9 MB  67.0 MB  Rack: /home/someone/sites/staging.example.com/current
### Processes: 6
### Total private dirty RSS: 151.05 MB (?)

Alongside this tool there's also passenger-status. This should be run as root and shows Passenger's loaded configuration. This is split into two sections, general and domains (applications).

someone@somewhere:~$ sudo passenger-status
[sudo] password for someone: 
============== General information ==============
max      = 2
count    = 2
active   = 0
inactive = 2
Waiting on global queue: 0

============== Application groups ===============
/home/someone/sites/example.com/current:
  App root: /home/someone/sites/example.com/current
  * PID: 20851   Sessions: 0    Processed: 801     Uptime: 18h 19m 9s

/home/someone/sites/staging.example.com/current:
  App root: /home/someone/sites/staging.example.com/current
  * PID: 21025   Sessions: 0    Processed: 9       Uptime: 18h 11m 19s

You can read more about what each section reports in the Passenger documentation, however some of it might be apparent from the changes you will have made to the configuration earlier. Both these tools should aid you in tuning your instances to better suit the machine you're running them on.

Even though these optimisations are for a small site you can scale it up pretty well to larger servers (to a certain degree). Obviously there is a lot more you will be doing if you're quickly outgrowing VPS's but we found these tips useful for a few quick wins on smaller instances.