How to using NGINX caching proxy with S3

Below is a guide on a module to compile with NGINX that will allow access to private/public s3 objects while cacheing locally based on demand. This will likely save on bandwidth costs via s3 and allow for a faster response time.

1. First we will install/compile the AWS Auth module with NGINX. Here is a corresponding bash script to get you started.

#!/bin/bash
#prepare environment with required applications 
cd /root
yum -y install gcc gcc-c++ make zlib-devel pcre-devel openssl-devel git wget unzip perl

#install this AWS Auth module
wget https://github.com/anomalizer/ngx_aws_auth/archive/master.zip
unzip master.zip

#nginx install
mkdir -p src && cd src
nginxVersion="1.5.7"  #or latest
wget http://nginx.org/download/nginx-$nginxVersion.tar.gz
tar -xzf nginx-$nginxVersion.tar.gz

cd "nginx-$nginxVersion"
./configure --user=nginx --group=nginx --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx 
--conf-path=/etc/nginx/nginx.conf --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock 
--error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log 
--with-http_gzip_static_module        --with-http_stub_status_module 
--with-http_ssl_module --with-pcre --with-file-aio --with-http_realip_module 
--add-module=/root/ngx_aws_auth-master/

make
make install
useradd -r nginx

#add as daemon
wget -O /etc/init.d/nginx https://gist.github.com/sairam/5892520/raw/b8195a71e944d46271c8a49f2717f70bcd04bf1a/etc-init.d-nginx
chmod +x /etc/init.d/nginx

chkconfig nginx on
service nginx start

2. Setup the nginx.conf file to have the following configuration:

user nginx;
worker_processes  1;
error_log  /var/log/nginx/error.log;
pid        /var/run/nginx.pid;
events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main; 
    sendfile        on;  
    keepalive_timeout  65;
    proxy_cache_path /etc/nginx/cache keys_zone=mycache:10m; 
 server {
   listen     80;
   proxy_cache mycache;
   location / {
     proxy_pass http://[YOUR-S3-BUCKET].s3.amazonaws.com;
      aws_access_key [YOUR-S3-ACCESS-KEY];
      aws_secret_key [YOUR-S3-SECRET-KEY];
      s3_bucket [YOUR-S3-BUCKET];

      proxy_set_header Authorization $s3_auth_token;
      proxy_set_header x-amz-date $aws_date;
    }
  }
}

Note that the proxy_cache_path and proxy_cache directives indicate how your cacheing infrastructure will work (how often it refreshes etc). Be sure to replace the elements in bold and restart nginx after this config change.

3. Ensure your bucket policy does not accept any other requests than from this server (or the servers you’ve setup the proxy on etc).

4. Then you can access your S3 objects via a url like: http://[your-nginx-server]/path/to/object/in/s3.mp4


Comments are closed.