Secure and Fast Downloading¶

Module: mod_secdownload

• Table of contents
• Secure and Fast Downloading
  • Description
    • Important
  • Options
  • Examples
    • PHP Example
    • Ruby On Rails example, used in the context of a helper
    • Perl Example
    • Python example, usable with Django or any other Python web framework
    • C# Example
    • JAVA Example
    • Webserver

Description¶

There are multiple ways to handle secured download mechanisms:

1. use the webserver and the internal HTTP authentication
2. use the application to authenticate and send the file
   through the application

Both ways have limitations:

Webserver:

• + fast download
• + no additional system load
• -- inflexible authentication handling

Application:

• + integrated into the overall layout
• + very flexible permission management
• -- the download occupies an application thread/process

A simple way to combine the two ways could be:

1. app authenticates user and checks permissions to
download the file.
2. app redirects user to the file accessable by the webserver
for further downloading.
3. the webserver transfers the file to the user.

As the webserver doesn't know anything about the permissions
used in the app, the resulting URL would be available to every
user who knows the URL.

mod_secdownload removes this problem by introducing a way to
authenticate a URL for a specified time. The application has
to generate a token and a timestamp which are checked by the
webserver before it allows the file to be downloaded by the
webserver.

The generated URL has to have the format:

<uri-prefix>/<token>/<timestamp-in-hex>/<rel-path>
which looks like "yourserver.com/bf32df9cdb54894b22e09d0ed87326fc/435cc8cc/secure.tar.gz" 

<token> is an MD5 of

1. a secret string (user supplied)
2. <rel-path> (starts with /)
3. <timestamp-in-hex>

As you can see, the token is not bound to the user at all. The
only limiting factor is the timestamp which is used to
invalidate the URL after a given timeout (secdownload.timeout).

Important¶

Be sure to choose a another secret than the one used in the
examples, as this is the only part of the token that is not
known to the user.

Ensure that the token is also in hexadecimal. Depending on
the programming language you use, there might be no extra
step for this. For instance, in PHP, the MD5 function
returns the Hex value of the digest. If, however, you use a
language such as Java or Python, the extra step of converting
the digest into Hex is needed (see the Python example below).

If the user tries to fake the URL by choosing a random token,
status 403 'Forbidden' will be sent out.

If the timeout is reached, status 410 'Gone' will be
sent. This used to be 408 'Request Timeout' in earlier versions.

If token and timeout are valid, the <rel-path> is appended to
the configured (secdownload.document-root) and passed to the
normal internal file transfer functionality. This might lead to
status 200 or 404.

Options¶

  secdownload.secret        = <string>
  secdownload.document-root = <string>
  secdownload.uri-prefix    = <string>  (default: /)
  secdownload.timeout       = <short>   (default: 60 seconds)
  secdownload.algorithm     = <string>  ("md5", "hmac-sha1", "hmac-sha256")
  secdownload.path-segments = <number>
  secdownload.hash-querystr = "enable" | "disable" 

Examples¶

Your application has to generate the correct URLs.

PHP Example¶

  <?php

  $secret = "verysecret";
  $uri_prefix = "/dl/";

  # filename
  # please note file name starts with "/" 
  $f = "/secret-file.txt";

  # current timestamp
  $t = time();

  $t_hex = sprintf("%08x", $t);
  $m = md5($secret.$f.$t_hex);

  # generate link
  printf('<a href="%s%s/%s%s">%s</a>',
         $uri_prefix, $m, $t_hex, $f, $f);
  ?>

Ruby On Rails example, used in the context of a helper¶

  def gen_sec_link(rel_path)
    rel_path.sub!(/^([^\/])/,'/\1')     # Make sure it had a leading slash
    s_secret = 'verysecret'             # Secret string
    uri_prefix = '/dl/'                 # Arbitrary download prefix
    timestamp = "%08x" % Time.now.to_i  # Timestamp, to hex
    token = MD5::md5(s_secret + rel_path + timestamp).to_s    # Token Creation
    '%s%s/%s%s' % [uri_prefix, token, timestamp, rel_path]   # Return the properly formatted string
  end

So in a view or helper:

<%= link_to "Private Image", gen_sec_link("path/from/download-area/someimage.img") %>

Perl Example¶

use strict;
use Digest::MD5 qw< md5_hex >;

my $request = "whatever.txt";
print gen_sec_link($request), "\n";

sub gen_sec_link {
    my $relpath = shift;
    $relpath    =~ s:^([^/]):/$1:;  # make sure it has a leading slash
    my $prefix  = "/static/";       # download prefix
    my $secret  = "verysecret";     # secret string
    my $hextime = sprintf "%08x", time;
    my $token   = md5_hex($secret.$relpath.$hextime);
    return sprintf "%s%s/%s%s", $prefix, $token, $hextime, $relpath
}

Python example, usable with Django or any other Python web framework¶

  def gen_sec_link(rel_path):
      import time, hashlib
      secret = 'verysecret'
      uri_prefix = '/dl/'
      hextime = "%08x" % time.time()
      token = hashlib.md5(secret + rel_path + hextime).hexdigest()
      return '%s%s/%s%s' % (uri_prefix, token, hextime, rel_path)

Note: When using Django with non-ASCII file names (hashlib in Python 2.x does not handle non-ASCII strings properly, throwing the UnicodeDecodeError exception, unless bytestrings encoded in UTF-8 are supplied instead of the default unicode objects):

  def gen_securelink(rel_path):
      import time, hashlib
      from django.utils.http import urlquote
      rel_path = '/%s/%s' % (self.series.directory, self.filename)
      secret = "flylight" 
      uri_prefix = "http://media.ongoing.ru/download/" 
      hextime = "%08x" % time.time()
      token = hashlib.md5((secret + rel_path + hextime).encode('utf-8')).hexdigest()
      return '%s%s/%s%s' % (uri_prefix, token, hextime, urlquote(rel_path))

(urlquote() can be replaced by urllib methods, of course.)

C# Example¶

//import library

using System.Security.Cryptography;
using System.Text;

//function :
    public string GetHash(string hashMe) // function get MD5
    {
        MD5CryptoServiceProvider md5 = new MD5CryptoServiceProvider();
        UTF7Encoding encoder = new UTF7Encoding();
        Byte[] encStringBytes;
        encStringBytes = encoder.GetBytes(hashMe);
        encStringBytes = md5.ComputeHash(encStringBytes);
        string strHex = string.Empty;
        foreach (byte b in encStringBytes)       
            strHex += String.Format("{0:x2}", b);        
        return strHex;
    }
    public string GetCurrentEpochTimeInHex() // function get current epoch time in HEX
    {        
        DateTime EpochTime = new DateTime(1970, 1, 1, 0, 0, 0);
        TimeSpan diff = DateTime.UtcNow - EpochTime;
        return ((long)diff.TotalSeconds).ToString("x");
    }
    public string GenerateSecureLink(string rel_path) // ex : rel_path = "/secret-file.txt";
    {
        string t_hex = GetCurrentEpochTimeInHex();
        string serect = "verysecret";
        string uri_prefix = "/dl/";
        string m = GetHash(serect + rel_path + t_hex);
        return String.Format("{0}{1}/{2}{3}", uri_prefix, m, t_hex, rel_path);
    }

Reference make MD5 in C# : http://ok-cool.com/posts/read/125-php-md5-not-the-same-as-net-md5/

JAVA Example¶

//Usage
String url = new LighttpdAuthUrl("verysecret","/dl/",videoName).toString();

//Import Packages
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;

public class LighttpdAuthUrl {

    private final String token;
    private final String timestamp;
    private final String authpath;
    private final String filepath;

    public LighttpdAuthUrl(String secret, String authpath, String filepath) {
        this(secret, authpath, filepath, System.currentTimeMillis());
    }

    public LighttpdAuthUrl(String secret, String authpath, String filepath, long time) {
        this.timestamp = Long.toHexString(time/1000L);
        this.authpath = authpath;
        this.filepath = sanitizeFilePath(filepath);
        this.token = generateToken(secret);
    }

    public String sanitizeFilePath(String fpath) {
        if (fpath.charAt(0) != '/') {
            fpath = '/' + fpath;
        }
        return fpath;
    }

    private String generateToken(String secret) {
        return toMD5(secret+filepath+timestamp);
    }

    public static String toMD5(String source) {
        MessageDigest digest = null;
        try {
            digest = MessageDigest.getInstance("MD5");
        } catch (NoSuchAlgorithmException e) {
            throw new RuntimeException(e);
        }
        digest.update(source.getBytes());
        byte[] hash = digest.digest();
        return byteArrayToHexString(hash);
    }

    public static String byteToHexString(byte aByte) {
        String hex = Integer.toHexString(0xFF & aByte);
        return ((hex.length() == 1)?"0":"")+hex;
    }

    public static String byteArrayToHexString(byte[] hash) {
        StringBuffer hexString = new StringBuffer();
        for (int i=0;i<hash.length;i++) {
            hexString.append(byteToHexString(hash[i]));
        }
        return hexString.toString();
    }

    public static boolean isBlank(String str) {
        return str == null || str.length() == 0;
    }

    public String getToken() {
        return token;
    }

    public String getTimestamp() {
        return timestamp;
    }

    public String toString() {
        return toUriString();
    }

    public String toUriString() {
        return authpath + token + "/" + timestamp + filepath;
    }
}

Webserver¶

The server has to be configured in the same way. The URI prefix and
secret have to match: ::

  server.modules = ( ..., "mod_secdownload", ... )

  secdownload.secret          = "verysecret" 
  secdownload.document-root   = "/home/www/servers/download-area/" 
  secdownload.uri-prefix      = "/dl/" 
  secdownload.timeout         = 10