Project

General

Profile

Docs ModSecDownload » History » Revision 43

Revision 42 (maddingue, 2014-04-10 15:50) → Revision 43/46 (gstrauss, 2016-07-13 04:57)

h1. Secure and Fast Downloading 


 *Module: mod_secdownload* 

 {{>toc}} 

 h2. Description 


 There are multiple ways to handle secured download mechanisms: 

 # use the webserver and the internal HTTP authentication 
 # use the application to authenticate and send the file 
    through the application 

 Both ways have limitations: 

 Webserver: 
 * + fast download 
 * + no additional system load 
 * -- inflexible authentication handling 

 Application: 

 * + integrated into the overall layout 
 * + very flexible permission management 
 * -- the download occupies an application thread/process 

 A simple way to combine the two ways could be: 

 1. app authenticates user and checks permissions to 
    download the file. 
 2. app redirects user to the file accessable by the webserver 
    for further downloading. 
 3. the webserver transfers the file to the user. 

 As the webserver doesn't know anything about the permissions 
 used in the app, the resulting URL would be available to every 
 user who knows the URL. 

 mod_secdownload removes this problem by introducing a way to 
 authenticate a URL for a specified time. The application has 
 to generate a token and a timestamp which are checked by the 
 webserver before it allows the file to be downloaded by the 
 webserver. 

 The generated URL has to have the format: 

 <pre> 
 <uri-prefix>/<token>/<timestamp-in-hex>/<rel-path> 
 which looks like "yourserver.com/bf32df9cdb54894b22e09d0ed87326fc/435cc8cc/secure.tar.gz" 


 <token> is an MD5 of 

 1. a secret string (user supplied) 
 2. <rel-path> (starts with /) 
 3. <timestamp-in-hex> 

 </pre> 

 As you can see, the token is not bound to the user at all. The 
 only limiting factor is the timestamp which is used to 
 invalidate the URL after a given timeout (secdownload.timeout). 

 h3. Important 


 Be sure to choose a another secret than the one used in the 
 examples, as this is the only part of the token that is not 
 known to the user. 

 Ensure that the token is also in hexadecimal. Depending on  
 the programming language you use, there might be no extra  
 step for this. For instance, in PHP, the MD5 function  
 returns the Hex value of the digest. If, however, you use a  
 language such as Java    or Python, the extra step of converting  
 the digest into Hex is needed (see the Python example below). 



 If the user tries to fake the URL by choosing a random token, 
 status 403 'Forbidden' will be sent out. 

 If the timeout is reached, status 410 'Gone' will be 
 sent. This used to be 408 'Request Timeout' in earlier versions. 

 If token and timeout are valid, the <rel-path> is appended to 
 the configured (secdownload.document-root) and passed to the 
 normal internal file transfer functionality. This might lead to 
 status 200 or 404. 


 h2. Options 

 <pre> 
   secdownload.secret          = <string> 
   secdownload.document-root = <string> 
   secdownload.uri-prefix      = <string>    (default: /) 
   secdownload.timeout         = <short>     (default: 60 seconds) 
   secdownload.algorithm       = <string>    ("md5", "hmac-sha1", "hmac-sha256") 
 </pre> 


 h2. Examples 

 Your application has to generate the correct URLs.  

 h3. PHP Example 

 <pre> 
   <?php 
  
   $secret = "verysecret"; 
   $uri_prefix = "/dl/"; 
  
   # filename 
   # please note file name starts with "/" 
   $f = "/secret-file.txt"; 
  

  
   # current timestamp 
   $t = time(); 
  
   $t_hex = sprintf("%08x", $t); 
   $m = md5($secret.$f.$t_hex); 
  
   # generate link 
   printf('<a href="%s%s/%s%s">%s</a>', 
          $uri_prefix, $m, $t_hex, $f, $f); 
   ?> 
 </pre> 

 h3. Ruby On Rails example, used in the context of a helper 

 <pre> 
   def gen_sec_link(rel_path) 
     rel_path.sub!(/^([^\/])/,'/\1')       # Make sure it had a leading slash 
     s_secret = 'verysecret'               # Secret string 
     uri_prefix = '/dl/'                   # Arbitrary download prefix 
     timestamp = "%08x" % Time.now.to_i    # Timestamp, to hex 
     token = MD5::md5(s_secret + rel_path + timestamp).to_s      # Token Creation 
     '%s%s/%s%s' % [uri_prefix, token, timestamp, rel_path]     # Return the properly formatted string 
   end 
 </pre> 

 So in a view or helper:  

 <pre> 
 <%= link_to "Private Image", gen_sec_link("path/from/download-area/someimage.img") %> 
 </pre> 

 

 h3. Perl Example 

 <pre> 
 use strict; 
 use Digest::MD5 qw< md5_hex >; 

 my $request = "whatever.txt"; 
 print gen_sec_link($request), "\n"; 

 sub gen_sec_link { 
     my $relpath = shift; 
     $relpath      =~ s:^([^/]):/$1:;    # make sure it has a leading slash 
     my $prefix    = "/static/";         # download prefix 
     my $secret    = "verysecret";       # secret string 
     my $hextime = sprintf "%08x", time; 
     my $token     = md5_hex($secret.$relpath.$hextime); 
     return sprintf "%s%s/%s%s", $prefix, $token, $hextime, $relpath 
 } 
 </pre> 

 

 h3. Python example, usable with Django or any other Python web framework 

 <pre> 
   def gen_sec_link(rel_path): 
       import time, hashlib 
       secret = 'verysecret' 
       uri_prefix = '/dl/' 
       hextime = "%08x" % time.time() 
       token = hashlib.md5(secret + rel_path + hextime).hexdigest() 
       return '%s%s/%s%s' % (uri_prefix, token, hextime, rel_path) 
 </pre> 

 *Note:* When using Django with non-ASCII file names (hashlib in Python 2.x does not handle non-ASCII strings properly, throwing the UnicodeDecodeError exception, unless bytestrings encoded in UTF-8 are supplied instead of the default unicode objects): 

 <pre> 
   def gen_securelink(rel_path): 
       import time, hashlib 
       from django.utils.http import urlquote 
       rel_path = '/%s/%s' % (self.series.directory, self.filename) 
       secret = "flylight" 
       uri_prefix = "http://media.ongoing.ru/download/" 
       hextime = "%08x" % time.time() 
       token = hashlib.md5((secret + rel_path + hextime).encode('utf-8')).hexdigest() 
       return '%s%s/%s%s' % (uri_prefix, token, hextime, urlquote(rel_path)) 
 </pre> 

 (urlquote() can be replaced by urllib methods, of course.) 

 h3. C# Example 

 <pre> 
 //import library 

 using System.Security.Cryptography; 
 using System.Text; 

 //function : 
     public string GetHash(string hashMe) // function get MD5 
     { 
         MD5CryptoServiceProvider md5 = new MD5CryptoServiceProvider(); 
         UTF7Encoding encoder = new UTF7Encoding(); 
         Byte[] encStringBytes; 
         encStringBytes = encoder.GetBytes(hashMe); 
         encStringBytes = md5.ComputeHash(encStringBytes); 
         string strHex = string.Empty; 
         foreach (byte b in encStringBytes)        
             strHex += String.Format("{0:x2}", b);         
         return strHex; 
     } 
     public string GetCurrentEpochTimeInHex() // function get current epoch time in HEX 
     {         
         DateTime EpochTime = new DateTime(1970, 1, 1, 0, 0, 0); 
         TimeSpan diff = DateTime.UtcNow - EpochTime; 
         return ((long)diff.TotalSeconds).ToString("x"); 
     } 
     public string GenerateSecureLink(string rel_path) // ex : rel_path = "/secret-file.txt"; 
     { 
         string t_hex = GetCurrentEpochTimeInHex(); 
         string serect = "verysecret"; 
         string uri_prefix = "/dl/"; 
         string m = GetHash(serect + rel_path + t_hex); 
         return String.Format("{0}{1}/{2}{3}", uri_prefix, m, t_hex, rel_path); 
     } 
 </pre> 

 Reference make MD5 in C# : http://ok-cool.com/posts/read/125-php-md5-not-the-same-as-net-md5/ 

 h3. JAVA Example 

 <pre> 

 //Usage 
 String url = new LighttpdAuthUrl("verysecret","/dl/",videoName).toString(); 


 //Import Packages 
 import java.security.MessageDigest; 
 import java.security.NoSuchAlgorithmException; 



 public class LighttpdAuthUrl { 

	 private final String token; 
	 private final String timestamp; 
	 private final String authpath; 
	 private final String filepath; 
	
	
	 public LighttpdAuthUrl(String secret, String authpath, String filepath) { 
		 this(secret, authpath, filepath, System.currentTimeMillis()); 
	 } 
	
	
	 public LighttpdAuthUrl(String secret, String authpath, String filepath, long time) { 
		 this.timestamp = Long.toHexString(time/1000L); 
		 this.authpath = authpath; 
		 this.filepath = sanitizeFilePath(filepath); 
		 this.token = generateToken(secret); 
	 } 
	
	 public String sanitizeFilePath(String fpath) { 
		 if (fpath.charAt(0) != '/') { 
			 fpath = '/' + fpath; 
		 } 
		 return fpath; 
	 } 
	
	 private String generateToken(String secret) { 
		 return toMD5(secret+filepath+timestamp); 
	 } 
	
	 public static String toMD5(String source) { 
         MessageDigest digest = null; 
         try { 
             digest = MessageDigest.getInstance("MD5"); 
         } catch (NoSuchAlgorithmException e) { 
             throw new RuntimeException(e); 
         } 
         digest.update(source.getBytes()); 
         byte[] hash = digest.digest(); 
         return byteArrayToHexString(hash); 
     } 
	
	 public static String byteToHexString(byte aByte) { 
		 String hex = Integer.toHexString(0xFF & aByte); 
		 return ((hex.length() == 1)?"0":"")+hex; 
	 } 

	 public static String byteArrayToHexString(byte[] hash) { 
		 StringBuffer hexString = new StringBuffer(); 
		 for (int i=0;i<hash.length;i++) { 
			 hexString.append(byteToHexString(hash[i])); 
		 } 
		 return hexString.toString(); 
	 } 
	
	 public static boolean isBlank(String str) { 
		 return str == null || str.length() == 0; 
	 } 
	
	 public String getToken() { 
		 return token; 
	 } 

	 public String getTimestamp() { 
		 return timestamp; 
	 } 

	 public String toString() { 
		 return toUriString(); 
	 } 
	
	 public String toUriString() { 
		 return authpath + token + "/" + timestamp + filepath; 
	 } 
 } 
 </pre> 

 h3. Webserver 

 The server has to be configured in the same way. The URI prefix and 
 secret have to match: :: 

 <pre> 

   server.modules = ( ..., "mod_secdownload", ... ) 
  
   secdownload.secret            = "verysecret" 
   secdownload.document-root     = "/home/www/servers/download-area/" 
   secdownload.uri-prefix        = "/dl/" 
   secdownload.timeout           = 10 
 </pre>