HTTP class

Background

The Hypertext Transfer Protocol (HTTP) is an application-level protocol for distributed, collaborative, hypermedia information systems. HTTP functions as a request-response protocol in the client-server computing model. A web browser, for example, may be the client and an application running on a computer hosting a web site may be the server. The client submits an HTTP request message to the server. The server, which provides resources such as HTML files and other content, or performs other functions on behalf of the client, returns a response message to the client. The response contains completion status information about the request and may also contain requested content in its message body. HTTP resources are identified and located on the network by Uniform Resource Locators (URLs) using the http or https URI schemes.

For more information, see http://www.w3.org/Protocols/.

Overview

The HTTP class implements a standard HTTP client as defined in RFCs 1945 and 2616. It can be used for communication with an HTTP server and it supports the HTTP as well as the HTTPS protocols.

HTTP communication involves the client sending requests to the server and the server sending a response back. The HTTP class allows using different request methods, defining authentication and defining additional parameters when sending requests, and it allows capturing the response into a file.

The current implementation supports the HTTP request methods PUT, GET, POST and DELETE. The methods HEAD, OPTIONS, TRACE, CONNECT and PATCH are not available, but they are also less important.

The HTTP class supports the HTTP Basic authentication scheme through the user and password properties. Other authentication schemes (Digest, NTLM, Negotiate, Proprietary, OAuth) can be implemented by using the authScheme property.

The addParameter() method can be used to add extra parameters to the request and attaching file is done with the setAttachedFile() method.

The server response is written into a file when the localFilePath property is set. This is whatever the server has responded; it can be the HTML source of the requested URL, JSON or XML data returned by a REST API call, or a file or document that was downloaded. If this property is not specified, the server response can be obtained as byte array using the getServerResponse() method.

The HTTP class also defines a number of constants that are available in the autocomplete list.

Constructing

HTTP( connectionType : String )

Constructs a new HTTP object. The connectionType argument can be “NoSSL” or “SSL” for HTTP or HTTPS protocol respectively. It is also possible to use the predefined constants HTTP.NoSSL and HTTP.SSL. If the connectionType is omitted, by default “NoSSL” is used.

var theHTTP = new HTTP( HTTP.SSL );

Properties

url : String

Contains the URL to be used for request. The URL must be URI-encoded. The method HTTP.encodeURI() can be used to encode the URL.

theHTTP.url = “https://api.dropbox.com/1/fileops/delete”;

The URL must specify the protocol to be used (http:// or https://) and it must be in line with the connectionType specified when constructing the HTTP object.

enableMime : Boolean

This property specifies if MIME encoding should be used (as specified in RFC1867).

Note: This property is relevant only when POST request is used.

authScheme : String

This property will tell the class which type of authorization to perform when the user and password properties are set.

This property should be set to "NoneAuth" (predefined constant HTTP.NoneAuth) when no authentication is to be performed.
By default, this property is "BasicAuth" (HTTP.BasicAuth), and if the user and password properties are set, the class will attempt basic authentication.
If authScheme is set to "DigestAuth" (HTTP.DigestAuth), "NTLMAuth" (HTTP.NTLMAuth) or "NegotiateAuth" (HTTP.NegotiateAuth), digest, NTLM or Negotiate authentication will be attempted instead.
If authScheme is set to "ProprietaryAuth" (HTTP.ProprietaryAuth), the authorization token must be supplied through the authorization property.
If authScheme is set to "OauthAuth" (HTTP.OauthAuth), the authorization string must be supplied through the authorization property.

theHTTP.authScheme = HTTP.OauthAuth;

authorization : String

If the authorization property contains a non-empty string, an Authorization HTTP request header is added to the request. This header conveys authorization information to the server. A common use for this property is to specify the OAuth authorization string.

The authScheme property defines the authentication scheme used. In the case of HTTP Basic Authentication (default), the user and password are automatically Base64 encoded by Switch, and the result in the form "Basic [encoded-user-password]" is used for authorization.

theHTTP.authorization = “authorization string”;

user : String

This property contains a user name if authentication is to be used.

If authScheme is set to HTTP Basic Authentication, the user and password are Base64 encoded, and the result in the form "Basic [encoded-user-password]" is used for authorization.
If authScheme is set to HTTP Digest Authentication, the user and password properties are used to respond to the HTTP Digest Authentication challenge from the server.
If authScheme is set to NTLM, NTLM authentication will be attempted.

theHTTP.user = “User1”;

password : String

This property contains a password if authentication is to be used.

theHTTP.password = “password1”;

localFilePath : String

The path to a local file that the server response is written to. This can be the file or document being downloaded, JSON or XML data or whatever the server has responded with. If this property is not specified the server response can be obtained as byte array using the getServerResponse() method.

theHTTP.localFilePath = job.createPathWithExtension( ”pdf”, false );

timeOut : Number

If the timeOut property is set to "0", all operations will run uninterrupted until successful completion or until an error condition is encountered.

If timeOut is set to a positive value, the class will wait for the operation to complete and if timeOut expires before the operation is complete, the class fails with an error.

The default value for the timeOut property is "60" (seconds).

useProxy : Boolean

This property specifies if a proxy server should be used.

proxyServer : Boolean

If a proxy proxyServer is given, the HTTP request is sent to the proxy instead of the server specified in the url property.

If the proxyServer property is set to a Domain Name, a DNS request is initiated and upon successful termination of the request, the corresponding address is used for sending the HTTP request. If the search is not successful, an error is returned.

proxyPort : Number

This property contains the TCP port for the proxy server (default: 80).

proxyUser : String

This property contains a user name, if authentication is to be used for the proxy.

If proxyAuthScheme is set to Basic Authentication, the proxyUser and proxyPassword are Base64 encoded and the proxy authentication token will be generated in the form "Basic [encoded-user-password]".
If proxyAuthScheme is set to Digest Authentication, the proxyUser and proxyPassword properties are used to respond to the Digest Authentication challenge from the server.
If proxyAuthScheme is set to NTLM Authentication, the proxyUser and proxyPassword properties are used to authenticate through NTLM negotiation.

proxyPassword : String

This property contains a password if authentication is to be used for the proxy.

proxyAuthScheme : String

This property is used to tell the class which type of authorization to perform when connecting to the proxy. This is only used when the proxyUser and proxyPassword properties are set.

proxyAuthScheme should be set to "NoneAuth" (predefined constant HTTP.NoneAuth) when no authentication is expected.
By default, proxyAuthScheme is "BasicAuth" (HTTP.BasicAuth), and if the proxyUser and proxyPassword properties are set, the component will attempt basic authentication.
If proxyAuthScheme is set to "DigestAuth" (HTTP.DigestAuth), digest authentication will be attempted instead.
If proxyAuthScheme is set to "ProprietaryAuth" (HTTP.ProprietaryAuth), the authorization token will not be generated by the class.
If proxyAuthScheme is set to "NTLMAuth" (HTTP.NTLMAuth), NTLM authentication will be used.

HTTPVersion : String

This property contains the HTTP version from the first line of the last server response. The HTTP protocol specifies the structure of this line as: [HTTP version] [Status Code] [Description].

This property is read-only and becomes available after the HTTP request is finished.

statusCode : Number

This property contains the Status Code from the first line of the last server response. The HTTP protocol specifies the structure of this line as: [HTTP version] [Status Code] [Description]. For the list of HTTP status codes, see http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html.

This property is read-only and becomes available after the HTTP request is finished.

statusDescription : String

This property contains the Description from the first line of the last server response. The HTTP protocol specifies the structure of this line as: [HTTP version] [Status Code] [Description].

This property is read-only and becomes available after the HTTP request is finished.

lastError : String

This property contains the last error string.

This property is read-only and becomes available after the HTTP request is finished.

finishedStatus : String

This property contains the request finished status. The possible statuses are:

"Ok" (predefined constant HTTP.Ok) - the request was executed successfully.
"Failed" (HTTP.Failed) – the request failed to execute.
"Interrupted" (HTTP.Interrupted) – the request was interrupted.

Note that when finishedStatus returns “Ok” it does not automatically mean that the request yielded the desired result. The URL provided could have been redirected to another URL. In such a case the HTTP exchange with the server was successful, but nothing happened. It is therefore advisable to also read the properties statusCode and statusDescription to get more details about the completed request. In case of failed status, the lastError property should provide more details about the problem.

if( theHTTP.finishedStatus == HTTP.Failed )
{      
      job.fail( ”The request failed: %1”, theHTTP.lastError );
      return;
}

resumeEnabled : Boolean

Relevant only for GET request. If set to true, Switch will try to resume the interrupted/broken download if possible. The files containing not finished downloads are stored in the folder defined by the tempFolderPath property.

tempFolderPath : String

Relevant only for GET request. This property defines a folder where not finished downloads are stored.

By default this property is empty. In case resumeEnabled is set to true, this property must be set as well. It should contain a folder path pointing to an existing writable folder. Switch does not clean up this folder automatically; it removes the files containing not finished downloads after they are completed by some future request.

followRedirects : Boolean

Relevant only for GET request. This property determines what happens when the server issues a redirect. If this property is set to false, the HTTP call returns an error if the server responds with an "Object Moved" message (HTTP status codes in the 300 range). If this property is set to true (default), the new URL for the object is retrieved automatically.

Note: To avoid endless loops, the maximum number of redirects is set to 10. A redirection between protocols (for example HTTP to HTTPS) is not allowed and will generate an error.

Static functions

encodeURI( inURI : String ) : String: This function encodes inURI by replacing each instance of certain characters by one, two, three, or four escape sequences representing the UTF-8 encoding of the character. It assumes that the URI is a complete URI, so it does not encode reserved characters that have special meaning in the URI. It replaces all characters except the following: alphabetic, decimal digits, - _ . ! ~ * ' ( ) ; , / ? : @ & = + $ #

decodeURI( inURI : String ) : String: This function decodes inURI by replacing each escape sequence in the encoded URI with the character that it represents.

encodeURIComponent( inURIComponent : String ) : String

This function encodes inURIComponent by replacing each instance of certain characters by one, two, three, or four escape sequences representing the UTF-8 encoding of the character.

It replaces all characters except the following: alphabetic, decimal digits, - _ . ! ~ * ' ( )

decodeURIComponent( inURIcomponent : String ) : String: This function decodes inURIcomponent by replacing each escape sequence in the encoded URI component with the character that it represents.

Member functions

setAttachedFile( filePath : String , fileVariable : String )

Sets the path to the file that is appended to the HTTP request, if the HTTP POST or PUT methods are used (through the post() or put() methods). The second argument is optional. It is taken into account only when POST request is sent and MIME encoding is enabled. It specifies the name of the HTTP form data variable that is used by the receiving HTTP server to identify the correct file part of the uploaded MIME package. If this argument is omitted, the default value "file" will be used.

theHTTP.setAttachedFile( job.getPath() );

Warning:

In case of a POST request with MIME encoding enabled, this call identifies the MIME part by adding the following MIME part header:

Content-Disposition: form-data; filename=“name of the file that is being attached”; name=“second parameter”

If the file being uploaded is job.getPath(), the filename field will also have the unique ID of the job. To work around this limitation, a temporary copy of the job without the unique ID has to be made and that temporary file should be the attached file:

var tmpFile = job.createPathWithName(job.getName());
if(!s.copy(job.getPath(), tmpFile))
{
   job.fail( "Could not copy the job to the temporary location: %1", tmpFile );
   return;
}
theHTTP.setAttachedFile(tmpFile);

addParameter( key : String, value: String )

Adds parameters to the HTTP request.

For the POST and PUT requests, the parameters are included into the HTTP request after the HTTP headers.
For the GET request, the parameters are appended to the URL after ‘?’; when calling a REST API it is common to use URLs that look like this: http://www.xyz.com/updatedatabase?key1=value1&key2=value2. The HTTPS protocol encrypts everything after the ‘?’.
For DELETE requests, the parameters are ignored.

The method HTTP.encodeURIComponent() can be used to encode parameters. Note that when adding parameters, the key and value must always be URI-encoded, except when POST requests with MIME encoding are used (the property enableMime is true). In this case the parameters are put in the MIME data and in most cases they should not be URI-encoded by the script.

theHTTP.addParameter( ”root”, ”auto” );
theHTTP.addParameter( ”path”, HTTP.encodeURIComponent( job.getName() );

resetParameters()

Clears parameters previously added by the addParameter() method.

theHTTP.resetParameters();

post()

This method posts data to the HTTP server using the HTTP POST method. The data includes the file that was specified using the setAttachedFile() method and also parameters added using the addParameter() method. If the property localFilePath is specified, the server response is written to this file; otherwise the response can be obtained as a byte array using the getServerResponse() method.

The 'Content-Type' HTTP request header is automatically added to the request with the value "application/x-www-form-urlencoded" (which means the request contains HTML form input data).

theHTTP.post();

Note: This method is asynchronous. It means it does not wait till the request is finished. To wait for the request completion, a script writer has to use the waitForFinished() method.

put()

This method sends data to the HTTP server using the HTTP PUT method. The data includes the file that was specified using the setAttachedFile() method and also parameters added using the addParameter() method. If the property localFilePath is specified, the server response is written to this file; otherwise the response can be obtained as a byte array using the getServerResponse() method.

The 'Content-Type' HTTP request header is automatically added to the request with the value "application/x-www-form-urlencoded" (which means the request contains HTML form input data).

theHTTP.put();

Note: This method is asynchronous. It means it does not wait till the request is finished. To wait for the request completion, a script writer has to use the waitForFinished() method.

get()

This property fetches the document using the HTTP GET method. If the property localFilePath is specified, the server response is written to this file; otherwise the response can be obtained as a byte array using the getServerResponse() method.

theHTTP.get();

Note: This method is asynchronous. It means it does not wait till the request is finished. To wait for the request completion, a script writer has to use the waitForFinished() method.

del()

This method is used to delete an object at the URL specified by using the HTTP DELETE method. If the property localFilePath is specified, the server response is written to this file; otherwise the response can be obtained as byte array using getServerResponse() method.

theHTTP.del();

Note: This method is asynchronous. It means it does not wait till the request is finished. To wait for the request completion, a script writer has to use the waitForFinished() method.

progress() : Number

Returns the progress (in percent) of the current request. This method should be used in combination with the waitForFinished() method.

waitForFinished( delay : Number ) : Boolean

This method is used to wait for the request completion. It is used after calling the post(), put(), get() or del() method.

The delay argument specifies the timeout in seconds that is the maximum time the waitForFinished() may block the script execution. Return value ‘true’ means the request has finished, otherwise ‘false’ is returned. Note that the method returns ‘true’ immediately after request has finished without waiting for the timeout to happen.

This method is convenient in combination with the progress() method to inform the user about the request progress.

job.log( 4, "Starting request...", 100 ); // max is 100%
while( !theHTTP.waitForFinished( 1 ) )
{
  job.log( 5, "Executing request", theHTTP.progress() );
}
  job.log( 6, "Finished executing request..." );

getServerResponse() : ByteArray

Returns the byte array containing the server response. This byte array can contain the content of the file or document being downloaded, JSON or XML data or whatever (he server has responded with.

Note: Avoid using this method when downloading large files because this may cause memory issues. In such cases, the localFilePath property should be used instead.

getHeaderNames() : String[]

Returns the array of all HTTP header names present in the server response.

var theNames = theHTTP.getHeaderNames();

getHeaderValue( headerName : String ) : ByteArray

Returns the byte array containing the HTTP header content. If the header is missing, the empty byte array is returned.

var theValue = theHTTP.getHeaderValue( HTTP.ContentDisposition );
job.log( 1, "Content-Disposition: %1", theValue.toString("ISO 8859-1") );
theValue = theHTTP.getHeaderValue( “Cache-Control” );
job.log( 1, "Cache-Control: %1", theValue.toString("ISO 8859-1") );

addHeader( headerName : String, headerValue : String )

This method can be used to specify additional headers for the HTTP request. This method should be used with caution. If there are invalid headers, HTTP requests may fail.

theHTTP.addHeader( ”x-metadata”, ”size=12345; path=/” );
theHTTP.addHeader( ”userlogin”, ”OK” );

resetHeaders()

Clears headers previously added by the addHeader() method.

theHTTP.resetHeaders();

interrupt()

Interrupts the current request to the server.

If this is GET request downloading some file, then such interrupted download may be resumed if the same GET request is executed later. To enable resuming, the resumeEnabled property must be set to true and tempFolderPath property must be defined.

theHTTP.interrupt();

Example

POST request to upload a file to Dropbox

function jobArrived( s : Switch, job : Job )
{
  var theHTTP = new HTTP( HTTP.SSL ); 

  theHTTP.authScheme = HTTP.OauthAuth;
  theHTTP.authorization = "authorization string";
  theHTTP.setAttachedFile( job.getPath() );
  theHTTP.url = "https://api-content.dropbox.com/1/files_put/auto/Test.pdf"; 

  theHTTP.post(); 

  job.log( 4, "Upload started", 100);
  while( !theHTTP.waitForFinished( 3 ) )
  {
  job.log( 5, "Uploading...", theHTTP.progress() );
  }
  job.log( 6, "Upload finished" ); 
  job.log( 1, "Server response: %1", theHTTP.getServerResponse().toString( "UTF-8" ) );
  if( theHTTP.finishedStatus == HTTP.Ok && theHTTP.statusCode == 200 )
  {
      job.log( 1, "Upload completed successfully" );
  }
  else
  {
      job.fail( "Upload failed with the status code %1", theHTTP.statusCode );
      return;
  }
  job.sendToSingle( job.getPath() );
}

GET request to download a file from Dropbox

function jobArrived( s : Switch, job : Job )
{
  var theHTTP = new HTTP( HTTP.SSL ); 

  theHTTP.authScheme = HTTP.OauthAuth;
  theHTTP.authorization = "authorization string";
  theHTTP.url = "https://api-content.dropbox.com/1/files/auto/Test.pdf";
  theHTTP.localFilePath = job.createPathWithExtension( "pdf", false ); 

  theHTTP.get(); 

  job.log( 4, "Download started", 100 );
  while( !theHTTP.waitForFinished( 3 ) )
  {
      job.log( 5, "Downloading...", theHTTP.progress() );
  }
  job.log( 6, "Download finished" ); 

  if( theHTTP.finishedStatus == HTTP.Ok && theHTTP.statusCode == 200
          && File.exists( theHTTP.localFilePath ) )
  {
      job.log( 1, "Download completed successfully" );
      job.sendToSingle( theHTTP.localFilePath );
  }
  else
  {
      job.fail( "Download failed with the status code %1", theHTTP.statusCode );
      return;
   }
   job.sendToNull( job.getPath() );
}

GET request to delete a file from Dropbox

function jobArrived( s : Switch, job : Job )
{
  var theHTTP = new HTTP( HTTP.SSL ); 

  theHTTP.authScheme = HTTP.OauthAuth;
  theHTTP.authorization = "authorization string";
  theHTTP.url = "https://api.dropbox.com/1/fileops/delete";
  theHTTP.addParameter( "root", "auto" );
  theHTTP.addParameter( "path", "Test.pdf" );

  theHTTP.get(); 

  while( !theHTTP.waitForFinished( 3 ) )
  {
        job.log( 1, "Delete in progress" );
  } 

  job.log( 1, "Server response: %1", theHTTP.getServerResponse().toString( "UTF-8" ) );

  if( theHTTP.finishedStatus == HTTP.Ok && theHTTP.statusCode == 200 )
  {
        job.log( 1, "Delete completed successfully" );
  }
  else
  {     job.fail( "Delete failed with the status code %1", theHTTP.statusCode );
        return;
  }
  job.sendToSingle( job.getPath() );
}