Web Client
Contents
Introduction
Treck's HTTP client consists of a set of APIs for the user application to connect to a web server, send a request and retrieve the response from the web server. It supports both blocking and non-blocking modes of operation. For non-blocking mode, user can choose to use polling mode or event callbacks of the HTTP client.
Treck's HTTP client runs over TCP. It's designed to meet the requirement of Universal Plug and Play. Universal Plug and Play control points may invoke SOAP actions on a device's services and receive results or errors back. The actions, results and errors are encapsulated in SOAP, sent via HTTP requests, and received via HTTP responses. GENA also requires a HTTP client over TCP to connect to the control point's subscription URL and send event notifications.
Limitations
Receiving chunked encoding
Receiving of chunked encoding is supported. If the body is chunk encoded, it will be decoded. However, in Universal Plug and Play context, there is no chance for the HTTP client to receive a chunk-encoded body.
Low level APIs
The HTTP client API is at a low level that requires user to know about the structure of the HTTP request and response message, which normally consists of the method/status line, headers and the body. The user is required to build all the HTTP headers as part of the request message. The HTTP client API meets the requirements of Universal Plug and Play SOAP and GENA, since they both need a way to send and receive the exact methods/status code and headers.
This may not be desirable for users who simply wish to retrieve a HTTP resource at a given URL, and who do not want to deal with the details of the HTTP protocol. In that case, a higher level API could be built on top of the APIs descried in this document.
Single thread only
For a given HTTP client connection handle, these APIs should only be called from a single thread.
Connection management APIs
This group of APIs opens, connects to the HTTP server and closes the connection. It also allows the user to set options on the socket used by the connection.
- tfHttpcUserOpen()
- Allocate a HTTP client connection structure, create a TCP socket, store it in the client connection structure and return the connection handle to the user.
- tfHttpcUserConnect()
- Connect to a specific HTTP server.
- tfHttpcUserClose()
- Close the TCP connection to the HTTP server by closing the socket, and free the client connection structure.
- tfHttpcUserSetSockOpt()
- Enable the user to set socket options on the socket used by the HTTP client connection. This is a wrapper function for BSD setsockopt.
- tfHttpcUserSetSsl()
- Enable SSL on the HTTP client connection and specify the target server name.
- tfHttpcUserRegisterResponseHeader()
- Register interested HTTP headers. When response headers are received, the HTTP client saves the registered header types so that they can be retrieved by the user later on.
- tfHttpcUserSendRequestMethodLine()
- Send a HTTP method line to the HTTP server.
- Blocking Mode
- The user will be blocked until the method line has been sent out completely.
- Non-blocking Mode
- This API will only send what the socket send queue allows and returns immediately. It must be called multiple times with the same parameter inputs as long as it returns TM_EWOULDBLOCK.
- Blocking Mode
- Send a HTTP method line to the HTTP server.
- tfHttpcUserSendRequestHeaders()
- Send one or more headers to the HTTP server.
- Blocking Mode
- The user will be blocked until the headers have been sent out completely.
- Non-blocking Mode
- This API will only send what the socket send queue allows and returns immediately. It must be called multiple times with those unsent headers inputs as long as it returns TM_EWOULDBLOCK.
- Blocking Mode
- Send one or more headers to the HTTP server.
- tfHttpcUserSendRequestBody()
- Send a buffer of body to the HTTP server.
- Blocking Mode
- The user will be blocked until the given buffer has been sent out completely.
- Non-blocking Mode
- This API will only send what the socket send queue allows and returns immediately. User may call this API multiple times to send out the body until TM_ENOERROR is returned.
- Blocking Mode
- Send a buffer of body to the HTTP server.
- tfHttpcUserGetResponseStatusLine()
- Receive status line from the HTTP server. Parse the status line and pass the Http version, status code and reason phrase to the user.
- Blocking Mode
- The user will be blocked until the status line has been received, or timed out, or an error occurred.
- Non-blocking Mode
- This API must be called multiple times with the same parameter inputs as long as it returns TM_EWOULDBLOCK;
- Blocking Mode
- Receive status line from the HTTP server. Parse the status line and pass the Http version, status code and reason phrase to the user.
- tfHttpcUserGetResponseHeaders()
- Receive headers from the HTTP server. Parse the header, store the header value in an array of strings and pass the pointer of the array to the user.
- Blocking Mode
- The user will be blocked until all headers have been received, or timed out, or an error occurred.
- Non-blocking Mode
- This API must be called multiple times with the same parameter inputs as long as it returns TM_EWOULDBLOCK;
- Blocking Mode
- Receive headers from the HTTP server. Parse the header, store the header value in an array of strings and pass the pointer of the array to the user.
- tfHttpcUserGetResponseBody()
- Receive a buffer of body content from the HTTP server. Copy the received data into user provided buffer. This API may be called multiple times to receive the entire body for both blocking and non-blocking mode until the received data length is 0 and the returned error code is TM_ENOERROR, which indicates the end of the received response body.
APIs to store and retrieve the user data
These two APIs can be used together for the user to store and retrieve user data associated with a specific HTTP client connection.
- tfHttpcUserStoreVariable()
- To save a generic union variable on the HTTP client connection, so that it can be retrieved later by tfHttpcUserRetrieveVariable()
- tfHttpcUserRetrieveVariable()
- To retrieve the generic union variable on the HTTP client connection saved by tfHttpcUserStoreVariable(). If user hasn't called tfHttpcUserStoreVariable() before, a generic union with internal value set to 0 will be retrieved.
Special API for non-blocking mode only
- tfHttpcUserExecute()
- When using the HTTP client in event callbacks for non-blocking mode, this API must be called periodically to check the socket connection status, sending status and receiving status.
Authentication and Authorization
If a Web Client application tries to access a protected resource on the server without providing the appropriate authorization, the server will respond with a 401 Unauthorized status that can be determined from the tfHttpcUserGetResponseStatusLine() call. When returning a 401 response, the server must also supply one or more challenges via the WWW-Authenticate header, which can be retrieved after calling tfHttpcUserGetResponseHeaders(). The application must parse the WWW-Authenticate header and send a new request for the resource that includes an Authorization header with an appropriate response to the server's challenge. If the server accepts the client's response, the requested resource will be returned.
Parsing the server's challenge and generating a response can be, well, challenging. You will need to become familiar with RFC 2617, HTTP Authentication: Basic and Digest Access Authentication, to understand the remainder of this section.
There are two supported challenge types: Basic and Digest. The Basic challenge requires that you respond with a username and password, encoded as a Base64 string. The Digest challenge requires that you perform a sequence of cryptographic digests on specific data from you (e.g. username and password) and from the WWW-Authenticate header.
To assist the application, the following authentication helper functions are available:
- tfHttpcUserGetResponseAuthParams()
- Retrieve all the parameters from a WWW-Authenticate response header in the form of argv style arrays.
- tfHttpcUserBuildAuthResponse()
- Build Digest Authentication responses from input strings.
- tfHttpcUserFreeObject()
- Free the parameter structure returned from a successful tfHttpcUserGetResponseAuthParams() call.
Note: | Before calling tfHttpcUserGetResponseAuthParams() you must ensure two things. First, you must tell the Web Client to save the WWW-Authenticate header by calling tfHttpcUserRegisterResponseHeader() with header type TM_HTTP_HEADER_WWW_AUTHENTICATE or TM_HTTP_HEADER_ALL. Second, you must retrieve all response headers by calling tfHttpcUserGetResponseHeaders(). |
By default, the Treck Web Client supports both Basic and Digest Authentication. If you do not need one or both and wish to reduce the Treck code size, you can uncomment the associated configuration macro in your trsystem.h:
#define TM_DISABLE_HTTPC_BASIC_AUTH #define TM_DISABLE_HTTPC_DIGEST_AUTH
A simple example that responds to the first challenge from the server's WWW-Authenticate header:
. . . ttHttpcUserConHandle clientHandle; ttHttpcAuthArgVectorPtr argVecPtr; char authHdr[500]; char hashA1[TM_HTTPC_AUTH_DIGEST_ALLOC]; char hashA2[TM_HTTPC_AUTH_DIGEST_ALLOC]; char hashDigest[TM_HTTPC_AUTH_DIGEST_ALLOC]; char * digestInputs[6]; char * realmPtr; char * noncePtr; int respStatus; int errorCode; . . . /* Create the Web client */ clientHandle = tfHttpcUserOpen(...); assert(errorCode == TM_ENOERROR); . . . /* Make sure we save the WWW-Authenticate header from the server */ errorCode = tfHttpcUserRegisterResponseHeader(..., TM_HTTP_HEADER_WWW_AUTHENTICATE, ...); assert(errorCode == TM_ENOERROR); . . . /* Send a Web request */ errorCode = tfHttpcUserSendRequestMethodLine(...); assert(errorCode == TM_ENOERROR); errorCode = tfHttpcUserSendRequestHeaders(...); assert(errorCode == TM_ENOERROR); . . . /* Get the status */ errorCode = tfHttpcUserGetResponseStatusLine(..., &respStatus, ...); assert(errorCode == TM_ENOERROR); . . . /* Get the response headers */ errorCode = tfHttpcUserGetResponseHeaders(...); assert(errorCode == TM_ENOERROR); . . . /* Provide authorization, if required */ if (respStatus == 401) { . . . /* Get the WWW-Authenticate parameters */ errorCode = tfHttpcUserGetResponseAuthParams(clientHandle, &argVecPtr); assert(errorCode == TM_ENOERROR); assert(argVecPtr->argVecCount > 0); if (argVecPtr->argIdVecPtr[0] == TM_HTTPC_AUTH_PARAM_BASIC) { /* Format a Basic response */ sprintf(authHdr, "Basic %s", myBase64Encoder("username:password")); } else if (argVecPtr->argIdVecPtr[0] == TM_HTTPC_AUTH_PARAM_DIGEST) { /* Find the WWW-Authenticate parameter values (assuming they exist) */ realmPtr = argVecPtr->argValueVecPtr[myArgFinder(argVecPtr->argIdVecPtr, TM_HTTPC_AUTH_PARAM_REALM)]; noncePtr = argVecPtr->argValueVecPtr[myArgFinder(argVecPtr->argIdVecPtr, TM_HTTPC_AUTH_PARAM_NONCE)]; . . . /* Compute Digest H(A1) value */ digestInputs[0] = "username"; digestInputs[1] = realmPtr; digestInputs[2] = "password"; errorCode = tfHttpcUserBuildAuthResponse(clientHandle, hashA1, digestInputs, 3); assert(errorCode == TM_ENOERROR); /* Compute Digest H(A2) value */ digestInputs[0] = "GET"; digestInputs[1] = "/webpage.html"; errorCode = tfHttpcUserBuildAuthResponse(clientHandle, hashA2, digestInputs, 2); assert(errorCode == TM_ENOERROR); /* Compute Digest response */ digestInputs[0] = hashA1; digestInputs[1] = noncePtr; digestInputs[2] = "27"; /* Nonce count parameter */ digestInputs[3] = "abcd"; /* Client nonce value */ digestInputs[4] = "auth"; /* QOP selected value */ digestInputs[5] = hashA2; errorCode = tfHttpcUserBuildAuthResponse(clientHandle, hashDigest, digestInputs, 6); assert(errorCode == TM_ENOERROR); /* Format a Digest response */ sprintf(authHdr, "Digest realm=%s, username=%s, nonce=%s, uri=%s, response=%s, qop=%s, nc=%s, cnonce=%s", realmPtr, "username", noncePtr, "/webpage.html", hashDigest, "auth", "27", "abcd"); } . . . /* Free the WWW-Authenticate parameter structure */ tfHttpcUserFreeObject(argVecPtr); . . . /* Send the same Web request with TM_HTTP_HEADER_AUTHORIZATION set to authHdr */ errorCode = tfHttpcUserSendRequestMethodLine(...); assert(errorCode == TM_ENOERROR); errorCode = tfHttpcUserSendRequestHeaders(...); assert(errorCode == TM_ENOERROR); . . . /* Get the status */ errorCode = tfHttpcUserGetResponseStatusLine(..., &respStatus, ...); assert(errorCode == TM_ENOERROR); . . . } assert(respStatus == 200); . . .
Usage scenario
Figure 1: HTTP Client Usage Scenario
Function References
Example
HTTP client application
The example can be found in the "examples" subdirectory of the Treck TCP product distribution as <txhttpc.c>. This example contains the application for both blocking mode and non-blocking mode. It sends a request that contains method line, headers and body to a web server, and receives response from the web server.
Acronyms
Acronym | Meaning |
GENA | General Event Notification Architecture |
HTML | Hypertext Markup Language |
HTTP | Hypertext Transfer Protocol |
SOAP | Simple Object Access Protocol |
SWDS | Software Design Specification |
URL | Uniform Resource Locator |