Cross-platform https server with non-blocking sockets. Part 2

This article is a continuation of the articles:
The easiest cross-platform server with ssl support
Cross-platform https server with non-blocking sockets
In these articles, I gradually from an unpretentious example that is part of OpenSSL try to make a full-fledged single-threaded web server.
In the previous article, I “taught” the server to accept the connection from one client and send back the html page with the request headers.
Today I will correct the server code so that it can handle connections from an arbitrary number of clients in one thread.

To begin, I will break the code into two files: serv.cpp and server.h
At the same time, the serv.cpp file will contain such a “highly integrated” code:

#include "server.h" int main() { server::CServer(); return 0; } 

Yes, you can kick me with your feet, but I still wrote, I write and I will write code in the header files if it is convenient for me. For this I actually love with ++, that it gives the freedom of choice, but this is a separate conversation ...

Go to the file server.h
In its beginning, I transferred all the headers, macros and definitions that were previously in serv.cpp, and added a couple more headings from STL:
')

 #ifndef _SERVER #define _SERVER #include <stdio.h> #include <stdlib.h> #include <memory.h> #include <errno.h> #include <sys/types.h> #ifndef WIN32 #include <unistd.h> #include <sys/socket.h> #include <netinet/in.h> #include <arpa/inet.h> #include <netdb.h> #else #include <io.h> #include <Winsock2.h> #pragma comment(lib, "ws2_32.lib") #endif #include <openssl/rsa.h> /* SSLeay stuff */ #include <openssl/crypto.h> #include <openssl/x509.h> #include <openssl/pem.h> #include <openssl/ssl.h> #include <openssl/err.h> #include <vector> #include <string> #include <sstream> #include <map> #include <memory> #ifdef WIN32 #define SET_NONBLOCK(socket) \ if (true) \ { \ DWORD dw = true; \ ioctlsocket(socket, FIONBIO, &dw); \ } #else #include <fcntl.h> #define SET_NONBLOCK(socket) \ if (fcntl( socket, F_SETFL, fcntl( socket, F_GETFL, 0 ) | O_NONBLOCK ) < 0) \ printf("error in fcntl errno=%i\n", errno); #define closesocket(socket) close(socket) #define Sleep(a) usleep(a*1000) #define SOCKET int #define INVALID_SOCKET -1 #endif /* define HOME to be dir for key and cert files... */ #define HOME "./" /* Make these what you want for cert & key files */ #define CERTF HOME "ca-cert.pem" #define KEYF HOME "ca-cert.pem" #define CHK_ERR(err,s) if ((err)==-1) { perror(s); exit(1); } 

Next, we first create the CServer and CClient classes inside the namespace server:

 using namespace std; namespace server { class CClient { //   SOCKET m_hSocket; //        vector<unsigned char> m_vRecvBuffer; //        vector<unsigned char> m_vSendBuffer; //    OpenSSL SSL_CTX* m_pSSLContext; SSL* m_pSSL; //       explicit CClient(const CClient &client) {} public: CClient(const SOCKET hSocket) : m_hSocket(hSocket), m_pSSL(NULL), m_pSSLContext(NULL) {} ~CClient() { if(m_hSocket != INVALID_SOCKET) closesocket(m_hSocket); if (m_pSSL) SSL_free (m_pSSL); if (m_pSSLContext) SSL_CTX_free (m_pSSLContext); } }; class CServer { //      map<SOCKET, shared_ptr<CClient> > m_mapClients; //       explicit CServer(const CServer &server) {} public: CServer() {} }; } #endif 

As you can see, this is just a blank for our server. Let's start slowly filling this blank with a code, most of which is already in the previous article .
For each client, its own SSL context is initiated; obviously, this should be done in the constructor of the CClient class.

  CClient(const SOCKET hSocket) : m_hSocket(hSocket), m_pSSL(NULL), m_pSSLContext(NULL) { #ifdef WIN32 const SSL_METHOD *meth = SSLv23_server_method(); #else SSL_METHOD *meth = SSLv23_server_method(); #endif m_pSSLContext = SSL_CTX_new (meth); if (!m_pSSLContext) ERR_print_errors_fp(stderr); if (SSL_CTX_use_certificate_file(m_pSSLContext, CERTF, SSL_FILETYPE_PEM) <= 0) ERR_print_errors_fp(stderr); if (SSL_CTX_use_PrivateKey_file(m_pSSLContext, KEYF, SSL_FILETYPE_PEM) <= 0) ERR_print_errors_fp(stderr); if (!SSL_CTX_check_private_key(m_pSSLContext)) fprintf(stderr,"Private key does not match the certificate public key\n"); } 

Library initialization, creation and binding of the listening socket will be transferred with minimal changes to the CServer constructor:

  CServer() { #ifdef WIN32 WSADATA wsaData; if ( WSAStartup( MAKEWORD( 2, 2 ), &wsaData ) != 0 ) { printf("Could not to find usable WinSock in WSAStartup\n"); return; } #endif SSL_load_error_strings(); SSLeay_add_ssl_algorithms(); /* ----------------------------------------------- */ /* Prepare TCP socket for receiving connections */ SOCKET listen_sd = socket (AF_INET, SOCK_STREAM, 0); CHK_ERR(listen_sd, "socket"); SET_NONBLOCK(listen_sd); struct sockaddr_in sa_serv; memset (&sa_serv, '\0', sizeof(sa_serv)); sa_serv.sin_family = AF_INET; sa_serv.sin_addr.s_addr = INADDR_ANY; sa_serv.sin_port = htons (1111); /* Server Port number */ int err = ::bind(listen_sd, (struct sockaddr*) &sa_serv, sizeof (sa_serv)); CHK_ERR(err, "bind"); /* Receive a TCP connection. */ err = listen (listen_sd, 5); CHK_ERR(err, "listen"); } 

Further in the same constructor, I propose to accept incoming TCP connections.
No one has so far brought me a single argument against, so we will listen to TCP connections in an infinite loop, as in the previous article.
After each call to accept, we can do something with the newly connected and already connected clients by calling the callback function.
Add the following code to the CServer constructor after the listen function:

 while(true) { Sleep(1); struct sockaddr_in sa_cli; size_t client_len = sizeof(sa_cli); #ifdef WIN32 const SOCKET sd = accept (listen_sd, (struct sockaddr*) &sa_cli, (int *)&client_len); #else const SOCKET sd = accept (listen_sd, (struct sockaddr*) &sa_cli, &client_len); #endif Callback(sd); } 

And immediately after the constructor, the actual callback function:

  private: void Callback(const SOCKET hSocket) { if (hSocket != INVALID_SOCKET) m_mapClients[hSocket] = shared_ptr<CClient>(new CClient(hSocket)); //   auto it = m_mapClients.begin(); while (it != m_mapClients.end()) //   { if (!it->second->Continue()) // -   m_mapClients.erase(it++); //   false,    else it++; } } 

On this code class CServer finished! All other application logic will be in the CClient class.
It is important to note that for speed-critical projects, instead of going through all the clients in the loop, you need to go through only those clients whose sockets are ready for reading or writing.
It's easy to do this using the select functions in Windows or epoll in Linux. I will show how this is done in the next article,
In the meantime (risking again to run into criticism) I’ll confine myself to a simple cycle.

Moving on to the main workhorse of our server: the CClient class.
The CClient class should keep in itself not only information about its socket, but also information about what stage its interaction with the server is at.
Add the following code to the CClient class definition:

  private: //    .     . enum STATES { S_ACCEPTED_TCP, S_ACCEPTED_SSL, S_READING, S_ALL_READED, S_WRITING, S_ALL_WRITED }; STATES m_stateCurrent; //    //      void SetState(const STATES state) {m_stateCurrent = state;} const STATES GetState() const {return m_stateCurrent;} public: //      const bool Continue() { if (m_hSocket == INVALID_SOCKET) return false; switch (GetState()) { case S_ACCEPTED_TCP: break; case S_ACCEPTED_SSL: break; case S_READING: break; case S_ALL_READED: break; case S_WRITING: break; case S_ALL_WRITED: break; default: return false; } return true; } 

Here, Continue () is so far only a stub function, just below we will teach her to perform all actions with the connected client.

In the constructor, change:

 CClient(const SOCKET hSocket) : m_hSocket(hSocket), m_pSSL(NULL), m_pSSLContext(NULL) 

on

 CClient(const SOCKET hSocket) : m_hSocket(hSocket), m_pSSL(NULL), m_pSSLContext(NULL), m_stateCurrent(S_ACCEPTED_TCP) 

Depending on the current state, the client calls different functions. We agree that client states can only be changed in the constructor and in the Continue () function, this will slightly increase the size of the code, but it will greatly facilitate its debugging.

So the first state that the client receives when created in the constructor: S_ACCEPTED_TCP.
Let's write a function that will be called by the client as long as it has this state:
For this line:

  case S_ACCEPTED_TCP: break; 

Change to the following:

  case S_ACCEPTED_TCP: { switch (AcceptSSL()) { case RET_READY: printf ("SSL connection using %s\n", SSL_get_cipher (m_pSSL)); SetState(S_ACCEPTED_SSL); break; case RET_ERROR: return false; } return true; } 

And also add the following code to the CClient class:

  private: enum RETCODES { RET_WAIT, RET_READY, RET_ERROR }; const RETCODES AcceptSSL() { if (!m_pSSLContext) //     SSL return RET_ERROR; if (!m_pSSL) { m_pSSL = SSL_new (m_pSSLContext); if (!m_pSSL) return RET_ERROR; SSL_set_fd (m_pSSL, m_hSocket); } const int err = SSL_accept (m_pSSL); const int nCode = SSL_get_error(m_pSSL, err); if ((nCode != SSL_ERROR_WANT_READ) && (nCode != SSL_ERROR_WANT_WRITE)) return RET_READY; return RET_WAIT; } 

Now the AcceptSSL () function will be called by the client until an encrypted connection occurs or an error occurs.

1. In case of an error, the CClient :: AcceptSSL () function returns the RET_ERROR code in its calling CClient :: Continue () function, which in this case returns false the CServer :: Callback function that caused it, which in this case will remove the client from the server memory.
2. In case of a successful connection, the CClient :: AcceptSSL () function will return the RET_READY code to the CClient :: Continue () function that called it, which in this case will change the client state to S_ACCEPTED_SSL.

Now add the state processing function S_ACCEPTED_SSL. For this line

 case S_ACCEPTED_SSL: break; 

fix for the following:

  case S_ACCEPTED_SSL: { switch (GetSertificate()) { case RET_READY: SetState(S_READING); break; case RET_ERROR: return false; } return true; } 

And add a function to CClient:

  const RETCODES GetSertificate() { if (!m_pSSLContext || !m_pSSL) //     SSL return RET_ERROR; /* Get client's certificate (note: beware of dynamic allocation) - opt */ X509* client_cert = SSL_get_peer_certificate (m_pSSL); if (client_cert != NULL) { printf ("Client certificate:\n"); char* str = X509_NAME_oneline (X509_get_subject_name (client_cert), 0, 0); if (!str) return RET_ERROR; printf ("\t subject: %s\n", str); OPENSSL_free (str); str = X509_NAME_oneline (X509_get_issuer_name (client_cert), 0, 0); if (!str) return RET_ERROR; printf ("\t issuer: %s\n", str); OPENSSL_free (str); /* We could do all sorts of certificate verification stuff here before deallocating the certificate. */ X509_free (client_cert); } else printf ("Client does not have certificate.\n"); return RET_READY; } 

This function, unlike the previous one, will be called only once and will return either RET_ERROR or RET_READY to CClient :: Continue. Accordingly, CClient :: Continue will return either false, or it will change the client state to S_READING.

Then everything is the same: change the code

  case S_READING: break; case S_ALL_READED: break; case S_WRITING: break; 

on this:

  case S_READING: { switch (ContinueRead()) { case RET_READY: SetState(S_ALL_READED); break; case RET_ERROR: return false; } return true; } case S_ALL_READED: { switch (InitRead()) { case RET_READY: SetState(S_WRITING); break; case RET_ERROR: return false; } return true; } case S_WRITING: { switch (ContinueWrite()) { case RET_READY: SetState(S_ALL_WRITED); break; case RET_ERROR: return false; } return true; } 

And add the appropriate state processing functions:

  const RETCODES ContinueRead() { if (!m_pSSLContext || !m_pSSL) //     SSL return RET_ERROR; unsigned char szBuffer[4096]; const int err = SSL_read (m_pSSL, szBuffer, 4096); //      if (err > 0) { //     m_vRecvBuffer m_vRecvBuffer.resize(m_vRecvBuffer.size()+err); memcpy(&m_vRecvBuffer[m_vRecvBuffer.size()-err], szBuffer, err); //  http     const std::string strInputString((const char *)&m_vRecvBuffer[0]); if (strInputString.find("\r\n\r\n") != -1) return RET_READY; return RET_WAIT; } const int nCode = SSL_get_error(m_pSSL, err); if ((nCode != SSL_ERROR_WANT_READ) && (nCode != SSL_ERROR_WANT_WRITE)) return RET_ERROR; return RET_WAIT; } const RETCODES InitRead() { if (!m_pSSLContext || !m_pSSL) //     SSL return RET_ERROR; //      const std::string strInputString((const char *)&m_vRecvBuffer[0]); // html     const std::string strHTML = "<html><body><h2>Hello! Your HTTP headers is:</h2><br><pre>" + strInputString.substr(0, strInputString.find("\r\n\r\n")) + "</pre></body></html>"; //    http  std::ostringstream strStream; strStream << "HTTP/1.1 200 OK\r\n" << "Content-Type: text/html; charset=utf-8\r\n" << "Content-Length: " << strHTML.length() << "\r\n" << "\r\n" << strHTML.c_str(); // ,    m_vSendBuffer.resize(strStream.str().length()); memcpy(&m_vSendBuffer[0], strStream.str().c_str(), strStream.str().length()); return RET_READY; } const RETCODES ContinueWrite() { if (!m_pSSLContext || !m_pSSL) //     SSL return RET_ERROR; int err = SSL_write (m_pSSL, &m_vSendBuffer[0], m_vSendBuffer.size()); if (err > 0) { //    ,      if (err == m_vSendBuffer.size()) return RET_READY; //    ,      ,     vector<unsigned char> vTemp(m_vSendBuffer.size()-err); memcpy(&vTemp[0], &m_vSendBuffer[err], m_vSendBuffer.size()-err); m_vSendBuffer = vTemp; return RET_WAIT; } const int nCode = SSL_get_error(m_pSSL, err); if ((nCode != SSL_ERROR_WANT_READ) && (nCode != SSL_ERROR_WANT_WRITE)) return RET_ERROR; return RET_WAIT; } 

Our server is only designed to show the client the headers of its http request.
After the server has fulfilled its purpose, it can close the connection and forget about the client.
Therefore, it remains to make the last small change in our code:

 case S_ALL_WRITED: break; 

need to fix on

  case S_ALL_WRITED: return false; 

That's all! Now we have a cross-platform single-threaded https server on non-blocking sockets, which can handle an arbitrary (limited only by memory and operating system settings) number of connections.

The archive with the project for Visual Studio 2012 can be downloaded here: 01.3s3s.org
To compile to Linux, you need to copy the following files into one directory: serv.cpp, server.h, ca-cert.pem and type in the command line: “g ++ -std = c ++ 0x -L / usr / lib -lssl -lcrypto serv .cpp

Continuation