Logging over HTTP websockets

I was recently looking for proven ways to do logging in C++ and read Petru Marginean’s article Logging in C++. I wanted a log that told me where a message came from and at what level it was logged. The article is rather old, but presents a good way to organize logging. My problem was that with a timestamp and bunch of additional information, the log message starts to become quite big and it makes for a messy text file. I figured that a great way to view generated text data is the browser. Then I went a bit overboard.

I. Overview

The boring way to do this would be to just generate an .html file instead of a simple log text file. However my application would have benefited a lot from live logging, so I decided to embed a lightweight web server into the logging system and send log messages over a WebSocket to a local .html page opened in a browser. The log messages are received by JavaScript as JSON objects containing information such as a time stamp, the actual log message, as well as the file and line it came from. I also required Lua script log messages to be supported by the system, so I used the standard Lua debug library to extract the same kind of information for Lua scripts.

II. Logging

If you haven’t already, I suggest you go and read Logging in C++ for a thorough understanding of why this design was chosen. The article goes over developing it step by step and makes a good case for each part of the design. It is very simple and it easily lets you integrate it into existing code bases. Mashing together the interfaces of the web server and the logging system went a lot smoother than I expected when I was writing this.

I have chosen to support the following log levels (in order of importance):


I felt like these suited my needs, but you can of course choose other ones (the joy of writing things from scratch). Note that this solution only supports a certain set of compile-time defined log levels (in other words, a simple enum) and you can’t dynamically add new log levels at runtime.

A nice feature to have during debugging is to be able to define a global log level that stops low priority messages from being logged. For this purpose, we can let the user #define a GLOBAL_LOG_LEVEL. In case they don’t, we’ll go ahead and define it to be some arbitrary reasonable level.


We check whether the message level is higher than this global level in the core macro of the logging system. It compares the levels and, if the message should be logged, branches to an output string stream, to which we can write the message.

#define LOG(level,type) if (level > GLOBAL_LOG_LEVEL) ; \
else Log<type>(level,__LINE__,__FILE__,__FUNCTION__).getStream()

The template<class T> Log type is a short-lived temporary object that is used to assemble the message.

template<class T>
class Log
Log(LogLevel Level, unsigned int Line, const char* File, const char* Function);

std::ostringstream& getStream();
static LogLevel& reportingLevel();
static std::string logLevelAsString(LogLevel level);

std::ostringstream os;

unsigned int line;
std::string file;
std::string function;
LogLevel messageLevel;
std::chrono::high_resolution_clock::time_point logTime;

inline std::string getTime();

Log(const Log&);
Log& operator=(const Log&);

A Log object holds all of the information about a log message. The public constructor simply initializes its data members. I should point out one detail of my implementation – I am using c++11. Or at least Visual Studio 2012’s interpretation of it. For example, std::chrono is part of <chrono>, a new header in the 2011 standard. I, for one, really like the new standard and hope more of it gets implemented soon.

The interesting part of the object is std::ostringstream os; (which is what .getStream() returns in the macro from before). ostringstream, istringstream, and its combined version stringstream are among the great little pocket-knife types in the standard library. ostringstream is an ostream that outputs to a string. For comparison, the cout object’s type is a descendant of ostream. This means that we get free access to the familiar and juicy << operator (or if you want to go more hardcore, there’s the write method). By returning a reference to our string stream we allow the caller to simply compound the message using <<, in similar fashion to standard console output.

The macro I mentioned earlier would essentially expand to the following (curly brackets added for additional drama):

if (level > GLOBAL_LOG_LEVEL)
Log<type>(level,__LINE__,__FILE__,__FUNCTION__).getStream() // rest of expression that follows after the macro

The destructor of Log is the one that does the actual logging. Once the expression the macro was used in finishes being evaluated, the Log object we created will be destroyed as we leave the else block. Here’s what the destructor looks like:

template<class T>

If you haven’t noticed so far, Log is a template class. The template argument has to be a type that implements the following monstrosity of a method in order to handle an incoming log message:

static void logMessage(std::chrono::high_resolution_clock::time_point logTime, LogLevel level, std::string file, unsigned int line, std::string function, std::string message);

Again, we see the same set of information as before being passed along. The idea is that we can extend the system by implementing different strategies to handle logging messages. The logMessage method is static, so we are limited to strategies that can be implemented in terms of static or singleton objects.

Note: This certainly is true if we look at this code as a standalone library. In practical usage however, we have control over the lifetime of such objects, so we can simply handle the initialization of a certain strategy early in the execution of the program before any other systems might generate log messages. As you may have guessed from my usage of the word “strategy”, this implementation is a simple version of the strategy design pattern implemented using templates instead of a type hierarchy.

A benefit of this logging system design is that with a few macros it can become very easy and intuitive to use. We can define a macro for every strategy we have implemented and then use it similarly to cout.

// WebsocketLog is the class that implements the web server strategy
#define WLOG(level) Log<WebsocketLog>(level,__LINE__,__FILE__,__FUNCTION__).getStream()
// Note that I didn't bother checking against the global level with the if statement trick from the previous macro. We can use the default macro or define a new one with separate global logging levels. If you have more than one logging output stream you can compound different logging strategies together either with a macro or with a compound strategy type.

// And then logging a message using the WebsocketLog strategy:
WLOG(LOG_ERROR) << "Error message!";

Log page with a single error message

I think at this point it is fairly trivial to implement a strategy which simply outputs to a file, or even to a different file depending on level, so I’ll end the discussion on the logging system here. You can look at the WebsocketLog section to see what one example implementation looks like.

III. Embedding a web server

I started writing my own web server, since in theory I don’t need full compliance with some huge standard, but just to answer a few basic requests and handle a handful of file handles (say that 5 times quickly). I soon realized that that is a whole load of low level data juggling code that I could do without in my life and, naturally, turned to the internet. I found Mongoose (RED: find link) – a lightweight (like – one .c and one .h file ^light^, as well as memory wise light) web server designed for embedding with a C interface. I can’t recommend it enough: it is extremely easy to setup and has a very low learning curve. There is a short list of methods you can call on the flat structure that represents the web server. All external interaction with the server is exposed through callbacks. It also does not make any assumptions about the data you will be sending/receiving. It will not try to read any files, but rather just invoke the callback you defined with the requested URL as an argument and let you write to an output stream.

Note: I have to say that despite the fact that Mongoose fits my needs perfectly and it’s overall a great library, I had some minor problems with it. I was trying to keep track of who is connecting to the server and needed to store the information in the connection structure in the WebsocketServer.I found that some of the details I needed were stored in a structure that is only forward declared in the header file, I would assume because it’s only supposed to be passed around as a pointer, not actually inspected by the user of the library. Basically, when you start the Mongoose server, you can give it a data pointer which it stores internally. Unfortunately, this data pointer is only passed directly to some callbacks and the ones I needed weren’t among them. They did however have a pointer to a Mongoose connection structure. Since connections are tied to a certain server instance they keep track of its address internally. Since, however, the structure was only forward declared in the header file to prevent tampering, I did not have visibility over the server pointer, so I had to write myself a couple of helper functions within Mongoose.c to extract the associated user data from Mongoose’s structures.

Let’s look at the WebsocketLog class, which implements the discussed strategy:

class WebsocketLog
static WebsocketServer*& webserver();

static void logMessage(std::chrono::high_resolution_clock::time_point logTime,
LogLevel level, std::string file, unsigned int line, std::string function, std::string message)
Json::Value jsonMessage;
jsonMessage["type"] = "log";
jsonMessage["logLine"] = Json::Value(Json::arrayValue);
jsonMessage["logLine"][0] = getTime(logTime);
jsonMessage["logLine"][1] = Log<WebsocketLog>::logLevelAsString(level);
jsonMessage["logLine"][2] = message;
jsonMessage["logLine"][3] = file;
jsonMessage["logLine"][4] = line;

The static webserver method allows us to use an external singleton WebsocketServer object. Since this object manages the connections to the clients it makes sense to give control over its lifetime to the user of the library. This is probably the most important thing to keep track of when doing low level memory manipulation – keeping track of who “owns” memory (along with rigorous use of the const keyword * I once decided to completely specify a basic one argument method according to its behaviour as far as modifying objects goes – I had to use const 5 times in that one function declaration). logMessage is the main method the strategy type has to implement. In this case I build a JSON message with the callback’s arguments and broadcast it to all connected clients through the webserver. Speaking of…

class WebsocketServer
void broadcastMessage(const Json::Value& message);
bool acceptNewMessages;
MTQueue<Json::Value> messageQueue;
struct mg_context* mongooseContext;
pthread_mutex_t mutex;
std::vector<struct mg_connection*> clients;
pthread_t thread;
pthread_t pingThread;
pthread_mutex_t pingMutex;
pthread_cond_t pingCond;

friend void* websocketServerThread(void* websocketServerPtr);
friend void* websocketPingThread(void* websocketServerPtr);
friend void websocketReadyHandler(struct mg_connection* connection);
friend int websocketDataHandler(struct mg_connection* connection,
int flags, char* data, unsigned int dataLength);
friend void websocketClosedHandler(struct mg_connection* connection);

Aside from a simple public interface, the WebsocketServer only contains a small amount of state information and bunch of friend callback declarations for thread main functions and Mongoose callbacks.

Initializing Mongoose is fairly straight-forward:


const char* options[] =
"listening_ports", "8080",

struct mg_callbacks callbacks;
callbacks.websocket_ready = websocketReadyHandler;
callbacks.websocket_data = websocketDataHandler;
callbacks.connection_closed = websocketClosedHandler;

mongooseContext = mg_start(&callbacks,this,options);

After initializing some thread synchronization objects I prepare the structures needed to build the main Mongoose object – a context structure. The “websocketServerThread” thread is the main server logic that handles broadcasting messages and keeps track of clients. The “websocketPing” one is actually there just to broadcast a pulse message every 5 seconds to prevent clients’ browsers from closing the websocket connection due to innactivity. The destructor will then go on to clean up by calling mg_stop() on the Mongoose context and join the spawned threads into the main one , but not before broadcasting a final “stop” message to the clients. This helps clean up the opened websockets on the Javascript side.

Broadcasting a message is done by simply pushing a JSON object to a queue. websocketServerThread monitors this queue for new messages and broadcasts them as soon as possible. I am using the JsonCpp library. As you may have noticed in the WebsocketLog::logMessage above, building messages with it is very intuitive. Parsing data into objects is just as easy, so overall – a pretty good library.

I check every outgoing message for a type, so that I can gracefully close when the user broadcasts a “stop” message, as well as handle the already mentioned pulse messages. There is actually a way to simply ping a websocket with the purpose of keeping it alive. When writing to the websocket you can specify an opcode that describes what the data is. Regular messages are WEBSOCKET_OPCODE_TEXT in Mongoose’s language, but you can also send blank WEBSOCKET_OPCODE_PING messages. Here’s the interesting case, when we are actually going to be sending out data:

Json::FastWriter writer;
string messageSerialized = writer.write(message);
for (auto client : ws->clients)

The JsonCpp library provides different writers like a human readable one, or the FastWriter I used, which compacts the generated textual representation of an object. Note that Mongoose is not thread safe, so we have to manually protect it with a mutex. JSON objects are also used when processing incoming messages. Whenever a message is received, all I do is check if it’s a “close” message. For my needs I didn’t need any other messages from the client to the application, so this was enough. The websocketDataHandler is the callback that Mongoose calls whenever a chunk of data has arrived. I parse it into a JSON object and return 0 or 1 to tell the server if it should keep the socket opened. You could easily integrate more functionality at this point to let the application expose some sort of REST or JavaScript API for further debugging or automated logging.

int websocketDataHandler(struct mg_connection* connection,
int flags, char* data, unsigned int dataLength)
Json::Value message;
Json::Reader reader;
if (message.get("type","n/a").asString().compare("close") == 0)
return 0;
return 1;

IV. Client side

As already mentioned, I used a single .html page stored locally for the interface. In particular, I used jQuery along with the dataTables plugin to create a simple interface that allows you to connect to different host applications and view the log they are producing in real time. I have to say jQuery made it incredibly easy to put the page together. I handle creating and talking over websockets, continuous reconnection attempts (it automatically will reconnect to the address+port it last used as soon as an application becomes responsive at it), as well as searching and filtering the messages displayed and saving log sessions to cookies. All of this is implemented in about 250 lines of JavaScript on top of 40 lines of HTML. Be warned – client side development is not my expertise. I was learning about websockets as I was writing this and outside of the warm type safe environment of C++, JavaScript does not offer much common sense protection, so the code is far from perfect, but it gets the job done.

Most of the code resides within the JavaScript object that the dataTables plugin uses to represent the table. Since JavaScript does not have a notion of types, I use it to store any methods I need to manipulate the data and then have the callbacks actually do a lot of the work for me.

Let’s look at connecting to the application through a WebSocket:

var tryConnect = function()
var serverURL = $("#serverURL").val();
if (serverURL == "") throw "Server URL cannot be empty!";
ws = new WebSocket("ws://"+$("#serverURL").val());
ws.onopen = function()
$("#connectButton").button("option",{label: "Disconnect",icons: {primary: 'ui-icon-circle-check'}})
if ($("#clearOnConnect").is(":checked"))
connected = true;
ws.onmessage = function (evt)
if (typeof(evt.data) == "string" && evt.data.length != 0)
var message = JSON.parse(evt.data);
if (message["type"] == "log")
ws.onclose = function()
$("#connectButton").button("option",{label: "Connect",icons: {primary: 'ui-icon-circle-close'}})
connected = false;
if ($("#autoReconnect").is(":checked") && !manualDisconnect && !stopWaiting)
timeout = window.setTimeout(autoReconnectFunction,5000);
$("#connectButton").button("option",{label: "Waiting",icons: {primary: 'ui-icon-refresh'}});
waiting = true;
manualDisconnect = false;

var autoReconnectFunction = function()
try { tryConnect(); } catch (e) { /* ignore errors */ }

These two functions are pretty much all you need to receive log messages. tryConnect simply initializes a WebSocket to the address and sets the appropriate callbacks for handling the connection. Upon connecting we check if the user wants us to clear the log messages already in the table or just append. When receiving a message we check if it is a text string and parse into a JSON object that goes straight into the table as a record. If the connection is closed for some reason we do some clean up work and change the UI to reflect the current connection status. The autoReconnectFunction is put on a timer and periodically calls tryConnect if the user has checked “Auto Reconnect”.

All of the fancier buttons for viewing the logs are implemented as direct uses of functionality already provided by the data table. With the appropriate filtering and searching functions the table can understand the data it’s showing and offer some neat features out of the box.

Finally, we use a cookie to store some data on the client. This allows the user to save his settings (the state of all the toggle buttons) and save the last session of logs so that they are reopened the next time he uses the page.

LiveLoggingOverHTTP_image1All of that probably constitutes about half of the code, with the rest being mostly boring UI handling stuff. Also I came up with the colors before considering a large screen of text glaring at you, so that could use some work.

V. Lua log messages

In Lua, we would want to simply register a function that will use the WLOG macro to log a string message. Lua offers debug functions that let you inspect the current state of execution.

lua_Debug activationRecord;
Log<WebsocketLog> log(l,activationRecord.currentline,activationRecord.source,activationRecord.name);

lud_Debug is the main structure we’re interested in. The two Lua functions populate it with information about the stack and current location within the code. Then I just check all the arguments passed to the function and convert any appropriate ones to strings that get appended to WLOG’s stream.

This minimum amount of work lets me use the logging system from within Lua seamlessly. In fact Lua’s debug library offers even more features than you get from the native environment like execution stack and data stack inspection, which can all be integrated into the log message with a few lines of codes if required.

VI. Possible uses

I developed this mostly because I needed a “fancy” logging system for native C++ and did not want to get weighted down by existing ones (some of which would have literally brought more code into the project than it itself had). The main feature I was looking for was monitoring logs in real time and I felt like cout’ing and printf’ing things was a bit too limiting.
The solution I propose could even be used to show others problems with a piece of code without requiring them to build and run your version of it. And you could even take the idea further. Considering the current state of web-development technologies (both front and back-end wise) creating a simple server that keeps track of logs from individual machines and can display them to programmers during the development stages of a project isn’t going to be too difficult to create. Heck, if you tell it where the source files are located, it can even provide links to specific resources, even cross-reference logs with old versions from source control. I remember an article on AltDevBlogADay that talked about integrating links to objects into your toolset so that when team members talk they don’t have to refer to “that one object over in x file has a problem with y texture”, but can just copy paste links executed by a protocol handler on the dev system. How cool would it be to send someone a link to a live session of your running application, so that he can view the logs as they happen. Once the application closes, the server can save the session and then just save a full transcript if someone requests the same link afterwards. For something like game development you can even implement a log screen method that attaches a screenshot, which can readily be viewed by a browser.

I have provided my implementation of the discussed logging method on my github page:


Article permalink: http://mtsvetkov.eu/logging-over-http-websockets/