Whether to inspect a server response, optimize network usage or just to fiddle with new REST API it almost always make sense to use an application that can display http requests and responses for investigation. On Windows there is the great Fiddler. While it’s possible to use Fiddler on other platforms using virtualization software I think it’s an overkill. Fortunately there are alternatives. Charles looks like the most advanced one offering most if not all features available in Fiddler. There is also a little less popular HTTP Scoop. Both Charles and HTTP Scoop aren’t free but in my opinion they are worth the price especially if used often. Command line lovers might find mitmproxy suit their needs. If you only need basic features ngrok might serve you well. To dive a bit deeper and see http traffic on a tcp level WireShark is indispensable.
As you can see there are plenty of tools available to help understand what is happening on http level. As I was learning about http caching a question popped to my head. How hard would it be to actually build a simple http sniffer?
proxy-mirror – a simple http inspector
As it turned out it’s actually not that hard to built one thus proxy-mirror came to be. Of course this is a very simplistic http sniffer – nowhere near to tools I mentioned above both
in terms of features and reliability – but it works (at least in most scenarios :-)). It’s open source and I learned couple of new things about HTTP while implementing it – more on that in future posts. Here are some screen shoots of the the tool:
As you might have guessed from screenshots proxy-mirror is a web application. Right now it’s not very easy to try it out – the instructions are in the Readme – I’ll try to fix it soon.
How it works
I wanted to run proxy-mirror on many platforms and rely on a foundation that has both great support for http and building web applications. I picked node.js over java or ruby mainly because I wanted to sharpen my skills in it.
You can think of proxy-mirror as 2 logical components. The first is a regular http proxy that you can use by configuring your browser or system. I didn’t want to focus on building that first so I utilized a great node module call http-proxy. With it’s helped you can have a system wide proxy in couple of minutes. The http proxy component emits events whenever a request or response goes through it. Those events are consumed by the second logical component – a web application built with express.js. The information about http traffic received from events is then pushed to browser part through socket.io where a simple SPA built with my beloved AngularJS displays information about them.
Features
Right now proxy-mirror has only couple of features:
- http and https support – although the latter one requires additional setup
- simple session list – a grid with list of request/response pairs that you can inspect
- a detailed view – both for request and response that can display headers, message body and a preview right in the browser
Although it still is a very simple (and buggy) application I actually found that it has most of features I frequently use – probably except for filtering. Maybe someday it will become a reasonable alternative for Charles - at least for me.
I think building an http sniffer is a great exercise in the process of learning how http(s) works. I guess the famous saying about journey being more more important than the destination fits nice here.