Wrapping up a short series on capturing device network traffic, I'm happy to present mitmproxy-java, a small library which allows convenient access to network requests of devices made in the middle of your test runs. It has the following features which I was unable to find using other methods in Java:
- Starts the proxy server as a background task.
- Allows for passing in a lambda function which gets called for every intercepted message, allowing you to manage the recording of data any way you see fit.
- Allows for modifying the responses from the proxy, so you can inject arbitrary data and states into your app.
- Captures HTTPS traffic even when ip addresses are used instead of host names.
The first three advantages come from the wrapper code in mitmproxy-java which is basically a Java version of the great Node.js module I found for the same purpose: mitmproxy-node. The last bullet point comes from the use of mitmproxy.
Traditionally, the testing community mostly seems to use Browsermob Proxy, but I found it has not been maintained recently and can't support Android emulators due to the issue with HTTPS traffic and ip addresses. I'm hoping that people will be able to find mitmproxy-java as a suitable upgrade.
But please help! I put a lot of work into it but I'm not a Java expert. The way I currently handle exceptions isn't friendly. Hop onto github and submit pull requests or make issues if you run into trouble. If the community is supportive, we can improve it further.
Oh, this should work for Selenium too, if you set up the browsers to proxy correctly.
For those just tuning in, see the past two articles on capturing network traffic to learn about what we're doing and how it works:
- Capturing iOS Simulator Network Traffic with Appium
- Capturing Android Emulator Network Traffic with Appium
Those two articles also go through the setup needed for configuring devices, this post will focus on setting up mitmproxy-java and how to write the Java test code.
While mitmproxy-java will start the proxy server for us programmatically, we need to install mitmproxy ourselves, just like we did in the previous articles. Make sure to install with pip3 since installing with other methods, misses some python dependencies which we need.
When running mitmproxy-java, we need to supply it with the location of the mitmdump executable. mitmdump is installed automatically when you install mitmproxy and is a commandline version of mitmproxy which isn't interactive and runs in the background. Let's get that location and make a note of it for later.
For me the output is /usr/local/bin/mitmdump.
Next, we need to install the Python websockets module. the way mitmproxy-java works, is it starts mitmdump with a special Python plugin which is included inside the mitmproxy-java jar. This plugin runs inside mitmdump and connects to a websocket server hosted by mitmproxy-java. The Python code then transfers request/response data to the Java code over the websocket.
That should be all the setup we need on our host machine, now on to the actual test code.
Writing a Test Using mitmproxy-java
Include the mitmproxy-java jar in your project. The library is hosted on Maven Central Repository.
Add the following to your pom file:
Or in your build.gradle file, for Gradle users:
You can now access two classes in your test code: MitmproxyJava - The class which starts and stops the proxy. InterceptedMessage - A class used to represent messages intercepted by the proxy. A "message" includes both an HTTP request, and its matching response.
The constructor for MitmproxyJava takes two arguments. The first is a String with the path to the mitmdump executable on your computer. We got this value earlier in the setup section. The second argument is a lambda function which the MitmproxyJava instance will call every time it intercepts a network request. You can do anything you like with the InterceptedMessage passed in. In the following example, we create a List of InterceptedMessage objects and instantiate a new MitmproxyJava instance. every intercepted message gets added to our list, which is in scope for the rest of the test.
Notice that we return the message from the lambda function. If we forget to return it, no worries, this is the implicit behavior. If you block or throw an error though, then the message response never completes its journey to your test device.
You can also modify the response in the InterceptedMessage. Modifying m.responseHeaders and setting different bytes in the content of m.responseBody will result in overwriting the data which the device receives in response to its request.
Now that we've instantiated our MitmproxyJava object, all we need to do is call
to start the proxy server and start collecting responses. This method call runs in a separate thread. Call
to shut down.
The proxy, by default, runs on localhost:8080 just like in the examples from the previous articles. One future feature should be to allow configuration of this port.
Here's an example of an entire test for Android and iOS, using mitmproxy-java:
Full source code for this example can be found with all our example code on Github.