Making the Mozilla automation infrastructure run reliably for each checkin on mobile devices has been my primary focus for the last few years. Last year at this time we were just trying to get Android automation up and running, all tools and harnesses had been written and ready to run. The core buildbot code for running the tests was in place. The problem was that we just had so many failures of the devices (NVidia Tegra development boards) and the tests.
So as the months went on from last December and up through August, we really made little progress. A few tests were fixed, some disabled, some checks in place to make the boards stay online, but really no consistent set of test results.
There were a couple things that fixed our problems:
1) a rock star intern (:jchen) who found and fixed some workarounds with the OS so fennec wouldn’t crash all the time (issues with networking and libc).
2) a weekly meeting started by :blassey to go over all the bugs, status, issues, future work, and other items.
Both of these items are signs that the mobile development team was serious about testing and wanting to see Android unittests become a part of Mozilla. While this seems trivial, it was next to impossible to keep tests running smoothly without support from the entire team.
Enjoy the reliable unit tests on Android!