Yesterday I recorded a session where I looked at alerts from an uplift. I did a lot of rambling and not a lot of content, but there are a few interesting differences between uplift alerts and normal alerts:
- a 6 week snapshot
- we try to match them up historically
- there should be pre-existing bugs
- a lot of time there are odd things
- we need to account for differences in build/configs between trunk and aurora/beta.
If you want to take a look at this, the link is on air.mozilla.com.
I do plan to do more episodes soon, a few topics of interest:
- understanding the noisy tests (bi-modal, noise)
- comparing talos on android to the new ported tests to autophone
- running talos locally and interpreting the results
- looking at intermittent failures in general- maybe with a focus on Talos issues.