Performance Alerts – by the numbers

If you have ever received an automated mail about a performance regression, and then 10 more, you probably are frustrated by the volume of alerts.  6 months ago, I started looking at the alerts and filing bugs, and 10 weeks ago a little tool was written to help out.

What have I seen in 10 weeks:

1926 alerts on mozilla.dev.tree-management for Talos resulting in 58 bugs filed (or 1 bug/33 alerts):

*keep in mind that many alerts are improvements, as well as duplicated between trees and pgo/nonpgo

 

Now for some numbers as we uplift.  How are we doing from one release to another?  Are we regressing, Improving?  These are all questions I would like to answer in the coming weeks.

Firefox 30 uplift, m-c -> Aurora:

• 26 – regressions (4 TART, 4 SVG, 3 TS, Paint, and many more)
  • 2 remaining bugs not resolved as we are now on Beta (bug 990183, bug 990194)

• 6 – improvements (CART, Windows 8 TPaint)

 

Firefox 31 uplift, m-c -> Aurora (tracking bug 990085):

• 23 – regressions (5 CART, 3 TART, 2 SVG, 2 TP5, 2 TResize, and a few others)
  • CART regressions are mostly fixed already!
• 10 – improvements (Kraken/V8, Linux64 SVG, OSX 10.8 TART)

 

Is this useful information?

Are there questions you would rather I answer with this data?