As many of you know there have been numerous times lately that Tinderbox has become unresponsive, sometimes to the point of going down completely for a period of time. This post will attempt to summarize the issues and what's being done about them. The biggest issue is load (surprise!). In a period of a few years we've gone from a few active trees with tens of columns between them to tens of active trees with hundreds of columns between them. Unsurprisingly, this has made the Tinderbox server a lot busier. The biggest load items are:

A bit of profiling has also been done in bug 585814 to try to find specific hotspots. We've already done a few things to help with Tinderbox load: Other ways we're looking at improving the situation: