Bitten server performance problems: Bitten slave's repository check goes back through entire repository history

[fbrettschneider@…]

From the IRC discussion:

• [13:48] <me> ... whenever a bitten slave logs "No pending builds" on command line the server load is 100% for about 3 seconds. And therefore my Trac server is quite busy all time long since every such slave check is called about every minute and I have 30 slaves running
• [13:51] <me> When I exit all slaves, the server is not busy anymore. I wonder why such check for pending builds gives such server CPU load
• [13:53] <Hodgestar> When a slave connects Bitten checks the respository for changes. Maybe that's causing the load?
• [13:53] <Hodgestar> What are the 30 slaves for? (out of interest)
• [13:54] <me> 6 svn branches, each one having several platforms
• [14:01] <me> What happens on such repository check? Does it also need to have a look into myproject/log/bitten ? I noticed it contains about 85000 build log files, you know, my build number has grown to about 30000
• [14:03] <Hodgestar> Do you have build_all set to True, btw?
• [14:04] <me> yes
• [14:06] <Hodgestar> I think this causes Bitten to go back through the entire repository history each time. Not good. :(
• [14:07] <Hodgestar> Well, at least the entire history for the path the configuration applies to.
• [14:08] <Hodgestar> Bumping min_rev or setting build_all to False may provide temporary workarounds (and help confirm that my guess is correct).

...

• [14:19] <me> Bitten could internally hold a minimum counter it must go back on checking
• [14:22] <Hodgestar> The problem is that someone could in theory go far back into the history and invalidate a build.

...

• [14:32] <me> well now, testwise, I cut min_rev of all my branches to "current SVN revision minus 500". Let's see if it becomes faster
• [14:42] <me> Yes, now the task manager shows about the same number of CPU load peaks in the timing diagram, but it's just a subsecond anymore