bitten-master dies, subversion repository in bad shape

[stava@…]

Get a lot of these:

Traceback (most recent call last):

File "/usr/bin/bitten-master", line 7, in ?

sys.exit(

File "build/bdist.linux-i686/egg/bitten/master.py", line 437, in main File "build/bdist.linux-i686/egg/bitten/util/beep.py", line 149, in run File "build/bdist.linux-i686/egg/bitten/master.py", line 61, in _enqueue_builds File "build/bdist.linux-i686/egg/bitten/queue.py", line 152, in populate File "/usr/lib/python2.4/site-packages/trac/env.py", line 162, in get_repository

repos = Subversion Repository?(repos_dir, authz, self.log)

File "/usr/lib/python2.4/site-packages/trac/versioncontrol/svn_fs.py", line 192, in init

self.repos = repos.svn_repos_open(self.path, self.pool())

File "/usr/lib/python2.4/site-packages/libsvn/repos.py", line 47, in svn_repos_open

return apply(_repos.svn_repos_open, args)

libsvn._core.Subversion Exception?: ('Berkeley DB error for filesystem /var/svn/addons/db while opening environment:\nDB_RUNRECOVERY: Fatal error, run database recovery', 160029) Exception exceptions.Attribute Error?: "'Subversion Repository?' object has no attribute 'log'" in <bound method Subversion Repository?.del of <trac.versioncontrol.svn_fs.Subversion Repository? object at 0xb77334ec>> ignored

After that the subversion repository wont work without recovery. This problem seems unrelated to any builds going on. It seems to happen in the polling sequence. When running bitten-master in one session and a bitten-slave in another, no builds to be produced, it will still bail out after a while, sometimes 1 minute, sometimes 20 minutes.

We're using Trac 0.9.6, Bitten 0.5.3, and subversion 1.3.1.

[andreas.jabs@…]

are you sure this is bittens fault? This looks more like a problem in the subversion wrapper of trac/python...

Is /var/svn mounted from somewhere? You know you should not access berkeley-db repositories via nfs/smb, because of locks. If so, try FSFS, this repos type can lock even on network filesystems. you need to dump & load to convert a repos, refer to subversion manuals. dont know if it helps, but i don't have such problems here. (bitten trunk, trac trunk, svn 1.24)

[stava@…]

/var/svn is native, i.e. not mounted over nfs. I'll try to do some more tests and get back to you.

[stava@…]

You're right (I think), it would seem that my Trac is unstable, the swig library bails out on occasion. Doesn't seem to have anything to do with bitten as such, so you should close this ticket.

Thanks for your help anyways. /Lars

[stava@…]

Yup, all testing done, Bitten works just fine after upgrading to subversion 1.4.0. I have no idea what went on there, but now both my Trac and Bitten are stable and runs like a charm.

It would seem that I'm not allowed to close this ticket, so I guess someone else has to.

Thanks/Lars?

[immanuel@…]

The problem might be related to file access rights.

When I began to use bitten, it sometimes took only minutes before the svn repository needed a recovery (even when there were no pending builds).

The reason was, that my svn repository had the owner "wwwrun", because it used to be accessed only by apache. I started bitten-master as root, and it accesses the svn repository via the "file:///" method. If locks are created during the access of bitten-master, these locks cannot be removed anymore by the user "wwwrun". This causes svn to come into an inconsistent state.

I solved the problem by running bitten-master as www-run:

su -s /bin/sh wwrun -c "bitten-master ..."

Since then I never had to recover the svn database again.

[stava@…]

That makes sense, that's exactly the setup I have.

However, after upgrading to subversion 1.4.0 I haven't seen any problems and bitten master and slave are as stable as ever.

But I will indeed switch from root user to wwwrun just to make sure.

Thanks for the tip. /Lars Stavholm

[wbell]

Closing, seems to be resolved.