[PATCH] bitten master crashes when unittest's method includes a non-ASCII encoding docstring

[Xie Yanbo <xieyanbo@…>]

If a testcase method includes an utf-8 encoding short description like this:

class Utf8DocstingTestCase(TestCase):
    def test_something(self):
        '''test utf-8 Chinese character, 测试汉字'''
        pass

Bitten will crash with:

Traceback (most recent call last):
  File "/usr/lib64/python2.5/site-packages/trac/web/main.py", line 450, in _disp
atch_request
    dispatcher.dispatch(req)
  File "/usr/lib64/python2.5/site-packages/trac/web/main.py", line 206, in dispa
tch
    resp = chosen_handler.process_request(req)
  File "/usr/lib64/python2.5/site-packages/bitten/master.py", line 130, in proce
ss_request
    return self._process_build_step(req, config, build)
  File "/usr/lib64/python2.5/site-packages/bitten/master.py", line 324, in _proc
ess_build_step
    report.insert(db=db)
  File "/usr/lib64/python2.5/site-packages/bitten/model.py", line 936, in insert
    in item.items()])
  File "/usr/lib64/python2.5/site-packages/trac/db/util.py", line 80, in execute
many
    return self.cursor.executemany(sql_escape_percent(sql), args)
  File "/usr/lib64/python2.5/site-packages/trac/db/sqlite_backend.py", line 64, 
in executemany
    args or [])
  File "/usr/lib64/python2.5/site-packages/trac/db/sqlite_backend.py", line 51, 
in _rollback_on_error
    return function(self, *args, **kwargs)
ProgrammingError: You must not use 8-bit bytestrings unless you use a text_facto
ry that can interpret 8-bit bytestrings (like text_factory = str). It is highly 
recommended that you instead just switch your application to Unicode strings.

cause sqlite does not accept the utf8 string.

My environment:

• Bitten: 0.6b2
• Python: 2.5
• pysqlite: 2.5.6
• OS: Gentoo Linux

Patch attached.

[osimons]

Thanks for report.

Seems very late to deal with different variations of string when building strings for SQL execute? I think a more correct approach would be to make sure Report.items don't get populated with anything but Unicode strings. That means converting as strings are received from input (ie. xml from slave). Which I kind of thought we did, but this must have slipped through somehow (although I have not verified it myself).

[Xie Yanbo <xieyanbo@…>]

you are right, how about this:

bitten/master.py

@@ -319 +319 @@
                 item = {'type': item_elem.name}
                 item.update(item_elem.attr)
                 for child_elem in item_elem.children():
-                    item[child_elem.name] = child_elem.gettext()
+                    item[child_elem.name] = child_elem.gettext().decode('utf8')
                 report.items.append(item)
             report.insert(db=db)

[osimons]

Thanks, feels better. A unittest would also be really nice... :-)