Attach files to a build

[Murphi]

It would be great if one could automatically attach files created by a test run to the build.

[anonymous]

+1 from me.

It would be perfect if artefacts (libs, docs, whatever) created by a build were automatically sent to the master and offered for download. This would allow for a nice history of nightly builds of components/libraries (plus we wouldn't have to continue the close-to hack we're currently using for this purpose)

15. Trenner

[anonymous]

Ok, I should've read this before submitting my comment. Who would've guessed that this has already been taken care of... I'm quite looking forward to seing it implemented ;)

15. Trenner

[petrowsky@…]

Hello,

I just installed Bitten on our server and would like to know when this feature will be available. We could really use this right now. Is there any way to help you implement that in the near future?

Mr. Trenner: Could you tell us/me how you "close-to hacked" the builds?

Tim Petrowsky

[anonymous]

Tim,

Our current solution doesn't upload the artifacts to the server, it simply depends on the fact that client and slave run on the same machine. We usually include a "package" step into our recipes that creates the artifacts and moves them to a place where they can be downloaded from. Some additional comfort is gained from a wiki macro that lists the artifact directory and exposes links to the files. The macro looks like this:

TRAC_ENV/wiki-macros/artifacts.py:

#!/usr/bin/python
#-*- coding: utf-8 -*-
#
# Copyright (C) 2007 Ole Trenner, <ole@jayotee.de>
# All rights reserved.
#
# This software is licensed as described in the file COPYING, which
# you should have received as part of this distribution.

import os
import StringIO

ARTIFACT_DIR = '/var/www/artifacts/'
BASE_URL = '/trac/chrome/site/artifacts/'
ARG_LAST_ONLY = 'last_only'

def execute(hdf, args, env):
    artifacts = os.listdir(ARTIFACT_DIR);
    artifacts.sort()
    if (args == ARG_LAST_ONLY):
        artifacts = [artifacts[-1],]
    output = StringIO.StringIO()
    output.write('<ul>')
    for name, url in [(name, BASE_URL + name) for name in artifacts]:
        output.write('<li><a href="%s">%s</a></li>' % (url, name))
    output.write('</ul>')
    return output.getvalue()

You invoke the macro like this:

[[artifacts(last_only)]]

Cheers, Ole Trenner

[aelse@…]

I am adding a +1 to this also. The docs (linked to by O. Trenner a few posts up) don't indicate the exact syntax to use, so I take it that this remains unimplemented. It would be very useful to have this feature working. For now i'm going to have to implement a hack similar to Ole's.

Any idea when this is likely to come about?

[thomas.moschny@…]

+1 for this feature.

[anonymous]

+1 for this feature... We want this!

[osimons]

I suppose we could reuse the Trac Attachment module to handle attachments, and add a generic command to upload artifacts of any kind:

<attach file="local/path" filename="display name" author="(req.authname)" desription="my file"/>

They will display and be handled as regular attachments - listed on the build page, and manipulated in same way as attachments for Ticket and Wiki modules. Including deleting them individually, or adding new attachments manually through web page.

It needs some work:

• Need to implement IResourceManager to make builds become full Trac resources.
• Add regular attachment code to make attachments work through web interface.
• Add delete of attachments when builds are deleted
• Do we delete attachments when builds gets invalidated?
• Add the upload command and receiving/inserting code that adds attachments automatically.

Doable. I'll put it on my to-do at least, and we'll see.

[osimons]

Patch for adding attachments - both through web and using new <attach/> command.

[osimons]

I've made a patch for this. Turned out really well I think.

It introduces regular Trac attachment support via web interaface for attaching artifacts to both build (example an .exe result per build) and configuration level (adding a 'latest.exe' to the overview page for the configuration).

Adding attachments is also supported through recipes. The new <attach/> command currently has the following docs (also part of the patch):

---

<attach>

Attach a file to the build or configuration as regular attachment. An already existing attachment on the same resource with same base filename will be replaced.

Parameters

Name	Description
file	Path to the XML file containing the report data, relative to the project directory.
resource	The resource to attach the file to. Either 'build' (default) or 'config'. Optional.
description	Attachment description. Optional.

---

Unittests remain - quite a few tests needed, actually... And also, testing by those with interest in this feature - feedback welcome!

[osimons]

Tests are now also added. Should be ready for commit...

[osimons]

Updated patch. This should now be complete - code, tests, docs.

[mgood]

Nice work, this is something I've wanted plenty of times in the past. I've looked over the patch and it looks good, though I have a few minor concerns:

Firstly, in tests/master you can replace this block:

try:
    module.process_request(req)
    self.fail('Expected RequestDone')
except RequestDone:

with this:

self.assertRaises(RequestDone, module.process_request, req)

What use cases did you have in mind for attaching files to the config instead of the build? The builds don't always arrive in sequential order, so if the most recent build overwrites the attachments they could actually be for an old version of the code. This could be confusing since the config page shows the build status of the most recent code version and you can't easily tell if the attachments are from that build or not.

Also, did you test how big of files this can handle? This is probably ok for now, but sending the contents directly in the XML won't work so well for large files.

[osimons]

Thanks for reviewing!

Replying to mgood:

Firstly, in tests/master you can replace this block:

You're right. I just copied the pattern used by all the other master tests. But your suggestion makes the tests look 'cleaner', so I've updated my test to use it.

What use cases did you have in mind for attaching files to the config instead of the build?

Well, attaching files to config is useful in inself (from web also). I am aware of the issue that about overwriting, and it will only work ~consistently when only building latest (at least for overwriting, but you could of course use ${build} as part of the filename and trim list manually). And, as attachments are not versioned there is of course no way of knowing what build created it. I'll write a note about this in the docs - use with care...

Also, did you test how big of files this can handle?

Nope. Have no clue - it consumed the 'normal' sizes files I threw at it in development. I'm working on the assumption that it should be OK...

[osimons]

Attachment support added in [703]. Thanks for ideas and review!