Re: Automatic test framework for media fragment servers

Looks good at first sight. I think you may find you need to adjust
some things as you go. But I can't spot anything wrong with it right
away.

Also, I think this is a good approach to validate the workings of a
server. I think to validate a client, we may need to sniff HTTP
packets for given URIs given a validated server.

Cheers,
Silvia.



On Tue, May 25, 2010 at 11:02 PM, Davy Van Deursen
<davy.vandeursen@ugent.be> wrote:
> Dear all,
>
>
>
> in the context of Erik's action-164, we are currently working on a
> validation tool for Media Fragment servers, based on input from corrib.
> Currently, our validation tool performs the following steps:
>
> 1. syntax validation of the Media Fragment URI (e.g., #t=banaan is a syntax
> error)
>
> 2. semantic validation of the Media Fragment URI (e.g., #t=20,10 is a
> semantic error)
>
> 3. if the media fragment is syntactically and semantically valid, construct
> a valid Range header and perform the request
>
>                 3.1 check if the HTTP response code is right
>
>                 3.2 check the HTTP response headers (i.e.,
> Content-Range-Mapping) syntactically and semantically
>
> 4. if the media fragment is syntactically or semantically invalid, construct
> an invalid Range header and perform the request. Check if the server reacts
> in the right way.
>
> 5. functional testing (in case of a valid media fragment)
>
>                 5.1 download the full resource
>
>                 5.2 download the resource in pieces using byte range
> requests and reconstruct the full resource
>
>                 5.3 download the resource in pieces using time range
> requests and reconstruct the full resource
>
>                 5.4 download the resource in pieces using track range
> requests and reconstruct the full resource
>
>                 5.5 compare the retrieved resources bit-wise
>
> 6. visual testing (in case of a valid media fragment): play the media
> fragment in the browser (probably using a valid MF player :-))
>
>
>
> It is important to notice that we do not test Media Fragment players (i.e.,
> the translation from Media Fragment URI to HTTP Range header), as mentioned
> in the minutes of the last phone conf [1]; our validation tool only
> validates servers.
>
>
>
> Now, in order to get this working with corrib RDF output, we need the
> following information out of corrib:
>
> - input media fragment (already available)
>
> - http response code (not explicitly available)
>
> - visual result description (+/- available)
>
> - media resource (not available) -> a test case is dependent on the
> underlying media resource
>
>
>
> Therefore, we extended [2] with the necessary properties and created [3] to
> describe the media resources used in the test cases. The description of the
> first test case should then look as follows:
>
>
>
> <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
>
>   xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
>
>   xmlns:dcterms="http://purl.org/dc/terms/"
>
>   xmlns:t="http://www.w3.org/2006/03/test-description#"
>
>   xmlns:ctag="http://commontag.org/ns#"
>
>   xmlns:mftc="http://www.w3.org/2008/WebVideo/Fragments/TC/mftc#">
>
>
>
> <rdf:Description
> rdf:about="http://www.w3.org/2008/WebVideo/Fragments/TestCases/v1-mediafrag/Testd050755546ab4712eab90bb768e7ed6a">
>
>     <rdf:type
> rdf:resource="http://www.w3.org/2006/03/test-description#TestCase"/>
>
>     <rdfs:seeAlso
> rdf:resource="http://www.w3.org/2009/05/13-mediafrag-minutes.html#item05"/>
>
>     <dcterms:title>plain fragment</dcterms:title>
>
>     <dcterms:contributor rdf:resource="http://sw-app.org/mic.xhtml#i"/>
>
>     <t:reviewStatus
> rdf:resource="http://www.w3.org/2006/03/test-description#approved"/>
>
>     <t:input>#</t:input>
>
>     <t:purpose></t:purpose>
>
>     <ctag:tagged
> rdf:resource="http://www.w3.org/2008/WebVideo/Fragments/TC/mftc#UndefinedTag"/>
>
>     <mftc:expectedHTTPStatusCode
> rdf:resource="http://www.w3.org/2008/http-statusCodes#statusCode200"/>
>
>     <mftc:expectedVisualResults>entire
> representation</mftc:expectedVisualResults>
>
>     <mftc:mediaResource
> rdf:resource="http://www.w3.org/2008/WebVideo/Fragments/TC/media#fragf2f.mp4"/>
>
>     <mftc:mediaResource
> rdf:resource="http://www.w3.org/2008/WebVideo/Fragments/TC/media#fragf2f.ogv"/>
>
> </rdf:Description>
>
> </rdf:RDF>
>
>
>
> @All: what do you think of this approach?
>
> @Michael: do you think you could extend corrib to cover this information?
>
>
>
> Best regards,
>
>
>
> Davy
>
>
>
> [1] http://www.w3.org/2010/05/19-mediafrag-minutes.html#item03
>
> [2] http://www.w3.org/2008/WebVideo/Fragments/TC/mftc#
>
> [3] http://www.w3.org/2008/WebVideo/Fragments/TC/media#
>
>
>
> --
>
> Davy Van Deursen
>
>
>
> Ghent University - IBBT
>
> Faculty of Engineering
>
> Department of Electronics and Information Systems
>
> Multimedia Lab
>
>
>
> Gaston Crommenlaan 8 bus 201
>
> B-9050 Ledeberg-Ghent
>
> Belgium
>
>
>
> t: +32 9 33 14893
>
> f: +32 9 33 14896
>
> t secr: +32 9 33 14911
>
> e: davy.vandeursen@ugent.be
>
> URL: http://multimedialab.elis.ugent.be/dvdeurse
>
>

Received on Tuesday, 25 May 2010 13:24:49 UTC