Re: HTTP Testing Resources

Not sure how relevant is this, but we have found nghttp (the cli utility
that comes with ngttp2) together with the `-v` option invaluable as part of
our functional test suite; mainly to verify that the server does HTTP/2
priorities and flow control correctly.



On Tue, Apr 9, 2019 at 3:08 PM Sawood Alam <ibnesayeed@gmail.com> wrote:

> Yes, the code was open-sourced from the day one under MIT license at
> https://github.com/ibnesayeed/webserver-tester
> <https://github..com/ibnesayeed/webserver-tester> (sorry, I forgot to
> provide the link in the previous email).
>
> I agree that it is almost impossible to create one tool that can test
> every aspect of a web server. In the case of our class test suites, we
> provided static files and CGI scripts with predefined directory structure
> for students to serve with their web server implementations.
>
> Before writing this library we looked for many other ways people have
> implemented HTTP testing, but we did not find anything quite what we were
> looking for, i.e., something that takes a raw HTTP message as input (which
> was essential for the course). Many implementations require some sort of
> representation or a data structure or provide helper methods to build
> request piece by piece.
>
> We have an instance of our custom tester instance tailored for the course
> still deployed at https://cs531.cs.odu.edu/ where one can enter a host
> (e.g., "odu.edu") in the "test a Web Server" block and run provided test
> suits against it. Alternatively, first enter "msiddique" (which is the ID
> of one of the students) in the first text input field under the "Deploy a
> Web Server" block then deploy it. This will fetch the code of that
> student's implementation from the private repo at GitHub, build a Docker
> image, deploy it, and made that instance accessible at "cs531-msiddique"
> host name from our tester instance (which itself is running in a
> container). The block where tests are performed will automatically be
> populated with this host name. Running those tests will show what aspects
> were tested, what test cases passed, where things failed, and any request
> and response data will be rendered for manual inspection. Similar
> information is available from the CLI as well.
>
> Best,
>
> --
> Sawood Alam
> Department of Computer Science
> Old Dominion University
> Norfolk VA 23529
>
>
>
> On Mon, Apr 8, 2019 at 8:13 PM Mark Nottingham <mnot@mnot.net> wrote:
>
>> Hi Sawood,
>>
>> That sounds interesting; have you published them yet? If so, I don't see
>> any harm in adding them.
>>
>> For what it's worth, the approach you outline reminds me of how
>> cache-tests.fyi works; it defines test cases in JavaScript data structures,
>> and you can run them using a NodeJS CLI or in a browser.
>>
>> Also, part of the problem of server testing is that many behaviours are
>> dependent upon a particular resource, because of how it interfaces with the
>> server (filesystem vs. CGI, for example) and also because of
>> resource-specific handling. REDbot tries to address that by running checks
>> against individual URLs, but I think there's also space for a tool that can
>> be used to check how "back-ends" such as filesystems, CGI, FastCGI, as well
>> as Web frameworks (e.g., Drupal, Wordpress) handle various aspects of the
>> protocol.
>>
>> Cheers,
>>
>>
>> > On 9 Apr 2019, at 10:03 am, Sawood Alam <ibnesayeed@gmail.com> wrote:
>> >
>> > Hi,
>> >
>> > Last year I wrote a web server tester library along with a handful of
>> custom test suites to automate testing/grading of students' assignments of
>> the Web Server Design course. While I think it is relevant here, I would
>> wait for some feedback before I decide to add it to the wiki. Here is how
>> it works:
>> >
>> > First, write a template file (say, "malformed-header.http") for the
>> request data (which can be a single HTTP message or multiple messages put
>> together for pipeline requests) as following:
>> >
>> > GET /foo HTTP/1.1
>> > Host: <HOSTPORT>
>> > Header with missing colon
>> >
>> > Then write a test method in a test suite class, inherited from the base
>> tester class, as following:
>> >
>> > @make_request("malformed-header.http")
>> > def test_bad_request_header(self, report):
>> > """Test whether the server recognizes malformed headers"""
>> >     self.check_status_is(report, 400)
>> >
>> > From there, the test can be run against any HTTP server using the
>> included CLI or Web UI.
>> >
>> > I am planning to improve it further when I will offer this course in
>> the Fall 2019.
>> >
>> > Best,
>> >
>> > --
>> > Sawood Alam
>> > Department of Computer Science
>> > Old Dominion University
>> > Norfolk VA 23529
>> >
>> >
>> >
>> > On Mon, Apr 8, 2019 at 7:43 PM Mark Nottingham <mnot@mnot.net> wrote:
>> > Hi everyone,
>> >
>> > We've started collecting resources that might be helpful to test HTTP
>> implementations here:
>> >   https://github.com/httpwg/wiki/wiki/HTTP-Testing-Resources
>> >
>> > Thoughts / additions?
>> >
>> > Cheers,
>> >
>> > --
>> > Mark Nottingham   https://www.mnot.net/
>> >
>> >
>>
>> --
>> Mark Nottingham   https://www.mnot.net/
>>
>>

-- 
Alcides Viamontes E.
ShimmerCat AB, CTO
(+46) 722294542