Re: HTTP Testing Resources

Yes, the code was open-sourced from the day one under MIT license at
https://github.com/ibnesayeed/webserver-tester (sorry, I forgot to provide
the link in the previous email).

I agree that it is almost impossible to create one tool that can test every
aspect of a web server. In the case of our class test suites, we provided
static files and CGI scripts with predefined directory structure for
students to serve with their web server implementations.

Before writing this library we looked for many other ways people have
implemented HTTP testing, but we did not find anything quite what we were
looking for, i.e., something that takes a raw HTTP message as input (which
was essential for the course). Many implementations require some sort of
representation or a data structure or provide helper methods to build
request piece by piece.

We have an instance of our custom tester instance tailored for the course
still deployed at https://cs531.cs.odu.edu/ where one can enter a host
(e.g., "odu.edu") in the "test a Web Server" block and run provided test
suits against it. Alternatively, first enter "msiddique" (which is the ID
of one of the students) in the first text input field under the "Deploy a
Web Server" block then deploy it. This will fetch the code of that
student's implementation from the private repo at GitHub, build a Docker
image, deploy it, and made that instance accessible at "cs531-msiddique"
host name from our tester instance (which itself is running in a
container). The block where tests are performed will automatically be
populated with this host name. Running those tests will show what aspects
were tested, what test cases passed, where things failed, and any request
and response data will be rendered for manual inspection. Similar
information is available from the CLI as well.

Best,

--
Sawood Alam
Department of Computer Science
Old Dominion University
Norfolk VA 23529



On Mon, Apr 8, 2019 at 8:13 PM Mark Nottingham <mnot@mnot.net> wrote:

> Hi Sawood,
>
> That sounds interesting; have you published them yet? If so, I don't see
> any harm in adding them.
>
> For what it's worth, the approach you outline reminds me of how
> cache-tests.fyi works; it defines test cases in JavaScript data structures,
> and you can run them using a NodeJS CLI or in a browser.
>
> Also, part of the problem of server testing is that many behaviours are
> dependent upon a particular resource, because of how it interfaces with the
> server (filesystem vs. CGI, for example) and also because of
> resource-specific handling. REDbot tries to address that by running checks
> against individual URLs, but I think there's also space for a tool that can
> be used to check how "back-ends" such as filesystems, CGI, FastCGI, as well
> as Web frameworks (e.g., Drupal, Wordpress) handle various aspects of the
> protocol.
>
> Cheers,
>
>
> > On 9 Apr 2019, at 10:03 am, Sawood Alam <ibnesayeed@gmail.com> wrote:
> >
> > Hi,
> >
> > Last year I wrote a web server tester library along with a handful of
> custom test suites to automate testing/grading of students' assignments of
> the Web Server Design course. While I think it is relevant here, I would
> wait for some feedback before I decide to add it to the wiki. Here is how
> it works:
> >
> > First, write a template file (say, "malformed-header.http") for the
> request data (which can be a single HTTP message or multiple messages put
> together for pipeline requests) as following:
> >
> > GET /foo HTTP/1.1
> > Host: <HOSTPORT>
> > Header with missing colon
> >
> > Then write a test method in a test suite class, inherited from the base
> tester class, as following:
> >
> > @make_request("malformed-header.http")
> > def test_bad_request_header(self, report):
> > """Test whether the server recognizes malformed headers"""
> >     self.check_status_is(report, 400)
> >
> > From there, the test can be run against any HTTP server using the
> included CLI or Web UI.
> >
> > I am planning to improve it further when I will offer this course in the
> Fall 2019.
> >
> > Best,
> >
> > --
> > Sawood Alam
> > Department of Computer Science
> > Old Dominion University
> > Norfolk VA 23529
> >
> >
> >
> > On Mon, Apr 8, 2019 at 7:43 PM Mark Nottingham <mnot@mnot.net> wrote:
> > Hi everyone,
> >
> > We've started collecting resources that might be helpful to test HTTP
> implementations here:
> >   https://github.com/httpwg/wiki/wiki/HTTP-Testing-Resources
> >
> > Thoughts / additions?
> >
> > Cheers,
> >
> > --
> > Mark Nottingham   https://www.mnot.net/
> >
> >
>
> --
> Mark Nottingham   https://www.mnot.net/
>
>