Re: stitching together APIs

On Tue, Oct 2, 2012 at 11:11 AM, Annette Greiner <amgreiner@lbl.gov> wrote:

> Hi folks,
> To frame the discussion for the October 11 conference call, I've started
> thinking about how to go about putting together a first draft of a standard
> API. It seems to me that it would be logical to simply blend the two APIs
> we currently have, NEWT and the iPlant API (Agave). There's a lot they have
> in common, though of course they have different terms for things. I would
> suggest we choose our terms based on three principles:
> coherence: terms in the API should have grammatical commonality with other
> terms of similar function in the API
> clarity: terms should be unambiguous
> memorability: terms should be easy to associate mentally with their
> meaning in the API
> cross-center generalizability: terms should make sense in the context of
> any HPC center
>
>
Good points. One step toward the last one is to make a fake HPC center
stubbed out in the software itself. This serves two purposes. 1) you get to
try the software or develop on your laptop without touching the guts of
your HPC center. 2) It provides a common meeting ground for all of us as a
plain vanilla idealization of an HPC center. To be a little more specific I
am suggesting that auth, data, and job functions should have stub
implementations that operate locally and while ineffectual they should be
processed in a way that mimics a real HPC center.

auth: just use an install-time configured password with a test user
data: just move local files on disk
jobs: just run the command (fork/exec).
KVP store: use a couch or mongo local instance.

Once we have that stub implementation down and packaged people can download
and try the API without herculean efforts.

We'll also need to discuss the scope of the standard API. How much should
> it cover? Clearly, centers should be free to do their own implementations;
> we are just defining a set of REST calls that can be re-used across
> implementations. But what functions should be left out of the standard? I'm
> thinking here of functions that are not specific to HPC. One example is the
> iPlant PostIt, which generates disposable URLs. I think that's a great
> service to offer people, but I would suggest we leave it out of a standard
> for HPC, since it isn't a function that arises from the HPC context. The
> iPlant Apps and Profile features strike me similarly. NEWT has a liststore
> feature that could also be seen as a non-HPC aspect of that API.
>
>
The guiding model for NEWT thus far has been to stick to the core things
you see in HPC center documentation. How do I log in, how do I move files,
how do I run things. We don't need to be rigid about that but having a
guiding principle with a decent level of simplicity seems prudent.

We've also advocated an exception mechanism whereby you can step outside
the API and do whatever you like. That provides some demarcation as to
where the API stops and where custom machinery begins.

-David

What do other people think? How should we define what is in/out of the spec?
> -Annette
> --
> Annette Greiner
> Outreach, Software, and Programming Group
> NERSC, LBNL
> amgreiner@lbl.gov
> 510-495-2935
>
>
>
>
>
>