⌈⌋ branch:  freshcode


Artifact Content

Artifact e11500b873c14fcb003578464667016b31e40218:

Wiki page [API2] by glen 2015-08-10 11:58:08.
D 2015-08-10T11:58:08.796
L API2
N text/x-markdown
P 7fd956b6d9000159e2af6aaa84450523b83733a0
U glen
W 15274
State: ***design***, *testing*


## Simpler JSON API

To update and submit project records, there's going to be a simplified
JSON interface.

 * The testing API endpoint is <kbd>`https://test.freshcode.club/…`</kbd>,  
   which can be used for browsing as well.

 * The regular API endpoint is <kbd>`https://api.freshcode.club/…`</kbd>,  
   but is *currently* fixated on the test database too.


## URL scheme

Updating and release publishing can now be combined into one PUT/POST
request. The URL dict can now be embedded everywhere or passed alongside.


method	| URL path						|function+purpose
--------------------------------------------------------------------------------
GET	|`/projects/<name>.json`				|query all fields
PUT	|`/projects/<name>.json`				|update_core+release
CREATE	|`/projects/<name>.json`				|new_project+update
<s>POST	|<s>`/projects/<name>/releases.json`</s>		|<s>publish</s>	
DELETE	|`/projects/<name>/releases/<verstr>.json`*?auth_code=*	|withdraw version	
<s>PUSH	|<s>`/projects/<name>/urls.json`</s>			|<s>update_urls</s>
GET	|`/feed/<name>.json`					|release_list+federation

The <kbd>PUT</kbd> method likewise works with <kbd>POST</kbd> or
<kbd>PUSH</kbd> ("PUSH" because it's a programming site, and entries are
basically a stack). Fields are equivalent for the initializing <kbd>CREATE</kbd>.


## JSON object keys

Where the JSON field names are mapped as follows:

<style>
 .content table { width: 95%; }
 .content table tr td:nth-child(5) { color: #99c; font-style: italic; }
 .content table tr td:nth-child(3) { font-weight: bold; color: #272; } 
 .content table tr td:nth-child(1) { color: #bbb; font-weight: bold; }
</style>


 works with   |  freshcode DB   |  JSON field     | freecode-submit  | Sample data
------------------------------------------------------------------------------------------
 CREATE       |  name           |  `<name>`       | Project          | unix-name
   PUT        |  title          |  title          | Project          | Foo Bar Gtk+
   PUT        |  summary        |  summary        | Summary          | short slogan
   PUT        |  description    |  description    | Description      | Long  text...
   PUT        |  tags           | `project_tags`  | Project-Tag-List | c, c++, cli
   PUT        |  license        | `license_list`  | License-List     | MITL, GNU GPL
   PUT, URL   |  homepage       |  urls {…}       | Homepage-URL     | http://exmpl
   PUT, URL   |  homepage       |  urls {…}       | Website-URL      | http://exmpl
   PUT, URL   |  image:         |  urls {…}       | Screenshot-URL   | http://png
   PUT, URL   |  autoupdate_url |  urls {…}       | Changelog-URL    | .../NEWS.md
   PUT, URL   |  urls {…}       |  urls {…}       | *******-URL      | Title = http://...
     RELEASE  |  download       |  download       | Download-URL     | http://sf.net
     RELEASE  |  version        |  version        | Version          | 1.0.3-rc5
     RELEASE  |  changes        |  changes        | Changes          | Added Xy. Fixed Zy.
     RELEASE  |  hidden         | `hide`          | Hide             | 0
     RELEASE  |  state          | `release_tags`  | Release-Tag-List | stable
     RELEASE  |  scope          | `release_tags`  | Release-Tag-List | minor, bugfix
 CREATE       |  submitter      |  author         | Author           | Bob
 CREATE       |  submitter_img  |  -              | Author           | Bob, bob@gitub
      -       |  editor_note    |  -              | -                | -  
      -       |  flag           |  -              | -                | -  
 CREATE       |  lock           |  `<auth_code>`  | <kbd>.netrc</kbd>| -
      -       |  social_links   |  -              | -                | -  
      -       |  autoupdate_*   |  -              | -                | -  
       DELETE |  name           |  `<name>`       | -P name          | unix-name
       DELETE |  deleted        | `<versionstr>`  | -d               | 1.0.3-rc5  


- Renamed: `permalink` → `name`, and `changelog` → `changes`
- New: `title`, `summary` (oneliner), `urls` as dict {…}, `author` for `submitter`
- Changes: `project_tag_list` is now `project_tag_list` and CSV text instead of
  a JSON list, same for `license_list` with spacing irrelevant
- `release_tags` could perhaps be split up, but the update handler itself separates
  "stable/beta/prerelease" from "minor/major/bugfix/feature/security" etc.
- Internal / Unassigned yet: `editor_note`, `flag`, `lock`, `social_links`,
  `autoupdate_***`


### Authorization:

All requests can append an `?auth_code=` to request URLs to carry a plain password.
Which is primarily used with the <kbd>DELETE</kbd> method. The <kbd>GET</kbd> method
only hides a few internal fields when unauthorized.

<kbd>POST</kbd>/<kbd>PUT</kbd>/etc. requests instead should wrap the
password outermost in the JSON payload:

    {
        "auth_code": "plain_pw_123",
        "project": {
            "key": "Value",
            ...
        }
    }


## Query <kbd>GET</kbd> `/projects/<name>.json`

Returns an almost literal database dump:

    {
        "project": {
            "name": "foo-bar",
            "title": "Foo Bar",
            "summary": "iCal-compatible calendaring app",
            "description": "Here comes a very long description ...",
            "image": "",
            "author": "Bob",
            "license_list": "MITL, CC-BY-SA",
            "project_tags": "desktop, c++, qt5, calendar",
            "homepage": "http://example.org",
            "download": "http://example.org/dwnld/",
            "urls": {
                "Homepage": "http://example.org",
                "Changelog": "https://example.org/downl/NEWS.md",
                "DEB": "http://example.org/downl/foo-bar_1.deb",
                "Download": "http://example.org/dwnld/"
            },
            "version": "5.0.1",
            "state": "stable",
            "scope": "minor bugfix",
            "release_tags": "stable, minor bugfix",
            "changes": "Fixed the thing, and added the stuff.",
            "download": "http://example.org/dwnld/",
            "t_published": "1427461501",
            "t_changed": "1452413135"
        },
        "$feed-license": "CC-BY-SA 3.0",
        "$feed-origin": "http://freshcode.club/"
    }

The release fields (second half in the example) represent the latest
known version.

  * The URLs are partly duplicated. Because "homepage" and "download" are
    core fields, they show up in two places.
  * This GET response blob might carry a few more internal fields (autoupdate_*,
    lock, t_published, flag, editor_note) if an `?auth_code=` was supplied.


## Update core+release <kbd>PUT</kbd> `/projects/<name>.json`

For changing general project information, all fields can be sent wrapped
in the same structure. The only difference is the required `auth_code`.

    {
        "auth_code": "plain_pw_123",
        "project": {
            "title": "New Title",
            "summary": "Shorter project slogan",
            "description": "New and longer description...",
            "image": "http://new.png",
            "license_list": "BSDL",
            "project_tags": "qt6, dlang, desktop, calendar",
            "author": "Elise Exemplary, dj_ical@launchpad",
            "urls": {
                "homepage": "http://google-code.com",
                "DEB": "http://example.org/SECOND-REALEASE.deb",
                "download": "http://example.org/dwnld/"
            },
            "version": "6.0.2",
            "release_tags": "stable, minor bugfix, security",
            "changes": "Fixed the thing, and added the stuff.",
            "download": "http://example.org/dwnld/",
        }
    }

Now interestingly, this scheme can be used to:
 
 1. Just update the basic project description.
 2. Or publish new release "version" and "changes" right along.

Either, or, and both.


#### Sectioned variant

As variation of that, the JSON struct ***may*** be split into:

    {
        "auth_code": "plain_pw_123",
        "project": {
            "title": "New Title",
            "summary": "Shorter project slogan",
            "description": "New and longer description...",
            "image": "http://new.png",
            "license_list": "BSDL",
            "project_tags": "qt6, dlang, desktop, calendar",
            "urls": {
                "homepage": "http://google-code.com",
                "DEB": "http://example.org/SECOND-REALEASE.deb",
                "download": "http://example.org/dwnld/"
            }
        },
        "release": {
            "version": "6.0.2",
            "release_tags": "stable, minor bugfix",
            "changes": "Fixed the thing, and added the stuff.",
            "download": "http://example.org/dwnld/",
            "hide": false,
        }
    }

It'll simply be rejoined server-side. Which doesn't make a difference
to the internal database scheme. Might be more convenient to implement.


#### URL section moved out

And as further variation, you can also split out the `url:{}` dict:

    {
        "auth_code": "plain_pw_123",
        "project": {
            "title": "New Title",
            "description": "................"
            "editor_note": "Please reset the social bookmark counter!
                            Project moved from Sourceforge to GitHub."
        },
        "urls": {
            "homepage": "http://example.org/",
            "proprietary-hoster": "http://github.com/proj/name/",
            "PYZ-package": "http://example.org/calendar.pyzw",
        }
        "release": {
            "version": "7.5.3",
            "changes": "Rome sprang up...",
        }
    }

The API is pretty much indifferent if this wraps up just the "project"
base information, a lone "urls" dict, or just a "release" or any
combination thereof.


## URLs dict <kbd>GET</kbd>/<kbd>PUSH</kbd> `/projects/<name>/urls.json`

There's still a separate API endpoint for just retrieving or updating
the URLs though:

    {
        "auth_code": "plain_pw_123",
        "urls": {
            "Homepage": "http://example.org/",
            "Screenshot": "http://launchpad.com/proj/screenshot.png",
            "Custom-link": "http://example.com/share/help/C/ping/index.page",
        }
    }

Note that a projects "homepage" or "download" or "image" URL only get
updated when listed herein. All other project links get completely
discarded and replaced by whatever is in the new list.

 * Incoming link titles are *free-form*.
 * The case is preserved for custom entries.
 * Non-word characters are replaced.
 * Titles will always be stored with "Dashed-Names" (it's internally
   a `Key=Url` text/yaml field).
 * The core URLs (homepage, download, screenshot) however are always
   lowercased (when returned on GET requests).
 * The API is indifferent if you pass the "homepage" or "download"
   link as literal `project:{…}` dict entry, or wrapped in `urls:{…}`.


## Release publishing <kbd>POST</kbd> `/projects/<name>/releases.json`

This is entirely redundant now. But a new release can be published
with that alternative URL scheme. It's basically identical to
"PUT `/project/<name>.json`" now:

    {
        "auth_code": "plain_pw_123",
        "release": {
            "version": "0.0.1",
            "release_tags": "beta, cleanup",
            "changes": "Initial release. No docs or code.",
            "hide": true,
        }
    }

Note that even the "hide" flag is already available with regular
project PUT / core updates.


## Withdraw_release <kbd>DELETE</kbd> `/projects/<name>/<versionstr>.json`

Now release retraction became simpler as well. There are no longer
arbitrary numeric IDs exposed. (They were faked anyway.)

Instead the *literal version number*  can now be used to delete
a release entry.

  -  <kbd>DELETE</kbd> `/projects/<name>/`*1.2.3-rc2*`.json?auth_code=pw123`

The version string has to be URL-encoded as needed of course.


## New_project <kbd>CREATE</kbd> `/projects/<name>.json`

The JSON payload for creating a new project might look familiar.
Well in fact, *it is* identical to the regular PUT payload:

    {
        "auth_code": "this:is-a/brand#new+password_567",
        "project": {
            "title": "Initial Title",
            "summary": "ONELINER",
            "description": "Description REQUIRED.",
            "project_tags": "bash, script",
            "license": null,
            "author": "Username, user@localhost",
            "urls": {
                "screenshot": null,
                "Homepage": "http://example.org/",
            },
            "version": null,
            "release_tags": "initial, security bugfix",
            "changes": "..."
        }
    }

The `null` fields are just for brevity. In fact the first project
submission *can and **should** even* contain a full record - including
the current release version/info. (CREATE cannot be used to register
all-empty or stub project records.)

It can however also just be completed with the next <kbd>PUT</kbd>/Update.
The only requirements for <kbd>CREATE</kbd> are a new Unix project`<name>`
(taken from the request URL), and a "`title`" and "`description`",
"`license`" or "`project_tags`", and a "`homepage`" URL.

And obviously this is the only time the ‹`auth_code`› is used for
populating the internal authorization hash. All other requests just
compare it, but CREATE does the creative step.

Client SSL cert as CREATE spamguard: **Undecided** / *Testing*.  
See also [/doc/trunk/doc/submit.pem](doc/trunk/doc/submit.pem) for public
authorization key.


## Responses

Successful submissions will be answered with HTTP Status 201 and:

    {
        "success": true
    }

Whereas errors will typically carry a HTTP status error and some vaguely
interesting notes in:

    {
        "error": "Missing project title, summary, description, homepage.",
        "project": {
            "name": null
        },
        "data": null
    }

HTTP status codes:

 * "200 No content" for unknow project names.
 * "201 Created" for successful publication.
 * "401 Unauthorized" for an invalid `auth_code`.
 * "410 Obsolete" for GET /proj/releases.json.
 * "495 Client Cert" on CREATE requests without submit.pem
 * "500 DB Error" for server failures.
 * "501 Not implemented" for unknown request methods/paths.
 * "503 Unavailable" is unlikely to happen.


## Security considerations

Now that's slightly irrelevant for API usage, in particular for just polling
or updating your own project records.

  - But the naming scheme on freshcode also permits project names like
    `commonmark.js` or `table-generator.sh`.  So some care should be taken
    before using the returned project names as literal filenames.
  - Description fields are mostly just filtered for printable characters. 
  - Only Homepage and Download URLs are guaranteed to be HTTP references.



Z c8c577480a74d7fda52e245c208ef972