⌈⌋ branch:  freshcode


Update of "API2"

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview

Artifact ID: e11500b873c14fcb003578464667016b31e40218
Page Name:API2
Date: 2015-08-10 11:58:08
Original User: glen
Mimetype:text/x-markdown
Parent: 7fd956b6d9000159e2af6aaa84450523b83733a0
Content

State: design, testing

Simpler JSON API

To update and submit project records, there's going to be a simplified JSON interface.

  • The testing API endpoint is https://test.freshcode.club/…,
    which can be used for browsing as well.

  • The regular API endpoint is https://api.freshcode.club/…,
    but is currently fixated on the test database too.

URL scheme

Updating and release publishing can now be combined into one PUT/POST request. The URL dict can now be embedded everywhere or passed alongside.

method URL path function+purpose
GET /projects/<name>.json query all fields
PUT /projects/<name>.json update_core+release
CREATE /projects/<name>.json new_project+update
POST /projects/<name>/releases.json publish
DELETE /projects/<name>/releases/<verstr>.json?auth_code= withdraw version
PUSH /projects/<name>/urls.json update_urls
GET /feed/<name>.json release_list+federation

The PUT method likewise works with POST or PUSH ("PUSH" because it's a programming site, and entries are basically a stack). Fields are equivalent for the initializing CREATE.

JSON object keys

Where the JSON field names are mapped as follows:

works with freshcode DB JSON field freecode-submit Sample data
CREATE name <name> Project unix-name
PUT title title Project Foo Bar Gtk+
PUT summary summary Summary short slogan
PUT description description Description Long text...
PUT tags project_tags Project-Tag-List c, c++, cli
PUT license license_list License-List MITL, GNU GPL
PUT, URL homepage urls {…} Homepage-URL http://exmpl
PUT, URL homepage urls {…} Website-URL http://exmpl
PUT, URL image: urls {…} Screenshot-URL http://png
PUT, URL autoupdate_url urls {…} Changelog-URL .../NEWS.md
PUT, URL urls {…} urls {…} *******-URL Title = http://...
RELEASE download download Download-URL http://sf.net
RELEASE version version Version 1.0.3-rc5
RELEASE changes changes Changes Added Xy. Fixed Zy.
RELEASE hidden hide Hide 0
RELEASE state release_tags Release-Tag-List stable
RELEASE scope release_tags Release-Tag-List minor, bugfix
CREATE submitter author Author Bob
CREATE submitter_img - Author Bob, bob@gitub
- editor_note - - -
- flag - - -
CREATE lock <auth_code> .netrc -
- social_links - - -
- autoupdate_* - - -
DELETE name <name> -P name unix-name
DELETE deleted <versionstr> -d 1.0.3-rc5
  • Renamed: permalinkname, and changelogchanges
  • New: title, summary (oneliner), urls as dict {…}, author for submitter
  • Changes: project_tag_list is now project_tag_list and CSV text instead of a JSON list, same for license_list with spacing irrelevant
  • release_tags could perhaps be split up, but the update handler itself separates "stable/beta/prerelease" from "minor/major/bugfix/feature/security" etc.
  • Internal / Unassigned yet: editor_note, flag, lock, social_links, autoupdate_***

Authorization:

All requests can append an ?auth_code= to request URLs to carry a plain password. Which is primarily used with the DELETE method. The GET method only hides a few internal fields when unauthorized.

POST/PUT/etc. requests instead should wrap the password outermost in the JSON payload:

{
    "auth_code": "plain_pw_123",
    "project": {
        "key": "Value",
        ...
    }
}

Query GET /projects/<name>.json

Returns an almost literal database dump:

{
    "project": {
        "name": "foo-bar",
        "title": "Foo Bar",
        "summary": "iCal-compatible calendaring app",
        "description": "Here comes a very long description ...",
        "image": "",
        "author": "Bob",
        "license_list": "MITL, CC-BY-SA",
        "project_tags": "desktop, c++, qt5, calendar",
        "homepage": "http://example.org",
        "download": "http://example.org/dwnld/",
        "urls": {
            "Homepage": "http://example.org",
            "Changelog": "https://example.org/downl/NEWS.md",
            "DEB": "http://example.org/downl/foo-bar_1.deb",
            "Download": "http://example.org/dwnld/"
        },
        "version": "5.0.1",
        "state": "stable",
        "scope": "minor bugfix",
        "release_tags": "stable, minor bugfix",
        "changes": "Fixed the thing, and added the stuff.",
        "download": "http://example.org/dwnld/",
        "t_published": "1427461501",
        "t_changed": "1452413135"
    },
    "$feed-license": "CC-BY-SA 3.0",
    "$feed-origin": "http://freshcode.club/"
}

The release fields (second half in the example) represent the latest known version.

  • The URLs are partly duplicated. Because "homepage" and "download" are core fields, they show up in two places.
  • This GET response blob might carry a few more internal fields (autoupdate*, lock, tpublished, flag, editor_note) if an ?auth_code= was supplied.

Update core+release PUT /projects/<name>.json

For changing general project information, all fields can be sent wrapped in the same structure. The only difference is the required auth_code.

{
    "auth_code": "plain_pw_123",
    "project": {
        "title": "New Title",
        "summary": "Shorter project slogan",
        "description": "New and longer description...",
        "image": "http://new.png",
        "license_list": "BSDL",
        "project_tags": "qt6, dlang, desktop, calendar",
        "author": "Elise Exemplary, dj_ical@launchpad",
        "urls": {
            "homepage": "http://google-code.com",
            "DEB": "http://example.org/SECOND-REALEASE.deb",
            "download": "http://example.org/dwnld/"
        },
        "version": "6.0.2",
        "release_tags": "stable, minor bugfix, security",
        "changes": "Fixed the thing, and added the stuff.",
        "download": "http://example.org/dwnld/",
    }
}

Now interestingly, this scheme can be used to:

  1. Just update the basic project description.
  2. Or publish new release "version" and "changes" right along.

Either, or, and both.

Sectioned variant

As variation of that, the JSON struct may be split into:

{
    "auth_code": "plain_pw_123",
    "project": {
        "title": "New Title",
        "summary": "Shorter project slogan",
        "description": "New and longer description...",
        "image": "http://new.png",
        "license_list": "BSDL",
        "project_tags": "qt6, dlang, desktop, calendar",
        "urls": {
            "homepage": "http://google-code.com",
            "DEB": "http://example.org/SECOND-REALEASE.deb",
            "download": "http://example.org/dwnld/"
        }
    },
    "release": {
        "version": "6.0.2",
        "release_tags": "stable, minor bugfix",
        "changes": "Fixed the thing, and added the stuff.",
        "download": "http://example.org/dwnld/",
        "hide": false,
    }
}

It'll simply be rejoined server-side. Which doesn't make a difference to the internal database scheme. Might be more convenient to implement.

URL section moved out

And as further variation, you can also split out the url:{} dict:

{
    "auth_code": "plain_pw_123",
    "project": {
        "title": "New Title",
        "description": "................"
        "editor_note": "Please reset the social bookmark counter!
                        Project moved from Sourceforge to GitHub."
    },
    "urls": {
        "homepage": "http://example.org/",
        "proprietary-hoster": "http://github.com/proj/name/",
        "PYZ-package": "http://example.org/calendar.pyzw",
    }
    "release": {
        "version": "7.5.3",
        "changes": "Rome sprang up...",
    }
}

The API is pretty much indifferent if this wraps up just the "project" base information, a lone "urls" dict, or just a "release" or any combination thereof.

URLs dict GET/PUSH /projects/<name>/urls.json

There's still a separate API endpoint for just retrieving or updating the URLs though:

{
    "auth_code": "plain_pw_123",
    "urls": {
        "Homepage": "http://example.org/",
        "Screenshot": "http://launchpad.com/proj/screenshot.png",
        "Custom-link": "http://example.com/share/help/C/ping/index.page",
    }
}

Note that a projects "homepage" or "download" or "image" URL only get updated when listed herein. All other project links get completely discarded and replaced by whatever is in the new list.

  • Incoming link titles are free-form.
  • The case is preserved for custom entries.
  • Non-word characters are replaced.
  • Titles will always be stored with "Dashed-Names" (it's internally a Key=Url text/yaml field).
  • The core URLs (homepage, download, screenshot) however are always lowercased (when returned on GET requests).
  • The API is indifferent if you pass the "homepage" or "download" link as literal project:{…} dict entry, or wrapped in urls:{…}.

Release publishing POST /projects/<name>/releases.json

This is entirely redundant now. But a new release can be published with that alternative URL scheme. It's basically identical to "PUT /project/<name>.json" now:

{
    "auth_code": "plain_pw_123",
    "release": {
        "version": "0.0.1",
        "release_tags": "beta, cleanup",
        "changes": "Initial release. No docs or code.",
        "hide": true,
    }
}

Note that even the "hide" flag is already available with regular project PUT / core updates.

Withdraw_release DELETE /projects/<name>/<versionstr>.json

Now release retraction became simpler as well. There are no longer arbitrary numeric IDs exposed. (They were faked anyway.)

Instead the literal version number can now be used to delete a release entry.

  • DELETE /projects/<name>/1.2.3-rc2.json?auth_code=pw123

The version string has to be URL-encoded as needed of course.

New_project CREATE /projects/<name>.json

The JSON payload for creating a new project might look familiar. Well in fact, it is identical to the regular PUT payload:

{
    "auth_code": "this:is-a/brand#new+password_567",
    "project": {
        "title": "Initial Title",
        "summary": "ONELINER",
        "description": "Description REQUIRED.",
        "project_tags": "bash, script",
        "license": null,
        "author": "Username, user@localhost",
        "urls": {
            "screenshot": null,
            "Homepage": "http://example.org/",
        },
        "version": null,
        "release_tags": "initial, security bugfix",
        "changes": "..."
    }
}

The null fields are just for brevity. In fact the first project submission can and should even contain a full record - including the current release version/info. (CREATE cannot be used to register all-empty or stub project records.)

It can however also just be completed with the next PUT/Update. The only requirements for CREATE are a new Unix project<name> (taken from the request URL), and a "title" and "description", "license" or "project_tags", and a "homepage" URL.

And obviously this is the only time the ‹auth_code› is used for populating the internal authorization hash. All other requests just compare it, but CREATE does the creative step.

Client SSL cert as CREATE spamguard: Undecided / Testing.
See also /doc/trunk/doc/submit.pem for public authorization key.

Responses

Successful submissions will be answered with HTTP Status 201 and:

{
    "success": true
}

Whereas errors will typically carry a HTTP status error and some vaguely interesting notes in:

{
    "error": "Missing project title, summary, description, homepage.",
    "project": {
        "name": null
    },
    "data": null
}

HTTP status codes:

  • "200 No content" for unknow project names.
  • "201 Created" for successful publication.
  • "401 Unauthorized" for an invalid auth_code.
  • "410 Obsolete" for GET /proj/releases.json.
  • "495 Client Cert" on CREATE requests without submit.pem
  • "500 DB Error" for server failures.
  • "501 Not implemented" for unknown request methods/paths.
  • "503 Unavailable" is unlikely to happen.

Security considerations

Now that's slightly irrelevant for API usage, in particular for just polling or updating your own project records.

  • But the naming scheme on freshcode also permits project names like commonmark.js or table-generator.sh. So some care should be taken before using the returned project names as literal filenames.
  • Description fields are mostly just filtered for printable characters.
  • Only Homepage and Download URLs are guaranteed to be HTTP references.