State: design, testing

Simpler JSON API

To update and submit project records, there's going to be a simplified JSON interface.

• The testing API endpoint is https://test.freshcode.club/…,
  which can be used for browsing as well.

• The regular API endpoint is https://api.freshcode.club/…,
  but is currently fixated on the test database too.

URL scheme

Updating and release publishing can now be combined into one PUT/POST request. The URL dict can now be embedded everywhere or passed alongside.

method	URL path	function+purpose
GET	/projects/<name>.json	query all fields
PUT	/projects/<name>.json	update_core+release
CREATE	/projects/<name>.json	new_project+update
POST	/projects/<name>/releases.json	publish
DELETE	/projects/<name>/releases/<verstr>.json?auth_code=	withdraw version
PUSH	/projects/<name>/urls.json	update_urls
GET	/feed/<name>.json	release_list+federation

The PUT method likewise works with POST or PUSH ("PUSH" because it's a programming site, and entries are basically a stack). Fields are equivalent for the initializing CREATE.

JSON object keys

Where the JSON field names are mapped as follows:

<style> .content table { width: 95%; } .content table tr td:nth-child(5) { color: #99c; font-style: italic; } .content table tr td:nth-child(3) { font-weight: bold; color: #272; } .content table tr td:nth-child(1) { color: #bbb; font-weight: bold; } </style>

works with	freshcode DB	JSON field	freecode-submit	Sample data
CREATE	name	<name>	Project	unix-name
PUT	title	title	Project	Foo Bar Gtk+
PUT	summary	summary	Summary	short slogan
PUT	description	description	Description	Long text...
PUT	tags	project_tags	Project-Tag-List	c, c++, cli
PUT	license	license_list	License-List	MITL, GNU GPL
PUT, URL	homepage	urls {…}	Homepage-URL	http://exmpl
PUT, URL	homepage	urls {…}	Website-URL	http://exmpl
PUT, URL	image:	urls {…}	Screenshot-URL	http://png
PUT, URL	autoupdate_url	urls {…}	Changelog-URL	.../NEWS.md
PUT, URL	urls {…}	urls {…}	*******-URL	Title = http://...
RELEASE	download	download	Download-URL	http://sf.net
RELEASE	version	version	Version	1.0.3-rc5
RELEASE	changes	changes	Changes	Added Xy. Fixed Zy.
RELEASE	hidden	hide	Hide	0
RELEASE	state	release_tags	Release-Tag-List	stable
RELEASE	scope	release_tags	Release-Tag-List	minor, bugfix
CREATE	submitter	author	Author	Bob
CREATE	submitter_img	-	Author	Bob, bob@gitub
-	editor_note	-	-	-
-	flag	-	-	-
CREATE	lock	<auth_code>	.netrc	-
-	social_links	-	-	-
-	autoupdate_*	-	-	-
DELETE	name	<name>	-P name	unix-name
DELETE	deleted	<versionstr>	-d	1.0.3-rc5

• Renamed: permalink → name, and changelog → changes
• New: title, summary (oneliner), urls as dict {…}, author for submitter
• Changes: project_tag_list is now project_tag_list and CSV text instead of a JSON list, same for license_list with spacing irrelevant
• release_tags could perhaps be split up, but the update handler itself separates "stable/beta/prerelease" from "minor/major/bugfix/feature/security" etc.
• Internal / Unassigned yet: editor_note, flag, lock, social_links, autoupdate_***

Authorization:

All requests can append an ?auth_code= to request URLs to carry a plain password. Which is primarily used with the DELETE method. The GET method only hides a few internal fields when unauthorized.

POST/PUT/etc. requests instead should wrap the password outermost in the JSON payload:

{
    "auth_code": "plain_pw_123",
    "project": {
        "key": "Value",
        ...
    }
}

Query GET /projects/<name>.json

Returns an almost literal database dump:

{
    "project": {
        "name": "foo-bar",
        "title": "Foo Bar",
        "summary": "iCal-compatible calendaring app",
        "description": "Here comes a very long description ...",
        "image": "",
        "author": "Bob",
        "license_list": "MITL, CC-BY-SA",
        "project_tags": "desktop, c++, qt5, calendar",
        "homepage": "http://example.org",
        "download": "http://example.org/dwnld/",
        "urls": {
            "Homepage": "http://example.org",
            "Changelog": "https://example.org/downl/NEWS.md",
            "DEB": "http://example.org/downl/foo-bar_1.deb",
            "Download": "http://example.org/dwnld/"
        },
        "version": "5.0.1",
        "state": "stable",
        "scope": "minor bugfix",
        "release_tags": "stable, minor bugfix",
        "changes": "Fixed the thing, and added the stuff.",
        "download": "http://example.org/dwnld/",
        "t_published": "1427461501",
        "t_changed": "1452413135"
    },
    "$feed-license": "CC-BY-SA 3.0",
    "$feed-origin": "http://freshcode.club/"
}

The release fields (second half in the example) represent the latest known version.

• The URLs are partly duplicated. Because "homepage" and "download" are core fields, they show up in two places.
• This GET response blob might carry a few more internal fields (autoupdate_*, lock, t_published, flag, editor_note) if an ?auth_code= was supplied.

Update core+release PUT /projects/<name>.json

For changing general project information, all fields can be sent wrapped in the same structure. The only difference is the required auth_code.

{
    "auth_code": "plain_pw_123",
    "project": {
        "title": "New Title",
        "summary": "Shorter project slogan",
        "description": "New and longer description...",
        "image": "http://new.png",
        "license_list": "BSDL",
        "project_tags": "qt6, dlang, desktop, calendar",
        "author": "Elise Exemplary, dj_ical@launchpad",
        "urls": {
            "homepage": "http://google-code.com",
            "DEB": "http://example.org/SECOND-REALEASE.deb",
            "download": "http://example.org/dwnld/"
        },
        "version": "6.0.2",
        "release_tags": "stable, minor bugfix, security",
        "changes": "Fixed the thing, and added the stuff.",
        "download": "http://example.org/dwnld/",
    }
}

Now interestingly, this scheme can be used to:

1. Just update the basic project description.
2. Or publish new release "version" and "changes" right along.

Either, or, and both.

Sectioned variant

As variation of that, the JSON struct may be split into:

{
    "auth_code": "plain_pw_123",
    "project": {
        "title": "New Title",
        "summary": "Shorter project slogan",
        "description": "New and longer description...",
        "image": "http://new.png",
        "license_list": "BSDL",
        "project_tags": "qt6, dlang, desktop, calendar",
        "urls": {
            "homepage": "http://google-code.com",
            "DEB": "http://example.org/SECOND-REALEASE.deb",
            "download": "http://example.org/dwnld/"
        }
    },
    "release": {
        "version": "6.0.2",
        "release_tags": "stable, minor bugfix",
        "changes": "Fixed the thing, and added the stuff.",
        "download": "http://example.org/dwnld/",
        "hide": false,
    }
}

It'll simply be rejoined server-side. Which doesn't make a difference to the internal database scheme. Might be more convenient to implement.

URL section moved out

And as further variation, you can also split out the url:{} dict:

{
    "auth_code": "plain_pw_123",
    "project": {
        "title": "New Title",
        "description": "................"
        "editor_note": "Please reset the social bookmark counter!
                        Project moved from Sourceforge to GitHub."
    },
    "urls": {
        "homepage": "http://example.org/",
        "proprietary-hoster": "http://github.com/proj/name/",
        "PYZ-package": "http://example.org/calendar.pyzw",
    }
    "release": {
        "version": "7.5.3",
        "changes": "Rome sprang up...",
    }
}

The API is pretty much indifferent if this wraps up just the "project" base information, a lone "urls" dict, or just a "release" or any combination thereof.

URLs dict GET/PUSH /projects/<name>/urls.json

There's still a separate API endpoint for just retrieving or updating the URLs though:

{
    "auth_code": "plain_pw_123",
    "urls": {
        "Homepage": "http://example.org/",
        "Screenshot": "http://launchpad.com/proj/screenshot.png",
        "Custom-link": "http://example.com/share/help/C/ping/index.page",
    }
}

Note that a projects "homepage" or "download" or "image" URL only get updated when listed herein. All other project links get completely discarded and replaced by whatever is in the new list.

• Incoming link titles are free-form.
• The case is preserved for custom entries.
• Non-word characters are replaced.
• Titles will always be stored with "Dashed-Names" (it's internally a Key=Url text/yaml field).
• The core URLs (homepage, download, screenshot) however are always lowercased (when returned on GET requests).
• The API is indifferent if you pass the "homepage" or "download" link as literal project:{…} dict entry, or wrapped in urls:{…}.

Release publishing POST /projects/<name>/releases.json

This is entirely redundant now. But a new release can be published with that alternative URL scheme. It's basically identical to "PUT /project/<name>.json" now:

{
    "auth_code": "plain_pw_123",
    "release": {
        "version": "0.0.1",
        "release_tags": "beta, cleanup",
        "changes": "Initial release. No docs or code.",
        "hide": true,
    }
}

Note that even the "hide" flag is already available with regular project PUT / core updates.

Withdraw_release DELETE /projects/<name>/<versionstr>.json

Now release retraction became simpler as well. There are no longer arbitrary numeric IDs exposed. (They were faked anyway.)

Instead the literal version number can now be used to delete a release entry.

• DELETE /projects/<name>/1.2.3-rc2.json?auth_code=pw123

The version string has to be URL-encoded as needed of course.

New_project CREATE /projects/<name>.json

The JSON payload for creating a new project might look familiar. Well in fact, it is identical to the regular PUT payload:

{
    "auth_code": "this:is-a/brand#new+password_567",
    "project": {
        "title": "Initial Title",
        "summary": "ONELINER",
        "description": "Description REQUIRED.",
        "project_tags": "bash, script",
        "license": null,
        "author": "Username, user@localhost",
        "urls": {
            "screenshot": null,
            "Homepage": "http://example.org/",
        },
        "version": null,
        "release_tags": "initial, security bugfix",
        "changes": "..."
    }
}

The null fields are just for brevity. In fact the first project submission can and should even contain a full record - including the current release version/info. (CREATE cannot be used to register all-empty or stub project records.)

It can however also just be completed with the next PUT/Update. The only requirements for CREATE are a new Unix project<name> (taken from the request URL), and a "title" and "description", "license" or "project_tags", and a "homepage" URL.

And obviously this is the only time the ‹auth_code› is used for populating the internal authorization hash. All other requests just compare it, but CREATE does the creative step.

Client SSL cert as CREATE spamguard: Undecided / Testing.
See also /doc/trunk/doc/submit.pem for public authorization key.

Responses

Successful submissions will be answered with HTTP Status 201 and:

{
    "success": true
}

Whereas errors will typically carry a HTTP status error and some vaguely interesting notes in:

{
    "error": "Missing project title, summary, description, homepage.",
    "project": {
        "name": null
    },
    "data": null
}

HTTP status codes:

• "200 No content" for unknow project names.
• "201 Created" for successful publication.
• "401 Unauthorized" for an invalid auth_code.
• "410 Obsolete" for GET /proj/releases.json.
• "495 Client Cert" on CREATE requests without submit.pem
• "500 DB Error" for server failures.
• "501 Not implemented" for unknown request methods/paths.
• "503 Unavailable" is unlikely to happen.

Security considerations

Now that's slightly irrelevant for API usage, in particular for just polling or updating your own project records.

• But the naming scheme on freshcode also permits project names like commonmark.js or table-generator.sh. So some care should be taken before using the returned project names as literal filenames.
• Description fields are mostly just filtered for printable characters.
• Only Homepage and Download URLs are guaranteed to be HTTP references.