Check-in [3ffaf89706]
Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Patched fc-submit for Python3 and using `requests` instead of `urllib2`; which is kind of necessary due to required SNI for SSL/TLS, usage of verify=cacert.pem should be possibly patched in as well |
---|---|
Downloads: | Tarball | ZIP archive | SQL archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA1: |
3ffaf89706cdbb20c9b1a9b7817ae840 |
User & Date: | mario 2014-07-30 14:25:15 |
Context
2014-07-31
| ||
21:56 | New additions: Linux AppFinder, Find Best OpenSource, FOSS Wiki, Libre Projects, and Zwodnik, a few more links in the sidebar. check-in: 0232a9a82d user: mario tags: trunk | |
2014-07-30
| ||
14:25 | Patched fc-submit for Python3 and using `requests` instead of `urllib2`; which is kind of necessary due to required SNI for SSL/TLS, usage of verify=cacert.pem should be possibly patched in as well check-in: 3ffaf89706 user: mario tags: trunk | |
14:18 | Made `?auth_code` optional for dispatching onto freecode API handler (else it would show up regular page instead of filtered JSON). check-in: 74758b8842 user: mario tags: trunk | |
Changes
Added doc/cacert.pem.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 | -----BEGIN CERTIFICATE----- MIIHPTCCBSWgAwIBAgIBADANBgkqhkiG9w0BAQQFADB5MRAwDgYDVQQKEwdSb290 IENBMR4wHAYDVQQLExVodHRwOi8vd3d3LmNhY2VydC5vcmcxIjAgBgNVBAMTGUNB IENlcnQgU2lnbmluZyBBdXRob3JpdHkxITAfBgkqhkiG9w0BCQEWEnN1cHBvcnRA Y2FjZXJ0Lm9yZzAeFw0wMzAzMzAxMjI5NDlaFw0zMzAzMjkxMjI5NDlaMHkxEDAO BgNVBAoTB1Jvb3QgQ0ExHjAcBgNVBAsTFWh0dHA6Ly93d3cuY2FjZXJ0Lm9yZzEi MCAGA1UEAxMZQ0EgQ2VydCBTaWduaW5nIEF1dGhvcml0eTEhMB8GCSqGSIb3DQEJ ARYSc3VwcG9ydEBjYWNlcnQub3JnMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIIC CgKCAgEAziLA4kZ97DYoB1CW8qAzQIxL8TtmPzHlawI229Z89vGIj053NgVBlfkJ 8BLPRoZzYLdufujAWGSuzbCtRRcMY/pnCujW0r8+55jE8Ez64AO7NV1sId6eINm6 zWYyN3L69wj1x81YyY7nDl7qPv4coRQKFWyGhFtkZip6qUtTefWIonvuLwphK42y fk1WpRPs6tqSnqxEQR5YYGUFZvjARL3LlPdCfgv3ZWiYUQXw8wWRBB0bF4LsyFe7 w2t6iPGwcswlWyCR7BYCEo8y6RcYSNDHBS4CMEK4JZwFaz+qOqfrU0j36NK2B5jc G8Y0f3/JHIJ6BVgrCFvzOKKrF11myZjXnhCLotLddJr3cQxyYN/Nb5gznZY0dj4k epKwDpUeb+agRThHqtdB7Uq3EvbXG4OKDy7YCbZZ16oE/9KTfWgu3YtLq1i6L43q laegw1SJpfvbi1EinbLDvhG+LJGGi5Z4rSDTii8aP8bQUWWHIbEZAWV/RRyH9XzQ QUxPKZgh/TMfdQwEUfoZd9vUFBzugcMd9Zi3aQaRIt0AUMyBMawSB3s42mhb5ivU fslfrejrckzzAeVLIL+aplfKkQABi6F1ITe1Yw1nPkZPcCBnzsXWWdsC4PDSy826 YreQQejdIOQpvGQpQsgi3Hia/0PsmBsJUUtaWsJx8cTLc6nloQsCAwEAAaOCAc4w ggHKMB0GA1UdDgQWBBQWtTIb1Mfz4OaO873SsDrusjkY0TCBowYDVR0jBIGbMIGY gBQWtTIb1Mfz4OaO873SsDrusjkY0aF9pHsweTEQMA4GA1UEChMHUm9vdCBDQTEe MBwGA1UECxMVaHR0cDovL3d3dy5jYWNlcnQub3JnMSIwIAYDVQQDExlDQSBDZXJ0 IFNpZ25pbmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcNAQkBFhJzdXBwb3J0QGNhY2Vy dC5vcmeCAQAwDwYDVR0TAQH/BAUwAwEB/zAyBgNVHR8EKzApMCegJaAjhiFodHRw czovL3d3dy5jYWNlcnQub3JnL3Jldm9rZS5jcmwwMAYJYIZIAYb4QgEEBCMWIWh0 dHBzOi8vd3d3LmNhY2VydC5vcmcvcmV2b2tlLmNybDA0BglghkgBhvhCAQgEJxYl aHR0cDovL3d3dy5jYWNlcnQub3JnL2luZGV4LnBocD9pZD0xMDBWBglghkgBhvhC AQ0ESRZHVG8gZ2V0IHlvdXIgb3duIGNlcnRpZmljYXRlIGZvciBGUkVFIGhlYWQg b3ZlciB0byBodHRwOi8vd3d3LmNhY2VydC5vcmcwDQYJKoZIhvcNAQEEBQADggIB ACjH7pyCArpcgBLKNQodgW+JapnM8mgPf6fhjViVPr3yBsOQWqy1YPaZQwGjiHCc nWKdpIevZ1gNMDY75q1I08t0AoZxPuIrA2jxNGJARjtT6ij0rPtmlVOKTV39O9lg 18p5aTuxZZKmxoGCXJzN600BiqXfEVWqFcofN8CCmHBh22p8lqOOLlQ+TyGpkO/c gr/c6EWtTZBzCDyUZbAEmXZ/4rzCahWqlwQ3JNgelE5tDlG+1sSPypZt90Pf6DBl Jzt7u0NDY8RD97LsaMzhGY4i+5jhe1o+ATc7iwiwovOVThrLm82asduycPAtStvY sONvRUgzEv/+PDIqVPfE94rwiCPCR/5kenHA0R6mY7AHfqQv0wGP3J8rtsYIqQ+T SCX8Ev2fQtzzxD72V7DX3WnRBnc0CkvSyqD/HMaMyRa+xMwyN2hzXwj7UfdJUzYF CpUCTPJ5GhD22Dp1nPMd8aINcGeGG7MW9S/lpOt5hvk9C8JzC6WZrG/8Z7jlLwum GCSNe9FINSkYQKyTYOGWhlC0elnYjyELn8+CkcY7v2vcB5G5l1YjqrZslMZIBjzk zk6q5PYvCdxTby78dOs6Y5nCpqyJvKeyRKANihDjbPIky/qbn3BHLt4Ui9SyIAmW omTxJBzcoTWcFbLUvFUufQb1nA5V9FrWk9p2rSVzTMVD -----END CERTIFICATE----- |
Changes to doc/fc-submit.
|
| | | | | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | #!/usr/bin/env python3 """ freshcode-submit -- script transactions with the Freshcode.club server """ import sys, re, requests, json, netrc, email.parser, optparse version = "3.0" class Update: "Encapsulate dictionaries describing a project metadata update." def __init__(self): self.name = None self.per_project = {} self.urlassoc = [] |
︙ | ︙ | |||
34 35 36 37 38 39 40 | # 404 Not Found - The requested resource was not found # 409 Conflict - The validation of your submitted data failed, please check # the response body for error pointers # 500 Server Error - The request hit a problem and was aborted, please report # as a bug if it persists # 503 Service Unavailable - You hit your API credit limit | | | | < < < < < < < < < | < < | 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | # 404 Not Found - The requested resource was not found # 409 Conflict - The validation of your submitted data failed, please check # the response body for error pointers # 500 Server Error - The request hit a problem and was aborted, please report # as a bug if it persists # 503 Service Unavailable - You hit your API credit limit def RequestWithMethod(method, url, **kwargs): """requests.request is really a drop-in replacement here; with TLS-SNI support""" # Here verify="cacert.pem" would better solve the certificate issue return requests.request(method, url, verify=False, **kwargs) class FreecodeSessionException(Exception): "Carry exception state when a session blows up." def __init__(self, msg): Exception.__init__(self) self.msg = msg |
︙ | ︙ | |||
73 74 75 76 77 78 79 | self.permalink = None self.id = None self.project_data = None # If user didn't supply credentials, fetch from ~/.netrc if not self.auth: try: credentials = netrc.netrc() | | | | | | | | < | | | | | | | | | | 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 | self.permalink = None self.id = None self.project_data = None # If user didn't supply credentials, fetch from ~/.netrc if not self.auth: try: credentials = netrc.netrc() except netrc.NetrcParseError as e: raise FreecodeSessionException("ill-formed .netrc: %s:%s %s" \ % (e.filename, e.lineno, e.msg)) except IOError as e: raise FreecodeSessionException(("missing .netrc file %s" % \ str(e).split()[-1])) ret = credentials.authenticators("freshcode") if not ret: raise FreecodeSessionException("no credentials for Freshcode") _login, self.auth, _password = ret def on_project(self, name): "Select project by Freecode shortname." if self.verbose: print(("Selecting project: %s" % name)) self.project = name pquery = FreecodeSession.server + "projects/%s.json?auth_code=%s" \ % (self.project, self.auth) handle = RequestWithMethod("GET", url=pquery) content = json.loads(handle.text) self.project_data = content['project'] #if self.verbose: # print "Project data: %s" % self.project_data self.permalink = self.project_data['permalink'] self.id = self.project_data['id'] def edit_request(self, url, method="GET", request=None, force=False): "Wrap a JSON object with the auth code and ship it as a request" if request is None: request = {} url = FreecodeSession.server + url data = {"auth_code" : self.auth} data.update(request) data = json.dumps(data) headers = {"Content-Type" : "application/json"} if self.verbose: print(("Request URL:", method, url)) #if self.verbose: # print "Request headers:", headers if self.verbose: print(("Request data:", data)) if self.emit_enable or force: req = RequestWithMethod(method=method, url=url, data=data, headers=headers) if self.verbose: print(req.status_code, req.url, req.headers) content = req.text if self.verbose: print(("Response:", content)) return content def publish_release(self, data): "Add a new release to the current project." if self.verbose: print(("Publishing %s release: %s" % (self.project, repr(data)))) self.edit_request("projects/" + self.permalink + "/releases.json", "POST", {"release": data}) def withdraw_release(self, dversion): "Withdraw a specified release from the current project." if self.verbose: print("Withdrawing %s release: %s" % (self.project, dversion)) releases = self.edit_request("projects/%s/releases/pending.json" \ % self.permalink, force=True) releases = json.loads(releases) for release in releases: properties = release["release"] if properties.get("version") == dversion: vid = properties["id"] break else: raise FreecodeSessionException("couldn't find release %s"%dversion) deletelink = "projects/%s/releases/%s.json" % (self.permalink, vid) self.edit_request(deletelink, "DELETE", {}) def update_core(self, coredata): "Update the core data for a project." if self.verbose: print("Core data update for %s is: %s" % (self.project, coredata)) self.edit_request("projects/" + self.permalink + ".json", "PUT", {"project": coredata}) def update_urls(self, urlassoc): "Update URL list for a project." if self.verbose: print("URL list update for %s is: %s" % (self.project, urlassoc)) # First, get permalinks for all existing URLs uquery = FreecodeSession.server + "projects/%s/urls.json?auth_code=%s" \ % (self.permalink, self.auth) handle = RequestWithMethod("GET", url=uquery) content = json.loads(handle.text) permadict = content['urls'] # Just send the new dict over self.edit_request("projects/%s/urls.json" % (self.permalink), "PUSH", {"urls" : dict(urlassoc)}) class FreecodeMetadataFactory: |
︙ | ︙ | |||
196 197 198 199 200 201 202 | projectwide = ('name', 'description', 'oneliner', 'license_list', 'project_tag_list') def __init__(self): | | | 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 | projectwide = ('name', 'description', 'oneliner', 'license_list', 'project_tag_list') def __init__(self): self.message_parser = email.parser.Parser() self.argument_parser = optparse.OptionParser( \ usage="usage: %prog [options]") for (msg_field, shortopt, rpc_field) in FreecodeMetadataFactory.freecode_field_map: self.argument_parser.add_option("-" + shortopt, "--" + msg_field.lower(), dest=rpc_field, help="Set the %s field"%msg_field) |
︙ | ︙ | |||
239 240 241 242 243 244 245 | data = {} urls = {} (options, _args) = self.argument_parser.parse_args() # Stuff from stdin if present prior_version = data.get("version") if not (options.query or options.showversion) and options.read: message = self.message_parser.parse(stream) | | | | | | | | 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 | data = {} urls = {} (options, _args) = self.argument_parser.parse_args() # Stuff from stdin if present prior_version = data.get("version") if not (options.query or options.showversion) and options.read: message = self.message_parser.parse(stream) for (key, value) in list(message.items()): value = re.sub("\n +", " ", value).strip() if key.endswith("-URL"): key = key.replace("-", " ") urls.update({key[:-4] : value}) else: if key.endswith("List"): value = [x.strip() for x in value.split()] data.update({FreecodeMetadataFactory.header_to_field(key) : value}) if not 'changelog' in data: payload = message.get_payload().strip() if payload: data['changelog'] = payload + "\n" if prior_version and data.get("version") != prior_version: raise FreecodeSessionException("Version conflict on stdin.") # Merge in options from the command line; # they override what's on stdin. controls = ('query', 'delete', 'read', 'dryrun', 'verbose', 'showversion') prior_version = data.get("version") for (key, value) in list(options.__dict__.items()): if key not in controls and value != None: data[key] = value del options.__dict__[key] if prior_version and data.get("version") != prior_version and not options.delete: raise FreecodeSessionException("Version conflict in options.") # Hidden flag special handling if "hidden_from_frontpage" in data: data["hidden_from_frontpage"] = data["hidden_from_frontpage"] in ("Y", "y") # Now merge in the URLs, doing symbol substitution urllist = [] for (label, furl) in list(urls.items()): for (k, v) in list(data.items()): if type(v) == type(""): furl = furl.replace('${' + k + '}', v) urllist.append((label, furl)) # Sort out what things go where update = Update() if options.showversion: pass elif options.query: update.name = options.query else: update.name = data.pop('name') update.urlassoc = urllist for (k, v) in list(data.items()): if k in FreecodeMetadataFactory.projectwide: # Hack to get around a namespace collision if k == "project_release_tag": k = "release_tag" update.per_project[k] = v else: update.per_release[k] = v |
︙ | ︙ | |||
306 307 308 309 310 311 312 | # Some switches shouldn't be passed to the server query = 'query' in options.__dict__ and options.query verbose = 'verbose' in options.__dict__ and options.verbose delete = 'delete' in options.__dict__ and options.delete dryrun = 'dryrun' in options.__dict__ and options.dryrun showversion = 'showversion' in options.__dict__ and options.showversion if showversion: | | | | | | | | | | | | | | | | | | | | | | | | | | 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 | # Some switches shouldn't be passed to the server query = 'query' in options.__dict__ and options.query verbose = 'verbose' in options.__dict__ and options.verbose delete = 'delete' in options.__dict__ and options.delete dryrun = 'dryrun' in options.__dict__ and options.dryrun showversion = 'showversion' in options.__dict__ and options.showversion if showversion: print("freshcode-submit", version) raise SystemExit(0) # Time to ship the update. # Establish session session = FreecodeSession(verbose=int(verbose), emit_enable=not dryrun) try: session.on_project(update.name) except ValueError as e: print(e) print("freshcode-submit: looks like a server-side problem at freshcode.club; bailing out.", file=sys.stderr) raise SystemExit(1) if options.query: print("Project: %s" % session.project_data["name"]) print("Summary: %s" % session.project_data["oneliner"]) print("Description: %s" % session.project_data["description"].replace("\n", "\n ").rstrip()) print("License-List: %s" % ",".join(session.project_data["license_list"])) print("Project-Tag-List: %s" % ",".join(session.project_data["tag_list"])) for assoc in session.project_data['approved_urls']: #print "Querying", assoc["redirector"] #req = RequestWithMethod(method="HEAD", # url=assoc["redirector"], # data={}, # headers={}) #handle = urllib2.urlopen(req) #print "===" #print handle.info() #print "===" print("%s-URL: %s" % (assoc["label"].replace(" ","-"), assoc["redirector"])) if 'recent_releases' in session.project_data and session.project_data['recent_releases']: most_recent = session.project_data['recent_releases'][0] print("Version: %s" % most_recent['version']) print("Tag-List: %s" % ",".join(most_recent['tag_list'])) if most_recent.get('hidden_from_frontpage'): print("Hide: Y") else: print("Hide: N") print("") print(most_recent["changelog"]) else: # OK, now actually add or delete the release. if update.per_project: session.update_core(update.per_project) if update.urlassoc: session.update_urls(update.urlassoc) if delete: session.withdraw_release(update.per_release['version']) elif update.per_release and list(update.per_release.keys())!=["version"]: session.publish_release(update.per_release) except FreecodeSessionException as e: print("freshcode-submit:", e.msg, file=sys.stderr) sys.exit(1) except requests.exceptions.HTTPError as f: print("freshcode-submit: HTTPError %s" % (f.code), file=sys.stderr) print(f.read(), file=sys.stderr) sys.exit(1) except requests.exceptions.RequestException as f: print("freshcode-submit: URLError %s" % (f.reason,), file=sys.stderr) sys.exit(1) # end |