Check-in [0afbbc5c26]
Overview
Comment: | Implement "buffy" mode for just keeping Xiphs YP.XML in memory once traversed. |
---|---|
Downloads: | Tarball | ZIP archive | SQL archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA1: |
0afbbc5c263476a04dcc0e7ae74763d8 |
User & Date: | mario on 2015-05-03 09:35:10 |
Other Links: | manifest | tags |
Context
2015-05-03
| ||
14:07 | Remove print statement. check-in: cf6b582c7e user: mario tags: trunk | |
09:35 | Implement "buffy" mode for just keeping Xiphs YP.XML in memory once traversed. check-in: 0afbbc5c26 user: mario tags: trunk | |
09:25 | Consolidate bitrate filter in main update_streams() method. Fix conjoined category strings. check-in: 2793a3e6f8 user: mario tags: trunk | |
Changes
Modified channels/xiph.py from [73380fcbf4] to [6db1c9f004].
︙ | ︙ | |||
26 27 28 29 30 31 32 | # The category list is hardwired in this plugin. And there # are three station fetching modes now: # # → "JSON cache" retrieves a refurbished JSON station list, # both sliceable genres and searchable. # # → "Clunky XML" fetches the olden YP.XML, which is really | | > | 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 | # The category list is hardwired in this plugin. And there # are three station fetching modes now: # # → "JSON cache" retrieves a refurbished JSON station list, # both sliceable genres and searchable. # # → "Clunky XML" fetches the olden YP.XML, which is really # slow, then slices out genres. No search. With the secret # "buffy" mode keeps all streams buffered. # # → "Forbidden Fruits" extracts from dir.xiph.org HTML pages, # with homepages and listener/max infos available. Search # is also possible. # # The bitrate filter can strip any low-quality entries, but # retains `0` entries (which just lack meta information and |
︙ | ︙ | |||
113 114 115 116 117 118 119 | data = ahttp.get(self.json_url, params=params) #log.DATA(data) #-- extract l = [] data = json.loads(data) for e in data: | < | 114 115 116 117 118 119 120 121 122 123 124 125 126 127 | data = ahttp.get(self.json_url, params=params) #log.DATA(data) #-- extract l = [] data = json.loads(data) for e in data: if not len(l) or l[-1]["title"] != e["stream_name"]: l.append({ "title": e["stream_name"], "url": e["listen_url"], "format": e["type"], "bitrate": int(e["bitrate"]), "genre": e["genre"], |
︙ | ︙ | |||
143 144 145 146 147 148 149 | # But it's a huge waste of memory to keep it around for unused # categories. Extracting all streams{} at once would be worse. Yet # enabling this buffer method prevents partial reloading.. if conf.xiph_source != "buffy": buffy = [] # Get XML blob | > | > | | 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 | # But it's a huge waste of memory to keep it around for unused # categories. Extracting all streams{} at once would be worse. Yet # enabling this buffer method prevents partial reloading.. if conf.xiph_source != "buffy": buffy = [] # Get XML blob if not buffy: yp = ahttp.get(self.xml_url, statusmsg="Brace yourselves, still downloading the yp.xml blob.") else: yp = "<none/>" self.status("Yes, XML parsing isn't much faster either.", timeout=20) for entry in xml.dom.minidom.parseString(yp).getElementsByTagName("entry"): buffy.append({ "title": x(entry, "server_name"), "url": x(entry, "listen_url"), "format": self.mime_fmt(x(entry, "server_type")[6:]), "bitrate": bitrate(x(entry, "bitrate")), |
︙ | ︙ | |||
166 167 168 169 170 171 172 | }) self.status("This. Is. Happening. Now.") # Filter out a single subtree l = [] if cat: rx = re.compile(cat.lower()) | < < | < > | | 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 | }) self.status("This. Is. Happening. Now.") # Filter out a single subtree l = [] if cat: rx = re.compile(cat.lower()) l = [row for row in buffy if rx.search(row["genre"])] # Search is actually no problem. Just don't want to. Nobody is using the YP.XML mode anyway.. elif search: pass # Result category return l # Fetch directly from website. Which Xiph does not approve of; but |
︙ | ︙ |