Forum - View topicRelease scraper problem
|
Author | Message | ||
---|---|---|---|
BigOnAnime
Encyclopedia Editor
Posts: 1230 Location: Minnesota, USA |
|
||
I've brought this up in the thread for release corrections a few times, but I might as well bring it up here too to bring more awareness to the problem.
Whenever I and seemingly others try to add Blu-ray and DVD releases to the encyclopedia now, it fails to retrieve the plot summary and extras from the Rightstuf listing. This results in many entries I'd consider "blank" like these. release#30996 release#30884 release#29990 release#29989 I've seen entries that unfortunately failed to list extras in the past (which is now rather frustrating for OOP releases Rightstuf doesn't list anymore), but they at least had the plot summary. I've been hesitating a bit to add some releases (I've been noticing the encyclopedia is unfortunately starting to get a little out of date with DVD's/BD's, particularly non-FUNi releases) because of this problem. I don't want to bring up every single entry with this problem in the release corrections thread. Is there any way to fix this problem so normal users adding releases works as it should? One of my main uses of the encyclopedia is for the release entries, so this problem is really frustrating. |
|||
BigOnAnime
Encyclopedia Editor
Posts: 1230 Location: Minnesota, USA |
|
||
Bumping in hopes this gets more noticed. I brought this up a month ago on this same page. I see the issue still hasn't been fixed, and it's gotten even more frustrating as I'm seeing more and more releases suffer from the issue. Like I added this just a few days ago, and it just looks awful and is rather uninformative. "What is it about? What special features can I find on it? What episodes does it contain?". If the Rightstuf listing gets removed in the future, you won't know. It already frustrates me with OOP releases I'm interested in when I see the extras were never listed (and since most of those are no longer listed on Rightstuf, I can't use them as a resource either), so that's where I'm coming from here.
Also many in these posts still look the same as that Love Live! S2 release. animenewsnetwork.com/bbs/phpBB2/viewtopic.php?p=4737649#4737649 animenewsnetwork.com/bbs/phpBB2/viewtopic.php?p=4745396#4745396 animenewsnetwork.com/bbs/phpBB2/viewtopic.php?p=4753245#4753245 Also strangely, with this (which needs to be changed to a combo pack, whoever added this for some reason thought it was a DVD when it's not, I have not yet tried to add the DVD-only release as it would only cause confusion atm), it's getting the plot summary from the Barnes & Noble listing instead of the Rightstuf one (which is way more complete). If it helps much, IIRC when Rightstuf first made the site change, you had different tabs on the product page, however that's no longer the case, the reviews, extras, plot summary, everything is on the same page, there's no tabs you have to click on to see more info. |
|||
Dan42
Chief Encyclopedist
Posts: 3782 Location: Montreal |
|
||
Good thing you did, because I hadn't noticed. I just fixed the scraper to extract the description from Rightstuf. Existing description-less releases will get updated over the next 24 hours or so. |
|||
Shiroi Hane
Encyclopedia Editor
Posts: 7580 Location: Wales |
|
||
Thank goodness. I was hoping for a system fix because that was a manual job that was only ever going to get longer.
|
|||
BigOnAnime
Encyclopedia Editor
Posts: 1230 Location: Minnesota, USA |
|
||
Indeed, and I already brought up a bunch of examples, just about anything that wasn't a FUNi release was looking blank. Also glad to see anything added prior to the fix will get updated.
One of my main uses of the encyclopedia over the years has been the release info, so this was a rather huge deal to me. Thank you for fixing the problem. |
|||
BigOnAnime
Encyclopedia Editor
Posts: 1230 Location: Minnesota, USA |
|
||
Looks like AnimeCastle underwent a site redesign recently, so it looks like now all the AnimeCastle links in the database for releases are now broken. An example of what a new URL looks like from the site. Tried adding this link for that release, and I got an error to no surprise.
http://www.animecastle.com/brynhildr-in-the-darkness-blu-ray-complete-collection/ So that may need to get fixed, though then again, lately AnimeCastle seems to no longer be carrying that much for newer releases anymore. Also as I noted recently in the releases correction thread...
|
|||
Dan42
Chief Encyclopedist
Posts: 3782 Location: Montreal |
|
||
This is too much of a mess, I'm just going to remove AnimeCastle. |
|||
Shiroi Hane
Encyclopedia Editor
Posts: 7580 Location: Wales |
|
||
It's been a while since I've searched Rakuten for a UPC and got a result. Wondering why, I browsed to the anime section, sorted by release date and was surprised to see stuff like Sailor Stars and The Daichis on the first page (Shirobako and what look like a couple of Japanese imports being the only actually recent releases there).
It looks like they have gone marketplace only (IIRC Rakuten UK did the same before closing entirely), with all the most recent stuff I've found being sold by "Importcds". |
|||
bglassbrook
Posts: 1243 Location: Gaithersburg, MD |
|
||
I was rather taken aback when it actually found some of the manga I was adding this/last week. The more I think about it, there may only have been a marketplace listing at the time for the one I had to do manually. A shame, for a while they were actually a fairly reliable hit on super-advance pre-order titles.
|
|||
BigOnAnime
Encyclopedia Editor
Posts: 1230 Location: Minnesota, USA |
|
||
Well the scraper is now having issues with Rightstuf again. Rightstuf recently had a minor site redesign and now here's how it's broken the scraper.
First off, it can no longer find the UPC so I'm unable to add stuff with just the Rightstuf link (a problem for NISA, Ponycan USA, and Aniplex USA releases). The UPC for stuff is now listed as SKU. Note how these match. http://www.rightstufanime.com/Punch-Line-Blu-ray https://images-na.ssl-images-amazon.com/images/I/8114WEyrPTL._SL1280_.jpg Second, it's now unable to pick up the description. Not sure why given the only main change the descriptions have gone under is now when you have longer descriptions, you have to click "see more" to see the rest of it. Lastly, the scraper seems unable to tell if an item is out of stock or OOP. Like for example this is OOP, but the scraper thinks Rightstuf still has it in-stock. release#28820 http://www.rightstufanime.com/Persona-4-The-Animation-Complete-Collectors-Edition-BD-DVD-Hyb |
|||
BigOnAnime
Encyclopedia Editor
Posts: 1230 Location: Minnesota, USA |
|
||
Bump, this issue still hasn't been fixed after nearly a month.
|
|||
BigOnAnime
Encyclopedia Editor
Posts: 1230 Location: Minnesota, USA |
|
||
Bumping again, two months later and the scraper is still having those same problems with RightStuf.
|
|||
All times are GMT - 5 Hours |
||
|
Powered by phpBB © 2001, 2005 phpBB Group