The Scraper Update of 6/25/2018

DJDavid98
Non-Fungible Trixie -
Kinship Through Differences - Celebrated the 11th anniversary of MLP:FIM!
Verified Pegasus - Show us your gorgeous wings!
Preenhub - We all know what you were up to this evening~
Philomena - For helping others attend the 2021 community collab
Twinkling Balloon - Took part in the 2021 community collab.
Ten years of changes - Celebrated the 10th anniversary of MLP:FiM!
My Little Pony - 1992 Edition
Friendship, Art, and Magic (2020) - Took part in the 2020 Community Collab
The Magic of Friendship Grows - For helping others attend the 2020 Community Collab

Disy
@Background Pony #EF38  
I’ve made the necessary changes, and the scraper should now always use the :orig URL. I’m not sure what Twitter is even doing with their image version as this point, but I guess this one’s the best. I left the suffix off because I thought both with and without resulted in the same image, but you’ve proven that that’s not the case.
TexasUberAlles
Duck - If report count was a score, he'd have the biggest score
Silly Pony - Celebrated the 13th anniversary of MLP:FIM, and 40 years of MLP!
Shimmering Smile - Celebrated the 10th anniversary of Equestria Girls!
Lunar Supporter - Helped forge New Lunar Republic's freedom in the face of the Solar Empire's oppressive tyrannical regime (April Fools 2023).
Roseluck - Had their OC in the 2023 Derpibooru Collab.
Flower Trio - Helped others get their OC into the 2023 Derpibooru Collab.
King Sombra - Celebrated the 10th anniversary of The Crystal Empire!
A Lovely Nightmare Night - Celebrated the 12th anniversary of MLP:FIM!
Princess of Love - Extra special version for those who participated in the Canterlot Wedding 10th anniversary event by contributing art.
Tree of Harmony - Drew someone's OC for the 2022 Community Collab

Tumblr does the same kind of pointless janking about with image sizes to fit its display format as well; it famously ruins any image that’s taller than a certain aspect ratio– I wanna say something like 2x3, so 500x750 or 1280x1920– which is why a lot of artists who post on both Tumblr and DA will post a single tall image on DA but break it up into multiple panels on Tumblr. I do that manually when reblogging tall/thin comics from the DA bookmarklet, for example, because otherwise they look like garbage.
stsyn
Lunar Supporter - Helped forge New Lunar Republic's freedom in the face of the Solar Empire's oppressive tyrannical regime (April Fools 2023).
Elements of Harmony - Had an OC in the 2022 Community Collab
Non-Fungible Trixie -
Twinkling Balloon - Took part in the 2021 community collab.
Notoriously Divine Tagger - Consistently uploads images above and beyond the minimum tag requirements. And/or additionally, bringing over the original description from the source if the image has one. Does NOT apply to the uploader adding several to a dozen tags after originally uploading with minimum to bare tagging.
Best Artist - Providing quality, Derpibooru-exclusive artwork
Fine Arts - Two hundred uploads with a score of over a hundred (Safe/Suggestive)
Economist -
Cool Crow - "Caw!" An awesome tagger
The Power of Love - Given to a publicly verified artist with an image under their artist’s tag that has reached 1000 upvotes

Moderator
@TexasUberAlles  
This “pointless janking” is calling “Data saving”.  
And also you won’t be able to view single long image without opening it in new tab.  
(And also browser may crash with images bigger than 5Kx5K on machines with 2gb of RAM).
TexasUberAlles
Duck - If report count was a score, he'd have the biggest score
Silly Pony - Celebrated the 13th anniversary of MLP:FIM, and 40 years of MLP!
Shimmering Smile - Celebrated the 10th anniversary of Equestria Girls!
Lunar Supporter - Helped forge New Lunar Republic's freedom in the face of the Solar Empire's oppressive tyrannical regime (April Fools 2023).
Roseluck - Had their OC in the 2023 Derpibooru Collab.
Flower Trio - Helped others get their OC into the 2023 Derpibooru Collab.
King Sombra - Celebrated the 10th anniversary of The Crystal Empire!
A Lovely Nightmare Night - Celebrated the 12th anniversary of MLP:FIM!
Princess of Love - Extra special version for those who participated in the Canterlot Wedding 10th anniversary event by contributing art.
Tree of Harmony - Drew someone's OC for the 2022 Community Collab

StSyaN  
Yeah, but they do it in the dumbest way possible; instead of just reducing the height until it’s at the maximum allowed px, it appears as if they’re doing that first, then resampling from there and adjusting the width to the nearest Tumblr display standard (540, 500, 400, 250, or 125). It just ends up making tall images look like pixelated garbage.
 
They’re essentially shouting “ENHANCE!” at the screen like a poorly written techno-thriller.
stsyn
Lunar Supporter - Helped forge New Lunar Republic's freedom in the face of the Solar Empire's oppressive tyrannical regime (April Fools 2023).
Elements of Harmony - Had an OC in the 2022 Community Collab
Non-Fungible Trixie -
Twinkling Balloon - Took part in the 2021 community collab.
Notoriously Divine Tagger - Consistently uploads images above and beyond the minimum tag requirements. And/or additionally, bringing over the original description from the source if the image has one. Does NOT apply to the uploader adding several to a dozen tags after originally uploading with minimum to bare tagging.
Best Artist - Providing quality, Derpibooru-exclusive artwork
Fine Arts - Two hundred uploads with a score of over a hundred (Safe/Suggestive)
Economist -
Cool Crow - "Caw!" An awesome tagger
The Power of Love - Given to a publicly verified artist with an image under their artist’s tag that has reached 1000 upvotes

Moderator
@TexasUberAlles  
Not only by width, 1280x1920 is a limit, and 2000x10000 will be rescaled to 384x1920. But actually it shows this resolution only in separate view, where you cannot scroll, and even in desktop Full HD monitor you really will see ~180x900px until open separate (so there is no reason to provide bigger resolution, except for only one image per post (and I remember something in theme guideline for big images…)). In general dashboard they provide maximum 540x810 because of data saving.
 
And, well, it’s how it works absolutely everywhere. Even on Derpi
Brokedownandmadeone
Lunar Supporter - Helped forge New Lunar Republic's freedom in the face of the Solar Empire's oppressive tyrannical regime (April Fools 2023).
Non-Fungible Trixie -
Preenhub - We all know what you were up to this evening~
My Little Pony - 1992 Edition
Artistic Detective - For awesome dedication to sleuthing out and maintaining artist tags and links
Notoriously Divine Tagger - Consistently uploads images above and beyond the minimum tag requirements. And/or additionally, bringing over the original description from the source if the image has one. Does NOT apply to the uploader adding several to a dozen tags after originally uploading with minimum to bare tagging.
Cool Crow - "Caw!" An awesome tagger
Wallet After Summer Sale -
Equality - In our state, we're all equal here!
Not a Llama - Happy April Fools Day!

BigThirsty ponybooru.org
@DJDavid98  
I was actually going to check a picture to show the same example, but some better BP did it much more promptly than I did. (Thank you BP for nipping that in the bud ASAP where I didn’t/couldn’t) And thanks David, for whipping up the update so soon.
radpanic
Solar Supporter - Fought against the New Lunar Republic rebellion on the side of the Solar Deity (April Fools 2023).
Fine Arts - Two hundred uploads with a score of over a hundred (Safe/Suggestive)
Perfect Pony Plot Provider - Uploader of 10+ images with 350 upvotes or more (Questionable/Explicit)
Non-Fungible Trixie -
Not a Llama - Happy April Fools Day!

never big enough
Found an oddity while uploading from Tumblr. It appears that if an artist posts a single picture to Tumblr, then edits the post to add more pictures in the description, then the scraper will only return the very first image when you attempt to fetch the pictures. Examples:
 
https://besttubahorse.tumblr.com/post/173552135736/besttubahorse-besttubahorse-shes-used-to-the (safe)  
https://chromchill.tumblr.com/post/167914864796 (contains vore art, but not gross or grimdark)
Background Pony #A150
I’m noticing the scrapper isn’t fetching images from Tumblr in their full size res anymore.
Background Pony #F25E
Not sure if it’s intended, but does uploading using media.tumblr 1280 links with, or without using fetch button use the _raw file? For instance, the source for >>1799088 (merged) (nsfw) was 1280x1815, which was the same as the one in the source shown as literally https://78.media.tumblr.com/75446e71da0281e595921d3ad449f0ed/tumblr_pcz2kcg0GJ1ty8gvqo2_1280.jpg
 
However, converting it to the raw format gave a size of 3000x4253. Was it because the source was those media.tumblr links?
TexasUberAlles
Duck - If report count was a score, he'd have the biggest score
Silly Pony - Celebrated the 13th anniversary of MLP:FIM, and 40 years of MLP!
Shimmering Smile - Celebrated the 10th anniversary of Equestria Girls!
Lunar Supporter - Helped forge New Lunar Republic's freedom in the face of the Solar Empire's oppressive tyrannical regime (April Fools 2023).
Roseluck - Had their OC in the 2023 Derpibooru Collab.
Flower Trio - Helped others get their OC into the 2023 Derpibooru Collab.
King Sombra - Celebrated the 10th anniversary of The Crystal Empire!
A Lovely Nightmare Night - Celebrated the 12th anniversary of MLP:FIM!
Princess of Love - Extra special version for those who participated in the Canterlot Wedding 10th anniversary event by contributing art.
Tree of Harmony - Drew someone's OC for the 2022 Community Collab

Yeah, that was why Derpi made the push for everyone to use the raw versions a while back, because the 1280 versions everyone had been using to that point are limited to 1280x n regardless of the originally uploaded file’s resolution.
Jondor
Perfect Pony Plot Provider - Uploader of 10+ images with 350 upvotes or more (Questionable/Explicit)
Wallet After Summer Sale -
Bronze Bit -
Happy Derpy! - For Patreon supporters

Ship all the ponies!
Tumblr has screwed around with things again and raw images are no longer accessible using the data.tumblr.com domain and _raw suffix
Background Pony #F25E
@TexasUberAlles  
Are we talking about the same problem? I thought we were talking about the _raw links being broken; they seem working just yesterday, rather than just weeks ago.
 
Edit: Thanks for the tumblr mass downloader btw; never knew it existed. 🙂
TexasUberAlles
Duck - If report count was a score, he'd have the biggest score
Silly Pony - Celebrated the 13th anniversary of MLP:FIM, and 40 years of MLP!
Shimmering Smile - Celebrated the 10th anniversary of Equestria Girls!
Lunar Supporter - Helped forge New Lunar Republic's freedom in the face of the Solar Empire's oppressive tyrannical regime (April Fools 2023).
Roseluck - Had their OC in the 2023 Derpibooru Collab.
Flower Trio - Helped others get their OC into the 2023 Derpibooru Collab.
King Sombra - Celebrated the 10th anniversary of The Crystal Empire!
A Lovely Nightmare Night - Celebrated the 12th anniversary of MLP:FIM!
Princess of Love - Extra special version for those who participated in the Canterlot Wedding 10th anniversary event by contributing art.
Tree of Harmony - Drew someone's OC for the 2022 Community Collab

@Background Pony #EF38  
Jondor made it sound like he was trying to do the old trick of just swapping _raw for 1280 and data for 78.media in an image URL, and that doesn’t work anymore because now the raw files are in the s3.amazonaws.com/data.tumblr.com/xxxxxxxxxxxxxxxxxxxxxx/tumblr_xxxxxxxxxxxxxxxx_raw.jpg format.
Interested in advertising on Derpibooru? Click here for information!
Ministry of Image - Fanfiction Printing

Help fund the $15 daily operational cost of Derpibooru - support us financially!

This topic has been locked to new posts from non-moderators.

Locked

Lock reason: This turned into a feature request thread