Is there a way to download all images relating to a certain tag?
barbeque
do you even read the last few pages of the main Stuff That Needs To Be Done thread. Also at the bottom of every page there is an “API Docs” link which does exactly what you need.
Bottomline: script.
Bottomline: script.
JP
I miss the show so much
@Itsthinking
The API is your friend. The query you want to do appears to be
Parse the JSON you get back and extract the image numbers. Iterate over them and access the image via
to get the actual URL you can download. You’ll probably want to plug in your auth key you can see in your account page.
Should fit well in under a 100 lines of Python or similar, depending on how much error handling you do.
And remember to rate limit it so you don’t DoS the servers.
The API is your friend. The query you want to do appears to be
http://derpiboo.ru/search.json?q=<tags here, separated with plus signs>
Parse the JSON you get back and extract the image numbers. Iterate over them and access the image via
http://derpiboo.ru/<number>.json
to get the actual URL you can download. You’ll probably want to plug in your auth key you can see in your account page.
Should fit well in under a 100 lines of Python or similar, depending on how much error handling you do.
And remember to rate limit it so you don’t DoS the servers.
barbeque
@JP
actually just the
And yeah, less than 100 lines is easily done. I’m doing it in PHP with lots of whitespace, comments, and some custom stuff, and it’s 130 lines.
actually just the
search.json
gives quite a lot of info already (I know I’m using that only, and not the per-image JSON’s). I’m not sure the ‘suppress favorites arrays’ and ‘suppress comment arrays’ even work. The actual download URL is in there as well.And yeah, less than 100 lines is easily done. I’m doing it in PHP with lots of whitespace, comments, and some custom stuff, and it’s 130 lines.
Background Pony #8DA1
Can someone give a tutorial or at least a link to one that shows how to install and run scripts? For someone who knows nothing about scripts I mean.
Somewhere to learn about scripts would be nice too.
Somewhere to learn about scripts would be nice too.
barbeque
@Background Pony @Itsthinking
I’ve actually been using mine for a few months now, but I’d rather not share scripts with (no offense) inexperienced users. If mods want to see it as well, I guess I can pastebin it or something, but my point is that if everyone started doing this (and there are abusive users) it would cause a massive serverload.
Also, if it’s just a one-off artist dump, I guess I could run it and upload a zip somewhere. PM me about that.
I’ve actually been using mine for a few months now, but I’d rather not share scripts with (no offense) inexperienced users. If mods want to see it as well, I guess I can pastebin it or something, but my point is that if everyone started doing this (and there are abusive users) it would cause a massive serverload.
Also, if it’s just a one-off artist dump, I guess I could run it and upload a zip somewhere. PM me about that.
Carcer
I actually don’t think the load from people doing this would be that bad. There will always be ways that abusive users can attempt to DoS the site - efficient use of the API is easier than serving HTML and now that the image serving done from static files through nginx without involving the webapp even mass image downloading by large numbers of people shouldn’t cause that much load on the server.
But don’t quote me on all that. I’d let Clover weigh in before taking my words as absolute fact.
Inexplicably in power
If mods want to see it as well, I guess I can pastebin it or something, but my point is that if everyone started doing this (and there are abusive users) it would cause a massive serverload.
I actually don’t think the load from people doing this would be that bad. There will always be ways that abusive users can attempt to DoS the site - efficient use of the API is easier than serving HTML and now that the image serving done from static files through nginx without involving the webapp even mass image downloading by large numbers of people shouldn’t cause that much load on the server.
But don’t quote me on all that. I’d let Clover weigh in before taking my words as absolute fact.
Background Pony #ECFE
I wish there was someone I could just pay to do this stuff for me. I’ve spent hours on it, and I still can’t figure it out.
Vree
I totes read this as “Is there a way to downvote all images relating to a certain tag?”
I was impressed how you guys were all acting so helpful.
I was impressed how you guys were all acting so helpful.
John Sheppard
Is there anywhere you can go to learn how to do this properly?
I’ve missed out on about a years worth of good art and I want to download it all in one bulk.
Is there anyone how can show the step by steps of doing this?
Ive been trying and I still cant figure it out.
I’ve missed out on about a years worth of good art and I want to download it all in one bulk.
Is there anyone how can show the step by steps of doing this?
Ive been trying and I still cant figure it out.
misspelledletter
@John Sheppard
I’ve actually written a tool to do this, here’s a link.
https://github.com/woodenphone/Derpibooru-dl
Make sure to read the readme at the bottom of the page, the text-file based interface assumes you have that info.
If you’re on windows you can run the pre-built version linked from the readme without installing any extra libraries.
I’ve actually written a tool to do this, here’s a link.
https://github.com/woodenphone/Derpibooru-dl
Make sure to read the readme at the bottom of the page, the text-file based interface assumes you have that info.
If you’re on windows you can run the pre-built version linked from the readme without installing any extra libraries.
barbeque
@John Sheppard
If you have any sort of coding experience (or maybe want to try something new), you can also use the API (link at bottom of page)
Most pages (ie, the important ones) also have a
/1.json?nofav=1&nocomments=1
but also /search.json?q=safe&nofav=1&nocomments=1
each picture has a
Always remember to add
If you have any sort of coding experience (or maybe want to try something new), you can also use the API (link at bottom of page)
Most pages (ie, the important ones) also have a
.json
variant, eg /1.json?nofav=1&nocomments=1
but also /search.json?q=safe&nofav=1&nocomments=1
each picture has a
representations
element which contains a full
element, which is the direct link to the image.Always remember to add
nofav=1
and nocomments=1
if you don’t need them per se (makes things faster for you, and puts less load on the servers) as well as use some reasonable limit (use only 1 simultaneous connection, limit it to say 2 MB/sec.; you’re not the only one using the site)Interested in advertising on Derpibooru? Click here for information!
Help fund the $15 daily operational cost of Derpibooru - support us financially!