[Userscript] semi-Automated Derpibooru Uploader

Background Pony #0E61
@Markiz93
There is a different technique, where you get different pieces of the image at a time, and you have to examine them to see if they are full quality or not. If they are, reassemble them into the full image. It doesn’t work on every post, only sometimes, so it’s not part of the script. mdashlw does it on their own.
stsyn
Lunar Supporter - Helped forge New Lunar Republic's freedom in the face of the Solar Empire's oppressive tyrannical regime (April Fools 2023).
Elements of Harmony - Had an OC in the 2022 Community Collab
Non-Fungible Trixie -
Twinkling Balloon - Took part in the 2021 community collab.
Notoriously Divine Tagger - Consistently uploads images above and beyond the minimum tag requirements. And/or additionally, bringing over the original description from the source if the image has one. Does NOT apply to the uploader adding several to a dozen tags after originally uploading with minimum to bare tagging.
Best Artist - Providing quality, Derpibooru-exclusive artwork
Fine Arts - Two hundred uploads with a score of over a hundred (Safe/Suggestive)
Economist -
Cool Crow - "Caw!" An awesome tagger
The Power of Love - Given to a publicly verified artist with an image under their artist’s tag that has reached 1000 upvotes

Moderator
Updated script to work again.
Also fixed this old thing which kinda performs mentioned technique. Keep in mind, that resulted image still may be in non-adequate quality — compare top left corner of result with bottom right corner, if bottom right is very blurry, then trick didn’t work.
mdashlw
Pixel Perfection - I still call her Lightning Bolt
Cool Crow - "Caw!" An awesome tagger
Magnificent Metadata Maniac - #1 Assistant
Notoriously Divine Tagger - Consistently uploads images above and beyond the minimum tag requirements. And/or additionally, bringing over the original description from the source if the image has one. Does NOT apply to the uploader adding several to a dozen tags after originally uploading with minimum to bare tagging.
Lunar Supporter - Helped forge New Lunar Republic's freedom in the face of the Solar Empire's oppressive tyrannical regime (April Fools 2023).

@stsyn
it’s really hard to tell manually. reliable way is to check png headers.
“Pixels per unit” value in pHYs chunk of the PNG header on https://www.nayuki.io/page/png-file-chunk-inspector must not be 1000. If it’s 1000, that chunk was jpeg-ified. If it’s not 1000, it’s pixel identical to original image.
I’m not sure how to implement this in browser js though.
Also, the script always uses fullview dimensions for chunk width/height and adjusts x and y accordingly. I’ve found that adjusting chunk width & height (making it smaller in last bottom/right-most chunks) tends to be more reliable.
stsyn
Lunar Supporter - Helped forge New Lunar Republic's freedom in the face of the Solar Empire's oppressive tyrannical regime (April Fools 2023).
Elements of Harmony - Had an OC in the 2022 Community Collab
Non-Fungible Trixie -
Twinkling Balloon - Took part in the 2021 community collab.
Notoriously Divine Tagger - Consistently uploads images above and beyond the minimum tag requirements. And/or additionally, bringing over the original description from the source if the image has one. Does NOT apply to the uploader adding several to a dozen tags after originally uploading with minimum to bare tagging.
Best Artist - Providing quality, Derpibooru-exclusive artwork
Fine Arts - Two hundred uploads with a score of over a hundred (Safe/Suggestive)
Economist -
Cool Crow - "Caw!" An awesome tagger
The Power of Love - Given to a publicly verified artist with an image under their artist’s tag that has reached 1000 upvotes

Moderator
I’m not sure how to implement this in browser js though.
It’s still possible to fetch image as binary and parse that binary in browser JS, however, it’s kinda complex task and I’m not sure, if I do this soon. Anyway, nice info to have, thanks. Surprised, that wix is being honest with what junk it serves.
Also, the script always uses fullview dimensions for chunk width/height and adjusts x and y accordingly. I’ve found that adjusting chunk width & height (making it smaller in last bottom/right-most chunks) tends to be more reliable.
Hmm, maybe will play with it as well.
Background Pony #1D52
I remember you had more options of the scrypt in DA, now 2 of them are gone.
Background Pony #0E61
Token kek was one of them.
Yes, deviantArt changed so that no longer works.

@mdashlw, @stsyn
I found the library “pngtoy” (unknown how good or not it is) and was able to use it to automate the checks, starting from an older 0.3.0 version of DAF (I also make no claims about my own code quality). You definitely have to check each section - some posts will have some sections full-quality and some sections degraded - and it’s hit-or-miss if a post will have all sections in full quality. If the original image format was a GIF or JPEG, I do not think the check can be used.
// @require      https://github.com/xgds/pngtoy/raw/master/pngtoy.min.js

// This might have come from Stack Overflow?  I forgot, and I didn't make a comment so it probably wasn't verbatim
  function loadFileAsArrayBuffer(url){
    return new Promise((resolve, reject) => {
      var xhr = new XMLHttpRequest();
      xhr.open('GET', url, true);
      xhr.responseType = 'arraybuffer';
      xhr.onload = function(e) {
        if (this.status === 200) {
          resolve(this.response);
        } else {
          reject(this.response);
        }
      };
      xhr.send();
    })
  };

// Changed lines within function unwatermark
      offsetX += Math.floor(width / 4 + 0.5);
      offsetY += Math.floor(height / 6 + 0.5);

// Do the diff yourself to see what changed plz
  async function combine(data) {
    const fullview = data.media.types.find(item => item.t === 'fullview');
    const targetWidth = data.extended.originalFile.width;
    const targetHeight = data.extended.originalFile.height;
    const maxWidth = Math.floor(fullview.w);
    const maxHeight = Math.floor(fullview.h);
    let canvas = createElement('canvas', { width: targetWidth, height: targetHeight});
    let ctx = canvas.getContext('2d');
    const url = (oX, oY, mW, mH) => data.media.baseUri + `/v1/crop/x_${oX},y_${oY},w_${mW},h_${mH},q_100/${data.media.prettyName}-pre.png?token=` + data.media.token[0];

    const pngtoy = new PngToy();
    let warnings = 0;
    let errors = 0;

    let offsetX = 0;
    let offsetY = 0;
    let currentWidth = maxWidth;
    let currentHeight = maxHeight;

    while (offsetX < targetWidth) {
      offsetY = 0;
      while (offsetY < targetHeight) {
        currentWidth = Math.min(maxWidth, targetWidth - offsetX);
        currentHeight = Math.min(maxHeight, targetHeight - offsetY);
        const arrayBuffer = await loadFileAsArrayBuffer(url(offsetX, offsetY, currentWidth, currentHeight));

        pngtoy.fetch(arrayBuffer).then(() => {
          const phys = pngtoy.getChunk('pHYs');
          if (!phys || !phys.ppuX || !phys.ppuY)
            warnings++;
          else if (phys.ppuX == 1000 || phys.ppuY == 1000)
            errors++;
        });

        // https://gist.github.com/candycode/f18ae1767b2b0aba568e
        const arrayBufferView = new Uint8Array(arrayBuffer);
        const blob = new Blob([arrayBufferView], { type: "image/png" });
        const urlCreator = window.URL || window.webkitURL;
        const imageUrl = urlCreator.createObjectURL(blob);

        await (async () => {
          return new Promise((resolve, reject) => {
            const image = new Image;
            image.onload = function() {
              const temp = createElement('canvas', { width: targetWidth, height: targetHeight});
              const tempCtx = temp.getContext('2d');

              tempCtx.drawImage(image, offsetX, offsetY);
              tempCtx.drawImage(canvas, 0, 0);

              ctx = tempCtx;
              canvas = temp;
              image.remove();
              URL.revokeObjectURL(imageUrl);
              resolve();
            }
            image.onerror = reject;
            image.src = imageUrl;
          });
        })();

        offsetY += Math.floor(maxHeight);
      }
      offsetX += Math.floor(maxWidth);
    }

    (warnings == 0) ? warnings = '' : warnings = warnings + ' MYSTERIES! - ';
    (errors == 0) ? errors = '' : errors = errors + ' LOW QUALITY!!! - ';
    spawn(warnings + errors + 'Open full (Ctrl+ = download)', (event) => openOrDownload(canvas.toDataURL(), data.media.prettyName + '-full.png'));
  }
stsyn
Lunar Supporter - Helped forge New Lunar Republic's freedom in the face of the Solar Empire's oppressive tyrannical regime (April Fools 2023).
Elements of Harmony - Had an OC in the 2022 Community Collab
Non-Fungible Trixie -
Twinkling Balloon - Took part in the 2021 community collab.
Notoriously Divine Tagger - Consistently uploads images above and beyond the minimum tag requirements. And/or additionally, bringing over the original description from the source if the image has one. Does NOT apply to the uploader adding several to a dozen tags after originally uploading with minimum to bare tagging.
Best Artist - Providing quality, Derpibooru-exclusive artwork
Fine Arts - Two hundred uploads with a score of over a hundred (Safe/Suggestive)
Economist -
Cool Crow - "Caw!" An awesome tagger
The Power of Love - Given to a publicly verified artist with an image under their artist’s tag that has reached 1000 upvotes

Moderator
@Background Pony #0E61
I had to patch original library, because wix seems to mess with several chunks (which makes image technically incorrect, but stil viewable).
Anyway, huge thanks for submission, script updated and now shows if image is partially in low quality.
mdashlw
Pixel Perfection - I still call her Lightning Bolt
Cool Crow - "Caw!" An awesome tagger
Magnificent Metadata Maniac - #1 Assistant
Notoriously Divine Tagger - Consistently uploads images above and beyond the minimum tag requirements. And/or additionally, bringing over the original description from the source if the image has one. Does NOT apply to the uploader adding several to a dozen tags after originally uploading with minimum to bare tagging.
Lunar Supporter - Helped forge New Lunar Republic's freedom in the face of the Solar Empire's oppressive tyrannical regime (April Fools 2023).

It’s a bit annoying that people upload images here with visible artifacts due to low quality chunks using this method. I guess it’s still much better than 1280px but if there’s ever a proper method to get original images from deviantart, it wouldn’t be easily possible to tell which images need reuploading.
E.g.: >>3313297 (nsfw) you can easily tell the good-bad chunks border.
Background Pony #0E61
@mdashlw
:\ That’s a shame that’s happening, especially since, at least on that one example you linked, I think (based on the file name) the uploader is a long time major contributor to the site… (Since it was posted BP, might be a Rule #8 to say who)
stsyn
Lunar Supporter - Helped forge New Lunar Republic's freedom in the face of the Solar Empire's oppressive tyrannical regime (April Fools 2023).
Elements of Harmony - Had an OC in the 2022 Community Collab
Non-Fungible Trixie -
Twinkling Balloon - Took part in the 2021 community collab.
Notoriously Divine Tagger - Consistently uploads images above and beyond the minimum tag requirements. And/or additionally, bringing over the original description from the source if the image has one. Does NOT apply to the uploader adding several to a dozen tags after originally uploading with minimum to bare tagging.
Best Artist - Providing quality, Derpibooru-exclusive artwork
Fine Arts - Two hundred uploads with a score of over a hundred (Safe/Suggestive)
Economist -
Cool Crow - "Caw!" An awesome tagger
The Power of Love - Given to a publicly verified artist with an image under their artist’s tag that has reached 1000 upvotes

Moderator
Added Pixiv fetcher (only on derpi as alternative fetch). Login is not needed to fetch R-rated images.
Suddenly I realised, that direct copy-pasting of URL doesn’t work, and you have to download image and manually select it from file system. Fetcher allows to skip this step (however, because of manual downloading, it’s kinda slow).
Right now, multiple selections are not supported.
stsyn
Lunar Supporter - Helped forge New Lunar Republic's freedom in the face of the Solar Empire's oppressive tyrannical regime (April Fools 2023).
Elements of Harmony - Had an OC in the 2022 Community Collab
Non-Fungible Trixie -
Twinkling Balloon - Took part in the 2021 community collab.
Notoriously Divine Tagger - Consistently uploads images above and beyond the minimum tag requirements. And/or additionally, bringing over the original description from the source if the image has one. Does NOT apply to the uploader adding several to a dozen tags after originally uploading with minimum to bare tagging.
Best Artist - Providing quality, Derpibooru-exclusive artwork
Fine Arts - Two hundred uploads with a score of over a hundred (Safe/Suggestive)
Economist -
Cool Crow - "Caw!" An awesome tagger
The Power of Love - Given to a publicly verified artist with an image under their artist’s tag that has reached 1000 upvotes

Moderator
@Background Pony #0E61
There are used to be some sort of in-site proxies ~5 years ago, which allowed to browse booru while it was blocked. They worked in a way that instead of encoding site to the url path, they transformed domain of visited site into random, but very specific gibberish sub-domain (while there were several proxies worked that way, they all had same algorithms and letters matched).
One of them is for derpibooru.org, another one — trixiebooru.org, and both had www.* versions.
I guess I can omit them, since these proxies stopped working anyway.
Posted Report
Interested in advertising on Derpibooru? Click here for information!
Champions of Equestria

Help fund the $15 daily operational cost of Derpibooru - support us financially!

Syntax quick reference: **bold** *italic* ||hide text|| `code` __underline__ ~~strike~~ ^sup^ %sub%

Detailed syntax guide