Database dump survey

JP
Silly Pony - Celebrated the 13th anniversary of MLP:FIM, and 40 years of MLP!
Shimmering Smile - Celebrated the 10th anniversary of Equestria Girls!
Solar Guardian - Refused to surrender in the face of the Lunar rebellion and showed utmost loyalty to the Solar Empire (April Fools 2023).
Roseluck - Had their OC in the 2023 Derpibooru Collab.
King Sombra - Celebrated the 10th anniversary of The Crystal Empire!
A Lovely Nightmare Night - Celebrated the 12th anniversary of MLP:FIM!
Princess of Love - Extra special version for those who participated in the Canterlot Wedding 10th anniversary event by contributing art.
Elements of Harmony - Had an OC in the 2022 Community Collab
Non-Fungible Trixie -
Notoriously Divine Tagger - Consistently uploads images above and beyond the minimum tag requirements. And/or additionally, bringing over the original description from the source if the image has one. Does NOT apply to the uploader adding several to a dozen tags after originally uploading with minimum to bare tagging.

I miss the show so much
When can we expect the first dump to be released? I’m itching to do some tag-related statistics stuff (like what tags I’m most often adding to images).
 
>inb4 tag change histories are not part of the dump :-(
 
Also a question about the dump format. The “PostgreSQL custom dump format” part worries me a bit. AFAIK PGSQL dumps can be converted into sqlite dumps, but if it’s a “custom” format, then I don’t know.
Background Pony #B575
Sorry, maybe I’m asking the wrong place.  
Does this dump include all uploaded images?  
Is there already such a dump?
byte[]
Solar Supporter - Fought against the New Lunar Republic rebellion on the side of the Solar Deity (April Fools 2023).
Non-Fungible Trixie -
Verified Pegasus - Show us your gorgeous wings!
Preenhub - We all know what you were up to this evening~
An Artist Who Rocks - 100+ images under their artist tag
Artist -

Philomena Contributor
@JP  
When can we expect the first dump to be released? I’m itching to do some tag-related statistics stuff (like what tags I’m most often adding to images).
 
Probably a few months. I’m still working out the gremlins in the schema structure. I also need some time to verify that the dumps we make contain everything users wanted and don’t leak any private information.
 
>inb4 tag change histories are not part of the dump :-(
 
It was requested, so I’ll add them.
 
Also a question about the dump format. The “PostgreSQL custom dump format” part worries me a bit. AFAIK PGSQL dumps can be converted into sqlite dumps, but if it’s a “custom” format, then I don’t know.
 
See the documentation for pg_dump. The custom dump format, despite being non-portable, is the most flexible, because you can choose which tables you want to restore when you load it, and you can restore in parallel (assuming it wouldn’t swamp your drive). It’s also the fastest for me to make, since I can run it in parallel. You can’t do either of these things with the SQL format dump.
mjangelvortex
Silly Pony - Celebrated the 13th anniversary of MLP:FIM, and 40 years of MLP!
Lunar Supporter - Helped forge New Lunar Republic's freedom in the face of the Solar Empire's oppressive tyrannical regime (April Fools 2023).
Roseluck - Had their OC in the 2023 Derpibooru Collab.
King Sombra - Celebrated the 10th anniversary of The Crystal Empire!
A Lovely Nightmare Night - Celebrated the 12th anniversary of MLP:FIM!
Princess of Love - Extra special version for those who participated in the Canterlot Wedding 10th anniversary event by contributing art.
Tree of Harmony - Drew someone's OC for the 2022 Community Collab
Elements of Harmony - Had an OC in the 2022 Community Collab
Non-Fungible Trixie -
Kinship Through Differences - Celebrated the 11th anniversary of MLP:FIM!

Lady of Ships and Birbs
@ZizzyDizzyMC  
I wouldn’t mind to be able to see some of my deleted comments. There were two images here that were deleted recently that I commented on that I wouldn’t mind being able to see.
byte[]
Solar Supporter - Fought against the New Lunar Republic rebellion on the side of the Solar Deity (April Fools 2023).
Non-Fungible Trixie -
Verified Pegasus - Show us your gorgeous wings!
Preenhub - We all know what you were up to this evening~
An Artist Who Rocks - 100+ images under their artist tag
Artist -

Philomena Contributor
Preliminary testing shows that the public dump should be about 2.5GB, and it takes about 25 minutes to generate on our side. (This doesn’t require any downtime.)
Ralek
Daring Do Dakimakura - Attended a Derpibooru panel at a MLP convention
Smile - Derpi Supporter
Sigma Butt -
Magnificent Metadata Maniac - #1 Assistant
Non-Fungible Trixie -
My Little Pony - 1992 Edition
Wallet After Summer Sale -
Heart Gem -
A Tale For The Ages - Celebrated MLP's 35th Anniversary and FiM's 8th Anniversary
Notoriously Divine Tagger - Consistently uploads images above and beyond the minimum tag requirements. And/or additionally, bringing over the original description from the source if the image has one. Does NOT apply to the uploader adding several to a dozen tags after originally uploading with minimum to bare tagging.

Smile
This would definitely be useful, I know I’ve used the API to grab large chunks of data before.  
If it hasn’t been suggested, maybe publicize your method of reverse image searches and include the hashes in the dumps?
Background Pony #D733
If I right understood, with this dump any developer could use Derpibooru archive directly, without user interface?
 
We want create our own Derpibooru interface - with modern design and russian language support (I’m Brony Webmaster from Russia, yes). Also, standard Derpibooru interface is blocked in Russia (because foalcon denied in our country, same as child porn), but DerpiCDN is available - so we may make our interface with totally foalcon excluding, to avoid blocking it (or may be with hidden cheatcode or premium account to unlock foalcon).
 
That database dumps probably much helps us to create our custom Derpibooru interface. Good luck!
byte[]
Solar Supporter - Fought against the New Lunar Republic rebellion on the side of the Solar Deity (April Fools 2023).
Non-Fungible Trixie -
Verified Pegasus - Show us your gorgeous wings!
Preenhub - We all know what you were up to this evening~
An Artist Who Rocks - 100+ images under their artist tag
Artist -

Philomena Contributor
@Background Pony #DD72  
To be clear, these archives will not include any user data beyond what would normally be displayed by the site. That means if you want to use authentication, you’ll have to roll your own system.
Background Pony #2ECF
Can you by any chance also include data that are made look like deleted (like data about deleted images, comments on deleted images, deleted comments, deleted forum posts and deletion reasons)?
Interested in advertising on Derpibooru? Click here for information!
Ministry of Image - Fanfiction Printing

Help fund the $15 daily operational cost of Derpibooru - support us financially!

Syntax quick reference: **bold** *italic* ||hide text|| `code` __underline__ ~~strike~~ ^sup^ %sub%

Detailed syntax guide