Database dump survey

byte[]

Philomena Contributor
Hello all,

To make it easier for developers and analysts to access the information they want from the site, we are planning to offer database dumps for our public-facing data.1

Though this could change in the future, currently the plan is to offer nightly dumps in the PostgreSQL custom dump format. This will make it easy for anyone who wants to use the data to simply restore the dump and have a fully operational schema with indices and foreign key constraints. In addition, these dumps will be consistent and represent a perfect snapshot of the database at the time they are taken.

We would like to know from the community what data would be most useful to include in these dumps. Please comment below with the data you would like and your usecase for it, or any questions you may have.



1: We will not be offering any data that may deanonymize background ponies, or that would otherwise be considered personally identifying information.
Posted Report
NotADeliciousPotato
An Artist Who Rocks - 100+ images under their artist tag
My Little Pony - 1992 Edition
Friendship, Art, and Magic (2020) - Took part in the 2020 Community Collab
The Magic of Friendship Grows - For helping others attend the 2020 Community Collab
Magnificent Metadata Maniac - Numbers don't line up outwardly but they do inwardly.
Dream Come True! - Participated in the MLP 9th Anniversary Event
Wallet After Summer Sale -
Artist -

Site Moderator
Equus ferus caballus
I'd be down with that.
(Don't think it'd have anything that you can't already see from the website's normal user interface; it'll just be stuff like # of favorites a picture has, etc.)
Posted Report
Luna
Luna - For Patreon supporters
Woona - Derpi Supporter
My Little Pony - 1992 Edition
Best Art Program Ever - I get the badge because I made the event >:V
The Magic of Friendship Grows - For helping others attend the 2020 Community Collab
Friendship, Art, and Magic (2020) - Took part in the 2020 Community Collab
A Perfectly Normal Pony - <meow> byte[] permission to suck your dick
Dream Come True! - Participated in the MLP 9th Anniversary Event
Toola Roola - For helping others attend the 2019 Community Collab
Investor - Lootboxes are just another form of gambling

Luna Best Pony~
Interactions, comments, image deduplication data, hashes, representation paths for image objects, tags, tag aliases/implications data.
Anything else is secondary in my book.
Posted Report
ZizzyDizzyMC
My Little Pony - 1992 Edition
Friendship, Art, and Magic (2020) -
Economist -
Friendship, Art, and Magic (2019) - Celebrated Derpibooru's seventh year anniversary with friends.
Friendship, Art, and Magic (2018) - Celebrated Derpibooru's six year anniversary with friends.
Not a Llama - Happy April Fools Day!
A Perfectly Normal Pony - <ZizzyDizzyMC> you're really really triggering me

The Pony Archive Owner
I'd like this, you know, because it'd be easier on your servers.
edit: Full public facing dump, archiving everything. I already scrape every page recursively so this just makes it easier on your server with less queries from myself and like 50-60 other archivers that constantly download and update.
This would not stop new image queries however.

Deleted comments would be welcome in the dumps — it would take a lot of effort to stir up shit from a deleted comment pulled from a DB dump, it would also provide better transparency. If not, comment archivers will continue to run most likely resulting in more queries.
ZizzyDizzyMC
My Little Pony - 1992 Edition
Friendship, Art, and Magic (2020) -
Economist -
Friendship, Art, and Magic (2019) - Celebrated Derpibooru's seventh year anniversary with friends.
Friendship, Art, and Magic (2018) - Celebrated Derpibooru's six year anniversary with friends.
Not a Llama - Happy April Fools Day!
A Perfectly Normal Pony - <ZizzyDizzyMC> you're really really triggering me

The Pony Archive Owner
@Communist Starlight
A database dump is just a large excel file.
A series of tables that have things that go to other things.

Key:value pairs generally but it can also be Key:key:key:Value:value:value:value:value Like this:

imageid|image title|image tags|image uploader|boolean nsfw

Please note that's an dumb'd down example and is likely not anywhere close to their actual database.
Posted Report
Background Pony #2ECF
I'd also be interested in one such thing. Including image data, comments, comment history, tags, changes in tags, forums, forum topics and forum posts.

Usecase would be archiving.
Posted Report
Darkest Lunar Flower
Luna - For Patreon supporters
My Little Pony - 1992 Edition
Friendship, Art, and Magic (2020) - Took part in the 2020 Community Collab
The Magic of Friendship Grows - For helping others attend the 2020 Community Collab
Dream Come True! - Participated in the MLP 9th Anniversary Event
Bronze Bit -
Toola Roola - For helping others attend the 2019 Community Collab
Wallet After Summer Sale -
Happy Derpy! - For Patreon supporters
Woona - Best Princess

Your real life Luna.
Exact time of posts?

Also rating tags numbers and their relation to characters. For example who has a higher percentage of suggestive images Celestia or Luna.
Posted Report
SviMik
Speaking Fancy - Badge given to members that help with translations

Is it possible to share the code for reverse search along with the hashes it use?

Usecase:
I have my own picture archive, and I'd like to correlate them with derpibooru in order to
1. Automatically suggest a derpibooru link in my gallery
2. Maybe consider uploading images that are missing on derpibooru
Posted Report
Joey
Dawnsong - Derpi Supporter
My Little Pony - 1992 Edition
Best Art Program Ever - For artists who were courageous enough to draw something for the April Fools day.
The Magic of Friendship Grows - For helping others attend the 2020 Community Collab
Friendship, Art, and Magic (2020) - Took part in the 2020 Community Collab
Charitable Contributor - Contributed to a series of art packs that helped raise over $10,000 for charity
Dream Come True! - Participated in the MLP 9th Anniversary Event
Element of Generosity - For users who have donated to the site
Heart Gem -
A Really Classy Artist - 250+ images under their artist tag

PM me your cute OCs
@Darkest Lunar Flower
You can find the exact time by hovering over the bit that says "Posted 3 minutes ago with your cursor", and it'll show you the GMT timestamp of the post.

As for the other stuff — that's statistics. People will be able to calculate that with this info (and I'm sure some people will), but we'll just be providing the RAW data. Dumping a database is pretty easy and straightforward, but running a bunch of calculations and analytics against it is a bit more resource-intensive, and we don't wanna put additional strain on the server.
Posted Report
Interested in advertising on Derpibooru? Click here for information!
Ministry of Image - Fanfiction Printing

Derpibooru costs over $25 a day to operate - help support us financially!

Syntax quick reference: *bold* _italic_ [spoiler]hide text[/spoiler] @code@ +underline+ -strike- ^sup^ ~sub~