The 2020 Community Collab is posted! Thank you to everyone for submitting over 1600 characters to this years collab! Click here to see it!

Feature suggestions and discussion [READ THE FIRST POST]

Derpy Whooves
Not a Llama - Happy April Fools Day!
Economist -
Artistic Detective - For awesome dedication to sleuthing out and maintaining artist tags and links

Site Moderator
Artist Concierge
@JP
For what it's worth, from the mod side of staff the view into the dev space with the existing code base and all its legacy issues has often been a little pic related.



Just my personal opinion, but the legacy code on the site hasn't always been "just fine". Teams of people have been spending a lot of their personal time keeping the site running. And if they say it needs to move to a new code base, I trust them.
byte[]

Site Administrator
Site Developer
@JP
I actually wanted one day to see the Booru On Rails’ source code, but looks like I never will :-(
Don't worry, we're still releasing it. I've committed to a hard deadline on the release — Dec 25, 2019.

The old code is tested and proven to work. The new code… it’s not. Rewriting working code in a new language (that I’m not even sure what it is called) is a known way to fail.

I'm painfully aware of this. My plan is to develop and test incrementally. We'll be running Derpibooru and Philomena in parallel for a while to assert that Philomena works without major hiccups. This will be hosted at philomena.derpibooru.org.

How come the old code is now, suddenly, too slow, when it has worked just fine for 8 years?
This is something I'll be covering in an upcoming patreon post; hang tight.
GenericArchangel
Wallet After Summer Sale -
Magical Inkwell - Wrote MLP fanfiction consisting of at least around 1.5k words, and has a verified link to the platform of their choice
A Perfectly Normal Pony - I will get that badge one day even if it's the death of me.

Site Moderator
Free the Tantabus
@byte[]
Okay really dumb question: this site’s name won’t change, right? The wording of that makes it sound like the site’s name will change, but that can’t be right.
JP
The End wasn't The End - Found a new home after the great exodus of 2012
Magnificent Metadata Maniac - #1 Assistant
Friendship, Art, and Magic (5 Years) - Celebrated Derpibooru's five year anniversary with friends.
Not a Llama - Happy April Fools Day!
Birthday Cake - Celebrated MLP's 7th birthday
Cool Crow - "Caw!" An awesome tagger
Friendship, Art, and Magic (6 Years) - Celebrated Derpibooru's six year anniversary with friends.
A Tale For The Ages - Celebrated MLP's 35th Anniversary and FiM's 8th Anniversary
Artist -
Friendship, Art, and Magic (7 Years) - Celebrated Derpibooru's seventh year anniversary with friends.

@byte[]
I wish I'd know enough about database design and Rails' inner workings to properly comment on that post. I can only hope hope the transition goes well.

One thing, is the performance drop (which I haven't noticed at all) you mentioned also the reason why the metadata statistics on our profile pages stopped updating a few weeks ago? It's not a big deal, but it's a fun little statistic to watch.

I tried to generate similar statistics using the database dumps, but I couldn't figure out how to group multiple tag edits into one "metadata edit" as the site counts them. There's only one running ID plus timestamps, but trying to group edits using those feels hacky.
byte[]

Site Administrator
Site Developer
@JP
why the metadata statistics on our profile pages stopped updating a few weeks ago
It's a bug, I'm not quite sure why it's happening

I tried to generate similar statistics using the database dumps but I couldn’t figure out how to group multiple tag edits into one "metadata edit" as the site counts them
That's because they're calculated for each PUT to /images/:image_id/tags. That information vanishes after the request is made.

Try select user_id, date_trunc('minute', created_at) as time from tag_changes group by user_id, time
JP
The End wasn't The End - Found a new home after the great exodus of 2012
Magnificent Metadata Maniac - #1 Assistant
Friendship, Art, and Magic (5 Years) - Celebrated Derpibooru's five year anniversary with friends.
Not a Llama - Happy April Fools Day!
Birthday Cake - Celebrated MLP's 7th birthday
Cool Crow - "Caw!" An awesome tagger
Friendship, Art, and Magic (6 Years) - Celebrated Derpibooru's six year anniversary with friends.
A Tale For The Ages - Celebrated MLP's 35th Anniversary and FiM's 8th Anniversary
Artist -
Friendship, Art, and Magic (7 Years) - Celebrated Derpibooru's seventh year anniversary with friends.

@byte[]
Try select user_id, date_trunc('minute', created_at) as time from tag_changes group by user_id, time
Nice, I think I can figure out the rest. If not, I'll ask.

Regarding Philomena, if we go to philomena.derpibooru.org (once it's up (it's not yet)) and log in and use the site from there, will our actions there get mirrored back to the main site and vice-versa, in realtime? Or will it be a separate site, like Arcaire's Derpi Staging?
Wiimeiser
Artist -
Not a Llama - Happy April Fools Day!
An Artist Who Rocks - 100+ images under their artist tag
A Tale For The Ages - Celebrated MLP's 35th Anniversary and FiM's 8th Anniversary
Wallet After Summer Sale -
Dream Come True! - Participated in the MLP 9th Anniversary Event

(Foil Hat)
@cheezedoodle
I think it got split off into Rumblebooru and Puppybooru at some point? Something about NSFW. But I think Kerfufflebooru still has promise. Just running with the joke here
Background Pony #BA79
When Anonymous Uploader Reports duplicate, actual account is revealed.
Similar to tag changes, instead of account show something like "Background Pony #???? (Anonymous Source/Target Uploader)".
PUBLIQclopAccountant
Magical Inkwell - Wrote MLP fanfiction consisting of at least around 1.5k words, and has a verified link to the platform of their choice

Pubilq Phirm
First a new request (may be a duplicate or already implemented and I don't know where it is in the UI): a button to clear all notifications. I just got back from an internet vacation and came back to 80+ notifications.

@byte[]
@byte[]

And here's that Patreon post.

Another Phoenix is better than Rails post. Unlike the one I’m more familiar with, I do not expect this one to create pointless partisan debate.
byte[]

Site Administrator
Site Developer
@PUBLIQclopAccountant
You'll have to forgive me for asking, but I don't know what prior material you're referring to (and if it was written primarily for the readers of orange site, then maybe that's for the best)
Background Pony #00F1
After the switch to Philomena is done, how about adding soundcloud embeds and maybe also touching up how the external links work?
OneMoreAnonymous

Well I didn't expect my question to blow the lid on hot hot site rewrite. What a scoop!

A few questions, if you please. I searched for answers for the latter 2, but found nothing.

1) You seem to be only too glad to say goodbye to RoR. Still, does it have any features you find useful which Phoenix doesn't have?

2) In the Patreon post, you wrote:
Other websites are generally tied directly into a usage-based pricing model, and we don't do that.

Any particular reason for this? If I had to hazard a guess, I'd say it's because that model increases in price too much as it scales upwards, so sites growing quickly and/or with a lot of room for growth are better off using fixed prices.

3) I tried using the /tags.json API endpoint, but it seems the only query parameter it accepts is page, is that right?

4) Can I please have a list of all characters which are parsed as their names e.g. \ as fwslash?
byte[]

Site Administrator
Site Developer
1. Lots. I miss being able to call finder and query methods on models, and the ability to simply patch out bad behavior in external code (Ruby encourages monkey-patching). The ecosystem is also a lot less mature for Elixir, though we do get to take advantage of Erlang packages, which is nice.

2. We don't do it primarily because dealing with money sucks. If dealing with money didn't suck, then it might be nice to even give artists a financial reward for posting here. But getting PCI certified costs a lot of money, and we can't use external payment providers because we host porn, and none of them will work with porn.

3. I think so.

4. Please pull them from a dump.
Joey
Dawnsong - Derpi Supporter
The End wasn't The End - Found a new home after the great exodus of 2012
Artist -
Happy Derpy! -
Flexible - Bendy Pony
Best Artist - Best Artist - Providing quality, Derpibooru exclusive artwork
An Artist Who Rocks -
Perfect Pony Plot Provider - 10+ uploads with over 350 upvotes or more (Questionable/Explicit)
Friendship, Art, and Magic (5 Years) - Celebrated Derpibooru's five year anniversary with friends.
The Power of Love - Given to a publicly verified artist with an image under their artist’s tag that has reached 1000 upvotes

Site Administrator
Public Relations
Nerd Pony
@OneMoreAnonymous
2) In the Patreon post, you wrote:
Other websites are generally tied directly into a usage-based pricing model, and we don't do that.

Any particular reason for this? If I had to hazard a guess, I'd say it's because that model increases in price too much as it scales upwards, so sites growing quickly and/or with a lot of room for growth are better off using fixed prices.


Let me add some additional fluff to what byte wrote. Our primary server, aka the site's most expensive asset, costs roughly 450 Euro per month. For that fixed, predictable price, we get a system with ~10TB of SSD storage, 500Mbps unmetered bandwidth, 96GB of RAM, and a fuckton of CPU cores.

According to Cloudflare, in the past thirty days, our server transferred approximately 65TB of data (Overall, the site consumed approximately 150TB of bandwidth, but 85TB of it came from Cloudflare's cache instead of our server). According to AWS's "Simple" Monthly Cost Calculator, 65TB of outbound data transfer would cost roughly 5400 Euro per month.

Yeah, it varies month to month and different providers charge different rates and all that, but still — when bandwidth alone would be 10-12x higher than the monthly cost of a dedicated server that includes everything, usage-based pricing is not viable for us.
byte[]

Site Administrator
Site Developer
@byte[]
@Joey

It struck me that we are likely talking about different things, so let me just further clarify what I meant by "usage-based pricing model." When I say that, I'm not talking about the cost to us. I'm talking about a cost to users to use the site (in the form of advertising, or a monthly fee).

I will note, separately, that most RoR websites are commercial, that they do have usage-based pricing, and they can spend much more money on their servers due to generating large amounts of revenue. We are not, and I never intend for us to be commercial. We're just here to look at ponies.
OneMoreAnonymous

@byte[]
But getting PCI certified costs a lot of money, and we can't use external payment providers because we host porn, and none of them will work with porn.

As far as monthly fees and donations go, I'm sure you have already checked into them and discarded the choice, but there are payment processors that deal with adult sites. Well, credit card processors at least.

About ads, I was gonna ask about Project Wonderful, but I see they went tits-up a year ago. RIP in F.

@byte[]
4. Please pull them from a dump.


My first idea was to gather them using regex search voodoo on /tags.json, that's why I asked about the lack of endpoints for it. And now it just occurred to me that there might be characters which the software parses but haven't been used in any tags yet. Or would the dump contain that information too?

@Joey
Jesus Christ, that isn't even highway robbery, it's fucking enslavement. I keep forgetting that unmetered bandwidth is common, nowadays.

Thank you both for the answers.

Edit: I just found a small bug which, I guess, must have been reported by now, but anyway. "dump":/pages/data_dumps. parses the URL correctly by excluding the period, but the "dump":/pages/data_dumps.[/bq] in this post borked it, so I had to add a whitespace after the period.
Damaged
Happy Derpy! - For Patreon supporters
Not a Llama - Happy April Fools Day!
Magical Inkwell - Wrote MLP fanfiction consisting of at least around 1.5k words, and has a verified link to the platform of their choice
Equality - In our state, we do not stand out.
Wallet After Summer Sale -
Magnificent Metadata Maniac -
Notoriously Divine Tagger - Consistently uploads images above and beyond the minimum tag requirements. And/or additionally, bringing over the original description from the source if the image has one. Does NOT apply to the uploader adding several to a dozen tags after originally uploading with minimum to bare tagging.
Perfect Pony Plot Provider - Uploader of 10+ images with 350 upvotes or more (Questionable/Explicit)
Fine Arts - Two hundred uploads with a score of over a hundred (Safe/Suggestive)

Site Developer
Word Bug
@OneMoreAnonymous
You don't need to do voodoo on it. /tags.json is valid JSON. You can just load it with any language's JSON parser and get an array out the other side.

But as stated, loading up PostgreSQL and grabbing the database dumps will get you this list in a lot shorter time. (Even accounting for needing a crash course in SQL along the way.)
Joey
Dawnsong - Derpi Supporter
The End wasn't The End - Found a new home after the great exodus of 2012
Artist -
Happy Derpy! -
Flexible - Bendy Pony
Best Artist - Best Artist - Providing quality, Derpibooru exclusive artwork
An Artist Who Rocks -
Perfect Pony Plot Provider - 10+ uploads with over 350 upvotes or more (Questionable/Explicit)
Friendship, Art, and Magic (5 Years) - Celebrated Derpibooru's five year anniversary with friends.
The Power of Love - Given to a publicly verified artist with an image under their artist’s tag that has reached 1000 upvotes

Site Administrator
Public Relations
Nerd Pony
@Joey
Asking here because I can't do any coding on an iPhone, and that's all I have at the moment.

Can we add autocorrect="off" and spellcheck="false" to the search field at the top of the page? It's a real bitch trying to search for artist, character, and oc tags since phones love turning them into the closest English word instead.

This has been done
Aryanne's Jewish Sex Slave

Banned
I think it would be a good idea to have a system where users could specify that they only wanted to receive notifications for comments that are replying to them, preferably on an image-by-image basis. That way, if a user didn't want to get dragged into drama on an image where there was an argument going on, but they did have something they wanted to say and wanted to know what others thought about it, it wouldn't be an "all or nothing" scenario. I saw in the first post that the mods were "planning on" something like that, but since it's been 6 years I assume they gave up, so maybe it's impractical or something.
Interested in advertising on Derpibooru? Click here for information!
Ministry of Image - Fanfiction Printing

Derpibooru costs over $25 a day to operate - help support us financially!

Syntax quick reference: *bold* _italic_ [spoiler]hide text[/spoiler] @code@ +underline+ -strike- ^sup^ ~sub~