Pony with extreme care Tag images of/about the latest early episodes with spoiler:s09e22, spoiler:s09e23, ..., or spoiler:s09e26 and don't post spoilers about those episodes in other image threads.

Feature suggestions and discussion [READ THE FIRST POST]

Started by The Frowning Pony
Posted
5,747 replies
Login to subscribe to responses
Derpy Whooves's avatar
Posts: 3420
Derpy Whooves
Not a Llama - Happy April Fools Day!
Economist

Site Moderator
Artist Concierge
@JP
For what it’s worth, from the mod side of staff the view into the dev space with the existing code base and all its legacy issues has often been a little pic related.



Just my personal opinion, but the legacy code on the site hasn’t always been "just fine". Teams of people have been spending a lot of their personal time keeping the site running. And if they say it needs to move to a new code base, I trust them.
byte[]'s avatar
Posts: 1761
byte[]
Site Administrator
Site Developer
@JP
I actually wanted one day to see the Booru On Rails’ source code, but looks like I never will :-(
Don’t worry, we’re still releasing it. I’ve committed to a hard deadline on the release – Dec 25, 2019.

The old code is tested and proven to work. The new code… it’s not. Rewriting working code in a new language (that I’m not even sure what it is called) is a known way to fail.

I’m painfully aware of this. My plan is to develop and test incrementally. We’ll be running Derpibooru and Philomena in parallel for a while to assert that Philomena works without major hiccups. This will be hosted at philomena.derpibooru.org.

How come the old code is now, suddenly, too slow, when it has worked just fine for 8 years?
This is something I’ll be covering in an upcoming patreon post; hang tight.
Posted Report
GenericArchangel's avatar
Posts: 3783
GenericArchangel
Wallet After Summer Sale
Magical Inkwell - Wrote MLP fanfiction consisting of at least around 1.5k words, and has a verified link to the platform of their choice

Free the Tantabus
@byte[]
Okay really dumb question: this site’s name won’t change, right? The wording of that makes it sound like the site’s name will change, but that can’t be right.
Posted Report
JP's avatar
Posts: 1590
JP
The End wasn't The End - Found a new home after the great exodus of 2012
Magnificent Metadata Maniac - #1 Assistant
Friendship, Art, and Magic (5 Years) - Celebrated Derpibooru's five year anniversary with friends.
Not a Llama - Happy April Fools Day!
Birthday Cake - Celebrated MLP's 7th birthday
Cool Crow - "Caw!" An awesome tagger
Friendship, Art, and Magic (6 Years) - Celebrated Derpibooru's six year anniversary with friends.
A Tale For The Ages - Celebrated MLP's 35th Anniversary and FiM's 8th Anniversary
Artist
Friendship, Art, and Magic (7 Years) - Celebrated Derpibooru's seventh year anniversary with friends.
Best Artist - Providing quality, Derpibooru-exclusive artwork
Wallet After Summer Sale

@byte[]
I wish I’d know enough about database design and Rails’ inner workings to properly comment on that post. I can only hope hope the transition goes well.

One thing, is the performance drop (which I haven’t noticed at all) you mentioned also the reason why the metadata statistics on our profile pages stopped updating a few weeks ago? It’s not a big deal, but it’s a fun little statistic to watch.

I tried to generate similar statistics using the database dumps, but I couldn’t figure out how to group multiple tag edits into one "metadata edit" as the site counts them. There’s only one running ID plus timestamps, but trying to group edits using those feels hacky.
Posted Report
byte[]'s avatar
Posts: 1761
byte[]
Site Administrator
Site Developer
@JP
why the metadata statistics on our profile pages stopped updating a few weeks ago
It’s a bug, I’m not quite sure why it’s happening

I tried to generate similar statistics using the database dumps but I couldn’t figure out how to group multiple tag edits into one "metadata edit" as the site counts them
That’s because they’re calculated for each PUT to /images/:image_id/tags. That information vanishes after the request is made.

Try select user_id, date_trunc('minute', created_at) as time from tag_changes group by user_id, time
Posted Report
JP's avatar
Posts: 1590
JP
The End wasn't The End - Found a new home after the great exodus of 2012
Magnificent Metadata Maniac - #1 Assistant
Friendship, Art, and Magic (5 Years) - Celebrated Derpibooru's five year anniversary with friends.
Not a Llama - Happy April Fools Day!
Birthday Cake - Celebrated MLP's 7th birthday
Cool Crow - "Caw!" An awesome tagger
Friendship, Art, and Magic (6 Years) - Celebrated Derpibooru's six year anniversary with friends.
A Tale For The Ages - Celebrated MLP's 35th Anniversary and FiM's 8th Anniversary
Artist
Friendship, Art, and Magic (7 Years) - Celebrated Derpibooru's seventh year anniversary with friends.
Best Artist - Providing quality, Derpibooru-exclusive artwork
Wallet After Summer Sale

@byte[]
Try select user_id, date_trunc('minute', created_at) as time from tag_changes group by user_id, time
Nice, I think I can figure out the rest. If not, I’ll ask.

Regarding Philomena, if we go to philomena.derpibooru.org (once it’s up (it’s not yet)) and log in and use the site from there, will our actions there get mirrored back to the main site and vice-versa, in realtime? Or will it be a separate site, like Arcaire’s Derpi Staging?
Posted Report
Wiimeiser's avatar
Posts: 10565
Wiimeiser
Artist
Not a Llama - Happy April Fools Day!
An Artist Who Rocks - 100+ images under their artist tag
A Tale For The Ages - Celebrated MLP's 35th Anniversary and FiM's 8th Anniversary
Wallet After Summer Sale

(Foil Hat)
@cheezedoodle
I think it got split off into Rumblebooru and Puppybooru at some point? Something about NSFW. But I think Kerfufflebooru still has promise. Just running with the joke here
Posted Report
PUBLIQclopAccountant's avatar
Posts: 31
PUBLIQclopAccountant
Magical Inkwell - Wrote MLP fanfiction consisting of at least around 1.5k words, and has a verified link to the platform of their choice

Pubilq Phirm
First a new request (may be a duplicate or already implemented and I don’t know where it is in the UI): a button to clear all notifications. I just got back from an internet vacation and came back to 80+ notifications.

@byte[]
@byte[]

And here’s that Patreon post.

Another Phoenix is better than Rails post. Unlike the one I’m more familiar with, I do not expect this one to create pointless partisan debate.
Posted Report
byte[]'s avatar
Posts: 1761
byte[]
Site Administrator
Site Developer
@PUBLIQclopAccountant
You’ll have to forgive me for asking, but I don’t know what prior material you’re referring to (and if it was written primarily for the readers of orange site, then maybe that’s for the best)
Posted Report
Background Pony #6288
After the switch to Philomena is done, how about adding soundcloud embeds and maybe also touching up how the external links work?
Posted Report
Posts: 41
OneMoreAnonymous
Well I didn’t expect my question to blow the lid on hot hot site rewrite. What a scoop!

A few questions, if you please. I searched for answers for the latter 2, but found nothing.

1) You seem to be only too glad to say goodbye to RoR. Still, does it have any features you find useful which Phoenix doesn’t have?

2) In the Patreon post, you wrote:
Other websites are generally tied directly into a usage-based pricing model, and we don’t do that.

Any particular reason for this? If I had to hazard a guess, I’d say it’s because that model increases in price too much as it scales upwards, so sites growing quickly and/or with a lot of room for growth are better off using fixed prices.

3) I tried using the /tags.json API endpoint, but it seems the only query parameter it accepts is page, is that right?

4) Can I please have a list of all characters which are parsed as their names e.g. \ as -fwslash-?
Posted Report
byte[]'s avatar
Posts: 1761
byte[]
Site Administrator
Site Developer
1. Lots. I miss being able to call finder and query methods on models, and the ability to simply patch out bad behavior in external code (Ruby encourages monkey-patching). The ecosystem is also a lot less mature for Elixir, though we do get to take advantage of Erlang packages, which is nice.

2. We don’t do it primarily because dealing with money sucks. If dealing with money didn’t suck, then it might be nice to even give artists a financial reward for posting here. But getting PCI certified costs a lot of money, and we can’t use external payment providers because we host porn, and none of them will work with porn.

3. I think so.

4. Please pull them from a dump.
Posted Report
Joey's avatar
Posts: 4205
Joey
Dawnsong - Derpi Supporter
The End wasn't The End - Found a new home after the great exodus of 2012
Happy Derpy!
Artist
Flexible - Bendy Pony
Best Artist - Best Artist - Providing quality, Derpibooru exclusive artwork
An Artist Who Rocks
Perfect Pony Plot Provider - 10+ uploads with over 350 upvotes or more (Questionable/Explicit)
Friendship, Art, and Magic (5 Years) - Celebrated Derpibooru's five year anniversary with friends.
The Power of Love - Given to a publicly verified artist with an image under their artist’s tag that has reached 1000 upvotes
Friendly Griffon - For helping others attend the fifth anniversary party
Not a Llama - Happy April Fools Day!
Gold Patron - Gold Patron
Birthday Cake - Celebrated MLP's 7th birthday
Helpful Owl - Drew someone's OC for the 2018 Community Collab
Friendship, Art, and Magic (6 Years) - Celebrated Derpibooru's six year anniversary with friends.
Equality - In our state, we do not stand out.
A Tale For The Ages - Celebrated MLP's 35th Anniversary and FiM's 8th Anniversary
Friendship, Art, and Magic (7 Years) - Celebrated Derpibooru's seventh year anniversary with friends
Friendship, Art, and Magic (7 Years) - Celebrated Derpibooru's seventh year anniversary with friends
Thread Starter - Started a thread with over 100 pages (DCC2019)
Silver Bit
Heart Gem
Magnificent Metadata Maniac - #1 Assistant
Wallet After Summer Sale
Toola Roola - For helping others attend the 2019 Community Collab

Site Developer
Nerd Pony
@OneMoreAnonymous
2) In the Patreon post, you wrote:
Other websites are generally tied directly into a usage-based pricing model, and we don’t do that.

Any particular reason for this? If I had to hazard a guess, I’d say it’s because that model increases in price too much as it scales upwards, so sites growing quickly and/or with a lot of room for growth are better off using fixed prices.


Let me add some additional fluff to what byte wrote. Our primary server, aka the site’s most expensive asset, costs roughly 450 Euro per month. For that fixed, predictable price, we get a system with ~10TB of SSD storage, 500Mbps unmetered bandwidth, 96GB of RAM, and a fuckton of CPU cores.

According to Cloudflare, in the past thirty days, our server transferred approximately 65TB of data (Overall, the site consumed approximately 150TB of bandwidth, but 85TB of it came from Cloudflare’s cache instead of our server). According to AWS’s "Simple" Monthly Cost Calculator, 65TB of outbound data transfer would cost roughly 5400 Euro per month.

Yeah, it varies month to month and different providers charge different rates and all that, but still – when bandwidth alone would be 10-12x higher than the monthly cost of a dedicated server that includes everything, usage-based pricing is not viable for us.
Posted Report
byte[]'s avatar
Posts: 1761
byte[]
Site Administrator
Site Developer
@byte[]
@Joey

It struck me that we are likely talking about different things, so let me just further clarify what I meant by "usage-based pricing model." When I say that, I’m not talking about the cost to us. I’m talking about a cost to users to use the site (in the form of advertising, or a monthly fee).

I will note, separately, that most RoR websites are commercial, that they do have usage-based pricing, and they can spend much more money on their servers due to generating large amounts of revenue. We are not, and I never intend for us to be commercial. We’re just here to look at ponies.
Posted Report
Posts: 41
OneMoreAnonymous
@byte[]
But getting PCI certified costs a lot of money, and we can’t use external payment providers because we host porn, and none of them will work with porn.

As far as monthly fees and donations go, I’m sure you have already checked into them and discarded the choice, but there are payment processors that deal with adult sites. Well, credit card processors at least.

About ads, I was gonna ask about Project Wonderful, but I see they went tits-up a year ago. RIP in F.

@byte[]
4. Please pull them from a dump.


My first idea was to gather them using regex search voodoo on /tags.json, that’s why I asked about the lack of endpoints for it. And now it just occurred to me that there might be characters which the software parses but haven’t been used in any tags yet. Or would the dump contain that information too?

@Joey
Jesus Christ, that isn’t even highway robbery, it’s fucking enslavement. I keep forgetting that unmetered bandwidth is common, nowadays.

Thank you both for the answers.

Edit: I just found a small bug which, I guess, must have been reported by now, but anyway. "dump":/pages/data_dumps. parses the URL correctly by excluding the period, but the "dump":/pages/data_dumps.[/bq] in this post borked it, so I had to add a whitespace after the period.
Damaged's avatar
Posts: 446
Damaged
Happy Derpy! - For Patreon supporters
Not a Llama - Happy April Fools Day!
Magical Inkwell - Wrote MLP fanfiction consisting of at least around 1.5k words, and has a verified link to the platform of their choice
Equality - In our state, we do not stand out.
Wallet After Summer Sale

Word Bug
@OneMoreAnonymous
You don’t need to do voodoo on it. /tags.json is valid JSON. You can just load it with any language’s JSON parser and get an array out the other side.

But as stated, loading up PostgreSQL and grabbing the database dumps will get you this list in a lot shorter time. (Even accounting for needing a crash course in SQL along the way.)
Posted Report
Interested in advertising on Derpibooru? Click here for information!
Champions of Equestria

Derpibooru costs over $25 a day to operate - help support us financially!

Syntax quick reference: *bold* _italic_ [spoiler]hide text[/spoiler] @code@ +underline+ -strike- ^sup^ ~sub~