Yup, AJ even says to the Kirin "we're both from Equestria!", so they are outside of it for sure.
Twilight's verse in the Crystal Empire History song is about a "Crystal Kingdom anthem" so it appears to be an allied kingdom. It is now ruled by Equestrian royals, so who knows. But on other hand, having anthems appears to be a city/town thing, too - Cloudsdale has an anthem and they are part of Equestria for sure, despite floating above it. And Spike mentions the Ponyville anthem.
There was that time where Flam said "we don't care if the whole kingdom of Canterlot helps out!" even though that is the capitol of Equestria, not a separate kingdom. But that was season 2 so the hard lore was still a bit wonky. Or maybe every town and city are considered kingdoms (Ponyville DOES have a massive tree castle on its outskirts since the s4 finale), hence the anthems, and they're all under the Equestrian umbrellla, ruled by Celestia? Honestly your guess is as good as mine.