DerpyFast 
I don’t think Starlight is the writer’s pet or that she’s part of some conspiracy. I think something much simpler is going on.
First, you have to talk about the show’s ambition as time went on.
Season 1 was originally intended to be an episodic show without much continuity. It teased The Gala, but it just sort of happened at the end of the season.
Season 2 established some new stuff, but the season itself had little to no payoff.
Season 3 had the first inkling of an arc, by setting something up and building up to it. It also had the characters actually advance their goals for the first time since Season 1.
Season 4 was boatloads more ambitious, with a big, 9-part story arc that explored The Elements more deeply and advanced The Mane Six’s goals. Sadly, the actual exploration of The Elements didn’t yield very much.
Season 5 gave us a more ideologically-motivated villain, and explored the theme of moving past uncertainty or hardships, which was reincorporated in the season finale. It also wrapped up the arcs of Luna and Diamond Tiara.
Seasons 5 and 6 expanded the world, establishing the Yaks, and giving us more information on the Griffons, the Dragons and the Changelings.
Season 6 had Starlight undergo a character arc where she compensates for her lack of confidence in interacting with others with magic, until the finale strips her of her magic and forces her to co-operate with others.
Seasons 6 and 7 wrapped up the arcs of those in The Mane Six that had them, and established new character directions for them.
Season 7 used Starlight to explore the idea of conviction, while also using a number of different characters to explore a theme of transitioning to a new stage in life. It also fleshed out other pre-existing characters, and gave us the Pillars of Equestria arc.
So needless to say, the show has gotten a lot more ambitious since it first started. And therein lies the secret. The show started out a lot less ambitious, and a lot more episodic. For an episodic show, you want static characters; i.e. characters that don’t change a great deal; so that you can always have the same personalities to work with. They might apply previous lessons they’d learned, but other than Twilight in episodes 1 and 2, wouldn’t really undergo character arcs. They had goals, but those were “someday” goals, that would probably be achieved in an epilogue when the show finally ended.
The Mane Six’s goals have several problems. First of all, with the exception of Fluttershy, and Twilight during the series premier, none of the characters have to undergo a whole lot of change to reach their goals. Rarity is a skilled dressmaker, and it was only a matter of time before the world took notice. Rainbow Dash was already the best flier in Equestria, and has demonstrated leadership qualities, so her getting into The Wonderbolts was also only a matter of time. People were speculating that Twilight would become an alicorn as early as Season 1.
The other problem is the lack of a way to gauge character progress. Rarity had been chugging along, then Boom! She can open a boutique in Canterlot. A season goes by and Boom! She opens one in Manehattan. Now she’s got a fashion empire. Same thing with Twilight, Rainbow Dash, and Fluttershy. There’s not really a concrete sense of progression until they just tell us the character has progressed.
Enter Starlight Glimmer. Starlight had the luxury of being established into a much richer and better-defined universe than The Mane Six. She was also built from the ground up to take advantage of how ambitious the show had gotten. Where the Mane Six were written to be static, she was written to be more dynamic. With the way she’s constructed, the writers can show a more nuanced character change by modulating how frequently and how intensely her flaws manifest. This has led to her character arc having more clearly-defined stages. They would go even further with Tempest Shadow, by representing her growth with a recurring motif.
Going back to Season 7 and Conviction. Celestial advice sets up the theme by having Twilight and Celestia share the doubts they had as teachers. All Bottled Up has Starlight learning the importance of speaking out. A Royal Problem has Starlight going with her gut and it paying off. To Change A Changeling has Starlight going with her gut, but not thinking things through starts to catch up with her. Then in Shadow Play, she’s putting a lot more thought into potential consequences, and she more readily speaks up about her doubts.
It’s not perfect, but it’s the sort of nuanced character progression you can’t really get with static characters. And while Elements like Kindness, Honesty, Loyalty and Generosity are all themes you could absolutely spend multiple episodes across a season exploring, the show kind of kneecapped that potential by having them already be the embodiment of these elements. Four out of six times, they had to introduce a new character to give us a more nuanced look at the element in question.
In conclusion, I don’t think she’s the writers’ pet. The Mane Six and Starlight are just products of different eras of the show’s run, and as a project takes on a life of its own, people will develop more powerful and specialized tools to better tackle it.
First, you have to talk about the show’s ambition as time went on.
Season 1 was originally intended to be an episodic show without much continuity. It teased The Gala, but it just sort of happened at the end of the season.
Season 2 established some new stuff, but the season itself had little to no payoff.
Season 3 had the first inkling of an arc, by setting something up and building up to it. It also had the characters actually advance their goals for the first time since Season 1.
Season 4 was boatloads more ambitious, with a big, 9-part story arc that explored The Elements more deeply and advanced The Mane Six’s goals. Sadly, the actual exploration of The Elements didn’t yield very much.
Season 5 gave us a more ideologically-motivated villain, and explored the theme of moving past uncertainty or hardships, which was reincorporated in the season finale. It also wrapped up the arcs of Luna and Diamond Tiara.
Seasons 5 and 6 expanded the world, establishing the Yaks, and giving us more information on the Griffons, the Dragons and the Changelings.
Season 6 had Starlight undergo a character arc where she compensates for her lack of confidence in interacting with others with magic, until the finale strips her of her magic and forces her to co-operate with others.
Seasons 6 and 7 wrapped up the arcs of those in The Mane Six that had them, and established new character directions for them.
Season 7 used Starlight to explore the idea of conviction, while also using a number of different characters to explore a theme of transitioning to a new stage in life. It also fleshed out other pre-existing characters, and gave us the Pillars of Equestria arc.
So needless to say, the show has gotten a lot more ambitious since it first started. And therein lies the secret. The show started out a lot less ambitious, and a lot more episodic. For an episodic show, you want static characters; i.e. characters that don’t change a great deal; so that you can always have the same personalities to work with. They might apply previous lessons they’d learned, but other than Twilight in episodes 1 and 2, wouldn’t really undergo character arcs. They had goals, but those were “someday” goals, that would probably be achieved in an epilogue when the show finally ended.
The Mane Six’s goals have several problems. First of all, with the exception of Fluttershy, and Twilight during the series premier, none of the characters have to undergo a whole lot of change to reach their goals. Rarity is a skilled dressmaker, and it was only a matter of time before the world took notice. Rainbow Dash was already the best flier in Equestria, and has demonstrated leadership qualities, so her getting into The Wonderbolts was also only a matter of time. People were speculating that Twilight would become an alicorn as early as Season 1.
The other problem is the lack of a way to gauge character progress. Rarity had been chugging along, then Boom! She can open a boutique in Canterlot. A season goes by and Boom! She opens one in Manehattan. Now she’s got a fashion empire. Same thing with Twilight, Rainbow Dash, and Fluttershy. There’s not really a concrete sense of progression until they just tell us the character has progressed.
Enter Starlight Glimmer. Starlight had the luxury of being established into a much richer and better-defined universe than The Mane Six. She was also built from the ground up to take advantage of how ambitious the show had gotten. Where the Mane Six were written to be static, she was written to be more dynamic. With the way she’s constructed, the writers can show a more nuanced character change by modulating how frequently and how intensely her flaws manifest. This has led to her character arc having more clearly-defined stages. They would go even further with Tempest Shadow, by representing her growth with a recurring motif.
Going back to Season 7 and Conviction. Celestial advice sets up the theme by having Twilight and Celestia share the doubts they had as teachers. All Bottled Up has Starlight learning the importance of speaking out. A Royal Problem has Starlight going with her gut and it paying off. To Change A Changeling has Starlight going with her gut, but not thinking things through starts to catch up with her. Then in Shadow Play, she’s putting a lot more thought into potential consequences, and she more readily speaks up about her doubts.
It’s not perfect, but it’s the sort of nuanced character progression you can’t really get with static characters. And while Elements like Kindness, Honesty, Loyalty and Generosity are all themes you could absolutely spend multiple episodes across a season exploring, the show kind of kneecapped that potential by having them already be the embodiment of these elements. Four out of six times, they had to introduce a new character to give us a more nuanced look at the element in question.
In conclusion, I don’t think she’s the writers’ pet. The Mane Six and Starlight are just products of different eras of the show’s run, and as a project takes on a life of its own, people will develop more powerful and specialized tools to better tackle it.
