This is a modified, expanded version of a post I have made elsewhere in an attempt to explain the end of Rebellion, in particular Homura's part in it, her motivations, her attitudes, and so on. It is complicated so this will be long.
Before we dive into this, one thing that must be understood is the use of the ballet version, specifically (as evidenced by Homulilly's Clara dolls) of "The Nutcracker" for symbolism in Rebellion. It features a girl, Clara, who takes pity on the nutcracker with a broken jaw, a tool that can no longer fulfill its function, but she still loves it. She takes it to her room where it comes alive in time to fight to protect her from an army of mice that arrives. On the verge of losing she rescues it from the Mouse King long enough for the nutcracker to defeat him. The nutcracker then transforms into a prince and takes Clara away to his magical world where they can live happily.
On to the movie. Homura was on the verge of death due to overtaxing her soul gem. Kyubey put her soul gem in an isolation field to keep the law of cycles from reaching her, but allowed her to invite things in. Since she claimed to know the law of cycles personally, they presumed she could invite it in so they could observe it and control it.
Three scenes stand out as important in what happens next. The first is the conversation between Homura and Sayaka after the Mami battle where Sayaka forces Homura to question if she is okay with destroying that version of Mitakihara city where everyone is having a good time working together, fighting nightmares, not in any real danger, etc. The next is the flower field scene, wherein Madoka indicates, perhaps through lack of specifics in Homura's questioning, leaving behind everyone would hurt her and that she could never leave everyone behind. Finally, the conversation between Homura and Kyubey after she learns the truth, wherein he reveals his intent to observe the law of cycles, thus interfere, thus control it, thus forcing magical girls to become witches again. She stands by her wish to protect Madoka even now, and would choose death and exile into permanent despair over letting the Incubators have her. Rage and coobie annihilation ensue, Madoka and crew free Homura, and come to finish the job.
So, Homura has a choice - she can either go with Madoka or she can do something else. Based on conversation 2, the flower field, she believes Madoka is suffering where she is, that being unable to reach or interact with her family and friends (magical girl or otherwise) is causing Madoka to suffer, something Homura cannot abide. Based on conversation 3, Incubator, she knows that the Incubators believe that once they observed her, they could completely negate her wish and the Incubators have now observed her, something that would move Madoka suffering for all magical girls to Madoka suffering for no reason at all. She knows she created a world where Madoka was happy for a while and she is aware she has the power to do it again, this time on a much grander scale, not through illusions but through bending the fabric of the universe to the cause of Madoka's happiness, and, per the conversation with Sayaka, creates a version of Mitakihara City (and everything else) where all of them can have fun times together, hell, even the familiars seemed to be having a pleasant coexistence with humans. She also now has the power to bring the Incubators to heel, ensuring they can do nothing to Madoka ever again by making them deal with the curses personally.
So, summed up, the choices in Homura’s mind are either leave Madoka suffering, soon without purpose if the incubators have their way but honor her wish as she once tried to do, or sully Madoka's wish, but save her from despair and protect her from the Incubators. This to her is a terrible choice, a no win situation, because either way she feels like she is betraying Madoka. She chooses the latter and hates herself for it, more on that later.
Madoka arrives, Homura grabs her hands and uses her power to divide the law of cycles, specifically, she took "the records of who Madoka used to be before she became the law of cycles" and split it off from the rest. She then rewrote the universe to create one in which Madoka could like a happy human life. We know this is not merely a witch barrier because of her discussion with Kyubey - that this was the second time she had seen a new principle born into the universe. Likewise, through her conversation with Sayaka we know Wraiths are the enemy of choice rather than witches, "(M)aybe when all the Wraiths are destroyed I will" which indicates the Law of Cycles continues to function even without Madoka guiding it.
One of the key symbolic purposes of this universe from a storytelling perspective is that it is, in theory, the first time everyone got what they wanted. Madoka lives happily with her family and goes to school with people she can see every day and be friends with. Mami is no longer alone, having Nagisa with her. Kyoko and Sayaka are together, Sayaka both not being dead and able to hear Kyosuke playing the violin again. Nagisa has cheese. Homura finally seems to have succeeded in protecting Madoka and Madoka's happiness. So why does she want to die?
Homura has always considered herself useless baggage. Going back to the first timeline, she labeled herself a burden, unable to do anything right, always embarrassing herself. She was alone and friendless until Madoka and never had a sense of self-worth. Rebellion isn't even remotely subtle in depicting Homura's depression as having reached suicidal levels. Everything about a witch in some way reflects her feelings, the labyrinth, the shape, the familiars, etc. So, what can we learn about Homura?
To begin with, Homura is the nutcracker witch - a broken tool that can no longer fulfill its purpose, in a word, useless but for the love a girl has for it. Homura made her wish to fight for Madoka - something no longer possible because Madoka does not, and never did, exist in this universe, rather on a higher plane. Recall Homura's remarks in episode 11, "protecting Madoka was the only thing she had left to guide her" and "I lost myself a long time ago." We see this reflected in the form of her witch, a creature missing half its jaw, whose teeth fall out, who is condemned to death by its own mind for terrible crimes, crimes we will get to in a bit. The final form of her labyrinth itself was marching to her own execution by guillotine, led by her own familiars.
Speaking of, her most powerful familiars, the Clara Dolls (incidentally, how we know which version of the Nutcracker we are dealing with) are reflections of her feelings about herself. Their names are Inferiority, Cowardice, Coldheartedness, Laziness, Vanity, Pride, Pessimism, Liar, Selfishness, Slander, Blockhead, Jealousy, Stupid-Looking, and Stubbornness. Each is as powerful individually as a magical girl. There is a place reserved for a 15th, Love, but it is suspiciously absent. So, what do they do other than fight mice? Well, to begin with, we see repeated uses of them associated with empty shoes and jumping off cliffs, cultural touchstones for suicide. They attempt to get the attention of Kyoko who tosses them an apple, which Homura forces them not to accept, a pattern also seen with Homura and Mami, an action to get attention and then denying herself the interaction because she believes she is unworthy of it. Likewise, the repeated utterances of "Fort Da", an element from Freudian psychology dealing with, in part, a child who repeatedly destroyed themself by hiding from their reflection, denial of pleasure, and masochism. The Clara Dolls throw tomatoes at her as she proclaims herself to be evil, throwing tomatoes at someone generally being associated with giving a low quality performance. They fake tears for her supposed execution. In short, her familiars harass Homura at every turn because her feelings for herself are hatred.
Why? It goes without saying, Homura idolizes Madoka. She refers to her as “as sacred as a goddess” and has an actual shrine to Madoka in her labyrinth. She defied Madoka's wish, the wish she swore to keep fighting for because it was a world Madoka wanted to protect, and she turned her back on it, first by letting her ascend to Madokami, then becoming a witch and creating a happy dream in her soul gem, and finally by pulling Madoka from Magical Girl heaven. Regardless of cause, regardless of benefit (and by every measure from what we see of Homuworld is better for all involved than the previous one (so far as after all the cracks in Madokaworld were not evident until Rebellion so there may be deep flaws here too we just haven't seen)), she still turned her back on the wish and world Madoka swore to protect and that, to Homura, is unforgivable which is why she looks so miserable all the time. This is not helped by the final exchange with Madoka where Madoka affirms she holds the world precious and that breaking the rules because you want to is wrong, leading Homura to conclude, in true Homu fashion, that some day Madoka will hate her (you will be my enemy etc.)
Final concluding notes.