The Awesome Is Strong in This One
We'll start with the whole stunning, shocking awesomeness of what went down in the ending: Rey goes through the wringer, tries to deny her destiny, gets tortured by a freaky emo Sith Lord, almost sees her only friend die, and then gets into a Whack-at-Each-Other-Until-Someone-Falls-Over lightsaber duel in the middle of a crumbling planet-size Starkiller Base.
All to keep one Luke Skywalker safe and protected until she can get to him.
After all that, we finally arrive at Hidden Island Planet to find the guy that most of us are presumably all pumped to see—the last of the Jedi and perhaps the only remaining hope for the new order—found by a young woman who seems destined to take up his mantle.
There's the sheer emotional impact of seeing Luke again for the first time since Return of the Jedi. He was the last piece of the original film's cast to arrive (we still have our fingers crossed for a Lando Calrissian sighting in Episode VIII), and considering that the film's iconic text scroll literally starts with his name, he's the one we've been presumably hoping to see the most.
When we last left him, he was still a young man (Mark Hamill was 31 when Return of the Jedi opened). So we spend the movie biting our nails down to nubbins and mulling over a bunch of questions: what would Luke look like now? What kind of secrets is he carrying? Dude, his nephew just iced his best friend (who's also the nephew's dad), and thanks to the miracle of the Force, he got a front-row seat to the whole thing. What's that going to do to a guy whose flirtations with the dark side have gotten pretty intense over the years?
The movie has been letting us stew on that for two hours—building in every frame without any kind of release—then condensing the power of it into a few choice shots. We first see Luke with his back to us and then, as Rey approaches, he slowly turns to reveal his face.
Young Luke got old quickly, and not just because he's sporting a Grizzly Adams beard. His face is cracked and grief-stricken. His eyes hold the pain of having messed up big time and never being able to live it down. The wise, hopeful young man at the end of Jedi has been replaced by someone who couldn't avoid life's curveballs and now has to face the cost.
But in the middle of it all comes this girl, who's got baggage of her own to carry but also has some chips in this game and maybe—just maybe—can make it all right.
The dramatic impact of that comes because we've spent the whole movie building up to it. Orson Welles once talked about how he pulled off that trick for his movie The Third Man. Like Luke, everyone spends the whole time talking about Welles' character, and like Luke, we don't actually see him until late in the proceedings:
What matters in that kind of role is not how many lines you have, but how few. What counts is how much the other characters talk about you. (Source)
You can see that play out here: Luke is quite literally the definition of doing more with less. Brr—every time he pulls that hood back, it gives us chills.
How It Serves the Narrative
As we've said elsewhere, Luke is something of a MacGuffin in this story: an object being used to drive the plot forward instead of a character in his own right.
Oh, they drop hints here and there about Kylo Ren's massacre of his students, his guilt and horror, and the fact that everyone is desperately looking for him, but by and large, we're focused on the rest of the cast.
That makes him a great way to keep the plot moving forward, but it also helps the filmmakers resolve a key issue in terms of drama. Franchises like this are always trying to tell a larger story—stretching the narrative out over multiple movies, with each movie acting as a single chapter. (Star Wars actually kind of pioneered the format with Episodes IV-VI.)
But in doing so, you need to be careful not to leave the audience hanging too much. The film still needs to have a beginning, a middle, and an end…even if it's leaving a whole lot of story on the table for next time.
Luke's appearance in The Force Awakens hits a great balance between giving us a great stand-alone movie and leaving us ready to go for next time. The whole film is about finding him…and now we have.
So, the narrative tension in this film—as a stand-alone story—is resolved. In fact, the dramatic sucker punch of his appearance provides an emotional release that makes a perfect way to wrap things up. How are you going to top a moment like that? The Force Awakens knows that it doesn't even have to try.
But at the same time, it leaves a whole truckload of unanswered questions to cover in later films—everything from "what's Luke's ultimate game plan?" to "what went so wrong with Ben Solo's training?" The Force Awakens has the good sense to leave those for future entries—not because they don't need answering, but because it already has enough narrative oomph to stick the dismount and leave us hungry for more.