It's later that night. The club's closed, and Rick has apparently never heard that adage about "never drinking alone." As he sits there, hoping Ilsa will return, Sam tries to get him to leave, but Rick isn't interested in moving.
In fact, his motor capabilities are probably now so impaired that such a thing isn't even possible.
"Of all the gin joints in all the towns in all the world, she walks into mine." Yep. This guy's heart has definitely been through the wringer.
Rick orders Sam to play "As Time Goes By" again, and Sam reluctantly does so.