20-Year Prospect
If you really want to be a janitor for the next twenty years, then we've got some good news: you probably don't have to worry about your dream job going anywhere. There will never be a shortage of buildings to maintain. In fact, the janitorial field is on track for a 12% job growth, which puts it in line with most other fields (source).
Of course, there's always the threat of robots encroaching on your territory, but you still have some time before that becomes a real issue. Roombas just can't scrub a toilet like you can. The more immediate threat to your career is burnout.
This job is just plain hard, both on the body and the mind, and you'll have to be super-strong to make it twenty years. Sure, it's possible to do this work for two decades. But you might find that you want to switch to something a little less demanding.