OK, so I finally flagged down my upstairs neighbors.
It was early and I was still in my nightcap and pajamas, so I looked like a crazy person, but if you've been following, my previous attempt at a more sane encounter was unsuccessful.
I asked them if they can hear the weird neighbor following them into the bathroom and turning the water on and off because I've noticed someone else start to go back and forth with him. They said it wasn't them (I don't believe that), but that they can hear stuff at night.
I told them that instead of going back and forth, can you talk to the property manager and can you tell them that the lady beneath you can hear the same thing.
The whole thing was one of the more awkward human encounters that I've had in my entire life.