This has made me realise why although I love tarot cards and casting lots and tea leaves and stuff, I hate horoscopes with an absolute passion: Because the tarot cards conceptually deal with "I" (Whether that's I-the-reader or I-the-client) and horoscopes are about "YOU" (You-the-passer-by).
Your coworker who is incredibly into reading tea leaves or geomancy: If they want to pull you into that, they need to sit you down, offer to read for you, ask you questions. You need to engage with the process, and it is the norm not to read for someone who isn't present or interested, and any reading is temporary and keeps changing (Ofc if you keep reading for one person over and over and the same motifs keep appearing, you might start to get an association between them and a figure- but they have to keep engaging for that to happen!) - If you don't want to keep playing, all you have to do is stop sitting to have your cards read.
But if your coworker is into horoscopes, once they know your vague birth month they can start offering up vague readings basically indefinitely - ie "Oh, such a Taurus!" And read that into your every move. Even if they don't have your birth day, they can project astrological events onto you ("You need to be cautious about that new job offer because of Mercury in retrograde" No Tarquin, I want to grab it with both hands because I have manoeuvred for years to get it and it's finally happening, etc).
And we don't seem to have a good social script for stopping it that doesn't devolve into "Let people enjoy things/at least it isn't the Catholic church/astrology never hurt anyone/being annoyed by this is antifeminist and antiqueer" even though "Please stop speculating on my life through the lens of your spiritual practice that I'm not involved in" should be enough.