I’ve seen this happen several times, and every time I had to summon the floor supervisor to get paid. Indeed, in my experience dealers never pay the Ante bonus, as they are supposed to, when the dealer wins. Note the -2 instead of -1 because of both the a-2-3 and q-k-a straights. There are a total of combin(52,3)=22,100 to arrange 3 cards out of 52.
So, the total number of king high combinations is 54*60=3240.
However, four of these 64 result in a flush so there are 64-4=60 combinations of suits left. Next, there are four suits for each rank, or 4 3=64 possible combinations of suits. However, one of these combinations is king-queen-jack, resulting in a straight, so subtracting that combination there are 54 left that do not form a straight. The number of ways to arrange 2 out of 11 is combin(11,2) = 55. The king high hand must have two different of these ranks. The number of ranks less than king is 11. The probability will be the number of king high hands divided by the total number of hands. I'll just work out the king high first and briefly show the formula for the other two.