This has happened to me about 3 times so far in recent games. I will clear a monster zoo and except for one monster that I can never ever find. I'll start clearing all adjacent rooms as well, and so on, until eventually I will have strayed too far away from the zoo and lose out on the reward. I don't know if the count is incorrect, if a creature spawned inside a wall, or what (I've tried using AoEs at various walls and so far no luck). Ack nm lol. Found the critter 3 rooms away this time back in the direction I came from (he must have fled from my scary awesomeness). At least I know what's going on now.
This problem is particularly bad if you have to drop back two or three rooms to find a chokepoint, or if there are teleporting monsters in the zoo. It can usually be fixed, as you've found, by searching the area thoroughly, but it's still very annoying. A feature that marks zoo monsters, rather than zoo rooms, on the map would fix it nicely...
I did think it may have been a teleporting critter. And that, in fact, has happened in the past, where it would teleport past closed doors (which is why I tried every adjacent door). However, in this case, the creature was not a teleporter -- he simply decided to run far away from me for some reason. In any case, yes, that would work, as would changing the AI so that fleeing or wandering zoo creatures would never stray more than one or even two room's distance from a monster zoo. Or you could give credit for a kill if the creature did run too far away.
This is a real nightmare when you have invisible teleporters. Like on level 14. Spent 15 minutes trying to track down the last two monsters. One of them dropped off the counter for no apparent reason. I never did find the second one, because whilst checking to see if it got stuck in any walls, my spell template happened to extend beyond the edge of the map and crashed the game.
if you're having trouble finding the last monster in a zoo, it's almost always because a random monster blinked behind a door or a wall.
True that that does happen a lot, it's not the only way. As I said, I've seen non-teleporting critters flee from me as well and somehow wind up three rooms away. Then again, I just noticed something today -- I found a teleport blink trap earlier, and because I had such an awful (-5) chance of disarming it, and because it was blocking my only path to the other side of a room, and because it was responsible for a most horrible death in a previous run-through, I woke up and lured a critter into it. He teleported all over the place, and disappeared eventually -- no idea where he wound up.