Probably poor organization.
I'm also becoming increasingly convinced that the people who wrote enemy AI had little-to-no contact with the devs who actually implemented the fights in the field, except for info on scene backgrounds (which some enemies are 'tied' to). There are just too many cases of complex, difficult-to-debug code that does extraordinary things (like transform enemies, spawn new ones etc.) or makes arbitrarily precise choices... that get perhaps 30 seconds per game, on average. Not to mention the abundance of creatures near difficult sections (where reloading saves and restarting the random number path reduces the range of encounters). Likewise, I think enemy design was fairly autonomous - that's why there's comparatively so many creatures in FFVII. I mean, look at the enemy rosters:
VII: 265
VIII: 193
IX: 197
X: 148
I get the strong impression that enemy design was done on its own, with a load of creatures created without much sense of context, then just kinda 'dumped' on the desk of higher-level field designers. I've heard the theory that a large number of creatures makes the experience varied, and I can buy the idea that Square wanted to showcase the new Sony technology, but it's an awful lot of design, animation and debugging for comparatively little return.
I'd like to know more about the development of the game, but the info is likely lost, and at any rate, it's difficult enough getting a company to confess its flaws internally, yet alone to the public.