Well, the biggest change in 4.0 was supposed to better pathfinding where the AI would retreat out of cover (depending on moral, and how heavy the suppression) and properly fall back to next cover. However, there was issues with the pathfinding, etc, and by 4.02-03 (a year or so later) BF decided to simply have troops die in cover...Basically, it's now up to the player to fall back a turn or two ahead of time and not rely on Tac AI to do.