As the complexity and the scale of networks continue to grow, the management of the network operations and security defense has become a challenging task for network administrators, and many network devices may not be updated timely, leaving the network vulnerable to potential attacks. Moreover, the static nature of our existing network infrastructure allows attackers to have enough time to study the static configurations of the network and to launch well-crafted attacks at their convenience while defenders have to work around the clock to defend the network. This asymmetry, in terms of time and money invested, has given attackers greater advantage than defenders and has made the security defense even more challenging. It calls for new and innovative ideas to fix the problem. Moving Target Defense (MTD) is one of the innovative ideas which implements diverse and dynamic configurations of network systems with the goal of puzzling the exact attack surfaces available to attackers. As a result, the system status with the MTD strategy is unpredictable to attackers, hard to exploit, and is more resilient to various forms of attacks. There are existing survey papers on various MTD techniques, but to the best of our knowledge, insufficient focus was given on the architectural perspective of MTD strategies or some new technologies such as Internet of Things (IoT). This paper presents a comprehensive survey on MTD and implementation strategies from the perspective of the architecture of the complete network system, covering the motivation for MTD, the explanation of main MTD concepts, ongoing research efforts of MTD and its implementation at each level of the network system, and the future research opportunities offered by new technologies such as Software-Defined Networking (SDN) and Internet of Things (IoT).