For the deployment of autonomous robotic systems in mission- and safety-critical underwater environments, aspects such as reasoning and planning need to be designed to operate in highly dynamic, uncertain environments while assuring a safe and reliable operation. Systems are often deployed without a prior safety assessment or developed with safety analysis as a separate engineering process. In this paper, to tackle these challenges, we propose an initial research vision and plan with the envisioned contributions towards designing an approach for system-wide modeling and Model-Based Testing to support safety assessments of autonomous underwater robots.