UAVs used for infrastructure inspections require positioning and ranging to protect against collision accidents with structures in worse GNSS positioning environments such as spaces under a bridge. In this study, we focus on Visual Odometry to achieve positioning without GNSS. Each camera estimates rotation and translation parameters with Visual Odometry processing. In addition, we discuss an adjustment control of accumulation errors in Visual Odometry. First, relocalized positions are detected as a loop closure point in Visual Odometry processing. Second, detected positions are used to estimate an error adjustment value. Finally, the accumulation errors are gradually rectified after the relocalization in a flight. Because the error correction with loop closure causes sudden behavior-changing of UAV, the error correction is processed step by step. Through the accuracy evaluation experiment of Visual Odometry, we confirmed the position estimation accuracy was less than 0.02 [m] in indoor environments. Moreover, we confirmed our methodology can achieve an accumulation error control to improve relocalization processing in Visual Odometry for stable flights of indoor UAVs.