The BugWright2 project
Autonomous Robotic Inspection and Maintenance on Ship Hulls
New integration week in Greece
[Integration Week]
👷 For our third in-person Integration week, partners went back to Athens, Greece, in November 2022. This time partners focused on technical development with the mock up (Perama shypyard), collect data photos and videos for the Review meeting (RP2) at the end of November 2022.
The third integration week of the BugWright2 project was dedicated to the network and localization integration of the robotic and human agents involved in the hull monitoring task. This includes the three kinds of robots: magnetic crawlers, underwater ROVs, flying drones and human operators equipped with Augmented Reality gear (headset or tablet for instance). All these agents need to communicate their data in a common network and report their observation in a consistent localization. At this stage, this will be addressed by linking the inspected structure frame to the GPS UTM coordinate frame through AR tags observed by the drones. The integration week was also an opportunity to test long-range wireless teleoperation over the industrial test site, to define the mission specifications and to evaluate the prototype user interface.
Similar to our other in-person Integration week, partners had their own objectives. Here’s what we did during the week:
The CNRS team coordinated these activities and prepared the magnetic crawler for long-range wifi teleoperation. It also assisted the integration of the localization systems by preparing a frame alignment software that would link the 3D map in GPS UTM frames produced by the drones with the local frame of the inspected structure. In addition, a detailed 3D scan of the mock-up was acquired with our total station.
NTNU and BEYE, working on the Pioneer, had the main objective of performing autonomously an inspection of a section of a real ship using the X3 ROV. This would allow a visual documentation of the hull enabling the operator/inspector to have a look without doing anything. We started to test in the mock-up to make sure everything was working and to collect data simulating an inspection inside a ballast tank. We quickly moved on tests in the sea, to first, assess the navigation autonomy of the ROV without obstacles around – we had to use more time than expected on that because of minor issues related to the depth control. After this, we performed manual inspections to have an idea of how the data looks like in such scenario. We could at the same time test the acoustics when close to the hull because the X3 is equipped with a sonar which is the main sensor for the navigation along the hull. Once the data collected and analysed, we performed the mission autonomously and it was a great success. We were able to repeat multiple times an autonomous inspection of a ~30x5meters section of a ship with clear visual documentation. This proves the feasibility of the proposed setup and allows to move forward in the tests. During the post-processing of the data, we could also apply deep learning tools to automatically detect the ship parts and faults on the hull based on the collected data.
For this integration week UPORTO was aiming at testing forwarding the data collected with the X3 to the VR interface and doing some water testing near a ship. Due to DVL breaking down on the drone UPORTO was limited on what to test, but nevertheless with cooperation with NTNU we did test some navigation code. UPORTO was also able to sit down face to face and converge on a common planning interface for all the drones, allowing a common connection with the VR C2.
Being only the third integration week done in person due to the pandemic, was a great event to converge on ideas and way forward. – Paulo Dias, senior researcher & project manager (UPORTO)
For the team of UNI-KLU, the objective was to enable the UAV to acquire initialization data for localization anchors used in the navigation tasks of the other robotic platforms. For this specific integration week, the focus was on the initialization of UWB anchors and AR markers on the mock-up ship hull. The initialized positions should sever then for coordinate frame alignment between the different platform types as well as the VR reconstruction. The multi-sensor fusion framework MaRS is used to provide localization information for the UAV using any available sensing modality (GNSS, vision, etc). The challenges during the week were the windy and very confined spaces such that GNSS signals were highly distorted. Manual flights and post-processed visual localization was nevertheless possible. The initialization routine for the UWB anchors and AR markers was tested and the relatively large initialization errors further encouraged to modularly include such anchors in MaRS after initialization for continuous position refinement. The successfully gathered data will now further be analyzed to focus on MaRS and initialization routine improvements.
RWTH tested the previously developed user interface with the robotic partners. RWTH achieved a simultaneous connection to all the robotic platforms. Nathalie, Jan and Thomas (from Trier University) got new input from the partners for further user interface iterations. A mission planning protocol with all partners was established. Additionally, we worked on the tracking marker recognition for the AR glasses to align the virtual and real world. In the end this will allow RWTH to overlay the real world with digital information on the correct places of the hull.
Regarding Wi-Fi, INSA conducted performance and range testing for the robotic wireless network. We were interested to see how reliably we can control a robot wirelessly in an industrial environment that challenges radio communications with large metallic structures and interference from industrial equipment. We controlled the magnetic crawler over wireless network, and we were able to improve the network performance that allows wire-like robot performance. The link was tested up to 113 meters range in line of sight. This is the first step to test the full integration of the robots for autonomous inspection.
During the integration week, Trier University (UT) worked on Human Factors in autonomous ship inspections. UT presented relevant user interface and HRM needs and concepts. This helped to disseminate the current status of development to the consortium and to collect further input from experts and end users. Discussions on general and specific parts of the hybrid interface (desktop + VR) delivered valuable information for further user interface design and evaluation. Moreover, UT evaluated predictors of successful human-autonomy teaming and investigated HRM needs for implementation.
On the legal side: WMU wanted to get a deep insight into the technical challenges encountered with magnetic crawlers, ROVs and drones at the current stage of technology; to collect data about the hazards associated with RITs before, during, and after the ship’s inspection. No challenges encountered and data collected through interviews with the BUGWRIGHT2 consortium members. As a result, a joint scholarly publication is under development with RINA.
A new colleague from RINA got familiarised with the partners of the consortium and attained a better understanding of the work of each partner and the progress achieved during the last months; supported WMU in developing and performing a survey regarding the main hazards of the platforms considered in the project with the aim of developing a framework that will be disseminated through a journal article; identified areas where that RINA could support the other partners, such as provide the checklists used for remote survey, provide feedback on the VR platform in order to adopt the technical terminology used from the surveyors; discussed potential involvement of RINA surveyors for the evaluation of the platforms. The main methodological steps for T9.4, which RINA is leading were discussed and defined with the WP leader. This was very important in order to start organising the work that needs to be done in the upcoming months.
The Consortium would like to thanks Glafcos Marine Ltd, DANAOS and Starbulk for their welcome, support and help during the week.
#h2020 #euproject #integrationweek #fieldtests #research #euproject #dronetechnology #innovation #testing #data #robots #augmentedreality
Demonstration of adding wireless connectivity for Magnetic Crawler that performs LiDAR scan and ultrasound sensing on large metallic structures such as ship hulls and containers. The test was performed in Danaos shipyard in Piraeus, Greece
