The project aims at realizing a network of laboratories spread on the national territory (including in its first prototype implementation the labs of the Research Units) each of which providing the possibility of accessing experiments that can be remotely conducted thanks to a suitable application based on extended reality and running on mobile devices, as shown in Fig.1.
According to the block diagram presented in Fig.2, the framework will be expanded by defining and implementing innovative techniques and strategies for both instrument remote control and information sharing with the users.
The first step is the definition, design and implementation of new methods for 3D image scanning and reconstruction of the instrument. To this aim, a reverse engineering approach will be considered, by properly integrating acquisition techniques and systems (e.g., 3D scanners or advanced devices) with 3D image analysis methods.
3D models of the instruments will be then imported and rendered in the view/control interface; to this aim, the best strategies will be identified, defined and implemented. To ensure the largest diffusion, use and subsequent extension of the framework, both the display and the code that implements the operations associated with user actions will be carried out in Unity environment, one of the most used environments for the creation of augmented reality applications.
To broaden the scope of measurement devices and instruments that can be introduced into the remote lab framework, we plan to define, design and implement an appropriate control proxy capable of acting as an interpreter between the messages sent by the vision/control interface and the protocols typically used for communication with the instruments (as an example, GPIB, LXI, USB or Serial). The control proxy would be implemented by means of either a low-cost hardware interface or a gateway device suitably defined to the purpose. To ensure scalability, reliability and security, messages from the users’ device will be transmitted and received according to the Message Queue Telemetry Transport (MQTT) protocol, thus making any instrument an accessible resource through the IoT paradigm, notwithstanding the verification of the transmission security conditions.
Also the block of the physical experiments/instruments will be expanded by defining, designing and implementing proper solutions to carry out measurements on different circuits without changing the topology of the measurement station. This will be made possible by introducing appropriate reconfigurable devices capable of emulating various electrical and electronic circuits. Extensive research will be conducted to make such devices not only remotely controllable but, more importantly, include the definition of their parameters in the augmented reality environment.
A behavior observer block will be designed to continuously observe the state of the physical system, such to reproduce this state in the augmented reality environment. In the case of a simple bench static instrument, the behavior could be represented by its screen, which can be captured and played in the augmented reality environment. In some cases, the behavior can involve mechanical parameters. The physical system can be in this case a robot, where its state is represented by the angular positions of its joints. Such a state will be captured by the behavior observer block and given, with a proper data representation to the augmented reality viewer which will proceed to the modification of the 3D model, to make it appearing according to the new behavior.
The behavior observer, together with the control proxy will thus allow to implement a complete digital twin of the physical system.
Besides the main scientific objectives presented so far, further aspects will be targeted thanks to the executed research activities of the three Units. In particular, a network of laboratories in the Universities of Benevento, Cosenza and Naples will be created by sharing resources and knowledge through an integrated informative system that will allow the access to the virtual instruments through AR and to the description of the experiences specifically designed for the AR didactic.
The laboratory network will be accessed by means of a mobile application, which will be specifically developed within the project and will present and make available all the available equipment and didactical experiences.
New didactical material, specifically designed for the new AR didactic paradigm, will be developed. It will consist of video lessons, report examples, didactic agreement including the objectives and meta-objectives of the AR experience.
From a metrological point of view, a fundamental issue will be the investigation of the delay introduced by the hardware solutions designed to transform traditional measurement instruments into IoT measurement instruments. The delay can, in fact, affect the final AR experience of the students and its characterization will allow the design of suitably strategies at hardware and software levels to reduce it to acceptable values.
An application package and guidelines will be prepared and made freely available to allow other laboratories and institutions to connect their equipment and experiences to the network. Moreover, all the codes developed within the project will be made available on a software repository.
A further objective of the proposed project strategy is sustainability, which declined in terms of both reducing e-waste generation and improving the green footprint of measurement execution. Regarding the first point, the project plans to connect even very old instruments in a typical internet of things network, reusing them and avoiding their disposal due to obsolescence. Regarding the second point, it is easy to demonstrate that the greatest production of pollution in the execution of measurements is associated with the travel required to reach the laboratory location; allowing remote access eliminates this source of pollution. Last but no least the inclusive aspects by opening the STEM disciplines to people that due to the work or limited resource cannot access to Laboratories. This would boost also the Long Life Learning of all STEM workers.
