Mikado Find is a complete package of camera and software for recognizing 3D objects.
1. Introduction
The point cloud from any Ensenso 3D camera is scanned against the CAD model. Position, orientation, and matching score of the identified parts are provided through the integrated OPC-UA interface. The intuitive calibration process allows for quick and easy setup at the facility.
2. Versions
This article is applicable for Mikado Find version > 1.1
horstFX version > 2021.07
Python version > 3.73
3. Setting up the System
For videos and documentation on setting up the system, please visit the following link: https://www.optonic.com/en/products/mikado/find/
4. Communication
Mikado Find communicates through an OPC-UA interface using a Python OPC-UA Client. The installation and operation details are further explained in the following documentation. OPC UA. To test the communication, it is recommended to download the OPC-UA GUI from https://github.com/FreeOpcUa/opcua-client-gui.
The Python script can be executed directly on the control cabinet. Details on how it is automatically called via horstFX are described in the section on Executing External Python Scripts.
5. Example Program
The following program serves as a comprehensive programming example. The Python script for communication via OPC UA is directly called from the horstFX program. The relevant commands for connecting to the camera, loading the camera configuration, and sending the trigger command are sent from horstFX to the Python script using TCP IP Socket. Subsequently, the Python script forwards this information to the camera via OPC UA. The object coordinates of the detected objects are then transmitted from the camera to the Python script, which sends this data back to horstFX using TCP IP.
In horstFX, a workspace control then takes place. This involves checking whether the robot is within a desired range when approaching a gripping point and not colliding, for example, with the cell wall. If the workspace is valid, the object is grasped and placed in a pallet. Once the pallet is full, the objects are randomly thrown back into the box for an endless loop.
5.1. Python Script
The Python script is designed in a way that typically does not require modification for other program sequences. The focus should be on ensuring synchronization between horstFX and this script.
#!/usr/bin/python3
import sys
import time
import socket
sys.path.insert(0, "..")
from opcua import Client
from opcua import ua
from numpy import uint32
# definition of function to execute commands via OPC UA
def executeCommand(cmd):
while outBusyBitNode.get_value(): # wait for busy bit = false
time.sleep(0.001)
inCommandNoNode.set_value(cmd, ua.VariantType.Int32) # set Trigger
while not outBusyBitNode.get_value(): # wait for busy bit = true
time.sleep(0.001)
inCommandNoNode.set_value(0, ua.VariantType.Int32) # reset trigger
while outBusyBitNode.get_value(): # wait for busy bit = false
time.sleep(0.001)
if __name__ == "__main__":
# Connection to Camera via OPC-UA
client = Client("opc.tcp://192.168.0.50:4840") # connect using an IP Adress
# Socket Connection
HOST = "0.0.0.0" # IP HORST
PORT = 3000 # Port HORST
try:
# connect to camera via OPC-UA
print("connect to camera")
client.connect()
# connect to horstFX va Socket
s = socket.socket() # Create a socket object.
s.bind((HOST, PORT)) # Bind the port.
print("socket listen for client")
s.listen(5) # Awaiting client connection.
c, addr = s.accept() # Establish a connection with the client.
print("socket accepted by client")
doStop = False
# get Objects node, this is where we should put our nodes
root = client.get_root_node()
# populating address space
outBusyBitNode = root.get_child(["0:Objects", "2:Outputs", "2:BusyBit"])
outResultNoNode = root.get_child(["0:Objects", "2:Outputs", "2:ResultNo"])
outResultsNode = root.get_child(["0:Objects", "2:Outputs", "2:Results.50"])
outErrorNoNode = root.get_child(["0:Objects", "2:Outputs", "2:ErrorNo"])
inCameraNameNode = root.get_child(["0:Objects", "2:Inputs", "2:CameraName"])
inRotationModeNode = root.get_child(["0:Objects", "2:Inputs", "2:RotationMode"])
inCommandNoNode = root.get_child(["0:Objects", "2:Inputs", "2:CommandNo"])
inConfigNameNode = root.get_child(["0:Objects", "2:Inputs", "2:ConfigName"])
inSTLNameNode = root.get_child(["0:Objects", "2:Inputs", "2:STLName"])
# open camera
print("open camera")
cameraName = "204929" # name of the camera
inCameraNameNode.set_value(cameraName) # set camaera name
executeCommand(6) # call execute function to execute the command
if outErrorNoNode.get_value() == 0:
print("connected to camera")
c.sendall("Connection to the camera established successfully.".encode('utf-8'))
else :
print("connection to camera could not be established")
c.sendall("Connection to the camera could not be established.".encode('utf-8'))
# When is the camera fully opened? Is there a command for this from the camera?
#time.sleep(8)
# load camera configuration
configName = c.recv(1024).decode() # Reads a message from HORST via a socket connection.
print("load configuration:", configName);
inConfigNameNode.set_value(configName) # set config file name
executeCommand(1) # call execute function to execute the command
if outErrorNoNode.get_value() == 0:
print("configuration has been loaded")
c.sendall("Configuration successfully loaded".encode('utf-8'))
else :
print("configuration could not be loaded")
c.sendall("Error loading the configuration occurred.".encode('utf-8'))
#time.sleep(5)
inRotationModeNode.set_value("sxyz") # set Rotation Mode
while (not doStop):
try:
msg = c.recv(1024) # Reads a message from HORST via a socket connection.
#************************************************************************************************trigger camera*******************************************************************************************
if msg == b'Trigger': # check message of HORST
executeCommand(4) # call execute function to execute the command
#******************************************************************************'***************get camera results****************************************************************************************'
# print("Error Number: ", outErrorNoNode.get_value())
if outErrorNoNode.get_value() == 0: # check camera for errors
print("Result Number: ", outResultNoNode.get_value())
if outResultNoNode.get_value() != 0: # check if camera has detected objects
#cam_values_str = str(outResultsNode.get_value())
cam_values = outResultsNode.get_value()
# sort the detected objects regarding the object score
cam_values.sort(key=lambda x: float(x[0]), reverse=True)
print("Camera Object Values:\n", cam_values)
cam_values_str = str(cam_values)
c.sendall(cam_values_str.encode('utf-8')) # send complete camera value message
#c.sendall(str(outResultsNode.get_value()[0][0]).encode('utf-8')) # send just one camera value
else:
c.sendall("Kein Objekt gefunden".encode('utf-8'))
else :
c.sendall("Fehler".encode('utf-8'))
if outErrorNoNode.get_value() == 1:
ErrorMsg = "Unknown system error"
elif outErrorNoNode.get_value() == 2:
ErrorMsg = "Unable to open the camera."
elif outErrorNoNode.get_value() == 3:
ErrorMsg = "No stereo camera connected"
elif outErrorNoNode.get_value() == 4:
ErrorMsg = "Camera Error"
elif outErrorNoNode.get_value() == 5:
ErrorMsg = "Search failed"
elif outErrorNoNode.get_value() == 6:
ErrorMsg = "Search did not find a corresponding model."
elif outErrorNoNode.get_value() == 7:
ErrorMsg = "Model generation failed."
elif outErrorNoNode.get_value() == 8:
ErrorMsg = "Setting the 3D reference point failed."
elif outErrorNoNode.get_value() == 9:
ErrorMsg = "Configuration could not be loaded"
elif outErrorNoNode.get_value() == 10:
ErrorMsg = "Configuration could not be saved"
elif outErrorNoNode.get_value() == 11:
ErrorMsg = "Command not implemented"
elif outErrorNoNode.get_value() == 12:
ErrorMsg = "Invalid index for retrieving the results"
elif outErrorNoNode.get_value() == 13:
ErrorMsg = "STL file not found"
c.sendall(ErrorMsg.encode('utf-8'))
else:
print("Program interrupted by HORST")
doStop = True
except KeyboardInterrupt:
print("Interrupted!")
doStop = True
finally:
client.disconnect()
c.close()
5.2. horstFX Script
From the horstFX example, some of the functions can be directly utilized for various applications. In some cases, simply adjusting the waypoints may suffice.