Sinumerik Edge Sensor Adapter¶
Introduction and Goals¶
Sensor Adapter (SA) is a summary name for a system designed to enable the collection, transmission, and processing of data that can be collected by a production-related sensor in an industrial environment. We first aimed to support TCP / IP-based sensors connected to the EdgeBox network connector. Of these, we focus on the processing of images provided by the industrial IP Camera.
The goal is to create a generic SA that provides a unified API for sensor manufacturers to connect their sensors to the Sinumerik Edge environment. Another goal is to provide software developers with secure and efficient access to sensor data from machinelan for their data processing applications.
The sensors can send high-frequency (HF) data with a sampling frequency of up to 44.1 kHz and a depth of 16 bits. A typical example of this is the audio information collected by microphones. In contrast, sensors can send low-frequency (LF) 1-100 Hz data, which can be orders of magnitude larger data than high-frequency data. A good example of this is the high-resolution image information sent by cameras. The processing and transmission of HF and LF data based on these requires a slightly different approach, but the structure and main functions of the system are the same.
IP Camera technical requirements¶
- TCP/IP based communication
- RTSP data transfer
- H.264 codec
- Dual channel support (primary and secondary stream)
Definition of the Image_stream¶
The Image_stream is a series of preprocessed images (pictures) from the IP camera’s primary or secondary stream in raw (uncompressed) RGB format. A subscribed consumer will get the oldest available Image_stream picture from the buffer.
Resolution: Any of the IP camera’s standard resolutions (e.g. 1920*1080, 1280*720, …, 320*200)
Framerate: Any real number. (e.g. 25, 20, 10, 2, 0.5, 0.1, etc.)
Note: If the setting is not equal to the camera’s own framerate, the last available picture from the camera will be used.
Format: RGB (uncompressed)
Note: The SA’s CPU and RAM usage is depending on resolution and framerate of stream.
Definition of the Live_stream¶
The Live_stream is a series of preprocessed images (pictures) from the IP camera’s primary or secondary stream in (compressed) JPEG format. A subscribed consumer will get the next available package of JPG images.
Note: the default limit of the Live_stream’s buffer size is 300 frames.
Resolution: Any of the IP camera’s standard resolutions (e.g. 704*576, 640*480, 320*200)
Framerate: Any integer number. (e.g. 30, 25, 20, 10, 5, etc.) Note: If the setting is not equal to the camera’s own framerate, the last available picture from the camera will be used.
Format: JPEG
Note: The SA’s CPU and RAM usage is depending on resolution and framerate of the rtsp stream.
Installation¶
Select the application "sensoradapter" with the released version related to the firmware version installed on your SINUMERIK EDGE to install it on your connected SINUMERIK EDGE device. The current application is available in two different versions that differ in the resources available: - 1CPU: It is limited to use maximum 100% CPU - 2CPU: It is limited to use maximum 200% CPU
Note: the Sinumerik EdgeBox (Picobox (IPC127E), Nanobox (IPC227E) and Microbox(IPC427E)) contains 4 CPU cores (400%)
Please refer to chapter Application Management for detailed information how to install an application on your SINUMERIK EDGE device.
Configuration¶
Initial Configuration of the IP camera¶
1. Software config
Before connecting the IP camera to the EdgeBox with a LAN cable, it needs to be accurately preconfigured as follows:
- Connect it with a standard LAN cable (UTP Cat5e) to a PC (e.g. a simple Windows laptop)
- Open the camera’s built-in web page with a web browser. The default IP address (or site name), username and password can be found in the camera’s own documentation.
- Set up the desired IP address (MachineLan), change the default username and password.
- Set up the maximum necessary image resolution and fps values for the primary and secondary streams.
Note: do not use unnecessarily big values for the quality, it can cause unnecessary CPU load on the Edgebox.
- Set up the RTSP URL and port (if it can’t be changed, just note it down for later SA configuration).
2. Hardware config
- Connect the Camera with a standard LAN cable (UTP Cat5e) to the IPC (e.g. Edge NanoBox or MicroBox)
Configuration of Sensor Adapter¶
Before configuring the SA, please ensure that the IP Camera(s) configured to provide only the amount of images that is absolutely necessary, in minimum required resolution. For configuration, please use the Insights Hub configuration UI. You shall fill out the correct databus connection values, primary and secondary stream credentials & URLs, subscription parameters.
Scenario 1 - Connect to 1 camera using its primary stream (FullHD, 2 fps)¶
Sensoradapter metaconfig - Example #1¶
{
"databusConfig": {
"connectionstring": "rnd:databus.databus.indapp-net.industrialedge.io:8883",
"credentials": {
"password": "",
"username": ""
},
"permissions": []
},
"datasourceConfig": {
"providedDatasource": [
{
"datasourceId": "ExampleCamera",
"meta": {
"driver_parameters": {
"image": {
"rtsp_username": "[username]",
"rtsp_password": "[password]",
"rtsp_url": "[rtsp://]<camera_ip_address>[:rtsp_port][/path/to/image]"
}
}
},
"services": {
"subscription-service/v1": {
"subscriptions": [
{
"datapoints": [
{
"address": "image"
},
{
"address": "timestamp"
},
{
"address": "pan"
},
{
"address": "sensoradapter_subscription_message_count"
}
],
"merge": true,
"messageId": "ExampleCamera1",
"quality": "sensor_500ms"
}
]
}
},
"type": "camera"
}
]
},
"lifecycleConfig": {
"appStart": false
},
"machineConfig": {
"subscriptions": {}
},
"system_service": false
}
Sensoradapter metaconfig - Explanation #1¶
- Line 3: databusConfig\connectionstring: Port 8883 defines encrypted channel.
For non-encrypted connection use 1883.Note: Encrypted channel is enforced. - Line 13: datasourceConfig\providedDatasource\datasourceId: "ExampleCamera" is a user-defined ID for the camera the sensor driver will connect to. In order to connect to this camera instance, use the same on client side in requiredDatasource.
- Line 15: datasourceConfig\providedDatasource\driver_config: In order to read an rtsp stream from the IP camera, fill in image with “rtsp_url” and if necessary "rtsp_username" and "rtsp_password" fields in the given format, along with subscribing to “image” datapoint (line 29.). A true value from the configuration can look like this:
"driver_parameters": { "image": { "rtsp_username": "myuser", "rtsp_password": "myStr0ngPassword", "rtsp_url": "rtsp://192.168.1.100:554/channel_1" } }Note: As you'll see in later scenarios, in order to create a live stream, use “livestream” in the given format, along with subscribing to “livestream” datapoint. Both streams are optional and can function independently from each other. As a result, you can use either of the streams or both simultaneously. Preferably camera's secondary stream is used as a SA live_stream source, primary stream is used as a SA image_stream source.
- Line 25: datasourceConfig\providedDatasource\services\subscription-service/v1\subscriptions: Datapoints, quality and messageID attributes need to match in sensoradapter config and Consumer application metaconfig.
- Line 27: datasourceConfig\providedDatasource\services\subscription-service/v1\subscriptions\datapoints: In order to read the image stream, subscribe to “image” datapoint, along with filling in driver_config with image_rtsp_source.
Note: As in the later scenarios you'll see, in case you want to read the live stream, subscribe to the “livestream” datapoint, along with filling in driver_config with livestream_rtsp_source.
- Line 32: datasourceConfig\providedDatasource\services\subscription-service/v1\subscriptions\datapoints: timestamp value is for internal use only! As of version 0.2.0, this attribute represents a timestamp of the sensor payload.
- Line 35: datasourceConfig\providedDatasource\services\subscription-service/v1\subscriptions\datapoints: You can get some PTZ parameters from the camera like pan, tilt, and zoom. However, these are not implemented yet (as of version 0.4.1).
- Line 38: datasourceConfig\providedDatasource\services\subscription-service/v1\subscriptions\datapoints: sensoradapter_subscription_message_count represents the sequence number of the message related to the subscription.
- Line 43: datasourceConfig\providedDatasource\services\subscription-service/v1\subscriptions\quality: You can set the sampling frequency in the given format. Supports up to 5 messages per second (sensor_200ms). One message can contain 1 imagestream image, and a sequence of images for the livestream corresponding to the time elapsed since the previous message.
- Line 48: datasourceConfig\providedDatasource: type specifies the driver to be used in the sensor adapter for the given data-source. In this case, “camera” is a driver supporting IP cameras with h264 streams over RTSP.
- Line 53: lifecycleConfig\appStart: In case you want to manually start your application, set appStart to false. For automatic start, please set the value to true.
- Line 58: system_service: As Sensor Adapter is not a system application, this attribute needs to be “false”.
Consumer application metaconfig - Example #1¶
{
"databusConfig": {
"connectionstring": "rnd:databus.databus.indapp-net.industrialedge.io:8883",
"credentials": {
"password": "",
"username": ""
},
"permissions": []
},
"datasourceConfig": {
"requiredDatasource": [
{
"datasourceId": "ExampleCamera",
"meta": {
"driver_parameters": {}
},
"services": {
"subscription-service/v1": {
"subscriptions": [
{
"datapoints": [
{
"address": "image"
},
{
"address": "timestamp"
},
{
"address": "pan"
},
{
"address": "sensoradapter_subscription_message_count"
}
],
"messageId": "ExampleCamera1",
"messageName": "example_camera_image",
"quality": "sensor_500ms"
}
]
}
},
"type": "camera"
}
]
},
"machineConfig": {
"subscriptions": {}
},
"lifecycleConfig": {
"appStart": false
},
"system_service": false
}
Consumer application metaconfig - Explanation #1¶
- Line 13: datasourceConfig\requiredDatasource\datasourceId: Connect to the camera (ExampleCamera) defined in Sensoradapter metaconfig (providedDatasources).
- Line 19: datasourceConfig\requiredDatasource\services\subscription-service/v1\subscriptions: Datapoints, quality and messageID attributes need to match in Sensoradapter metaconfig and client config.
- Line 23: datasourceConfig\requiredDatasource\services\subscription-service/v1\subscriptions\datapoints: Subscribing to image datapoint, which is the primary stream.
- Line 36: datasourceConfig\requiredDatasource\services\subscription-service/v1\subscriptions\datapoints: While developing the source code of the application, value of messageName needs to be included in messageIdList when registering Consumer on Databus.
Scenario 2 - Connect to 1 camera using its primary (FullHD, 0.5 fps) and its secondary (VGA, 15 fps) stream¶
Sensoradapter metaconfig - Example #2¶
{
"databusConfig": {
"connectionstring": "rnd:databus.databus.indapp-net.industrialedge.io:8883",
"credentials": {
"password": "",
"username": ""
},
"permissions": []
},
"datasourceConfig": {
"providedDatasource": [
{
"datasourceId": "ExampleCamera",
"meta": {
"driver_parameters": {
"image": {
"rtsp_username": "[username]",
"rtsp_password": "[password]",
"rtsp_url": "[rtsp://]<camera_ip_address>[:rtsp_port][/path/to/image]"
},
"livestream": {
"rtsp_username": "[username]",
"rtsp_password": "[password]",
"rtsp_url": "[rtsp://]<camera_ip_address>[:rtsp_port][/path/to/livestream]"
"buffer_size": [max_frame_count]
}
}
},
"services": {
"subscription-service/v1": {
"subscriptions": [
{
"datapoints": [
{
"address": "image"
},
{
"address": "timestamp"
},
{
"address": "pan"
},
{
"address": "sensoradapter_subscription_message_count"
},
{
"address": "livestream"
}
],
"merge": true,
"messageId": "ExampleCamera1",
"quality": "sensor_2000ms"
}
]
}
},
"type": "camera"
}
]
},
"lifecycleConfig": {
"appStart": false
},
"machineConfig": {
"subscriptions": {}
},
"system_service": true
}
Sensoradapter metaconfig - Explanation #2¶
- Line 15: datasourceConfig\providedDatasource\meta\driver_parameters: In order to use image- and live streams at the same time, fill in driver_parameters according to the above demonstrated format. The buffer_size parameter defines the maximum number of images in the livestream buffer. buffer_size=-1 turns off the buffer limit. The default value is 300 (if the parameter is not set). A true value from the configuration can look like this:
"driver_parameters": { "image": { "rtsp_username": "myuser", "rtsp_password": "myStr0ngPassword", "rtsp_url": "rtsp://192.168.1.100:554/channel_1" }, "livestream": { "rtsp_username": "myuser", "rtsp_password": "myStr0ngPassword", "rtsp_url": "rtsp://192.168.1.100:554/channel_2" "buffer_size": 180 } } - Line 35: datasourceConfig\requiredDatasource\services\subscription-service/v1\subscriptions\datapoints: Don’t forget to subscribe to “image” datapoint, as you want to use image stream.
- Line 47: datasourceConfig\requiredDatasource\services\subscription-service/v1\subscriptions\datapoints: Don’t forget to subscribe to “livestream” datapoint, as you want to use live stream.
- Line 52: datasourceConfig\requiredDatasource\services\subscription-service/v1\subscriptions\quality: For example 0.5 messages per second set as sampling rate. A true value from the configuration can look like this:
Consumer application metaconfig - Example #2¶
{
"databusConfig": {
"connectionstring": "rnd:databus.databus.indapp-net.industrialedge.io:8883",
"credentials": {
"password": "",
"username": ""
},
"permissions": []
},
"datasourceConfig": {
"requiredDatasource": [
{
"datasourceId": "ExampleCamera",
"meta": {
"driver_parameters": {}
},
"services": {
"subscription-service/v1": {
"subscriptions": [
{
"datapoints": [
{
"address": "image"
},
{
"address": "timestamp"
},
{
"address": "pan"
},
{
"address": "sensoradapter_subscription_message_count"
},
{
"address": "livestream"
}
],
"messageId": "ExampleCamera1",
"messageName": "example_camera_image",
"quality": "sensor_2000ms"
}
]
}
},
"type": "camera"
}
]
},
"machineConfig": {
"subscriptions": {}
},
"lifecycleConfig": {
"appStart": false
},
"system_service": false
}
Scenario 3 - Connect to 2 cameras using:¶
- Camera 1 primary (FullHD, 0.5 fps) and secondary (VGA, 15 fps) streams
- Camera 2 secondary (VGA, 15 fps) stream
Sensoradapter metaconfig - Example #3¶
{
"databusConfig": {
"connectionstring": "rnd:databus.databus.indapp-net.industrialedge.io:8883",
"credentials": {
"password": "",
"username": ""
},
"permissions": []
},
"datasourceConfig": {
"providedDatasource": [
{
"datasourceId": "ExampleCamera",
"meta": {
"driver_parameters": {
"image": {
"rtsp_username": "[username]",
"rtsp_password": "[password]",
"rtsp_url": "[rtsp://]<camera_ip_address>[:rtsp_port][/path/to/image]"
},
"livestream": {
"rtsp_username": "[username]",
"rtsp_password": "[password]",
"rtsp_url": "[rtsp://]<camera_ip_address>[:rtsp_port][/path/to/livestream]"
"buffer_size": [max_frame_count]
}
}
},
"services": {
"subscription-service/v1": {
"subscriptions": [
{
"datapoints": [
{
"address": "image"
},
{
"address": "timestamp"
},
{
"address": "pan"
},
{
"address": "livestream"
},
{
"address": "sensoradapter_subscription_message_count"
}
],
"merge": true,
"messageId": "ExampleCamera1",
"quality": "sensor_2000ms"
}
]
}
},
"type": "camera"
},
{
"datasourceId": "OtherExampleCamera",
"meta": {
"driver_parameters": {
"livestream": {
"rtsp_username": "[username]",
"rtsp_password": "[password]",
"rtsp_url": "[rtsp://]<camera_ip_address>[:rtsp_port][/path/to/livestream]"
"buffer_size": [max_frame_count]
}
}
},
"services": {
"subscription-service/v1": {
"subscriptions": [
{
"datapoints": [
{
"address": "timestamp"
},
{
"address": "pan"
},
{
"address": "livestream"
},
{
"address": "sensoradapter_subscription_message_count"
}
],
"merge": true,
"messageId": "OtherExampleCamera1",
"quality": "sensor_2000ms"
}
]
}
},
"type": "camera"
}
]
},
"lifecycleConfig": {
"appStart": false
},
"machineConfig": {
"subscriptions" : {}
},
"system_service": true
}
Sensoradapter metaconfig - Explanation #3¶
- Line 13: datasourceConfig\providedDatasource\datasourceId: The first camera instance, called "ExampleCamera".
Note: In order to use both camera instances, you have to subscribe to both datasourceIDs (see: line 47.).
- Line 15: datasourceConfig\providedDatasource\meta\driver_parameters: A true value from the configuration can look like this:
"driver_parameters": { "image": { "rtsp_username": "myuser", "rtsp_password": "myStr0ngPassword", "rtsp_url": "rtsp://192.168.1.100:554/channel_1" }, "livestream": { "rtsp_username": "myuser", "rtsp_password": "myStr0ngPassword", "rtsp_url": "rtsp://192.168.1.100:554/channel_2" } }Note: The livestream_buffer_size parameter is missing, so it's value will be the default 300.
- Line 60: datasourceConfig\providedDatasource\datasourceId: The second camera instance, in this example called "OtherExampleCamera".
- Line 62: datasourceConfig\providedDatasource\meta\driver_parameters: Live stream can be used independently from image stream. A true value from the configuration can look like this:
"driver_parameters": { "livestream": { "rtsp_username": "myuser", "rtsp_password": "myStr0ngPassword", "rtsp_url": "rtsp://192.168.1.99:554/Streaming/Channels/102" "buffer_size": 180 } }Note: The IP address necessarily differs from the first camera instance (see: line 15.). The camera manufacturer can also differ, so the "/path/to/livestream" can be different.
- Line 83: datasourceConfig\providedDatasource\services\subscription-service/v1\subscriptions\datapoints: Datapoint for the livestream used as a source for consumer apps.
Consumer_1 application metaconfig - Example #3/1¶
{
"databusConfig": {
"connectionstring": "rnd:databus.indapp-net.industrialedge.io:8883",
"credentials": {
"password": "",
"username": ""
},
"permissions": []
},
"datasourceConfig": {
"requiredDatasource": [
{
"datasourceId": "ExampleCamera",
"meta": {
"driver_parameters": {}
},
"services": {
"subscription-service/v1": {
"subscriptions": [
{
"datapoints": [
{
"address": "image"
},
{
"address": "timestamp"
},
{
"address": "pan"
},
{
"address": "sensoradapter_subscription_message_count"
},
{
"address": "livestream"
}
],
"messageId": "ExampleCamera1",
"messageName": "example_camera_image",
"quality": "sensor_2000ms"
}
]
}
},
"type": "camera"
}
]
},
"machineConfig": {
"subscriptions" : {}
},
"lifecycleConfig": {
"appStart": false
},
"system_service": false
}
Consumer_2 application metaconfig - Example #3/2¶
{
"databusConfig": {
"connectionstring": "rnd:databus.indapp-net.industrialedge.io:8883",
"credentials": {
"password": "",
"username": ""
},
"permissions": []
},
"datasourceConfig": {
"requiredDatasource": [
{
"datasourceId": "OtherExampleCamera",
"meta": {
"driver_parameters": {}
},
"services": {
"subscription-service/v1": {
"subscriptions": [
{
"datapoints": [
{
"address": "timestamp"
},
{
"address": "pan"
},
{
"address": "livestream"
},
{
"address": "sensoradapter_subscription_message_count"
}
],
"merge": true,
"messageId": "OtherExampleCamera1",
"messageName": "other_example_camera_image",
"quality": "sensor_2000ms"
}
]
}
},
"type": "camera"
}
]
},
"machineConfig": {
"subscriptions": {}
},
"lifecycleConfig": {
"appStart": false
},
"system_service": false
}
Non-functional requirements (NFRs)¶
Sensor Adapter’s performance is measured by its latency on the databus’s consumer (customer’s application) side. The sensoradapter indapp has two versions that differ only in the amount of resources that can be used. The SA 1CPU version limited to 100% of CPU, the SA 2CPU version can use 200% CPU (a max CPU value is 400% in a box with 4 CPU cores). RAM limit is 512 MB in both cases. The image processing performance is highly dependent on:
- the hardware configuration (nanobox, picobox or microbox is used)
- the camera configuration (primary & secondary RTSP stream’s resolution and fps values)
- The Sensor Adapter’s configuration (quality, buffer size)
- The one- or two-CPU version of SA is used
> *__Example:__ if your application will use 800x600 resolution images in every 10 seconds, then **do not use** higher resolution (for example wqhd (2560 x 1440)) and higher framerate (for example 60 fps) settings on the camera side, because it will consume unnecessary CPU-time on Edgebox. Try to set-up the camera to minimal data output regarding to your actual usecase (800x600 pixel @ 0.1 fps).*
Any questions left?
Except where otherwise noted, content on this site is licensed under the The Siemens Inner Source License - 1.1.