Skip to content

Deploying and Upgrading LPC Environment

This section explains the process for upgrading and deploying the LPC system in a customer's environment. The goal is to provide an organized framework for performing an upgrade.

Prerequisites

The following requirements must be met before proceeding with the deployment:

  • Kubernetes Version: v1.26.13
  • Longhorn Version: v1.4.2
  • Operating System Version: Rocky Linux 8 / CentOS Linux 8 / RedHat Linux 8

Upgrade preparation

  • Prepare "Helm Charts" and "ArgoCD/Deploy" scripts for the customer environment.
  • Prepare "jumphost", VPN or network deployment admission conditions.
  • Backup of current configurations.
  • Validate system status and verify prerequisites.

Upgrading dependencies

Component Task Details
Kafka Upgrade Upgrade PVC name, download, and edit Kafka charts.
Upgrade Kafka version.
PGO for UTS Separate UTS database.
(No resources to perform this, ignore for now.)
PGO for IoT Separate read/write operations for IoT.
RabbitMQ Configuration Delete and unbind specified queues.

Restarting services

Restart necessary services that use Kafka, including the "Kafka connector" and "InsightsHub services".

Upgrading Insights Hub services

Upgrade "InsightsHub services" and ensure that services are running correctly and perform troubleshooting if needed.

Testing Post-Upgrade

Perform tests to ensure that the upgrade is successful, focus on system stability, data integrity and functional validation of Insights Hub.

Preparation and environment validation

  • Specify the required CPU, memory and disk space resources for the system.
  • Ensure that the Helm Charts and ArgoCD/Deploy scripts are correctly set up for the customer environment.
  • Check that ArgoCD auto-sync is disabled.
  • Check that there is sufficient memory and CPU to support the upgrade process. Example: PGO: + (16C/32G)*3
  • Verify that images are pre-pulled.

Environment validation

Perform the necessary validation steps, including checks for network connectivity, DNS configuration and time synchronization:

  • Verify network connectivity from Siemens site to the customer Kubernetes (k8s) API server.
  • Ensure customer site can access the InsightsHub image registry.
  • Validate time synchronization across all Kubernetes nodes.
  • Validate the running status of all Kubernetes (k8s) pods.
  • Check the InsightsHub UI status.

Backup instructions

This explains how to back up existing data and configurations before starting the deployment process.

  • Backup PGO data.
  • Backup CoreDNS configmap configuration.
  • Backup resolv.conf configuration.
  • Backup ingress controller configmap configuration.

Backend services upgrade details

Overview of required upgrades

Identify the backend services for InsightsHub that require upgrades, including their version changes.

Service Name Version Change Comments
bitnami/kafka 20.0.0 -> 28.2.5 Restart all services using Kafka.
uts/pgo pgo: 5.3.0
crunchy-postgres:ubi8-14.6-2
UTS database will be separated.
iot/pgo pgo: 5.3.0
crunchy-postgres:ubi8-14.6-2
IoT read/write operations are separated.

Kafka Upgrade

Upgrade bitnami Kafka from version 20.0.0 to 28.2.5.

PGO Upgrade

Node Expansion for PGO

Node Name Current CPU Current Memory Future CPU Future Memory
pgo-node1 16 32 32 64
pgo-node2 16 32 32 64
pgo-node3 16 32 32 64

Migrating UTS Database to new PGO cluster

  1. Set up a new PGO cluster (mdsp-pg-uts) on the same nodes as the existing mdsp-pg-all cluster.
  2. Deploy UTS services using the new UTS database endpoint.

    Note: The credentials for the new UTS PGO cluster are not yet updated, hence, the UTS services will not connect to the new cluster at this stage.

  3. Backup the UTS databases from the mdsp-pg-all PGO cluster, including the following databases:

    1. utsdb
    2. utsdatalakedb
    3. ruleenginedb
  4. Restore the backup databases into the mdsp-pg-uts PGO cluster.

    Note: The restoration time depends on the size of the databases.

  5. Update the postgres user password in the mdsp-pg-uts PGO cluster. After the password is updated, the UTS services can connect to the new cluster.

  6. Restart the UTS services to ensure the new database credentials are properly applied.

Infra secrets for new UTS/PGO

Secret Name Variable
mdsp-secret-infra-credentials uts_pg_sql_host
mdsp-secret-infra-credentials uts_pg_sql_password
mdsp-secret-infra-credentials uts_pg_sql_port
mdsp-secret-infra-credentials uts_pg_sql_user

Infra Secrets for new IoT/PGO

Secret Name Variable
mdsp-secret-infra-credentials iot_pg_sql_ro_user
mdsp-secret-infra-credentials iot_pg_sql_ro_host
mdsp-secret-infra-credentials iot_pg_sql_ro_password
mdsp-secret-infra-credentials iot_pg_sql_rw_user
mdsp-secret-infra-credentials iot_pg_sql_rw_host
mdsp-secret-infra-credentials iot_pg_sql_rw_password

Configuring updates

Service Name Configuration Updates
bitnami/rabbitmq Delete specific queues and unbind exchanges

Operations

The queue management and exchange unbinding components are detailed below:

Action Entity Name
Delete Queue asset-notifications-topic.iot-bulk-data-stream
asset-notifications-topic.iot-bulk-data-svc-asset-cache
notification-job-exchange.notification-job-queue
Unbind Exchanges conn -> provisioning-notification-topic.dpmservice
conn -> provisioning-notification-topic.emmservice

Deprecated configurations

Note

Identify any deprecated configurations that need to be removed, along with cleanup instructions.

Updating business services

This section explains the steps required to upgrade InsightsHub's business services. It includes Helm chart upgrades, rolling updates for workloads, pod restarts and applying new configurations.

  • Upgrade core services
  • Upgrade IOT services
  • Upgrade advanced services
  • Upgrade connection services
  • Upgrade other services
  • Validate and fix issues in affected environments (faw-vw,sunny-optical,maco…)

Verifying Post-Upgrade checklist

Provide a checklist to validate that all services are functioning correctly after the update:

  • Run the Docker suite.
  • Validate tenant login on the Insights Hub UI.
  • Review the Docker suite running result report.

Release Notes

Refer to the official documentation for detailed release notes: Insights Hub Release Notes.


Last update: January 31, 2025