Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Info

Applicable to on-premise deployments

Purpose of this article

This article documents each backend component that requires an admin or integration user will have and how to update/change their password change process documented here. Please follow the steps in order to update/change a password accordingly.

Info

You are expected to have access directly to the Kubernetes or Openshift pods and have the ability to delete/create pods and deployments to progress this this.

This is also considered a DISRUPTIVE change and will cause outage on certain components during the update process.

1. Keycloak

1.1 Updating Postgres Password

Before updating the Postgres password at the secret level, you will need to first update the instance password.

This is because even if you update the secret, the database instance was already created prior to the secret update and hence would have created the relevant admin or integration users with the current secret.

Before undertaking this change, please ensure there is no one using the platform or if anyone is using the platform you will need to notify them of a disruption. Changing the password of the instance will cause a cascading effect and the following components will cease to function and go into a CrashLoop:

  1. keycloak

This may also cause a cascading effect that will cause the following to go into a pending state due to health checks:

  1. keycloak-internal-gateway

1.1.1. Changing the Password on the Existing Instance

Log into your Postgres pod and access the psql command line

Code Block
languagebash
### Native Kubernetes
kubectl exec -it <keycloak postgres pod> /bin/bash
bash
psql -U postgres -d postgres

### OpenShift
oc rsh <keycloak postgres pod>
bash
psql -U postgres -d postgres

Once you are in update the postgres user password to the new password using the following Postgres SQL command

Code Block
languagesql
\password postgres

It will prompt you for the new password. Once this is complete, proceed to the next section to validate the change.

1.1.2. Updating Deployment Specifications and Re-Deploy

Update the POSTGRES_PASSWORD variable in the secrets yaml for Postgres keycloak-credentials.yaml to the new password.

Apply the secrets change using

Code Block
### Native Kubernetes
kubectl apply -f keycloak/k8s/keycloak-credentials.yaml

### OpenShift
oc apply -f keycloak/k8s/keycloak-credentials.yaml

Validate the

...

Table of Contents

 secrets was updated using

Code Block
### Native Kubernetes
kubectl get secrets

### OpenShift
oc get secrets

Once the change has been applied you will need to restart all dependant pods on this secret in the next section.

1.1.3. Restart Dependant Pods

You will need to restart the following pods to take on the new Postgres password

  1. keycloak

These pods should no longer be in a CrashLoop state after the restart. The keycloak-internal-gateway should also stabilise.

1.2. Updating Admin Password

This can be done through the console via the keycloak Admin portal which be accessed via

Code Block
<DOMAIN_URL>/keycloak

The default password is as prescribed by what’s in the secrets configuration.

  1. Login as the Admin user for Keycloak and head to the top right hand corner to “Manage Account”.

  2. Head to the password section on the left hand side pannel.

  3. Simply update the password here.

  4. Once the password is updated make sure you update the keycloak-credentials.yaml secret file KEYCLOAK_PASSWORD to match and reapply it.

    Code Block
    ### Native Kubernetes
    kubectl apply -f keycloak-credentials.yaml
    
    ### OpenShift
    oc apply -f keycloak-credentials.yaml
  5. Restart the deployment pod only to ensure the password takes effect.

    Code Block
    ### Native Kubernetes
    kubectl delete pod <keycloak deployment pod>
    
    ### OpenShift
    oc delete pod <keycloak deployment pod>

1.3. Bulk load keycloak users

Use the loader in --create=true to either create users in keycloak (when using keycloak as a standalone user repository). NB to rerun using --create=true you will need to delete the users from keycloak.

Use the loader in --create=false when updating keycloak users that have been loaded from another source such as AD / LDAP integrations.

Code Block
cd /opt/server
python batch_load_keycloak.py --keycloak-mapping=/opt/sample/demo/KEYCLOAKMAPPING.csv --default-password=<default password> --create=true

1.3. Changing Client Keys for Service Authentication

  1. From https://DOMAIN_URI/keycloak/auth click the Administration Console and login.

  2. On the left hand panel go to Client

    Image Added

     

  3. Now click client starting with 0b3a... and head to Credentials, click the Regenerate Secret button and copy the Secret

    Image Added

     

  4. In cerebrum/k8s/cerebrum-auth-credentials.yaml

    1. Copy the secret value to OIDC_CLIENT_SECRET

    2. Set ENABLE_AUTH to 'true'

      Code Block
      apiVersion: v1
      kind: Secret
      metadata:
        name: cerebrum-auth-credentials
      type: Opaque
      stringData:
        KEYCLOAK_AUTH_URL: http://keycloak-internal-gateway-cluster-ip-service/keycloak/auth
        OIDC_OPENID_REALM: kada
        OIDC_CLIENT_SECRET: <COPY SECRET HERE>
        OIDC_CLIENT_ID: 0b3a...
        ENABLE_AUTH: 'true'

       

  5. Repeat the same key generation process [steps 2-4] for the client starting withf6b2....

  6. Copy the secret.

  7. In solr/k8s/solr-auth-credentials.yaml

    1. Copy the secret value to OIDC_CLIENT_SECRET.

    2. Update KEYCLOAK_AUTH_URL set <REPLACE WITH PROJECT NAMESPACE> to the kubernetes project namespace.

    3. Save.

      Code Block
      apiVersion: v1
      kind: Secret
      metadata:
        name: solr-auth-credentials
      type: Opaque
      stringData:
        KEYCLOAK_AUTH_URL: http://keycloak-internal-gateway-cluster-ip-service.<REPLACE WITH PROJECT NAMESPACE>.svc.cluster.local/keycloak/auth
        OIDC_OPENID_REALM: kada
        OIDC_CLIENT_SECRET: <COPY SECRET HERE>
        OIDC_CLIENT_ID: f6b2...

2. Postgres

2.1. Updating Postgres Password

Before updating the Postgres password at the secret level, you will need to first update the instance password.

This is because even if you update the secret, the database instance was already created prior to the secret update and hence would have created the relevant admin or integration users with the current secret.

Before undertaking this change, please ensure there is no one using the platform or if anyone is using the platform you will need to notify them of a disruption. Changing the password of the instance will cause a cascading effect and the following components will cease to function and go into a CrashLoop:

  1. cerebrum-api-server

  2. cerebrum-batch

  3. cerebrum-scheduler

  4. cerebrum-worker

2.1.1 Changing the Password on the Existing Instance

Log into your Postgres pod and access the psql command line

Code Block
languagebash
### Native Kubernetes
kubectl exec -it <postgres pod> /bin/bash
bash
psql -U postgres -d postgres

### OpenShift
oc rsh <postgres pod>
bash
psql -U postgres -d postgres

Once you are in update the postgres user password to the new password using the following Postgres SQL command

Code Block
languagesql
\password postgres

It will prompt you for the new password. Once this is complete, proceed to the next section to validate the change.

2.1.2. Updating Deployment Specifications and Re-Deploy

Update the POSTGRES_PASS variable in the secrets yaml for Postgres credentials.yaml to the new password.

Apply the secrets change using

Code Block
### Native Kubernetes
kubectl apply -f postgres/k8s/credentails.yaml

### OpenShift
oc apply -f postgres/k8s/credentails.yaml

Validate the secrets was updated using

Code Block
### Native Kubernetes
kubectl get secrets

### OpenShift
oc get secrets

Once the change has been applied you will need to restart all dependant pods on this secret in the next section.

2.1.3. Restart Dependant Pods

You will need to restart the following pods to take on the new Postgres password

  1. cerebrum-api-server

  2. cerebrum-batch

  3. cerebrum-scheduler

  4. cerebrum-worker

These pods should no longer be in a CrashLoop state after the restart.