10.3 Vertica

The Vertica Cluster Might be Down with Insufficient Disk Space

Explanation: The Vertica cluster might be down because of insufficient free space in the Vertica data disk. For more information, check the Vertica log files available in the following location:

/vastorage/home/dbadmin/zenworks/v_zenworks_node0001_catalog/vertica.log

Log Message: Init Session:xxxxxxxxx <ERROR> @v_zenworks_node0001: xxxx/xxxx: Could not write to [/vastorage/home/dbadmin/zenworks/v_zenworks_node0001_data/xxxxxxxxxxx]: Volume [/vastorage/home/dbadmin/zenworks/v_zenworks_node0001_data] has insufficient space.

Action: Ensure that you have enough free disk space in Vertica nodes. For more information, see ZENworks Vertica system requirements.

Vertica recommends that the disk utilization per node be no more than sixty percent (60%). For more information, see Vertica Disk Space Requirements.

Free disk space is required to run system activities like merge and purge operations and dashboard queries. The amount of disk space needed depends on the queries, the amount of data on the node, and the number of concurrent users on the system.

Shutting Down One Node in a Muti-Node Vertica Cluster Does Not Stop the Entire Cluster

Explanation: In a multi-node cluster, unable to connect to Vertica after restarting one of the nodes (without shutting down Vertica), and Vertica services are running on the other nodes but not accepting connections.

Action:

Perform the following steps:

1. From a working Vertica node, run: /opt/novell/vertica/vertica-controller.sh do-maintenance.
2. After the above command completes, run the following command from the node that got rebooted:

/opt/novell/vertica/vertica-controller.sh start-node

Both of these commands might take time to complete.

For more information, see Stopping and Starting Vertica Services

After partially disabling Vertica in the zone, upgrade to the next ZENworks release version fails

Explanation: Consider a scenario, where Vertica is configured on a Primary Server, and then needs to be disabled and removed from the zone. While disabling Vertica, a few errors occurred, due to which Vertica was only partially disabled. Subsequently, upgrade to the next ZENworks release version fails.

Action: Before proceeding with the Upgrade process, ensure that you have fully configured or fully disabled Vertica. While disabling Vertica in your zone, ensure that all the steps documented in Disabling and Removing Vertica from the Zone are followed and no errors occur while executing the commands.

However, if you are still unable to disable Vertica completely from the zone, then run the following query:

delete from preglobalupgradelog where schematype='vertica'

Exception logged in the loader messages while syncing data from ZENworks to the Vertica database

Source: Vertica, ZENworks

Explanation: The following exceptions observed in loader-messages.log while syncing data from ZENWorks to Vertica database.

Failed to create rejected files  : java.io.IOException: No such file or directory
  at java.io.UnixFileSystem.createFileExclusively(Native Method)
  at java.io.File.createNewFile(File.java:1023)
  at com.novell.zenworks.commons.VerticaCopyStreamImpl.createErrorHandlingFiles(VerticaCopyStreamImpl.java:75)
  at com.novell.zenworks.commons.VerticaCopyStreamImpl.<init>(VerticaCopyStreamImpl.java:54)
  at com.novell.zenworks.streamprocessors.DimensionProcessor.getCopyStream(DimensionProcessor.java:108)
  at com.novell.zenworks.streamprocessors.BaseProcessor.process(BaseProcessor.java:132)
  at com.novell.zenworks.kafka.consumers.ZenKafkaConsumer.processBatch(ZenKafkaConsumer.java:231)
  at com.novell.zenworks.kafka.consumers.ZenKafkaConsumer.run(ZenKafkaConsumer.java:171)
  at java.lang.Thread.run(Thread.java:750)

Possible Cause: This issue can occur if Vertica was enabled in the zone on ZENWorks 2020 Update 2 or earlier versions, and if var/opt/microfocus/log/zenworks/vertica is a symbolic link to var/opt/novell/log/zenworks/vertica inside the ZENLoader container.

Action:

Perform the following steps:

Access the ZENLoader container by running the following command:

Docker exec -it zenloader bask
Navigate to the /var/opt/microfocus/log/zenworks directory.
Remove the Vertica directory by running the following command:

rm -r vertica
Exit the command prompt.
Restart loader service.

A node in the Vertica cluster shuts down

Explanation: To provide fault tolerance, Vertica can be run as a cluster of nodes. However, at times the nodes in the cluster can abruptly shut down due to issues related to network connectivity and so on.

Action: Follow the steps documented in the Vertica documentation, to bring up the database when one of the nodes shuts down.

When nodes responsible for bringing a Vertica cluster down are restarted, then the remaining nodes are no longer a part of the cluster

Explanation: The K-factor specified while configuring Vertica, sets the fault tolerance for the Vertica cluster, where K can be set as 0 or 1. The Vertica cluster will be down, if more than ‘K’ number of nodes are down. For example, if the K factor is 1 in a 3 node cluster and 2 nodes are down, then the Vertica cluster will be down. When you restart services on these 2 nodes, the third node (that was up and running) will no longer be a part of the cluster.

Action: The command to restart the Vertica servers should be run for all the servers in the Vertica cluster. For more information on how to restart the servers in the Vertica cluster, see the Vertica documentation.