6.1 Managing CIS

Cloud Integrated Storage (CIS) management console allows you to move the NSS data to cloud storage. It also allows you to define policies for data migration and view statistic information on migration, files migrated and recalled.

IMPORTANT:The CIS management login page (https://<OES server IP address or the host name>:8344) works only if CIS is configured.

Prerequisites

Before you start using the CIS management console, ensure that the following prerequisites are met:

  • Supported web browsers:

    • Mozilla Firefox

    • Google Chrome

    • Internet Explorer

    • Microsoft Edge

    • Apple Safari

  • Ensure that a cloud account is created and you have details of the access key and secret key.

  • If you use cloud account with SSL support and create certificate signed by different CA (other than eDirectory CA), copy the CA bundle (.pem format) in /etc/opt/novell/cis/certs/rootCAs and add the CA bundle file name in CLOUD_CA_BUNDLE_NAME in the /etc/opt/novell/cis/config file.

    For example:

    CLOUD_CA_BUNDLE_NAME="Cloud1-CA.pem"

    NOTE:If you have multiple cloud accounts with SSL support, concatenate CAs (.pem format) of the cloud accounts and add the concatenated CA bundle name in CLOUD_CA_BUNDLE_NAME in the /etc/opt/novell/cis/config file.

    After configuring the /etc/opt/novell/cis/config file, ensure to restart the data service using the following command:

    systemctl restart oes-cis-data.service

    In data scale scenario, repeat the same steps on all the servers where data service is running. The config file path in data scale is /etc/opt/novell/cis-scale.

    After configuring the /etc/opt/novell/cis-scale/config file, ensure to restart the data scale service using the following command:

    systemctl restart oes-cis-dataatscale.service

    NOTE:This prerequisite is not applicable in AWS S3, as secure communication is taken care without the CLOUD_CA_BUNDLE_NAME parameter.

  • Ensure that all the CIS services and its infrastructure services are up and running.

  • Verify whether the CIS agents are up and running. The CIS agents includes the following:

    oes-cis-agent.service

    oes-cis-recall-agent.service

    oes-cis-scanner.service

    For example,

    blr7-user1:/lib/modules/4.4.21-69-default/kernel # systemctl status oes-cis-agent.service
● oes-cis-agent.service - CIS agent for OES
   Loaded: loaded (/usr/lib/systemd/system/oes-cis-agent.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2017-06-14 19:01:09 IST; 1 day 23h ago
 Main PID: 17177 (cisagents)
    Tasks: 107 (limit: 512)
   CGroup: /system.slice/oes-cis-agent.service
           └─17177 /opt/novell/cis/bin/cisagents

Jun 16 10:53:43 blr7-user1 cisagents[17177]: Entry name = /media/nss/TEST1/DESKTOP.AFP/ICON/736F646D5458455
Jun 16 10:53:43 blr7-user1 cisagents[17177]: Entry name = /media/nss/TEST1/DESKTOP.AFP/ICON/736F646D666C726
Jun 16 10:53:43 blr7-user1 cisagents[17177]: Entry name = /media/nss/TEST1/~DFSINFO.8-P
Jun 16 11:07:26 blr7-user1 cisagents[17177]: 2017/06/16 11:07:26 Number of Components: 3
Jun 16 11:07:26 blr7-user1 cisagents[17177]: Entry name = /media/nss/TEST1/BB/krb5.conf
Jun 16 11:07:26 blr7-user1 cisagents[17177]: Entry name = /media/nss/TEST1/DESKTOP.AFP/ICON/736F646D414E494
Jun 16 11:07:26 blr7-user1 cisagents[17177]: Entry name = /media/nss/TEST1/DESKTOP.AFP/ICON/736F646D4C50504
Jun 16 11:07:26 blr7-user1 cisagents[17177]: Entry name = /media/nss/TEST1/DESKTOP.AFP/ICON/736F646D5458455
Jun 16 11:07:26 blr7-user1 cisagents[17177]: Entry name = /media/nss/TEST1/DESKTOP.AFP/ICON/736F646D666C726
Jun 16 11:07:26 blr7-user1 cisagents[17177]: Entry name = /media/nss/TEST1/~DFSINFO.8-P

    Similarly, verify the status of other two CIS agents.

    If CIS agents are not running, restart all 3 agents by entering the following command:

    systemctl start oes-cis-agent.service

    systemctl start oes-cis-recall-agent.service

    systemctl start oes-cis-scanner.service

Data Migration Using CIS Management Console

  1. Configure the cloud account. For more information, see Section 6.1.2, Accounts.

  2. Create a policy. For more information, see Section 6.1.3, Policies.

  3. Configure the tier. For more information, see Section 6.1.4, Tiers.

  4. View the status of data migration. For more information, see Section 6.1.5, Dashboard.

6.1.1 Insights

The CIS welcome page provides the network level view of total number of volumes available on the OES servers that is configured with CIS, total number of files available, and total space used by those files. Based on the access time or modification time, the Data Summary section displays the percentage of hot and cold data available on all the volumes and lets you do the following:

  • View the percentage of hot and cold data (based on access time or modification time) by moving the data slider.

  • Set the age of the hot and cold data (based on access time and modification time) by moving the time slider.

Click to view the updated scan details of all the CIS agents. You can use this page to discover the top five volumes that contain more cold data, the top five users with more cold data, and the top five file types available as part of that cold data. Overall, this page provides the user with the insights of data available in their organization and use this information to create policies.

Click any volume in Top Cold Volumes or the HOT/COLD chart to view the volume specific summary. Click on volumes count at the top to go to volume summary information page.

6.1.2 Accounts

Configure and manage the cloud account to which you are planning to migrate the data.

Configuring Cloud Account

Click the Accounts tab, the Cloud Accounts page is displayed.

  1. Click to configure the cloud account.

  2. Specify the following:

    1. Account Name: Specify the cloud account name.

    2. Account Type: Select one of the following:

      • AWS S3

      • S3 Compatible

        Endpoint: This parameter is displayed only if you select the S3 Compatible. Specify the URL of the cloud server.

    3. Region: Select the region name where the cloud server is available.

    4. Access Key: Specify the access key of the cloud account.

    5. Secret Key: Specify the secret key of the cloud account.

  3. Click SAVE.

Managing Cloud Accounts

Click the Accounts tab to view all the configured cloud accounts.

  • To modify the cloud account information:

    1. Click on the cloud account you want to modify.

    2. Click UPDATE.

  • To delete the cloud account:

    1. Click on the cloud account you want to delete.

      NOTE:You cannot delete the account on which the data migration is already performed.

    2. Click OK.

6.1.3 Policies

The data is migrated using the policies that are created based on the last accessed time, modified time, file type, file size, and so on.

Creating a Policy

Click the Policies tab, the Policies page is displayed.

  1. Click to create a policy.

  2. Specify the name and description for the policy.

  3. Select the required rule.

    The following are sample use cases:

    • Use Case 1: To migrate data that is not accessed for more than 120 days.

      Rules

      Value

      Unit

      Operation

      Files not accessed since

      120

      days

      End

      OR

      Rules

      Value

      Unit

      Operation

      Files not accessed since

      4

      months

      End

    • Use Case 2: To migrate data that is not modified for the last two years.

      Rules

      Value

      Unit

      Operation

      Files not modified since

      2

      years

      End

    • Use Case 3: To migrate all the files with .doc and .pdf extension.

      Rules

      Value

      Operation

      File name matches

      *.doc,*.pdf

      End

    • Use Case 4: Consider you have six files with names data, user1_data, 2017_data, 16-may_data, may-17_data, and data32. To migrate files with file names containing a matching string.

      Rules

      Value

      Operation

      File name matches

      *data*

      End

      After running this policy, all the six files are successfully migrated.

    • Use Case 5: To migrate all the files that are less than 1 MB in size.

      Rules

      Value

      Unit

      Operation

      File size is less than

      1

      MB

      End

    • Use Case 6: To migrate all the files and folders that are available in a specific file path.

      Rules

      Value

      Operation

      File path contains

      /data1/secret

      End

      NOTE:Ensure that the path provided is from the root of the volume.

  4. (Optional) Create multiple rules for the same policy using the following:

    • And: Performs And operation between the selected and next rule.

    • Or: Performs Or operation between the selected and next rule.

    • New group: Adds a new group and use And or Or option between these two groups. You can create any number of groups and add multiple rules for the same group.

    • Delete: Deletes the selected rule.

    • End: Deletes all the rules that follows the selected rule.

    For example, to migrate all the data except PDF files larger than 10 MB and less than 6 months old:

    Rule

    Value

    Unit

    Operation

    File not accessed since

    6

    months

    And

    File size is less than

    10

    MB

    And

    File name does not match

    *.pdf

    -

    End

  5. Click SAVE.

Managing Policies

Click the Policies tab to view all the policies.

  • To modify the policy:

    1. Click policy name or on the policy you want to modify.

    2. Click UPDATE.

  • To delete the policy:

    1. Click on the policy you want to delete.

      NOTE:You cannot delete the policy that you have used to migrate data.

    2. Click OK.

6.1.4 Tiers

It includes:

Tiers

By configuring the cloud tier, you are associating the primary storage (data on OES server) and cloud storage to perform the data migration. Using tier configuration, you can run the policy at a scheduled time.

Configuring Cloud Tiers

Click the Tiers tab, then click Tiers.

  1. Click to configure the tier.

  2. Specify the following:

    1. Server: Select the OES server. This lists the OES servers that are configured with CIS server where agent is running and also includes the cluster resources.

    2. Volume: Select a volume.

    3. Endpoint: Select the required cloud account name.

    4. Bucket Name: Specify the bucket name used to store the migrated data. The bucket name can be obtained from your cloud account.

    5. Region: Select the region where the specified bucket name is available.

    6. Encryption: To enable this parameter, configure the encryption settings. If enabled, encrypts the migrated data in the cloud storage.

      For more information on encryption settings, see Encryption.

    7. Policy: Select the required policy to be applied.

    8. Schedule: Select the schedule type based on how frequently the policy should be run for the tier. It includes the following: Daily, Weekly, Monthly, Once, and None.

      As a best practice, you can limit the duration of run to four hours everyday or run during the weekend to minimize the load on the OES servers. It is recommended that you do not migrate the data when users are accessing the data.

      To limit the duration of the schedule run, select Time duration for the schedule run option and specify the time duration.

  3. Click SAVE.

NOTE:The secondary volume or CBV (Cloud Backed Volume) is automatically created after the tier configuration.

Managing Cloud Tiers

Click the Tiers tab to view all the cloud tiers. Before to estimate the total migrate or recall data before the actual run.

  • If the schedule type is None, click to start the data migration.

  • To stop the data migration, click the rotating icon .

    The next time, you start the data migration process for the same tier, the remaining data is migrated to the cloud storage.

  • To rerun the policy for the tier, click .

  • To view statistics for the tier:

    1. Click .

      Displays information of the migrated files from the previous job run (each time the schedule starts at a specified time, a new job run is created). The statistic information includes:

      • Status: Provides the status of data migration.

      • Start Time: Provides the data migration start time.

      • End Time: Provides the data migration completion time.

      • Files Migrated: The total number of files migrated.

      • Data Size: Lists the overall size of the data migrated for each volume.

      For more information on detailed reports, see Dashboard.

    2. If no files are migrated, click Last Run to view the statistic information for the previous job run.

  • To modify the tier:

    1. Click on the tier you want to modify.

    2. Select the policy type for tier run. It includes:

      • Migration Policy: Migrates the data (that satisfies the policy selected) to the cloud.

      • Recall Policy: Recalls the data (that satisfies the policy selected) from the cloud.

      • Free Space Calculation: Calculates (dry run) the amount of data that will be migratedto the cloud. Before performing the migration, ensure that enough space is available on the cloud.

      • Recall Space Estimation: Calculates (dry run) the amount of data that will be recalledfrom the cloud to your Primary volume. Before performing the recall, ensure that enough space is available on your Primary volume.

    3. Click UPDATE.

  • To delete the tier:

    1. Click on the tier you want to delete.

      If data migration is not performed on this tier, click OK to delete.

      If data migration is performed on this tier, select the desired action.

      • Recall files and delete: This recalls all files (migrated as part of this tier) from the cloud and automatically deletes the tier.

      • Force delete: This deletes the tier and CBV volume associated with this tier without recalling files. The data (migrated as part of this tier) is lost and cannot be recovered.

    2. If you select the Recall files and delete action, choose the schedule to recall files and then click Recall.

      OR

      If you select the Force delete action, click Delete.

Migrate from DST

Based on the DST migrate tier details, the data is migrated from the DST shadow volume to the cloud storage. By default, an internal policy is applied to migrate all files in the DST shadow volume.

Configuring DST Migrate Tier

Click the Tiers tab, then click Migrate from DST.

NOTE:Before configuring the DST tier, ensure to disable the DST policies. For more information, see Section 7.0, Migrating DST Volumes to Cloud.

  1. Click to configure the tier.

  2. Specify the following:

    1. Server: Select the OES server. This lists the OES servers that are configured with CIS server where agent is running and also includes the cluster resources.

    2. Volume: Select a volume.

    3. Endpoint: Select the required cloud account name.

    4. Bucket Name: Specify the bucket name used to store the migrated data. The bucket name can be obtained from your cloud account.

    5. Region: Select the region where the specified bucket name is available.

    6. Encryption: To enable this parameter, configure the encryption settings. If enabled, encrypts the migrated data in the cloud storage.

      For more information on encryption settings, see Encryption.

    7. Schedule: Select the schedule type based on how frequently the policy should be run for the tier. It includes the following: Daily, Weekly, Monthly, Once, and None.

      As a best practice, you can limit the duration of run to four hours everyday or run during the weekend to minimize the load on the OES servers. It is recommended that you do not migrate the data when users are accessing the data.

      To limit the duration of the schedule run, select Time duration for the schedule run option and specify the time duration.

  3. Click SAVE.

NOTE:

  • The secondary volume or CBV (Cloud Backed Volume) is automatically created after the tier configuration.

  • If the primary volume is AD enabled, ensure that the CBV created for the corresponding primary volume is also AD enabled using NSS tools or utilities.

Managing DST Migrate Tiers

Click the Tiers tab to view all the DST migrate tiers.

  • If the schedule type is None, click to start the DST migration.

  • To stop the DST migration, click the rotating icon .

    The next time, you start the DST migration process for the same tier, the remaining data is migrated to the cloud storage.

  • To rerun the policy for the DST tier, click .

  • To view statistics for the DST tier:

    1. Click .

      Displays information of the migrated files from the previous job run (each time the schedule starts at a specified time, a new job run is created). The statistic information includes:

      • Status: Provides the status of data migration.

      • Start Time: Provides the data migration start time.

      • End Time: Provides the data migration completion time.

      • Files Migrated: The total number of files migrated.

      • Data Size: Lists the overall size of the data migrated for each volume.

      For more information on detailed reports, see Dashboard.

    2. If no files are migrated, click Latest Migration to view the statistic information for the previous job run.

  • To modify the DST tier:

    1. Click on the DST tier you want to modify.

    2. Click UPDATE.

  • To move the DST tier to cloud tier, perform the following:

    1. After migrating all the files to cloud, remove the DST pair. For more information, see Section 7.0, Migrating DST Volumes to Cloud.

    2. Click on the DST tier.

    3. Select the required policy and click MIGRATE.

      The DST tier is no longer listed in Migrate from DST. Instead, it is listed in Tiers.

  • To delete the DST tier:

    1. Click on the DST tier you want to delete.

    2. Click OK.

6.1.5 Dashboard

The Dashboard page displays the following:

  • File Size: Total size of files migrated and recalled.

  • File Count: Total number of files migrated and recalled.

  • Graphical Representation of Files Migrated and Recalled: Displays a line graph for both files migrated and recalled. The horizontal axis (x-axis) represents the migrated or recalled time, whereas the vertical axis (y-axis) represents the data size. Click on the files migrated or recalled value and zoom in to view the exact time the individual files are moved.

  • Select Date: Click on date icon at the top right corner to select a date and to view the information of files migrated and recalled on the selected dates. The date selection option is available on Dashboard and Statistics page.

Click the file size or file count to go to Volumes page. The Volumes page displays the total size of files migrated, recalled, and a graphical representation of the same in the form of pie graph for each volume.

Click on a volume or pie graph to go to Statistics page. The Statistics page displays the file size, file count, and a graphical representation of files migrated and recalled for a specific volume. Click More Details to view the detailed information of files migrated and recalled. The migrated files are displayed based on the run ID (ID generated for every policy/job run on each tier). Select the run ID to view the information of migrated files for a specific job (policy run for a specific volume).

6.1.6 Roles

Configure roles for the user or group objects that belongs to eDirectory to manage CIS.

NOTE:Active Directory user or group objects are not supported.

Configuring Roles

NOTE:Before configuring the Roles, create a proxy user using iManager and specify that user in Proxy User Name under the Settings > Proxy User and Context tab. For more information, see Proxy User and Context.

Click the Roles tab, the Roles page is displayed.

  1. Click to configure the role.

  2. Specify the following:

    1. Name: Specify the eDirectory object name.

    2. Type: Select either User or Group.

    3. Role: Select the required access:

      • Read Admin: The user or group objects can only view the cloud account, policy and tier information.

      • Execute Admin: The user or group objects can configure cloud account, create policy, and configure cloud tier.

      • Root Admin: The user or group objects can configure cloud account, create policy, configure cloud tier, configure roles and modify the CIS server and agent settings.

  3. Click SAVE.

Managing Roles

Click the Roles tab to view all the roles configured.

  • To modify the role:

    1. Click on the role you want to modify.

    2. Click UPDATE.

  • To delete the role:

    1. Click on the role you want to delete.

    2. Click OK.

6.1.7 Agents

This page allows you to set the configuration for OES server agents and scanners. Click the Agents tab, the Agent Settings page is displayed. It includes:

Global Configuration for All Agents

This section lists the common configuration for all the CIS agents on OES servers that is connected to CIS server. It includes the following:

  • Port: Specify the port through which all the CIS agents on OES servers communicate to CIS server. The default value is 8000.

    NOTE:If you are modifying the port, ensure to close the existing port.

  • Log Level: Select the log level for all the CIS agents. The options are: Panic, Fatal, Error, Warn, Info, and Debug. The default log level is Info.

  • Throttling: Click toggle button (On) to regulate the data transfer rate of the recalled files. By default, it is Off. It includes:

    • Duration: Specify the time interval in seconds within which the specified number of files in File Limit should be recalled. The default value is 60 seconds.

    • File Limit: Specify the total number of files to be recalled within the specified Duration. The default value is 100.

    For example, consider a scenario where the Duration as 30 seconds and File Limit as 60, which indicates maximum 60 files can be recalled at any given point in time depending on the file size and network bandwidth. Assuming that if 60 files are recalled in first 10 seconds, then the time taken to recall the remaining files is based on the following rule:

    Recall rate per file = Duration/File Limit = "n" seconds, which means 30/60 = 0.5 seconds.

    NOTE:Ensure to restart the recall agent for throttling changes to take effect.

  • Online/Offline: Enables or disables the data recall on all the CIS agents. By default, it is set to Online.

  • Enabled/Disabled: Enables or disables the data migration and recall on all the CIS agents on OES server that is connected to CIS. By default, it is Enabled.

    If disabled, it brings down all the CIS agents. To enable the CIS agents again, set toggle button to Enabled, click SAVE and then manually login to the OES servers and restart the CIS agent service

After setting the global configuration, click SAVE.

Configuration for Each Agent

This section lists all the CIS agents on OES server that is connected to the CIS server. It provides the setting parameters for individual CIS agent.

NOTE:The agents in the highlighted rows will be configured with the global agent configuration.

  • Agent Name: Displays the CIS agent name on OES server that is connected to CIS.

  • Data Server: Select the required CIS data server URI through which the CIS agent should communicate to. By default, the CIS server URI is displayed

  • Port: Specify the port through which the CIS agent on OES server should communicate to CIS data server. The default value is 8000.

    NOTE:If you are modifying the port, ensure to close the existing port.

  • Log Level: Select the log level for the CIS agent. The options are: Panic, Fatal, Error, Warn, Info, and Debug. The default log level is Info.

  • Agent State: This includes the following:

    • Online/Offline: Enables or disables the data recall on that CIS agent. By default, it is set to Online.

    • Enabled/Disabled: Enables or disables the data migration and recall on that CIS agent. By default, it is Enabled.

      If disabled, it brings down the CIS agent. To enable the CIS agent again, set toggle button to Enabled, click SAVE and then manually login to the OES server and restart the CIS agent service.

After setting the agent configuration, click SAVE.

Global Configuration for All Scanners

This section lists the common configuration for all the CIS scanners on OES servers that is connected to CIS server. It includes the following:

  • Schedule: Select the schedule type based on how frequently the scanner should run on OES servers. It includes the following: Daily, Weekly, Monthly, Once, and None.

    To limit the duration of the schedule run, select Time duration for the schedule run option and specify the time duration.

  • Incremental Scanning: Allows you to perform a full scan or a incremental scan on the OES volumes. By default, it is Disabled. If enabled, performs the differential scan from the previous full scan on the OES volumes.

After setting the global configuration, click SAVE.

Configuration for Each Scanner

This section lists all the CIS scanners on the OES servers that is connected to the CIS server. It provides the setting parameters for individual CIS scanner.

  • To immediately trigger a file scan, go to specific OES server and click .

  • To modify the scheduled scan:

    1. Click on the OES server you want to modify.

    2. Select the schedule type.

    3. Click UPDATE.

After setting the agent configuration, click SAVE.

6.1.8 Settings

This page allows you to set the configuration for CIS server. Click the Settings tab, the CIS settings page is displayed. It includes:

Proxy User and Context

Includes the following:

IMPORTANT:The root admin must have the following rights:

  • Rights to modify the cishost-info attribute on the server context.

  • Supervisory rights on the proxy user of eDirectory object, if root admin needs to update the proxy DN.

  • CIS Server Context: By default, it is obtained from the CIS configuration. Displays the fully distinguished name of the context under which the OES server objects that can connect to the CIS server reside. For example, ou=wdc,o=acme is set as context indicates that any server within this context can connect to CIS.

  • Administrator Search Context: Specify the context of the administrator user or group object where you can configure roles for a specific user or group. For example, if o=acme is set as search context, the authentication object is searched only within this context.

  • Proxy User Name: Specify the proxy user used by the CIS server for users or groups lookup. The proxy user should have read and compare permissions for CN attribute on the Administrator Search Context configured. The proxy user password is reset and maintained by the CIS server. This proxy user should be used to manage only CIS.

After configuring the context settings, click SAVE.

General

Includes the following:

  • Secondary Volume Suffix Pattern: Specify the name that needs to be suffixed with the secondary volume name. The default value is _CBV. For example, if the primary volume name is VOL1, the secondary volume name is VOL1_CBV.

  • Log Level: Select the log level for CIS services. The options are: Panic, Fatal, Error, Warn, Info, and Debug. The default log level is Info.

After configuring the general settings, click SAVE.

Encryption

CIS supports pool-based encryption and generates a pool of keys for data encryption. It includes the following:

  • Key Size: Select the AES encryption key size based on your requirement.

  • Pool Size: Specify a valid integer. Based on the specified value, the corresponding pool of keys are generated and used for data encryption. The minimum pool size value is 127.

After configuring the encryption settings, click SAVE to generate the pool of keys. To again generate a new pool of keys for a different Pool Size values, click REGENERATE.

6.1.9 Data Servers

The data servers page displays the following:

  • Lists all the gateway servers and the data servers that are connected to the individual gateway server. It also displays the corresponding CIS agents configured for all the gateway servers.

  • Lists all the standalone data servers and corresponding CIS agents configured to a standalone data server.

6.1.10 Health Indicator

The CIS health indicator shows the current server health status as healthy, partially healthy, or not healthy. Click the server health icon at the top right corner to display the CIS Health Status window.

The CIS health status displays the status of the following:

  • Summary: Displays the overall status of the CIS server.

  • Services: Displays the status of all the CIS services.

  • Gateway: Displays the connection status between the CIS gateway and CIS services.

  • Infrastructure: Displays the status of ZooKeeper, Elasticsearch, Kafka, and Database.

  • Configuration: Displays the status of docker daemon, docker network for CIS, CIS network in iptables - Firewall, network masquerading for CIS, CIS ports, and logging service.

If the server health status is not good, click FIX to fix the issues. Similarly, to get the updated health status of CIS server, click REFRESH.