Setting up SMC server redundancy

With SMC server redundancy, service continuity can be guaranteed during a failure of the SMC server. Redundancy involves the use of two SMC servers, on which the configuration is synchronized:

  • The main node,

  • The backup node.

When redundancy has been set up, the connection with SNS firewalls that are connected to SMC will continue in any of the following cases:

  • The main node encounters an issue and firewalls can no longer access SMC,

  • The connection between the main node and all firewalls has been disrupted,

  • When you voluntarily shut down the main node, to conduct maintenance operations, for example.

The firewalls will then automatically connect to the backup node. You must then connect to the backup node to manage and monitor the firewalls.

When the main node resumes operation or is accessible again, the firewalls will connect to it again without the need for any manual action. The main node will then retrieve the configuration from the backup node.

We recommend always using the main node when it is available. If the connection is disrupted between the main node and only some firewalls, the firewalls in question will connect to the backup node. In this case, we recommend searching for the causes of the disruption and establishing the connection again to continue managing the firewalls from the main node. Do note that if you manage these firewalls temporarily from the backup node while the main node is still available, the configuration on the main node will overwrite any changes that you make to the configuration when both nodes are synchronized. For more information on how synchronization works, refer to the section Understanding synchronization between two nodes.

The configurations on the main and backup nodes are synchronized every hour in two phases:

  • The configuration of the main node is exported five minutes past every hour (e.g., 9:05 a.m.),

  • The exported configuration is sent to the backup node fifteen minutes past every hour (e.g., 9:15 a.m.). The configurations will then be synchronized.

The frequency of synchronizations cannot be configured.

During synchronization, all data required for firewall monitoring and management will be replicated. The synchronized configuration does not include the following elements:

  • The SMC server license. You must install a license on each node.

  • The IP and DNS configurations of the SMC server.

  • The "root" account password.

  • The configuration of the NTP synchronization.

  • Any custom system configuration created by an administrator.

Configurations will be synchronized only if changes have been made to the configuration within the hour, either via the web administration interface, the public API or an smc-* command.

As a result, if you make changes directly to any of the files in the /data/config folder (e.g., the file cfgcheck.ini or smc-webservices.local), they will only be synchronized the next time changes are made to the configuration either via the web administration interface, the public API or an smc-* command.

When the following operations are performed via the web administration interface, they will not warrant a synchronization:

As such, the first three operations must be performed manually on both nodes.

To set up redundancy, follow the requirements and recommendations below:

  • Both nodes must be installed and initialized in line with the procedure indicated in the Installation guide.

  • We recommend installing both nodes in the same virtual environment for optimal operation.

  • They must both be equipped with the same version of SMC, otherwise redundancy will not function.

  • We recommend that you scale virtual machines with the exact same settings (vCPU and RAM).

  • Both nodes must apply the same NTP configuration to function on the same time and in the same time zone. For more information, refer to the section Changing the SMC server time zone and date.

  • Each node needs to have its own license, with both supporting the same number of firewalls. If either node does not have a license, redundancy will not function.

  • There must not be any common IP addresses between both nodes.

  • SSH communication must be enabled between both nodes. For more information, see the next section.

The SCP protocol, which is used for synchronizing both nodes, requires authentication via SSH key in order to function.

You must generate a key pair for each node, then forward the public key to the opposite node.

Follow the procedure below, and ensure that you use the paths and file names indicated:

  1. Connect to the first node in SSH.

  2. Run the following command to generate the key pair:
    ssh-keygen -t ecdsa -b 256 -f /data/redundancy/keys/redundancy -N ""

  3. Run the following command to forward the public key to the opposite node:
    scp /data/redundancy/keys/redundancy.pub root@<REMOTE_IP>:/data/ssh/authorized_keys.root

  4. Repeat the operation on the second node.

Ensure that you have met the above requirements before enabling redundancy:

  1. Connect to the main node in SSH.

  2. Run the following command: smc-redundancy --secundaryIP <BACKUP_NODE_IP>.

    Redundancy is now enabled and the first synchronization from the main node to the backup node will immediately start.

  3. To confirm that redundancy functions, refer to the file /var/log/redundancy.log at any time.

NOTE
The main node uses the IP address of its first network interface to communicate with the backup node. The backup node uses the IP address indicated in the command above.
If the IP address of either node is changed, redundancy will no longer function, and must be enabled again with the new address.

If you are using external severs in your configuration, such as a remote syslog server to send SMC logs, or an LDAP or Radius server to authenticate administrators, ensure that both nodes can communicate with these servers. Both nodes must be able to reach the IP address or the domain name of the external server.

Once redundancy has been enabled, the IP addresses of both SMC servers to contact must be indicated to the firewalls. This information is sent to firewalls through the SMC connecting package:

  • On firewalls that were already connected to the main node before redundancy was set up, you must generate and install new connecting package containing the addresses of both nodes, as indicated in the procedure below.

  • On new firewalls, follow the steps below.

To indicate the IP addresses of both SMC servers in a firewall's connecting package:

  1. Follow the usual procedure for connecting a firewall. For more information, refer to the section Connecting SNS firewalls to the SMC server.

  2. In the section Information to connect to the SMC server, indicate the IP addresses of both servers:
    Information to connect to the server - redundancy

In the example above, the main node has the address 105.0.0.100 and the backup node has the address 105.0.0.101.

The command smc-import-firewalls makes it possible to generate several connecting packages simultaneously. For more information, refer to the section Importing firewalls in command line.

To permanently stop redundancy, disable synchronization between both nodes, then modify the connecting packages so that the firewalls can no longer connect to the backup node.

In other cases, such as when a backup of the SMC server's configuration is being restored, you must temporarily disable synchronization, then enable it again. In this case, the connecting packages do not need to be modified. For more information on restoring backups, refer to the section Managing SMC backups and SNS firewalls.

Disabling synchronization

  1. Connect to the main node in SSH.

  2. Run the following command: smc-redundancy --disable.

Editing connecting packages

Once synchronization has been disabled between both nodes, we recommend that you generate new connecting packages for each firewall connected to SMC, by indicating the IP address of the only SMC server to which they must now connect.

In this way, the firewalls will no longer attempt to connect to the former backup node with a configuration that is no longer synchronized or which may have been deleted.

Enabling synchronization again

To enable synchronization between two nodes again after it has been temporarily disabled:

  1. Connect to the main node in SSH.

  2. Run the following command: smc-redundancy --enable.

To update your SMC servers when redundancy has been set up, follow the steps below in this order:

  1. Shut down the backup node.

  2. Update the main node as explained in the section Updating the SMC server.

  3. Wait for the update to end and for firewalls to reconnect.

  4. Start the backup node and update it.

We recommend that you perform updates in time slots that do not coincide with the synchronization of both nodes. For more information, refer to the section Understanding synchronization between two nodes.

During updates, nodes cannot be synchronized in order to avoid data loss.

If you wish to restore the backup of an SMC configuration while redundancy is enabled, follow the steps below:

  1. Disable synchronization between both nodes according to the relevant procedure.

  2. Restore the backup on each node. The backup must originate from the node that is to be restored. For more information, please refer to the section Saving and restoring the SMC server configuration.

  3. Once the backups have been restored on both nodes, ensure that both servers have restarted and have the same configuration.

  4. After the backup has been restored, both nodes will have the same IP address. Change the address range to restore the initial IP configuration.

  5. Enable synchronization again by following the relevant procedure.

If you have enabled redundancy and automatic backups of the server's and firewalls' configurations, backups made by one node cannot be retrieved on the other node.

SMC can be used as an Active Update distribution point.

If you wish to use this feature and redundancy has been enabled, follow the procedure indicated in Using the SMC Active Update server on each firewall cluster by indicating the information of each node. Firewalls will then have the IP addresses and certificates with which they can use both nodes as Active Update distribution points.

If you wish to manually update the Active Update databases using the Update bases now button in the web administration interface, or via the databases' download script, this operation must be performed on both nodes. For more information, see the section Downloading Active Update databases.