Intersite Application¶

Overview¶

The Intersite application allows application policies to be applied across ACI fabrics. This is configured on an Endpoint Group (EPG) basis. When endpoints appear in an EPG that is configured as an intersite EPG, the endpoint (EP) and EPG information is propagated to the remote sites as per the intersite policy.

On the remote sites, the EP and EPG information is used at the ingress L3out interface to reclassify the endpoint traffic into the appropriate EPG. Since multiple sites may be involved, the information may be propagated to more than 1 remote site as per the intersite policy. Likewise, each site may use more than 1 L3out interface to communicate with the other sites so the endpoint and EPG information may be installed on more than 1 L3out interface within the site.

Installation¶

The Intersite application runs on top of the acitoolkit and comes as part of its applications suite. Installation of the acitoolkit can be performed as per the instructions here. It is recommended that the acitoolkit be installed using the git clone commands. Once installed, the Intersite application can be found in the directory acitoolkit/applications/multisite.

Usage¶

For each site, a single instance of the Intersite application is to be run. This instance will handle exporting the EP and EPG information from the local site to the remote sites as per the Intersite policy. Since most communication will be bidirectional, this implies that there will be a similar policy on the remote site application exporting the EPs on the desired EPGs back to the local site. The Intersite application is run as a standard python script and can be called as shown below:

python intersite.py -h

usage: intersite.py [-h] [--config CONFIG] [--generateconfig]
                    [--debug [{verbose,warnings,critical}]]

ACI Multisite Tool

optional arguments:
  -h, --help            show this help message and exit
  --config CONFIG       Configuration file
  --generateconfig      Generate an empty example configuration file
  --maxlogfiles MAXLOGFILES
                        Maximum number of log files (default is 10)
  --debug [{verbose,warnings,critical}]
                        Enable debug messages.

The Intersite policy that defines the operation of the tool is provided using the --config command line option. This option is followed by the configuration filename that contains the intersite policy. The intersite policy is described using JSON and the details are described in the Configuration File section.

A sample configuration file can be generated by using the --generateconfig command line option. Specifying this option will produce the following output and quit the application. The sample JSON will be found in the file named sample_config.json and the contents are described in the Configuration File section.

python intersite.py --generateconfig

Sample configuration file written to sample_config.json
Replicate the site JSON for each site.
    Valid values for use_https and local are 'True' and 'False'
    One site must have local set to 'True'
Replicate the export JSON for each exported contract.

The --debug and --maxlogfiles options are described in the Logging section.

Configuration File¶

The configuration file defines the policy that is used to configure the tool, connect to the local and remote APICs, and the EPG export policies. The sample configuration file generated by the --generateconfig command line option described in the Usage section is shown below.

{
    "config": [
        {
            "site": {
                "username": "",
                "name": "",
                "ip_address": "",
                "password": "",
                "local": "",
                "use_https": ""
            }
        },
        {
            "export": {
                "epg": "",
                "app": "",
                "tenant": "",
                "remote_epg": "",
                "remote_sites": [
                    {
                        "site": {
                            "interfaces": [
                                {
                                    "l3out": {
                                        "consumes_interface": [
                                            {
                                                "cif_name": ""
                                            }
                                        ],
                                        "name": "",
                                        "protected_by": [
                                            {
                                                "taboo_name": ""
                                            }
                                        ],
                                        "provides": [
                                            {
                                                "contract_name": ""
                                            }
                                        ],
                                        "consumes": [
                                            {
                                                "contract_name": ""
                                            }
                                        ],
                                        "tenant": ""
                                    }
                                }
                            ],
                            "name": ""
                        }
                    }
                ]
            }
        }
    ]
}

It should be noted that the configuration consists of site and EPG export policies.

Site Policy¶

The site policy defines how the tool will communicate with the APIC of each fabric. There is a single site object in the JSON configuration for each site that the tool will communicate. Each site consists of the following items:

username: The username used to login to the APIC of this site
name: The name of this Site used in the context of the tool to correlate which site the EPG export policy is referring.
ip_address: The IP address of the APIC of this site
password: The password used to login to the APIC of this site
local: True or False. True if this site is considered the local site for this tool. Endpoints are only exported from the local site.
use_https: True or False. True if the tool should use https to login to the APIC.

EPG Export Policy¶

The EPG export policy defines which EPG to export EPs and where the EPs should be exported. There is at most a single EPG export policy for any EPG. The EPG policy contains the following items:

epg: The name of the EPG to export endpoints
app: The name of the Application Profile that contains the EPG.
tenant: The name of the tenant that contains the EPG.
remote_epg: The name to be used as the EPG (l3extInstP) on the remote site outside L3 interface (l3extOut).
remote_sites: The remote site policy for this exported EPG.

The remote_sites policy contains one or more remote site policies where the EPG will be exported. Each remote site policy contains the following items:

name: The name of the remote site. This should correspond to one of the sites defined in the Site policies.
interfaces: The list of remote L3out interface policies on the remote site to install the endpoints for this EPG.

Each L3out interface policy contains the following items which describes how the endpoints should be configured on that particular L3out interface.

name: The name of this L3out interface. This should match the name configured in the remote site APIC.
tenant: The tenant name that contains the L3out interface. This should match the name configured in the remote site APIC. It should be noted that this may be a different tenant than the EPG on the local site.
provides: This contains a list of zero or more contract names that the EP should provide when configured on the remote site.
consumes: This contains a list of zero or more contract names that the EP should consume when configured on the remote site.
consumes_interface: This contains a list of zero or more contract interface names that the EP should consume when configured on the remote site.
protected_by: This contains a list of zero or more taboo names that the EP should be protected by when configured on the remote site.
noclean: True of False value that determines whether ‘stale’ contracts should be removed from a remote EPG on script start up (default False, stale contracts are removed).

Command Shell¶

When the Intersite application is running, there is a small simple command line shell that provides basic interaction with the tool. The following commands are available at the command prompt.

configfile <filename>     Set the configuration filename to the specified filename.
show configfile           Show the current setting of the configuration filename.
show config               Show the current JSON configuration that the tool is using.
reloadconfig              Reload the JSON configuration from the configuration file. This is used to enable new configuration additions, changes, or deletions.
show debug                Show the current level setting of the debug messages.
debug critical            Sets the debug message level to critical.
debug warnings            Sets the debug message level to warnings.
debug verbose             Sets the debug message level to verbose.
help [cmd]                Displays help for any command.
quit                      Quit the Intersite tool.

REST API¶

The Intersite application can be accessed through a simple server application using a REST API. When run as a server, the application usage is run as follows:

python intersite_rest_server.py -h
usage: intersite_rest_server.py [-h] [--config CONFIG]
                                [--maxlogfiles MAXLOGFILES] [--generateconfig]
                                [--debug [{verbose,warnings,critical}]]

ACI Multisite Tool

optional arguments:
  -h, --help            show this help message and exit
  --config CONFIG       Configuration file
  --maxlogfiles MAXLOGFILES
                        Maximum number of log files (default is 10)
  --generateconfig      Generate an empty example configuration file
  --debug [{verbose,warnings,critical}]
                        Enable debug messages.
  --ip IP               IP address to listen on.
  --port PORT           Port number to listen on.

An example of running the application would be:

python intersite_rest_server.py --config my_config.json

By default, the server runs on the loopback IP address 127.0.0.1 and the port 5000. This can be changed via the --ip and --port command line options.

The REST API allows retrieving and setting the configuration of the tool. This is done through a single URL, namely /config.

By sending HTTP GET requests to /config, the configuration can be retrieved.

By sending JSON configuration in an HTTP POST or HTTP PUT to /config, the configuration can be changed. This JSON configuration is the same format as decribed in the Configuration File section. If the configuration is not valid JSON, a 400 error response code will be returned.

The REST API is protected by basic HTTP authentication with the default username set to admin and the default password set to acitoolkit.

Some examples using the REST API via curl:

curl –i –u admin:acitoolkit –H “Content-Type: application/json” -X PUT –d@my_config.json http://localhost:5000/config
curl –i –u admin:acitoolkit –X GET http://localhost:5000/config

Automator¶

The intersite application can be run in a basic automated fashion which will identify EPG’s with an assigned tag and automatically create the appropriate L3out objects and contracts from the templated configuration. The usefulness of this utility is limited by the requirement of tenant’s needing to exist in both sites (APIC’s) consistantly.

python intersite_automator.py -h
usage: intersite_automator.py [-h] [--config CONFIG] [--generateconfig]
                          [--debug [{verbose,info,warnings,critical}]]
                          [--stdout]

ACI Multisite Automation Tool

optional arguments:
   -h, --help            show this help message and exit
  --config CONFIG       Configuration file in JSON format
  --generateconfig      Generate an empty example configuration file
  --debug [{verbose,info,warnings,critical}]
                       Enable printing of debug messages
  --stdout              Output all log events to stdout

To make use of this modified version of the intersite application, you will need to use a different configuration file. The automator removes the need for statically defining the individual export policies but requires a new ‘automator’ specific configuration.

{
"config": [
    {
        "site": {
            "name": "",
            "username": "",
            "password": "",
            "ip_address": "",
            "use_https": "",
            "local": ""
        }
    },
    {
        "site": {
            "name": "",
            "username": "",
            "password": "",
            "ip_address": "",
            "use_https": "",
            "local": ""
        }
    }
],

"automator": {
    "check_interval": "60",
    "search_filter": "replicated",

    "remote_l3out": {
        "tenant": "Layer3OutTenant",
        "interface_name": "L3Out.Site1-Site2.SERVER",
        "network_name": "Site2.%{tenant}.%{app}.%{epg}"
    },

    "remote_contracts": {
        "consume_contract": [{
            "name": "CT.%{tenant}.%{app}.%{epg}.to.Site2",
            "default_filter": "east-west-allow-all",

            "export_to_epg_owner": "True",
            "export_name": "x.CT.%{tenant}.%{app}.%{epg}.to.Site2"
        }],
        "provide_contract": [{
            "name": "CT.%{tenant}.%{app}.%{epg}.to.Site2",
            "default_filter": "east-west-allow-all",

            "export_to_epg_owner": "True",
            "export_name": "x.CT.%{tenant}.%{app}.%{epg}.to.Site2"
        }],
        "consume_int_contract": []
    }
}
}

The site specific configuration hasn’t changed, and matches the definition in the previous section of this documentation

check_interval: integer - The time in seconds that intersite_automator will sleep between searching for EPG’s with the search_filter tag (this should be a >0 value to reduce API calls on the APIC)
search_filter: string - The tag that intersite_automator will search for (on EPG’s) to identify which EPG’s to replicate.
remote_l3out->tenant: string - The tenant that owns the L3out object that will have the ‘remote EPGs’ created under.
remote_l3out->interface_name: string - The interface that will have the ‘remote EPGs’ created under.
remote_l3out->network_name: string - The network name pattern (see pattern information below) that will be used to create the new remote EPG.
remote_contracts-><contract type>->name: string - The contract name pattern (see pattern information below) that will be attached to the newly created remote EPG (this will be created if it does not exist)
remote_contracts-><contract type>->default_filter: string - A default filter that will be attached to a contract that is created by the intersite automator
remote_contracts-><contract type>->export_to_epg_owner: boolean string - Determines whether or not the intersite automator script should export this contract to the EPG owner. This is useful if your L3out object sits in a different tenant as the EPG but also assumes that there is some level of consistancy across multiple APICs.
remote_contracts-><contract type>->export_name: string - The pattern to use when exporting the contract to the EPG owner.

Patterns: Certain configuration fields allow the use of a very primitive variable substitution. This enables you to create EPGs and Contracts with dynamic names. The following variables are currently supported:

${tenant} - The name of the tenant that owns the EPG
${app} - The name of the application that the EPG belongs to
${epg} - The name of the EPG that is being replicated

Logging¶

The Intersite application supports logging of various debug messages. The messages are categorized by levels and the application allows the level to be set by the user through the --debug option. The lowest level is critical and will only log the most critical messages. The next level is warnings and will log all of the critical messages as well as those deemed suspect. The verbose level gives the previous levels plus additional information that provides deep insight into what has occurred in the tool.

The debug messages are stored in the file intersite.log. When the file reaches 5 MB in size, the log file will rollover to a new file up to the configured maximum number of log files. By default, this maximum is 10 but the number can be changed by the user via the --maxlogfiles command line option. Once the maximum number of log files has been reached, the oldest log file will be deleted.

APIC Object Model¶

The following objects are used from the APIC Object Model to enable the functionality of the Intersite tool.

The EPG that is configured to have its endpoints exported to the remote sites is defined using the classes fvTenant, fvAp, and fvAEPg.

The endpoints are tracked by subscribing to notifications from the fvCEp and fvStCEp classes and examining the EPG membership of those endpoints.

The endpoints are installed on the remote site by using the classes l3extInstP and l3extSubnet under l3extOut. The endpoint address is installed as a /32 entry within the l3extSubnet. The l3extInstP contains the children fvRsProv, fvRsCons, fvRsConsIf, and fvRsProtBy to associate the endpoint with the appropriate contracts, taboos, and contract interfaces.

Additionally, a tagInst object is placed on every l3extInstP that contains specially formatted string within the opaque tagInst name field. This string is formatted as follows: isite:tenant_name:app_name:epg_name:site_name where the tenant, app, and epg names are that where the original endpoint is connected and the site_name is that of the site that installed the l3extInstP.

Importing within other applications¶

Each instance of the tool is represented by the Collector class. Within the Collector class, there will be multiple instances of the Site class, namely the LocalSite and RemoteSite classes.

If you wish to include the Intersite application in your own app, the policies can be defined as python dictionaries directly and passed to the LocalSite instance using the add_policy() function. This will allow the Monitor class to begin exporting the EP of the specified EPG according to the policy. Similarly, the external configuration file can be updated and the reload_config() function can be called within the Collector class.