ZFSSA-IB

ZFS Storage Appliance Infiniband

To attach your ZFSSA using infiniband to an EXA for example you might want more than one "virtual" datalink on your HCA using multiple IB partitions with same pkey. It is (yet?) not possible to allow the use of the same pkey when creating two IB partitions (datalinks) that point to the same IB device using the BUI and the CLI property is hidden; don't know why; but I could find a workaround in: Oracle Support Document 2087231.1 (Guidelines When Using ZFS Storage in an Exadata Environment) https://support.oracle.com/epmos/faces/DocumentDisplay?id=2087231.1

In my example I will create a bunch of datalinks to enable an active-active IPMP failover configuration for both controllers. (you will have to start with ibpart1, "0" does not work)

zsexa0101a:configuration net datalinks>partition
zsexa0101a:configuration net datalinks partition (uncommitted)> set li    <-- tab tab
linkmode  links
zsexa0101a:configuration net datalinks partition (uncommitted)> set link  <-- tab tab
linkmode  links       <-- it is not there ;-| 
zsexa0101a:configuration net datalinks partition (uncommitted)> set linkname=ibpart5
                      linkname = ibpart5 (uncommitted)
zsexa0101a:configuration net datalinks partition (uncommitted)> show
Properties:
                         class = partition
                         label = Untitled Datalink
                         links = (unset)
                          pkey = (unset)
                      linkmode = cm
 
zsexa0101a:configuration net datalinks partition (uncommitted)> set links=ibp0
                         links = ibp0 (uncommitted)
zsexa0101a:configuration net datalinks partition (uncommitted)> set pkey=ffff
                          pkey = ffff (uncommitted)
zsexa0101a:configuration net datalinks partition (uncommitted)> show
Properties:
                         class = partition
                         label = Untitled Datalink
                         links = ibp0 (uncommitted)
                          pkey = ffff (uncommitted)
                      linkmode = cm
 
zsexa0101a:configuration net datalinks partition (uncommitted)> commit
zsexa0101a:configuration net datalinks> show
Datalinks:
 
DATALINK       CLASS       LINKS       STATE   ID      LABEL
aggr1          aggregation i40e0       up      -        zsexa01-LACP
                           i40e4
ibpart1        partition   ibp0        up      -        zsexa01-IB0
ibpart2        partition   ibp2        up      -        zsexa01-IB1
ibpart3        partition   ibp0        up      -        zsexa01-IB0
ibpart4        partition   ibp2        up      -        zsexa01-IB1
ibpart5        partition   ibp0        up      -       Untitled Datalink
igb0           device      igb0        up      -       Motherboard-igb0
pffff_ibp0     partition   ibp0        up      -        zsexa01-IB0
pffff_ibp2     partition   ibp2        up      -        zsexa01-IB1
vnic1          vnic        igb0        up      -        zsexa0101a-VNIC
vnic2          vnic        igb0        up      -        zsexa0102a-VNIC
vnic3          vnic        aggr1       up      -        zsexa0101c-VNIC
vnic4          vnic        aggr1       up      -        zsexa0102c-VNIC