LDOM
From pressy's brainbackup
Contents
LDOM
My first T5-2 impressions with the new Logical Domain Manager 3. These are only quick&dirty notes for myself... Based on this blogs, thanks: https://blogs.oracle.com/cmt/entry/what_s_up_with_ldoms
Simple Setup
Activation
I installed Solaris 11.1 on my T5, so the LDOM software was already installed... First I had to "activate" the LDOM partitioning and some services like the virtual network terminal server daemon. To create new LDOMs, you need to remove HW from the primary domain.
# ldm add-vconscon port-range=5000-5100 primary-console primary: # svcadm enable vntsd # svcs vntsd STATE STIME FMRI online 17:38:35 svc:/ldoms/vntsd:default # ldm set-vcpu 128 primary # ldm start-reconf primary <-- reconfigure after reboot # ldm set-memory 127G primary # ldm add-config initial # ldm add-vds primary-vds primary # ldm add-vswitch net-dev=net0 switch-primary primary # init 6
After the reboot we have resources for other guest domains...
Create Guest LDOM
You can always choose the names by yourself, I think that could become complicated in a huge environment... this is a small configuration with on a ZVOL as hard drive based on my data ZPOOL:
# ldm create sol10-test01 # ldm list-devices -S <-- list free resources # ldm set-vcpu 128 sol10-test01 # ldm set-memory 128G sol10-test01 # zfs create -o mountpoint=none data/ldoms # zfs create -V 128G data/ldoms/sol10-test01.hdd0 # ldm add-vdsdev /dev/zvol/dsk/data/ldoms/sol10-test01.hdd0 sol10-test01.hdd0@primary-vds # ldm add-vdisk hdd0 sol10-test01.hdd0@primary-vds sol10-test01 # ldm add-vnet net0 switch-primary sol10-test01 # telnet localhost 5000
Install LDOM
I've installed Solaris 10 from a DVD image:
# ldm stop sol10-test01
# ldm unbind sol10-test01
# ldm add-vdsdev /downloads/sol-10-u11-ga-sparc-dvd.iso sol10u11.iso@primary-vds
# ldm add-vdisk sol10u11.iso sol10u11.iso@primary-vds sol10-test01
# ldm bind sol10-test01
# ldm start sol10-test01
# telnet localhost 5000
{0} ok boot sol10u11.iso:f
And yes, it worked ;)
# telnet localhost 5000
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to <name>.
Escape character is '^]'.
{0} ok banner
SPARC T5-2, No Keyboard
Copyright (c) 1998, 2013, Oracle and/or its affiliates. All rights reserved.
OpenBoot 4.35.0.a, 128.0000 GB memory available, Serial #833XXX86.
Ethernet address 0:14:4f:f8:XX:XX, Host ID: 84fXXX9a.
{0} ok
{0} ok boot
Boot device: disk:a File and args:
SunOS Release 5.10 Version Generic_148888-05 64-bit
Copyright (c) 1983, 2013, Oracle and/or its affiliates. All rights reserved.
Hostname: sol10-test01
sol10-test01 console login: root
Password:
Last login: Tue Jul 2 17:49:28 from 192.168.50.101
Jul 3 15:12:30 sol10-test01 login: ROOT LOGIN /dev/console
Oracle Corporation SunOS 5.10 Generic Patch January 2005
Sourcing //.profile-EIS.....
# uname -a
SunOS sol10-test01 5.10 Generic_148888-05 sun4v sparc sun4v
# psrinfo -pv
The physical processor has 128 virtual processors (0-127)
SPARC-T5 (chipid 0, clock 3600 MHz)
# prtconf | grep Memory
Memory size: 131072 Megabytes
Split I/O Domain
This Time, reseted to default, I will setup an own I/O domain.
Primary Setup
First of all, we need free resources, so let's shrink the service domain
root@t5pd:~# ldm list
NAME STATE FLAGS CONS VCPU MEMORY UTIL NORM UPTIME
primary active -n-cv- UART 256 255G 0.0% 0.0% 3h 19m
root@t5pd:~# ldm list-services
root@t5pd:~#
root@t5pd:~# ldm add-vconscon port-range=5000-5100 primary-console primary
root@t5pd:~# ldm list-services
VCC
NAME LDOM PORT-RANGE
primary-console primary 5000-5100
root@t5pd:~# ldm list
NAME STATE FLAGS CONS VCPU MEMORY UTIL NORM UPTIME
primary active -n-cv- UART 256 255G 0.0% 0.0% 3h 19m
root@t5pd:~#
root@t5pd:~# svcadm enable vntsd
root@t5pd:~# svcs -l vntsd
fmri svc:/ldoms/vntsd:default
name virtual network terminal server
enabled true
state online
next_state none
state_time Thu Sep 19 13:20:31 2013
logfile /var/svc/log/ldoms-vntsd:default.log
restarter svc:/system/svc/restarter:default
contract_id 125
manifest /lib/svc/manifest/platform/sun4v/vntsd.xml
dependency optional_all/error svc:/milestone/network (online)
dependency optional_all/none svc:/system/system-log (online)
root@t5pd:~#
I/O domain
OK... we will split "the backplane", so what devices are needed for the current installation...
root@t5pd:~# zpool status
pool: rpool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
c0t5000CCA0160BCAB8d0s0 ONLINE 0 0 0
errors: No known data errors
root@t5pd:~# mpathadm show lu /dev/rdsk/c0t5000CCA0160BCAB8d0s0
Logical Unit: /dev/rdsk/c0t5000CCA0160BCAB8d0s2
mpath-support: libmpscsi_vhci.so
Vendor: HITACHI
Product: H109030SESUN300G
Revision: A31A
Name Type: unknown type
Name: 5000cca0160bcab8
Asymmetric: no
Current Load Balance: round-robin
Logical Unit Group ID: NA
Auto Failback: on
Auto Probing: NA
Paths:
Initiator Port Name: w508002000159c458
Target Port Name: w5000cca0160bcab9
Override Path: NA
Path State: OK
Disabled: no
Target Ports:
Name: w5000cca0160bcab9
Relative ID: 0
root@t5pd:~# mpathadm show initiator-port w508002000159c458
Initiator Port: w508002000159c458
Transport Type: unknown
OS Device File: /devices/pci@300/pci@1/pci@0/pci@2/scsi@0/iport@1
Initiator Port: w508002000159c458
Transport Type: unknown
OS Device File: /devices/pci@300/pci@1/pci@0/pci@2/scsi@0/iport@2
root@t5pd:~# ipadm
NAME CLASS/TYPE STATE UNDER ADDR
lo0 loopback ok -- --
lo0/v4 static ok -- 127.0.0.1/8
net0 ip ok -- --
net0/v4 static ok -- 192.168.56.200/24
net5 ip ok -- --
net5/v4 static ok -- 169.254.182.77/24
root@t5pd:~# dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net5 Ethernet up 10 full usbecm2
net0 Ethernet up 1000 full ixgbe0
net2 Ethernet unknown 0 unknown ixgbe2
net1 Ethernet unknown 0 unknown ixgbe1
net3 Ethernet unknown 0 unknown ixgbe3
root@t5pd:~# ls -l /dev/ixgbe0
lrwxrwxrwx 1 root root 53 Sep 17 18:24 /dev/ixgbe0 -> ../devices/pci@300/pci@1/pci@0/pci@1/network@0:ixgbe0
soo... we need pci@300 for my devices...
root@t5pd:~# ldm list -l -o physio primary
NAME
primary
FLAGS
normal,delayed(),control,vio-service
IO
DEVICE PSEUDONYM OPTIONS
pci@340 pci_1
pci@300 pci_0
pci@3c0 pci_3
pci@380 pci_2
pci@340/pci@1/pci@0/pci@4 /SYS/MB/PCIE5
pci@340/pci@1/pci@0/pci@5 /SYS/MB/PCIE6
pci@340/pci@1/pci@0/pci@6 /SYS/MB/PCIE7
pci@300/pci@1/pci@0/pci@4 /SYS/MB/PCIE1
pci@300/pci@1/pci@0/pci@2 /SYS/MB/SASHBA0
pci@300/pci@1/pci@0/pci@1 /SYS/MB/NET0
pci@3c0/pci@1/pci@0/pci@7 /SYS/MB/PCIE8
pci@3c0/pci@1/pci@0/pci@2 /SYS/MB/SASHBA1
pci@3c0/pci@1/pci@0/pci@1 /SYS/MB/NET2
pci@380/pci@1/pci@0/pci@5 /SYS/MB/PCIE2
pci@380/pci@1/pci@0/pci@6 /SYS/MB/PCIE3
pci@380/pci@1/pci@0/pci@7 /SYS/MB/PCIE4
-------------> I will remove pci@3c0 for my I/O Domain (alias pci_3).
root@t5pd:~# ldm remove-io pci_3 primary
------------------------------------------------------------------------------
Notice: The primary domain is in the process of a delayed reconfiguration.
Any changes made to the primary domain will only take effect after it reboots.
------------------------------------------------------------------------------
root@t5pd:~#
root@t5pd:~# ldm set-vcpu -c 1 primary
------------------------------------------------------------------------------
Notice: The primary domain is in the process of a delayed reconfiguration.
Any changes made to the primary domain will only take effect after it reboots.
------------------------------------------------------------------------------
root@t5pd:~# ldm set-memory 8G primary
------------------------------------------------------------------------------
Notice: The primary domain is in the process of a delayed reconfiguration.
Any changes made to the primary domain will only take effect after it reboots.
------------------------------------------------------------------------------
root@t5pd:~#
root@t5pd:~# ldm list-config
factory-default [next poweron]
root@t5pd:~# ldm add-config mpr-split-io
root@t5pd:~# ldm list-config
factory-default
mpr-split-io [current]
root@t5pd:~#
root@t5pd:~# init 6
[...]
root@t5pd:~# psrinfo
0 on-line since 09/20/2013 14:50:18
1 on-line since 09/20/2013 14:50:22
2 on-line since 09/20/2013 14:50:22
3 on-line since 09/20/2013 14:50:22
4 on-line since 09/20/2013 14:50:22
5 on-line since 09/20/2013 14:50:22
6 on-line since 09/20/2013 14:50:22
7 on-line since 09/20/2013 14:50:22
root@t5pd:~# psrinfo -p
1
root@t5pd:~# prtconf | grep Memory
Memory size: 8192 Megabytes
root@t5pd:~#
root@t5pd:~# ldm list -l -o physio primary
NAME
primary
IO
DEVICE PSEUDONYM OPTIONS
pci@340 pci_1
pci@300 pci_0
pci@380 pci_2
pci@340/pci@1/pci@0/pci@4 /SYS/MB/PCIE5
pci@340/pci@1/pci@0/pci@5 /SYS/MB/PCIE6
pci@340/pci@1/pci@0/pci@6 /SYS/MB/PCIE7
pci@300/pci@1/pci@0/pci@4 /SYS/MB/PCIE1
pci@300/pci@1/pci@0/pci@2 /SYS/MB/SASHBA0
pci@300/pci@1/pci@0/pci@1 /SYS/MB/NET0
pci@380/pci@1/pci@0/pci@5 /SYS/MB/PCIE2
pci@380/pci@1/pci@0/pci@6 /SYS/MB/PCIE3
pci@380/pci@1/pci@0/pci@7 /SYS/MB/PCIE4
root@t5pd:~#
root@t5pd:~# ldm list-io
NAME TYPE BUS DOMAIN STATUS
---- ---- --- ------ ------
pci_1 BUS pci_1 primary
pci_0 BUS pci_0 primary
pci_3 BUS pci_3
pci_2 BUS pci_2 primary
/SYS/MB/PCIE5 PCIE pci_1 primary EMP
/SYS/MB/PCIE6 PCIE pci_1 primary EMP
/SYS/MB/PCIE7 PCIE pci_1 primary EMP
/SYS/MB/PCIE1 PCIE pci_0 primary EMP
/SYS/MB/SASHBA0 PCIE pci_0 primary OCC
/SYS/MB/NET0 PCIE pci_0 primary OCC
/SYS/MB/PCIE8 PCIE pci_3 UNK
/SYS/MB/SASHBA1 PCIE pci_3 UNK
/SYS/MB/NET2 PCIE pci_3 UNK
/SYS/MB/PCIE2 PCIE pci_2 primary EMP
/SYS/MB/PCIE3 PCIE pci_2 primary EMP
/SYS/MB/PCIE4 PCIE pci_2 primary EMP
/SYS/MB/NET0/IOVNET.PF0 PF pci_0 primary
/SYS/MB/NET0/IOVNET.PF1 PF pci_0 primary
root@t5pd:~#
OK, now let's create the second domain with the "2nd half" of the box
root@t5pd:~# ldm create io-dom
root@t5pd:~# ldm set-vcpu -c 1 io-dom
root@t5pd:~# ldm set-memory 8G io-dom
root@t5pd:~# ldm add-io pci_3 io-dom
root@t5pd:~#
root@t5pd:~# ldm list
NAME STATE FLAGS CONS VCPU MEMORY UTIL NORM UPTIME
primary active -n-cv- UART 8 8G 0.0% 0.0% 19m
io-dom inactive ------ 8 8G
root@t5pd:~#
root@t5pd:~# ldm bind io-dom
root@t5pd:~# ldm list
NAME STATE FLAGS CONS VCPU MEMORY UTIL NORM UPTIME
primary active -n-cv- UART 8 8G 1.0% 0.9% 23m
io-dom bound ------ 5000 8 8G
root@t5pd:~#
root@t5pd:~# ldm start io-dom
LDom io-dom started
root@t5pd:~#
root@t5pd:~# telnet localhost 5000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Connecting to console "io-dom" in group "io-dom" ....
Press ~? for control options ..
{0} ok
{0} ok
{0} ok banner
SPARC T5-2, No Keyboard
Copyright (c) 1998, 2013, Oracle and/or its affiliates. All rights reserved.
OpenBoot 4.35.3, 8.0000 GB memory available, Serial #83xxxxxx.
Ethernet address 0:14:xx:xx:xx:xx, Host ID: 84xxxxxx.
{0} ok
Now we need to install that I/O domain. Best way would be an AI. Had a small issue, because the "net" alias was not set. But after setting the OBP-devalias it worked to boot/install:
{0} ok banner
SPARC T5-2, No Keyboard
Copyright (c) 1998, 2013, Oracle and/or its affiliates. All rights reserved.
OpenBoot 4.35.3, 8.0000 GB memory available, Serial #83xxxxxx.
Ethernet address 0:14:xx:xx:xx:xx, Host ID: 84xxxxxx.
{0} ok
{0} ok boot net2:dhcp - install
Boot device: /pci@3c0/pci@1/pci@0/pci@1/network@0:dhcp File and args: - install
1000 Mbps full duplex Link up
Timed out waiting for BOOTP/DHCP reply
<time unavailable> wanboot info: WAN boot messages->console
<time unavailable> wanboot crit: No network device available for wanboot!
<time unavailable> wanboot crit: (Ensure that the 'net' alias is set correctly)
Type 'go' to resume
{0} ok
{0} ok
{0} ok nvalias net /pci@3c0/pci@1/pci@0/pci@1/network@0
{0} ok boot net:dhcp - install
NOTICE: Entering OpenBoot.
NOTICE: Fetching Guest MD from HV.
NOTICE: Starting additional cpus.
NOTICE: Initializing LDC services.
NOTICE: Probing PCI devices.
NOTICE: Finished PCI probing.
SPARC T5-2, No Keyboard
Copyright (c) 1998, 2013, Oracle and/or its affiliates. All rights reserved.
OpenBoot 4.35.3, 8.0000 GB memory available, Serial #83378927.
Ethernet address 0:14:4f:f8:42:ef, Host ID: 84f842ef.
Boot device: /pci@3c0/pci@1/pci@0/pci@1/network@0:dhcp File and args: - install
1000 Mbps full duplex Link up
<time unavailable> wanboot info: WAN boot messages->console
<time unavailable> wanboot info: configuring /pci@3c0/pci@1/pci@0/pci@1/network@0:dhcp
1000 Mbps full duplex Link up
<time unavailable> wanboot info: Starting DHCP configuration
<time unavailable> wanboot info: DHCP configuration succeeded
<time unavailable> wanboot progress: wanbootfs: Read 368 of 368 kB (100%)
<time unavailable> wanboot info: wanbootfs: Download complete
Fri Sep 20 13:21:43 wanboot progress: miniroot: Read 251058 of 251058 kB (100%)
Fri Sep 20 13:21:43 wanboot info: miniroot: Download complete
SunOS Release 5.11 Version 11.1 64-bit
Copyright (c) 1983, 2012, Oracle and/or its affiliates. All rights reserved.
Remounting root read/write
Probing for device nodes ...
Preparing network image for use
Downloading solaris.zlib
--2013-09-20 13:23:04-- http://192.168.56.113:5555/export/install/sparc-ai//solaris.zlib
Connecting to 192.168.56.113:5555... connected.
HTTP request sent, awaiting response... 200 OK
Length: 158720512 (151M) [text/plain]
Saving to: `/tmp/solaris.zlib'
100%[======================================>] 158,720,512 112M/s in 1.4s
2013-09-20 13:23:06 (112 MB/s) - `/tmp/solaris.zlib' saved [158720512/158720512]
Downloading solarismisc.zlib
--2013-09-20 13:23:06-- http://192.168.56.113:5555/export/install/sparc-ai//solarismisc.zlib
Connecting to 192.168.56.113:5555... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12876800 (12M) [text/plain]
Saving to: `/tmp/solarismisc.zlib'
100%[======================================>] 12,876,800 --.-K/s in 0.1s
2013-09-20 13:23:06 (111 MB/s) - `/tmp/solarismisc.zlib' saved [12876800/12876800]
Downloading .image_info
--2013-09-20 13:23:06-- http://192.168.56.113:5555/export/install/sparc-ai//.image_info
Connecting to 192.168.56.113:5555... connected.
HTTP request sent, awaiting response... 200 OK
Length: 88 [text/plain]
Saving to: `/tmp/.image_info'
100%[======================================>] 88 --.-K/s in 0s
2013-09-20 13:23:06 (7.44 MB/s) - `/tmp/.image_info' saved [88/88]
Done mounting image
Configuring devices.
Hostname: solaris
Service discovery phase initiated
Service name to look up: default-sparc
############
### a.s.o ####
############
Super, now I have two LDOMs which can provide I/O... let's configure a LDOM using both pci complexes
root@t5pd:~# ldm list-io
NAME TYPE BUS DOMAIN STATUS
---- ---- --- ------ ------
pci_1 BUS pci_1 primary
pci_0 BUS pci_0 primary
pci_3 BUS pci_3 io-dom
pci_2 BUS pci_2 primary
/SYS/MB/PCIE5 PCIE pci_1 primary EMP
/SYS/MB/PCIE6 PCIE pci_1 primary EMP
/SYS/MB/PCIE7 PCIE pci_1 primary EMP
/SYS/MB/PCIE1 PCIE pci_0 primary EMP
/SYS/MB/SASHBA0 PCIE pci_0 primary OCC
/SYS/MB/NET0 PCIE pci_0 primary OCC
/SYS/MB/PCIE8 PCIE pci_3 io-dom EMP
/SYS/MB/SASHBA1 PCIE pci_3 io-dom OCC
/SYS/MB/NET2 PCIE pci_3 io-dom OCC
/SYS/MB/PCIE2 PCIE pci_2 primary EMP
/SYS/MB/PCIE3 PCIE pci_2 primary EMP
/SYS/MB/PCIE4 PCIE pci_2 primary EMP
/SYS/MB/NET0/IOVNET.PF0 PF pci_0 primary
/SYS/MB/NET0/IOVNET.PF1 PF pci_0 primary
/SYS/MB/NET2/IOVNET.PF0 PF pci_3 io-dom
/SYS/MB/NET2/IOVNET.PF1 PF pci_3 io-dom
root@t5pd:~#
root@t5pd:~# ldm add-vds iodom-vds primary
root@t5pd:~# ldm add-vds iodom-vds io-dom
root@t5pd:~# ldm add-vswitch net-dev=net0 switch-primary primary
root@t5pd:~# ldm add-vswitch net-dev=net0 switch-io io-dom
root@t5pd:~#
root@t5pd:~# ldm create ldom01
root@t5pd:~# ldm set-vcpu -c 1 ldom01
root@t5pd:~# ldm set-memory 8g ldom01
root@t5pd:~#
root@t5pd:~# ldm add-vdsdev /dev/zvol/dsk/rpool/ldom/ldom01/ldom01-prim01vol ldom01-prim01vol@primary-vds
root@t5pd:~# ldm add-vdsdev /dev/zvol/dsk/rpool/ldom/ldom01/ldom01-iodom01vol ldom01-iodom01vol@iodom-vds
root@t5pd:~#
root@t5pd:~# ldm add-vdisk hdd0 ldom01-prim01vol@primary-vds ldom01
root@t5pd:~# ldm add-vdisk hdd1 ldom01-iodom01vol@iodom-vds ldom01
root@t5pd:~#
root@t5pd:~# ldm add-vnet net0 switch-primary ldom01
root@t5pd:~# ldm add-vnet net1 switch-io ldom01
root@t5pd:~# ldm set-vsw linkprop=phys-state switch-primary
root@t5pd:~# ldm set-vsw linkprop=phys-state switch-io
root@t5pd:~# ldm set-vnet linkprop=phys-state net0 ldom01
root@t5pd:~# ldm set-vnet linkprop=phys-state net1 ldom01
root@t5pd:~#
root@t5pd:~# ldm list
NAME STATE FLAGS CONS VCPU MEMORY UTIL NORM UPTIME
primary active -n-cv- UART 8 8G 0.1% 0.1% 3h 22m
io-dom active -n--v- 5000 8 8G 0.0% 0.0% 16m
ldom01 inactive ------ 8 8G
root@t5pd:~# ldm bind ldom01
root@t5pd:~#
root@t5pd:~# ldm list
NAME STATE FLAGS CONS VCPU MEMORY UTIL NORM UPTIME
primary active -n-cv- UART 8 8G 0.8% 0.8% 3h 23m
io-dom active -n--v- 5000 8 8G 0.1% 0.1% 16m
ldom01 bound ------ 5001 8 8G
root@t5pd:~# ldm start ldom01
LDom ldom01 started
root@t5pd:~# telnet localhost 5001
SPARC T5-2, No Keyboard
Copyright (c) 1998, 2013, Oracle and/or its affiliates. All rights reserved.
OpenBoot 4.35.3, 8.0000 GB memory available, Serial #83490061.
Ethernet address 0:14:4f:f9:f5:d, Host ID: 84f9f50d.
{0} ok
{0} ok
{0} ok show-disks
a) /virtual-devices@100/channel-devices@200/disk@1
b) /virtual-devices@100/channel-devices@200/disk@0
c) /iscsi-hba/disk
q) NO SELECTION
Enter Selection, q to quit: q
{0} ok
{0} ok devalias
hdd1 /virtual-devices@100/channel-devices@200/disk@1
hdd0 /virtual-devices@100/channel-devices@200/disk@0
net1 /virtual-devices@100/channel-devices@200/network@1
net0 /virtual-devices@100/channel-devices@200/network@0
net /virtual-devices@100/channel-devices@200/network@0
disk /virtual-devices@100/channel-devices@200/disk@0
virtual-console /virtual-devices/console@1
name aliases
{0} ok
[...install...]
huhu... let's use a HA setup
root@ldom01:~# echo | format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c3d0 <Unknown-Unknown-0001-128.00GB>
/virtual-devices@100/channel-devices@200/disk@0
1. c3d1 <Unknown-Unknown-0001-128.00GB>
/virtual-devices@100/channel-devices@200/disk@1
Specify disk (enter its number): Specify disk (enter its number):
root@ldom01:~# zpool attach rpool c3d0 c3d1
root@ldom01:~# zpool status
pool: rpool
state: ONLINE
scan: resilvered 7.91G in 0h3m with 0 errors on Fri Sep 20 18:42:02 2013
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c3d0 ONLINE 0 0 0
c3d1 ONLINE 0 0 0
errors: No known data errors
root@ldom01:~#
root@ldom01:~#
root@ldom01:~# ipadm show-if
IFNAME CLASS STATE ACTIVE OVER
lo0 loopback ok yes --
net0 ip ok yes --
root@ldom01:~#
root@ldom01:~# dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net1 Ethernet unknown 0 unknown vnet1
net0 Ethernet up 0 unknown vnet0
root@ldom01:~#
root@ldom01:~# route -p delete default 192.168.56.1 -ifp net0
delete net default: gateway 192.168.56.1: not in table
delete persistent net default: gateway 192.168.56.1
root@ldom01:~# route -p add default 192.168.56.1
add net default: gateway 192.168.56.1: entry exists
add persistent net default: gateway 192.168.56.1
root@ldom01:~# svcadm restart routing-setup
root@ldom01:~#
root@ldom01:~# ipadm create-ip net1
root@ldom01:~# ipadm create-ipmp ipmp0
root@ldom01:~# ipadm add-ipmp -i net0 -i net1 ipmp0
root@ldom01:~# ipadm create-addr -T static -a 192.168.56.202/24 ipmp0/v4
root@ldom01:~# ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
ipmp0/v4 static ok 192.168.56.202/24
lo0/v6 static ok ::1/128
root@ldom01:~#
root@ldom01:~# ipmpstat -a
ADDRESS STATE GROUP INBOUND OUTBOUND
:: down ipmp0 -- --
ldom01 up ipmp0 net0 net1 net0
root@ldom01:~# ipmpstat -i
INTERFACE ACTIVE GROUP FLAGS LINK PROBE STATE
net1 yes ipmp0 ------- up disabled ok
net0 yes ipmp0 --mbM-- up disabled ok
root@ldom01:~#