From Sebastien.Roy@Sun.COM Fri Jul 10 11:42:04 2009 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6AIg4rl009988 for ; Fri, 10 Jul 2009 11:42:04 -0700 (PDT) Received: from brmea-mail-2.sun.com (brmea-mail-2.Sun.COM [192.18.98.43]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6AIg4TB013561 for ; Fri, 10 Jul 2009 11:42:04 -0700 (PDT) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6AIg3vS006586 for ; Fri, 10 Jul 2009 18:42:03 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMK00100WJTFP00@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Fri, 10 Jul 2009 12:42:03 -0600 (MDT) Received: from [129.148.174.103] ([unknown] [129.148.174.103]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMK00B80XA2T7A0@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Fri, 10 Jul 2009 12:42:03 -0600 (MDT) Date: Fri, 10 Jul 2009 14:40:40 -0400 From: Sebastien Roy Subject: VRRP Update [PSARC/2009/388 FastTrack timeout 07/17/2009] Sender: Sebastien.Roy@Sun.COM To: psarc-ext Cc: Cathy Zhou Message-id: <1247251240.10474.96.camel@strat> Organization: Sun Microsystems X-Mailer: Evolution 2.26.2 Status: RO Content-Length: 19223 I'm submitting this fast-track for Cathy Zhou, it times out on 07/17/2009. This case depends on PSARC/2008/693. 2008/693 requested "micro" release binding, but no incompatible changes are introduced, and "patch" would be more appropriate (although no backport is planned). As such, this case both requests "patch" binding, and updates 2008/693 to have "patch" binding as well. The materials directory contains this specification (vrrp_psarc) as well as the documents listed in the References section. VRRP Update: Summary ======= This case describes several design issues of the original VRRP case (PSARC/2008/693 VRRP) and the proposals to address those issues. Problem area ============ Specifically, the problems with the existing VRRP design are: 1) Incorrect false accept_mode support According to the VRRP protocol, accept_mode can be set to either true or false by an administrator over a non-address-owner VRRP virtual router. If the accept_mode is set to be false, when the VRRP virtual router becomes the master (non-backup), this virtual router must not accept packets destined to the virtual IP addresses which are configured on this router. But the router must respond to the ARP request/ND solicitations for the virtual IP addresses. The existing VRRP design described two approaches to support the false accept_mode: a. For non-link-local virtual IP addresses, add the ARP/ND cache using the SIOCSXARP/SIOCLIFSETND ioctl in order to respond to the ARP request/ND solicitations. Note that these virtual IP addresses are not brought up on the VRRP interfaces. b. For link-local virtual IP address, "a new interface flag IFF_LL_NOACCEPT is introduced to mark a VNIC as non-accept mode. On receiving ioctl SIOCSLIFFLAGS request for IFF_LL_NOACCEPT, the IRE entry for the link-local address of the interface will be marked as IRE_MARK_NOACCEPT. If an ire which is looked up for a link local destination address turns out to have this flag marked, the input packets will be dropped in ip_rput_data_v6()". The above approaches do not work. Note that with approach (a), no IP address is brought up on the specific interface (in the IPv4 case), the ill of this interface will not be "bound" and the SIOCSXARP ioctl will simply fail. Furthermore, both approaches would drop all the unicast packets including the unicast Neighbor solicitation, which makes the Neighbor Unreachability Detection mechanism unusable. 2) Duplication of administrative interfaces and system services The administration model proposed by PSARC/2009/693 assumes that VRRP service has full management over the VRRP IP interfaces, the primary IP addresses and and the virtual IP addresses used by the specific VRRP router. The vrrpd daemon plumbs/unplumbs the VRRP interfaces and configures/unconfigures the associated IP addresses based on the current state of the VRRP router. But the design does not consider the interaction with the other existing IP interfaces configuration tools (network/physical service, ifconfig etc.), and the potential impacts of such interaction is unkown. More important, since the existing design assumes the entire control over the VRRP interface IP addresses, it makes the vrrpadm another administrative tool to configure IP addresses, which will never be as flexible as the existing tool. E.g., do we need to extend vrrpadm to configure one specific IP address to be the "preferred" address? Likewise, since the vrrpd internally create/destroy the special VRRP VNICs, we will lose the flexibilities provided by the existing "dladm" command and the feature it provides (flows, bandwidth, priority etc.). 3) No exclusive-zone support With the old VRRP PSARC case, the administrator specifies the name of the interface that will be managed by VRRP, and vrrpd would internally create and plumb a special VRRP VNIC over that interface and configure virtual IP addresses over that VNIC. This makes the VRRP support in an exclusive-zone problematic, since creating VNIC in an non-global zone is not supported. 4) No VLAN support The same design makes VRRP support over VLAN problematic as well, since the current VRRP design blindly try to create VNIC over the specified interface, and creating VNIC over an VLAN is not allowed either. Proposal Overview ================= - vrrpadm changes 1) Add "-router" to each vrrpadm subcommand Following the precedent of "dladm", all the vrrpadm subcommands will contain the object of the operation - the "router". For example, "vrrpadm create" will be changed to "vrrpadm create-router", and "vrrpadm show" will be changed to "vrrpadm show-router". 2) Change the "startup/shutdown" subcommands to "enable-router/disable-router" "startup" sounds like something that needs to be done on each reboot, which is not the case. - MAC_CAPAB_VRRP, DL_CAPAB_VRRP, IFF_VRRP and SO_VRRP For each VRRP router, a special VRRP VNIC is created with the special VRRP virtual MAC address. All the IP addresses reside on this VNIC are regarded as virtual IP addresses protected by the VRRP router. The vrrpd daemon brings up those addresses when the router becomes master and brings down the addresses when the router becomes backup. In other words, vrrpd has full management of the up/down state of the virtual IP addresses and no other applications and services are allowed to change the up/down state of those IP addresses. A new mac capability MAC_CAPAB_VRRP will be introduced and the special VRRP VNICs will have such capability. The VNICs will then advertise a new DL_CAPAB_VRRP DLPI capability as part of the DL_CAPABILITY_REQ/ACK negotiation with IP. IP will learn that the corresponding ill is VRRP capable and mark each IP addresses configured over such ill with a IFF_VRRP flag, to indicate that is a VRRP virtual IP address. To make VRRP service the only authorization to bring up and down the virtual IP addresses, a new SO_VRRP socket option will be introduced. Socket has the SO_VRRP socket option set is a VRRP control socket, and only VRRP control sockets are allowed to change the IFF_UP flag of a VRRP virtual IP address (with IFF_VRRP set). Other attempts to change the IFF_UP flag will fail. The priv_sys_ip_config privilege is required to set the SO_VRRP socket option. For now, the vrrpd daemon will be the only application to set the SO_VRRP socket option on the socket changing the IFF_UP flag of the virtual IP addresses, in order to manage their up/down state based on the state of the VRRP router. - IFF_NOACCEPT When a VRRP router becomes master, if its accept_mode is false, all the virtual IP addresses will be brought up but with the new IFF_NOACCEPT flag set by vrrpd. IP will mark all the local IREs associated with these IFF_NOACCEPT IP addresses to be "no_accept", and all the received unicast local packets will be dropped, with the exception of the Neighbor Solicitation packets and Neighbor Advertisement packets. This allows the Neighbor Unreachability Detection mechanism work as expected in the false accept_mode. The IFF_NOACCEPT flag can only be set on a IP addresses if its IFF_VRRP flag is set. Further, the same SO_VRRP socket option must be set to change the IFF_NOACCEPT flag over a IP address. - Interact with the existing data-link and IP administrative tools To address issue 2, VRRP configuration will be integrated with the existing IP administrative model seamlessly: the existing data-link and IP administrative tools and service will be able to be used to create/delete the VRRP special VNICs, plumb/unplumb the physical interfaces (which own the primary IP address) and VRRP VNICs (which own the virtual IP addresses), and configure the primary and virtual IP addresses needed by a VRRP router. The vrrpadm command will still be needed to create/delete a VRRP router, and configure the primary arguments required by the VRRP protocol: the VRID, the address family, the interface over which the VRRP router is created on, the priority, the advertisement interval etc. The "dladm create-vnic" subcommand will be extended to create the VRRP special VNICs where the virtual IP addresses reside on. A new mac address keyword "vrrp" will be introduced and will be used to create VRRP VNICs[2]. A new vnic_mac_addr_type_t VNIC_MAC_ADDR_TYPE_VRID will be added and the dladm_vnic_create() API will be extended accordingly: dladm_status_t dladm_vnic_create(dladm_handle_t handle, const char *vnic, datalink_id_t linkid, vnic_mac_addr_type_t type, uchar_t * mac_addr, uint_t mac_len, int *mac_slot, uint_t mac_prefix_len, uint16_t vid, vrid_t vrid, int af, datalink_id_t *vnic_id_out, dladm_arg_list_t *proplist, uint32_t flags); To track the VRRP virtual IP addresses, the vrrpd daemon will determine the VRRP special VNIC used by a specific VRRP router based on the VRID, the IP address family (IPv4 or IPv6) and the physical interface (including VLANs and aggregations) the router is created on, and regard all the IP addresses configured over the VNIC as the virtual IP addresses associated with this VRRP router. Since the existing PF_ROUTE event opcodes only report the changes of the set of "UP" IP addresses, and the vrrpd daemon needs to track all the virtual IP addresses configured over each VRRP router, regardless the IP address is brought up or not. Therefore, two new PF_ROUTE event opcodes will be introduced to report the changes of the IP addresses configuration, including the IP addresses that have not been brought up: - RTM_CHGADDR The RTM_CHGADDR event will be generated when a new IP address is newly configured (added or updated to). - RTM_FREEADDR The RTM_FREEADDR event will be generated when a IP address is removed from the configuration. The message format of the above routing socket events will be the same as the format of the RTM_NEWADDR/RTM_DELADDR messages. Note that both events will not report the unspecified (all-zero) IP addresses. Further, vrrpd will also track all the "UP" IP addresses configured over the physical interface, and select one as the primary IP address which will be used to send the VRRP advertisement. Below gives an example which creates an IPv4 VRRP router (vrrp1) with VRID 12, priority 100 and false accept_mode over the bge1 data-link. The virtual IP addresses are 11.1.1.1/24, and the primary IP address in the VRRP advertisement is 11.1.1.100. Note that the IP addresses in this example could be configured by any other IP configuration tools/services other than "ifconfig". # dladm create-vnic -m vrrp -V 12 -A inet -l bge1 vrrp_vnic1 # vrrpadm create-router -V 12 -l bge1 -A inet -p 100 -o no_accept vrrp1 # ifconfig vrrp_vnic1 plumb 11.1.1.1/24 # ifconfig bge1 plumb 11.1.1.100/24 up Since in some cases, the VRRP configuration is pretty simple and the administrator would prefer to have one single administrative tool to configure everything related to a VRRP router. Therefore, we give the option to an administrator to do that using only vrrpadm: the optional "-a" option can be used to specify a list of virtual IP addresses when creating a VRRP router, the "-P" option can be used to specify the primary IP address that is used to send VRRP advertisement packets. Further, if the "-f" option is specified, vrrpadm will create/plumb the VNIC if that has not been done: # vrrpadm create-router -V 12 -l bge1 -A inet -p 100 -o no_accept \ -a 11.1.1.1/24 -P 11.1.1.100/24 -f vrrp1 In this case, vrrpadm will create and plumb the VNIC, configure the virtual IP address over it and bring up the primary IP adddress over bge1. From this point on, the system will behave exactly the same as if the VNIC and and virtual IP addresses were configured by the other tools/services. Note that the administrator has to unconfigure all the virtual IP addresses and delete the VNIC using other tools to completely cleanup the VRRP configuration. More details of the vrrpadm and dladm configuration changed are discussed in [1] and [2]. - VLAN support The administrative model described above allows the administrator to configure the VRRP router over a VLAN. Note that the VRRP special VNIC has to be created using the "-v" option, which specifies the VLAN ID. For example: # dladm create-vnic -m vrrp -V 14 -A inet -l bge1 -v 2 vrrp_vnic2 # dladm create-vlan -l bge1 -v 2 vlan1 # vrrpadm create-router -V 14 -l vlan1 -A inet -p 100 -o no_accept vrrp2 # ifconfig vrrp_vnic2 plumb 12.1.1.1/24 # ifconfig vlan1 plumb 12.1.1.100/24 up - Exclusive zone support In the new design, VNICs will no longer be created internally by the vrrpd daemon, instead, one can create the VRRP special VNIC in the global-zone and assign the VNIC to the non-global zone where the VRRP router is configured. The VRRP (network/vrrp/default) service will be started in the non-global and it will start the vrrpd daemon. - Other miscellaneous changes * Least privileges of the vrrpd daemon In the VRRP SMF manifest, the vrrpd will be set to be run by the "root" user, and its privilege property will be set to only include the "basic" privilege and the following privileges: - priv_sys_config Required to post VRRP sysevents. Note that this privilege is only needed in the global zone since sysevents are not supported in the non-global zone. - priv_net_rawaccess Required to hold the physical data-link (which owns the primary IP address) and the vnic (which owns the virtual IP addresses) open to prevent them from being deleted. - priv_net_icmpaccess Required to open the RAW socket - priv_sys_ip_config Required to bring up/down the virtual IP addresses and set the SO_VRRP socket option * solaris.network.vrrp authorization A new solaris.network.vrrp authorization will be introduced and will be required to configure the VRRP service. Note that it will only be needed by the "write" operation but not the "read-only" operation (e.g., "vrrpadm show"). The solaris.network.vrrp authorization will be added to the "Network Management" profile. * Interaction with the existing IP address autoconfiguration tools in.ndpd (IPv6 autoconfiguration) and dhcpagent (DHCP client) are the two major IP address autoconfiguration tools exist in Solaris. Because the master and the backup VRRP routers (VNICs) share the same mac address, this will simply confuses in.ndpd and dhcpagent and eventually cause unexpected results. Therefore, IPv6 autoconfiguration and DHCP configuration will not be supported over VRRP VNICs. If an administrator configures either IPv6 autoconfiguration or DHCP over a VRRP VNIC, since neither in.ndpd or dhcp client sets the SO_VRRP socket option, the attempt to bring up the auto-configured IP address will fail, which causes the failure of the autoconfiguration operation. An ongoing project (ipadm) will make static IPv6 address configuration and IPv6 autoconfiguration exclusive to each other on a specific IPv6 interface. Since VRRP configuration requires static IP address configuration, it will prevent in.ndpd from trying to autoconfigure over a VRRP VNIC from the beginning. * Service dependencies The vrrpd daemon is started by the svc:/network/vrrp:default service, which depends on the svc:/network/physical service and the svc:/system/filesystem/usr service. The former service is required to configure the data-links and the IP addresses needed by the specific VRRP router, and the latter service is required since the vrrpd, vrrpadm and libvrrpadm.so.1 binaries reside under the /usr directory. * Removing the protected service support In the old VRRP design, there is a notion of protected services: an administrator can specify a set of SMF services to be protected by a VRRP router. To support that, vrrpd tracks the state of those SMF services and when vrrpd finds the service being offline/disabled, vrrpd changes the state of the VRRP router so the router will no longer be the master. Another backup router becomes the master and the SMF service can be started there. After discussion with the team that potentially would consume this feature (the ILB team), we found that this is an incomplete solution which does not simplify the management complexity. Ideally, the HA support configuration solution would be: configuring the following on the owner of the virtual IP addresses: - the virtual IP address(es) - the VRRP VRID - the IP addresses of the backup(s) - the ILB server groups, rules, hc, etc - some security configiguration (ssh keys) that allow automatic synchronization of things with the backups and all the other configurations and sync up between master and backup should be automatic. Clearly, the protected service support proposed by the old VRRP design does not help with the ideal world, therefore, we decide to remove the protected service support. Interface Table =============== Interface Classification Comments ================== ============== ======== IFF_NOACCEPT Committed no_accept VRRP mode IFF_VRRP Committed VRRP virtual IP address SO_VRRP Committed VRRP control socket option RTM_CHGADDR Committed New PF_ROUTE event RTM_FREEADDR Committed New PF_ROUTE event solaris.network.vrrp Committed authorization required by authorization VRRP configuration /usr/sbin/vrrpadm Committed VRRP administration tool vrrpadm show-router output Uncommitted dladm create-vnic -m vrrp Committed create VRRP vnics dladm_vnic_create() Consolidation Private VNIC_MAC_ADDR_TYPE_VRID Consolidation New vnic_mac_addr_type_t Private MAC_CAPAB_VRRP Project Private MAC layer capability DL_CAPBB_VRRP Project Private DLPI capability svc:/network/vrrp:default Project Private The service that starts vrrpd /usr/sbin/vrrpd Project Private VRRP daemon /usr/lib/libvrrpadm.so Project Private VRRP library /etc/vrrp.conf Project Private References ========== [1] VRRP design specification [2] vrrpadm(1M), vrrpd(1M), dladm(1M) manpages From carlsonj@workingcode.com Sat Jul 11 14:24:56 2009 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6BLOtG0019623 for ; Sat, 11 Jul 2009 14:24:55 -0700 (PDT) Received: from brmea-mail-1.sun.com (brmea-mail-1.Sun.COM [192.18.98.31]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6BLOtK1014464 for ; Sat, 11 Jul 2009 14:24:55 -0700 (PDT) Received: from relay44i.sun.com ([192.5.209.118]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6BLKpXS022504 for ; Sat, 11 Jul 2009 21:24:55 GMT Received: from mmp42es.mmp.us.syntegra.com ([160.41.221.11] [160.41.221.11]) by relay44i.sun.com with ESMTP id BT-MMP-336689 for psarc-ext@sac.sfbay.sun.com; Sat, 11 Jul 2009 21:24:55 Z Received: from relay43i.sun.com (relay43i.sun.com [192.5.209.74]) by mmp42es.mmp.us.syntegra.com with ESMTP id BT-MMP-80625913; Sat, 11 Jul 2009 21:24:54 Z Received: from carlson.workingcode.com ([75.150.68.97] [75.150.68.97]) by relay4i.sun.com with ESMTP id BT-MMP-2126875; Sat, 11 Jul 2009 21:24:54 Z Received: from [192.168.254.177] (dhcp-177 [192.168.254.177]) (authenticated bits=0) by carlson.workingcode.com (8.14.2+Sun/8.14.3) with ESMTP id n6BLOrSh027340 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 11 Jul 2009 17:24:53 -0400 (EDT) Message-ID: <4A590324.8050908@workingcode.com> Date: Sat, 11 Jul 2009 17:24:52 -0400 From: James Carlson User-Agent: Thunderbird 2.0.0.21 (X11/20090323) To: Sebastien Roy CC: psarc-ext , Cathy Zhou Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/17/2009] References: <1247251240.10474.96.camel@strat> In-Reply-To: <1247251240.10474.96.camel@strat> X-Brightmail-Tracker: AAAAAA== X-DCC-dmv.com-Metrics: carlson 1181; Body=3 Fuz1=3 Fuz2=3 X-Antispam: No, score=-2.6/5.0, scanned in 0.138sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Status: RO Content-Length: 7019 Sebastien Roy wrote: > I'm submitting this fast-track for Cathy Zhou, it times out on > 07/17/2009. This case depends on PSARC/2008/693. 2008/693 requested > "micro" release binding, but no incompatible changes are introduced, and > "patch" would be more appropriate (although no backport is planned). As > such, this case both requests "patch" binding, and updates 2008/693 to > have "patch" binding as well. First of all, "yay!" The new design looks a lot better. I'm not sure, though, if this is really a fast-track. There's a lot of subtlety here, and it's in some areas (interface flags, routing sockets) where we've managed to make some complicated and sometimes "wrong" decisions in the past, and where there are interoperability concerns. > - IFF_NOACCEPT > > When a VRRP router becomes master, if its accept_mode is false, all > the virtual IP addresses will be brought up but with the new > IFF_NOACCEPT flag set by vrrpd. IP will mark all the local IREs > associated with these IFF_NOACCEPT IP addresses to be "no_accept", > and all the received unicast local packets will be dropped, with > the exception of the Neighbor Solicitation packets and Neighbor > Advertisement packets. This allows the Neighbor Unreachability > Detection mechanism work as expected in the false accept_mode. This seems to describe IFF_NOACCEPT as a per-ipif flag. Is that the right granularity? It seems odd to me that we could potentially configure the system to have some addresses that respond and some that do not. The point of "no accept" mode is to behave as a real VRRP system. If you want that, I don't think you'd want it just some of the time, would you? > The IFF_NOACCEPT flag can only be set on a IP addresses if its > IFF_VRRP flag is set. Further, the same SO_VRRP socket option must > be set to change the IFF_NOACCEPT flag over a IP address. What's the rationale for requiring SO_VRRP? Nothing else knows about IFF_NOACCEPT, so (unlike IFF_UP) it can't be set "accidentally" by anyone else. (And you clearly can't *stop* appropriately privileged applications from setting SO_VRRP on their own.) A separate question on the IFF_UP limitations: I'm not sure why that's being controlled by the VRRP daemon. Why not allow IFF_UP to be the normal administrative control on the address, and just have the VRRP daemon control whether the address is "live" on the network (i.e., turning on and off ARP and ND for the address, as well as controlling input and output)? This proposal looks workable, but it still looks a little more complex to administer than I'd expected. > Since the existing PF_ROUTE event opcodes only report the changes > of the set of "UP" IP addresses, and the vrrpd daemon needs to > track all the virtual IP addresses configured over each VRRP router, > regardless the IP address is brought up or not. Therefore, two > new PF_ROUTE event opcodes will be introduced to report the changes > of the IP addresses configuration, including the IP addresses that > have not been brought up: > > - RTM_CHGADDR > > The RTM_CHGADDR event will be generated when a new IP address > is newly configured (added or updated to). > > - RTM_FREEADDR > > The RTM_FREEADDR event will be generated when a IP address is > removed from the configuration. These don't look like messages on other systems. Are there any parallels with existing BSD extensions? (I assume that "configure" means "plumb an ipif" here.) > Note that both events will not report the unspecified (all-zero) > IP addresses. I'm not sure what that means. Why would those need to be special? Or is that just an implementation limitation? Aren't all ipifs constructed with zero address when first plumbed ... ? > Since in some cases, the VRRP configuration is pretty simple and > the administrator would prefer to have one single administrative > tool to configure everything related to a VRRP router. Therefore, > we give the option to an administrator to do that using only > vrrpadm: the optional "-a" option can be used to specify a list > of virtual IP addresses when creating a VRRP router, the "-P" > option can be used to specify the primary IP address that > is used to send VRRP advertisement packets. Further, if the "-f" > option is specified, vrrpadm will create/plumb the VNIC if that > has not been done: > > # vrrpadm create-router -V 12 -l bge1 -A inet -p 100 -o no_accept \ > -a 11.1.1.1/24 -P 11.1.1.100/24 -f vrrp1 When the user does this, where is the information stored? Does "create-router" write the address into /etc/hostname.*, and so on? > - priv_net_rawaccess > > Required to hold the physical data-link (which owns the > primary IP address) and the vnic (which owns the virtual > IP addresses) open to prevent them from being deleted. Why is that necessary? > If an administrator configures either IPv6 autoconfiguration or > DHCP over a VRRP VNIC, since neither in.ndpd or dhcp client sets > the SO_VRRP socket option, the attempt to bring up the > auto-configured IP address will fail, which causes the failure > of the autoconfiguration operation. Shouldn't these things be modified to check for IFF_VRRP and emit reasonable messages and/or just avoid touching them? Relying on SO_VRRP and ioctl failure seems reasonable for protection from third party applications, but I think our own applications should probably do better. > After discussion with the team that potentially would consume > this feature (the ILB team), Note that Quagga routing could consume this feature as well. (Perhaps even sooner than ILB, as we've already got Quagga ...) > we found that this is an > incomplete solution which does not simplify the management > complexity. Ideally, the HA support configuration solution > would be: > > configuring the following on the owner of the virtual IP addresses: > - the virtual IP address(es) > - the VRRP VRID > - the IP addresses of the backup(s) > - the ILB server groups, rules, hc, etc > - some security configiguration (ssh keys) that allow > automatic synchronization of things with the backups > > and all the other configurations and sync up between master and > backup should be automatic. > > Clearly, the protected service support proposed by the old VRRP > design does not help with the ideal world, therefore, we decide > to remove the protected service support. That seems reasonable. Is there some way that SMF or Quagga could be configured to signal VRRP when a routing daemon fails? > /usr/sbin/vrrpd Project Private VRRP daemon Nit: this probably belongs in /usr/lib instead. No user or administrator should be starting VRRP on his own, should he? -- James Carlson 42.703N 71.076W From Cathy.Zhou@Sun.COM Mon Jul 13 11:35:50 2009 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6DIZox1026582 for ; Mon, 13 Jul 2009 11:35:50 -0700 (PDT) Received: from sineb-mail-2.sun.com ([192.18.19.7]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6DIZkWf030811 for ; Mon, 13 Jul 2009 11:35:47 -0700 (PDT) Received: from fe-apac-05.sun.com (fe-apac-05.sun.com [192.18.19.176] (may be forged)) by sineb-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6DIZff8002800 for ; Mon, 13 Jul 2009 18:35:41 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMQ00J00GPE7Z00@mail-apac.sun.com> for psarc-ext@sac.sfbay.sun.com; Tue, 14 Jul 2009 02:35:41 +0800 (SGT) Received: from [129.146.108.16] ([unknown] [129.146.108.16]) by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMQ008YBGZFCHJ0@mail-apac.sun.com> for psarc-ext@sac.sfbay.sun.com; Tue, 14 Jul 2009 02:35:41 +0800 (SGT) Date: Mon, 13 Jul 2009 11:33:53 -0700 From: Cathy Zhou Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/17/2009] In-reply-to: <4A590324.8050908@workingcode.com> Sender: Cathy.Zhou@Sun.COM To: James Carlson Cc: Sebastien Roy , psarc-ext Message-id: <4A5B7E11.3000103@sun.com> References: <1247251240.10474.96.camel@strat> <4A590324.8050908@workingcode.com> User-Agent: Thunderbird 2.0.0.12 (X11/20080422) Status: RO Content-Length: 8871 James, Thanks for your comments. See inline: > >> - IFF_NOACCEPT >> >> When a VRRP router becomes master, if its accept_mode is false, all >> the virtual IP addresses will be brought up but with the new >> IFF_NOACCEPT flag set by vrrpd. IP will mark all the local IREs >> associated with these IFF_NOACCEPT IP addresses to be "no_accept", >> and all the received unicast local packets will be dropped, with >> the exception of the Neighbor Solicitation packets and Neighbor >> Advertisement packets. This allows the Neighbor Unreachability >> Detection mechanism work as expected in the false accept_mode. > > This seems to describe IFF_NOACCEPT as a per-ipif flag. Is that the > right granularity? It seems odd to me that we could potentially > configure the system to have some addresses that respond and some that > do not. The point of "no accept" mode is to behave as a real VRRP > system. If you want that, I don't think you'd want it just some of > the time, would you? You are right. I can add a per-ill flag. But since I would also like this flag to be observable by the administrator, I will keep the IPIF_NOACCEPT flag so that ifconfig bge0:1 will be able to show this address is set to be "no_accept". > >> The IFF_NOACCEPT flag can only be set on a IP addresses if its >> IFF_VRRP flag is set. Further, the same SO_VRRP socket option must >> be set to change the IFF_NOACCEPT flag over a IP address. > > What's the rationale for requiring SO_VRRP? Nothing else knows about > IFF_NOACCEPT, so (unlike IFF_UP) it can't be set "accidentally" by > anyone else. (And you clearly can't *stop* appropriately privileged > applications from setting SO_VRRP on their own.) First, an application could potentially make a mistake by not getting ifflag before setting ifflag, which could potentially clear IFF_NOACCEPT. Second, this is also related to setting/clearing the IFF_UP flag over an VRRP VNIC: a non-accept VRRP router usually sets IFF_UP and IFF_NOACCEPT using the same socket when it becomes master, having SO_VRRP is to make the socket setting both flags to becomes a VRRP control socket session. > A separate question on the IFF_UP limitations: I'm not sure why that's > being controlled by the VRRP daemon. Why not allow IFF_UP to be the > normal administrative control on the address, and just have the VRRP > daemon control whether the address is "live" on the network (i.e., > turning on and off ARP and ND for the address, as well as controlling > input and output)? > > This proposal looks workable, but it still looks a little more complex > to administer than I'd expected. You mean another flag to control the liveness of the network? We thought about that option, and eventually we choose today's approach is that there is ambiguity of the "IFF_UP" flag. It can be seens as an administrative flag, but some application also believes it shows the current state of the address: once it is UP, the address should be usable. > >> Since the existing PF_ROUTE event opcodes only report the changes >> of the set of "UP" IP addresses, and the vrrpd daemon needs to >> track all the virtual IP addresses configured over each VRRP router, >> regardless the IP address is brought up or not. Therefore, two >> new PF_ROUTE event opcodes will be introduced to report the changes >> of the IP addresses configuration, including the IP addresses that >> have not been brought up: >> >> - RTM_CHGADDR >> >> The RTM_CHGADDR event will be generated when a new IP address >> is newly configured (added or updated to). >> >> - RTM_FREEADDR >> >> The RTM_FREEADDR event will be generated when a IP address is >> removed from the configuration. > > These don't look like messages on other systems. Are there any > parallels with existing BSD extensions? I looked but didn't see things similar to it. BSD does provide a IF_ANNOUNCE message to indicate the plumbing/unplumbing of the interface but that cannot satisfy my needs. > > (I assume that "configure" means "plumb an ipif" here.) Right. > >> Note that both events will not report the unspecified (all-zero) >> IP addresses. > > I'm not sure what that means. Why would those need to be special? Or > is that just an implementation limitation? > > Aren't all ipifs constructed with zero address when first plumbed ... ? It is related to implementation. Usually, the first zero-address is the result of the SIOCLIFADDIF, but the IP address is set as the result of the SIOCSLIFADDR. We only generate the RTM_CHGADDR as a result of the second operation. Also, I can imagine that application could care about the plumbing event of the interface, but I assume that no application would care about the first zero-IP addresses, to me, that zero-address is a side effect of our implementation. > >> Since in some cases, the VRRP configuration is pretty simple and >> the administrator would prefer to have one single administrative >> tool to configure everything related to a VRRP router. Therefore, >> we give the option to an administrator to do that using only >> vrrpadm: the optional "-a" option can be used to specify a list >> of virtual IP addresses when creating a VRRP router, the "-P" >> option can be used to specify the primary IP address that >> is used to send VRRP advertisement packets. Further, if the "-f" >> option is specified, vrrpadm will create/plumb the VNIC if that >> has not been done: >> >> # vrrpadm create-router -V 12 -l bge1 -A inet -p 100 -o no_accept \ >> -a 11.1.1.1/24 -P 11.1.1.100/24 -f vrrp1 > > When the user does this, where is the information stored? Does > "create-router" write the address into /etc/hostname.*, and so on? The vrrp configuration stores in /etc/vrrp.conf, and we will make use of libipadm to do the ip configuration persistence. > >> - priv_net_rawaccess >> >> Required to hold the physical data-link (which owns the >> primary IP address) and the vnic (which owns the virtual >> IP addresses) open to prevent them from being deleted. > > Why is that necessary? To prevent the data-link from being deleted or renamed. Note that we don't require the underlying interface and VNIC to be plumbed when the router is enabled. Instead, the PF_ROUTE socket will track all the IP addresses configured over the underlying interface and VNIC and determine the primary IP address and virtual IP addresses. > >> If an administrator configures either IPv6 autoconfiguration or >> DHCP over a VRRP VNIC, since neither in.ndpd or dhcp client sets >> the SO_VRRP socket option, the attempt to bring up the >> auto-configured IP address will fail, which causes the failure >> of the autoconfiguration operation. > > Shouldn't these things be modified to check for IFF_VRRP and emit > reasonable messages and/or just avoid touching them? Sure. I will see what I can do. > > Relying on SO_VRRP and ioctl failure seems reasonable for protection > from third party applications, but I think our own applications should > probably do better. > >> After discussion with the team that potentially would consume >> this feature (the ILB team), > > Note that Quagga routing could consume this feature as well. (Perhaps > even sooner than ILB, as we've already got Quagga ...) > > > we found that this is an >> incomplete solution which does not simplify the management >> complexity. Ideally, the HA support configuration solution >> would be: >> configuring the following on the owner of the virtual IP >> addresses: - the virtual IP address(es) >> - the VRRP VRID >> - the IP addresses of the backup(s) >> - the ILB server groups, rules, hc, etc >> - some security configiguration (ssh keys) that allow >> automatic synchronization of things with the backups >> >> and all the other configurations and sync up between master and >> backup should be automatic. >> >> Clearly, the protected service support proposed by the old VRRP >> design does not help with the ideal world, therefore, we decide >> to remove the protected service support. > > That seems reasonable. Is there some way that SMF or Quagga could be > configured to signal VRRP when a routing daemon fails? VRRP also needs to know that which virtual addresses those services are using and find out the specific VRRP router. In theory, I think that can be done and can be a useful feature, but this will be out of the scope of the current VRRP/ILB project. > >> /usr/sbin/vrrpd Project Private VRRP daemon > > Nit: this probably belongs in /usr/lib instead. No user or > administrator should be starting VRRP on his own, should he? Sure. I can change that. Thanks - Cathy From carlsonj@workingcode.com Tue Jul 14 05:56:40 2009 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6ECuc1U017847 for ; Tue, 14 Jul 2009 05:56:40 -0700 (PDT) Received: from brmea-mail-2.sun.com (brmea-mail-2.Sun.COM [192.18.98.43]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6ECubxJ039603 for ; Tue, 14 Jul 2009 05:56:37 -0700 (PDT) Received: from relay15i.sun.com (ip125.net129179-4.block1.us.syntegra.com [129.179.4.125]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6ECsQer011402 for ; Tue, 14 Jul 2009 12:56:37 GMT Received: from mmp14es.mmp.us.syntegra.com ([160.41.208.14] [160.41.208.14]) by relay15i.sun.com with ESMTP id BT-MMP-7598 for psarc-ext@sac.sfbay.sun.com; Tue, 14 Jul 2009 12:56:37 Z Received: from relay11i.sun.com (relay11i.sun.com [129.179.4.121]) by mmp14es.mmp.us.syntegra.com with ESMTP id BT-MMP-523310; Tue, 14 Jul 2009 12:56:36 Z Received: from carlson.workingcode.com ([75.150.68.97] [75.150.68.97]) by relay1i.sun.com with ESMTP id BT-MMP-819607; Tue, 14 Jul 2009 12:56:35 Z Received: from [10.50.24.188] (gate.abinitio.com [65.170.40.132]) (authenticated bits=0) by carlson.workingcode.com (8.14.2+Sun/8.14.3) with ESMTP id n6ECuYEA004683 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 14 Jul 2009 08:56:35 -0400 (EDT) Message-ID: <4A5C8082.9040508@workingcode.com> Date: Tue, 14 Jul 2009 08:56:34 -0400 From: James Carlson User-Agent: Thunderbird 2.0.0.22 (X11/20090605) To: Cathy Zhou CC: Sebastien Roy , psarc-ext Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/17/2009] References: <1247251240.10474.96.camel@strat> <4A590324.8050908@workingcode.com> <4A5B7E11.3000103@sun.com> In-Reply-To: <4A5B7E11.3000103@sun.com> X-Brightmail-Tracker: AAAAAA== X-DCC-dmv.com-Metrics: carlson 1181; Body=3 Fuz1=3 Fuz2=3 X-Antispam: No, score=-2.6/5.0, scanned in 0.128sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Status: RO Content-Length: 7373 Cathy Zhou wrote: >> This seems to describe IFF_NOACCEPT as a per-ipif flag. Is that the >> right granularity? It seems odd to me that we could potentially >> configure the system to have some addresses that respond and some that >> do not. The point of "no accept" mode is to behave as a real VRRP >> system. If you want that, I don't think you'd want it just some of >> the time, would you? > You are right. I can add a per-ill flag. But since I would also like > this flag to be observable by the administrator, I will keep the > IPIF_NOACCEPT flag so that ifconfig bge0:1 will be able to show this > address is set to be "no_accept". I don't understand. The IFF_* flag space is subdivided into flags on ipifs (IPIF_*), flags on ills (ILLF_*), and flags on phyints (PHYINT_*). Those are all OR'd together when delivered to user space. Thus, at least for observability, it does not matter where you put the flag. What does matter is that you choose the right granularity based on what the flag is supposed to report and/or control. If it's something that describes the behavior of the ill, then it should be an ILLF_* flag. If it's something that needs to be set separately for each address on the ill, then it needs to be an IPIF_* flag. Having an ill flag and "keeping" the IPIF_NOACCEPT definition sounds like a syntax error to me. (I don't honestly know whether it needs to be on the ill or the ipif. My guess is ill, but I brought up the question because it didn't seem to be stated clearly in the materials.) >>> The IFF_NOACCEPT flag can only be set on a IP addresses if its >>> IFF_VRRP flag is set. Further, the same SO_VRRP socket option must >>> be set to change the IFF_NOACCEPT flag over a IP address. >> >> What's the rationale for requiring SO_VRRP? Nothing else knows about >> IFF_NOACCEPT, so (unlike IFF_UP) it can't be set "accidentally" by >> anyone else. (And you clearly can't *stop* appropriately privileged >> applications from setting SO_VRRP on their own.) > First, an application could potentially make a mistake by not getting > ifflag before setting ifflag, which could potentially clear > IFF_NOACCEPT. Applications can do that with any flag, and we haven't needed to go to that sort of protection mechanism in the past. Moreover, as a new flag, this would likely be in the upper 32 bits. That means that the standard BSD ioctls can't touch it at all. You need the Solaris-specific SIOC[GS]LIF* family instead. That issue alone ropes off a large population of potentially troublesome applications. (It just seems odd to me. Why is there no SO_IPMP wrapper around IFF_IPMP or SO_DHCP around IFF_DHCPRUNNING?) > Second, this is also related to setting/clearing the > IFF_UP flag over an VRRP VNIC: a non-accept VRRP router usually sets > IFF_UP and IFF_NOACCEPT using the same socket when it becomes master, > having SO_VRRP is to make the socket setting both flags to becomes a > VRRP control socket session. It feels like too much complication to me, and very much unlike all the other cases where we've had to deal with subsystems that manage IP interfaces ... but I'm not going to argue it further. >> A separate question on the IFF_UP limitations: I'm not sure why that's >> being controlled by the VRRP daemon. Why not allow IFF_UP to be the >> normal administrative control on the address, and just have the VRRP >> daemon control whether the address is "live" on the network (i.e., >> turning on and off ARP and ND for the address, as well as controlling >> input and output)? >> >> This proposal looks workable, but it still looks a little more complex >> to administer than I'd expected. > You mean another flag to control the liveness of the network? We thought > about that option, and eventually we choose today's approach is that > there is ambiguity of the "IFF_UP" flag. It can be seens as an > administrative flag, but some application also believes it shows the > current state of the address: once it is UP, the address should be usable. Yes. My argument, though, is that it doesn't actually matter if the address is marked "UP" but (for reasons outside the application's control) is not currently usable on the wire. At most, lack of network connection is a reason to avoid that address when doing source address selection -- but, then, you almost certainly should be avoiding these VRRP-controlled addresses when doing source address selection for outbound connections anyway. >>> The RTM_FREEADDR event will be generated when a IP address is >>> removed from the configuration. >> >> These don't look like messages on other systems. Are there any >> parallels with existing BSD extensions? > I looked but didn't see things similar to it. BSD does provide a > IF_ANNOUNCE message to indicate the plumbing/unplumbing of the interface > but that cannot satisfy my needs. It'd probably help to describe that message and why it doesn't provide the information needed. (In fact, that message is exactly what I'd expected was needed ...) >>> Note that both events will not report the unspecified (all-zero) >>> IP addresses. >> >> I'm not sure what that means. Why would those need to be special? Or >> is that just an implementation limitation? >> >> Aren't all ipifs constructed with zero address when first plumbed ... ? > It is related to implementation. Usually, the first zero-address is the > result of the SIOCLIFADDIF, but the IP address is set as the result of > the SIOCSLIFADDR. We only generate the RTM_CHGADDR as a result of the > second operation. > > Also, I can imagine that application could care about the plumbing event > of the interface, but I assume that no application would care about the > first zero-IP addresses, to me, that zero-address is a side effect of > our implementation. Hmm. OK. >>> # vrrpadm create-router -V 12 -l bge1 -A inet -p 100 -o no_accept \ >>> -a 11.1.1.1/24 -P 11.1.1.100/24 -f vrrp1 >> >> When the user does this, where is the information stored? Does >> "create-router" write the address into /etc/hostname.*, and so on? > The vrrp configuration stores in /etc/vrrp.conf, and we will make use of > libipadm to do the ip configuration persistence. Ah, ok. That latter part is what I was after. >>> - priv_net_rawaccess >>> >>> Required to hold the physical data-link (which owns the >>> primary IP address) and the vnic (which owns the virtual >>> IP addresses) open to prevent them from being deleted. >> >> Why is that necessary? > To prevent the data-link from being deleted or renamed. Note that we > don't require the underlying interface and VNIC to be plumbed when the > router is enabled. Instead, the PF_ROUTE socket will track all the IP > addresses configured over the underlying interface and VNIC and > determine the primary IP address and virtual IP addresses. Why do you care whether the underlying link is deleted or renamed? How does that work with DR? >> That seems reasonable. Is there some way that SMF or Quagga could be >> configured to signal VRRP when a routing daemon fails? > VRRP also needs to know that which virtual addresses those services are > using and find out the specific VRRP router. > > In theory, I think that can be done and can be a useful feature, but > this will be out of the scope of the current VRRP/ILB project. OK. From Cathy.Zhou@Sun.COM Tue Jul 14 11:04:54 2009 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6EI4sYs003073 for ; Tue, 14 Jul 2009 11:04:54 -0700 (PDT) Received: from sineb-mail-2.sun.com ([192.18.19.7]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6EI4pIK000300 for ; Tue, 14 Jul 2009 11:04:51 -0700 (PDT) Received: from fe-apac-06.sun.com (fe-apac-06.sun.com [192.18.19.177] (may be forged)) by sineb-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6EI4j87022081 for ; Tue, 14 Jul 2009 18:04:45 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMS00J00A1J1F00@mail-apac.sun.com> for psarc-ext@sac.sfbay.sun.com; Wed, 15 Jul 2009 02:04:45 +0800 (SGT) Received: from [129.146.108.16] ([unknown] [129.146.108.16]) by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMS001VKA7VHE70@mail-apac.sun.com> for psarc-ext@sac.sfbay.sun.com; Wed, 15 Jul 2009 02:04:45 +0800 (SGT) Date: Tue, 14 Jul 2009 11:02:57 -0700 From: Cathy Zhou Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/17/2009] In-reply-to: <4A5C8082.9040508@workingcode.com> Sender: Cathy.Zhou@Sun.COM To: James Carlson Cc: Sebastien Roy , psarc-ext Message-id: <4A5CC851.9010107@sun.com> References: <1247251240.10474.96.camel@strat> <4A590324.8050908@workingcode.com> <4A5B7E11.3000103@sun.com> <4A5C8082.9040508@workingcode.com> User-Agent: Thunderbird 2.0.0.12 (X11/20080422) Status: RO Content-Length: 3205 James Carlson wrote: > Cathy Zhou wrote: > >>> This seems to describe IFF_NOACCEPT as a per-ipif flag. Is that the >>> right granularity? It seems odd to me that we could potentially >>> configure the system to have some addresses that respond and some that >>> do not. The point of "no accept" mode is to behave as a real VRRP >>> system. If you want that, I don't think you'd want it just some of >>> the time, would you? >>> >> You are right. I can add a per-ill flag. But since I would also like >> this flag to be observable by the administrator, I will keep the >> IPIF_NOACCEPT flag so that ifconfig bge0:1 will be able to show this >> address is set to be "no_accept". >> > > I don't understand. > > The IFF_* flag space is subdivided into flags on ipifs (IPIF_*), flags > on ills (ILLF_*), and flags on phyints (PHYINT_*). Those are all OR'd > together when delivered to user space. > > Thus, at least for observability, it does not matter where you put the flag. > Sorry that I wasn't clear. I meant to keep IFF_NOACCEPT and change its definition to be: #define ILLF_NOACCEPT IFF_NOACCEPT > Yes. My argument, though, is that it doesn't actually matter if the > address is marked "UP" but (for reasons outside the application's > control) is not currently usable on the wire. At most, lack of network > connection is a reason to avoid that address when doing source address > selection -- but, then, you almost certainly should be avoiding these > VRRP-controlled addresses when doing source address selection for > outbound connections anyway. > Sorry, why cannot application use the virtual IP addresses as the source IP address, if the router is currently master? >> I looked but didn't see things similar to it. BSD does provide a >> IF_ANNOUNCE message to indicate the plumbing/unplumbing of the interface >> but that cannot satisfy my needs. >> > It'd probably help to describe that message and why it doesn't provide > the information needed. (In fact, that message is exactly what I'd > expected was needed ...) > As far as I know, the IFF_ANNOUCE only indicate the plumbing/unplumbing of the interface, not the address (the logical interface). That is why we cannot use it for our purpose. > >>>> - priv_net_rawaccess >>>> >> To prevent the data-link from being deleted or renamed. Note that we >> don't require the underlying interface and VNIC to be plumbed when the >> router is enabled. Instead, the PF_ROUTE socket will track all the IP >> addresses configured over the underlying interface and VNIC and >> determine the primary IP address and virtual IP addresses. >> > > Why do you care whether the underlying link is deleted or renamed? > > How does that work with DR? > When the VRRP router is first enabled, we check which VNIC is the associated VNIC and make sure it is created (otherwise, enable-router would fail). Then we listen to the PF_ROUTE messages to track all the IP addresses over this VNIC based on its interface name. If the VNICs is deleted or renamed in between, that would be a problem. Thanks for remind me of DR. I believe adding a router_rcm plugin is possible. Thanks - Cathy From carlsonj@workingcode.com Tue Jul 14 11:43:49 2009 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6EIhmZe004036 for ; Tue, 14 Jul 2009 11:43:48 -0700 (PDT) Received: from brmea-mail-4.sun.com (brmea-mail-4.Sun.COM [192.18.98.36]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6EIhmIx025599 for ; Tue, 14 Jul 2009 11:43:48 -0700 (PDT) Received: from relay11i.sun.com (ip121.net129179-4.block1.us.syntegra.com [129.179.4.121]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6EIfdEw015721 for ; Tue, 14 Jul 2009 18:43:48 GMT Received: from mmp11es.mmp.us.syntegra.com ([160.41.208.11] [160.41.208.11]) by relay11i.sun.com with ESMTP id BT-MMP-1349892 for psarc-ext@sac.sfbay.sun.com; Tue, 14 Jul 2009 18:43:48 Z Received: from relay13i.sun.com (relay13i.sun.com [129.179.4.123]) by mmp11es.mmp.us.syntegra.com with ESMTP id BT-MMP-741610; Tue, 14 Jul 2009 18:43:47 Z Received: from carlson.workingcode.com ([75.150.68.97] [75.150.68.97]) by relay1i.sun.com with ESMTP id BT-MMP-24340204; Tue, 14 Jul 2009 18:43:47 Z Received: from [10.50.24.188] (gate.abinitio.com [65.170.40.132]) (authenticated bits=0) by carlson.workingcode.com (8.14.2+Sun/8.14.3) with ESMTP id n6EIhZTc005615 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 14 Jul 2009 14:43:36 -0400 (EDT) Message-ID: <4A5CD1D7.4050104@workingcode.com> Date: Tue, 14 Jul 2009 14:43:35 -0400 From: James Carlson User-Agent: Thunderbird 2.0.0.22 (X11/20090605) To: Cathy Zhou CC: Sebastien Roy , psarc-ext Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/17/2009] References: <1247251240.10474.96.camel@strat> <4A590324.8050908@workingcode.com> <4A5B7E11.3000103@sun.com> <4A5C8082.9040508@workingcode.com> <4A5CC851.9010107@sun.com> In-Reply-To: <4A5CC851.9010107@sun.com> X-Brightmail-Tracker: AAAAAA== X-DCC-dmv.com-Metrics: carlson 1181; Body=3 Fuz1=3 Fuz2=3 X-Antispam: No, score=-2.6/5.0, scanned in 0.124sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Status: RO Content-Length: 3088 Cathy Zhou wrote: > James Carlson wrote: >> Thus, at least for observability, it does not matter where you put the >> flag. >> > Sorry that I wasn't clear. I meant to keep IFF_NOACCEPT and change its > definition to be: > > #define ILLF_NOACCEPT IFF_NOACCEPT OK. >> Yes. My argument, though, is that it doesn't actually matter if the >> address is marked "UP" but (for reasons outside the application's >> control) is not currently usable on the wire. At most, lack of network >> connection is a reason to avoid that address when doing source address >> selection -- but, then, you almost certainly should be avoiding these >> VRRP-controlled addresses when doing source address selection for >> outbound connections anyway. >> > Sorry, why cannot application use the virtual IP addresses as the source > IP address, if the router is currently master? At least for the routing version of VRRP (that is, "no accept" mode), the virtual IP address is for the convenience of configuring routing on dumb hosts, and intentionally not for regular communication. Obviously, communication using that IP address won't work at all from the non-address-owning node, because when it's master, it's not allowed to accept packets there. From the address-owning node, it's possible, but usually you have better (less volatile) addresses to use. An address that moves around leaves questions about who exactly you're talking to. In the non-routing mode ("accept" mode), I'm not sure what the right practices are. That mode of operation is new (it's still just a draft, I think) and still isn't clear to me. >>> I looked but didn't see things similar to it. BSD does provide a >>> IF_ANNOUNCE message to indicate the plumbing/unplumbing of the interface >>> but that cannot satisfy my needs. >>> >> It'd probably help to describe that message and why it doesn't provide >> the information needed. (In fact, that message is exactly what I'd >> expected was needed ...) >> > As far as I know, the IFF_ANNOUCE only indicate the plumbing/unplumbing > of the interface, not the address (the logical interface). That is why > we cannot use it for our purpose. Ah, so you want the address on "plumbed-but-down" interfaces, is that it? Why can't IFF_ANNOUNCE on Solaris include an address ... ? >> Why do you care whether the underlying link is deleted or renamed? >> >> How does that work with DR? >> > When the VRRP router is first enabled, we check which VNIC is the > associated VNIC and make sure it is created (otherwise, enable-router > would fail). Then we listen to the PF_ROUTE messages to track all the IP > addresses over this VNIC based on its interface name. If the VNICs is > deleted or renamed in between, that would be a problem. Can they be deleted while IP is configured? I'm still confused about what the problems are. It'd be good to have a description of the problems in the specification to make it clear why these things are being held open. > Thanks for remind me of DR. I believe adding a router_rcm plugin is > possible. It may be necessary. From Cathy.Zhou@Sun.COM Tue Jul 14 12:11:17 2009 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6EJBHcw008102 for ; Tue, 14 Jul 2009 12:11:17 -0700 (PDT) Received: from sineb-mail-2.sun.com ([192.18.19.7]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6EJBGSL044202 for ; Tue, 14 Jul 2009 12:11:16 -0700 (PDT) Received: from fe-apac-06.sun.com (fe-apac-06.sun.com [192.18.19.177] (may be forged)) by sineb-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6EJBAMp022723 for ; Tue, 14 Jul 2009 19:11:10 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMS00D00D8GJ900@mail-apac.sun.com> for psarc-ext@sac.sfbay.sun.com; Wed, 15 Jul 2009 03:11:10 +0800 (SGT) Received: from [129.146.108.16] ([unknown] [129.146.108.16]) by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMS0015EDALHE80@mail-apac.sun.com> for psarc-ext@sac.sfbay.sun.com; Wed, 15 Jul 2009 03:11:10 +0800 (SGT) Date: Tue, 14 Jul 2009 12:09:22 -0700 From: Cathy Zhou Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/17/2009] In-reply-to: <4A5CD1D7.4050104@workingcode.com> Sender: Cathy.Zhou@Sun.COM To: James Carlson Cc: Sebastien Roy , psarc-ext Message-id: <4A5CD7E2.6050606@sun.com> References: <1247251240.10474.96.camel@strat> <4A590324.8050908@workingcode.com> <4A5B7E11.3000103@sun.com> <4A5C8082.9040508@workingcode.com> <4A5CC851.9010107@sun.com> <4A5CD1D7.4050104@workingcode.com> User-Agent: Thunderbird 2.0.0.12 (X11/20080422) Status: RO Content-Length: 2582 > > At least for the routing version of VRRP (that is, "no accept" mode), > the virtual IP address is for the convenience of configuring routing on > dumb hosts, and intentionally not for regular communication. > > Obviously, communication using that IP address won't work at all from > the non-address-owning node, because when it's master, it's not allowed > to accept packets there. > > From the address-owning node, it's possible, but usually you have better > (less volatile) addresses to use. An address that moves around leaves > questions about who exactly you're talking to. > > In the non-routing mode ("accept" mode), I'm not sure what the right > practices are. That mode of operation is new (it's still just a draft, > I think) and still isn't clear to me. > That's a good point. I will change the source selection logic to not including the no_accepted IP addresses. >> As far as I know, the IFF_ANNOUCE only indicate the plumbing/unplumbing >> of the interface, not the address (the logical interface). That is why >> we cannot use it for our purpose. >> > Ah, so you want the address on "plumbed-but-down" interfaces, is that it? > > Why can't IFF_ANNOUNCE on Solaris include an address ... ? > I am nervous to make such changes since that will probably confuse applications consuming the IFF_ANNOUCE messages. >>> When the VRRP router is first enabled, we check which VNIC is the >>> associated VNIC and make sure it is created (otherwise, enable-router >>> would fail). Then we listen to the PF_ROUTE messages to track all the IP >>> addresses over this VNIC based on its interface name. If the VNICs is >>> deleted or renamed in between, that would be a problem. >>> > Can they be deleted while IP is configured? > No, it cannot. But since we don't control the management of the IP addresses, one could unplumb the VNIC while the VRRP router is enabled, and even delete the VNIC and then probably create a inrelevant link with the old VNIC name. That is what we want to avoid. Since the vrrpd would assume that the new link is the VNIC that we need track the IP addresses on and make the wrong assumption what would be the virtual IP addresses. > I'm still confused about what the problems are. It'd be good to have a > description of the problems in the specification to make it clear why > these things are being held open. > Sure, I will try to make it clear. > >> Thanks for remind me of DR. I believe adding a router_rcm plugin is >> possible. >> > > It may be necessary. > I'll add it. Thanks - Cathy > > From carlsonj@workingcode.com Tue Jul 14 17:20:20 2009 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6F0KK0c024896 for ; Tue, 14 Jul 2009 17:20:20 -0700 (PDT) Received: from brmea-mail-2.sun.com (brmea-mail-2.Sun.COM [192.18.98.43]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6F0KKYk046382 for ; Tue, 14 Jul 2009 17:20:20 -0700 (PDT) Received: from relay41i.sun.com ([192.5.209.70]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6F0KJZN018863 for ; Wed, 15 Jul 2009 00:20:19 GMT Received: from mmp42es.mmp.us.syntegra.com ([160.41.221.11] [160.41.221.11]) by relay41i.sun.com with ESMTP id BT-MMP-2575851 for psarc-ext@sac.sfbay.sun.com; Wed, 15 Jul 2009 00:20:19 Z Received: from relay42i.sun.com (relay42i.sun.com [192.5.209.72]) by mmp42es.mmp.us.syntegra.com with ESMTP id BT-MMP-87123456; Wed, 15 Jul 2009 00:20:18 Z Received: from carlson.workingcode.com ([75.150.68.97] [75.150.68.97]) by relay4i.sun.com with ESMTP id BT-MMP-1360878; Wed, 15 Jul 2009 00:20:18 Z Received: from dhcp-160.workingcode.com (dhcp-160 [192.168.254.160]) (authenticated bits=0) by carlson.workingcode.com (8.14.2+Sun/8.14.3) with ESMTP id n6F0JsNT006692 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 14 Jul 2009 20:19:55 -0400 (EDT) Message-ID: <4A5D2092.5070304@workingcode.com> Date: Tue, 14 Jul 2009 20:19:30 -0400 From: James Carlson User-Agent: Thunderbird 2.0.0.22 (Macintosh/20090605) To: Cathy Zhou CC: Sebastien Roy , psarc-ext Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/17/2009] References: <1247251240.10474.96.camel@strat> <4A590324.8050908@workingcode.com> <4A5B7E11.3000103@sun.com> <4A5C8082.9040508@workingcode.com> <4A5CC851.9010107@sun.com> <4A5CD1D7.4050104@workingcode.com> <4A5CD7E2.6050606@sun.com> In-Reply-To: <4A5CD7E2.6050606@sun.com> X-Brightmail-Tracker: AAAAAA== X-DCC-EATSERVER-Metrics: carlson 1166; Body=3 Fuz1=3 Fuz2=3 X-Antispam: No, score=-2.6/5.0, scanned in 0.080sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Status: RO Content-Length: 2882 Cathy Zhou wrote: >> From the address-owning node, it's possible, but usually you have better >> (less volatile) addresses to use. An address that moves around leaves >> questions about who exactly you're talking to. >> >> In the non-routing mode ("accept" mode), I'm not sure what the right >> practices are. That mode of operation is new (it's still just a draft, >> I think) and still isn't clear to me. >> > > That's a good point. I will change the source selection logic to not > including the no_accepted IP addresses. I'm not sure that any VRRP addresses are appropriate for automatic selection -- but I guess that's a topic for a deeper discussion about using VRRP for application protection. >>> As far as I know, the IFF_ANNOUCE only indicate the plumbing/unplumbing >>> of the interface, not the address (the logical interface). That is why >>> we cannot use it for our purpose. >>> >> Ah, so you want the address on "plumbed-but-down" interfaces, is that it? >> >> Why can't IFF_ANNOUNCE on Solaris include an address ... ? >> > I am nervous to make such changes since that will probably confuse > applications consuming the IFF_ANNOUCE messages. I don't see how that's possible, given the bit vector used for the extensible parts of a routing socket message ... but ok. >>>> When the VRRP router is first enabled, we check which VNIC is the >>>> associated VNIC and make sure it is created (otherwise, enable-router >>>> would fail). Then we listen to the PF_ROUTE messages to track all >>>> the IP >>>> addresses over this VNIC based on its interface name. If the VNICs is >>>> deleted or renamed in between, that would be a problem. >>>> >> Can they be deleted while IP is configured? >> > No, it cannot. But since we don't control the management of the IP > addresses, one could unplumb the VNIC while the VRRP router is enabled, > and even delete the VNIC and then probably create a inrelevant link with > the old VNIC name. That is what we want to avoid. Since the vrrpd would > assume that the new link is the VNIC that we need track the IP addresses > on and make the wrong assumption what would be the virtual IP addresses. The user could do all sorts of things that might make a mess of the daemons that manage automatic interfaces. I don't think that means that it's a good idea to try to prevent such administrative changes. Unless I'm missing something, this sounds like a very different practice from what we've had in place for pretty much all the rest of networking, and it seems to require elevated and otherwise unnecessary privileges in order to accomplish. I'm not sure what to make of it, but it sounds like too much work to me. If the user deliberately thwarts the VRRP daemon by reconfiguring things in ways documented not to work, what's the worst that happens? It doesn't work properly until restarted, right? From Cathy.Zhou@Sun.COM Tue Jul 14 17:43:42 2009 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6F0hgYU025088 for ; Tue, 14 Jul 2009 17:43:42 -0700 (PDT) Received: from sineb-mail-1.sun.com ([192.18.19.6]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6F0hcvr056350 for ; Tue, 14 Jul 2009 17:43:39 -0700 (PDT) Received: from fe-apac-05.sun.com (fe-apac-05.sun.com [192.18.19.176] (may be forged)) by sineb-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6F0hXe9028265 for ; Wed, 15 Jul 2009 00:43:33 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMS00E00SGWL100@mail-apac.sun.com> for psarc-ext@sac.sfbay.sun.com; Wed, 15 Jul 2009 08:43:33 +0800 (SGT) Received: from [129.146.108.16] ([unknown] [129.146.108.16]) by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMS006MASO8A3C0@mail-apac.sun.com> for psarc-ext@sac.sfbay.sun.com; Wed, 15 Jul 2009 08:43:22 +0800 (SGT) Date: Tue, 14 Jul 2009 17:41:33 -0700 From: Cathy Zhou Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/17/2009] In-reply-to: <4A5D2092.5070304@workingcode.com> Sender: Cathy.Zhou@Sun.COM To: James Carlson Cc: Sebastien Roy , psarc-ext Message-id: <4A5D25BD.80006@sun.com> References: <1247251240.10474.96.camel@strat> <4A590324.8050908@workingcode.com> <4A5B7E11.3000103@sun.com> <4A5C8082.9040508@workingcode.com> <4A5CC851.9010107@sun.com> <4A5CD1D7.4050104@workingcode.com> <4A5CD7E2.6050606@sun.com> <4A5D2092.5070304@workingcode.com> User-Agent: Thunderbird 2.0.0.12 (X11/20080422) Status: RO Content-Length: 1681 > Can they be deleted while IP is configured? > >> No, it cannot. But since we don't control the management of the IP >> addresses, one could unplumb the VNIC while the VRRP router is enabled, >> and even delete the VNIC and then probably create a inrelevant link with >> the old VNIC name. That is what we want to avoid. Since the vrrpd would >> assume that the new link is the VNIC that we need track the IP addresses >> on and make the wrong assumption what would be the virtual IP addresses. >> > > The user could do all sorts of things that might make a mess of the > daemons that manage automatic interfaces. I don't think that means that > it's a good idea to try to prevent such administrative changes. > > Unless I'm missing something, this sounds like a very different practice > from what we've had in place for pretty much all the rest of networking, > and it seems to require elevated and otherwise unnecessary privileges in > order to accomplish. I'm not sure what to make of it, but it sounds > like too much work to me. > If the user deliberately thwarts the VRRP daemon by reconfiguring things > in ways documented not to work, what's the worst that happens? It > doesn't work properly until restarted, right? > I am not sure what kind of intentional misconfiguration is in your mind. I think it is possible in some cases, one has to delete and recreate the specific vrrp router in order to make the router work properly. In general, I'd like that either administrator is able to figure out the misconfiguration by looking at the "vrrpadm show-router output", or vrrpd can detect such mis-configuration and prompt a warning. Thanks - Cathy From Sebastien.Roy@Sun.COM Thu Jul 16 15:17:03 2009 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6GMH3Ib029335 for ; Thu, 16 Jul 2009 15:17:03 -0700 (PDT) Received: from brmea-mail-2.sun.com (brmea-mail-2.Sun.COM [192.18.98.43]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6GMH2QF024614 for ; Thu, 16 Jul 2009 15:17:02 -0700 (PDT) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6GMH2Lc016865 for ; Thu, 16 Jul 2009 22:17:02 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMW00B00AUWIG00@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Thu, 16 Jul 2009 16:17:02 -0600 (MDT) Received: from [129.148.174.103] ([unknown] [129.148.174.103]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMW00IX1B85AID0@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Thu, 16 Jul 2009 16:16:54 -0600 (MDT) Date: Thu, 16 Jul 2009 18:15:26 -0400 From: Sebastien Roy Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/17/2009] In-reply-to: <4A5D25BD.80006@sun.com> Sender: Sebastien.Roy@Sun.COM To: Cathy Zhou Cc: James Carlson , psarc-ext Message-id: <1247782526.9890.43.camel@strat> Organization: Sun Microsystems X-Mailer: Evolution 2.26.2 References: <1247251240.10474.96.camel@strat> <4A590324.8050908@workingcode.com> <4A5B7E11.3000103@sun.com> <4A5C8082.9040508@workingcode.com> <4A5CC851.9010107@sun.com> <4A5CD1D7.4050104@workingcode.com> <4A5CD7E2.6050606@sun.com> <4A5D2092.5070304@workingcode.com> <4A5D25BD.80006@sun.com> Status: RO Content-Length: 134 Since the project team is updating the spec to address issues discussed thus far, I've placed this case in waiting need spec. -Seb From Sebastien.Roy@Sun.COM Wed Jul 22 07:27:43 2009 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6MERhf8002281 for ; Wed, 22 Jul 2009 07:27:43 -0700 (PDT) Received: from brmea-mail-2.sun.com (brmea-mail-2.Sun.COM [192.18.98.43]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6MERhaV043930 for ; Wed, 22 Jul 2009 07:27:43 -0700 (PDT) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6MERgi7007577 for ; Wed, 22 Jul 2009 14:27:42 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KN600200SWWGG00@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Wed, 22 Jul 2009 08:27:42 -0600 (MDT) Received: from [129.148.174.103] ([unknown] [129.148.174.103]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KN600D9ZTHI9F20@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Wed, 22 Jul 2009 08:27:18 -0600 (MDT) Date: Wed, 22 Jul 2009 10:25:47 -0400 From: Sebastien Roy Subject: VRRP Update [PSARC/2009/388 FastTrack timeout 07/29/2009] Sender: Sebastien.Roy@Sun.COM To: psarc-ext Cc: Cathy Zhou Message-id: <1248272747.14938.13.camel@strat> Organization: Sun Microsystems X-Mailer: Evolution 2.26.2 Status: RO Content-Length: 711 The project team has supplied new materials for this case addressing issues brought up by Jim Carlson. I've thus reset the timer on this case. It expires on 07/29/2009. The changes include: 1. Make it clear that IFF_NOACCEPT is a per-ill flag. 2. Make it clear that all vrrpadm operations are persistent 3. Change /usr/sbin/vrrpd to /usr/lib/vrrpd 4. Describe changes needed to support DR 5. Change one of the dependency service from svc:/system/filesystem/usr to svc:/system/filesystem/minimal, since we need filesystem/minimal to mount /var/run directory, where the AF_UNIX socket file resides. 6. Add back-up router configuration example The updated materials are in the materials directory. -Seb From carlsonj@workingcode.com Thu Jul 23 08:55:18 2009 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6NFtFfM003343 for ; Thu, 23 Jul 2009 08:55:17 -0700 (PDT) Received: from brmea-mail-2.sun.com (brmea-mail-2.Sun.COM [192.18.98.43]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6NFtFum000988 for ; Thu, 23 Jul 2009 08:55:15 -0700 (PDT) Received: from relay14i.sun.com (ip124.net129179-4.block1.us.syntegra.com [129.179.4.124]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6NFm87r010928 for ; Thu, 23 Jul 2009 15:55:14 GMT Received: from mmp14es.mmp.us.syntegra.com ([160.41.208.14] [160.41.208.14]) by relay14i.sun.com with ESMTP id BT-MMP-113832 for psarc-ext@sac.sfbay.sun.com; Thu, 23 Jul 2009 15:55:14 Z Received: from relay12i.sun.com (relay12i.sun.com [129.179.4.122]) by mmp14es.mmp.us.syntegra.com with ESMTP id BT-MMP-61502; Thu, 23 Jul 2009 15:55:13 Z Received: from carlson.workingcode.com ([75.150.68.97] [75.150.68.97]) by relay1i.sun.com with ESMTP id BT-MMP-167390; Thu, 23 Jul 2009 15:32:59 Z Received: from [10.50.24.188] (gate.abinitio.com [65.170.40.132]) (authenticated bits=0) by carlson.workingcode.com (8.14.2+Sun/8.14.3) with ESMTP id n6NFUk0M001234 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 23 Jul 2009 11:30:46 -0400 (EDT) Message-ID: <4A688226.6040302@workingcode.com> Date: Thu, 23 Jul 2009 11:30:46 -0400 From: James Carlson User-Agent: Thunderbird 2.0.0.22 (X11/20090605) To: Sebastien Roy CC: psarc-ext , Cathy Zhou Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/29/2009] References: <1248272747.14938.13.camel@strat> In-Reply-To: <1248272747.14938.13.camel@strat> X-Brightmail-Tracker: AAAAAA== X-DCC-URT-Metrics: carlson 1060; Body=3 Fuz1=3 Fuz2=3 X-Antispam: No, score=0.0/5.0, scanned in 0.270sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Status: RO Content-Length: 966 Sebastien Roy wrote: > The project team has supplied new materials for this case addressing > issues brought up by Jim Carlson. I've thus reset the timer on this > case. It expires on 07/29/2009. > > The changes include: > > 1. Make it clear that IFF_NOACCEPT is a per-ill flag. > 2. Make it clear that all vrrpadm operations are persistent > 3. Change /usr/sbin/vrrpd to /usr/lib/vrrpd > 4. Describe changes needed to support DR > 5. Change one of the dependency service from svc:/system/filesystem/usr > to svc:/system/filesystem/minimal, since we need filesystem/minimal to > mount /var/run directory, where the AF_UNIX socket file resides. > 6. Add back-up router configuration example I still don't follow the need to hold the VNIC open so that renaming and DR are broken. If you haven't talked this over in detail with meem, I suggest doing so. The rest looks good to me. -- James Carlson 42.703N 71.076W From Cathy.Zhou@Sun.COM Thu Jul 23 10:24:13 2009 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6NHODb0007940 for ; Thu, 23 Jul 2009 10:24:13 -0700 (PDT) Received: from sineb-mail-2.sun.com ([192.18.19.7]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6NHO7nH004135 for ; Thu, 23 Jul 2009 10:24:07 -0700 (PDT) Received: from fe-apac-06.sun.com (fe-apac-06.sun.com [192.18.19.177] (may be forged)) by sineb-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6NHO1IC000032 for ; Thu, 23 Jul 2009 17:24:01 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KN800H00W4VOU00@mail-apac.sun.com> for psarc-ext@sac.sfbay.sun.com; Fri, 24 Jul 2009 01:24:01 +0800 (SGT) Received: from [129.146.108.16] ([unknown] [129.146.108.16]) by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KN80015BWC0J260@mail-apac.sun.com> for psarc-ext@sac.sfbay.sun.com; Fri, 24 Jul 2009 01:24:01 +0800 (SGT) Date: Thu, 23 Jul 2009 10:22:08 -0700 From: Cathy Zhou Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/29/2009] In-reply-to: <4A688226.6040302@workingcode.com> Sender: Cathy.Zhou@Sun.COM To: James Carlson Cc: Sebastien Roy , psarc-ext , Peter.Memishian@Sun.COM Message-id: <4A689C40.7060603@sun.com> References: <1248272747.14938.13.camel@strat> <4A688226.6040302@workingcode.com> User-Agent: Thunderbird 2.0.0.12 (X11/20080422) Status: RO Content-Length: 2042 Not sure whether Meem is following up our discussion here. I will copy him on this email and give a summary as below: In the current VRRP design, once a VRRP router is enabled, we will hold the underlying data-link and the VRRP router associated VNIC open, to prevent them from being renamed/deleted. The problem of not doing so is that the administrator could delete the underlying data-link (since it could be an aggregation or a VLAN) or delete the VNIC or rename them or recreate other links with those names, but since at the same time, vrrpd still listens to the PF_ROUTE socket messages and try to track all the IP addresses over the old data-link names, vrrpd could assume incorrect primary address and virtual addresses set. Therefore, the VRRP design proposes to hold the underlying data-link and the VNIC open. Further, to make DR work, a vrrp_rcm plugin needs to be implemented to disable the router when the associated device is DRed-out and re-enable the router once the device is DRed back. Meem/James, please let me know if you see a problem with this approach. Thanks - Cathy > Sebastien Roy wrote: > >> The project team has supplied new materials for this case addressing >> issues brought up by Jim Carlson. I've thus reset the timer on this >> case. It expires on 07/29/2009. >> >> The changes include: >> >> 1. Make it clear that IFF_NOACCEPT is a per-ill flag. >> 2. Make it clear that all vrrpadm operations are persistent >> 3. Change /usr/sbin/vrrpd to /usr/lib/vrrpd >> 4. Describe changes needed to support DR >> 5. Change one of the dependency service from svc:/system/filesystem/usr >> to svc:/system/filesystem/minimal, since we need filesystem/minimal to >> mount /var/run directory, where the AF_UNIX socket file resides. >> 6. Add back-up router configuration example >> > > I still don't follow the need to hold the VNIC open so that renaming and > DR are broken. If you haven't talked this over in detail with meem, I > suggest doing so. > > The rest looks good to me. > > From Sebastien.Roy@Sun.COM Thu Jul 30 13:17:53 2009 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6UKHrWd028096 for ; Thu, 30 Jul 2009 13:17:53 -0700 (PDT) Received: from brmea-mail-1.sun.com (brmea-mail-1.Sun.COM [192.18.98.31]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n6UKHrUC051591 for ; Thu, 30 Jul 2009 13:17:53 -0700 (PDT) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n6UKHqD5005160 for ; Thu, 30 Jul 2009 20:17:52 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KNM002002LMBS00@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Thu, 30 Jul 2009 14:17:52 -0600 (MDT) Received: from [129.148.174.103] ([unknown] [129.148.174.103]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KNM004P131K6JE0@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Thu, 30 Jul 2009 14:17:44 -0600 (MDT) Date: Thu, 30 Jul 2009 16:16:08 -0400 From: Sebastien Roy Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/29/2009] In-reply-to: <4A689C40.7060603@sun.com> Sender: Sebastien.Roy@Sun.COM To: Cathy Zhou Cc: James Carlson , psarc-ext , Peter.Memishian@Sun.COM Message-id: <1248984968.2352.56.camel@strat> Organization: Sun Microsystems X-Mailer: Evolution 2.26.2 References: <1248272747.14938.13.camel@strat> <4A688226.6040302@workingcode.com> <4A689C40.7060603@sun.com> Status: RO Content-Length: 190 As discussed during this week's PSARC meeting, I'm extending the timer for this case for an additional week to give Cathy adequate time to bring closure to the remaining open issue. -Seb From Sebastien.Roy@Sun.COM Wed Aug 5 12:19:18 2009 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n75JJIID008462 for ; Wed, 5 Aug 2009 12:19:18 -0700 (PDT) Received: from brmea-mail-2.sun.com (brmea-mail-2.Sun.COM [192.18.98.43]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n75JJHUo037755 for ; Wed, 5 Aug 2009 12:19:17 -0700 (PDT) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n75JJG0h008927 for ; Wed, 5 Aug 2009 19:19:16 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KNX00H00446RA00@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Wed, 05 Aug 2009 13:19:16 -0600 (MDT) Received: from [129.148.174.103] ([unknown] [129.148.174.103]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KNX00GLR4BRWDF0@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Wed, 05 Aug 2009 13:19:03 -0600 (MDT) Date: Wed, 05 Aug 2009 15:17:22 -0400 From: Sebastien Roy Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/29/2009] In-reply-to: <1248984968.2352.56.camel@strat> Sender: Sebastien.Roy@Sun.COM To: Cathy Zhou Cc: psarc-ext , Peter.Memishian@Sun.COM Message-id: <1249499842.16261.67.camel@strat> Organization: Sun Microsystems X-Mailer: Evolution 2.26.2 References: <1248272747.14938.13.camel@strat> <4A688226.6040302@workingcode.com> <4A689C40.7060603@sun.com> <1248984968.2352.56.camel@strat> Status: RO Content-Length: 541 On Thu, 2009-07-30 at 16:16 -0400, Sebastien Roy wrote: > As discussed during this week's PSARC meeting, I'm extending the timer > for this case for an additional week to give Cathy adequate time to > bring closure to the remaining open issue. The new timer is set to expire today and the remaining issues haven't reached closure yet. In addition to that, others have contacted me stating that a full review might be more appropriate. I've thus derailed this case. I'll contact the project team to schedule an Inception review. -Seb From peter.memishian@sun.com Wed Aug 5 13:00:12 2009 Received: from triplex.SFBay.Sun.COM (triplex.SFBay.Sun.COM [129.146.108.208]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n75K0CBE010040 for ; Wed, 5 Aug 2009 13:00:12 -0700 (PDT) Received: from triplex.SFBay.Sun.COM (localhost [127.0.0.1]) by triplex.SFBay.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n75K0BYd013110; Wed, 5 Aug 2009 13:00:11 -0700 (PDT) Received: (from meem@localhost) by triplex.SFBay.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n75K0Biv013107; Wed, 5 Aug 2009 13:00:11 -0700 (PDT) X-Authentication-Warning: triplex.SFBay.Sun.COM: meem set sender to peter.memishian@sun.com using -f MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <19065.58571.722903.543879@gargle.gargle.HOWL> Date: Wed, 5 Aug 2009 13:00:11 -0700 From: Peter Memishian To: Cathy Zhou Cc: James Carlson , Sebastien Roy , psarc-ext , Peter.Memishian@sun.com Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/29/2009] In-Reply-To: <4A689C40.7060603@sun.com> References: <1248272747.14938.13.camel@strat> <4A688226.6040302@workingcode.com> <4A689C40.7060603@sun.com> X-Mailer: VM 7.19 under 21.4 (patch 21) "Educational Television" XEmacs Lucid Reply-To: peter.memishian@sun.com Status: RO Content-Length: 2659 > Not sure whether Meem is following up our discussion here. I will copy > him on this email and give a summary as below: > > In the current VRRP design, once a VRRP router is enabled, we will hold > the underlying data-link and the VRRP router associated VNIC open, to > prevent them from being renamed/deleted. The problem of not doing so is > that the administrator could delete the underlying data-link (since it > could be an aggregation or a VLAN) or delete the VNIC or rename them or > recreate other links with those names, but since at the same time, vrrpd > still listens to the PF_ROUTE socket messages and try to track all the > IP addresses over the old data-link names, vrrpd could assume incorrect > primary address and virtual addresses set. > > Therefore, the VRRP design proposes to hold the underlying data-link and > the VNIC open. Further, to make DR work, a vrrp_rcm plugin needs to be > implemented to disable the router when the associated device is DRed-out > and re-enable the router once the device is DRed back. > > Meem/James, please let me know if you see a problem with this approach. That sounds fine to me, though I'd generally expect the VNICs to be plumbed by IP which would also prevent them from being renamed/deleted; in what situations is that not the case? Also, other daemons that hold datalinks open (e.g., wpad), we do not have any specific interlock with RCM -- the admin must stop the service and then reissue the DR operation. I'm not sure why we're holding vrrpd to a more stringent requirement, but I'm not opposed to it. > >> The project team has supplied new materials for this case addressing > >> issues brought up by Jim Carlson. I've thus reset the timer on this > >> case. It expires on 07/29/2009. > >> > >> The changes include: > >> > >> 1. Make it clear that IFF_NOACCEPT is a per-ill flag. > >> 2. Make it clear that all vrrpadm operations are persistent > >> 3. Change /usr/sbin/vrrpd to /usr/lib/vrrpd What's the reason we settled on /usr/lib/vrrpd instead of /usr/lib/inet/vrrpd? > >> 4. Describe changes needed to support DR > >> 5. Change one of the dependency service from svc:/system/filesystem/usr > >> to svc:/system/filesystem/minimal, since we need filesystem/minimal to > >> mount /var/run directory, where the AF_UNIX socket file resides. > >> 6. Add back-up router configuration example > >> > > > > I still don't follow the need to hold the VNIC open so that renaming and > > DR are broken. If you haven't talked this over in detail with meem, I > > suggest doing so. > > > > The rest looks good to me. -- meem From Cathy.Zhou@Sun.COM Wed Aug 5 13:05:57 2009 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n75K5vrQ010256 for ; Wed, 5 Aug 2009 13:05:57 -0700 (PDT) Received: from sineb-mail-2.sun.com ([192.18.19.7]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n75K5uX6051907 for ; Wed, 5 Aug 2009 13:05:56 -0700 (PDT) Received: from fe-apac-05.sun.com (fe-apac-05.sun.com [192.18.19.176] (may be forged)) by sineb-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n75K5oCt009485 for ; Wed, 5 Aug 2009 20:05:50 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KNX00C006095A00@mail-apac.sun.com> for psarc-ext@sac.sfbay.sun.com; Thu, 06 Aug 2009 04:05:50 +0800 (SGT) Received: from [129.146.108.16] ([unknown] [129.146.108.16]) by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KNX003NC6HOIKE0@mail-apac.sun.com> for psarc-ext@sac.sfbay.sun.com; Thu, 06 Aug 2009 04:05:50 +0800 (SGT) Date: Wed, 05 Aug 2009 13:03:50 -0700 From: Cathy Zhou Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/29/2009] In-reply-to: <19065.58571.722903.543879@gargle.gargle.HOWL> Sender: Cathy.Zhou@Sun.COM To: Peter.Memishian@Sun.COM Cc: James Carlson , Sebastien Roy , psarc-ext Message-id: <4A79E5A6.8040607@sun.com> References: <1248272747.14938.13.camel@strat> <4A688226.6040302@workingcode.com> <4A689C40.7060603@sun.com> <19065.58571.722903.543879@gargle.gargle.HOWL> User-Agent: Thunderbird 2.0.0.12 (X11/20080422) Status: RO Content-Length: 2168 Peter Memishian wrote: > > Not sure whether Meem is following up our discussion here. I will copy > > him on this email and give a summary as below: > > > > In the current VRRP design, once a VRRP router is enabled, we will hold > > the underlying data-link and the VRRP router associated VNIC open, to > > prevent them from being renamed/deleted. The problem of not doing so is > > that the administrator could delete the underlying data-link (since it > > could be an aggregation or a VLAN) or delete the VNIC or rename them or > > recreate other links with those names, but since at the same time, vrrpd > > still listens to the PF_ROUTE socket messages and try to track all the > > IP addresses over the old data-link names, vrrpd could assume incorrect > > primary address and virtual addresses set. > > > > Therefore, the VRRP design proposes to hold the underlying data-link and > > the VNIC open. Further, to make DR work, a vrrp_rcm plugin needs to be > > implemented to disable the router when the associated device is DRed-out > > and re-enable the router once the device is DRed back. > > > > Meem/James, please let me know if you see a problem with this approach. > > That sounds fine to me, though I'd generally expect the VNICs to be > plumbed by IP which would also prevent them from being renamed/deleted; in > what situations is that not the case? Also, other daemons that hold > datalinks open (e.g., wpad), we do not have any specific interlock with > RCM -- the admin must stop the service and then reissue the DR operation. > I'm not sure why we're holding vrrpd to a more stringent requirement, but > I'm not opposed to it. > Yes, once the VNIC is plumbed, it will not be renamed/deleted. But vrrpd cannot be assured that the VNIC is always plumbed when the VRRP router is created. Administrators can always unplumb the VNIC whenever they want, even the VRRP router that relies on the VNIC is already enabled. > What's the reason we settled on /usr/lib/vrrpd instead of > /usr/lib/inet/vrrpd? > If /usr/lib/inet/vrrpd is a better place. I can certainly change that. Thanks for your comments. - Cathy From peter.memishian@sun.com Wed Aug 5 13:15:56 2009 Received: from triplex.SFBay.Sun.COM (triplex.SFBay.Sun.COM [129.146.108.208]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n75KFusU010317 for ; Wed, 5 Aug 2009 13:15:56 -0700 (PDT) Received: from triplex.SFBay.Sun.COM (localhost [127.0.0.1]) by triplex.SFBay.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n75KFuQs013147; Wed, 5 Aug 2009 13:15:56 -0700 (PDT) Received: (from meem@localhost) by triplex.SFBay.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n75KFuKv013144; Wed, 5 Aug 2009 13:15:56 -0700 (PDT) X-Authentication-Warning: triplex.SFBay.Sun.COM: meem set sender to peter.memishian@sun.com using -f MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <19065.59516.337131.255838@gargle.gargle.HOWL> Date: Wed, 5 Aug 2009 13:15:56 -0700 From: Peter Memishian To: Cathy Zhou Cc: Peter.Memishian@sun.com, James Carlson , Sebastien Roy , psarc-ext Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/29/2009] In-Reply-To: <4A79E5A6.8040607@sun.com> References: <1248272747.14938.13.camel@strat> <4A688226.6040302@workingcode.com> <4A689C40.7060603@sun.com> <19065.58571.722903.543879@gargle.gargle.HOWL> <4A79E5A6.8040607@sun.com> X-Mailer: VM 7.19 under 21.4 (patch 21) "Educational Television" XEmacs Lucid Reply-To: peter.memishian@sun.com Status: RO Content-Length: 1022 > Yes, once the VNIC is plumbed, it will not be renamed/deleted. But vrrpd > cannot be assured that the VNIC is always plumbed when the VRRP router > is created. Administrators can always unplumb the VNIC whenever they > want, even the VRRP router that relies on the VNIC is already enabled. Sure -- so this is just a safeguard against an edge case, which I see as mostly an implementation matter and not really core to the architecture. I'm fine with it. > > What's the reason we settled on /usr/lib/vrrpd instead of > > /usr/lib/inet/vrrpd? > > > If /usr/lib/inet/vrrpd is a better place. I can certainly change that. I know there have been some religious wars about the fact that the directory is named /usr/lib/inet and not /usr/lib/net, but for all intents and purposes we generally use that directory for networking daemons, regardless of whether they are TCP/IP-specific. That said, if others feel strongly about using /usr/lib instead of /usr/lib/inet, I won't argue further about it. -- meem From carlsonj@workingcode.com Wed Aug 5 13:18:57 2009 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n75KIvvX010338 for ; Wed, 5 Aug 2009 13:18:57 -0700 (PDT) Received: from sca-ea-mail-2.sun.com (sca-ea-mail-2.Sun.COM [192.18.43.25]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n75KIvsC063345 for ; Wed, 5 Aug 2009 13:18:57 -0700 (PDT) Received: from relay14i.sun.com (ip124.net129179-4.block1.us.syntegra.com [129.179.4.124]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n75K9PXe022618 for ; Wed, 5 Aug 2009 20:18:52 GMT Received: from mmp14es.mmp.us.syntegra.com ([160.41.208.14] [160.41.208.14]) by relay14i.sun.com with ESMTP id BT-MMP-516848 for psarc-ext@sac.sfbay.sun.com; Wed, 5 Aug 2009 20:18:52 Z Received: from relay11i.sun.com (relay11i.sun.com [129.179.4.121]) by mmp14es.mmp.us.syntegra.com with ESMTP id BT-MMP-639768; Wed, 5 Aug 2009 20:18:47 Z Received: from carlson.workingcode.com ([75.150.68.97] [75.150.68.97]) by relay1i.sun.com with ESMTP id BT-MMP-18408069; Wed, 5 Aug 2009 20:17:15 Z Received: from [10.50.24.188] (gate.abinitio.com [65.170.40.132]) (authenticated bits=0) by carlson.workingcode.com (8.14.2+Sun/8.14.3) with ESMTP id n75KGYe4001836 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 5 Aug 2009 16:16:34 -0400 (EDT) Message-ID: <4A79E8A1.2070703@workingcode.com> Date: Wed, 05 Aug 2009 16:16:33 -0400 From: James Carlson User-Agent: Thunderbird 2.0.0.22 (X11/20090605) To: Cathy Zhou CC: Peter.Memishian@sun.com, Sebastien Roy , psarc-ext Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/29/2009] References: <1248272747.14938.13.camel@strat> <4A688226.6040302@workingcode.com> <4A689C40.7060603@sun.com> <19065.58571.722903.543879@gargle.gargle.HOWL> <4A79E5A6.8040607@sun.com> In-Reply-To: <4A79E5A6.8040607@sun.com> X-Brightmail-Tracker: AAAAAA== X-DCC-EATSERVER-Metrics: carlson 1166; Body=4 Fuz1=4 Fuz2=4 X-Antispam: No, score=-1.1/5.0, scanned in 0.418sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Status: RO Content-Length: 2886 Cathy Zhou wrote: > Peter Memishian wrote: >> > Not sure whether Meem is following up our discussion here. I will >> copy > him on this email and give a summary as below: >> > > In the current VRRP design, once a VRRP router is enabled, we >> will hold > the underlying data-link and the VRRP router associated >> VNIC open, to > prevent them from being renamed/deleted. The problem >> of not doing so is > that the administrator could delete the >> underlying data-link (since it > could be an aggregation or a VLAN) >> or delete the VNIC or rename them or > recreate other links with >> those names, but since at the same time, vrrpd > still listens to the >> PF_ROUTE socket messages and try to track all the > IP addresses over >> the old data-link names, vrrpd could assume incorrect > primary >> address and virtual addresses set. >> > > Therefore, the VRRP design proposes to hold the underlying >> data-link and > the VNIC open. Further, to make DR work, a vrrp_rcm >> plugin needs to be > implemented to disable the router when the >> associated device is DRed-out > and re-enable the router once the >> device is DRed back. >> > > Meem/James, please let me know if you see a problem with this >> approach. >> >> That sounds fine to me, though I'd generally expect the VNICs to be >> plumbed by IP which would also prevent them from being >> renamed/deleted; in >> what situations is that not the case? Also, other daemons that hold >> datalinks open (e.g., wpad), we do not have any specific interlock with >> RCM -- the admin must stop the service and then reissue the DR operation. >> I'm not sure why we're holding vrrpd to a more stringent requirement, but >> I'm not opposed to it. >> > Yes, once the VNIC is plumbed, it will not be renamed/deleted. But vrrpd > cannot be assured that the VNIC is always plumbed when the VRRP router > is created. Administrators can always unplumb the VNIC whenever they > want, even the VRRP router that relies on the VNIC is already enabled. It's not that it's a "more stringent requirement" -- my original question here doesn't really have anything to do with DR -- it's that I don't understand the need to block out link renames or deletes or creates from within vrrpd. If the link gets renamed, and the VRRP configuration no longer works, so what? Isn't that exactly what the user asked you to do? I don't expect vrrpd to be magic. If I configure a virtual router with link "foobar0" and then rename it without restarting vrrpd, I'd expect that it won't work. Having the system actively _stop_ me from renaming things seems unnecessarily draconian, and likely just makes it harder to manage. We don't do this for other services that are critically dependent on link names (such as routing daemons), so why is VRRP different? -- James Carlson 42.703N 71.076W From Sebastien.Roy@Sun.COM Wed Aug 5 13:25:15 2009 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n75KPFgK010519 for ; Wed, 5 Aug 2009 13:25:15 -0700 (PDT) Received: from brmea-mail-4.sun.com (brmea-mail-4.Sun.COM [192.18.98.36]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n75KPEkY001979 for ; Wed, 5 Aug 2009 13:25:14 -0700 (PDT) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n75KPEqT025510 for ; Wed, 5 Aug 2009 20:25:14 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KNX0080071JOS00@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Wed, 05 Aug 2009 14:25:14 -0600 (MDT) Received: from [129.148.174.103] ([unknown] [129.148.174.103]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KNX00HXQ7DZUXC0@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Wed, 05 Aug 2009 14:25:12 -0600 (MDT) Date: Wed, 05 Aug 2009 16:23:31 -0400 From: Sebastien Roy Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/29/2009] In-reply-to: <19065.59516.337131.255838@gargle.gargle.HOWL> Sender: Sebastien.Roy@Sun.COM To: Peter.Memishian@Sun.COM Cc: Cathy Zhou , James Carlson , psarc-ext Message-id: <1249503811.16261.105.camel@strat> Organization: Sun Microsystems X-Mailer: Evolution 2.26.2 References: <1248272747.14938.13.camel@strat> <4A688226.6040302@workingcode.com> <4A689C40.7060603@sun.com> <19065.58571.722903.543879@gargle.gargle.HOWL> <4A79E5A6.8040607@sun.com> <19065.59516.337131.255838@gargle.gargle.HOWL> Status: RO Content-Length: 1289 On Wed, 2009-08-05 at 13:15 -0700, Peter Memishian wrote: > > Yes, once the VNIC is plumbed, it will not be renamed/deleted. But vrrpd > > cannot be assured that the VNIC is always plumbed when the VRRP router > > is created. Administrators can always unplumb the VNIC whenever they > > want, even the VRRP router that relies on the VNIC is already enabled. > > Sure -- so this is just a safeguard against an edge case, which I see as > mostly an implementation matter and not really core to the architecture. > I'm fine with it. The addition of the RCM module is architecture, and the justification of its introduction is because VRRP is trying to prevent accidental misconfiguration of VNICs in use by VRRP. Weighing the cost of introducing an RCM module against the benefit of preventing an edge case is likely a judgment call, which is why this discussion has dragged on. My personal opinion is that simply not holding the VNICs open would yield a simpler architecture at the risk of administrators accidentally breaking their VRRP routers by renaming or deleting VNICs. Note that they can break VRRP by doing other things that this case doesn't prevent. Since the case is derailed, this will likely be one of the topics of discussion during the in-person review. -Seb From Cathy.Zhou@Sun.COM Wed Aug 5 13:27:34 2009 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n75KRY5m010555 for ; Wed, 5 Aug 2009 13:27:34 -0700 (PDT) Received: from sineb-mail-1.sun.com ([192.18.19.6]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n75KRXhV003340 for ; Wed, 5 Aug 2009 13:27:34 -0700 (PDT) Received: from fe-apac-06.sun.com (fe-apac-06.sun.com [192.18.19.177] (may be forged)) by sineb-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n75KRSha021128 for ; Wed, 5 Aug 2009 20:27:28 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KNX00I007GWI400@mail-apac.sun.com> for psarc-ext@sac.sfbay.sun.com; Thu, 06 Aug 2009 04:27:28 +0800 (SGT) Received: from [129.146.108.16] ([unknown] [129.146.108.16]) by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KNX003HT7HQVJ50@mail-apac.sun.com> for psarc-ext@sac.sfbay.sun.com; Thu, 06 Aug 2009 04:27:28 +0800 (SGT) Date: Wed, 05 Aug 2009 13:25:28 -0700 From: Cathy Zhou Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/29/2009] In-reply-to: <4A79E8A1.2070703@workingcode.com> Sender: Cathy.Zhou@Sun.COM To: James Carlson Cc: Peter.Memishian@Sun.COM, Sebastien Roy , psarc-ext Message-id: <4A79EAB8.9010403@sun.com> References: <1248272747.14938.13.camel@strat> <4A688226.6040302@workingcode.com> <4A689C40.7060603@sun.com> <19065.58571.722903.543879@gargle.gargle.HOWL> <4A79E5A6.8040607@sun.com> <4A79E8A1.2070703@workingcode.com> User-Agent: Thunderbird 2.0.0.12 (X11/20080422) Status: RO Content-Length: 1460 I don't agree. The VRRP router is created with the specified VRID and address family. Ideally, if the VNIC is renamed, vrrpd should simply find the new VNIC name and start to listen to the PF_ROUTE messages for the new interface name and tracks the IP addresses configured on that new interface as well. But we don't have the mechanism available to be informed with the renaming operation, so vrrpd will behave incorrectly. Also, if the VRRP VNIC does not exist, the router enabling operation would simply fail. Therefore, once the router is enabled, the VNIC should not be deleted for consistency purpose. Thanks - Cathy > It's not that it's a "more stringent requirement" -- my original > question here doesn't really have anything to do with DR -- it's that I > don't understand the need to block out link renames or deletes or > creates from within vrrpd. If the link gets renamed, and the VRRP > configuration no longer works, so what? Isn't that exactly what the > user asked you to do? > > I don't expect vrrpd to be magic. If I configure a virtual router with > link "foobar0" and then rename it without restarting vrrpd, I'd expect > that it won't work. Having the system actively _stop_ me from renaming > things seems unnecessarily draconian, and likely just makes it harder to > manage. > > We don't do this for other services that are critically dependent on > link names (such as routing daemons), so why is VRRP different? > > From carlsonj@workingcode.com Wed Aug 5 13:31:09 2009 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n75KV9bI010627 for ; Wed, 5 Aug 2009 13:31:09 -0700 (PDT) Received: from brmea-mail-4.sun.com (brmea-mail-4.Sun.COM [192.18.98.36]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n75KV9Hw020001 for ; Wed, 5 Aug 2009 13:31:09 -0700 (PDT) Received: from relay42i.sun.com ([192.5.209.72]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n75KOhVS025248 for ; Wed, 5 Aug 2009 20:31:08 GMT Received: from mms49es.mms.us.syntegra.com ([160.41.221.232] [160.41.221.232]) by relay42i.sun.com with ESMTP id BT-MMP-1879604 for psarc-ext@sac.sfbay.sun.com; Wed, 5 Aug 2009 20:31:08 Z Received: from relay44i.sun.com (relay44i.sun.com [192.5.209.118]) by mms49es.mms.us.syntegra.com with ESMTP id BT-MMP-38961225; Wed, 5 Aug 2009 20:31:07 Z Received: from carlson.workingcode.com ([75.150.68.97] [75.150.68.97]) by relay4i.sun.com with ESMTP id BT-MMP-9106136; Wed, 5 Aug 2009 20:31:07 Z Received: from [10.50.24.188] (gate.abinitio.com [65.170.40.132]) (authenticated bits=0) by carlson.workingcode.com (8.14.2+Sun/8.14.3) with ESMTP id n75KV6Xg001947 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 5 Aug 2009 16:31:06 -0400 (EDT) Message-ID: <4A79EC0A.2070100@workingcode.com> Date: Wed, 05 Aug 2009 16:31:06 -0400 From: James Carlson User-Agent: Thunderbird 2.0.0.22 (X11/20090605) To: Cathy Zhou CC: Peter.Memishian@sun.com, Sebastien Roy , psarc-ext Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/29/2009] References: <1248272747.14938.13.camel@strat> <4A688226.6040302@workingcode.com> <4A689C40.7060603@sun.com> <19065.58571.722903.543879@gargle.gargle.HOWL> <4A79E5A6.8040607@sun.com> <4A79E8A1.2070703@workingcode.com> <4A79EAB8.9010403@sun.com> In-Reply-To: <4A79EAB8.9010403@sun.com> X-Brightmail-Tracker: AAAAAA== X-DCC-EATSERVER-Metrics: carlson 1166; Body=4 Fuz1=4 Fuz2=4 X-Antispam: No, score=0.0/5.0, scanned in 0.491sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Status: RO Content-Length: 1099 Cathy Zhou wrote: > I don't agree. The VRRP router is created with the specified VRID and > address family. Ideally, if the VNIC is renamed, vrrpd should simply > find the new VNIC name and start to listen to the PF_ROUTE messages for > the new interface name and tracks the IP addresses configured on that > new interface as well. > > But we don't have the mechanism available to be informed with the > renaming operation, so vrrpd will behave incorrectly. That's not different from dozens of other parts of the system. Arguably, VRRP is far less important than IP filters and routing. Why does it need to be protected specially? > Also, if the VRRP VNIC does not exist, the router enabling operation > would simply fail. Therefore, once the router is enabled, the VNIC > should not be deleted for consistency purpose. I'm still puzzled. This sounds like a job for sysevents or something similar (though there are probably Zones concerns with that), but I don't see why nailing it to the wall is the right answer. -- James Carlson 42.703N 71.076W From Sebastien.Roy@Sun.COM Wed Aug 5 13:34:18 2009 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n75KYI49010837 for ; Wed, 5 Aug 2009 13:34:18 -0700 (PDT) Received: from brmea-mail-4.sun.com (brmea-mail-4.Sun.COM [192.18.98.36]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n75KYIpl022594 for ; Wed, 5 Aug 2009 13:34:18 -0700 (PDT) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n75KYIUO029409 for ; Wed, 5 Aug 2009 20:34:18 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KNX00F007HYRP00@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Wed, 05 Aug 2009 14:34:18 -0600 (MDT) Received: from [129.148.174.103] ([unknown] [129.148.174.103]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KNX003X57T4CB60@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Wed, 05 Aug 2009 14:34:16 -0600 (MDT) Date: Wed, 05 Aug 2009 16:32:35 -0400 From: Sebastien Roy Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/29/2009] In-reply-to: <4A79EAB8.9010403@sun.com> Sender: Sebastien.Roy@Sun.COM To: Cathy Zhou Cc: James Carlson , Peter.Memishian@Sun.COM, psarc-ext Message-id: <1249504355.16261.112.camel@strat> Organization: Sun Microsystems X-Mailer: Evolution 2.26.2 References: <1248272747.14938.13.camel@strat> <4A688226.6040302@workingcode.com> <4A689C40.7060603@sun.com> <19065.58571.722903.543879@gargle.gargle.HOWL> <4A79E5A6.8040607@sun.com> <4A79E8A1.2070703@workingcode.com> <4A79EAB8.9010403@sun.com> Status: RO Content-Length: 411 I don't think this discussion is progressing constructively at this point. Let's shift this discussion to another forum with the interested parties (including myself, Jim Carlson, and Meem please), where the result can serve to prepare for the Inception review (remember, this case is derailed). If there are still open issues for the Inception, then we can resolve them in person during the meeting. -Seb From peter.memishian@sun.com Wed Aug 5 13:48:01 2009 Received: from triplex.SFBay.Sun.COM (triplex.SFBay.Sun.COM [129.146.108.208]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n75Km1cx011210 for ; Wed, 5 Aug 2009 13:48:01 -0700 (PDT) Received: from triplex.SFBay.Sun.COM (localhost [127.0.0.1]) by triplex.SFBay.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n75KlvJu013203; Wed, 5 Aug 2009 13:47:57 -0700 (PDT) Received: (from meem@localhost) by triplex.SFBay.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n75KlvDd013200; Wed, 5 Aug 2009 13:47:57 -0700 (PDT) X-Authentication-Warning: triplex.SFBay.Sun.COM: meem set sender to peter.memishian@sun.com using -f MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <19065.61437.357668.35442@gargle.gargle.HOWL> Date: Wed, 5 Aug 2009 13:47:57 -0700 From: Peter Memishian To: Sebastien Roy Cc: Peter.Memishian@sun.com, Cathy Zhou , James Carlson , psarc-ext Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/29/2009] In-Reply-To: <1249503811.16261.105.camel@strat> References: <1248272747.14938.13.camel@strat> <4A688226.6040302@workingcode.com> <4A689C40.7060603@sun.com> <19065.58571.722903.543879@gargle.gargle.HOWL> <4A79E5A6.8040607@sun.com> <19065.59516.337131.255838@gargle.gargle.HOWL> <1249503811.16261.105.camel@strat> X-Mailer: VM 7.19 under 21.4 (patch 21) "Educational Television" XEmacs Lucid Reply-To: peter.memishian@sun.com Status: RO Content-Length: 852 > The addition of the RCM module is architecture The existence of RCM and the RCM module framework is architecture; whether we choose to implement a new RCM module to solve a given problem seems like an implementation detail to me. > misconfiguration of VNICs in use by VRRP. Weighing the cost of > introducing an RCM module against the benefit of preventing an edge case > is likely a judgment call, which is why this discussion has dragged on. The only architectural matters I see here are the semantics of rename and DR in the presence of VRRP. We should reach agreement on what those should be and leave it to the project team to decide how to implement those semantics. Of course, if the project team feels the advised semantics are too onerous, then more discussion is warranted -- but that does not seem to be the case here. -- meem From Sebastien.Roy@Sun.COM Wed Aug 5 14:18:11 2009 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n75LIBKp012867 for ; Wed, 5 Aug 2009 14:18:11 -0700 (PDT) Received: from brmea-mail-4.sun.com (brmea-mail-4.Sun.COM [192.18.98.36]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n75LIBm8039253 for ; Wed, 5 Aug 2009 14:18:11 -0700 (PDT) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n75LIBw8018773 for ; Wed, 5 Aug 2009 21:18:11 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KNX00F007HYRP00@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Wed, 05 Aug 2009 15:18:11 -0600 (MDT) Received: from [129.148.174.103] ([unknown] [129.148.174.103]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KNX00KX09U1N580@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Wed, 05 Aug 2009 15:18:02 -0600 (MDT) Date: Wed, 05 Aug 2009 17:16:21 -0400 From: Sebastien Roy Subject: Re: VRRP Update [PSARC/2009/388 FastTrack timeout 07/29/2009] In-reply-to: <19065.61437.357668.35442@gargle.gargle.HOWL> Sender: Sebastien.Roy@Sun.COM To: Peter.Memishian@Sun.COM Cc: Cathy Zhou , James Carlson , psarc-ext Message-id: <1249506981.16261.150.camel@strat> Organization: Sun Microsystems X-Mailer: Evolution 2.26.2 References: <1248272747.14938.13.camel@strat> <4A688226.6040302@workingcode.com> <4A689C40.7060603@sun.com> <19065.58571.722903.543879@gargle.gargle.HOWL> <4A79E5A6.8040607@sun.com> <19065.59516.337131.255838@gargle.gargle.HOWL> <1249503811.16261.105.camel@strat> <19065.61437.357668.35442@gargle.gargle.HOWL> Status: RO Content-Length: 378 On Wed, 2009-08-05 at 13:47 -0700, Peter Memishian wrote: > The only architectural matters I see here are the semantics of rename and > DR in the presence of VRRP. We should reach agreement on what those > should be and leave it to the project team to decide how to implement > those semantics. Agreed, and that's the approach we'll take when we discuss this offline. -Seb From Sebastien.Roy@Sun.COM Wed Aug 12 11:18:53 2009 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7CIIrOw013947 for ; Wed, 12 Aug 2009 11:18:53 -0700 (PDT) Received: from brmea-mail-4.sun.com (brmea-mail-4.Sun.COM [192.18.98.36]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n7CIIqvW024534 for ; Wed, 12 Aug 2009 11:18:53 -0700 (PDT) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n7CIIq23025682 for ; Wed, 12 Aug 2009 18:18:52 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KO900F00YR86100@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Wed, 12 Aug 2009 12:18:52 -0600 (MDT) Received: from [129.148.174.103] ([unknown] [129.148.174.103]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KOA00L0U06WGL70@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Wed, 12 Aug 2009 12:18:33 -0600 (MDT) Date: Wed, 12 Aug 2009 14:16:47 -0400 From: Sebastien Roy Subject: PSARC/2009/388 VRRP Update approved Sender: Sebastien.Roy@Sun.COM To: psarc-ext Cc: Cathy Zhou Message-id: <1250101007.11355.14.camel@strat> Organization: Sun Microsystems X-Mailer: Evolution 2.26.2 Status: RO Content-Length: 561 As was previously noted in the case log, I derailed 2009/388 last week, and it was reviewed during today's PSARC meeting. The issue that Jim Carlson had brought up regarding the interaction with VNICs was resolved offline and reflected in the materials that the committee voted on (VNICs will not be held open by the VRRP daemon). Jim Walker requested a spec update to introduce VRRP more generally in the man page. The case was approved, with Garrett D'Amore, Sebastien Roy, and Kais Belgaied voting to approve, and other members not participating. -Seb From Cathy.Zhou@Sun.COM Wed Aug 26 14:36:50 2009 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7QLao9A001868 for ; Wed, 26 Aug 2009 14:36:50 -0700 (PDT) Received: from sineb-mail-1.sun.com ([192.18.19.6]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n7QLansv008645 for ; Wed, 26 Aug 2009 14:36:50 -0700 (PDT) Received: from fe-apac-05.sun.com (fe-apac-05.sun.com [192.18.19.176] (may be forged)) by sineb-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n7QLaiOC011538 for ; Wed, 26 Aug 2009 21:36:44 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KP0001006HB7X00@mail-apac.sun.com> for psarc-ext@sac.sfbay.sun.com; Thu, 27 Aug 2009 05:36:44 +0800 (SGT) Received: from [129.146.108.16] ([unknown] [129.146.108.16]) by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KP0009V16P67OD0@mail-apac.sun.com> for psarc-ext@sac.sfbay.sun.com; Thu, 27 Aug 2009 05:36:44 +0800 (SGT) Date: Wed, 26 Aug 2009 14:34:33 -0700 From: Cathy Zhou Subject: minor change to [PSARC/2009/388] - VRRP update Sender: Cathy.Zhou@Sun.COM To: psarc-ext Cc: Cathy.Zhou@Sun.COM, Sebastien Roy Message-id: <4A95AA69.7090509@sun.com> User-Agent: Thunderbird 2.0.0.12 (X11/20080422) Status: RO Content-Length: 1167 I am proposing to defer the implementation of the "-f", "-a" and "-P" options for the "vrrpadm create-router" subcommand in this case in order to remove the case dependency between PSARC/2009/388 (VRRP update) and PSARC/2009/306 (ipadm and libipadm). A separate PSARC fast-track will be filed to add the "-f", "-a" and "-P" options back that itself depend on PSARC/2009/306. Note that without the "-f", "-a" and "-P" options, the VRRP VNICs will still be able to be created using the dladm command, and the primary IP address and the virtual IP addresses of a VRRP router will still be able to be configured using other IP configuration tools (such as ifconfig or the /etc/hostname.* files). Basically, the IP addresses configured by the "vrrpadm -P" and "vrrpadm -a" command are not different from those configured by other IP tools. The vrrpd daemon simply tracks all the IP addresses configured over the underlying link and the VNIC, and determines the primary IP address and virtual IP addresses, no matter how those IP addresses are configured. The PSARC materials are updated accordingly and will be placed in the materials directory later. Thanks - Cathy From Sebastien.Roy@Sun.COM Mon Aug 31 15:50:07 2009 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7VMo7oI007363 for ; Mon, 31 Aug 2009 15:50:07 -0700 (PDT) Received: from brmea-mail-4.sun.com (brmea-mail-4.Sun.COM [192.18.98.36]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n7VMo73S057867 for ; Mon, 31 Aug 2009 15:50:07 -0700 (PDT) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n7VMo73f002034 for ; Mon, 31 Aug 2009 22:50:07 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KP900L00JAFIE00@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Mon, 31 Aug 2009 16:50:07 -0600 (MDT) Received: from [129.148.174.103] ([unknown] [129.148.174.103]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KP900DWZJFH50F0@mail-amer.sun.com> for psarc-ext@sac.sfbay.sun.com; Mon, 31 Aug 2009 16:50:06 -0600 (MDT) Date: Mon, 31 Aug 2009 18:48:07 -0400 From: Sebastien Roy Subject: Re: minor change to [PSARC/2009/388] - VRRP update In-reply-to: <4A95AA69.7090509@sun.com> Sender: Sebastien.Roy@Sun.COM To: Cathy Zhou Cc: psarc-ext Message-id: <1251758887.2316.145.camel@strat> Organization: Sun Microsystems X-Mailer: Evolution 2.26.3 References: <4A95AA69.7090509@sun.com> Status: RO Content-Length: 525 On Wed, 2009-08-26 at 14:34 -0700, Cathy Zhou wrote: > I am proposing to defer the implementation of the "-f", "-a" and "-P" > options for the "vrrpadm create-router" subcommand in this case in > order to remove the case dependency between PSARC/2009/388 > (VRRP update) and PSARC/2009/306 (ipadm and libipadm). A separate > PSARC fast-track will be filed to add the "-f", "-a" and "-P" options > back that itself depend on PSARC/2009/306. I've placed the updated materials in final.materials in the case directory. -Seb From Cathy.Zhou@Sun.COM Wed Oct 28 11:08:48 2009 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n9SI8mRD022681 for ; Wed, 28 Oct 2009 11:08:48 -0700 (PDT) Received: from sineb-mail-1.sun.com ([192.18.19.6]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id n9SI8lRl028053 for ; Wed, 28 Oct 2009 11:08:48 -0700 (PDT) Received: from fe-apac-05.sun.com (fe-apac-05.sun.com [192.18.19.176] (may be forged)) by sineb-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n9SI8gxC014781 for ; Wed, 28 Oct 2009 18:08:42 GMT MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KS800B00KZQDZ00@mail-apac.sun.com> for psarc-ext@sac.sfbay.sun.com; Thu, 29 Oct 2009 02:08:42 +0800 (SGT) Received: from [129.146.108.16] ([unknown] [129.146.108.16]) by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KS800DPXL2GO830@mail-apac.sun.com> for psarc-ext@sac.sfbay.sun.com; Thu, 29 Oct 2009 02:08:42 +0800 (SGT) Date: Wed, 28 Oct 2009 11:05:55 -0700 From: Cathy Zhou Subject: Re: minor change to [PSARC/2009/388] - VRRP update In-reply-to: <4A95AA69.7090509@sun.com> Sender: Cathy.Zhou@Sun.COM To: psarc-ext Cc: Cathy Zhou , Sebastien Roy Message-id: <4AE88803.7090704@sun.com> References: <4A95AA69.7090509@sun.com> User-Agent: Thunderbird 2.0.0.12 (X11/20080422) Status: RO Content-Length: 1954 FYI. I made some more minor changes to the case material, mostly to address the arc review opinions for PSARC/2008/693 (VRRP). Specifically, the changes are: a. Change "-o no_preempt" to "-o un_preempt" for vrrpadm options (jdc-3). b. Speicfy the characters restrictions for VRRP router names (the legal characters in a router name are: alphanumeric (a-z, A-Z, 0-9) and underscore ('_')) (jdc-2) c. Add vrrp sysevent to the interface table (Consolidation Private). (jdc-5) d. Change the path of vrrp.conf from /etc/vrrp.conf to /etc/inet/vrrp.conf (comments during the code review) e. Add more protocol details to vrrpadm(1M) manpage.(jdc-98) Seb already helped me to put the new material to the case directory. Thanks - Cathy > I am proposing to defer the implementation of the "-f", "-a" and "-P" > options for the "vrrpadm create-router" subcommand in this case in > order to remove the case dependency between PSARC/2009/388 > (VRRP update) and PSARC/2009/306 (ipadm and libipadm). A separate > PSARC fast-track will be filed to add the "-f", "-a" and "-P" options > back that itself depend on PSARC/2009/306. > > Note that without the "-f", "-a" and "-P" options, the VRRP VNICs will > still be able to be created using the dladm command, and the primary IP > address and the virtual IP addresses of a VRRP router will still be able > to be configured using other IP configuration tools (such as ifconfig or > the /etc/hostname.* files). Basically, the IP addresses configured by the > "vrrpadm -P" and "vrrpadm -a" command are not different from those > configured by other IP tools. The vrrpd daemon simply tracks all the IP > addresses configured over the underlying link and the VNIC, and > determines the primary IP address and virtual IP addresses, no matter > how those IP addresses are configured. > > The PSARC materials are updated accordingly and will be placed in > the materials directory later. > > Thanks > - Cathy >