From sacadmin Mon Mar 12 21:36:56 2007 Received: from sparc-build.SFBay.Sun.COM (sparc-build.SFBay.Sun.COM [10.5.23.87]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l2D4auiW027166; Mon, 12 Mar 2007 21:36:56 -0700 (PDT) Received: from sparc-build.SFBay.Sun.COM (localhost [127.0.0.1]) by sparc-build.SFBay.Sun.COM (8.13.3+Sun/8.13.3) with ESMTP id l2D4SeRC014228; Mon, 12 Mar 2007 20:28:40 -0800 (PST) Received: (from is@localhost) by sparc-build.SFBay.Sun.COM (8.13.3+Sun/8.13.3/Submit) id l2D4Se8G014218; Mon, 12 Mar 2007 20:28:40 -0800 (PST) Date: Mon, 12 Mar 2007 20:28:40 -0800 (PST) From: Ienup Sung Message-Id: <200703130428.l2D4Se8G014218@sparc-build.SFBay.Sun.COM> To: PSARC@sac.sfbay.sun.com Subject: UTF-8 text preparation functions [PSARC/2007/149 Timeout: 03/19/2007] Status: RO Content-Length: 2071 Subject: PSARC FastTrack [03/19/2007]: UTF-8 text preparation functions Template Version: @(#)sac_nextcase %I% %G% SMI This information is Copyright 2007 Sun Microsystems, Inc. 1. Introduction 1.1. Project/Component Working Name: UTF-8 text preparation functions 1.2. Name of Document Author/Supplier: Author: Ienup Sung 1.3 Date of This Document: 12 March, 2007 4. Technical Description OVERVIEW This project proposes to add six new kernel functions for the support of UTF-8 text preparations as described in [1]. The interfaces proposed will do case foldings and normalizations on UTF-8 text as specified in the Unicode standard [2, 3]. The interfaces are requested by SNAS Technologies group and file system groups such as NFS and PCFS to support necessary standards and also to be compatible with file systems from other vendors. Unicode case foldings and normalizations together will require less than 80KB of data to contain necessary case mappings and equivalency mappings for a version of Unicode. The project proposes to support Unicode versions 3.2 and 5.0 as they are the only two notable versions of Unicode that we need to support for our products. Unicode normally comes out with a major version every two years or so. The latest Unicode 5.0 came out at November of 2006. INTERFACE STABILITY AND RELEASE BINDING There is no notable imported interfaces. The following are the exported interfaces: Interface Stability Reference u8_textprep_open(), Committed [1] u8_textprep_buf(), u8_textprep_close(), u8_textprep_str(), u8_strcmp(), u8_validate() The project team asks for Micro/Patch release binding. REFERENCES [1] Man pages at the case directory in flat text files: materials/*.9f [2] The Unicode standard: http://www.unicode.org/standard/standard.html [3] Unicode Standard Annex #15: UNICODE NORMALIZATION FORMS http://www.unicode.org/reports/tr15/ 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack From sacadmin Tue Mar 13 04:48:04 2007 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l2DBm3OC010459 for ; Tue, 13 Mar 2007 04:48:03 -0700 (PDT) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.0+Sun/8.14.0) with ESMTP id l2DBm3qG002232; Tue, 13 Mar 2007 07:48:03 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.0+Sun/8.14.0/Submit) id l2DBm2LK002229; Tue, 13 Mar 2007 07:48:02 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <17910.36722.589951.785514@gargle.gargle.HOWL> Date: Tue, 13 Mar 2007 07:48:02 -0400 From: James Carlson To: Ienup Sung Cc: PSARC@sac.sfbay.sun.com Subject: Re: UTF-8 text preparation functions [PSARC/2007/149 Timeout: 03/19/2007] In-Reply-To: <200703130428.l2D4Se8G014218@sparc-build.SFBay.Sun.COM> References: <200703130428.l2D4Se8G014218@sparc-build.SFBay.Sun.COM> X-Mailer: VM 7.01 under Emacs 21.3.1 Status: RO Content-Length: 1549 Ienup Sung writes: > Unicode case foldings and normalizations together will require less than > 80KB of data to contain necessary case mappings and equivalency mappings for > a version of Unicode. The project proposes to support Unicode versions 3.2 and > 5.0 as they are the only two notable versions of Unicode that we need to > support for our products. > > Unicode normally comes out with a major version every two years or so. > The latest Unicode 5.0 came out at November of 2006. What happens going forward? Do we increase the size of the kernel by 80KB every two years for new tables? When can we ever get rid of old versions? Why do kernel modules need to ask for specific Unicode variants rather than just always using "the latest?" Is the implication that there's just no compatibility story for U8, or is something else at play? > u8_textprep_open(), Committed [1] > u8_textprep_buf(), > u8_textprep_close(), > u8_textprep_str(), > u8_strcmp(), > u8_validate() Also committed interfaces are the U8_* #defines. Please provide #defines for the magic numbers 320 and 500. Does u8_textprep_open do a blocking memory allocation? (If so, then suggest possible non-blocking flag, and NULL rather than -1 as error return.) u8_textprep_str should not return (size_t)-1. Consider using ssize_t instead. -- James Carlson, Solaris Networking Sun Microsystems / 1 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From sacadmin Tue Mar 13 12:44:13 2007 Received: from sfbaymail1sca.SFBay.Sun.COM (sfbaymail1sca.SFBay.Sun.COM [129.145.154.35]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l2DJiD7x029082 for ; Tue, 13 Mar 2007 12:44:13 -0700 (PDT) Received: from nwk-ea-fw-1.sun.com (nwkes-gis-mail-2.SFBay.Sun.COM [10.4.134.6]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l2DJiDDR007710 for ; Tue, 13 Mar 2007 12:44:13 -0700 (PDT) Received: from d1-sfbay-09.sun.com ([192.18.39.119]) by nwk-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l2DJi8Wr018281 for ; Tue, 13 Mar 2007 11:44:08 -0800 (PST) Received: from conversion-daemon.d1-sfbay-09.sun.com by d1-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JEU00C01XH6QF00@d1-sfbay-09.sun.com> (original mail from Ienup.Sung@Sun.COM) for PSARC@sac.sfbay.sun.com; Tue, 13 Mar 2007 12:44:08 -0700 (PDT) Received: from [192.168.1.93] ([129.150.17.160]) by d1-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JEU0022UXHHWPRR@d1-sfbay-09.sun.com> for PSARC@sac.sfbay.sun.com; Tue, 13 Mar 2007 12:44:06 -0700 (PDT) Date: Tue, 13 Mar 2007 12:43:21 -0700 From: Ienup Sung Subject: Re: UTF-8 text preparation functions [PSARC/2007/149 Timeout: 03/19/2007] In-reply-to: <17910.36722.589951.785514@gargle.gargle.HOWL> Sender: Ienup.Sung@Sun.COM To: PSARC@sac.sfbay.sun.com Reply-to: Ienup.Sung@Sun.COM Message-id: <45F6FED9.8060106@sun.com> Organization: Sun Microsystems, Inc. MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en References: <200703130428.l2D4Se8G014218@sparc-build.SFBay.Sun.COM> <17910.36722.589951.785514@gargle.gargle.HOWL> User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.7) Gecko/20041221 Status: RO Content-Length: 3377 James Carlson wrote at 03/13/07 04:48: > What happens going forward? Do we increase the size of the kernel by > 80KB every two years for new tables? When can we ever get rid of old > versions? Only if there is any need for such new tables. Nothing can be guaranteed but based on the standardization works going on in various forums today, we will be pretty much set for the next minimum two to five years or even more for the supported functionalities. Once supported/added, I think we cannot get rid of the existing tables easily unless we decided not to support a certain version of Unicode and also referencing standards such as NFSv4, IDNA, SASLprep, and so on. Also, to open up better chance of getting rid of the version data, the case would need to set the interface stability for the Unicode data separately as Uncommitted (or even Volatile perhaps) today. Please let me know if that's what you've intended and I'll update the spec. Otherwise, I'd like to keep the spec as it is. > Why do kernel modules need to ask for specific Unicode variants rather > than just always using "the latest?" Is the implication that there's > just no compatibility story for U8, or is something else at play? This is due to we need such versions. For instance, Unicode 3.2 is being used by NFSv4, Stringprep, SASLprep (last time I checked), and so on. On the other hand, MSFT Windows Vista supports Unicode 4.1 with their NTFS right now (in their to-uppercase based case-insensitive file name support) and it will be updated soon to Unicode 5.0 and at that time we will be ready for them. > Also committed interfaces are the U8_* #defines. > > Please provide #defines for the magic numbers 320 and 500. By supplying the man pages with ATTRIBUTES section saying only Committed for the entire man pages, I thought everything in the man pages are Committed including macros, errnos, behaviors, and so on. I updated the spec.txt and man pages to update the 320 and the 500 and also to explicitly specify the stability on the macros at the exported interfaces sub-section. > Does u8_textprep_open do a blocking memory allocation? (If so, then > suggest possible non-blocking flag, and NULL rather than -1 as error > return.) Yes. And following your suggestion, I've update the spec and man page. I am though still keeping a single failure return value of (u8_textprep_t)-1 for the case of no kernel memory return and instead added ENOMEM errno. (FWIW, I was also thought about this but then didn't add that.) > u8_textprep_str should not return (size_t)-1. Consider using ssize_t > instead. Thanks for pointing this out. The original intention of having it with type size_t is due to that I was hoping to return the number of illegal or incomplete characters found during the text preparation which I was omitting up until now. I thus updated the RETURN VALUES section with the following addition: When U8_TEXTPREP_IGNORE_INVALID option value is specified, it returns | the number of illegal or incomplete characters found during the text | preparation. When U8_TEXTPREP_IGNORE_INVALID is not specified and | the text preparation is entirely successful, consequently, | the function returns 0. .......................................... | Updated spec.txt and man pages are placed at the materials directory. Ienup From sacadmin Tue Mar 13 13:19:45 2007 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l2DKJjPS000517 for ; Tue, 13 Mar 2007 13:19:45 -0700 (PDT) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.0+Sun/8.14.0) with ESMTP id l2DKJjd8005921; Tue, 13 Mar 2007 16:19:45 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.0+Sun/8.14.0/Submit) id l2DKJjXL005918; Tue, 13 Mar 2007 16:19:45 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <17911.1888.819672.257187@gargle.gargle.HOWL> Date: Tue, 13 Mar 2007 16:19:44 -0400 From: James Carlson To: Ienup.Sung@sun.com Cc: PSARC@sac.sfbay.sun.com Subject: Re: UTF-8 text preparation functions [PSARC/2007/149 Timeout: 03/19/2007] In-Reply-To: <45F6FED9.8060106@sun.com> References: <200703130428.l2D4Se8G014218@sparc-build.SFBay.Sun.COM> <17910.36722.589951.785514@gargle.gargle.HOWL> <45F6FED9.8060106@sun.com> X-Mailer: VM 7.01 under Emacs 21.3.1 Status: RO Content-Length: 3993 Ienup Sung writes: [...] > Please let me know if that's what you've intended and I'll update > the spec. Otherwise, I'd like to keep the spec as it is. No; I'm just trying to understand the nature of what's delivered. > > Why do kernel modules need to ask for specific Unicode variants rather > > than just always using "the latest?" Is the implication that there's > > just no compatibility story for U8, or is something else at play? > > This is due to we need such versions. For instance, Unicode 3.2 is being > used by NFSv4, Stringprep, SASLprep (last time I checked), and so on. On > the other hand, MSFT Windows Vista supports Unicode 4.1 with their NTFS > right now (in their to-uppercase based case-insensitive file name support) > and it will be updated soon to Unicode 5.0 and at that time we will be > ready for them. I guess that gets close to answering my underlying question. The underlying question is this: is a 'Unicode 3.2' byte string something that a 'Unicode 5.0' parser can understand, or are these really completely independent items with mutually incompatible bits? If there's some compatibility story (e.g., "all valid Unicode 3.2 byte strings are valid Unicode 5.0 strings, but not all 5.0 strings are valid 3.2 strings"), then all we really need is the newest one. Otherwise, it looks like we're stuck with the standards these people are producing. > > Also committed interfaces are the U8_* #defines. > > > > Please provide #defines for the magic numbers 320 and 500. > > By supplying the man pages with ATTRIBUTES section saying only Committed > for the entire man pages, I thought everything in the man pages are > Committed including macros, errnos, behaviors, and so on. OK. It wasn't clear to me, since you had a separate table in your fast-track text just listing the function names alone. > I updated the spec.txt and man pages to update the 320 and the 500 and > also to explicitly specify the stability on the macros at the exported > interfaces sub-section. Looks good; thanks. > > Does u8_textprep_open do a blocking memory allocation? (If so, then > > suggest possible non-blocking flag, and NULL rather than -1 as error > > return.) > > Yes. And following your suggestion, I've update the spec and man page. > > I am though still keeping a single failure return value of (u8_textprep_t)-1 > for the case of no kernel memory return and instead added ENOMEM errno. OK. It's worth pointing out that casting arbitrary integers to pointer values doesn't necessarily work on all CPUs. (Yes, I realize that some ugly old interfaces depend on it, and that it does work _today_.) > (FWIW, I was also thought about this but then didn't add that.) > > > > u8_textprep_str should not return (size_t)-1. Consider using ssize_t > > instead. > > Thanks for pointing this out. The original intention of having it with type > size_t is due to that I was hoping to return the number of illegal or > incomplete characters found during the text preparation which I was omitting > up until now. I thus updated the RETURN VALUES section with the following > addition: > > When U8_TEXTPREP_IGNORE_INVALID option value is specified, it returns | > the number of illegal or incomplete characters found during the text | > preparation. When U8_TEXTPREP_IGNORE_INVALID is not specified and | > the text preparation is entirely successful, consequently, | > the function returns 0. .......................................... | That's an important change, but I don't understand how it addresses my comment. Why is returning -1 via an unsigned type the right thing to do? If the return value is actually signed, why not use a signed type for it? For example, see strfmon(3C). -- James Carlson, Solaris Networking Sun Microsystems / 1 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From sacadmin Tue Mar 13 13:57:33 2007 Received: from sfbaymail1sca.SFBay.Sun.COM (sfbaymail1sca.SFBay.Sun.COM [129.145.154.35]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l2DKvXdi002946 for ; Tue, 13 Mar 2007 13:57:33 -0700 (PDT) Received: from nwk-ea-fw-1.sun.com (nwkes-gis-mail-1.SFBay.Sun.COM [10.4.134.5]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l2DKvXjC012154 for ; Tue, 13 Mar 2007 13:57:33 -0700 (PDT) Received: from d1-sfbay-10.sun.com ([192.18.39.120]) by nwk-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l2DKvSV9009157 for ; Tue, 13 Mar 2007 12:57:28 -0800 (PST) Received: from conversion-daemon.d1-sfbay-10.sun.com by d1-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JEV007010U80L00@d1-sfbay-10.sun.com> (original mail from Ienup.Sung@Sun.COM) for PSARC@sac.sfbay.sun.com; Tue, 13 Mar 2007 13:57:28 -0700 (PDT) Received: from [192.168.1.93] ([129.150.17.160]) by d1-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JEV00M6I0VRD7OH@d1-sfbay-10.sun.com> for PSARC@sac.sfbay.sun.com; Tue, 13 Mar 2007 13:57:27 -0700 (PDT) Date: Tue, 13 Mar 2007 13:56:43 -0700 From: Ienup Sung Subject: Re: UTF-8 text preparation functions [PSARC/2007/149 Timeout: 03/19/2007] In-reply-to: <17911.1888.819672.257187@gargle.gargle.HOWL> Sender: Ienup.Sung@Sun.COM To: PSARC@sac.sfbay.sun.com Reply-to: Ienup.Sung@Sun.COM Message-id: <45F7100B.6040405@sun.com> Organization: Sun Microsystems, Inc. MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en References: <200703130428.l2D4Se8G014218@sparc-build.SFBay.Sun.COM> <17910.36722.589951.785514@gargle.gargle.HOWL> <45F6FED9.8060106@sun.com> <17911.1888.819672.257187@gargle.gargle.HOWL> User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.7) Gecko/20041221 Status: RO Content-Length: 3438 James Carlson wrote at 03/13/07 13:19: >>>Why do kernel modules need to ask for specific Unicode variants rather >>>than just always using "the latest?" Is the implication that there's >>>just no compatibility story for U8, or is something else at play? >> >>This is due to we need such versions. For instance, Unicode 3.2 is being >>used by NFSv4, Stringprep, SASLprep (last time I checked), and so on. On >>the other hand, MSFT Windows Vista supports Unicode 4.1 with their NTFS >>right now (in their to-uppercase based case-insensitive file name support) >>and it will be updated soon to Unicode 5.0 and at that time we will be >>ready for them. > > > I guess that gets close to answering my underlying question. > > The underlying question is this: is a 'Unicode 3.2' byte string > something that a 'Unicode 5.0' parser can understand, or are these > really completely independent items with mutually incompatible bits? > > If there's some compatibility story (e.g., "all valid Unicode 3.2 byte > strings are valid Unicode 5.0 strings, but not all 5.0 strings are > valid 3.2 strings"), then all we really need is the newest one. > Otherwise, it looks like we're stuck with the standards these people > are producing. One thing in the interface level that is clear to me is that we need to distinguish between the two versions. Yes, Unicode 5.0 adds 3,868 new characters over Unicode 3.2 defined characters. Actual implementation is a different story and, yes, I totally agree with you and this also goes without saying that we will make sure to share the data as much as possible so that the memory requirement will be minimum possible even though I said in the spec that, in the worst case, it could be two sets of data one for each version. >>>u8_textprep_str should not return (size_t)-1. Consider using ssize_t >>>instead. >> >>Thanks for pointing this out. The original intention of having it with type >>size_t is due to that I was hoping to return the number of illegal or >>incomplete characters found during the text preparation which I was omitting >>up until now. I thus updated the RETURN VALUES section with the following >>addition: >> >> When U8_TEXTPREP_IGNORE_INVALID option value is specified, it returns | >> the number of illegal or incomplete characters found during the text | >> preparation. When U8_TEXTPREP_IGNORE_INVALID is not specified and | >> the text preparation is entirely successful, consequently, | >> the function returns 0. .......................................... | > > > That's an important change, but I don't understand how it addresses my > comment. > > Why is returning -1 via an unsigned type the right thing to do? If > the return value is actually signed, why not use a signed type for it? > > For example, see strfmon(3C). The reason why I chose the size_t as the type of the return value is due to the function returns a positive integer not signed as it cannot be negative in normal circumstances (as in favor of "strongly typed" although C language isn't) to prevent/warn at least during compilation time on the possible mis-use of the variable that could be used as an index to a data structure. And we also need to use the return value to indicate the failure and hence used (size_t)-1 similarly like iconv(3C). <-- I presume one of the old (enough) interfaces. Would this be a unreasonable/unnecessary idea? Ienup From sacadmin Tue Mar 13 14:21:06 2007 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l2DLL52u003709 for ; Tue, 13 Mar 2007 14:21:05 -0700 (PDT) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.0+Sun/8.14.0) with ESMTP id l2DLL5GY006336; Tue, 13 Mar 2007 17:21:05 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.0+Sun/8.14.0/Submit) id l2DLL5XE006333; Tue, 13 Mar 2007 17:21:05 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <17911.5568.390811.345311@gargle.gargle.HOWL> Date: Tue, 13 Mar 2007 17:21:04 -0400 From: James Carlson To: Ienup.Sung@sun.com Cc: PSARC@sac.sfbay.sun.com Subject: Re: UTF-8 text preparation functions [PSARC/2007/149 Timeout: 03/19/2007] In-Reply-To: <45F7100B.6040405@sun.com> References: <200703130428.l2D4Se8G014218@sparc-build.SFBay.Sun.COM> <17910.36722.589951.785514@gargle.gargle.HOWL> <45F6FED9.8060106@sun.com> <17911.1888.819672.257187@gargle.gargle.HOWL> <45F7100B.6040405@sun.com> X-Mailer: VM 7.01 under Emacs 21.3.1 Status: RO Content-Length: 3170 Ienup Sung writes: > > If there's some compatibility story (e.g., "all valid Unicode 3.2 byte > > strings are valid Unicode 5.0 strings, but not all 5.0 strings are > > valid 3.2 strings"), then all we really need is the newest one. > > Otherwise, it looks like we're stuck with the standards these people > > are producing. > > One thing in the interface level that is clear to me is that we need to > distinguish between the two versions. > > Yes, Unicode 5.0 adds 3,868 new characters over Unicode 3.2 defined > characters. > > Actual implementation is a different story and, yes, I totally agree with > you and this also goes without saying that we will make sure to share > the data as much as possible so that the memory requirement will be minimum > possible even though I said in the spec that, in the worst case, it could be > two sets of data one for each version. I'm not really talking so much about the internal implementation, but rather the effect on the consumers of this interface. As a Committed interface, what should kernel consumers really do here? Should they nail themselves to the wall and specify a particular Unicode version number -- meaning that as the world around them evolves, and 3.2 is no longer relevant, they slowly fall apart -- or should they just specify "use latest" -- and thus presumably be beset by the sorts of compatibility problems you seem to be guarding against? I don't understand how this is really usable. An upward compatibility story makes sense to me, where each revision just builds on the previous one, and the only thing a kernel module might need to know is whether the system supports a high enough version number and (for network peer negotiation purposes) the single exact number used by the kernel. I can't make sense out of a use-specific-version-only (multiple versions) story, nor can I see how any reasonable standards body could ignore the issue (write-only text?). If it makes sense to the people who use Unicode, though, and this is what they really want, then I guess I'll just give up. > > Why is returning -1 via an unsigned type the right thing to do? If > > the return value is actually signed, why not use a signed type for it? > > > > For example, see strfmon(3C). > > The reason why I chose the size_t as the type of the return value is > due to the function returns a positive integer not signed as it cannot be > negative in normal circumstances (as in favor of "strongly typed" although > C language isn't) to prevent/warn at least during compilation time on > the possible mis-use of the variable that could be used as an index to > a data structure. > > And we also need to use the return value to indicate the failure and hence > used (size_t)-1 similarly like iconv(3C). <-- I presume one of the old > (enough) interfaces. How about read(2)? > Would this be a unreasonable/unnecessary idea? It still seems icky to me, but it's a nit. Drive on. -- James Carlson, Solaris Networking Sun Microsystems / 1 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From sacadmin Tue Mar 13 15:04:33 2007 Received: from sfbaymail2sca.sfbay.sun.com (sfbaymail2sca.SFBay.Sun.COM [129.145.155.42]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l2DM4XBB007457 for ; Tue, 13 Mar 2007 15:04:33 -0700 (PDT) Received: from nwk-ea-fw-1.sun.com (nwkes-gis-mail-1.SFBay.Sun.COM [10.4.134.5]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l2DM4Xxc002385 for ; Tue, 13 Mar 2007 15:04:33 -0700 (PDT) Received: from d1-sfbay-10.sun.com ([192.18.39.120]) by nwk-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l2DM4SOk017777 for ; Tue, 13 Mar 2007 14:04:28 -0800 (PST) Received: from conversion-daemon.d1-sfbay-10.sun.com by d1-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JEV00A013VZZ500@d1-sfbay-10.sun.com> (original mail from Ienup.Sung@Sun.COM) for PSARC@sac.sfbay.sun.com; Tue, 13 Mar 2007 15:04:28 -0700 (PDT) Received: from [192.168.1.93] ([129.150.17.160]) by d1-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JEV00MP93ZDDBFF@d1-sfbay-10.sun.com> for PSARC@sac.sfbay.sun.com; Tue, 13 Mar 2007 15:04:26 -0700 (PDT) Date: Tue, 13 Mar 2007 15:03:41 -0700 From: Ienup Sung Subject: Re: UTF-8 text preparation functions [PSARC/2007/149 Timeout: 03/19/2007] In-reply-to: <17911.5568.390811.345311@gargle.gargle.HOWL> Sender: Ienup.Sung@Sun.COM Cc: PSARC@sac.sfbay.sun.com Reply-to: Ienup.Sung@Sun.COM Message-id: <45F71FBD.2030208@sun.com> Organization: Sun Microsystems, Inc. MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en References: <200703130428.l2D4Se8G014218@sparc-build.SFBay.Sun.COM> <17910.36722.589951.785514@gargle.gargle.HOWL> <45F6FED9.8060106@sun.com> <17911.1888.819672.257187@gargle.gargle.HOWL> <45F7100B.6040405@sun.com> <17911.5568.390811.345311@gargle.gargle.HOWL> User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.7) Gecko/20041221 Status: RO Content-Length: 3440 James Carlson wrote at 03/13/07 14:21: > Ienup Sung writes: > >>>If there's some compatibility story (e.g., "all valid Unicode 3.2 byte >>>strings are valid Unicode 5.0 strings, but not all 5.0 strings are >>>valid 3.2 strings"), then all we really need is the newest one. >>>Otherwise, it looks like we're stuck with the standards these people >>>are producing. >> >>One thing in the interface level that is clear to me is that we need to >>distinguish between the two versions. >> >>Yes, Unicode 5.0 adds 3,868 new characters over Unicode 3.2 defined >>characters. >> >>Actual implementation is a different story and, yes, I totally agree with >>you and this also goes without saying that we will make sure to share >>the data as much as possible so that the memory requirement will be minimum >>possible even though I said in the spec that, in the worst case, it could be >>two sets of data one for each version. > > > I'm not really talking so much about the internal implementation, but > rather the effect on the consumers of this interface. > > As a Committed interface, what should kernel consumers really do here? > > Should they nail themselves to the wall and specify a particular > Unicode version number -- meaning that as the world around them > evolves, and 3.2 is no longer relevant, they slowly fall apart -- or > should they just specify "use latest" -- and thus presumably be beset > by the sorts of compatibility problems you seem to be guarding > against? > > I don't understand how this is really usable. An upward compatibility > story makes sense to me, where each revision just builds on the > previous one, and the only thing a kernel module might need to know is > whether the system supports a high enough version number and (for > network peer negotiation purposes) the single exact number used by the > kernel. > > I can't make sense out of a use-specific-version-only (multiple > versions) story, nor can I see how any reasonable standards body could > ignore the issue (write-only text?). If it makes sense to the people > who use Unicode, though, and this is what they really want, then I > guess I'll just give up. I had similar (but not exactly the same) short discussion with ZFS team. For ZFS, my suggestion was to use either U8_UNICODE_500 or U8_UNICODE_LATEST with a notice that if you use U8_UNICODE_LATEST that means your software is good to go for any future releases/versions but also be aware that you're allowing characters that may not exist in the today's Unicode version. For anyone who is implementing a specific protocol that requires a specific Unicode version such as NFSv4, I would suggest to use U8_UNICODE_320 since that's what the protocol is explictily calling out for unfortunately... And so the proposal aims two sets of consumers: one who has to stick to a specific version (and in this case either Unciode 3.2 and 5.0) or one who can just ride on the latest. Re some protocols calling out for a specific Unicode version, I also think that is a mistake (some may need to but not so rather blindly). One good thing though that is going on at IETF IDNA-UPATE and EAI forums and Unicode consortium is that people are somehow talking to each other and trying to amend that mistake for future. It may take perhaps another year or more to settle onto something better BTW... > It still seems icky to me, but it's a nit. Drive on. Thanks for allowing this... Ienup From sacadmin Wed Mar 14 11:02:17 2007 Received: from binky.central.sun.com (binky.Central.Sun.COM [129.153.128.104]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l2EI2HQW013505 for ; Wed, 14 Mar 2007 11:02:17 -0700 (PDT) Received: from binky.central.sun.com (localhost [127.0.0.1]) by binky.central.sun.com (8.13.8+Sun/8.13.6) with ESMTP id l2EI1cJN021645; Wed, 14 Mar 2007 13:01:38 -0500 (CDT) Received: (from nw141292@localhost) by binky.central.sun.com (8.13.8+Sun/8.13.8/Submit) id l2EI1bF3021644; Wed, 14 Mar 2007 13:01:37 -0500 (CDT) X-Authentication-Warning: binky.central.sun.com: nw141292 set sender to Nicolas.Williams@sun.com using -f Date: Wed, 14 Mar 2007 13:01:37 -0500 From: Nicolas Williams To: James Carlson Cc: Ienup Sung , PSARC@sac.sfbay.sun.com Subject: Re: UTF-8 text preparation functions [PSARC/2007/149 Timeout: 03/19/2007] Message-ID: <20070314180137.GP19939@Sun.COM> References: <200703130428.l2D4Se8G014218@sparc-build.SFBay.Sun.COM> <17910.36722.589951.785514@gargle.gargle.HOWL> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <17910.36722.589951.785514@gargle.gargle.HOWL> User-Agent: Mutt/1.5.7i Status: RO Content-Length: 512 On Tue, Mar 13, 2007 at 07:48:02AM -0400, James Carlson wrote: > Why do kernel modules need to ask for specific Unicode variants rather > than just always using "the latest?" Is the implication that there's > just no compatibility story for U8, or is something else at play? Internet protocols nowadays specify how to handle codepoints that are, from one peer's point of view, unassigned. Often this involves rejecting the use of unassigned codepoints where strings containing them would be stored. Nico -- From sacadmin Wed Mar 14 11:17:53 2007 Received: from sfbaymail2sca.sfbay.sun.com (sfbaymail2sca.SFBay.Sun.COM [129.145.155.42]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l2EIHrBo014048 for ; Wed, 14 Mar 2007 11:17:53 -0700 (PDT) Received: from nwk-ea-fw-1.sun.com (nwkes-gis-mail-2.SFBay.Sun.COM [10.4.134.6]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l2EIHrTC019574 for ; Wed, 14 Mar 2007 11:17:53 -0700 (PDT) Received: from d1-sfbay-09.sun.com ([192.18.39.119]) by nwk-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l2EIHmq6012872 for ; Wed, 14 Mar 2007 10:17:48 -0800 (PST) Received: from conversion-daemon.d1-sfbay-09.sun.com by d1-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JEW00101O398J00@d1-sfbay-09.sun.com> (original mail from Ienup.Sung@Sun.COM) for PSARC@sac.sfbay.sun.com; Wed, 14 Mar 2007 11:17:47 -0700 (PDT) Received: from [192.168.1.93] ([129.150.16.97]) by d1-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JEW00CQJO5N9F3O@d1-sfbay-09.sun.com> for PSARC@sac.sfbay.sun.com; Wed, 14 Mar 2007 11:17:47 -0700 (PDT) Date: Wed, 14 Mar 2007 11:17:01 -0700 From: Ienup Sung Subject: Re: UTF-8 text preparation functions [PSARC/2007/149 Timeout: 03/19/2007] In-reply-to: <45F71FBD.2030208@sun.com> Sender: Ienup.Sung@Sun.COM To: PSARC@sac.sfbay.sun.com Reply-to: Ienup.Sung@Sun.COM Message-id: <45F83C1D.5030405@sun.com> Organization: Sun Microsystems, Inc. MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en References: <200703130428.l2D4Se8G014218@sparc-build.SFBay.Sun.COM> <17910.36722.589951.785514@gargle.gargle.HOWL> <45F6FED9.8060106@sun.com> <17911.1888.819672.257187@gargle.gargle.HOWL> <45F7100B.6040405@sun.com> <17911.5568.390811.345311@gargle.gargle.HOWL> <45F71FBD.2030208@sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.7) Gecko/20041221 Status: RO Content-Length: 140 This case was approved at today's PSARC meeting. Approved spec.txt and man pages can be found from materials directory of the case. Ienup From sacadmin Fri Mar 16 16:13:24 2007 Received: from binky.central.sun.com (binky.Central.Sun.COM [129.153.128.104]) by sac.sfbay.sun.com (8.13.6+Sun/8.13.6) with ESMTP id l2GNDO33015533 for ; Fri, 16 Mar 2007 16:13:24 -0700 (PDT) Received: from binky.central.sun.com (localhost [127.0.0.1]) by binky.central.sun.com (8.13.8+Sun/8.13.6) with ESMTP id l2GNChYX022519; Fri, 16 Mar 2007 18:12:43 -0500 (CDT) Received: (from nw141292@localhost) by binky.central.sun.com (8.13.8+Sun/8.13.8/Submit) id l2GNChAS022518; Fri, 16 Mar 2007 18:12:43 -0500 (CDT) X-Authentication-Warning: binky.central.sun.com: nw141292 set sender to Nicolas.Williams@sun.com using -f Date: Fri, 16 Mar 2007 18:12:43 -0500 From: Nicolas Williams To: Ienup Sung Cc: PSARC@sac.sfbay.sun.com Subject: Re: UTF-8 text preparation functions [PSARC/2007/149 Timeout: 03/19/2007] Message-ID: <20070316231243.GE22445@Sun.COM> References: <200703130428.l2D4Se8G014218@sparc-build.SFBay.Sun.COM> <17910.36722.589951.785514@gargle.gargle.HOWL> <45F6FED9.8060106@sun.com> <17911.1888.819672.257187@gargle.gargle.HOWL> <45F7100B.6040405@sun.com> <17911.5568.390811.345311@gargle.gargle.HOWL> <45F71FBD.2030208@sun.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <45F71FBD.2030208@sun.com> User-Agent: Mutt/1.5.7i Status: RO Content-Length: 966 On Tue, Mar 13, 2007 at 03:03:41PM -0700, Ienup Sung wrote: > Re some protocols calling out for a specific Unicode version, I also > think that is a mistake (some may need to but not so rather blindly). I don't agree. Protocols that need to forbid or map certain problematic codepoints can't know whether future codepoint assignments will create additional problems that may require additions to the forbidden/mapped codepoint tables. Now, NFSv4 does prohibit some codepoints and maps others in filesystem object names. Given that it does this it's reasonable to expect that the prohibited and mapping tables for that stringprep may need updates for Unicode versions > 3.2. Speacking of stringprep: NFSv4 will need not just Unicode normalizations but actual stringprep support for at least the nfs4_cs_prep and nfs4_cis_prep profiles. At first glance these APIs don't provide all of that, but could through the addition of flags for textprep_open(). Nico -- From sacadmin Mon May 14 14:45:29 2007 Received: from sfbaymail2sca.sfbay.sun.com (sfbaymail2sca.SFBay.Sun.COM [129.145.155.42]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l4ELjTwO026173 for ; Mon, 14 May 2007 14:45:29 -0700 (PDT) Received: from nwk-ea-fw-1.sun.com (nwkes-gis-mail-1.SFBay.Sun.COM [10.4.134.5]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l4ELiK9C010753 for ; Mon, 14 May 2007 14:44:20 -0700 (PDT) Received: from d1-sfbay-09.sun.com ([192.18.39.119]) by nwk-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l4ELiF1M007664 for ; Mon, 14 May 2007 14:44:15 -0700 (PDT) Received: from conversion-daemon.d1-sfbay-09.sun.com by d1-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JI100101W8W5R00@d1-sfbay-09.sun.com> (original mail from Ienup.Sung@Sun.COM) for PSARC@sac.sfbay.sun.com; Mon, 14 May 2007 14:44:15 -0700 (PDT) Received: from [192.168.1.93] ([129.150.18.187]) by d1-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JI100336WDQW1HH@d1-sfbay-09.sun.com> for PSARC@sac.sfbay.sun.com; Mon, 14 May 2007 14:44:15 -0700 (PDT) Date: Mon, 14 May 2007 14:41:42 -0700 From: Ienup Sung Subject: Two man page tweakings for PSARC/2007/149 UTF-8 text preparation functions In-reply-to: <200703130428.l2D4Se8G014218@sparc-build.SFBay.Sun.COM> Sender: Ienup.Sung@Sun.COM To: PSARC@sac.sfbay.sun.com Reply-to: Ienup.Sung@Sun.COM Message-id: <4648D796.2070603@sun.com> Organization: Sun Microsystems, Inc. MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en References: <200703130428.l2D4Se8G014218@sparc-build.SFBay.Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.7) Gecko/20041221 Status: RO Content-Length: 1676 During the implementation of the project, I realized that there are two minor mitakes in the man pages and so I just applied the following changes as shown with diffs output. Please let me know if this would cause/qualify for a new review on the case. Ienup 1. u8_strcmp.9f change at DESCRIPTION section: --- PSARC/2007/149/materials/u8_strcmp.9f- Mon May 14 14:37:31 2007 +++ PSARC/2007/149/materials/u8_strcmp.9f Mon May 14 14:39:27 2007 @@ -102,8 +102,8 @@ defined by the corresponding version of the Unicode Standard. When multiple comparison options are specified, Unicode Normalization - is performed first before case-sensitive or case-insensitive comparison| - is performed. | + is performed after case-sensitive or case-insensitive processing is | + performed. | RETURN VALUES The function returns an integer greater than, equal to, or less than 0, 2. u8_validate.9f typo change at RETURN VALUES section: --- PSARC/2007/149/materials/u8_validate.9f- Mon May 14 14:37:22 2007 +++ PSARC/2007/149/materials/u8_validate.9f Mon May 14 14:40:13 2007 @@ -92,7 +92,7 @@ If u8str is a null pointer, u8_validate() returns 0. If u8str is not a null pointer, u8_validate() returns the number of bytes that constitute the characters if the next n or fewer bytes form - valid characters), or -1 if there is an validation failure and + valid characters, or -1 if there is an validation failure and may set errno to indicate the error. From sacadmin Mon May 21 13:52:21 2007 Received: from sfbaymail2sca.sfbay.sun.com (sfbaymail2sca.SFBay.Sun.COM [129.145.155.42]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l4LKqKcn009012 for ; Mon, 21 May 2007 13:52:20 -0700 (PDT) Received: from nwk-ea-fw-1.sun.com (nwkes-gis-mail-1.SFBay.Sun.COM [10.4.134.5]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l4LKp99W027998 for ; Mon, 21 May 2007 13:51:09 -0700 (PDT) Received: from d1-sfbay-10.sun.com ([192.18.39.120]) by nwk-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l4LKp3me013044 for ; Mon, 21 May 2007 13:51:03 -0700 (PDT) Received: from conversion-daemon.d1-sfbay-10.sun.com by d1-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JIE00401SK1CC00@d1-sfbay-10.sun.com> (original mail from Ienup.Sung@Sun.COM) for psarc-record@sac.sfbay.sun.com; Mon, 21 May 2007 13:51:03 -0700 (PDT) Received: from [192.168.1.93] ([129.150.37.26]) by d1-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JIE006E9SL2H9K0@d1-sfbay-10.sun.com> for psarc-record@sac.sfbay.sun.com; Mon, 21 May 2007 13:51:03 -0700 (PDT) Date: Mon, 21 May 2007 13:48:18 -0700 From: Ienup Sung Subject: PSARC/2007/149 UTF-8 text preparation functions -- some minor modifications at man pages Sender: Ienup.Sung@Sun.COM To: psarc-record@sac.sfbay.sun.com Reply-to: Ienup.Sung@Sun.COM Message-id: <46520592.4040500@sun.com> Organization: Sun Microsystems, Inc. MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.7) Gecko/20041221 Status: RO Content-Length: 3198 In the following man pages approved, the project team mentioned that the support for Unicode Normalization Forms NFC and NFKC are not yet implemented. During the project implementation, however, the project team was able to also support the NFC and NFKD due to a proper allocation of test resources on the NFC and the NFKC related testing: PSARC/2007/149/materials/ u8_strcmp.9f u8_textprep_open.9f u8_textprep_str.9f Hence, I just updated the man page removing the "Currently not implemented." lines from the above mentioned man pages. Since this doesn't really change any architecture or interface in practical sense, I'm just recording the changes without sending out any explicit email to PSARC. The diffs are shown at below. Ienup 1. diff -u u8_strcmp.9f- u8_strcmp.9f --- u8_strcmp.9f- Mon May 21 13:42:22 2007 +++ u8_strcmp.9f Mon May 21 13:43:10 2007 @@ -46,15 +46,11 @@ by using Unicode Normalization Form C. - Currently not implemented. - U8_STRCMP_NFKD Do string comparison after s1 and s2 have been normalized by using Unicode Normalization Form KD. - Currently not implemented. - U8_STRCMP_NFKC Do string comparison after s1 and s2 have been normalized by using Unicode Normalization 2. diff -u u8_textprep_open.9f- u8_textprep_open.9f --- u8_textprep_open.9f- Mon May 21 13:42:32 2007 +++ u8_textprep_open.9f Mon May 21 13:43:18 2007 @@ -33,13 +33,9 @@ U8_TEXTPREP_NFC Apply Unicode Normalization Form C. - Currently not implemented. - U8_TEXTPREP_NFKD Apply Unicode Normalization Form KD. - Currently not implemented. - U8_TEXTPREP_NFKC Apply Unicode Normalization Form KC. 3. diff -u u8_textprep_str.9f- u8_textprep_str.9f --- u8_textprep_str.9f- Mon May 21 13:42:40 2007 +++ u8_textprep_str.9f Mon May 21 13:43:25 2007 @@ -75,13 +75,9 @@ U8_TEXTPREP_NFC Apply Unicode Normalization Form C. - Currently not implemented. - U8_TEXTPREP_NFKD Apply Unicode Normalization Form KD. - Currently not implemented. - U8_TEXTPREP_NFKC Apply Unicode Normalization Form KC. From sacadmin Wed May 23 13:46:37 2007 Received: from binky.central.sun.com (binky.Central.Sun.COM [129.153.128.104]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l4NKkbtJ013249 for ; Wed, 23 May 2007 13:46:37 -0700 (PDT) Received: from binky.central.sun.com (localhost [127.0.0.1]) by binky.central.sun.com (8.13.8+Sun/8.13.6) with ESMTP id l4NKi2GY026639; Wed, 23 May 2007 15:44:02 -0500 (CDT) Received: (from nw141292@localhost) by binky.central.sun.com (8.13.8+Sun/8.13.8/Submit) id l4NKi2AO026638; Wed, 23 May 2007 15:44:02 -0500 (CDT) X-Authentication-Warning: binky.central.sun.com: nw141292 set sender to Nicolas.Williams@sun.com using -f Date: Wed, 23 May 2007 15:44:02 -0500 From: Nicolas Williams To: Ienup Sung Cc: PSARC@sac.sfbay.sun.com Subject: Re: UTF-8 text preparation functions [PSARC/2007/149 Timeout: 03/19/2007] Message-ID: <20070523204401.GW23226@Sun.COM> References: <200703130428.l2D4Se8G014218@sparc-build.SFBay.Sun.COM> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200703130428.l2D4Se8G014218@sparc-build.SFBay.Sun.COM> User-Agent: Mutt/1.5.7i Status: RO Content-Length: 449 A couple of comments (yes, they are very late, they come from doing a code review): - EBADF means something very different according to intro(2) than it is used to mean in this interface. If it's easy to add new errno symbols and values for use in a kernel-land API, should this be pursued? - At least one parameter name, 'list' in u8_validate(), is not very meaningful -- 'prohibited_list' would be. Oh well, I guess. Nico -- From sacadmin Fri Jun 29 15:01:02 2007 Received: from binky.Central.Sun.COM (binky.Central.Sun.COM [129.153.128.104]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l5TM11PO023275 for ; Fri, 29 Jun 2007 15:01:02 -0700 (PDT) Received: from localhost.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l5TLwAqp004166; Fri, 29 Jun 2007 16:58:10 -0500 (CDT) Received: (from nw141292@localhost) by localhost.Central.Sun.COM (8.14.1+Sun/8.14.1/Submit) id l5TLw6LU004165; Fri, 29 Jun 2007 16:58:06 -0500 (CDT) X-Authentication-Warning: localhost.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f Date: Fri, 29 Jun 2007 16:58:06 -0500 From: Nicolas Williams To: Ienup Sung Cc: PSARC@sac.sfbay.sun.com Subject: Re: UTF-8 text preparation functions [PSARC/2007/149 Timeout: 03/19/2007] Message-ID: <20070629215806.GE2719@Sun.COM> References: <200703130428.l2D4Se8G014218@sparc-build.SFBay.Sun.COM> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200703130428.l2D4Se8G014218@sparc-build.SFBay.Sun.COM> User-Agent: Mutt/1.5.7i Status: RO Content-Length: 43 BTW, we have a need for this in user-land. From sacadmin Fri Jun 29 15:36:50 2007 Received: from sfbaymail2sca.sfbay.sun.com (sfbaymail2sca.SFBay.Sun.COM [129.145.155.42]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l5TMaoCk024556 for ; Fri, 29 Jun 2007 15:36:50 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l5TMZ3t0014233 for ; Fri, 29 Jun 2007 15:35:04 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id l5TMYwfa019447 for ; Fri, 29 Jun 2007 15:34:58 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JKF007014ZVQ200@fe-sfbay-09.sun.com> (original mail from Ienup.Sung@Sun.COM) for PSARC@sac.sfbay.sun.com; Fri, 29 Jun 2007 15:34:58 -0700 (PDT) Received: from [129.145.155.141] by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JKF00JP45E37Q70@fe-sfbay-09.sun.com> for PSARC@sac.sfbay.sun.com; Fri, 29 Jun 2007 15:34:52 -0700 (PDT) Date: Fri, 29 Jun 2007 15:34:51 -0700 From: Ienup Sung Subject: Re: UTF-8 text preparation functions [PSARC/2007/149 Timeout: 03/19/2007] In-reply-to: <20070629215806.GE2719@Sun.COM> Sender: Ienup.Sung@Sun.COM To: PSARC@sac.sfbay.sun.com Reply-to: Ienup.Sung@Sun.COM Message-id: <4685890B.4010108@Sun.COM> Organization: Sun Microsystems, Inc. MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en References: <200703130428.l2D4Se8G014218@sparc-build.SFBay.Sun.COM> <20070629215806.GE2719@Sun.COM> User-Agent: Mozilla/5.0 (X11; U; SunOS sun4v; en-US; rv:1.7) Gecko/20070109 Status: RO Content-Length: 397 Nicolas Williams wrote at 06/29/07 14:58: > BTW, we have a need for this in user-land. Hi Nicolas, Thanks for the requirement. Let's talk off-line on the requirement and timeline. (As you know, the current code is portable and so can be placed into common. The project team will need to gain PSARC approval and also ON and G11N management approvals on several things as you can imagine.) Ienup