From ienup.sung@oracle.com Fri May 7 09:53:34 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o47GrYr8028389 for ; Fri, 7 May 2010 09:53:34 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o47GrXO7028802 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 7 May 2010 11:53:33 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L22006096X9C600@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 07 May 2010 09:53:33 -0700 (PDT) Received: from sca-ea-mail-4.sun.com ([192.18.43.22]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L22004AF6X9IY20@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 07 May 2010 09:53:33 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by sca-ea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o47GrWUY026609; Fri, 07 May 2010 16:53:33 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o47GAkxb018304; Fri, 07 May 2010 16:53:28 +0000 (GMT) Received: from abhmt003.oracle.com by acsmt354.oracle.com with ESMTP id 246977771273251205; Fri, 07 May 2010 09:53:25 -0700 Received: from [192.168.1.93] (/129.150.249.10) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 07 May 2010 09:53:25 -0700 Date: Fri, 07 May 2010 09:50:16 -0700 From: Ienup Sung Subject: Libc iconv enhancement [PSARC/2010/160 FastTrack timeout 05/14/2010] To: PSARC-ext@sun.com Cc: i18n-discuss@opensolaris.org Reply-to: ienup.sung@oracle.com Message-id: <4BE444C8.1090309@oracle.com> Organization: Oracle Corporation MIME-version: 1.0 Content-type: text/plain; charset=UTF-8; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090209.4BE44589.002E:SCFMA4539814,ss=1,fgs=0 User-Agent: Thunderbird 2.0.0.17 (X11/20081010) Status: RO Content-Length: 6777 Template Version: @(#)sac_nextcase 1.70 03/30/10 SMI This information is Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. 1. Introduction 1.1. Project/Component Working Name: Libc iconv enhancement 1.2. Name of Document Author/Supplier: Author: Ienup Sung 1.3 Date of This Document: 07 May, 2010 4. Technical Description OVERVIEW This project extends the current iconv API framework at libc so that it will be compatible with GNU libiconv. It also adds a few additional Solaris specific features such as iconvstr(3C) and additional iconv code conversion behaviors on non-identical and illegal byte sequences as described at below: - iconv_open(3C) will understand and properly handle "" (i.e., empty string), "char", and "wchar_t" as possible values for fromcode and tocode arguments as shown with change bars at [2] and also described in [3]. - iconv_open(3C) will parse fromcode and tocode arguments, understand, and properly handle transliteration and code conversion behavior modification requests when one or more of the following indicators are appended to the names pointed to by the arguments as described with change bars at [2] and also described in [3]: "//ILLEGAL_DISCARD" "//ILLEGAL_REPLACE_HEX" "//ILLEGAL_RESTORE_HEX" "//NON_IDENTICAL_DISCARD" "//NON_IDENTICAL_REPLACE_HEX" "//NON_IDENTICAL_RESTORE_HEX" "//NON_IDENTICAL_TRANSLITERATE" "//IGNORE" (An alias to "//NON_IDENTICAL_DISCARD//ILLEGAL_DISCARD".) "//REPLACE_HEX" (An alias to "//NON_IDENTICAL_REPLACE_HEX//ILLEGAL_REPLACE_HEX".) "//RESTORE_HEX" (An alias to "//NON_IDENTICAL_RESTORE_HEX//ILLEGAL_RESTORE_HEX".) "//TRANSLIT" (An alias to "//NON_IDENTICAL_TRANSLITERATE".) The "//IGNORE" and the "//TRANSLIT" are primarily provided for GNU libiconv compatibility. - The current path-through iconv tocode and fromcode names will go through alias matching mechanism so that the path-through iconvs will also recognize and support alias names. (As an example, iconv_open("8859-1", "ISO-8859-1") will also activate the path-through iconv code conversion unless there is a corresponding iconv code conversion module in the current system.) - iconvctl(3C) will be added as a new function with some additional Solaris extensions as described in [2] and [3]. This is also to be compatible with the same GNU libiconv API. - iconvstr(3C) will be added as a new function as described in [2] and [3]. This is to be compatible with the kernel iconv framework and, in particular, kiconvstr(9F) [10] and also to provide a string-based code conversion scheme in addition to the current buffer-based code conversion scheme of iconv(3C). The main focus of the function at this point is to be compatible with kiconvstr(9F). To support the above, we will add three more Contracted Consolidation Private interfaces defined between libc (owned by ON) and iconv code conversion shared object modules (owned by G11N mostly) as described in [3] and [4]: - _icv_open_attr() - _icv_iconvctl() - _icv_iconvstr() Without any modifications, existing iconv code conversion shared object modules and geniconvtbl binary table driven iconv code conversions will still be supported as they are and be compatible with the new and enhanced iconv framework; they will just not provide the new functionalities specified in this spec. To provide the new functionalities mentioned in this spec, iconv code conversion shared object modules and geniconvtbl binary tables must be updated to supply the necessary changes as specified in [2] and [3]. Once the case is approved by PSARC, the project team will also seek the approvals on the [4] from the responsible engineering managers of the related parties. As a side note, we will do follow-up projects that will update the existing iconv code conversion shared object modules and geniconvtbl-based binary tables and also add necessary iconv code conversions and name aliases to be more compatible with other platforms and systems in terms of code conversion coverage and support of additional codeset/charset names. Lastly, there are iconv_open_into(3C) API and also some extra features in iconv(1) CLI from GNU libiconv that are not included in this project at this time. They are planned to be done as a future project or two in near future as necessary. INTERFACE STABILITY AND RELEASE BINDING There is no notable interface imported. Exported interfaces are: Interface Stability Note --------- --------- ---- iconv_open(3C), Committed [2] iconv(3C), iconvctl(3C), iconvstr(3C), iconv.h(3HEAD), geniconvtbl(4) _icv_open_attr(), Contracted [3], [4] _icv_iconvctl(), Consolidation _icv_iconvstr() Private This project seeks Micro/Patch release binding. REFERENCES [1] Related CRs: 6803313 Solaris iconv does not support transliteration (Rails need transliteration to work on Solaris) 6912982 iconv_open() should allow an empty string / "" / as any argument 6913721 alias supported is wanted for pass-through iconv tocode and fromcode names Bug 11957 - iconv doesn't support wchar_t as an encoding name (http://defect.opensolaris.org/bz/show_bug.cgi?id=11957) [2] Updated and new man pages at the materials directory of the case: iconv_open.3c, iconv_open.3c.diff, iconv.3c, iconv.3c.diff, iconvctl.3c (new man page), iconvstr.3c (new man page), iconv.h.3head, iconv.h.3head.diff, geniconvtbl.4, geniconvtbl.4.diff (See geniconvtbl.4.diff first since geniconvtbl.4 has text with change bars from PSARC/2001/659 too.) [3] Localization guide at the materials directory of the case: iconv-l10n-guide.txt [4] Contract template file at the materials directory of the case: contract-template.txt [5] PSARC/1993/153 iconv/iconv_open/iconv_close [6] PSARC/1999/292 Addition of geniconvtbl(1) [7] PSARC/2001/072 GNU gettext support (For /usr/lib/iconv/alias and alias support mechanism at iconv.) [8] PSARC/2001/659 Non-identical character conversion support in geniconvtbl(1) [9] PSARC/2007/173 kiconv [10] PSARC/2009/561 Pass-through iconv code conversion 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: G11N 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From ienup.sung@oracle.com Mon May 10 12:48:10 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4AJmAMn001294 for ; Mon, 10 May 2010 12:48:10 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4AJm47v024149 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 10 May 2010 14:48:10 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2700C1JZ0AID00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 10 May 2010 12:48:10 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2700FB1Z089ED0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 10 May 2010 12:48:08 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4AJm74U022821 for ; Mon, 10 May 2010 19:48:07 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4AGgQxR015666; Mon, 10 May 2010 19:48:02 +0000 (GMT) Received: from abhmt007.oracle.com by acsmt355.oracle.com with ESMTP id 228877611273520850; Mon, 10 May 2010 12:47:30 -0700 Received: from [192.168.1.93] (/129.150.247.105) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 10 May 2010 12:47:30 -0700 Date: Mon, 10 May 2010 12:44:18 -0700 From: Ienup Sung Subject: Fwd: Libc iconv enhancement [PSARC/2010/160 FastTrack timeout 05/17/2010] To: PSARC-ext@sun.com Cc: i18n-discuss@opensolaris.org Reply-to: ienup.sung@oracle.com Message-id: <4BE86212.20302@oracle.com> Organization: Oracle Corporation MIME-version: 1.0 Content-type: text/plain; charset=UTF-8; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090206.4BE862F3.0093:SCFMA4539814,ss=1,fgs=0 User-Agent: Thunderbird 2.0.0.17 (X11/20081010) Status: RO Content-Length: 7190 I'm re-sending this case due to that, including mailer failures, sac.sfbay wasn't operating properly last week. The timer has been extended from 5/14 (Fri) to 5/17 (Mon). Ienup -------- Original Message -------- Subject: Libc iconv enhancement [PSARC/2010/160 FastTrack timeout 05/14/2010] Date: Fri, 07 May 2010 09:50:16 -0700 From: Ienup Sung Organization: Oracle Corporation To: PSARC-ext CC: i18n-discuss Template Version: @(#)sac_nextcase 1.70 03/30/10 SMI This information is Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. 1. Introduction 1.1. Project/Component Working Name: Libc iconv enhancement 1.2. Name of Document Author/Supplier: Author: Ienup Sung 1.3 Date of This Document: 07 May, 2010 4. Technical Description OVERVIEW This project extends the current iconv API framework at libc so that it will be compatible with GNU libiconv. It also adds a few additional Solaris specific features such as iconvstr(3C) and additional iconv code conversion behaviors on non-identical and illegal byte sequences as described at below: - iconv_open(3C) will understand and properly handle "" (i.e., empty string), "char", and "wchar_t" as possible values for fromcode and tocode arguments as shown with change bars at [2] and also described in [3]. - iconv_open(3C) will parse fromcode and tocode arguments, understand, and properly handle transliteration and code conversion behavior modification requests when one or more of the following indicators are appended to the names pointed to by the arguments as described with change bars at [2] and also described in [3]: "//ILLEGAL_DISCARD" "//ILLEGAL_REPLACE_HEX" "//ILLEGAL_RESTORE_HEX" "//NON_IDENTICAL_DISCARD" "//NON_IDENTICAL_REPLACE_HEX" "//NON_IDENTICAL_RESTORE_HEX" "//NON_IDENTICAL_TRANSLITERATE" "//IGNORE" (An alias to "//NON_IDENTICAL_DISCARD//ILLEGAL_DISCARD".) "//REPLACE_HEX" (An alias to "//NON_IDENTICAL_REPLACE_HEX//ILLEGAL_REPLACE_HEX".) "//RESTORE_HEX" (An alias to "//NON_IDENTICAL_RESTORE_HEX//ILLEGAL_RESTORE_HEX".) "//TRANSLIT" (An alias to "//NON_IDENTICAL_TRANSLITERATE".) The "//IGNORE" and the "//TRANSLIT" are primarily provided for GNU libiconv compatibility. - The current path-through iconv tocode and fromcode names will go through alias matching mechanism so that the path-through iconvs will also recognize and support alias names. (As an example, iconv_open("8859-1", "ISO-8859-1") will also activate the path-through iconv code conversion unless there is a corresponding iconv code conversion module in the current system.) - iconvctl(3C) will be added as a new function with some additional Solaris extensions as described in [2] and [3]. This is also to be compatible with the same GNU libiconv API. - iconvstr(3C) will be added as a new function as described in [2] and [3]. This is to be compatible with the kernel iconv framework and, in particular, kiconvstr(9F) [10] and also to provide a string-based code conversion scheme in addition to the current buffer-based code conversion scheme of iconv(3C). The main focus of the function at this point is to be compatible with kiconvstr(9F). To support the above, we will add three more Contracted Consolidation Private interfaces defined between libc (owned by ON) and iconv code conversion shared object modules (owned by G11N mostly) as described in [3] and [4]: - _icv_open_attr() - _icv_iconvctl() - _icv_iconvstr() Without any modifications, existing iconv code conversion shared object modules and geniconvtbl binary table driven iconv code conversions will still be supported as they are and be compatible with the new and enhanced iconv framework; they will just not provide the new functionalities specified in this spec. To provide the new functionalities mentioned in this spec, iconv code conversion shared object modules and geniconvtbl binary tables must be updated to supply the necessary changes as specified in [2] and [3]. Once the case is approved by PSARC, the project team will also seek the approvals on the [4] from the responsible engineering managers of the related parties. As a side note, we will do follow-up projects that will update the existing iconv code conversion shared object modules and geniconvtbl-based binary tables and also add necessary iconv code conversions and name aliases to be more compatible with other platforms and systems in terms of code conversion coverage and support of additional codeset/charset names. Lastly, there are iconv_open_into(3C) API and also some extra features in iconv(1) CLI from GNU libiconv that are not included in this project at this time. They are planned to be done as a future project or two in near future as necessary. INTERFACE STABILITY AND RELEASE BINDING There is no notable interface imported. Exported interfaces are: Interface Stability Note --------- --------- ---- iconv_open(3C), Committed [2] iconv(3C), iconvctl(3C), iconvstr(3C), iconv.h(3HEAD), geniconvtbl(4) _icv_open_attr(), Contracted [3], [4] _icv_iconvctl(), Consolidation _icv_iconvstr() Private This project seeks Micro/Patch release binding. REFERENCES [1] Related CRs: 6803313 Solaris iconv does not support transliteration (Rails need transliteration to work on Solaris) 6912982 iconv_open() should allow an empty string / "" / as any argument 6913721 alias supported is wanted for pass-through iconv tocode and fromcode names Bug 11957 - iconv doesn't support wchar_t as an encoding name (http://defect.opensolaris.org/bz/show_bug.cgi?id=11957) [2] Updated and new man pages at the materials directory of the case: iconv_open.3c, iconv_open.3c.diff, iconv.3c, iconv.3c.diff, iconvctl.3c (new man page), iconvstr.3c (new man page), iconv.h.3head, iconv.h.3head.diff, geniconvtbl.4, geniconvtbl.4.diff (See geniconvtbl.4.diff first since geniconvtbl.4 has text with change bars from PSARC/2001/659 too.) [3] Localization guide at the materials directory of the case: iconv-l10n-guide.txt [4] Contract template file at the materials directory of the case: contract-template.txt [5] PSARC/1993/153 iconv/iconv_open/iconv_close [6] PSARC/1999/292 Addition of geniconvtbl(1) [7] PSARC/2001/072 GNU gettext support (For /usr/lib/iconv/alias and alias support mechanism at iconv.) [8] PSARC/2001/659 Non-identical character conversion support in geniconvtbl(1) [9] PSARC/2007/173 kiconv [10] PSARC/2009/561 Pass-through iconv code conversion 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: G11N 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From ienup.sung@oracle.com Mon May 10 13:40:04 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o47GrYr8028389 for ; Fri, 7 May 2010 09:53:34 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o47GrXO7028802 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Fri, 7 May 2010 11:53:33 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L22006096X9C600@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 07 May 2010 09:53:33 -0700 (PDT) Received: from sca-ea-mail-4.sun.com ([192.18.43.22]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L22004AF6X9IY20@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Fri, 07 May 2010 09:53:33 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by sca-ea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o47GrWUY026609; Fri, 07 May 2010 16:53:33 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o47GAkxb018304; Fri, 07 May 2010 16:53:28 +0000 (GMT) Received: from abhmt003.oracle.com by acsmt354.oracle.com with ESMTP id 246977771273251205; Fri, 07 May 2010 09:53:25 -0700 Received: from [192.168.1.93] (/129.150.249.10) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 07 May 2010 09:53:25 -0700 Date: Fri, 07 May 2010 09:50:16 -0700 From: Ienup Sung Subject: Libc iconv enhancement [PSARC/2010/160 FastTrack timeout 05/14/2010] To: PSARC-ext@sun.com Cc: i18n-discuss@opensolaris.org Reply-to: ienup.sung@oracle.com Message-id: <4BE444C8.1090309@oracle.com> Organization: Oracle Corporation MIME-version: 1.0 Content-type: text/plain; charset=UTF-8; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090209.4BE44589.002E:SCFMA4539814,ss=1,fgs=0 User-Agent: Thunderbird 2.0.0.17 (X11/20081010) Status: RO Content-Length: 6777 Template Version: @(#)sac_nextcase 1.70 03/30/10 SMI This information is Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. 1. Introduction 1.1. Project/Component Working Name: Libc iconv enhancement 1.2. Name of Document Author/Supplier: Author: Ienup Sung 1.3 Date of This Document: 07 May, 2010 4. Technical Description OVERVIEW This project extends the current iconv API framework at libc so that it will be compatible with GNU libiconv. It also adds a few additional Solaris specific features such as iconvstr(3C) and additional iconv code conversion behaviors on non-identical and illegal byte sequences as described at below: - iconv_open(3C) will understand and properly handle "" (i.e., empty string), "char", and "wchar_t" as possible values for fromcode and tocode arguments as shown with change bars at [2] and also described in [3]. - iconv_open(3C) will parse fromcode and tocode arguments, understand, and properly handle transliteration and code conversion behavior modification requests when one or more of the following indicators are appended to the names pointed to by the arguments as described with change bars at [2] and also described in [3]: "//ILLEGAL_DISCARD" "//ILLEGAL_REPLACE_HEX" "//ILLEGAL_RESTORE_HEX" "//NON_IDENTICAL_DISCARD" "//NON_IDENTICAL_REPLACE_HEX" "//NON_IDENTICAL_RESTORE_HEX" "//NON_IDENTICAL_TRANSLITERATE" "//IGNORE" (An alias to "//NON_IDENTICAL_DISCARD//ILLEGAL_DISCARD".) "//REPLACE_HEX" (An alias to "//NON_IDENTICAL_REPLACE_HEX//ILLEGAL_REPLACE_HEX".) "//RESTORE_HEX" (An alias to "//NON_IDENTICAL_RESTORE_HEX//ILLEGAL_RESTORE_HEX".) "//TRANSLIT" (An alias to "//NON_IDENTICAL_TRANSLITERATE".) The "//IGNORE" and the "//TRANSLIT" are primarily provided for GNU libiconv compatibility. - The current path-through iconv tocode and fromcode names will go through alias matching mechanism so that the path-through iconvs will also recognize and support alias names. (As an example, iconv_open("8859-1", "ISO-8859-1") will also activate the path-through iconv code conversion unless there is a corresponding iconv code conversion module in the current system.) - iconvctl(3C) will be added as a new function with some additional Solaris extensions as described in [2] and [3]. This is also to be compatible with the same GNU libiconv API. - iconvstr(3C) will be added as a new function as described in [2] and [3]. This is to be compatible with the kernel iconv framework and, in particular, kiconvstr(9F) [10] and also to provide a string-based code conversion scheme in addition to the current buffer-based code conversion scheme of iconv(3C). The main focus of the function at this point is to be compatible with kiconvstr(9F). To support the above, we will add three more Contracted Consolidation Private interfaces defined between libc (owned by ON) and iconv code conversion shared object modules (owned by G11N mostly) as described in [3] and [4]: - _icv_open_attr() - _icv_iconvctl() - _icv_iconvstr() Without any modifications, existing iconv code conversion shared object modules and geniconvtbl binary table driven iconv code conversions will still be supported as they are and be compatible with the new and enhanced iconv framework; they will just not provide the new functionalities specified in this spec. To provide the new functionalities mentioned in this spec, iconv code conversion shared object modules and geniconvtbl binary tables must be updated to supply the necessary changes as specified in [2] and [3]. Once the case is approved by PSARC, the project team will also seek the approvals on the [4] from the responsible engineering managers of the related parties. As a side note, we will do follow-up projects that will update the existing iconv code conversion shared object modules and geniconvtbl-based binary tables and also add necessary iconv code conversions and name aliases to be more compatible with other platforms and systems in terms of code conversion coverage and support of additional codeset/charset names. Lastly, there are iconv_open_into(3C) API and also some extra features in iconv(1) CLI from GNU libiconv that are not included in this project at this time. They are planned to be done as a future project or two in near future as necessary. INTERFACE STABILITY AND RELEASE BINDING There is no notable interface imported. Exported interfaces are: Interface Stability Note --------- --------- ---- iconv_open(3C), Committed [2] iconv(3C), iconvctl(3C), iconvstr(3C), iconv.h(3HEAD), geniconvtbl(4) _icv_open_attr(), Contracted [3], [4] _icv_iconvctl(), Consolidation _icv_iconvstr() Private This project seeks Micro/Patch release binding. REFERENCES [1] Related CRs: 6803313 Solaris iconv does not support transliteration (Rails need transliteration to work on Solaris) 6912982 iconv_open() should allow an empty string / "" / as any argument 6913721 alias supported is wanted for pass-through iconv tocode and fromcode names Bug 11957 - iconv doesn't support wchar_t as an encoding name (http://defect.opensolaris.org/bz/show_bug.cgi?id=11957) [2] Updated and new man pages at the materials directory of the case: iconv_open.3c, iconv_open.3c.diff, iconv.3c, iconv.3c.diff, iconvctl.3c (new man page), iconvstr.3c (new man page), iconv.h.3head, iconv.h.3head.diff, geniconvtbl.4, geniconvtbl.4.diff (See geniconvtbl.4.diff first since geniconvtbl.4 has text with change bars from PSARC/2001/659 too.) [3] Localization guide at the materials directory of the case: iconv-l10n-guide.txt [4] Contract template file at the materials directory of the case: contract-template.txt [5] PSARC/1993/153 iconv/iconv_open/iconv_close [6] PSARC/1999/292 Addition of geniconvtbl(1) [7] PSARC/2001/072 GNU gettext support (For /usr/lib/iconv/alias and alias support mechanism at iconv.) [8] PSARC/2001/659 Non-identical character conversion support in geniconvtbl(1) [9] PSARC/2007/173 kiconv [10] PSARC/2009/561 Pass-through iconv code conversion 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: G11N 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From is@sac.sfbay.sun.com Mon May 10 13:43:57 2010 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4AKhvlw023187 for ; Mon, 10 May 2010 13:43:57 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4AKhuST017464; Mon, 10 May 2010 13:43:57 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L280040H1L85000@nwk-avmta-1.sfbay.Sun.COM>; Mon, 10 May 2010 13:43:56 -0700 (PDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L28001ZO1L8H230@nwk-avmta-1.sfbay.Sun.COM>; Mon, 10 May 2010 13:43:56 -0700 (PDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4AKc6uW009599; Mon, 10 May 2010 13:43:55 -0700 (PDT) Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o47GS446028100; Fri, 07 May 2010 09:28:04 -0700 (PDT) Received: (from is@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id o47GS4pA028096; Fri, 07 May 2010 09:28:04 -0700 (PDT) Date: Fri, 07 May 2010 09:28:04 -0700 (PDT) From: Ienup Sung Subject: Libc iconv enhancement [PSARC/2010/160 FastTrack timeout 05/14/2010] To: PSARC-ext@sun.com Cc: i18n-discuss@opensolaris.org Message-id: <201005071628.o47GS4pA028096@sac.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 6713 Template Version: @(#)sac_nextcase 1.70 03/30/10 SMI This information is Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. 1. Introduction 1.1. Project/Component Working Name: Libc iconv enhancement 1.2. Name of Document Author/Supplier: Author: Ienup Sung 1.3 Date of This Document: 07 May, 2010 4. Technical Description OVERVIEW This project extends the current iconv API framework at libc so that it will be compatible with GNU libiconv. It also adds a few additional Solaris specific features such as iconvstr(3C) and additional iconv code conversion behaviors on non-identical and illegal byte sequences as described at below: - iconv_open(3C) will understand and properly handle "" (i.e., empty string), "char", and "wchar_t" as possible values for fromcode and tocode arguments as shown with change bars at [2] and also described in [3]. - iconv_open(3C) will parse fromcode and tocode arguments, understand, and properly handle transliteration and code conversion behavior modification requests when one or more of the following indicators are appended to the names pointed to by the arguments as described with change bars at [2] and also described in [3]: "//ILLEGAL_DISCARD" "//ILLEGAL_REPLACE_HEX" "//ILLEGAL_RESTORE_HEX" "//NON_IDENTICAL_DISCARD" "//NON_IDENTICAL_REPLACE_HEX" "//NON_IDENTICAL_RESTORE_HEX" "//NON_IDENTICAL_TRANSLITERATE" "//IGNORE" (An alias to "//NON_IDENTICAL_DISCARD//ILLEGAL_DISCARD".) "//REPLACE_HEX" (An alias to "//NON_IDENTICAL_REPLACE_HEX//ILLEGAL_REPLACE_HEX".) "//RESTORE_HEX" (An alias to "//NON_IDENTICAL_RESTORE_HEX//ILLEGAL_RESTORE_HEX".) "//TRANSLIT" (An alias to "//NON_IDENTICAL_TRANSLITERATE".) The "//IGNORE" and the "//TRANSLIT" are primarily provided for GNU libiconv compatibility. - The current path-through iconv tocode and fromcode names will go through alias matching mechanism so that the path-through iconvs will also recognize and support alias names. (As an example, iconv_open("8859-1", "ISO-8859-1") will also activate the path-through iconv code conversion unless there is a corresponding iconv code conversion module in the current system.) - iconvctl(3C) will be added as a new function with some additional Solaris extensions as described in [2] and [3]. This is also to be compatible with the same GNU libiconv API. - iconvstr(3C) will be added as a new function as described in [2] and [3]. This is to be compatible with the kernel iconv framework and, in particular, kiconvstr(9F) [10] and also to provide a string-based code conversion scheme in addition to the current buffer-based code conversion scheme of iconv(3C). The main focus of the function at this point is to be compatible with kiconvstr(9F). To support the above, we will add three more Contracted Consolidation Private interfaces defined between libc (owned by ON) and iconv code conversion shared object modules (owned by G11N mostly) as described in [3] and [4]: - _icv_open_attr() - _icv_iconvctl() - _icv_iconvstr() Without any modifications, existing iconv code conversion shared object modules and geniconvtbl binary table driven iconv code conversions will still be supported as they are and be compatible with the new and enhanced iconv framework; they will just not provide the new functionalities specified in this spec. To provide the new functionalities mentioned in this spec, iconv code conversion shared object modules and geniconvtbl binary tables must be updated to supply the necessary changes as specified in [2] and [3]. Once the case is approved by PSARC, the project team will also seek the approvals on the [4] from the responsible engineering managers of the related parties. As a side note, we will do follow-up projects that will update the existing iconv code conversion shared object modules and geniconvtbl-based binary tables and also add necessary iconv code conversions and name aliases to be more compatible with other platforms and systems in terms of code conversion coverage and support of additional codeset/charset names. Lastly, there are iconv_open_into(3C) API and also some extra features in iconv(1) CLI from GNU libiconv that are not included in this project at this time. They are planned to be done as a future project or two in near future as necessary. INTERFACE STABILITY AND RELEASE BINDING There is no notable interface imported. Exported interfaces are: Interface Stability Note --------- --------- ---- iconv_open(3C), Committed [2] iconv(3C), iconvctl(3C), iconvstr(3C), iconv.h(3HEAD), geniconvtbl(4) _icv_open_attr(), Contracted [3], [4] _icv_iconvctl(), Consolidation _icv_iconvstr() Private This project seeks Micro/Patch release binding. REFERENCES [1] Related CRs: 6803313 Solaris iconv does not support transliteration (Rails need transliteration to work on Solaris) 6912982 iconv_open() should allow an empty string / "" / as any argument 6913721 alias supported is wanted for pass-through iconv tocode and fromcode names Bug 11957 - iconv doesn't support wchar_t as an encoding name (http://defect.opensolaris.org/bz/show_bug.cgi?id=11957) [2] Updated and new man pages at the materials directory of the case: iconv_open.3c, iconv_open.3c.diff, iconv.3c, iconv.3c.diff, iconvctl.3c (new man page), iconvstr.3c (new man page), iconv.h.3head, iconv.h.3head.diff, geniconvtbl.4, geniconvtbl.4.diff (See geniconvtbl.4.diff first since geniconvtbl.4 has text with change bars from PSARC/2001/659 too.) [3] Localization guide at the materials directory of the case: iconv-l10n-guide.txt [4] Contract template file at the materials directory of the case: contract-template.txt [5] PSARC/1993/153 iconv/iconv_open/iconv_close [6] PSARC/1999/292 Addition of geniconvtbl(1) [7] PSARC/2001/072 GNU gettext support (For /usr/lib/iconv/alias and alias support mechanism at iconv.) [8] PSARC/2001/659 Non-identical character conversion support in geniconvtbl(1) [9] PSARC/2007/173 kiconv [10] PSARC/2009/561 Pass-through iconv code conversion 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: G11N 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From michael.kearney@oracle.com Tue May 11 06:47:58 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4BDlvdO016201 for ; Tue, 11 May 2010 06:47:57 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4BDlvNe010146 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 11 May 2010 08:47:57 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2900J0RCZXCV00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 11 May 2010 06:47:57 -0700 (PDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2900JHQCZW6P00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 11 May 2010 06:47:56 -0700 (PDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4BDltST023437 for ; Tue, 11 May 2010 13:47:55 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4BDlk8D026820; Tue, 11 May 2010 13:47:47 +0000 (GMT) Received: from abhmt016.oracle.com by acsmt353.oracle.com with ESMTP id 231342831273585577; Tue, 11 May 2010 06:46:17 -0700 Received: from [129.156.38.62] (/129.156.38.62) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 11 May 2010 06:46:16 -0700 Date: Tue, 11 May 2010 14:46:14 +0100 From: Michael Kearney Subject: Re: Fwd: Libc iconv enhancement [PSARC/2010/160 FastTrack timeout 05/17/2010] In-reply-to: <4BE86212.20302@oracle.com> To: ienup.sung@oracle.com Cc: PSARC-ext@sun.com, i18n-discuss@opensolaris.org Message-id: <4BE95FA6.3010903@oracle.com> MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_wSNuQUgF0EtQ5Ma5+4HXXw)" X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090204.4BE96008.0142:SCFMA4539814,ss=1,fgs=0 References: <4BE86212.20302@oracle.com> User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) Status: RO Content-Length: 18625 This is a multi-part message in MIME format. --Boundary_(ID_wSNuQUgF0EtQ5Ma5+4HXXw) Content-type: text/plain; charset=UTF-8; format=flowed Content-transfer-encoding: 7BIT I've reviewed the case and can give it a +1. Regards, Michael On 5/10/2010 8:44 PM, Ienup Sung wrote: > I'm re-sending this case due to that, including mailer failures, > sac.sfbay > wasn't operating properly last week. > > The timer has been extended from 5/14 (Fri) to 5/17 (Mon). > > Ienup > > -------- Original Message -------- > Subject: Libc iconv enhancement [PSARC/2010/160 FastTrack timeout > 05/14/2010] > Date: Fri, 07 May 2010 09:50:16 -0700 > From: Ienup Sung > Organization: Oracle Corporation > To: PSARC-ext > CC: i18n-discuss > > > Template Version: @(#)sac_nextcase 1.70 03/30/10 SMI > This information is Copyright (c) 2010, Oracle and/or its affiliates. > All rights > reserved. > 1. Introduction > 1.1. Project/Component Working Name: > Libc iconv enhancement > 1.2. Name of Document Author/Supplier: > Author: Ienup Sung > 1.3 Date of This Document: > 07 May, 2010 > 4. Technical Description > > OVERVIEW > > This project extends the current iconv API framework at libc so that > it will > be compatible with GNU libiconv. It also adds a few additional Solaris > specific > features such as iconvstr(3C) and additional iconv code conversion > behaviors on > non-identical and illegal byte sequences as described at below: > > - iconv_open(3C) will understand and properly handle "" (i.e., empty > string), > "char", and "wchar_t" as possible values for fromcode and tocode > arguments as > shown with change bars at [2] and also described in [3]. > > - iconv_open(3C) will parse fromcode and tocode arguments, understand, > and > properly handle transliteration and code conversion behavior > modification > requests when one or more of the following indicators are appended to > the names pointed to by the arguments as described with change bars > at [2] and also described in [3]: > > "//ILLEGAL_DISCARD" > "//ILLEGAL_REPLACE_HEX" > "//ILLEGAL_RESTORE_HEX" > > "//NON_IDENTICAL_DISCARD" > "//NON_IDENTICAL_REPLACE_HEX" > "//NON_IDENTICAL_RESTORE_HEX" > "//NON_IDENTICAL_TRANSLITERATE" > > "//IGNORE" (An alias to > "//NON_IDENTICAL_DISCARD//ILLEGAL_DISCARD".) > "//REPLACE_HEX" (An alias to > "//NON_IDENTICAL_REPLACE_HEX//ILLEGAL_REPLACE_HEX".) > "//RESTORE_HEX" (An alias to > "//NON_IDENTICAL_RESTORE_HEX//ILLEGAL_RESTORE_HEX".) > "//TRANSLIT" (An alias to "//NON_IDENTICAL_TRANSLITERATE".) > > The "//IGNORE" and the "//TRANSLIT" are primarily provided for GNU > libiconv > compatibility. > > - The current path-through iconv tocode and fromcode names will go > through > alias matching mechanism so that the path-through iconvs will also > recognize > and support alias names. (As an example, iconv_open("8859-1", > "ISO-8859-1") > will also activate the path-through iconv code conversion unless > there is > a corresponding iconv code conversion module in the current system.) > > - iconvctl(3C) will be added as a new function with some additional > Solaris > extensions as described in [2] and [3]. This is also to be > compatible with > the same GNU libiconv API. > > - iconvstr(3C) will be added as a new function as described in [2] and > [3]. > This is to be compatible with the kernel iconv framework and, in > particular, > kiconvstr(9F) [10] and also to provide a string-based code > conversion > scheme in addition to the current buffer-based code conversion > scheme of > iconv(3C). The main focus of the function at this point is to be > compatible > with kiconvstr(9F). > > To support the above, we will add three more Contracted Consolidation > Private > interfaces defined between libc (owned by ON) and iconv code > conversion shared > object modules (owned by G11N mostly) as described in [3] and [4]: > > - _icv_open_attr() > > - _icv_iconvctl() > > - _icv_iconvstr() > > Without any modifications, existing iconv code conversion shared object > modules and geniconvtbl binary table driven iconv code conversions > will still > be supported as they are and be compatible with the new and enhanced > iconv > framework; they will just not provide the new functionalities > specified in > this spec. > > To provide the new functionalities mentioned in this spec, iconv code > conversion shared object modules and geniconvtbl binary tables must be > updated > to supply the necessary changes as specified in [2] and [3]. > > Once the case is approved by PSARC, the project team will also seek > the approvals on the [4] from the responsible engineering managers of > the related parties. > > As a side note, we will do follow-up projects that will update the > existing > iconv code conversion shared object modules and geniconvtbl-based > binary tables > and also add necessary iconv code conversions and name aliases to be more > compatible with other platforms and systems in terms of code conversion > coverage and support of additional codeset/charset names. > > Lastly, there are iconv_open_into(3C) API and also some extra features > in iconv(1) CLI from GNU libiconv that are not included in this > project at this > time. They are planned to be done as a future project or two in near > future > as necessary. > > > INTERFACE STABILITY AND RELEASE BINDING > > There is no notable interface imported. Exported interfaces are: > > Interface Stability Note > --------- --------- ---- > iconv_open(3C), Committed [2] > iconv(3C), > iconvctl(3C), > iconvstr(3C), > iconv.h(3HEAD), > geniconvtbl(4) > > _icv_open_attr(), Contracted [3], [4] > _icv_iconvctl(), Consolidation > _icv_iconvstr() Private > > This project seeks Micro/Patch release binding. > > > REFERENCES > > [1] Related CRs: > 6803313 Solaris iconv does not support transliteration (Rails > need > transliteration to work on Solaris) > 6912982 iconv_open() should allow an empty string / "" / as > any argument > 6913721 alias supported is wanted for pass-through iconv > tocode and > fromcode names > Bug 11957 - iconv doesn't support wchar_t as an encoding name > (http://defect.opensolaris.org/bz/show_bug.cgi?id=11957) > [2] Updated and new man pages at the materials directory of the case: > iconv_open.3c, iconv_open.3c.diff, > iconv.3c, iconv.3c.diff, > iconvctl.3c (new man page), > iconvstr.3c (new man page), > iconv.h.3head, iconv.h.3head.diff, > geniconvtbl.4, geniconvtbl.4.diff (See geniconvtbl.4.diff > first since > geniconvtbl.4 has text with change bars from PSARC/2001/659 too.) > [3] Localization guide at the materials directory of the case: > iconv-l10n-guide.txt > [4] Contract template file at the materials directory of the case: > contract-template.txt > [5] PSARC/1993/153 iconv/iconv_open/iconv_close > [6] PSARC/1999/292 Addition of geniconvtbl(1) > [7] PSARC/2001/072 GNU gettext support > (For /usr/lib/iconv/alias and alias support mechanism at iconv.) > [8] PSARC/2001/659 Non-identical character conversion support in > geniconvtbl(1) > [9] PSARC/2007/173 kiconv > [10] PSARC/2009/561 Pass-through iconv code conversion > > > 6. Resources and Schedule > 6.4. Steering Committee requested information > 6.4.1. Consolidation C-team Name: > G11N > 6.5. ARC review type: FastTrack > 6.6. ARC Exposure: open > > > -- * Michael Kearney * Principal Software Engineer *Oracle Corp.* MS UBRM05-390, 500 Eldorado Blvd Broomfield, CO 80021 US Phone 303-272-2402 Fax 303-272-6554 Email Michael.Kearney@Oracle.COM --Boundary_(ID_wSNuQUgF0EtQ5Ma5+4HXXw) Content-type: text/html; charset=UTF-8 Content-transfer-encoding: 8BIT I've reviewed the case and can give it a +1.

    Regards,
       Michael

On 5/10/2010 8:44 PM, Ienup Sung wrote:
I'm re-sending this case due to that, including mailer failures, sac.sfbay
wasn't operating properly last week.

The timer has been extended from 5/14 (Fri) to 5/17 (Mon).

Ienup

-------- Original Message --------
Subject: Libc iconv enhancement [PSARC/2010/160 FastTrack timeout 05/14/2010]
Date: Fri, 07 May 2010 09:50:16 -0700
From: Ienup Sung
Organization: Oracle Corporation
To: PSARC-ext
CC: i18n-discuss


Template Version: @(#)sac_nextcase 1.70 03/30/10 SMI
This information is Copyright (c) 2010, Oracle and/or its affiliates. All rights
reserved.
1. Introduction
    1.1. Project/Component Working Name:
     Libc iconv enhancement
    1.2. Name of Document Author/Supplier:
     Author:  Ienup Sung
    1.3  Date of This Document:
    07 May, 2010
4. Technical Description

OVERVIEW

This project extends the current iconv API framework at libc so that it will
be compatible with GNU libiconv. It also adds a few additional Solaris specific
features such as iconvstr(3C) and additional iconv code conversion behaviors on
non-identical and illegal byte sequences as described at below:

- iconv_open(3C) will understand and properly handle "" (i.e., empty string),
  "char", and "wchar_t" as possible values for fromcode and tocode arguments as
  shown with change bars at [2] and also described in [3].

- iconv_open(3C) will parse fromcode and tocode arguments, understand, and
  properly handle transliteration and code conversion behavior modification
  requests when one or more of the following indicators are appended to
  the names pointed to by the arguments as described with change bars
  at [2] and also described in [3]:

        "//ILLEGAL_DISCARD"
        "//ILLEGAL_REPLACE_HEX"
        "//ILLEGAL_RESTORE_HEX"

        "//NON_IDENTICAL_DISCARD"
        "//NON_IDENTICAL_REPLACE_HEX"
        "//NON_IDENTICAL_RESTORE_HEX"
        "//NON_IDENTICAL_TRANSLITERATE"

        "//IGNORE" (An alias to "//NON_IDENTICAL_DISCARD//ILLEGAL_DISCARD".)
        "//REPLACE_HEX" (An alias to
                "//NON_IDENTICAL_REPLACE_HEX//ILLEGAL_REPLACE_HEX".)
        "//RESTORE_HEX" (An alias to
                "//NON_IDENTICAL_RESTORE_HEX//ILLEGAL_RESTORE_HEX".)
        "//TRANSLIT" (An alias to "//NON_IDENTICAL_TRANSLITERATE".)

  The "//IGNORE" and the "//TRANSLIT" are primarily provided for GNU libiconv
  compatibility.

- The current path-through iconv tocode and fromcode names will go through
  alias matching mechanism so that the path-through iconvs will also recognize
  and support alias names. (As an example, iconv_open("8859-1", "ISO-8859-1")
  will also activate the path-through iconv code conversion unless there is
  a corresponding iconv code conversion module in the current system.)

- iconvctl(3C) will be added as a new function with some additional Solaris
  extensions as described in [2] and [3]. This is also to be compatible with
  the same GNU libiconv API.

- iconvstr(3C) will be added as a new function as described in [2] and [3].
  This is to be compatible with the kernel iconv framework and, in particular,
  kiconvstr(9F) [10]  and  also to provide  a string-based  code conversion
  scheme in addition to the current buffer-based code conversion scheme of
  iconv(3C). The main focus of the function at this point is to be compatible
  with kiconvstr(9F).

To support the above, we will add three more Contracted Consolidation Private
interfaces defined between libc (owned by ON) and iconv code conversion shared
object modules (owned by G11N mostly) as described in [3] and [4]:

- _icv_open_attr()

- _icv_iconvctl()

- _icv_iconvstr()

Without any modifications, existing iconv code conversion shared object
modules and geniconvtbl binary table driven iconv code conversions will still
be supported as they are and be compatible with the new and enhanced iconv
framework; they will just not provide the new functionalities specified in
this spec.

To provide the new functionalities mentioned in this spec, iconv code
conversion shared object modules and geniconvtbl binary tables must be updated
to supply the necessary changes as specified in [2] and [3].

Once the case is approved by PSARC, the project team will also seek
the approvals on the [4] from the responsible engineering managers of
the related parties.

As a side note, we will do follow-up projects that will update the existing
iconv code conversion shared object modules and geniconvtbl-based binary tables
and also add necessary iconv code conversions and name aliases to be more
compatible with other platforms and systems in terms of code conversion
coverage and support of additional codeset/charset names.

Lastly, there are iconv_open_into(3C) API and also some extra features
in iconv(1) CLI from GNU libiconv that are not included in this project at this
time. They are planned to be done as a future project or two in near future
as necessary.


INTERFACE STABILITY AND RELEASE BINDING

There is no notable interface imported. Exported interfaces are:

    Interface                     Stability             Note
    ---------                     ---------             ----
    iconv_open(3C),               Committed             [2]
        iconv(3C),
        iconvctl(3C),
        iconvstr(3C),
        iconv.h(3HEAD),
        geniconvtbl(4)

    _icv_open_attr(),             Contracted            [3], [4]
        _icv_iconvctl(),              Consolidation
        _icv_iconvstr()               Private

This project seeks Micro/Patch release binding.


REFERENCES

[1] Related CRs:
        6803313 Solaris iconv does not support transliteration (Rails need
                transliteration to work on Solaris)
        6912982 iconv_open() should allow an empty string / "" / as any argument
        6913721 alias supported is wanted for pass-through iconv tocode and
                fromcode names
        Bug 11957 -  iconv doesn't support wchar_t as an encoding name
        (http://defect.opensolaris.org/bz/show_bug.cgi?id=11957)
[2] Updated and new man pages at the materials directory of the case:
        iconv_open.3c, iconv_open.3c.diff,
        iconv.3c, iconv.3c.diff,
        iconvctl.3c (new man page),
        iconvstr.3c (new man page),
        iconv.h.3head, iconv.h.3head.diff,
        geniconvtbl.4, geniconvtbl.4.diff (See geniconvtbl.4.diff first since
        geniconvtbl.4 has text with change bars from PSARC/2001/659 too.)
[3] Localization guide at the materials directory of the case:
        iconv-l10n-guide.txt
[4] Contract template file at the materials directory of the case:
        contract-template.txt
[5] PSARC/1993/153 iconv/iconv_open/iconv_close
[6] PSARC/1999/292 Addition of geniconvtbl(1)
[7] PSARC/2001/072 GNU gettext support
        (For /usr/lib/iconv/alias and alias support mechanism at iconv.)
[8] PSARC/2001/659 Non-identical character conversion support in geniconvtbl(1)
[9] PSARC/2007/173 kiconv
[10] PSARC/2009/561 Pass-through iconv code conversion


6. Resources and Schedule
    6.4. Steering Committee requested information
       6.4.1. Consolidation C-team Name:
        G11N
    6.5. ARC review type: FastTrack
    6.6. ARC Exposure: open




--
Michael Kearney
Principal Software Engineer

Oracle Corp.
MS UBRM05-390, 500 Eldorado Blvd
Broomfield, CO 80021 US
Phone 303-272-2402
Fax 303-272-6554
Email Michael.Kearney@Oracle.COM
--Boundary_(ID_wSNuQUgF0EtQ5Ma5+4HXXw)-- From ienup.sung@oracle.com Mon May 17 17:04:40 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4I04dfj013381 for ; Mon, 17 May 2010 17:04:40 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4I04caG021971 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 17 May 2010 19:04:39 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2L00A6X9JQ0Y00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 17 May 2010 17:04:38 -0700 (PDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2L007N19JO4P30@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 17 May 2010 17:04:36 -0700 (PDT) Received: from rcsinet13.oracle.com (rcsinet13.oracle.com [148.87.113.125]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4I04ZXN011443 for ; Tue, 18 May 2010 00:04:35 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by rcsinet13.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4HHrGiq027785; Tue, 18 May 2010 00:04:28 +0000 (GMT) Received: from abhmt021.oracle.com by acsmt355.oracle.com with ESMTP id 272519041274141046; Mon, 17 May 2010 17:04:06 -0700 Received: from [192.168.1.93] (/129.150.236.55) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 17 May 2010 17:04:06 -0700 Date: Mon, 17 May 2010 17:02:35 -0700 From: Ienup Sung Subject: Re: Fwd: Libc iconv enhancement [PSARC/2010/160 FastTrack timeout 05/17/2010] In-reply-to: <4BE86212.20302@oracle.com> To: PSARC-ext@sun.com Cc: i18n-discuss@opensolaris.org Reply-to: ienup.sung@oracle.com Message-id: <4BF1D91B.5050000@oracle.com> Organization: Oracle Corporation MIME-version: 1.0 Content-type: text/plain; charset=UTF-8; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090202.4BF1D98D.01B0:SCFMA4539814,ss=1,fgs=0 References: <4BE86212.20302@oracle.com> User-Agent: Thunderbird 2.0.0.17 (X11/20081010) Status: RO Content-Length: 105 The timer for this case expired with a +1 and no open issue. I marked this case closed approved. Ienup From sacadmin Wed May 19 10:10:58 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4JHAvh6008542 for ; Wed, 19 May 2010 10:10:57 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4JHAusR003051; Wed, 19 May 2010 10:10:57 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2O00J0PFQ92600@nwk-avmta-2.sfbay.sun.com>; Wed, 19 May 2010 10:10:57 -0700 (PDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2O00J8TFQ81600@nwk-avmta-2.sfbay.sun.com>; Wed, 19 May 2010 10:10:56 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id o4JHAuP6014804; Wed, 19 May 2010 17:10:56 +0000 (GMT) Received: from acsmt354.oracle.com (acsmt354.oracle.com [141.146.40.154]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4JH6Nfe027794; Wed, 19 May 2010 17:10:54 +0000 (GMT) Received: from abhmt018.oracle.com by acsmt354.oracle.com with ESMTP id 281708031274288995; Wed, 19 May 2010 10:09:55 -0700 Received: from [192.168.1.93] (/129.150.250.205) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 19 May 2010 10:09:55 -0700 Date: Wed, 19 May 2010 10:08:47 -0700 From: Ienup Sung Subject: Re: [Fwd: [Fwd: Re: Fwd: Libc iconv enhancement [PSARC/2010/160 FastTrack timeout 05/17/2010]]] In-reply-to: To: psarc-record@Sun.COM Cc: Joseph George , Shinobu Matsuzuka Reply-to: ienup.sung@oracle.com Message-id: <4BF41B1F.7020803@oracle.com> Organization: Oracle Corporation MIME-version: 1.0 Content-type: text/plain; charset=UTF-8; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt354.oracle.com [141.146.40.154] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090205.4BF41B9F.011A:SCFMA4539814,ss=1,fgs=0 References: <4BF1EC30.5020305@oracle.com> <4BF22955.9070408@oracle.com> User-Agent: Thunderbird 2.0.0.17 (X11/20081010) Status: RO Content-Length: 2724 Thanks very much, Joseph and Shinobu. I created the contract-01 file at the case directory of PSARC/2010/160 and hereby recording the agreement to the contract in the mail archive of the case. Ienup Joseph George wrote at 05/18/10 18:53: > Hi Ienup, > > I approve. > > Thanks, > Joe > > On May 18, 2010, at 11:14 AM, Shinobu Matsuzuka wrote: > >> Hi Ienup, >> >> I approve. >> >> Thanks, >> -shinobu >> >> On 5/17/10 6:24 PM, Ienup Sung wrote: >>> >>> -------- Original Message -------- >>> Subject: [Fwd: Re: Fwd: Libc iconv enhancement [PSARC/2010/160 FastTrack >>> timeout 05/17/2010]] >>> Date: Mon, 17 May 2010 18:16:44 -0700 >>> From: Ienup Sung >>> Reply-To: ienup.sung >>> Organization: Oracle Corporation >>> To: joe.g, shinobu.matsuzuka >>> >>> Hello Joseph, Shinobu, >>> >>> The following PSARC case got approval today. This project enhances our >>> iconv framework at libc so that it will be able to compete with GNU >>> libiconv >>> and also compatible with their implementation. >>> >>> To do so, I had to add a few additional internal methods between the libc >>> and the iconv shared objects that are dynamically loaded into memory during >>> runtime based on the iconv_open(tocode, fromcode) arguments. >>> >>> This also causes us to classify them as Contracted Consolidation Private >>> interfaces and due to that, we need to establish a contract between >>> ON Sustaining and G11N as described at the attached contract. It is >>> basically >>> a routine and standard contract except it states that G11N will maintain >>> the new feature (since G11N is adding the new feature) while it is owned by >>> ON Sustaining. >>> >>> Please review and, if okay with you, approve the contract and thanks very >>> much. >>> >>> Once I get "I approve" emails from both of you, I'll record it to PSARC >>> with you cc'd in. >>> >>> For more details on the technical details, please refer to spec.txt at >>> below and also man pages and iconv-l10n-guide.txt at the same directory: >>> >>> http://sac.sfbay/arc/PSARC/2010/160/materials/ >>> spec.txt >>> iconv-l10n-guide.txt >>> *.3c, *.3head, *.4 >>> >>> This was reviewed by GTO within G11N and Nobutomo was cc'd in that review >>> which held before the PSARC review. >>> >>> Ienup >>> >>> -------- Original Message -------- >>> Subject: Re: Fwd: Libc iconv enhancement [PSARC/2010/160 FastTrack timeout >>> 05/17/2010] >>> Date: Mon, 17 May 2010 17:02:35 -0700 >>> From: Ienup Sung >>> Reply-To: ienup.sung >>> Organization: Oracle Corporation >>> To: PSARC-ext >>> CC: i18n-discuss >>> References: <4BE86212.20302@oracle.com> >>> >>> The timer for this case expired with a +1 and no open issue. >>> >>> I marked this case closed approved. >>> >>> Ienup >>> >