Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mapping for IEEE references in bibxml6 to IEEE dataset #136

Closed
Tracked by #129
ronaldtse opened this issue Feb 18, 2022 · 10 comments
Closed
Tracked by #129

Mapping for IEEE references in bibxml6 to IEEE dataset #136

ronaldtse opened this issue Feb 18, 2022 · 10 comments
Assignees
Labels
enhancement New feature or request

Comments

@ronaldtse
Copy link
Collaborator

ronaldtse commented Feb 18, 2022

This task will fix ietf-ribose#31

reference.IEEE.730_2014.xml
reference.IEEE.754.1985.xml
reference.IEEE.802-1A.1990.xml
reference.IEEE.802-1D.1990.xml
reference.IEEE.802-1D.1991.xml
reference.IEEE.802-1D.1993.xml
reference.IEEE.802-1G.1992.xml
reference.IEEE.802-1Y.1990.xml
reference.IEEE.802-2.1984.xml
reference.IEEE.802-2.1985.xml
reference.IEEE.802-2.1989.xml
reference.IEEE.802-2.1994.xml
reference.IEEE.802-3.1985.xml
reference.IEEE.802-3.1988.xml
reference.IEEE.802-3.1990.xml
reference.IEEE.802-3.1996.xml
reference.IEEE.802-3.1998.xml
reference.IEEE.802-4.1988.xml
reference.IEEE.802-5.1989.xml
reference.IEEE.802-5.1995.xml
reference.IEEE.802-6.1990.xml
reference.IEEE.802-11.1999.xml
reference.IEEE.802-11A.1999.xml
reference.IEEE.802-11B.1999.xml
reference.IEEE.802-11D.2001.xml
reference.IEEE.802-11F.2003.xml
reference.IEEE.802-11G.2003.xml
reference.IEEE.802-11H.2003.xml
reference.IEEE.802-11I.2004.xml
reference.IEEE.802-12.1995.xml
reference.IEEE.802.1AB_2009.xml
reference.IEEE.802.1AB_2009_COR_1_2013.xml
reference.IEEE.802.1AB_2009_COR_2_2015.xml
reference.IEEE.802.1AC_2012.xml
reference.IEEE.802.1AD_2005.xml
reference.IEEE.802.1AEBN_2011.xml
reference.IEEE.802.1AEBW_2013.xml
reference.IEEE.802.1AE_2006.xml
reference.IEEE.802.1AG_2007.xml
reference.IEEE.802.1AJ_2009.xml
reference.IEEE.802.1AK_2007.xml
reference.IEEE.802.1AP_2008.xml
reference.IEEE.802.1AQ_2012.xml
reference.IEEE.802.1AR_2009.xml
reference.IEEE.802.1AS_2011.xml
reference.IEEE.802.1AS_2011_COR_1_2013.xml
reference.IEEE.802.1AXBK_2012.xml
reference.IEEE.802.1AX_2008.xml
reference.IEEE.802.1AX_2014.xml
reference.IEEE.802.1BA_2011.xml
reference.IEEE.802.1BR_2012.xml
reference.IEEE.802.1D_2004.xml
reference.IEEE.802.1H_1997_EDITION_ISO_IEC_TR11802_5_1997_.xml
reference.IEEE.802.1QAT_2010.xml
reference.IEEE.802.1QAU_2010.xml
reference.IEEE.802.1QAV_2009.xml
reference.IEEE.802.1QAW_2009.xml
reference.IEEE.802.1QAY_2009.xml
reference.IEEE.802.1QAZ_2011.xml
reference.IEEE.802.1QBB_2011.xml
reference.IEEE.802.1QBC_2011.xml
reference.IEEE.802.1QBE_2011.xml
reference.IEEE.802.1QBF_2011.xml
reference.IEEE.802.1QBG_2012.xml
reference.IEEE.802.1QBV_2015.xml
reference.IEEE.802.1QCD_2015.xml
reference.IEEE.802.1Q_2005.xml
reference.IEEE.802.1Q_2005_COR1_2008.xml
reference.IEEE.802.1Q_2011_COR_2_2012.xml
reference.IEEE.802.1Q_2012.xml
reference.IEEE.802.1Q_2014.xml
reference.IEEE.802.1Qcc-2018.xml
reference.IEEE.802.1XBX_2014.xml
reference.IEEE.802.1X_2010.xml
reference.IEEE.802.3.1_2011.xml
reference.IEEE.802.3.1_2013.xml
reference.IEEE.802.3AN_2006.xml
reference.IEEE.802.3AP_2007.xml
reference.IEEE.802.3AQ_2006.xml
reference.IEEE.802.3AS_2006.xml
reference.IEEE.802.3BD_2011.xml
reference.IEEE.802.3BJ_2014.xml
reference.IEEE.802.3BK_2013.xml
reference.IEEE.802.3BM_2015.xml
reference.IEEE.802.3_2012.xml
reference.IEEE.802.5T_2000.xml
reference.IEEE.802.5V_2001.xml
reference.IEEE.802.5W_2000.xml
reference.IEEE.802.11AA_2012.xml
reference.IEEE.802.11AC_2013.xml
reference.IEEE.802.11AD_2012.xml
reference.IEEE.802.11AE_2012.xml
reference.IEEE.802.11AF_2013.xml
reference.IEEE.802.11_2012.xml
reference.IEEE.802.15.1_2005.xml
reference.IEEE.802.15.2_2003.xml
reference.IEEE.802.15.3B_2005.xml
reference.IEEE.802.15.3C_2009.xml
reference.IEEE.802.15.3_2003.xml
reference.IEEE.802.15.4A_2007.xml
reference.IEEE.802.15.4C_2009.xml
reference.IEEE.802.15.4D_2009.xml
reference.IEEE.802.15.4E_2012.xml
reference.IEEE.802.15.4F_2012.xml
reference.IEEE.802.15.4G_2012.xml
reference.IEEE.802.15.4J_2013.xml
reference.IEEE.802.15.4K_2013.xml
reference.IEEE.802.15.4M_2014.xml
reference.IEEE.802.15.4P_2014.xml
reference.IEEE.802.15.4_2011.xml
reference.IEEE.802.15.5_2009.xml
reference.IEEE.802.15.6_2012.xml
reference.IEEE.802.15.7_2011.xml
reference.IEEE.802.16.1A_2013.xml
reference.IEEE.802.16.1B_2012.xml
reference.IEEE.802.16.1_2012.xml
reference.IEEE.802.16.2_2004.xml
reference.IEEE.802.16K_2007.xml
reference.IEEE.802.16M_2011.xml
reference.IEEE.802.16P_2012.xml
reference.IEEE.802.16Q_2015.xml
reference.IEEE.802.16_CONFORMANCE04_2006.xml
reference.IEEE.802.17A_2004.xml
reference.IEEE.802.17B_2007.xml
reference.IEEE.802.17C_2010.xml
reference.IEEE.802.17_2004.xml
reference.IEEE.802.17_2011.xml
reference.IEEE.802.19.1_2014.xml
reference.IEEE.802.20.2_2010.xml
reference.IEEE.802.20.3_2010.xml
reference.IEEE.802.20A_2010.xml
reference.IEEE.802.20B_2010.xml
reference.IEEE.802.20_2008.xml
reference.IEEE.802.21A_2012.xml
reference.IEEE.802.21B_2012.xml
reference.IEEE.802.21C_2014.xml
reference.IEEE.802.21D_2015.xml
reference.IEEE.802.21_2008.xml
reference.IEEE.802.22.1_2010.xml
reference.IEEE.802.22.2_2012.xml
reference.IEEE.802.22A_2014.xml
reference.IEEE.802.22_2011.xml
reference.IEEE.802.1990.xml
reference.IEEE.802A_2003.xml
reference.IEEE.802_2001.xml
reference.IEEE.802_2014.xml
reference.IEEE.828_2012.xml
reference.IEEE.829_2008.xml
reference.IEEE.982.1_2005.xml
reference.IEEE.1003-1G.1997.xml
reference.IEEE.1003-2.1992.xml
reference.IEEE.1003.1_2013_EDITION.xml
reference.IEEE.1003.13_2003.xml
reference.IEEE.1003.26_2003.xml
reference.IEEE.1008_1987.xml
reference.IEEE.1012_2012.xml
reference.IEEE.1016_2009.xml
reference.IEEE.1028_2008.xml
reference.IEEE.1044_2009.xml
reference.IEEE.1061_1998.xml
reference.IEEE.1062_1998.xml
reference.IEEE.1074_2006.xml
reference.IEEE.1175.1_2002.xml
reference.IEEE.1175.2_2006.xml
reference.IEEE.1175.3_2004.xml
reference.IEEE.1175.4_2008.xml
reference.IEEE.1220_2005.xml
reference.IEEE.1228_1994.xml
reference.IEEE.1278-1.1995.xml
reference.IEEE.1278-2.1995.xml
reference.IEEE.1284-1.1997.xml
reference.IEEE.1320.1_1998.xml
reference.IEEE.1320.2_1998.xml
reference.IEEE.1394.1995.xml
reference.IEEE.1462_1998.xml
reference.IEEE.1490_2011.xml
reference.IEEE.1517_2010.xml
reference.IEEE.1633_2008.xml
reference.IEEE.1636.1_2013.xml
reference.IEEE.1636.99_2013.xml
reference.IEEE.1647_2011.xml
reference.IEEE.1874_2013.xml
reference.IEEE.2001_2002.xml
reference.IEEE.3333.1.1_2015.xml
reference.IEEE.8802.1AR_2014.xml
reference.IEEE.8802_1AB_2014.xml
reference.IEEE.8802_1AE_2013.xml
reference.IEEE.8802_1AS_2014.xml
reference.IEEE.8802_1X_2013.xml
reference.IEEE.8802_3_2014.xml
reference.IEEE.8802_5_1998.xml
reference.IEEE.8802_11_2012.xml
reference.IEEE.8802_11_2012_AMD.1_2014.xml
reference.IEEE.8802_11_2012_AMD.2_2014.xml
reference.IEEE.8802_11_2012_AMD.3_2014.xml
reference.IEEE.8802_22_2015.xml
reference.IEEE.12207_2008.xml
reference.IEEE.13210_1994.xml
reference.IEEE.14102_2010.xml
reference.IEEE.14143.1_2000.xml
reference.IEEE.14471_2010.xml
reference.IEEE.14515_1_2000.xml
reference.IEEE.15026_1_2014.xml
reference.IEEE.15026_2_2011.xml
reference.IEEE.15026_3_2013.xml
reference.IEEE.15026_4_2013.xml
reference.IEEE.15288.1_2014.xml
reference.IEEE.15288.2_2014.xml
reference.IEEE.15288_2008.xml
reference.IEEE.15288_2015.xml
reference.IEEE.15289_2015.xml
reference.IEEE.15939_2008.xml
reference.IEEE.16085_2006.xml
reference.IEEE.16326_2009.xml
reference.IEEE.20000_1_2013.xml
reference.IEEE.20000_2_2013.xml
reference.IEEE.23026_2015.xml
reference.IEEE.24748_1_2011.xml
reference.IEEE.24748_2_2012.xml
reference.IEEE.24748_3_2012.xml
reference.IEEE.24765_2010.xml
reference.IEEE.24774_2012.xml
reference.IEEE.26511_2012.xml
reference.IEEE.26512_2011.xml
reference.IEEE.26513_2010.xml
reference.IEEE.26514_2010.xml
reference.IEEE.26515_2012.xml
reference.IEEE.26531_2015.xml
reference.IEEE.29119_1_2013.xml
reference.IEEE.29119_2_2013.xml
reference.IEEE.29119_3_2013.xml
reference.IEEE.29148_2011.xml
reference.IEEE.31320_1_2012.xml
reference.IEEE.31320_2_2012.xml
reference.IEEE.42010_2011.xml
reference.IEEE.90003_2008.xml
reference.IEEE.ISO_IEC_14515_1_2000_2003.1_2000.xml
reference.IEEE.ISO_IEC_14519_2001_1003.5_1999.xml
reference.IEEE.ISO_IEC_14764_IEEE_STD_14764_2006.xml
reference.IEEE.ISO_IEC_26702_IEEE_STD_1220_2005_FIRST_EDITION_2007_07_15.xml
reference.IEEE.ISO_IEC_42010_IEEE_STD_1471_2000_FIRST_EDITION_2007_07_15.xml
reference.IEEE.ISO_IEC_IEEE_9945.xml
reference.IEEE.P802-1A.1989.xml
reference.IEEE.P802-1Q.1998.xml
reference.IEEE.P802-3K.1992.xml
reference.IEEE.P802-3P.1992.xml
reference.IEEE.P802-5D.1989.xml
reference.IEEE.P802.11U.xml
reference.IEEE.P1363.1998.xml
reference.IEEE.P1394A.1995.xml
reference.IEEE.P1394B.1995.xml
reference.IEEE.P8021A.1989.xml
reference.IEEE.P8021D.1989.xml
@ronaldtse
Copy link
Collaborator Author

Here's the mapping. There are entries that are marked "UNKNOWN" or "NOT AVAILABLE IN DATASET".

ieee-mapping.yaml.zip

Warning that some of the "NOT AVAILABLE IN DATASET" entries are due to relaton/relaton-ieee#16, so they may become available once we are able to parse all IEEE entries.

@ronaldtse
Copy link
Collaborator Author

ronaldtse commented Apr 10, 2022

This is the new set of mappings from bibxml6 filenames to IEEE dataset's title attribute. Only the following items are missing.

ieee-mapping.yaml.zip

These are the document identifiers that we cannot find in the IEEE dataset: not available on ieee.org and cannot be found anywhere (even when relaton/relaton-ieee#16 is complete). We need to find a way to resolve these. The last two possibly point to ISO co-published copies, but IEEE's dataset doesn't contain them.

reference.IEEE.802-1D.1991.xml: UNKNOWN; ONLY INTERNET SOURCE IS bibxml2
reference.IEEE.802-1D.1993.xml: UNKNOWN; ONLY INTERNET SOURCE IS bibxml2
reference.IEEE.802-1Y.1990.xml: UNKNOWN; there is a 802.1Y but is a completely different document.
reference.IEEE.P802-1A.1989.xml: UNKNOWN, ONLY INTERNET SOURCE IS bibxml2
reference.IEEE.P8021A.1989.xml: UNKNOWN, ONLY INTERNET SOURCE IS bibxml2
reference.IEEE.802-3.1990.xml: DOES NOT EXIST IN IEEE DATASET - ISO/IEC 8802-3:1990
reference.IEEE.802-3.1996.xml: DOES NOT EXIST IN IEEE DATASET - ISO/IEC 8802-3:1996

@ronaldtse
Copy link
Collaborator Author

ronaldtse commented Apr 11, 2022

I've investigated these last 7 entries:

  • reference.IEEE.802-1D.1991.xml: Possibly IEEE Std 802.1D-1990?
  • reference.IEEE.802-1D.1993.xml: There is IEEE Std 802.1D-1990, ANSI/IEEE Std 802.1D, 1998 Edition, and IEEE Std 802.1D-2004 (Revision of IEEE Std 802.1D-1998) but no 1993.
  • reference.IEEE.802-1Y.1990.xml: There is a textual reference to P802.1y in the dataset from the entry of IEEE Std 802.1D-2004 (Revision of IEEE Std 802.1D-1998), but it is not available as an entry in the IEEE dataset or on the IEEE website. The only reference to this is the bibxml2/bibxml6 datasets.
  • reference.IEEE.P802-1A.1989.xml: This reference cannot be found in the IEEE dataset or the IEEE website. The only reference to this is the bibxml2/bibxml6 datasets.
  • reference.IEEE.P8021A.1989.xml: possibly a variation of the above entry.
  • reference.IEEE.802-3.1990.xml: ISO/IEC 8802-3:1990 - ANSI/IEEE Std 802.3-1990 Edition existed but not in the IEEE dataset, and cannot be found on the IEEE website. There are textual references to this document in the IEEE dataset but it does not exist as an entry.
  • reference.IEEE.802-3.1996.xml: This document only exists as ISO/IEC 8802-3:1996 on ISO's site. In the IEEE dataset, there is only IEEE Std 802.3, 1998 Edition.

I have also verified that all bibxml2/reference.IEEE.* entries exist in bibxml6/.

The recommendations are:

  • Mismatched edition years. reference.IEEE.802-1D.1991.xml, reference.IEEE.802-1D.1993.xml correct to 1990.
  • Published by ISO not IEEE: reference.IEEE.802-3.1990.xml, reference.IEEE.802-3.1996.xml change to the corresponding IEEE published entry, not the ISO version.
  • For the items outside of the official IEEE dataset (which are also not available on the IEEE website), either:
    • we continue making them available under bibxml-misc
    • remove them because they are likely incorrect entries

@rjsparks we will need guidance on next steps here. Thanks!

@ronaldtse ronaldtse added the enhancement New feature or request label Apr 11, 2022
@ronaldtse ronaldtse added this to BibXML Apr 11, 2022
@ronaldtse ronaldtse self-assigned this Apr 11, 2022
@ronaldtse ronaldtse moved this to Urgent in BibXML Apr 11, 2022
@ronaldtse
Copy link
Collaborator Author

ronaldtse commented Apr 11, 2022

(Once this task is done we can remove all bibxml2/reference.IEEE.* files from the bibxml-data-archive repo.)

@rjsparks
Copy link
Member

@TonyLHansen Comments on the choice in the last bullet above?

@russhousley
Copy link

I do not think we should deviate away from the IEEE dataset. This will keep it simple when IEEE releases updates.

Any supplemental entries can be put in bibxml-misc.

@strogonoff
Copy link
Collaborator

Here's the mapping. There are entries that are marked "UNKNOWN" or "NOT AVAILABLE IN DATASET".

ieee-mapping.yaml.zip

These mappings seem to reference many document identifiers not present in IEEE bibliography source🤔

  • IEEE Std 1062, 1998 Edition
  • IEEE Std 1061-1998
  • IEEE Std 1012-2012 (Revision of IEEE Std 1012-2004) - Redline
  • IEEE Std 1003.1, 2013 Edition (incorporates IEEE Std 1003.1-2008, and IEEE Std 1003.1-2008/Cor 1-2013)

and others.

I’m not sure whether it has to do with wrong identifier format, or documents are missing altogether.

@strogonoff
Copy link
Collaborator

strogonoff commented Jun 25, 2022

Answered my own question.

Mappings for bibxml6 were added to bibxml-data-archive in ietf-ribose/bibxml-data-archive@c6b1281. Most reference nonexistent docids and so will fall back to xml2rfc archive data.

Repository owner moved this from Urgent to Done in BibXML Jun 25, 2022
@ronaldtse
Copy link
Collaborator Author

These mappings seem to reference many document identifiers not present in IEEE bibliography source🤔

  • IEEE Std 1062, 1998 Edition
  • IEEE Std 1061-1998
  • IEEE Std 1012-2012 (Revision of IEEE Std 1012-2004) - Redline
  • IEEE Std 1003.1, 2013 Edition (incorporates IEEE Std 1003.1-2008, and IEEE Std 1003.1-2008/Cor 1-2013)

and others.

I’m not sure whether it has to do with wrong identifier format, or documents are missing altogether.

We actually need to fix the relaton-data-ieee document identifiers to show the correct format. It is time to integrate relaton-ieee with pubid-ieee.

@strogonoff
Copy link
Collaborator

I’m happy if we do that, but I’m not sure if this will address the invalid mappings in short term…

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants