Merge pull request #5 from pfnet-research/cpa_audit-data-inclusion

feat: direct inclusion of CAP_AUDIT data

README.md

@@ -142,6 +142,13 @@ Or cite directory this repository:
 }
 ```
 
+# Note:
+cpa_audit data comes from an existing collection of Japanese CPA Audit exam questions and answers [1].
+In addition, this dataset was built using data from the [Institute of Certified Public Accountants and Auditing Oversight Board Web site](https://www.fsa.go.jp/cpaaob/kouninkaikeishi-shiken/index.html) and is [subject to a CC-BY 4.0 license](https://www.fsa.go.jp/cpaaob/copyright.html).
+We got special permission to include this data directly for this evaluation. Thanks to their contribution.
+
+[1] Tatsuki Masuda, Kei Nakagawa, Takahiro Hoshino, Can ChatGPT pass the JCPA exam?: Challenge for the short-answer method test on Auditing, JSAI Technical Report, Type 2 SIG, 2023, Volume 2023, Issue FIN-031, Pages 81-88, Released on J-STAGE October 12, 2023, Online ISSN 2436-5556, https://doi.org/10.11517/jsaisigtwo.2023.FIN-031_81
+
 # Contribution
 This project is owned by [Preferred Networks](https://www.preferred.jp) and maintained by [Masanori Hirano](https://mhirano.jp).
 

jlm_fin_eval/datasets/cpa/README.md

@@ -1,9 +1,10 @@
 # CPA
 ## CAP Audit
 公認会計士試験短答式試験監査論データセットです。詳しくは[1]をご参照ください。
+また、このデータセットは、[公認会計士・監査審査会ホームページ](https://www.fsa.go.jp/cpaaob/kouninkaikeishi-shiken/index.html)のデータを用いて構築されており、[CC-BY 4.0ライセンスに従います](https://www.fsa.go.jp/cpaaob/copyright.html)。
 
 This data is a collection of Japanese CPA Audit exam questions and answers [1].
-
+In addition, this dataset was built using data from the [Institute of Certified Public Accountants and Auditing Oversight Board Web site](https://www.fsa.go.jp/cpaaob/kouninkaikeishi-shiken/index.html) and is [subject to a CC-BY 4.0 license](https://www.fsa.go.jp/cpaaob/copyright.html).
 
 [1] 増田 樹, 中川 慧, 星野 崇宏, ChatGPTは公認会計士試験を突破できるか？: 短答式試験監査論への挑戦, 人工知能学会第二種研究会資料, 2023, 2023 巻, FIN-031 号, p. 81-88, 公開日 2023/10/12,Online ISSN 2436-5556, https://doi.org/10.11517/jsaisigtwo.2023.FIN-031_81
 

jlm_fin_eval/datasets/cpa/cpa_audit.py

@@ -5,7 +5,7 @@
     "Japanese CPA Audit dataset. This contains short-answer method test on Auditing."
 )
 _CITATION = "Tatsuki MASUDA, Kei NAKAGAWA, Takahiro HOSHINO, Can ChatGPT pass the JCPA exam?: Challenge for the short-answer method test on Auditing, JSAI Technical Report, Type 2 SIG, 2023, Volume 2023, Issue FIN-031, Pages 81-88, Released on J-STAGE October 12, 2023, Online ISSN 2436-5556, https://doi.org/10.11517/jsaisigtwo.2023.FIN-031_81"
-_LICENSE = "Unknown"
+_LICENSE = "MIT License"
 _HOMEPAGE = "https://www.jstage.jst.go.jp/article/jsaisigtwo/2023/FIN-031/2023_81/_article/-char/ja"
 
 
@@ -49,9 +49,7 @@ def _split_generators(self, dl_manager):
             datasets.SplitGenerator(
                 name=datasets.Split.TEST,  # type: ignore
                 gen_kwargs={
-                    "filepath": dl_manager.download(
-                        "https://drive.google.com/uc?export=download&id=1Wf3YybZ9MAPA33Z6B9-yhZiToOVnFLJH"
-                    ),
+                    "filepath": dl_manager.download("CPA_AUDIT.xlsx"),
                     "split": datasets.Split.TEST,
                 },
             )