NTCIR11 MedNLP 2

IMPORTANT NOTICE

*How to submit your result(s):

Please send your results to the following e-mail address with your submission files attached: mednlp.office@gmail.com

*Submission file format:

The submission file format is the very same format of the training files.
Your submission files should include tags and attributes just as of the training data (mednlp2-train_v0.xml), except tags.

(e.g. 1) Your submission files for TASK 1 should include tags and modality attributes (modality="negation", "suspicion", "family").
(e.g. 2) Your submission files for TASK 2 should includes tag, modality attributes (negation, suspicion, family), and ICD attributes (icd="XXXX").

Note: You are encouraged to submit results with all above attributes, but you may submit without modality attributes for TASK 1 and 2 (In that case, the evaluation will be less detailed.).

*Validation of submitted files:

We will perform formal validation as soon as we receive your submission.
In case of any format irregularities found in your submission results, we will let you know directly.

*Test data for TASK 2:

We will provide the gold standard data after all the TASK 1 results are submitted (instead of the highest f-score data, which we have previously announced).

*The number of results you can submit:

You may submit up to 3 results for TASK 1.

There are 2 possible submissions for TASK 2.

(The gold standard data includes modality attributes.
The modality attributions are not mandatory, but strongly recommended.):

1) Adding ICD attributes to the test data of TASK 1 (test.xml).
2) Adding ICD attributes to the gold standard data which will be provided after TASK 1 submission.

*You may submit up to 3 results for each possible submission (up to 6 results in total).

If you have any questions, please feel free to contact us.
mednlp.office@gmail.com

Welcome to MedNLP 2

Recently, more and more medical records are written in electronic format in place of paper, which leads to a higher importance of information processing technique in medical fields. In this proposed pilot task, participants are supposed to retrieve important information from medical documents in Japanese. This is one of the elemental technologies to develop computational systems for supporting a wide range of medical services.

Our goal is to promote and support to generate practical tools and systems applicable in the medical industry, which will support medical decisions and treatments by physicians and medical staffs. While a short-term objective of this pilot task is to evaluate basic techniques to information extraction in medical fields, the actual objective is to offer a forum for achieving the goal with community-based approach, that is, to gather people who are interested in this issue, and to facilitate their communication and discussion to clarify issues to be solved and to define element technologies.

Task

Participants are expected to extract information from medical reports written by physicians and from past medical exams. There are 3 tasks for this competition:

Task 1) extraction of complaint and diagnosis Task (extract complaint and diagnosis from the text)

Task 2) normalization of complaint and diagnosis Task (give icd-10 code on complaint and diagnosis)

Task 3) free Task (any creative ideas other than above are welcomed)

You can register each task independently. That allows you to participate flexibly: e.g. enter for Task 1) only, enter for Task 1) and Task 3), or enter for all 3 tasks.

Dataset

The dataset for this pilot task contains medical history summaries of imaginary patients written by physicians:

Please click HERE for the actual examples. (Opens new window/tab.)
Please click HERE for the Annotation Guideline. (pdf / Opens new window/tab.)

Input Data Sample (original text):

工場に勤めている64歳の男性。
2025年月8月2日（来院5日前）頃から腹痛が生じるとともに，食欲不振，嘔気・嘔吐出現した。
体幹は温かく、ショック状態。
明らかな運動麻痺はみられず。
翌日，意識障害出現し，腎機能障害の増悪を認め， 8月9日18時10分に心肺停止。
8月9日21時44分死亡確認。

Output Data Sample (tagged text including ICD and modality):

工場に勤めている64歳の男性。
<t>2025年月8月2日（来院5日前）頃</t>から <c icd="R104">腹痛</c>が生じるとともに， <c icd="R630">食欲不振</c>， <c icd="R11_">嘔気</c> ・ <c icd="R11_">嘔吐</c>出現した。
体幹は温かく、<c icd="R579">ショック状態</c>。
明らかな <c icd="G839" modality="negation">運動麻痺</c> はみられず。
翌日， <c icd="R402">意識障害</c>出現し， <c icd="N289">腎機能障害</c>の増悪を認め， <t>8月9日18時10分</t>に <c icd="I469">心肺停止</c>。
<t>8月9日21時44分</t> <c icd="R99_">死亡確認</c>。

Important Dates

Jan. 20, 2014	Task Participants' Registration Due
Feb. 10, 2014	Annotation Guideline distribution
Mar. 10, 2014	Training Corpus distribution (Please submit the Agreement to download the corpus)
Mar. 20, 2014	Dry Run (Recommended but optional)
Jul. 11, 2014	Distribution of Test-data for TASK 1 (NER) & TASK 3 (Free)
Jul. 18, 2014	Submission due for the system output for TASK 1 (NER)
Jul. 25, 2014	Distribution of test-data for TASK 2 (the output result that received the highest f-score in Task 1)
Aug. 1, 2014	Submission due for the system output for TASK 2 (Normalization/Coding) & TASK 3 (Free)
Aug. 1, 2014	Early draft overview papers by TOs released to participants
Sep. 1, 2014	Draft participant paper submission due
Oct. 1, 2014	Task overview paper submission due
Nov. 1, 2014	All camera-ready copy for the Proceedings due
Dec. 9-12, 2014	NTCIR-11 Conference, NII, Tokyo, Japan

Registration

Please register via registration link to participate in this pilot task.

Organizer etc.

Organaizer

	ARAMAKI Eiji, Ph.D. [Chair] （Kyoto University， JST PRESTO）
	MORITA Mizuki, Ph.D. [Chair] （The University of Tokyo， National Institute of Biomedical Innovation）
	KANO Yoshinobu, Ph.D. （JST PRESTO， National Institute of Informatics）
	OHKUMA Tomoko, Ph.D. (FUJI XEROX)

Adviser

MASUICHI Hiroshi, Ph.D. (FUJI XEROX)

Annotator

SHIKATA Shuko

KUBO Kay

SHIMAMOTO Yumiko

Misc

† Please click here for the terminologies and the descriptions used in medical reports.

† Please click here to become the member of our mailing list.

IMPORTANT NOTICE

*How to submit your result(s):

*Submission file format:

*Validation of submitted files:

*Test data for TASK 2:

*The number of results you can submit: