AOP-Wiki_XML_to_RDF_conversion.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <b>AOP-Wiki XML conversion to RDF</b>\n",
    "Author: Marvin Martens\n",
    "\n",
    "The [AOP-Wiki](https://aopwiki.org/) is the central repository for qualitative descriptions of AOPs, and releases its database every three months in XML format. This Jupyter notebook makes the conversion of the AOP-Wiki XML into RDF with Turtle (ttl) syntax. \n",
    "\n",
    "It downloads and parses the AOP-Wiki XML file with the ElementTree XML API Python library, and stores all its components in nested dictionaries for the all subjects which form the basis of the existing AOP-Wiki, being the AOPs, KEs,  KERs,  stressors,  chemicals,  taxonomy,  cell-terms,  organ-terms,  and  the  KE  components, which comprise of Biological Processes (BPs),  Biological Objects (BOs) and Biological Actions (BAs).  During the filling of those dictionaries, semantic annotations are being added for  the  subjects,  the  relationship  (predicate)  to  their  property  (object),  and  for  the  properties themselves when meant to represent an identifier or ontology term.\n",
    "\n",
    "<img src=\"Overview AOP-Wiki RDF.svg\" style=\"width: 650px;\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b>Step #1: imports</b>\n",
    "First, all required Python libraries are imported. It will `pip install` libraries if the imports are not found on the system."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: pip in /home/marvin/anaconda3/lib/python3.12/site-packages (24.2)\n"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "\n",
    "!{sys.executable} -m pip install --upgrade pip \n",
    "from xml.etree.ElementTree import parse\n",
    "import re\n",
    "TAG_RE = re.compile(r'<[^>]+>')\n",
    "import requests\n",
    "import datetime\n",
    "import urllib\n",
    "import gzip\n",
    "import shutil\n",
    "import os\n",
    "import stat\n",
    "import time\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/bin/bash: line 1: pipreqsnb: command not found\n"
     ]
    }
   ],
   "source": [
    "!pipreqsnb . "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook includes the mapping of identifiers for chemicals and genes. To make this possible, the URL to the BridgeDb service should be defined in the `bridgedb` variable, and include the `/Human/`. The quickest way to execute the code is by using a local BridgeDb service launched with the BridgeDb Docker image using the [instructions](https://github.com/bridgedb/docker). Alternatively, the live web version can be used by defining the `bridgedb` variable as 'https://webservice.bridgedb.org/Human/'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "bridgedb = 'https://webservice.bridgedb.org/Human/'#'http://localhost:8180/Human/'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b>Step #2: Getting the AOP-Wiki XML</b>\n",
    "Next, the last version of the AOP-Wiki XML is defined in the `aopwikixmlfilename` variable, which can be found in the [download page of the AOP-Wiki](https://aopwiki.org/downloads/). This file is downloaded, unzipped, and opened, after which the ElementTree XML API parses it, making it ready for extracting its contents from the `root`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Today's date: 2024-08-20\n"
     ]
    }
   ],
   "source": [
    "from datetime import date\n",
    "\n",
    "today = date.today()\n",
    "print(\"Today's date:\", today)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/marvin/anaconda3/lib/python3.12/site-packages/urllib3/connectionpool.py:1099: InsecureRequestWarning: Unverified HTTPS request is being made to host 'aopwiki.org'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "aopwikixmlfilename = 'aop-wiki-xml-'+str(today)\n",
    "response = requests.get('https://aopwiki.org/downloads/aop-wiki-xml.gz', verify=False)\n",
    "with open(aopwikixmlfilename, 'wb') as f:\n",
    "    f.write(response.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The XML will be extracted to the folder defined within the variable `filepath` in the next block of code, which is by defailt `/data` relative to the location of the Jupyter notebook. All datafiles used and produced with this notebook will be placed there."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "filepath = 'data/'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "File data/aop-wiki-xml-2024-08-20 opened\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    with gzip.open(aopwikixmlfilename, 'rb') as f_in:\n",
    "        with open(filepath+aopwikixmlfilename, 'wb') as f_out:\n",
    "            shutil.copyfileobj(f_in, f_out)\n",
    "            print('File ' + filepath+aopwikixmlfilename + ' opened')\n",
    "except:\n",
    "    print('Check if the filepath is correct:\\n' + filepath+aopwikixmlfilename)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The AOP-Wiki XML is parsed correctly, and contains 6279 entities\n"
     ]
    }
   ],
   "source": [
    "tree = parse(filepath + aopwikixmlfilename)\n",
    "root = tree.getroot()\n",
    "print('The AOP-Wiki XML is parsed correctly, and contains ' + str(len(root)) + ' entities')\n",
    "\n",
    "aopxml = '{http://www.aopkb.org/aop-xml}'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b>Step #3: extracting information from the XML</b>\n",
    "The next section extracts all information from the main 11 AOP-Wiki entities shown in Figure 1. These are stored in nested dictionaries, while using ontological annotations as keys for semantic mapping of the information. Note that the cell-terms and organ-terms are included in the KE block of code.\n",
    "\n",
    "First, all reference identifiers for AOPs, KEs, KERs and stressors need to be extracted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The AOP-Wiki XML contains 487 identifiers for the entity AOP\n",
      "\n",
      "The AOP-Wiki XML contains 1455 identifiers for the entity KE\n",
      "\n",
      "The AOP-Wiki XML contains 2042 identifiers for the entity KER\n",
      "\n",
      "The AOP-Wiki XML contains 658 identifiers for the entity Stressor\n"
     ]
    }
   ],
   "source": [
    "refs = {'AOP': {}, 'KE': {}, 'KER': {}, 'Stressor': {}}\n",
    "for ref in root.find(aopxml + 'vendor-specific').findall(aopxml + 'aop-reference'):\n",
    "    refs['AOP'][ref.get('id')] = ref.get('aop-wiki-id')\n",
    "for ref in root.find(aopxml + 'vendor-specific').findall(aopxml + 'key-event-reference'):\n",
    "    refs['KE'][ref.get('id')] = ref.get('aop-wiki-id')\n",
    "for ref in root.find(aopxml + 'vendor-specific').findall(aopxml + 'key-event-relationship-reference'):\n",
    "    refs['KER'][ref.get('id')] = ref.get('aop-wiki-id')\n",
    "for ref in root.find(aopxml + 'vendor-specific').findall(aopxml + 'stressor-reference'):\n",
    "    refs['Stressor'][ref.get('id')] = ref.get('aop-wiki-id')\n",
    "for item in refs:\n",
    "    print('\\nThe AOP-Wiki XML contains ' + str(len(refs[item])) + ' identifiers for the entity ' + item)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Adverse Outcome Pathways"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n",
      "\n",
      "A total of 487 Adverse Outcome Pathways have been parsed.\n"
     ]
    }
   ],
   "source": [
    "aopdict = {}\n",
    "kedict = {}\n",
    "for AOP in root.findall(aopxml + 'aop'):\n",
    "    aopdict[AOP.get('id')] = {}\n",
    "    aopdict[AOP.get('id')]['dc:identifier'] = 'aop:' + refs['AOP'][AOP.get('id')]\n",
    "    aopdict[AOP.get('id')]['rdfs:label'] = '\"AOP ' + refs['AOP'][AOP.get('id')] + '\"'\n",
    "    aopdict[AOP.get('id')]['foaf:page'] = '<https://identifiers.org/aop/' + refs['AOP'][AOP.get('id')] + '>'\n",
    "    aopdict[AOP.get('id')]['dc:title'] = '\"' + AOP.find(aopxml + 'title').text + '\"'\n",
    "    aopdict[AOP.get('id')]['dcterms:alternative'] = AOP.find(aopxml + 'short-name').text\n",
    "    aopdict[AOP.get('id')]['dc:description'] = []\n",
    "    if AOP.find(aopxml + 'background') is not None:\n",
    "        aopdict[AOP.get('id')]['dc:description'].append('\"\"\"' + TAG_RE.sub('', AOP.find(aopxml + 'background').text) + '\"\"\"')\n",
    "    if AOP.find(aopxml + 'authors').text is not None:\n",
    "        aopdict[AOP.get('id')]['dc:creator'] = '\"\"\"' + TAG_RE.sub('', AOP.find(aopxml + 'authors').text) + '\"\"\"'\n",
    "    if AOP.find(aopxml + 'abstract').text is not None:\n",
    "        aopdict[AOP.get('id')]['dcterms:abstract'] = '\"\"\"' + TAG_RE.sub('', AOP.find(aopxml + 'abstract').text) + '\"\"\"'\n",
    "    if AOP.find(aopxml + 'status').find(aopxml + 'wiki-status') is not None:\n",
    "        aopdict[AOP.get('id')]['dcterms:accessRights'] = '\"' + AOP.find(aopxml + 'status').find(aopxml + 'wiki-status').text + '\"' \n",
    "    if AOP.find(aopxml + 'status').find(aopxml + 'oecd-status') is not None:\n",
    "        aopdict[AOP.get('id')]['oecd-status'] =  '\"' + AOP.find(aopxml + 'status').find(aopxml + 'oecd-status').text + '\"' \n",
    "    if AOP.find(aopxml + 'status').find(aopxml + 'saaop-status') is not None:\n",
    "        aopdict[AOP.get('id')]['saaop-status'] =  '\"' + AOP.find(aopxml + 'status').find(aopxml + 'saaop-status').text + '\"' \n",
    "    aopdict[AOP.get('id')]['oecd-project'] = AOP.find(aopxml + 'oecd-project').text\n",
    "    aopdict[AOP.get('id')]['dc:source'] = AOP.find(aopxml + 'source').text\n",
    "    aopdict[AOP.get('id')]['dcterms:created'] = AOP.find(aopxml + 'creation-timestamp').text\n",
    "    aopdict[AOP.get('id')]['dcterms:modified'] = AOP.find(aopxml + 'last-modification-timestamp').text\n",
    "    for appl in AOP.findall(aopxml + 'applicability'):\n",
    "        for sex in appl.findall(aopxml + 'sex'):\n",
    "            if 'pato:0000047' not in aopdict[AOP.get('id')]:\n",
    "                aopdict[AOP.get('id')]['pato:0000047'] = [[sex.find(aopxml + 'evidence').text, sex.find(aopxml + 'sex').text]]\n",
    "            else:\n",
    "                aopdict[AOP.get('id')]['pato:0000047'].append([sex.find(aopxml + 'evidence').text, sex.find(aopxml + 'sex').text])\n",
    "        for life in appl.findall(aopxml + 'life-stage'):\n",
    "            if 'aopo:LifeStageContext' not in aopdict[AOP.get('id')]:\n",
    "                aopdict[AOP.get('id')]['aopo:LifeStageContext'] = [[life.find(aopxml + 'evidence').text, life.find(aopxml + 'life-stage').text]]\n",
    "            else:\n",
    "                aopdict[AOP.get('id')]['aopo:LifeStageContext'].append([life.find(aopxml + 'evidence').text, life.find(aopxml + 'life-stage').text])\n",
    "    aopdict[AOP.get('id')]['aopo:has_key_event'] = {}\n",
    "    if AOP.find(aopxml + 'key-events') is not None:\n",
    "        for KE in AOP.find(aopxml + 'key-events').findall(aopxml + 'key-event'):\n",
    "            aopdict[AOP.get('id')]['aopo:has_key_event'][KE.get('key-event-id')] = {}\n",
    "            aopdict[AOP.get('id')]['aopo:has_key_event'][KE.get('key-event-id')]['dc:identifier'] = 'aop.events:' + refs['KE'][KE.get('key-event-id')]\n",
    "    aopdict[AOP.get('id')]['aopo:has_key_event_relationship'] = {}\n",
    "    if AOP.find(aopxml + 'key-event-relationships') is not None:\n",
    "        for KER in AOP.find(aopxml + 'key-event-relationships').findall(aopxml + 'relationship'):\n",
    "            aopdict[AOP.get('id')]['aopo:has_key_event_relationship'][KER.get('id')] = {}\n",
    "            aopdict[AOP.get('id')]['aopo:has_key_event_relationship'][KER.get('id')]['dc:identifier'] = 'aop.relationships:' + refs['KER'][KER.get('id')]\n",
    "            aopdict[AOP.get('id')]['aopo:has_key_event_relationship'][KER.get('id')]['adjacency'] = KER.find(aopxml + 'adjacency').text\n",
    "            aopdict[AOP.get('id')]['aopo:has_key_event_relationship'][KER.get('id')]['quantitative-understanding-value'] = KER.find(aopxml + 'quantitative-understanding-value').text\n",
    "            aopdict[AOP.get('id')]['aopo:has_key_event_relationship'][KER.get('id')]['aopo:has_evidence'] = KER.find(aopxml + 'evidence').text\n",
    "    aopdict[AOP.get('id')]['aopo:has_molecular_initiating_event'] = {}\n",
    "    for MIE in AOP.findall(aopxml + 'molecular-initiating-event'):\n",
    "        aopdict[AOP.get('id')]['aopo:has_molecular_initiating_event'][MIE.get('key-event-id')] = {}\n",
    "        aopdict[AOP.get('id')]['aopo:has_molecular_initiating_event'][MIE.get('key-event-id')]['dc:identifier'] = 'aop.events:' + refs['KE'][MIE.get('key-event-id')]\n",
    "        aopdict[AOP.get('id')]['aopo:has_key_event'][MIE.get('key-event-id')] = {}\n",
    "        aopdict[AOP.get('id')]['aopo:has_key_event'][MIE.get('key-event-id')]['dc:identifier'] = 'aop.events:' + refs['KE'][MIE.get('key-event-id')]\n",
    "        if MIE.find(aopxml + 'evidence-supporting-chemical-initiation').text is not None:\n",
    "            kedict[MIE.get('key-event-id')] = {}\n",
    "            aopdict[AOP.get('id')]['dc:description'].append('\"\"\"' + TAG_RE.sub('', MIE.find(aopxml + 'evidence-supporting-chemical-initiation').text) + '\"\"\"')\n",
    "    aopdict[AOP.get('id')]['aopo:has_adverse_outcome'] = {}\n",
    "    for AO in AOP.findall(aopxml + 'adverse-outcome'):\n",
    "        aopdict[AOP.get('id')]['aopo:has_adverse_outcome'][AO.get('key-event-id')] = {}\n",
    "        aopdict[AOP.get('id')]['aopo:has_adverse_outcome'][AO.get('key-event-id')]['dc:identifier'] = 'aop.events:' + refs['KE'][AO.get('key-event-id')]\n",
    "        aopdict[AOP.get('id')]['aopo:has_key_event'][AO.get('key-event-id')] = {}\n",
    "        aopdict[AOP.get('id')]['aopo:has_key_event'][AO.get('key-event-id')]['dc:identifier'] = 'aop.events:' + refs['KE'][AO.get('key-event-id')]\n",
    "        if AO.find(aopxml + 'examples').text is not None:\n",
    "            kedict[AO.get('key-event-id')] = {}\n",
    "            aopdict[AOP.get('id')]['dc:description'].append('\"\"\"' + TAG_RE.sub('', AO.find(aopxml + 'examples').text) + '\"\"\"')\n",
    "    aopdict[AOP.get('id')]['nci:C54571'] = {}\n",
    "    if AOP.find(aopxml + 'aop-stressors') is not None:\n",
    "        for stressor in AOP.find(aopxml + 'aop-stressors').findall(aopxml + 'aop-stressor'):\n",
    "            aopdict[AOP.get('id')]['nci:C54571'][stressor.get('stressor-id')] = {}\n",
    "            aopdict[AOP.get('id')]['nci:C54571'][stressor.get('stressor-id')]['dc:identifier'] = 'aop.stressor:' + refs['Stressor'][stressor.get('stressor-id')]\n",
    "            aopdict[AOP.get('id')]['nci:C54571'][stressor.get('stressor-id')]['aopo:has_evidence'] = stressor.find(aopxml + 'evidence').text\n",
    "    if AOP.find(aopxml + 'overall-assessment').find(aopxml + 'description').text is not None:\n",
    "        aopdict[AOP.get('id')]['nci:C25217'] = '\"\"\"' + TAG_RE.sub('', AOP.find(aopxml + 'overall-assessment').find(aopxml + 'description').text) + '\"\"\"'\n",
    "    if AOP.find(aopxml + 'overall-assessment').find(aopxml + 'key-event-essentiality-summary').text is not None:\n",
    "        aopdict[AOP.get('id')]['nci:C48192'] = '\"\"\"' + TAG_RE.sub('', AOP.find(aopxml + 'overall-assessment').find(aopxml + 'key-event-essentiality-summary').text) + '\"\"\"'\n",
    "    if AOP.find(aopxml + 'overall-assessment').find(aopxml + 'applicability').text is not None:\n",
    "        aopdict[AOP.get('id')]['aopo:AopContext'] = '\"\"\"' + TAG_RE.sub('', AOP.find(aopxml + 'overall-assessment').find(aopxml + 'applicability').text) + '\"\"\"'\n",
    "    if AOP.find(aopxml + 'overall-assessment').find(aopxml + 'weight-of-evidence-summary').text is not None:\n",
    "        aopdict[AOP.get('id')]['aopo:has_evidence'] = '\"\"\"' + TAG_RE.sub('', AOP.find(aopxml + 'overall-assessment').find(aopxml + 'weight-of-evidence-summary').text) + '\"\"\"'\n",
    "    if AOP.find(aopxml + 'overall-assessment').find(aopxml + 'quantitative-considerations').text is not None:\n",
    "        aopdict[AOP.get('id')]['edam:operation_3799'] = '\"\"\"' + TAG_RE.sub('', AOP.find(aopxml + 'overall-assessment').find(aopxml + 'quantitative-considerations').text) + '\"\"\"'\n",
    "    if AOP.find(aopxml + 'potential-applications').text is not None:\n",
    "        aopdict[AOP.get('id')]['nci:C25725'] = '\"\"\"' + TAG_RE.sub('', AOP.find(aopxml + 'potential-applications').text) + '\"\"\"'\n",
    "print('Done!\\n\\nA total of ' + str(len(aopdict)) + ' Adverse Outcome Pathways have been parsed.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Chemicals\n",
    "For the chemicals in the AOP-Wiki, we added BridgeDb mappings for increased coverage of chemical databases for which we used the already present CAS identifers. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n",
      "\n",
      "A total of 397 chemicals have been parsed.\n"
     ]
    }
   ],
   "source": [
    "chedict = {}\n",
    "listofchebi = []\n",
    "listofchemspider = []\n",
    "listofwikidata = []\n",
    "listofchembl = []\n",
    "listofdrugbank = []\n",
    "listofpubchem = []\n",
    "listoflipidmaps = []\n",
    "listofhmdb = []\n",
    "listofkegg = []\n",
    "listofcas = []\n",
    "listofinchikey = []\n",
    "listofcomptox = []\n",
    "\n",
    "for che in root.findall(aopxml + 'chemical'):\n",
    "    chedict[che.get('id')] = {}\n",
    "    if che.find(aopxml + 'casrn') is not None:\n",
    "        if 'NOCAS' not in che.find(aopxml + 'casrn').text:  # all NOCAS ids are taken out, so no issues as subjects\n",
    "            chedict[che.get('id')]['dc:identifier'] = 'cas:' + che.find(aopxml + 'casrn').text\n",
    "            listofcas.append('cas:' + che.find(aopxml + 'casrn').text)\n",
    "            chedict[che.get('id')]['cheminf:000446'] = '\"' + che.find(aopxml + 'casrn').text + '\"'\n",
    "            a = requests.get(bridgedb+'xrefs/Ca/'+che.find(aopxml + 'casrn').text).text.split('\\n')\n",
    "            dictionaryforchemical = {}\n",
    "            if 'html' not in a:\n",
    "                for item in a:\n",
    "                    b = item.split('\\t')\n",
    "                    if len(b) == 2:\n",
    "                        if b[1] not in dictionaryforchemical:\n",
    "                            dictionaryforchemical[b[1]] = []\n",
    "                            dictionaryforchemical[b[1]].append(b[0])\n",
    "                        else:\n",
    "                            dictionaryforchemical[b[1]].append(b[0])\n",
    "            if 'ChEBI' in dictionaryforchemical:\n",
    "                chedict[che.get('id')]['cheminf:000407'] = []\n",
    "                for chebi in dictionaryforchemical['ChEBI']:\n",
    "                    if \"chebi:CHEBI:\"+chebi[6:] not in listofchebi:\n",
    "                        listofchebi.append(\"chebi:\"+chebi)\n",
    "                    chedict[che.get('id')]['cheminf:000407'].append(\"chebi:\"+chebi)\n",
    "            if 'Chemspider' in dictionaryforchemical:\n",
    "                chedict[che.get('id')]['cheminf:000405'] = []\n",
    "                for chemspider in dictionaryforchemical['Chemspider']:\n",
    "                    if \"chemspider:\"+chemspider not in listofchemspider:\n",
    "                        listofchemspider.append(\"chemspider:\"+chemspider)\n",
    "                    chedict[che.get('id')]['cheminf:000405'].append(\"chemspider:\"+chemspider)\n",
    "            if 'Wikidata' in dictionaryforchemical:\n",
    "                chedict[che.get('id')]['cheminf:000567'] = []\n",
    "                for wd in dictionaryforchemical['Wikidata']:\n",
    "                    if \"wikidata:\"+wd not in listofwikidata:\n",
    "                        listofwikidata.append(\"wikidata:\"+wd)\n",
    "                    chedict[che.get('id')]['cheminf:000567'].append(\"wikidata:\"+wd)\n",
    "            if 'ChEMBL compound' in dictionaryforchemical:\n",
    "                chedict[che.get('id')]['cheminf:000412'] = []\n",
    "                for chembl in dictionaryforchemical['ChEMBL compound']:\n",
    "                    if \"chembl.compound:\"+chembl not in listofchembl:\n",
    "                        listofchembl.append(\"chembl.compound:\"+chembl)\n",
    "                    chedict[che.get('id')]['cheminf:000412'].append(\"chembl.compound:\"+chembl)\n",
    "            if 'PubChem-compound' in dictionaryforchemical:\n",
    "                chedict[che.get('id')]['cheminf:000140'] = []\n",
    "                for pub in dictionaryforchemical['PubChem-compound']:\n",
    "                    if \"pubchem.compound:\"+pub not in listofpubchem:\n",
    "                        listofpubchem.append(\"pubchem.compound:\"+pub)\n",
    "                    chedict[che.get('id')]['cheminf:000140'].append(\"pubchem.compound:\"+pub)\n",
    "            if 'DrugBank' in dictionaryforchemical:\n",
    "                chedict[che.get('id')]['cheminf:000406'] = []\n",
    "                for drugbank in dictionaryforchemical['DrugBank']:\n",
    "                    if \"drugbank:\"+drugbank not in listofdrugbank:\n",
    "                        listofdrugbank.append(\"drugbank:\"+drugbank)\n",
    "                    chedict[che.get('id')]['cheminf:000406'].append(\"drugbank:\"+drugbank)\n",
    "            if 'KEGG Compound' in dictionaryforchemical:\n",
    "                chedict[che.get('id')]['cheminf:000409'] = []\n",
    "                for kegg in dictionaryforchemical['KEGG Compound']:\n",
    "                    if \"kegg.compound:\"+kegg not in listofkegg:\n",
    "                        listofkegg.append(\"kegg.compound:\"+kegg)\n",
    "                    chedict[che.get('id')]['cheminf:000409'].append(\"kegg.compound:\"+kegg)\n",
    "            if 'LIPID MAPS' in dictionaryforchemical:\n",
    "                chedict[che.get('id')]['cheminf:000564'] = []\n",
    "                for lipidmaps in dictionaryforchemical['LIPID MAPS']:\n",
    "                    if \"lipidmaps:\"+lipidmaps not in listoflipidmaps:\n",
    "                        listoflipidmaps.append(\"lipidmaps:\"+lipidmaps)\n",
    "                    chedict[che.get('id')]['cheminf:000564'].append(\"lipidmaps:\"+lipidmaps)\n",
    "            if 'HMDB' in dictionaryforchemical:\n",
    "                chedict[che.get('id')]['cheminf:000408'] = []\n",
    "                for hmdb in dictionaryforchemical['HMDB']:\n",
    "                    if \"hmdb:\"+hmdb not in listofhmdb:\n",
    "                        listofhmdb.append(\"hmdb:\"+hmdb)\n",
    "                    chedict[che.get('id')]['cheminf:000408'].append(\"hmdb:\"+hmdb)\n",
    "        else:\n",
    "            chedict[che.get('id')]['dc:identifier'] = '\"' + che.find(aopxml + 'casrn').text + '\"'\n",
    "    if che.find(aopxml + 'jchem-inchi-key') is not None:\n",
    "        chedict[che.get('id')]['cheminf:000059'] = 'inchikey:' + str(che.find(aopxml + 'jchem-inchi-key').text)\n",
    "        listofinchikey.append('inchikey:' + str(che.find(aopxml + 'jchem-inchi-key').text))\n",
    "    if che.find(aopxml + 'preferred-name') is not None:\n",
    "        chedict[che.get('id')]['dc:title'] = '\"' + che.find(aopxml + 'preferred-name').text + '\"'\n",
    "    if che.find(aopxml + 'dsstox-id') is not None:\n",
    "        chedict[che.get('id')]['cheminf:000568'] = 'comptox:' + che.find(aopxml + 'dsstox-id').text\n",
    "        listofcomptox.append('comptox:' + che.find(aopxml + 'dsstox-id').text)\n",
    "    if che.find(aopxml + 'synonyms') is not None:\n",
    "        chedict[che.get('id')]['dcterms:alternative'] = []\n",
    "        for synonym in che.find(aopxml + 'synonyms').findall(aopxml + 'synonym'):\n",
    "            chedict[che.get('id')]['dcterms:alternative'].append(synonym.text[:-1])\n",
    "print('Done!\\n\\nA total of ' + str(len(chedict)) + ' chemicals have been parsed.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stressors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n",
      "\n",
      "A total of 658 Stressors have been parsed.\n"
     ]
    }
   ],
   "source": [
    "strdict = {}\n",
    "for stressor in root.findall(aopxml + 'stressor'):\n",
    "    strdict[stressor.get('id')] = {}\n",
    "    strdict[stressor.get('id')]['dc:identifier'] = 'aop.stressor:' + refs['Stressor'][stressor.get('id')]\n",
    "    strdict[stressor.get('id')]['rdfs:label'] = '\"Stressor ' + refs['Stressor'][stressor.get('id')] + '\"'\n",
    "    strdict[stressor.get('id')]['foaf:page'] = '<https://identifiers.org/aop.stressor/' + refs['Stressor'][stressor.get('id')] + '>'\n",
    "    strdict[stressor.get('id')]['dc:title'] = '\"' + stressor.find(aopxml + 'name').text + '\"'\n",
    "    if stressor.find(aopxml + 'description').text is not None:\n",
    "        strdict[stressor.get('id')]['dc:description'] = '\"\"\"' + TAG_RE.sub('', stressor.find(aopxml + 'description').text) + '\"\"\"'\n",
    "    strdict[stressor.get('id')]['dcterms:created'] = stressor.find(aopxml + 'creation-timestamp').text\n",
    "    strdict[stressor.get('id')]['dcterms:modified'] = stressor.find(aopxml + 'last-modification-timestamp').text\n",
    "    strdict[stressor.get('id')]['aopo:has_chemical_entity'] = []\n",
    "    strdict[stressor.get('id')]['linktochemical'] = []\n",
    "    if stressor.find(aopxml + 'chemicals') is not None:\n",
    "        for chemical in stressor.find(aopxml + 'chemicals').findall(aopxml + 'chemical-initiator'):\n",
    "            strdict[stressor.get('id')]['aopo:has_chemical_entity'].append('\"' + chemical.get('user-term') + '\"')\n",
    "            strdict[stressor.get('id')]['linktochemical'].append(chemical.get('chemical-id'))\n",
    "print('Done!\\n\\nA total of ' + str(len(strdict)) + ' Stressors have been parsed.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Taxonomy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n",
      "\n",
      "A total of 278 taxonomies have been parsed.\n"
     ]
    }
   ],
   "source": [
    "taxdict = {}\n",
    "for tax in root.findall(aopxml + 'taxonomy'):\n",
    "    taxdict[tax.get('id')] = {}\n",
    "    taxdict[tax.get('id')]['dc:source'] = tax.find(aopxml + 'source').text\n",
    "    taxdict[tax.get('id')]['dc:title'] = tax.find(aopxml + 'name').text\n",
    "    if taxdict[tax.get('id')]['dc:source'] == 'NCBI':\n",
    "        taxdict[tax.get('id')]['dc:identifier'] = 'ncbitaxon:' + tax.find(aopxml + 'source-id').text\n",
    "    elif taxdict[tax.get('id')]['dc:source'] is not None:\n",
    "        taxdict[tax.get('id')]['dc:identifier'] = '\"' + tax.find(aopxml + 'source-id').text + '\"'\n",
    "    else:\n",
    "        taxdict[tax.get('id')]['dc:identifier'] = '\"' + tax.find(aopxml + 'source-id').text + '\"'\n",
    "print('Done!\\n\\nA total of ' + str(len(taxdict)) + ' taxonomies have been parsed.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Key Event Components\n",
    "Which comprise of the Biological Actions, Biological Processes, Biological Objects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n",
      "A total of 12 Biological Activity annotations have been parsed.\n"
     ]
    }
   ],
   "source": [
    "bioactdict = {None: {}}\n",
    "bioactdict[None]['dc:identifier'] = None\n",
    "bioactdict[None]['dc:source'] = None\n",
    "bioactdict[None]['dc:title'] = None\n",
    "for bioact in root.findall(aopxml + 'biological-action'):\n",
    "    bioactdict[bioact.get('id')] = {}\n",
    "    bioactdict[bioact.get('id')]['dc:source'] = '\"' + bioact.find(aopxml + 'source').text + '\"'\n",
    "    bioactdict[bioact.get('id')]['dc:title'] = '\"' + bioact.find(aopxml + 'name').text + '\"'\n",
    "    bioactdict[bioact.get('id')]['dc:identifier'] = '\"WIKI:' + bioact.find(aopxml + 'source-id').text + '\"'\n",
    "print('Done!\\nA total of ' + str(len(bioactdict)) + ' Biological Activity annotations have been parsed.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n",
      "\n",
      "A total of 499 Biological Process annotations have been parsed.\n"
     ]
    }
   ],
   "source": [
    "bioprodict = {None: {}}\n",
    "bioprodict[None]['dc:identifier'] = None\n",
    "bioprodict[None]['dc:source'] = None\n",
    "bioprodict[None]['dc:title'] = None\n",
    "for biopro in root.findall(aopxml + 'biological-process'):\n",
    "    bioprodict[biopro.get('id')] = {}\n",
    "    bioprodict[biopro.get('id')]['dc:source'] = '\"' + biopro.find(aopxml + 'source').text + '\"'\n",
    "    bioprodict[biopro.get('id')]['dc:title'] = '\"' + biopro.find(aopxml + 'name').text + '\"'\n",
    "    if bioprodict[biopro.get('id')]['dc:source'] == '\"GO\"':\n",
    "        bioprodict[biopro.get('id')]['dc:identifier'] = 'go:' + biopro.find(aopxml + 'source-id').text[3:] \n",
    "    elif bioprodict[biopro.get('id')]['dc:source'] == '\"MI\"':\n",
    "        bioprodict[biopro.get('id')]['dc:identifier'] = 'mi:' + biopro.find(aopxml + 'source-id').text\n",
    "    elif bioprodict[biopro.get('id')]['dc:source'] == '\"MP\"':\n",
    "        bioprodict[biopro.get('id')]['dc:identifier'] = 'mp:' + biopro.find(aopxml + 'source-id').text[3:]\n",
    "    elif bioprodict[biopro.get('id')]['dc:source'] == '\"MESH\"':\n",
    "        bioprodict[biopro.get('id')]['dc:identifier'] = 'mesh:' + biopro.find(aopxml + 'source-id').text\n",
    "    elif bioprodict[biopro.get('id')]['dc:source'] == '\"HP\"':\n",
    "        bioprodict[biopro.get('id')]['dc:identifier'] = 'hp:' + biopro.find(aopxml + 'source-id').text[3:]\n",
    "    elif bioprodict[biopro.get('id')]['dc:source'] == '\"PCO\"':\n",
    "        bioprodict[biopro.get('id')]['dc:identifier'] = 'pco:' + biopro.find(aopxml + 'source-id').text[4:]\n",
    "    elif bioprodict[biopro.get('id')]['dc:source'] == '\"NBO\"':\n",
    "        bioprodict[biopro.get('id')]['dc:identifier'] = 'nbo:' + biopro.find(aopxml + 'source-id').text[4:]\n",
    "    elif bioprodict[biopro.get('id')]['dc:source'] == '\"VT\"':\n",
    "        bioprodict[biopro.get('id')]['dc:identifier'] = 'vt:' + biopro.find(aopxml + 'source-id').text[3:]\n",
    "    elif bioprodict[biopro.get('id')]['dc:source'] == '\"RBO\"':\n",
    "        bioprodict[biopro.get('id')]['dc:identifier'] = 'rbo:' + biopro.find(aopxml + 'source-id').text[4:]\n",
    "    elif bioprodict[biopro.get('id')]['dc:source'] == '\"NCI\"':\n",
    "        bioprodict[biopro.get('id')]['dc:identifier'] = 'nci:' + biopro.find(aopxml + 'source-id').text[4:]\n",
    "    else:\n",
    "        bioprodict[biopro.get('id')]['dc:identifier'] = biopro.find(aopxml + 'source-id').text\n",
    "print('Done!\\n\\nA total of ' + str(len(bioprodict)) + ' Biological Process annotations have been parsed.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n",
      "\n",
      "A total of 453 Biological Object annotations have been parsed.\n"
     ]
    }
   ],
   "source": [
    "bioobjdict = {None: {}}\n",
    "bioobjdict[None]['dc:identifier'] = None\n",
    "bioobjdict[None]['dc:source'] = None\n",
    "bioobjdict[None]['dc:title'] = None\n",
    "objectstoskip = []\n",
    "prolist = []\n",
    "for bioobj in root.findall(aopxml + 'biological-object'):\n",
    "    bioobjdict[bioobj.get('id')] = {}\n",
    "    bioobjdict[bioobj.get('id')]['dc:source'] = '\"' + bioobj.find(aopxml + 'source').text + '\"'\n",
    "    bioobjdict[bioobj.get('id')]['dc:title'] = '\"' + bioobj.find(aopxml + 'name').text + '\"'\n",
    "    if bioobjdict[bioobj.get('id')]['dc:source'] == '\"PR\"':\n",
    "        bioobjdict[bioobj.get('id')]['dc:identifier'] = 'pr:' + bioobj.find(aopxml + 'source-id').text[3:]\n",
    "        prolist.append('pr:' + bioobj.find(aopxml + 'source-id').text[3:])\n",
    "    elif bioobjdict[bioobj.get('id')]['dc:source'] == '\"CL\"':\n",
    "        bioobjdict[bioobj.get('id')]['dc:identifier'] = 'cl:' + bioobj.find(aopxml + 'source-id').text[3:]\n",
    "    elif bioobjdict[bioobj.get('id')]['dc:source'] == '\"MESH\"':\n",
    "        bioobjdict[bioobj.get('id')]['dc:identifier'] = 'mesh:' + bioobj.find(aopxml + 'source-id').text\n",
    "    elif bioobjdict[bioobj.get('id')]['dc:source'] == '\"GO\"':\n",
    "        bioobjdict[bioobj.get('id')]['dc:identifier'] = 'go:' + bioobj.find(aopxml + 'source-id').text[3:] \n",
    "    elif bioobjdict[bioobj.get('id')]['dc:source'] == '\"UBERON\"':\n",
    "        bioobjdict[bioobj.get('id')]['dc:identifier'] = 'uberon:' + bioobj.find(aopxml + 'source-id').text[7:]\n",
    "    elif bioobjdict[bioobj.get('id')]['dc:source'] == '\"CHEBI\"':\n",
    "        bioobjdict[bioobj.get('id')]['dc:identifier'] = 'chebio:' + bioobj.find(aopxml + 'source-id').text[6:]\n",
    "    elif bioobjdict[bioobj.get('id')]['dc:source'] == '\"MP\"':\n",
    "        bioobjdict[bioobj.get('id')]['dc:identifier'] = 'mp:' + bioobj.find(aopxml + 'source-id').text[3:]\n",
    "    elif bioobjdict[bioobj.get('id')]['dc:source'] == '\"FMA\"':\n",
    "        bioobjdict[bioobj.get('id')]['dc:identifier'] = 'fma:' + bioobj.find(aopxml + 'source-id').text[4:]\n",
    "    elif bioobjdict[bioobj.get('id')]['dc:source'] == '\"PCO\"':\n",
    "        bioobjdict[bioobj.get('id')]['dc:identifier'] = 'pco:' + bioobj.find(aopxml + 'source-id').text[4:]\n",
    "    else:\n",
    "        bioobjdict[bioobj.get('id')]['dc:identifier'] = '\"' + bioobj.find(aopxml + 'source-id').text + '\"'\n",
    "print('Done!\\n\\nA total of ' + str(len(bioobjdict)) + ' Biological Object annotations have been parsed.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Biological Objects containing terms from the Protein Ontology are mapped to protein identifiers with the PR mapping file `promapping.txt`, which was downloaded from the [Protein Consortium website](https://proconsortium.org/download/current/), which provides matching identifiers from Entrez Gene, HGNC and UniProt. The file location should be the `filepath` variable defined in Step #2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('data/promapping.txt', <http.client.HTTPMessage at 0x7e6d69258cb0>)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pro = \"promapping.txt\"\n",
    "urllib.request.urlretrieve('https://proconsortium.org/download/current/promapping.txt', 'data/promapping.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Last Modified Time :  Tue Aug 20 21:10:51 2024\n"
     ]
    }
   ],
   "source": [
    "fileStatsObj = os.stat (filepath + pro)\n",
    "PromodificationTime = time.ctime ( fileStatsObj [ stat.ST_MTIME ] )\n",
    "print(\"Last Modified Time : \", PromodificationTime )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This step added 686 identifiers for 146 Protein Ontology terms\n"
     ]
    }
   ],
   "source": [
    "f = open(filepath+pro, \"r\")\n",
    "prodict = {}\n",
    "hgnclist = []\n",
    "uniprotlist = []\n",
    "ncbigenelist = []\n",
    "for line in f:\n",
    "    a = line.split('\\t')\n",
    "    key = 'pr:'+a[0][3:]\n",
    "    if key in prolist:\n",
    "        if not key in prodict:\n",
    "            prodict[key] = []\n",
    "        if 'HGNC:' in a[1]:\n",
    "            prodict[key].append('hgnc:'+a[1][5:])\n",
    "            hgnclist.append('hgnc:'+a[1][5:])\n",
    "        if 'NCBIGene:' in a[1]:\n",
    "            prodict[key].append('ncbigene:'+a[1][9:])\n",
    "            ncbigenelist.append('ncbigene:'+a[1][9:])\n",
    "        if 'UniProtKB:' in a[1]:\n",
    "            prodict[key].append('uniprot:'+a[1].split(',')[0][10:])\n",
    "            uniprotlist.append('uniprot:'+a[1].split(',')[0][10:])\n",
    "        if prodict[key]==[]:\n",
    "            del prodict[key]\n",
    "f.close()\n",
    "print('This step added ' + str(len(hgnclist)+len(ncbigenelist)+len(uniprotlist)) + ' identifiers for ' + str(len(prodict)) + ' Protein Ontology terms')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Key Events\n",
    "The KEs also include the entities for cell-terms and organ-terms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n",
      "\n",
      "A total of 1455 Key Events have been parsed.\n"
     ]
    }
   ],
   "source": [
    "listofkedescriptions = []\n",
    "for ke in root.findall(aopxml + 'key-event'):\n",
    "    if not ke.get('id') in kedict:\n",
    "        kedict[ke.get('id')] = {}\n",
    "    kedict[ke.get('id')]['dc:identifier'] = 'aop.events:' + refs['KE'][ke.get('id')]\n",
    "    kedict[ke.get('id')]['rdfs:label'] = '\"KE ' + refs['KE'][ke.get('id')] + '\"'\n",
    "    kedict[ke.get('id')]['foaf:page'] = '<https://identifiers.org/aop.events/' + refs['KE'][ke.get('id')] + '>'\n",
    "    kedict[ke.get('id')]['dc:title'] = '\"' + ke.find(aopxml + 'title').text + '\"'\n",
    "    kedict[ke.get('id')]['dcterms:alternative'] = ke.find(aopxml + 'short-name').text\n",
    "    kedict[ke.get('id')]['nci:C25664'] = '\"\"\"' + ke.find(aopxml + 'biological-organization-level').text + '\"\"\"'\n",
    "    if ke.find(aopxml + 'description').text is not None:\n",
    "        kedict[ke.get('id')]['dc:description'] = '\"\"\"' + TAG_RE.sub('', ke.find(aopxml + 'description').text) + '\"\"\"'\n",
    "#    if ke.find(aopxml + 'evidence-supporting-taxonomic-applicability').text is not None:\n",
    "#        kedict[ke.get('id')]['dc:description'] = '\"\"\"' + TAG_RE.sub('', ke.find(aopxml + 'evidence-supporting-taxonomic-applicability').text) + '\"\"\"'\n",
    "    if ke.find(aopxml + 'measurement-methodology').text is not None:\n",
    "        kedict[ke.get('id')]['mmo:0000000'] = '\"\"\"' + TAG_RE.sub('', ke.find(aopxml + 'measurement-methodology').text) + '\"\"\"'\n",
    "    kedict[ke.get('id')]['biological-organization-level'] = ke.find(aopxml + 'biological-organization-level').text\n",
    "    kedict[ke.get('id')]['dc:source'] = ke.find(aopxml + 'source').text\n",
    "    for appl in ke.findall(aopxml + 'applicability'):\n",
    "        for sex in appl.findall(aopxml + 'sex'):\n",
    "            if 'pato:0000047' not in kedict[ke.get('id')]:\n",
    "                kedict[ke.get('id')]['pato:0000047'] = [[sex.find(aopxml + 'evidence').text, sex.find(aopxml + 'sex').text]]\n",
    "            else:\n",
    "                kedict[ke.get('id')]['pato:0000047'].append([sex.find(aopxml + 'evidence').text, sex.find(aopxml + 'sex').text])\n",
    "        for life in appl.findall(aopxml + 'life-stage'):\n",
    "            if 'aopo:LifeStageContext' not in kedict[ke.get('id')]:\n",
    "                kedict[ke.get('id')]['aopo:LifeStageContext'] = [[life.find(aopxml + 'evidence').text, life.find(aopxml + 'life-stage').text]]\n",
    "            else:\n",
    "                kedict[ke.get('id')]['aopo:LifeStageContext'].append([life.find(aopxml + 'evidence').text, life.find(aopxml + 'life-stage').text])\n",
    "        for tax in appl.findall(aopxml + 'taxonomy'):\n",
    "            if 'ncbitaxon:131567' not in kedict[ke.get('id')]:\n",
    "                if 'dc:identifier' in taxdict[tax.get('taxonomy-id')]:\n",
    "                    kedict[ke.get('id')]['ncbitaxon:131567'] = [[tax.get('taxonomy-id'), tax.find(aopxml + 'evidence').text, taxdict[tax.get('taxonomy-id')]['dc:identifier'], taxdict[tax.get('taxonomy-id')]['dc:source'], taxdict[tax.get('taxonomy-id')]['dc:title']]]\n",
    "            else:\n",
    "                if 'dc:identifier' in taxdict[tax.get('taxonomy-id')]:\n",
    "                    kedict[ke.get('id')]['ncbitaxon:131567'].append([tax.get('taxonomy-id'), tax.find(aopxml + 'evidence').text, taxdict[tax.get('taxonomy-id')]['dc:identifier'], taxdict[tax.get('taxonomy-id')]['dc:source'], taxdict[tax.get('taxonomy-id')]['dc:title']])\n",
    "    if ke.find(aopxml + 'biological-events') is not None:\n",
    "        kedict[ke.get('id')]['biological-event'] = {}\n",
    "        kedict[ke.get('id')]['biological-event']['go:0008150'] = []\n",
    "        kedict[ke.get('id')]['biological-event']['pato:0001241'] = []\n",
    "        kedict[ke.get('id')]['biological-event']['pato:0000001'] = []\n",
    "        for event in ke.find(aopxml + 'biological-events').findall(aopxml + 'biological-event'):\n",
    "            if event.get('process-id') is not None:\n",
    "                kedict[ke.get('id')]['biological-event']['go:0008150'].append(bioprodict[event.get('process-id')]['dc:identifier'])\n",
    "            if event.get('object-id') is not None:\n",
    "                kedict[ke.get('id')]['biological-event']['pato:0001241'].append(bioobjdict[event.get('object-id')]['dc:identifier'])\n",
    "            if event.get('action-id') is not None:\n",
    "                kedict[ke.get('id')]['biological-event']['pato:0000001'].append(bioactdict[event.get('action-id')]['dc:identifier'])\n",
    "    if ke.find(aopxml + 'cell-term') is not None:\n",
    "        kedict[ke.get('id')]['aopo:CellTypeContext'] = {}\n",
    "        kedict[ke.get('id')]['aopo:CellTypeContext']['dc:source'] = '\"' + ke.find(aopxml + 'cell-term').find(aopxml + 'source').text + '\"'\n",
    "        kedict[ke.get('id')]['aopo:CellTypeContext']['dc:title'] = '\"' + ke.find(aopxml + 'cell-term').find(aopxml + 'name').text + '\"'\n",
    "        if kedict[ke.get('id')]['aopo:CellTypeContext']['dc:source'] == '\"CL\"':\n",
    "            kedict[ke.get('id')]['aopo:CellTypeContext']['dc:identifier'] = ['cl:' + ke.find(aopxml + 'cell-term').find(aopxml + 'source-id').text[3:], ke.find(aopxml + 'cell-term').find(aopxml + 'source-id').text]\n",
    "        elif kedict[ke.get('id')]['aopo:CellTypeContext']['dc:source'] == '\"UBERON\"':\n",
    "            kedict[ke.get('id')]['aopo:CellTypeContext']['dc:identifier'] = ['uberon:' + ke.find(aopxml + 'cell-term').find(aopxml + 'source-id').text[7:], ke.find(aopxml + 'cell-term').find(aopxml + 'source-id').text]\n",
    "        else:\n",
    "            kedict[ke.get('id')]['aopo:CellTypeContext']['dc:identifier'] = ['\"' + ke.find(aopxml + 'cell-term').find(aopxml + 'source-id').text + '\"', 'placeholder']\n",
    "    if ke.find(aopxml + 'organ-term') is not None:\n",
    "        kedict[ke.get('id')]['aopo:OrganContext'] = {}\n",
    "        kedict[ke.get('id')]['aopo:OrganContext']['dc:source'] = '\"' + ke.find(aopxml + 'organ-term').find(aopxml + 'source').text + '\"'\n",
    "        kedict[ke.get('id')]['aopo:OrganContext']['dc:title'] = '\"' + ke.find(aopxml + 'organ-term').find(aopxml + 'name').text + '\"'\n",
    "        if kedict[ke.get('id')]['aopo:OrganContext']['dc:source'] == '\"UBERON\"':\n",
    "            kedict[ke.get('id')]['aopo:OrganContext']['dc:identifier'] = ['uberon:' + ke.find(aopxml + 'organ-term').find(aopxml + 'source-id').text[7:], ke.find(aopxml + 'organ-term').find(aopxml + 'source-id').text]\n",
    "        else:\n",
    "            kedict[ke.get('id')]['aopo:OrganContext']['dc:identifier'] = [\n",
    "                '\"' + ke.find(aopxml + 'organ-term').find(aopxml + 'source-id').text + '\"', 'placeholder']\n",
    "    if ke.find(aopxml + 'key-event-stressors') is not None:\n",
    "        kedict[ke.get('id')]['nci:C54571'] = {}\n",
    "        for stressor in ke.find(aopxml + 'key-event-stressors').findall(aopxml + 'key-event-stressor'):\n",
    "            kedict[ke.get('id')]['nci:C54571'][stressor.get('stressor-id')] = {}\n",
    "            kedict[ke.get('id')]['nci:C54571'][stressor.get('stressor-id')]['dc:identifier'] = strdict[stressor.get('stressor-id')]['dc:identifier']\n",
    "            kedict[ke.get('id')]['nci:C54571'][stressor.get('stressor-id')]['aopo:has_evidence'] = stressor.find(aopxml + 'evidence').text\n",
    "print('Done!\\n\\nA total of ' + str(len(kedict)) + ' Key Events have been parsed.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Key Event Relationships"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n",
      "\n",
      "A total of 2042 Key Event Relationships have been parsed.\n"
     ]
    }
   ],
   "source": [
    "kerdict = {}\n",
    "for ker in root.findall(aopxml + 'key-event-relationship'):\n",
    "    kerdict[ker.get('id')] = {}\n",
    "    kerdict[ker.get('id')]['dc:identifier'] = 'aop.relationships:' + refs['KER'][ker.get('id')]\n",
    "    kerdict[ker.get('id')]['rdfs:label'] = '\"KER ' + refs['KER'][ker.get('id')] + '\"'\n",
    "    kerdict[ker.get('id')]['foaf:page'] = '<https://identifiers.org/aop.relationships/' + refs['KER'][ker.get('id')] + '>'\n",
    "    kerdict[ker.get('id')]['dc:source'] = ker.find(aopxml + 'source').text\n",
    "    kerdict[ker.get('id')]['dcterms:created'] = ker.find(aopxml + 'creation-timestamp').text\n",
    "    kerdict[ker.get('id')]['dcterms:modified'] = ker.find(aopxml + 'last-modification-timestamp').text\n",
    "    if ker.find(aopxml + 'description').text is not None:\n",
    "        kerdict[ker.get('id')]['dc:description'] = '\"\"\"' + TAG_RE.sub('', ker.find(aopxml + 'description').text) + '\"\"\"'\n",
    "    for weight in ker.findall(aopxml + 'weight-of-evidence'):\n",
    "        if weight.find(aopxml + 'biological-plausibility').text is not None:\n",
    "            kerdict[ker.get('id')]['nci:C80263'] = '\"\"\"' + TAG_RE.sub('', weight.find(aopxml + 'biological-plausibility').text) + '\"\"\"'\n",
    "        if weight.find(aopxml + 'emperical-support-linkage').text is not None:\n",
    "            kerdict[ker.get('id')]['edam:data_2042'] = '\"\"\"' + TAG_RE.sub('', weight.find(aopxml + 'emperical-support-linkage').text) + '\"\"\"'\n",
    "        if weight.find(aopxml + 'uncertainties-or-inconsistencies').text is not None:\n",
    "            kerdict[ker.get('id')]['nci:C71478'] = '\"\"\"' + TAG_RE.sub('', weight.find(aopxml + 'uncertainties-or-inconsistencies').text) + '\"\"\"'\n",
    "    kerdict[ker.get('id')]['aopo:has_upstream_key_event'] = {}\n",
    "    kerdict[ker.get('id')]['aopo:has_upstream_key_event']['id'] = ker.find(aopxml + 'title').find(aopxml + 'upstream-id').text\n",
    "    kerdict[ker.get('id')]['aopo:has_upstream_key_event']['dc:identifier'] = 'aop.events:' + refs['KE'][ker.find(aopxml + 'title').find(aopxml + 'upstream-id').text]\n",
    "    kerdict[ker.get('id')]['aopo:has_downstream_key_event'] = {}\n",
    "    kerdict[ker.get('id')]['aopo:has_downstream_key_event']['id'] = ker.find(aopxml + 'title').find(aopxml + 'downstream-id').text\n",
    "    kerdict[ker.get('id')]['aopo:has_downstream_key_event']['dc:identifier'] = 'aop.events:' + refs['KE'][ker.find(aopxml + 'title').find(aopxml + 'downstream-id').text]\n",
    "    for appl in ker.findall(aopxml + 'taxonomic-applicability'):\n",
    "        for sex in appl.findall(aopxml + 'sex'):\n",
    "            if 'pato:0000047' not in kerdict[ker.get('id')]:\n",
    "                kerdict[ker.get('id')]['pato:0000047'] = [[sex.find(aopxml + 'evidence').text, sex.find(aopxml + 'sex').text]]\n",
    "            else:\n",
    "                kerdict[ker.get('id')]['pato:0000047'].append([sex.find(aopxml + 'evidence').text, sex.find(aopxml + 'sex').text])\n",
    "        for life in appl.findall(aopxml + 'life-stage'):\n",
    "            if 'aopo:LifeStageContext' not in kerdict[ker.get('id')]:\n",
    "                kerdict[ker.get('id')]['aopo:LifeStageContext'] = [[life.find(aopxml + 'evidence').text, life.find(aopxml + 'life-stage').text]]\n",
    "            else:\n",
    "                kerdict[ker.get('id')]['aopo:LifeStageContext'].append([life.find(aopxml + 'evidence').text, life.find(aopxml + 'life-stage').text])\n",
    "        for tax in appl.findall(aopxml + 'taxonomy'):\n",
    "            if 'ncbitaxon:131567' not in kerdict[ker.get('id')]:\n",
    "                if 'dc:identifier' in taxdict[tax.get('taxonomy-id')]:\n",
    "                    kerdict[ker.get('id')]['ncbitaxon:131567'] = [[tax.get('taxonomy-id'), tax.find(aopxml + 'evidence').text, taxdict[tax.get('taxonomy-id')]['dc:identifier'], taxdict[tax.get('taxonomy-id')]['dc:source'], taxdict[tax.get('taxonomy-id')]['dc:title']]]\n",
    "            else:\n",
    "                if 'dc:identifier' in taxdict[tax.get('taxonomy-id')]:\n",
    "                    kerdict[ker.get('id')]['ncbitaxon:131567'].append([tax.get('taxonomy-id'), tax.find(aopxml + 'evidence').text, taxdict[tax.get('taxonomy-id')]['dc:identifier'], taxdict[tax.get('taxonomy-id')]['dc:source'], taxdict[tax.get('taxonomy-id')]['dc:title']])\n",
    "print('Done!\\n\\nA total of ' + str(len(kerdict)) + ' Key Event Relationships have been parsed.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b>Step #4: Writing the AOP-Wiki RDF</b>\n",
    "This step involves the writing of the central RDF file, containing all information from the AOP-Wiki XML, written in Turtle (ttl) syntax."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "g = open(filepath + 'AOPWikiRDF.ttl', 'w', encoding='utf-8')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing prefixes\n",
    "The first thing is writing the Prefixes of all ontologies and database identifiers, which go in the top of the document. That is followed by the writing of all entities of the AOP-Wiki described in Figure 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2589"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "g.write('@prefix dc: <http://purl.org/dc/elements/1.1/> .\\n@prefix dcterms: <http://purl.org/dc/terms/> .\\n@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .\\n@prefix owl: <http://www.w3.org/2002/07/owl#> .\\n@prefix foaf: <http://xmlns.com/foaf/0.1/> .\\n@prefix aop: <https://identifiers.org/aop/> .\\n@prefix aop.events: <https://identifiers.org/aop.events/> .\\n@prefix aop.relationships: <https://identifiers.org/aop.relationships/> .\\n@prefix aop.stressor: <https://identifiers.org/aop.stressor/> .\\n@prefix aopo: <http://aopkb.org/aop_ontology#> .\\n@prefix skos: <http://www.w3.org/2004/02/skos/core#> . \\n@prefix cas: <https://identifiers.org/cas/> .\\n@prefix inchikey: <https://identifiers.org/inchikey/> .\\n@prefix pato: <http://purl.obolibrary.org/obo/PATO_> .\\n@prefix ncbitaxon: <http://purl.bioontology.org/ontology/NCBITAXON/> .\\n@prefix cl: <http://purl.obolibrary.org/obo/CL_> .\\n@prefix uberon: <http://purl.obolibrary.org/obo/UBERON_> .\\n@prefix go: <http://purl.obolibrary.org/obo/GO_> .\\n@prefix mi: <http://purl.obolibrary.org/obo/MI_> .\\n@prefix mp: <http://purl.obolibrary.org/obo/MP_> .\\n@prefix mesh: <http://purl.org/commons/record/mesh/> .\\n@prefix hp: <http://purl.obolibrary.org/obo/HP_> .\\n@prefix pco: <http://purl.obolibrary.org/obo/PCO_> .\\n@prefix nbo: <http://purl.obolibrary.org/obo/NBO_> .\\n@prefix vt: <http://purl.obolibrary.org/obo/VT_> .\\n@prefix pr: <http://purl.obolibrary.org/obo/PR_> .\\n@prefix chebio: <http://purl.obolibrary.org/obo/CHEBI_> .\\n@prefix fma: <http://purl.org/sig/ont/fma/fma> .\\n@prefix cheminf: <http://semanticscience.org/resource/CHEMINF_> .\\n@prefix nci: <http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#> .\\n@prefix comptox: <https://identifiers.org/comptox/> .\\n@prefix mmo: <http://purl.obolibrary.org/obo/MMO_> .\\n@prefix chebi: <https://identifiers.org/chebi/> .\\n@prefix chemspider: <https://identifiers.org/chemspider/> .\\n@prefix wikidata: <https://identifiers.org/wikidata/> .\\n@prefix chembl.compound: <https://identifiers.org/chembl.compound/> .\\n@prefix pubchem.compound: <https://identifiers.org/pubchem.compound/> .\\n@prefix drugbank: <https://identifiers.org/drugbank/> .\\n@prefix kegg.compound: <https://identifiers.org/kegg.compound/> .\\n@prefix lipidmaps: <https://identifiers.org/lipidmaps/> .\\n@prefix hmdb: <https://identifiers.org/hmdb/> .\\n@prefix ensembl: <https://identifiers.org/ensembl/> .\\n@prefix edam: <http://edamontology.org/> .\\n@prefix hgnc: <https://identifiers.org/hgnc/>.\\n@prefix ncbigene: <https://identifiers.org/ncbigene/>.\\n@prefix uniprot: <https://identifiers.org/uniprot/>.\\n@prefix rbo: <http://purl.obolibrary.org/obo/RBO_>.\\n\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing Adverse Outcome Pathway triples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n"
     ]
    }
   ],
   "source": [
    "for aop in aopdict:\n",
    "    g.write(aopdict[aop]['dc:identifier'] + '\\n\\ta\\taopo:AdverseOutcomePathway ;\\n\\tdc:identifier\\t' + aopdict[aop]['dc:identifier'] + ' ;\\n\\trdfs:label\\t' + aopdict[aop]['rdfs:label'] + ' ;\\n\\trdfs:seeAlso\\t' + aopdict[aop]['foaf:page'] + ' ;\\n\\tfoaf:page\\t' + aopdict[aop]['foaf:page'] + ' ;\\n\\tdc:title\\t' + aopdict[aop]['dc:title'] + ' ;\\n\\tdcterms:alternative\\t\"' + aopdict[aop]['dcterms:alternative'] + '\" ;\\n\\tdc:source\\t\"' + aopdict[aop]['dc:source'] + '\" ;\\n\\tdcterms:created\\t\"' + aopdict[aop]['dcterms:created'] + '\" ;\\n\\tdcterms:modified\\t\"' + aopdict[aop]['dcterms:modified'] + '\"')\n",
    "    if 'dc:description' in aopdict[aop]:\n",
    "        if not aopdict[aop]['dc:description'] == []:\n",
    "            g.write(' ;\\n\\tdc:description\\t' + ','.join(aopdict[aop]['dc:description']))\n",
    "    if 'nci:C25217' in aopdict[aop]:\n",
    "        g.write(' ;\\n\\tnci:C25217\\t' + aopdict[aop]['nci:C25217'])\n",
    "    if 'nci:C48192' in aopdict[aop]:\n",
    "        g.write(' ;\\n\\tnci:C48192\\t' + aopdict[aop]['nci:C48192'])\n",
    "    if 'aopo:AopContext' in aopdict[aop]:\n",
    "        g.write(' ;\\n\\taopo:AopContext\\t' + aopdict[aop]['aopo:AopContext'])\n",
    "    if 'aopo:has_evidence' in aopdict[aop]:\n",
    "        g.write(' ;\\n\\taopo:has_evidence\\t' + aopdict[aop]['aopo:has_evidence'])\n",
    "    if 'edam:operation_3799' in aopdict[aop]:\n",
    "        g.write(' ;\\n\\tedam:operation_3799\\t' + aopdict[aop]['edam:operation_3799'])\n",
    "    if 'nci:C25725' in aopdict[aop]:\n",
    "        g.write(' ;\\n\\tnci:C25725\\t' + aopdict[aop]['nci:C25725'])\n",
    "    if 'dc:creator' in aopdict[aop]:\n",
    "        g.write(' ;\\n\\tdc:creator\\t' + aopdict[aop]['dc:creator'])\n",
    "    if 'dcterms:accessRights' in aopdict[aop]:\n",
    "         g.write(' ;\\n\\tdcterms:accessRights\\t' + aopdict[aop]['dcterms:accessRights'])\n",
    "    if 'oecd-status' in aopdict[aop]:\n",
    "         g.write(' ;\\n\\tnci:C25688\\t' + aopdict[aop]['oecd-status'])\n",
    "    if 'saaop-status' in aopdict[aop]:\n",
    "         g.write(' ;\\n\\tnci:C25688\\t' + aopdict[aop]['saaop-status'])\n",
    "    if 'dcterms:abstract' in aopdict[aop]:\n",
    "        g.write(' ;\\n\\tdcterms:abstract\\t' + aopdict[aop]['dcterms:abstract'])\n",
    "    listofthings = []\n",
    "    for KE in aopdict[aop]['aopo:has_key_event']:\n",
    "        listofthings.append(aopdict[aop]['aopo:has_key_event'][KE]['dc:identifier'])\n",
    "    if not listofthings == []:\n",
    "        g.write(' ;\\n\\taopo:has_key_event\\t' + (','.join(listofthings)))\n",
    "    listofthings = []\n",
    "    for KER in aopdict[aop]['aopo:has_key_event_relationship']:\n",
    "        listofthings.append(aopdict[aop]['aopo:has_key_event_relationship'][KER]['dc:identifier'])\n",
    "    if not listofthings == []:\n",
    "        g.write(' ;\\n\\taopo:has_key_event_relationship\\t' + (','.join(listofthings)))\n",
    "    listofthings = []\n",
    "    for mie in aopdict[aop]['aopo:has_molecular_initiating_event']:\n",
    "        listofthings.append(aopdict[aop]['aopo:has_molecular_initiating_event'][mie]['dc:identifier'])\n",
    "    if not listofthings == []:\n",
    "        g.write(' ;\\n\\taopo:has_molecular_initiating_event\\t' + (','.join(listofthings)))\n",
    "    listofthings = []\n",
    "    for ao in aopdict[aop]['aopo:has_adverse_outcome']:\n",
    "        listofthings.append(aopdict[aop]['aopo:has_adverse_outcome'][ao]['dc:identifier'])\n",
    "    if not listofthings == []:\n",
    "        g.write(' ;\\n\\taopo:has_adverse_outcome\\t' + (','.join(listofthings)))\n",
    "    listofthings = []\n",
    "    for stressor in aopdict[aop]['nci:C54571']:\n",
    "        listofthings.append(aopdict[aop]['nci:C54571'][stressor]['dc:identifier'])\n",
    "    if not listofthings == []:\n",
    "        g.write(' ;\\n\\tnci:C54571\\t' + (','.join(listofthings)))\n",
    "    listofthings = []\n",
    "    if 'pato:0000047' in aopdict[aop]:\n",
    "        for sex in aopdict[aop]['pato:0000047']:\n",
    "            listofthings.append('\"' + sex[1] + '\"')\n",
    "        if not listofthings == []:\n",
    "            g.write(' ;\\n\\tpato:0000047\\t' + (','.join(listofthings)))\n",
    "    listofthings = []\n",
    "    if 'aopo:LifeStageContext' in aopdict[aop]:\n",
    "        for lifestage in aopdict[aop]['aopo:LifeStageContext']:\n",
    "            listofthings.append('\"' + lifestage[1] + '\"')\n",
    "        if not listofthings == []:\n",
    "            g.write(' ;\\n\\taopo:LifeStageContext\\t' + (','.join(listofthings)))\n",
    "    g.write(' .\\n\\n')\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing Key Event triples\n",
    "This step also includes the extraction of the cell-terms and organ-terms, which are written to the file later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n"
     ]
    }
   ],
   "source": [
    "cterm = {}\n",
    "oterm = {}\n",
    "for ke in kedict:\n",
    "    g.write(kedict[ke]['dc:identifier'] + '\\n\\ta\\taopo:KeyEvent ;\\n\\tdc:identifier\\t' + kedict[ke]['dc:identifier'] + ' ;\\n\\trdfs:label\\t' + kedict[ke]['rdfs:label'] + ' ;\\n\\tfoaf:page\\t' + kedict[ke]['foaf:page'] + ' ;\\n\\trdfs:seeAlso\\t' + kedict[ke]['foaf:page'] + ' ;\\n\\tdc:title\\t' + kedict[ke]['dc:title'] + ' ;\\n\\tdcterms:alternative\\t\"' + kedict[ke]['dcterms:alternative'] + '\" ;\\n\\tdc:source\\t\"' + kedict[ke]['dc:source'] + '\"')\n",
    "    if 'dc:description' in kedict[ke]:\n",
    "        g.write(' ;\\n\\tdc:description\\t' + kedict[ke]['dc:description'])\n",
    "    if 'mmo:0000000' in kedict[ke]:\n",
    "        g.write(' ;\\n\\tmmo:0000000\\t' + kedict[ke]['mmo:0000000'])\n",
    "    if 'nci:C25664' in kedict[ke]:\n",
    "        g.write(' ;\\n\\tnci:C25664\\t' + kedict[ke]['nci:C25664'])\n",
    "    listofthings = []\n",
    "    if 'pato:0000047' in kedict[ke]:\n",
    "        for sex in kedict[ke]['pato:0000047']:\n",
    "            listofthings.append('\"' + sex[1] + '\"')\n",
    "        if not listofthings == []:\n",
    "            g.write(' ;\\n\\tpato:0000047\\t' + (','.join(listofthings)))\n",
    "    listofthings = []\n",
    "    if 'aopo:LifeStageContext' in kedict[ke]:\n",
    "        for lifestage in kedict[ke]['aopo:LifeStageContext']:\n",
    "            listofthings.append('\"' + lifestage[1] + '\"')\n",
    "        if not listofthings == []:\n",
    "            g.write(' ;\\n\\taopo:LifeStageContext\\t' + (','.join(listofthings)))\n",
    "    listofthings = []\n",
    "    if 'ncbitaxon:131567' in kedict[ke]:\n",
    "        for taxonomy in kedict[ke]['ncbitaxon:131567']:\n",
    "            listofthings.append(taxonomy[2])\n",
    "        if not listofthings == []:\n",
    "            g.write(' ;\\n\\tncbitaxon:131567\\t' + (','.join(listofthings)))\n",
    "    listofthings = []\n",
    "    if 'nci:C54571' in kedict[ke]:\n",
    "        for stressor in kedict[ke]['nci:C54571']:\n",
    "            listofthings.append(kedict[ke]['nci:C54571'][stressor]['dc:identifier'])\n",
    "        if not listofthings == []:\n",
    "            g.write(' ;\\n\\tnci:C54571\\t' + (','.join(listofthings)))\n",
    "    if 'aopo:CellTypeContext' in kedict[ke]:\n",
    "        g.write(' ;\\n\\taopo:CellTypeContext\\t' + kedict[ke]['aopo:CellTypeContext']['dc:identifier'][0])\n",
    "        if not kedict[ke]['aopo:CellTypeContext']['dc:identifier'][0] in cterm:\n",
    "            cterm[kedict[ke]['aopo:CellTypeContext']['dc:identifier'][0]] = {}\n",
    "            cterm[kedict[ke]['aopo:CellTypeContext']['dc:identifier'][0]]['dc:source'] = kedict[ke]['aopo:CellTypeContext']['dc:source']\n",
    "            cterm[kedict[ke]['aopo:CellTypeContext']['dc:identifier'][0]]['dc:title'] = kedict[ke]['aopo:CellTypeContext']['dc:title']\n",
    "    if 'aopo:OrganContext' in kedict[ke]:\n",
    "        g.write(' ;\\n\\taopo:OrganContext\\t' + kedict[ke]['aopo:OrganContext']['dc:identifier'][0])\n",
    "        if not kedict[ke]['aopo:OrganContext']['dc:identifier'][0] in oterm:\n",
    "            oterm[kedict[ke]['aopo:OrganContext']['dc:identifier'][0]] = {}\n",
    "            oterm[kedict[ke]['aopo:OrganContext']['dc:identifier'][0]]['dc:source'] = kedict[ke]['aopo:OrganContext']['dc:source']\n",
    "            oterm[kedict[ke]['aopo:OrganContext']['dc:identifier'][0]]['dc:title'] = kedict[ke]['aopo:OrganContext']['dc:title']\n",
    "    if 'biological-event' in kedict[ke]:\n",
    "        if len(kedict[ke]['biological-event']['go:0008150']) > 0:\n",
    "            g.write(' ;\\n\\tgo:0008150\\t' + (','.join(kedict[ke]['biological-event']['go:0008150'])))\n",
    "        if len(kedict[ke]['biological-event']['pato:0000001']) > 0:\n",
    "            g.write(' ;\\n\\tpato:0000001\\t' + (','.join(kedict[ke]['biological-event']['pato:0000001'])))\n",
    "        if len(kedict[ke]['biological-event']['pato:0001241']) > 0:\n",
    "            g.write(' ;\\n\\tpato:0001241\\t' + (','.join(kedict[ke]['biological-event']['pato:0001241'])))\n",
    "    listofthings = []\n",
    "    for aop in aopdict:\n",
    "        if ke in aopdict[aop]['aopo:has_key_event']:\n",
    "            listofthings.append(aopdict[aop]['dc:identifier'])\n",
    "    if not listofthings == []:\n",
    "        g.write(' ;\\n\\tdcterms:isPartOf\\t' + (','.join(listofthings)))\n",
    "    g.write(' .\\n\\n')\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing Key Event Relationship triples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n"
     ]
    }
   ],
   "source": [
    "for ker in kerdict:\n",
    "    g.write(kerdict[ker]['dc:identifier'] + '\\n\\ta\\taopo:KeyEventRelationship ;\\n\\tdc:identifier\\t' + kerdict[ker]['dc:identifier'] + ' ;\\n\\trdfs:label\\t' + kerdict[ker]['rdfs:label'] + ' ;\\n\\tfoaf:page\\t' + kerdict[ker]['foaf:page'] + ' ;\\n\\trdfs:seeAlso\\t' + kerdict[ker]['foaf:page'] + ' ;\\n\\tdcterms:created\\t\"' + kerdict[ker]['dcterms:created'] + '\" ;\\n\\tdcterms:modified\\t\"' + kerdict[ker]['dcterms:modified'] + '\" ;\\n\\taopo:has_upstream_key_event\\t' + kerdict[ker]['aopo:has_upstream_key_event']['dc:identifier'] + ' ;\\n\\taopo:has_downstream_key_event\\t' + kerdict[ker]['aopo:has_downstream_key_event']['dc:identifier'])\n",
    "    if 'dc:description' in kerdict[ker]:\n",
    "        g.write(' ;\\n\\tdc:description\\t' + kerdict[ker]['dc:description'])\n",
    "    if 'nci:C80263' in kerdict[ker]:\n",
    "        g.write(' ;\\n\\tnci:C80263\\t' + kerdict[ker]['nci:C80263'])\n",
    "    if 'edam:data_2042' in kerdict[ker]:\n",
    "        g.write(' ;\\n\\tedam:data_2042\\t' + kerdict[ker]['edam:data_2042'].replace(\"\\\\\", \"\"))\n",
    "    if 'nci:C71478' in kerdict[ker]:\n",
    "        g.write(' ;\\n\\tnci:C71478\\t' + kerdict[ker]['nci:C71478'].replace(\"\\\\\", \"\"))\n",
    "    listofthings = []\n",
    "    if 'pato:0000047' in kerdict[ker]:\n",
    "        for sex in kerdict[ker]['pato:0000047']:\n",
    "            listofthings.append('\"' + sex[1] + '\"')\n",
    "        if not listofthings == []:\n",
    "            g.write(' ;\\n\\tpato:0000047\\t' + (','.join(listofthings)))\n",
    "    listofthings = []\n",
    "    if 'aopo:LifeStageContext' in kerdict[ker]:\n",
    "        for lifestage in kerdict[ker]['aopo:LifeStageContext']:\n",
    "            listofthings.append('\"' + lifestage[1] + '\"')\n",
    "        if not listofthings == []:\n",
    "            g.write(' ;\\n\\taopo:LifeStageContext\\t' + (','.join(listofthings)))\n",
    "    listofthings = []\n",
    "    if 'ncbitaxon:131567' in kerdict[ker]:\n",
    "        for taxonomy in kerdict[ker]['ncbitaxon:131567']:\n",
    "            listofthings.append(taxonomy[2])\n",
    "        if not listofthings == []:\n",
    "            g.write(' ;\\n\\tncbitaxon:131567\\t' + (','.join(listofthings)))\n",
    "    listofthings = []\n",
    "    for aop in aopdict:\n",
    "        if ker in aopdict[aop]['aopo:has_key_event_relationship']:\n",
    "            listofthings.append(aopdict[aop]['dc:identifier'])\n",
    "    if not listofthings == []:\n",
    "        g.write(' ;\\n\\tdcterms:isPartOf\\t' + (','.join(listofthings)))\n",
    "    g.write(' .\\n\\n')\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing Taxonomy triples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n"
     ]
    }
   ],
   "source": [
    "for tax in taxdict:\n",
    "    if 'dc:identifier' in taxdict[tax]:\n",
    "        if '\"' not in taxdict[tax]['dc:identifier']:\n",
    "            g.write(taxdict[tax]['dc:identifier'] + '\\n\\ta\\tncbitaxon:131567 ;\\n\\tdc:identifier\\t' + taxdict[tax]['dc:identifier'] + ' ;\\n\\tdc:title\\t\"' + taxdict[tax]['dc:title'])\n",
    "            if taxdict[tax]['dc:source'] is not None:\n",
    "                g.write('\" ;\\n\\tdc:source\\t\"' + taxdict[tax]['dc:source'])\n",
    "            g.write('\" .\\n\\n')\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing Stressor triples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n"
     ]
    }
   ],
   "source": [
    "for stressor in strdict:\n",
    "    g.write(strdict[stressor]['dc:identifier'] + '\\n\\ta\\tnci:C54571 ;\\n\\tdc:identifier\\t' + strdict[stressor]['dc:identifier'] + ' ;\\n\\trdfs:label\\t' + strdict[stressor]['rdfs:label'] + ' ;\\n\\tfoaf:page\\t' + strdict[stressor]['foaf:page'] + ' ;\\n\\tdc:title\\t' + strdict[stressor]['dc:title'] + ' ;\\n\\tdcterms:created\\t\"' + strdict[stressor]['dcterms:created'] + '\" ;\\n\\tdcterms:modified\\t\"' + strdict[stressor]['dcterms:modified'] + '\"')\n",
    "    if 'dc:description' in strdict[stressor]:\n",
    "        g.write(' ;\\n\\tdc:description\\t' + strdict[stressor]['dc:description'])\n",
    "    listofthings = []\n",
    "    for chem in strdict[stressor]['linktochemical']:\n",
    "        listofthings.append(chedict[chem]['dc:identifier'])\n",
    "    if not listofthings == []:\n",
    "        g.write(' ;\\n\\taopo:has_chemical_entity\\t' + ','.join(listofthings))\n",
    "    listofthings = []\n",
    "    for ke in kedict:\n",
    "        if 'nci:C54571' in kedict[ke]:\n",
    "            if stressor in kedict[ke]['nci:C54571']:\n",
    "                listofthings.append(kedict[ke]['dc:identifier'])\n",
    "    for item in listofthings:\n",
    "        for ke in kedict:\n",
    "            if kedict[ke]['dc:identifier'] == item:\n",
    "                for aop in aopdict:\n",
    "                    if ke in aopdict[aop]['aopo:has_key_event'] and aopdict[aop]['dc:identifier'] not in listofthings:\n",
    "                        listofthings.append(aopdict[aop]['dc:identifier'])\n",
    "    for aop in aopdict:\n",
    "        if stressor in aopdict[aop]['nci:C54571']:\n",
    "                if not aopdict[aop]['dc:identifier'] in listofthings:\n",
    "                    listofthings.append(aopdict[aop]['dc:identifier'])\n",
    "    if not listofthings == []:\n",
    "        g.write(' ;\\n\\tdcterms:isPartOf\\t' + (','.join(listofthings)))\n",
    "    g.write(' .\\n\\n')\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing Biological Process triples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n"
     ]
    }
   ],
   "source": [
    "for pro in bioprodict:\n",
    "    if pro is not None:\n",
    "        g.write(bioprodict[pro]['dc:identifier'] + '\\ta\\tgo:0008150 ;\\n\\tdc:identifier\\t' + bioprodict[pro]['dc:identifier'] + ' ;\\n\\tdc:title\\t' + bioprodict[pro]['dc:title'] + ' ;\\n\\tdc:source\\t' + bioprodict[pro]['dc:source'] + ' . \\n\\n')\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing Biological Object triples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n"
     ]
    }
   ],
   "source": [
    "for obj in bioobjdict:\n",
    "    if obj is not None and not \"N/A\" and 'TAIR' not in bioobjdict[obj]['dc:identifier']:\n",
    "        g.write(bioobjdict[obj]['dc:identifier'] + '\\ta\\tpato:0001241 ;\\n\\tdc:identifier\\t' + bioobjdict[obj]['dc:identifier'] + ' ;\\n\\tdc:title\\t' + bioobjdict[obj]['dc:title'] + ' ;\\n\\tdc:source\\t' + bioobjdict[obj]['dc:source'])\n",
    "        if bioobjdict[obj]['dc:identifier'] in prodict:\n",
    "            g.write(' ;\\n\\tskos:exactMatch\\t'+','.join(prodict[bioobjdict[obj]['dc:identifier']]))\n",
    "        g.write('. \\n\\n')\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing Biological Action triples\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n"
     ]
    }
   ],
   "source": [
    "for act in bioactdict:\n",
    "    if act is not None:\n",
    "        if '\"' not in bioactdict[act]['dc:identifier']:\n",
    "            g.write(bioactdict[act]['dc:identifier'] + '\\ta\\tpato:0000001 ;\\n\\tdc:identifier\\t' + bioactdict[act]['dc:identifier'] + ' ;\\n\\tdc:title\\t' + bioactdict[act]['dc:title'] + ' ;\\n\\tdc:source\\t' + bioactdict[act]['dc:source'] + ' . \\n\\n')\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing Cell term triples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n"
     ]
    }
   ],
   "source": [
    "for item in cterm:\n",
    "    if '\"' not in item:\n",
    "        g.write(item + '\\ta\\taopo:CellTypeContext ;\\n\\tdc:identifier\\t' + item + ' ;\\n\\tdc:title\\t' + cterm[item]['dc:title'] + ' ;\\n\\tdc:source\\t' + cterm[item]['dc:source'] + ' .\\n\\n')\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing Organ term triples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n"
     ]
    }
   ],
   "source": [
    "for item in oterm:\n",
    "    if '\"' not in item:\n",
    "        g.write(item + '\\ta\\taopo:OrganContext ;\\n\\tdc:identifier\\t' + item + ' ;\\n\\tdc:title\\t' + oterm[item]['dc:title'] + ' ;\\n\\tdc:source\\t' + oterm[item]['dc:source'] + ' .\\n\\n')\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing Chemical triples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n"
     ]
    }
   ],
   "source": [
    "for che in chedict:\n",
    "    if 'dc:identifier' in chedict[che] and '\"' not in chedict[che]['dc:identifier']:\n",
    "        g.write(chedict[che]['dc:identifier'] + '\\n\\tdc:identifier\\t' + chedict[che]['dc:identifier'])\n",
    "        if 'cheminf:000446' in chedict[che]:\n",
    "            g.write(' ;\\n\\ta\\tcheminf:000000, cheminf:000446 ;\\n\\tcheminf:000446\\t' + chedict[che]['cheminf:000446'])\n",
    "        if not chedict[che]['cheminf:000059'] == 'inchikey:None':\n",
    "            g.write(' ;\\n\\tcheminf:000059\\t' + chedict[che]['cheminf:000059'])\n",
    "        if 'dc:title' in chedict[che]:\n",
    "            g.write(' ;\\n\\tdc:title\\t' + chedict[che]['dc:title'])\n",
    "        if 'cheminf:000568' in chedict[che]:\n",
    "            g.write(' ;\\n\\tcheminf:000568\\t' + str(chedict[che]['cheminf:000568']))\n",
    "        listofexactmatches = []\n",
    "        if 'cheminf:000407' in chedict[che]:\n",
    "            listofexactmatches.append(','.join(chedict[che]['cheminf:000407']))\n",
    "        if 'cheminf:000405' in chedict[che]:\n",
    "            listofexactmatches.append(','.join(chedict[che]['cheminf:000405']))\n",
    "        if 'cheminf:000567' in chedict[che]:\n",
    "            listofexactmatches.append(','.join(chedict[che]['cheminf:000567']))\n",
    "        if 'cheminf:000412' in chedict[che]:\n",
    "            listofexactmatches.append(','.join(chedict[che]['cheminf:000412']))\n",
    "        if 'cheminf:000140' in chedict[che]:\n",
    "            listofexactmatches.append(','.join(chedict[che]['cheminf:000140']))\n",
    "        if 'cheminf:000406' in chedict[che]:\n",
    "            listofexactmatches.append(','.join(chedict[che]['cheminf:000406']))\n",
    "        if 'cheminf:000408' in chedict[che]:\n",
    "            listofexactmatches.append(','.join(chedict[che]['cheminf:000408']))\n",
    "        if 'cheminf:000409' in chedict[che]:\n",
    "            listofexactmatches.append(','.join(chedict[che]['cheminf:000409']))\n",
    "        if 'cheminf:000564' in chedict[che]:\n",
    "            listofexactmatches.append(','.join(chedict[che]['cheminf:000564']))\n",
    "        if 'cheminf:000407' in chedict[che] or 'cheminf:000405' in chedict[che] or 'cheminf:000567' in chedict[che] or 'cheminf:000412' in chedict[che] or 'cheminf:000140' in chedict[che] or 'cheminf:000406' in chedict[che] or 'cheminf:000408' in chedict[che] or 'cheminf:000409' in chedict[che] or 'cheminf:000564' in chedict[che]:\n",
    "            g.write(' ;\\n\\tskos:exactMatch\\t'+','.join(listofexactmatches))\n",
    "        listofthings = []\n",
    "        if 'dcterms:alternative' in chedict[che]:\n",
    "            for alt in chedict[che]['dcterms:alternative']:\n",
    "                listofthings.append('\"' + alt + '\"')\n",
    "            g.write(' ;\\n\\tdcterms:alternative\\t' + ','.join(listofthings))\n",
    "        listofthings = []\n",
    "        for stressor in strdict:\n",
    "            if 'aopo:has_chemical_entity' in strdict[stressor]:\n",
    "                if che in strdict[stressor]['linktochemical']:\n",
    "                    listofthings.append(strdict[stressor]['dc:identifier'])\n",
    "        if not listofthings == []:\n",
    "            g.write(' ;\\n\\tdcterms:isPartOf\\t' + (','.join(listofthings)))\n",
    "        g.write(' .\\n\\n')\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "n = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing mapped Chemical identifiers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "391\n",
      "788\n",
      "1185\n",
      "1984\n",
      "2390\n",
      "2792\n",
      "3125\n",
      "3528\n",
      "3738\n",
      "4040\n",
      "4074\n",
      "4482\n",
      "Done!\n"
     ]
    }
   ],
   "source": [
    "for cas in listofcas:\n",
    "    g.write(cas + '\\tdc:source\\t\"CAS\".\\n\\n')\n",
    "    n += 1\n",
    "print(n)\n",
    "for inchikey in listofinchikey:\n",
    "    g.write(inchikey + '\\tdc:source\\t\"InChIKey\".\\n\\n')\n",
    "    n += 1\n",
    "print(n)\n",
    "    \n",
    "for comptox in listofcomptox:\n",
    "    g.write(comptox + '\\tdc:source\\t\"CompTox\".\\n\\n')\n",
    "    n += 1\n",
    "print(n)\n",
    "\n",
    "for chebi in listofchebi:\n",
    "    g.write(chebi + '\\ta\\tcheminf:000407 ;\\n\\tcheminf:000407\\t\"'+chebi[6:]+'\";\\n\\tdc:identifier\\t\"'+chebi+'\";\\n\\tdc:source\\t\"ChEBI\".\\n\\n')\n",
    "    n += 1\n",
    "print(n)\n",
    "for chemspider in listofchemspider:\n",
    "    g.write(chemspider + '\\ta\\tcheminf:000405 ;\\n\\tcheminf:000405\\t\"'+chemspider[11:]+'\";\\n\\tdc:identifier\\t\"'+chemspider+'\";\\n\\tdc:source\\t\"ChemSpider\".\\n\\n')\n",
    "    n += 1\n",
    "print(n)\n",
    "for wd in listofwikidata:\n",
    "    g.write(wd + '\\ta\\tcheminf:000567 ;\\n\\tcheminf:000567\\t\"'+wd[9:]+'\";\\n\\tdc:identifier\\t\"'+wd+'\";\\n\\tdc:source\\t\"Wikidata\".\\n\\n')\n",
    "    n += 1\n",
    "print(n)\n",
    "for chembl in listofchembl:\n",
    "    g.write(chembl + '\\ta\\tcheminf:000412 ;\\n\\tcheminf:000412\\t\"'+chembl[16:]+'\";\\n\\tdc:identifier\\t\"'+chembl+'\";\\n\\tdc:source\\t\"ChEMBL\".\\n\\n')\n",
    "    n += 1\n",
    "print(n)\n",
    "for pubchem in listofpubchem:\n",
    "    g.write(pubchem + '\\ta\\tcheminf:000140 ;\\n\\tcheminf:000140\\t\"'+pubchem[17:]+'\";\\n\\tdc:identifier\\t\"'+pubchem+'\";\\n\\tdc:source\\t\"PubChem\".\\n\\n')\n",
    "    n += 1\n",
    "print(n)\n",
    "for drugbank in listofdrugbank:\n",
    "    g.write(drugbank + '\\ta\\tcheminf:000406 ;\\n\\tcheminf:000406\\t\"'+drugbank[9:]+'\";\\n\\tdc:identifier\\t\"'+drugbank+'\";\\n\\tdc:source\\t\"DrugBank\".\\n\\n')\n",
    "    n += 1\n",
    "print(n)\n",
    "for kegg in listofkegg:\n",
    "    g.write(kegg + '\\ta\\tcheminf:000409 ;\\n\\tcheminf:000409\\t\"'+kegg[14:]+'\";\\n\\tdc:identifier\\t\"'+kegg+'\";\\n\\tdc:source\\t\"KEGG\".\\n\\n')\n",
    "    n += 1\n",
    "print(n)\n",
    "for lipidmaps in listoflipidmaps:\n",
    "    g.write(lipidmaps + '\\ta\\tcheminf:000564 ;\\n\\tcheminf:000564\\t\"'+lipidmaps[10:]+'\";\\n\\tdc:identifier\\t\"'+lipidmaps+'\";\\n\\tdc:source\\t\"LIPID MAPS\".\\n\\n')\n",
    "    n += 1\n",
    "print(n)\n",
    "for hmdb in listofhmdb:\n",
    "    g.write(hmdb + '\\ta\\tcheminf:000408 ;\\n\\tcheminf:000408\\t\"'+hmdb[5:]+'\";\\n\\tdc:identifier\\t\"'+hmdb+'\";\\n\\tdc:source\\t\"HMDB\".\\n\\n')\n",
    "    n += 1\n",
    "print(n)\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing mapped Gene identifiers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n"
     ]
    }
   ],
   "source": [
    "for hgnc in hgnclist:\n",
    "    g.write(hgnc + '\\ta\\tedam:data_2298, edam:data_1025 ;\\n\\tedam:data_2298\\t\"'+hgnc[5:]+'\";\\n\\tdc:identifier\\t\"'+hgnc+'\";\\n\\tdc:source\\t\"HGNC\".\\n\\n')\n",
    "\n",
    "for entrez in ncbigenelist:\n",
    "    g.write(entrez + '\\ta\\tedam:data_1027, edam:data_1025 ;\\n\\tedam:data_1027\\t\"'+entrez[9:]+'\";\\n\\tdc:identifier\\t\"'+entrez+'\";\\n\\tdc:source\\t\"Entrez Gene\".\\n\\n')\n",
    "\n",
    "for uniprot in uniprotlist:\n",
    "    g.write(uniprot + '\\ta\\tedam:data_2291, edam:data_1025 ;\\n\\trdfs:seeAlso <http://purl.uniprot.org/uniprot/' + uniprot[8:] + '>;\\n\\towl:sameAs <http://purl.uniprot.org/uniprot/' + uniprot[8:] + '>;\\n\\tedam:data_2291\\t\"'+uniprot[8:]+'\";\\n\\tdc:identifier\\t\"'+uniprot+'\";\\n\\tdc:source\\t\"UniProt\".\\n\\n')\n",
    "    \n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing class labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>URI</th>\n",
       "      <th>label</th>\n",
       "      <th>description</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>&lt;http://rdfs.org/ns/void#Dataset&gt;</td>\n",
       "      <td>Dataset</td>\n",
       "      <td>-</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>&lt;http://edamontology.org/data_1027&gt;</td>\n",
       "      <td>Gene ID (NCBI)</td>\n",
       "      <td>An NCBI unique identifier of a gene.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>&lt;http://edamontology.org/data_1025&gt;</td>\n",
       "      <td>Gene identifier</td>\n",
       "      <td>An identifier of a gene, such as a name/symbol...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>&lt;http://edamontology.org/data_2298&gt;</td>\n",
       "      <td>Gene ID (HGNC)</td>\n",
       "      <td>Identifier for a gene approved by the HUGO Gen...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>&lt;http://purl.org/obo/owl/GO#0008150&gt;</td>\n",
       "      <td>biological_process</td>\n",
       "      <td>A biological process represents a specific obj...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>&lt;http://aopkb.org/aop_ontology#CellTypeContext&gt;</td>\n",
       "      <td>Cell-term</td>\n",
       "      <td>-</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>&lt;http://aopkb.org/aop_ontology#OrganContext&gt;</td>\n",
       "      <td>Organ-term</td>\n",
       "      <td>-</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>&lt;http://semanticscience.org/resource/CHEMINF_0...</td>\n",
       "      <td>ChEBI identifier</td>\n",
       "      <td>Database identifier used by ChEBI.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>&lt;http://semanticscience.org/resource/CHEMINF_0...</td>\n",
       "      <td>ChEMBL identifier</td>\n",
       "      <td>Identifier used by the ChEMBL database for com...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>&lt;http://semanticscience.org/resource/CHEMINF_0...</td>\n",
       "      <td>ChemSpider identifier</td>\n",
       "      <td>Database identifier used by ChemSpider.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>&lt;http://semanticscience.org/resource/CHEMINF_0...</td>\n",
       "      <td>DrugBank identifier</td>\n",
       "      <td>Database identifier used by DrugBank.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>&lt;http://semanticscience.org/resource/CHEMINF_0...</td>\n",
       "      <td>HMDB identifier</td>\n",
       "      <td>Database identifier used by Human Metabolome D...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>&lt;http://semanticscience.org/resource/CHEMINF_0...</td>\n",
       "      <td>KEGG identifier</td>\n",
       "      <td>Database identifier used by KEGG.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>&lt;http://semanticscience.org/resource/CHEMINF_0...</td>\n",
       "      <td>LipidMaps identifier</td>\n",
       "      <td>Identifier used by the LipidMaps database, htt...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>&lt;http://semanticscience.org/resource/CHEMINF_0...</td>\n",
       "      <td>PubChem compound identifier (CID)</td>\n",
       "      <td>-</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>&lt;http://semanticscience.org/resource/CHEMINF_0...</td>\n",
       "      <td>Wikidata identifier</td>\n",
       "      <td>Database identifier used by Wikidata.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>&lt;http://rdfs.org/ns/void#Linkset&gt;</td>\n",
       "      <td>Linkset</td>\n",
       "      <td>-</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>&lt;http://aopkb.org/aop_ontology#KeyEvent&gt;</td>\n",
       "      <td>Key Event</td>\n",
       "      <td>-</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>&lt;http://aopkb.org/aop_ontology#KeyEventRelatio...</td>\n",
       "      <td>Key Event Relationship</td>\n",
       "      <td>-</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>&lt;http://edamontology.org/data_1033&gt;</td>\n",
       "      <td>Ensembl gene ID</td>\n",
       "      <td>Unique identifier for a gene (or other feature...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>&lt;http://ncicb.nci.nih.gov/xml/owl/EVS/Thesauru...</td>\n",
       "      <td>Stressor</td>\n",
       "      <td>An agent, stimulus, activity, or event that ca...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>&lt;http://aopkb.org/aop_ontology#AdverseOutcomeP...</td>\n",
       "      <td>Adverse Outcome Pathway</td>\n",
       "      <td>-</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>&lt;http://semanticscience.org/resource/CHEMINF_0...</td>\n",
       "      <td>chemical entity</td>\n",
       "      <td>A chemical entity is any molecular entity or c...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>&lt;http://semanticscience.org/resource/CHEMINF_0...</td>\n",
       "      <td>CAS registry number</td>\n",
       "      <td>Identifier used by the Chemical Abstracts Serv...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>&lt;http://edamontology.org/data_2291&gt;</td>\n",
       "      <td>UniProt ID</td>\n",
       "      <td>An identifier of a polypeptide in the UniProt ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>&lt;http://purl.bioontology.org/ontology/NCBITAXO...</td>\n",
       "      <td>cellular organisms</td>\n",
       "      <td>-</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>&lt;http://purl.obolibrary.org/obo/PATO_0001241&gt;</td>\n",
       "      <td>physical object quality</td>\n",
       "      <td>A quality which inheres in a continuant</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                  URI  \\\n",
       "0                   <http://rdfs.org/ns/void#Dataset>   \n",
       "1                 <http://edamontology.org/data_1027>   \n",
       "2                 <http://edamontology.org/data_1025>   \n",
       "3                 <http://edamontology.org/data_2298>   \n",
       "4                <http://purl.org/obo/owl/GO#0008150>   \n",
       "5     <http://aopkb.org/aop_ontology#CellTypeContext>   \n",
       "6        <http://aopkb.org/aop_ontology#OrganContext>   \n",
       "7   <http://semanticscience.org/resource/CHEMINF_0...   \n",
       "8   <http://semanticscience.org/resource/CHEMINF_0...   \n",
       "9   <http://semanticscience.org/resource/CHEMINF_0...   \n",
       "10  <http://semanticscience.org/resource/CHEMINF_0...   \n",
       "11  <http://semanticscience.org/resource/CHEMINF_0...   \n",
       "12  <http://semanticscience.org/resource/CHEMINF_0...   \n",
       "13  <http://semanticscience.org/resource/CHEMINF_0...   \n",
       "14  <http://semanticscience.org/resource/CHEMINF_0...   \n",
       "15  <http://semanticscience.org/resource/CHEMINF_0...   \n",
       "16                  <http://rdfs.org/ns/void#Linkset>   \n",
       "17           <http://aopkb.org/aop_ontology#KeyEvent>   \n",
       "18  <http://aopkb.org/aop_ontology#KeyEventRelatio...   \n",
       "19                <http://edamontology.org/data_1033>   \n",
       "20  <http://ncicb.nci.nih.gov/xml/owl/EVS/Thesauru...   \n",
       "21  <http://aopkb.org/aop_ontology#AdverseOutcomeP...   \n",
       "22  <http://semanticscience.org/resource/CHEMINF_0...   \n",
       "23  <http://semanticscience.org/resource/CHEMINF_0...   \n",
       "24                <http://edamontology.org/data_2291>   \n",
       "25  <http://purl.bioontology.org/ontology/NCBITAXO...   \n",
       "26      <http://purl.obolibrary.org/obo/PATO_0001241>   \n",
       "\n",
       "                                label  \\\n",
       "0                             Dataset   \n",
       "1                      Gene ID (NCBI)   \n",
       "2                     Gene identifier   \n",
       "3                      Gene ID (HGNC)   \n",
       "4                  biological_process   \n",
       "5                           Cell-term   \n",
       "6                          Organ-term   \n",
       "7                    ChEBI identifier   \n",
       "8                   ChEMBL identifier   \n",
       "9               ChemSpider identifier   \n",
       "10                DrugBank identifier   \n",
       "11                    HMDB identifier   \n",
       "12                    KEGG identifier   \n",
       "13               LipidMaps identifier   \n",
       "14  PubChem compound identifier (CID)   \n",
       "15                Wikidata identifier   \n",
       "16                            Linkset   \n",
       "17                          Key Event   \n",
       "18             Key Event Relationship   \n",
       "19                    Ensembl gene ID   \n",
       "20                           Stressor   \n",
       "21            Adverse Outcome Pathway   \n",
       "22                    chemical entity   \n",
       "23                CAS registry number   \n",
       "24                         UniProt ID   \n",
       "25                 cellular organisms   \n",
       "26            physical object quality   \n",
       "\n",
       "                                          description  \n",
       "0                                                   -  \n",
       "1                An NCBI unique identifier of a gene.  \n",
       "2   An identifier of a gene, such as a name/symbol...  \n",
       "3   Identifier for a gene approved by the HUGO Gen...  \n",
       "4   A biological process represents a specific obj...  \n",
       "5                                                   -  \n",
       "6                                                   -  \n",
       "7                  Database identifier used by ChEBI.  \n",
       "8   Identifier used by the ChEMBL database for com...  \n",
       "9             Database identifier used by ChemSpider.  \n",
       "10              Database identifier used by DrugBank.  \n",
       "11  Database identifier used by Human Metabolome D...  \n",
       "12                  Database identifier used by KEGG.  \n",
       "13  Identifier used by the LipidMaps database, htt...  \n",
       "14                                                  -  \n",
       "15              Database identifier used by Wikidata.  \n",
       "16                                                  -  \n",
       "17                                                  -  \n",
       "18                                                  -  \n",
       "19  Unique identifier for a gene (or other feature...  \n",
       "20  An agent, stimulus, activity, or event that ca...  \n",
       "21                                                  -  \n",
       "22  A chemical entity is any molecular entity or c...  \n",
       "23  Identifier used by the Chemical Abstracts Serv...  \n",
       "24  An identifier of a polypeptide in the UniProt ...  \n",
       "25                                                  -  \n",
       "26            A quality which inheres in a continuant  "
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "df = pd.read_csv(filepath + 'typelabels.txt')\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "for row,index in df.iterrows():\n",
    "    g.write('\\n\\n'+index['URI']+'\\trdfs:label\\t\"'+index['label'])\n",
    "    if index['description'] != '-':\n",
    "        g.write('\";\\n\\tdc:description\\t\"\"\"'+index['description']+'\"\"\".')\n",
    "    else:\n",
    "        g.write('\".')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Close the file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The AOP-Wiki RDF file is created!\n"
     ]
    }
   ],
   "source": [
    "g.close()\n",
    "print(\"The AOP-Wiki RDF file is created!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b>Step #5: Gene ID text-mapping (HGNC)</b>\n",
    "In order to identify genes present in the textual descriptions of Key Events (KEs) and Key Event Relationships (KERs), HGNC identifier mapping was performed. [Genenames.org](https://www.genenames.org/) is the curated online repository for HGNC nomenclature, and it allows custom downloads for all HGNC entries, including approved symbols and names, previous symbols and synonyms. \n",
    "\n",
    "## Step #5A - Parsing the custom HGNC file\n",
    "This starts with loading the custom download file, which was named `HGNCgenes.txt` and stored in the path defined in Step #2. Next, its contents are extracted and stored in a dictionary called `genedict1`, while variants are created for every gene name and gene symbol for more effective mapping of genes. These variants are stored in `genedict2`, which is used for more effective mapping of genes in Step #5B. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "HGNCfilename = 'HGNCgenes.txt'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Last Modified Time :  Tue Aug 20 08:51:09 2024\n"
     ]
    }
   ],
   "source": [
    "fileStatsObj = os.stat (filepath + HGNCfilename)\n",
    "HGNCmodificationTime = time.ctime ( fileStatsObj [ stat.ST_MTIME ] )\n",
    "print(\"Last Modified Time : \", HGNCmodificationTime )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A total of 41893 genes are included for mappings\n"
     ]
    }
   ],
   "source": [
    "HGNCgenes = open(filepath + HGNCfilename, 'r')\n",
    "symbols = [' ','(',')','[',']',',','.']\n",
    "genedict1 = {}\n",
    "genedict2 = {}\n",
    "b = 0\n",
    "for line in HGNCgenes:\n",
    "    if not 'HGNC ID\tApproved symbol\tApproved name\tPrevious symbols\tSynonyms\tAccession numbers\tEnsembl ID(supplied by Ensembl)'in line:\n",
    "        a = line[:-1].split('\\t')\n",
    "        if not '@' in a[1]: #gene clusters contain a '@' in their symbol, which are filtered out\n",
    "            genedict1[a[1]] = []\n",
    "            genedict2[a[1]] = []\n",
    "            genedict1[a[1]].append(a[1])\n",
    "            if not a[2] == '':\n",
    "                genedict1[a[1]].append(a[2])\n",
    "            for item in a[3:]:\n",
    "                if not item == '':\n",
    "                    for name in item.split(', '):\n",
    "                        genedict1[a[1]].append(name)\n",
    "            for item in genedict1[a[1]]:\n",
    "                for s1 in symbols:\n",
    "                    for s2 in symbols:\n",
    "                        genedict2[a[1]].append((s1+item+s2))\n",
    "HGNCgenes.close()\n",
    "print(\"A total of \" + str(len(genedict2)) + \" genes are included for mappings\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step #5B - HGNC identifier mapping\n",
    "Genes are mapped for descriptions of KEs and KERs, and for the biological plausibility and emperical support sections of KERs. First, these are screened for any overlap with all possible gene symbols and names captured in genedict1. Then, all positive matches are checked by mapping with all variants of those genes, ensuring the correct mapping. All matches are stored in the kedict and kerdict dictionaries. Also, all mapped genes are stored in a list called hgnclist."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Key Events"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Gene mapping on Key Events is can take a minute...\n",
      "Done!\n",
      "In total, 894 genes were mapped to descriptions of Key Events\n"
     ]
    }
   ],
   "source": [
    "hgnclist = []\n",
    "print(\"Gene mapping on Key Events is can take a minute...\")\n",
    "for ke in root.findall(aopxml + 'key-event'):\n",
    "    geneoverlapdict = {}\n",
    "    if ke.find(aopxml + 'description').text is not None:\n",
    "        geneoverlapdict[ke.get('id')] = []\n",
    "        for key in genedict2:\n",
    "            a = 0\n",
    "            for item in genedict1[key]:\n",
    "                if item in kedict[ke.get('id')]['dc:description']:\n",
    "                    a = 1\n",
    "            if a == 1:\n",
    "                for item in genedict2[key]:\n",
    "                    if item in kedict[ke.get('id')]['dc:description'] and not 'hgnc:' + genedict2[key][1][1:-1] in geneoverlapdict[ke.get('id')]:\n",
    "                        geneoverlapdict[ke.get('id')].append('hgnc:' + genedict2[key][1][1:-1])\n",
    "                        if 'hgnc:' + genedict2[key][1][1:-1] not in hgnclist:\n",
    "                            hgnclist.append('hgnc:' + genedict2[key][1][1:-1])\n",
    "        if not geneoverlapdict[ke.get('id')]:\n",
    "            del geneoverlapdict[ke.get('id')]\n",
    "    if ke.get('id') in geneoverlapdict:\n",
    "        kedict[ke.get('id')]['edam:data_1025'] = geneoverlapdict[ke.get('id')]\n",
    "print(\"Done!\\nIn total, \" + str(len(hgnclist))+ \" genes were mapped to descriptions of Key Events\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Key Event Relationships"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Gene mapping on Key Events is can take a couple of minutes...\n",
      "Done!\n",
      "In total, 1483 genes were mapped to descriptions of Key Events and Key Event Relationships\n"
     ]
    }
   ],
   "source": [
    "print(\"Gene mapping on Key Events is can take a couple of minutes...\")\n",
    "for ker in root.findall(aopxml + 'key-event-relationship'):\n",
    "    geneoverlapdict = {}\n",
    "    geneoverlapdict[ker.get('id')] = []\n",
    "    if ker.find(aopxml + 'description').text is not None:\n",
    "        for key in genedict2:\n",
    "            a = 0\n",
    "            for item in genedict1[key]:\n",
    "                if item in kerdict[ker.get('id')]['dc:description']:\n",
    "                    a = 1\n",
    "            if a == 1:\n",
    "                for item in genedict2[key]:\n",
    "                    if item in kerdict[ker.get('id')]['dc:description'] and not 'hgnc:' + genedict2[key][1][1:-1] in geneoverlapdict[ker.get('id')]:\n",
    "                        geneoverlapdict[ker.get('id')].append('hgnc:' + genedict2[key][1][1:-1])\n",
    "                        if 'hgnc:' + genedict2[key][1][1:-1] not in hgnclist:\n",
    "                            hgnclist.append('hgnc:' + genedict2[key][1][1:-1])\n",
    "    for weight in ker.findall(aopxml + 'weight-of-evidence'):\n",
    "        if weight.find(aopxml + 'biological-plausibility').text is not None:\n",
    "            for key in genedict2:\n",
    "                a = 0\n",
    "                for item in genedict1[key]:\n",
    "                    if item in kerdict[ker.get('id')]['nci:C80263']:\n",
    "                        a = 1\n",
    "                if a== 1:\n",
    "                    for item in genedict2[key]:\n",
    "                        if item in kerdict[ker.get('id')]['nci:C80263'] and not 'hgnc:' + genedict2[key][1][1:-1] in geneoverlapdict[ker.get('id')]:\n",
    "                            geneoverlapdict[ker.get('id')].append('hgnc:' + genedict2[key][1][1:-1])\n",
    "                            if 'hgnc:' + genedict2[key][1][1:-1] not in hgnclist:\n",
    "                                hgnclist.append('hgnc:' + genedict2[key][1][1:-1])\n",
    "        if weight.find(aopxml + 'emperical-support-linkage').text is not None:\n",
    "            for key in genedict2:\n",
    "                a = 0\n",
    "                for item in genedict1[key]:\n",
    "                    if item in kerdict[ker.get('id')]['edam:data_2042']:\n",
    "                        a = 1\n",
    "                if a== 1:\n",
    "                    for item in genedict2[key]:\n",
    "                        if item in kerdict[ker.get('id')]['edam:data_2042'] and not 'hgnc:' + genedict2[key][1][1:-1] in geneoverlapdict[ker.get('id')]:\n",
    "                            geneoverlapdict[ker.get('id')].append('hgnc:' + genedict2[key][1][1:-1])\n",
    "                            if 'hgnc:' + genedict2[key][1][1:-1] not in hgnclist:\n",
    "                                hgnclist.append('hgnc:' + genedict2[key][1][1:-1])\n",
    "    if not geneoverlapdict[ker.get('id')]:\n",
    "        del geneoverlapdict[ker.get('id')]\n",
    "    if ker.get('id') in geneoverlapdict:\n",
    "        kerdict[ker.get('id')]['edam:data_1025'] = geneoverlapdict[ker.get('id')]\n",
    "print(\"Done!\\nIn total, \" + str(len(hgnclist))+ \" genes were mapped to descriptions of Key Events and Key Event Relationships\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step #5C - Identifier mapping for other databases\n",
    "BridgeDb was used to additional identifiers from other databases, including Entrez gene, Ensembl, and UniProt IDs. By a request call, identifiers are returned, which are stored in the dictionary called `geneiddict`. The BridgeDb service URL has already been defined in Step #3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Gene identifiers mapped:\n",
      "1458 Entrez gene IDs\n",
      "7317 Uniprot IDs\n",
      "1456 Ensembl IDs\n"
     ]
    }
   ],
   "source": [
    "geneiddict = {}\n",
    "listofentrez = []\n",
    "listofensembl = []\n",
    "listofuniprot = []\n",
    "\n",
    "for gene in hgnclist:\n",
    "    a = requests.get(bridgedb + 'xrefs/H/' + gene[5:]).text.split('\\n')\n",
    "    dictionaryforgene = {}\n",
    "    if 'html' not in a:\n",
    "        for item in a:\n",
    "            b = item.split('\\t')\n",
    "            if len(b) == 2:\n",
    "                if b[1] not in dictionaryforgene:\n",
    "                    dictionaryforgene[b[1]] = []\n",
    "                    dictionaryforgene[b[1]].append(b[0])\n",
    "                else:\n",
    "                    dictionaryforgene[b[1]].append(b[0])\n",
    "    geneiddict[gene] = []\n",
    "    if 'Entrez Gene' in dictionaryforgene:\n",
    "        for entrez in dictionaryforgene['Entrez Gene']:\n",
    "            if 'ncbigene:'+entrez not in listofentrez:\n",
    "                listofentrez.append(\"ncbigene:\"+entrez)\n",
    "            geneiddict[gene].append(\"ncbigene:\"+entrez)\n",
    "    if 'Ensembl' in dictionaryforgene:\n",
    "        for ensembl in dictionaryforgene['Ensembl']:\n",
    "            if 'ensembl:' + ensembl not in listofensembl:\n",
    "                listofensembl.append(\"ensembl:\"+ensembl)\n",
    "            geneiddict[gene].append(\"ensembl:\"+ensembl)\n",
    "    if 'Uniprot-TrEMBL' in dictionaryforgene:\n",
    "        for uniprot in dictionaryforgene['Uniprot-TrEMBL']:\n",
    "            if 'uniprot:'+uniprot not in listofuniprot:\n",
    "                listofuniprot.append(\"uniprot:\"+uniprot)\n",
    "            geneiddict[gene].append(\"uniprot:\"+uniprot)\n",
    "print(\"Gene identifiers mapped:\\n\" + str(len(listofentrez)) + \" Entrez gene IDs\\n\" + str(len(listofuniprot)) + \" Uniprot IDs\\n\" + str(len(listofensembl)) + \" Ensembl IDs\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step #5D - Writing output file\n",
    "The final step involves the writing of the RDF file in Turtle syntax. After writing the prefixes used for predicates and identifier types, all gene mapping links stored in the kedict and kerdict are written, followed by the HGNC IDs and matched IDs for other databases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "g = open(filepath + 'AOPWikiRDF-Genes.ttl', 'w', encoding='utf-8')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing prefixes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "595"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "g.write('@prefix dc: <http://purl.org/dc/elements/1.1/> .\\n@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .\\n@prefix aop.events: <https://identifiers.org/aop.events/> .\\n@prefix aop.relationships: <https://identifiers.org/aop.relationships/> .\\n@prefix skos: <http://www.w3.org/2004/02/skos/core#> . \\n@prefix ensembl: <https://identifiers.org/ensembl/> .\\n@prefix edam: <http://edamontology.org/> .\\n@prefix hgnc: <https://identifiers.org/hgnc/>.\\n@prefix ncbigene: <https://identifiers.org/ncbigene/>.\\n@prefix uniprot: <https://identifiers.org/uniprot/>.\\n@prefix owl: <http://www.w3.org/2002/07/owl#>.\\n\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing Key Event triples\n",
    "These triples only contain the mappings with genes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n",
      "\n",
      "Number of Key Events with genes mapped to their descriptions: 348\n"
     ]
    }
   ],
   "source": [
    "n = 0\n",
    "for ke in kedict:\n",
    "    if 'edam:data_1025' in kedict[ke]:\n",
    "        n += 1\n",
    "        g.write(kedict[ke]['dc:identifier']+'\\tedam:data_1025\\t' + ','.join(kedict[ke]['edam:data_1025'])+' .\\n\\n')\n",
    "print(\"Done!\\n\\nNumber of Key Events with genes mapped to their descriptions: \" + str(n))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing Key Event Relationship triples\n",
    "These triples only contain the mappings with genes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done!\n",
      "\n",
      "Number of Key Event Relationships with genes mapped to their descriptions: 549\n"
     ]
    }
   ],
   "source": [
    "n = 0\n",
    "for ker in kerdict:\n",
    "    if 'edam:data_1025' in kerdict[ker]:\n",
    "        n += 1\n",
    "        g.write(kerdict[ker]['dc:identifier']+'\\tedam:data_1025\\t' + ','.join(kerdict[ker]['edam:data_1025'])+' .\\n\\n')\n",
    "print(\"Done!\\n\\nNumber of Key Event Relationships with genes mapped to their descriptions: \" + str(n))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writing Gene identifier triples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1483 HGNC triples written.\n",
      "1458 Entrez gene triples written.\n",
      "1456 Ensembl triples written.\n",
      "7317 UniProt triples written.\n",
      "Done!\n"
     ]
    }
   ],
   "source": [
    "for hgnc in hgnclist:\n",
    "    g.write(hgnc + '\\ta\\tedam:data_2298, edam:data_1025 ;\\n\\tedam:data_2298\\t\"'+hgnc[5:]+'\";\\n\\tdc:identifier\\t\"'+hgnc+'\";\\n\\tdc:source\\t\"HGNC\"')\n",
    "    if not geneiddict[hgnc] == []:\n",
    "        g.write(' ;\\n\\tskos:exactMatch\\t'+','.join(geneiddict[hgnc]))\n",
    "    g.write('.\\n\\n')\n",
    "print(str(len(hgnclist))+\" HGNC triples written.\")\n",
    "for entrez in listofentrez:\n",
    "    g.write(entrez + '\\ta\\tedam:data_1027, edam:data_1025 ;\\n\\tedam:data_1027\\t\"'+entrez[9:]+'\";\\n\\tdc:identifier\\t\"'+entrez+'\";\\n\\tdc:source\\t\"Entrez Gene\".\\n\\n')\n",
    "print(str(len(listofentrez))+\" Entrez gene triples written.\")\n",
    "for ensembl in listofensembl:\n",
    "    g.write(ensembl + '\\ta\\tedam:data_1033, edam:data_1025 ;\\n\\tedam:data_1033\\t\"'+ensembl[8:]+'\";\\n\\tdc:identifier\\t\"'+ensembl+'\";\\n\\tdc:source\\t\"Ensembl\".\\n\\n')\n",
    "print(str(len(listofensembl))+ \" Ensembl triples written.\")\n",
    "for uniprot in listofuniprot:\n",
    "    g.write(uniprot + '\\ta\\tedam:data_2291, edam:data_1025 ;\\n\\tedam:data_2291\\t\"'+uniprot[8:]+'\";\\n\\tdc:identifier\\t\"'+uniprot+'\";\\n\\tdc:source\\t\"UniProt\".\\n\\n')\n",
    "print(str(len(listofuniprot))+ \" UniProt triples written.\\nDone!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Close the file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The AOP-Wiki RDF Genes file is created!\n"
     ]
    }
   ],
   "source": [
    "g.close()\n",
    "print(\"The AOP-Wiki RDF Genes file is created!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <b>Step #6: Creating the VoID file</b>\n",
    "The last file contains the metadata of the original data, script, and tools used for the creation of the AOP-Wiki RDF files. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The version of the BridgeDb mapping files: \n",
      " Gene/Proteins: Ensembl:108\n",
      " Chemicals: HMDB-CHEBI-WIKIDATA:HMDB5.0.20211102-CHEBI211-WIKIDATA20220707\n"
     ]
    }
   ],
   "source": [
    "a = requests.get(bridgedb + 'properties').text.split('\\n')\n",
    "info = {}\n",
    "for item in a:\n",
    "    if not item.split('\\t')[0] in info:\n",
    "        info[item.split('\\t')[0]] = []\n",
    "    if len(item.split('\\t')) == 2:\n",
    "        info[item.split('\\t')[0]].append(item.split('\\t')[1])\n",
    "print('The version of the BridgeDb mapping files: \\n Gene/Proteins: '\n",
    "      + str(info['DATASOURCENAME'][0]) + ':' + str(info['DATASOURCEVERSION'][0]) + '\\n Chemicals: '\n",
    "      + str(info['DATASOURCENAME'][5]) + ':' + str(info['DATASOURCEVERSION'][5]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The date: 2024-08-20 21:23:13.171514\n"
     ]
    }
   ],
   "source": [
    "x = datetime.datetime.now()\n",
    "print('The date: ' + str(x))\n",
    "y = str(x)\n",
    "y = y[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The VoID file is created!\n"
     ]
    }
   ],
   "source": [
    "g = open(filepath + 'AOPWikiRDF-Void.ttl', 'w', encoding='utf-8')\n",
    "g.write('@prefix : <https://aopwiki.rdf.bigcat-bioinformatics.org/> .\\n@prefix dc: <http://purl.org/dc/elements/1.1/> .\\n@prefix dcterms: <http://purl.org/dc/terms/> .\\n@prefix void:  <http://rdfs.org/ns/void#> .\\n@prefix pav:   <http://purl.org/pav/> .\\n@prefix rdfs:  <http://www.w3.org/2000/01/rdf-schema#> .\\n@prefix dcat:  <http://www.w3.org/ns/dcat#> .\\n@prefix foaf:  <http://xmlns.com/foaf/0.1/> .\\n@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .\\n@prefix freq:  <http://purl.org/cld/freq/> .')\n",
    "g.write('\\n:AOPWikiRDF.ttl\\ta\\tvoid:Dataset ;\\n\\tdc:description\\t\"AOP-Wiki RDF data from the AOP-Wiki database\" ;\\n\\tpav:createdOn\\t\"' + y + '\"^^xsd:date;\\n\\tdcterms:modified\\t\"' + y +'\"^^xsd:date ;\\n\\tpav:createdWith\\t\"' + str(aopwikixmlfilename) + '\", :Promapping ;\\n\\tpav:createdBy\\t<https://zenodo.org/badge/latestdoi/146466058> ;\\n\\tfoaf:homepage\\t<https://aopwiki.org> ;\\n\\tdcterms:accuralPeriodicity  freq:quarterly ;\\n\\tdcat:downloadURL\\t<https://aopwiki.org/downloads/' + str(aopwikixmlfilename) + '> .\\n\\n:AOPWikiRDF-Genes.ttl\\ta\\tvoid:Dataset ;\\n\\tdc:description\\t\"AOP-Wiki RDF extension with gene mappings based on approved names and symbols\" ;\\n\\tpav:createdOn\\t\"' + str(x) + '\" ;\\n\\tpav:createdWith\\t\"' + str(aopwikixmlfilename) + '\", :HGNCgenes ;\\n\\tpav:createdBy\\t<https://zenodo.org/badge/latestdoi/146466058> ;\\n\\tdcterms:accuralPeriodicity  freq:quarterly ;\\n\\tfoaf:homepage\\t<https://aopwiki.org> ;\\n\\tdcat:downloadURL\\t<https://aopwiki.org/downloads/' + str(aopwikixmlfilename) + '>, <https://www.genenames.org/download/custom/> . \\n\\n:HGNCgenes.txt\\ta\\tvoid:Dataset, void:Linkset ;\\n\\tdc:description\\t\"HGNC approved symbols and names for genes\" ;\\n\\tdcat:downloadURL\\t<https://www.genenames.org/download/custom/> ;\\n\\tpav:importedOn\\t\"'+HGNCmodificationTime+'\" .\\n\\n<https://proconsortium.org/download/current/promapping.txt>\\ta\\tvoid:Dataset, void:Linkset;\\n\\tdc:description\\t\"PRotein ontology mappings to protein database identifiers\";\\n\\tdcat:downloadURL\\t<https://proconsortium.org/download/current/promapping.txt>;\\n\\tpav:importedOn\\t\"'+PromodificationTime+'\".')\n",
    "g.close()\n",
    "print(\"The VoID file is created!\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}