BioGTP_pedromedeiros.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "Essa é uma implementação simples do BioGPT a partir do [hugging face](https://huggingface.co/docs/transformers/model_doc/biogpt). O funcionamento dele, para quem não conhece o Colab, é bem simples: Dê play nas duas primeiras seções para que os pacotes sejam instalados e importados e nos dois últimos para rodar a aplicação.\n",
        "\n",
        "\n",
        "Para trocar a seed (a frase que irá motivar a busca e gerar uma mensagem) basta alterar a frase no trecho\n",
        "- `sentence = \"Influenza is a virus\"` Para o modo de geração de um parágrafo:\n",
        "\n",
        "- `generator(\"E. coly in metagenomics results\", max_length=500, num_return_sequences=10, do_sample=True)` para geração de frases individuais.\n",
        "\n",
        "\n",
        "\n",
        "Ele não utiliza nenhuma API, então todo processamento dele ocorre nesse notebook, que tem uma quantidade bastante limitada de memória RAM, o que pode fazer com que ele esgote os recursos antes de demonstrar uma resposta caso sua requisição seja muito complexa.\n",
        "\n",
        "Caso você queira aprender mais, vá nos links abaixo:\n",
        "\n",
        "- [ArxIv do projeto](https://arxiv.org/abs/2210.10341)\n",
        "- [GitHub do projeto](https://github.com/microsoft/BioGPT)"
      ],
      "metadata": {
        "id": "01pQKHzseYm7"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "X3dNyL2p_ZgM",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f4a90b51-fe92-4134-a728-112c69a02a9d"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.3/6.3 MB\u001b[0m \u001b[31m53.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m880.6/880.6 KB\u001b[0m \u001b[31m32.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m190.3/190.3 KB\u001b[0m \u001b[31m9.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.6/7.6 MB\u001b[0m \u001b[31m63.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Building wheel for sacremoses (setup.py) ... \u001b[?25l\u001b[?25hdone\n"
          ]
        }
      ],
      "source": [
        "!pip install -q -U transformers sacremoses"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "from transformers import BioGptTokenizer, BioGptForCausalLM, set_seed, pipeline"
      ],
      "metadata": {
        "id": "snh4_vy7BJwO"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "tokenizer = BioGptTokenizer.from_pretrained(\"microsoft/biogpt\")\n",
        "model = BioGptForCausalLM.from_pretrained(\"microsoft/biogpt\")\n",
        "\n",
        "sentence = \"Influenza is a virus\"\n",
        "inputs = tokenizer(sentence, return_tensors=\"pt\")\n",
        "\n",
        "set_seed(42)\n",
        "\n",
        "with torch.no_grad():\n",
        "    beam_output = model.generate(**inputs,\n",
        "                                min_length=100,\n",
        "                                max_length=1024,\n",
        "                                num_beams=5,\n",
        "                                early_stopping=True\n",
        "                                )\n",
        "tokenizer.decode(beam_output[0], skip_special_tokens=True)"
      ],
      "metadata": {
        "id": "ZEGcJC6FBKNt",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 122
        },
        "outputId": "593f5114-7e8b-4d0c-a0cf-c06280111827"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'Influenza is a virus of the family Paramyxoviridae, which causes seasonal epidemics and occasional pandemics of varying severity in humans and animals, and is a significant cause of morbidity and mortality in humans and domestic and wild animals, including birds, reptiles, amphibians, fish, and marine mammals, as well as in domestic and wild animals and humans in contact with these animals and their products (e.g., eggs, meat, milk, and honey). (1-3) Influenza A viruses (IAVs) are classified into six antigenic groups (A, B, C, D, E, and F) based on their hemagglutinin (HA) proteins.'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "generator = pipeline('text-generation', model=model, tokenizer=tokenizer)\n",
        "set_seed(42)\n",
        "generator(\"E. coly in metagenomics results\", max_length=500, num_return_sequences=10, do_sample=True)"
      ],
      "metadata": {
        "id": "yHrPWFKnrTfm",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "013f2d3f-a786-4309-a70e-304e3a7b981e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[{'generated_text': 'E. coly in metagenomics results means that the functional composition of a microbial community is significantly modified than in metagenomic libraries, and this has great practical implications.'},\n",
              " {'generated_text': 'E. coly in metagenomics results means more opportunities for microbial engineering, but also more challenges.'},\n",
              " {'generated_text': 'E. coly in metagenomics results means it is an interesting application of the theory of ecological systems.'},\n",
              " {'generated_text': 'E. coly in metagenomics results means to have a greater access to basic research.'},\n",
              " {'generated_text': 'E. coly in metagenomics results means it is difficult to assess whether such results are trustworthy and do not change the course of the research.'},\n",
              " {'generated_text': 'E. coly in metagenomics results means that the whole-sample pipeline will be improved, but the methods and applications for metagenomics are still not adequate.'},\n",
              " {'generated_text': 'E. coly in metagenomics results means that only a limited number of DNA fragments are currently of potential interest for genome based studies.'},\n",
              " {'generated_text': 'E. coly in metagenomics results means that all the genes encoding the exopeptidases are probably also present in these bacteria.'},\n",
              " {'generated_text': 'E. coly in metagenomics results means new ways to explore microorganisms.'},\n",
              " {'generated_text': 'E. coly in metagenomics results means that the abundance of these taxa increases after the harvest process.'}]"
            ]
          },
          "metadata": {},
          "execution_count": 36
        }
      ]
    }
  ]
}