lots of changes #48
Draft
lots of changes #48
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The URL for the fungal category of the Alliance genome has been updated for accuracy. The delay in the UI testing code has been increased to accommodate changes in page loading times. Screenshots will now be saved in the SGD directory improving file organization.
This commit adds the new test configuration file config.json for the SGD test suite. The configuration file details various server endpoints and their database records for testing purposes. It includes both main and fungal databases, each with their specific URLs and nucleotide and protein sequences.
Added a new test case for FB2024_02 in the UI testing config and made enhancements to the testing scripts. Improvements involve adding extra failure tolerance in element interaction and storing screenshots in organized output directories. Also updated the utilities to fetch and print unique sorted list of items from the URL for better clarity.
Consolidated import statements for readability and removed an unused function call to `route53_check` to streamline the code. This improves maintainability and reduces potential confusion for future modifications.
Refactored `validate_fasta` to remove Biopython dependency, using a manual method to validate FASTA files. Added `list_databases_from_config` to list database names from YAML or JSON config files. Cleaned up `create_blast_db.py` by removing redundant comments and improving error handling for JSON processing.
Upgraded multiple dependencies to newer versions, including attrs to 24.2.0, black to 23.12.1, boto3 to 1.35.46, and boto3-stubs to 1.35.46. Adjusted package extras to align with the new versions and ensure compatibility.
Introduced a new utility function, needs_parse_seqids, to determine if the -parse_seqids flag is needed for BLAST. Updated create_blast_db.py to use this function and include the flag conditionally in the makeblastdb command based on FASTA header analysis.
Introduce several new utilities for downloading, storing, and processing FASTA files, such as process_entry and process_files. Enhanced makeblastdb integration and error handling, code cleaning, and modularization for improved maintainability.
Introduce a conditional check to skip MD5 verification for ZFIN in both FTP and HTTP file downloads. Also, refactor the process_entry function for better logging and flow control during checks.
Refactored the logic for handling FTP and HTTP file downloads for clarity by breaking long lines into multiple lines. Additionally, adjusted string formatting for output status to improve readability.
Introduce a new CLI option `--keep-files` to control whether downloaded FASTA files are retained after processing. Also, refactor import statements and add a new helper function `process_entry` for better code organization.
Removed the `keep_files` option from the `create_dbs` function and associated code. Refactored the imports and adjusted the code formatting in `utils.py` for clarity and consistency. These changes help to streamline the code and improve readability.
Removed redundant comments and excessive logging in utils.py functions for clarity. Also, improved import statements in create_blast_db.py to enhance readability. The refactoring maintains existing functionality while making the code easier to read and maintain.
Reorganized imports and added missing newlines for readability in `utils.py` and `create_blast_db.py`. This improves code structure and ensures consistency with Python PEP8 standards.
Refactor utility functions to include comprehensive logging and error handling. Improved the logging format for better traceability and added optional file storage during downloads. Streamlined functions by removing unused imports and redundant checks.
Introduced optional cleanup of FASTA files in processing functions and command-line arguments. Modified logging to include the new parameters and ensure continuation even if errors occur during cleanup. Enhanced type hinting in `src/utils.py` and related files in `src/create_blast_db.py`.
Introduced an `update_genome_browser_map` function to manage genome browser mappings in Ruby and JSON formats. Integrated this function into the BLAST DB creation process to ensure mappings are updated if relevant configuration data is present.
Replace console output with `terminal.py` module functions for consistent logging and progress tracking. Add enhanced progress and summary display for JSON entry processing, and reorganize code structure for better readability and maintainability.
Refactor logging and status updates to use `print_status` for better user feedback. Introduce comprehensive error handling and add progress bars for cleanup operations. Streamline imports and improve functionality for Slack message sending and file downloading.
Implemented `copy_config_file` to manage configuration files and enhanced `update_genome_browser_map` for better logging and file handling. Updated `process_json_entries` and `process_entry` to integrate new functions, perform unzipping, and handle config file management more efficiently.
Refactored the update_genome_browser_map function to improve readability and directory structure. Enhanced error handling and logging during the makeblastdb process for better operational visibility.
This commit cleans up unnecessary blank lines in `tests/UI/utils.py` and `tests/CLI/test_cli.py` for better readability and consistency. These changes do not affect the functionality but improve code quality.
Refactored `update_genome_browser_map` function to include improved logging and error handling. Added real-time feedback using both logger and rich print to display messages, including debug information about current operations and handling of file writes. Streamlined directory and file creation process, and ensured robust reading and writing of JSON and Ruby files.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.