Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add rucio upload script and rucio configuration file #27

Open
wants to merge 20 commits into
base: main
Choose a base branch
from

Conversation

rahmans1
Copy link
Contributor

Briefly, what does this PR introduce?

Add rucio registration code snippet to run script

What kind of change does this PR introduce?

  • Bug fix (issue #__)
  • New feature (issue #__)
  • Documentation update
  • Other: __

Please check if this PR fulfills the following:

  • Tests for the changes have been added
  • Documentation has been added / updated
  • Changes have been communicated to collaborators

Does this PR introduce breaking changes? What changes might users need to make to their code?

Does this PR change default behavior?

@rahmans1 rahmans1 marked this pull request as draft December 14, 2024 04:19
@rahmans1 rahmans1 requested a review from wdconinc December 14, 2024 04:19
scripts/run.sh Outdated Show resolved Hide resolved
@rahmans1 rahmans1 requested a review from wdconinc December 14, 2024 21:11
@rahmans1 rahmans1 marked this pull request as ready for review December 14, 2024 21:11
@rahmans1
Copy link
Contributor Author

We are still missing something may be.

jug_dev> rahmans@ifarm2401:/w/eic-scshelf2104/users/rahmans/simulation_campaign_hepmc3/scripts$ ./register_to_rucio.py -f /work/eic2/EPIC/main/CI/SINGLE/e-/100MeV/130to177deg/e-_100MeV_130to177deg.edm4hep.root --du $(xrdfs root://dtn-eic.jlab.org stat /work/eic2/EPIC/main/CI/SINGLE/e-/100MeV/130to177deg/e-_100MeV_130to177deg.edm4hep.root | grep 'Size' | awk -F' ' '{print $2}')
/opt/local/lib/python3.11/site-packages/paramiko/pkey.py:100: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from this module in 48.0.0.
  "cipher": algorithms.TripleDES,
/opt/local/lib/python3.11/site-packages/paramiko/transport.py:259: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from this module in 48.0.0.
  "class": algorithms.TripleDES,
Traceback (most recent call last):
  File "/w/eic-scshelf2104/users/rahmans/simulation_campaign_hepmc3/scripts/./register_to_rucio.py", line 23, in <module>
    client=Client()
           ^^^^^^^^
  File "/opt/local/lib/python3.11/site-packages/rucio/client/client.py", line 76, in __init__
    super(Client, self).__init__(**args)
  File "/opt/local/lib/python3.11/site-packages/rucio/client/baseclient.py", line 121, in __init__
    self.host = config_get('client', 'rucio_host')
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/lib/python3.11/site-packages/rucio/common/config.py", line 212, in config_get
    raise err
  File "/opt/local/lib/python3.11/site-packages/rucio/common/config.py", line 190, in config_get
    return convert_type_fnc(get_config().get(section, option))
                            ^^^^^^^^^^^^
  File "/opt/local/lib/python3.11/site-packages/rucio/common/config.py", line 768, in get_config
    __CONFIG = Config()
               ^^^^^^^^
  File "/opt/local/lib/python3.11/site-packages/rucio/common/config.py", line 792, in __init__
    raise ConfigNotFound(
rucio.common.exception.ConfigNotFound: Configuration not found.
Details: Could not load Rucio configuration file. Rucio looked in the following paths for a configuration file, in order:
        /opt/rucio/etc/rucio.cfg
jug_dev> rahmans@ifarm2401:/w/eic-scshelf2104/users/rahmans/simulation_campaign_hepmc3/scripts$

@rahmans1 rahmans1 force-pushed the feature-register-to-rucio branch from c2b4985 to a5371da Compare December 20, 2024 04:29
@rahmans1 rahmans1 changed the title Add rucio registration code snippet to run script Add rucio upload script and rucio configuration file Dec 20, 2024
.github/workflows/run_prod.yml Outdated Show resolved Hide resolved
rucio_host = https://rucio-server.jlab.org:443
auth_host = https://rucio-server.jlab.org:443
auth_type = x509_proxy
client_x509_proxy = $X509_USER_PROXY
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work when specified as an environment variable?

wdconinc
wdconinc previously approved these changes Dec 20, 2024
Copy link
Contributor

@wdconinc wdconinc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, if it works with the environment variable in the config.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants