Issue with MongoDB Atlas style DBURL

[uvaaland]

mLab has been acquired by MongoDB, Inc which means that all new users on www.mlab.com will be users on MongoDB Atlas instead. It is still possible to set up a free database, but the url format is different.

mongodb://<USER>:<PASSWORD>@cluster0-shard-00-00-8dd3w.mongodb.net:27017,cluster0-shard-00-01-8dd3w.mongodb.net:27017,cluster0-shard-00-02-8dd3w.mongodb.net:27017/test?ssl=true&replicaSet=Cluster0-shard-0&authSource=admin&retryWrites=true

When trying to use this as RADICAL_PILOT_DBURL the following error occurs:

db.net:27017,cluster0-shard-00-02-8dd3w.mongodb.net:27017'
Traceback (most recent call last):
  File "/home/vagrant/scratch/entk/venv/lib/python2.7/site-packages/radical/entk/appman/appmanager.py", line 309, in run
    self._resource_manager._submit_resource_request()
  File "/home/vagrant/scratch/entk/venv/lib/python2.7/site-packages/radical/entk/execman/rp/resource_manager.py", line 125, in _s$
bmit_resource_request
    self._session = rp.Session(dburl=self._mlab_url, uid=self._sid)
  File "/home/vagrant/scratch/entk/venv/lib/python2.7/site-packages/radical/pilot/session.py", line 269, in __init__
    % (dburl, ex))
RuntimeError: Couldn't create new session (database URL 'mongodb://uvaaland:[email protected]$
t:27017,cluster0-shard-00-01-8dd3w.mongodb.net:27017,cluster0-shard-00-02-8dd3w.mongodb.net:27017/test?ssl=true&replicaSet=Cluste$
0-shard-0&authSource=admin&retryWrites=true' incorrect?): invalid literal for int() with base 10: '27017,cluster0-shard-00-01-8dd$
w.mongodb.net:27017,cluster0-shard-00-02-8dd3w.mongodb.net:27017'
Traceback (most recent call last):
  File "get_started.py", line 59, in <module>
    appman.run()
  File "/home/vagrant/scratch/entk/venv/lib/python2.7/site-packages/radical/entk/appman/appmanager.py", line 309, in run
    self._resource_manager._submit_resource_request()
  File "/home/vagrant/scratch/entk/venv/lib/python2.7/site-packages/radical/entk/execman/rp/resource_manager.py", line 125, in _su
bmit_resource_request
    self._session = rp.Session(dburl=self._mlab_url, uid=self._sid)
  File "/home/vagrant/scratch/entk/venv/lib/python2.7/site-packages/radical/pilot/session.py", line 269, in __init__
    % (dburl, ex))
RuntimeError: Couldn't create new session (database URL 'mongodb://<USER>:[email protected]
t:27017,cluster0-shard-00-01-8dd3w.mongodb.net:27017,cluster0-shard-00-02-8dd3w.mongodb.net:27017/test?ssl=true&replicaSet=Cluster
0-shard-0&authSource=admin&retryWrites=true' incorrect?): invalid literal for int() with base 10: '27017,cluster0-shard-00-01-8dd3
w.mongodb.net:27017,cluster0-shard-00-02-8dd3w.mongodb.net:27017'

Which suggests that there is an issue with the way that EnTK tries to parse the URL.

To verify that the problem originates in the EnTK layer I tried to open up a Python shell and connect to the database with the above url, and I was able to do that successfully.

Just wanted to open a ticket on this since mLab is moving to MongoDB Atlas, so it would be good to make sure that EnTK can also deal with this url format.

session.tar.gz

Attached are all the session files and verbose.log

[andre-merzky]

I am actually not sure this is a valid URL format... mlab should also offer a short version of a connection URL (w/o the explicit sharding locations) - can you please try that one? Thanks!

[mturilli]

@uvaaland is this still an issue?

[uvaaland]

As far as I know. Vivek provided me with an MLAB url which I have been working on. I could try it again with the new url format and update the ticket accordingly.

[mturilli]

Thanks Uno, no worries. The important thing is that you are up and running. We will pick up testing from here on.

[mturilli]

We should:

1. Find an alternative to mlab
2. Document how to create a container with MongoDB
3. Create a template on JetStream and provide users a dedicated MongoDB instance
4. Try to use former mlab and document changes in API

[lee212]

@uvaaland , I will close this ticket for now as it has not been updated for a while. Also supporting mlab/atlas won't be a problem since there is a dedicated mongodb server from RADICAL team to use but feel free to reopen if this still should be addressed and leave a comment.