Connect to OmegaML MongoDB

daria

Greetings!

On our project, we want to run some async tasks using Celery and the local MongoDB container.

We have initialized a new Celery app with such backend as:

BACKEND_URL = 'mongodb://admin:foobar@0.0.0.0:27017/omega'

MongoDB is running in a container, which we are using to proceed requests to omegaml - e.g. om.datasets(...) etc

The container was build using:

# mongodb
MONGO_INITDB_ROOT_USERNAME=admin
MONGO_INITDB_ROOT_PASSWORD=foobar

And omega worker is using:

OMEGA_BROKER=amqp://rabbitmq:5672/
OMEGA_MONGO_URL=mongodb://admin:foobar@mongodb/omega

But, when our local Celery app trying to call the OmegaML MongoDB:

om.datasets.get(metadata_store_name)

we get the next error:

pymongo.errors.OperationFailure: Authentication failed., full error: {'ok': 0.0, 'errmsg': 'Authentication failed.', 'code': 18, 'codeName': 'AuthenticationFailed'}

Basically, we are trying to connect to MongoDB connected to OmegaML. Seems like, the authentication mechanism is performed on the OmegaML side, namely in the OmegaStore class.

The authentication flow is not clear. Is it enough just to pass a variable as:

 OMEGA_MONGO_URL=mongodb://admin:foobar@mongodb/omega   ?

Or should we perform some additional settings to connect to the MongoDB used by OmegaML?

Are there any options to check if authentication was successful, or maybe you can briefly describe the flow, how the OmegaML + Celery + MongoDB works together?

daria

HI!

I just do not use a CELERY_RESULT_BACKEND for our local Celery app and run tasks without specifying the MongoDB. Just use RabbitMQ as a broker.

Everything works fine now!

support

we are trying to connect to MongoDB connected to OmegaML

The omegaml worker and the client both need to specify the same OMEGA_MONGO_URL and the same OMEGA_BROKER in order to connect.

the authentication mechanism is performed on the OmegaML side, namely in the OmegaStore class.

MongoDB authentication is performed upon first access to any of om.datasets|models|jobs|scripts etc. The actual authentication is done by the MongoDB server, not the client. omegaml just uses the OMEGA_MONGO_URL to connect.

how the OmegaML + Celery + MongoDB works together?

omegaml is imported and instantiated as Python module on both the client (e.g. your code, Jupyter Notebook) and by the Celery worker processes. Each Celery worker opens its own connection to the MongoDB server. Upon submitting a runtime task (e.g. om.runtime.model('mymodel').predict(...) omegaml sends a task request to a Celery worker, via Rabbitmq. The worker picks up the request and uses its omegaml instance to connect to the database, in particular using the OMEGA_MONGO_URL.

CELERY_RESULT_BACKEND for our local Celery app and run tasks without specifying the MongoDB. 

This may work, however this way it is not possible to get track task results in the client. Currently omegaml uses amqp as the result backend (see OMEGA_CELERY_DEFAULTS).

Reference: https://omegaml.github.io/omegaml/admin/configuration.html#celery-cluster-configuration