OmegaStore.put - wrong pipeline type

daria · October 2020

I am trying to invoke om.models.put with pipeline and model_store_name:

if model_type_name == "CTGAN":
    pipeline = CTGANSynthesizer()

om.models.put(pipeline, model_store_name)

Which pass us to omegaml.store.base.OmegaStore.putThis method put() takes as one of the args an obj.

obj =  <ctgan.synthesizer.CTGANSynthesizer at 0x7fd6423a2240>

So technically, this obj is our pipeline and a CTGANSynthesizer instance at the same time.

But then, inside a method OmegaStore.put we check:

is_estimator(obj),
s_spark_mllib(obj),
is_dataframe(obj) etc.

None of the checks perform CTGANSynthesizer. As the result I get an error:

TypeError: type <class 'ctgan.synthesizer.CTGANSynthesizer'> not supported

Seems like there should be not just a CTGANSynthesizer obj but some other type.

What type of pipeline should it bee?

support · October 2020

om.mondels.put(obj, 'name') works by calling each registered plugin's PluginClass.supports(obj, name) method. If the plugin returns True, the plugin's .put() method is called.

The methods you mention are part of the omega|ml's built in scikit-learn and spark model handling. The fact that these methods are called would mean that your plugin is either not registered or it does not recognize the pipeline object as being an object it supports.

You can check that your plugin is recognized by calling the following function. It should return an instance of your backend. If it does not, your backend is either not registered or it does not recognize your pipeline as being an object it can store (your code, your decision 😉 ).

# check the backend is registered and knows about the pipeline object
mybackend = om.models.get_backend_byobj(pipeline)
assert mybackend is not None

To see which backends are registered, check as follows. It should include your backend class:

om.defaults.OMEGA_STORE_BACKENDS                                                                                                          
Out[4]: 
{'sklearn.joblib': omegaml.backends.scikitlearn.ScikitLearnBackend,
 'ndarray.bin': omegaml.backends.npndarray.NumpyNDArrayBackend,
 'virtualobj.dill': omegaml.backends.virtualobj.VirtualObjectBackend,
 'pandas.rawdict': omegaml.backends.rawdict.PandasRawDictBackend,
 'python.file': omegaml.backends.rawfiles.PythonRawFileBackend,
 'python.package': omegaml.backends.package.localpip.PythonPackageData,
 'pipsrc.package': omegaml.backends.package.remotepip.PythonPipSourcedPackageData,
 'pandas.csv': omegaml.backends.externaldata.PandasExternalData,
 'sqlalchemy.conx': omegaml.backends.sqlalchemy.SQLAlchemyBackend}

The process of how plugins work, along with an example, is described in more detail in this article and in the documentation.

daria · October 2020

Thank you for the response!

OmegaStore.put - wrong pipeline type

Comments