OmegaStore.put - wrong pipeline type

I am trying to invoke om.models.put with pipeline and model_store_name:

if model_type_name == "CTGAN":
    pipeline = CTGANSynthesizer()

om.models.put(pipeline, model_store_name)

Which pass us to method put() takes as one of the args an obj.

obj =  <ctgan.synthesizer.CTGANSynthesizer at 0x7fd6423a2240>

So technically, this obj is our pipeline and a CTGANSynthesizer instance at the same time.

But then, inside a method OmegaStore.put we check:

  • is_estimator(obj),
  • s_spark_mllib(obj),
  • is_dataframe(obj) etc.

None of the checks perform CTGANSynthesizer. As the result I get an error:

TypeError: type <class 'ctgan.synthesizer.CTGANSynthesizer'> not supported

Seems like there should be not just a CTGANSynthesizer obj but some other type.

What type of pipeline should it bee?


  • edited October 2020

    om.mondels.put(obj, 'name') works by calling each registered plugin's PluginClass.supports(obj, name) method. If the plugin returns True, the plugin's .put() method is called.

    The methods you mention are part of the omega|ml's built in scikit-learn and spark model handling. The fact that these methods are called would mean that your plugin is either not registered or it does not recognize the pipeline object as being an object it supports.

    You can check that your plugin is recognized by calling the following function. It should return an instance of your backend. If it does not, your backend is either not registered or it does not recognize your pipeline as being an object it can store (your code, your decision 😉 ).

    # check the backend is registered and knows about the pipeline object
    mybackend = om.models.get_backend_byobj(pipeline)
    assert mybackend is not None

    To see which backends are registered, check as follows. It should include your backend class:

    {'sklearn.joblib': omegaml.backends.scikitlearn.ScikitLearnBackend,
     'ndarray.bin': omegaml.backends.npndarray.NumpyNDArrayBackend,
     'virtualobj.dill': omegaml.backends.virtualobj.VirtualObjectBackend,
     'pandas.rawdict': omegaml.backends.rawdict.PandasRawDictBackend,
     'python.file': omegaml.backends.rawfiles.PythonRawFileBackend,
     'python.package': omegaml.backends.package.localpip.PythonPackageData,
     'pipsrc.package': omegaml.backends.package.remotepip.PythonPipSourcedPackageData,
     'pandas.csv': omegaml.backends.externaldata.PandasExternalData,
     'sqlalchemy.conx': omegaml.backends.sqlalchemy.SQLAlchemyBackend}

    The process of how plugins work, along with an example, is described in more detail in this article and in the documentation.

  • Thank you for the response!

Sign In or Register to comment.