Creating model metadata
Greetings!
Let us imagine, that we want to fit the model. The flow to fit the model is the following:
- We save a model name and pipeline:
om.models.put(pipeline, model_store_name)
2. Save a metadata for model:
om.datasets.put(metadata_store, model_meta_store_name, append=False)
Where metadata_store is:
"model_name": model_store_name, "model_type_name": model_type_name, "model_fitted_dataset": model_fitted_dataset, "model_fitted_epochs": model_fitted_epochs
3. Fit the model.
om.runtime.model(model_store_name).fit(...)
The questions are next:
1. Could we store metadata after fitting the model, not before it?
As, if we create metadata before fitting the model, we will still have this meta data even when the result of fitting was a FAILURE. As the result: we have data, but have no model.
2. What is the best way to store models metadata?
Currently all the metadata (for original. synthetic datasets and model) is stored using:
om.datasets.put(metadata_store, model_meta_store_name, append=False)
Seems like it would be better to store models metadata by models, datasets metadata by datasets.
What could be the best approach?
3. How is models metadata created?
We can get models metadata using:
meta = om.models.metadata(model_store_name)
or add additional fields by:
meta.attributes.setdefault('tasks', {})
meta.attributes['tasks'].update({synth_dataset_store_name: task.id})
But how is initial models metadata created?
Comments
But how is initial models metadata created?
The metadata is created on storing the model (so before fit is called via the runtime):
We can also directly add custom attributes when storing the model, using the
attributes=dict(...)parameter as follows, where thedict(...)part is the data you want to add tometadata.attributes:meta = om.models.put(mdl, 'mymodel', attributes=dict(foo='bar')) meta.attributes => { 'foo': 'bar', ... }Specifying
.put(..., attributes=...)is a short-cut and functionality equivalent of the following:models.put(mdl, 'mymodel') meta = om.models.metadata('mymodel') meta.attributes['foo'] = 'bar' meta.save()Save a metadata for model:
om.datasets.put(metadata_store, model_meta_store_name, append=False)
Metadata is a first-class citizen in omega|ml, it gets created automatically, for every object that is stored. There is no need to create an additional dataset.
Could we store metadata after fitting the model, not before it? As, if we create metadata before fitting the model, we will still have this meta data even when the result of fitting was a FAILURE. As the result: we have data, but have no model.
No, storing Metadata cannot be delayed after the model has been fitted, as the runtime won't know about the model unless there is a Metadata object. The FAILURE status is a result of the runtime task, not a property of the model. Thus when you remember the task id as part of the model's Metadata, you can also find out about its state.
Seems like it would be better to store models metadata by models, datasets metadata by datasets.
Yes, every model and every dataset has its own Metadata.
Many thanks for such a detailed answer!
Just want to clarify:
if I want to store a custom model data like:
I can save it after the model was fitted, by adding it to the already created metadata (by om.models.put() )
Right?
I can save it after the model was fitted, by adding it to the already created metadata (by om.models.put() )
Yes, the Metadata exists as long as the model was not dropped. You can update the Metadata as follows:
Thank you for the answer!