Get task status

edited November 2020 in Ask anything

Greetings!

I have a question concerning task status:

I want to show a list of all the synthetic datasets to the User with the status of generating (as it can take a wile, User can see when it is Finished).

Currently, we have 1 endpoint to make a fit-generate and 1 endpoint to list all the synthetic datasets. Inside the fit-generate endpoint I can call a result.status (just after I call .predict ) and get a "Pending", "SUCCESS" etc.

But how can I get this status inside the list datasets, when all that I can do is just call:

om.datasets.get(meta_synthetic_dataset_store_name)[0]

Can I store the status data inside the created metadata before generating?

synth_meta_store = om.datasets.get(meta_synthetic_dataset_store_name)[0]
synth_meta_store['task_result'] = task_result.status
om.datasets.put(synth_meta_store, meta_synthetic_dataset_store_name, append=False)

Or is there another way?

Comments

  • edited December 2020

    Can I store the status data inside the created metadata before generating?

    Storing the task id in the metadata of an *existing* dataset or model, is possible like so:

    task = om.runtime.model('mymodel').predict()
    meta = om.models.metadata('mymodel')
    meta.attributes.setdefault('tasks', {})
    meta.attributes['tasks'].update({'expected-dataset-name': task.id}) 
    meta.save()
    

    To check the task status later:

    meta = om.models.metadata('mymodel')
    taskid = meta.attributes['tasks']['expected-dataset-name']
    task = om.runtime.celeryapp.AsyncResult(taskid)
    status = task.status # this will either PENDING, RUNNING, SUCCESS, FAILURE
    

    Note I used the model, instead of the dataset, to store the task status for two reasons:

    1. the dataset does not exist yet, so there is no metadata entry yet.
    2. the task is really related to the model so IMHO that's actually the better place

    However you can store the same metadata on any other object. For example you could create a dummy "task-status" dataset and put the metadata there, e.g.

    meta = om.datasets.put({}, 'task-status')
    ... (same as above)
    

    PS. As this is quite a common question we are considering a built-in feature for easier task tracking in one of the upcoming releases.

  • edited December 2020

    Thanks! It works!

    Will  store the task status in the model metadata.

    P.S. In case anyone will use the code - there were some misspellings:

    ...
    # the 'r' was missed
    meta.attributes['tasks'].update({'expected-dataset-name': task.id})
    ...
    
    ...
    # not a 'task['taskid'] but just 'taskid'
    task = om.runtime.celeryapp.AsyncResult(taskid)
    status = task.status # this will either PENDING, RUNNING, SUCCESS, FAILURE
    


  • Thank you. Updated the original answer.

Sign In or Register to comment.