Can "cell_types" be more than three?
Recently I am trying to finetune a new model for disease classification with "hyperparameter optimization with raytune for disease classification", at first I chose six different cell types, but the error occured and showed the number of cell_types couldn't be more than three. I wonder the reason about this, and is there any solution can support more than three cell_types? Thank you! : )
filter dataset for given cell_type
def if_cell_type(example):
cell_types = ["Fibroblast1", "Fibroblast2", "Macrophage"]
return any(example["cell_type"].startswith(cell_type) for cell_type in cell_types)
Thank you for your question! There should be no issue with more than 6 cell types. I don’t see the error in your post so cannot help troubleshoot further but I would suggest looking at the documentation for filtering in Hugging Face Datasets.
https://huggingface.co/docs/datasets/en/process
You may also try using the hyperparameter tuning that is part of the classifier module. If you pull the new version you can follow the cell state classification example and provide the filtering dictionary for the module to perform the filtering.