Integration with django-filter: 'Meta.fields' must not contain non-model field names

[mfoglio]

Hello, I have the following models representing some Data that can be created by an Annotator which can either be a human UserAnnotator or an AI AiModelAnnotator

class Annotator(PolymorphicModel):
    pass


class UserAnnotator(Annotator):
    user = models.ForeignKey(
        get_user_model(),
        on_delete=models.PROTECT,
        default=None
    )

class AiModelAnnotator(Annotator):
    ai_model = models.CharField(max_length=255)
    version = models.CharField(max_length=16, default=None, null=True)

class Data(models.Model):
    annotator = models.ForeignKey(Annotator, on_delete=models.PROTECT)

I am using django-rest-framework and django-filter to expose an API endpoint where I can look for data produced by a specific AI model.
The serializers are the following:

class AiModelAnnotatorSerializer(serializers.ModelSerializer):
    class Meta:
        model = AiModelAnnotator
        fields = '__all__'


class UserAnnotatorSerializer(serializers.ModelSerializer):
    class Meta:
        model = UserAnnotator
        fields = '__all__'


class AnnotatorSerializer(serializers.ModelSerializer):
    class Meta:
        model = Annotator
        fields = '__all__'


class AnnotatorPolymorphicSerializer(PolymorphicSerializer):
    model_serializer_mapping = {
        Annotator: AnnotatorSerializer,
        AiModelAnnotator: AiModelAnnotatorSerializer,
        UserAnnotator: UserAnnotatorSerializer,
    }

class AnnotationSerializer(serializers.ModelSerializer):
    annotator = serializers.PrimaryKeyRelatedField(queryset=Annotator.objects.all())
    class Meta:
        model = Annotation
        fields = '__all__'

And the viewset is:

class AnnotationTrainingViewSet(
        mixins.CreateModelMixin,
        mixins.ListModelMixin,
        mixins.RetrieveModelMixin,
        viewsets.GenericViewSet
    ):
    queryset = Annotation.objects.all()
    serializer_class = AnnotatorPolymorphicSerializer
    filterset_fields = ('annotator', 'annotator__ai_model')

When I try to call the GET endpoint with the error 'Meta.fields' must not contain non-model field names: annotator__ai_model which is caused by adding annotator__ai_model to the AnnotationTrainingViewSet. Without specifying annotator__ai_model the code works fine but of course I can't use django-filter as I'd like to.
Is this the expected behavior? Is django-filter supported?

[lvlgl]

I am also searching for a solution to this issue. While I have been able to find workarounds for some cases, these workarounds are not able to accommodate different types of lookups. Therefore, I am hoping that @vdboor will provide an answer soon clarifying how we can properly solve this problem

[JohananOppongAmoateng]

@bckohan might be wrong but i dont think there is a realistic to patch this in django-polymorphic patches might need to be sent to django-filter and also the patch would need to detct that an fk is polymorphic and behave appropriately

[bckohan]

I assume the op is using django-rest-polymorphic. This issue might be fixable there? Which means it would be fixable here when #655 is done.

[JohananOppongAmoateng]

I think so might not hurt to come back once #655 is done to verify

[bckohan]

@mfoglio @lvlgl You can get this to work but you need to implement your own subclass of FilterSet:

The problem you're having is that Data.objects is not a polymorphic queryset and you're trying to filter on a subfield of the polymorphic relation that is not part of the base class so the normal Django/django-filter logic fails.

I would consider a PR that added a PolymorphicFilterSet, but I think that'd be a lot of work to avoid writing a little bit of extra boiler plate like:

from .serializers import AnnotationSerializer
from .models import Data, AiModelAnnotator
from rest_framework import viewsets, mixins
from django_filters.rest_framework import DjangoFilterBackend
import django_filters


class DataFilterSet(django_filters.FilterSet):
    """FilterSet for Data model with polymorphic annotator filtering."""

    annotator__ai_model = django_filters.CharFilter(method="filter_by_ai_model")

    class Meta:
        model = Data
        fields = ["annotator"]

    def filter_by_ai_model(self, queryset, name, value):
        return queryset.filter(annotator__in=AiModelAnnotator.objects.filter(ai_model=value))


class AnnotationTrainingViewSet(
    mixins.CreateModelMixin,
    mixins.ListModelMixin,
    mixins.RetrieveModelMixin,
    viewsets.GenericViewSet,
):
    queryset = Data.objects.all()
    serializer_class = AnnotationSerializer
    filter_backends = [DjangoFilterBackend]
    filterset_class = DataFilterSet