Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Inconsistent validation with complex number #10430

Open
1 task done
Tracked by #10516
AntoineD opened this issue Sep 17, 2024 · 5 comments · May be fixed by pydantic/pydantic-core#1574
Open
1 task done
Tracked by #10516

Inconsistent validation with complex number #10430

AntoineD opened this issue Sep 17, 2024 · 5 comments · May be fixed by pydantic/pydantic-core#1574
Assignees
@ambroseling
Labels
bug V2 Bug related to Pydantic V2 good first issue help wanted Pull Request welcome

Comments

@AntoineD
Copy link

Initial Checks

  • I confirm that I'm using Pydantic V2

Description

A field expecting a list of float number is valid for a numpy array of complex number, but a field expecting a float number is not valid for a complex number.

I would expect the behavior to be the same, i.e. the complex number is a valid float number.

The output of the example code below is (forget about the ComplexWarning):

/x/python3.11/site-packages/pydantic/main.py:211: ComplexWarning: Casting complex values to real discards the imaginary part
  validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
Traceback (most recent call last):
  File "/x/scratch_10.py", line 10, in <module>
    Model(x=1j, y=array([1.]))
  File "/x/python3.11/site-packages/pydantic/main.py", line 211, in __init__
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for Model
x
  Input should be a valid number [type=float_type, input_value=1j, input_type=complex]
    For further information visit https://errors.pydantic.dev/2.9/v/float_type

Example Code

from numpy import array
from pydantic import BaseModel


class Model(BaseModel):
    x: float
    y: list[float]

Model(x=1., y=array([1.]))
Model(x=1j, y=array([1.]))

Python, Pydantic & OS Version

pydantic version: 2.9.0
        pydantic-core version: 2.23.2
          pydantic-core build: profile=release pgo=false
                 install path: /x/python3.11/site-packages/pydantic
               python version: 3.11.10 (main, Sep  9 2024, 00:00:00) [GCC 14.2.1 20240801 (Red Hat 14.2.1-1)]
                     platform: Linux-6.10.9-200.fc40.x86_64-x86_64-with-glibc2.39
             related packages: typing_extensions-4.12.2 typing_extensions-4.12.2
                       commit: unknown
@AntoineD AntoineD added bug V2 Bug related to Pydantic V2 pending Awaiting a response / confirmation labels Sep 17, 2024
@sydney-runkle
Copy link
Member

Yep, looks like a bug 👍. Can take a look later this week.

@sydney-runkle sydney-runkle removed the pending Awaiting a response / confirmation label Sep 17, 2024
@sydney-runkle sydney-runkle self-assigned this Sep 17, 2024
@sydney-runkle sydney-runkle mentioned this issue Sep 30, 2024
Closed
10 tasks
@sydney-runkle
Copy link
Member

So, I'm not opposed to supporting complex -> float coercion in lax mode, though this isn't high priority for us. PRs welcome adding support in pydantic-core :).

@rawwar

This comment was marked as outdated.

Sign in to view
@sydney-runkle sydney-runkle assigned rawwar and unassigned sydney-runkle Oct 9, 2024
@rawwar rawwar removed their assignment Oct 16, 2024
@ambroseling
Copy link

hi @sydney-runkle may I try tackling this issue, thanks!

@ambroseling ambroseling linked a pull request Dec 8, 2024 that will close this issue
Open
4 tasks
@changhc
Copy link
Contributor

changhc commented Dec 9, 2024 •
edited
Loading

So I just actually tried out the sample code and played around with that complex coercion when I was reading pydantic/pydantic-core#1574. It seems that this is a numpy-only behaviour and thus I'm not sure if we should implement this for all complex numbers in pydantic.

A field expecting a list of float number is valid for a numpy array of complex number

For example, numpy.array([1j]) has dtype=complex128, not the builtin complex type. These numpy complex types perform the coercion and raise a numpy ComplexWarning in such cases. If you try your sample code with

Model(x=numpy.complex128(1, 2), y=array([1.]))

pydantic won't raise any error either and you'll see the coercion.

a field expecting a float number is not valid for a complex number

It is not valid for a complex number instantiated with the builtin complex type but such fields do accept numpy.complex64/complex128. This, however, is because numpy handles the coercion, not pydantic.

I don't think we should follow this numpy case because coercions can be done differently:

  • Simply dropping the imaginary part: this is how numpy does it
  • Accepting only complex numbers with a zero imaginary part: this is what I imagine coercion to be like for complex numbers
  • No coercion at all: this is the default behaviour of the builtin float type, e.g. float(complex(1, 2)) gives an error, even when the imaginary part is 0

and there's no point for pydantic to simply follow how it's done in numpy. Users might expect different outcomes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ambroseling ambroseling

Labels
bug V2 Bug related to Pydantic V2 good first issue help wanted Pull Request welcome
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Consistent validation with complex number to float type coercion ambroseling/pydantic-core
5 participants
@AntoineD @changhc @rawwar @sydney-runkle @ambroseling