8.1.0

[seperman]

• Removing deprecated lines from setup.py
• Added prefix option to pretty()
• Fixes hashing of numpy boolean values.
• Fixes slots comparison when the attribute doesn't exist.
• Relaxing orderly-set reqs
• Added Python 3.13 support
• Only lower if clean_key is instance of str
• Only lower if clean_key is instance of str Only lower if clean_key is instance of str #504
• Fixes issue where the key deep_distance is not returned when both compared items are equal
• Fixes issue where the key deep_distance is not returned when both compared items are equal Fixes issue where the key deep_distance is not returned when both compared items are equal #510
• Fixes exclude_paths fails to work in certain cases
• exclude_paths fails to work exclude_paths fails to work #509
• Fixes to_json() method chokes on standard json.dumps() kwargs such as sort_keys
• to_dict() method chokes on standard json.dumps() kwargs to_dict() method chokes on standard json.dumps() kwargs  #490
• Fixes accessing the affected_root_keys property on the diff object returned by DeepDiff fails when one of the dicts is empty
• In version 8.0.1, accessing the affected_root_keys property on the diff object returned by DeepDiff fails when one of the dicts is empty In version 8.0.1, accessing the affected_root_keys property on the diff object returned by DeepDiff fails when one of the dicts is empty #508

[codecov]

Codecov Report

Attention: Patch coverage is 93.93939% with 2 lines in your changes missing coverage. Please review.

Project coverage is 96.52%. Comparing base (6d8a4c7) to head (32d60a9).

Files with missing lines	Patch %	Lines
deepdiff/deephash.py	66.66%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #483      +/-   ##
==========================================
- Coverage   96.70%   96.52%   -0.19%     
==========================================
  Files          14       14              
  Lines        3946     3971      +25     
==========================================
+ Hits         3816     3833      +17     
- Misses        130      138       +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[seperman]

Hi @artemisart Thanks for reviewing!
Sorry I was too busy. I'm going to make a release over the weekend.

[eamanu]

Hello!

I'm patching this branch in Debian to have deepdiff workin in python3.13. I have these tests failing. I'm didn't check in depth if is an error on my setup, or perhaps is something that can help you in something.

=================================== FAILURES ===================================
___________ TestDeltaCompareFunc.test_list_of_alphabet_and_its_delta ___________

self = <tests.test_delta.TestDeltaCompareFunc object at 0x7f83493da1e0>

    def test_list_of_alphabet_and_its_delta(self):
        l1 = "A B C D E F G D H".split()
        l2 = "B C X D H Y Z".split()
        diff = DeepDiff(l1, l2)
    
        # Problem: The index of values_changed should be either all for AFTER removals or BEFORE removals.
        # What we have here is that F & G transformation to Y and Z is not compatible with A and E removal
        # it is really meant for the removals to happen first, and then have indexes in L2 for values changing
        # rather than indexes in L1. Here what we need to have is:
        # A B C D E F G D H
        # A B C-X-E
        # B C D F G D H  # removal
    
        # What we really need is to report is as it is in difflib for delta specifically:
        # A B C D E F G D H
        # B C D E F G D H     delete    t1[0:1] --> t2[0:0]    ['A'] --> []
        # B C D E F G D H     equal     t1[1:3] --> t2[0:2] ['B', 'C'] --> ['B', 'C']
        # B C X D H           replace   t1[3:7] --> t2[2:3] ['D', 'E', 'F', 'G'] --> ['X']
        # B C X D H           equal     t1[7:9] --> t2[3:5] ['D', 'H'] --> ['D', 'H']
        # B C X D H Y Z       insert    t1[9:9] --> t2[5:7]       [] --> ['Y', 'Z']
    
        # So in this case, it needs to also include information about what stays equal in the delta
        # NOTE: the problem is that these operations need to be performed in a specific order.
        # DeepDiff removes that order and just buckets all insertions vs. replace vs. delete in their own buckets.
        # For times that we use Difflib, we may want to keep the information for the array_change key
        # just for the sake of delta, but not for reporting in deepdiff itself.
        # that way we can re-apply the changes as they were reported in delta.
    
        delta = Delta(diff)
        assert l2 == l1 + delta
        with pytest.raises(ValueError) as exc_info:
            l1 == l2 - delta
        assert "Please recreate the delta with bidirectional=True" == str(exc_info.value)
    
        delta2 = Delta(diff, bidirectional=True)
        assert l2 == l1 + delta2
        assert l1 == l2 - delta2
    
        dump = Delta(diff, bidirectional=True).dumps()
        delta3 = Delta(dump, bidirectional=True)
    
        assert l2 == l1 + delta3
        assert l1 == l2 - delta3
    
        dump4 = Delta(diff, bidirectional=True, serializer=json_dumps).dumps()
>       delta4 = Delta(dump4, bidirectional=True, deserializer=json_loads)

tests/test_delta.py:2440: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
deepdiff/delta.py:130: in __init__
    self.diff = _deserializer(diff, safe_to_import=safe_to_import)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

obj = '{"values_changed": {"root[3]": {"new_value": "X", "old_value": "D", "new_path": "root[2]"}, "root[5]": {"new_value": ...3, 7, 2, 3, ["D", "E", "F", "G"], ["X"]], ["equal", 7, 9, 3, 5, null, null], ["insert", 9, 9, 5, 7, [], ["Y", "Z"]]]}}'
safe_to_import = None

    def _deserializer(obj, safe_to_import=None):
        result = deserializer(obj)
        if result.get('_iterable_opcodes'):
            _iterable_opcodes = {}
            for path, op_codes in result['_iterable_opcodes'].items():
                _iterable_opcodes[path] = []
                for op_code in op_codes:
                    _iterable_opcodes[path].append(
>                       Opcode(
                            **op_code
                        )
                    )
E                   TypeError: deepdiff.helper.Opcode() argument after ** must be a mapping, not list

deepdiff/delta.py:102: TypeError
___ TestDeepDiffText.test_exclude_path_when_prefix_of_exclude_path_matches1 ____

self = <tests.test_diff_text.TestDeepDiffText object at 0x7f834934f440>

    def test_exclude_path_when_prefix_of_exclude_path_matches1(self):
        diff = DeepDiff({}, {'foo': '', 'bar': ''}, exclude_paths=['foo', 'bar'])
>       assert not diff
E       AssertionError: assert not {'values_changed': {'root': {'new_value': {'foo': '', 'bar': ''}, 'old_value': {}}}}

tests/test_diff_text.py:1555: AssertionError
_____________________ TestDeepDiffText.test_bad_attribute ______________________

self = <tests.test_diff_text.TestDeepDiffText object at 0x7f8349355c70>

    def test_bad_attribute(self):
        class Bad:
            __slots__ = ['x', 'y']
    
            def __getattr__(self, key):
                raise AttributeError("Bad item")
    
            def __str__(self):
                return "Bad Object"
    
        t1 = Bad()
        t2 = Bad()
    
        ddiff = DeepDiff(t1, t2)
        result = {'unprocessed': ['root: Bad Object and Bad Object']}
>       assert result == ddiff
E       AssertionError: assert {'unprocessed... Bad Object']} == {}
E         
E         Left contains 1 more item:
E         {'unprocessed': ['root: Bad Object and Bad Object']}
E         Use -v to get more diff

tests/test_diff_text.py:1791: AssertionError
_________ TestDeepDiffText.test_group_by_with_none_key_and_ignore_case _________

self = <tests.test_diff_text.TestDeepDiffText object at 0x7f8349356570>

    def test_group_by_with_none_key_and_ignore_case(self):
        """Test that group_by works with None keys when ignore_string_case is True"""
        dict1 = [{'txt_field': 'FULL_NONE', 'group_id': None}, {'txt_field': 'FULL', 'group_id': 'a'}]
        dict2 = [{'txt_field': 'PARTIAL_NONE', 'group_id': None}, {'txt_field': 'PARTIAL', 'group_id': 'a'}]
    
>       diff = DeepDiff(
            dict1,
            dict2,
            ignore_order=True,
            group_by='group_id',
            ignore_string_case=True
        )

tests/test_diff_text.py:2230: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
deepdiff/diff.py:332: in __init__
    self._diff(root, parents_ids=frozenset({id(t1)}), _original_type=_original_type)
deepdiff/diff.py:1669: in _diff
    self._diff_dict(level, parents_ids, local_tree=local_tree)
deepdiff/diff.py:605: in _diff_dict
    t1_clean_to_keys = self._get_clean_to_keys_mapping(keys=t1_keys, level=level)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = {}, keys = [None, 'a']
level = <root t1:{None: {'txt...}, t2:{None: {'txt...}>

    def _get_clean_to_keys_mapping(self, keys, level):
        """
        Get a dictionary of cleaned value of keys to the keys themselves.
        This is mainly used to transform the keys when the type changes of keys should be ignored.
    
        TODO: needs also some key conversion for groups of types other than the built-in strings and numbers.
        """
        result = dict_()
        for key in keys:
            if self.ignore_string_type_changes and isinstance(key, bytes):
                clean_key = key.decode('utf-8')
            elif self.use_enum_value and isinstance(key, Enum):
                clean_key = key.value
            elif isinstance(key, numbers):
                type_ = "number" if self.ignore_numeric_type_changes else key.__class__.__name__
                clean_key = self.number_to_string(key, significant_digits=self.significant_digits,
                                                  number_format_notation=self.number_format_notation)
                clean_key = KEY_TO_VAL_STR.format(type_, clean_key)
            else:
                clean_key = key
            if self.ignore_string_case:
>               clean_key = clean_key.lower()
E               AttributeError: 'NoneType' object has no attribute 'lower'

deepdiff/diff.py:560: AttributeError
___________ TestDeepDiffText.test_affected_root_keys_when_dict_empty ___________

self = <tests.test_diff_text.TestDeepDiffText object at 0x7f8349356750>

    def test_affected_root_keys_when_dict_empty(self):
        diff = DeepDiff({}, {1:1, 2:2}, threshold_to_diff_deeper=0)
        assert [1, 2] == diff.affected_root_keys
    
        diff2 = DeepDiff({}, {1:1, 2:2})
>       assert [] == diff2.affected_root_keys
E       assert [] == [not present]
E         
E         Right contains one more item: not present
E         Use -v to get more diff

tests/test_diff_text.py:2251: AssertionError
_________________________ TestDeepHashPrep.test_polars _________________________

self = <tests.test_hash.TestDeepHashPrep object at 0x7f8349367530>

    def test_polars(self):
>       import polars as pl
E       ModuleNotFoundError: No module named 'polars'

tests/test_hash.py:795: ModuleNotFoundError
_________ TestSerialization.test_serialization_text_force_builtin_json _________

self = <tests.test_serialization.TestSerialization object at 0x7f83489e9d90>

    def test_serialization_text_force_builtin_json(self):
        ddiff = DeepDiff(t1, t2)
>       with pytest.raises(TypeError) as excinfo:
E       Failed: DID NOT RAISE <class 'TypeError'>

tests/test_serialization.py:52: Failed
_______________ TestDeepDiffPretty.test_namedtuple_seriazliation _______________

self = <tests.test_serialization.TestDeepDiffPretty object at 0x7f8348a291c0>

    def test_namedtuple_seriazliation(self):
        op_code = Opcode(tag="replace", t1_from_index=0, t1_to_index=1, t2_from_index=10, t2_to_index=20)
        serialized = json_dumps(op_code)
        expected = '{"tag":"replace","t1_from_index":0,"t1_to_index":1,"t2_from_index":10,"t2_to_index":20,"old_values":null,"new_values":null}'
>       assert serialized == expected
E       assert '["replace", ..., null, null]' == '{"tag":"repl...values":null}'
E         
E         - {"tag":"replace","t1_from_index":0,"t1_to_index":1,"t2_from_index":10,"t2_to_index":20,"old_values":null,"new_values":null}
E         + ["replace", 0, 1, 10, 20, null, null]

tests/test_serialization.py:412: AssertionError
____________________ TestDeepDiffPretty.test_reversed_list _____________________

self = <tests.test_serialization.TestDeepDiffPretty object at 0x7f8348a28530>

    def test_reversed_list(self):
        items = reversed([1, 2, 3])
    
        serialized = json_dumps(items)
        serialized2 = json_dumps(items)
    
>       assert '[3,2,1]' == serialized
E       AssertionError: assert '[3,2,1]' == '[3, 2, 1]'
E         
E         - [3, 2, 1]
E         ?    -  -
E         + [3,2,1]

tests/test_serialization.py:420: AssertionError
=============================== warnings summary ===============================
tests/test_serialization.py:391
  /build/reproducible-path/deepdiff-8.0.1/.pybuild/cpython3_3.12_deepdiff/build/tests/test_serialization.py:391: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
    (7, datetime.datetime.utcnow(), datetime.datetime.fromisoformat),

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

------------------------------------------------------------------------------------------------ benchmark: 3 tests ------------------------------------------------------------------------------------------------
Name (time in us)                                  Min                 Max               Mean            StdDev             Median               IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_lfu[items0-3-expected_results0-1.333]     14.6970 (1.0)      313.0990 (2.98)     16.1242 (1.0)      6.5514 (1.27)     15.5090 (1.0)      0.5010 (1.0)        92;334       62.0184 (1.0)        7977           1
test_lfu[items1-3-expected_results1-1.666]     17.5230 (1.19)     105.0680 (1.0)      19.5968 (1.22)     5.1401 (1.0)      18.8250 (1.21)     1.3930 (2.78)      287;366       51.0287 (0.82)      25482           1
test_lfu[items2-3-expected_results2-3.333]     23.6950 (1.61)     161.3240 (1.54)     25.4352 (1.58)     5.6073 (1.09)     24.8160 (1.60)     0.5810 (1.16)      247;987       39.3155 (0.63)      23376           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean
=========================== short test summary info ============================
FAILED tests/test_delta.py::TestDeltaCompareFunc::test_list_of_alphabet_and_its_delta
FAILED tests/test_diff_text.py::TestDeepDiffText::test_exclude_path_when_prefix_of_exclude_path_matches1
FAILED tests/test_diff_text.py::TestDeepDiffText::test_bad_attribute - Assert...
FAILED tests/test_diff_text.py::TestDeepDiffText::test_group_by_with_none_key_and_ignore_case
FAILED tests/test_diff_text.py::TestDeepDiffText::test_affected_root_keys_when_dict_empty
FAILED tests/test_hash.py::TestDeepHashPrep::test_polars - ModuleNotFoundErro...
FAILED tests/test_serialization.py::TestSerialization::test_serialization_text_force_builtin_json
FAILED tests/test_serialization.py::TestDeepDiffPretty::test_namedtuple_seriazliation
FAILED tests/test_serialization.py::TestDeepDiffPretty::test_reversed_list - ...
============= 9 failed, 937 passed, 10 skipped, 1 warning in 4.74s =============

Please ignore the polars package error. Any help or clue is welcome :)

[seperman]

Hi @eamanu
All these tests pass for me on Ubuntu 24.04.1 and Python 3.13.
What version of Debian are you using?

[eamanu]

Hi @seperman

Thanks for your response. I'm in Debian sid. Now, as you merged I will wait the release to package it.

I suspect that I missed something while patching the PR.

[eamanu]

Hi @seperman,

The issue was because orjson is a dependency. In despite that deepdiff has a import error manage, the tests need it, if not they fail.

[seperman]

Hi @eamanu
Yeah, you need to install the requirements-dev.txt before running the tests. You can take a look at the github actions that run the tests.