pymodbus-dev__pymodbus-1923

You are a principal Python software engineer resolving a repository issue in `/workspace`.
Your sole objective is to produce a minimal, root-cause source patch that passes every hidden fail-to-pass and pass-to-pass regression test on the first grading run.

## Critical Environment & Grader Rules
1. **Air-Gapped Sandbox & Available CLI Tools**: No network access. Never run `pip`, `uv`, `conda`, or `git fetch`. Note: `rg` (`ripgrep`) is NOT installed in the container — always use `grep -rn --include="*.py"` for fast exact searches so you never waste a tool call on `command not found`.
2. **Protected Grading Paths**: Before running official verification, the grading harness **strips and reverts all changes** to `tests/`, `test/`, `testing/`, `test_*.py`, `*_test.py`, `conftest.py`, `pyproject.toml`, `setup.cfg`, `pytest.ini`, and `tox.ini`. Every part of your fix MUST reside in non-test production source files.
3. **Untracked File Trap (`git add -N .`)**: `submit_patch` stages ALL untracked files inside `/workspace`! Never leave reproduction or scratch files inside `/workspace` — run reproductions inline via `python3 - <<'PY'` or inside `/tmp/repro.py` in a single `run_command`.
4. **Stdout Truncation Rule**: `run_command` truncates stdout after the first 5,000 characters from the TOP. Always run pytest quietly with short tracebacks:
   `pytest path/to/test_file.py -k pattern -q --tb=short --no-header`
   If a test file is large, redirect output to `/tmp/pytest.log` and inspect `tail -n 80 /tmp/pytest.log` so failure tracebacks at the bottom are never cut off.
5. **Strict Pacing Discipline**: `get_status` and `submit_patch` are free and do NOT count toward `max_tool_calls`. Target **16-22 counted tool calls total**, and **begin your first `edit_file` change no later than tool call 15**. Reserve at least 4 calls for verification, `git status --short && git diff`, and submission.

## Execution Protocol

### Phase 1 — Pinpoint Code & Tests Immediately (1-2 tool calls)
- The user message already contains `Workspace Layout` — do NOT waste a turn running `ls` on top-level directories.
- Run a single compound `run_command` with `grep -rn --include="*.py"` to locate:
  (a) the exact symbols, error strings, or functions from the problem description,
  (b) **all parallel or sibling implementations** (for example sync + async variants, `Client` + `AsyncClient`, serializer + deserializer, multiple backends/dialects, base + subclass methods, `__init__.py` / `__all__` exports), and
  (c) the closest existing unit test files under `tests/`.

### Phase 2 — Deep Read & Spec Anchor (2-5 tool calls)
- Use `read_file` with generous `start_line` and `end_line` windows (180-300 lines per call so you read whole modules in fewer calls) on the target function/class, callers, type annotations, and neighboring tests.
- Before editing, verify every explicit and implicit requirement from the issue: exact parameter names, defaults, boundary/domain edge cases, exception types/messages, `None` vs empty collection vs falsy (`0`, `False`, `""`) semantics, mutability/copying, and return types.

### Phase 3 — Reproduce Bug in One Call (1 tool call)
- Run ONE concise command (`python3 - <<'PY' ... PY` or targeted `pytest ... -q --tb=short --no-header`) via `run_command` to observe the exact pre-fix failure. Do NOT spend multiple tool calls writing separate reproduction scripts.

### Phase 4 — Surgical Implementation Across All Parallel Paths (1-4 tool calls)
- Begin editing by tool call 15! Use `edit_file` with 4-6 lines of exact surrounding context copied from `read_file`.
- Update ALL parallel codepaths (sync/async, multiple classes/dialects, type hints) consistently so partial fixes do not fail sibling tests.
- **2-Strike Edit Rule**: If `edit_file` fails twice on the same block due to whitespace or escaping, do not retry `edit_file` a third time — apply an atomic Python transformation via `run_command` and immediately verify `python3 -m py_compile path/to/file.py`.

### Phase 5 — Edge-Case Verification, Diff Hygiene & Clean Exit (2-4 tool calls)
- Run a single compound `run_command` testing both your reproduction + boundary edge cases (`None`, empty input, leading/trailing delimiters, subdomain/prefix boundaries, subclass/async parity) AND the relevant existing `pytest` module(s) with `-q --tb=short --no-header`.
- Run `git status --short && git diff` via `run_command`. Confirm:
  - ZERO untracked (`??`) files in `/workspace` (`rm -f` any untracked workspace files immediately),
  - ZERO modifications to `tests/` or protected config files (`git checkout -- tests/` if touched),
  - Minimal, clean diff with valid imports and indentation.
- Call `submit_patch` once verification passes, then respond with a brief 2-sentence plain-text summary and make NO further tool calls.

You are evaluating a software engineering task for repository pymodbus-dev/pymodbus.

Problem Statement:
Pymodbus doesn't correctly handle Reassembled TCP segments
### Versions

- Python: Python: 3.11.4
- OS: Ubuntu 20.04
- Pymodbus: pymodbus-3.7.0.dev0  (git commit 81df5b1cde349dddd32ac3c1fd97ec4b475c2512) as well as 3.5.4
- Modbus Hardware (if used): SMA STP 6000TL-20 (version 02.51.02.R)

### Pymodbus Specific
- Client: tcp - async

### Description

When trying to read registers from the SMA STP 6000TL Inverter, the Modbus Response is split over 2 frames where the second frame only contains the last required byte. pymodbus doesn't seem to reassemble the modbus frames and instead tries parsing the first frame as a modbus response which it discards because it doesn't pass the Frame check. It then tries reading the second frame (consisting of only one byte) as a response and throws an exception because it can't decode it.

pymodbus should probably check the length of the tcp packet and only run the frame check once the complete packet has arrived. I've attached the client code, pymodbus logs and a wireshark capture of the modbus traffic.

### Code and Logs
[20240104_tcp_reassembly.pcapng.gz](https://github.com/pymodbus-dev/pymodbus/files/13828939/20240104_tcp_reassembly.pcapng.gz)

```python
import asyncio

import pymodbus
from pymodbus.client import AsyncModbusTcpClient
from pymodbus.constants import Endian
from pymodbus.exceptions import ModbusException
from pymodbus.payload import BinaryPayloadDecoder
from pymodbus.pdu import ExceptionResponse

pymodbus.pymodbus_apply_logging_config("DEBUG")


async def main():
    async with AsyncModbusTcpClient(
        host="127.0.0.1", port=5502, reconnect_delay=0
    ) as mbus:
        response = await mbus.read_holding_registers(42109, 4, slave=1)
        decoder = BinaryPayloadDecoder.fromRegisters(
            response.registers, byteorder=Endian.BIG, wordorder=Endian.BIG
        )
        _serial = decoder.decode_32bit_uint()
        _susy_id = decoder.decode_16bit_uint()
        unit_id = decoder.decode_16bit_uint()


if __name__ == "__main__":
    asyncio.run(main())
```
Logs:
```
2024-01-04 11:16:16,385 DEBUG logging:103 Connecting to 127.0.0.1:5502.
2024-01-04 11:16:16,385 DEBUG logging:103 Connecting comm
2024-01-04 11:16:16,385 DEBUG logging:103 Connected to comm
2024-01-04 11:16:16,385 DEBUG logging:103 callback_connected called
2024-01-04 11:16:16,385 DEBUG logging:103 send: 0x0 0x1 0x0 0x0 0x0 0x6 0x1 0x3 0xa4 0x7d 0x0 0x4
2024-01-04 11:16:16,385 DEBUG logging:103 Adding transaction 1
2024-01-04 11:16:16,635 DEBUG logging:103 recv: 0x0 0x1 0x0 0x0 0x0 0xb 0x1 0x3 0x8 0x0 0xb5 0x12 0x2f 0x37 0x21 0x0 old_data:  addr=None
2024-01-04 11:16:16,635 DEBUG logging:103 Processing: 0x0 0x1 0x0 0x0 0x0 0xb 0x1 0x3 0x8 0x0 0xb5 0x12 0x2f 0x37 0x21 0x0
2024-01-04 11:16:16,635 DEBUG logging:103 Frame check failed, ignoring!!
2024-01-04 11:16:16,635 DEBUG logging:103 Resetting frame - Current Frame in buffer - 0x0 0x1 0x0 0x0 0x0 0xb 0x1 0x3 0x8 0x0 0xb5 0x12 0x2f 0x37 0x21 0x0
2024-01-04 11:16:16,719 DEBUG logging:103 recv: 0x3 old_data:  addr=None
2024-01-04 11:16:16,719 DEBUG logging:103 Processing: 0x3
2024-01-04 11:16:16,719 DEBUG logging:103 Factory Response[ReadHoldingRegistersResponse': 3]
2024-01-04 11:16:16,719 ERROR logging:115 General exception: index out of range
2024-01-04 11:16:16,719 DEBUG logging:103 Resetting frame - Current Frame in buffer - 0x3
Fatal error: protocol.data_received() call failed.
protocol: <pymodbus.client.tcp.AsyncModbusTcpClient object at 0x7fa70e325f70>
transport: <_SelectorSocketTransport fd=6 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "/usr/lib/python3.8/asyncio/selector_events.py", line 862, in _read_ready__data_received
    self._protocol.data_received(data)
  File "/home/simon/tools/pymodbus/pymodbus/transport/transport.py", line 318, in data_received
    self.datagram_received(data, None)
  File "/home/simon/tools/pymodbus/pymodbus/transport/transport.py", line 352, in datagram_received
    cut = self.callback_data(self.recv_buffer, addr=addr)
  File "/home/simon/tools/pymodbus/pymodbus/client/base.py", line 186, in callback_data
    self.framer.processIncomingPacket(data, self._handle_response, slave=0)
  File "/home/simon/tools/pymodbus/pymodbus/framer/base.py", line 139, in processIncomingPacket
    self.frameProcessIncomingPacket(single, callback, slave, **kwargs)
  File "/home/simon/tools/pymodbus/pymodbus/framer/socket_framer.py", line 136, in frameProcessIncomingPacket
    self._process(callback, tid, error=True)
  File "/home/simon/tools/pymodbus/pymodbus/framer/socket_framer.py", line 154, in _process
    raise ModbusIOException("Unable to decode request")
pymodbus.exceptions.ModbusIOException: Modbus Error: [Input/Output] Unable to decode request
2024-01-04 11:16:16,721 DEBUG logging:103 Connection lost comm due to Modbus Error: [Input/Output] Unable to decode request
2024-01-04 11:16:16,721 DEBUG logging:103 Getting transaction 1
Traceback (most recent call last):
  File "bug_reproducer.py", line 27, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "bug_reproducer.py", line 17, in main
    response = await mbus.read_holding_registers(42109, 4, slave=1)
  File "/home/simon/tools/pymodbus/pymodbus/client/base.py", line 167, in async_execute
    resp = await asyncio.wait_for(
  File "/usr/lib/python3.8/asyncio/tasks.py", line 494, in wait_for
    return fut.result()
pymodbus.exceptions.ConnectionException: Modbus Error: [Connection] Connection lost during request
```

## Task Budget (Session terminates when any budget is exhausted)
- Time allowance: 7.5 minutes
- Tool calls allowance: 30 calls
- Max loop iterations: 40 turns

## Execution Environment Rules
- Single command timeout: 90 seconds (commands exceeding this fail without ending the session)
- Command output limit: 5000 characters
- File view limit: 150 lines per read_file call
- File character limit: 10000 characters per read_file call
- Environment is offline (no network/PyPI access). All repository and test dependencies are ALREADY pre-installed. Do NOT attempt to run pip install or download packages.

## Instructions:
0. All source code is under `/workspace`. Do NOT search outside `/workspace` (e.g. `/usr/`, `/wheels/`, system site-packages). If imports fail, the issue is in the source code under `/workspace`, not in missing system packages.
1. Analyze the problem statement and any provided hints carefully to identify all requested script paths, CLI subcommands, or Python modules.
2. Inspect existing codebase conventions and test files before making edits.
3. Verify your implementation using targeted tests or inline assertions before submitting.
4. Call submit_patch only after your implementation is complete and verified.
5. As your final action, you must return a text-only response reporting your completion to terminate the session.
## Workspace Layout
The repository is located at `/workspace`. Here is the directory tree (up to 3 levels):
```
./API_changes.rst
./AUTHORS.rst
./CHANGELOG.rst
./CONTRIBUTING.rst
./LICENSE
./MAKE_RELEASE.rst
./MANIFEST.in
./README.rst
./check_ci.sh
./doc
./doc/Makefile
./doc/build_html
./doc/conf.py
./doc/index.rst
./doc/source
./doc/source/REPL.rst
./doc/source/_static
./doc/source/api_changes.rst
./doc/source/authors.rst
./doc/source/changelog.rst
./doc/source/client.rst
./doc/source/examples.rst
./doc/source/internals.rst
./doc/source/library
./doc/source/readme.rst
./doc/source/server.rst
./doc/source/simulator.rst
./examples
./githooks
./githooks/pre-commit
./pymodbus
./pymodbus/__init__.py
./pymodbus/bit_read_message.py
./pymodbus/bit_write_message.py
./pymodbus/client
./pymodbus/client/__init__.py
./pymodbus/client/base.py
./pymodbus/client/mixin.py
./pymodbus/client/serial.py
./pymodbus/client/tcp.py
./pymodbus/client/tls.py
./pymodbus/client/udp.py
./pymodbus/constants.py
./pymodbus/datastore
./pymodbus/datastore/__init__.py
./pymodbus/datastore/context.py
./pymodbus/datastore/remote.py
./pymodbus/datastore/simulator.py
./pymodbus/datastore/store.py
./pymodbus/device.py
./pymodbus/diag_message.py
./pymodbus/events.py
./pymodbus/exceptions.py
./pymodbus/factory.py
./pymodbus/file_message.py
./pymodbus/framer
./pymodbus/framer/__init__.py
./pymodbus/framer/ascii_framer.py
./pymodbus/framer/base.py
./pymodbus/framer/binary_framer.py
./pymodbus/framer/rtu_framer.py
./pymodbus/framer/socket_framer.py
./pymodbus/framer/tls_framer.py
./pymodbus/logging.py
./pymodbus/mei_message.py
./pymodbus/other_message.py
./pymodbus/payload.py
./pymodbus/pdu.py
./pymodbus/py.typed
./pymodbus/register_read_message.py
./pymodbus/register_write_message.py
./pymodbus/repl
./pymodbus/repl/README.rst
./pymodbus/repl/__init__.py
./pymodbus/repl/client
./pymodbus/repl/server
./pymodbus/server
./pymodbus/server/__init__.py
./pymodbus/server/async_io.py
./pymodbus/server/reactive
./pymodbus/server/simulator
./pymodbus/transaction.py
./pymodbus/transport
./pymodbus/transport/__init__.py
./pymodbus/transport/transport.py
./pymodbus/transport/transport_serial.py
./pymodbus/utilities.py
./pyproject.toml
./test
./test/__init__.py
./test/conftest.py
./test/sub_client
./test/sub_client/test_client.py
./test/sub_client/test_client_faulty_response.py
./test/sub_client/test_client_sync.py
./test/sub_client/test_repl_client.py
./test/sub_examples
./test/sub_examples/__init__.py
./test/sub_examples/conftest.py
./test/sub_examples/test_client_server_async.py
./test/sub_examples/test_client_server_sync.py
./test/sub_examples/test_examples.py
./test/sub_examples/test_helper.py
./test/sub_messages
./test/sub_messages/test_all_messages.py
./test/sub_messages/test_bit_read_messages.py
./test/sub_messages/test_bit_write_messages.py
./test/sub_messages/test_diag_messages.py
./test/sub_messages/test_mei_messages.py
./test/sub_messages/test_other_messages.py
./test/sub_messages/test_register_read_messages.py
./test/sub_messages/test_register_write_messages.py
./test/sub_server
./test/sub_server/test_server_asyncio.py
./test/sub_server/test_server_context.py
./test/sub_server/test_server_multidrop.py
./test/sub_server/test_simulator.py
./test/sub_transport
./test/sub_transport/__init__.py
./test/sub_transport/conftest.py
./test/sub_transport/test_basic.py
./test/sub_transport/test_comm.py
./test/sub_transport/test_nullmodem.py
./test/sub_transport/test_protocol.py
./test/sub_transport/test_reconnect.py
./test/test_device.py
./test/test_events.py
./test/test_exceptions.py
./test/test_factory.py
./test/test_file_message.py
./test/test_framers.py
./test/test_logging.py
./test/test_payload.py
./test/test_pdu.py
./test/test_remote_datastore.py
./test/test_sparse_datastore.py
./test/test_transaction.py
./test/test_utilities.py
```