refactor(network): Remove dynamic allocations for NetPacket#2866
refactor(network): Remove dynamic allocations for NetPacket#2866Caball009 wants to merge 4 commits into
Conversation
|
| Filename | Overview |
|---|---|
| Core/GameEngine/Source/GameNetwork/NetPacket.cpp | Refactored ConstructBigCommandPacketList → ConstructBigCommandList. Contains a correctness regression: all chunk refs share the same wrapperMsg pointer, so only the last chunk survives the list's deduplication logic for any command requiring ≥2 chunks. |
| Core/GameEngine/Include/GameNetwork/NetPacket.h | Removes MemoryPoolObject base and related typedefs; changes constructor signature to const TransportMessage&. Clean header changes, no issues. |
| Core/GameEngine/Source/GameNetwork/Connection.cpp | Replaces pointer-based NetPacket usage with stack-allocated instances in sendNetCommandMsg and doSend. Logic is equivalent to the old code; addMessage nullptr guard added correctly. |
| Core/GameEngine/Source/GameNetwork/ConnectionManager.cpp | Replaces heap-allocated NetPacket with a stack-local in doRelay; removes now-unnecessary end-of-function cleanup for packet/cmdList. The outer cmdList (from getReadyCommands) remains correctly scoped and deleted. |
| Core/GameEngine/Source/Common/System/GameMemoryInitPools_Generals.inl | Removes NetPacket pool entry, consistent with removing MemoryPoolObject inheritance. |
| Core/GameEngine/Source/Common/System/GameMemoryInitPools_GeneralsMD.inl | Removes NetPacket pool entry, consistent with removing MemoryPoolObject inheritance. |
Sequence Diagram
sequenceDiagram
participant C as Connection
participant NCL as NetCommandList (returned)
participant WM as wrapperMsg (single object)
C->>NCL: ConstructBigCommandList(tempref)
Note over NCL,WM: Loop iteration 1
WM->>WM: setID(ID_1), setData(chunk0)
NCL->>NCL: addMessage(chunkRef1 → wrapperMsg) ✓ inserted
Note over NCL,WM: Loop iteration 2
WM->>WM: setID(ID_2), setData(chunk1)
Note over NCL,WM: chunkRef1 now also shows ID_2 (same object!)
NCL->>NCL: addMessage(chunkRef2 → wrapperMsg) ✗ duplicate detected → deleted
Note over NCL,WM: Loop iteration N
WM->>WM: setID(ID_N), setData(chunkN-1)
NCL->>NCL: addMessage(chunkRefN → wrapperMsg) ✗ duplicate detected → deleted
NCL-->>C: list with 1 entry (last chunk data only)
C->>C: m_netCommandList.addMessage(wrapperMsg) — only last chunk queued
Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 1
Core/GameEngine/Source/GameNetwork/NetPacket.cpp:108-138
**All chunk refs share the same mutable `wrapperMsg` — big commands silently truncated to one chunk**
Every `chunkRef` created in the loop points to the *same* `wrapperMsg` object. Because `wrapperMsg` is mutated in place on each iteration (new command ID, new chunk number, new data), all refs that were already inserted into `commandList` immediately observe the new state. The deduplication logic in `addMessage(NetCommandRef*&)` compares `m_lastMessageInserted->getCommand()` and `msg->getCommand()`, but both are literally the same pointer, so every chunk after the first is discarded as a "duplicate" and its `chunkRef` is deleted. The list ends up with a single entry holding the last chunk's data and the last generated command ID.
The old code worked because `addCommand` serialized each chunk's state into a raw byte buffer inside a per-chunk `NetPacket`, and the caller then called `getCommandList()` to deserialize independent `NetCommandMsg` objects from those bytes. The new path eliminates that serialization round-trip, but must instead allocate a distinct `NetWrapperCommandMsg` per chunk (so each ref owns its own snapshot of the state) to preserve correctness.
Any game command large enough to span two or more network chunks will be silently dropped to its last piece, breaking reassembly on the receiver.
Reviews (8): Last reviewed commit: "Added nullptr check." | Re-trigger Greptile
|
What is the motivation for removing memory pool? The main purpose of memory pool is cache and defrag friendlyness. |
The purpose of a memory pool is to mitigate the downsides of frequent dynamic allocations. It's unnecessary if you don't need dynamic allocations. |
|
But it is still allocated with |
9b43034 to
e1fead4
Compare
Rewritten so that this is no longer the case. |
|
@greptileai re-review this pull request. |
e1fead4 to
757d5cf
Compare
757d5cf to
e396279
Compare
|
Needs rebase and todo. |
3085796 to
27c0487
Compare
| while (currentChunk < numChunks) { | ||
| NetPacket *packet = newInstance(NetPacket); | ||
|
|
||
| UnsignedInt dataSizeThisPacket = commandSizePerPacket; | ||
| if ((bufferSize - bigPacketCurrentOffset) < dataSizeThisPacket) { | ||
| dataSizeThisPacket = bufferSize - bigPacketCurrentOffset; | ||
| UnsignedInt dataSizeThisChunk = commandSizePerPacket; | ||
| if ((bufferSize - bigPacketCurrentOffset) < dataSizeThisChunk) { | ||
| dataSizeThisChunk = bufferSize - bigPacketCurrentOffset; | ||
| } | ||
| NetCommandDataChunk bigPacket(dataSizeThisPacket); | ||
| memcpy(bigPacket.data(), bigPacketData + bigPacketCurrentOffset, bigPacket.size()); | ||
|
|
||
| NetCommandDataChunk chunkData(dataSizeThisChunk); | ||
| memcpy(chunkData.data(), bigPacketData + bigPacketCurrentOffset, chunkData.size()); | ||
|
|
||
| if (DoesCommandRequireACommandID(wrapperMsg->getNetCommandType())) { | ||
| wrapperMsg->setID(GenerateNextCommandID()); | ||
| } | ||
|
|
||
| wrapperMsg->setPlayerID(msg->getPlayerID()); | ||
| wrapperMsg->setExecutionFrame(msg->getExecutionFrame()); | ||
|
|
||
| wrapperMsg->setChunkNumber(currentChunk); | ||
| wrapperMsg->setNumChunks(numChunks); | ||
| wrapperMsg->setDataOffset(bigPacketCurrentOffset); | ||
| wrapperMsg->setData(bigPacket); | ||
| wrapperMsg->setData(chunkData); | ||
| wrapperMsg->setTotalDataLength(bufferSize); | ||
| wrapperMsg->setWrappedCommandID(msg->getID()); | ||
|
|
||
| bigPacketCurrentOffset += dataSizeThisPacket; | ||
|
|
||
| NetCommandRef *ref = NEW_NETCOMMANDREF(wrapperMsg); | ||
| ref->setRelay(ref->getRelay()); | ||
|
|
||
| if (packet->addCommand(ref) == FALSE) { | ||
| DEBUG_LOG_LEVEL(DEBUG_LEVEL_NET, ("NetPacket::BeginBigCommandPacketList - failed to add a wrapper command to the packet")); // I still have a drinking problem. | ||
| } | ||
| bigPacketCurrentOffset += dataSizeThisChunk; | ||
|
|
||
| packetList.push_back(packet); | ||
|
|
||
| deleteInstance(ref); | ||
| ref = nullptr; | ||
| NetCommandRef* chunkRef = NEW_NETCOMMANDREF(wrapperMsg); | ||
| chunkRef->setRelay(ref->getRelay()); | ||
|
|
||
| commandList->addMessage(chunkRef); | ||
| ++currentChunk; | ||
| } |
There was a problem hiding this comment.
All chunk refs share the same mutable
wrapperMsg — big commands silently truncated to one chunk
Every chunkRef created in the loop points to the same wrapperMsg object. Because wrapperMsg is mutated in place on each iteration (new command ID, new chunk number, new data), all refs that were already inserted into commandList immediately observe the new state. The deduplication logic in addMessage(NetCommandRef*&) compares m_lastMessageInserted->getCommand() and msg->getCommand(), but both are literally the same pointer, so every chunk after the first is discarded as a "duplicate" and its chunkRef is deleted. The list ends up with a single entry holding the last chunk's data and the last generated command ID.
The old code worked because addCommand serialized each chunk's state into a raw byte buffer inside a per-chunk NetPacket, and the caller then called getCommandList() to deserialize independent NetCommandMsg objects from those bytes. The new path eliminates that serialization round-trip, but must instead allocate a distinct NetWrapperCommandMsg per chunk (so each ref owns its own snapshot of the state) to preserve correctness.
Any game command large enough to span two or more network chunks will be silently dropped to its last piece, breaking reassembly on the receiver.
Prompt To Fix With AI
This is a comment left during a code review.
Path: Core/GameEngine/Source/GameNetwork/NetPacket.cpp
Line: 108-138
Comment:
**All chunk refs share the same mutable `wrapperMsg` — big commands silently truncated to one chunk**
Every `chunkRef` created in the loop points to the *same* `wrapperMsg` object. Because `wrapperMsg` is mutated in place on each iteration (new command ID, new chunk number, new data), all refs that were already inserted into `commandList` immediately observe the new state. The deduplication logic in `addMessage(NetCommandRef*&)` compares `m_lastMessageInserted->getCommand()` and `msg->getCommand()`, but both are literally the same pointer, so every chunk after the first is discarded as a "duplicate" and its `chunkRef` is deleted. The list ends up with a single entry holding the last chunk's data and the last generated command ID.
The old code worked because `addCommand` serialized each chunk's state into a raw byte buffer inside a per-chunk `NetPacket`, and the caller then called `getCommandList()` to deserialize independent `NetCommandMsg` objects from those bytes. The new path eliminates that serialization round-trip, but must instead allocate a distinct `NetWrapperCommandMsg` per chunk (so each ref owns its own snapshot of the state) to preserve correctness.
Any game command large enough to span two or more network chunks will be silently dropped to its last piece, breaking reassembly on the receiver.
How can I resolve this? If you propose a fix, please make it concise.
This PR makes two changes to remove all dynamic allocations for
NetPacket:NetPacket::ConstructBigCommandListso thatNetPacketdoesn't have to be memory pooled.NetPacket, so that it can be stack allocated.TODO: