Python API about the Shared Memory Transport #42

Activity

1. duchengyao commented on Oct 26, 2022

   

   duchengyao

   on Oct 26, 2022

   More actions

   Did you find a solution >_<
2. jsan-rt commented on Oct 26, 2022

   

   jsan-rt

   on Oct 26, 2022

   Contributor

   More actions

   Hi @cdp-study , @duchengyao ,

   At this moment it's not possible to define the user transports using the Python API. However, by default Fast DDS uses both an UDPv4 transport and a Shared Memory transport, the latter being preferred for data transfer if communication is possible.

   Could you provide us with some more information (like the IDL used to generate the type) or a minimal reproducer?
3. jsan-rt

   added

   need more info

   on Oct 26, 2022
4. duchengyao commented on Oct 27, 2022

   

   duchengyao

   on Oct 27, 2022

   Last edited by duchengyao

   More actions

   @jsantiago-eProsima Thanks for reply.

   For example, I'd like to receive data from example/HelloWorldExampleSharedMem .

   1. I modified HelloWorld.idl #L5 to char data[1048576]; because of this issue
   2. Generated and build using fastddsgen -python HelloWorld.idl mkdir build; cmake ..; make
   3. Copied HelloWorldExample.py to build folder. And modified the topic name to "HelloWorldSharedMemTopic", "HelloWorld".

   Then, I run publisher of C++ and subscriber of python. The subscriber cannot receive any data.

   But after delete this line pqos.transport().use_builtin_transports = false; #L57 ,the subscriber could receive data.

   So, maybe the python binding cannot receive data of shared memory.
   Is there the way to receive data of HelloWorldExampleSharedMem without modify the code of C++?

   Thanks!
5. jsan-rt commented on Oct 27, 2022

   

   jsan-rt

   on Oct 27, 2022

   Contributor

   More actions

   You could use an XML profile like the one shown here to setup the available descriptors for your participant. This XML can be loaded either from a file or from a string.

   If the profile is not set as default, you can specify it manually on participant creation with the create_participant_with_profile call.

   👍React with 👍1Reacted by S1NH

   🎉React with 🎉1Reacted by S1NH
6. duchengyao commented on Nov 15, 2022

   

   duchengyao

   on Nov 15, 2022

   Last edited by duchengyao

   More actions

   @jsantiago-eProsima I have successfully used shared memory via XML, thanks!

   BTW,
   Converting SWIG type to numpy or list will consume a lot of time. I can't find the solution about that, could you please provide an example, thank you ^_^

   python:
   
   data.raw_data()
   >>    <...;proxy of <Swig Object of type 'std::array< float,512*16000 > *' at 0x7fb...> >
   
   a = list(data.raw_data())        # cost long time
   b = np.array(data.raw_data())    # cost long time
   

   And I try using ctypes to accelerate speed of converting data.

   a = np.frombuffer((ctypes.c_float*512*16000).from_address(0x7fb..),np.float32)
   

   But I cannot get the address using variable. Can I get the address like data.raw_data().get_address() ?
7. JesusPoderoso commented on Nov 6, 2023

   

   JesusPoderoso

   on Nov 6, 2023

   Last edited by JesusPoderoso

   Contributor

   More actions

   Hi @duchengyao, sorry for the late response. The data conversion has been improved in this PR, which introduces a get_buffer() method that returns the data buffer. The time cost reduction is significant, but its usage is quite similar to the solution you reached:

   a = np.frombuffer((ctypes.c_float*512*16000).from_address(int(data.get_buffer())),np.float32)
   

   But I cannot get the address using variable. Can I get the address like data.raw_data().get_address() ?

   Regarding this, the get_buffer() method can be used on the defined types: data.address().get_buffer().
8. JesusPoderoso commented on Nov 27, 2023

   

   JesusPoderoso

   on Nov 27, 2023

   Contributor

   More actions

   I am moving this issue to the support discussion forum as per our CONTRIBUTING guidelines.
9. eProsima

   locked and limited conversation to collaborators on Nov 27, 2023
10. JesusPoderoso

    converted this issue into a discussion #72 on Nov 27, 2023