[meta-xilinx] Linux/FreeRTOS AMP vrings

[Edward Wingate]

I wanted to add some possibly relevant information.  To get AMP and
rpmsg/vrings to work with Linux and FreeRTOS, I started working from
the AMP example in ug978.  In remoteproc_kernel.h, there's is a
VRING_ADDR_MASK #defined to 0xFFFFFF.  This is used to mask the MSB of
the vring data buffer address before being used by read_message() or
send_message().  I didn't understand why this was needed and it
wouldn't have worked with the address range I wanted to put the vrings
at (0x1e40xxxx).  CPU1 would have tried to read/write 0x400000.  For
now, I kept the mask but set it to 0xFFFFFFFF and basically turned it
into a noop.

But I still don't understand why the physical address of the vring
data buffer needed to be modified in ug978 before use by CPU1.  Can
anyone clarify?  Just to get it working on  my system, I could
probably subtract 0x01000000 from the data buffer address so that CPU1
writes data to the same place Linux is reading vring data from.  But
I'd like to understand what's going on.  Is some manipulation of the
data buffer physical address expected?  (ug978 masked off the MSB, I
need to subtract 0x01000000...)



On Thu, Aug 27, 2015 at 1:15 PM, Edward Wingate <edwingate8 at gmail.com> wrote:
> I'm still having some vring sync issues between Linux (CPU0) and CPU1.
> I've gotten the vring descriptors to line up, but the data buffer
> addresses in the descriptors are not in sync.  CPU1 is writing vring0
> data to 0x1f440000, but Linux wants to read vring0 data from
> 0x1e440000.  I'm not sure where/how to configure them to match.
>
> On my system, Linux is mapping 32MB of RAM at 0x1e000000 is to virtual
> address 0x9f000000.
> From /proc/vmallocinfo:
> 0x9f000000-0xa1001000 33558528 dma_declare_coherent_memory+0x58/0xfc
> phys=1e000000 ioremap
>
> The vring0 descriptor locations are in sync; both Linux and CPU1 are
> accessing them at physical address 0x1e400000 (Linux VA 0x9f400000).
> Linux appears to have the vring0 data buffers at 0x9f440000, physical
> address 0x1e440000.
>
> However, the vring0 descriptors are initialized with data buffer
> addresses of 0x1f440000, 0x1f440200, 0x1f440400, etc, instead of
> 0x1e440xxx.
>
> So when CPU1 sends a message to Linux, it'll write the data to
> 0x1f440xxx.  But when Linux gets the kick and retrieves the message,
> it does so at 0x9f4400xxx, which is physical address 0x1e440xxx, not
> 0x1f440xxx.
>
> Why are the vring descriptors initialized with address 0x1f440xxx, but
> Linux tries to read 0x9f440xxx, which is physical address 0x1e440xxx?
> It seems like either the vring descriptor addresses should be
> initialized with 0x1e440xxx, or Linux should be reading at VA
> 0xA04400xxx. Where does initialization of vring descriptors take place
> in the kernel (so I can try to see what's going on)?
>
> I'd appreciate any help or information on where to look/debug next.
> Below are remoteproc/vring log output with the addresses I mentioned
> above.
>
> Log output:
> remoteproc0: carveout va 9f000000, dma 1e000000, len 0x400000
> remoteproc0: vdev rsc: id 7, dfeatures 1, cfg len 0, 2 vrings
> remoteproc0: vdev rsc: vring0: da 1e400000, qsz 256, align 4096
> remoteproc0: vdev rsc: vring1: da 1e404000, qsz 256, align 4096
> remoteproc0: vring0: va 9f400000 dma 1e400000 size 3000 idr 0
> remoteproc0: vring0: va 9f400000 qsz 256 notifyid 0
> remoteproc0: vring1: va 9f404000 dma 1e404000 size 3000 idr 1
> remoteproc0: vring1: va 9f404000 qsz 256 notifyid 1
> virtio_rpmsg_bus virtio0: buffers: va 9f440000, dma 0x1e440000
>