[meta-xilinx] Linux/FreeRTOS AMP vrings

[Edward Wingate]

On Wed, Sep 2, 2015 at 1:26 PM, Wendy Liang <wendy.liang at xilinx.com> wrote:
> rpmsg_probe() from drivers/rpmsg/virtio_rpmsg_bus.c will allocate the buffers, and in the end
> it will call virtqueue_add() from drivers/virtio/virtio_ring.c to set the addr.
> You can add some print into those functions to see what value from Linux side it get.

I already did so in both linux-xlnx 3.14 and 3.19 and things are as
described in my previous message.
The 3 prints I added/looked at are:

1) Existing print in rpmsg_probe() showing vring buffer allocation at:
virtio_rpmsg_bus virtio0: buffers: va 9f440000, dma 0x1e440000

2) Here is the loop in rpmsg_probe() that calls virtqueue_add():
for (i = 0; i < vrp->num_bufs / 2; i++) {
    struct scatterlist sg;
    cpu_addr = vrp->rbufs + i * RPMSG_BUF_SIZE;
    cpu_addr_dma = rbufs_guest_addr_kva + i*RPMSG_BUF_SIZE;

    dev_dbg(&vdev->dev, "buf %d, cpu_addr %p, cpu_addr_dma, %p\n", i,
cpu_addr, cpu_addr_dma);
    sg_init_one(&sg, cpu_addr_dma, RPMSG_BUF_SIZE);

    err = virtqueue_add_inbuf(vrp->rvq, &sg, 1, cpu_addr,
                            GFP_KERNEL);
    WARN_ON(err); /* sanity check; this can't really happen */
}

I added the dev_dbg() print and this is the result:
virtio_rpmsg_bus virtio0: buf 0, cpu_addr 9f440000, cpu_addr_dma, 9f440000
virtio_rpmsg_bus virtio0: buf 1, cpu_addr 9f440200, cpu_addr_dma, 9f440200
virtio_rpmsg_bus virtio0: buf 2, cpu_addr 9f440400, cpu_addr_dma, 9f440400
... [snipped]
virtio_rpmsg_bus virtio0: buf 253, cpu_addr 9f45fa00, cpu_addr_dma, 9f45fa00
virtio_rpmsg_bus virtio0: buf 254, cpu_addr 9f45fc00, cpu_addr_dma, 9f45fc00
virtio_rpmsg_bus virtio0: buf 255, cpu_addr 9f45fe00, cpu_addr_dma, 9f45fe00

I don't what to make of the fact that cpu_addr and cpu_addr_dma are
set to the same virtual address value.

3) Here is where virtqueue_add() sets the buffer addresses in the
vring descriptors.
No looping occurs here in this case because (out_sgs + in_sgs) == 1,
as can be verified by looking at virtqueue_add_inbuf() call above.

for (; n < (out_sgs + in_sgs); n++) {
    for (sg = sgs[n]; sg; sg = sg_next(sg)) {
        desc[i].flags = cpu_to_virtio16(_vq->vdev, VRING_DESC_F_NEXT |
VRING_DESC_F_WRITE);
        desc[i].addr = cpu_to_virtio64(_vq->vdev, sg_phys(sg));
        desc[i].len = cpu_to_virtio32(_vq->vdev, sg->length);
        pr_debug("desc %d addr %llx\n", i, desc[i].addr);
        prev = i;
        i = virtio16_to_cpu(_vq->vdev, desc[i].next);
    }
}

I added the pr_debug() after desc[i].addr is set and here are the results:
desc 0 addr 1f440000
desc 1 addr 1f440200
desc 2 addr 1f440400
... [snipped]
desc 253 addr 1f45fa00
desc 254 addr 1f45fc00
desc 255 addr 1f45fe00

virtqueue_add() is populating the vring descriptors with buffer
addresses of 0x1f440xxx instead of the expected 0x1e440xxx.

Trying to decipher what sg_phys() does is where I get lost, but I
think that's where the 0x1f440xxx addresses are ultimately coming
from.