pciproxy code

[Mattias Nissler]

Hi,

as some of you might remember I have been working on writing some PCI  
proxying code for MOL. This makes it possibly to have PCI devices  
from the host system show up in the virtual machine created by MOL  
and thus using drivers of the guest OS for those devices (the  
original intention was to get the AirPort Extreme cards running since  
there is no Linux driver; I only tested that device yet). But note: I  
don't think there is a chance of getting the AirPort Extreme devices  
working using the original Apple driver because of a DMA issue, see  
below. However, I thought I should share the code anyway, you can  
grab the patch at http://www-user.rhrk.uni-kl.de/~nissler/mol/ 
index.html.

Now some things I want to discuss:

1. Currently the code blocks PCI bus mastering. The problem with that  
is that the PCI device would be able to write to/read from arbitrary  
memory locations. The memory addresses come from the device driver  
which uses that feature to transfer data from/to the card (network  
packets received or to be sent in case of AirPort Extreme). Now this  
causes trouble: The card only knows about physical addresses which  
are totally different in the physical system in which the card is  
living, compared to the physical addresses of the virtual mac. So  
allowing those DMA transfers would result in overwriting arbitrary  
memory locations, which will probably crash the host OS, right? To  
get it working, one would have to keep those to address spaces in  
sync. This means that MOL would have to grab some part of physical  
RAM while it is running and mapping it to the same location in the  
physical address space of the virtual machine. In my understanding  
this is not what MOL is currently doing. Can anyone confirm this?  
Other ideas? Is there some possibilty to rewrite memory addresses at  
the PCI host bridge?

2. For the driver to access the card, I wanted to map the PCI I/O  
memory directly into the physical address space of the virtual  
machine using the mmu mapping facilities MOL provides. I've found  
that this causes trouble with MOLs iopage implementation: Because  
there is not ordinary memory on the PCI board you can't access any  
address, but only special ones. However, when a DSI exception occurs  
(i.e. page fault or protection violation) MOL checks whether there is  
some special signature in the page that indicates the access is to be  
satisfied via MOLs userspace process. When MOL maps the i/o memory it  
does this read-only, so the driver will cause a protection violation  
at the first write attempt. MOL tries to check the page for the magic  
numbers and immediately causes a machine check exception. This is  
because the PCI machinery signals a bus error, since MOL accessed an  
address that isn't backed by memory on the PCI device. Bad. The  
workaround I have implemented is an additional mapping flag called  
MAPPING_FORCE_WRITABLE that will cause the pages of the mapping to be  
mapped in read/write, so no protection violation can occur. Not very  
nice, suggestions?

3. If somebody wants to try out (with AirPort Extreme, you will at  
least see the device in the OS X system profiler), get the patch (at  
the location mentioned above) and apply it against the rsync version  
of MOL. To get it running you will have to do some configuration (see  
the comments in Doc/config/molrc.osx). If you have problems, just  
drop me a line.

4. IRQ handling: The problem is how to forward IRQs from the physical  
device occurring in the host OS to the virtual machine. I have  
implemented IRQ mapping support (kernel module and userspace parts,  
see the hostirq.c files). This is quite independent of the pciproxy  
parts and could be used for other things as well, see include/hostirq.h.

5. Still reading? Well, now what about AirPort Extreme? Only that  
much: Disassembling, modifying and redistributing Apples driver is  
prohibited by their license agreement. If that wouldn't be the case,  
I could make available a modified version that doesn't use DMA but  
ordinary mmio reads/writes. For those of you who are familar with  
assembly there is a chance... Or maybe someone finds a driver version  
that isn't determined to use DMA.

Comments, suggestions, experiences with other PCI devices? Just mail me.

Mattias