macrobug

Well, after years of procrastination, it’s about to happen.

This is it.

I’m about to release a product.

In fact, it’s already been available for download from this website for weeks, and has been diligently tested by a couple of testers (special thanks to Adrian in particular!) It’s still nowhere near final-quality, but it’s good enough that I want more people to download it and play with it. And, if people like it enough to buy it, that’s fine with me.

So, Macrobug enters a new phase - the terrifying phase of actually having to sell things. I’ll try to post relatively frequently to recount my experiences!

First steps:

1. Post on blog - check!
2. E-mail lots of contacts about it - check!
3. Cross fingers very hard… check…

Well, I still can’t find any software which will allow me to use the standard HP drivers with the DeskJet. (See parts one and two for the explanation). But, there’s another open-source alternative to Gutenprint’s HP driver: HPIJS - which works great! It’s still not brilliantly fast, but it supports multiple paper types, and most importantly, duplex printing - so I’m happy!

Part two of my saga of trying to get my HP DeskJet 940C to print nicely from MacOS X despite it being attached to my ReadyNAS. Bonjour Browser, we can see that the Airport Express is advertising:

• _pdl-datastream._tcp., a “Printer PDL Data Stream”, and
• _riousbprint._tcp., a “Remote I/O USB Printer”

So, not IPP at all.

What we need to do, then, is get the ReadyNAS to advertise itself as a remotely controllable USB device, instead of (as well as?) an IPP printer. But do we need _pdl-datastream, or _riousbprint, or both?

Specifically, this is how it appears over Bonjour from the AirportExpress:

_pdl-datastream._tcp.
192.168.0.6:9100
txtvers=1
qtotal=1
note=Living Room
product=(HEWLETT-PACKARD DESKJET 940C)
pdl=application/vnd.hp-PCL
priority=5
usb_MFG=HEWLETT-PACKARD
usb_MDL=DESKJET 940C
usb_C
ty=HEWLETT_PACKARD DESKJET 940C

_riousbprint._tcp.
192.168.0.6:10000
txtvers=1
qtotal=1
note=Living Room
product=(HEWLETT-PACKARD DESKJET 940C)
rp=HEWLETT-PACKARD DESKJET 940C HU1B96N1ZZBH
pdl=application/MLC,application/PCL,application/PML
priority=1
usb_MFG=HE

Well, looking at a Wireshark dump of the transaction, we’re talking to port 10000: the remote USB service. Unsurprising, really. To be honest it’s more surprising that the Airport Express provides the other service (which makes it look like an HP JetDirect to PCs, I suppose).

So, we “just” need a package we can install on the ReadyNAS using apt, which exposes the USB device using this protocol. Sadly I can’t find such a Linux project, and nor can I find documentation on the protocol so I can’t have a go at writing one myself. Which is probably a good thing, as I have more important things I should be doing!

Summary: it’s not impossible after all, somebody just needs to write some software!

I’ve just bought a ReadyNAS NV+, a NAS box onto which I will do all my backups. It’s great: amongst other things you can log into it using SSH, and use apt-get to install packages, which opens up lots of opportunities such as running my Subversion server on it.

I also wanted to share my old-ish HP DeskJet 940C inkjet from it. Upon connecting it to the USB port on the ReadyNAS, it appeared in Bonjour and I was able to print with it from my Mac - but only using the Gutenprint driver, rather than HP’s own driver. This was not a problem when it was attached to my old Power Mac G4 sitting in the corner… but I’d hoped to retire that. Unfortunately, the Gutenprint driver is slower, doesn’t support different types of paper, nor duplex. It’s good, but it’s not good enough.

I’ve spent a couple of hours researching the problem and have concluded that it’s impossible to solve. Here’s my reasoning. If anyone can find a way round these problems that would be terrific!

How printing works on MacOS X

MacOS X uses a printing system called CUPS. This knows about the printers installed, and manages queues of documents for them. You can control CUPS using the Printers preference page, or by logging into your own Mac on port 631 using a web browser.

The settings for each CUPS printer consist of three things behind the scenes:

1. An entry in /etc/cups/printers.conf which says very little except…
2. A URL to which the printer data should be delivered
3. A PPD (Postscript Printer Description file) in /etc/cups/ppd

The key part is the PPD. This specifies all the capabilities of the printer. It also specifies how to process the data for the printer… and this is done in terms of the “cupsFilter” element. It describes whether the printer can accept a given MIME type directly. If not, it can also specify that the printer can accept that MIME type if the data is fed through a certain programme. CUPS will chain together these “filters” to convert from the original data type to one which the printer can actually accept. For example,a PICT image might be converted to PDF, then to a general bitmap, then to a bitmap format suitable for the printer.

HP Sources

Here are the cupsFilter details for my HP DeskJet 940c…

So, for Gutenprint, a general CUPS raster (which CUPS intrinsically knows how to accept) is converted by the program “rastertogutenprint5.1″ to the data format specific for the printer. CUPS then sends that across the network to the ReadyNas, and all is well.

If you’re using the standard HP driver locally, it uses a command “PrintJobMgr” to send the data to the actual printer. This takes a PDF and does something to convert it to HP’s format.

If you’re using an HP printer connected to a remote Mac, things are very different - it just sends the PDF, PICT or any image format directly to the other Mac - which will then send it through CUPS again, eventually resulting in a PDF form of the data making its way into PrintJobMgr on that distant Mac.

So what’s this mysterious PrintJobMgr? Unfortunately that’s where it all gets a bit depressing. Back in the early days of MacOS X (before 10.2) it didn’t use CUPS - instead it used a “Printer Manager” framework, codenamed Tioga. It appears that PrintJobMgr is a call from CUPS into that older system. That means, the process of converting from PDF (etc.) to the HP-specific data stream is not embedded in a nice executable which we can call from a cupsFilter before sending the data to the ReadyNAS. There’s simply no way to get the data out of the HP driver so that we can send it across the network.

So the HP driver is out - we’d have to run it on the ReadyNAS which means we’d have to run OS X on the ReadyNAS.

What about using “gutenprint.5.1″ to convert to the HP data format, but using the rest of the HP PPD file to configure the available options? I haven’t tried this, but I think it’s a non-starter. According to the Gutenprint user’s manual, the rest of the PPD is very specific to the Gutenprint driver. It specifically says that rastertogutenprint5.1 will fail if you use a PPD other than the one which comes with Gutenprint.

Unfortunately, it seems that to use a DeskJet attached to the ReadyNAS we have no choice but to use Gutenprint.

Update: it’s just occurred to me that the Airport Express is supposed to support printer sharing. I can’t believe it’s running the HP printer driver internally. So perhaps there’s a way… more on that later!

Eh? What? Adrian’s blogging again?

Yes. Macrobug has not gone away, just been dormant for the past few months whilst I’ve been working on other things.

But, fear not, because I am about to launch a product!

I’ve debugged why I was unable to display the memory relating to Android binder transactions coming from the kernel to user-side. It turns out that Binder is marking the pages as VM_IO, and the kernel code behind strace’s umoven function was unwilling to dump pages marked as such.

To fix this, just alter drivers/binder/binder.c line 585 (on my copy)…
vma->vm_flags |= VM_RESERVED | VM_READ | VM_RAND_READ | VM_IO | VM_DONTCOPY | VM_DONTEXPAND;
Remove the “VM_IO” from that list.

This appears to me to be a valid change, because the pages aren’t really I/O… they’re used for communicating within the Linux device, not outside of it.

Thanks to Motz for the time-saving instructions on how to build the Android kernel.

Anyway, I now have a complete record of the OpenBinder transactions sent and received by the ’service’ command. I want to do some cleanups, especially around the area of offsets into the transaction data, so I can nicely show the object references involved. I will eventually get around to posting the output (and of course the changes) here.

Then I’ll apply the same tools (again) to the normal Android startup, and after that, I’ll try to feed the output into my existing Macrobug tools and come up with a flowchart of the startup process of Android. At least that’s the plan.

First, a random notice. There’s an Android-Internals wiki, which hopefully will eventually start to describe everything we know about Android internals.

Secondly, here’s the most recent version of my openbinder.c code for strace. (Again, combine with tweaks to the makefiles etc. I’ve previously posted).

openbinder.c

And thirdly, thanks to Davanum’s discover of the ’service’ command, we can start to craft some nice simple OpenBinder interactions and see exactly how they work behind the scenes. Results are promising! I now understand the messages sent from the service command to the kernel, although I have yet to decode the replies from the kernel. More on that later.

Without further ado, here is a transcript using the above strace code of the following command: service call phone 2 s16 “123″. Captured using simply strace -o /tmp/strace.out service call phone 2 s16 “123″.

1. General process startup
• execve(”/system/bin/service”, [”service”, “call”, “phone”, “2″, “s16″, “123″], [/* 8 vars */]) = 0
• getpid() = 751
• syscall_983045(0xb0016b48, 0xb0013760, 0×3e4, 0, 0xbeef9e4c, 0×6, 0, 0xf0005, 0xb0013760, 0, 0, 0xbeef9e34, 0, 0xbeef9de8, 0xb0000d89, 0xb00016ec, 0×10, 0xb0016b48, 0, 0, 0, 0xc8c8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) = 0
• gettid() = 751
• sigaction(SIGILL, {0xb0001469, [], SA_RESTART}, {SIG_DFL}, 0) = 0
• sigaction(SIGABRT, {0xb0001469, [], SA_RESTART}, {SIG_DFL}, 0) = 0
• sigaction(SIGBUS, {0xb0001469, [], SA_RESTART}, {SIG_DFL}, 0) = 0
• sigaction(SIGFPE, {0xb0001469, [], SA_RESTART}, {SIG_DFL}, 0) = 0
• sigaction(SIGSEGV, {0xb0001469, [], SA_RESTART}, {SIG_DFL}, 0) = 0
• sigaction(SIGSTKFLT, {0xb0001469, [], SA_RESTART}, {SIG_DFL}, 0) = 0
• open(”libutils.so”, O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
• open(”/system/lib/libutils.so”, O_RDONLY|O_LARGEFILE) = 3
• lseek(3, -8, SEEK_END) = 425880
• read(3, “\0\0\320\251PRE “, 8) = 8
• mmap2(0xa9d00000, 425984, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0xa9d00000
• close(3) = 0
• open(”libz.so”, O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
• open(”/system/lib/libz.so”, O_RDONLY|O_LARGEFILE) = 3
• lseek(3, -8, SEEK_END) = 57252
• read(3, “\0\0\220\257PRE “, 8) = 8
• mmap2(0xaf900000, 57344, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0xaf900000
• close(3) = 0
• open(”libc.so”, O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
• open(”/system/lib/libc.so”, O_RDONLY|O_LARGEFILE) = 3
• lseek(3, -8, SEEK_END) = 231908
• read(3, “\0\0\340\257PRE “, 8) = 8
• mmap2(0xafe00000, 233472, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0xafe00000
• close(3) = 0
• mmap2(0xafe39000, 45056, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, 0, 0) = 0xafe39000
• mprotect(0xafe00000, 221184, PROT_READ|PROT_EXEC) = 0
• open(”libstdc++.so”, O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
• open(”/system/lib/libstdc++.so”, O_RDONLY|O_LARGEFILE) = 3
• lseek(3, -8, SEEK_END) = 3812
• read(3, “\0\0\320\257PRE “, 8) = 8
• mmap2(0xafd00000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0xafd00000
• close(3) = 0
• open(”libm.so”, O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
• open(”/system/lib/libm.so”, O_RDONLY|O_LARGEFILE) = 3
• lseek(3, -8, SEEK_END) = 131596
• read(3, “\0\0\300\257PRE “, 8) = 8
• mmap2(0xafc00000, 135168, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0xafc00000
• close(3) = 0
• mprotect(0xafc00000, 126976, PROT_READ|PROT_EXEC) = 0
• mprotect(0xaf900000, 53248, PROT_READ|PROT_EXEC) = 0
• open(”libcutils.so”, O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
• open(”/system/lib/libcutils.so”, O_RDONLY|O_LARGEFILE) = 3
• lseek(3, -8, SEEK_END) = 54548
• read(3, “\0\0\260\257PRE “, 8) = 8
• mmap2(0xafb00000, 57344, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0xafb00000
• close(3) = 0
• mmap2(0xafb0e000, 61440, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, 0, 0) = 0xafb0e000
• mprotect(0xafb00000, 53248, PROT_READ|PROT_EXEC) = 0
• mprotect(0xa9d00000, 344064, PROT_READ|PROT_EXEC) = 0
• brk(0) = 0xa000
• brk(0xa000) = 0xa000
• brk(0xb000) = 0xb000
• sched_get_priority_min(SCHED_RR) = 1
• sched_get_priority_max(SCHED_RR) = 99
• mprotect(0, 0, PROT_READ|PROT_EXEC) = 0
• gettid() = 751
• syscall_983045(0xbeef9d3c, 0, 0×20, 0, 0xbeeda000, 0xbeef9df0, 0xbeef9e30, 0xf0005, 0xbeef9e30, 0, 0×8858, 0×8854, 0, 0xbeef9d30, 0xafe20a53, 0xafe0949c, 0×60000010, 0xbeef9d3c, 0, 0, 0, 0xc8c8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) = 0
• SYS_281(0×1, 0×1, 0, 0xafe40d20, 0xafe371bc) = 3
• SYS_283(0×3, 0xbeef9c9a, 0×13, 0, 0×3) = 0
• SYS_297(0×3, 0xbeef9cd4, 0, 0×1, 0xbeef9cd4) = 4
• mmap2(NULL, 32768, PROT_READ, MAP_SHARED, 4, 0) = 0×40000000
• close(4) = 0
• close(3) = 0
• open(”/data/malloc_debug”, O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
2. Open the device for OpenBinder
• open(”/dev/binder”, O_RDWR|O_LARGEFILE) = 3
• fcntl64(3, F_SETFD, FD_CLOEXEC) = 0
3. Check what OpenBinder version is in use. The kernel replies 5.
• ioctl(3, BINDER_VERSION, {protocol_version=5}) = 0
4. State that the maximum threads we’re happy for OpenBinder to create in our process is 15. (At least, I think that’s what this is doing).
• ioctl(3, BINDER_SET_MAX_THREADS, 15) = 0
5. Map OpenBinder’s global memory to a space in our process. Later on, we’ll see what appears in this address space. Note that this is read-only; this space appears to be user for communications from the kernel Binder driver to this service executable, but not for data going from the executable to the server.
• mmap2(NULL, 8388608, PROT_READ, MAP_PRIVATE|MAP_NORESERVE, 3, 0) = 0×40008000
6. Get some details about this process, which presumably are going to be needed for the OpenBinder transactions. The process priority is definitely needed, because when a thread makes an OpenBinder request to another process, the request runs within a thread in the destination process at the same priority as the requesting thread.
• getpid() = 751
• getuid32() = 0
• getpid() = 751
• getuid32() = 0
• getpriority(PRIO_PROCESS, 0) = 20
7. Start an OpenBinder transaction to talk to the default service.

   Specifically, we create a transaction whose target is handle 0. I’m not sure what handle 0 means in OpenBinder, but presumably it’s some general OpenBinder framework or service. Logically, that makes sense, because OpenBinder handles must either refer to some local object (in which case they’re just a pointer) or some remote object… in which case the handle must already have been supplied by the OpenBinder driver in some previous transaction or reply. Since we’re the first, and since it’s clearly not a pointer, handle 0 must mean something special. The request we make has operation code 0. Again, I assume that’s just some general registration request.

   The request body has 44 bytes of data, of which none are a reference to any other object or entity in the Binder world, whether in our process or elsewhere.

   The contents of the 44 bytes is the string we see containing the word ‘default’. This is the request body - really, the parameters to the function which is being called across the process boundary. The contents of these do not appear to be self-describing, and presumably therefore rely on a mutual knowledge of the interface description. Since I don’t know what this service is, I can’t speculate what data it’s providing. But, I can reveal a few things. The first four bytes are the process ID (0×000002ef == 751). I speculate that the subsequent four bytes of 0×0 are the UID. After that, we’ve got the string length (0×07 bytes) then the string. The strings always seem to end in a null (0×00 0×00) even though they’re length-prefixed. Then we see 0×85,0×2a,0×62,0×73… which is a mystery but I see it (or similar) in lots of these bodies so it would be nice to figure it out. Finally, there’s 0×08 0×00 0×00 0×00, then another empty word (0×00 0×00 0×00 0×00).

   The kernel driver replies to say that the transaction was complete. It doesn’t return any data. The purpose of the brNOOP is explained in the OpenBinder documentation available on Dianne Hackborn’s website.

   • ioctl(3, BINDER_WRITE_READ, {write_size=40,write_consumed=40,write_buffer=0xa438,read_size=256,read_consumed=8,read_buffer=0xa328,write_data=[bcTRANSACTION({target=0×00000000, cookie=0×0000a204,code=0×00000000, flags=0×00000000,priority=80,data_size=44, offsets_size=0, data={buffer=0×0000a540,offsets=0×0000a300,*buffer= [0xef,0×02,0×00,0×00,0×00,0×00,0×00,0×00,0×07,0×00,0×00,0×00,0×64,0×00,0×65,0×00,0×66,0×00,0×61, 0×00,0×75,0×00,0×6c,0×00,0×74,0×00,0×00,0×00,0×85,0×2a,0×62,0×73,0×08,0×00,0×00,0×00,0×00,0×00,0×00,0×00,0×00,0×00,0×00,0×00, “…………d.e.f.a.u.l.t….*bs…………”], *offsets=[”"]}})],read_data={[brNOOP,brTRANSACTION_COMPLETE]}) = 0
8. And now we start another OpenBinder transaction. In this one, we (the service process) send nothing, but we offer up a buffer of 256 bytes to receive a reply. This ioctl will then block until our target has prepared the reply, at which point the kernel driver will fill in our buffer and give the reply to us. Sadly, this is where my strace hacks aren’t good enough. Although I can show the data sent from the process to the kernel, I can’t show the reply received… the address is 0×4008000, which is at the start of the mmap’ped /dev/binder memory. strace is not happy to give me that memory, and I haven’t yet looked into why. Maybe I’ll get there in the end.

   Meanwhile, though, we can see a few things. The reply is 24 bytes long, of which there is one flat_binder_object at a certain offset into the buffer. That flat_binder_object will contain a handle to some sort of service discovery object in some other process. At a guess, that handle is going to be 0×01.

   • ioctl(3, BINDER_WRITE_READ, {write_size=0,write_consumed=0,write_buffer=0xa438,read_size=256,read_consumed=44,read_buffer=0xa328, write_data=[],read_data={[brNOOP,brREPLY({flags=0×00000000,priority=80,data_size=24,offsets_size=4, data={buffer=0×40008000,offsets=0×40008018,*buffer=[…]}})]}) = 0
9. Our next OpenBinder interaction says we want to keep a reference to that handle - 0×01. We also acquire it (I’m not sure what that means) and state that we’ve finished with the buffer we read from the global memory. All Binder things are reference-counted… presumably the fact it was in a buffer meant there was a reference to the service discovery object, then when we INCREF’ed, we claimed another one, and destroyed the buffer thus removing the original one. At any rate, the reference count never went to 0 so Binder won’t have destroyed the original object.
   • ioctl(3, BINDER_WRITE_READ, {write_size=24,write_consumed=24,write_buffer=0xa438,read_size=0,read_consumed=0,read_buffer=0xa9d2cf57, write_data=[bcINCREFS(target=0×00000001),bcACQUIRE(target=0×00000001),bcFREE_BUFFER(ptr=0×40008000)],read_data={[]}) = 0
10. So now we ask for our priority, etc. all over again. That suggests that the OpenBinder user-side code is treating all of the above as one single “transaction” (though I don’t want to overload that word) and now we’re starting another one.
• getpid() = 751
• getuid32() = 0
• getpriority(PRIO_PROCESS, 0) = 20
11. And here is that next transaction. The target is 0×1, which I believe to be that reference to the service discovery object we asked for. We’re asking for opcode 2, which presumably means we want to get a reference to a named object. The body this time is simple - it’s just the PID, probable UID, and a string containing the name of the service (”phone” in Unicode).
• ioctl(3, BINDER_WRITE_READ, {write_size=40,write_consumed=40,write_buffer=0xa438,read_size=256,read_consumed=8,read_buffer=0xa328,write_data=[ bcTRANSACTION({target=0×00000001,cookie=0×00000000,code=0×00000002,flags=0×00000000, priority=80,data_size=24,offsets_size=0,data={buffer=0×0000a660,offsets=0×00000000,*buffer= [0xef,0×02,0×00,0×00,0×00,0×00,0×00,0×00,0×05,0×00,0×00,0×00,0×70,0×00,0×68,0×00,0×6f,0×00,0×6e,0×00,0×65,0×00,0×00,0×00, “…………p.h.o.n.e…”],*offsets=[”"]}})], read_data={[brNOOP,brTRANSACTION_COMPLETE]}) = 0
12. And we wait for a reply. Again, we get one; it’s 24 bytes long and again contains one reference. I bet that reference is a reference to the phone service, and I bet it’s handle 0×02.
• ioctl(3, BINDER_WRITE_READ, {write_size=0,write_consumed=0,write_buffer=0xa438,read_size=256,read_consumed=44,read_buffer=0xa328,write_data=[],read_data= {[brNOOP,brREPLY({flags=0×00000000,priority=80,data_size=24,offsets_size=4,data={buffer=0×40008000,offsets=0×40008018, *buffer=[…]}})]}) = 0
13. And the start of another ‘transaction’.
• getpid() = 751
• getuid32() = 0
• getpriority(PRIO_PROCESS, 0) = 20
14. Now the really meaty bit where we actually make the request. We increase the reference count on that ‘phone’ object we got given, free the buffer, and then start a transaction to the phone object (target 0×2). The opcode is 2, which is what we specified on the command-line. The body contains purely the string we specified on the command-line: 123.
• ioctl(3, BINDER_WRITE_READ, {write_size=64,write_consumed=64,write_buffer=0xa438,read_size=256,read_consumed=8,read_buffer=0xa328, write_data=[bcINCREFS(target=0×00000002),bcACQUIRE(target=0×00000002), bcFREE_BUFFER(ptr=0×40008000),bcTRANSACTION({target=0×00000002,cookie=0×0000001b, code=0×00000002,flags=0×00000000,priority=80,data_size=20,offsets_size=0, data={buffer=0×0000a6e0,offsets=0×00000000,*buffer= [0xef,0×02,0×00,0×00,0×00,0×00,0×00,0×00,0×03,0×00,0×00,0×00,0×31,0×00,0×32,0×00,0×33,0×00,0×00,0×00, “…………1.2.3…”],*offsets=[”"]}})], read_data={[brNOOP,brTRANSACTION_COMPLETE]}) = 0
15. Finally, we block to wait for a reply. We’re given 8 bytes containing no references to any objects… and I bet those bytes describe a NULL parcel.
• ioctl(3, BINDER_WRITE_READ, {write_size=0,write_consumed=0,write_buffer=0xa438,read_size=256,read_consumed=44,read_buffer=0xa328,write_data=[], read_data={[brNOOP,brREPLY({flags=0×00000000,priority=80,data_size=8,offsets_size=0,data={buffer=0×40008000,offsets=0×40008008, *buffer=[…]}})]}) = 0
16. And we’re done.
• writev(1, [{”Result: Parcel(NULL)\n”, 21}], 1) = 21
• exit_group(0) = ?

Where to go next? I want to:

• Figure out how to get strace to ’see’ inside the mmap’ped /dev/binder, so I can see the replies coming back from the kernel (and thus ultimately from other processes
• Play around a bit to see if I can work out any more (e.g. the meaning of the cookies, or whether the second word of the transaction body really is the UID, or whether ints make a difference to the other data in the body)
• Use the knowledge gained from this to make a stab at understanding some of the (massive) logs gained from applying the same strace tools to runtime and Zygote and the things they spawn
• See if I can build the Binder shell and run it on Android. It’s like ’service’ only more sophisticated.
• Look more into the known, documented stuff about the AIDLs and the Service classes on the Java (Dalvik) side. I’d like to think the opcodes are a direct mapping into an AIDL. I’d like to try creating my own service and interacting with it. I haven’t really looked into the Java side stuff so I’ve no idea if it’s possible.

The odds of me getting around to any of the above any time soon are slim!

Frequently appears in strace listings of what Android is up to. Appears to be setting the thread local storage.

I’ve also found that strace’ing Zygote and the things it spawns is (possibly predictably) much more fruitful in terms of IPC messages than strace’ing the runtime. However I have don’t yet have enough time to figure out how I dig down into the structure enough to find out the destination process for a transaction, from the data available. Once I get that far I’d like to think I can knock up a simple flow diagram of the messages passed between all the Android threads and we can start to get a decent picture of how the whole system operates.

One day…

Here are some superficial thoughts on Android (which I don’t know much about) versus Symbian OS (which I do).

Some of that’s probably rubbish, and may be based on misinterpreting bits of the interesting podcast. I’m happy to accept corrections, but then again, only six people read this blog anyway so this post was mostly for my own interest :-)

In other news, Motz has figured out the correct linker settings for the Android loader!

Notes from the recent Android podcast…

• Dalvik VM can mmap dex files.
• Zygote loads VM before forking, so copy-on-write effective and all apps share lots of pages.
• ‘am’ command doesn’t work that way; can get idea of benefit; takes 2 seconds to launch because it has to load all classes.
• Different user for each application. Your data is UID’d to you on the filesystem too.C
• Content providers use URL scheme. Every bit of data on device has a unique name.
• Intents are just a generic way of naming something to do. e.g. I’d like a content. Then system finds something like the contacts app. Similar for playing an MP3, any media player app can fulfil intent.
• Intents are broadcast; anyone can reply.
• Embedding in UIs not supported.
• XML gets compiled down. Generic binary format for XML, the compiler can reduce any XML.
• Reference HW has 64MB RAM, 128MB flash. No virtual memory practically. Yet Linux is aimed around VM. For example when creating a thread, Linux allocates a heap of virtual address space for the program (but only allocates physical pages later). If RAM unavailable, would have to kill app, can’t fail the malloc.
• So component model allows kernel to identify which applications aren’t needed, and kill them if there are insufficient pages.
• Threads: tried hard to avoid needing an app to have multiple threads. Everything designed to happen in main thread. But of course you need to remain responsive in main thread.
• A service is not a main thread either. When you start a service it runs on the main thread still.
• UI framework not thread-safe, for efficiency. Must access UI object from main thread. Use Handler object to do that.
• Handler also allows timer services; can request an event in the future.
• Current UI is somewhat of a placeholder