Archive for December, 2007

Gotcha!

Friday, December 28th, 2007

I’ve debugged why I was unable to display the memory relating to Android binder transactions coming from the kernel to user-side. It turns out that Binder is marking the pages as VM_IO, and the kernel code behind strace’s umoven function was unwilling to dump pages marked as such.

To fix this, just alter drivers/binder/binder.c line 585 (on my copy)…
vma->vm_flags |= VM_RESERVED | VM_READ | VM_RAND_READ | VM_IO | VM_DONTCOPY | VM_DONTEXPAND;
Remove the “VM_IO” from that list.

This appears to me to be a valid change, because the pages aren’t really I/O… they’re used for communicating within the Linux device, not outside of it.

Thanks to Motz for the time-saving instructions on how to build the Android kernel.

Anyway, I now have a complete record of the OpenBinder transactions sent and received by the ‘service’ command. I want to do some cleanups, especially around the area of offsets into the transaction data, so I can nicely show the object references involved. I will eventually get around to posting the output (and of course the changes) here.

Then I’ll apply the same tools (again) to the normal Android startup, and after that, I’ll try to feed the output into my existing Macrobug tools and come up with a flowchart of the startup process of Android. At least that’s the plan.

Decoding a ‘service’ call in Android

Saturday, December 15th, 2007

First, a random notice. There’s an Android-Internals wiki, which hopefully will eventually start to describe everything we know about Android internals.

Secondly, here’s the most recent version of my openbinder.c code for strace. (Again, combine with tweaks to the makefiles etc. I’ve previously posted).

openbinder.c

And thirdly, thanks to Davanum’s discover of the ‘service’ command, we can start to craft some nice simple OpenBinder interactions and see exactly how they work behind the scenes. Results are promising! I now understand the messages sent from the service command to the kernel, although I have yet to decode the replies from the kernel. More on that later.

Without further ado, here is a transcript using the above strace code of the following command: service call phone 2 s16 "123". Captured using simply strace -o /tmp/strace.out service call phone 2 s16 "123".

  1. General process startup
    • execve(“/system/bin/service”, [“service”, “call”, “phone”, “2”, “s16”, “123”], [/* 8 vars */]) = 0
    • getpid() = 751
    • syscall_983045(0xb0016b48, 0xb0013760, 0x3e4, 0, 0xbeef9e4c, 0x6, 0, 0xf0005, 0xb0013760, 0, 0, 0xbeef9e34, 0, 0xbeef9de8, 0xb0000d89, 0xb00016ec, 0x10, 0xb0016b48, 0, 0, 0, 0xc8c8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) = 0
    • gettid() = 751
    • sigaction(SIGILL, {0xb0001469, [], SA_RESTART}, {SIG_DFL}, 0) = 0
    • sigaction(SIGABRT, {0xb0001469, [], SA_RESTART}, {SIG_DFL}, 0) = 0
    • sigaction(SIGBUS, {0xb0001469, [], SA_RESTART}, {SIG_DFL}, 0) = 0
    • sigaction(SIGFPE, {0xb0001469, [], SA_RESTART}, {SIG_DFL}, 0) = 0
    • sigaction(SIGSEGV, {0xb0001469, [], SA_RESTART}, {SIG_DFL}, 0) = 0
    • sigaction(SIGSTKFLT, {0xb0001469, [], SA_RESTART}, {SIG_DFL}, 0) = 0
    • open(“libutils.so”, O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
    • open(“/system/lib/libutils.so”, O_RDONLY|O_LARGEFILE) = 3
    • lseek(3, -8, SEEK_END) = 425880
    • read(3, “\0\0\320\251PRE “, 8) = 8
    • mmap2(0xa9d00000, 425984, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0xa9d00000
    • close(3) = 0
    • open(“libz.so”, O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
    • open(“/system/lib/libz.so”, O_RDONLY|O_LARGEFILE) = 3
    • lseek(3, -8, SEEK_END) = 57252
    • read(3, “\0\0\220\257PRE “, 8) = 8
    • mmap2(0xaf900000, 57344, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0xaf900000
    • close(3) = 0
    • open(“libc.so”, O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
    • open(“/system/lib/libc.so”, O_RDONLY|O_LARGEFILE) = 3
    • lseek(3, -8, SEEK_END) = 231908
    • read(3, “\0\0\340\257PRE “, 8) = 8
    • mmap2(0xafe00000, 233472, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0xafe00000
    • close(3) = 0
    • mmap2(0xafe39000, 45056, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, 0, 0) = 0xafe39000
    • mprotect(0xafe00000, 221184, PROT_READ|PROT_EXEC) = 0
    • open(“libstdc++.so”, O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
    • open(“/system/lib/libstdc++.so”, O_RDONLY|O_LARGEFILE) = 3
    • lseek(3, -8, SEEK_END) = 3812
    • read(3, “\0\0\320\257PRE “, 8) = 8
    • mmap2(0xafd00000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0xafd00000
    • close(3) = 0
    • open(“libm.so”, O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
    • open(“/system/lib/libm.so”, O_RDONLY|O_LARGEFILE) = 3
    • lseek(3, -8, SEEK_END) = 131596
    • read(3, “\0\0\300\257PRE “, 8) = 8
    • mmap2(0xafc00000, 135168, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0xafc00000
    • close(3) = 0
    • mprotect(0xafc00000, 126976, PROT_READ|PROT_EXEC) = 0
    • mprotect(0xaf900000, 53248, PROT_READ|PROT_EXEC) = 0
    • open(“libcutils.so”, O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
    • open(“/system/lib/libcutils.so”, O_RDONLY|O_LARGEFILE) = 3
    • lseek(3, -8, SEEK_END) = 54548
    • read(3, “\0\0\260\257PRE “, 8) = 8
    • mmap2(0xafb00000, 57344, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0xafb00000
    • close(3) = 0
    • mmap2(0xafb0e000, 61440, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, 0, 0) = 0xafb0e000
    • mprotect(0xafb00000, 53248, PROT_READ|PROT_EXEC) = 0
    • mprotect(0xa9d00000, 344064, PROT_READ|PROT_EXEC) = 0
    • brk(0) = 0xa000
    • brk(0xa000) = 0xa000
    • brk(0xb000) = 0xb000
    • sched_get_priority_min(SCHED_RR) = 1
    • sched_get_priority_max(SCHED_RR) = 99
    • mprotect(0, 0, PROT_READ|PROT_EXEC) = 0
    • gettid() = 751
    • syscall_983045(0xbeef9d3c, 0, 0x20, 0, 0xbeeda000, 0xbeef9df0, 0xbeef9e30, 0xf0005, 0xbeef9e30, 0, 0x8858, 0x8854, 0, 0xbeef9d30, 0xafe20a53, 0xafe0949c, 0x60000010, 0xbeef9d3c, 0, 0, 0, 0xc8c8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) = 0
    • SYS_281(0x1, 0x1, 0, 0xafe40d20, 0xafe371bc) = 3
    • SYS_283(0x3, 0xbeef9c9a, 0x13, 0, 0x3) = 0
    • SYS_297(0x3, 0xbeef9cd4, 0, 0x1, 0xbeef9cd4) = 4
    • mmap2(NULL, 32768, PROT_READ, MAP_SHARED, 4, 0) = 0x40000000
    • close(4) = 0
    • close(3) = 0
    • open(“/data/malloc_debug”, O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
  2. Open the device for OpenBinder
    • open(“/dev/binder”, O_RDWR|O_LARGEFILE) = 3
    • fcntl64(3, F_SETFD, FD_CLOEXEC) = 0
  3. Check what OpenBinder version is in use. The kernel replies 5.
    • ioctl(3, BINDER_VERSION, {protocol_version=5}) = 0
  4. State that the maximum threads we’re happy for OpenBinder to create in our process is 15. (At least, I think that’s what this is doing).
    • ioctl(3, BINDER_SET_MAX_THREADS, 15) = 0
  5. Map OpenBinder’s global memory to a space in our process. Later on, we’ll see what appears in this address space. Note that this is read-only; this space appears to be user for communications from the kernel Binder driver to this service executable, but not for data going from the executable to the server.
    • mmap2(NULL, 8388608, PROT_READ, MAP_PRIVATE|MAP_NORESERVE, 3, 0) = 0x40008000
  6. Get some details about this process, which presumably are going to be needed for the OpenBinder transactions. The process priority is definitely needed, because when a thread makes an OpenBinder request to another process, the request runs within a thread in the destination process at the same priority as the requesting thread.
    • getpid() = 751
    • getuid32() = 0
    • getpid() = 751
    • getuid32() = 0
    • getpriority(PRIO_PROCESS, 0) = 20
  7. Start an OpenBinder transaction to talk to the default service.

    Specifically, we create a transaction whose target is handle 0. I’m not sure what handle 0 means in OpenBinder, but presumably it’s some general OpenBinder framework or service. Logically, that makes sense, because OpenBinder handles must either refer to some local object (in which case they’re just a pointer) or some remote object… in which case the handle must already have been supplied by the OpenBinder driver in some previous transaction or reply. Since we’re the first, and since it’s clearly not a pointer, handle 0 must mean something special. The request we make has operation code 0. Again, I assume that’s just some general registration request.

    The request body has 44 bytes of data, of which none are a reference to any other object or entity in the Binder world, whether in our process or elsewhere.

    The contents of the 44 bytes is the string we see containing the word ‘default’. This is the request body – really, the parameters to the function which is being called across the process boundary. The contents of these do not appear to be self-describing, and presumably therefore rely on a mutual knowledge of the interface description. Since I don’t know what this service is, I can’t speculate what data it’s providing. But, I can reveal a few things. The first four bytes are the process ID (0x000002ef == 751). I speculate that the subsequent four bytes of 0x0 are the UID. After that, we’ve got the string length (0x07 bytes) then the string. The strings always seem to end in a null (0x00 0x00) even though they’re length-prefixed. Then we see 0x85,0x2a,0x62,0x73… which is a mystery but I see it (or similar) in lots of these bodies so it would be nice to figure it out. Finally, there’s 0x08 0x00 0x00 0x00, then another empty word (0x00 0x00 0x00 0x00).

    The kernel driver replies to say that the transaction was complete. It doesn’t return any data. The purpose of the brNOOP is explained in the OpenBinder documentation available on Dianne Hackborn’s website.

    • ioctl(3, BINDER_WRITE_READ, {write_size=40,write_consumed=40,write_buffer=0xa438,read_size=256,read_consumed=8,read_buffer=0xa328,write_data=[bcTRANSACTION({target=0x00000000, cookie=0x0000a204,code=0x00000000, flags=0x00000000,priority=80,data_size=44, offsets_size=0, data={buffer=0x0000a540,offsets=0x0000a300,*buffer= [0xef,0x02,0x00,0x00,0x00,0x00,0x00,0x00,0x07,0x00,0x00,0x00,0x64,0x00,0x65,0x00,0x66,0x00,0x61, 0x00,0x75,0x00,0x6c,0x00,0x74,0x00,0x00,0x00,0x85,0x2a,0x62,0x73,0x08,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00, “…………d.e.f.a.u.l.t….*bs…………”], *offsets=[“”]}})],read_data={[brNOOP,brTRANSACTION_COMPLETE]}) = 0
  8. And now we start another OpenBinder transaction. In this one, we (the service process) send nothing, but we offer up a buffer of 256 bytes to receive a reply. This ioctl will then block until our target has prepared the reply, at which point the kernel driver will fill in our buffer and give the reply to us. Sadly, this is where my strace hacks aren’t good enough. Although I can show the data sent from the process to the kernel, I can’t show the reply received… the address is 0x4008000, which is at the start of the mmap’ped /dev/binder memory. strace is not happy to give me that memory, and I haven’t yet looked into why. Maybe I’ll get there in the end.

    Meanwhile, though, we can see a few things. The reply is 24 bytes long, of which there is one flat_binder_object at a certain offset into the buffer. That flat_binder_object will contain a handle to some sort of service discovery object in some other process. At a guess, that handle is going to be 0x01.

    • ioctl(3, BINDER_WRITE_READ, {write_size=0,write_consumed=0,write_buffer=0xa438,read_size=256,read_consumed=44,read_buffer=0xa328, write_data=[],read_data={[brNOOP,brREPLY({flags=0x00000000,priority=80,data_size=24,offsets_size=4, data={buffer=0x40008000,offsets=0x40008018,*buffer=[…]}})]}) = 0
  9. Our next OpenBinder interaction says we want to keep a reference to that handle – 0x01. We also acquire it (I’m not sure what that means) and state that we’ve finished with the buffer we read from the global memory. All Binder things are reference-counted… presumably the fact it was in a buffer meant there was a reference to the service discovery object, then when we INCREF’ed, we claimed another one, and destroyed the buffer thus removing the original one. At any rate, the reference count never went to 0 so Binder won’t have destroyed the original object.
    • ioctl(3, BINDER_WRITE_READ, {write_size=24,write_consumed=24,write_buffer=0xa438,read_size=0,read_consumed=0,read_buffer=0xa9d2cf57, write_data=[bcINCREFS(target=0x00000001),bcACQUIRE(target=0x00000001),bcFREE_BUFFER(ptr=0x40008000)],read_data={[]}) = 0
  10. So now we ask for our priority, etc. all over again. That suggests that the OpenBinder user-side code is treating all of the above as one single “transaction” (though I don’t want to overload that word) and now we’re starting another one.
    • getpid() = 751
    • getuid32() = 0
    • getpriority(PRIO_PROCESS, 0) = 20
  11. And here is that next transaction. The target is 0x1, which I believe to be that reference to the service discovery object we asked for. We’re asking for opcode 2, which presumably means we want to get a reference to a named object. The body this time is simple – it’s just the PID, probable UID, and a string containing the name of the service (“phone” in Unicode).
    • ioctl(3, BINDER_WRITE_READ, {write_size=40,write_consumed=40,write_buffer=0xa438,read_size=256,read_consumed=8,read_buffer=0xa328,write_data=[ bcTRANSACTION({target=0x00000001,cookie=0x00000000,code=0x00000002,flags=0x00000000, priority=80,data_size=24,offsets_size=0,data={buffer=0x0000a660,offsets=0x00000000,*buffer= [0xef,0x02,0x00,0x00,0x00,0x00,0x00,0x00,0x05,0x00,0x00,0x00,0x70,0x00,0x68,0x00,0x6f,0x00,0x6e,0x00,0x65,0x00,0x00,0x00, “…………p.h.o.n.e…”],*offsets=[“”]}})], read_data={[brNOOP,brTRANSACTION_COMPLETE]}) = 0
  12. And we wait for a reply. Again, we get one; it’s 24 bytes long and again contains one reference. I bet that reference is a reference to the phone service, and I bet it’s handle 0x02.
    • ioctl(3, BINDER_WRITE_READ, {write_size=0,write_consumed=0,write_buffer=0xa438,read_size=256,read_consumed=44,read_buffer=0xa328,write_data=[],read_data= {[brNOOP,brREPLY({flags=0x00000000,priority=80,data_size=24,offsets_size=4,data={buffer=0x40008000,offsets=0x40008018, *buffer=[…]}})]}) = 0
  13. And the start of another ‘transaction’.
    • getpid() = 751
    • getuid32() = 0
    • getpriority(PRIO_PROCESS, 0) = 20
  14. Now the really meaty bit where we actually make the request. We increase the reference count on that ‘phone’ object we got given, free the buffer, and then start a transaction to the phone object (target 0x2). The opcode is 2, which is what we specified on the command-line. The body contains purely the string we specified on the command-line: 123.
    • ioctl(3, BINDER_WRITE_READ, {write_size=64,write_consumed=64,write_buffer=0xa438,read_size=256,read_consumed=8,read_buffer=0xa328, write_data=[bcINCREFS(target=0x00000002),bcACQUIRE(target=0x00000002), bcFREE_BUFFER(ptr=0x40008000),bcTRANSACTION({target=0x00000002,cookie=0x0000001b, code=0x00000002,flags=0x00000000,priority=80,data_size=20,offsets_size=0, data={buffer=0x0000a6e0,offsets=0x00000000,*buffer= [0xef,0x02,0x00,0x00,0x00,0x00,0x00,0x00,0x03,0x00,0x00,0x00,0x31,0x00,0x32,0x00,0x33,0x00,0x00,0x00, “…………1.2.3…”],*offsets=[“”]}})], read_data={[brNOOP,brTRANSACTION_COMPLETE]}) = 0
  15. Finally, we block to wait for a reply. We’re given 8 bytes containing no references to any objects… and I bet those bytes describe a NULL parcel.
    • ioctl(3, BINDER_WRITE_READ, {write_size=0,write_consumed=0,write_buffer=0xa438,read_size=256,read_consumed=44,read_buffer=0xa328,write_data=[], read_data={[brNOOP,brREPLY({flags=0x00000000,priority=80,data_size=8,offsets_size=0,data={buffer=0x40008000,offsets=0x40008008, *buffer=[…]}})]}) = 0
  16. And we’re done.
    • writev(1, [{“Result: Parcel(NULL)\n”, 21}], 1) = 21
    • exit_group(0) = ?

Where to go next? I want to:

  • Figure out how to get strace to ‘see’ inside the mmap’ped /dev/binder, so I can see the replies coming back from the kernel (and thus ultimately from other processes
  • Play around a bit to see if I can work out any more (e.g. the meaning of the cookies, or whether the second word of the transaction body really is the UID, or whether ints make a difference to the other data in the body)
  • Use the knowledge gained from this to make a stab at understanding some of the (massive) logs gained from applying the same strace tools to runtime and Zygote and the things they spawn
  • See if I can build the Binder shell and run it on Android. It’s like ‘service’ only more sophisticated.
  • Look more into the known, documented stuff about the AIDLs and the Service classes on the Java (Dalvik) side. I’d like to think the opcodes are a direct mapping into an AIDL. I’d like to try creating my own service and interacting with it. I haven’t really looked into the Java side stuff so I’ve no idea if it’s possible.

The odds of me getting around to any of the above any time soon are slim!

syscall_983045

Monday, December 10th, 2007

Frequently appears in strace listings of what Android is up to. Appears to be setting the thread local storage.

I’ve also found that strace’ing Zygote and the things it spawns is (possibly predictably) much more fruitful in terms of IPC messages than strace’ing the runtime. However I have don’t yet have enough time to figure out how I dig down into the structure enough to find out the destination process for a transaction, from the data available. Once I get that far I’d like to think I can knock up a simple flow diagram of the messages passed between all the Android threads and we can start to get a decent picture of how the whole system operates.

One day…

Android versus Symbian OS

Saturday, December 8th, 2007

Here are some superficial thoughts on Android (which I don’t know much about) versus Symbian OS (which I do).

Android Symbian OS
Business model
Revenue stream for creator Advertising, fairy dust? Fixed license fee per phone (with a few quirks, all publicly known
Source code availability Available to nobody yet. Available to all, in theory, one day. Available only to Symbian OS phone manufacturers, plus partners who pay a lot.
Source code modifiability Freely modifiable (although I seem to remember constraints for Open Handset Alliance members, presumably related to keeping the APIs sufficiently similar between devices to keep a common platform for developers Modifiable, but with similar restrictions aimed at keeping a common platform.
End-user openness End devices will probably accept Java-language applications. No evidence that they will accept native applications. All end devices accept Java-language applications, and virtually all also accept native applications.
Current market position in smartphone market Zero, but has Google behind it Overwhelmingly dominant in EMEA and Japan, reasonable success elsewhere except America. (American technology writers just don’t realise Symbian exists, and that it’s America which is unusual…)
Technical Basis
API language Java C++ (J2ME Java also available for a subset of APIs)
Standards-compliance Highly POSIXy for native handset software. Proprietary for high-level Java APIs. Proprietary. POSIX layer available, but it limits interaction with the rest of the OS so much that it’s only really used for porting software.
Kernel Linux with minor changes Proprietary
Basic user library BSD-derived libc Proprietary
User interface Unknown; current is a ‘placeholder’ None; UIs developed by Nokia (S60), DoCoMo (MOAP) and UIQ/Motorola/Sony Ericsson (UIQ). Other UIs have existed in the past.
Typical filesystem YAFFS2 Proprietary.
Toolchain for native software Standard GCC, possibly requiring prelinking and other oddities Proprietary, built on top of GCC
Binary format Standard ELF Proprietary
Toolchain for typical third party software Ant-based, dex compiler, etc. Same toolchain for entire device
Debugging/profiling/investigation tools Standard UNIX tools Not a lot
Inter-process communication Unclear. D-bus for some layers. OpenBinder for others. Possibly something different again for messages flowing between the Dalvik-side processes. Proprietary; based on client-server.
Features
Pre-emptive multithreading Yes Yes
Memory protection between processes Yes Yes
Pre-emptible kernel Yes Yes
Demand paging Yes Yes
Virtual memory (page outs) Probably not according to Dianne Hackborn; they are anticipating devices with 64MB RAM and 128MB flash Not yet, but it may not be long; Symbian OS has been in use on phones with hard disks for some time
Shared libraries Yes Yes
Copy-on-write Yes N/A (‘fork’ is not used)
Reference counting of kernel objects and proper cleanup if a process dies Yes Yes
Approaches to problems
Flash space taken by many copies of symbol names for dynamic linkage Unknown; maybe just don’t use too many native programs Link by ordinal not by name
Memory allocation Allocate a big hunk of virtual address space for process. On write, try to free up enough physical RAM pages to give it the memory it needs. Allocate minimum virtual and physical address space; heap algorithms in user library know how to request more RAM from kernel.
Memory full behaviour Kernel will kill other processes if necessary to relinquish physical RAM. If no physical RAM is available, program is killed. On some Symbian systems, user-side heap library may request other apps to exit. If no memory really is available, malloc equivalent throws an exception and application should be able to handle it.
Memory management within user code Check for NULLs from malloc if you’re lucky, but you’ve probably got spare virtual address space anyway so it’s unlikely you’ll be told of a memory allocation failure Incredibly anal rules about memory management
Security Each process runs as its own user. Unclear how this prevents access to certain APIs that could do destructive things. Processes have capabilities and can only access APIs appropriate to their capabilities. Applications must be certified and are then signed such that they cannot access APIs beyond their capabilities.
Overhead of loading binaries Store application code as dex files which can just be mmap’ped in Store application code as native code which can just be mmap’ed in (or equivalent)
Overhead of interpreter startup Application interpreter starts once (Zygote) and all other processes fork from that No interpreter – native applications
Componentisation Software uses ‘intents’ to say what it wishes to do. Other software can fulfil the intent. Applications typically rigidly defined (but can load plug-ins).
XML and text files are inefficient Compile XML down to a binary representation Compile resource files down to a binary representation
Overhead of multiple threads Remove need for multiple threads, but allow developers to use them if they wish. Provide event loop which runs in main thread and can respond to most events. Allow events to be posted onto event loop using handler object. Remove need for multiple threads, but allow developers to use them if they wish. Provide object-oriented event loop in main thread (‘active scheduler’), using ‘active objects’ to respond to each type of incoming event.
Overhead of making APIs and libraries thread-safe Insist they are used only from main thread Run in a different process; use IPC for all UI requests. And indeed pretty much anything else.

Some of that’s probably rubbish, and may be based on misinterpreting bits of the interesting podcast. I’m happy to accept corrections, but then again, only six people read this blog anyway so this post was mostly for my own interest :-)

In other news, Motz has figured out the correct linker settings for the Android loader!

Android podcast notes

Saturday, December 8th, 2007

Notes from the recent Android podcast…

  • Dalvik VM can mmap dex files.
  • Zygote loads VM before forking, so copy-on-write effective and all apps share lots of pages.
  • ‘am’ command doesn’t work that way; can get idea of benefit; takes 2 seconds to launch because it has to load all classes.
  • Different user for each application. Your data is UID’d to you on the filesystem too.C
  • Content providers use URL scheme. Every bit of data on device has a unique name.
  • Intents are just a generic way of naming something to do. e.g. I’d like a content. Then system finds something like the contacts app. Similar for playing an MP3, any media player app can fulfil intent.
  • Intents are broadcast; anyone can reply.
  • Embedding in UIs not supported.
  • XML gets compiled down. Generic binary format for XML, the compiler can reduce any XML.
  • Reference HW has 64MB RAM, 128MB flash. No virtual memory practically. Yet Linux is aimed around VM. For example when creating a thread, Linux allocates a heap of virtual address space for the program (but only allocates physical pages later). If RAM unavailable, would have to kill app, can’t fail the malloc.
  • So component model allows kernel to identify which applications aren’t needed, and kill them if there are insufficient pages.
  • Threads: tried hard to avoid needing an app to have multiple threads. Everything designed to happen in main thread. But of course you need to remain responsive in main thread.
  • A service is not a main thread either. When you start a service it runs on the main thread still.
  • UI framework not thread-safe, for efficiency. Must access UI object from main thread. Use Handler object to do that.
  • Handler also allows timer services; can request an event in the future.
  • Current UI is somewhat of a placeholder

Updated openbinder.c for strace

Wednesday, December 5th, 2007

Here’s my latest openbinder.c file (use with the other changes in my previous post).

openbinder.c

Much more sensible output this time. I was right to be suspicious about all the nulls – it turned out I was only transferring the first byte of the data. But still no strings yet… next is to decode the replies from the kernel, then to read the kernel module source a little more closely to work out if we can decode any of the stuff in ‘buffer’. Plus, of course, to keep working on it to see if I’ve made any other daft mistakes :-)

Example output:

22:34:57.044758 ioctl(7, BINDER_WRITE_READ, {write_size=56,write_consumed=56,write_buffer=0x153f8,read_size=256, read_consumed=8,read_buffer=0x152e8,write_data= [bcINCREFS(target=0x00000015)bcACQUIRE(target=0x00000015) bcREPLY({cookie=0x00014130,code=0x00000000,flags=0x00000000,priority=80,data_size=12, offsets_size=0,data={buffer=0x00015970,offsets=0x00000000, *buffer=[0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0x00,0x00,0x00,0x00,............]}})], read_data={...}) = 0

22:34:57.867208 ioctl(7, BINDER_WRITE_READ, {write_size=40,write_consumed=40,write_buffer=0x153f8,read_size=256,read_consumed=8, read_buffer=0x152e8,write_data=[bcREPLY({cookie=0x00014130,code=0x00000000, flags=0x00000000,priority=80,data_size=24,offsets_size=4,data={buffer=0x00015e18, offsets=0x00014130,*buffer= [0xff,0xff,0xff,0xff,0xff,0xff,0xff,0xff,0x85,0x2a,0x68,0x73,0x08,0x00,0x00,0x00,0x0e,0x00,0x00,0x00,0x04,0xfe,0x4f,0x10, .........*hs..........O.]}})],read_data={...}) = 0

So far the only decent string I’ve seen in any of the buffers is /dev/input/event0… which might give another clue about where to head next.

Beginning to decode Android IPC

Sunday, December 2nd, 2007

I’ve had a quick look into the OpenBinder ioctls zinging around within the Android emulator. Here’s a summary of what I’ve done, and the results I’ve had (such as they are… it’s not been too fruitful).

First, the context. We know that Android IPC uses a system called OpenBinder, thanks both to Benno’s pioneering investigations, and the fact that one of the OpenBinder developers kindly posted a load of information on the web! But, we also know that the user-side OpenBinder is not based on the existing MPL version, or Google would have had to release the code already. (For this reason a lot of my terminology is probably wrong – it looks like perhaps the MPL version is OpenBinder, whilst we’ve got some other type of Binder, perhaps a not-quite-Open-yet-Binder).

In fact, the only code we’ve got relating to the OpenBinder in use within Android are the code for the module, provided within the Android Linux kernel. That includes two header files – include/linux/binder_module.h being the important one – and about 10 files code for the kernel-side driver.

According to the OpenBinder website, the kernel driver is responsible for reference-counting as well as the pure data-copying aspect of IPC. So, with any luck, the kernel-side code will have to understand the data structures passed back and forth, and therefore so can we.

Anyway, this is what I did.

  1. First, I modified strace to output some information about OpenBinder. I only bothered with the BINDER_WRITE_READ ioctl, and essentially just printed out the payload in binary, for the moment. My changes are here – sorry it’s not a nice easy patch, but all the makefiles etc. got changed too and I didn’t want to include them in the patch. strace-4.5.15-diffs
  2. I built this using my Scratchbox as per my previous instructions. You’ll need to run automake first to rebuild the makefiles to include the extra source code file.
  3. I then followed Benno’s instructions to boot Android without the automatic startup of Zygote and the runtime. I ran my modified strace in exactly the same way as Benno to get the output I wanted.

Enough! What are the results? Well… not very spectacular yet.

Here’s a typical ioctl:

ioctl(7, BINDER_WRITE_READ, {write_size=8,write_consumed=8,write_buffer=0x14eb0,read_size=256, read_consumed=44,read_buffer=0x14da0, write_data={0x4,0xc1,0xb,0x0,0x84,0xc1,0xb,0x0,}, read_data={0xe,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0, 0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0, 0x0,0x0,0x0,0x0,0x0,0x0,0x0,0xb9,0xfc,0x1,0x0,0xe,0x0,0x0,0x0,0xb1,0xfc,0x1,0x0, 0xe,0x0,0x0,0x0,0xa9,0xfc,0x1,0x0,0xe,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0, 0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0, 0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0, 0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0, 0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0, 0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0, 0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0, 0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0, 0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0, 0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,}) = 0

My thoughts about that (in no particular order):

  • I don’t like the fact that the write_buffer and read-buffer addresses are so small. I am not at all familiar with the Linux memory map, though, so maybe Linux process memory really does start all the way down there. And, their type is signed long. (In standard OpenBinder it’s void*, according to the documentation). Comments suggest that this strange type is used to allow for mixed 32/64-bit systems, but I still slightly worry that these parameters aren’t addresses at all, but could be offsets into some shared mmapped area or something.
  • However, each time I tried to fetch the data at this address, it worked. So this makes me think they’re real addresses.
  • I think these data areas should contain lists of commands and replies, respectively. The commands and replies are variable-length, so without some analysis it’s hard to work out second and subsequent commands/replies. The first byte is an opcode from BinderDriverCommandProtocol. I’m encouraged that the first byte is never 0x0, which would be bcNOOP, and is small enough to be likely to be an enumeration. So again, I’m happy these are the command streams.
  • But I’m surprised we’re seeing so much null data. I’d also hoped that we’d see some strings related to the objects being bound to. But perhaps they’re pointed to by pointers within the data stream (certainly, the main ‘transaction’ command usually contains pointers to the real data).
  • Maybe the read_data is usually blank; the client always allocates 256 bytes for a reply. Quite often the first opcode in the reply is 0xe, no-op, which might back that up. Although I’m not clear how the no-op operation causes the rest of the buffer to be ignored.

The code within the kernel module does parse and try to understand this data, and it does match the understanding above.

My overall conclusion: this approach does work, and it’s just a matter of getting time to implementing code within strace fully to parse the Binder command stream. I reckon we should be able to get as far as tracking individual object lifetimes, and what method are called on which objects. I reckon we should be able to get the names of the objects (as that’s part of the binding) but I haven’t yet worked out how binder specifies the methods which are called… it’s probably by ordinal into the interface rather than by name, so perhaps we won’t be able to work out what methods are being called. We’ll see, though. It would be great if we could see what methods were being called by one process on another.

You might wonder why I stopped here… mostly because I ran out of time, but also because I planned to do the analysis using Perl from this point onwards. But I think we’re going to need to access pointers to other bits of memory, so strace is probably the way to go.

(It might be easier just to rebuild the kernel with binder logging turned on, but I haven’t yet tried rebuilding the kernel for Android).

Here is an archive of the strace logs retrieved. Logs

I hope I will eventually get around to implementing much more code within strace to dig into what Binder is up to, although I reckon Benno’s bound to get there first :-)