Archive for the ‘Business’ Category

News from Macrobugland

Tuesday, September 22nd, 2009

Some of you may have heard that I have taken a full-time job at another company, so Macrobug is proceeding as a background activity. But that’s not all the news!

  • The stray event scanner has morphed (finally!) into a generic static analysis tool. As well as checking for all kinds of active object problems, it can find problems related to Symbian OS kernel handles… and in fact, anything else – it’s fully extensible! (For the techies – all you need to do is define an Eclipse plug-in which does two things. First, you define certain ‘actions’ taken by different function calls; secondly, you define a ‘device state model’ which gets fed every possible sequence of those actions called by the code you’re analysing).
  • Although it’s terrific, this generic static analysis tool probably isn’t as terrific as some of the existing tools out there. So Macrobug is also hoping to convert its existing analysis engines – for active objects and kernel handles – into checkers for an existing product. Perhaps a commercial one, or perhaps something open-source. If you’re interested, get in touch!
  • Macrobug (the company) might be starting a small extra website on the side soon… something more leisure oriented. Watch this space!

If you’ve been reading since the days I first started the company, thanks! Can I recommend that you take a look at the blog of Transmission Begins, a company run by my good friends John and Morgan?

Paternity leave and perspective

Monday, January 19th, 2009

Probably the greatest thing about being self-employed (or being a director of one’s own company, no matter how big it is) is the flexibility to take time off when it suits you, as opposed to the rules put in place by your organisation. So, I’ve taken the whole of January off as paternity leave. It’s great. I feel greatly sorry for my friends who are also becoming Dads, and have had to go back to work after just two weeks. It can’t be good for them, or their kids, or probably their work either – I know how sleepy I’d be if I had to stick to 9-5 hours.

Anyway, this month off is also the longest period I’ve had away from my Macrobug work – ever. Unsurprisingly, that gives a good sense of perspective about the endeavour.

When I started off, I actually had a fairly solid game-plan. It was more detailed than I ever wrote in this blog, and ultimately would have led to Macrobug being the domain specialist for a particular type of tool, across all platforms on all types of devices. Starting on Symbian OS was a good stepping-stone because the tool in question was more easily possible on Symbian OS, and in fact would have been most useful on Symbian OS compared to other platforms.

However, problems reared their ugly heads. Serious commercial development on Symbian OS is expensive, ultimately. It involves paying Symbian heaps of money to get a version of Symbian OS with source code (or, in my case, spending equivalent heaps of time trying to negotiate a free version with various conditions attached), as well as paying a lot of money for the compiler etc.

It would be nice to think that with Nokia’s purchase of Symbian, and the resulting spirit of openness, many of these problems would disappear. I do believe they will. But this has been accompanied by a marked reduction in my customer base. Previously I was aiming at Symbian, UIQ, Motorola, Sony Ericsson and ultimately I hoped to market to Nokia once I’d had experience. Of that list, only Nokia and Sony Ericsson are left (and most of the bits of Sony Ericsson I know have disappeared too). I have no suitable contacts in the other Symbian OS licensees which are sure to pop up. Symbian’s merge with Nokia has also changed parts of their tools roadmap on which I was relying.

The other thing that became clear about my original tools plans was that their benefits are not easy to quantify financially. I can’t definitively say “this tool will save you £100,000 per year”. That’s a serious problem, because even though the tools are really useful and would save heaps of time (and money) for their purchasers, it’s hard to prove that.

So developing the tools that I had originally in mind on Symbian OS looks tricky. In the mean time, then, I’ve been doing other things. I’ve developed a completely different series of tools. They’re very Symbian-specific, which is a shame. But the benefits they produce are financially quantifiable, and I have successfully sold them. In addition, of course, I’ve been doing lots of tools-related contracts and trying to establish Macrobug as a place to come if you have a need for a Symbian-related tool. This has been moderately successful, and actually very enjoyable. (That’s why I’m only now getting this sense of perspective when I take time off). But as everyone knows, doing endless contracts does not equal a career.

The perspective I’ve gained from my month off, then, suggests that I’m at a crossroads. Do I:

  1. Continue doing contracts a lot of the time.
  2. Develop tools which are Symbian-specific but whose benefits are financially quantifiable. Successfully sell those tools – maybe.
  3. Develop tools which correspond to my original plan, and could enable Macrobug to grow to be a cross-platform vendor, dominant in its own small field.
  4. Give up and do something else, probably a full-time job.

I don’t know the answer. I fear that I might have to go for the third option if I have any ambition!

But having kids when self-employed also has another impact: it makes it very appealing to go for the financially safer options – which are any of the others!


Thursday, December 11th, 2008

The smartphone software market has been busy lately. Android comes along; Nokia open-sources Symbian OS; the iPhone becomes wildly popular.

Ten years ago, Symbian was formed. It was the “open” alternative. Now, people look at Symbian and – until the Foundation announcement – ask what on earth is open about Symbian? Surely it’s more closed than Android etc.? I get asked these questions all the time.

Ten years ago, things were easy. The alternatives were closed platforms built by the phone manufacturers themselves: NOS, OSE etc. They had no APIs for developers (well, Java on phones was just beginning to take off) and they were entirely closed. Symbian was open in two key ways: firstly, it had APIs which third parties could use to add to the device software; and secondly, all the source code was available to the licensees, and the development costs could therefore be shared. That was radical ten years ago in the mobile marketplace!

Today, the industry has moved on. “Open” today doesn’t mean just sharing source code between a few large companies – it means publishing it on the web and letting anyone change it. But we’re going to see the same battle fought on exactly the same things: the openness of the APIs, and the degree to which the code is open source. Again, the Symbian Foundation is pushing the envelope. On both counts, it is in principle more ‘open’ than any of the alternatives.

Allow me to explain by way of a diagram.

Ade’s openness graph

Some caveats for nit-pickers.

  1. Symbian (pre-foundation) had different API classifications – available to all, partners only, or nobody. Even if you were using just the ‘available to all’ category, it still had more APIs than Android (where you can’t run native code on real devices) or the iPhone (where you can’t even run a background task). When the foundation becomes active, presumably all the ‘partners only’ APIs will become available to everyone, which will wipe the floor with the competitors.
  2. No, you really can’t run native code on Android devices. Yet..
  3. No, you really can’t run background processes on an iPhone..
  4. I have no idea about LiMo and friends. I’m assuming they fit the mould of Motorola’s Linux Java platform, where the kernel is GPLed but the only APIs available are Java. This may be a gross disservice to LiMo, as I think it is intended to have native APIs. But to be honest, I don’t count it as one of the major players at the moment.

What does this tell us?

For one thing, Android is substantially less open than Symbian OS on both counts. This may change, of course. But right now, there’s nothing to stop handset makers taking the Android code, and altering it willy-nilly to create purely proprietary software. Furthermore, you, as a third party developer, can’t run native code. So you can’t port your existing software. You can’t talk to hardware, beyond the ways that Google gives you. You certainly can’t run emulators or different execution environments such as Python.

Even before the Foundation move, it could be argued that the Symbian APIs were more open than the Android ones. Symbian and Nokia have jumped through a lot of hoops to produce a reasonable POSIX compatibility layer which enables lots of existing software to work on Symbian devices, relatively unchanged.

So: Symbian currently looks like the most open platform. Sadly, Symbian has some major practical issues to sort out which prevent the platform from appearing open. In addition, the APIs are certainly hard to develop against (but since Symbian is the only platform that allows different runtime environments such as Python, it could be argued that doesn’t matter in the long term).

Finally, credit where it’s due: Android and Google really forced Nokia to make this change. Full credit to Nokia, though, for such a bold step.

(All trademarks and logos are owned by their respective owners).

Results of Stray Event Scanner evaluation at a customer site

Tuesday, August 19th, 2008

As I mentioned, about a week ago I visited a customer site to find out if our Stray Event Scanner tool actually did the trick, on their codebase. That customer has kindly said I can post some results here.

Results were mixed!

Proportion of this customer’s codebase scanned 5% (estimate).
False positives reported by the tool 47
Of which are things I might be able to fix 37
are intrinsic limitations 8
are already fixed in a later development version of the tool 2
Interestingly, 35 of these false-positives were in one smallish area of code. It seems that some Symbian OS code conforms to the norms expected by the tool, whilst some code really gets is unnecessarily overexcited. All the rest of the code we scanned only produced 12 false-positives.
Real bugs found by the tool 14
Of which tool was precise about the problem 6
tool was wrong, or vague, about the specific problem but did identify the right bit of faulty code 8
Actual bugs filed in this customer’s defect-tracking system 10 (because the other four were duplicates)

Conclusions. Good things first…

  • As expected, the tool found plenty of bugs. Nearly all of these bugs were real problems which could cause E32USER-CBase 46 panics under some circumstances, which are awful to diagnose.
  • In fact one of them was a duplicate of an existing E32USER-CBase 46 bug, which they’d been searching for for some time, but had been unable to find. Result!
  • The tool broadly worked as planned. There were no hopeless incompatibilities with their source code.
  • With the exception of that one component, there were far fewer false-positives than I realised.

Bad things…

  • There were lots of false-positives in that one component. Many of these will be impossible to remove using the current analysis method.
  • In some cases the tool was not very accurate about specifying exactly what was wrong with the code. It would identify the right area of code, but a lot of examination and thought was required to identify whether there really was a problem there. Again, there’s not much that can be done about this: stray event problems are intrinsically complicated and somebody technically experienced will be required to analyse the results.

Next steps…

  • See how many of those false-positives I can remove easily
  • Fix a few other minor bugs that cropped up
  • Decide whether the tool is ready to leave the beta stage!

Off to demonstrate tomorrow…

Sunday, August 10th, 2008

A customer has given me a contract to deploy my tools on their codebase, to see whether they come up with useful results. After that, maybe they’ll talk about licencing the tool… very exciting! But of course I’ve failed to fix all the things I had wanted to fix in the time leading up to this demonstration, so I know it won’t meet their criteria, quite. Sugar!

Stray Event Scanner Launched

Wednesday, July 9th, 2008

Macrobug is proud to present its first product for actual sale: the Stray Event Scanner. This is a source code analysis tool which does one thing, and does it well: it finds the causes of the dreaded Stray Event panic. Why are they dreaded? Well, by the time the crash has happened, the reason is lost in the mists of time, so there’s no way to diagnose them except to read reams of source code and spot the mistake.

The Macrobug Stray Event Scanner does that for you. It uses GCCXML to build your source code, and then analyses the results to ensure that each any every time you make a request, you are somehow getting ready for the reply to be received.

Problem marker from Stray Event Scanner

And yes, the problem messages are so descriptive they don’t even fit on this website.

Download the demonstration version now, or buy the real one!

Crashing – how not to do it

Wednesday, July 9th, 2008

If you’re a Symbian OS device creator (that means, you make phones) you’ll want to read this paper. It includes interesting statistics about which crashes occur most often during the device creation process, and tips about which products can identify them early using source code analysis. Of course, the conclusion is, you should buy our new product – but also, you should buy Coverity™ or Klocwork™!

Imminent launch!

Wednesday, July 9th, 2008

Well, after years of procrastination, it’s about to happen.

This is it.

I’m about to release a product.

In fact, it’s already been available for download from this website for weeks, and has been diligently tested by a couple of testers (special thanks to Adrian in particular!) It’s still nowhere near final-quality, but it’s good enough that I want more people to download it and play with it. And, if people like it enough to buy it, that’s fine with me.

So, Macrobug enters a new phase – the terrifying phase of actually having to sell things. I’ll try to post relatively frequently to recount my experiences!

First steps:

  1. Post on blog – check!
  2. E-mail lots of contacts about it – check!
  3. Cross fingers very hard… check…

It’s springtime!

Tuesday, April 29th, 2008

Eh? What? Adrian’s blogging again?

Yes. Macrobug has not gone away, just been dormant for the past few months whilst I’ve been working on other things.

But, fear not, because I am about to launch a product!

Android versus Symbian OS

Saturday, December 8th, 2007

Here are some superficial thoughts on Android (which I don’t know much about) versus Symbian OS (which I do).

Android Symbian OS
Business model
Revenue stream for creator Advertising, fairy dust? Fixed license fee per phone (with a few quirks, all publicly known
Source code availability Available to nobody yet. Available to all, in theory, one day. Available only to Symbian OS phone manufacturers, plus partners who pay a lot.
Source code modifiability Freely modifiable (although I seem to remember constraints for Open Handset Alliance members, presumably related to keeping the APIs sufficiently similar between devices to keep a common platform for developers Modifiable, but with similar restrictions aimed at keeping a common platform.
End-user openness End devices will probably accept Java-language applications. No evidence that they will accept native applications. All end devices accept Java-language applications, and virtually all also accept native applications.
Current market position in smartphone market Zero, but has Google behind it Overwhelmingly dominant in EMEA and Japan, reasonable success elsewhere except America. (American technology writers just don’t realise Symbian exists, and that it’s America which is unusual…)
Technical Basis
API language Java C++ (J2ME Java also available for a subset of APIs)
Standards-compliance Highly POSIXy for native handset software. Proprietary for high-level Java APIs. Proprietary. POSIX layer available, but it limits interaction with the rest of the OS so much that it’s only really used for porting software.
Kernel Linux with minor changes Proprietary
Basic user library BSD-derived libc Proprietary
User interface Unknown; current is a ‘placeholder’ None; UIs developed by Nokia (S60), DoCoMo (MOAP) and UIQ/Motorola/Sony Ericsson (UIQ). Other UIs have existed in the past.
Typical filesystem YAFFS2 Proprietary.
Toolchain for native software Standard GCC, possibly requiring prelinking and other oddities Proprietary, built on top of GCC
Binary format Standard ELF Proprietary
Toolchain for typical third party software Ant-based, dex compiler, etc. Same toolchain for entire device
Debugging/profiling/investigation tools Standard UNIX tools Not a lot
Inter-process communication Unclear. D-bus for some layers. OpenBinder for others. Possibly something different again for messages flowing between the Dalvik-side processes. Proprietary; based on client-server.
Pre-emptive multithreading Yes Yes
Memory protection between processes Yes Yes
Pre-emptible kernel Yes Yes
Demand paging Yes Yes
Virtual memory (page outs) Probably not according to Dianne Hackborn; they are anticipating devices with 64MB RAM and 128MB flash Not yet, but it may not be long; Symbian OS has been in use on phones with hard disks for some time
Shared libraries Yes Yes
Copy-on-write Yes N/A (‘fork’ is not used)
Reference counting of kernel objects and proper cleanup if a process dies Yes Yes
Approaches to problems
Flash space taken by many copies of symbol names for dynamic linkage Unknown; maybe just don’t use too many native programs Link by ordinal not by name
Memory allocation Allocate a big hunk of virtual address space for process. On write, try to free up enough physical RAM pages to give it the memory it needs. Allocate minimum virtual and physical address space; heap algorithms in user library know how to request more RAM from kernel.
Memory full behaviour Kernel will kill other processes if necessary to relinquish physical RAM. If no physical RAM is available, program is killed. On some Symbian systems, user-side heap library may request other apps to exit. If no memory really is available, malloc equivalent throws an exception and application should be able to handle it.
Memory management within user code Check for NULLs from malloc if you’re lucky, but you’ve probably got spare virtual address space anyway so it’s unlikely you’ll be told of a memory allocation failure Incredibly anal rules about memory management
Security Each process runs as its own user. Unclear how this prevents access to certain APIs that could do destructive things. Processes have capabilities and can only access APIs appropriate to their capabilities. Applications must be certified and are then signed such that they cannot access APIs beyond their capabilities.
Overhead of loading binaries Store application code as dex files which can just be mmap’ped in Store application code as native code which can just be mmap’ed in (or equivalent)
Overhead of interpreter startup Application interpreter starts once (Zygote) and all other processes fork from that No interpreter – native applications
Componentisation Software uses ‘intents’ to say what it wishes to do. Other software can fulfil the intent. Applications typically rigidly defined (but can load plug-ins).
XML and text files are inefficient Compile XML down to a binary representation Compile resource files down to a binary representation
Overhead of multiple threads Remove need for multiple threads, but allow developers to use them if they wish. Provide event loop which runs in main thread and can respond to most events. Allow events to be posted onto event loop using handler object. Remove need for multiple threads, but allow developers to use them if they wish. Provide object-oriented event loop in main thread (‘active scheduler’), using ‘active objects’ to respond to each type of incoming event.
Overhead of making APIs and libraries thread-safe Insist they are used only from main thread Run in a different process; use IPC for all UI requests. And indeed pretty much anything else.

Some of that’s probably rubbish, and may be based on misinterpreting bits of the interesting podcast. I’m happy to accept corrections, but then again, only six people read this blog anyway so this post was mostly for my own interest :-)

In other news, Motz has figured out the correct linker settings for the Android loader!