Archive for November, 2006

More on that Symbian OS Parallels bug

Thursday, November 23rd, 2006

So, now I’m using this blog like my personal wiki, but perhaps there’s somebody out there who wants to know.

For the S60 3rd edition SDK, here’s my fix for the bug I talked about a while back.

Open up \epoc32\release\winscw\udeb\ecust.dll in your favourite hex editor. Find this bit:

0002c10: 894d bcc7 45c0 0009 3d00 6a0f c745 fcff .M..E...=.j..E..
0002c20: ffff ffff 1538 4241 0050 c745 fcff ffff .....8BA.P.E....
0002c30: ffff 153c 4241 008d 45c4 50c7 45fc ffff ...<BA..E.P.E...
0002c40: ffff ff15 4042 4100 83f8 000f 84a8 0000 ....@BA.........
0002c50: 008b 45c0 4848 4848 4848 4848 4848 4848 ..E.HHHHHHHHHHHH

See the 84a8 above?

That ’84’ corresponds to the second byte of the instruction :00402C4B 0F84A8000000 je 00402CF9, which jumps around the problematic code if QueryPerformanceCounter gives the wrong answer.

We want to always jump around the problematic code. So, change the 84 to 81. That will change it from “jump if equal” to “jump if not overflow”. My x86 assembler is useless, but hopefully the overflow bit will never be set, and the code will always be avoided. Parallels still haven’t replied to say whether it’s a bug in their VM or in Symbian OS or both.

Sorry to any non-techies reading.

There is a moral to this story, though, for anybody reading within Symbian: this is the sort of thing us ISVs have to go through, in order to get Symbian OS debugged without access to the source code. Great eh?!

Scrum

Tuesday, November 21st, 2006

Computing projects are typically managed using standard project-management methodologies, such as GANTT charts. This doesn’t work too well for a variety of reasons:

  1. In any non-trivial computing project it’s impossible to identify the subtasks involved until you’ve more-or-less done them
  2. Computing projects are particularly difficult for the clients to envisage, since they’re so abstract. Therefore, the requirements change throughout the life of the project.
  3. Interrupting a computer programmer, or changing their goal, causes them to forget lots of important information about what they were doing, and hugely reduces their efficiency. It’s said that diverting a programmer’s attention for half an hour destroys the equivalent of a day’s productivity.
  4. Project management is hard.

There are various methodologies in computing for “agile” development, which are intended to solve these problems and others. I only really know enough to use two of them – test-driven development and Scrum. Test-driven development is simple: the idea is, before you write the real code, you write the test code. This helps you think about the purpose and interface of your real code, and also means that when you finally write the real code, you can be certain when you have finished and it’s bug-free, because all the tests pass. Which is nice. It also fits the typical project lifecycle a bit better, because tests should be based off the design or specification rather than off the actual code, which means it makes sense to write them before you write the real code. (Of course, most project managers secretly don’t want you to write test code anyway, because it means your project takes longer. But they’ll all deny that.)

However Scrum is much harder to explain, as it’s a project management thing. It has a number of interconnected principles. Here’s some of them…

  1. There is a master set of requirements called the Product Backlog.
  2. The development team does everything in 30-day “sprints”
  3. The requirements for a sprint are set at the beginning and cannot be changed
  4. However a sprint can be cancelled if its requirements prove to be irrelevant
  5. There are defined-length meetings at the start of the sprint to set its objectives, selected from the Product Backlog. This implies an acceptance that the requirements cannot be perfect.
  6. The team manages itself within the Sprint. There is no pro-active management.
  7. All deliverables in a Sprint must be finished and ready to deploy in a product. e.g. documentation and test code must be ready. Therefore your customers can assess what you’re producing and affect your Product Backlog appropriately for future sprints.
  8. Progress within a Sprint is tracked using a “burndown graph” which represents the time remaining on each task in the Sprint. The time remaining on each task can go up as well as down, based on developers’ current estimates – but hopefully, across all tasks, it tends down to about 0 at the end of the 30-day sprint.
  9. There are defined roles for a Product Manager-type person and for a Scrum co-ordinator (who just enforces the process, rather than actively manages)

Here’s an example of a burndown graph. (The numbers along the bottom are the 30 days of the sprint, which Excel has eaten. Bless it. The y axis is ‘time remaining’ in hours).

Example burndown graph

There are other principles – and I’m sure that, to be fully effective, you can’t arbitrarily omit the ones you don’t like – but this subset seems to serve me well for a team of one: me. I’ve been introduced to Scrum by my contracting work. To be honest it works better there – too many of the Macrobug tasks are too experimental, and, a bit like test-driven development, it doesn’t work too well if you’re just flailing around trying to make things work. Still, I’m quite sure it works better than the typical approach with rigid GANTT charts, and I hope one day I get a chance to use it in a team of more than one!

Things I like:

  • The theory that, once you’ve planned your objectives for the 30 days, they won’t get changed and you won’t have to do lots of wasteful context-switching. This benefit only really applies if the organisation fully adopts Scrum, which my contracting job doesn’t really.
  • If you discover that it’s going to take more time to do a task than you expected, that’s fine. You just record it, and the graph will show your product manager that there’s going to be a problem. Much less troublesome than aiming for arbitrary targets.
  • The fact that you release things in 30-day increments which are supposed to be “finished”. That’s a bit like the well-known open source principal, “release early and release often”. It’s good.

In short, it seems like the best compromise between not interrupting the programmers, versus giving the customers/stakeholders frequent opportunities to affect the product progress.

Situation: normal

Tuesday, November 21st, 2006

Things are going OK in the world of Macrobug. Not great, but not really bad. Right now I am struggling on two fronts: one of them commercial, which I can’t really talk about, and the other is technical.

On the technical side I have apparently encountered a performance problem. I knew this was coming – I need to capture a lot of information and I fully expected that it would place unreasonable demands on the system being tested. I’m at that stage now, and need to work around it.

Anyway, I’m now considering going full-time on Macrobug. I still don’t necessarily believe I can make a business out of it, but I’m increasingly sure I’ll never know whilst I’m working part-time. So, for a couple of weeks (illogically?) I’m going to be contracting full-time in order to bring in some cash that I’ll be able to live off, whilst I try to get Macrobug going.

To be clear: this does not mean that I think I’ve overcome all the potential obstacles to Macrobug. Far from it! But there we go: I think I should still give it a try.

This plan might all radically change any time, of course :-)

Another Parallels tip

Thursday, November 9th, 2006

I do a lot of my work in the Windows side of my Mac, with Windows occupying the whole screen. For that reason, I can’t see all the usual notifications of incoming events on the Mac side (e-mails, Skypes, chats etc.) I’ve found that Growl is a good solution to this. Growl is a general Mac notification system, into which many applications plug. Its nicely-crafted bubbles of incoming events appear even on the Windows side of the computer.

Hints for development under Parallels

Saturday, November 4th, 2006

A couple of hints for those using ParallelsI use a Mac. For all my Symbian development needs, I use a virtualization product called Parallels running Windows XP. It’s great!

Hint One: networking from Windows to the Mac

I keep my source code in a Subversion repository running under MacOS X. Accessing this from the PC environment is fine, but for the fact that the IP address of the Mac changes. That’s particularly true of the latest version of Parallels, where there’s a special ‘host only’ mode which enables the two environments to communicate even when there is no external network. However, installing Bonjour for Windows appears to allow you to refer to your Mac using its local DNS name. For example, my Mac appears (to my PC) as adrian-taylors-macbook.local. Even to things as obscure as the Subclipse Eclipse plug-in which I’m sure has no special knowledge of Bonjour.

Hint Two: Creating a TrueCrypt volume

TrueCrypt is a system for encrypting whole volumes under Windows or Linux. Such a volume can either be contained in a file on some other volume (as is normally most appropriate under a normal Windows PC), or a whole hardware device or partition can be encrypted.Under Parallels, the latter works beautifully. Just configure Parallels to have an extra hard disk:

Extra Hard Disk in Parallels

TrueCrypt has an option to encrypt a whole device in its encrypted volume creation wizard. However, if you select a whole volume it will warn you that Windows may overwrite the first few bytes of the drive. So you must first create a partition on it. Any old FAT or NTFS partition will do. Use Windows Control Panel -> Administrative Tools -> Computer Management -> Storage -> Disk Management (an obscure location!) Once you’ve created the FAT/NTFS partition, you can then tell TrueCrypt to create an encrypted volume on this partition.

Creating an encrypted TrueCrypt volume

  It works beautifully and is reasonably fast – the Symbian emulator only takes a few seconds to load even when none of it is cached.(Note that I haven’t actually tried using a TrueCrypt volume in a file on my main Parallels volume, so it may be quicker, but I’d be very surprised).

Hint Three: Getting the Symbian emulator to actually boot

I have found two problems getting the Symbian emulator to boot under Parallels. About half the time, it crashes before even displaying its window. After some investigation, it appears that the emulator is getting a high-resolution time from Windows, then executing 400000 instructions, then getting the time again – in order to determine how quickly the host PC executes instructions. The problem is that Parallels’ high-resolution timer appears to be insufficiently high resolution*, and the times come back the same. This confuses the code which fails with a divide-by-zero exception. There’s no particular workaround unless you have access to Symbian OS source code (in which case, comment out Wins::CalibrateCpuSpeed() within src\cedar\generic\base\wins\specific\variant.cpp, cd .., bldmake bldfiles, abld build winscw udeb vwins). However I’ve e-mailed Parallels to see if they can improve their timer, and will let Symbian know too.

The other problem is that, under certain Symbian OS emulators, they crash when they discover the PC has no serial port. (They try to use it to find a modem). To fix this, simply add a virtual serial port under Parallels. It can be connected to a file, or anything… that solves that problem.

(* I have no evidence that their timer is lower resolution than a real PC. Maybe my PC is just very fast, and lots of people with Core 2 Duo PCs are having trouble with the Symbian OS emulator.) 

Parallels is, incidentally, great. It is absolutely as fast as a real PC in normal usage, and even uses some of MacOS X’s cool graphics effects when you switch between Mac and PC – the whole screen ‘flips round’ in front of you.

Perspective

Saturday, November 4th, 2006

I’ve been a bit grumpy of late because I feel my effort hasn’t really been getting me anywhere. Really this is just due to a lack of perspective about the overall development task, and where I am in it.

Yesterday I found a cunning way to give myself that perspective… look at the project plan I wrote months ago!!

I feel much better now. It turns out I have actually achieved some stuff in the past month or two.

My contracting job has also converted me from the traditional GANTT-chart project methodology to the new-fangled buzzword-compliant agile-whatever-that-means Scrum methodology. (I may take the mickey, but I do actually think it’s quite good.) So I’ve also gone through and converted it to a Scrum “product backlog”. This might also mean I look at my project plan more frequently than once every two months :-)

I might write more about Scrum some day.

MacBook Pro, and why I keep going on about it

Friday, November 3rd, 2006

There were a number of problems with trying to do development on my old laptop:

  1. It was Claire’s and she wanted it back.
  2. The screen was 1024×768, which isn’t quite enough to productively fit all the Eclipse windows in.
  3. It was slow.
  4. It had no Z key.
  5. It turned off randomly (quite frequently too)
  6. It had woefully little RAM: 512MB. That’s enough for one session of Eclipse/Carbide. It’s just about enough for one session of Eclipse/Carbide plus one copy of Symbian OS being debugged. But it definitely wasn’t enough for a session of Eclipse/Carbide debugging another session of Eclipse/Carbide, debugging Symbian OS.

For all these reasons, my change-test-debug cycle was at least 15 minutes. That’s madness. It’s impossible to avoid getting distracted whilst Eclipse is taking 10 minutes to load. So that’s why I’m very excited that I’ve got a new Intel Mac laptop which can run Windows via Parallels. (And is currently doing so!)

Windows licensing

Friday, November 3rd, 2006

There are two versions of Windows XP Pro: OEM and full retail. They’re identical, but the OEM version is only valid when sold with a new PC, in theory. On the grounds that I want to be fully legitimate, I got the (more expensive) full retail version.

Alas, the CD was scratched.

Stewart kindly loaned me his XP CD, but like everyone elses’ in the planet, it was an OEM version. So I couldn’t install it. It wouldn’t accept my product activation key.

Massively annoying. I try to do the right thing by Microsoft by getting the expensive version, then they mess me about by not accepting my (expensive, full) activation key for the cheaper version of Windows. I can understand it the other way round, but this way round sucks.

Obviously when I phoned up customer service they were totally unwilling to help. Grrr.

I know I’m ranting a bit here, and that it’s not really so bad, but there we go. It annoyed me because I really wanted to get Windows working on this MacBook Pro.

Fortunately, in the end, the third computer I tried was able to read the CD and, a day after I started trying, I got Windows installed. Hooray! (It’s not even as if I like Windows… oh for a MacOS-compatible Symbian OS development environment.)

Personality types

Wednesday, November 1st, 2006

I am the sort of neat person who likes to get everything “ship-shape and Bristol fashion”. My wife will tell you that I am too tidy. Funnily enough I wouldn’t agree with that, but I would agree that untidy things nag at me until they’re fixed.

I have tended to think this is a good trait, but I am becoming increasingly sure that this personality type is somewhat incompatible with trying to make a start-up. There’s simply so many niggly things to worry about (as you’ll have gathered from this blog) that to be on top of them means you have no time to actually start your start-up (as you’ll also have gathered). I think that somebody like me has to let some things go, and ignore some very important matters, or they will be unable to get their company off the ground.

I suspect this is the reason why lots of small companies get in trouble with the HM Revenue etc. for not filing tax returns on time, for example. Or (I suspect, though I have no evidence) why a lot of them don’t make enough backups and lose all their data. There’s just no alternative: you get nowhere if you look after everything.

I think I am on top of most of this stuff now, but I am not sure I should have been quite so diligent. I should probably have just let things go wrong, and waited until I got angry letters from HMRC before thinking about tax, accountants etc. And waiting until I lost my data before making a back-up strategy. And waiting until I got sued by Symbian before starting to negotiate about IP, licenses etc.

Oh well. I’d still prefer to be in the situation I’m in now, I think. Just. In some ways I’d rather be at the bottom of that organisational mountain, if it meant I was at the top of my initial-coding-phase mountain. A mountain that just doesn’t seem to get any smaller.