What makes people stop looking objectively at the end result?
Today’s mental conundrum:
January 5th, 2012Dee Hock Quote on Compensation
December 7th, 2011Money motivates neither the best people, nor the best in people. It can move the body and influence the mind, but it cannot touch the heart or move the spirit.
Writing your own Geohash algorthim, part 2
November 16th, 2011Ok, so earlier, I started investigating how to write a geohash algorithm in Javascript. I’m sure you’ve been waiting in eager for the results. Well dear readers, now is the time.
Below, is a geohash algorithm, based on the Maidenhead Locator system but with a few tweaks.
Firstly to get more accuracy with less characters, I replaced the complicated Maidenhead codes for different characters with one set of 64 different characters for each character. They range from the lowercase alphabet to the uppercase, then the base 10 numbers, and the greater than and less than symbols.
With this method, effectively the globe is split into 4096 squares (64×64) and then the longitude and latitude are interleaved together, so every odd character is Longitude and every even character is Latitude. The process is then recursive, based on the level of precision you set. So after the first square is picked - 2 characters, the selected square is then split again into 4096 squares and stored as another 2 characters, this goes on so on and so forth.
A 1 degree arc of the earth’s surface is roughly 111,000 meters long, which means you need 5 decimal places to get around 1 meter worth of accuracy with your location. The following code takes a 5 decimal place longitude and effectively transforms it into a geohash and back again.
The result it, we can take the latitude, 47.64932, convert it into a geohash of kwGK and then back again to 47.64932.
In part 3, I will looking at creating a formal proof of this algorithm, possibly using induction proofs.
var lat,
latMin,
latMax,
base64A,
reduce,
calculateBoundaries,
reverseGeoHash,
geoHash,
reversed;
latMin = -90;
latMax = 90;
base64A = "<>0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
lat = 47.64932;
reduce = function(boundaries, bucketSize, value, precision, geohash) {
boundaries = calculateBoundaries(boundaries, bucketSize, geohash);
var bucket = Math.floor((value - boundaries.min)*(bucketSize/(boundaries.max-boundaries.min)));
return precision-1 > 0 ? base64A[bucket] + reduce(boundaries, bucketSize, value, precision-1, base64A[bucket]) : base64A[bucket];
};
calculateBoundaries = function(boundaries, bucketSize, geohash) {
if(geohash.length > 0) {
var lastGeoHash = base64A.indexOf(geohash[geohash.length-1]);
var range = boundaries.max-boundaries.min;
boundaries.max = (lastGeoHash+1)*(range/bucketSize)+boundaries.min;
boundaries.min = (lastGeoHash*(range/bucketSize))+boundaries.min;
}
return boundaries;
};
reverseGeoHash = function(boundaries, bucketSize, geohash) {
for(var i = 0; i < geoHash.length; i++) {
boundaries = calculateBoundaries(boundaries, bucketSize, geohash[i]);
}
return Math.round((((boundaries.max-boundaries.min)/2)+boundaries.min)*100000)/100000;
};
geoHash = reduce({min:latMin, max:latMax}, base64A.length, lat, 4, 0);
reversed = reverseGeoHash({min:latMin, max:latMax}, base64A.length, geoHash);
console.log(geoHash, reversed, lat);
Conway’s Law
November 9th, 2011Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure.
Writing your own Geohash algorthim, part 1
October 26th, 2011Sometimes you find yourself with a random computing science seeming problem that you just want to roll up your sleeves and get you hands dirty with. I had one of those moments, about 4 months ago.
Now, four months may seeeem like a really long time, but as a classic lean saying say’s, results are not the point. It may have taken four months, but in those four months I learned some valuable lessons about computer programming. I thought it would be worth sharing them.
The problem.
The first part of the process is to frame the problem. On maps.nokia.com we have some functionality that means whenever you drag the map, the url bar updates with the new latitude/longitude.
Which is fine.
Unless.
You live in China.
In China you are not allowed to display latitude/longitude. Instead you must hide it from the sight of your viewers, incase they see it and realise the world is not the distorted place they’ve been told it is by their government. Go figure. So, my problem was how to save the location, via the url bar, without using the lat/long.
Googling this topic leads you to three likely looking solutions.
Geohash
The geohash appears to be a relatively new (2008) discovery, invented by Gustavo Niemeyer, you can find the implementation at Geohash.org.
I spent sometime replicating this technique in Javascript, essentially, it involves, systematic storing of deltas (or I guess diff’s for people who know source control). The starting premise is that for example, a latitude exists between -90 and 90 degrees. The first delta identifies whether it is in the first or second half of that range. So if I took the latitude, 52.51607, I would programatically ask, is that between -90 and 0 or 0 and 90, clearly it would be the latter so my first delta would be 1, if it had been -52.51607 then my delta would be 0. Doing this recursively I can then create a pattern of 1’s and 0’s that slowly drill down to my desired latitude. Then, once i’ve done that, I can take that binary string and convert it to integers between 0 and 32 and then using base32 represent it as a string.
It means for the coordinate pair 57.64911,10.40744, you end up with a geohash of u4pruydqqvj which is a saving of 6 characters (including points and the comma delimiter). Not bad. But what are the other options?
Patent No : 7, 302, 343
Working for Nokia, I have luxury of being able to tap into a vast treasure trove of both Microsoft and Nokia patents without the fear of being sued. Now, I’m not saying I think software patents are a good thing, but it was a relief when I found this patent which is described as compact text encoding of Longitude/Latitude was owned by Microsoft rather than Apple. It seemed to be exactly what I was looking for.
This technique takes the long/lats and converts them into non-negative numbers. So for example 47.64932 converts to 18,583,657 to be honest, don’t ask me why this works, I followed the formulas in the paper and boom, got the right number, but the theory behind converting to a non-negative number is lost on me. Then with this integer, you can generate a base-n (so in this case they used base32 again) string.
So with the example co-ordinate pair, 47.6493, -122.12926 you end up with a hash of ry7cx4tp95, a saving of nine whole characters! Pretty damn impressive, but there was one final method I wanted to research, mainly because it was named after a place in my homeland.
The Maidenhead Locator System
Developed in 1980 in Maidenhead, England, this system was designed by VHF managers and used by amateur radio enthusiasts. The Maidenhead Locator system was created with the use case of being able to transmit a location within 12km accuracy in a simple 6 character string using morse code.
It essentially works on the idea of converting the world map into a grid, based on degrees and assigning them a letter or a number. Each character in the string and precisely positioned and different locations have different boundaries. An example of a Maidenhead Locator system hash is:
BL11bh16
Stolen from wikipedia, below is a run down of what each of the characters does.
- Character pairs encode longitude first, and then latitude.
- The first pair (a field) encodes with base 18 and the letters “A” to “R”.
- The second pair (square) encodes with base 10 and the digits “0″ to “9″.
- The third pair (subsquare) encodes with base 24 and the letters “A” to “X”.
- The fourth pair (extended square) encodes with base 10 and the digits “0″ to “9″.
Interesting, a system invented in the 80’s that achieves a small, human readable hash of a lat/long with only 8 characters but alas with one large draw back, which is that it’s not as accurate as I need.
Having invested a large amount of time replicating in Javascript each of these techniques, I realised that what I need was a hybrid system, using the Maidenhead locator concept as a basis. In part 2, I will explore the javascript I need to write a geohash which matches the precision of the Geohash and Microsoft patent solution with the number of characters from the Maidenhead system.
Magic potions Vs. Continuous improvement
September 15th, 2011In order to understand something, sometimes it’s easier to understand it by defining what the opposite is.
Recently, I was trying to explain to someone the fundamental principle behind continuous improvement was. Do you ever run into the scenario where you find yourself in a conversation and you hear:
“We need to build an X, it will solve all our problems and make our code bug free.”
Conversations like this, immediately sound off alarm bells in my head, because what’s essentially being suggested is a magic potion, a concoction to cure all ills, an elixir of eternal life or just plain old powered tiger scrotum.
The overuse of hyperbole and setting truly unrealistic expectations in software development only results in continuous disappointment, and slowly kills a team spirit and stakeholder trust.
Aim for steady incremental continuos improvement with small achievable micro goals, know when to pat yourself and your team on the back when they have achieved them, and make sure going off solo on a magical quest to slay a dragon get’s stigmatised and not rewarded.
What’s the standard?
August 8th, 2011When writing code we spend a lot of time standardising things. Language specifications mainly. General syntax. API’s. Data Formats. The Web. We standardise all manner of things to make it more convenient for anyone from anywhere to make things that everyone can use.
Being so used to all these standards, I was quite surprised when I suddenly realised, that we don’t standardise Country names.
Think about it, in English, the country Germany is named, well, Germany. In German, it’s called Deutschland, in French it’s called Allemagne, in Danish it’s called Tyskland - and these are all countries that are next door! I haven’t even begun to look-up what German is in Arabic or Chinese.
So who’s correct? Well, the name that the country gave itself of-course!
Quite frankly, it puzzels me why anyone would even go to the extent of creating a different names from the ones they were given. I can just imagine the conversation:
“hello!, where are you from?”
“I am from español.”“Oh really? Ah yes, I see….
Well sorry old boy, español doesn’t really have a very British ring to it, how about we call you… hmmm, Spanish?Yes you look like a Spanishy sort of a fellow. Now on to the next, where are you from… Éire?… hmmmm
Anyway, I’m sure each of these names have a long established tradition and reason behind them, not just arbitrary like I suggest, but really, doesn’t this ring of complexity for complexities sake? Can’t we go about tidying up the languages we use the most?
You are solving the wrong problem.
July 27th, 2011I really like stories.
In fact, I more than like them. Stories are an amazingly versatile tool for passing on wisdom, knowledge, ideas and illustrating principles in a way everyone can understand. They can be especially useful in web development and I try to use them as often as possible for just this purpose, even if the story is not about development itself (indeed sometimes it’s better if it’s not).
So I was delighted when I came across a story on Aza Raskin’s blog that illustrates beautifully the principle of continuous deployment using the topic of airplane construction.
Read it. Absorb it. Pass it on to your team, and make sure you’re solving the right problem…
A short story about the value of automated testing.
July 9th, 2011For the last few weeks, my team has been working on some pretty fundamental changes for Nokia maps on the web.
But that’s the short term. For the last six months my team has been working hard on getting a suite of automated tests up and running, and I’m proud to say we now have close to 1000 Jasmine unit tests and 200 cucumber acceptance scenarios running on every commit.
So I was delighted, when I heard that this week, one of the new changes had broken 52 of our acceptance tests.
Why should I be delighted?
Doesn’t that mean the team now has to go away and fix 52 acceptance tests? Well, yes, and that was certainly the way the team saw it, until I said:
“Now imagine we didn’t have those tests. What would have happened?”
And then we all started to realise, the power of automated testing.
So if we didn’t have automated testing, what would have actually happened?
Well, someone would have gone into the code. Found the piece of code that needed changing. And changed it. Probably tested it in the one place they knew about. Committed it. And moved on. Completely oblivious to the fact that they’d broken the application in 51 other different places.
Then the build would have gone to QA, the QA’s would have found the 52 different breakages, raised 52 new bugs, and sent them back to the developers.
Then the developers would have gone into the code, tried to fix those 52 bugs, and probably caused another 50 regressions. Which would then go to the QA team, be raised and go back to the developers for fixing. Eventually this hugely inefficient cycle would have continued until the build gets to an extremely brittle stability, a bit like building a house of cards, and it would have gone live.
However, with the tests, those bugs were caught. Immediately. And fixed. Immediately. Those 52 broken acceptance tests, saved us potentially months and months of rework.
So the moral of the story is, when you reduce the feedback cycle of a bug, with the saftey net of automated tests, it allows you to develop in confidence, giving you warning to what you’ve broken and giving you the opportunity to fix it then and there (however having the discipline to do this is another story). In Lean, this is referred to as ‘building integrity and quality into the product from the beginning of the cycle’, rather than trying to add it in at the end with a torturous QA cycle.
Faking nature in your work environment.
July 5th, 2011
Once upon a time I worked in a digital agency, in London, in Canary Wharf, also known as “Thatcher’s armpit”. The office was fairly standard, it was on the 14th floor and on all the other floors were home to banking institutions. Our office looked a bit like a bank. The tables were those horrible standard tables you get in offices with the stupid little walls to segregate you from talking to one another. The ceiling was a hanging ceiling made of the nasty white tiles with strip lighting that made you look like you had a terminal disease. For all intensive purposes it felt like a bank.
But there was one redeeming feature of this office, that made it not so bank-like, the back wall was covered in wallpaper that had a forest scene painted printed on it.
At the time I was a hardcore lonestar programmer (not something I would recommend these days), coding late, headphones on, many hours spent in my head and in the machine in front of me. After an hour or so of intensive logic thought based activity, often, I would get up to go get a glass of water and relax and more often than not, I would spend a few moments inspecting the wall paper.
Something about it, enveloped you. The level of detail, everytime you looked at it, you would find something new, something you hadn’t seen before. It was refreshing, regenerating, revitalizing. And then almost effortlessly you jumped back into the code and back into the flow.
Years later, I am reading the book: 125 design principles, a recommendation from my father, and I was amused to find very early on a mention of ‘the biophilia effect’ and more specifically the ‘attention restoration theory’.The later theory states:
Attention Restoration Theory (ART) asserts that people can concentrate better after spending time in nature, or even looking at scenes of nature. Natural environments abound with “soft fascinations” which a person can reflect upon in “effortless attention”, such as clouds moving across the sky, leaves rustling in a breeze or water bubbling over rocks in a stream.
The book further describes the science of how nature scenery affects us as creatures and should be specifically “in environments where learning, healing and concentration are paramount”. So now it begins to make sense why back in that office, that wall paper made so much of a difference to my working environment. Programming is nothing if not something that requires exceptional concentration abilities, and the ability to be able to quickly relax and regenerate is exceedingly important.
So, now I find myself as the manager and I understand the ‘why’, so lets get on with the ‘how’. Below is a picture of my new attention restoring office.
The best company I found to help is wallpapermurals.co.uk, the pictures are good quality and indeed help you to relax when you most need it.
I highly recommend you go forth, and even if you work in the drabbest of banks, add a little regenerative imagery to your wall. It will make you feel that tiny bit better.