I’m not terribly keen on this for all the reasons that you might expect, but I’ve been putting it down to lack of exercise (I do about 15 minutes walking to a from work every day, that’s it), and a natural aging spread effect.

This leaves me only three choices, join a gym, start a diet or accept being that fat geek!

I kept considering joining a gym, but I’m not a terribly motivated person, except by money, and a gym seems like a lot of effort and costs a lot of money, so it’s out for both those reasons. I’ve tried belonging to a gym before, but the time commitment is something I’d find hard to keep up, if I could find one at or nearby work that I could use at lunchtimes, maybe I’d go for it.

Anyway that left me with dieting, but a lot of diets require willpower, dedication and effort, none of which I have in vast quantities. I was speaking to my friend Dan Catt about it recently when he was regaling me with details of his diet. He’d lost of a lot of weight on the diet, but it was one of these extreme diets, where you go into ketosis, and your body starts eating itself to keep your energy levels up. Besides sounding awful it comes with a lot of side effects that I didn’t really want, and worst, if you slipped up and ate one single thing that was not on the diet, your body would drop out of ketosis, and you’d take a few days to get back in, which meant hunger, headaches, dizziness and nausea all over again. No thanks to that!

But it got me thinking, maybe I could just reduce my calorie intake, and see how that would go. Luckily Dan (or possibly somebody else, we don’t know who) had a whole bunch of Cambridge Weight Plan milkshakes left, and swapping out a great cooked lunch for a milkshake sounded unappatising, but at least doable. Note that the shakes are essentially unpurchasable unless you are seing their nutritionists. Their recommended diet is the uber extreme diet, which I wouldn’t recommend even trying if you are anything like me. I’m running out, so I’m going to try some weight watchers shakes, or might have to hunt around for some good meal replacement shakes.

But before I started my diet, I needed to find out what I was actually eating. This is the essence of being a good developer, measure everything and most iportantly measure before you act. Ideally I wanted 24/7 ganglia monitoring of all the respective statistics of my body, but I figured I’d start with just two simple emasurements.

1. What was it that I was actually eating, what was my calorie intake, and therefore which meals/snacks/foods contributed the most
2. How much did I actually weigh, and how much should I weigh?

• They have android and iphone apps. This means when out and about, with either my ipad, phone or laptop I can record the food I’ve eaten.
• They have a great database of calorie information for foods. I tried a number of other apps and a lot of them were very US-centric. I searched for tesco and sainsbury’s own brand foods and SparkPeople actually had most items.
• They have a user submitted calorie information search. Even if their official recipe isn’t there, I can search what other users have submitted, and they tend to have pretty much everything.

So using the app I started tracking for a week what I was eating. After a few days it became clear to me that I had absoutely no idea about calories, there were things I thought had little that were high calorie, and things I thought had lots that had hardly any! The main thing I noticed was that I was eating on average at least 2500 calories a day. That’s about 500 too many (or the upper end of a mans diet, but I’m essentially sedentary, I shouldn’t eat 2500), and on many days when I treated myself I was reaching 3000 to 4000 calories.

So I decided to start the diet, and my rules were simple.

1. Track everything I eat
2. Weigh myself once a week on mondays
3. Reduce the high calorie snacks and foodstuffs that I’m already eating
4. Replace my lunch with a low calorie milkshake.
5. Don’t affect the foods my family eats at dinner time too much

So what did I do?

Firstly it seemed that an awful lot of my calories were coming from milk, I was having latte’s, cappucino’s and just plain white coffee a lot during the day. Coffee with milk is around 25 calories per cup, black coffee is between 2-5 (the experts seem to vary their opinions). I was having an easy 6 coffee’s a day (free at the guardian), so that’s an easy 100 calories to save. A large latte (as I was having every morning thanks to my new Nespresso pod coffee machibne), was about 200 calories.

Breakfast milk also was an issue, theoretically corn flakes with 1/2 cup of milk is only 140 calories, but I tried an experiment and weighed my portion. I was eating more or less a double portion, and my milk was whole fat or semi-skimmed not skimmed. Again I was looking at 300-400 calories for breakfast.

So I removed the latte’s, capucinos etc and only had them as treats, still once a week or so, but my morning coffee is americano style, and I can cope with that just fine.

I switched breakfast to 2 slices of toast with philadelphia light on it, total of 229 calories. I bought the mini tubs of philidelphia light at first, since 1 entire tub is exactly 35g and 55 calories, and for me spreads over 2 slices of toast just fine. That meant I was sure of how much I was eating.

Lunch is a cambridge diet shake, 140 calories in one of those, and they fill me up just fine it seems. I don’t have one every day, for a start there is a long standing ritual at my workplace to go for all-day breakfast on a thursday lunchtime, that I’ve been doing for over 3 years now, and I can’t break the habit.

Dinner is whatever my wife has cooked for us, and this is the area where I did slightly change my diet. In advance of deciding to actually do the diet, I was thinking of losing weight, and I was also reminded by Aral about how awesome vegetarian diets can be. I used to be vegetarian for 5 years or so, and I really enjoyed the interesting diet. So on January 1st I switched back to being vegetarian. Now this wasn’t entirely for a slimming reason, but one of the reasons that I did it was because generally speaking, vegetarian food is lighter in calories and better for you, if you make sure to keep a good nutritious diet going.

So dinner for me might be spaghetti bolognese, made with vegetarian soya mince, whcih I estimate at about 300 – 350 calories. That’s actually a much lighter dinner than I would normally have calorie wise, but the idea is that I’m free to enjoy up to a normal 1000 calorie big dinner, including a couple of glasses of wine and still be in or around the 1500 calorie range.

So how did it all go?

Well I did 3 weeks of the diet before I had to stop (more details below), and then I essentially cut out the shakes and dieting proper for about 6 weeks, and I was simply counting the calories and eating salads and light lunches a lot. I’ve since decided to start the diet proper again, and this is the 1 weeks mark.

During the diet, I went from 181.1 to 174.2 at the lowest, coming back up to 176 at worst but hovering around 175. That’s a loss of 13 pounds in 3 weeks, and keeping stable at the end. Last week I weighed in at 175 pounds, today I’m 171.6, despite the easter splurge this week.

Issues

On the diet I didn’t feel particularly hungry, but I did encounter a few problems.

Firstly: The diet shakes.

They do not look terribly appetising. I also made a massive mistake on the first day of going to lunch with all my work mates and sitting with them and trying to drink the shake. This was my first taste of the shake, and while they are actually quite nice, they take a bit of getting used to. My workmates, rather naturally, were not terribly encouraging, pointing out how disgusting it looked, and talking about hte delicious meals that they had purchased. I think I’ve already mentioned that I don’t have a lot of willpower, and I ended up binning the shake and buying a jacket potato with beans and cheese instead (approx 500 calories). The next day I stayed at my desk and forced myself to drink the shake alone.

After the week was up, and I’d gotten used to the shakes, I was able to rejoin my work mates, partly because I’d gotten used to it, and partly because I’d seen my weight drop, so knew it was worth doing!

The other thing, as referenced above is that I was lucky that there was about 15-20 diet shakes left over from Dan’s diet that he kindly gave me.  Getting hold of those shake powders isn’t just a case of popping into tescos.

Secondly: Dizziness, Nausea.

If you don’t eat enough your body does not appreciate it. I’ve had a few days where I’ve been presenting at morning conference, or talking over lunch and I’ve been stressed or much more active than normal. In those cases I have gotten brief waves of feeling sick or dizzy. Every time I’ve had a big glass of water and normally a go-ahead or weightwatchers snackbar, and felt just fine.

Make sure you keep a stash of snacks around that you can eat if it comes to it. I often want to snack at about 4pm, and I let myself have 1 snack a day at 4pm. Because I tend to be so busy at work I’ve often forgotten so I don’t eat it every day, but it’s nice to have, and at around 100-150 calories they are pretty good snacks to have.

Thirdly: Drinking

It’s a common british work culture to go for drinks after work on a fairly regular basis, and the Guardian has a pub/wine bar in the building itself, so it’s even easier to nip out for just one drink. Unfortunately I’m not very good at just one drink! Alcohol is high in calories, a normal sized glass of red wine is around 90 calories, and a pint of bitter has nearly 200 calories. A recent evening out involved me drinking 4 pints, which is the same number of calories as I’d had all day, and meant I was around 2200 calories by the time I got on the train (eating a small burrito – 600 calories, as my dinner)

Finally: Travel and conferences

I had a hectic fortnight recently where I attended multiple events in San Francisco, then flew back to attend InfoQ’s QCon in London. I know that I couldn’t possibly do the shakes with a schedule like that, and I tend to be far more active, on my feet and lot, and having to do a lot fo talking and walking. I gave up the diet, ate whatever was put in front of me (vegetarian of course), and just didn’t worry about it. It took me a week or so to recover from the conferences and get back into recording eating and therefore watching my weight as well, and I expected that I’d have ut on a lot of weight, but it turned out to only put on about 2 pounds of weight, which is not bad for completely abandoning the diet. I went back down by a pound once I started measuring again.

Summary

So whats so geeky about dieting then? Well it seems that everybody has a fad they want to use, red days/green days, points systems, the chocolate cake for breakfast diet. But like many geeks I realised that the first step is measurement. Once you can measure your system (calories and weight) you can start to change the things you do and see what difference it makes. Sitting and counting calories on an iPad or phone at the dinner table, estimating portion sizes and even entire meals and starting to understand food better, that’s all part of the geekiness.

Defining what devops is can be hard, so instead I thought I’d feature a few of the things that devops isn’t.

“I’ve renamed our Ops team the DevOps team”

This is not DevOps, you’ve renamed your operations team, but that is all! You may as well print a million pound note or command the sea to stop for all the good it will do.

“But our devops team run puppet/chef/CFEngine to manage their systems”

This is not devops, this is just ops using new tools. You Just configure your tool in a ruby based DSL (or use CFengine), so now you have two problems instead of just the one.

“We split our ops team into Infrastructure and DevOps. Our infrastructure team rack and stack, our devops team manage puppet/chef…”

Congratulations! You’ve recognised that some operations people don’t want to do development tasks, how happy for you. But all you’ve done is split them out of the Ops team, but your ops team still sounds like an ops team to me.

“Ok, so we got rid of our ops team, and our devs have root now”

This is not just not devops, but it’s pretty damn stupid. It turns out that people have specialisations. I wouldn’t expect a java developer to be as productive with backbone.js as one of our javascript engineers, and I wouldn’t expect every developer to be able to setup and configure apache or nginx. Full stack engineers are great, and if you’ve got a full team of them then you have a great resource there, but even then everybody will have things that they prefer doing and things they are more efficient at doing.

“Ok, so we got rid of our ops team, our devs have root now, and we hired a DevOp (or DevOps) to manage configuration management. Better?”

So now you got rid of a team, replaced them with a new team with a different name, less authority and expect your team to be functional? Get out of here!

“I see DevOps for what I think it is, a power grab. Revenge for not giving ‘devs’ root when they wanted it”

I think you are assuming that DevOps is a developer movement, and that it’s all about power. You are so far wrong that I’m pretty confused right now. However this is also an accurate view of how some people view the devops movement, sometimes those people are grumpy sysadmins who don’t want to do that configuration management thing, but that might be exactly how senior business decision makers are viewing devops.

My experience of DevOps is that the general trend is not of developers wanting to do sysadmin work or have root, but of existing sysadmins wanting to make their lives easier by working more closely with developers. All of the people who I would consider to be doing “Devops” are brilliant sysadmins who understand something bigger than the inner workings of IPv4, BGP, Apache and nginx.

So lets make a simple list, DevOps is not about:

• Less racking and stacking hardware
• Developers having root
• Using configuration management tools (like Puppet or Chef)
• Continuous Deployment (10 times a day)
• Getting rid of the Ops team

But surely I hear you ask, that’s what all the blogosphere has been talking about, how DevOps will revolutionise Ops, and that all of these things are possible? Well that is kind of true, it looks like organisations that embrace DevOps might do some of those things, they also might not, it depends on the requirements of the organisation.

Ok, so I’ve talked a lot about misconceptions of DevOps, so what do I think DevOps actually is?

“You are DevOps when your operations team and your developer team are measured and rewarded by the same metric, did you help the business deliver value today”.

The traditional view that pits devs versus ops is that ops teams are measured on reliability and availability (essentially uptime), whereas devs are measured on number of stories delivered. These two goals are in direct opposition and it leads to developers wanting root access to get around the pesky ops team, and very grumpy sysadmins at 4am fixing bugs that devs didn’t see.

Both are bad measurements, they are proxy metrics for a real metric which is business value. Downtime is probably bad for your business, except that there are cases where it isn’t. The Apple store is taken down during apple keynotes. Do you think that the Apple ops team looses their bonus over this? Downtime is a proxy for business value, so greater downtime probably means lower business value.

Delivering new stories is a great metric if every story delivers value, but deploying 20 times a day is completely wasted if what you are deploying are minor ideas that one internal stakeholder wants that have no impact. Again, deploying stories is a proxy for business value, and generally the higher the rate, the greater the business value.

See that’s the problem with DevOps, it’s so hard to get right because you need a team that is willing to focus around what the business actually needs, be willing to measure themselves and each other and work out what the business value is.

To do DevOps successfully you need to be able to measure and estimate the business value of a new story, and the business risk to each deployment. Teams that do that are likely to start deploying more often, use configuration management and may even move towards removing the ops team, but in each case it will only happen if doing that action creates more business value than risk.

Of course with DevOps there is the eternal truth that if you get 5 DevOps in a room to debate “What is DevOps” you’d get 7 answers out at the end, and this is one of mine, so you can take it with a pinch of salt but my main point is that DevOps is not a set of practices, it’s not what you do, it’s why you do it.

[Updated for spelling and clarity in a few sentences, thanks to @mart_brooks and @tomnomnom]

I blathered on the podcast for a bit again.

This article titled “Tech Weekly podcast: when books go social” was written by Presented by Charles Arthur and produced by Matt Hill, for guardian.co.uk on Tuesday 3rd January 2012 17.36 UTC

How do you discover new books? Matteo Berlucchi is tired of buying books from a certain online retailer only to be recommended very similar reads. He tells us why his aNobii.com is different.

Charles Arthur sits in the presenter hotseat for this week’s programme. He’s joined by Josh Halliday and Michael Brunton-Spall to discuss the week’s developments, including what Rupert Murdoch’s supposed Twitter feed tells us about the verification process and why those network crashes on New Year’s Eve look to be a thing of the past.

Plus, Aleks Krotoski talks to Chad Dickerson from vintage retailer Etsy about its rise and rise, and how to scale up a business without losing your customer focus.

Don’t forget to…

• Comment below
• Mail the producer tech@guardian.co.uk
• Get our Twitter feed for programme updates or follow our Twitter list
• Like our Facebook page
• See our pics on Flickr/Post your tech pics

      <a rel="nofollow" href="http://oas.guardian.co.uk/RealMedia/ads/click_nx.ads/guardianapis.com/technology/oas.html/@Bottom">
          <img alt="Ads by The Guardian" src="http://oas.guardian.co.uk/RealMedia/ads/adstream_nx.ads/guardianapis.com/technology/oas.html/@Bottom"></img>
      </a>

  </div><img src='http://hits.guardian.co.uk/b/ss/guardiangu-api/1/H.20.3/98867?ns=guardian&amp;pageName=Tech+Weekly+podcast%3A+when+books+go+social+Audio+1683419&amp;ch=Technology&amp;c2=67197&amp;c4=Technology%2CSocial+media%2CSocial+networking%2CBooks%2CInternet%2CGoogle+%28Technology%29%2CTablet+computers%2CMicrosoft+%28Technology%29%2CRupert+Murdoch+%28Media%29%2CTwitter+%28Technology%29%2CMobile+phones+%28Technology%29%2CiPlayer%2CApps&amp;c3=guardian.co.uk&amp;c6=Presented+by+Charles+Arthur+and+produced+by+Matt+Hill&amp;c7=12-Jan-03&amp;c8=1683419&amp;c9=Audio' width='1' height='1' /><!-- Guardian Watermark: technology/blog/audio/2012/jan/03/social-media-socialnetworking|2012-05-18T22:55:34Z|f84029f6f8c74b451e53fdd3ea23508a7efe1d40 --><p>guardian.co.uk &#169; Guardian News &amp; Media Limited 2010</p> <p>Published via the <a href="http://www.guardian.co.uk/open-platform/news-feed-wordpress-plugin" target="_blank" title="Guardian plugin page">Guardian News Feed</a> <a href="http://wordpress.org/extend/plugins/the-guardian-news-feed/" target="_blank" title="Wordress plugin page">plugin</a> for WordPress.</p><!-- END GUARDIAN WATERMARK -->

]]> http://www.brunton-spall.co.uk/post/2012/01/09/tech-weekly-podcast-when-books-go-social/feed/ 0 http://www.brunton-spall.co.uk/post/2011/12/02/map-map-and-flatmap-in-scala/ http://www.brunton-spall.co.uk/post/2011/12/02/map-map-and-flatmap-in-scala/#comments Fri, 02 Dec 2011 10:56:39 +0000 bruntonspall http://www.brunton-spall.co.uk/?p=1348

Scala (stairs) by Paolo Campioni

One of the things I like about Scala is it’s collections framework. As a non CS graduate I only very lightly covered functional programming at university and I’d never come across it until Scala. One the benefits of Scala is that the functional programming concepts can be introduced slowly to the programmer. One of the first places you’ll start to use functional constructs is with the collections framework.

Chances are your first collection will be a list of items and we might want to apply a function to each item in the list in some way.

Map works by applying a function to each element in the list.

scala> val l = List(1,2,3,4,5)

scala> l.map( x => x*2 )
res60: List[Int] = List(2, 4, 6, 8, 10)

So there are some occasions where you want to return a sequence or list from the function, for example an Option

scala> def f(x: Int) = if (v > 2) Some(v) else None

scala> l.map(x => f(x))
res63: List[Option[Int]] = List(None, None, Some(3), Some(4), Some(5))

flatMap works applying a function that returns a sequence for each element in the list, and flattening the results into the original list. This is easier to show than to explain:

scala> def g(v:Int) = List(v-1, v, v+1)
g: (v: Int)List[Int]

scala> l.map(x => g(x))
res64: List[List[Int]] = List(List(0, 1, 2), List(1, 2, 3), List(2, 3, 4), List(3, 4, 5), List(4, 5, 6))

scala> l.flatMap(x => g(x))
res65: List[Int] = List(0, 1, 2, 1, 2, 3, 2, 3, 4, 3, 4, 5, 4, 5, 6)

This comes in really useful with the built in Option class because an option can be considered a sequence that is either empty or has 1 item.

scala> l.map(x => f(x))
res66: List[Option[Int]] = List(None, None, Some(3), Some(4), Some(5))

scala> l.flatMap(x => f(x))
res67: List[Int] = List(3, 4, 5)

So with that all covered, lets look at how you can apply those concepts to a Map. Now a map can be implemented a number of different ways, but regardless of how it is implemented it can be thought of as a sequence of Tuples, where a tuple is a pair of items, the key and the value.

scala> val m = Map(1 -> 2, 2 -> 4, 3 -> 6)
m: scala.collection.immutable.Map[Int,Int] = Map(1 -> 2, 2 -> 4, 3 -> 6)

scala> m.toList
res69: List[(Int, Int)] = List((1,2), (2,4), (3,6))

We can access a tuple by accessing the inner variables _1 and _2

scala> val t = (1,2)
t: (Int, Int) = (1,2)

scala> t._1
res70: Int = 1

scala> t._2
res71: Int = 2

So we want to think about using map and flatMap on our Map, but because of the way a map works it often doesn’t make quite the same sense, we probably don’t want to apply a function to the tuple, but to the value side of the tuple, leaving the key as is, so for example we might want to double all the values. Map provides us with a function to do exactly that.

scala> m.mapValues(v => v*2)
res73: scala.collection.immutable.Map[Int,Int] = Map(1 -> 4, 2 -> 8, 3 -> 12)

scala> m.mapValues(v => f(v))
res74: scala.collection.immutable.Map[Int,Option[Int]] = Map(1 -> None, 2 -> Some(4), 3 -> Some(6))

But in my case I wanted to do something more like flat map in this case, I want a map to come out that misses out the key 1 because it’s value is None. flatMap doesn’t work on maps like mapValues, it get’s passed the tuple and if it returns a List single items you’ll get a list back, but if you return a tuple you’ll get a Map back.

scala> m.flatMap(e => List(e._2))
res85: scala.collection.immutable.Iterable[Int] = List(2, 4, 6)

scala> m.flatMap(e => List(e))
res86: scala.collection.immutable.Map[Int,Int] = Map(1 -> 2, 2 -> 4, 3 -> 6)

Ok so we are pretty close to using options with flatMap, we need to filter out our None’s, we can do returning a list with just e => f(e._2) and we’ll get the list of values without the None’s, but that isn’t really what I want. What I need to do is return an Option containing a tuple. So here’s our updated function:

scala> def h(k:Int, v:Int) = if (v > 2) Some(k->v) else None
h: (k: Int, v: Int)Option[(Int, Int)]

and here’s how we might call it:

scala> m.flatMap ( e => h(e._1,e._2) )
res109: scala.collection.immutable.Map[Int,Int] = Map(2 -> 4, 3 -> 6)

but this is pretty ugly, all those _1 and _2′s make me sad. If only there was a nice way of unapplying the tuple into variables. Given that this works in python and in a number of places in scala I thought this code should work:

scala> m.flatMap ( (k,v) => h(k,v) )
<console>:10: error: wrong number of parameters; expected = 1

I spent way too long today looking at this (in 5 minute chunks broken by meetings to be fair), before I gave in and asked a coworker what the hell I was missing. The answer is seems is that an unapply is normally only executed in a PartialFunction, which in scala is most easily defined as a case statement. So this is the code that works as expected:

scala> m.flatMap { case (k,v) => h(k,v) }
res108: scala.collection.immutable.Map[Int,Int] = Map(2 -> 4, 3 -> 6)

Note that we switch to using curly braces, indicating a function block rather than parameters, and the function is a case statement. This means that the function block we pass to flatMap is a partialFunction that is only invoked for items that match the case statement, and in the case statement the unapply method on tuple is called to extract the contents of the tuple into the variables. This form of variable extraction is very common, and you’ll see it used a lot.

There is of course another way of writing that code that doesn’t use flatMap. Since what we are doing is removing all members of the map that don’t match a predicate, this is a use for the filter method:

scala> m.filter( e => f(e._2) != None )
res114: scala.collection.immutable.Map[Int,Int] = Map(2 -> 4, 3 -> 6)

scala> m.filter { case (k,v) => f(v) != None }
res115: scala.collection.immutable.Map[Int,Int] = Map(2 -> 4, 3 -> 6)

scala> m.filter { case (k,v) => f(v).isDefined }
res116: scala.collection.immutable.Map[Int,Int] = Map(2 -> 4, 3 -> 6)

Unfortunately due to the way the web (and Facebook) works, if somebody copies the URL in their browser and posts it to twitter you’ll be looking at the same thing as them, and if that was a guardian Facebook app, but you haven’t logged into Facebook, Facebook wont tell us anything so we can’t redirect you back to the guardian.

However, most modern browsers support special bookmarks with javascript in them, called Bookmarklets, and since the original Facebook url has the important path data needed for the guardian site, it’s possible to manipulate the current url to take you back to the guardian site without logging into Facebook.

Basically, drag this link: to your bookmarks toolbar up above this page, then click this link: Are these the 10 heaviest albums ever made? and then click the new fb->g link on your toolbar and you should go to the guardian site.

This will only work if you are not logged into facebook.

How does it work?

It gets the current url, and looks for a parameter called cancel_url.  That’s where the url for the original apps.facebook.com is.  Next we replace apps.facebook.com/theguardian with www.guardian.co.uk, then we update window.location with the result.  This causes your browser to load the new url, and you should be on the guardian site.

Note that it does no error checking, it does not handle unusual guardian urls (that might not start www.guardian.co.uk) nor does it check whether there is a cancel_url before acting.

You can view the source and repurpose for your own uses at jsfiddle.

This, it turns out, is really easy to implement if you run Nginx as the front to your website.

You simply need to update your server configuration to include the following snippet:

location ~ ^/?+$ {
  rewrite ^ https://plus.google.com/YOUR_NUMBER permanent;
}

This will ensure that your domain.tld/+ issues a redirect to your Google+ profile.

Hope this helps

So something bugs me (I know, big shock), and it’s API documentation like this:

The document will be returned as:
ID: Integer
Name: String
Published: Date/Time

My position on this is simple, in an API (we’ll come back to internal design later), an ID should always be of String type.

So why is this a big deal? I mean you implement this with a database table with autoincrement right, so it is a number underneath?

There’s a few issues with this and I’ll deal with them in reverse stupidity order.

Numbers are more than identifiers

A number represents a mathematical concept. It implies strict ordering (2 is always > than 1), it implies a set of operations (1+2=3) and it implies an infinite set. If you provide external developers with numbers they are going to assume that these implications are available and might make some form of sense.

You would hope that most people wouldn’t assume that something like twitter ids can be added or multiplied and expect anything sensible from them, but that comparison function, that’s a bit more tricky. It’s true that tweet 1 was sent before tweet 2, and even that tweet 1,000,000,000 was sent before tweet 2,000,000,000. But as twitter scales guaranteeing that ordering becomes harder and harder in a massively parallel system, and there is no value in a client ordering tweets this way. Each tweet has a date/time field on it indicating when it was sent, and ordering based on that field is far more semantically correct (and what the user expects).

Numbers aren’t equal in all languages

If you argue that representing something as a number is saving you memory, because an integer only occupies say 2 bytes, you are probably prematurely optimising. Number representation is a very complex subject, one which I know only a little, but I know enough to know that I should never make assumptions about how a number is represented internally.

I come from a C background, and despite it being fairly common knowledge how big various numbers are stored, we have a sizeof operator for a reason. Compiling your program on a slightly different architecture could change your integers from 2 bytes to 4 bytes to 8 bytes to anything inbetween. It’s a pretty common source of bugs for new developers to assume that unsigned int a is 4 bytes in size.

When you provide a number in an API, You have no idea who is consuming the API or what system they are running on. It might be an x86 EC2 instance, or it might be a mobile device, it might be a super computer, or a graphic compute unit.

It’s fine in your system when you create your 4 billionth item and exceed the 32bit MAXINT (4,294,967,295 for unsigned ints), but will it be fine for all your users? If your samples and current users get numbers in the low 100′s, but 2 years down the line you have a wildly successful product and your id’s are now into the billions upon billions? Do you want people writing API libraries to have to worry and think about this?

One fantastic issue that twitter discovered fairly recently is that javascript and therefore many json libraries are required to represent integers as a 53 bit number, that’s a pretty large number, but it’s several orders of magnitude less than most people would expect.

You make database scaling hard

Autoincrementing primary keys are fine if your datastore is a single logical machine (a highly connected cluster), but as you add parallelism, there are a number of situations where you might want to be able to allocate id’s from more than one cluster at the same time without synchronisation between them.

If your ID’s in your table are not autoincrementing integers, but are an identifier that maybe looks like “DC1-DB1-0001″ (or some other similar system), an identifier that takes into account the datacenter and database identifiers, reconciliation for tweet creation is perfectly possible.

You lock yourself into technologies

Autoincrementing primary keys are a feature of most relational databases. Some support it natively, some support it with triggers and stored procedures. However for scalability reasons, most non-relational databases don’t support them. Mongo for example provides primary keys that are OID’s or Object ID’s. These are a UUID4 type identifier that takes into account the host, the time and a few other things at creation time. BigTable provides horrible “keys” that are totally opaque, I have no idea how a key is generated (nor should I need to know).

Adding an autoincrementing key on top of these technologies is often possible, but it is often not ideal.

Conclusion

So there is a number of good reasons not to use numbers as an identifier in your API. Just use a string, trust me on this, you’ll use sightly more bytes, but you wont have to convert the integer into a string at API render time, and your client wont convert it back. Your string will be opaque to the user, meaning you can change primary identifier format willy nilly and providing the old and new format don’t conflict it won’t matter. Nobody will try to use false ordering on your ID’s when they should use a field instead, and it wont make me angry and ranty when I read your API documentation.

Addendum

I’m not saying don’t use a number in your database schema. It’s entirely possible for your ID’s that you know you’ll only ever generate a few hundred ID’s and that you wont have a date/time field, but order of creation is important and ordering the ID’s is the simplest way to do so. There are many valid reasons to use a numerical id internally for various things, but when you convert it into an API or external system, just turn that number into a string.

I strongly suspect that there will be people who will see your “{ ‘id’:’123′ }” and will turn it into a number themselves, but then later if you return a document “{ ‘id’: ’01-007′}” they can’t really complain, because you always told them it was an opaque string.

The bit of Scala code I’m particularly proud of is around waiting for a port to be opened.  During our deployment scripts we restart our application servers, then we want the scripts to wait until the server is up and able to serve requests before we move on.

Writing this kind of code is not terribly difficult, but testing that it works can be very tricky.  I decided to take the blackbox kind of approach to this, and my test spins up a second thread that listens on the port in question so that we can execute the real code that attempts to connect to the port.  In java or most other languages this would be something fairly long winded, but in my Scala test it looks like this:

The spawn function is provided by concurrent.ops and is built into Scala and represents a throwaway thread that we aren’t terribly interested in.  We don’t want to join on it, we just want it to start up and execute. The code inside creates a new java.net.ServerSocket, calls the blocking accept method which blocks until it gets a connection, closes the connection and then closes the original listening connection. We can then create and execute our port connection code and it should validly connect to the port 9998 on localhost and work beautifully.

The next test needed to be the failure case, if the waitForPort task cannot connect to a port after a specified timeout it should error, preferably by throwing an exception of some form. Testing this was also fairly easy, I wrote the traditional try { blah(); fail()} catch… stuff, then checked with Graham if there was a better way, and of course there was:

ScalaTest with the ShouldMatchers makes for a simply beautiful test that is clear to read and was easy to write.

However when it came to implementing the method, I struggled a little bit.  I’ve never learnt functional programming, so closures and maps and so forth are still a bit alien to me, so I ended up writing some very java style code:

The isSocketOpen simply tries to open a socket and catches any exceptions to return false, and true if it opened successfully. Having written this, knowing there was a better solution, I asked again, and this prompted Phil Wills to upload his blogpost about the Stream class and lazy evaluation. I won’t cover the details here, but essentially using the return statement in Scala is frowned upon because of the way it works, and it’s more idiomatic Scala to use Options and flatMap, here’s the updated code:

Here we’ve changed isSocketOpen to checkSocketOpen and made it return an option.  At this point I also made checkSocketOpen do the sleep, which we could have done before, but is necessary now.

To explain, this code takes the range 1 to 10, which is a Seq of ints (1,2,3…) and we flatMap across it with a function that returns an option.  If you haven’t seen this before, here’s a quick overview of what the flatMap actually does:

map applies a function on each item in a collection and returns a transformed collection, essentially [a,b].map(f) is the same as calling [f(a), f(b)].

If our function f returns an Option, this is either Some(true) or None.  An Option can really be thought of as a sequence that either contains 1 item or is empty, so Some(true) is the same as [true] and None is the same as [].

This means our assuming that our function f returns None for everything but c, the map on [a, b, c, d] will return something like [ [], [], [true], [] ].  That’s often not a lot of use for us, and what we are interested in is stuff that returned something.

flatMap is a function that essentially flattens the returned sequences catenated together, so if f(x) = [1,2] and f(y) = [4,5], then [x, y] map f == [ [1,2], [4,5] ] but [x, y] flatMap f == [ 1, 2, 3, 4 ].  This also has the handy shortcut that [] flattens into nothing, so essentially gets skipped, so our flatMap on [a, b, c, d] should return [true].

Our function checkSocketOpen has a side effect of pausing for an amount of time though, and since flatMap executes the function once for each item in the original list, this means that we will always pause for the entire connection timeout no matter whether we successfully connect the first time or not. (kind of, actually if we only pause on a failed connection, and if we successfully connect each time, we wont pause, but we will create 10 connections rather than just the one we need to confirm the port is open).

Phil in his blog has a very simple fix for this, which is to wrap our original sequence in a Stream class.  Stream is another Scala provided class, and in this case it is a sequence that is lazily evaluated.  Essentially this means that the flatMap doesn’t call the checkSocketOpen on all items immediately but instead only calls it when you try to get to the first item.  This means that headOption executes each function one after another until one of them returns true or it gets to the end of the sequence, it then returns the option based on what happened.

This means that if the first call returned none, it moved on and called f again over and over until it returned true, then the headOption stopped and never called the remaining calls, making this system stop as soon as the first connection was made.

Success! We’ve removed the return statement and made the code significantly harder to read and understand.  I’ve just taken several hundred words explaining why it works and I’m still not entirely clear on the details.

So we started wondering if there was an alternative way of doing this, something that would still be idiomatic Scala but that would be obvious to junior or learning Scala programmers like myself. Graham and I wondered if what a recursive solution would look like, so we came up with this code:

We decided very early on to inline the checkSocketOpen function as recursive itself, so although this looks like more lines of code, it now combines both functions.

This function, when called at the bottom with checkOpen(0) initialises currentTry to 0, then tries to open a socket, if it is successful it just returns, if attempting to open a socket throws an exception it sleeps for a bit, then calls checkOpen with currentTry incremented by one.

All recursive functions have to have a guard, which is to say some statement that prevents infinite recursion (unless that’s what you want anyway), this function checks to find out if currentTry is greater than MAXCONNECTIONATTEMPTS and if it is, sys.error throws an exception which bubbles all the way back up through 10 copies of checkOpen before exiting properly.

If you know that your range is bounded fairly low, (such as 10 retry attempts) this is a perfectly acceptable way of writing this function.  We suspect because of the way we’ve written the try catch that it’s not possible to make this function tail recursive, which would effectively allow infinite (or lots) recursive calls, but I’m sure it’s possible to adjust the try catch into a subfunction that might make it tail recursive if you really wanted to.

So there you go, that’s been a day of Scala learning for me, I now know the issues with the return statement, the Stream classes which might be useful later, and I’ve written my first truly recursive program in anger.  I’m feeling a little bit proud.

Of course there are improvements to make, making the function tail recursive would be education as I’m aware of tail recursion but am not entirely sure how to actually make a function tail recursive.  I’d also love to make that ServerSocket stuff into one line again, originally I had ServerSocket(port).accept().close() as one line, but I discovered in testing the stream stuff that the ServerSocket hangs around and keeps accepting connections if you don’t close it, which I can’t do in a single line without keeping a reference to it.

If you are interested in our new deploy system Magenta and the upcoming RifRaf web application for it, then I suggest you keep an eye out on our github repository and feel free to send patches, advice and help our way.

You’ve probably read journalists who love it and journalists that hated it, but since the cat appears to be out of the bag and the invitations are flowing fast and furious I thought I’d write a quick few bits about how to get started using it.

First of all your invitation.  If somebody you know has invited you, there’s a good chance that you didn’t realise it was an invitation.  There are 2 forms of invitation.

As a “member of the press” (by association not directly) I got a real invitation from the Google PR team, which appeared to come from one of our developers.  It looked a bit like this and it was a real invitation.

A real full invite

You can see clearly that it is an invitation and you can sign up from the mail easily.

During the first few hours, Google ‘accidentally’ left a send invitation feature into Google+, which was quickly noticed by the press and the number of people coming in massively increased.  Google quickly disabled the send an invite functionality, and we don’t know when it’s coming back.

However there is another way to invite people to Google+ and they get an email that looks like this:

This is actually an invite

Firstly this is not an official way of inviting someone, and may not work.  This email doesn’t say invitation except in tiny print at the bottom of the email, and instead looks like an opportunity to see the marketing bumph about Google+.

But clicking that button will result in your seeing one of two screens, if you are unlucky, it appears that Google are disabling signups globally for a few hours at a time, and you’ll get the “We are overloaded” page.

When Google is not allowing invites, this is what you see

If you get this try again in a few hours (or minutes) and you might get lucky and get the “Make an account” page (I don’t have a screenshot I’m afraid, you can’t miss it, it asks for your name and has a big signup button).

First steps in Google+

People are sharing with me

If you unlucky and only know one or two geeks then it’s possible that the only person who has added you is the person who invited you.  Never mind, you can find friends all on your own.

First of all, you can simply mine your gmail contact book.  This is particularly easy to do, and Google have put a lot of thought into it.

A list of friends

In the section that looks like this under your circles, you can see all of your contacts who are not yet on Google+.  At first you will probably instinctively start dragging contacts down to the relevant circle one at a time.  This is a fairly inefficient way of sorting your contacts.

Click to select multiple friends

Instead simply click all of the contacts that you want in a single circle (say the 25 members of my family I wanted to select) and then drag one of them, you should see that they all come down and group together to be dropped into a circle.  This makes life a lot easier when sorting out those tricky large groups.

Creating a new circle of friends

Create lots of circles, you can add people to multiple circles, and it’s very easy to rename, delete and even move people from one circle to another.  I currently have circles for my church, my friends from Milton Keynes, my friends from School, people I’ve worked with, and people I currently work with at the Guardian.

Speaking of people you went to school with, I don’t really use GMail (especially not my gmail account itself) to email people, so I only had 60 − 80 addresses in there.  For a number of my contacts, I instead use Facebook.  At the time I’m writing this, adding your Facebook contacts requires a couple of steps.  I expect this to change eventually, but for now, it turns out that Yahoo works as a very good middleman.

Login to Yahoo Mail for your Yahoo account (or create one if you don’t have one).  Click the contacts link, and select the Import Contacts link from the Popular Tools that you should see.

You can now select Facebook, enter your Facebook details and Yahoo will happily slurp all of your Facebook friends into your Yahoo email list.  Handy huh?

Find and invite from Yahoo

Now back in Google+, Under the circles tab, select the Find and Invite link and you should be able to add your Yahoo contacts.  Authorise it from your Yahoo account by following the instructions and suddenly all of your Facebook friends should be available in the Find and Invite tab.

People who are in paul's circles

Finally, once you’ve got a few friends who are on Google+, you can start looking at their circles to see if there are any friends that you are missing.  In pretty much any section of Google+ you can click somebodies name and get to their profile page.  On the left is a quick view of a few people in their circle, and people who have added them to their circles (and people you’ve got in common).

People in pauls circles

If you hit the view more, you can get a nice view of all their contacts with an easy way to add them to your own circles.  With this you can very easily start adding a large number of people who are already on Google circles.

Doing all of this should give you a fair number of people into your circles, but you might be missing a number of the less geeky people in your life.  You might have them in your circles, but they don’t have a Google+ profile yet.  How disappointing.  So lets discuss invites.I cannot guarantee that this works, the only sure way to send invites is to wait for Google to fully enable invites.   For now the following seems to send out the magic email.

Email your friends

Go to your stream and start sharing something.  Remove the Public/All Circles users that are currently in the list and add a circle that has members you want to invite.  You should see an option to email people not on Google+, tick it and confirm the pop up that comes up when you click share.

This will cause each of those people to get an email indicating that they’ve been mentioned, and they can click the link in the email and hopefully they’ll get in and sign up.

I hope you found this useful, and feel free to find me on Google+ and say hello or thanks.

This article titled “Google’s Chrome browser hits 160m users – but what does it mean for the web?” was written by Michael Brunton-Spall, for guardian.co.uk on Wednesday 11th May 2011 19.38 UTC

We have decisively moved from a world of one browser provided by your operating system to a world where people install the browser of their choice. The increasing growth of Firefox usage has risen to the point where 31% of users on the Guardian website use Firefox and only 37% use Microsoft’s Internet Explorer (split into 22% IE8, 9% IE7 and approx 3% each for IE6 and IE9).

Google’s entry into the market has increased usage over time: one year ago at the previous Google IO, there were 70m Chrome users worldwide; today we were told there are over 160m. Unusually for a browser, Google has committed to a forced upgrade process and releases updates every six weeks.

Why? To combat the problem that more than 50% of Windows computers around the world are still running Windows XP – an operating system that is approaching 10 years old – and a browser that’s often the same age.

Google has been trumpeting its Chrome browser achievements at Google IO, showing off what a wonderful platform it is for the web. Over the last year, the team has improved the performance of the browser when running complex web applications by improving the performance of the underlying Javascript engine that web developers use to add logic to a simple web page. Additional improvements in security have made the “Google web” a safer place to be – which can only be good for the web, and for users.

The biggest announcements they have made is massive improvements to the graphics functionality in your browser. Implementations of the HTML5 Canvas technology that powers many of the graphical bits of the web are 10 times faster than they were in previous versions, and those improvements don’t require the developers to upgrade their application.

This is good news for game developers such as Rovio, who are able to build Angry Birds for the web, and on new builds of the Chrome browser it will run faster.

If developers want much, much faster graphics, the Chrome browser team has spent time vastly improving the WebGL graphics library available in Chrome. This is a subsystem of the browser, often called part of the HTML5 specification, but actually a separate specification in its own right, that allows developers to build 3D applications and utilise the high performance graphics hardware available in your machine.

Google’s improvements in Chrome means that WebGL applications can run up to 10 times faster than the equivalent Canvas implementation.

The dark side of all of these improvements is that we are likely to start seeing “Optimized for Google Chrome” badges on the web, as many of these features won’t work across all browsers. That wasn’t good with Internet Explorer in the bad old days of the Netscape wars, and wouldn’t be good today. Some things such as Canvas work in most browsers (but not in the most common versions of Internet Explorer), but WebGL’s implementation is inconsistent across all the browsers.

Google’s advancements are welcome for experimental applications, and developers building applications for the Chrome App Store, but their impact on the wider web will probably be negligible, since cross-browser implementation is considered so important by many organisations.

guardian.co.uk © Guardian News & Media Limited 2010

Published via the Guardian News Feed plugin for WordPress.