Category Archives: tech

Open social networks?

blogging, social networks, tech

At work I’m building a custom vertical social network. It’s interesting work, and so I’ve been following some of the stuff about how social networks need to become “open”

I had a half-written post about how an “open” social network means such different things to different people that it is pretty much meaningless. Dare says it much better, than I could.

I still think someone needs to point out what a crap article the Wired piece “Slap in the Facebook: It’s Time for Social Networks to Open Up” is.

They spend a long time listing different web tools you can use to build some kind of nice looking website, and then miss the “social network” bit of building a social network.

A social network isn’t about a stupid frigging BLOG (yes, I’m quite aware of the irony of saying that on a blog). It’s about the personal interactions and relationships the software enables. Go and listen to some of the Danah Boyd podcasts recorded at the education.au seminars – you’ll note she talks about the social pressures of how to order your friends lists, how bands on MySpace are identity markers and how the “wall” is useful as a publicly witnessed space. There’s nothing in about blogging or social bookmarking or group calendaring – as useful as those things may be.

So anyway – the social network I’m building is going to be as open as I can make it – but it’s MY definition of open. Specifically, it’s going to make it as easy as possible to use external applications like blogs, and yet still tie them into your identity on the system. That sounds pretty obvious of course, but that doesn’t mean it is wrong.

Hmm.. I seem to be discussing work projects on here a lot more than I used to.. not sure what that means.

Guessing is much quicker than debugging

java, tech

My previous post I’ve already tried the ‘waving a dead chicken over our servers’ trick attracted a bit of attention, and quite a number of suggestions – thanks to all who contributed. The suggestions seemed to fall into four main categories:

  1. Database tuning.
    • This is a good suggestion, and is something we’ve done a fair bit of. In this case it doesn’t really help because the problem wasn’t performance but stability.
  2. Introduce a caching layer
    • We’d already done this, twice. We initially used an ehcache caching filter to fix some pretty serious performance problems. We later added some OSCache JSP cache tags in some critical areas in some templates (it was the addition of OSCache which caused the performance boost seen in my post on monitoring performance using the Google Webmaster Tools). As it turned out this combination may have been what caused our problem.
  3. Rewrite everything
    • Thanks. Let me know when you get a job in the real world.
  4. Debug the problem
    • This is what I figured we’d have to do. It’s something I was attempted to avoid because the issue seemed to be threading related, and we couldn’t reproduce it anywhere except our production environment.

We did have one stroke of good luck. We were able to predict when the site would stop working by monitoring the number of threads Apache was using and we could use this information to preemptively restart the site. We were able to modify the restart script to generate stack traces for all the JMV’s threads (kill -SIGQUIT <jvm pid>).

Since it looked like I’d actually have to start debugging this problem I started looking through the stack traces and I noticed that lots of the threads were in the ehcache filter. Now this wasn’t necessarily a bad thing, since all http request would be passed though it. However, it did make debugging harder, was easy to remove (just comment it out in the web.xml) and did have some potential to be a source of problems – in particular the cache-invalidation part.

So we took a punt and removed the filter and… it fixed the problem. Yay! I’m a genius and all that.

Except…. now the CMS is crashing with a NullPointerException deep in the data persistence layer. There’s also the small problem that I don’t have a clue why that change fixed it. Using the ehcache filter on its own worked fine, and there is no programmatic interaction between the ehcache and oscache code.

There is an alleged fix for the NullPointerException – but we have to take a point release of the CMS, and then patch it with a service pack to get it. Our previous experience with upgrades have been less than confidence inspiring.

In the mean time we have a script watching the site and restarting it when it crashes. It’s kind of like failover, without the over bit.

“You should just buy Google”

java, tech

Me to Unix Admin @ work: Hey – so I’m doing estimates for a proposal which needs between 30 and 250 TB of storage – what do you know about mass storage?

Unix Admin: Hmm.. you should just buy Google….

Okay then! But seriously – Amazon S3 seems the obvious solution, and I’m also looking at OmniDrive. Any other suggestions are welcome. No hardware suggestions, though – I don’t have enough confidence in our operations group to do it in house (mainly because we don’t have an operations group…)

Recommendations for Australian Contractor service companies?

personal, random, tech

I’m currently employed as a contractor, which means I need to have my own company (I operate as a sole trader). For a variety of reasons this sucks, and I’m interested in any recommendations for companies which act as contractor shell companies (I’m not sure what the proper terminology is). What I’m looking for is a company which employs me as an employee, and the company I actually work for pays my contract rate to. Then the shell company handles all the tax obligations, etc.

I’ve heard of a few companies in Australia which do this, but the only one I remembered to save is Entity Solutions. Anyone got any other recommendations (or experience with them)?

Firefox accessibility tool for checking secure sites

tech

I’d previously mentioned that we was looking for a XUL programmer at work. Fortunately, we found someone who – while he didn’t know XUL – knew enough Javascript to be able to pick it up.

I’ve now been able to do a screencast of the project we were working on – a Firefox plugin for checking accessibility for sites which require a login – which is pretty cool if you are a web developer and care about accessibility. More details on my work blog.

Desktop Tower Defense Strategies

random, tech

Somewhat to my surprise, it seems that more people want to read about Desktop Tower Defense than they do about anything else I’ve written. Over the last few days roughly 40% of the traffic to this blog has been to my previous post on DTD.

While I’m a long way from an excellent player, I’ve developed my strategies to the point where I’m consistently scoring just below 6000 points.

My first tip is that it’s a mistake to try and build a really long maze. When I started playing I tried to make the longest maze possible, covering the board with towers. That’s a mistake because you just can’t get the firepower you need to stop the creeps on higher levels. Instead, concentrate on upgrading towers in the center of the board. I find its better to upgrade a couple of towers to the maximum level, a fair few to medium levels, but I leave plenty untouched. I usually aim to have two Squirt Towers upgraded to Typhoon Towers separated by a Bash Tower.

My second tip is to use Swarm Towers. For a while I tried to avoid that, and thought that I should be able to deal with flying creeps just using Squirt Towers. It might be possible, but I sure haven’t got it to work. One problem with the Squirt Tower only strategy is that typically they are also dealing with other creeps at the same time the flying creeps need to be shot down. I’ve found that three appropriately upgraded Swarm Towers give enough support to my existing Squirt Towers to enable me to shoot down all flying creeps.

My final tip it to be aggressive in the use of the “Send next level” button. It’s pretty easy to pick up 500+ points like this.

I think these three tips are the difference between scoring 3500 and scoring 5900+. If I had to guess I’d say the first tip is worth over 1000 points, and the second two are worth roughly 500 points each.

So there you go – Nick’s quick Desktop Tower Defense Strategy guide. Feel free to add your own tips here, and to add your scores to the “badmagicnumber” group…

Performance monitoring with Google Webmaster Tools

tech

So I was recently involved in some performance tuning for a fairly large Australian website. One of the problems with performance tuning is that typically there’s no historical metrics to compare any changes against.

Fortunately, Google records this infomation as part of their crawl, and if you have signed to use the Google Webmaster Tools you can see this data.

In our case, this was very useful, because we were able to demonstrate a pretty significant improvement (download time (ms) vs Date):

Download Times

(year, I know – I’m still not really happy with that performance either, but there are some infrastructure problems which are proving difficult to work around….)