Paul Hinks' Blog

I just discovered this trick, and I’m recording it here so other people don’t have to search as much as I did to find it. To dump the contents of a bean b use this code:

org.apache.commons.beanutils.BeanUtils.describe(b).toString()

This uses reflection to dump the bean field names and values – very useful in debugging. describe returns a HashMap of fields and values – also useful if you want to check that specific fields have been set properly.

[ad#co-1]

There have been quite a few sites set up recently that provide lists of apps on the Apple iPhone AppStore – apptism.com is probably the best known. Obviously they are accessing the AppStore and pulling the data down by masquerading as iTunes. I decided to find out how they are doing it.
In order to monitor the network connection between iTunes and the AppStore I needed a network packet sniffer. I googled the best solution and found that OSX ships with tcpdump, which does a good job of tracking network traffic – the command I used was

sudo tcpdump -s 0 -A -i en0 port 80

which gave me enough details to see what was happening.
I then started up iTunes and went to the AppStore page, and I was able to see all the messages passing between the two – it looks like iTunes AppStore feature functions very similarly to a web browser, although it doesn’t use HTML – it is based on a proprietary,and quite complex, xml format. There are different ways of finding the apps in iTunes – by clicking a category, and then paging through, which is painfully slow although it shows details of each app, including the price, rating and a description. The alternative is to use the browse by category list, which just lists the app name, price and genre. Since I plan on pulling down all the app details, I thought I’d start with the browse list and then drill down through the categories to the individual apps.

The url iTunes uses for browsing is
http://ax.itunes.apple.com/WebObjects/MZStore.woa/wa/browse?path=/36
This returns a list of categories and associated IDs in pairs – the meat of the xml response looks like this:
<key>infoURL</key><string>http://ax.itunes.apple.com/WebObjects/MZStore.woa/wa/viewGenre?id=</string>
<key>items</key>
<array>
<dict>
<key>itemName</key><string>Books</string>
<key>itemId</key><integer>6018</integer>
</dict>
<dict>
<key>itemName</key><string>Business</string>
<key>itemId</key><integer>6000</integer>
</dict>
<dict>
<key>itemName</key><string>Education</string>
<key>itemId</key><integer>6017</integer>
</dict>
...


The first item is a URL which can be used together with the category id to generate a list of apps in that category. The categories and their ids follow in <dict> pairs.

These URLs can be tested with curl or in a web browser. I’ll cover getting category entries and individual app details in a later posting.
[ad#co-1]

I re-read my recent post on agile development and I think it sounded rather more negative that I intended. I don’t wish to give the impression that I am opposed to agile methodologies – quite the contrary. I don’t think I’ve ever worked in a development environment that I would not describe as agile – I think that kind of approach is the norm in Silicon Valley, where I have worked almost my entire career. I think the large corporate IT departments are the ones that embraced the waterfall, highly structured and inflexible approaches that agile methodologies (henceforward referred to as simply ‘agile’ or ‘am’) is advocated to replace, and I would never advocate that approach. Having said that, I think there are different ways to implement am and different reasons for doing so. I have seen them introduced in four companies, and only in one was it done for the right reason and in none was it a classic replacement of waterfall methods.
In the first case, the company was failing and in an attempt to find a scapegoat the development team was deemed to be “out of control” and so a project manager attempted to rein it in by imposing agile on them. This was a complete failure because the development team was in fact doing a great job, and resented being bullied by someone who turned out to be too controlling and ultimately incompetent. The company eventually failed, certainly hastened by the departure of most of the development team who refused to buckle under.
In another case a new manager was hired to run an existing team, and in order to make his mark he also tried to impose agile methodologies, again in a way that increased the reporting and meeting burden on the developers and caused resentment and dissatisfaction. That team also started to disintegrate but thankfully the new manager was replaced before any real damage was done.
In the one successful case, the development team did not really change its approach at all – they had been used to working on short-term projects, and were releasing daily updates to their product. The change really applied to the project management and executive teams who learned whole new ways to communicate. They used the story approach to better define their requirements, and were able to understand the scheduling impact of their projects and to gain insight into the development process by becoming more involved in the planning and estimating processes. And of course they got to see exactly how things were progressing by turning up at scrum meetings.
So I think the moral is that a successful agile process allows everybody space to do what they do best, without any power struggles or imposition of one group’s will onto another’s. And mostly, I think it often requires a change in mindset of the customers more than it does the developers.
Oh, and the prayer meeting thing? It’s probably just me, but when I see a bunch or people standing in a room round a table, often with heads bowed, it’s not a rugby match that comes to mind.
[ad#co-1]

I have a window with a variable number of sets of radio buttons I need to get the values of, so I have some javascript that does a few iterations of ids and uses eval to read the values since i don’t know in advance what the radio buttons are called. Sometimes it works flawlessly. sometimes Firefox just disappears, and sometimes this happens:

Firefox telling me there's something wrong with my javascript.

Safari seems to work ok, and of course all the javascript debugging tools I have are part of Firefox. Oh well.

Every company I have worked at for the last 5 years has tried to implement some kind of Agile development process. It seems to be the new mantra, the silver bullet of software development, similar to the way Object-Oriented Programming was ten, no wait fifteen years ago. Of course OOP was a development methodology and Agile is more of a management process, but that in many ways makes it more appealing to managers since they can understand it better. Each company I have seen seems to have taken different parts of the process and incorporated them in different ways into their existing process. All have had some kind of meeting which they called a scrum, usually an existing meeting with a new name. Most have instigated “sprints”, usually of 2 weeks, to implement suitably small pieces of functionality. Usually the enthusiasm wears off fairly quickly, often when a major re-write is required or something else that doesn’t fit the model and daily meetings are reduced to people repeating the same thing every day for weeks on end. Which is fine by me. I’m sure the whole Agile thing is perfectly appropriate for some sub-set of environments and projects, but for those that it isn’t it’s really painful to try to make it work. And I really dislike doing something because somebody thinks I ought to be doing it. Plus it makes me feel like a Toyota factory worker singing the company song in the morning. I don’t think the development process at any of these places was broken before they were “agile”, and I don’t think any of them were significantly better while it was. If a project is well managed and the lines of communication between different stakeholders are open (which they should be) then a project is generally successful. If it isn’t and they aren’t then it generally won’t be, and it will take more than a few buzzwords to fix it. Keeping a team lean and focused is important, if you have to focus on that then go ahead and implement Agile. If you’re already lean and focused, and delivering, then it ain’t broke – don’t fix it.
[ad]

A while ago I posted that I’d tried Ruby and didn’t like it, thanks very much. But I’ve been back and tried it again and it’s beginning to grow on me.

To be honest, going back to Java was a little hard. It was a bit like going back to C++ after working in Java – it suddenly takes a lot of code to do something very simple. So I admit it’s true – Ruby does get a lot more done with less lines of code. The major problem I had with using Ruby was really Rails – it’s attitude is do it my way or don’t do it and I didn’t want to do it its way. I have another project now to prototype a simple cloud API, the only requirement was that it should be REST-ful and it would be good if it was in Ruby. So I did some googling and discovered Sinatra, a very lightweight framework that seems to have been designed specifically for writing REST-ful APIs in Ruby. And it is very cool. I don’t find the framework getting in the way at all – I just define the http action and URL and it handles directing the request to my code and then promptly gets out of the way. I intend to post the code to my prototype as soon as it’s presentable, but for anyone with a development background in Java thinking about dipping a toe into the world of Ruby I heartily recommend Sinatra as a starting point. Unfortunately I don’t see any easy solution for the performance issues I mentioned.

I had a client request an application in Ruby. I explained that I’ve never used Ruby, but that I’d do it if that was what was required. Having got that cleared up I downloaded Ruby and Rails and got started. Actually I discovered that Leopard has it pre-installed, but anyhow…

I know I’ve been out of the loop for a few years as far as new tools go, blame it on working in a very conservative environment. I was regarded as  radical for promoting Hibernate. I heard a lot of great things about Ruby I just never got round to using it.

The requirement involves an RSS reader filtering apps into a database with a UI displaying the most recent matched articles. Pretty simple stuff, and I figured a couple of days work in Java. But with the added problem of working in a language that I’m not familiar with I estimated a week. Turned out to be pretty accurate, but unfortunately that’s also about how long the thing takes to run.

It turns out that Ruby is a pretty cool language.  I do like some of its features, especially the whole class worh of getters and setters (and variable declarations) in a single line. But it’s interpreted, ffs. It’s S-L-O-W. Especially reading and parsing files. I don’t know why nobody’s produced a Ruby JIT yet, but it’s sorely needed.

I also found that if you want to get things done with Ruby you have to do it Ruby’s way – it’s all about programming by convention and if you don’t know (or like) the conventions you’re out of luck and life is going to be tough.

My overall impression is that Ruby is great for the things it claims to be great for (simple stand-alone web apps), but it’s not going to take over the rest of the development arena any time soon.

I created a page based on the technique I outlined in my earlier post about a simple CMS with PHP and Javascript serving static pages – it’s a recipe book app (I’ve always wanted to create one, I love food and cooking and I’m pretty opinionated about them), and it also demonstrates the technique pretty well. It’s a work in progress, so don’t expect great content just yet. That takes time, which I don’t have a lot of right now.

So I’m playing with PHP (it’s not so bad, but I’m not convinved its performance is up to the uses I’m seeing it put to), and I’ve thought of a cool way to do simple content management with Ajax and static pages. I don’t know if this is a common technique, but I think it would be useful in some situations. I created a top level page with a div area that I want to paste content into, and use PHP to scan a directory looking for subdirectories that start with a specific pattern (in this case “r_”), and create a menu item for each one found. For each of these I create a link to a javascript method to load a file called index.php from that directory. It’s kind of like using frames, but a lot cleaner. I never really liked frames anyway. And it’s really easy to add new items – I just add a new subdirectory containing an index.php with the content of the second div. I can also add links to other directories get loaded teh same way but that don’t get displayed in the menu. The code follows:

I’ve build a simple page with two div areas, the first contains the PHP code to scan the directory, the second will be used to display the dynamic content:


<body>
<div id="menu">
//root path containing CMS items as sub-directories
$path = ".";
//open the directory
$dir_handle = @opendir($path) or die("Unable to open $path");
//loop through directory entries
while ($file = readdir($dir_handle))
{
if (strstr($file, "r_"))
{
$file=substr($file, 2, strlen($file)-1);
echo "$file
";
}
}
//close the directory
closedir($dir_handle);
</div>
<div id="content">
</div>

And the following javascript:

//synchronous javascript request handler
function sJax (url) {
if (window.XMLHttpRequest) { // Mozilla, Safari,...
http_request = new XMLHttpRequest();
} else if (window.ActiveXObject) { // IE
try {
http_request = new ActiveXObject("Msxml2.XMLHTTP");
} catch (e) {
try {
http_request = new ActiveXObject("Microsoft.XMLHTTP");
} catch (e) {}
}
}
if (!http_request) {
alert("SJAX action failed.");
return false;
}
http_request.open('GET', url, false);
http_request.send(null);
if(http_request.status == 200) {
return http_request;
} else {
return null;
}
}
function load(url)
{
document.getElementById("content").innerHTML = sJax(url).responseText;
}
function loadlink(link)
{
var url = link + '/index.php';
load(url);
}
function loadDir(path)
{
var url = 'r_' + path + '/path.php';
load(url);
}


My old MacBook has served me well for several years now, but I think it’s time to upgrade. The power cord is looking a bit ropey (it needs twisting to get the juice running). I also need to be able to run VMWare (a new project requires me to host a linux VM image to develop against, a very cool idea). I need more speed and storage. And I want a shiny new computer. So I ordered a new MacBook Pro today from Amazon – with Prime membership and same day delivery it will arrive this evening. Incredible.
Update – yes it’s very cool. Amazingly fast. Love it.