Imperfection
Before bomber planes came in to existence, WWI aircraft crew used to take a sack of bombs in to their biplane cockpits & lob them over the side after flying in to enemy territory [1]. I like practical actions in the face of technical inperfection.
[1] according to a Timewatch DVD I just watched.
Lab UK code_swarm
Matt kindly put our current project SVN logs in to code_swarm. We should add it to our project documentation.
project visualization with codeswarm from Matt Haynes on Vimeo.Pretty amazing visualisation. It could do with a bit of curation to help explain key points/people (’who was Mike?’, ‘when was r0.1′, ‘why the big explosion of activity on Oct 20th?’ etc.), a bit like the Python example on the code_swarm site.
Perhaps it could read & display some conventions from the log messages or release notes, but curation is quite a personal thing and only really doable to those with a intimate knowledge of the project, so here it goes…
Looking through our logs, mid-to-end of October marked the end of the technical prototype, where Al left and the project started in earnest with designers, flash developers, project managers all starting to make commits, which explains the flurry of activity.
Matt H, the main developer is a sort of persistant floating presence, he’s joined by Darren in mid-November who replaced Al. Mike & Tom arrive for a few weeks then gradually fade away as they were contracted in to complete specialist parts of the build.
Pierre, the senior designer, fades out as the build goes on, indicating (in our department, at least) design is very much an up-front activity with lesser input the longer the project exists. As we start v2 of the project around April I’d expect the design activity to increase a lot more.
There’s a nice long period of inactivity for xmas.
The very tall lines in the histogram along the foot of the video are external dependencies (flex, phpdoc etc.) rather than huge commits by a single developer, otherwise they might indicate a developer has been saving up code on their working copy and committing it all at once, which of course would be naughty of them.
Peter B stays too long in the visualisation for my liking, he only committed once, and that was a mistake (we share our repository with the rest of the BBC), so we should probably filter him out of our logs.
I expect the project to settle down in to fortnightly bursts of activity to match our release pattern as we work toward a live release.
Web Fonts & Wingdings
I noticed Opera have just shipped an alpha version of their next release that includes support for CSS3 web fonts, which amongst other things adds the ability to distribute fonts with your pages, TTF, OTF etc.
Most people use fonts to express the alphabet, punctuation etc. but typographers also ship some very pretty symbol typefaces known as dingbats.
Here’s a demo of using an embedded dingbat typeface to add some bullet makers to a list via css3.
After setting up the font description,
@font-face { font-family: "Sarus"; src: url("http://commuterjoy.co.uk/resources/ttf/sarus.ttf") format("truetype"); }I can attach a letter to any node using the :before selector,
li:before { font-family: Sarus; font-size: 2em; content: "A"; }Then toggle the color & style of it using the :hover selector.
li:hover::before { color:red; content: "B"; }The best thing about native web fonts (unlike, say, image icons) is that they scale well and respond to standard CSS properties like ‘color’, ‘font-weight’, try upping the font size of the example.
Maven Notes
I’m trying out Maven at work, it’s not much fun.
It solves some of your problems like JAR dependencies quite neatly and the subscriber concept works conceptually (ie. in Ant terms, targets listen out for standardised events rather than having to chain them together by hand each build), but all the documentation I’ve read explains things in such a roundabout way that it just confuses matters. Here’s a helpful quote from the O’Reilly Maven book (page 182),
The interesting phase in the clean lifecycle is the clean phase. The Clean plugin’s clean goal (clean:clean) is bound to the clean phase in the clean lifecycle. The clean:clean goal deletes the output of a build by deleting the build directory.
Got that? Rather than,
<target name="clean"> <delete dir="${build.dir}"/> <target>One good thing: Maven has a published XSD, which I don’t think Ant has ever bothered with. Maybe I’ll get it in a week or two.
EC2 running Windows running on OSX
I copied what Paul did.
It’s ok for browser testing, I guess you want to build a custom image with your favourite browsers rather than the off-the-shelf one. My main exploration is to see if I can use it to host an instance of Selenium RC to which I can point our CI server, otherwise we’ll have to continue to use a bespoke underthedesktm solution.
Model Villages & Thoughts on Tinkering
I’ve been playing with TUAW’s and Receding Hairline’s fake tilt-shift. The illusion of a shallow depth of field makes your photo’s look like they were taken above a model village.
A not-to-great example from the French coast …
It’s reminded me of how much I like playing with Photoshop tutorials (I hardly ever open it these days, unless I need to chop or resize a JPG!) and how you can lose and entire evening tinkering with the sliders, filters, channels, gradients to get it just right … It feels very creative even thought you are essentially just adjusting numbers that are fed in to Adobe’s canned CS3 image algorithms.
Duck, Tequila and Chili Soup
Was hoping to do chicken marinaded in Tequila but I accidentally defrosted the duck instead. There’s seemingly not a lot of South American duck recipes, so soup to the rescue.
- 750(ish) pints of strong duck stock - some duck fat - 2 duck legs, cooked and meat shredded - 2 onions, chopped - 100ml Tequila - 3 red chillies (or more) - 4 garlic cloves - squirt of tomato purée - 1 juiced lime - 1 chopped carrot - 100ml cream cheese (or maybe some strong cheese) ~~~~~~~~~~~~~~~~ - sauté the onions in the Tequila, put to one side - fry the garlic, then the chilli in a little duck fat - add the stock and the onions, tomato purée, carrot - cook for 25 mins - add the shredded duck and the lime juice - blend slightly (10 secs) to thicken
Inspired by Garlic Breath.
Poppystar Chairs (iPhone camera test)
Not great …
Eversley
Phew, another weekend, another dandy time spent watching my friends scorching the enamel from their teeth with burning liqueur.
Idea for Nicer RegExp Errors
It would be nicer if RegExp objects assigned the mismatched portions of the test string to an array in a similar fashion to which they store portions of successful pattern matches.
var foo = /^[a-z]+$/; foo.exec("34asdf"); // returns nullSo, the developer knows the input string (’34asdf’) is invalid but not why. It would much more useful to the calling method to know the specifics of why the string didn’t match the pattern and, say, in the cases of interfaces and web forms be able to communicate with precision to the user the reason why their input has failed not just that it has.
For example, given the regex test above, it would be useful to be able to look up a failure property that has been populated as a result of the test,
foo.failures = [ "substring '3' found at position 1, expected '[a-z]'", ... ];
I’m not aware of any language that implements regex’s doing something like this. In JS you’d have to either throw away your regex pattern and write a lot of custom substring checking functions or split your regex in to multiple parts, but by doing this you lose some of their concise beauty.
France 2008
In the spirit of our Dublin trip here’s a list of stuff we did so I don’t forget.
Took a trip to San Sebastian and spent a night in Villa Soro. Very friendly, pretty hotel.
Ate at Andoni Aduriz’s Mugaritz. The veal enveloped in a vine-infused charcoal from the Naturan menu was one of the most interesting things I’ve ever tasted.
Ordered jambon, anchovies, and olives (in Spanglish with a slight French accent as dictated by the babel fish) in the food hall under San Sebastian’s main square.
Watched surfers on Playa de la Concha.Tripped to Biarritz. Reminded me how nice UK seaside resorts could be but generally aren’t. The waves looked to be about 6-7ft by my untrained eye with more bodyboarders than surfers. There’s an exciting tunnel you can rush through as the waves crash against the sea wall every 15 seconds, only a little spoilt by a local vagrant using it as a shower facility.
Took a suprisingly easy drive up the Pyrenees (our last drive to Col de Port was fraught with hairpin bends, narrow roads, and agitated French drivers) and a 5 hours round trek through meadows and an ever-increasing incline to the Cirque de Gavarnie. We went back an alternate way only to find it blocked by a warning (”pour votre sécurité”) so we took a shortcut back to the other trail through the UNESCO protected flora and fauna.
Thanks to the ever more useful N95 we found a petrol station next to Blagnac Airport so Avis can’t charge us some exorbitant fee for topping up the tank. Ha Ha.
Tarball without hidden directories
If you wanted to make a tarball but you have a bunch of .svn directories stuffing up the place this will make the tar.gz but exclude hidden directories,
tar -c --exclude '.*' -f - path/to/files | gzip > /path/to/archive.tar.gz
Patching
I feel a bit dumb for only having just discovered the patch command.
My local file system is full of copies of bits of code that have gradually morphed from what I set out to do to what I ended up with, a path strewn with fruitless (but brave, nonetheless!) diversions. Using Subversion does solve the problem managing ever changing files over time it’s pretty bothersome to keep jumping around the revision history in your local working copy or even attempting to compare and run multiple versions of the same file at the same time. Even remembering which revision does what is a bit of a struggle unless you’ve got a good commit message convention.
I think patch can make this easier by allowing you to store your experiments from the main trunk code as a little library of diff snippets.
Eg. Let’s say you have a JavaScript file called ‘original’,
// -- original -- // returns a charArray of a string String.prototype.toCharArray = function(){ var a = this.split(""); return a; }If you copied the above file and added an experiment to it you might end up with this,
//-- new -- // returns a charArray of a string String.prototype.toCharArray = function(){ if ( this.length == 0 ) // don't want empty arrays return false; var a = this.split(""); return a; }You can now diff the two files and store the result as a patch file …
diff orginal new > lengthcheck.patch
… the contents of which looks something like this,
//-- lengthcheck.patch -- 2a3,4 > if ( this.length == 0 ) // don't want empty arrays > return false;
At some later date you can patch your code with following command.
patch -b orginal lengthcheck.patch
The -b switch makes a backup of your code. Patch will prompt you if it finds a problem and store any rejected patches in a seperate file for you to inspect.
The idea being that in the course of, say, a 2 day hacking session, you can keep the core code in your SVN trunk directory while the deviations, for better or worse, can be stored in a library of diff’s that you can periodically merge in and out of your mainline development.
Spring Garden
Python
I think I might learn how to do things in Python, sysadmin tasks, mini web apps and the like.
I’ve come to know a healthy amount about Perl in the past few years, mainly due to it being the only language officially supported at work, but it has some things I’ve not really got on with.
It’s error handling is a bit rubbish if you are used to the try/catch/throw style of some other languages. Errors in Perl are mostly handled by adding conditionals around (or on the end of) a bunch of statements.
# if there's a problem opening 'foo.txt' then exit with the error open( file, 'foo.txt' ) or die $!;
The OO stuff in Perl feels a bit contrived and it’s easy to cheat or pick up bad habits. Some of Perl’s basic functions remain resolute in their non-OOness…
# adding an item to an array, passing the array as an argument to the push function my @foo = ("a", "b", "c"); push(@foo, "d");The feeling of tacked-on OO also manifests itself in calling a classes methods. You have to remember to pluck the object out from an implied argument before operating on it. Normally you’d expect to be able to use a ‘this’-like reference without having to manage this sort of low-level stuff yourself.
# if foo was a method of some class, $class would hold a reference to the calling object. sub foo{ my $class = @_; }Perhaps the main reason I don’t want to keep using Perl is that it hasn’t seemed to introduce much of interest to the language over the 4 or 5 years since I’ve known it. Most other languages I know have had pretty significant upgrades and improvements in that time (XSLT, JavaScript …). In that time Perl has had a few minor version number patches but I can’t see anything to motivate a casual user like myself to upgrade, so I just stick with whatever is on the box I’m using.
Maybe Python won’t do these things any better, but I won’t know until I try.
Updated
I forgot one other thing. Because I don’t write Perl every day I find it a real struggle to remember the specifics of the often dense and syntax. For example to get the length of an array you need to remember the $# convention, which you eventually remember after the first few times, but something like ‘[array].length’ would be more obvious.
There’s lots of little ticks like this $_ (implied variable), $! (error message), @_ (arguments to an subroutine) that you don’t use so often as a casual developer and have to scout around to trigger your memory…
# assign the length of array 'foo' to $a my $a = $#foo;
XML
Jeff’s rant about the ubiquity of XML got me thinking about the things I like about XML, so here’s some notes…
1. Standard API’s for parsing. - DOM, SAX, E4X, StAX etc. These are all attempts to provide standard (ie. langauge agnostic, by consensus) ways of reading/writing the XML data.
I know JSON and YAML etc. all come with their own parsers from which you can munge the YAML/JSON in to some internal data structure or object and set about looping over it and extracting bits out in whatever way you see fit but the API-like approach of defining how you should access data XML makes more sense to me. I’d rather developers I work with use standard ways of processing and accessing data than each doing their own thing.
Having said that, DOM isn’t perfect, so there’s many libraries (like JQuery) that provide convience access methods, and guess what? These differ per language. Argh!
2. Vocabularies. With XML I can define my own vocabularly and mix parts of existing vocabularies. Ascribing meaning the data you are working with forces you to think outside of how the infomation should be structured, more about what it represents. I’ve found JSON and YAML over-literal in their representations of data, so you end up designing formats that looks like data structures, which *is* great for many situations but loses something of the semantics.
3. Type checking. By defining vocabularies (either in RelaxNG or XML Schema) you will probably end up using XML Schema data types, making it easy to tightly define (and enforce the integrity) of the data you are working with. There’s some really helpful default types (like ID + IDREFS) that solve specific problems when working with XML as well as the usual date, duration, uri types.
4. Same-langauge Schema. One useful byproduct of XML Schema (or RelaxNG) documents being defined in the same terms as the data themselves (ie. XML - a common complaint) is that they can very naturally become part of the transformation process (or, say, unit testing process).
Say your Schema includes an implicit attribute with a default value and your XML source documents sometimes include it, sometimes not. The knowledge of this particular attribute’s behaviour and properties can be written in to the XML processing language without having to be overly specific about the details.
# if the attribute doesn't exist and is defined as mandatory in the schema, # then go and fetch the schema value and output it. IF not(foo/@bar) and doc(schema.xml//element[@name = foo]/optional/attribute[@name = 'bar']) PRINT doc(schema.xml/...) END
I think JSON schemas (being valid JSON themselves) will probably benefit from the same approach.
5. XPath 2.0. When using other formats I’ve never understood how to get the data I want from the JSON/YAML/CSV data structure other than having to write little subroutines with temporary data structures, loops etc. to extract, join, compare, transform the info in to what I want.
That’s sometimes ok, but XPath (particularly XPath 2.0 used with Saxon 9) elimates this problem for me by providing a hugely expressive set of statements for selecting, sorting, and querying parts of the document combined with some more mundane things like regexp and a whole variety of string functions.
I know a lot of people were put off from using XML as general container formats by XSLT & XPath 1.0 but I found version 2.0 feels so much more natural to author without having to jump off to using extension functions every other statement.
I’m not a complete XML zealot. The project I’m doing at the moment uses a variety of XML (for source data & communication from web services), JSON (for browser loaded data) and CSV (for producers to edit), whatever fits really.
XSLT 2, Saxon 9 and Ant error
"Cannot write an implicit result document if an explicit result document has been written to the same URI: file:/path/to/my/file.xml" at net.sf.saxon.Controller.checkImplicitResultTree
Odd error of the week. Ant, or rather Saxon, seems to run over everything in the basedir twice, and at the second pass creates the above error message. I’m using the following to run the transforms over an directory of xml documents,
<xslt basedir="${project.trunk}/xml/" destdir="${project.home}www/" extension=".xml" style="${project.home}/xsl/foo.xsl" classpath="${ant.lib}/saxon9.jar;${ant.lib}/ant-trax.jar" processor="trax" force="true" > <param name="foo" expression="hello world"/> </xslt>I fixed it by adding a include directive (xslt supports implict filesets) as a child of the xslt task.
<include name="**/*.xml"/>
Grep For Non-blank Lines
Find all the *.as files in the src folder and print every line that isn’t just whitespace characters (spaces, tabs etc.)
find ./src -name *.as -print -exec grep -Rv ^[[:space:]]*$ {} \; | wc -lUseful for counting lines-of-code.
Ant Error & Fix
Using Ant 1.7 on OSX 1.4 with commons-net-1.4.1.jar in my lib path and the built in FTP task behind our work firewall when I get the following message:
java.net.SocketException: SOCKS: Connection not allowed by ruleset
I fixed it by adding the full address of our firewall to the ‘Bypass proxy settings …’ list in the System Preferences -> Network -> Connection Type -> Proxies.
Kludsky
Isn’t this grand? If I had $400 spare I’d buy the entire Archive Type collection.