Converting Blogger to Markdown

I’ve been using Blogger happily for three years or so, since I migrated the blog from LiveJournal and decided to actually invest some time writing. I’m happy with it because I just type stuff into Blogger and It Just Works. I’m happy because I can use my Google credentials to sign in. I’m happy because now I can pretend my two Google+ accounts exist for a purpose, by getting Blogger to automatically share my content there.

A couple of things have been problematic for the whole time I’ve been using it though:

  1. Code looks like crap, no matter what you do.
  2. Pictures are awkwardly jammed in to the prose like a geek mingling at a Marketing event.

The first problem I’ve tried to solve a number of ways, with custom CSS at a blog- and a post- level. I was super happy when I discovered gist, it gave me lovely content highlighting without all the nasty CSS. It’s still not ideal in a blogger world though, as the gist doesn’t appear in your WYSIWYG editor, leading you to all sorts of tricks to try not to accidentally delete it. Also I was too lazy to migrate old code over, so now my blog is a mish-mash of code styles, particular where I changed global CSS mulitple times, leaving old code in a big fat mess. There’s a lesson to be learned there somewhere.

The second problem, photos, I just gave up on. I decided I would end up wasting too much time trying to make the thing look pretty, and I’d never get around to posting anything. So my photos are always dropped randomly into the blogs - it’s better than a whole wall of prose (probably).

But I’ve been happy overall, the main reason being I don’t have to maintain anything, I don’t have to worry about my web server going down, I don’t have versions of a blog platform to maintain, patch, upgrade; I can Just Write.

But last week my boss and my colleague were both on at me to try Hugo, a site generator created by my boss. I was resistent because I do not want to maintain my own blog platform, but then Christian explained how I can write my posts in markdown, use Hugo to generate the content, and then host it github pages. It sounded relatively painless.

I’ve been considering a move to something that supports markdown for a while, for the following reasons:

  1. These days I write at least half of my posts on the plane, so I use TextEdit to write the content, and later paste this into blogger and add formatting. It would be better if I could write markdown to begin with.
  2. Although I’ve always disliked wiki-type syntax for documentation, markdown is actually not despicable, and lets me add simple formatting easily without getting in my way or breaking my flow.

So I spent a few days playing with Hugo to see what it was, how it worked, and whether it was going to help me. I’ve come up with a few observations:

Hugo really is lightning fast. If I add a .md file in the appropriate place, and with the Hugo server running on my local machine it will turn this into real HTML in (almost) less time than it takes for me to refresh the browser on the second monitor. Edits to existing files appear almost instantly, so I can write a post and preview it really easily. It beats the hell out of blogger’s Preview feature, which I always need to use if I’m doing anything other than posting simple prose.

It’s awesome to type my blog in IntelliJ. Do you find yourself trying to use IntelliJ shortcuts in other editors? The two I miss the most when I’m not in IntelliJ are Cmd+Y to delete a line, and Ctrl+Shift+J to bring the next line up. Writing markdown in IntelliJ with my usual shortcuts (and the markdown plugin) is really easy and productive. Plus, of course, you get IntelliJ’s ability to paste from any item in the clipboard history. And I don’t have to worry about those random intervals when blogger tells me it hasn’t saved my content, and I have no idea if I will just lose hours of work.

I now own my own content. It never really occurred to me before that all the effort I’ve put into three years of regular blogging is out there, on some Google servers somewhere, and I don’t have a copy of that material. That’s dumb, that doesn’t reflect how seriously I take my writing. Now I have that content here, on my laptop, and it’s also backed up in Github, both as raw markdown and as generated HTML, and versioned. Massive massive win.

I have more control over how things are rendered, and I can customise the display much more. This has drawbacks though too, as it’s exactly this freedom-to-play that I worry will distract me from actual writing.

As with every project that’s worth trying, it wasn’t completely without pain. I followed the (surprisingly excellent) documentation, as well as these guidelines, but I did run into some fiddly bits:

  1. I couldn’t quite get my head around the difference between my Hugo project code and my actual site content to begin with: how to put them into source control and how to get my site on github pages. I’ve ended up with two projects on github, even though the generated code is technically a subtree of the Hugo project. I think I’m happy with that.
  2. I’m not really sure about the difference between tags, keywords, and topics, if I’m honest. Maybe this is something I’ll grow into.
  3. I really need to spend some time on the layout and design, I don’t want to simply rip off Steve’s original layout. Plus there are things I would like to have on the main page which are missing.
  4. I needed to convert my old content to the new format
  5. Final migration from old to new (incomplete)

To address the last point first, I’m not sure yet if I will take the plunge and do full redirection from Blogger to the new github pages site (and redirect my domains too), for a while I’m going to run both in parallel and see how I feel.

As for the fourth point, I didn’t find a tool for migrating Blogger blogs into markdown that didn’t require me to install some other tool or language, and there was nothing that was specifically Hugo-shaped, so I surprised myself and did what every programmer would - I wrote my own. Surprising because I’m not normally that sort of person - I like to use tools that other people have written, I like things that Just Work, I spend all my time coding for my job so I can’t be bothered to devote extra time to it. But my recent experiences with Groovy had convinced me that I could write a simple Groovy parser that would take my exported blog (in Atom XML format) and turn it into a series of markdown files. And I was right, I could. So I’ve created a new github project, atom-to-hugo. It’s very rough, but a) it works and b) it even has tests. And documentation.

I don’t know what’s come over me lately, I’ve been a creative, coding machine.

In summary, I’m pretty happy with the new way of working, but it’s going to take me a while to get used to it and decide if it’s the way I want to go. At the very least, I now have my Blogger content as something markdown-ish.

But there are a couple of things I miss about Blogger:

  1. I actually like the way it shows the blog archive on the right hand side, split into months and years. I use that to motivate me to blog more if a month looks kinda empty
  2. While Google Analytics is definitely more powerful than the simple blogger analytics, I find them an easier way to get a quick insight into whether people are reading the blog, and which paths they take to find it.

I don’t think either of these are showstoppers, I should be able to work around both of them.

Sevilla MUG March Madness

Last night the Sevilla MUG had our March Madness event. This was our largest event yet, with 36 people signed up. Although the aim of March Madness is to have a MongoDB Engineer at all the user groups this month, that’s not such a big deal for us as I live here, so this was also the first event where I wasn’t the main attraction - [Javier] (https://twitter.com/JvrBaena) gave a really great talk about the lessons learnt at SocialBro after using MongoDB in production for the last couple of years.

The slides for my introduction to Replica Sets and Sharding:

And the slides for the main attraction:

The event went really well, everyone seemed engaged, and our first talk in Spanish seemed to encourage more questions than normal. We were also in a new venue, and although I love the central location of our previous venue and the friendliness of the owners, the menu of the new location seemed to be a massive win.

QCon London 2014

Wow. My 4th QCon London. That’s not bad. And every time, it’s a different experience (if you must, see my blogs for 2013, 2012, and even 2007 (part 1 & part 2 - how cute was I? "agile seems like a jolly good idea; automated testing appears to be important")).

I can’t even tell you what I did on the first day, I was mostly panicking about my presentation - I was inspired after my trip to New York last month to change my talk at the last (responsible?) minute and do a live coding session, something much more technical than my recent talks. I’ll leave the details for a separate blog post though, when the video comes out.

The thing that stands out for me from Wednesday though was Damian Conway programming Conway's Game of Life in Klingon. Yeah. Just find the video and watch it, the man is a genius.

Damian Conway, Life, The Universe, and Everything

The Thursday keynote was inspiring too from a totally different point of view - Tim Lister of Peopleware fame shared stories from his career, and I came away from that really happy I work as a technologist, but with an increased desire to learn off other amazing people.

Tim Lister @QCon London 2014

Not Only Java track - I’m on the programme committee for QCon, and this year we wanted to cover leading edge technologies (as always) but we didn’t want to slice things into strict technology silos (interruption: argh! the person in front of me nearly destroyed my laptop by suddenly moving their chair back! Why do people bother in economy on a morning flight?). So I wanted the Java track to be more representative of what today’s Java programmers care about - for the programmers of course, but also because I know there are architects and team leads at QCon who might not realise how things have moved on with the language, and how much polyglot programming we do these days.

Martin kicked it off with a great history lesson on the progress (or occasionally, lack of it) in Java. He begged us to study and understand Set Theory, to use async design, to think of the users of our APIs, and, most of all, to design nice, clean code.

Next up, Eva took us through the fundamentals of Garbage Collection - this might not seem like a cutting edge subject these days, but it’s one of the most misunderstood subjects for Java programmers. Eva gave us a really great, understandable view of the different types of garbage collectors, how they work, and their pros and cons. She left us with a call to arms to not simply let other people try and solve this problem, but to get stuck in and contribute ourselves, via the OpenJDK.

Trisha Gee @QCon London 2014

After lunch was my nerve-wracking live coding session, putting together a full stack end-to-end web app using AngularJS, HTML5, Java and MongoDB. It only went wrong twice, and people seemed to like it. I’ll post the video in another blog post as soon as it's publicly available. Code is available on github.

We’ve been playing with the open spaces idea at QCon. The Java one only had a few people in it, but that gave everyone a chance to speak at least. We covered Java 8 (the Good and the Bad); other JVM languages; and UIs for Java (Javascript or GWT?). And I plugged the work the LJC does in London, of course.

After this Bodil blew me away creating a My Little Pony game using RX in the browser. ‘Nuff said.

Finally, Simon Ritter gave us a view of the Java 8 features most likely to impact the way Java developers think about software design - lambdas and streams. I thought this was a really great introduction to the concepts if you haven’t seen them before, and with concrete examples that showed how we should be using them. If you're not already looking at lambdas and streams, you should be - even if you're not going to be using Java 8 yet, it's worth getting a heads up on how it's going to impact our programming style.

I’m very pleased with the way the Java track turned out on the day - every speaker was first class, a wide range of topics was covered, and I, for one, learnt something in every presentation.

To finish off the day, Emma Langman gave an awesome keynote about how people are the messy bit of your system, and how they’re never rational and you shouldn’t expect them to be. I also highly recommend this talk, especially if you’re a techy and you’ve found yourself in some sort of management or team lead position.

Sadly I couldn't stay for day three of the conference, I had to fly off to the Joy of Coding to re-give the live coding presentation there. Because, if you’re going to do something terrifying and doomed to failure, you might as well do it twice.

QCon is an expensive conference, especially compared to the developer-friendly prices of something like DevoxxUK, but for getting a big picture of where the industry is, of things you might be missing, for learning hard core technical skills and understanding the important of the fluffy-people-stuff, and finally for meeting a wide range of people from developers to CTOs, I think you'd be hard pushed to find something better in London. IMHO (and remember, I did disclose I'm on the programme committee).

And although it's really hard work putting together the programme for a track like this, and although both times I've said I'm Never Doing It Again, when it goes this well it makes you want to do it all over again. After a break. A looong break.

Sparking innovation in an established company

I’ve been running into BSkyB a bunch this week - firstly I was invited to kick off their innovation tech talks series last Monday, then I kept meeting Sky people (that makes it sound like they’re aliens) at QCon.
It seems Sky is keen to create/foster a culture of innovation, which is an ambitious goal for such a large company.  So I was given a very vague brief for Monday’s presentation (“Can you do something Mongo-ish, maybe a bit of performance and concurrency stuff, and talk about overall best practice?”) I gave them an early view of a presentation I’m working on titled something like “Why do we keep reinventing the wheel?” (anyone who saw me at JFokus might realise that I cunningly swapped out most of the body of the “What do you mean, backwards compatibility?” talk and tried out this topic instead).
Picture courtesy of +Russell Miles 

This was supposed to be a 20 minute intro talk (OK, so I clocked it at 40 minutes in rehearsal) with the rest of the time devoted to questions on any subject I know about, or even anything I have an opinion on.  Sounds dangerous, but means I can always move fast on the answers I don’t know in order to cover more ground - cunning, you see?  Actually, that sounds like it would be more fun if alcohol were involved.

Anyway.  The talk bit of the session took up most of the time, because I had really good questions during the course of the presentation.  It’s nice to talk to a much smaller group (there were maybe 30ish people, giving up their free time after work, like an internal user group).  But I also had questions on remote working, on creating a culture that’s appealing for developers to come and work in, and on MongoDB.

I don’t know how many companies do things like this, or lunchtime brown bags, or something, but I think they’re a great idea.  This was a mix of people from different departments, different teams, working on different technologies.  It was a good way to get people from across the organisation talking with each other and sharing best practice between teams.  Many organisations have people working in their little silo, and we (MongoDB) often find ourselves introducing teams from within the same organisation to each other - there’s no other mechanism for one team to find out about the other.  I guess that’s what white-tower architects were supposed to be for - to know what everyone’s up to and make sure people are doing the “right” thing.  Given this doesn’t seem to work, what else can we do to share knowledge internally?  How else can we get the right people to meet each other?

The Joy of Coding


Dick Wall - The Tao, of the Joy, of Coding

I’m on a plane (there’s a surprise) on my way back from The Joy of Coding.  It’s the title that attracted me to this conference, and it was a great little one day event with awesome people like Dan North, Erik Meijer and Dick Wall presenting.  What I liked about the themes is they were broadly technology agnostic, pulling back to a level which made me remember why I’m a developer.  It inspired me to do more research and more thinking, and less panicking and trying to write code.  That might seem odd, as someone who goes to as many conferences as I do should get loads more research and thinking time than most developers living under the whip, but I seem to spend so much time bouncing from thing to thing I don’t get space to just sit and think, or to read things in any depth.

The conference organisers made it easy to focus on the bigger things - the venue was great, with a nice space for mingling with people but nooks and corners for doing work or for video conferences.  The hotel they chose was one of the best I’ve stayed at - the room had a sauna in it.  A sauna!  I spent the evening using every toiletry they provided, and moving between the sauna, the steam/shower cabinet and the jacuzzi bath.  I won’t pretend I did a lot of thinking, but it was a really awesome way to top of a crazy week, and I feel almost sane.

So if I’m going to leave you with one thing from my experience at the Joy of Coding, it’s to Stop.  Stop and think.  Stop and ask.  Stop and read.  What are you doing?  Why are you doing it?

And most of all, do you feel Joy when you’re coding?

PS There are some really great photos from the conference on Flickr.