Joe Turner

A developers toolbox?

Yesterday I posted about DIBI, a web conference I attended recently. One speaker, Gareth Rushgrove, spoke about how a modern web developer has many tools available to them to do their job. The title "web developer" should actually be replaced with just "developer" as we now have to deal with much more than just the web.

So what tools are there that could make your web apps run faster and more reliably? Just a few mentioned during the talk were: -

1. Nginx - nginx.org

   A very lightweight, FAST web server. An interesting quote mentioned during the talk was by Chris Lea,

    

   “Apache is like Microsoft Word, it has a million options but you only need six. Nginx does those six things, and does five of them 50 times faster than Apache”.

    

2. Memcached - memcached.org

   A high performance distributed caching system intended to increase speed of dynamic web apps by relieving database load. It is also useful in that it scales as your server resource grows.

3. Apache Solr - lucene.apache.org/solr

   How many of us are guilty of building a website, then dropping on a search box right at the end and not spending any time thinking about how we are actually going to obtain the results. Running direct database queries is not only slow, but very expensive! Plus, as your site grows, the search will come back and bite you and eventually, take your server(s) down.

   Enter Solr, a fast, open source search platform from the Apache Lucerne project. Features include full-text search; hit highlighting and even handles rich document formats like Word and PDF.

   Written in Java, it runs as a standalone full-text search server on something like Tomcat. It has REST HTTP/XML and JSON APIs that means it can be accessed from more or less any programming langauge without any need for Java coding.

4. RabbitMQ - rabbitmq.com

   Traditionally the web has always done everything synchronously. A request is sent to a server, data is processed and finally the result is sent back. This is fine if the request is minor, but what if the server elected to process the request takes 15 seconds? Is the user expected to wait, or can they go on and send further requests? By queueing messages it takes load off servers, which can just pop a message off the queue when it has spare capacity, and also means users aren’t sat waiting for a result to be returned.

5. Data storage with CouchDB - couchdb.apache.org

   Sometimes a full relational database is overkill when you just want to store a large amount of data. CouchDB is schema-free meaning that as your data evolves, it will evolve with you. Normally with relational databases like MySQL you would have to migrate existing data to fit into a new schema. With couchDB this isn’t the case, as no schema is enforced, new data can be safely added alongside the old.

6. Hadoop - hadoop.apache.org

   Hadoop is useful tool for distributing tasks across many nodes in completely different geographical locations. Whatever you would use hadoop for, you would need Hadoop common, plus one or more of its subprojects such as HDFS (Hadoop Distributed File System); HBase a scalable distributed database for storing large tables of data or MapReduce, a framework for distributing processing of large data sets across multiple nodes.

7. Cucumber - cukes.info

   A Behaviour Driven Development tool, Cucumber allows ‘tests’ to be written in the form of plain text feature descriptions with scenarios. These scenarios can then signed off by project stakeholders, business analyists and collegues. Production code can then be written to make the ‘stories’ pass. Cucumber is written in Ruby, but can be used to test many other languages such as Java, C#, Python and PHP.

8. Puppet - puppetlabs.com

   More and more developers these days need to also be some kind of sysadmin. Whether its being a full on linux geek or not, setting up a server for production use is now a required skill. This is where Puppet comes into play. No matter how many times you do it, installing and configuring a server is a pain. By using puppet, you can declare multiple setups depending you what you will be using the node for. Do you need a bare bones install with SSH running? What about extending this node declaration and adding on Nginx? With puppet this is as simple as extending a class in PHP!

DIBI 2010 : Design it, Build it

Held at The Sage in Gateshead, DIBI is one very few, if not the first, to bring designers & developers under one roof for a day and allow them to listen to some of the industries leading names talk about their experiences and passions.

What made the event great was that both design & development 'tracks' ran side by side, so you could jump between halls depending on what topics you were interested in without worrying about what your 'official' job title was. So if you were developer that wanted to hear about iPhone UI design, or a designer that had an underlying passion to bring their designs to life with a bit jQuery magic, it didn't matter.

Here's a run down of my particular day and a brief idea of what I heard!

Jon McLoone

Using Wolfram|Alpha to build computational applications

To start the day, both tracks got a brief insight into what Wolfram|Atlpha is; what its purpose is; why it isn't, and never will be a 'Google killer' and the small question of how Wolfram Research can make money from it. A few examples of data that can be extracted from Wolfram|Alpha impressed the audience, and Jon answered any questions about the application comprehensively.

Elliott Kember @elliottkember

Pimp your app with jQuery

The Kiwi with a unhealthy obsession for the colour pink kicked off the developer track with a great whirlwind talk around the incredibly powerful tool, jQuery. In 30 minutes Elliot managed to cram in several ideas on how you can implement jQuery to really put some zing into your web apps. Simple tasks like show and hide, to animating elements in or out of focus, to posting Forms with AjaxForm. A difficult topic to explain even if the majority of the audience were already users of the framework and Elliot managed to do this clearly, showing a few tricks along the way to save time when battering the keyboard.

David Singleton @dsingleton

Last.fm vs Xbox

So you have a kickass start-up like last.fm and Microsoft goes and creates an app to hook into your system from millions of living rooms across the globe through their Xbox. Great right? Hell yeah, just the slight issue of how the hell do you cope with the increase in traffic, especially the launch day spike?

David explained the process last.fm took to estimate what traffic they may receive, and after the ‘Oh Sh*t’ moment, how they would deal with this increase. From profiling their API with tools like Kcachegrind; politely asking Microsoft to rework their app so it didn’t send 3 requests just to make the ‘love’ button glow, to heavy caching at several stages of the application stack. David gave some great information and showed that even successful web apps have their problems, such as server fires, or security issues at data centres…..

Gareth Rushgrove @garethr

You're going to need a bigger toolbox

Currently a developer a Glasses Direct, Gareth gave a a brief talk about the many tools a developer has to work with in 2010. In years gone by, it was simply a case of knowing PHP, mySQL and Apache. With those three tools you could pretty much do anything! This still applies, but you now have many alternatives, plus the many, many tools available to make everything faster and more reliable which can only mean one thing, the end user is happier because websites are fast, always online and aren’t waiting for things to happen.

Websites that are known and used by most frequent web users actually have many tools to make up their stack. A few examples are: -

• LastFM
  • PHP
  • C++
  • Java
  • Hadoop
  • Python
• Twitter
  • Ruby
  • Scala
  • Java
  • C++
• Facebook
  • PHP
  • Erlang
  • C
  • MySQL
  • Cassandra

The point here is, you need to know when you stack isn’t up to the job. Are you using the current toolkit because it’s what you know, rather than using a tool that could do a better job?

Gareth showed that many of the tools that some shy away from because they are ‘complicated’, can be implemented relatively simply and costing nothing but time. There was a strong warning in the conclusion of the talk…don’t go bounding in and install all these tools on your production server without testing and learning them first. If you do, something WILL go wrong and your knowledge won’t be large enough to dig you out.

Read more about some of tools Gareth mentioned during his talk here.

Peter-Paul Koch (PPK) @ppk

The touch action

PPK started his 30 minute slot telling us how a few years ago he was bored of the work he was doing. Day in, day out testing in browsers on computers. Luckily he was released from this boredom when he was offered a job at Vodafone testing mobile browsers. He now has many more browsers to play with, through several interfaces such as the mouse, keyboard, trackball and touch. Peter showed how depending on the mobile device being used, different events were fired. For example, how do you fire the hover event with a touch interface such as on the iPhone? How do you replicate the swipe gesture on the iPhone using a mouse or keyboard?

Dan Rubin @danrubin

Blending Usability testing with interface design, prototyping and rapid interaction

Called in at the last minute, Dan gave a polished talk on how he and a team of usability testers took a website that looked AWFUL, but amazingly already had a large turnover, applied a few tried and tested techniques and rules and proved to the client through two sessions with users, that the interface can make or break a conversion.

If you are a designer and told you can’t change the look of the website as it already works to create a turnover of $5 million per year, it must be difficult to make vast improvements. But by applying rules such as a grid layout, and organising the pages with some thought, users found it much easier to complete an order. Even small alterations requested by the client, made during a testing session, made the checkout system completely unusable to customers. This type of testing helps prove many core rules that need to be adhered to in order to make a website ‘useable’.

Andy Clarke @malarkey

Harboiled Web design

Known by many to have strong opinions about what should be done during the design process, Andy gave a great talk giving listeners a brief idea of what his new book ‘Hardboiled web design’ is about.

Using CSS3 and HTML5 shouldn’t be shyed away from, and it isn’t bad to use vender prefix CSS rules to help enhance your websites look. Adding small animations, 3D transformations, or even small effects like text shadows and border radiuses shouldn’t be overlooked just because CSS3 isn’t ‘a standard’.

Browser testing should no longer be a process of getting a ruler on a screen and checking everything lines up exactly the same in each browser, but rather using the tools available in any particular browser to make the experience as good as possible. Why should people using Safari 4 be held back by users stuck in 2001 with IE6! Instead use a tool like Modernizr and detect what you can and can’t do to enhance the website for this visitor. Browser testing in 2010 should be more about checking that a visitor can get the best experience that their browser allows them to. So if the visitor is viewing with Safari 4, add some animation and transitions, and it not, don’t.

A great quote that sums up everything he believes in is “Only geeks have more than one browser installed! Your mum won’t be checking that a webpage looks the same in Safari and IE”.

For its first year, DIBI was a great success, and also being my first visit to the North East, I can also say that was pretty good too! For anyone wanting to go see speakers talk about a whole variety of topics to do with the web, I highly recommend next years DIBI, which is again at The Sage in Gateshead, on 8th June. See the website for more details.

Start of a new beginning!

I have finally got round to splitting out my 2 main domains into separate websites/blogs/dumping grounds.

I plan to use my this site, http://www.joeturner.info for anything to do with the digital sector, and less for my personal rants and raves. Those will happen over at http://www.mentalreality.co.uk. So, want to read interesting geeky things, stick around here. Want to see what I'm thinking about world on the whole, head over to MentalReality

I'm sure iI'll keep up this idea for at least, oh I don't know, a month?