jurriaanpersyn.com

A month from now, the first Netlog Developer day will be held. On this one-day, free admission event we’ll be discussing oa. the OpenSocial API, the gaming platform and monetization possibilities on Netlog.

OpenSocial defines a common API for social applications across multiple websites. With standard JavaScript and HTML, developers can create apps that access a social network’s friends and update feeds.
Netlog is one of the social networks supporting this standard. So, if you’re interested in developing social applications and while-doing-that would like to learn a little more about Netlog, this event is for you. Topics will also include performance and scaling, giving you an insight on how a site like Netlog is built.

In the afternoon there will be a codelab, where Google and Netlog developers will help you out experimenting with the available API’s.
Parallel with the codelab, there will also be a “Brand Integration Day”, where creative agencies and media buyers are invited to learn all about leveraging Netlog’s Brand Integration Platform and key customers will share their experience with Netlog.

A lot has happened during the past months, and we continue to work on improving our platform – for users and partners alike. We’d be thrilled to see you on April 2 to learn what we’ve done and share some of your thoughts.

The Developer Day kicks off at 9 am on April 2nd at Kinepolis in Brussels. Interested? Register here.

Keep an eye on the Netlog Developer blog for more info.

This article accompanies the slides from a presentation on database sharding. Sharding is a technique used for horizontal scaling of databases we are using at Netlog. If you’re interested in high performance, scalability, MySQL, php, caching, partitioning, Sphinx, federation or Netlog, read on …

This presentation was given at the second day of FOSDEM 2009 in Brussels. FOSDEM is an annual conference on open source software with about 5000 hackers. I was invited by Kris Buytaert and Lenz Grimmer to give a talk in the MySQL Dev Room. The talk was based on an earlier talk I gave at BarcampGent 2.

Overview

• Who am I?
• What is Netlog?
• A history of scaling database systems
• Hitting limits
• Sharding basics
• Sharding schemes
• Implications
• Existing solutions
• Implementation
• Tackling the problems
• Final thoughts
• Slides
• Resources

More… »

Here are the slides from yesterday’s presentation about horizontal database scaling through sharding at the mySQL dev room at FOSDEM 2009.

I’ve got a ton of notes and remarks to these slides, which will become available here soon.

Every so often, when in JavasScript-code-mode, there's the need to make some decisions based on data that's not available in the document itself. Say your templating engine comes up with the following HTML, which is basically a list of friends linking to their respective profiles;

Now you'd want to mix in some extra-js-fu, by displaying a mini profile when you click any of the links. For that you'll be using your AJAX-API that needs eg. a user-identification integer or string as parameter. How is your JavaScript gonna know what this userID is for each of the listed friends? Right, you need some extra information for this.

Different approaches

There's a few ways of including data you need on a js-application level to your DOM-tree, some of which I list here;

• Hidden form elements:
  You could nest some forms with hidden elements containing the data you need into your DOM. But I guess for the use case here, that would make your html quite cluttered and you'll have quite some overhead traversing the DOM to find the elements related to the click-event of your link. On the upside though, you can store about any value in those form elements as long as they're properly escaped. Example HTML:
  PLAIN TEXT
  HTML:

  1. <a href="/wolksken">Laura Bogaert</a><form><input type="hidden" name="userid" value="1043" /></form>

• Custom attributes:
  Why not just add some custom userID-attributes to the elements you need? It's very readable and flexible, it works in all browsers that support XHTML and - as with form elements - allows for about any type of text string, thus is an excellent solution. And in fact, the X in XHTML is for extensible right? But of course, there's the Standards Police™, who might not like that you're adding new custom attributes that aren't in the original specification, to put you off from using this technique. Example HTML:
  PLAIN TEXT
  HTML:

  1. <a href="/wolksken" userid="1043">Laura Bogaert</a>

• Abuse an existing attribute:
  You could hide the fact you actually want to use custom attributes, by just abusing some of the existing ones. Let's for instance take the rel="" attribute; an existing attribute, available on anchors, left mostly unused; if we use it to store a userid in, validators won't complain, so we're all set? Well, quite an ugly hack, isn't it? Not semantic, not always available and you still have no easy solution for key-value pairs. Example HTML:
  PLAIN TEXT
  HTML:

  1. <a href="/wolksken" rel="1043">Laura Bogaert</a>

• (Ab)use your class-attribute:
  There is one attribute that does fit for using to save data though, since it's available on every element, flexible enough, and it (can) even make sense on a semantic level. While class names will in 99% of the cases play a purely visual role, you could use them for application logic too. Isn't that what microformats are doing? Since you're using class names, you have to deal with some of it's limitations though; they can't have spaces, and some browsers might not like all characters. Example HTML:
  PLAIN TEXT
  HTML:

  1. <a href="/wolksken" class="userid_1043">Laura Bogaert</a>

Each of these 4 techniques have their own reasons to exist, and I'm not here to tell you which one is best, or which one you should use. I'm here to share two helper methods I've written for the class names approach.

Extending Prototype with classname data helpers

So, we're adding some extra data to our class-attributes. The templating engine does this in the format of "_".

If we want to access these key-value pairs in JavaScript we go to the DOM-element, loop over it's class names, split the keys and values, and return the value for the queried key. Likewise you might want to have a class data setter method too.

For this I wrote two methods, getClassData(key) and setClassData(key, value), for Prototype, my weapon of choice for JavaScript development. These methods are getters and setters for those key-value pairs, and thanks to Element.addMethods, available on every DOM-element. Here's an example of it's use:

Here's the source code:

Here's a seperate file with the script: classdata-extensions-01.js.

The getClassData will always return values as strings. You could build in some type-hinting methods, but you'd have to implement that same protocol in your templating engine too then, so I decided to leave it out here. The data value is now being url-escaped to add it to the classes, but of course there's some limitations there too.

Here's to hoping these 2 methods might provide you some use ...

This saturday (nov 29th), Johan Ronsse, is hosting a second edition of BarCamp, at the IBBT building in Ghent, Belgium. BarCamp is an "ad-hoc gathering born from the desire for people to share and learn in an open environment". This "un-conference" with discussions, demos and interaction from the participants, is a great way to meet some of the local people in our industry. There's always a good dose of talented people around, so we're looking forward to share and learn ...

At the last edition in 2007, Lennart and Jurriaan spoke about frontend best practices. This time a few more of the Netlog developers and designers will be present, giving talks about HTML5, iPhone development, Sphinx, Memcached and sharding.

In "HTML5, In a big nutshell", Lennart - lead web designer at Netlog - will introduce us to the next version of html. And there's plenty of cool stuff to get excited about! An overview of Web Forms 2.0 and Web Applications 1.0, will get you up to date on the current draft of new tags as <m>, <progress> and <video>.
Fresh from releasing the first version of Netlog's iPhone app, Lieven will do an introductory talk on mobile development for the iPhone, with a few insights on how we tried to get it right building Netlog's app.
Jayme and Jurriaan will shed some light on a few of the more advanced performance techniques we're using here at Netlog. We'll explain a practical example of a horizontally federated database system (sharding), and how we tackle some of its issues with Sphinx and Memcached.

A few of the other presentation we're looking forward to include: "Chinese Internet Market Overview", Mollom, GIT and "UTF-8: The Secret of Character Encoding".

The BarCamp attendee list is full, but keep an eye out for barcamp related blog posts and presentations, on this blog and the rest of the interwebs. Full details on #barcampghent over at barcampgent2.wikispaces.com.