So here’s the story about how to have the page displayed first, ads later. One may argue that you want your money machine up (ha!) before giving away the goods. One could also argue that the ads coming in late will draw more attention. I say: A slow site is like no site.

And as with any story, the solution comes in the end. If you want to know who the killer is, and don’t care about the drama, just skip to the end.

Trial #1: Absolute positioning

Spoiler: This didn’t work for me. But it may work for you. I don’t know if it’s because I’m using a lot of Javascript in my page or not.

The idea was that if the browser doesn’t need to know what’s inside the ad box, it will go on rendering the page. Since an absolutely positioned section is out of the flow, and won’t change any other part’s placement, I hoped that the browser would be smart enough not to wait for the ads. Or stupid enough to overlook the inconsistency in scripting. I was wrong.

Anyhow, this is the code I used:

<div style="position: relative; height: 620px; width: 200px; overflow: hidden; ">
<div style="position: absolute; top: 0px; left: 0px">
<script type="text/javascript"><!--
google_ad_client = "pub-xxxxxxx";
google_ad_slot = "xxxxxxxx";
google_ad_width = 160;
google_ad_height = 600;
//-->
</script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script>
</div></div>

The outer DIV allocates the necessary space on the page. The inner DIV is absolutely positioned, so no matter what comes in there, everything else will remain in place.

What really happened: IE6 went right on with the rest of the page, Firefox still waited for the ads to load. Even though IE played it nice this time, it’s actually a bug in IE.

The thing is, that I have a lot of Javascript embedded in my page. Since the Google ad code consists of Javascript itself, the browser must wait until that code is loaded before running any other Javascript code (which may, as far as the browser is concerned, depend on it). Firefox waited to prevent unpredictable execution of following Javascript code, IE just went on. Ignorance is bliss.

This way or another, the problem remains. I mean, even Microsoft will fix this bug some day.

(non)-Trial #2:

The idea is simple. If the Google ad code appears just before the closing </body>, the browser must be pretty boneheaded to wait for it. So now the trick is to get the ad in the right place.

Proposed trick:  Create a <div> container at the end of  the document, and push it to its place with absolute positioning. With Javascript or without. This can be done only if one knows the exact pixel position in terms of the entire document. There is no safe way, that I know of, to get the absolute position of an element by its ID. Neither to get it into a known position within a containing block other than the one you’re currenly inside.

There are scripts out there which claim to do that, but they don’t rely on any standard. And getting a block of ads in the wrong place is too bad to risk.

And I haven’t even mentioned what happens when the page is resized. Unless all surrounding elements are nailed in place with absolute positioning, bad things can happen…

Anyhow, this method didn’t fit my case. I dropped it.

Trial #3: Hijacking document.write()

One widely proposed solution is to hack document.write(), which Google uses to implant the extra HTML. I don’t like it very much, because it’s well, hacky. But I tried. And then I saw what the script in show_ads.js produces:

<script src="http://pagead2.googlesyndication.com/pagead/expansion_embed.js">
</script>
<script src="http://googleads.g.doubleclick.net/pagead/test_domain.js">
</script>
<script>window.google_render_ad();</script>

So show_ads.js writes a script that loads other scripts? Doing what? Call other?

This was just enough to scare me off. If hijacking means to recursively run scripts loaded by scripts, there’s a good chance to really mess things up. So I gave this idea up as well.

Trial #4: Moving the DOM element

This is the way to do it. Let the browser reach the ad segment at the end of the document, and then manipulate the document tree, moving the element to its right position. This is pretty intrusive, and it turns out that it pushes the browser’s tolerance to the limit.

But first, let’s look at the code. It has two parts. First, we put this where we really want the ads:

<div style="position: relative; height: 620px; width: 200px; overflow: hidden; ">
<div id="googletarget" style="position: absolute; top: 0px; left: 0px">
</div></div>

That’s pretty much like what I did in trial #1, with two important differences: The google ad isn’t here, of course, and I’ve given the inner DIV an ID, so I can target it later. Absolute positioning is still important, because I don’t want the page to flicker when the ad comes in.

Then, at the end of the document (before closing </body>) we have:

<div style="display: none;"><div id="googleads">
<script type="text/javascript"><!--
google_ad_client = "pub-xxxxxxx";
google_ad_slot = "xxxxxxxx";
google_ad_width = 160;
google_ad_height = 600;
//-->
</script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script>
</div></div>
<script type="text/javascript">
document.getElementById("googletarget").appendChild(document.getElementById("googleads"));
document.getElementById("googleads").style.display="block";
</script>

That rings an old bell too. It’s exactly like the original script, now embedded in other DIVs and with an extra script snippet, which moves the ads to their right position using appendChild(). W3C is very clear about that appendChild() never duplicates a node, but moves it in case it’s already placed in the document tree.

And the DIV, to which the ads are loaded into, is wrapped by another DIV which is set to “display: none”. This makes them invisible in their original position, so nothing flickers at the bottom of the page and the scroll bars don’t get crazy. And even better, if the appendChild() fails (because the browser is old or primitive), the ads remain invisible. Which is rare enough to neglect businesswise, and harmless in terms of how badly we’ve messed up the page.

In IE6, it was necessary to set the display mode back to “block” after moving it. That’s what the line after the appendChild() is for.

By the way, “visibility: hidden;” wasn’t enough, because the Google data contains an IFRAME, which doesn’t respect the external DIV’s visibility status.

And in the extreme case, in which the browser doesn’t respect the “display: hidden” attribute, we will have some ugliness at the bottom of the page. Functionally, it’s still harmless, and graphically it’s not so bad compared with what such a browser’s user sees every day.

So, this looks like a bulletproof solution. Of course it worked on any fairly updated browser I tested with. What could possibly go wrong?

The answer came to me when I tried in on an Mozilla 0.99 (seven years old). Don’t think I’m using it for myself, but testing on a steam engine is a good way to find weak spots. In our case, moving the ads around caused the browser to display a blank page. All was fine just before the call to appendChild(), but then the page went blank, and the browser started to behave strangely.

This is bad news. I’m trying to make a page load slightly faster, and I might make it unloadable (to some) instead. Even if I don’t believe that someone really uses a browser from 2002, a new, cutting-edge and somewhat buggy browser might fall in the same hole.

The problem, as it turned out, is that the element I move around consists of scripts. Mozilla 0.99 executes the script once when it’s loaded, and once again when it’s moved. Modern browsers execute the script once. IE6 is in the middle: It indeed runs the script once, but loads it twice. Not harmful to the user, but this indicates that moving a script around is not an issue of the past.

It also looks like there was an issue with appendChild() and scripts around the development of Firefox 3.1, but given the relatively little attention it got (and the fact that the relevant remark in the docs has vanished), I suppose this is a non-issue.

To see how the browser behaves, I wrote a small snippet:

<div id="taker"></div>
<p>Hello, world</p>
<div id="loser">
<div id="node">
<script>
alert("Script running");
document.write("I was first!");
</script>
</div></div>
<script>
var node = document.getElementById("node");
alert("Now swapping");
document.getElementById("taker").appendChild(node);
</script>

The idea is that the two rows swap places by virtue of appendChild(). The “script running” alert should appear only once, of course. On Mozilla 0.99 it popped up twice, and the page ended up with “I was first” written before and after “Hello world”, which sort-of explains why nothing worked with Google ads. Other browsers I checked, including Konqueror 3.0.0 from 2002 ran the script once, as expected.

Conclusion

The solution I picked was appendChild(), of course. But I also realize that moving around an element which contains a script pushes the browser to its limits. On the other hand, AJAX web sites are becoming increasingly popular, so browsers are more and more likely to support this properly in the future.

For my own use, I’ve decided to do this trick only on browsers I’ve tested. This is possible since the page is produced by a server-side script, which checks the browser version. This will make the page load slightly slower on non-mainstream browsers, but on the other hand, guarantee that the page loads at all.

SQL DATETIME sucks

MySQL, among other databases, has a column type called DATETIME. Its name seems to mislead people into thinking that it’s suitable for storing time of events. Or suitable for anything.

This is a general SQL thing, by the way, but I’ll demonstrate it on MySQL.

I often find this column type in other people’s database schemas, and I wonder if the designer gave it a thought before using it. It’s true, that in the beginning it looks simple:

mysql> CREATE TABLE stupid_date ( thedate DATETIME, PRIMARY KEY (thedate) );
Query OK, 0 rows affected (0.04 sec)

mysql> INSERT INTO stupid_date(thedate) VALUES ( NOW() );
Query OK, 1 row affected (0.03 sec)

mysql> SELECT * FROM stupid_date;
+---------------------+
| thedate             |
+---------------------+
| 2009-03-15 14:01:43 |
+---------------------+
1 row in set (0.00 sec)

That was way too cute, wasn’t it? We also have the NOW() function, which fits in exactly, and puts the current time! Yay! Yay! And if the timestamp looks old-fashioned to you, I suppose there is a reason for that.

But wait, there are two major problems. The first one is that the time is given in the host’s local time. That was fair enough before the internet was invented. But today a web server can be a continent away. DATETIME will show you the local time of the server, not yours. There are SQL functions to convert timezones, of course. Are you sure that you want to deal with them? What happens when you want to move your database to a server in another timezone? What about daylight saving time? Local time is one big YUCK.

(Update: As commented below, the real stupidity is to use NOW(), and not UTC_TIMESTAMP(). The latter gives the UTC time, as its name implies)

Problem number two: Most applications don’t care what the absolute time is. The current time is commonly used to calculate how much time has elapsed since a certain event. To filter elements according to if they were X seconds before now. Is the user still logged in? Has 24 hours elapsed since the last warning email was sent? And so on.

“Solution”: The SQL language supplies a variety of COBOL-like functions to calculate whatever we can ask for. And also an opportunity to get things horribly wrong, because the SQL statement became way too complicated.

Use POSIX time() instead

Sorry, didn’t mean to scare you off. It’s really simple: Any modern operating system, even Windows, will readily supply you with the number of seconds since January 1, 1970, midnight, UTC (that is, more or less GMT). This is also called “seconds since the Epoch” or “UNIX time”.

No matter where the computer is, what timezone it uses or what programming language you’re using, this simple integer representation will show the same number at any given moment.

You can, in fact, obtain this number from MySQL directly:

mysql> SELECT UNIX_TIMESTAMP(thedate) FROM stupid_date;
+-------------------------+
| UNIX_TIMESTAMP(thedate) |
+-------------------------+
|              1237118503 |
+-------------------------+
1 row in set (0.00 sec)

This means, that 1237118503 seconds elapsed since the Epoch (which is a global time point) until 14:01:43 in Israeli LOCAL time of the day I wrote this post. So now we have an integer number to work with, which is handy for calculations, but things will still get messy if we try to move the database to another server.

Store the number instead

If we are interested in working with integers, why not store the integer itself in the database? We could go:

mysql> CREATE TABLE smart_date ( thedate INTEGER UNSIGNED, PRIMARY KEY (thedate) );
Query OK, 0 rows affected (0.00 sec)

mysql> INSERT INTO smart_date(thedate) VALUES (1237118503);
Query OK, 1 row affected (0.00 sec)

mysql> SELECT * FROM smart_date;
+------------+
| thedate    |
+------------+
| 1237118503 |
+------------+
1 row in set (0.00 sec)

That wasn’t very impressive, was it? The first question would be “OK, how do I get this magic number, now that I don’t have the NOW() function?”

The short and not-so-clever answer is that you could always use MySQL’s UNIX_TIMESTAMP( NOW() ) for this. The better answer is that no matter which scripting or programming language you’re using, this number is very easy to obtain. I’ll show examples below.

As for the magnitude of this number, yes, it’s pretty big. But it will fit a signed 32-bit integer until year 2038. I presume that nobody will use 32-bit integers by then.

And finally, one could argue that DATETIME is convenient when reading from the database directly. True. But for that specific issue we have the FROM_UNIXTIME() function:

mysql> SELECT FROM_UNIXTIME(thedate) FROM smart_date;
+------------------------+
| FROM_UNIXTIME(thedate) |
+------------------------+
| 2009-03-15 14:01:43    |
+------------------------+
1 row in set (0.00 sec)

And again, this is given in the computer’s local time. Which is fine, because it’s intended to be read by humans. In particular, humans who easily translate time differences between their server and themselves.

Obtaining Epoch time

Just to prove that it’s easy to know what the “Epoch time” is in any language, here are a few examples. Wherever it’s really simple, I’m showing how to convert this format to human-readable format.

In Perl:

print time();
print scalar localtime time(); # Local time for humans

In PHP:

<?php
echo time();
echo date('r', time() ); // Local time for humans
?>

In Python:

from time import time;
print time();

(note that the time is returned as a float number with higher precision)

In C:

#include <time.h>
#include <stdio.h>

int main () {
  int now = time(NULL);

  printf("%d seconds since the Epoch\n", now);
  return 0;
}

In JavaScript:

<script language="JavaScript" type="text/javascript">
now = new Date();
alert( now.getTime() / 1000 );
</script>

In this case, the time is shown with a fractional resolution.

The JavaScript example is not really useful for a database application, because the time is measured at the computer showing the page. In a website application, this is just anybody’s computer clock, which may be wrong. But it’s yet another example of how this time representation is widely available.

Conclusion

Drop those DATETIME columns from your tables, and use a simple, robust and handy format to represent time. Don’t let the database play around with a sensitive issue like time, and don’t risk getting confused by different functions when calculating time differences. Just because the DATETIME column type exists, it doesn’t mean there is a reason to use it.

Enjoy the database on what it’s best at: Storing and collecting information.